Skip to contentSkip to conversation

scsi: enable reserved commands for LLDDs

RFC PATCH Superseded Hannes Reinecke 8 participants 104 messages Apr 30, 2020 – Jun 8, 2020

Subsystems (11)

v3 of 9 Superseded Latest is v9 →

lore
Share & tools
lore
#0 lore raw

v3 cover scsi: enable reserved commands for LLDDs

Hi all,

quite some drivers use internal commands for various purposes, most commonly sending TMFs or querying the HBA status.
While these commands use the same submission mechanism than normal I/O commands, they will not be counted as outstanding commands, requiring those drivers to implement their own mechanism to figure out outstanding commands.
The block layer already has the concept of 'reserved' tags for precisely this purpose, namely non-I/O tags which live off a separate tag pool. That guarantees that these commands can always be sent, and won't be influenced by tag starvation from the I/O tag pool. This patchset enables the use of reserved tags for the SCSI midlayer by allocating a virtual LUN for the HBA itself which just serves as a resource to allocate valid tags from.
This removes quite some hacks which were required for some drivers (eg. fnic or snic).
Conversion of the SAS drivers hisi_sas, pm8001, and mv_sas are compile tested only; I'd be grateful if someone could give these patches a spin on that hardware, too.

The entire patchset can be found at

git://git.kernel.org/pub/scm/linux/kernel/git/hare/scsi-devel.git reserved-tags.v3

As usual, comments and reviews are welcome.

Changes to v2:
- Update patches from John Garry
- Use virtual LUN as suggested by Christoph
- Improve SCSI Host device to present a real SCSI device
- Implement 'persistent' commands for AENs
- Convert Megaraid SAS

Changes to v1:
- Make scsi_{get, put}_reserved_cmd() for Scsi host
- Previously we separate scsi_{get, put}_reserved_cmd() for sdev and scsi_host_get_reserved_cmd() for the host
- Fix how Scsi_Host.can_queue is set in the virtio-scsi change
- Drop Scsi_Host.use_reserved_cmd_q
- Drop scsi_is_reserved_cmd()
- Add support in libsas and associated HBA drivers
- Allocate reserved command in slow task
- Switch hisi_sas to use reserved Scsi command
- Reorder the series a little
- Some tidying

Hannes Reinecke (39):
scsi: add 'nr_reserved_cmds' field to the SCSI host template
scsi: add scsi_{get,put}_reserved_cmd()
scsi: Implement scsi_cmd_is_reserved()
csiostor: use reserved command for LUN reset
scsi: add scsi_cmd_from_priv()
virtio_scsi: use reserved commands for TMF
fnic: use reserved commands
fnic: use scsi_host_busy_iter() to traverse commands
scsi: use real inquiry data when initialising devices
scsi: make host device a first-class citizen
hpsa: move hpsa_hba_inquiry after scsi_add_host()
hpsa: use reserved commands
hpsa: use scsi_host_busy_iter() to traverse outstanding commands
hpsa: drop refcount field from CommandList
aacraid: use private commands
aacraid: use scsi_host_busy_iter() to traverse commands
megaraid_sas: kill this_id and init_id
megaraid_sas: use shost_priv()
megaraid_sas: avoid using megaraid_lookup_instance()
megaraid_sas: separate out megasas_set_max_sectors()
megaraid_sas: megaraid_sas: reshuffle SCSI host allocation
block: implement persistent commands
scsi: add a 'persistent' argument to scsi_get_reserved_cmd()
megaraid_sas: separate out megasas_prepare_aen()
megaraid_sas: use reserved commands
megaraid_sas_fusion: rearrange mfi and mpt frame pools
megaraid_sas_fusion: sanitize command lookup
megaraid_sas: use scsi_host_busy_iter to traverse outstanding commands
snic: use reserved commands
snic: use tagset iter for traversing commands
mv_sas: kill mvsas_debug_issue_ssp_tmf()
pm8001: kill pm8001_issue_ssp_tmf()
pm8001: kill 'dev' argument from pm8001_exec_internal_task_abort()
pm8001: use libsas-provided domain devices for SATA
libsas: add SCSI target pointer to struct domain_device
libsas: add tag to struct sas_task
hisi_sas: use task tag to reference the slot
mv_sas: use reserved tags and drop private tag allocation
pm8001: use block-layer tags for ccb allocation

John Garry (2):
scsi: libsas,hisi_sas,mvsas,pm8001: Allocate Scsi_cmd for slow task
scsi: hisi_sas: Use libsas slow task SCSI command

 block/blk-mq.c                              |  27 +-
 drivers/scsi/aacraid/aachba.c               |  14 +-
 drivers/scsi/aacraid/aacraid.h              |   9 +-
 drivers/scsi/aacraid/commctrl.c             |  57 +-
 drivers/scsi/aacraid/comminit.c             |  34 +-
 drivers/scsi/aacraid/commsup.c              |  76 +--
 drivers/scsi/aacraid/dpcsup.c               |   2 +-
 drivers/scsi/aacraid/linit.c                | 181 +++---
 drivers/scsi/csiostor/csio_init.c           |   1 +
 drivers/scsi/csiostor/csio_scsi.c           |  48 +-
 drivers/scsi/fnic/fnic_scsi.c               | 943 ++++++++++++----------------
 drivers/scsi/hisi_sas/hisi_sas_main.c       | 116 ++--
 drivers/scsi/hisi_sas/hisi_sas_v3_hw.c      |   5 +-
 drivers/scsi/hpsa.c                         | 367 +++++------
 drivers/scsi/hpsa.h                         |   3 +-
 drivers/scsi/hpsa_cmd.h                     |   1 -
 drivers/scsi/libsas/sas_ata.c               |   4 +
 drivers/scsi/libsas/sas_expander.c          |   7 +-
 drivers/scsi/libsas/sas_init.c              |  63 +-
 drivers/scsi/libsas/sas_scsi_host.c         |   4 +-
 drivers/scsi/megaraid/megaraid_sas.h        |  14 +-
 drivers/scsi/megaraid/megaraid_sas_base.c   | 664 +++++++++++---------
 drivers/scsi/megaraid/megaraid_sas_fusion.c | 431 +++++++------
 drivers/scsi/megaraid/megaraid_sas_fusion.h |   5 -
 drivers/scsi/mvsas/mv_init.c                |  10 +-
 drivers/scsi/mvsas/mv_sas.c                 | 136 ++--
 drivers/scsi/mvsas/mv_sas.h                 |  13 +-
 drivers/scsi/pm8001/pm8001_hwi.c            | 133 ++--
 drivers/scsi/pm8001/pm8001_init.c           |  30 +-
 drivers/scsi/pm8001/pm8001_sas.c            | 192 +++---
 drivers/scsi/pm8001/pm8001_sas.h            |  13 +-
 drivers/scsi/pm8001/pm80xx_hwi.c            | 131 ++--
 drivers/scsi/scsi_lib.c                     |  73 +++
 drivers/scsi/scsi_scan.c                    |  80 ++-
 drivers/scsi/scsi_sysfs.c                   |   3 +-
 drivers/scsi/snic/snic.h                    |   4 +-
 drivers/scsi/snic/snic_main.c               |   8 +
 drivers/scsi/snic/snic_scsi.c               | 386 ++++++------
 drivers/scsi/virtio_scsi.c                  | 105 ++--
 include/linux/blk-mq.h                      |   3 +
 include/linux/blk_types.h                   |   4 +
 include/scsi/libsas.h                       |  11 +-
 include/scsi/scsi_cmnd.h                    |  23 +
 include/scsi/scsi_device.h                  |   4 +
 include/scsi/scsi_host.h                    |   9 +
 45 files changed, 2190 insertions(+), 2257 deletions(-)
Signature · 1 line
2.16.4
#1 lore raw

RFC PATCH v3 1/41 scsi: add 'nr_reserved_cmds' field to the SCSI host template

From: Hannes Reinecke <[email protected]>

Quite a lot of drivers are using management commands internally, which typically use the same hardware tag pool (ie they are being allocated from the same hardware resources) as the 'normal' I/O commands. These commands are set aside before allocating the block-mq tag bitmap, so they'll never show up as busy in the tag map.
The block-layer, OTOH, already has 'reserved_tags' to handle precisely this situation.
So this patch adds a new field 'nr_reserved_cmds' to the SCSI host template to instruct the block layer to set aside a tag space for these management commands by using reserved_tags.

---
 drivers/scsi/scsi_lib.c  | 1 +
 include/scsi/scsi_host.h | 6 ++++++
 2 files changed, 7 insertions(+)
drivers/scsi/scsi_lib.c +1 -0
index 47835c4b4ee0..5358f553f526 100644
@@ -1885,6 +1885,7 @@ int scsi_mq_setup_tags(struct Scsi_Host *shost)
 		shost->tag_set.ops = &scsi_mq_ops_no_commit;
 	shost->tag_set.nr_hw_queues = shost->nr_hw_queues ? : 1;
 	shost->tag_set.queue_depth = shost->can_queue;
+	shost->tag_set.reserved_tags = shost->nr_reserved_cmds;
 	shost->tag_set.cmd_size = cmd_size;
 	shost->tag_set.numa_node = NUMA_NO_NODE;
 	shost->tag_set.flags = BLK_MQ_F_SHOULD_MERGE;
include/scsi/scsi_host.h +6 -0
index 822e8cda8d9b..37bb7d74e4c4 100644
@@ -599,6 +599,12 @@ struct Scsi_Host {
 	 * is nr_hw_queues * can_queue.
 	 */
 	unsigned nr_hw_queues;
+
+	/*
+	 * Number of commands to reserve, if any.
+	 */
+	unsigned nr_reserved_cmds;
+
 	unsigned active_mode:2;
 	unsigned unchecked_isa_dma:1;
 
Signature · 1 line
2.16.4
Reviewed-by on v3 1/41
#2 lore raw

Re: [PATCH RFC v3 01/41] scsi: add 'nr_reserved_cmds' field to the SCSI host template

Quoted 13 lines · Hannes Reinecke

From: Hannes Reinecke <[email protected]>

Quite a lot of drivers are using management commands internally, which typically use the same hardware tag pool (ie they are being allocated from the same hardware resources) as the 'normal' I/O commands. These commands are set aside before allocating the block-mq tag bitmap, so they'll never show up as busy in the tag map.
The block-layer, OTOH, already has 'reserved_tags' to handle precisely this situation.
So this patch adds a new field 'nr_reserved_cmds' to the SCSI host template to instruct the block layer to set aside a tag space for these management commands by using reserved_tags.

Quoted 1 line

It may be worth adding this field to scsi_host_template. And we should also prob mention this in Documentation/scsi/scsi_mid_low_api.txt

Apart from that, thanks:

Quoted 35 lines
---
  drivers/scsi/scsi_lib.c  | 1 +
  include/scsi/scsi_host.h | 6 ++++++
  2 files changed, 7 insertions(+)
drivers/scsi/scsi_lib.c +1 -0
index 47835c4b4ee0..5358f553f526 100644
@@ -1885,6 +1885,7 @@ int scsi_mq_setup_tags(struct Scsi_Host *shost)
  		shost->tag_set.ops = &scsi_mq_ops_no_commit;
  	shost->tag_set.nr_hw_queues = shost->nr_hw_queues ? : 1;
  	shost->tag_set.queue_depth = shost->can_queue;
+	shost->tag_set.reserved_tags = shost->nr_reserved_cmds;
  	shost->tag_set.cmd_size = cmd_size;
  	shost->tag_set.numa_node = NUMA_NO_NODE;
  	shost->tag_set.flags = BLK_MQ_F_SHOULD_MERGE;
include/scsi/scsi_host.h +6 -0
index 822e8cda8d9b..37bb7d74e4c4 100644
@@ -599,6 +599,12 @@ struct Scsi_Host {
  	 * is nr_hw_queues * can_queue.
  	 */
  	unsigned nr_hw_queues;
+
+	/*
+	 * Number of commands to reserve, if any.
+	 */
+	unsigned nr_reserved_cmds;
+
  	unsigned active_mode:2;
  	unsigned unchecked_isa_dma:1;
 
#3 lore raw

Re: [PATCH RFC v3 01/41] scsi: add 'nr_reserved_cmds' field to the SCSI host template

Quoted 14 lines · John Garry
Quoted 13 lines · Hannes Reinecke

From: Hannes Reinecke <[email protected]>

Quite a lot of drivers are using management commands internally, which typically use the same hardware tag pool (ie they are being allocated from the same hardware resources) as the 'normal' I/O commands. These commands are set aside before allocating the block-mq tag bitmap, so they'll never show up as busy in the tag map.
The block-layer, OTOH, already has 'reserved_tags' to handle precisely this situation.
So this patch adds a new field 'nr_reserved_cmds' to the SCSI host template to instruct the block layer to set aside a tag space for these management commands by using reserved_tags.

Quoted 4 lines
Quoted 1 line

It may be worth adding this field to scsi_host_template. And we should also prob mention this in Documentation/scsi/scsi_mid_low_api.txt

Right, indeed, will be doing so.
Haven't done it as long as this is still an RFC; guess when we'll get the SAS bits sorted (hint, hint :-) and no further objections are coming wrt the overall design I'll be sending out a 'real' patchset with the documentation bits sorted, too.

Quoted 4 lines

Apart from that, thanks:

Thanks for the review!

Cheers,

Hannes

Signature · 4 lines
Dr. Hannes Reinecke            Teamlead Storage & Networking
[email protected]                               +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer
Reviewed-by on v3 1/41
#4 lore raw

Re: [PATCH RFC v3 01/41] scsi: add 'nr_reserved_cmds' field to the SCSI host template

In reply to Hannes Reinecke
Quoted 12 lines · Hannes Reinecke

From: Hannes Reinecke <[email protected]>

Quite a lot of drivers are using management commands internally, which typically use the same hardware tag pool (ie they are being allocated from the same hardware resources) as the 'normal' I/O commands. These commands are set aside before allocating the block-mq tag bitmap, so they'll never show up as busy in the tag map.
The block-layer, OTOH, already has 'reserved_tags' to handle precisely this situation.
So this patch adds a new field 'nr_reserved_cmds' to the SCSI host template to instruct the block layer to set aside a tag space for these management commands by using reserved_tags.

#5 lore raw

Re: [PATCH RFC v3 01/41] scsi: add 'nr_reserved_cmds' field to the SCSI host template

In reply to Hannes Reinecke

drivers/scsi/scsi_lib.c int scsi_mq_setup_tags(struct Scsi_Host *shost) · line 1888

Quoted 9 lines · Hannes Reinecke
drivers/scsi/scsi_lib.c +1 -0
index 47835c4b4ee0..5358f553f526 100644
@@ -1885,6 +1885,7 @@ int scsi_mq_setup_tags(struct Scsi_Host *shost)
 		shost->tag_set.ops = &scsi_mq_ops_no_commit;
 	shost->tag_set.nr_hw_queues = shost->nr_hw_queues ? : 1;
 	shost->tag_set.queue_depth = shost->can_queue;
+	shost->tag_set.reserved_tags = shost->nr_reserved_cmds;

Insteda of just passing through the value I think we should remove them from can_queue here - as seen in the few patches the current behavior of summing both up seems to cause a fair amount of confusion.

Also I'd merge this with the patch to actually allocate reserved command, as that is one actual unit of useful functionality.

#6 lore raw

Re: [PATCH RFC v3 01/41] scsi: add 'nr_reserved_cmds' field to the SCSI host template

Quoted 16 lines · Christoph Hellwig

drivers/scsi/scsi_lib.c int scsi_mq_setup_tags(struct Scsi_Host *shost) · line 1888

Quoted 9 lines · Hannes Reinecke
drivers/scsi/scsi_lib.c +1 -0
index 47835c4b4ee0..5358f553f526 100644
@@ -1885,6 +1885,7 @@ int scsi_mq_setup_tags(struct Scsi_Host *shost)
  		shost->tag_set.ops = &scsi_mq_ops_no_commit;
  	shost->tag_set.nr_hw_queues = shost->nr_hw_queues ? : 1;
  	shost->tag_set.queue_depth = shost->can_queue;
+	shost->tag_set.reserved_tags = shost->nr_reserved_cmds;

Insteda of just passing through the value I think we should remove them from can_queue here - as seen in the few patches the current behavior of summing both up seems to cause a fair amount of confusion.

Yes, I would very much prefer that.

Quoted 3 lines

Also I'd merge this with the patch to actually allocate reserved command, as that is one actual unit of useful functionality.

Right, will do.

Cheers,

Hannes

Signature · 4 lines
Dr. Hannes Reinecke            Teamlead Storage & Networking
[email protected]                               +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer
#7 lore raw

RFC PATCH v3 2/41 scsi: add scsi_{get,put}_reserved_cmd()

In reply to Hannes Reinecke

Add helper functions to retrieve SCSI commands from the reserved tag pool.

---
 drivers/scsi/scsi_lib.c    | 35 +++++++++++++++++++++++++++++++++++
 include/scsi/scsi_device.h |  3 +++
 2 files changed, 38 insertions(+)
drivers/scsi/scsi_lib.c +35 -0
index 5358f553f526..d0972744ea7f 100644
@@ -1901,6 +1901,41 @@ void scsi_mq_destroy_tags(struct Scsi_Host *shost)
 	blk_mq_free_tag_set(&shost->tag_set);
 }
 
+/**
+ * scsi_get_reserved_cmd - allocate a SCSI command from reserved tags
+ * @sdev: SCSI device from which to allocate the command
+ * @data_direction: Data direction for the allocated command
+ */
+struct scsi_cmnd *scsi_get_reserved_cmd(struct scsi_device *sdev,
+					int data_direction)
+{
+	struct request *rq;
+	struct scsi_cmnd *scmd;
+
+	rq = blk_mq_alloc_request(sdev->request_queue,
+				  data_direction == DMA_TO_DEVICE ?
+				  REQ_OP_SCSI_OUT : REQ_OP_SCSI_IN | REQ_NOWAIT,
+				  BLK_MQ_REQ_RESERVED);
+	if (IS_ERR(rq))
+		return NULL;
+	scmd = blk_mq_rq_to_pdu(rq);
+	scmd->request = rq;
+	return scmd;
+}
+EXPORT_SYMBOL_GPL(scsi_get_reserved_cmd);
+
+/**
+ * scsi_put_reserved_cmd - free a SCSI command allocated from reserved tags
+ * @scmd: SCSI command to be freed
+ */
+void scsi_put_reserved_cmd(struct scsi_cmnd *scmd)
+{
+	struct request *rq = blk_mq_rq_from_pdu(scmd);
+
+	blk_mq_free_request(rq);
+}
+EXPORT_SYMBOL_GPL(scsi_put_reserved_cmd);
+
 /**
  * scsi_device_from_queue - return sdev associated with a request_queue
  * @q: The request queue to return the sdev from
include/scsi/scsi_device.h +3 -0
index c3cba2aaf934..e74c7e671aa0 100644
@@ -457,6 +457,9 @@ static inline int scsi_execute_req(struct scsi_device *sdev,
 	return scsi_execute(sdev, cmd, data_direction, buffer,
 		bufflen, NULL, sshdr, timeout, retries,  0, 0, resid);
 }
+struct scsi_cmnd *scsi_get_reserved_cmd(struct scsi_device *sdev,
+					int data_direction);
+void scsi_put_reserved_cmd(struct scsi_cmnd *scmd);
 extern void sdev_disable_disk_events(struct scsi_device *sdev);
 extern void sdev_enable_disk_events(struct scsi_device *sdev);
 extern int scsi_vpd_lun_id(struct scsi_device *, char *, size_t);
Signature · 1 line
2.16.4
#8 lore raw

Re: [PATCH RFC v3 02/41] scsi: add scsi_{get,put}_reserved_cmd()

Quoted 21 lines · Hannes Reinecke

Add helper functions to retrieve SCSI commands from the reserved tag pool.

---
  drivers/scsi/scsi_lib.c    | 35 +++++++++++++++++++++++++++++++++++
  include/scsi/scsi_device.h |  3 +++
  2 files changed, 38 insertions(+)
drivers/scsi/scsi_lib.c +4 -0
index 5358f553f526..d0972744ea7f 100644
@@ -1901,6 +1901,41 @@ void scsi_mq_destroy_tags(struct Scsi_Host *shost)
  	blk_mq_free_tag_set(&shost->tag_set);
  }
 
+/**
+ * scsi_get_reserved_cmd - allocate a SCSI command from reserved tags
+ * @sdev: SCSI device from which to allocate the command
+ * @data_direction: Data direction for the allocated command

drivers/scsi/scsi_lib.c void scsi_mq_destroy_tags(struct Scsi_Host *shost) · line 1917

Quoted 10 lines
drivers/scsi/scsi_lib.c +10 -0
+ */
+struct scsi_cmnd *scsi_get_reserved_cmd(struct scsi_device *sdev,
+					int data_direction)
+{
+	struct request *rq;
+	struct scsi_cmnd *scmd;
+
+	rq = blk_mq_alloc_request(sdev->request_queue,
+				  data_direction == DMA_TO_DEVICE ?
+				  REQ_OP_SCSI_OUT : REQ_OP_SCSI_IN | REQ_NOWAIT,

after consulting: https://en.cppreference.com/w/c/language/operator_precedence
I think that the above expression can be written as:

(data_direction == DMA_TO_DEVICE) ? REQ_OP_SCSI_OUT : (REQ_OP_SCSI_IN | REQ_NOWAIT),

So is REQ_NOWAIT not needed with REQ_OP_SCSI_OUT ?

Please note that scripts/checkpatch.pl does _not_ complain when parentheses are used around bitwise operators in complex expressions. And I reckon this expression qualifies as complex.

Doug Gilbert

Note: in that reference the terniary operator gets this note: "The expression in the middle of the conditional operator (between ? and :) is parsed as if
parenthesized: its precedence relative to ?: is ignored". Who knew?

#9 lore raw

Re: [PATCH RFC v3 02/41] scsi: add scsi_{get,put}_reserved_cmd()

Quoted 35 lines · Douglas Gilbert
Quoted 20 lines · Hannes Reinecke

Add helper functions to retrieve SCSI commands from the reserved tag pool.

---
  drivers/scsi/scsi_lib.c    | 35 +++++++++++++++++++++++++++++++++++ include/scsi/scsi_device.h |  3 +++

  2 files changed, 38 insertions(+)
drivers/scsi/scsi_lib.c +4 -0
index 5358f553f526..d0972744ea7f 100644
@@ -1901,6 +1901,41 @@ void scsi_mq_destroy_tags(struct Scsi_Host *shost)
       blk_mq_free_tag_set(&shost->tag_set);
   }
+/**
+ * scsi_get_reserved_cmd - allocate a SCSI command from reserved tags
+ * @sdev: SCSI device from which to allocate the command
+ * @data_direction: Data direction for the allocated command

drivers/scsi/scsi_lib.c void scsi_mq_destroy_tags(struct Scsi_Host *shost) · line 1916

Quoted 10 lines
drivers/scsi/scsi_lib.c +10 -0
+ */
+struct scsi_cmnd *scsi_get_reserved_cmd(struct scsi_device *sdev,
+                    int data_direction)
+{
+    struct request *rq;
+    struct scsi_cmnd *scmd;
+
+    rq = blk_mq_alloc_request(sdev->request_queue,
+                  data_direction == DMA_TO_DEVICE ?
+                  REQ_OP_SCSI_OUT : REQ_OP_SCSI_IN | REQ_NOWAIT,

after consulting: https://en.cppreference.com/w/c/language/operator_precedence
I think that the above expression can be written as:

Quoted 4 lines

(data_direction == DMA_TO_DEVICE) ? REQ_OP_SCSI_OUT : (REQ_OP_SCSI_IN | REQ_NOWAIT),

So is REQ_NOWAIT not needed with REQ_OP_SCSI_OUT ?

Nope, that's an error on my side.
REQ_NOWAIT should be set for both directions.

I'll fix it up.

Cheers,

Hannes

Signature · 4 lines
Dr. Hannes Reinecke            Teamlead Storage & Networking
[email protected]                               +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer
#10 lore raw

Re: [PATCH RFC v3 02/41] scsi: add scsi_{get,put}_reserved_cmd()

In reply to Hannes Reinecke
Quoted 11 lines · Hannes Reinecke
drivers/scsi/scsi_lib.c +11 -0
+/**
+ * scsi_get_reserved_cmd - allocate a SCSI command from reserved tags
+ * @sdev: SCSI device from which to allocate the command
+ * @data_direction: Data direction for the allocated command
+ */
+struct scsi_cmnd *scsi_get_reserved_cmd(struct scsi_device *sdev,
+					int data_direction)
+{
+	struct request *rq;
+	struct scsi_cmnd *scmd;
+

drivers/scsi/scsi_lib.c void scsi_mq_destroy_tags(struct Scsi_Host *shost) · line 1924

Quoted 10 lines
drivers/scsi/scsi_lib.c +10 -0
+	rq = blk_mq_alloc_request(sdev->request_queue,
+				  data_direction == DMA_TO_DEVICE ?
+				  REQ_OP_SCSI_OUT : REQ_OP_SCSI_IN | REQ_NOWAIT,
+				  BLK_MQ_REQ_RESERVED);
+	if (IS_ERR(rq))
+		return NULL;
+	scmd = blk_mq_rq_to_pdu(rq);
+	scmd->request = rq;
+	return scmd;
+}

Isn't REQ_NOWAIT something the caller should decide about instead of always setting that flag? Additionally, I think some parentheses are missing. I think the compiler will interpret the blk_mq_alloc_request() call as follows, which is probably not what was intended:

rq = blk_mq_alloc_request(sdev->request_queue,

		  data_direction == DMA_TO_DEVICE ?
		  REQ_OP_SCSI_OUT : (REQ_OP_SCSI_IN | REQ_NOWAIT),
		  BLK_MQ_REQ_RESERVED);

Thanks,

Bart.

#11 lore raw

Re: [PATCH RFC v3 02/41] scsi: add scsi_{get,put}_reserved_cmd()

In reply to Hannes Reinecke

drivers/scsi/scsi_lib.c void scsi_mq_destroy_tags(struct Scsi_Host *shost) · line 1922

Quoted 8 lines · Hannes Reinecke
drivers/scsi/scsi_lib.c +8 -0
+	rq = blk_mq_alloc_request(sdev->request_queue,
+				  data_direction == DMA_TO_DEVICE ?
+				  REQ_OP_SCSI_OUT : REQ_OP_SCSI_IN | REQ_NOWAIT,
+				  BLK_MQ_REQ_RESERVED);
+	if (IS_ERR(rq))
+		return NULL;
+	scmd = blk_mq_rq_to_pdu(rq);
+	scmd->request = rq;

Should we just set scmd->device = sdev also for completeness?

Thanks,
John

Quoted 3 lines
drivers/scsi/scsi_lib.c +3 -0
+	return scmd;
+}
+EXPORT_SYMBOL_GPL(scsi_get_reserved_cmd);
#12 lore raw

Re: [PATCH RFC v3 02/41] scsi: add scsi_{get,put}_reserved_cmd()

Quoted 10 lines · John Garry
Quoted 8 lines · Hannes Reinecke

+    rq = blk_mq_alloc_request(sdev->request_queue,
+                  data_direction == DMA_TO_DEVICE ?
+                  REQ_OP_SCSI_OUT : REQ_OP_SCSI_IN | REQ_NOWAIT, +                  BLK_MQ_REQ_RESERVED);
+    if (IS_ERR(rq))
+        return NULL;
+    scmd = blk_mq_rq_to_pdu(rq);
+    scmd->request = rq;

Quoted 2 lines

Should we just set scmd->device = sdev also for completeness?

Inded, we should.

Cheers,

Hannes

Signature · 4 lines
Dr. Hannes Reinecke            Teamlead Storage & Networking
[email protected]                               +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer
#13 lore raw

Re: [PATCH RFC v3 02/41] scsi: add scsi_{get,put}_reserved_cmd()

In reply to Hannes Reinecke
Quoted 2 lines · Hannes Reinecke

Add helper functions to retrieve SCSI commands from the reserved tag pool.

I'm still quite worried about the fact that we have a pretty much half-initialized command that now goes down the whole stack.

#14 lore raw

Re: [PATCH RFC v3 02/41] scsi: add scsi_{get,put}_reserved_cmd()

Quoted 7 lines · Christoph Hellwig
Quoted 2 lines · Hannes Reinecke

Add helper functions to retrieve SCSI commands from the reserved tag pool.

I'm still quite worried about the fact that we have a pretty much half-initialized command that now goes down the whole stack.

Reserved commands just serve as a placeholder to get a valid tag from the block layer; the SCSI commands themselves are never ever passed through the whole stack, but rather allocated internally within the driver, and passed to the hardware by driver-specific means. So really the SCSI specific parts of the commands are never used.
We can add a check in queuecommand to abort reserved commands if that's what you worried about, though.

Cheers,

Hannes

Signature · 4 lines
Dr. Hannes Reinecke            Teamlead Storage & Networking
[email protected]                               +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer
#15 lore raw

Re: [PATCH RFC v3 02/41] scsi: add scsi_{get,put}_reserved_cmd()

Quoted 7 lines · Hannes Reinecke

Reserved commands just serve as a placeholder to get a valid tag from the block layer; the SCSI commands themselves are never ever passed through the whole stack, but rather allocated internally within the driver, and passed to the hardware by driver-specific means.
So really the SCSI specific parts of the commands are never used. We can add a check in queuecommand to abort reserved commands if that's what you worried about, though.

How about an interface that just returns a tag then, so that it can't be misused?

#16 lore raw

RFC PATCH v3 3/41 scsi: Implement scsi_cmd_is_reserved()

In reply to Hannes Reinecke

Add function to check if a SCSI command originates from the reserved tag pool and update scsi_put_reserved_cmd() to only free commands if they originate from the reserved tag pool.

---
 block/blk-mq.c           | 15 +++++++++++++++
 drivers/scsi/scsi_lib.c  |  7 +++++--
 include/linux/blk-mq.h   |  1 +
 include/scsi/scsi_cmnd.h | 13 +++++++++++++
 4 files changed, 34 insertions(+), 2 deletions(-)
block/blk-mq.c +15 -0
index 8e56884fd2e9..44482aaed11e 100644
@@ -604,6 +604,21 @@ static void __blk_mq_complete_request(struct request *rq)
 	put_cpu();
 }
 
+/**
+ * blk_mq_rq_is_reserved - Check for reserved request
+ *
+ * @rq: Request to check
+ *
+ * Returns true if the request originates from the reserved tag pool.
+ */
+bool blk_mq_rq_is_reserved(struct request *rq)
+{
+	struct blk_mq_hw_ctx *hctx = rq->mq_hctx;
+
+	return blk_mq_tag_is_reserved(hctx->tags, rq->tag);
+}
+EXPORT_SYMBOL_GPL(blk_mq_rq_is_reserved);
+
 static void hctx_unlock(struct blk_mq_hw_ctx *hctx, int srcu_idx)
 	__releases(hctx->srcu)
 {
drivers/scsi/scsi_lib.c +5 -2
index d0972744ea7f..30f9ca9fce22 100644
@@ -1930,9 +1930,12 @@ EXPORT_SYMBOL_GPL(scsi_get_reserved_cmd);
  */
 void scsi_put_reserved_cmd(struct scsi_cmnd *scmd)
 {
-	struct request *rq = blk_mq_rq_from_pdu(scmd);
+	struct request *rq;
 
-	blk_mq_free_request(rq);
+	if (scmd && scsi_cmd_is_reserved(scmd)) {
+		rq = blk_mq_rq_from_pdu(scmd);
+		blk_mq_free_request(rq);
+	}
 }
 EXPORT_SYMBOL_GPL(scsi_put_reserved_cmd);
 
include/linux/blk-mq.h +1 -0
index f389d7c724bd..c186dc25fc1c 100644
@@ -494,6 +494,7 @@ void blk_mq_requeue_request(struct request *rq, bool kick_requeue_list);
 void blk_mq_kick_requeue_list(struct request_queue *q);
 void blk_mq_delay_kick_requeue_list(struct request_queue *q, unsigned long msecs);
 bool blk_mq_complete_request(struct request *rq);
+bool blk_mq_rq_is_reserved(struct request *rq);
 bool blk_mq_bio_list_merge(struct request_queue *q, struct list_head *list,
 			   struct bio *bio, unsigned int nr_segs);
 bool blk_mq_queue_stopped(struct request_queue *q);
include/scsi/scsi_cmnd.h +13 -0
index 80ac89e47b47..2cd894fbdcfa 100644
@@ -159,6 +159,19 @@ static inline struct scsi_driver *scsi_cmd_to_driver(struct scsi_cmnd *cmd)
 	return *(struct scsi_driver **)cmd->request->rq_disk->private_data;
 }
 
+/**
+ * scsi_cmd_is_reserved - check for reserved scsi command
+ * @scmd: command to check
+ *
+ * Returns true if @scmd originates from the reserved tag pool.
+ */
+static inline bool scsi_cmd_is_reserved(struct scsi_cmnd *scmd)
+{
+	struct request *rq = blk_mq_rq_from_pdu(scmd);
+
+	return blk_mq_rq_is_reserved(rq);
+}
+
 extern void scsi_finish_command(struct scsi_cmnd *cmd);
 
 extern void *scsi_kmap_atomic_sg(struct scatterlist *sg, int sg_count,
Signature · 1 line
2.16.4
#17 lore raw

Re: [PATCH RFC v3 03/41] scsi: Implement scsi_cmd_is_reserved()

drivers/scsi/scsi_lib.c EXPORT_SYMBOL_GPL(scsi_get_reserved_cmd); · line 1939

Quoted 11 lines · Hannes Reinecke

void scsi_put_reserved_cmd(struct scsi_cmnd *scmd) {

drivers/scsi/scsi_lib.c +5 -2
-	struct request *rq = blk_mq_rq_from_pdu(scmd);
+	struct request *rq;
 
-	blk_mq_free_request(rq);
+	if (scmd && scsi_cmd_is_reserved(scmd)) {
+		rq = blk_mq_rq_from_pdu(scmd);
+		blk_mq_free_request(rq);
+	}
 }

The above looks weird to me. Why to tolerate that a caller passes NULL as argument to this function? Additionally, wouldn't a WARN_ON_ONCE(!scsi_cmd_is_reserved(scmd)) be more appropriate instead of the if (scsi_cmd_is_reserved(scmd)) { ... } ?

Thanks,

Bart.

#18 lore raw

Re: [PATCH RFC v3 03/41] scsi: Implement scsi_cmd_is_reserved()

In reply to Hannes Reinecke
Quoted 3 lines · Hannes Reinecke

Add function to check if a SCSI command originates from the reserved tag pool and update scsi_put_reserved_cmd() to only free commands if they originate from the reserved tag pool.

The SCSI bits should go into the previous patch. The block layer bits should be a separate prep patch before that.

Quoted 4 lines

+/**
+ * blk_mq_rq_is_reserved - Check for reserved request
+ *
+ * @rq: Request to check

No empty line before the parameter description, please.

drivers/scsi/scsi_lib.c EXPORT_SYMBOL_GPL(scsi_get_reserved_cmd); · line 1938

Quoted 9 lines

*/
void scsi_put_reserved_cmd(struct scsi_cmnd *scmd) {

drivers/scsi/scsi_lib.c +5 -0
+	struct request *rq;
 
+	if (scmd && scsi_cmd_is_reserved(scmd)) {
+		rq = blk_mq_rq_from_pdu(scmd);
+		blk_mq_free_request(rq);
+	}

The check looks weird. Passing a NULL cmnd here seems like an API abuse to start with, and !scsi_cmd_is_reserved should at best be a WARN_ON_ONCE.

So I think this should just be something like:

void scsi_put_reserved_cmd(struct scsi_cmnd *scmd)

{
	WARN_ON_ONCE(!scsi_cmd_is_reserved(scmd));
	blk_mq_free_request(blk_mq_rq_from_pdu(scmd));
}

include/scsi/scsi_cmnd.h static inline struct scsi_driver *scsi_cmd_to_driver(struct scsi_cmnd *cmd) · line 172

Quoted 11 lines
include/scsi/scsi_cmnd.h +11 -0
+/**
+ * scsi_cmd_is_reserved - check for reserved scsi command
+ * @scmd: command to check
+ *
+ * Returns true if @scmd originates from the reserved tag pool.
+ */
+static inline bool scsi_cmd_is_reserved(struct scsi_cmnd *scmd)
+{
+	struct request *rq = blk_mq_rq_from_pdu(scmd);
+
+	return blk_mq_rq_is_reserved(rq);

Can be shortened to:

	return blk_mq_rq_is_reserved(blk_mq_rq_from_pdu(scmd));
#19 lore raw

Re: [PATCH RFC v3 03/41] scsi: Implement scsi_cmd_is_reserved()

Quoted 8 lines · Christoph Hellwig
Quoted 3 lines · Hannes Reinecke

Add function to check if a SCSI command originates from the reserved tag pool and update scsi_put_reserved_cmd() to only free commands if they originate from the reserved tag pool.

The SCSI bits should go into the previous patch. The block layer bits should be a separate prep patch before that.

Okay.

Quoted 7 lines
Quoted 4 lines

+/**
+ * blk_mq_rq_is_reserved - Check for reserved request
+ *
+ * @rq: Request to check

No empty line before the parameter description, please.

Okay

Quoted 14 lines

drivers/scsi/scsi_lib.c EXPORT_SYMBOL_GPL(scsi_get_reserved_cmd); · line 1938

Quoted 9 lines
   */
  void scsi_put_reserved_cmd(struct scsi_cmnd *scmd)
  {
drivers/scsi/scsi_lib.c +5 -0
+	struct request *rq;
 
+	if (scmd && scsi_cmd_is_reserved(scmd)) {
+		rq = blk_mq_rq_from_pdu(scmd);
+		blk_mq_free_request(rq);
+	}

The check looks weird. Passing a NULL cmnd here seems like an API abuse to start with, and !scsi_cmd_is_reserved should at best be a WARN_ON_ONCE.

Quoted 8 lines

So I think this should just be something like:

void scsi_put_reserved_cmd(struct scsi_cmnd *scmd)

{
	WARN_ON_ONCE(!scsi_cmd_is_reserved(scmd));
	blk_mq_free_request(blk_mq_rq_from_pdu(scmd));
}

Will do.

Quoted 14 lines

include/scsi/scsi_cmnd.h static inline struct scsi_driver *scsi_cmd_to_driver(struct scsi_cmnd *cmd) · line 172

Quoted 11 lines
include/scsi/scsi_cmnd.h +11 -0
+/**
+ * scsi_cmd_is_reserved - check for reserved scsi command
+ * @scmd: command to check
+ *
+ * Returns true if @scmd originates from the reserved tag pool.
+ */
+static inline bool scsi_cmd_is_reserved(struct scsi_cmnd *scmd)
+{
+	struct request *rq = blk_mq_rq_from_pdu(scmd);
+
+	return blk_mq_rq_is_reserved(rq);

Can be shortened to:

Quoted 2 lines
	return blk_mq_rq_is_reserved(blk_mq_rq_from_pdu(scmd));

Same here.

Will be updating the patch.

Cheers,

Hannes

Signature · 4 lines
Dr. Hannes Reinecke            Teamlead Storage & Networking
[email protected]                               +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer
#20 lore raw

RFC PATCH v3 4/41 csiostor: use reserved command for LUN reset

In reply to Hannes Reinecke

When issuing a LUN reset we should be using a reserved command to avoid overwriting the original command.

---
 drivers/scsi/csiostor/csio_init.c |  1 +
 drivers/scsi/csiostor/csio_scsi.c | 48 +++++++++++++++++++++++----------------
 2 files changed, 30 insertions(+), 19 deletions(-)
drivers/scsi/csiostor/csio_init.c +1 -0
index 8dea7d53788a..5e1b0a24caf6 100644
@@ -622,6 +622,7 @@ csio_shost_init(struct csio_hw *hw, struct device *dev,
 	ln->dev_num = (shost->host_no << 16);
 
 	shost->can_queue = CSIO_MAX_QUEUE;
+	shost->nr_reserved_cmds = 1;
 	shost->this_id = -1;
 	shost->unique_id = shost->host_no;
 	shost->max_cmd_len = 16; /* Max CDB length supported */
drivers/scsi/csiostor/csio_scsi.c +29 -19

131 lines not on this page · raw message

Signature · 1 line
2.16.4
#21 lore raw

Re: [PATCH RFC v3 04/41] csiostor: use reserved command for LUN reset

Quoted 14 lines · Hannes Reinecke

When issuing a LUN reset we should be using a reserved command to avoid overwriting the original command.

---
 drivers/scsi/csiostor/csio_init.c |  1 +
 drivers/scsi/csiostor/csio_scsi.c | 48 +++++++++++++++++++++++----------------
 2 files changed, 30 insertions(+), 19 deletions(-)
drivers/scsi/csiostor/csio_init.c +0 -0
index 8dea7d53788a..5e1b0a24caf6 100644

drivers/scsi/csiostor/csio_init.c csio_shost_init(struct csio_hw *hw, struct device *dev, · line 625

Quoted 4 lines
drivers/scsi/csiostor/csio_init.c +1 -0
@@ -622,6 +622,7 @@ csio_shost_init(struct csio_hw *hw, struct device *dev,
 	ln->dev_num = (shost->host_no << 16);
 
 	shost->can_queue = CSIO_MAX_QUEUE;
+	shost->nr_reserved_cmds = 1;

->can_queue isn't increased by 1 given CSIO_MAX_QUEUE isn't changed, so setting shost->nr_reserved_cmds as 1 will cause io queue depth reduced by 1, that is supposed to not happen.

Any conversion not increasing ->can_queue should have this same issue, please check other conversions.

thanks,
Ming

#22 lore raw

Re: [PATCH RFC v3 04/41] csiostor: use reserved command for LUN reset

Quoted 17 lines · Ming Lei
Quoted 16 lines · Hannes Reinecke

When issuing a LUN reset we should be using a reserved command to avoid overwriting the original command.

---
  drivers/scsi/csiostor/csio_init.c |  1 +
  drivers/scsi/csiostor/csio_scsi.c | 48 +++++++++++++++++++++++----------------
  2 files changed, 30 insertions(+), 19 deletions(-)
drivers/scsi/csiostor/csio_init.c +0 -0
index 8dea7d53788a..5e1b0a24caf6 100644
@@ -622,6 +622,7 @@ csio_shost_init(struct csio_hw *hw, struct device *dev,
  	ln->dev_num = (shost->host_no << 16);
 
Quoted 7 lines

drivers/scsi/csiostor/csio_init.c csio_shost_init(struct csio_hw *hw, struct device *dev, · line 625

Quoted 2 lines
drivers/scsi/csiostor/csio_init.c +1 -0
  	shost->can_queue = CSIO_MAX_QUEUE;
+	shost->nr_reserved_cmds = 1;

->can_queue isn't increased by 1 given CSIO_MAX_QUEUE isn't changed, so setting shost->nr_reserved_cmds as 1 will cause io queue depth reduced by 1, that is supposed to not happen.

We cannot increase MAX_QUEUE arbitrarily as this is a compile time variable, which seems to relate to a hardware setting.

But I can see to update the reserved command functionality for allowing to fetch commands from the normal I/O tag pool; in the case of LUN reset it shouldn't make much of a difference as the all I/O is quiesced anyway.

Cheers,

Hannes

Signature · 4 lines
Dr. Hannes Reinecke            Teamlead Storage & Networking
[email protected]                               +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer
#23 lore raw

Re: [PATCH RFC v3 04/41] csiostor: use reserved command for LUN reset

Quoted 24 lines · Hannes Reinecke
Quoted 10 lines · Ming Lei
Quoted 9 lines · Hannes Reinecke

When issuing a LUN reset we should be using a reserved command to avoid overwriting the original command.

---
  drivers/scsi/csiostor/csio_init.c |  1 +
  drivers/scsi/csiostor/csio_scsi.c | 48 +++++++++++++++++++++++----------------
  2 files changed, 30 insertions(+), 19 deletions(-)
Quoted 13 lines

drivers/scsi/csiostor/csio_init.c csio_shost_init(struct csio_hw *hw, struct device *dev, · line 624

Quoted 8 lines
drivers/scsi/csiostor/csio_init.c +1 -0
index 8dea7d53788a..5e1b0a24caf6 100644
@@ -622,6 +622,7 @@ csio_shost_init(struct csio_hw *hw, struct device *dev,
  	ln->dev_num = (shost->host_no << 16);
  	shost->can_queue = CSIO_MAX_QUEUE;
+	shost->nr_reserved_cmds = 1;

->can_queue isn't increased by 1 given CSIO_MAX_QUEUE isn't changed, so setting shost->nr_reserved_cmds as 1 will cause io queue depth reduced by 1, that is supposed to not happen.

Quoted 6 lines

We cannot increase MAX_QUEUE arbitrarily as this is a compile time variable, which seems to relate to a hardware setting.

But I can see to update the reserved command functionality for allowing to fetch commands from the normal I/O tag pool; in the case of LUN reset it shouldn't make much of a difference as the all I/O is quiesced anyway.

It isn't related with reset.

This patch reduces active IO queue depth by 1 anytime no matter there is reset or not, and this way may cause performance regression.

Thanks,
Ming

#24 lore raw

Re: [PATCH RFC v3 04/41] csiostor: use reserved command for LUN reset

Quoted 11 lines · Ming Lei
Quoted 6 lines

We cannot increase MAX_QUEUE arbitrarily as this is a compile time variable, which seems to relate to a hardware setting.

But I can see to update the reserved command functionality for allowing to fetch commands from the normal I/O tag pool; in the case of LUN reset it shouldn't make much of a difference as the all I/O is quiesced anyway.

It isn't related with reset.

This patch reduces active IO queue depth by 1 anytime no matter there is reset or not, and this way may cause performance regression.

But isn't it the right thing to do? How else do we guarantee that there always is a tag available for the LU reset?

#25 lore raw

Re: [PATCH RFC v3 04/41] csiostor: use reserved command for LUN reset

Quoted 8 lines · Christoph Hellwig
Quoted 7 lines · Ming Lei
Quoted 6 lines

We cannot increase MAX_QUEUE arbitrarily as this is a compile time variable, which seems to relate to a hardware setting.

But I can see to update the reserved command functionality for allowing to fetch commands from the normal I/O tag pool; in the case of LUN reset it shouldn't make much of a difference as the all I/O is quiesced anyway.

Quoted 7 lines
Quoted 4 lines

It isn't related with reset.

This patch reduces active IO queue depth by 1 anytime no matter there is reset or not, and this way may cause performance regression.

But isn't it the right thing to do? How else do we guarantee that there always is a tag available for the LU reset?

If that is case, some of these patches should be bug-fix, but nothing about this kind of comment is provided. If it is true, please update the commit log and explain the current issue in detail, such as, what is the side-effect of 'overwriting the original command'?

And we might need to backport it to stable tree because storage error recovery is very key function.

Even though it is true, still not sure if this patch is the correct way to fix the issue cause IO performance drop might be caused.

Thanks,
Ming

#26 lore raw

Re: [PATCH RFC v3 04/41] csiostor: use reserved command for LUN reset

Quoted 22 lines · Ming Lei
Quoted 8 lines · Christoph Hellwig
Quoted 7 lines · Ming Lei
Quoted 6 lines

We cannot increase MAX_QUEUE arbitrarily as this is a compile time variable, which seems to relate to a hardware setting.

But I can see to update the reserved command functionality for allowing to fetch commands from the normal I/O tag pool; in the case of LUN reset it shouldn't make much of a difference as the all I/O is quiesced anyway.

Quoted 7 lines
Quoted 4 lines

It isn't related with reset.

This patch reduces active IO queue depth by 1 anytime no matter there is reset or not, and this way may cause performance regression.

But isn't it the right thing to do? How else do we guarantee that there always is a tag available for the LU reset?

If that is case, some of these patches should be bug-fix, but nothing about this kind of comment is provided. If it is true, please update the commit log and explain the current issue in detail, such as, what is the side-effect of 'overwriting the original command'?

Quoted 5 lines

And we might need to backport it to stable tree because storage error recovery is very key function.

Quoted 1 line

Even though it is true, still not sure if this patch is the correct

way to fix the issue cause IO performance drop might be caused.

You can't have it both ways.

The underlying problem is this:
The csiostor driver (and several others, too) require a valid hardware tag to send a LU reset command. Currently it tries to allocate a tag from the pool of free hardware tags.
However, experience shows that the majority of cases (in my personal experience _all_ of the cases) where we ever entered the error handler are due to command timeouts. If now all commands timed out, the tag space is full and we cannot get a free tag to send the LU reset command. Hence LU reset will currently fail in this case.
With the patchset we will always ensure to have at least one free tag
such that we can send the LU reset command. But, as correctly noted,
it will reduce the available tagspace and possibly reduce the performance.
But you really can have it both ways. Either you go for max performance and have the risk of starving the error handler, or you go for reliability and accept a (slightly) lower performance.
And, btw, I'm not sure if one could even measure the performance impact.
csiostor has 2048 tags per HBA with a 10G FCoE link. So it would require a latency of less than 8us with 4k I/O to saturate the HBA; for everything slower we wouldn't be seeing anything.

Cheers,

Hannes

Signature · 4 lines
Dr. Hannes Reinecke            Teamlead Storage & Networking
[email protected]                               +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer
#27 lore raw

Re: [PATCH RFC v3 04/41] csiostor: use reserved command for LUN reset

In reply to Christoph Hellwig
Quoted 8 lines · Christoph Hellwig
Quoted 7 lines · Ming Lei
Quoted 6 lines

We cannot increase MAX_QUEUE arbitrarily as this is a compile time variable, which seems to relate to a hardware setting.

But I can see to update the reserved command functionality for allowing to fetch commands from the normal I/O tag pool; in the case of LUN reset it shouldn't make much of a difference as the all I/O is quiesced anyway.

Quoted 8 lines
Quoted 4 lines

It isn't related with reset.

This patch reduces active IO queue depth by 1 anytime no matter there is reset or not, and this way may cause performance regression.

But isn't it the right thing to do? How else do we guarantee that there always is a tag available for the LU reset?

Precisely. One could argue that this is an issue with the current driver, too; if all tags have timed-out there is no way how we can send a LUN reset even now. Hence we need to reserve a tag for us to reliably send a LUN reset. And this was precisely the problem what sparked off this entire patchset; some drivers require a valid tag to send internal, non SCSI commands to the hardware.
And with the current design it requires some really ugly hacks to make this to work.

Cheers,

Hannes

Signature · 4 lines
Dr. Hannes Reinecke            Teamlead Storage & Networking
[email protected]                               +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer
#28 lore raw

Re: [PATCH RFC v3 04/41] csiostor: use reserved command for LUN reset

Quoted 17 lines · Hannes Reinecke
Quoted 8 lines · Christoph Hellwig
Quoted 7 lines · Ming Lei
Quoted 6 lines

We cannot increase MAX_QUEUE arbitrarily as this is a compile time variable, which seems to relate to a hardware setting.

But I can see to update the reserved command functionality for allowing to fetch commands from the normal I/O tag pool; in the case of LUN reset it shouldn't make much of a difference as the all I/O is quiesced anyway.

Quoted 8 lines
Quoted 4 lines

It isn't related with reset.

This patch reduces active IO queue depth by 1 anytime no matter there is reset or not, and this way may cause performance regression.

But isn't it the right thing to do? How else do we guarantee that there always is a tag available for the LU reset?

Quoted 7 lines

Precisely. One could argue that this is an issue with the current driver, too; if all tags have timed-out there is no way how we can send a LUN reset even now. Hence we need to reserve a tag for us to reliably send a LUN reset.
And this was precisely the problem what sparked off this entire patchset; some drivers require a valid tag to send internal, non SCSI commands to the hardware.

Could you explain a bit how you conclude that csio_scsi reset hander has to use one unique tag? At least we don't allocate request from block layer for ioctl(SG_SCSI_RESET), see scsi_ioctl_reset(). Also this patch doesn't use the reserved rq->tag too.

Quoted 2 lines

And with the current design it requires some really ugly hacks to make this to work.

You also don't explain how csio_eh_lun_reset_handler() is broken and where the ugly hack is in csio scsi too, and how this patch fixes the issue, could you document the exact reason in the commit log?

Thanks,
Ming

#29 lore raw

Re: [PATCH RFC v3 04/41] csiostor: use reserved command for LUN reset

Quoted 31 lines · Ming Lei
Quoted 17 lines · Hannes Reinecke
Quoted 8 lines · Christoph Hellwig
Quoted 7 lines · Ming Lei
Quoted 6 lines

We cannot increase MAX_QUEUE arbitrarily as this is a compile time variable, which seems to relate to a hardware setting.

But I can see to update the reserved command functionality for allowing to fetch commands from the normal I/O tag pool; in the case of LUN reset it shouldn't make much of a difference as the all I/O is quiesced anyway.

Quoted 8 lines
Quoted 4 lines

It isn't related with reset.

This patch reduces active IO queue depth by 1 anytime no matter there is reset or not, and this way may cause performance regression.

But isn't it the right thing to do? How else do we guarantee that there always is a tag available for the LU reset?

Quoted 7 lines

Precisely. One could argue that this is an issue with the current driver, too; if all tags have timed-out there is no way how we can send a LUN reset even now. Hence we need to reserve a tag for us to reliably send a LUN reset.
And this was precisely the problem what sparked off this entire patchset; some drivers require a valid tag to send internal, non SCSI commands to the hardware.

Could you explain a bit how you conclude that csio_scsi reset hander has to use one unique tag? At least we don't allocate request from block layer for ioctl(SG_SCSI_RESET), see scsi_ioctl_reset(). Also this patch doesn't use the reserved rq->tag too.

Quoted 7 lines
Quoted 2 lines

And with the current design it requires some really ugly hacks to make this to work.

You also don't explain how csio_eh_lun_reset_handler() is broken and where the ugly hack is in csio scsi too, and how this patch fixes the issue, could you document the exact reason in the commit log?

The problem is the ioctl path.
When issuing TMF commands from the ioctl path we currently do not have a valid SCSI command to pass to the various SCSI EH functions.
This requires the SCSI LLDDs to check for every EH function whether the passed in SCSI command is valid (ie coming from SCSI EH), or a made up one coming from the ioctl path. And having to code various ways to work around this issue.

With this patchset I'm implementing a standardized way how these functions can be coded. By using a reserved command the EH functions and the driver internal command handling will always have a valid command, so the workarounds can be removed. And this is also the first step to my final SCSI EH rewrite, where I'm planning to move the SCSI EH functions from having the SCSI command as argument to the respective object (ie LU reset will be having a SCSI device as argument, host reset a SCSI host etc.)

Cheers,

Hannes

Signature · 4 lines
Dr. Hannes Reinecke            Teamlead Storage & Networking
[email protected]                               +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer
#30 lore raw

Re: [PATCH RFC v3 04/41] csiostor: use reserved command for LUN reset

Quoted 39 lines · Hannes Reinecke
Quoted 31 lines · Ming Lei
Quoted 17 lines · Hannes Reinecke
Quoted 8 lines · Christoph Hellwig
Quoted 7 lines · Ming Lei
Quoted 6 lines

We cannot increase MAX_QUEUE arbitrarily as this is a compile time variable, which seems to relate to a hardware setting.

But I can see to update the reserved command functionality for allowing to fetch commands from the normal I/O tag pool; in the case of LUN reset it shouldn't make much of a difference as the all I/O is quiesced anyway.

Quoted 8 lines
Quoted 4 lines

It isn't related with reset.

This patch reduces active IO queue depth by 1 anytime no matter there is reset or not, and this way may cause performance regression.

But isn't it the right thing to do? How else do we guarantee that there always is a tag available for the LU reset?

Quoted 7 lines

Precisely. One could argue that this is an issue with the current driver, too; if all tags have timed-out there is no way how we can send a LUN reset even now. Hence we need to reserve a tag for us to reliably send a LUN reset.
And this was precisely the problem what sparked off this entire patchset; some drivers require a valid tag to send internal, non SCSI commands to the hardware.

Could you explain a bit how you conclude that csio_scsi reset hander has to use one unique tag? At least we don't allocate request from block layer for ioctl(SG_SCSI_RESET), see scsi_ioctl_reset(). Also this patch doesn't use the reserved rq->tag too.

Quoted 7 lines
Quoted 2 lines

And with the current design it requires some really ugly hacks to make this to work.

You also don't explain how csio_eh_lun_reset_handler() is broken and where the ugly hack is in csio scsi too, and how this patch fixes the issue, could you document the exact reason in the commit log?

Quoted 6 lines

The problem is the ioctl path.
When issuing TMF commands from the ioctl path we currently do not have a valid SCSI command to pass to the various SCSI EH functions. This requires the SCSI LLDDs to check for every EH function whether the passed in SCSI command is valid (ie coming from SCSI EH), or a made up one coming from the ioctl path.

Could you point out where the check is in csio driver?

Quoted 6 lines

And having to code various ways to work around this issue.

With this patchset I'm implementing a standardized way how these functions can be coded. By using a reserved command the EH functions and the driver internal command handling will always have a valid command, so the workarounds can be removed.

It depends if every HBA really requires unique tag for sending reset command.

If some HBAs don't require unique tag for resetting device or host, the reserved tag is wasted.

thanks,
Ming

#31 lore raw

Re: [PATCH RFC v3 04/41] csiostor: use reserved command for LUN reset

Quoted 40 lines · Ming Lei
Quoted 39 lines · Hannes Reinecke
Quoted 38 lines · Ming Lei
Quoted 17 lines · Hannes Reinecke
Quoted 8 lines · Christoph Hellwig
Quoted 7 lines · Ming Lei
Quoted 6 lines

We cannot increase MAX_QUEUE arbitrarily as this is a compile time variable, which seems to relate to a hardware setting.

But I can see to update the reserved command functionality for allowing to fetch commands from the normal I/O tag pool; in the case of LUN reset it shouldn't make much of a difference as the all I/O is quiesced anyway.

Quoted 8 lines
Quoted 4 lines

It isn't related with reset.

This patch reduces active IO queue depth by 1 anytime no matter there is reset or not, and this way may cause performance regression.

But isn't it the right thing to do? How else do we guarantee that there always is a tag available for the LU reset?

Quoted 7 lines

Precisely. One could argue that this is an issue with the current driver, too; if all tags have timed-out there is no way how we can send a LUN reset even now. Hence we need to reserve a tag for us to reliably send a LUN reset.
And this was precisely the problem what sparked off this entire patchset; some drivers require a valid tag to send internal, non SCSI commands to the hardware.

Could you explain a bit how you conclude that csio_scsi reset hander has to use one unique tag? At least we don't allocate request from block layer for ioctl(SG_SCSI_RESET), see scsi_ioctl_reset(). Also this patch doesn't use the reserved rq->tag too.

Quoted 2 lines

And with the current design it requires some really ugly hacks to make this to work.

You also don't explain how csio_eh_lun_reset_handler() is broken and where the ugly hack is in csio scsi too, and how this patch fixes the issue, could you document the exact reason in the commit log?

Quoted 9 lines
Quoted 6 lines

The problem is the ioctl path.
When issuing TMF commands from the ioctl path we currently do not have a valid SCSI command to pass to the various SCSI EH functions. This requires the SCSI LLDDs to check for every EH function whether the passed in SCSI command is valid (ie coming from SCSI EH), or a made up one coming from the ioctl path.

Could you point out where the check is in csio driver?

Okok, I see your point.

Indeed the csiostor driver doesn't use the 'tag' per se for submitting commands; it's just using the scsi command pointer as a tag to figure out if a completion has been send from the hw.

I'll be giving it a bit more thought, and will be dropping it for the next round (which will contain only the minimal changes to get the 'reserved_cmds' interface in).

Cheers,

Hannes

Signature · 4 lines
Dr. Hannes Reinecke            Teamlead Storage & Networking
[email protected]                               +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer
#32 lore raw

Re: [PATCH RFC v3 04/41] csiostor: use reserved command for LUN reset

Quoted 52 lines · Hannes Reinecke
Quoted 40 lines · Ming Lei
Quoted 39 lines · Hannes Reinecke
Quoted 38 lines · Ming Lei
Quoted 17 lines · Hannes Reinecke
Quoted 8 lines · Christoph Hellwig
Quoted 7 lines · Ming Lei
Quoted 6 lines

We cannot increase MAX_QUEUE arbitrarily as this is a compile time variable, which seems to relate to a hardware setting.

But I can see to update the reserved command functionality for allowing to fetch commands from the normal I/O tag pool; in the case of LUN reset it shouldn't make much of a difference as the all I/O is quiesced anyway.

Quoted 8 lines
Quoted 4 lines

It isn't related with reset.

This patch reduces active IO queue depth by 1 anytime no matter there is reset or not, and this way may cause performance regression.

But isn't it the right thing to do? How else do we guarantee that there always is a tag available for the LU reset?

Quoted 7 lines

Precisely. One could argue that this is an issue with the current driver, too; if all tags have timed-out there is no way how we can send a LUN reset even now. Hence we need to reserve a tag for us to reliably send a LUN reset.
And this was precisely the problem what sparked off this entire patchset; some drivers require a valid tag to send internal, non SCSI commands to the hardware.

Could you explain a bit how you conclude that csio_scsi reset hander has to use one unique tag? At least we don't allocate request from block layer for ioctl(SG_SCSI_RESET), see scsi_ioctl_reset(). Also this patch doesn't use the reserved rq->tag too.

Quoted 2 lines

And with the current design it requires some really ugly hacks to make this to work.

You also don't explain how csio_eh_lun_reset_handler() is broken and where the ugly hack is in csio scsi too, and how this patch fixes the issue, could you document the exact reason in the commit log?

Quoted 9 lines
Quoted 6 lines

The problem is the ioctl path.
When issuing TMF commands from the ioctl path we currently do not have a valid SCSI command to pass to the various SCSI EH functions. This requires the SCSI LLDDs to check for every EH function whether the passed in SCSI command is valid (ie coming from SCSI EH), or a made up one coming from the ioctl path.

Could you point out where the check is in csio driver?

Okok, I see your point.

Quoted 7 lines

Indeed the csiostor driver doesn't use the 'tag' per se for submitting commands; it's just using the scsi command pointer as a tag to figure out if a completion has been send from the hw.

I'll be giving it a bit more thought, and will be dropping it for the next round (which will contain only the minimal changes to get the 'reserved_cmds' interface in).

IMO, the 'reserved_cmds' interface is only needed in case that RESET command is transported from IO channel. Any HBA which has dedicated channel for sending RESET doesn't need this interface.

Thanks,
Ming

#33 lore raw

Re: [PATCH RFC v3 04/41] csiostor: use reserved command for LUN reset

In reply to Hannes Reinecke
Quoted 21 lines · Hannes Reinecke
Quoted 11 lines · Christoph Hellwig
Quoted 10 lines · Ming Lei
Quoted 9 lines

We cannot increase MAX_QUEUE arbitrarily as this is a compile time variable,
which seems to relate to a hardware setting.

But I can see to update the reserved command functionality for allowing to
fetch commands from the normal I/O tag pool; in the case of LUN reset it
shouldn't make much of a difference as the all I/O is quiesced anyway.

Quoted 9 lines
Quoted 5 lines

It isn't related with reset.

This patch reduces active IO queue depth by 1 anytime no matter there is reset
or not, and this way may cause performance regression.

But isn't it the right thing to do?  How else do we guarantee that there always is a tag available for the LU reset?

Quoted 9 lines

Precisely. One could argue that this is an issue with the current driver, too; if all tags have timed-out there is no way how we can send a LUN reset even now. Hence we need to reserve a tag for us to reliably send a LUN reset.
And this was precisely the problem what sparked off this entire patchset; some drivers require a valid tag to send internal, non SCSI commands to the hardware.
And with the current design it requires some really ugly hacks to make this to work.

Hi Hannes,

The above explanation seems incomplete to me. The code in drivers/scsi/scsi_error.c and several SCSI LLDs use scsi_eh_prep_cmnd() and scsi_eh_restore_cmnd() to reset a controller without allocating a new command. Has it been considered to use that approach in the csiostor driver?

Thanks,

Bart.

#34 lore raw

Re: [PATCH RFC v3 04/41] csiostor: use reserved command for LUN reset

Quoted 32 lines · Bart Van Assche
Quoted 21 lines · Hannes Reinecke
Quoted 11 lines · Christoph Hellwig
Quoted 10 lines · Ming Lei
Quoted 9 lines

We cannot increase MAX_QUEUE arbitrarily as this is a compile time variable,
which seems to relate to a hardware setting.

But I can see to update the reserved command functionality for allowing to
fetch commands from the normal I/O tag pool; in the case of LUN reset it
shouldn't make much of a difference as the all I/O is quiesced anyway.

Quoted 9 lines
Quoted 5 lines

It isn't related with reset.

This patch reduces active IO queue depth by 1 anytime no matter there is reset
or not, and this way may cause performance regression.

But isn't it the right thing to do?  How else do we guarantee that there always is a tag available for the LU reset?

Quoted 9 lines

Precisely. One could argue that this is an issue with the current driver, too; if all tags have timed-out there is no way how we can send a LUN reset even now. Hence we need to reserve a tag for us to reliably send a LUN reset.
And this was precisely the problem what sparked off this entire patchset; some drivers require a valid tag to send internal, non SCSI commands to the hardware.
And with the current design it requires some really ugly hacks to make this to work.

Quoted 8 lines

Hi Hannes,

The above explanation seems incomplete to me. The code in drivers/scsi/scsi_error.c and several SCSI LLDs use scsi_eh_prep_cmnd() and scsi_eh_restore_cmnd() to reset a controller without allocating a new command. Has it been considered to use that approach in the csiostor driver?

As outlined in the response to Ming, the problem is the ioctl path.
When called from ioctl we do _not_ have a valid command, hence drivers have to figure out if the command is valid (ie coming from SCSI EH), or invalid (ie coming from ioctl). I'm trying to unify both call paths by having the SCSI EH functions to always allocate a (reserved) SCSI command.

I have not moved this into the caller, as using a reserved command requires modifications of the driver itself (at least by setting 'nr_reserved_cmds').

Also, the above approach does not work when we run into a command timeout; which, from my experience, is the majority of cases.
For timeout commands we precisely can _not_ re-use the command tag, as the command itself is still assumed live somewhere (otherwise we could have aborted it, and we wouldn't have to call the EH functions...). But that means that any associated resources (like FC oxids) are still active on the wire, and re-using them would actually be a violating of the spec.

Cheers,

Hannes

Signature · 4 lines
Dr. Hannes Reinecke            Teamlead Storage & Networking
[email protected]                               +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer
#35 lore raw

RFC PATCH v3 5/41 scsi: add scsi_cmd_from_priv()

In reply to Hannes Reinecke

From: Hannes Reinecke <[email protected]>

Add a functon to retrieve the scsi_cmnd structure from the driver private allocation data.

---
 include/scsi/scsi_cmnd.h | 10 ++++++++++
 1 file changed, 10 insertions(+)
include/scsi/scsi_cmnd.h +10 -0
index 2cd894fbdcfa..d7bc368e1b4e 100644
@@ -153,6 +153,16 @@ static inline void *scsi_cmd_priv(struct scsi_cmnd *cmd)
 	return cmd + 1;
 }
 
+/*
+ * Return the scsi_cmnd structure located before the driver
+ * private allocation. Only works if cmd_size is set in the
+ * host template.
+ */
+static inline struct scsi_cmnd *scsi_cmd_from_priv(void *priv)
+{
+	return priv - sizeof(struct scsi_cmnd);
+}
+
 /* make sure not to use it with passthrough commands */
 static inline struct scsi_driver *scsi_cmd_to_driver(struct scsi_cmnd *cmd)
 {
Signature · 1 line
2.16.4
#36 lore raw

RFC PATCH v3 6/41 virtio_scsi: use reserved commands for TMF

In reply to Hannes Reinecke

From: Hannes Reinecke <[email protected]>

Set two commands aside for TMF, and use reserved commands to issue TMFs. With that we can drop the TMF memory pool.

---
 drivers/scsi/virtio_scsi.c | 105 ++++++++++++++++++---------------------------
 1 file changed, 41 insertions(+), 64 deletions(-)
drivers/scsi/virtio_scsi.c +41 -64
index 0e0910c5b942..26054c29d897 100644
@@ -35,10 +35,10 @@
 #define VIRTIO_SCSI_MEMPOOL_SZ 64
 #define VIRTIO_SCSI_EVENT_LEN 8
 #define VIRTIO_SCSI_VQ_BASE 2
+#define VIRTIO_SCSI_RESERVED_CMDS 2
 
 /* Command queue element */
 struct virtio_scsi_cmd {
-	struct scsi_cmnd *sc;
 	struct completion *comp;
 	union {
 		struct virtio_scsi_cmd_req       cmd;
@@ -86,9 +86,6 @@ struct virtio_scsi {
 	struct virtio_scsi_vq req_vqs[];
 };
 
-static struct kmem_cache *virtscsi_cmd_cache;
-static mempool_t *virtscsi_cmd_pool;
-
 static inline struct Scsi_Host *virtio_scsi_host(struct virtio_device *vdev)
 {
 	return vdev->priv;
@@ -108,7 +105,7 @@ static void virtscsi_compute_resid(struct scsi_cmnd *sc, u32 resid)
 static void virtscsi_complete_cmd(struct virtio_scsi *vscsi, void *buf)
 {
 	struct virtio_scsi_cmd *cmd = buf;
-	struct scsi_cmnd *sc = cmd->sc;
+	struct scsi_cmnd *sc = scsi_cmd_from_priv(cmd);
 	struct virtio_scsi_cmd_resp *resp = &cmd->resp.cmd;
 
 	dev_dbg(&sc->device->sdev_gendev,
@@ -406,7 +403,7 @@ static int __virtscsi_add_cmd(struct virtqueue *vq,
 			    struct virtio_scsi_cmd *cmd,
 			    size_t req_size, size_t resp_size)
 {
-	struct scsi_cmnd *sc = cmd->sc;
+	struct scsi_cmnd *sc = scsi_cmd_from_priv(cmd);
 	struct scatterlist *sgs[6], req, resp;
 	struct sg_table *out, *in;
 	unsigned out_num = 0, in_num = 0;
@@ -557,8 +554,6 @@ static int virtscsi_queuecommand(struct Scsi_Host *shost,
 	dev_dbg(&sc->device->sdev_gendev,
 		"cmd %p CDB: %#02x\n", sc, sc->cmnd[0]);
 
-	cmd->sc = sc;
-
 	BUG_ON(sc->cmd_len > VIRTIO_SCSI_CDB_SIZE);
 
 #ifdef CONFIG_BLK_DEV_INTEGRITY
@@ -590,17 +585,17 @@ static int virtscsi_queuecommand(struct Scsi_Host *shost,
 static int virtscsi_tmf(struct virtio_scsi *vscsi, struct virtio_scsi_cmd *cmd)
 {
 	DECLARE_COMPLETION_ONSTACK(comp);
-	int ret = FAILED;
 
 	cmd->comp = &comp;
+
 	if (virtscsi_add_cmd(&vscsi->ctrl_vq, cmd,
 			      sizeof cmd->req.tmf, sizeof cmd->resp.tmf, true) < 0)
-		goto out;
+		return FAILED;
 
 	wait_for_completion(&comp);
 	if (cmd->resp.tmf.response == VIRTIO_SCSI_S_OK ||
 	    cmd->resp.tmf.response == VIRTIO_SCSI_S_FUNCTION_SUCCEEDED)
-		ret = SUCCESS;
+		return SUCCESS;
 
 	/*
 	 * The spec guarantees that all requests related to the TMF have
@@ -613,33 +608,36 @@ static int virtscsi_tmf(struct virtio_scsi *vscsi, struct virtio_scsi_cmd *cmd)
 	 * REQ_ATOM_COMPLETE has been set.
 	 */
 	virtscsi_poll_requests(vscsi);
-
-out:
-	mempool_free(cmd, virtscsi_cmd_pool);
-	return ret;
+	return FAILED;
 }
 
 static int virtscsi_device_reset(struct scsi_cmnd *sc)
 {
+	struct scsi_device *sdev = sc->device;
+	struct scsi_cmnd *reset_sc;
 	struct virtio_scsi *vscsi = shost_priv(sc->device->host);
 	struct virtio_scsi_cmd *cmd;
+	int rc;
 
-	sdev_printk(KERN_INFO, sc->device, "device reset\n");
-	cmd = mempool_alloc(virtscsi_cmd_pool, GFP_NOIO);
-	if (!cmd)
+	sdev_printk(KERN_INFO, sdev, "device reset\n");
+	reset_sc = scsi_get_reserved_cmd(sdev, DMA_NONE);
+	if (!reset_sc)
 		return FAILED;
-
+	cmd = scsi_cmd_priv(reset_sc);
 	memset(cmd, 0, sizeof(*cmd));
 	cmd->req.tmf = (struct virtio_scsi_ctrl_tmf_req){
 		.type = VIRTIO_SCSI_T_TMF,
 		.subtype = cpu_to_virtio32(vscsi->vdev,
 					     VIRTIO_SCSI_T_TMF_LOGICAL_UNIT_RESET),
 		.lun[0] = 1,
-		.lun[1] = sc->device->id,
-		.lun[2] = (sc->device->lun >> 8) | 0x40,
-		.lun[3] = sc->device->lun & 0xff,
+		.lun[1] = sdev->id,
+		.lun[2] = (sdev->lun >> 8) | 0x40,
+		.lun[3] = sdev->lun & 0xff,
 	};
-	return virtscsi_tmf(vscsi, cmd);
+	rc = virtscsi_tmf(vscsi, cmd);
+	scsi_put_reserved_cmd(reset_sc);
+
+	return rc;
 }
 
 static int virtscsi_device_alloc(struct scsi_device *sdevice)
@@ -679,25 +677,31 @@ static int virtscsi_change_queue_depth(struct scsi_device *sdev, int qdepth)
 
 static int virtscsi_abort(struct scsi_cmnd *sc)
 {
-	struct virtio_scsi *vscsi = shost_priv(sc->device->host);
+	struct scsi_device *sdev = sc->device;
+	struct scsi_cmnd *reset_sc;
+	struct virtio_scsi *vscsi = shost_priv(sdev->host);
 	struct virtio_scsi_cmd *cmd;
+	int rc;
 
 	scmd_printk(KERN_INFO, sc, "abort\n");
-	cmd = mempool_alloc(virtscsi_cmd_pool, GFP_NOIO);
-	if (!cmd)
+	reset_sc = scsi_get_reserved_cmd(sdev, DMA_NONE);
+	if (!reset_sc)
 		return FAILED;
+	cmd = scsi_cmd_priv(reset_sc);
 
 	memset(cmd, 0, sizeof(*cmd));
 	cmd->req.tmf = (struct virtio_scsi_ctrl_tmf_req){
 		.type = VIRTIO_SCSI_T_TMF,
 		.subtype = VIRTIO_SCSI_T_TMF_ABORT_TASK,
 		.lun[0] = 1,
-		.lun[1] = sc->device->id,
-		.lun[2] = (sc->device->lun >> 8) | 0x40,
-		.lun[3] = sc->device->lun & 0xff,
+		.lun[1] = sdev->id,
+		.lun[2] = (sdev->lun >> 8) | 0x40,
+		.lun[3] = sdev->lun & 0xff,
 		.tag = cpu_to_virtio64(vscsi->vdev, (unsigned long)sc),
 	};
-	return virtscsi_tmf(vscsi, cmd);
+	rc = virtscsi_tmf(vscsi, cmd);
+	scsi_put_reserved_cmd(reset_sc);
+	return rc;
 }
 
 static int virtscsi_map_queues(struct Scsi_Host *shost)
@@ -865,6 +869,11 @@ static int virtscsi_probe(struct virtio_device *vdev)
 		goto virtscsi_init_failed;
 
 	shost->can_queue = virtqueue_get_vring_size(vscsi->req_vqs[0].vq);
+	shost->can_queue -= VIRTIO_SCSI_RESERVED_CMDS;
+	if (shost->can_queue <= 0) {
+		err = -ENOMEM;
+		goto scsi_add_host_failed;
+	}
 
 	cmd_per_lun = virtscsi_config_get(vdev, cmd_per_lun) ?: 1;
 	shost->cmd_per_lun = min_t(u32, cmd_per_lun, shost->can_queue);
@@ -878,6 +887,7 @@ static int virtscsi_probe(struct virtio_device *vdev)
 	shost->max_channel = 0;
 	shost->max_cmd_len = VIRTIO_SCSI_CDB_SIZE;
 	shost->nr_hw_queues = num_queues;
+	shost->nr_reserved_cmds = VIRTIO_SCSI_RESERVED_CMDS;
 
 #ifdef CONFIG_BLK_DEV_INTEGRITY
 	if (virtio_has_feature(vdev, VIRTIO_SCSI_F_T10_PI)) {
@@ -979,45 +989,12 @@ static struct virtio_driver virtio_scsi_driver = {
 
 static int __init init(void)
 {
-	int ret = -ENOMEM;
-
-	virtscsi_cmd_cache = KMEM_CACHE(virtio_scsi_cmd, 0);
-	if (!virtscsi_cmd_cache) {
-		pr_err("kmem_cache_create() for virtscsi_cmd_cache failed\n");
-		goto error;
-	}
-
-
-	virtscsi_cmd_pool =
-		mempool_create_slab_pool(VIRTIO_SCSI_MEMPOOL_SZ,
-					 virtscsi_cmd_cache);
-	if (!virtscsi_cmd_pool) {
-		pr_err("mempool_create() for virtscsi_cmd_pool failed\n");
-		goto error;
-	}
-	ret = register_virtio_driver(&virtio_scsi_driver);
-	if (ret < 0)
-		goto error;
-
-	return 0;
-
-error:
-	if (virtscsi_cmd_pool) {
-		mempool_destroy(virtscsi_cmd_pool);
-		virtscsi_cmd_pool = NULL;
-	}
-	if (virtscsi_cmd_cache) {
-		kmem_cache_destroy(virtscsi_cmd_cache);
-		virtscsi_cmd_cache = NULL;
-	}
-	return ret;
+	return register_virtio_driver(&virtio_scsi_driver);
 }
 
 static void __exit fini(void)
 {
 	unregister_virtio_driver(&virtio_scsi_driver);
-	mempool_destroy(virtscsi_cmd_pool);
-	kmem_cache_destroy(virtscsi_cmd_cache);
 }
 module_init(init);
 module_exit(fini);
Signature · 1 line
2.16.4
#37 lore raw

Re: [PATCH RFC v3 06/41] virtio_scsi: use reserved commands for TMF

Quoted 173 lines · Hannes Reinecke

From: Hannes Reinecke <[email protected]>

Set two commands aside for TMF, and use reserved commands to issue TMFs. With that we can drop the TMF memory pool.

---
 drivers/scsi/virtio_scsi.c | 105 ++++++++++++++++++---------------------------
 1 file changed, 41 insertions(+), 64 deletions(-)
drivers/scsi/virtio_scsi.c +34 -30
index 0e0910c5b942..26054c29d897 100644
@@ -35,10 +35,10 @@
 #define VIRTIO_SCSI_MEMPOOL_SZ 64
 #define VIRTIO_SCSI_EVENT_LEN 8
 #define VIRTIO_SCSI_VQ_BASE 2
+#define VIRTIO_SCSI_RESERVED_CMDS 2
 
 /* Command queue element */
 struct virtio_scsi_cmd {
-	struct scsi_cmnd *sc;
 	struct completion *comp;
 	union {
 		struct virtio_scsi_cmd_req       cmd;
@@ -86,9 +86,6 @@ struct virtio_scsi {
 	struct virtio_scsi_vq req_vqs[];
 };
 
-static struct kmem_cache *virtscsi_cmd_cache;
-static mempool_t *virtscsi_cmd_pool;
-
 static inline struct Scsi_Host *virtio_scsi_host(struct virtio_device *vdev)
 {
 	return vdev->priv;
@@ -108,7 +105,7 @@ static void virtscsi_compute_resid(struct scsi_cmnd *sc, u32 resid)
 static void virtscsi_complete_cmd(struct virtio_scsi *vscsi, void *buf)
 {
 	struct virtio_scsi_cmd *cmd = buf;
-	struct scsi_cmnd *sc = cmd->sc;
+	struct scsi_cmnd *sc = scsi_cmd_from_priv(cmd);
 	struct virtio_scsi_cmd_resp *resp = &cmd->resp.cmd;
 
 	dev_dbg(&sc->device->sdev_gendev,
@@ -406,7 +403,7 @@ static int __virtscsi_add_cmd(struct virtqueue *vq,
 			    struct virtio_scsi_cmd *cmd,
 			    size_t req_size, size_t resp_size)
 {
-	struct scsi_cmnd *sc = cmd->sc;
+	struct scsi_cmnd *sc = scsi_cmd_from_priv(cmd);
 	struct scatterlist *sgs[6], req, resp;
 	struct sg_table *out, *in;
 	unsigned out_num = 0, in_num = 0;
@@ -557,8 +554,6 @@ static int virtscsi_queuecommand(struct Scsi_Host *shost,
 	dev_dbg(&sc->device->sdev_gendev,
 		"cmd %p CDB: %#02x\n", sc, sc->cmnd[0]);
 
-	cmd->sc = sc;
-
 	BUG_ON(sc->cmd_len > VIRTIO_SCSI_CDB_SIZE);
 
 #ifdef CONFIG_BLK_DEV_INTEGRITY
@@ -590,17 +585,17 @@ static int virtscsi_queuecommand(struct Scsi_Host *shost,
 static int virtscsi_tmf(struct virtio_scsi *vscsi, struct virtio_scsi_cmd *cmd)
 {
 	DECLARE_COMPLETION_ONSTACK(comp);
-	int ret = FAILED;
 
 	cmd->comp = &comp;
+
 	if (virtscsi_add_cmd(&vscsi->ctrl_vq, cmd,
 			      sizeof cmd->req.tmf, sizeof cmd->resp.tmf, true) < 0)
-		goto out;
+		return FAILED;
 
 	wait_for_completion(&comp);
 	if (cmd->resp.tmf.response == VIRTIO_SCSI_S_OK ||
 	    cmd->resp.tmf.response == VIRTIO_SCSI_S_FUNCTION_SUCCEEDED)
-		ret = SUCCESS;
+		return SUCCESS;
 
 	/*
 	 * The spec guarantees that all requests related to the TMF have
@@ -613,33 +608,36 @@ static int virtscsi_tmf(struct virtio_scsi *vscsi, struct virtio_scsi_cmd *cmd)
 	 * REQ_ATOM_COMPLETE has been set.
 	 */
 	virtscsi_poll_requests(vscsi);
-
-out:
-	mempool_free(cmd, virtscsi_cmd_pool);
-	return ret;
+	return FAILED;
 }
 
 static int virtscsi_device_reset(struct scsi_cmnd *sc)
 {
+	struct scsi_device *sdev = sc->device;
+	struct scsi_cmnd *reset_sc;
 	struct virtio_scsi *vscsi = shost_priv(sc->device->host);
 	struct virtio_scsi_cmd *cmd;
+	int rc;
 
-	sdev_printk(KERN_INFO, sc->device, "device reset\n");
-	cmd = mempool_alloc(virtscsi_cmd_pool, GFP_NOIO);
-	if (!cmd)
+	sdev_printk(KERN_INFO, sdev, "device reset\n");
+	reset_sc = scsi_get_reserved_cmd(sdev, DMA_NONE);
+	if (!reset_sc)
 		return FAILED;
-
+	cmd = scsi_cmd_priv(reset_sc);
 	memset(cmd, 0, sizeof(*cmd));
 	cmd->req.tmf = (struct virtio_scsi_ctrl_tmf_req){
 		.type = VIRTIO_SCSI_T_TMF,
 		.subtype = cpu_to_virtio32(vscsi->vdev,
 					     VIRTIO_SCSI_T_TMF_LOGICAL_UNIT_RESET),
 		.lun[0] = 1,
-		.lun[1] = sc->device->id,
-		.lun[2] = (sc->device->lun >> 8) | 0x40,
-		.lun[3] = sc->device->lun & 0xff,
+		.lun[1] = sdev->id,
+		.lun[2] = (sdev->lun >> 8) | 0x40,
+		.lun[3] = sdev->lun & 0xff,
 	};
-	return virtscsi_tmf(vscsi, cmd);
+	rc = virtscsi_tmf(vscsi, cmd);
+	scsi_put_reserved_cmd(reset_sc);
+
+	return rc;
 }
 
 static int virtscsi_device_alloc(struct scsi_device *sdevice)
@@ -679,25 +677,31 @@ static int virtscsi_change_queue_depth(struct scsi_device *sdev, int qdepth)
 
 static int virtscsi_abort(struct scsi_cmnd *sc)
 {
-	struct virtio_scsi *vscsi = shost_priv(sc->device->host);
+	struct scsi_device *sdev = sc->device;
+	struct scsi_cmnd *reset_sc;
+	struct virtio_scsi *vscsi = shost_priv(sdev->host);
 	struct virtio_scsi_cmd *cmd;
+	int rc;
 
 	scmd_printk(KERN_INFO, sc, "abort\n");
-	cmd = mempool_alloc(virtscsi_cmd_pool, GFP_NOIO);
-	if (!cmd)
+	reset_sc = scsi_get_reserved_cmd(sdev, DMA_NONE);
+	if (!reset_sc)
 		return FAILED;
+	cmd = scsi_cmd_priv(reset_sc);
 
 	memset(cmd, 0, sizeof(*cmd));
 	cmd->req.tmf = (struct virtio_scsi_ctrl_tmf_req){
 		.type = VIRTIO_SCSI_T_TMF,
 		.subtype = VIRTIO_SCSI_T_TMF_ABORT_TASK,
 		.lun[0] = 1,
-		.lun[1] = sc->device->id,
-		.lun[2] = (sc->device->lun >> 8) | 0x40,
-		.lun[3] = sc->device->lun & 0xff,
+		.lun[1] = sdev->id,
+		.lun[2] = (sdev->lun >> 8) | 0x40,
+		.lun[3] = sdev->lun & 0xff,
 		.tag = cpu_to_virtio64(vscsi->vdev, (unsigned long)sc),
 	};
-	return virtscsi_tmf(vscsi, cmd);
+	rc = virtscsi_tmf(vscsi, cmd);
+	scsi_put_reserved_cmd(reset_sc);
+	return rc;
 }
 
 static int virtscsi_map_queues(struct Scsi_Host *shost)

drivers/scsi/virtio_scsi.c static int virtscsi_probe(struct virtio_device *vdev) · line 872

Quoted 4 lines
drivers/scsi/virtio_scsi.c +1 -0
@@ -865,6 +869,11 @@ static int virtscsi_probe(struct virtio_device *vdev)
 		goto virtscsi_init_failed;
 
 	shost->can_queue = virtqueue_get_vring_size(vscsi->req_vqs[0].vq);
+	shost->can_queue -= VIRTIO_SCSI_RESERVED_CMDS;

I guess it should have been '+=', right?

Thanks,
Ming

#38 lore raw

Re: [PATCH RFC v3 06/41] virtio_scsi: use reserved commands for TMF

In reply to Hannes Reinecke
Quoted 64 lines · Hannes Reinecke

From: Hannes Reinecke <[email protected]>

Set two commands aside for TMF, and use reserved commands to issue TMFs. With that we can drop the TMF memory pool.

---
 drivers/scsi/virtio_scsi.c | 105 ++++++++++++++++++---------------------------
 1 file changed, 41 insertions(+), 64 deletions(-)
drivers/scsi/virtio_scsi.c +3 -8
index 0e0910c5b942..26054c29d897 100644
@@ -35,10 +35,10 @@
 #define VIRTIO_SCSI_MEMPOOL_SZ 64
 #define VIRTIO_SCSI_EVENT_LEN 8
 #define VIRTIO_SCSI_VQ_BASE 2
+#define VIRTIO_SCSI_RESERVED_CMDS 2
 
 /* Command queue element */
 struct virtio_scsi_cmd {
-	struct scsi_cmnd *sc;
 	struct completion *comp;
 	union {
 		struct virtio_scsi_cmd_req       cmd;
@@ -86,9 +86,6 @@ struct virtio_scsi {
 	struct virtio_scsi_vq req_vqs[];
 };
 
-static struct kmem_cache *virtscsi_cmd_cache;
-static mempool_t *virtscsi_cmd_pool;
-
 static inline struct Scsi_Host *virtio_scsi_host(struct virtio_device *vdev)
 {
 	return vdev->priv;
@@ -108,7 +105,7 @@ static void virtscsi_compute_resid(struct scsi_cmnd *sc, u32 resid)
 static void virtscsi_complete_cmd(struct virtio_scsi *vscsi, void *buf)
 {
 	struct virtio_scsi_cmd *cmd = buf;
-	struct scsi_cmnd *sc = cmd->sc;
+	struct scsi_cmnd *sc = scsi_cmd_from_priv(cmd);
 	struct virtio_scsi_cmd_resp *resp = &cmd->resp.cmd;
 
 	dev_dbg(&sc->device->sdev_gendev,
@@ -406,7 +403,7 @@ static int __virtscsi_add_cmd(struct virtqueue *vq,
 			    struct virtio_scsi_cmd *cmd,
 			    size_t req_size, size_t resp_size)
 {
-	struct scsi_cmnd *sc = cmd->sc;
+	struct scsi_cmnd *sc = scsi_cmd_from_priv(cmd);
 	struct scatterlist *sgs[6], req, resp;
 	struct sg_table *out, *in;
 	unsigned out_num = 0, in_num = 0;
@@ -557,8 +554,6 @@ static int virtscsi_queuecommand(struct Scsi_Host *shost,
 	dev_dbg(&sc->device->sdev_gendev,
 		"cmd %p CDB: %#02x\n", sc, sc->cmnd[0]);
 
-	cmd->sc = sc;
-
 	BUG_ON(sc->cmd_len > VIRTIO_SCSI_CDB_SIZE);
 
 #ifdef CONFIG_BLK_DEV_INTEGRITY

drivers/scsi/virtio_scsi.c static int virtscsi_queuecommand(struct Scsi_Host *shost, · line 592

Quoted 9 lines
drivers/scsi/virtio_scsi.c +1 -1
@@ -590,17 +585,17 @@ static int virtscsi_queuecommand(struct Scsi_Host *shost,
 static int virtscsi_tmf(struct virtio_scsi *vscsi, struct virtio_scsi_cmd *cmd)
 {
 	DECLARE_COMPLETION_ONSTACK(comp);
-	int ret = FAILED;
 
 	cmd->comp = &comp;
+
 	if (virtscsi_add_cmd(&vscsi->ctrl_vq, cmd,
 			      sizeof cmd->req.tmf, sizeof cmd->resp.tmf, true) < 0)

virtscsi uses dedicated ->ctrl_vq to send TMF, and ->ctrl_vq isn't nothing to do with vqs(->vqs[]) for sending IO request.

Quoted 25 lines
drivers/scsi/virtio_scsi.c +4 -6
-		goto out;
+		return FAILED;
 
 	wait_for_completion(&comp);
 	if (cmd->resp.tmf.response == VIRTIO_SCSI_S_OK ||
 	    cmd->resp.tmf.response == VIRTIO_SCSI_S_FUNCTION_SUCCEEDED)
-		ret = SUCCESS;
+		return SUCCESS;
 
 	/*
 	 * The spec guarantees that all requests related to the TMF have
@@ -613,33 +608,36 @@ static int virtscsi_tmf(struct virtio_scsi *vscsi, struct virtio_scsi_cmd *cmd)
 	 * REQ_ATOM_COMPLETE has been set.
 	 */
 	virtscsi_poll_requests(vscsi);
-
-out:
-	mempool_free(cmd, virtscsi_cmd_pool);
-	return ret;
+	return FAILED;
 }
 
 static int virtscsi_device_reset(struct scsi_cmnd *sc)
 {
+	struct scsi_device *sdev = sc->device;

drivers/scsi/virtio_scsi.c static int virtscsi_tmf(struct virtio_scsi *vscsi, struct virtio_scsi_cmd *cmd) · line 623

Quoted 10 lines
drivers/scsi/virtio_scsi.c +4 -3
+	struct scsi_cmnd *reset_sc;
 	struct virtio_scsi *vscsi = shost_priv(sc->device->host);
 	struct virtio_scsi_cmd *cmd;
+	int rc;
 
-	sdev_printk(KERN_INFO, sc->device, "device reset\n");
-	cmd = mempool_alloc(virtscsi_cmd_pool, GFP_NOIO);
-	if (!cmd)
+	sdev_printk(KERN_INFO, sdev, "device reset\n");
+	reset_sc = scsi_get_reserved_cmd(sdev, DMA_NONE);

As commented above, virtio scsi doesn't use IO vqs to send TMF, so the above reserved command isn't necessary.

There might be other example which do not require IO tag to send reset command.

Thanks,
Ming

#39 lore raw

Re: [PATCH RFC v3 06/41] virtio_scsi: use reserved commands for TMF

Quoted 70 lines · Ming Lei
Quoted 69 lines · Hannes Reinecke

From: Hannes Reinecke <[email protected]>

Set two commands aside for TMF, and use reserved commands to issue TMFs. With that we can drop the TMF memory pool.

---
  drivers/scsi/virtio_scsi.c | 105 ++++++++++++++++++---------------------------
  1 file changed, 41 insertions(+), 64 deletions(-)
drivers/scsi/virtio_scsi.c +3 -9
index 0e0910c5b942..26054c29d897 100644
@@ -35,10 +35,10 @@
  #define VIRTIO_SCSI_MEMPOOL_SZ 64
  #define VIRTIO_SCSI_EVENT_LEN 8
  #define VIRTIO_SCSI_VQ_BASE 2
+#define VIRTIO_SCSI_RESERVED_CMDS 2
 
  /* Command queue element */
  struct virtio_scsi_cmd {
-	struct scsi_cmnd *sc;
  	struct completion *comp;
  	union {
  		struct virtio_scsi_cmd_req       cmd;
@@ -86,9 +86,6 @@ struct virtio_scsi {
  	struct virtio_scsi_vq req_vqs[];
  };
 
-static struct kmem_cache *virtscsi_cmd_cache;
-static mempool_t *virtscsi_cmd_pool;
-
  static inline struct Scsi_Host *virtio_scsi_host(struct virtio_device *vdev)
  {
  	return vdev->priv;
@@ -108,7 +105,7 @@ static void virtscsi_compute_resid(struct scsi_cmnd *sc, u32 resid)
  static void virtscsi_complete_cmd(struct virtio_scsi *vscsi, void *buf)
  {
  	struct virtio_scsi_cmd *cmd = buf;
-	struct scsi_cmnd *sc = cmd->sc;
+	struct scsi_cmnd *sc = scsi_cmd_from_priv(cmd);
  	struct virtio_scsi_cmd_resp *resp = &cmd->resp.cmd;
 
  	dev_dbg(&sc->device->sdev_gendev,
@@ -406,7 +403,7 @@ static int __virtscsi_add_cmd(struct virtqueue *vq,
  			    struct virtio_scsi_cmd *cmd,
  			    size_t req_size, size_t resp_size)
  {
-	struct scsi_cmnd *sc = cmd->sc;
+	struct scsi_cmnd *sc = scsi_cmd_from_priv(cmd);
  	struct scatterlist *sgs[6], req, resp;
  	struct sg_table *out, *in;
  	unsigned out_num = 0, in_num = 0;
@@ -557,8 +554,6 @@ static int virtscsi_queuecommand(struct Scsi_Host *shost,
  	dev_dbg(&sc->device->sdev_gendev,
  		"cmd %p CDB: %#02x\n", sc, sc->cmnd[0]);
 
-	cmd->sc = sc;
-
  	BUG_ON(sc->cmd_len > VIRTIO_SCSI_CDB_SIZE);
 
  #ifdef CONFIG_BLK_DEV_INTEGRITY
@@ -590,17 +585,17 @@ static int virtscsi_queuecommand(struct Scsi_Host *shost,
  static int virtscsi_tmf(struct virtio_scsi *vscsi, struct virtio_scsi_cmd *cmd)
  {
  	DECLARE_COMPLETION_ONSTACK(comp);
-	int ret = FAILED;
 
Quoted 8 lines

drivers/scsi/virtio_scsi.c static int virtscsi_queuecommand(struct Scsi_Host *shost, · line 592

Quoted 4 lines
drivers/scsi/virtio_scsi.c +1 -0
  	cmd->comp = &comp;
+
  	if (virtscsi_add_cmd(&vscsi->ctrl_vq, cmd,
  			      sizeof cmd->req.tmf, sizeof cmd->resp.tmf, true) < 0)

virtscsi uses dedicated ->ctrl_vq to send TMF, and ->ctrl_vq isn't nothing to do with vqs(->vqs[]) for sending IO request.

Indeed, you are right.
We should be handling this by adding a virtual LUN corresponding the the ctrl_vq and allocate commands from there.

Will be updating for the next round.

Cheers,

Hannes

Signature · 4 lines
Dr. Hannes Reinecke            Teamlead Storage & Networking
[email protected]                               +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer
#40 lore raw

Re: [PATCH RFC v3 06/41] virtio_scsi: use reserved commands for TMF

Quoted 73 lines · Hannes Reinecke
Quoted 63 lines · Ming Lei
Quoted 62 lines · Hannes Reinecke

From: Hannes Reinecke <[email protected]>

Set two commands aside for TMF, and use reserved commands to issue TMFs. With that we can drop the TMF memory pool.

---
  drivers/scsi/virtio_scsi.c | 105 ++++++++++++++++++---------------------------
  1 file changed, 41 insertions(+), 64 deletions(-)
drivers/scsi/virtio_scsi.c +3 -8
index 0e0910c5b942..26054c29d897 100644
@@ -35,10 +35,10 @@
  #define VIRTIO_SCSI_MEMPOOL_SZ 64
  #define VIRTIO_SCSI_EVENT_LEN 8
  #define VIRTIO_SCSI_VQ_BASE 2
+#define VIRTIO_SCSI_RESERVED_CMDS 2
  /* Command queue element */
  struct virtio_scsi_cmd {
-	struct scsi_cmnd *sc;
  	struct completion *comp;
  	union {
  		struct virtio_scsi_cmd_req       cmd;
@@ -86,9 +86,6 @@ struct virtio_scsi {
  	struct virtio_scsi_vq req_vqs[];
  };
-static struct kmem_cache *virtscsi_cmd_cache;
-static mempool_t *virtscsi_cmd_pool;
-
  static inline struct Scsi_Host *virtio_scsi_host(struct virtio_device *vdev)
  {
  	return vdev->priv;
@@ -108,7 +105,7 @@ static void virtscsi_compute_resid(struct scsi_cmnd *sc, u32 resid)
  static void virtscsi_complete_cmd(struct virtio_scsi *vscsi, void *buf)
  {
  	struct virtio_scsi_cmd *cmd = buf;
-	struct scsi_cmnd *sc = cmd->sc;
+	struct scsi_cmnd *sc = scsi_cmd_from_priv(cmd);
  	struct virtio_scsi_cmd_resp *resp = &cmd->resp.cmd;
  	dev_dbg(&sc->device->sdev_gendev,
@@ -406,7 +403,7 @@ static int __virtscsi_add_cmd(struct virtqueue *vq,
  			    struct virtio_scsi_cmd *cmd,
  			    size_t req_size, size_t resp_size)
  {
-	struct scsi_cmnd *sc = cmd->sc;
+	struct scsi_cmnd *sc = scsi_cmd_from_priv(cmd);
  	struct scatterlist *sgs[6], req, resp;
  	struct sg_table *out, *in;
  	unsigned out_num = 0, in_num = 0;
@@ -557,8 +554,6 @@ static int virtscsi_queuecommand(struct Scsi_Host *shost,
  	dev_dbg(&sc->device->sdev_gendev,
  		"cmd %p CDB: %#02x\n", sc, sc->cmnd[0]);
-	cmd->sc = sc;
-
  	BUG_ON(sc->cmd_len > VIRTIO_SCSI_CDB_SIZE);
  #ifdef CONFIG_BLK_DEV_INTEGRITY
@@ -590,17 +585,17 @@ static int virtscsi_queuecommand(struct Scsi_Host *shost,
  static int virtscsi_tmf(struct virtio_scsi *vscsi, struct virtio_scsi_cmd *cmd)
  {
  	DECLARE_COMPLETION_ONSTACK(comp);
Quoted 9 lines

drivers/scsi/virtio_scsi.c static int virtscsi_queuecommand(struct Scsi_Host *shost, · line 592

Quoted 5 lines
drivers/scsi/virtio_scsi.c +1 -1
-	int ret = FAILED;
  	cmd->comp = &comp;
+
  	if (virtscsi_add_cmd(&vscsi->ctrl_vq, cmd,
  			      sizeof cmd->req.tmf, sizeof cmd->resp.tmf, true) < 0)

virtscsi uses dedicated ->ctrl_vq to send TMF, and ->ctrl_vq isn't nothing to do with vqs(->vqs[]) for sending IO request.

Quoted 3 lines

Indeed, you are right.
We should be handling this by adding a virtual LUN corresponding the the ctrl_vq and allocate commands from there.

I am not sure why we need this kind of change, and looks the current code just works fine.

Thanks,
Ming

#41 lore raw

RFC PATCH v3 7/41 fnic: use reserved commands

In reply to Hannes Reinecke

From: Hannes Reinecke <[email protected]>

Remove hack to get tag for the reset command by using reserved commands.

---
 drivers/scsi/fnic/fnic_scsi.c | 147 ++++++++++++++----------------------------
 1 file changed, 49 insertions(+), 98 deletions(-)
drivers/scsi/fnic/fnic_scsi.c +49 -98
index b60795893994..228729013e21 100644
@@ -101,7 +101,7 @@ static const char *fnic_fcpio_status_to_str(unsigned int status)
 	return fcpio_status_str[status];
 }
 
-static void fnic_cleanup_io(struct fnic *fnic, int exclude_id);
+static void fnic_cleanup_io(struct fnic *fnic);
 
 static inline spinlock_t *fnic_io_lock_hash(struct fnic *fnic,
 					    struct scsi_cmnd *sc)
@@ -637,7 +637,7 @@ static int fnic_fcpio_fw_reset_cmpl_handler(struct fnic *fnic,
 	atomic64_inc(&reset_stats->fw_reset_completions);
 
 	/* Clean up all outstanding io requests */
-	fnic_cleanup_io(fnic, SCSI_NO_TAG);
+	fnic_cleanup_io(fnic);
 
 	atomic64_set(&fnic->fnic_stats.fw_stats.active_fw_reqs, 0);
 	atomic64_set(&fnic->fnic_stats.io_stats.active_ios, 0);
@@ -1359,7 +1359,7 @@ int fnic_wq_copy_cmpl_handler(struct fnic *fnic, int copy_work_to_do)
 	return wq_work_done;
 }
 
-static void fnic_cleanup_io(struct fnic *fnic, int exclude_id)
+static void fnic_cleanup_io(struct fnic *fnic)
 {
 	int i;
 	struct fnic_io_req *io_req;
@@ -1370,9 +1370,6 @@ static void fnic_cleanup_io(struct fnic *fnic, int exclude_id)
 	struct fnic_stats *fnic_stats = &fnic->fnic_stats;
 
 	for (i = 0; i < fnic->fnic_max_tag_id; i++) {
-		if (i == exclude_id)
-			continue;
-
 		io_lock = fnic_io_lock_tag(fnic, i);
 		spin_lock_irqsave(io_lock, flags);
 		sc = scsi_host_find_tag(fnic->lport->host, i);
@@ -2125,9 +2122,7 @@ static inline int fnic_queue_dr_io_req(struct fnic *fnic,
  * successfully aborted, 1 otherwise
  */
 static int fnic_clean_pending_aborts(struct fnic *fnic,
-				     struct scsi_cmnd *lr_sc,
-					 bool new_sc)
-
+				     struct scsi_cmnd *lr_sc)
 {
 	int tag, abt_tag;
 	struct fnic_io_req *io_req;
@@ -2148,7 +2143,7 @@ static int fnic_clean_pending_aborts(struct fnic *fnic,
 		 * ignore this lun reset cmd if issued using new SC
 		 * or cmds that do not belong to this lun
 		 */
-		if (!sc || ((sc == lr_sc) && new_sc) || sc->device != lun_dev) {
+		if (!sc || sc == lr_sc || sc->device != lun_dev) {
 			spin_unlock_irqrestore(io_lock, flags);
 			continue;
 		}
@@ -2287,38 +2282,6 @@ static int fnic_clean_pending_aborts(struct fnic *fnic,
 	return ret;
 }
 
-/**
- * fnic_scsi_host_start_tag
- * Allocates tagid from host's tag list
- **/
-static inline int
-fnic_scsi_host_start_tag(struct fnic *fnic, struct scsi_cmnd *sc)
-{
-	struct request_queue *q = sc->request->q;
-	struct request *dummy;
-
-	dummy = blk_mq_alloc_request(q, REQ_OP_WRITE, BLK_MQ_REQ_NOWAIT);
-	if (IS_ERR(dummy))
-		return SCSI_NO_TAG;
-
-	sc->tag = sc->request->tag = dummy->tag;
-	sc->host_scribble = (unsigned char *)dummy;
-
-	return dummy->tag;
-}
-
-/**
- * fnic_scsi_host_end_tag
- * frees tag allocated by fnic_scsi_host_start_tag.
- **/
-static inline void
-fnic_scsi_host_end_tag(struct fnic *fnic, struct scsi_cmnd *sc)
-{
-	struct request *dummy = (struct request *)sc->host_scribble;
-
-	blk_mq_free_request(dummy);
-}
-
 /*
  * SCSI Eh thread issues a Lun Reset when one or more commands on a LUN
  * fail to get aborted. It calls driver's eh_device_reset with a SCSI command
@@ -2335,19 +2298,19 @@ int fnic_device_reset(struct scsi_cmnd *sc)
 	spinlock_t *io_lock;
 	unsigned long flags;
 	unsigned long start_time = 0;
+	struct scsi_device *sdev = sc->device;
 	struct scsi_lun fc_lun;
 	struct fnic_stats *fnic_stats;
 	struct reset_stats *reset_stats;
 	int tag = 0;
 	DECLARE_COMPLETION_ONSTACK(tm_done);
-	int tag_gen_flag = 0;   /*to track tags allocated by fnic driver*/
-	bool new_sc = 0;
+	struct scsi_cmnd *reset_sc = NULL;
 
 	/* Wait for rport to unblock */
 	fc_block_scsi_eh(sc);
 
 	/* Get local-port, check ready and link up */
-	lp = shost_priv(sc->device->host);
+	lp = shost_priv(sdev->host);
 
 	fnic = lport_priv(lp);
 	fnic_stats = &fnic->fnic_stats;
@@ -2355,10 +2318,10 @@ int fnic_device_reset(struct scsi_cmnd *sc)
 
 	atomic64_inc(&reset_stats->device_resets);
 
-	rport = starget_to_rport(scsi_target(sc->device));
+	rport = starget_to_rport(scsi_target(sdev));
 	FNIC_SCSI_DBG(KERN_DEBUG, fnic->lport->host,
 		      "Device reset called FCID 0x%x, LUN 0x%llx sc 0x%p\n",
-		      rport->port_id, sc->device->lun, sc);
+		      rport->port_id, sdev->lun, sc);
 
 	if (lp->state != LPORT_ST_READY || !(lp->link_up))
 		goto fnic_device_reset_end;
@@ -2369,42 +2332,31 @@ int fnic_device_reset(struct scsi_cmnd *sc)
 		goto fnic_device_reset_end;
 	}
 
-	CMD_FLAGS(sc) = FNIC_DEVICE_RESET;
-	/* Allocate tag if not present */
+	reset_sc = scsi_get_reserved_cmd(sdev, DMA_NONE);
+	if (unlikely(!reset_sc))
+		goto fnic_device_reset_end;
 
-	tag = sc->request->tag;
-	if (unlikely(tag < 0)) {
-		/*
-		 * Really should fix the midlayer to pass in a proper
-		 * request for ioctls...
-		 */
-		tag = fnic_scsi_host_start_tag(fnic, sc);
-		if (unlikely(tag == SCSI_NO_TAG))
-			goto fnic_device_reset_end;
-		tag_gen_flag = 1;
-		new_sc = 1;
-	}
-	io_lock = fnic_io_lock_hash(fnic, sc);
+	CMD_FLAGS(reset_sc) = FNIC_DEVICE_RESET;
+	tag = reset_sc->request->tag;
+	io_lock = fnic_io_lock_hash(fnic, reset_sc);
 	spin_lock_irqsave(io_lock, flags);
-	io_req = (struct fnic_io_req *)CMD_SP(sc);
+	io_req = (struct fnic_io_req *)CMD_SP(reset_sc);
 
 	/*
-	 * If there is a io_req attached to this command, then use it,
-	 * else allocate a new one.
+	 * Allocate a new io_req.
 	 */
+	io_req = mempool_alloc(fnic->io_req_pool, GFP_ATOMIC);
 	if (!io_req) {
-		io_req = mempool_alloc(fnic->io_req_pool, GFP_ATOMIC);
-		if (!io_req) {
-			spin_unlock_irqrestore(io_lock, flags);
-			goto fnic_device_reset_end;
-		}
-		memset(io_req, 0, sizeof(*io_req));
-		io_req->port_id = rport->port_id;
-		CMD_SP(sc) = (char *)io_req;
+		spin_unlock_irqrestore(io_lock, flags);
+		goto fnic_device_reset_end;
 	}
+	memset(io_req, 0, sizeof(*io_req));
+	io_req->port_id = rport->port_id;
+	CMD_SP(reset_sc) = (char *)io_req;
+
 	io_req->dr_done = &tm_done;
-	CMD_STATE(sc) = FNIC_IOREQ_CMD_PENDING;
-	CMD_LR_STATUS(sc) = FCPIO_INVALID_CODE;
+	CMD_STATE(reset_sc) = FNIC_IOREQ_CMD_PENDING;
+	CMD_LR_STATUS(reset_sc) = FCPIO_INVALID_CODE;
 	spin_unlock_irqrestore(io_lock, flags);
 
 	FNIC_SCSI_DBG(KERN_DEBUG, fnic->lport->host, "TAG %x\n", tag);
@@ -2413,15 +2365,15 @@ int fnic_device_reset(struct scsi_cmnd *sc)
 	 * issue the device reset, if enqueue failed, clean up the ioreq
 	 * and break assoc with scsi cmd
 	 */
-	if (fnic_queue_dr_io_req(fnic, sc, io_req)) {
+	if (fnic_queue_dr_io_req(fnic, reset_sc, io_req)) {
 		spin_lock_irqsave(io_lock, flags);
-		io_req = (struct fnic_io_req *)CMD_SP(sc);
+		io_req = (struct fnic_io_req *)CMD_SP(reset_sc);
 		if (io_req)
 			io_req->dr_done = NULL;
 		goto fnic_device_reset_clean;
 	}
 	spin_lock_irqsave(io_lock, flags);
-	CMD_FLAGS(sc) |= FNIC_DEV_RST_ISSUED;
+	CMD_FLAGS(reset_sc) |= FNIC_DEV_RST_ISSUED;
 	spin_unlock_irqrestore(io_lock, flags);
 
 	/*
@@ -2432,16 +2384,16 @@ int fnic_device_reset(struct scsi_cmnd *sc)
 				    msecs_to_jiffies(FNIC_LUN_RESET_TIMEOUT));
 
 	spin_lock_irqsave(io_lock, flags);
-	io_req = (struct fnic_io_req *)CMD_SP(sc);
+	io_req = (struct fnic_io_req *)CMD_SP(reset_sc);
 	if (!io_req) {
 		spin_unlock_irqrestore(io_lock, flags);
 		FNIC_SCSI_DBG(KERN_DEBUG, fnic->lport->host,
-				"io_req is null tag 0x%x sc 0x%p\n", tag, sc);
+				"io_req is null tag 0x%x sc 0x%p\n", tag, reset_sc);
 		goto fnic_device_reset_end;
 	}
 	io_req->dr_done = NULL;
 
-	status = CMD_LR_STATUS(sc);
+	status = CMD_LR_STATUS(reset_sc);
 
 	/*
 	 * If lun reset not completed, bail out with failed. io_req
@@ -2451,16 +2403,16 @@ int fnic_device_reset(struct scsi_cmnd *sc)
 		atomic64_inc(&reset_stats->device_reset_timeouts);
 		FNIC_SCSI_DBG(KERN_DEBUG, fnic->lport->host,
 			      "Device reset timed out\n");
-		CMD_FLAGS(sc) |= FNIC_DEV_RST_TIMED_OUT;
+		CMD_FLAGS(reset_sc) |= FNIC_DEV_RST_TIMED_OUT;
 		spin_unlock_irqrestore(io_lock, flags);
-		int_to_scsilun(sc->device->lun, &fc_lun);
+		int_to_scsilun(sdev->lun, &fc_lun);
 		/*
 		 * Issue abort and terminate on device reset request.
 		 * If q'ing of terminate fails, retry it after a delay.
 		 */
 		while (1) {
 			spin_lock_irqsave(io_lock, flags);
-			if (CMD_FLAGS(sc) & FNIC_DEV_RST_TERM_ISSUED) {
+			if (CMD_FLAGS(reset_sc) & FNIC_DEV_RST_TERM_ISSUED) {
 				spin_unlock_irqrestore(io_lock, flags);
 				break;
 			}
@@ -2473,13 +2425,13 @@ int fnic_device_reset(struct scsi_cmnd *sc)
 				msecs_to_jiffies(FNIC_ABT_TERM_DELAY_TIMEOUT));
 			} else {
 				spin_lock_irqsave(io_lock, flags);
-				CMD_FLAGS(sc) |= FNIC_DEV_RST_TERM_ISSUED;
-				CMD_STATE(sc) = FNIC_IOREQ_ABTS_PENDING;
+				CMD_FLAGS(reset_sc) |= FNIC_DEV_RST_TERM_ISSUED;
+				CMD_STATE(reset_sc) = FNIC_IOREQ_ABTS_PENDING;
 				io_req->abts_done = &tm_done;
 				spin_unlock_irqrestore(io_lock, flags);
 				FNIC_SCSI_DBG(KERN_DEBUG, fnic->lport->host,
 				"Abort and terminate issued on Device reset "
-				"tag 0x%x sc 0x%p\n", tag, sc);
+				"tag 0x%x sc 0x%p\n", tag, reset_sc);
 				break;
 			}
 		}
@@ -2491,7 +2443,7 @@ int fnic_device_reset(struct scsi_cmnd *sc)
 				msecs_to_jiffies(FNIC_LUN_RESET_TIMEOUT));
 				break;
 			} else {
-				io_req = (struct fnic_io_req *)CMD_SP(sc);
+				io_req = (struct fnic_io_req *)CMD_SP(reset_sc);
 				io_req->abts_done = NULL;
 				goto fnic_device_reset_clean;
 			}
@@ -2506,7 +2458,7 @@ int fnic_device_reset(struct scsi_cmnd *sc)
 		FNIC_SCSI_DBG(KERN_DEBUG,
 			      fnic->lport->host,
 			      "Device reset completed - failed\n");
-		io_req = (struct fnic_io_req *)CMD_SP(sc);
+		io_req = (struct fnic_io_req *)CMD_SP(reset_sc);
 		goto fnic_device_reset_clean;
 	}
 
@@ -2517,7 +2469,7 @@ int fnic_device_reset(struct scsi_cmnd *sc)
 	 * the lun reset cmd. If all cmds get cleaned, the lun reset
 	 * succeeds
 	 */
-	if (fnic_clean_pending_aborts(fnic, sc, new_sc)) {
+	if (fnic_clean_pending_aborts(fnic, reset_sc)) {
 		spin_lock_irqsave(io_lock, flags);
 		io_req = (struct fnic_io_req *)CMD_SP(sc);
 		FNIC_SCSI_DBG(KERN_DEBUG, fnic->lport->host,
@@ -2528,35 +2480,34 @@ int fnic_device_reset(struct scsi_cmnd *sc)
 
 	/* Clean lun reset command */
 	spin_lock_irqsave(io_lock, flags);
-	io_req = (struct fnic_io_req *)CMD_SP(sc);
+	io_req = (struct fnic_io_req *)CMD_SP(reset_sc);
 	if (io_req)
 		/* Completed, and successful */
 		ret = SUCCESS;
 
 fnic_device_reset_clean:
 	if (io_req)
-		CMD_SP(sc) = NULL;
+		CMD_SP(reset_sc) = NULL;
 
 	spin_unlock_irqrestore(io_lock, flags);
 
 	if (io_req) {
 		start_time = io_req->start_time;
-		fnic_release_ioreq_buf(fnic, io_req, sc);
+		fnic_release_ioreq_buf(fnic, io_req, reset_sc);
 		mempool_free(io_req, fnic->io_req_pool);
 	}
 
 fnic_device_reset_end:
-	FNIC_TRACE(fnic_device_reset, sc->device->host->host_no,
-		  sc->request->tag, sc,
+	FNIC_TRACE(fnic_device_reset, sdev->host->host_no,
+		   reset_sc->request->tag, reset_sc,
 		  jiffies_to_msecs(jiffies - start_time),
-		  0, ((u64)sc->cmnd[0] << 32 |
+		  0, ((u64)reset_sc->cmnd[0] << 32 |
 		  (u64)sc->cmnd[2] << 24 | (u64)sc->cmnd[3] << 16 |
 		  (u64)sc->cmnd[4] << 8 | sc->cmnd[5]),
 		  (((u64)CMD_FLAGS(sc) << 32) | CMD_STATE(sc)));
 
 	/* free tag if it is allocated */
-	if (unlikely(tag_gen_flag))
-		fnic_scsi_host_end_tag(fnic, sc);
+	scsi_put_reserved_cmd(reset_sc);
 
 	FNIC_SCSI_DBG(KERN_DEBUG, fnic->lport->host,
 		      "Returning from device reset %s\n",
Signature · 1 line
2.16.4
#42 lore raw

Re: [PATCH RFC v3 07/41] fnic: use reserved commands

Quoted 147 lines · Hannes Reinecke

From: Hannes Reinecke <[email protected]>

Remove hack to get tag for the reset command by using reserved commands.

---
 drivers/scsi/fnic/fnic_scsi.c | 147 ++++++++++++++----------------------------
 1 file changed, 49 insertions(+), 98 deletions(-)
drivers/scsi/fnic/fnic_scsi.c +10 -47
index b60795893994..228729013e21 100644
@@ -101,7 +101,7 @@ static const char *fnic_fcpio_status_to_str(unsigned int status)
 	return fcpio_status_str[status];
 }
 
-static void fnic_cleanup_io(struct fnic *fnic, int exclude_id);
+static void fnic_cleanup_io(struct fnic *fnic);
 
 static inline spinlock_t *fnic_io_lock_hash(struct fnic *fnic,
 					    struct scsi_cmnd *sc)
@@ -637,7 +637,7 @@ static int fnic_fcpio_fw_reset_cmpl_handler(struct fnic *fnic,
 	atomic64_inc(&reset_stats->fw_reset_completions);
 
 	/* Clean up all outstanding io requests */
-	fnic_cleanup_io(fnic, SCSI_NO_TAG);
+	fnic_cleanup_io(fnic);
 
 	atomic64_set(&fnic->fnic_stats.fw_stats.active_fw_reqs, 0);
 	atomic64_set(&fnic->fnic_stats.io_stats.active_ios, 0);
@@ -1359,7 +1359,7 @@ int fnic_wq_copy_cmpl_handler(struct fnic *fnic, int copy_work_to_do)
 	return wq_work_done;
 }
 
-static void fnic_cleanup_io(struct fnic *fnic, int exclude_id)
+static void fnic_cleanup_io(struct fnic *fnic)
 {
 	int i;
 	struct fnic_io_req *io_req;
@@ -1370,9 +1370,6 @@ static void fnic_cleanup_io(struct fnic *fnic, int exclude_id)
 	struct fnic_stats *fnic_stats = &fnic->fnic_stats;
 
 	for (i = 0; i < fnic->fnic_max_tag_id; i++) {
-		if (i == exclude_id)
-			continue;
-
 		io_lock = fnic_io_lock_tag(fnic, i);
 		spin_lock_irqsave(io_lock, flags);
 		sc = scsi_host_find_tag(fnic->lport->host, i);
@@ -2125,9 +2122,7 @@ static inline int fnic_queue_dr_io_req(struct fnic *fnic,
  * successfully aborted, 1 otherwise
  */
 static int fnic_clean_pending_aborts(struct fnic *fnic,
-				     struct scsi_cmnd *lr_sc,
-					 bool new_sc)
-
+				     struct scsi_cmnd *lr_sc)
 {
 	int tag, abt_tag;
 	struct fnic_io_req *io_req;
@@ -2148,7 +2143,7 @@ static int fnic_clean_pending_aborts(struct fnic *fnic,
 		 * ignore this lun reset cmd if issued using new SC
 		 * or cmds that do not belong to this lun
 		 */
-		if (!sc || ((sc == lr_sc) && new_sc) || sc->device != lun_dev) {
+		if (!sc || sc == lr_sc || sc->device != lun_dev) {
 			spin_unlock_irqrestore(io_lock, flags);
 			continue;
 		}
@@ -2287,38 +2282,6 @@ static int fnic_clean_pending_aborts(struct fnic *fnic,
 	return ret;
 }
 
-/**
- * fnic_scsi_host_start_tag
- * Allocates tagid from host's tag list
- **/
-static inline int
-fnic_scsi_host_start_tag(struct fnic *fnic, struct scsi_cmnd *sc)
-{
-	struct request_queue *q = sc->request->q;
-	struct request *dummy;
-
-	dummy = blk_mq_alloc_request(q, REQ_OP_WRITE, BLK_MQ_REQ_NOWAIT);
-	if (IS_ERR(dummy))
-		return SCSI_NO_TAG;
-
-	sc->tag = sc->request->tag = dummy->tag;
-	sc->host_scribble = (unsigned char *)dummy;
-
-	return dummy->tag;
-}
-
-/**
- * fnic_scsi_host_end_tag
- * frees tag allocated by fnic_scsi_host_start_tag.
- **/
-static inline void
-fnic_scsi_host_end_tag(struct fnic *fnic, struct scsi_cmnd *sc)
-{
-	struct request *dummy = (struct request *)sc->host_scribble;
-
-	blk_mq_free_request(dummy);
-}
-
 /*
  * SCSI Eh thread issues a Lun Reset when one or more commands on a LUN
  * fail to get aborted. It calls driver's eh_device_reset with a SCSI command
@@ -2335,19 +2298,19 @@ int fnic_device_reset(struct scsi_cmnd *sc)
 	spinlock_t *io_lock;
 	unsigned long flags;
 	unsigned long start_time = 0;
+	struct scsi_device *sdev = sc->device;
 	struct scsi_lun fc_lun;
 	struct fnic_stats *fnic_stats;
 	struct reset_stats *reset_stats;
 	int tag = 0;
 	DECLARE_COMPLETION_ONSTACK(tm_done);
-	int tag_gen_flag = 0;   /*to track tags allocated by fnic driver*/
-	bool new_sc = 0;
+	struct scsi_cmnd *reset_sc = NULL;
 
 	/* Wait for rport to unblock */
 	fc_block_scsi_eh(sc);
 
 	/* Get local-port, check ready and link up */
-	lp = shost_priv(sc->device->host);
+	lp = shost_priv(sdev->host);
 
 	fnic = lport_priv(lp);
 	fnic_stats = &fnic->fnic_stats;
@@ -2355,10 +2318,10 @@ int fnic_device_reset(struct scsi_cmnd *sc)
 
 	atomic64_inc(&reset_stats->device_resets);
 
-	rport = starget_to_rport(scsi_target(sc->device));
+	rport = starget_to_rport(scsi_target(sdev));
 	FNIC_SCSI_DBG(KERN_DEBUG, fnic->lport->host,
 		      "Device reset called FCID 0x%x, LUN 0x%llx sc 0x%p\n",
-		      rport->port_id, sc->device->lun, sc);
+		      rport->port_id, sdev->lun, sc);
 
 	if (lp->state != LPORT_ST_READY || !(lp->link_up))
 		goto fnic_device_reset_end;

drivers/scsi/fnic/fnic_scsi.c int fnic_device_reset(struct scsi_cmnd *sc) · line 2335

Quoted 6 lines
drivers/scsi/fnic/fnic_scsi.c +1 -2
@@ -2369,42 +2332,31 @@ int fnic_device_reset(struct scsi_cmnd *sc)
 		goto fnic_device_reset_end;
 	}
 
-	CMD_FLAGS(sc) = FNIC_DEVICE_RESET;
-	/* Allocate tag if not present */
+	reset_sc = scsi_get_reserved_cmd(sdev, DMA_NONE);

scsi_get_reserved_cmd() returns NULL if .nr_reserved_cmds isn't passed, so I guess you forget to do that?

Thanks,
Ming

#43 lore raw

RFC PATCH v3 8/41 fnic: use scsi_host_busy_iter() to traverse commands

In reply to Hannes Reinecke

From: Hannes Reinecke <[email protected]>

Use scsi_host_busy_iter() to traverse commands instead of hand-crafted routines walking the command list.

---
 drivers/scsi/fnic/fnic_scsi.c | 804 +++++++++++++++++++-----------------------
 1 file changed, 359 insertions(+), 445 deletions(-)
drivers/scsi/fnic/fnic_scsi.c +359 -445
index 228729013e21..94d0db99d4ec 100644
@@ -1359,91 +1359,90 @@ int fnic_wq_copy_cmpl_handler(struct fnic *fnic, int copy_work_to_do)
 	return wq_work_done;
 }
 
-static void fnic_cleanup_io(struct fnic *fnic)
+static bool fnic_cleanup_io_iter(struct scsi_cmnd *sc, void *data,
+				 bool reserved)
 {
-	int i;
+	struct fnic *fnic = data;
 	struct fnic_io_req *io_req;
 	unsigned long flags = 0;
-	struct scsi_cmnd *sc;
 	spinlock_t *io_lock;
 	unsigned long start_time = 0;
 	struct fnic_stats *fnic_stats = &fnic->fnic_stats;
 
-	for (i = 0; i < fnic->fnic_max_tag_id; i++) {
-		io_lock = fnic_io_lock_tag(fnic, i);
-		spin_lock_irqsave(io_lock, flags);
-		sc = scsi_host_find_tag(fnic->lport->host, i);
-		if (!sc) {
-			spin_unlock_irqrestore(io_lock, flags);
-			continue;
-		}
+	io_lock = fnic_io_lock_tag(fnic, sc->request->tag);
+	spin_lock_irqsave(io_lock, flags);
 
-		io_req = (struct fnic_io_req *)CMD_SP(sc);
-		if ((CMD_FLAGS(sc) & FNIC_DEVICE_RESET) &&
-			!(CMD_FLAGS(sc) & FNIC_DEV_RST_DONE)) {
-			/*
-			 * We will be here only when FW completes reset
-			 * without sending completions for outstanding ios.
-			 */
-			CMD_FLAGS(sc) |= FNIC_DEV_RST_DONE;
-			if (io_req && io_req->dr_done)
-				complete(io_req->dr_done);
-			else if (io_req && io_req->abts_done)
-				complete(io_req->abts_done);
-			spin_unlock_irqrestore(io_lock, flags);
-			continue;
-		} else if (CMD_FLAGS(sc) & FNIC_DEVICE_RESET) {
-			spin_unlock_irqrestore(io_lock, flags);
-			continue;
-		}
-		if (!io_req) {
-			spin_unlock_irqrestore(io_lock, flags);
-			goto cleanup_scsi_cmd;
-		}
+	io_req = (struct fnic_io_req *)CMD_SP(sc);
+	if ((CMD_FLAGS(sc) & FNIC_DEVICE_RESET) &&
+	    !(CMD_FLAGS(sc) & FNIC_DEV_RST_DONE)) {
+		/*
+		 * We will be here only when FW completes reset
+		 * without sending completions for outstanding ios.
+		 */
+		CMD_FLAGS(sc) |= FNIC_DEV_RST_DONE;
+		if (io_req && io_req->dr_done)
+			complete(io_req->dr_done);
+		else if (io_req && io_req->abts_done)
+			complete(io_req->abts_done);
+		spin_unlock_irqrestore(io_lock, flags);
+		return true;
+	} else if (CMD_FLAGS(sc) & FNIC_DEVICE_RESET) {
+		spin_unlock_irqrestore(io_lock, flags);
+		return true;
+	}
+	if (!io_req) {
+		spin_unlock_irqrestore(io_lock, flags);
+		goto cleanup_scsi_cmd;
+	}
 
-		CMD_SP(sc) = NULL;
+	CMD_SP(sc) = NULL;
 
-		spin_unlock_irqrestore(io_lock, flags);
+	spin_unlock_irqrestore(io_lock, flags);
 
-		/*
-		 * If there is a scsi_cmnd associated with this io_req, then
-		 * free the corresponding state
-		 */
-		start_time = io_req->start_time;
-		fnic_release_ioreq_buf(fnic, io_req, sc);
-		mempool_free(io_req, fnic->io_req_pool);
+	/*
+	 * If there is a scsi_cmnd associated with this io_req, then
+	 * free the corresponding state
+	 */
+	start_time = io_req->start_time;
+	fnic_release_ioreq_buf(fnic, io_req, sc);
+	mempool_free(io_req, fnic->io_req_pool);
 
 cleanup_scsi_cmd:
-		sc->result = DID_TRANSPORT_DISRUPTED << 16;
-		FNIC_SCSI_DBG(KERN_DEBUG, fnic->lport->host,
-			      "%s: tag:0x%x : sc:0x%p duration = %lu DID_TRANSPORT_DISRUPTED\n",
-			      __func__, sc->request->tag, sc,
-			      (jiffies - start_time));
-
-		if (atomic64_read(&fnic->io_cmpl_skip))
-			atomic64_dec(&fnic->io_cmpl_skip);
-		else
-			atomic64_inc(&fnic_stats->io_stats.io_completions);
+	sc->result = DID_TRANSPORT_DISRUPTED << 16;
+	FNIC_SCSI_DBG(KERN_DEBUG, fnic->lport->host,
+		      "fnic_cleanup_io: tag:0x%x : sc:0x%p duration = %lu DID_TRANSPORT_DISRUPTED\n",
+		      sc->request->tag, sc, (jiffies - start_time));
 
-		/* Complete the command to SCSI */
-		if (sc->scsi_done) {
-			if (!(CMD_FLAGS(sc) & FNIC_IO_ISSUED))
-				shost_printk(KERN_ERR, fnic->lport->host,
-				"Calling done for IO not issued to fw: tag:0x%x sc:0x%p\n",
-				 sc->request->tag, sc);
+	if (atomic64_read(&fnic->io_cmpl_skip))
+		atomic64_dec(&fnic->io_cmpl_skip);
+	else
+		atomic64_inc(&fnic_stats->io_stats.io_completions);
 
-			FNIC_TRACE(fnic_cleanup_io,
-				  sc->device->host->host_no, i, sc,
-				  jiffies_to_msecs(jiffies - start_time),
-				  0, ((u64)sc->cmnd[0] << 32 |
-				  (u64)sc->cmnd[2] << 24 |
-				  (u64)sc->cmnd[3] << 16 |
-				  (u64)sc->cmnd[4] << 8 | sc->cmnd[5]),
-				  (((u64)CMD_FLAGS(sc) << 32) | CMD_STATE(sc)));
+	/* Complete the command to SCSI */
+	if (sc->scsi_done) {
+		if (!(CMD_FLAGS(sc) & FNIC_IO_ISSUED))
+			shost_printk(KERN_ERR, fnic->lport->host,
+				     "Calling done for IO not issued to fw: tag:0x%x sc:0x%p\n",
+				     sc->request->tag, sc);
+
+		FNIC_TRACE(fnic_cleanup_io,
+			   sc->device->host->host_no, sc->request->tag, sc,
+			   jiffies_to_msecs(jiffies - start_time),
+			   0, ((u64)sc->cmnd[0] << 32 |
+			       (u64)sc->cmnd[2] << 24 |
+			       (u64)sc->cmnd[3] << 16 |
+			       (u64)sc->cmnd[4] << 8 | sc->cmnd[5]),
+			   (((u64)CMD_FLAGS(sc) << 32) | CMD_STATE(sc)));
 
-			sc->scsi_done(sc);
-		}
+		sc->scsi_done(sc);
 	}
+	return true;
+}
+
+static void fnic_cleanup_io(struct fnic *fnic)
+{
+	scsi_host_busy_iter(fnic->lport->host,
+			    fnic_cleanup_io_iter, fnic);
 }
 
 void fnic_wq_copy_cleanup_handler(struct vnic_wq_copy *wq,
@@ -1554,143 +1553,140 @@ static inline int fnic_queue_abort_io_req(struct fnic *fnic, int tag,
 	return 0;
 }
 
-static void fnic_rport_exch_reset(struct fnic *fnic, u32 port_id)
+struct fnic_rport_abort_io_iter_data {
+	struct fnic *fnic;
+	u32 port_id;
+	int term_cnt;
+};
+
+static bool fnic_rport_abort_io(struct scsi_cmnd *sc, void *data, bool reserved)
 {
-	int tag;
-	int abt_tag;
-	int term_cnt = 0;
+	struct fnic_rport_abort_io_iter_data *iter_data = data;
+	struct fnic *fnic = iter_data->fnic;
+	int abt_tag = sc->request->tag;
 	struct fnic_io_req *io_req;
 	spinlock_t *io_lock;
 	unsigned long flags;
-	struct scsi_cmnd *sc;
 	struct reset_stats *reset_stats = &fnic->fnic_stats.reset_stats;
 	struct terminate_stats *term_stats = &fnic->fnic_stats.term_stats;
 	struct scsi_lun fc_lun;
 	enum fnic_ioreq_state old_ioreq_state;
 
-	FNIC_SCSI_DBG(KERN_DEBUG,
-		      fnic->lport->host,
-		      "fnic_rport_exch_reset called portid 0x%06x\n",
-		      port_id);
-
-	if (fnic->in_remove)
-		return;
-
-	for (tag = 0; tag < fnic->fnic_max_tag_id; tag++) {
-		abt_tag = tag;
-		io_lock = fnic_io_lock_tag(fnic, tag);
-		spin_lock_irqsave(io_lock, flags);
-		sc = scsi_host_find_tag(fnic->lport->host, tag);
-		if (!sc) {
-			spin_unlock_irqrestore(io_lock, flags);
-			continue;
-		}
+	io_lock = fnic_io_lock_tag(fnic, abt_tag);
+	spin_lock_irqsave(io_lock, flags);
 
-		io_req = (struct fnic_io_req *)CMD_SP(sc);
+	io_req = (struct fnic_io_req *)CMD_SP(sc);
 
-		if (!io_req || io_req->port_id != port_id) {
-			spin_unlock_irqrestore(io_lock, flags);
-			continue;
-		}
+	if (!io_req || io_req->port_id != iter_data->port_id) {
+		spin_unlock_irqrestore(io_lock, flags);
+		return true;
+	}
 
-		if ((CMD_FLAGS(sc) & FNIC_DEVICE_RESET) &&
-			(!(CMD_FLAGS(sc) & FNIC_DEV_RST_ISSUED))) {
-			FNIC_SCSI_DBG(KERN_DEBUG, fnic->lport->host,
+	if ((CMD_FLAGS(sc) & FNIC_DEVICE_RESET) &&
+	    (!(CMD_FLAGS(sc) & FNIC_DEV_RST_ISSUED))) {
+		FNIC_SCSI_DBG(KERN_DEBUG, fnic->lport->host,
 			"fnic_rport_exch_reset dev rst not pending sc 0x%p\n",
 			sc);
-			spin_unlock_irqrestore(io_lock, flags);
-			continue;
-		}
+		spin_unlock_irqrestore(io_lock, flags);
+		return true;
+	}
 
-		/*
-		 * Found IO that is still pending with firmware and
-		 * belongs to rport that went away
-		 */
-		if (CMD_STATE(sc) == FNIC_IOREQ_ABTS_PENDING) {
-			spin_unlock_irqrestore(io_lock, flags);
-			continue;
-		}
-		if (io_req->abts_done) {
-			shost_printk(KERN_ERR, fnic->lport->host,
+	/*
+	 * Found IO that is still pending with firmware and
+	 * belongs to rport that went away
+	 */
+	if (CMD_STATE(sc) == FNIC_IOREQ_ABTS_PENDING) {
+		spin_unlock_irqrestore(io_lock, flags);
+		return true;
+	}
+	if (io_req->abts_done) {
+		shost_printk(KERN_ERR, fnic->lport->host,
 			"fnic_rport_exch_reset: io_req->abts_done is set "
 			"state is %s\n",
 			fnic_ioreq_state_to_str(CMD_STATE(sc)));
-		}
+	}
 
-		if (!(CMD_FLAGS(sc) & FNIC_IO_ISSUED)) {
-			shost_printk(KERN_ERR, fnic->lport->host,
-				  "rport_exch_reset "
-				  "IO not yet issued %p tag 0x%x flags "
-				  "%x state %d\n",
-				  sc, tag, CMD_FLAGS(sc), CMD_STATE(sc));
-		}
-		old_ioreq_state = CMD_STATE(sc);
-		CMD_STATE(sc) = FNIC_IOREQ_ABTS_PENDING;
-		CMD_ABTS_STATUS(sc) = FCPIO_INVALID_CODE;
-		if (CMD_FLAGS(sc) & FNIC_DEVICE_RESET) {
-			atomic64_inc(&reset_stats->device_reset_terminates);
-			abt_tag = (tag | FNIC_TAG_DEV_RST);
-			FNIC_SCSI_DBG(KERN_DEBUG, fnic->lport->host,
-			"fnic_rport_exch_reset dev rst sc 0x%p\n",
-			sc);
-		}
+	if (!(CMD_FLAGS(sc) & FNIC_IO_ISSUED)) {
+		shost_printk(KERN_ERR, fnic->lport->host,
+			     "rport_exch_reset "
+			     "IO not yet issued %p tag 0x%x flags "
+			     "%x state %d\n",
+			     sc, abt_tag, CMD_FLAGS(sc), CMD_STATE(sc));
+	}
+	old_ioreq_state = CMD_STATE(sc);
+	CMD_STATE(sc) = FNIC_IOREQ_ABTS_PENDING;
+	CMD_ABTS_STATUS(sc) = FCPIO_INVALID_CODE;
+	if (CMD_FLAGS(sc) & FNIC_DEVICE_RESET) {
+		atomic64_inc(&reset_stats->device_reset_terminates);
+		abt_tag |= FNIC_TAG_DEV_RST;
+	}
+	FNIC_SCSI_DBG(KERN_DEBUG, fnic->lport->host,
+		      "fnic_rport_exch_reset dev rst sc 0x%p\n", sc);
+	BUG_ON(io_req->abts_done);
+
+	FNIC_SCSI_DBG(KERN_DEBUG, fnic->lport->host,
+		      "fnic_rport_reset_exch: Issuing abts\n");
 
-		BUG_ON(io_req->abts_done);
+	spin_unlock_irqrestore(io_lock, flags);
 
-		FNIC_SCSI_DBG(KERN_DEBUG, fnic->lport->host,
-			      "fnic_rport_reset_exch: Issuing abts\n");
+	/* Now queue the abort command to firmware */
+	int_to_scsilun(sc->device->lun, &fc_lun);
 
+	if (fnic_queue_abort_io_req(fnic, abt_tag,
+				    FCPIO_ITMF_ABT_TASK_TERM,
+				    fc_lun.scsi_lun, io_req)) {
+		/*
+		 * Revert the cmd state back to old state, if
+		 * it hasn't changed in between. This cmd will get
+		 * aborted later by scsi_eh, or cleaned up during
+		 * lun reset
+		 */
+		spin_lock_irqsave(io_lock, flags);
+		if (CMD_STATE(sc) == FNIC_IOREQ_ABTS_PENDING)
+			CMD_STATE(sc) = old_ioreq_state;
+		spin_unlock_irqrestore(io_lock, flags);
+	} else {
+		spin_lock_irqsave(io_lock, flags);
+		if (CMD_FLAGS(sc) & FNIC_DEVICE_RESET)
+			CMD_FLAGS(sc) |= FNIC_DEV_RST_TERM_ISSUED;
+		else
+			CMD_FLAGS(sc) |= FNIC_IO_INTERNAL_TERM_ISSUED;
 		spin_unlock_irqrestore(io_lock, flags);
+		atomic64_inc(&term_stats->terminates);
+		iter_data->term_cnt++;
+	}
+	return true;
+}
 
-		/* Now queue the abort command to firmware */
-		int_to_scsilun(sc->device->lun, &fc_lun);
+static void fnic_rport_exch_reset(struct fnic *fnic, u32 port_id)
+{
+	struct terminate_stats *term_stats = &fnic->fnic_stats.term_stats;
+	struct fnic_rport_abort_io_iter_data iter_data = {
+		.fnic = fnic,
+		.port_id = port_id,
+		.term_cnt = 0,
+	};
 
-		if (fnic_queue_abort_io_req(fnic, abt_tag,
-					    FCPIO_ITMF_ABT_TASK_TERM,
-					    fc_lun.scsi_lun, io_req)) {
-			/*
-			 * Revert the cmd state back to old state, if
-			 * it hasn't changed in between. This cmd will get
-			 * aborted later by scsi_eh, or cleaned up during
-			 * lun reset
-			 */
-			spin_lock_irqsave(io_lock, flags);
-			if (CMD_STATE(sc) == FNIC_IOREQ_ABTS_PENDING)
-				CMD_STATE(sc) = old_ioreq_state;
-			spin_unlock_irqrestore(io_lock, flags);
-		} else {
-			spin_lock_irqsave(io_lock, flags);
-			if (CMD_FLAGS(sc) & FNIC_DEVICE_RESET)
-				CMD_FLAGS(sc) |= FNIC_DEV_RST_TERM_ISSUED;
-			else
-				CMD_FLAGS(sc) |= FNIC_IO_INTERNAL_TERM_ISSUED;
-			spin_unlock_irqrestore(io_lock, flags);
-			atomic64_inc(&term_stats->terminates);
-			term_cnt++;
-		}
-	}
-	if (term_cnt > atomic64_read(&term_stats->max_terminates))
-		atomic64_set(&term_stats->max_terminates, term_cnt);
+	FNIC_SCSI_DBG(KERN_DEBUG,
+		      fnic->lport->host,
+		      "fnic_rport_exch_reset called portid 0x%06x\n",
+		      port_id);
+
+	if (fnic->in_remove)
+		return;
+
+	scsi_host_busy_iter(fnic->lport->host, fnic_rport_abort_io,
+			    &iter_data);
+	if (iter_data.term_cnt > atomic64_read(&term_stats->max_terminates))
+		atomic64_set(&term_stats->max_terminates, iter_data.term_cnt);
 
 }
 
 void fnic_terminate_rport_io(struct fc_rport *rport)
 {
-	int tag;
-	int abt_tag;
-	int term_cnt = 0;
-	struct fnic_io_req *io_req;
-	spinlock_t *io_lock;
-	unsigned long flags;
-	struct scsi_cmnd *sc;
-	struct scsi_lun fc_lun;
 	struct fc_rport_libfc_priv *rdata;
 	struct fc_lport *lport;
 	struct fnic *fnic;
-	struct fc_rport *cmd_rport;
-	struct reset_stats *reset_stats;
-	struct terminate_stats *term_stats;
-	enum fnic_ioreq_state old_ioreq_state;
 
 	if (!rport) {
 		printk(KERN_ERR "fnic_terminate_rport_io: rport is NULL\n");
@@ -1718,109 +1714,7 @@ void fnic_terminate_rport_io(struct fc_rport *rport)
 	if (fnic->in_remove)
 		return;
 
-	reset_stats = &fnic->fnic_stats.reset_stats;
-	term_stats = &fnic->fnic_stats.term_stats;
-
-	for (tag = 0; tag < fnic->fnic_max_tag_id; tag++) {
-		abt_tag = tag;
-		io_lock = fnic_io_lock_tag(fnic, tag);
-		spin_lock_irqsave(io_lock, flags);
-		sc = scsi_host_find_tag(fnic->lport->host, tag);
-		if (!sc) {
-			spin_unlock_irqrestore(io_lock, flags);
-			continue;
-		}
-
-		cmd_rport = starget_to_rport(scsi_target(sc->device));
-		if (rport != cmd_rport) {
-			spin_unlock_irqrestore(io_lock, flags);
-			continue;
-		}
-
-		io_req = (struct fnic_io_req *)CMD_SP(sc);
-
-		if (!io_req || rport != cmd_rport) {
-			spin_unlock_irqrestore(io_lock, flags);
-			continue;
-		}
-
-		if ((CMD_FLAGS(sc) & FNIC_DEVICE_RESET) &&
-			(!(CMD_FLAGS(sc) & FNIC_DEV_RST_ISSUED))) {
-			FNIC_SCSI_DBG(KERN_DEBUG, fnic->lport->host,
-			"fnic_terminate_rport_io dev rst not pending sc 0x%p\n",
-			sc);
-			spin_unlock_irqrestore(io_lock, flags);
-			continue;
-		}
-		/*
-		 * Found IO that is still pending with firmware and
-		 * belongs to rport that went away
-		 */
-		if (CMD_STATE(sc) == FNIC_IOREQ_ABTS_PENDING) {
-			spin_unlock_irqrestore(io_lock, flags);
-			continue;
-		}
-		if (io_req->abts_done) {
-			shost_printk(KERN_ERR, fnic->lport->host,
-			"fnic_terminate_rport_io: io_req->abts_done is set "
-			"state is %s\n",
-			fnic_ioreq_state_to_str(CMD_STATE(sc)));
-		}
-		if (!(CMD_FLAGS(sc) & FNIC_IO_ISSUED)) {
-			FNIC_SCSI_DBG(KERN_INFO, fnic->lport->host,
-				  "fnic_terminate_rport_io "
-				  "IO not yet issued %p tag 0x%x flags "
-				  "%x state %d\n",
-				  sc, tag, CMD_FLAGS(sc), CMD_STATE(sc));
-		}
-		old_ioreq_state = CMD_STATE(sc);
-		CMD_STATE(sc) = FNIC_IOREQ_ABTS_PENDING;
-		CMD_ABTS_STATUS(sc) = FCPIO_INVALID_CODE;
-		if (CMD_FLAGS(sc) & FNIC_DEVICE_RESET) {
-			atomic64_inc(&reset_stats->device_reset_terminates);
-			abt_tag = (tag | FNIC_TAG_DEV_RST);
-			FNIC_SCSI_DBG(KERN_DEBUG, fnic->lport->host,
-			"fnic_terminate_rport_io dev rst sc 0x%p\n", sc);
-		}
-
-		BUG_ON(io_req->abts_done);
-
-		FNIC_SCSI_DBG(KERN_DEBUG,
-			      fnic->lport->host,
-			      "fnic_terminate_rport_io: Issuing abts\n");
-
-		spin_unlock_irqrestore(io_lock, flags);
-
-		/* Now queue the abort command to firmware */
-		int_to_scsilun(sc->device->lun, &fc_lun);
-
-		if (fnic_queue_abort_io_req(fnic, abt_tag,
-					    FCPIO_ITMF_ABT_TASK_TERM,
-					    fc_lun.scsi_lun, io_req)) {
-			/*
-			 * Revert the cmd state back to old state, if
-			 * it hasn't changed in between. This cmd will get
-			 * aborted later by scsi_eh, or cleaned up during
-			 * lun reset
-			 */
-			spin_lock_irqsave(io_lock, flags);
-			if (CMD_STATE(sc) == FNIC_IOREQ_ABTS_PENDING)
-				CMD_STATE(sc) = old_ioreq_state;
-			spin_unlock_irqrestore(io_lock, flags);
-		} else {
-			spin_lock_irqsave(io_lock, flags);
-			if (CMD_FLAGS(sc) & FNIC_DEVICE_RESET)
-				CMD_FLAGS(sc) |= FNIC_DEV_RST_TERM_ISSUED;
-			else
-				CMD_FLAGS(sc) |= FNIC_IO_INTERNAL_TERM_ISSUED;
-			spin_unlock_irqrestore(io_lock, flags);
-			atomic64_inc(&term_stats->terminates);
-			term_cnt++;
-		}
-	}
-	if (term_cnt > atomic64_read(&term_stats->max_terminates))
-		atomic64_set(&term_stats->max_terminates, term_cnt);
-
+	fnic_rport_exch_reset(fnic, rport->port_id);
 }
 
 /*
@@ -2115,163 +2009,174 @@ static inline int fnic_queue_dr_io_req(struct fnic *fnic,
 	return ret;
 }
 
-/*
- * Clean up any pending aborts on the lun
- * For each outstanding IO on this lun, whose abort is not completed by fw,
- * issue a local abort. Wait for abort to complete. Return 0 if all commands
- * successfully aborted, 1 otherwise
- */
-static int fnic_clean_pending_aborts(struct fnic *fnic,
-				     struct scsi_cmnd *lr_sc)
+struct fnic_pending_aborts_iter_data {
+	struct fnic *fnic;
+	struct scsi_device *lun_dev;
+	int ret;
+};
+
+static bool fnic_pending_aborts_iter(struct scsi_cmnd *sc,
+				     void *data, bool reserved)
 {
-	int tag, abt_tag;
+	struct fnic_pending_aborts_iter_data *iter_data = data;
+	struct fnic *fnic = iter_data->fnic;
+	struct scsi_device *lun_dev = iter_data->lun_dev;
+	int abt_tag = sc->request->tag;
 	struct fnic_io_req *io_req;
 	spinlock_t *io_lock;
 	unsigned long flags;
-	int ret = 0;
-	struct scsi_cmnd *sc;
 	struct scsi_lun fc_lun;
-	struct scsi_device *lun_dev = lr_sc->device;
 	DECLARE_COMPLETION_ONSTACK(tm_done);
 	enum fnic_ioreq_state old_ioreq_state;
 
-	for (tag = 0; tag < fnic->fnic_max_tag_id; tag++) {
-		io_lock = fnic_io_lock_tag(fnic, tag);
-		spin_lock_irqsave(io_lock, flags);
-		sc = scsi_host_find_tag(fnic->lport->host, tag);
-		/*
-		 * ignore this lun reset cmd if issued using new SC
-		 * or cmds that do not belong to this lun
-		 */
-		if (!sc || sc == lr_sc || sc->device != lun_dev) {
-			spin_unlock_irqrestore(io_lock, flags);
-			continue;
-		}
-
-		io_req = (struct fnic_io_req *)CMD_SP(sc);
-
-		if (!io_req || sc->device != lun_dev) {
-			spin_unlock_irqrestore(io_lock, flags);
-			continue;
-		}
-
-		/*
-		 * Found IO that is still pending with firmware and
-		 * belongs to the LUN that we are resetting
-		 */
-		FNIC_SCSI_DBG(KERN_DEBUG, fnic->lport->host,
-			      "Found IO in %s on lun\n",
-			      fnic_ioreq_state_to_str(CMD_STATE(sc)));
+	if (sc->device != lun_dev)
+		return true;
+	if (reserved)
+		return true;
 
-		if (CMD_STATE(sc) == FNIC_IOREQ_ABTS_PENDING) {
-			spin_unlock_irqrestore(io_lock, flags);
-			continue;
-		}
-		if ((CMD_FLAGS(sc) & FNIC_DEVICE_RESET) &&
-			(!(CMD_FLAGS(sc) & FNIC_DEV_RST_ISSUED))) {
-			FNIC_SCSI_DBG(KERN_INFO, fnic->lport->host,
-				"%s dev rst not pending sc 0x%p\n", __func__,
-				sc);
-			spin_unlock_irqrestore(io_lock, flags);
-			continue;
-		}
+	io_lock = fnic_io_lock_tag(fnic, abt_tag);
+	spin_lock_irqsave(io_lock, flags);
+	io_req = (struct fnic_io_req *)CMD_SP(sc);
+	if (!io_req) {
+		spin_unlock_irqrestore(io_lock, flags);
+		return true;
+	}
 
-		if (io_req->abts_done)
-			shost_printk(KERN_ERR, fnic->lport->host,
-			  "%s: io_req->abts_done is set state is %s\n",
-			  __func__, fnic_ioreq_state_to_str(CMD_STATE(sc)));
-		old_ioreq_state = CMD_STATE(sc);
-		/*
-		 * Any pending IO issued prior to reset is expected to be
-		 * in abts pending state, if not we need to set
-		 * FNIC_IOREQ_ABTS_PENDING to indicate the IO is abort pending.
-		 * When IO is completed, the IO will be handed over and
-		 * handled in this function.
-		 */
-		CMD_STATE(sc) = FNIC_IOREQ_ABTS_PENDING;
+	/*
+	 * Found IO that is still pending with firmware and
+	 * belongs to the LUN that we are resetting
+	 */
+	FNIC_SCSI_DBG(KERN_DEBUG, fnic->lport->host,
+		      "Found IO in %s on lun\n",
+		      fnic_ioreq_state_to_str(CMD_STATE(sc)));
 
-		BUG_ON(io_req->abts_done);
+	if (CMD_STATE(sc) == FNIC_IOREQ_ABTS_PENDING) {
+		spin_unlock_irqrestore(io_lock, flags);
+		return true;
+	}
+	if ((CMD_FLAGS(sc) & FNIC_DEVICE_RESET) &&
+	    (!(CMD_FLAGS(sc) & FNIC_DEV_RST_ISSUED))) {
+		FNIC_SCSI_DBG(KERN_INFO, fnic->lport->host,
+			      "%s dev rst not pending sc 0x%p\n", __func__,
+			      sc);
+		spin_unlock_irqrestore(io_lock, flags);
+		return true;
+	}
 
-		abt_tag = tag;
-		if (CMD_FLAGS(sc) & FNIC_DEVICE_RESET) {
-			abt_tag |= FNIC_TAG_DEV_RST;
-			FNIC_SCSI_DBG(KERN_INFO, fnic->lport->host,
-				  "%s: dev rst sc 0x%p\n", __func__, sc);
-		}
+	if (io_req->abts_done)
+		shost_printk(KERN_ERR, fnic->lport->host,
+			     "%s: io_req->abts_done is set state is %s\n",
+			     __func__, fnic_ioreq_state_to_str(CMD_STATE(sc)));
+	old_ioreq_state = CMD_STATE(sc);
+	/*
+	 * Any pending IO issued prior to reset is expected to be
+	 * in abts pending state, if not we need to set
+	 * FNIC_IOREQ_ABTS_PENDING to indicate the IO is abort pending.
+	 * When IO is completed, the IO will be handed over and
+	 * handled in this function.
+	 */
+	CMD_STATE(sc) = FNIC_IOREQ_ABTS_PENDING;
 
-		CMD_ABTS_STATUS(sc) = FCPIO_INVALID_CODE;
-		io_req->abts_done = &tm_done;
-		spin_unlock_irqrestore(io_lock, flags);
+	BUG_ON(io_req->abts_done);
 
-		/* Now queue the abort command to firmware */
-		int_to_scsilun(sc->device->lun, &fc_lun);
+	if (CMD_FLAGS(sc) & FNIC_DEVICE_RESET) {
+		abt_tag |= FNIC_TAG_DEV_RST;
+		FNIC_SCSI_DBG(KERN_INFO, fnic->lport->host,
+			      "%s: dev rst sc 0x%p\n", __func__, sc);
+	}
 
-		if (fnic_queue_abort_io_req(fnic, abt_tag,
-					    FCPIO_ITMF_ABT_TASK_TERM,
-					    fc_lun.scsi_lun, io_req)) {
-			spin_lock_irqsave(io_lock, flags);
-			io_req = (struct fnic_io_req *)CMD_SP(sc);
-			if (io_req)
-				io_req->abts_done = NULL;
-			if (CMD_STATE(sc) == FNIC_IOREQ_ABTS_PENDING)
-				CMD_STATE(sc) = old_ioreq_state;
-			spin_unlock_irqrestore(io_lock, flags);
-			ret = 1;
-			goto clean_pending_aborts_end;
-		} else {
-			spin_lock_irqsave(io_lock, flags);
-			if (CMD_FLAGS(sc) & FNIC_DEVICE_RESET)
-				CMD_FLAGS(sc) |= FNIC_DEV_RST_TERM_ISSUED;
-			spin_unlock_irqrestore(io_lock, flags);
-		}
-		CMD_FLAGS(sc) |= FNIC_IO_INTERNAL_TERM_ISSUED;
+	CMD_ABTS_STATUS(sc) = FCPIO_INVALID_CODE;
+	io_req->abts_done = &tm_done;
+	spin_unlock_irqrestore(io_lock, flags);
 
-		wait_for_completion_timeout(&tm_done,
-					    msecs_to_jiffies
-					    (fnic->config.ed_tov));
+	/* Now queue the abort command to firmware */
+	int_to_scsilun(sc->device->lun, &fc_lun);
 
-		/* Recheck cmd state to check if it is now aborted */
+	if (fnic_queue_abort_io_req(fnic, abt_tag,
+				    FCPIO_ITMF_ABT_TASK_TERM,
+				    fc_lun.scsi_lun, io_req)) {
 		spin_lock_irqsave(io_lock, flags);
 		io_req = (struct fnic_io_req *)CMD_SP(sc);
-		if (!io_req) {
-			spin_unlock_irqrestore(io_lock, flags);
-			CMD_FLAGS(sc) |= FNIC_IO_ABT_TERM_REQ_NULL;
-			continue;
-		}
+		if (io_req)
+			io_req->abts_done = NULL;
+		if (CMD_STATE(sc) == FNIC_IOREQ_ABTS_PENDING)
+			CMD_STATE(sc) = old_ioreq_state;
+		spin_unlock_irqrestore(io_lock, flags);
+		return false;
+	} else {
+		spin_lock_irqsave(io_lock, flags);
+		if (CMD_FLAGS(sc) & FNIC_DEVICE_RESET)
+			CMD_FLAGS(sc) |= FNIC_DEV_RST_TERM_ISSUED;
+		spin_unlock_irqrestore(io_lock, flags);
+	}
+	CMD_FLAGS(sc) |= FNIC_IO_INTERNAL_TERM_ISSUED;
 
-		io_req->abts_done = NULL;
+	wait_for_completion_timeout(&tm_done, msecs_to_jiffies
+				    (fnic->config.ed_tov));
 
-		/* if abort is still pending with fw, fail */
-		if (CMD_ABTS_STATUS(sc) == FCPIO_INVALID_CODE) {
-			spin_unlock_irqrestore(io_lock, flags);
-			CMD_FLAGS(sc) |= FNIC_IO_ABT_TERM_DONE;
-			ret = 1;
-			goto clean_pending_aborts_end;
-		}
-		CMD_STATE(sc) = FNIC_IOREQ_ABTS_COMPLETE;
+	/* Recheck cmd state to check if it is now aborted */
+	spin_lock_irqsave(io_lock, flags);
+	io_req = (struct fnic_io_req *)CMD_SP(sc);
+	if (!io_req) {
+		spin_unlock_irqrestore(io_lock, flags);
+		CMD_FLAGS(sc) |= FNIC_IO_ABT_TERM_REQ_NULL;
+		return true;
+	}
 
-		/* original sc used for lr is handled by dev reset code */
-		if (sc != lr_sc)
-			CMD_SP(sc) = NULL;
+	io_req->abts_done = NULL;
+
+	/* if abort is still pending with fw, fail */
+	if (CMD_ABTS_STATUS(sc) == FCPIO_INVALID_CODE) {
 		spin_unlock_irqrestore(io_lock, flags);
+		CMD_FLAGS(sc) |= FNIC_IO_ABT_TERM_DONE;
+		iter_data->ret = FAILED;
+		return false;
+	}
+	CMD_STATE(sc) = FNIC_IOREQ_ABTS_COMPLETE;
 
-		/* original sc used for lr is handled by dev reset code */
-		if (sc != lr_sc) {
-			fnic_release_ioreq_buf(fnic, io_req, sc);
-			mempool_free(io_req, fnic->io_req_pool);
-		}
+	/* original sc used for lr is handled by dev reset code */
+	CMD_SP(sc) = NULL;
+	spin_unlock_irqrestore(io_lock, flags);
 
-		/*
-		 * Any IO is returned during reset, it needs to call scsi_done
-		 * to return the scsi_cmnd to upper layer.
-		 */
-		if (sc->scsi_done) {
-			/* Set result to let upper SCSI layer retry */
-			sc->result = DID_RESET << 16;
-			sc->scsi_done(sc);
-		}
+	/* original sc used for lr is handled by dev reset code */
+	fnic_release_ioreq_buf(fnic, io_req, sc);
+	mempool_free(io_req, fnic->io_req_pool);
+
+	/*
+	 * Any IO is returned during reset, it needs to call scsi_done
+	 * to return the scsi_cmnd to upper layer.
+	 */
+	if (sc->scsi_done) {
+		/* Set result to let upper SCSI layer retry */
+		sc->result = DID_RESET << 16;
+		sc->scsi_done(sc);
 	}
+	return true;
+}
+
+/*
+ * Clean up any pending aborts on the lun
+ * For each outstanding IO on this lun, whose abort is not completed by fw,
+ * issue a local abort. Wait for abort to complete. Return 0 if all commands
+ * successfully aborted, 1 otherwise
+ */
+static int fnic_clean_pending_aborts(struct fnic *fnic,
+				     struct scsi_cmnd *lr_sc)
 
+{
+	int ret = SUCCESS;
+	struct fnic_pending_aborts_iter_data iter_data = {
+		.fnic = fnic,
+		.lun_dev = lr_sc->device,
+		.ret = SUCCESS,
+	};
+
+	scsi_host_busy_iter(fnic->lport->host,
+			    fnic_pending_aborts_iter, &iter_data);
+	if (iter_data.ret == FAILED) {
+		ret = iter_data.ret;
+		goto clean_pending_aborts_end;
+	}
 	schedule_timeout(msecs_to_jiffies(2 * fnic->config.ed_tov));
 
 	/* walk again to check, if IOs are still pending in fw */
@@ -2725,6 +2630,46 @@ void fnic_exch_mgr_reset(struct fc_lport *lp, u32 sid, u32 did)
 
 }
 
+static bool fnic_abts_pending_iter(struct scsi_cmnd *sc, void *data,
+				   bool reserved)
+{
+	struct fnic_pending_aborts_iter_data *iter_data = data;
+	struct fnic *fnic = iter_data->fnic;
+	int cmd_state;
+	struct fnic_io_req *io_req;
+	spinlock_t *io_lock;
+	unsigned long flags;
+
+	if (reserved)
+		return true;
+
+	if (iter_data->lun_dev && sc->device != iter_data->lun_dev)
+		return true;
+
+	io_lock = fnic_io_lock_hash(fnic, sc);
+	spin_lock_irqsave(io_lock, flags);
+
+	io_req = (struct fnic_io_req *)CMD_SP(sc);
+	if (!io_req) {
+		spin_unlock_irqrestore(io_lock, flags);
+		return true;
+	}
+
+	/*
+	 * Found IO that is still pending with firmware and
+	 * belongs to the LUN that we are resetting
+	 */
+	FNIC_SCSI_DBG(KERN_INFO, fnic->lport->host,
+		      "Found IO in %s on lun\n",
+		      fnic_ioreq_state_to_str(CMD_STATE(sc)));
+	cmd_state = CMD_STATE(sc);
+	spin_unlock_irqrestore(io_lock, flags);
+	if (cmd_state == FNIC_IOREQ_ABTS_PENDING)
+		iter_data->ret = 1;
+
+	return iter_data->ret ? false : true;
+}
+
 /*
  * fnic_is_abts_pending() is a helper function that
  * walks through tag map to check if there is any IOs pending,if there is one,
@@ -2734,49 +2679,18 @@ void fnic_exch_mgr_reset(struct fc_lport *lp, u32 sid, u32 did)
  */
 int fnic_is_abts_pending(struct fnic *fnic, struct scsi_cmnd *lr_sc)
 {
-	int tag;
-	struct fnic_io_req *io_req;
-	spinlock_t *io_lock;
-	unsigned long flags;
-	int ret = 0;
-	struct scsi_cmnd *sc;
-	struct scsi_device *lun_dev = NULL;
+	struct fnic_pending_aborts_iter_data iter_data = {
+		.fnic = fnic,
+		.lun_dev = NULL,
+		.ret = 0,
+	};
 
 	if (lr_sc)
-		lun_dev = lr_sc->device;
+		iter_data.lun_dev = lr_sc->device;
 
 	/* walk again to check, if IOs are still pending in fw */
-	for (tag = 0; tag < fnic->fnic_max_tag_id; tag++) {
-		sc = scsi_host_find_tag(fnic->lport->host, tag);
-		/*
-		 * ignore this lun reset cmd or cmds that do not belong to
-		 * this lun
-		 */
-		if (!sc || (lr_sc && (sc->device != lun_dev || sc == lr_sc)))
-			continue;
-
-		io_lock = fnic_io_lock_hash(fnic, sc);
-		spin_lock_irqsave(io_lock, flags);
-
-		io_req = (struct fnic_io_req *)CMD_SP(sc);
+	scsi_host_busy_iter(fnic->lport->host,
+			    fnic_abts_pending_iter, &iter_data);
 
-		if (!io_req || sc->device != lun_dev) {
-			spin_unlock_irqrestore(io_lock, flags);
-			continue;
-		}
-
-		/*
-		 * Found IO that is still pending with firmware and
-		 * belongs to the LUN that we are resetting
-		 */
-		FNIC_SCSI_DBG(KERN_INFO, fnic->lport->host,
-			      "Found IO in %s on lun\n",
-			      fnic_ioreq_state_to_str(CMD_STATE(sc)));
-
-		if (CMD_STATE(sc) == FNIC_IOREQ_ABTS_PENDING)
-			ret = 1;
-		spin_unlock_irqrestore(io_lock, flags);
-	}
-
-	return ret;
+	return iter_data.ret;
 }
Signature · 1 line
2.16.4
#44 lore raw

RFC PATCH v3 9/41 scsi: use real inquiry data when initialising devices

In reply to Hannes Reinecke

Use dummy inquiry data when initialising devices and not just a some string.

---
 drivers/scsi/scsi_scan.c | 20 +++++++++++---------
 1 file changed, 11 insertions(+), 9 deletions(-)
drivers/scsi/scsi_scan.c +11 -9
index f2437a7570ce..4648fd3f80d9 100644
@@ -81,7 +81,13 @@
 #define SCSI_SCAN_TARGET_PRESENT	1
 #define SCSI_SCAN_LUN_PRESENT		2
 
-static const char *scsi_null_device_strs = "nullnullnullnull";
+static const unsigned char scsi_null_inquiry[36] = {
+	0x3f, 0x00, 0x05, 0x02, 0x5b, 0x00, 0x00, 0x00,
+	0x4c, 0x49, 0x4e, 0x55, 0x58, 0x20, 0x20, 0x20,
+	0x56, 0x49, 0x52, 0x54, 0x55, 0x41, 0x4c, 0x4c,
+	0x55, 0x4e, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20,
+	0x31, 0x2e, 0x30, 0x20
+};
 
 #define MAX_SCSI_LUNS	512
 
@@ -224,9 +230,10 @@ static struct scsi_device *scsi_alloc_sdev(struct scsi_target *starget,
 	if (!sdev)
 		goto out;
 
-	sdev->vendor = scsi_null_device_strs;
-	sdev->model = scsi_null_device_strs;
-	sdev->rev = scsi_null_device_strs;
+	sdev->type = scsi_null_inquiry[0] & 0x1f;
+	sdev->vendor = scsi_null_inquiry + 8;
+	sdev->model = scsi_null_inquiry + 16;
+	sdev->rev = scsi_null_inquiry + 32;
 	sdev->host = shost;
 	sdev->queue_ramp_up_period = SCSI_DEFAULT_RAMP_UP_PERIOD;
 	sdev->id = starget->id;
@@ -253,11 +260,6 @@ static struct scsi_device *scsi_alloc_sdev(struct scsi_target *starget,
 	 * slave_configure function */
 	sdev->max_device_blocked = SCSI_DEFAULT_DEVICE_BLOCKED;
 
-	/*
-	 * Some low level driver could use device->type
-	 */
-	sdev->type = -1;
-
 	/*
 	 * Assume that the device will have handshaking problems,
 	 * and then fix this field later if it turns out it
Signature · 1 line
2.16.4
#45 lore raw

Re: [PATCH RFC v3 09/41] scsi: use real inquiry data when initialising devices

Quoted 2 lines · Hannes Reinecke

Use dummy inquiry data when initialising devices and not just a some string.

Why? And what do the values mean?

#46 lore raw

Re: [PATCH RFC v3 09/41] scsi: use real inquiry data when initialising devices

Quoted 6 lines · Christoph Hellwig
Quoted 2 lines · Hannes Reinecke

Use dummy inquiry data when initialising devices and not just a some string.

Why? And what do the values mean?

This is so that we can pass the entire inquiry data to the scsi device, pretty much the same way 'normal' scsi devices do. As for the values they are:

standard INQUIRY: [qualifier indicates no connected LU]
PQual=1 Device_type=31 RMB=0 LU_CONG=0 version=0x05 [SPC-3] [AERC=0] [TrmTsk=0] NormACA=0 HiSUP=0 Resp_data_format=2 SCCS=0 ACC=0 TPGS=0 3PC=0 Protect=0 [BQue=0] EncServ=0 MultiP=0 [MChngr=0] [ACKREQQ=0] Addr16=0 [RelAdr=0] WBus16=0 Sync=0 [Linked=0] [TranDis=0] CmdQue=0

    length=36 (0x24), Peripheral device type: no physical device on this lu

Vendor identification: LINUX
Product identification: VIRTUALLUN
Product revision level: 1.0

I'll add a verbose description to the byte string.

Cheers,

Hannes

Signature · 4 lines
Dr. Hannes Reinecke            Teamlead Storage & Networking
[email protected]                               +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer
#47 lore raw

RFC PATCH v3 10/41 scsi: make host device a first-class citizen

In reply to Hannes Reinecke

Rather than having the device created by scsi_get_host_dev() as a weird semi-initialized device make it a first class citizen by implementing a minimal command emulation and provide (static) inquiry data.

---
 drivers/scsi/scsi_lib.c    | 24 +++++++++++++++++++
 drivers/scsi/scsi_scan.c   | 60 +++++++++++++++++++++++++++++++++++-----------
 drivers/scsi/scsi_sysfs.c  |  3 ++-
 include/scsi/scsi_device.h |  1 +
 include/scsi/scsi_host.h   |  3 +++
 5 files changed, 76 insertions(+), 15 deletions(-)
drivers/scsi/scsi_lib.c +24 -0
index 30f9ca9fce22..ce9f1d83aaee 100644
@@ -1528,6 +1528,30 @@ static int scsi_dispatch_cmd(struct scsi_cmnd *cmd)
 
 	}
 
+	if (unlikely(scsi_device_is_virtual(cmd->device))) {
+		unsigned char null_report_luns[16];
+
+		switch (cmd->cmnd[0]) {
+		case TEST_UNIT_READY:
+			cmd->result = (DID_OK << 16);
+			goto done;
+		case INQUIRY:
+			scsi_sg_copy_from_buffer(cmd, cmd->device->inquiry, 36);
+			cmd->result = (DID_OK << 16);
+			goto done;
+		case REPORT_LUNS:
+			memset(null_report_luns, 0, 16);
+			null_report_luns[3] = 0x08;
+			scsi_sg_copy_from_buffer(cmd, null_report_luns, 16);
+			cmd->result = (DID_OK << 16);
+			goto done;
+		default:
+			scsi_build_sense_buffer(0, cmd->sense_buffer,
+						ILLEGAL_REQUEST, 0x20, 0x00);
+			cmd->result = (DID_NO_CONNECT << 16);
+			goto done;
+		}
+	}
 	trace_scsi_dispatch_cmd_start(cmd);
 	rtn = host->hostt->queuecommand(host, cmd);
 	if (rtn) {
drivers/scsi/scsi_scan.c +46 -14

132 lines not on this page · raw message

drivers/scsi/scsi_sysfs.c +2 -1
index 163dbcb741c1..c54011c2cdda 100644
@@ -485,7 +485,8 @@ static void scsi_device_dev_release_usercontext(struct work_struct *work)
 		kfree_rcu(vpd_pg80, rcu);
 	if (vpd_pg89)
 		kfree_rcu(vpd_pg89, rcu);
-	kfree(sdev->inquiry);
+	if (!scsi_device_is_virtual(sdev))
+		kfree(sdev->inquiry);
 	kfree(sdev);
 
 	if (parent)
include/scsi/scsi_device.h +1 -0
index e74c7e671aa0..6039ce7d09d7 100644
@@ -203,6 +203,7 @@ struct scsi_device {
 	unsigned unmap_limit_for_ws:1;	/* Use the UNMAP limit for WRITE SAME */
 	unsigned rpm_autosuspend:1;	/* Enable runtime autosuspend at device
 					 * creation time */
+	unsigned hidden:1;		/* Do not register with sysfs */
 
 	bool offline_already;		/* Device offline message logged */
 
include/scsi/scsi_host.h +3 -0
index 37bb7d74e4c4..6961cbc3b2c0 100644
@@ -781,7 +781,10 @@ struct class_container;
  * from any high-level drivers.
  */
 extern void scsi_free_host_dev(struct scsi_device *);
+extern struct scsi_device *scsi_get_virtual_dev(struct Scsi_Host *,
+						unsigned int, unsigned int);
 extern struct scsi_device *scsi_get_host_dev(struct Scsi_Host *);
+extern bool scsi_device_is_virtual(struct scsi_device *);
 
 /*
  * DIF defines the exchange of protection information between
Signature · 1 line
2.16.4
#48 lore raw

Re: [PATCH RFC v3 10/41] scsi: make host device a first-class citizen

Quoted 4 lines · Hannes Reinecke

Rather than having the device created by scsi_get_host_dev() as a weird semi-initialized device make it a first class citizen by implementing a minimal command emulation and provide (static) inquiry data.

It may be worth mentioning in the commit message that gdth and sierra_ms are the only two drivers that create a "host device".

Thanks,

Bart.

#49 lore raw

RFC PATCH v3 11/41 hpsa: move hpsa_hba_inquiry after scsi_add_host()

In reply to Hannes Reinecke

Move hpsa_hba_inquiry to after scsi_add_host() so that the host is fully initialized.

---
 drivers/scsi/hpsa.c | 37 +++++++++++++++++++------------------
 1 file changed, 19 insertions(+), 18 deletions(-)
drivers/scsi/hpsa.c +19 -18
index 1e9302e99d05..fe2fd1c8f4e0 100644
@@ -5835,6 +5835,22 @@ static int hpsa_scsi_host_alloc(struct ctlr_info *h)
 	return 0;
 }
 
+static void hpsa_hba_inquiry(struct ctlr_info *h)
+{
+	int rc;
+
+#define HBA_INQUIRY_BYTE_COUNT 64
+	h->hba_inquiry_data = kmalloc(HBA_INQUIRY_BYTE_COUNT, GFP_KERNEL);
+	if (!h->hba_inquiry_data)
+		return;
+	rc = hpsa_scsi_do_inquiry(h, RAID_CTLR_LUNID, 0,
+		h->hba_inquiry_data, HBA_INQUIRY_BYTE_COUNT);
+	if (rc != 0) {
+		kfree(h->hba_inquiry_data);
+		h->hba_inquiry_data = NULL;
+	}
+}
+
 static int hpsa_scsi_add_host(struct ctlr_info *h)
 {
 	int rv;
@@ -5844,6 +5860,9 @@ static int hpsa_scsi_add_host(struct ctlr_info *h)
 		dev_err(&h->pdev->dev, "scsi_add_host failed\n");
 		return rv;
 	}
+
+	hpsa_hba_inquiry(h);
+
 	scsi_scan_host(h->scsi_host);
 	return 0;
 }
@@ -7920,22 +7939,6 @@ static int hpsa_pci_init(struct ctlr_info *h)
 	return err;
 }
 
-static void hpsa_hba_inquiry(struct ctlr_info *h)
-{
-	int rc;
-
-#define HBA_INQUIRY_BYTE_COUNT 64
-	h->hba_inquiry_data = kmalloc(HBA_INQUIRY_BYTE_COUNT, GFP_KERNEL);
-	if (!h->hba_inquiry_data)
-		return;
-	rc = hpsa_scsi_do_inquiry(h, RAID_CTLR_LUNID, 0,
-		h->hba_inquiry_data, HBA_INQUIRY_BYTE_COUNT);
-	if (rc != 0) {
-		kfree(h->hba_inquiry_data);
-		h->hba_inquiry_data = NULL;
-	}
-}
-
 static int hpsa_init_reset_devices(struct pci_dev *pdev, u32 board_id)
 {
 	int rc, i;
@@ -8844,8 +8847,6 @@ static int hpsa_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
 	/* Turn the interrupts on so we can service requests */
 	h->access.set_intr_mask(h, HPSA_INTR_ON);
 
-	hpsa_hba_inquiry(h);
-
 	h->lastlogicals = kzalloc(sizeof(*(h->lastlogicals)), GFP_KERNEL);
 	if (!h->lastlogicals)
 		dev_info(&h->pdev->dev,
Signature · 1 line
2.16.4
#50 lore raw

RFC PATCH v3 12/41 hpsa: use reserved commands

In reply to Hannes Reinecke

From: Hannes Reinecke <[email protected]>

Enable the use of reserved commands, and drop the hand-crafted command allocation.

---
 drivers/scsi/hpsa.c | 203 +++++++++++++++++++---------------------------------
 drivers/scsi/hpsa.h |   3 +-
 2 files changed, 77 insertions(+), 129 deletions(-)
drivers/scsi/hpsa.c +75 -128
index fe2fd1c8f4e0..8f23011aade9 100644
@@ -244,10 +244,6 @@ static struct hpsa_scsi_dev_t
 	*hpsa_find_device_by_sas_rphy(struct ctlr_info *h,
 		struct sas_rphy *rphy);
 
-#define SCSI_CMD_BUSY ((struct scsi_cmnd *)&hpsa_cmd_busy)
-static const struct scsi_cmnd hpsa_cmd_busy;
-#define SCSI_CMD_IDLE ((struct scsi_cmnd *)&hpsa_cmd_idle)
-static const struct scsi_cmnd hpsa_cmd_idle;
 static int number_of_controllers;
 
 static irqreturn_t do_hpsa_intr_intx(int irq, void *dev_id);
@@ -261,7 +257,7 @@ static int hpsa_compat_ioctl(struct scsi_device *dev, unsigned int cmd,
 #endif
 
 static void cmd_free(struct ctlr_info *h, struct CommandList *c);
-static struct CommandList *cmd_alloc(struct ctlr_info *h);
+static struct CommandList *cmd_alloc(struct ctlr_info *h, u8 direction);
 static void cmd_tagged_free(struct ctlr_info *h, struct CommandList *c);
 static struct CommandList *cmd_tagged_alloc(struct ctlr_info *h,
 					    struct scsi_cmnd *scmd);
@@ -342,7 +338,7 @@ static inline struct ctlr_info *shost_to_hba(struct Scsi_Host *sh)
 
 static inline bool hpsa_is_cmd_idle(struct CommandList *c)
 {
-	return c->scsi_cmd == SCSI_CMD_IDLE;
+	return c->scsi_cmd == NULL;
 }
 
 /* extract sense key, asc, and ascq from sense data.  -1 means invalid. */
@@ -2450,7 +2446,12 @@ static void hpsa_cmd_resolve_events(struct ctlr_info *h,
 	 * this command has completed.  Then, check to see if the handler is
 	 * waiting for this command, and, if so, wake it.
 	 */
-	c->scsi_cmd = SCSI_CMD_IDLE;
+	if (c->scsi_cmd && c->cmd_type == CMD_IOCTL_PEND) {
+		struct scsi_cmnd *scmd = c->scsi_cmd;
+
+		scsi_put_reserved_cmd(scmd);
+	}
+	c->scsi_cmd = NULL;
 	mb();	/* Declare command idle before checking for pending events. */
 	if (dev) {
 		atomic_dec(&dev->commands_outstanding);
@@ -2993,7 +2994,7 @@ static int hpsa_do_receive_diagnostic(struct ctlr_info *h, u8 *scsi3addr,
 	struct CommandList *c;
 	struct ErrorInfo *ei;
 
-	c = cmd_alloc(h);
+	c = cmd_alloc(h, XFER_READ);
 	if (fill_cmd(c, RECEIVE_DIAGNOSTIC, h, buf, bufsize,
 			page, scsi3addr, TYPE_CMD)) {
 		rc = -1;
@@ -3045,7 +3046,7 @@ static int hpsa_scsi_do_inquiry(struct ctlr_info *h, unsigned char *scsi3addr,
 	struct CommandList *c;
 	struct ErrorInfo *ei;
 
-	c = cmd_alloc(h);
+	c = cmd_alloc(h, XFER_READ);
 
 	if (fill_cmd(c, HPSA_INQUIRY, h, buf, bufsize,
 			page, scsi3addr, TYPE_CMD)) {
@@ -3073,7 +3074,7 @@ static int hpsa_send_reset(struct ctlr_info *h, struct hpsa_scsi_dev_t *dev,
 	struct CommandList *c;
 	struct ErrorInfo *ei;
 
-	c = cmd_alloc(h);
+	c = cmd_alloc(h, XFER_NONE);
 	c->device = dev;
 
 	/* fill_cmd can't fail here, no data buffer to map. */
@@ -3299,7 +3300,7 @@ static int hpsa_get_raid_map(struct ctlr_info *h,
 	struct CommandList *c;
 	struct ErrorInfo *ei;
 
-	c = cmd_alloc(h);
+	c = cmd_alloc(h, XFER_READ);
 
 	if (fill_cmd(c, HPSA_GET_RAID_MAP, h, &this_device->raid_map,
 			sizeof(this_device->raid_map), 0,
@@ -3341,7 +3342,7 @@ static int hpsa_bmic_sense_subsystem_information(struct ctlr_info *h,
 	struct CommandList *c;
 	struct ErrorInfo *ei;
 
-	c = cmd_alloc(h);
+	c = cmd_alloc(h, XFER_READ);
 
 	rc = fill_cmd(c, BMIC_SENSE_SUBSYSTEM_INFORMATION, h, buf, bufsize,
 		0, RAID_CTLR_LUNID, TYPE_CMD);
@@ -3372,7 +3373,7 @@ static int hpsa_bmic_id_controller(struct ctlr_info *h,
 	struct CommandList *c;
 	struct ErrorInfo *ei;
 
-	c = cmd_alloc(h);
+	c = cmd_alloc(h, XFER_READ);
 
 	rc = fill_cmd(c, BMIC_IDENTIFY_CONTROLLER, h, buf, bufsize,
 		0, RAID_CTLR_LUNID, TYPE_CMD);
@@ -3401,7 +3402,7 @@ static int hpsa_bmic_id_physical_device(struct ctlr_info *h,
 	struct CommandList *c;
 	struct ErrorInfo *ei;
 
-	c = cmd_alloc(h);
+	c = cmd_alloc(h, XFER_READ);
 	rc = fill_cmd(c, BMIC_IDENTIFY_PHYSICAL_DEVICE, h, buf, bufsize,
 		0, RAID_CTLR_LUNID, TYPE_CMD);
 	if (rc)
@@ -3473,7 +3474,7 @@ static void hpsa_get_enclosure_info(struct ctlr_info *h,
 		goto out;
 	}
 
-	c = cmd_alloc(h);
+	c = cmd_alloc(h, XFER_READ);
 
 	rc = fill_cmd(c, BMIC_SENSE_STORAGE_BOX_PARAMS, h, bssbp,
 			sizeof(*bssbp), 0, RAID_CTLR_LUNID, TYPE_CMD);
@@ -3729,7 +3730,7 @@ static int hpsa_scsi_do_report_luns(struct ctlr_info *h, int logical,
 	unsigned char scsi3addr[8];
 	struct ErrorInfo *ei;
 
-	c = cmd_alloc(h);
+	c = cmd_alloc(h, XFER_READ);
 
 	/* address the controller */
 	memset(scsi3addr, 0, sizeof(scsi3addr));
@@ -3872,7 +3873,7 @@ static unsigned char hpsa_volume_offline(struct ctlr_info *h,
 #define ASCQ_LUN_NOT_READY_FORMAT_IN_PROGRESS 0x04
 #define ASCQ_LUN_NOT_READY_INITIALIZING_CMD_REQ 0x02
 
-	c = cmd_alloc(h);
+	c = cmd_alloc(h, XFER_NONE);
 
 	(void) fill_cmd(c, TEST_UNIT_READY, h, NULL, 0, 0, scsi3addr, TYPE_CMD);
 	rc = hpsa_scsi_do_simple_cmd(h, c, DEFAULT_REPLY_QUEUE,
@@ -5523,7 +5524,6 @@ static void hpsa_cmd_init(struct ctlr_info *h, int index,
 	c->ErrDesc.Addr = cpu_to_le64((u64) err_dma_handle);
 	c->ErrDesc.Len = cpu_to_le32((u32) sizeof(*c->err_info));
 	c->h = h;
-	c->scsi_cmd = SCSI_CMD_IDLE;
 }
 
 static void hpsa_preinitialize_commands(struct ctlr_info *h)
@@ -5818,12 +5818,12 @@ static int hpsa_scsi_host_alloc(struct ctlr_info *h)
 
 	sh->io_port = 0;
 	sh->n_io_port = 0;
-	sh->this_id = -1;
 	sh->max_channel = 3;
 	sh->max_cmd_len = MAX_COMMAND_SIZE;
 	sh->max_lun = HPSA_MAX_LUN;
 	sh->max_id = HPSA_MAX_LUN;
-	sh->can_queue = h->nr_cmds - HPSA_NRESERVED_CMDS;
+	sh->can_queue = h->nr_cmds;
+	sh->nr_reserved_cmds = HPSA_NRESERVED_CMDS;
 	sh->cmd_per_lun = sh->can_queue;
 	sh->sg_tablesize = h->maxsgentries;
 	sh->transportt = hpsa_sas_transport_template;
@@ -5860,30 +5860,19 @@ static int hpsa_scsi_add_host(struct ctlr_info *h)
 		dev_err(&h->pdev->dev, "scsi_add_host failed\n");
 		return rv;
 	}
-
+	h->raid_ctrl_sdev = scsi_get_virtual_dev(h->scsi_host,
+						 HPSA_LEGACY_HBA_BUS + 1, 0);
+	if (!h->raid_ctrl_sdev) {
+		dev_err(&h->pdev->dev,
+			"allocate raid controller device failed\n");
+		return -ENOMEM;
+	}
 	hpsa_hba_inquiry(h);
 
 	scsi_scan_host(h->scsi_host);
 	return 0;
 }
 
-/*
- * The block layer has already gone to the trouble of picking out a unique,
- * small-integer tag for this request.  We use an offset from that value as
- * an index to select our command block.  (The offset allows us to reserve the
- * low-numbered entries for our own uses.)
- */
-static int hpsa_get_cmd_index(struct scsi_cmnd *scmd)
-{
-	int idx = scmd->request->tag;
-
-	if (idx < 0)
-		return idx;
-
-	/* Offset to leave space for internal cmds. */
-	return idx += HPSA_NRESERVED_CMDS;
-}
-
 /*
  * Send a TEST_UNIT_READY command to the specified LUN using the specified
  * reply queue; returns zero if the unit is ready, and non-zero otherwise.
@@ -5967,7 +5956,7 @@ static int wait_for_device_to_become_ready(struct ctlr_info *h,
 	int rc = 0;
 	struct CommandList *c;
 
-	c = cmd_alloc(h);
+	c = cmd_alloc(h, XFER_NONE);
 
 	/*
 	 * If no specific reply queue was requested, then send the TUR
@@ -6040,7 +6029,7 @@ static int hpsa_eh_device_reset_handler(struct scsi_cmnd *scsicmd)
 	if (lockup_detected(h)) {
 		snprintf(msg, sizeof(msg),
 			 "cmd %d RESET FAILED, lockup detected",
-			 hpsa_get_cmd_index(scsicmd));
+			 scsicmd->request->tag);
 		hpsa_show_dev_msg(KERN_WARNING, h, dev, msg);
 		rc = FAILED;
 		goto return_reset_status;
@@ -6050,7 +6039,7 @@ static int hpsa_eh_device_reset_handler(struct scsi_cmnd *scsicmd)
 	if (detect_controller_lockup(h)) {
 		snprintf(msg, sizeof(msg),
 			 "cmd %d RESET FAILED, new lockup detected",
-			 hpsa_get_cmd_index(scsicmd));
+			  scsicmd->request->tag);
 		hpsa_show_dev_msg(KERN_WARNING, h, dev, msg);
 		rc = FAILED;
 		goto return_reset_status;
@@ -6112,12 +6101,12 @@ static int hpsa_eh_device_reset_handler(struct scsi_cmnd *scsicmd)
 static struct CommandList *cmd_tagged_alloc(struct ctlr_info *h,
 					    struct scsi_cmnd *scmd)
 {
-	int idx = hpsa_get_cmd_index(scmd);
+	int idx = scmd->request->tag;
 	struct CommandList *c = h->cmd_pool + idx;
 
-	if (idx < HPSA_NRESERVED_CMDS || idx >= h->nr_cmds) {
+	if (idx < 0 || idx >= h->nr_cmds) {
 		dev_err(&h->pdev->dev, "Bad block tag: %d not in [%d..%d]\n",
-			idx, HPSA_NRESERVED_CMDS, h->nr_cmds - 1);
+			idx, 0, h->nr_cmds - 1);
 		/* The index value comes from the block layer, so if it's out of
 		 * bounds, it's probably not our bug.
 		 */
@@ -6154,80 +6143,50 @@ static void cmd_tagged_free(struct ctlr_info *h, struct CommandList *c)
 	 * else to free it, because it is accessed by index.
 	 */
 	(void)atomic_dec(&c->refcount);
+	c->scsi_cmd = NULL;
 }
 
-/*
- * For operations that cannot sleep, a command block is allocated at init,
- * and managed by cmd_alloc() and cmd_free() using a simple bitmap to track
- * which ones are free or in use.  Lock must be held when calling this.
- * cmd_free() is the complement.
- * This function never gives up and returns NULL.  If it hangs,
- * another thread must call cmd_free() to free some tags.
- */
-
-static struct CommandList *cmd_alloc(struct ctlr_info *h)
+static struct CommandList *cmd_alloc(struct ctlr_info *h, u8 direction)
 {
+	struct scsi_cmnd *scmd;
 	struct CommandList *c;
-	int refcount, i;
-	int offset = 0;
+	int idx;
 
-	/*
-	 * There is some *extremely* small but non-zero chance that that
-	 * multiple threads could get in here, and one thread could
-	 * be scanning through the list of bits looking for a free
-	 * one, but the free ones are always behind him, and other
-	 * threads sneak in behind him and eat them before he can
-	 * get to them, so that while there is always a free one, a
-	 * very unlucky thread might be starved anyway, never able to
-	 * beat the other threads.  In reality, this happens so
-	 * infrequently as to be indistinguishable from never.
-	 *
-	 * Note that we start allocating commands before the SCSI host structure
-	 * is initialized.  Since the search starts at bit zero, this
-	 * all works, since we have at least one command structure available;
-	 * however, it means that the structures with the low indexes have to be
-	 * reserved for driver-initiated requests, while requests from the block
-	 * layer will use the higher indexes.
-	 */
-
-	for (;;) {
-		i = find_next_zero_bit(h->cmd_pool_bits,
-					HPSA_NRESERVED_CMDS,
-					offset);
-		if (unlikely(i >= HPSA_NRESERVED_CMDS)) {
-			offset = 0;
-			continue;
-		}
-		c = h->cmd_pool + i;
-		refcount = atomic_inc_return(&c->refcount);
-		if (unlikely(refcount > 1)) {
-			cmd_free(h, c); /* already in use */
-			offset = (i + 1) % HPSA_NRESERVED_CMDS;
-			continue;
-		}
-		set_bit(i & (BITS_PER_LONG - 1),
-			h->cmd_pool_bits + (i / BITS_PER_LONG));
-		break; /* it's ours now. */
+	scmd = scsi_get_reserved_cmd(h->raid_ctrl_sdev, direction & XFER_WRITE ?
+				     DMA_TO_DEVICE : DMA_FROM_DEVICE);
+	if (!scmd) {
+		dev_warn(&h->pdev->dev, "failed to allocate reserved cmd\n");
+		return NULL;
 	}
-	hpsa_cmd_partial_init(h, i, c);
+	idx = scmd->request->tag;
+	c = cmd_tagged_alloc(h, scmd);
+	if (!c) {
+		dev_warn(&h->pdev->dev, "failed to allocate reserved cmd %u\n",
+			 idx);
+		scsi_put_reserved_cmd(scmd);
+		return NULL;
+	}
+	hpsa_cmd_partial_init(h, idx, c);
+	c->scsi_cmd = scmd;
 	c->device = NULL;
+	c->cmd_type = CMD_IOCTL_PEND;
+	dev_dbg(&h->pdev->dev, "using reserved cmd %u\n", idx);
 	return c;
 }
 
-/*
- * This is the complementary operation to cmd_alloc().  Note, however, in some
- * corner cases it may also be used to free blocks allocated by
- * cmd_tagged_alloc() in which case the ref-count decrement does the trick and
- * the clear-bit is harmless.
- */
 static void cmd_free(struct ctlr_info *h, struct CommandList *c)
 {
-	if (atomic_dec_and_test(&c->refcount)) {
-		int i;
+	struct scsi_cmnd *scmd = c->scsi_cmd;
 
-		i = c - h->cmd_pool;
-		clear_bit(i & (BITS_PER_LONG - 1),
-			  h->cmd_pool_bits + (i / BITS_PER_LONG));
+	if (!scmd) {
+		dev_warn(&h->pdev->dev, "freeing idle cmd\n");
+		return;
+	}
+	cmd_tagged_free(h, c);
+	if (c->cmd_type == CMD_IOCTL_PEND) {
+		dev_dbg(&h->pdev->dev, "returning reserved cmd %u\n",
+			scmd->request->tag);
+		scsi_put_reserved_cmd(scmd);
 	}
 }
 
@@ -6410,11 +6369,8 @@ static int hpsa_passthru_ioctl(struct ctlr_info *h, void __user *argp)
 			memset(buff, 0, iocommand.buf_size);
 		}
 	}
-	c = cmd_alloc(h);
+	c = cmd_alloc(h, iocommand.Request.Type.Direction);
 
-	/* Fill in the command type */
-	c->cmd_type = CMD_IOCTL_PEND;
-	c->scsi_cmd = SCSI_CMD_BUSY;
 	/* Fill in Command Header */
 	c->Header.ReplyQueue = 0; /* unused in simple mode */
 	if (iocommand.buf_size > 0) {	/* buffer to fill */
@@ -6543,10 +6499,8 @@ static int hpsa_big_passthru_ioctl(struct ctlr_info *h, void __user *argp)
 		data_ptr += sz;
 		sg_used++;
 	}
-	c = cmd_alloc(h);
+	c = cmd_alloc(h, ioc->Request.Type.Direction);
 
-	c->cmd_type = CMD_IOCTL_PEND;
-	c->scsi_cmd = SCSI_CMD_BUSY;
 	c->Header.ReplyQueue = 0;
 	c->Header.SGList = (u8) sg_used;
 	c->Header.SGTotal = cpu_to_le16(sg_used);
@@ -6668,7 +6622,7 @@ static void hpsa_send_host_reset(struct ctlr_info *h, u8 reset_type)
 {
 	struct CommandList *c;
 
-	c = cmd_alloc(h);
+	c = cmd_alloc(h, XFER_NONE);
 
 	/* fill_cmd can't fail here, no data buffer to map */
 	(void) fill_cmd(c, HPSA_DEVICE_RESET_MSG, h, NULL, 0, 0,
@@ -6689,8 +6643,6 @@ static int fill_cmd(struct CommandList *c, u8 cmd, struct ctlr_info *h,
 {
 	enum dma_data_direction dir = DMA_NONE;
 
-	c->cmd_type = CMD_IOCTL_PEND;
-	c->scsi_cmd = SCSI_CMD_BUSY;
 	c->Header.ReplyQueue = 0;
 	if (buff != NULL && size > 0) {
 		c->Header.SGList = 1;
@@ -8003,8 +7955,6 @@ static int hpsa_init_reset_devices(struct pci_dev *pdev, u32 board_id)
 
 static void hpsa_free_cmd_pool(struct ctlr_info *h)
 {
-	kfree(h->cmd_pool_bits);
-	h->cmd_pool_bits = NULL;
 	if (h->cmd_pool) {
 		dma_free_coherent(&h->pdev->dev,
 				h->nr_cmds * sizeof(struct CommandList),
@@ -8025,17 +7975,13 @@ static void hpsa_free_cmd_pool(struct ctlr_info *h)
 
 static int hpsa_alloc_cmd_pool(struct ctlr_info *h)
 {
-	h->cmd_pool_bits = kcalloc(DIV_ROUND_UP(h->nr_cmds, BITS_PER_LONG),
-				   sizeof(unsigned long),
-				   GFP_KERNEL);
 	h->cmd_pool = dma_alloc_coherent(&h->pdev->dev,
 		    h->nr_cmds * sizeof(*h->cmd_pool),
 		    &h->cmd_pool_dhandle, GFP_KERNEL);
 	h->errinfo_pool = dma_alloc_coherent(&h->pdev->dev,
 		    h->nr_cmds * sizeof(*h->errinfo_pool),
 		    &h->errinfo_pool_dhandle, GFP_KERNEL);
-	if ((h->cmd_pool_bits == NULL)
-	    || (h->cmd_pool == NULL)
+	if ((h->cmd_pool == NULL)
 	    || (h->errinfo_pool == NULL)) {
 		dev_err(&h->pdev->dev, "out of memory in %s", __func__);
 		goto clean_up;
@@ -8918,7 +8864,7 @@ static void hpsa_flush_cache(struct ctlr_info *h)
 	if (!flush_buf)
 		return;
 
-	c = cmd_alloc(h);
+	c = cmd_alloc(h, XFER_NONE);
 
 	if (fill_cmd(c, HPSA_CACHE_FLUSH, h, flush_buf, 4, 0,
 		RAID_CTLR_LUNID, TYPE_CMD)) {
@@ -8953,7 +8899,7 @@ static void hpsa_disable_rld_caching(struct ctlr_info *h)
 	if (!options)
 		return;
 
-	c = cmd_alloc(h);
+	c = cmd_alloc(h, XFER_READ);
 
 	/* first, get the current diag options settings */
 	if (fill_cmd(c, BMIC_SENSE_DIAG_OPTIONS, h, options, 4, 0,
@@ -9003,11 +8949,10 @@ static void __hpsa_shutdown(struct pci_dev *pdev)
 	struct ctlr_info *h;
 
 	h = pci_get_drvdata(pdev);
-	/* Turn board interrupts off  and send the flush cache command
-	 * sendcmd will turn off interrupt, and send the flush...
-	 * To write all data in the battery backed cache to disks
+	/*
+	 * Turn board interrupts off;
+	 * flush cache command has already been sent.
 	 */
-	hpsa_flush_cache(h);
 	h->access.set_intr_mask(h, HPSA_INTR_OFF);
 	hpsa_free_irqs(h);			/* init_one 4 */
 	hpsa_disable_interrupt_mode(h);		/* pci_init 2 */
@@ -9015,6 +8960,7 @@ static void __hpsa_shutdown(struct pci_dev *pdev)
 
 static void hpsa_shutdown(struct pci_dev *pdev)
 {
+	hpsa_flush_cache(pci_get_drvdata(pdev));
 	__hpsa_shutdown(pdev);
 	pci_disable_device(pdev);
 }
@@ -9059,6 +9005,7 @@ static void hpsa_remove_one(struct pci_dev *pdev)
 	 * when multipath is enabled. There can be SYNCHRONIZE CACHE
 	 * operations which cannot complete and will hang the system.
 	 */
+	hpsa_flush_cache(h);
 	if (h->scsi_host)
 		scsi_remove_host(h->scsi_host);		/* init_one 8 */
 	/* includes hpsa_free_irqs - init_one 4 */
drivers/scsi/hpsa.h +2 -1
index f8c88fc7b80a..d5fcecbc4401 100644
@@ -205,7 +205,6 @@ struct ctlr_info {
 	dma_addr_t		ioaccel2_cmd_pool_dhandle;
 	struct ErrorInfo 	*errinfo_pool;
 	dma_addr_t		errinfo_pool_dhandle;
-	unsigned long  		*cmd_pool_bits;
 	int			scan_finished;
 	u8			scan_waiting : 1;
 	spinlock_t		scan_lock;
@@ -215,6 +214,8 @@ struct ctlr_info {
 	spinlock_t devlock; /* to protect hba[ctlr]->dev[];  */
 	int ndevices; /* number of used elements in .dev[] array. */
 	struct hpsa_scsi_dev_t *dev[HPSA_MAX_DEVICES];
+	struct scsi_device *raid_ctrl_sdev;
+
 	/*
 	 * Performant mode tables.
 	 */
Signature · 1 line
2.16.4
#51 lore raw

RFC PATCH v3 13/41 hpsa: use scsi_host_busy_iter() to traverse outstanding commands

In reply to Hannes Reinecke

From: Hannes Reinecke <[email protected]>

Replace all hand-crafted command iterations with
scsi_host_busy_iter() calls.

---
 drivers/scsi/hpsa.c | 117 ++++++++++++++++++++++++++++++----------------------
 1 file changed, 67 insertions(+), 50 deletions(-)
drivers/scsi/hpsa.c +67 -50
index 8f23011aade9..51e503a668d5 100644
@@ -1812,30 +1812,26 @@ static int hpsa_add_device(struct ctlr_info *h, struct hpsa_scsi_dev_t *device)
 	return rc;
 }
 
-static int hpsa_find_outstanding_commands_for_dev(struct ctlr_info *h,
-						struct hpsa_scsi_dev_t *dev)
-{
-	int i;
-	int count = 0;
-
-	for (i = 0; i < h->nr_cmds; i++) {
-		struct CommandList *c = h->cmd_pool + i;
-		int refcount = atomic_inc_return(&c->refcount);
-
-		if (refcount > 1 && hpsa_cmd_dev_match(h, c, dev,
-				dev->scsi3addr)) {
-			unsigned long flags;
+struct hpsa_command_iter_data {
+	struct ctlr_info *h;
+	struct hpsa_scsi_dev_t *dev;
+	unsigned char *scsi3addr;
+	int count;
+};
 
-			spin_lock_irqsave(&h->lock, flags);	/* Implied MB */
-			if (!hpsa_is_cmd_idle(c))
-				++count;
-			spin_unlock_irqrestore(&h->lock, flags);
-		}
+static bool hpsa_find_outstanding_commands_iter(struct scsi_cmnd *sc,
+						void *data, bool reserved)
+{
+	struct hpsa_command_iter_data *iter_data = data;
+	struct ctlr_info *h = iter_data->h;
+	struct hpsa_scsi_dev_t *dev = iter_data->dev;
+	struct CommandList *c = h->cmd_pool + sc->request->tag;
 
-		cmd_free(h, c);
+	if (hpsa_cmd_dev_match(h, c, dev, dev->scsi3addr)) {
+		iter_data->count++;
+		return false;
 	}
-
-	return count;
+	return true;
 }
 
 #define NUM_WAIT 20
@@ -1845,13 +1841,20 @@ static void hpsa_wait_for_outstanding_commands_for_dev(struct ctlr_info *h,
 	int cmds = 0;
 	int waits = 0;
 	int num_wait = NUM_WAIT;
+	struct hpsa_command_iter_data iter_data = {
+		.h = h,
+		.dev = device,
+	};
 
 	if (device->external)
 		num_wait = HPSA_EH_PTRAID_TIMEOUT;
 
 	while (1) {
-		cmds = hpsa_find_outstanding_commands_for_dev(h, device);
-		if (cmds == 0)
+		iter_data.count = 0;
+		scsi_host_busy_iter(h->scsi_host,
+				    hpsa_find_outstanding_commands_iter,
+				    &iter_data);
+		if (iter_data.count == 0)
 			break;
 		if (++waits > num_wait)
 			break;
@@ -8152,27 +8155,34 @@ static void hpsa_undo_allocations_after_kdump_soft_reset(struct ctlr_info *h)
 	kfree(h);				/* init_one 1 */
 }
 
+static bool fail_all_outstanding_cmds_iter(struct scsi_cmnd *sc, void *data,
+					   bool reserved)
+{
+	struct hpsa_command_iter_data *iter_data = data;
+	struct ctlr_info *h = iter_data->h;
+	struct CommandList *c = h->cmd_pool + sc->request->tag;
+
+	c->err_info->CommandStatus = CMD_CTLR_LOCKUP;
+	finish_cmd(c);
+	atomic_dec(&h->commands_outstanding);
+	iter_data->count++;
+
+	return true;
+}
+
 /* Called when controller lockup detected. */
 static void fail_all_outstanding_cmds(struct ctlr_info *h)
 {
-	int i, refcount;
-	struct CommandList *c;
-	int failcount = 0;
+	struct hpsa_command_iter_data iter_data = {
+		.h = h,
+		.count = 0,
+	};
 
 	flush_workqueue(h->resubmit_wq); /* ensure all cmds are fully built */
-	for (i = 0; i < h->nr_cmds; i++) {
-		c = h->cmd_pool + i;
-		refcount = atomic_inc_return(&c->refcount);
-		if (refcount > 1) {
-			c->err_info->CommandStatus = CMD_CTLR_LOCKUP;
-			finish_cmd(c);
-			atomic_dec(&h->commands_outstanding);
-			failcount++;
-		}
-		cmd_free(h, c);
-	}
+	scsi_host_busy_iter(h->scsi_host,
+			    fail_all_outstanding_cmds_iter, &iter_data);
 	dev_warn(&h->pdev->dev,
-		"failed %d commands in fail_all\n", failcount);
+		"failed %d commands in fail_all\n", iter_data.count);
 }
 
 static void set_lockup_detected_for_all_cpus(struct ctlr_info *h, u32 value)
@@ -9472,22 +9482,29 @@ static int is_accelerated_cmd(struct CommandList *c)
 	return c->cmd_type == CMD_IOACCEL1 || c->cmd_type == CMD_IOACCEL2;
 }
 
+static bool hpsa_drain_accel_commands_iter(struct scsi_cmnd *sc, void *data,
+					   bool reserved)
+{
+	struct hpsa_command_iter_data *iter_data = data;
+	struct ctlr_info *h = iter_data->h;
+	struct CommandList *c = h->cmd_pool + sc->request->tag;
+
+	iter_data->count += is_accelerated_cmd(c);
+	return true;
+}
+
 static void hpsa_drain_accel_commands(struct ctlr_info *h)
 {
-	struct CommandList *c = NULL;
-	int i, accel_cmds_out;
-	int refcount;
+	struct hpsa_command_iter_data iter_data = {
+		.h = h,
+	};
 
 	do { /* wait for all outstanding ioaccel commands to drain out */
-		accel_cmds_out = 0;
-		for (i = 0; i < h->nr_cmds; i++) {
-			c = h->cmd_pool + i;
-			refcount = atomic_inc_return(&c->refcount);
-			if (refcount > 1) /* Command is allocated */
-				accel_cmds_out += is_accelerated_cmd(c);
-			cmd_free(h, c);
-		}
-		if (accel_cmds_out <= 0)
+		iter_data.count = 0;
+		scsi_host_busy_iter(h->scsi_host,
+				    hpsa_drain_accel_commands_iter,
+				    &iter_data);
+		if (iter_data.count <= 0)
 			break;
 		msleep(100);
 	} while (1);
Signature · 1 line
2.16.4
#52 lore raw

RFC PATCH v3 14/41 hpsa: drop refcount field from CommandList

In reply to Hannes Reinecke

Field is now unused, so drop it.

---
 drivers/scsi/hpsa.c     | 12 ++----------
 drivers/scsi/hpsa_cmd.h |  1 -
 2 files changed, 2 insertions(+), 11 deletions(-)
drivers/scsi/hpsa.c +2 -10
index 51e503a668d5..6f66cec0e2cc 100644
@@ -5514,8 +5514,8 @@ static void hpsa_cmd_init(struct ctlr_info *h, int index,
 {
 	dma_addr_t cmd_dma_handle, err_dma_handle;
 
-	/* Zero out all of commandlist except the last field, refcount */
-	memset(c, 0, offsetof(struct CommandList, refcount));
+	/* Zero out all of commandlist */
+	memset(c, 0, sizeof(struct CommandList));
 	c->Header.tag = cpu_to_le64((u64) (index << DIRECT_LOOKUP_SHIFT));
 	cmd_dma_handle = h->cmd_pool_dhandle + index * sizeof(*c);
 	c->err_info = h->errinfo_pool + index;
@@ -5537,7 +5537,6 @@ static void hpsa_preinitialize_commands(struct ctlr_info *h)
 		struct CommandList *c = h->cmd_pool + i;
 
 		hpsa_cmd_init(h, i, c);
-		atomic_set(&c->refcount, 0);
 	}
 }
 
@@ -6133,19 +6132,12 @@ static struct CommandList *cmd_tagged_alloc(struct ctlr_info *h,
 		return NULL;
 	}
 
-	atomic_inc(&c->refcount);
-
 	hpsa_cmd_partial_init(h, idx, c);
 	return c;
 }
 
 static void cmd_tagged_free(struct ctlr_info *h, struct CommandList *c)
 {
-	/*
-	 * Release our reference to the block.  We don't need to do anything
-	 * else to free it, because it is accessed by index.
-	 */
-	(void)atomic_dec(&c->refcount);
 	c->scsi_cmd = NULL;
 }
 
drivers/scsi/hpsa_cmd.h +0 -1
index 7825cbfea4dc..2575a396f1a5 100644
@@ -449,7 +449,6 @@ struct CommandList {
 
 	int abort_pending;
 	struct hpsa_scsi_dev_t *device;
-	atomic_t refcount; /* Must be last to avoid memset in hpsa_cmd_init() */
 } __aligned(COMMANDLIST_ALIGNMENT);
 
 /* Max S/G elements in I/O accelerator command */
Signature · 1 line
2.16.4
#53 lore raw

RFC PATCH v3 15/41 aacraid: use private commands

In reply to Hannes Reinecke

From: Hannes Reinecke <[email protected]>

Use private commands to allocate internal commands.

---
 drivers/scsi/aacraid/aachba.c   | 14 ++++----
 drivers/scsi/aacraid/aacraid.h  |  9 +++--
 drivers/scsi/aacraid/commctrl.c | 57 ++++++++++++++++---------------
 drivers/scsi/aacraid/comminit.c | 34 +++++++-----------
 drivers/scsi/aacraid/commsup.c  | 76 +++++++++++++++++------------------------
 drivers/scsi/aacraid/dpcsup.c   |  2 +-
 drivers/scsi/aacraid/linit.c    | 54 +++++++++++++++--------------
 7 files changed, 116 insertions(+), 130 deletions(-)
drivers/scsi/aacraid/aachba.c +8 -6
index eb72ac8136c3..64bf36528579 100644
@@ -359,7 +359,7 @@ int aac_get_config_status(struct aac_dev *dev, int commit_flag)
 	int status = 0;
 	struct fib * fibptr;
 
-	if (!(fibptr = aac_fib_alloc(dev)))
+	if (!(fibptr = aac_fib_alloc(dev, DMA_FROM_DEVICE)))
 		return -ENOMEM;
 
 	aac_fib_init(fibptr);
@@ -456,7 +456,7 @@ int aac_get_containers(struct aac_dev *dev)
 	struct aac_get_container_count_resp *dresp;
 	int maximum_num_containers = MAXIMUM_NUM_CONTAINERS;
 
-	if (!(fibptr = aac_fib_alloc(dev)))
+	if (!(fibptr = aac_fib_alloc(dev, DMA_FROM_DEVICE)))
 		return -ENOMEM;
 
 	aac_fib_init(fibptr);
@@ -732,9 +732,11 @@ static void _aac_probe_container1(void * context, struct fib * fibptr)
 static int _aac_probe_container(struct scsi_cmnd * scsicmd, int (*callback)(struct scsi_cmnd *))
 {
 	struct fib * fibptr;
+	struct aac_dev * dev;
 	int status = -ENOMEM;
 
-	if ((fibptr = aac_fib_alloc((struct aac_dev *)scsicmd->device->host->hostdata))) {
+	dev = (struct aac_dev *)scsicmd->device->host->hostdata;
+	if ((fibptr = aac_fib_alloc(dev, DMA_FROM_DEVICE))) {
 		struct aac_query_mount *dinfo;
 
 		aac_fib_init(fibptr);
@@ -1673,7 +1675,7 @@ static int aac_send_safw_bmic_cmd(struct aac_dev *dev,
 		return 0;
 
 	/* allocate FIB */
-	fibptr = aac_fib_alloc(dev);
+	fibptr = aac_fib_alloc(dev, DMA_BIDIRECTIONAL);
 	if (!fibptr)
 		return -ENOMEM;
 
@@ -2040,7 +2042,7 @@ int aac_get_adapter_info(struct aac_dev* dev)
 	struct aac_bus_info *command;
 	struct aac_bus_info_response *bus_info;
 
-	if (!(fibptr = aac_fib_alloc(dev)))
+	if (!(fibptr = aac_fib_alloc(dev, DMA_FROM_DEVICE)))
 		return -ENOMEM;
 
 	aac_fib_init(fibptr);
@@ -2087,7 +2089,7 @@ int aac_get_adapter_info(struct aac_dev* dev)
 		if (rcode >= 0)
 			memcpy(&dev->supplement_adapter_info, sinfo, sizeof(*sinfo));
 		if (rcode == -ERESTARTSYS) {
-			fibptr = aac_fib_alloc(dev);
+			fibptr = aac_fib_alloc(dev, DMA_FROM_DEVICE);
 			if (!fibptr)
 				return -ENOMEM;
 		}
drivers/scsi/aacraid/aacraid.h +6 -3
index e3e4ecbea726..5b3fd34363c0 100644
@@ -1291,13 +1291,16 @@ struct fsa_dev_info {
 };
 
 struct fib {
-	void			*next;	/* this is used by the allocator */
 	s16			type;
 	s16			size;
 	/*
 	 *	The Adapter that this I/O is destined for.
 	 */
 	struct aac_dev		*dev;
+	/*
+	 * The associated scsi command
+	 */
+	struct scsi_cmnd 	*scmd;
 	/*
 	 *	This is the event the sendfib routine will wait on if the
 	 *	caller did not pass one and this is synch io.
@@ -1552,7 +1555,6 @@ struct aac_dev
 	 */
 	struct fib              *fibs;
 
-	struct fib		*free_fib;
 	spinlock_t		fib_lock;
 
 	struct mutex		ioctl_mutex;
@@ -1597,6 +1599,7 @@ struct aac_dev
 	size_t			comm_size;
 
 	struct Scsi_Host	*scsi_host_ptr;
+	struct scsi_device	*scsi_host_dev;
 	int			maximum_num_containers;
 	int			maximum_num_physicals;
 	int			maximum_num_channels;
@@ -2686,7 +2689,7 @@ void aac_free_irq(struct aac_dev *dev);
 int aac_setup_safw_adapter(struct aac_dev *dev);
 const char *aac_driverinfo(struct Scsi_Host *);
 void aac_fib_vector_assign(struct aac_dev *dev);
-struct fib *aac_fib_alloc(struct aac_dev *dev);
+struct fib *aac_fib_alloc(struct aac_dev *dev, int direction);
 struct fib *aac_fib_alloc_tag(struct aac_dev *dev, struct scsi_cmnd *scmd);
 int aac_fib_setup(struct aac_dev *dev);
 void aac_fib_map_free(struct aac_dev *dev);
drivers/scsi/aacraid/commctrl.c +30 -27

94 lines not on this page · raw message

drivers/scsi/aacraid/comminit.c +12 -22
index 355b16f0b145..baca5883775b 100644
@@ -67,14 +67,13 @@ static int aac_alloc_comm(struct aac_dev *dev, void **commaddr, unsigned long co
 		(dev->comm_interface == AAC_COMM_MESSAGE_TYPE2) ||
 		(dev->comm_interface == AAC_COMM_MESSAGE_TYPE3 &&
 		!dev->sa_firmware)) {
-		host_rrq_size =
-			(dev->scsi_host_ptr->can_queue + AAC_NUM_MGT_FIB)
+		host_rrq_size = dev->scsi_host_ptr->can_queue
 				* sizeof(u32);
 		aac_init_size = sizeof(union aac_init);
 	} else if (dev->comm_interface == AAC_COMM_MESSAGE_TYPE3 &&
 		dev->sa_firmware) {
-		host_rrq_size = (dev->scsi_host_ptr->can_queue
-			+ AAC_NUM_MGT_FIB) * sizeof(u32)  * AAC_MAX_MSIX;
+		host_rrq_size = dev->scsi_host_ptr->can_queue
+			* sizeof(u32)  * AAC_MAX_MSIX;
 		aac_init_size = sizeof(union aac_init) +
 			(AAC_MAX_HRRQ - 1) * sizeof(struct _rrq);
 	} else {
@@ -179,8 +178,7 @@ static int aac_alloc_comm(struct aac_dev *dev, void **commaddr, unsigned long co
 			cpu_to_le32(INITFLAGS_DRIVER_USES_UTC_TIME |
 			INITFLAGS_DRIVER_SUPPORTS_PM);
 		init->r7.max_io_commands =
-			cpu_to_le32(dev->scsi_host_ptr->can_queue +
-					AAC_NUM_MGT_FIB);
+			cpu_to_le32(dev->scsi_host_ptr->can_queue);
 		init->r7.max_io_size =
 			cpu_to_le32(dev->scsi_host_ptr->max_sectors << 9);
 		init->r7.max_fib_size = cpu_to_le32(dev->max_fib_size);
@@ -327,7 +325,7 @@ int aac_send_shutdown(struct aac_dev * dev)
 
 	aac_wait_for_io_completion(dev);
 
-	fibctx = aac_fib_alloc(dev);
+	fibctx = aac_fib_alloc(dev, DMA_NONE);
 	if (!fibctx)
 		return -ENOMEM;
 	aac_fib_init(fibctx);
@@ -462,9 +460,7 @@ void aac_define_int_mode(struct aac_dev *dev)
 	    dev->pdev->device == PMC_DEVICE_S6 ||
 	    dev->sync_mode) {
 		dev->max_msix = 1;
-		dev->vector_cap =
-			dev->scsi_host_ptr->can_queue +
-			AAC_NUM_MGT_FIB;
+		dev->vector_cap = dev->scsi_host_ptr->can_queue;
 		return;
 	}
 
@@ -500,11 +496,9 @@ void aac_define_int_mode(struct aac_dev *dev)
 			dev->max_msix = msi_count;
 	}
 	if (dev->comm_interface == AAC_COMM_MESSAGE_TYPE3 && dev->sa_firmware)
-		dev->vector_cap = dev->scsi_host_ptr->can_queue +
-				AAC_NUM_MGT_FIB;
+		dev->vector_cap = dev->scsi_host_ptr->can_queue;
 	else
-		dev->vector_cap = (dev->scsi_host_ptr->can_queue +
-				AAC_NUM_MGT_FIB) / msi_count;
+		dev->vector_cap = dev->scsi_host_ptr->can_queue / msi_count;
 
 }
 struct aac_dev *aac_init_adapter(struct aac_dev *dev)
@@ -610,14 +604,10 @@ struct aac_dev *aac_init_adapter(struct aac_dev *dev)
 		host->sg_tablesize = status[2] >> 16;
 		dev->sg_tablesize = status[2] & 0xFFFF;
 		if (aac_is_src(dev)) {
-			if (host->can_queue > (status[3] >> 16) -
-					AAC_NUM_MGT_FIB)
-				host->can_queue = (status[3] >> 16) -
-					AAC_NUM_MGT_FIB;
-		} else if (host->can_queue > (status[3] & 0xFFFF) -
-				AAC_NUM_MGT_FIB)
-			host->can_queue = (status[3] & 0xFFFF) -
-				AAC_NUM_MGT_FIB;
+			if (host->can_queue > (status[3] >> 16))
+				host->can_queue = (status[3] >> 16);
+		} else if (host->can_queue > (status[3] & 0xFFFF))
+			host->can_queue = (status[3] & 0xFFFF);
 
 		dev->max_num_aif = status[4] & 0xFFFF;
 	}
drivers/scsi/aacraid/commsup.c +31 -45

178 lines not on this page · raw message

drivers/scsi/aacraid/dpcsup.c +1 -1
index a557aa629827..ae88e1d20c7b 100644
@@ -315,7 +315,7 @@ unsigned int aac_intr_normal(struct aac_dev *dev, u32 index, int isAif,
 		struct fib *fibctx;
 		struct aac_aifcmd *cmd;
 
-		fibctx = aac_fib_alloc(dev);
+		fibctx = aac_fib_alloc(dev, DMA_FROM_DEVICE);
 		if (!fibctx)
 			return 1;
 		aac_fib_init(fibctx);
drivers/scsi/aacraid/linit.c +28 -26

149 lines not on this page · raw message

Signature · 1 line
2.16.4
#54 lore raw

RFC PATCH v3 16/41 aacraid: use scsi_host_busy_iter() to traverse commands

In reply to Hannes Reinecke

Now that the block layer manages all commands we can use scsi_host_busy_iter() to traverse outstanding commands.

---
 drivers/scsi/aacraid/linit.c | 137 ++++++++++++++++++++-----------------------
 1 file changed, 62 insertions(+), 75 deletions(-)
drivers/scsi/aacraid/linit.c +62 -75
index 91a4780d9815..e3f8eac6cebe 100644
@@ -680,7 +680,7 @@ static int aac_eh_abort(struct scsi_cmnd* cmd)
 	struct scsi_device * dev = cmd->device;
 	struct Scsi_Host * host = dev->host;
 	struct aac_dev * aac = (struct aac_dev *)host->hostdata;
-	int count, found;
+	int count;
 	u32 bus, cid;
 	int ret = FAILED;
 
@@ -699,17 +699,13 @@ static int aac_eh_abort(struct scsi_cmnd* cmd)
 		 AAC_DRIVERNAME,
 		 host->host_no, sdev_channel(dev), sdev_id(dev), (int)dev->lun);
 
-		found = 0;
-		for (count = 0; count < host->can_queue; ++count) {
-			fib = &aac->fibs[count];
-			if (*(u8 *)fib->hw_fib_va != 0 &&
-				(fib->flags & FIB_CONTEXT_FLAG_NATIVE_HBA) &&
-				(fib->callback_data == cmd)) {
-				found = 1;
-				break;
-			}
-		}
-		if (!found)
+		if (cmd->request->tag >= host->can_queue)
+			return ret;
+
+		fib = &aac->fibs[cmd->request->tag];
+		if (*(u8 *)fib->hw_fib_va == 0 ||
+		    !(fib->flags & FIB_CONTEXT_FLAG_NATIVE_HBA) ||
+		    (fib->callback_data != cmd))
 			return ret;
 
 		/* start a HBA_TMF_ABORT_TASK TMF request */
@@ -767,12 +763,13 @@ static int aac_eh_abort(struct scsi_cmnd* cmd)
 			/* fall through */
 		case INQUIRY:
 		case READ_CAPACITY:
+		case TEST_UNIT_READY:
 			/*
 			 * Mark associated FIB to not complete,
 			 * eh handler does this
 			 */
-			for (count = 0; count < host->can_queue; ++count) {
-				struct fib *fib = &aac->fibs[count];
+			if (cmd->request->tag < host->can_queue) {
+				struct fib *fib = &aac->fibs[cmd->request->tag];
 
 				if (fib->hw_fib_va->header.XferState &&
 				(fib->flags & FIB_CONTEXT_FLAG) &&
@@ -785,32 +782,6 @@ static int aac_eh_abort(struct scsi_cmnd* cmd)
 				}
 			}
 			break;
-		case TEST_UNIT_READY:
-			/*
-			 * Mark associated FIB to not complete,
-			 * eh handler does this
-			 */
-			for (count = 0; count < host->can_queue; ++count) {
-				struct scsi_cmnd *command;
-				struct fib *fib = &aac->fibs[count];
-
-				command = fib->callback_data;
-
-				if ((fib->hw_fib_va->header.XferState &
-					cpu_to_le32
-					(Async | NoResponseExpected)) &&
-					(fib->flags & FIB_CONTEXT_FLAG) &&
-					((command)) &&
-					(command->device == cmd->device)) {
-					fib->flags |=
-						FIB_CONTEXT_FLAG_TIMED_OUT;
-					command->SCp.phase =
-						AAC_OWNER_ERROR_HANDLER;
-					if (command == cmd)
-						ret = SUCCESS;
-				}
-			}
-			break;
 		}
 	}
 	return ret;
@@ -1016,6 +987,36 @@ static int aac_eh_target_reset(struct scsi_cmnd *cmd)
 	return ret;
 }
 
+struct aac_abort_fibs_iter_data {
+	struct aac_dev *aac;
+	u32 bus;
+};
+
+static bool aac_abort_fibs_iter(struct scsi_cmnd *scmd, void *data, bool rsvd)
+{
+	struct aac_abort_fibs_iter_data *iter_data = data;
+	struct aac_dev *aac = iter_data->aac;
+	struct fib *fib = &aac->fibs[scmd->request->tag];
+
+	if (fib->hw_fib_va->header.XferState &&
+	    (fib->flags & FIB_CONTEXT_FLAG) &&
+	    (fib->flags & FIB_CONTEXT_FLAG_SCSI_CMD)) {
+		struct aac_hba_map_info *info;
+		u32 bus, cid;
+
+		bus = aac_logical_to_phys(scmd_channel(scmd));
+		if (bus != iter_data->bus)
+			return true;
+		cid = scmd_id(scmd);
+		info = &aac->hba_map[bus][cid];
+		if (bus >= AAC_MAX_BUSES || cid >= AAC_MAX_TARGETS ||
+		    info->devtype != AAC_DEVTYPE_NATIVE_RAW) {
+			fib->flags |= FIB_CONTEXT_FLAG_EH_RESET;
+			scmd->SCp.phase = AAC_OWNER_ERROR_HANDLER;
+		}
+	}
+	return true;
+}
 /*
  *	aac_eh_bus_reset	- Bus reset command handling
  *	@scsi_cmd:	SCSI command block causing the reset
@@ -1027,34 +1028,13 @@ static int aac_eh_bus_reset(struct scsi_cmnd* cmd)
 	struct Scsi_Host * host = dev->host;
 	struct aac_dev * aac = (struct aac_dev *)host->hostdata;
 	int count;
-	u32 cmd_bus;
 	int status = 0;
+	struct aac_abort_fibs_iter_data data = {
+		.aac = aac,
+		.bus = aac_logical_to_phys(scmd_channel(cmd)),
+	};
 
-
-	cmd_bus = aac_logical_to_phys(scmd_channel(cmd));
-	/* Mark the assoc. FIB to not complete, eh handler does this */
-	for (count = 0; count < host->can_queue; ++count) {
-		struct fib *fib = &aac->fibs[count];
-
-		if (fib->hw_fib_va->header.XferState &&
-		    (fib->flags & FIB_CONTEXT_FLAG) &&
-		    (fib->flags & FIB_CONTEXT_FLAG_SCSI_CMD)) {
-			struct aac_hba_map_info *info;
-			u32 bus, cid;
-
-			cmd = (struct scsi_cmnd *)fib->callback_data;
-			bus = aac_logical_to_phys(scmd_channel(cmd));
-			if (bus != cmd_bus)
-				continue;
-			cid = scmd_id(cmd);
-			info = &aac->hba_map[bus][cid];
-			if (bus >= AAC_MAX_BUSES || cid >= AAC_MAX_TARGETS ||
-			    info->devtype != AAC_DEVTYPE_NATIVE_RAW) {
-				fib->flags |= FIB_CONTEXT_FLAG_EH_RESET;
-				cmd->SCp.phase = AAC_OWNER_ERROR_HANDLER;
-			}
-		}
-	}
+	scsi_host_busy_iter(host, aac_abort_fibs_iter, &data);
 
 	pr_err("%s: Host bus reset request. SCSI hang ?\n", AAC_DRIVERNAME);
 
@@ -1558,23 +1538,28 @@ static struct scsi_host_template aac_driver_template = {
 	.no_write_same			= 1,
 };
 
-static void __aac_shutdown(struct aac_dev * aac)
+static bool __aac_shutdown_iter(struct scsi_cmnd *scmd, void *data, bool rsvd)
 {
-	int i;
+	struct aac_dev *aac = data;
+	struct fib *fib = &aac->fibs[scmd->request->tag];
+
+	if (!(fib->hw_fib_va->header.XferState &
+	      cpu_to_le32(NoResponseExpected | Async)) &&
+	    (fib->hw_fib_va->header.XferState & cpu_to_le32(ResponseExpected)))
+		complete(&fib->event_wait);
+	return true;
+}
 
+static void __aac_shutdown(struct aac_dev * aac)
+{
 	mutex_lock(&aac->ioctl_mutex);
 	aac->adapter_shutdown = 1;
 	mutex_unlock(&aac->ioctl_mutex);
 
 	if (aac->aif_thread) {
-		int i;
 		/* Clear out events first */
-		for (i = 0; i < aac->scsi_host_ptr->can_queue; i++) {
-			struct fib *fib = &aac->fibs[i];
-			if (!(fib->hw_fib_va->header.XferState & cpu_to_le32(NoResponseExpected | Async)) &&
-			    (fib->hw_fib_va->header.XferState & cpu_to_le32(ResponseExpected)))
-				complete(&fib->event_wait);
-		}
+		scsi_host_busy_iter(aac->scsi_host_ptr,
+				    __aac_shutdown_iter, aac);
 		kthread_stop(aac->thread);
 		aac->thread = NULL;
 	}
@@ -1585,6 +1570,8 @@ static void __aac_shutdown(struct aac_dev * aac)
 
 	if (aac_is_src(aac)) {
 		if (aac->max_msix > 1) {
+			int i;
+
 			for (i = 0; i < aac->max_msix; i++) {
 				free_irq(pci_irq_vector(aac->pdev, i),
 					 &(aac->aac_msix[i]));
Signature · 1 line
2.16.4
#55 lore raw

RFC PATCH v3 17/41 megaraid_sas: kill this_id and init_id

In reply to Hannes Reinecke

Unused.

---
 drivers/scsi/megaraid/megaraid_sas.h      |  2 --
 drivers/scsi/megaraid/megaraid_sas_base.c | 10 +++-------
 2 files changed, 3 insertions(+), 9 deletions(-)
drivers/scsi/megaraid/megaraid_sas.h +0 -2
index 83d8c4cb1ad5..d295a2036588 100644
@@ -2323,7 +2323,6 @@ struct megasas_instance {
 	struct megasas_pd_list          pd_list[MEGASAS_MAX_PD];
 	struct megasas_pd_list          local_pd_list[MEGASAS_MAX_PD];
 	u8 ld_ids[MEGASAS_MAX_LD_IDS];
-	s8 init_id;
 
 	u16 max_num_sge;
 	u16 max_fw_cmds;
@@ -2359,7 +2358,6 @@ struct megasas_instance {
 	wait_queue_head_t abort_cmd_wait_q;
 
 	struct pci_dev *pdev;
-	u32 unique_id;
 	u32 fw_support_ieee;
 	u32 threshold_reply_count;
 
drivers/scsi/megaraid/megaraid_sas_base.c +3 -7
index fb9c3ceed508..949ae49a6967 100644
@@ -3425,6 +3425,7 @@ static struct scsi_host_template megasas_template = {
 	.bios_param = megasas_bios_param,
 	.change_queue_depth = scsi_change_queue_depth,
 	.max_segment_size = 0xffffffff,
+	.this_id = -1,
 };
 
 /**
@@ -5494,9 +5495,7 @@ megasas_init_adapter_mfi(struct megasas_instance *instance)
 		goto fail_fw_init;
 
 	if (megasas_get_ctrl_info(instance)) {
-		dev_err(&instance->pdev->dev, "(%d): Could get controller info "
-			"Fail from %s %d\n", instance->unique_id,
-			__func__, __LINE__);
+		dev_err(&instance->pdev->dev, "Couldn't get controller info\n");
 		goto fail_fw_init;
 	}
 
@@ -6758,9 +6757,7 @@ static int megasas_io_attach(struct megasas_instance *instance)
 	/*
 	 * Export parameters required by SCSI mid-layer
 	 */
-	host->unique_id = instance->unique_id;
 	host->can_queue = instance->max_scsi_cmds;
-	host->this_id = instance->init_id;
 	host->sg_tablesize = instance->max_num_sge;
 
 	if (instance->fw_support_ieee)
@@ -7346,8 +7343,7 @@ static int megasas_probe_one(struct pci_dev *pdev,
 	 */
 	instance->pdev = pdev;
 	instance->host = host;
-	instance->unique_id = pdev->bus->number << 8 | pdev->devfn;
-	instance->init_id = MEGASAS_DEFAULT_INIT_ID;
+	host->unique_id = pdev->bus->number << 8 | pdev->devfn;
 
 	megasas_set_adapter_type(instance);
 
Signature · 1 line
2.16.4
#56 lore raw

RFC PATCH v3 18/41 megaraid_sas: use shost_priv()

In reply to Hannes Reinecke

Cleanup code by using shost_priv() instead of directly referencing the ->hostdata pointer.

---
 drivers/scsi/megaraid/megaraid_sas_base.c | 63 +++++++++++--------------------
 1 file changed, 22 insertions(+), 41 deletions(-)
drivers/scsi/megaraid/megaraid_sas_base.c +22 -41
index 949ae49a6967..b241a0ae9955 100644
@@ -1755,12 +1755,9 @@ megasas_build_and_issue_cmd(struct megasas_instance *instance,
 static int
 megasas_queue_command(struct Scsi_Host *shost, struct scsi_cmnd *scmd)
 {
-	struct megasas_instance *instance;
+	struct megasas_instance *instance = shost_priv(shost);
 	struct MR_PRIV_DEVICE *mr_device_priv_data;
 
-	instance = (struct megasas_instance *)
-	    scmd->device->host->hostdata;
-
 	if (instance->unload == 1) {
 		scmd->result = DID_NO_CONNECT << 16;
 		scmd->scsi_done(scmd);
@@ -1857,14 +1854,13 @@ void megasas_set_dynamic_target_properties(struct scsi_device *sdev,
 {
 	u16 pd_index = 0, ld;
 	u32 device_id;
-	struct megasas_instance *instance;
+	struct megasas_instance *instance = shost_priv(sdev->host);
 	struct fusion_context *fusion;
 	struct MR_PRIV_DEVICE *mr_device_priv_data;
 	struct MR_PD_CFG_SEQ_NUM_SYNC *pd_sync;
 	struct MR_LD_RAID *raid;
 	struct MR_DRV_RAID_MAP_ALL *local_map_ptr;
 
-	instance = megasas_lookup_instance(sdev->host->host_no);
 	fusion = instance->ctrl_context;
 	mr_device_priv_data = sdev->hostdata;
 
@@ -1937,7 +1933,7 @@ megasas_set_nvme_device_properties(struct scsi_device *sdev, u32 max_io_size)
 	struct megasas_instance *instance;
 	u32 mr_nvme_pg_size;
 
-	instance = (struct megasas_instance *)sdev->host->hostdata;
+	instance = shost_priv(sdev->host);
 	mr_nvme_pg_size = max_t(u32, instance->nvme_page_size,
 				MR_DEFAULT_NVME_PAGE_SIZE);
 
@@ -1961,10 +1957,9 @@ static void megasas_set_fw_assisted_qd(struct scsi_device *sdev,
 	u8 interface_type;
 	u32 device_qd = MEGASAS_DEFAULT_CMD_PER_LUN;
 	u32 tgt_device_qd;
-	struct megasas_instance *instance;
+	struct megasas_instance *instance = shost_priv(sdev->host);
 	struct MR_PRIV_DEVICE *mr_device_priv_data;
 
-	instance = megasas_lookup_instance(sdev->host->host_no);
 	mr_device_priv_data = sdev->hostdata;
 	interface_type  = mr_device_priv_data->interface_type;
 
@@ -2838,7 +2833,7 @@ static int megasas_generic_reset(struct scsi_cmnd *scmd)
 	int ret_val;
 	struct megasas_instance *instance;
 
-	instance = (struct megasas_instance *)scmd->device->host->hostdata;
+	instance = shost_priv(scmd->device->host);
 
 	scmd_printk(KERN_NOTICE, scmd, "megasas: RESET cmd=%x retries=%x\n",
 		 scmd->cmnd[0], scmd->retries);
@@ -2875,7 +2870,7 @@ blk_eh_timer_return megasas_reset_timer(struct scsi_cmnd *scmd)
 		return BLK_EH_DONE;
 	}
 
-	instance = (struct megasas_instance *)scmd->device->host->hostdata;
+	instance = shost_priv(scmd->device->host);
 	if (!(instance->flag & MEGASAS_FW_BUSY)) {
 		/* FW is busy, throttle IO */
 		spin_lock_irqsave(instance->host->host_lock, flags);
@@ -2942,10 +2937,9 @@ megasas_dump_fusion_io(struct scsi_cmnd *scmd)
 {
 	struct megasas_cmd_fusion *cmd;
 	union MEGASAS_REQUEST_DESCRIPTOR_UNION *req_desc;
-	struct megasas_instance *instance;
+	struct megasas_instance *instance = shost_priv(scmd->device->host);
 
 	cmd = (struct megasas_cmd_fusion *)scmd->SCp.ptr;
-	instance = (struct megasas_instance *)scmd->device->host->hostdata;
 
 	scmd_printk(KERN_INFO, scmd,
 		    "scmd: (0x%p)  retries: 0x%x  allowed: 0x%x\n",
@@ -3001,9 +2995,7 @@ megasas_dump_sys_regs(void __iomem *reg_set, char *buf)
 static int megasas_reset_bus_host(struct scsi_cmnd *scmd)
 {
 	int ret;
-	struct megasas_instance *instance;
-
-	instance = (struct megasas_instance *)scmd->device->host->hostdata;
+	struct megasas_instance *instance = shost_priv(scmd->device->host);
 
 	scmd_printk(KERN_INFO, scmd,
 		"OCR is requested due to IO timeout!!\n");
@@ -3035,9 +3027,7 @@ static int megasas_reset_bus_host(struct scsi_cmnd *scmd)
 static int megasas_task_abort(struct scsi_cmnd *scmd)
 {
 	int ret;
-	struct megasas_instance *instance;
-
-	instance = (struct megasas_instance *)scmd->device->host->hostdata;
+	struct megasas_instance *instance = shost_priv(scmd->device->host);
 
 	if (instance->adapter_type != MFI_SERIES)
 		ret = megasas_task_abort_fusion(scmd);
@@ -3057,9 +3047,7 @@ static int megasas_task_abort(struct scsi_cmnd *scmd)
 static int megasas_reset_target(struct scsi_cmnd *scmd)
 {
 	int ret;
-	struct megasas_instance *instance;
-
-	instance = (struct megasas_instance *)scmd->device->host->hostdata;
+	struct megasas_instance *instance = shost_priv(scmd->device->host);
 
 	if (instance->adapter_type != MFI_SERIES)
 		ret = megasas_reset_target_fusion(scmd);
@@ -3173,8 +3161,7 @@ fw_crash_buffer_store(struct device *cdev,
 	struct device_attribute *attr, const char *buf, size_t count)
 {
 	struct Scsi_Host *shost = class_to_shost(cdev);
-	struct megasas_instance *instance =
-		(struct megasas_instance *) shost->hostdata;
+	struct megasas_instance *instance = shost_priv(shost);
 	int val = 0;
 	unsigned long flags;
 
@@ -3192,8 +3179,7 @@ fw_crash_buffer_show(struct device *cdev,
 	struct device_attribute *attr, char *buf)
 {
 	struct Scsi_Host *shost = class_to_shost(cdev);
-	struct megasas_instance *instance =
-		(struct megasas_instance *) shost->hostdata;
+	struct megasas_instance *instance = shost_priv(shost);
 	u32 size;
 	unsigned long dmachunk = CRASH_DMA_BUF_SIZE;
 	unsigned long chunk_left_bytes;
@@ -3237,8 +3223,7 @@ fw_crash_buffer_size_show(struct device *cdev,
 	struct device_attribute *attr, char *buf)
 {
 	struct Scsi_Host *shost = class_to_shost(cdev);
-	struct megasas_instance *instance =
-		(struct megasas_instance *) shost->hostdata;
+	struct megasas_instance *instance = shost_priv(shost);
 
 	return snprintf(buf, PAGE_SIZE, "%ld\n", (unsigned long)
 		((instance->fw_crash_buffer_size) * 1024 * 1024)/PAGE_SIZE);
@@ -3249,8 +3234,7 @@ fw_crash_state_store(struct device *cdev,
 	struct device_attribute *attr, const char *buf, size_t count)
 {
 	struct Scsi_Host *shost = class_to_shost(cdev);
-	struct megasas_instance *instance =
-		(struct megasas_instance *) shost->hostdata;
+	struct megasas_instance *instance = shost_priv(shost);
 	int val = 0;
 	unsigned long flags;
 
@@ -3284,8 +3268,7 @@ fw_crash_state_show(struct device *cdev,
 	struct device_attribute *attr, char *buf)
 {
 	struct Scsi_Host *shost = class_to_shost(cdev);
-	struct megasas_instance *instance =
-		(struct megasas_instance *) shost->hostdata;
+	struct megasas_instance *instance = shost_priv(shost);
 
 	return snprintf(buf, PAGE_SIZE, "%d\n", instance->fw_crash_state);
 }
@@ -3302,7 +3285,7 @@ ldio_outstanding_show(struct device *cdev, struct device_attribute *attr,
 	char *buf)
 {
 	struct Scsi_Host *shost = class_to_shost(cdev);
-	struct megasas_instance *instance = (struct megasas_instance *)shost->hostdata;
+	struct megasas_instance *instance = shost_priv(shost);
 
 	return snprintf(buf, PAGE_SIZE, "%d\n", atomic_read(&instance->ldio_outstanding));
 }
@@ -3312,7 +3295,7 @@ fw_cmds_outstanding_show(struct device *cdev,
 				 struct device_attribute *attr, char *buf)
 {
 	struct Scsi_Host *shost = class_to_shost(cdev);
-	struct megasas_instance *instance = (struct megasas_instance *)shost->hostdata;
+	struct megasas_instance *instance = shost_priv(shost);
 
 	return snprintf(buf, PAGE_SIZE, "%d\n", atomic_read(&instance->fw_outstanding));
 }
@@ -3322,7 +3305,7 @@ enable_sdev_max_qd_show(struct device *cdev,
 	struct device_attribute *attr, char *buf)
 {
 	struct Scsi_Host *shost = class_to_shost(cdev);
-	struct megasas_instance *instance = (struct megasas_instance *)shost->hostdata;
+	struct megasas_instance *instance = shost_priv(shost);
 
 	return snprintf(buf, PAGE_SIZE, "%d\n", instance->enable_sdev_max_qd);
 }
@@ -3332,7 +3315,7 @@ enable_sdev_max_qd_store(struct device *cdev,
 	struct device_attribute *attr, const char *buf, size_t count)
 {
 	struct Scsi_Host *shost = class_to_shost(cdev);
-	struct megasas_instance *instance = (struct megasas_instance *)shost->hostdata;
+	struct megasas_instance *instance = shost_priv(shost);
 	u32 val = 0;
 	bool is_target_prop;
 	int ret_target_prop = DCMD_FAILED;
@@ -3364,8 +3347,7 @@ dump_system_regs_show(struct device *cdev,
 			       struct device_attribute *attr, char *buf)
 {
 	struct Scsi_Host *shost = class_to_shost(cdev);
-	struct megasas_instance *instance =
-			(struct megasas_instance *)shost->hostdata;
+	struct megasas_instance *instance = shost_priv(shost);
 
 	return megasas_dump_sys_regs(instance->reg_set, buf);
 }
@@ -3375,8 +3357,7 @@ raid_map_id_show(struct device *cdev, struct device_attribute *attr,
 			  char *buf)
 {
 	struct Scsi_Host *shost = class_to_shost(cdev);
-	struct megasas_instance *instance =
-			(struct megasas_instance *)shost->hostdata;
+	struct megasas_instance *instance = shost_priv(shost);
 
 	return snprintf(buf, PAGE_SIZE, "%ld\n",
 			(unsigned long)instance->map_id);
@@ -7334,7 +7315,7 @@ static int megasas_probe_one(struct pci_dev *pdev,
 		goto fail_alloc_instance;
 	}
 
-	instance = (struct megasas_instance *)host->hostdata;
+	instance = shost_priv(host);
 	memset(instance, 0, sizeof(*instance));
 	atomic_set(&instance->fw_reset_no_pci_access, 0);
 
Signature · 1 line
2.16.4
#57 lore raw

RFC PATCH v3 19/41 megaraid_sas: avoid using megaraid_lookup_instance()

In reply to Hannes Reinecke

When a scsi device is given we can dereference the hostdata pointer directly from the scsi host, no need to painstakingly looking it up.

---
 drivers/scsi/megaraid/megaraid_sas_base.c | 10 +++-------
 1 file changed, 3 insertions(+), 7 deletions(-)
drivers/scsi/megaraid/megaraid_sas_base.c +3 -7
index b241a0ae9955..43a179fc91f2 100644
@@ -2004,9 +2004,7 @@ static void megasas_set_static_target_properties(struct scsi_device *sdev,
 						 bool is_target_prop)
 {
 	u32 max_io_size_kb = MR_DEFAULT_NVME_MDTS_KB;
-	struct megasas_instance *instance;
-
-	instance = megasas_lookup_instance(sdev->host->host_no);
+	struct megasas_instance *instance = shost_priv(sdev->host);
 
 	/*
 	 * The RAID firmware may require extended timeouts.
@@ -2029,11 +2027,10 @@ static void megasas_set_static_target_properties(struct scsi_device *sdev,
 static int megasas_slave_configure(struct scsi_device *sdev)
 {
 	u16 pd_index = 0;
-	struct megasas_instance *instance;
+	struct megasas_instance *instance = shost_priv(sdev->host);
 	int ret_target_prop = DCMD_FAILED;
 	bool is_target_prop = false;
 
-	instance = megasas_lookup_instance(sdev->host->host_no);
 	if (instance->pd_list_not_supported) {
 		if (!MEGASAS_IS_LOGICAL(sdev) && sdev->type == TYPE_DISK) {
 			pd_index = (sdev->channel * MEGASAS_MAX_DEV_PER_CHANNEL) +
@@ -2069,10 +2066,9 @@ static int megasas_slave_configure(struct scsi_device *sdev)
 static int megasas_slave_alloc(struct scsi_device *sdev)
 {
 	u16 pd_index = 0;
-	struct megasas_instance *instance ;
+	struct megasas_instance *instance = shost_priv(sdev->host);
 	struct MR_PRIV_DEVICE *mr_device_priv_data;
 
-	instance = megasas_lookup_instance(sdev->host->host_no);
 	if (!MEGASAS_IS_LOGICAL(sdev)) {
 		/*
 		 * Open the OS scan to the SYSTEM PD
Signature · 1 line
2.16.4
#58 lore raw

RFC PATCH v3 20/41 megaraid_sas: separate out megasas_set_max_sectors()

In reply to Hannes Reinecke

Separate out the calculations for setting max_sectors into a separate function.

---
 drivers/scsi/megaraid/megaraid_sas_base.c | 55 +++++++++++++++++--------------
 1 file changed, 31 insertions(+), 24 deletions(-)
drivers/scsi/megaraid/megaraid_sas_base.c +31 -24
index 43a179fc91f2..00a1d5caf7d5 100644
@@ -5829,6 +5829,36 @@ megasas_alloc_irq_vectors(struct megasas_instance *instance)
 		megasas_set_high_iops_queue_affinity_hint(instance);
 }
 
+static void megasas_set_max_sectors(struct megasas_instance *instance)
+{
+	if (instance->fw_support_ieee)
+		instance->max_sectors_per_req = MEGASAS_MAX_SECTORS_IEEE;
+
+	/*
+	 * Check if the module parameter value for max_sectors can be used
+	 */
+	if (max_sectors && max_sectors < instance->max_sectors_per_req)
+		instance->max_sectors_per_req = max_sectors;
+	else {
+		if (max_sectors) {
+			if (((instance->pdev->device ==
+				PCI_DEVICE_ID_LSI_SAS1078GEN2) ||
+				(instance->pdev->device ==
+				PCI_DEVICE_ID_LSI_SAS0079GEN2)) &&
+				(max_sectors <= MEGASAS_MAX_SECTORS)) {
+				instance->max_sectors_per_req = max_sectors;
+			} else {
+				dev_info(&instance->pdev->dev,
+					 "max_sectors should be > 0 and <= %d"
+					 "(or < 1MB for GEN2 controller)\n",
+					 instance->max_sectors_per_req);
+			}
+		}
+	}
+
+	instance->host->max_sectors = instance->max_sectors_per_req;
+}
+
 /**
  * megasas_init_fw -	Initializes the FW
  * @instance:		Adapter soft state
@@ -6737,31 +6767,8 @@ static int megasas_io_attach(struct megasas_instance *instance)
 	host->can_queue = instance->max_scsi_cmds;
 	host->sg_tablesize = instance->max_num_sge;
 
-	if (instance->fw_support_ieee)
-		instance->max_sectors_per_req = MEGASAS_MAX_SECTORS_IEEE;
-
-	/*
-	 * Check if the module parameter value for max_sectors can be used
-	 */
-	if (max_sectors && max_sectors < instance->max_sectors_per_req)
-		instance->max_sectors_per_req = max_sectors;
-	else {
-		if (max_sectors) {
-			if (((instance->pdev->device ==
-				PCI_DEVICE_ID_LSI_SAS1078GEN2) ||
-				(instance->pdev->device ==
-				PCI_DEVICE_ID_LSI_SAS0079GEN2)) &&
-				(max_sectors <= MEGASAS_MAX_SECTORS)) {
-				instance->max_sectors_per_req = max_sectors;
-			} else {
-			dev_info(&instance->pdev->dev, "max_sectors should be > 0"
-				"and <= %d (or < 1MB for GEN2 controller)\n",
-				instance->max_sectors_per_req);
-			}
-		}
-	}
+	megasas_set_max_sectors(instance);
 
-	host->max_sectors = instance->max_sectors_per_req;
 	host->cmd_per_lun = MEGASAS_DEFAULT_CMD_PER_LUN;
 	host->max_channel = MEGASAS_MAX_CHANNELS - 1;
 	host->max_id = MEGASAS_MAX_DEV_PER_CHANNEL;
Signature · 1 line
2.16.4
#59 lore raw

RFC PATCH v3 21/41 megaraid_sas: megaraid_sas: reshuffle SCSI host allocation

In reply to Hannes Reinecke

Reshuffle SCSI host allocation such that the scsi host device can be allocated prior to the first command being sent.

---
 drivers/scsi/megaraid/megaraid_sas.h        |  3 +++
 drivers/scsi/megaraid/megaraid_sas_base.c   | 36 ++++++++++++++++++++---------
 drivers/scsi/megaraid/megaraid_sas_fusion.c |  5 ++++
 3 files changed, 33 insertions(+), 11 deletions(-)
drivers/scsi/megaraid/megaraid_sas.h +3 -0
index d295a2036588..b47306a66650 100644
@@ -2677,6 +2677,9 @@ int megasas_set_crash_dump_params(struct megasas_instance *instance,
 	u8 crash_buf_state);
 void megasas_free_host_crash_buffer(struct megasas_instance *instance);
 
+int megasas_io_attach(struct megasas_instance *instance);
+void megasas_io_detach(struct megasas_instance *instance);
+
 void megasas_return_cmd_fusion(struct megasas_instance *instance,
 	struct megasas_cmd_fusion *cmd);
 int megasas_issue_blocked_cmd(struct megasas_instance *instance,
drivers/scsi/megaraid/megaraid_sas_base.c +25 -11
index 00a1d5caf7d5..ddb8df03481f 100644
@@ -5468,6 +5468,13 @@ megasas_init_adapter_mfi(struct megasas_instance *instance)
 		goto fail_reply_queue;
 	}
 
+	/*
+	 * Notify the mid-layer about the new controller; do this
+	 * early so that we can allocate internal commands
+	 */
+	if (megasas_io_attach(instance))
+		goto fail_io_attach;
+
 	if (megasas_issue_init_mfi(instance))
 		goto fail_fw_init;
 
@@ -5490,7 +5497,8 @@ megasas_init_adapter_mfi(struct megasas_instance *instance)
 	return 0;
 
 fail_fw_init:
-
+	megasas_io_detach(instance);
+fail_io_attach:
 	dma_free_coherent(&instance->pdev->dev, reply_q_sz,
 			    instance->reply_queue, instance->reply_queue_h);
 fail_reply_queue:
@@ -6340,6 +6348,8 @@ static int megasas_init_fw(struct megasas_instance *instance)
 	if (tmp_sectors && (instance->max_sectors_per_req > tmp_sectors))
 		instance->max_sectors_per_req = tmp_sectors;
 
+	megasas_set_max_sectors(instance);
+
 	/* Check for valid throttlequeuedepth module parameter */
 	if (throttlequeuedepth &&
 			throttlequeuedepth <= instance->max_scsi_cmds)
@@ -6757,7 +6767,7 @@ static int megasas_start_aen(struct megasas_instance *instance)
  * megasas_io_attach -	Attaches this driver to SCSI mid-layer
  * @instance:		Adapter soft state
  */
-static int megasas_io_attach(struct megasas_instance *instance)
+int megasas_io_attach(struct megasas_instance *instance)
 {
 	struct Scsi_Host *host = instance->host;
 
@@ -6767,7 +6777,8 @@ static int megasas_io_attach(struct megasas_instance *instance)
 	host->can_queue = instance->max_scsi_cmds;
 	host->sg_tablesize = instance->max_num_sge;
 
-	megasas_set_max_sectors(instance);
+	/* Will be adjusted later */
+	host->max_sectors = MEGASAS_MAX_SECTORS_IEEE;
 
 	host->cmd_per_lun = MEGASAS_DEFAULT_CMD_PER_LUN;
 	host->max_channel = MEGASAS_MAX_CHANNELS - 1;
@@ -6788,6 +6799,15 @@ static int megasas_io_attach(struct megasas_instance *instance)
 	return 0;
 }
 
+/**
+ * megasas_io_detach -	Detaches this driver from the SCSI mid-layer
+ * @instance:		Adapter soft state
+ */
+void megasas_io_detach(struct megasas_instance *instance)
+{
+	scsi_remove_host(instance->host);
+}
+
 /**
  * megasas_set_dma_mask -	Set DMA mask for supported controllers
  *
@@ -7373,12 +7393,6 @@ static int megasas_probe_one(struct pci_dev *pdev,
 	megasas_mgmt_info.instance[megasas_mgmt_info.max_index] = instance;
 	megasas_mgmt_info.max_index++;
 
-	/*
-	 * Register with SCSI mid-layer
-	 */
-	if (megasas_io_attach(instance))
-		goto fail_io_attach;
-
 	instance->unload = 0;
 	/*
 	 * Trigger SCSI to scan our drives
@@ -7404,7 +7418,6 @@ static int megasas_probe_one(struct pci_dev *pdev,
 	return 0;
 
 fail_start_aen:
-fail_io_attach:
 	megasas_mgmt_info.count--;
 	megasas_mgmt_info.max_index--;
 	megasas_mgmt_info.instance[megasas_mgmt_info.max_index] = NULL;
@@ -7802,7 +7815,6 @@ static void megasas_detach_one(struct pci_dev *pdev)
 
 	if (instance->fw_crash_state != UNAVAILABLE)
 		megasas_free_host_crash_buffer(instance);
-	scsi_remove_host(instance->host);
 	instance->unload = 1;
 
 	if (megasas_wait_for_adapter_operational(instance))
@@ -7812,6 +7824,8 @@ static void megasas_detach_one(struct pci_dev *pdev)
 	megasas_shutdown_controller(instance, MR_DCMD_CTRL_SHUTDOWN);
 
 skip_firing_dcmds:
+	megasas_io_detach(instance);
+
 	/* cancel the delayed work if this work still in queue*/
 	if (instance->ev != NULL) {
 		struct megasas_aen_event *ev = instance->ev;
drivers/scsi/megaraid/megaraid_sas_fusion.c +5 -0
index bec3d4cca74f..483146051957 100644
@@ -1857,6 +1857,9 @@ megasas_init_adapter_fusion(struct megasas_instance *instance)
 	if (megasas_alloc_cmds_fusion(instance))
 		goto fail_alloc_cmds;
 
+	if (megasas_io_attach(instance))
+		goto fail_io_attach;
+
 	if (megasas_ioc_init_fusion(instance)) {
 		status_reg = instance->instancet->read_fw_status_reg(instance);
 		if (((status_reg & MFI_STATE_MASK) == MFI_STATE_FAULT) &&
@@ -1895,6 +1898,8 @@ megasas_init_adapter_fusion(struct megasas_instance *instance)
 	return 0;
 
 fail_ioc_init:
+	megasas_io_detach(instance);
+fail_io_attach:
 	megasas_free_cmds_fusion(instance);
 fail_alloc_cmds:
 	megasas_free_cmds(instance);
Signature · 1 line
2.16.4
#60 lore raw

RFC PATCH v3 22/41 block: implement persistent commands

In reply to Hannes Reinecke

Some LLDDs implement event handling by sending a command to the firmware, which then will be completed once the firmware wants to register an event.
So worst case a command is being sent to the firmware then the driver initializes, and will be returned once the driver unloads. To avoid these commands to block the queues during freezing or quiescing this patch implements support for 'persistent' commands, which will be excluded from blk_queue_enter() and blk_queue_exit() calls.

---
 block/blk-mq.c            | 12 +++++++++---
 include/linux/blk-mq.h    |  2 ++
 include/linux/blk_types.h |  4 ++++
 3 files changed, 15 insertions(+), 3 deletions(-)
block/blk-mq.c +9 -3
index 44482aaed11e..402cf104d183 100644
@@ -402,9 +402,14 @@ struct request *blk_mq_alloc_request(struct request_queue *q, unsigned int op,
 {
 	struct blk_mq_alloc_data alloc_data = { .flags = flags, .cmd_flags = op };
 	struct request *rq;
-	int ret;
+	int ret = 0;
 
-	ret = blk_queue_enter(q, flags);
+	if (flags & BLK_MQ_REQ_PERSISTENT) {
+		if (blk_queue_dying(q))
+			ret = -ENODEV;
+		alloc_data.cmd_flags |= REQ_PERSISTENT;
+	} else
+		ret = blk_queue_enter(q, flags);
 	if (ret)
 		return ERR_PTR(ret);
 
@@ -481,7 +486,8 @@ static void __blk_mq_free_request(struct request *rq)
 	if (sched_tag != -1)
 		blk_mq_put_tag(hctx->sched_tags, ctx, sched_tag);
 	blk_mq_sched_restart(hctx);
-	blk_queue_exit(q);
+	if (!(rq->cmd_flags & REQ_PERSISTENT))
+		blk_queue_exit(q);
 }
 
 void blk_mq_free_request(struct request *rq)
include/linux/blk-mq.h +2 -0
index c186dc25fc1c..a4b02196810c 100644
@@ -441,6 +441,8 @@ enum {
 	BLK_MQ_REQ_INTERNAL	= (__force blk_mq_req_flags_t)(1 << 2),
 	/* set RQF_PREEMPT */
 	BLK_MQ_REQ_PREEMPT	= (__force blk_mq_req_flags_t)(1 << 3),
+	/* mark request as persistent */
+	BLK_MQ_REQ_PERSISTENT	= (__force blk_mq_req_flags_t)(1 << 4),
 };
 
 struct request *blk_mq_alloc_request(struct request_queue *q, unsigned int op,
include/linux/blk_types.h +4 -0
index 70254ae11769..898e75e2e8b0 100644
@@ -336,6 +336,9 @@ enum req_flag_bits {
 	/* command specific flags for REQ_OP_WRITE_ZEROES: */
 	__REQ_NOUNMAP,		/* do not free blocks when zeroing */
 
+	/* Persistent firmware command, ignore for q_usage_counter */
+	__REQ_PERSISTENT,
+
 	__REQ_HIPRI,
 
 	/* for driver use */
@@ -362,6 +365,7 @@ enum req_flag_bits {
 #define REQ_CGROUP_PUNT		(1ULL << __REQ_CGROUP_PUNT)
 
 #define REQ_NOUNMAP		(1ULL << __REQ_NOUNMAP)
+#define REQ_PERSISTENT		(1ULL << __REQ_PERSISTENT)
 #define REQ_HIPRI		(1ULL << __REQ_HIPRI)
 
 #define REQ_DRV			(1ULL << __REQ_DRV)
Signature · 1 line
2.16.4
#61 lore raw

Re: [PATCH RFC v3 22/41] block: implement persistent commands

Quoted 3 lines · Hannes Reinecke

Some LLDDs implement event handling by sending a command to the firmware, which then will be completed once the firmware wants to register an event.

     ^^^^^^^^
     report?
Quoted 1 line

So worst case a command is being sent to the firmware then the

                                                        ^^^^
                                                        when?
Quoted 5 lines

driver initializes, and will be returned once the driver unloads. To avoid these commands to block the queues during freezing or quiescing this patch implements support for 'persistent' commands, which will be excluded from blk_queue_enter() and blk_queue_exit() calls.

How is it prevented that the SCSI timeout handler is activated for persistent commands?

Quoted 18 lines
---
 block/blk-mq.c            | 12 +++++++++---
 include/linux/blk-mq.h    |  2 ++
 include/linux/blk_types.h |  4 ++++
 3 files changed, 15 insertions(+), 3 deletions(-)
block/blk-mq.c +1 -1
index 44482aaed11e..402cf104d183 100644
@@ -402,9 +402,14 @@ struct request *blk_mq_alloc_request(struct request_queue *q, unsigned int op,
 {
 	struct blk_mq_alloc_data alloc_data = { .flags = flags, .cmd_flags = op };
 	struct request *rq;
-	int ret;
+	int ret = 0;
 

block/blk-mq.c struct request *blk_mq_alloc_request(struct request_queue *q, unsigned int op, · line 415

Quoted 10 lines
block/blk-mq.c +6 -1
-	ret = blk_queue_enter(q, flags);
+	if (flags & BLK_MQ_REQ_PERSISTENT) {
+		if (blk_queue_dying(q))
+			ret = -ENODEV;
+		alloc_data.cmd_flags |= REQ_PERSISTENT;
+	} else
+		ret = blk_queue_enter(q, flags);
 	if (ret)
 		return ERR_PTR(ret);
 

I think that not calling blk_queue_enter() for persistent commands means opening a giant can of worms. There is quite some code in the block layer that assumes that neither .queue_rq() nor the request completion code will be called if q_usage_counter == 0. Skipping the blk_queue_enter() call for persistent commands breaks that assumption. I think we need a better solution.

Thanks,

Bart.

#62 lore raw

Re: [PATCH RFC v3 22/41] block: implement persistent commands

Quoted 46 lines · Bart Van Assche
Quoted 3 lines · Hannes Reinecke

Some LLDDs implement event handling by sending a command to the firmware, which then will be completed once the firmware wants to register an event.

      ^^^^^^^^
      report?
Quoted 1 line

So worst case a command is being sent to the firmware then the

                                                         ^^^^
                                                         when?
Quoted 5 lines

driver initializes, and will be returned once the driver unloads. To avoid these commands to block the queues during freezing or quiescing this patch implements support for 'persistent' commands, which will be excluded from blk_queue_enter() and blk_queue_exit() calls.

How is it prevented that the SCSI timeout handler is activated for persistent commands?

Quoted 27 lines
---
  block/blk-mq.c            | 12 +++++++++---
  include/linux/blk-mq.h    |  2 ++
  include/linux/blk_types.h |  4 ++++
  3 files changed, 15 insertions(+), 3 deletions(-)
block/blk-mq.c +7 -2
index 44482aaed11e..402cf104d183 100644
@@ -402,9 +402,14 @@ struct request *blk_mq_alloc_request(struct request_queue *q, unsigned int op,
  {
  	struct blk_mq_alloc_data alloc_data = { .flags = flags, .cmd_flags = op };
  	struct request *rq;
-	int ret;
+	int ret = 0;
 
-	ret = blk_queue_enter(q, flags);
+	if (flags & BLK_MQ_REQ_PERSISTENT) {
+		if (blk_queue_dying(q))
+			ret = -ENODEV;
+		alloc_data.cmd_flags |= REQ_PERSISTENT;
+	} else
+		ret = blk_queue_enter(q, flags);
  	if (ret)
  		return ERR_PTR(ret);
Quoted 9 lines
Quoted 1 line

I think that not calling blk_queue_enter() for persistent commands means opening a giant can of worms. There is quite some code in the block layer that assumes that neither .queue_rq() nor the request completion code will be called if q_usage_counter == 0. Skipping the blk_queue_enter() call for persistent commands breaks that assumption. I think we need a better solution.

Well, yeah, maybe.
My aim here is that _all_ I/O requiring a tag from the hardware will be tracked by the blocklayer tagset. Only that will give the block-layer accurate information about outstanding commands, such that the ongoing CPU hotplug discussion can make the correct decisions and implement functions really covering all outstanding I/O. It also allows us to use the scsi_host_busy_iter() functions within the driver, and will get rid of the hand-crafted iterations the driver has to do now.

It worked reasonably well, until I encountered the infamous AEN commands, which actually require the opposite: _not_ to be tracked by the block layer at all, as the commands themselves are just placeholders to be returned by the firmware once an event occurs.
(And yes, I _do_ think this is a quite dangerous operation, because I can't quite see how one could reliably return this command in case of a firmware crash ...)
(But anyhow, that's how the firmware is written and we have to live with it.)

So I implemented this approach, to have tags which are ignored by the block layer. But I have to admit that this approach relies on quite some assumptions (like these tags are never actually submitted to the blocklayer itself, are never started etc), none of which are spelled out clearly (yet). An alternative approach would be to arbitrary decrease the tagset size by one (eg by shifting the tags by one), and use the free tag for AENs).
That would have to be coded within the driver, though.

If that's a solution which you like better I could give it a go.

Cheers,

Hannes

Signature · 4 lines
Dr. Hannes Reinecke            Teamlead Storage & Networking
[email protected]                               +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer
#63 lore raw

Re: [PATCH RFC v3 22/41] block: implement persistent commands

Quoted 61 lines · Hannes Reinecke
Quoted 22 lines · Bart Van Assche
Quoted 21 lines · Hannes Reinecke
block/blk-mq.c +7 -1
index 44482aaed11e..402cf104d183 100644
@@ -402,9 +402,14 @@ struct request *blk_mq_alloc_request(struct
 request_queue *q, unsigned int op,
   {
       struct blk_mq_alloc_data alloc_data = { .flags = flags,
 .cmd_flags = op };
       struct request *rq;
-    int ret;
+    int ret = 0;
   -    ret = blk_queue_enter(q, flags);
+    if (flags & BLK_MQ_REQ_PERSISTENT) {
+        if (blk_queue_dying(q))
+            ret = -ENODEV;
+        alloc_data.cmd_flags |= REQ_PERSISTENT;
+    } else
+        ret = blk_queue_enter(q, flags);
       if (ret)

          return ERR_PTR(ret);

Quoted 9 lines
Quoted 1 line

I think that not calling blk_queue_enter() for persistent commands means opening a giant can of worms. There is quite some code in the block layer that assumes that neither .queue_rq() nor the request completion code will be called if q_usage_counter == 0. Skipping the blk_queue_enter() call for persistent commands breaks that assumption. I think we need a better solution.

Well, yeah, maybe.
My aim here is that _all_ I/O requiring a tag from the hardware will be tracked by the blocklayer tagset. Only that will give the block-layer accurate information about outstanding commands, such that the ongoing CPU hotplug discussion can make the correct decisions and implement functions really covering all outstanding I/O.
It also allows us to use the scsi_host_busy_iter() functions within the driver, and will get rid of the hand-crafted iterations the driver has to do now.

It worked reasonably well, until I encountered the infamous AEN commands, which actually require the opposite: _not_ to be tracked by the block layer at all, as the commands themselves are just placeholders to be returned by the firmware once an event occurs.
(And yes, I _do_ think this is a quite dangerous operation, because I can't quite see how one could reliably return this command in case of a firmware crash ...)
(But anyhow, that's how the firmware is written and we have to live with it.)

So I implemented this approach, to have tags which are ignored by the block layer. But I have to admit that this approach relies on quite some assumptions (like these tags are never actually submitted to the blocklayer itself, are never started etc), none of which are spelled out clearly (yet).
An alternative approach would be to arbitrary decrease the tagset size by one (eg by shifting the tags by one), and use the free tag for AENs). That would have to be coded within the driver, though.

Quoted 1 line

If that's a solution which you like better I could give it a go.

How about dropping support for AEN commands entirely? As far as I know such a command has never been standardized. Additionally, all SCSI core code I'm familiar with supports unit attentions and does not rely on asynchronous events to be reported immediately.

If dropping support for AEN commands is not an option, how about aborting these commands before freezing a request queue?

Thanks,

Bart.

#64 lore raw

Re: [PATCH RFC v3 22/41] block: implement persistent commands

In reply to Hannes Reinecke
Quoted 9 lines · Hannes Reinecke

Some LLDDs implement event handling by sending a command to the firmware, which then will be completed once the firmware wants to register an event.
So worst case a command is being sent to the firmware then the driver initializes, and will be returned once the driver unloads. To avoid these commands to block the queues during freezing or quiescing this patch implements support for 'persistent' commands, which will be excluded from blk_queue_enter() and blk_queue_exit() calls.

This way is quite dangerous from block layer viewpoint, and it should have been done in driver/device specific way instead of polluting block layer.

thanks,
Ming

#65 lore raw

Re: [PATCH RFC v3 22/41] block: implement persistent commands

Quoted 11 lines · Ming Lei
Quoted 9 lines · Hannes Reinecke

Some LLDDs implement event handling by sending a command to the firmware, which then will be completed once the firmware wants to register an event.
So worst case a command is being sent to the firmware then the driver initializes, and will be returned once the driver unloads. To avoid these commands to block the queues during freezing or quiescing this patch implements support for 'persistent' commands, which will be excluded from blk_queue_enter() and blk_queue_exit() calls.

Quoted 4 lines

This way is quite dangerous from block layer viewpoint, and it should have been done in driver/device specific way instead of polluting block layer.

As already outlined in the reply to Bart, I'll be rewriting that requiring the drivers to set aside a separate tag and decrease the tagspace by one. That should work as well.

Cheers,

Hannes

Signature · 4 lines
Dr. Hannes Reinecke            Teamlead Storage & Networking
[email protected]                               +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer
#66 lore raw

RFC PATCH v3 23/41 scsi: add a 'persistent' argument to scsi_get_reserved_cmd()

In reply to Hannes Reinecke

To correctly handle AENs the corresponding command needs to be marked as 'persistent', so add an additional flag to scsi_get_reserved_cmd().

---
 drivers/scsi/aacraid/commsup.c    |  2 +-
 drivers/scsi/csiostor/csio_scsi.c |  2 +-
 drivers/scsi/fnic/fnic_scsi.c     |  2 +-
 drivers/scsi/hpsa.c               |  2 +-
 drivers/scsi/scsi_lib.c           | 20 +++++++++++++++-----
 drivers/scsi/virtio_scsi.c        |  4 ++--
 include/scsi/scsi_device.h        |  2 +-
 7 files changed, 22 insertions(+), 12 deletions(-)
drivers/scsi/aacraid/commsup.c +1 -1
index 8c6da75188a6..dc9b5613b89f 100644
@@ -244,7 +244,7 @@ struct fib *aac_fib_alloc(struct aac_dev *dev, int direction)
 	unsigned long flags;
 
 	spin_lock_irqsave(&dev->fib_lock, flags);
-	scmd = scsi_get_reserved_cmd(dev->scsi_host_dev, direction);
+	scmd = scsi_get_reserved_cmd(dev->scsi_host_dev, direction, false);
 	if (scmd)
 		fibptr = aac_fib_alloc_tag(dev, scmd);
 	spin_unlock_irqrestore(&dev->fib_lock, flags);
drivers/scsi/csiostor/csio_scsi.c +1 -1
index 273a8b952e69..69597fd15740 100644
@@ -2105,7 +2105,7 @@ csio_eh_lun_reset_handler(struct scsi_cmnd *cmnd)
 		goto fail;
 	}
 
-	reset_cmnd = scsi_get_reserved_cmd(sdev, DMA_NONE);
+	reset_cmnd = scsi_get_reserved_cmd(sdev, DMA_NONE, false);
 	if (!reset_cmnd) {
 		csio_err(hw, "No free TMF request\n");
 		goto fail;
drivers/scsi/fnic/fnic_scsi.c +1 -1
index 94d0db99d4ec..454f8c1c9e75 100644
@@ -2237,7 +2237,7 @@ int fnic_device_reset(struct scsi_cmnd *sc)
 		goto fnic_device_reset_end;
 	}
 
-	reset_sc = scsi_get_reserved_cmd(sdev, DMA_NONE);
+	reset_sc = scsi_get_reserved_cmd(sdev, DMA_NONE, false);
 	if (unlikely(!reset_sc))
 		goto fnic_device_reset_end;
 
drivers/scsi/hpsa.c +1 -1
index 6f66cec0e2cc..628752909cd3 100644
@@ -6148,7 +6148,7 @@ static struct CommandList *cmd_alloc(struct ctlr_info *h, u8 direction)
 	int idx;
 
 	scmd = scsi_get_reserved_cmd(h->raid_ctrl_sdev, direction & XFER_WRITE ?
-				     DMA_TO_DEVICE : DMA_FROM_DEVICE);
+				     DMA_TO_DEVICE : DMA_FROM_DEVICE, false);
 	if (!scmd) {
 		dev_warn(&h->pdev->dev, "failed to allocate reserved cmd\n");
 		return NULL;
drivers/scsi/scsi_lib.c +15 -5
index ce9f1d83aaee..8befa6b63fe5 100644
@@ -1929,17 +1929,27 @@ void scsi_mq_destroy_tags(struct Scsi_Host *shost)
  * scsi_get_reserved_cmd - allocate a SCSI command from reserved tags
  * @sdev: SCSI device from which to allocate the command
  * @data_direction: Data direction for the allocated command
+ * @persistent: Allocate a persistent command
  */
 struct scsi_cmnd *scsi_get_reserved_cmd(struct scsi_device *sdev,
-					int data_direction)
+					int data_direction, bool persistent)
 {
 	struct request *rq;
 	struct scsi_cmnd *scmd;
+	blk_mq_req_flags_t flags = 0;
+	int op = REQ_NOWAIT;
 
-	rq = blk_mq_alloc_request(sdev->request_queue,
-				  data_direction == DMA_TO_DEVICE ?
-				  REQ_OP_SCSI_OUT : REQ_OP_SCSI_IN | REQ_NOWAIT,
-				  BLK_MQ_REQ_RESERVED);
+	if (sdev->host->nr_reserved_cmds) {
+		flags |= BLK_MQ_REQ_RESERVED;
+		if (persistent)
+			flags |= BLK_MQ_REQ_PERSISTENT;
+	}
+	if (data_direction == DMA_TO_DEVICE)
+		op |= REQ_OP_SCSI_OUT;
+	else
+		op |= REQ_OP_SCSI_IN;
+
+	rq = blk_mq_alloc_request(sdev->request_queue, op, flags);
 	if (IS_ERR(rq))
 		return NULL;
 	scmd = blk_mq_rq_to_pdu(rq);
drivers/scsi/virtio_scsi.c +2 -2
index 26054c29d897..4024588d22e0 100644
@@ -620,7 +620,7 @@ static int virtscsi_device_reset(struct scsi_cmnd *sc)
 	int rc;
 
 	sdev_printk(KERN_INFO, sdev, "device reset\n");
-	reset_sc = scsi_get_reserved_cmd(sdev, DMA_NONE);
+	reset_sc = scsi_get_reserved_cmd(sdev, DMA_NONE, false);
 	if (!reset_sc)
 		return FAILED;
 	cmd = scsi_cmd_priv(reset_sc);
@@ -684,7 +684,7 @@ static int virtscsi_abort(struct scsi_cmnd *sc)
 	int rc;
 
 	scmd_printk(KERN_INFO, sc, "abort\n");
-	reset_sc = scsi_get_reserved_cmd(sdev, DMA_NONE);
+	reset_sc = scsi_get_reserved_cmd(sdev, DMA_NONE, false);
 	if (!reset_sc)
 		return FAILED;
 	cmd = scsi_cmd_priv(reset_sc);
include/scsi/scsi_device.h +1 -1
index 6039ce7d09d7..f4112de78045 100644
@@ -459,7 +459,7 @@ static inline int scsi_execute_req(struct scsi_device *sdev,
 		bufflen, NULL, sshdr, timeout, retries,  0, 0, resid);
 }
 struct scsi_cmnd *scsi_get_reserved_cmd(struct scsi_device *sdev,
-					int data_direction);
+					int data_direction, bool persistent);
 void scsi_put_reserved_cmd(struct scsi_cmnd *scmd);
 extern void sdev_disable_disk_events(struct scsi_device *sdev);
 extern void sdev_enable_disk_events(struct scsi_device *sdev);
Signature · 1 line
2.16.4
#67 lore raw

RFC PATCH v3 24/41 megaraid_sas: separate out megasas_prepare_aen()

In reply to Hannes Reinecke

Separate out code for preparing an AEN into a separate function.

---
 drivers/scsi/megaraid/megaraid_sas_base.c | 85 +++++++++++++++++--------------
 1 file changed, 46 insertions(+), 39 deletions(-)
drivers/scsi/megaraid/megaraid_sas_base.c +46 -39
index ddb8df03481f..8498e6a1d67c 100644
@@ -6506,6 +6506,51 @@ megasas_get_seq_num(struct megasas_instance *instance,
 	return ret;
 }
 
+static struct megasas_cmd *
+megasas_prepare_aen(struct megasas_instance *instance, u32 seq_num,
+		    union megasas_evt_class_locale *aen)
+{
+	struct megasas_cmd *cmd;
+	struct megasas_dcmd_frame *dcmd;
+
+	cmd = megasas_get_cmd(instance);
+	if (!cmd)
+		return NULL;
+
+	dcmd = &cmd->frame->dcmd;
+
+	memset(instance->evt_detail, 0, sizeof(struct megasas_evt_detail));
+
+	/*
+	 * Prepare DCMD for aen registration
+	 */
+	memset(dcmd->mbox.b, 0, MFI_MBOX_SIZE);
+
+	dcmd->cmd = MFI_CMD_DCMD;
+	dcmd->cmd_status = 0x0;
+	dcmd->sge_count = 1;
+	dcmd->flags = MFI_FRAME_DIR_READ;
+	dcmd->timeout = 0;
+	dcmd->pad_0 = 0;
+	dcmd->data_xfer_len = cpu_to_le32(sizeof(struct megasas_evt_detail));
+	dcmd->opcode = cpu_to_le32(MR_DCMD_CTRL_EVENT_WAIT);
+	dcmd->mbox.w[0] = cpu_to_le32(seq_num);
+	instance->last_seq_num = seq_num;
+	dcmd->mbox.w[1] = cpu_to_le32(aen->word);
+
+	megasas_set_dma_settings(instance, dcmd, instance->evt_detail_h,
+				 sizeof(struct megasas_evt_detail));
+
+	/*
+	 * Store reference to the cmd used to register for AEN. When an
+	 * application wants us to register for AEN, we have to abort this
+	 * cmd and re-register with a new EVENT LOCALE supplied by that app
+	 */
+	instance->aen_cmd = cmd;
+
+	return cmd;
+}
+
 /**
  * megasas_register_aen -	Registers for asynchronous event notification
  * @instance:			Adapter soft state
@@ -6521,7 +6566,6 @@ megasas_register_aen(struct megasas_instance *instance, u32 seq_num,
 {
 	int ret_val;
 	struct megasas_cmd *cmd;
-	struct megasas_dcmd_frame *dcmd;
 	union megasas_evt_class_locale curr_aen;
 	union megasas_evt_class_locale prev_aen;
 
@@ -6590,47 +6634,10 @@ megasas_register_aen(struct megasas_instance *instance, u32 seq_num,
 		}
 	}
 
-	cmd = megasas_get_cmd(instance);
-
+	cmd = megasas_prepare_aen(instance, seq_num, &curr_aen);
 	if (!cmd)
 		return -ENOMEM;
 
-	dcmd = &cmd->frame->dcmd;
-
-	memset(instance->evt_detail, 0, sizeof(struct megasas_evt_detail));
-
-	/*
-	 * Prepare DCMD for aen registration
-	 */
-	memset(dcmd->mbox.b, 0, MFI_MBOX_SIZE);
-
-	dcmd->cmd = MFI_CMD_DCMD;
-	dcmd->cmd_status = 0x0;
-	dcmd->sge_count = 1;
-	dcmd->flags = MFI_FRAME_DIR_READ;
-	dcmd->timeout = 0;
-	dcmd->pad_0 = 0;
-	dcmd->data_xfer_len = cpu_to_le32(sizeof(struct megasas_evt_detail));
-	dcmd->opcode = cpu_to_le32(MR_DCMD_CTRL_EVENT_WAIT);
-	dcmd->mbox.w[0] = cpu_to_le32(seq_num);
-	instance->last_seq_num = seq_num;
-	dcmd->mbox.w[1] = cpu_to_le32(curr_aen.word);
-
-	megasas_set_dma_settings(instance, dcmd, instance->evt_detail_h,
-				 sizeof(struct megasas_evt_detail));
-
-	if (instance->aen_cmd != NULL) {
-		megasas_return_cmd(instance, cmd);
-		return 0;
-	}
-
-	/*
-	 * Store reference to the cmd used to register for AEN. When an
-	 * application wants us to register for AEN, we have to abort this
-	 * cmd and re-register with a new EVENT LOCALE supplied by that app
-	 */
-	instance->aen_cmd = cmd;
-
 	/*
 	 * Issue the aen registration frame
 	 */
Signature · 1 line
2.16.4
#68 lore raw

RFC PATCH v3 25/41 megaraid_sas: use reserved commands

In reply to Hannes Reinecke

Implement support for reserved commands to allow the block layer to track all commands.

---
 drivers/scsi/megaraid/megaraid_sas.h        |   8 +-
 drivers/scsi/megaraid/megaraid_sas_base.c   | 142 +++++++++++++++++-----------
 drivers/scsi/megaraid/megaraid_sas_fusion.c |  33 +++----
 3 files changed, 106 insertions(+), 77 deletions(-)
drivers/scsi/megaraid/megaraid_sas.h +6 -2
index b47306a66650..bb765e715011 100644
@@ -2336,7 +2336,6 @@ struct megasas_instance {
 	struct megasas_aen_event *ev;
 
 	struct megasas_cmd **cmd_list;
-	struct list_head cmd_pool;
 	/* used to sync fire the cmd to fw */
 	spinlock_t mfi_pool_lock;
 	/* used to sync fire the cmd to fw */
@@ -2353,6 +2352,7 @@ struct megasas_instance {
 	struct semaphore ioctl_sem;
 
 	struct Scsi_Host *host;
+	struct scsi_device *host_dev;
 
 	wait_queue_head_t int_cmd_wait_q;
 	wait_queue_head_t abort_cmd_wait_q;
@@ -2680,11 +2680,15 @@ void megasas_free_host_crash_buffer(struct megasas_instance *instance);
 int megasas_io_attach(struct megasas_instance *instance);
 void megasas_io_detach(struct megasas_instance *instance);
 
+struct megasas_cmd *megasas_get_cmd(struct megasas_instance *instance,
+	int direction, bool persistent);
+void megasas_complete_cmd(struct megasas_instance *instance,
+	struct megasas_cmd *cmd, u8 alt_status);
 void megasas_return_cmd_fusion(struct megasas_instance *instance,
 	struct megasas_cmd_fusion *cmd);
 int megasas_issue_blocked_cmd(struct megasas_instance *instance,
 	struct megasas_cmd *cmd, int timeout);
-void __megasas_return_cmd(struct megasas_instance *instance,
+void megasas_return_cmd(struct megasas_instance *instance,
 	struct megasas_cmd *cmd);
 
 void megasas_return_mfi_mpt_pthr(struct megasas_instance *instance,
drivers/scsi/megaraid/megaraid_sas_base.c +86 -56

369 lines not on this page · raw message

drivers/scsi/megaraid/megaraid_sas_fusion.c +14 -19
index 483146051957..f5915c9f474c 100644
@@ -49,17 +49,10 @@
 
 
 extern void megasas_free_cmds(struct megasas_instance *instance);
-extern struct megasas_cmd *megasas_get_cmd(struct megasas_instance
-					   *instance);
-extern void
-megasas_complete_cmd(struct megasas_instance *instance,
-		     struct megasas_cmd *cmd, u8 alt_status);
 int
 wait_and_poll(struct megasas_instance *instance, struct megasas_cmd *cmd,
 	      int seconds);
 
-void
-megasas_return_cmd(struct megasas_instance *instance, struct megasas_cmd *cmd);
 int megasas_alloc_cmds(struct megasas_instance *instance);
 int
 megasas_clear_intr_fusion(struct megasas_instance *instance);
@@ -343,10 +336,11 @@ megasas_fusion_update_can_queue(struct megasas_instance *instance, int fw_boot_c
 	if (fw_boot_context == OCR_CONTEXT) {
 		cur_max_fw_cmds = cur_max_fw_cmds - 1;
 		if (cur_max_fw_cmds < instance->max_fw_cmds) {
-			instance->cur_can_queue =
-				cur_max_fw_cmds - (MEGASAS_FUSION_INTERNAL_CMDS +
-						MEGASAS_FUSION_IOCTL_CMDS);
+			instance->cur_can_queue = cur_max_fw_cmds;
 			instance->host->can_queue = instance->cur_can_queue;
+			instance->host->nr_reserved_cmds =
+				MEGASAS_FUSION_INTERNAL_CMDS +
+				MEGASAS_FUSION_IOCTL_CMDS;
 			instance->ldio_threshold = ldio_threshold;
 		}
 	} else {
@@ -1291,7 +1285,8 @@ megasas_sync_pd_seq_num(struct megasas_instance *instance, bool pend) {
 	pd_seq_h = fusion->pd_seq_phys[(instance->pd_seq_map_id & 1)];
 	pd_seq_map_sz = struct_size(pd_sync, seq, MAX_PHYSICAL_DEVICES - 1);
 
-	cmd = megasas_get_cmd(instance);
+	cmd = megasas_get_cmd(instance,
+		pend ? DMA_TO_DEVICE : DMA_FROM_DEVICE, pend);
 	if (!cmd) {
 		dev_err(&instance->pdev->dev,
 			"Could not get mfi cmd. Fail from %s %d\n",
@@ -1379,10 +1374,11 @@ megasas_get_ld_map_info(struct megasas_instance *instance)
 	u32 size_map_info;
 	struct fusion_context *fusion;
 
-	cmd = megasas_get_cmd(instance);
+	cmd = megasas_get_cmd(instance, DMA_FROM_DEVICE, false);
 
 	if (!cmd) {
-		dev_printk(KERN_DEBUG, &instance->pdev->dev, "Failed to get cmd for map info\n");
+		dev_printk(KERN_DEBUG, &instance->pdev->dev,
+			   "Failed to get cmd for map info\n");
 		return -ENOMEM;
 	}
 
@@ -1473,13 +1469,13 @@ megasas_sync_map_info(struct megasas_instance *instance)
 	dma_addr_t ci_h = 0;
 	u32 size_map_info;
 
-	cmd = megasas_get_cmd(instance);
+	cmd = megasas_get_cmd(instance, DMA_TO_DEVICE, true);
 
 	if (!cmd) {
-		dev_printk(KERN_DEBUG, &instance->pdev->dev, "Failed to get cmd for sync info\n");
+		dev_printk(KERN_DEBUG, &instance->pdev->dev,
+			   "Failed to get cmd for sync info\n");
 		return -ENOMEM;
 	}
-
 	fusion = instance->ctrl_context;
 
 	if (!fusion) {
@@ -1699,8 +1695,7 @@ void megasas_configure_queue_sizes(struct megasas_instance *instance)
 		instance->max_mpt_cmds = instance->max_fw_cmds;
 
 	instance->max_scsi_cmds = instance->max_fw_cmds - instance->max_mfi_cmds;
-	instance->cur_can_queue = instance->max_scsi_cmds;
-	instance->host->can_queue = instance->cur_can_queue;
+	instance->cur_can_queue = instance->max_fw_cmds;
 
 	fusion->reply_q_depth = 2 * ((max_cmd + 1 + 15) / 16) * 16;
 
@@ -4477,7 +4472,7 @@ megasas_issue_tm(struct megasas_instance *instance, u16 device_handle,
 
 	fusion = instance->ctrl_context;
 
-	cmd_mfi = megasas_get_cmd(instance);
+	cmd_mfi = megasas_get_cmd(instance, DMA_NONE, false);
 
 	if (!cmd_mfi) {
 		dev_err(&instance->pdev->dev, "Failed from %s %d\n",
Signature · 1 line
2.16.4
#69 lore raw

RFC PATCH v3 26/41 megaraid_sas_fusion: rearrange mfi and mpt frame pools

In reply to Hannes Reinecke

Rearrange the mfi frame pool to start at the beginning of the mpt frame pool. This means that the command index for the mpt and the mfi frame pool is identical (up to the size of the mfi frame pool, of course), and the 'sync_cmd_idx' structure entry can be dropped.

---
 drivers/scsi/megaraid/megaraid_sas_base.c   |  6 +++---
 drivers/scsi/megaraid/megaraid_sas_fusion.c | 28 +++++++++++-----------------
 drivers/scsi/megaraid/megaraid_sas_fusion.h |  5 -----
 3 files changed, 14 insertions(+), 25 deletions(-)
drivers/scsi/megaraid/megaraid_sas_base.c +3 -3
index 45adfd4b6b07..306c6495f1da 100644
@@ -345,7 +345,7 @@ megasas_return_cmd(struct megasas_instance *instance, struct megasas_cmd *cmd)
 		return;
 
 	if (fusion) {
-		blk_tags = instance->max_scsi_cmds + cmd->index;
+		blk_tags = cmd->index;
 		cmd_fusion = fusion->cmd_list[blk_tags];
 		megasas_return_cmd_fusion(instance, cmd_fusion);
 	}
@@ -2123,8 +2123,8 @@ static void megasas_complete_outstanding_ioctls(struct megasas_instance *instanc
 	if (fusion) {
 		for (i = 0; i < instance->max_fw_cmds; i++) {
 			cmd_fusion = fusion->cmd_list[i];
-			if (cmd_fusion->sync_cmd_idx != (u32)ULONG_MAX) {
-				cmd_mfi = instance->cmd_list[cmd_fusion->sync_cmd_idx];
+			if (i < instance->max_mfi_cmds) {
+				cmd_mfi = instance->cmd_list[i];
 				if (cmd_mfi->sync_cmd &&
 				    (cmd_mfi->frame->hdr.cmd != MFI_CMD_ABORT)) {
 					cmd_mfi->frame->hdr.cmd_status =
drivers/scsi/megaraid/megaraid_sas_fusion.c +11 -17
index f5915c9f474c..8e2ae44ab1f8 100644
@@ -982,10 +982,6 @@ megasas_alloc_cmds_fusion(struct megasas_instance *instance)
 		memset(cmd, 0, sizeof(struct megasas_cmd_fusion));
 		cmd->index = i + 1;
 		cmd->scmd = NULL;
-		cmd->sync_cmd_idx =
-		(i >= instance->max_scsi_cmds && i < instance->max_fw_cmds) ?
-				(i - instance->max_scsi_cmds) :
-				(u32)ULONG_MAX; /* Set to Invalid */
 		cmd->instance = instance;
 		cmd->io_request =
 			(struct MPI2_RAID_SCSI_IO_REQUEST *)
@@ -3612,7 +3608,7 @@ complete_cmd_fusion(struct megasas_instance *instance, u32 MSIxIndex,
 				megasas_complete_r1_command(instance, cmd_fusion);
 			break;
 		case MEGASAS_MPI2_FUNCTION_PASSTHRU_IO_REQUEST: /*MFI command */
-			cmd_mfi = instance->cmd_list[cmd_fusion->sync_cmd_idx];
+			cmd_mfi = instance->cmd_list[cmd_fusion->index - 1];
 			/* Poll mode. Dummy free.
 			 * In case of Interrupt mode, caller has reverse check.
 			 */
@@ -3836,8 +3832,7 @@ build_mpt_mfi_pass_thru(struct megasas_instance *instance,
 
 	fusion = instance->ctrl_context;
 
-	cmd = megasas_get_cmd_fusion(instance,
-			instance->max_scsi_cmds + mfi_cmd->index);
+	cmd = megasas_get_cmd_fusion(instance, mfi_cmd->index);
 
 	/*  Save the smid. To be used for returning the cmd */
 	mfi_cmd->context.smid = cmd->index;
@@ -4246,11 +4241,11 @@ static void megasas_refire_mgmt_cmd(struct megasas_instance *instance,
 	fusion = instance->ctrl_context;
 
 	/* Re-fire management commands.
-	 * Do not traverse complet MPT frame pool. Start from max_scsi_cmds.
+	 * Do not traverse complete MPT frame pool, only the MFI frame pool.
 	 */
-	for (j = instance->max_scsi_cmds ; j < instance->max_fw_cmds; j++) {
+	for (j = 0; j < instance->max_mfi_cmds; j++) {
 		cmd_fusion = fusion->cmd_list[j];
-		cmd_mfi = instance->cmd_list[cmd_fusion->sync_cmd_idx];
+		cmd_mfi = instance->cmd_list[j];
 		smid = le16_to_cpu(cmd_mfi->context.smid);
 		result = REFIRE_CMD;
 
@@ -4334,9 +4329,9 @@ megasas_return_polled_cmds(struct megasas_instance *instance)
 
 	fusion = instance->ctrl_context;
 
-	for (i = instance->max_scsi_cmds; i < instance->max_fw_cmds; i++) {
+	for (i = 0; i < instance->max_mfi_cmds; i++) {
 		cmd_fusion = fusion->cmd_list[i];
-		cmd_mfi = instance->cmd_list[cmd_fusion->sync_cmd_idx];
+		cmd_mfi = instance->cmd_list[i];
 
 		if (cmd_mfi->flags & DRV_DCMD_POLLED_MODE) {
 			if (megasas_dbg_lvl & OCR_DEBUG)
@@ -4367,7 +4362,7 @@ static int megasas_track_scsiio(struct megasas_instance *instance,
 	struct fusion_context *fusion;
 	fusion = instance->ctrl_context;
 
-	for (i = 0 ; i < instance->max_scsi_cmds; i++) {
+	for (i = instance->max_mfi_cmds; i < instance->max_fw_cmds; i++) {
 		cmd_fusion = fusion->cmd_list[i];
 		if (cmd_fusion->scmd &&
 			(cmd_fusion->scmd->device->id == id &&
@@ -4480,8 +4475,7 @@ megasas_issue_tm(struct megasas_instance *instance, u16 device_handle,
 		return -ENOMEM;
 	}
 
-	cmd_fusion = megasas_get_cmd_fusion(instance,
-			instance->max_scsi_cmds + cmd_mfi->index);
+	cmd_fusion = megasas_get_cmd_fusion(instance, cmd_mfi->index);
 
 	/*  Save the smid. To be used for returning the cmd */
 	cmd_mfi->context.smid = cmd_fusion->index;
@@ -4601,7 +4595,7 @@ static u16 megasas_fusion_smid_lookup(struct scsi_cmnd *scmd)
 
 	fusion = instance->ctrl_context;
 
-	for (i = 0; i < instance->max_scsi_cmds; i++) {
+	for (i = instance->max_mfi_cmds; i < instance->max_fw_cmds; i++) {
 		cmd_fusion = fusion->cmd_list[i];
 		if (cmd_fusion->scmd && (cmd_fusion->scmd == scmd)) {
 			scmd_printk(KERN_NOTICE, scmd, "Abort request is for"
@@ -4918,7 +4912,7 @@ int megasas_reset_fusion(struct Scsi_Host *shost, int reason)
 			dev_info(&instance->pdev->dev, "\nPending SCSI commands:\n");
 
 		/* Now return commands back to the OS */
-		for (i = 0 ; i < instance->max_scsi_cmds; i++) {
+		for (i = instance->max_mfi_cmds; i < instance->max_fw_cmds; i++) {
 			cmd_fusion = fusion->cmd_list[i];
 			/*check for extra commands issued by driver*/
 			if (instance->adapter_type >= VENTURA_SERIES) {
drivers/scsi/megaraid/megaraid_sas_fusion.h +0 -5
index d57ecc7f88d8..7f0abc66dbb1 100644
@@ -1105,11 +1105,6 @@ struct megasas_cmd_fusion {
 	u8 retry_for_fw_reset;
 	union MEGASAS_REQUEST_DESCRIPTOR_UNION  *request_desc;
 
-	/*
-	 * Context for a MFI frame.
-	 * Used to get the mfi cmd from list when a MFI cmd is completed
-	 */
-	u32 sync_cmd_idx;
 	u32 index;
 	u8 pd_r1_lb;
 	struct completion done;
Signature · 1 line
2.16.4
#70 lore raw

RFC PATCH v3 27/41 megaraid_sas_fusion: sanitize command lookup

In reply to Hannes Reinecke

Use megasas_get_cmd_fusion() to lookup a command from a scsi command, and use direct array lookup whenever we already have an index.

---
 drivers/scsi/megaraid/megaraid_sas_fusion.c | 56 +++++++++--------------------
 1 file changed, 17 insertions(+), 39 deletions(-)
drivers/scsi/megaraid/megaraid_sas_fusion.c +17 -39
index 8e2ae44ab1f8..6c62c7f647f2 100644
@@ -231,14 +231,20 @@ megasas_clear_intr_fusion(struct megasas_instance *instance)
 /**
  * megasas_get_cmd_fusion -	Get a command from the free pool
  * @instance:		Adapter soft state
+ * @scmd:		SCSI command
+ * @is_raid:		true if the command is a RAID1 fastpath command
  *
  * Returns a blk_tag indexed mpt frame
  */
-inline struct megasas_cmd_fusion *megasas_get_cmd_fusion(struct megasas_instance
-						  *instance, u32 blk_tag)
+inline struct megasas_cmd_fusion *
+megasas_get_cmd_fusion(struct megasas_instance *instance,
+		       struct scsi_cmnd *scmd, bool is_raid)
 {
 	struct fusion_context *fusion;
+	u32 blk_tag = scmd->request->tag;
 
+	if (is_raid)
+		blk_tag += instance->max_fw_cmds;
 	fusion = instance->ctrl_context;
 	return fusion->cmd_list[blk_tag];
 }
@@ -3396,7 +3402,7 @@ megasas_build_and_issue_cmd_fusion(struct megasas_instance *instance,
 		return SCSI_MLQUEUE_HOST_BUSY;
 	}
 
-	cmd = megasas_get_cmd_fusion(instance, scmd->request->tag);
+	cmd = megasas_get_cmd_fusion(instance, scmd, false);
 
 	if (!cmd) {
 		atomic_dec(&instance->fw_outstanding);
@@ -3436,8 +3442,7 @@ megasas_build_and_issue_cmd_fusion(struct megasas_instance *instance,
 	 *	to get new command
 	 */
 	if (cmd->r1_alt_dev_handle != MR_DEVHANDLE_INVALID) {
-		r1_cmd = megasas_get_cmd_fusion(instance,
-				(scmd->request->tag + instance->max_fw_cmds));
+		r1_cmd = megasas_get_cmd_fusion(instance, scmd, true);
 		megasas_prepare_secondRaid1_IO(instance, cmd, r1_cmd);
 	}
 
@@ -3832,7 +3837,7 @@ build_mpt_mfi_pass_thru(struct megasas_instance *instance,
 
 	fusion = instance->ctrl_context;
 
-	cmd = megasas_get_cmd_fusion(instance, mfi_cmd->index);
+	cmd = fusion->cmd_list[mfi_cmd->index];
 
 	/*  Save the smid. To be used for returning the cmd */
 	mfi_cmd->context.smid = cmd->index;
@@ -4475,7 +4480,7 @@ megasas_issue_tm(struct megasas_instance *instance, u16 device_handle,
 		return -ENOMEM;
 	}
 
-	cmd_fusion = megasas_get_cmd_fusion(instance, cmd_mfi->index);
+	cmd_fusion = fusion->cmd_list[cmd_mfi->index];
 
 	/*  Save the smid. To be used for returning the cmd */
 	cmd_mfi->context.smid = cmd_fusion->index;
@@ -4578,36 +4583,6 @@ megasas_issue_tm(struct megasas_instance *instance, u16 device_handle,
 
 }
 
-/*
- * megasas_fusion_smid_lookup : Look for fusion command correpspodning to SCSI
- * @instance: per adapter struct
- *
- * Return Non Zero index, if SMID found in outstanding commands
- */
-static u16 megasas_fusion_smid_lookup(struct scsi_cmnd *scmd)
-{
-	int i, ret = 0;
-	struct megasas_instance *instance;
-	struct megasas_cmd_fusion *cmd_fusion;
-	struct fusion_context *fusion;
-
-	instance = (struct megasas_instance *)scmd->device->host->hostdata;
-
-	fusion = instance->ctrl_context;
-
-	for (i = instance->max_mfi_cmds; i < instance->max_fw_cmds; i++) {
-		cmd_fusion = fusion->cmd_list[i];
-		if (cmd_fusion->scmd && (cmd_fusion->scmd == scmd)) {
-			scmd_printk(KERN_NOTICE, scmd, "Abort request is for"
-				" SMID: %d\n", cmd_fusion->index);
-			ret = cmd_fusion->index;
-			break;
-		}
-	}
-
-	return ret;
-}
-
 /*
 * megasas_get_tm_devhandle - Get devhandle for TM request
 * @sdev-		     OS provided scsi device
@@ -4655,7 +4630,8 @@ static u16 megasas_get_tm_devhandle(struct scsi_device *sdev)
 int megasas_task_abort_fusion(struct scsi_cmnd *scmd)
 {
 	struct megasas_instance *instance;
-	u16 smid, devhandle;
+	struct megasas_cmd_fusion *cmd_fusion;
+	u16 smid = 0, devhandle;
 	int ret;
 	struct MR_PRIV_DEVICE *mr_device_priv_data;
 	mr_device_priv_data = scmd->device->hostdata;
@@ -4684,7 +4660,9 @@ int megasas_task_abort_fusion(struct scsi_cmnd *scmd)
 
 	mutex_lock(&instance->reset_mutex);
 
-	smid = megasas_fusion_smid_lookup(scmd);
+	cmd_fusion = megasas_get_cmd_fusion(instance, scmd, false);
+	if (cmd_fusion)
+		smid = cmd_fusion->index;
 
 	if (!smid) {
 		ret = SUCCESS;
Signature · 1 line
2.16.4
#71 lore raw

RFC PATCH v3 28/41 megaraid_sas: use scsi_host_busy_iter to traverse outstanding commands

In reply to Hannes Reinecke

As the block layer now accounts for all outstanding commands we can use scsi_host_busy_iter() to traverse all commands.

---
 drivers/scsi/megaraid/megaraid_sas.h        |   1 +
 drivers/scsi/megaraid/megaraid_sas_base.c   | 265 +++++++++++++---------
 drivers/scsi/megaraid/megaraid_sas_fusion.c | 329 +++++++++++++++-------------
 3 files changed, 334 insertions(+), 261 deletions(-)
drivers/scsi/megaraid/megaraid_sas.h +1 -0
index bb765e715011..2a089d2ba37b 100644
@@ -2406,6 +2406,7 @@ struct megasas_instance {
 	struct megasas_cmd *jbod_seq_cmd;
 	unsigned long bar;
 	long reset_flags;
+	unsigned int reset_count;
 	struct mutex reset_mutex;
 	struct timer_list sriov_heartbeat_timer;
 	char skip_heartbeat_timer_del;
drivers/scsi/megaraid/megaraid_sas_base.c +156 -109

312 lines not on this page · raw message

drivers/scsi/megaraid/megaraid_sas_fusion.c +177 -152

411 lines not on this page · raw message

Signature · 1 line
2.16.4
#72 lore raw

RFC PATCH v3 29/41 snic: use reserved commands

In reply to Hannes Reinecke

From: Hannes Reinecke <[email protected]>

Use a reserved command for host and device reset.

---
 drivers/scsi/snic/snic.h      |   4 +-
 drivers/scsi/snic/snic_main.c |   8 +++
 drivers/scsi/snic/snic_scsi.c | 140 +++++++++++++++++-------------------------
 3 files changed, 66 insertions(+), 86 deletions(-)
drivers/scsi/snic/snic.h +2 -2
index de0ab5fc8474..7dc529ae8a90 100644
@@ -59,7 +59,6 @@
  */
 #define SNIC_TAG_ABORT		BIT(30)		/* Tag indicating abort */
 #define SNIC_TAG_DEV_RST	BIT(29)		/* Tag for device reset */
-#define SNIC_TAG_IOCTL_DEV_RST	BIT(28)		/* Tag for User Device Reset */
 #define SNIC_TAG_MASK		(BIT(24) - 1)	/* Mask for lookup */
 #define SNIC_NO_TAG		-1
 
@@ -278,6 +277,7 @@ struct snic {
 
 	/* Scsi Host info */
 	struct Scsi_Host *shost;
+	struct scsi_device *shost_dev;
 
 	/* vnic related structures */
 	struct vnic_dev_bar bar0;
@@ -380,7 +380,7 @@ int snic_queuecommand(struct Scsi_Host *, struct scsi_cmnd *);
 int snic_abort_cmd(struct scsi_cmnd *);
 int snic_device_reset(struct scsi_cmnd *);
 int snic_host_reset(struct scsi_cmnd *);
-int snic_reset(struct Scsi_Host *, struct scsi_cmnd *);
+int snic_reset(struct Scsi_Host *);
 void snic_shutdown_scsi_cleanup(struct snic *);
 
 
drivers/scsi/snic/snic_main.c +8 -0
index 14f4ce665e58..f520da64ec8e 100644
@@ -303,6 +303,7 @@ static int
 snic_add_host(struct Scsi_Host *shost, struct pci_dev *pdev)
 {
 	int ret = 0;
+	struct snic *snic = shost_priv(shost);
 
 	ret = scsi_add_host(shost, &pdev->dev);
 	if (ret) {
@@ -313,6 +314,12 @@ snic_add_host(struct Scsi_Host *shost, struct pci_dev *pdev)
 		return ret;
 	}
 
+	snic->shost_dev = scsi_get_virtual_dev(shost, 1, 0);
+	if (!snic->shost_dev) {
+		SNIC_HOST_ERR(shost,
+			      "snic: scsi_get_virtual_dev failed\n");
+		return -ENOMEM;
+	}
 	SNIC_BUG_ON(shost->work_q != NULL);
 	snprintf(shost->work_q_name, sizeof(shost->work_q_name), "scsi_wq_%d",
 		 shost->host_no);
@@ -385,6 +392,7 @@ snic_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
 
 		goto prob_end;
 	}
+	shost->nr_reserved_cmds = 2;
 	snic = shost_priv(shost);
 	snic->shost = shost;
 
drivers/scsi/snic/snic_scsi.c +56 -84

315 lines not on this page · raw message

Signature · 1 line
2.16.4
#73 lore raw

Re: [PATCH RFC v3 29/41] snic: use reserved commands

Quoted 66 lines · Hannes Reinecke

From: Hannes Reinecke <[email protected]>

Use a reserved command for host and device reset.

---
 drivers/scsi/snic/snic.h      |   4 +-
 drivers/scsi/snic/snic_main.c |   8 +++
 drivers/scsi/snic/snic_scsi.c | 140 +++++++++++++++++-------------------------
 3 files changed, 66 insertions(+), 86 deletions(-)
drivers/scsi/snic/snic.h +2 -2
index de0ab5fc8474..7dc529ae8a90 100644
@@ -59,7 +59,6 @@
  */
 #define SNIC_TAG_ABORT		BIT(30)		/* Tag indicating abort */
 #define SNIC_TAG_DEV_RST	BIT(29)		/* Tag for device reset */
-#define SNIC_TAG_IOCTL_DEV_RST	BIT(28)		/* Tag for User Device Reset */
 #define SNIC_TAG_MASK		(BIT(24) - 1)	/* Mask for lookup */
 #define SNIC_NO_TAG		-1
 
@@ -278,6 +277,7 @@ struct snic {
 
 	/* Scsi Host info */
 	struct Scsi_Host *shost;
+	struct scsi_device *shost_dev;
 
 	/* vnic related structures */
 	struct vnic_dev_bar bar0;
@@ -380,7 +380,7 @@ int snic_queuecommand(struct Scsi_Host *, struct scsi_cmnd *);
 int snic_abort_cmd(struct scsi_cmnd *);
 int snic_device_reset(struct scsi_cmnd *);
 int snic_host_reset(struct scsi_cmnd *);
-int snic_reset(struct Scsi_Host *, struct scsi_cmnd *);
+int snic_reset(struct Scsi_Host *);
 void snic_shutdown_scsi_cleanup(struct snic *);
 
 
drivers/scsi/snic/snic_main.c +7 -0
index 14f4ce665e58..f520da64ec8e 100644
@@ -303,6 +303,7 @@ static int
 snic_add_host(struct Scsi_Host *shost, struct pci_dev *pdev)
 {
 	int ret = 0;
+	struct snic *snic = shost_priv(shost);
 
 	ret = scsi_add_host(shost, &pdev->dev);
 	if (ret) {
@@ -313,6 +314,12 @@ snic_add_host(struct Scsi_Host *shost, struct pci_dev *pdev)
 		return ret;
 	}
 
+	snic->shost_dev = scsi_get_virtual_dev(shost, 1, 0);
+	if (!snic->shost_dev) {
+		SNIC_HOST_ERR(shost,
+			      "snic: scsi_get_virtual_dev failed\n");
+		return -ENOMEM;
+	}
 	SNIC_BUG_ON(shost->work_q != NULL);
 	snprintf(shost->work_q_name, sizeof(shost->work_q_name), "scsi_wq_%d",
 		 shost->host_no);

drivers/scsi/snic/snic_main.c snic_probe(struct pci_dev *pdev, const struct pci_device_id *ent) · line 395

Quoted 4 lines
drivers/scsi/snic/snic_main.c +1 -0
@@ -385,6 +392,7 @@ snic_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
 
 		goto prob_end;
 	}
+	shost->nr_reserved_cmds = 2;

Not see .can_queue is increased by 2 in this patch, please comment on the reason. Otherwise, IO performance drop may be caused.

Thanks,
Ming

#74 lore raw

Re: [PATCH RFC v3 29/41] snic: use reserved commands

Quoted 68 lines · Ming Lei
Quoted 67 lines · Hannes Reinecke

From: Hannes Reinecke <[email protected]>

Use a reserved command for host and device reset.

---
  drivers/scsi/snic/snic.h      |   4 +-
  drivers/scsi/snic/snic_main.c |   8 +++
  drivers/scsi/snic/snic_scsi.c | 140 +++++++++++++++++-------------------------
  3 files changed, 66 insertions(+), 86 deletions(-)
drivers/scsi/snic/snic.h +2 -2
index de0ab5fc8474..7dc529ae8a90 100644
@@ -59,7 +59,6 @@
   */
  #define SNIC_TAG_ABORT		BIT(30)		/* Tag indicating abort */
  #define SNIC_TAG_DEV_RST	BIT(29)		/* Tag for device reset */
-#define SNIC_TAG_IOCTL_DEV_RST	BIT(28)		/* Tag for User Device Reset */
  #define SNIC_TAG_MASK		(BIT(24) - 1)	/* Mask for lookup */
  #define SNIC_NO_TAG		-1
 
@@ -278,6 +277,7 @@ struct snic {
 
  	/* Scsi Host info */
  	struct Scsi_Host *shost;
+	struct scsi_device *shost_dev;
 
  	/* vnic related structures */
  	struct vnic_dev_bar bar0;
@@ -380,7 +380,7 @@ int snic_queuecommand(struct Scsi_Host *, struct scsi_cmnd *);
  int snic_abort_cmd(struct scsi_cmnd *);
  int snic_device_reset(struct scsi_cmnd *);
  int snic_host_reset(struct scsi_cmnd *);
-int snic_reset(struct Scsi_Host *, struct scsi_cmnd *);
+int snic_reset(struct Scsi_Host *);
  void snic_shutdown_scsi_cleanup(struct snic *);
 
 
drivers/scsi/snic/snic_main.c +7 -0
index 14f4ce665e58..f520da64ec8e 100644
@@ -303,6 +303,7 @@ static int
  snic_add_host(struct Scsi_Host *shost, struct pci_dev *pdev)
  {
  	int ret = 0;
+	struct snic *snic = shost_priv(shost);
 
  	ret = scsi_add_host(shost, &pdev->dev);
  	if (ret) {
@@ -313,6 +314,12 @@ snic_add_host(struct Scsi_Host *shost, struct pci_dev *pdev)
  		return ret;
  	}
 
+	snic->shost_dev = scsi_get_virtual_dev(shost, 1, 0);
+	if (!snic->shost_dev) {
+		SNIC_HOST_ERR(shost,
+			      "snic: scsi_get_virtual_dev failed\n");
+		return -ENOMEM;
+	}
  	SNIC_BUG_ON(shost->work_q != NULL);
  	snprintf(shost->work_q_name, sizeof(shost->work_q_name), "scsi_wq_%d",
  		 shost->host_no);
@@ -385,6 +392,7 @@ snic_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
 
Quoted 7 lines

drivers/scsi/snic/snic_main.c snic_probe(struct pci_dev *pdev, const struct pci_device_id *ent) · line 395

Quoted 3 lines
drivers/scsi/snic/snic_main.c +1 -0
  		goto prob_end;
  	}
+	shost->nr_reserved_cmds = 2;

Not see .can_queue is increased by 2 in this patch, please comment on the reason. Otherwise, IO performance drop may be caused.

snic is requiring a tag for both LU reset and Host reset.
We do require one tag for SCSI EH, and I'm setting aside another one for ioctl commands. Will be adding a comment here explaining things.

Cheers,

Hannes

Signature · 4 lines
Dr. Hannes Reinecke            Teamlead Storage & Networking
[email protected]                               +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer
#75 lore raw

RFC PATCH v3 30/41 snic: use tagset iter for traversing commands

In reply to Hannes Reinecke

From: Hannes Reinecke <[email protected]>

Use the tagset iter to traverse active commands during device and hba reset.

---
 drivers/scsi/snic/snic_scsi.c | 246 +++++++++++++++++++++---------------------
 1 file changed, 124 insertions(+), 122 deletions(-)
drivers/scsi/snic/snic_scsi.c +124 -122
index d42e178bfab3..01bec9d773ef 100644
@@ -1648,84 +1648,87 @@ snic_abort_cmd(struct scsi_cmnd *sc)
 	return ret;
 }
 
+struct snic_cmd_pending_iter_data {
+	struct snic *snic;
+	struct scsi_device *sdev;
+	int ret;
+};
 
-
-static int
-snic_is_abts_pending(struct snic *snic, struct scsi_cmnd *lr_sc)
+static bool
+snic_is_abts_pending_iter(struct scsi_cmnd *sc, void *data, bool reserved)
 {
+	struct snic_cmd_pending_iter_data *iter_data = data;
 	struct snic_req_info *rqi = NULL;
-	struct scsi_cmnd *sc = NULL;
-	struct scsi_device *lr_sdev = NULL;
 	spinlock_t *io_lock = NULL;
-	u32 tag;
 	unsigned long flags;
 
-	if (lr_sc)
-		lr_sdev = lr_sc->device;
+	if (reserved)
+		return true;
 
-	/* walk through the tag map, an dcheck if IOs are still pending in fw*/
-	for (tag = 0; tag < snic->max_tag_id; tag++) {
-		io_lock = snic_io_lock_tag(snic, tag);
+	if (iter_data->sdev && iter_data->sdev != sc->device)
+		return true;
 
-		spin_lock_irqsave(io_lock, flags);
-		sc = scsi_host_find_tag(snic->shost, tag);
-
-		if (!sc || (lr_sc && (sc->device != lr_sdev || sc == lr_sc))) {
-			spin_unlock_irqrestore(io_lock, flags);
-
-			continue;
-		}
+	io_lock = snic_io_lock_tag(iter_data->snic, sc->request->tag);
+	spin_lock_irqsave(io_lock, flags);
 
-		rqi = (struct snic_req_info *) CMD_SP(sc);
-		if (!rqi) {
-			spin_unlock_irqrestore(io_lock, flags);
+	rqi = (struct snic_req_info *) CMD_SP(sc);
+	if (!rqi) {
+		spin_unlock_irqrestore(io_lock, flags);
+		return true;
+	}
 
-			continue;
-		}
+	/*
+	 * Found IO that is still pending w/ firmware and belongs to
+	 * the LUN that is under reset, if lr_sc != NULL
+	 */
+	SNIC_SCSI_DBG(iter_data->snic->shost, "Found IO in %s on LUN\n",
+		      snic_ioreq_state_to_str(CMD_STATE(sc)));
 
-		/*
-		 * Found IO that is still pending w/ firmware and belongs to
-		 * the LUN that is under reset, if lr_sc != NULL
-		 */
-		SNIC_SCSI_DBG(snic->shost, "Found IO in %s on LUN\n",
-			      snic_ioreq_state_to_str(CMD_STATE(sc)));
+	if (CMD_STATE(sc) == SNIC_IOREQ_ABTS_PENDING)
+		iter_data->ret = 1;
+	spin_unlock_irqrestore(io_lock, flags);
 
-		if (CMD_STATE(sc) == SNIC_IOREQ_ABTS_PENDING) {
-			spin_unlock_irqrestore(io_lock, flags);
+	return true;
+}
 
-			return 1;
-		}
+static int
+snic_is_abts_pending(struct snic *snic, struct scsi_device *lr_sdev)
+{
+	struct snic_cmd_pending_iter_data iter_data = {
+		.snic = snic,
+		.sdev = lr_sdev,
+		.ret = 0,
+	};
 
-		spin_unlock_irqrestore(io_lock, flags);
-	}
+	/* walk through the tag map, an dcheck if IOs are still pending in fw*/
+	scsi_host_busy_iter(snic->shost,
+			    snic_is_abts_pending_iter, &iter_data);
 
-	return 0;
+	return iter_data.ret;
 } /* end of snic_is_abts_pending */
 
-static int
-snic_dr_clean_single_req(struct snic *snic,
-			 u32 tag,
-			 struct scsi_device *lr_sdev)
+static bool
+snic_dr_clean_single_req(struct scsi_cmnd *sc, void *data, bool reserved)
 {
 	struct snic_req_info *rqi = NULL;
 	struct snic_tgt *tgt = NULL;
-	struct scsi_cmnd *sc = NULL;
 	spinlock_t *io_lock = NULL;
 	u32 sv_state = 0, tmf = 0;
 	DECLARE_COMPLETION_ONSTACK(tm_done);
 	unsigned long flags;
 	int ret = 0;
+	struct snic_cmd_pending_iter_data *iter_data = data;
+	struct snic *snic = iter_data->snic;
 
-	io_lock = snic_io_lock_tag(snic, tag);
-	spin_lock_irqsave(io_lock, flags);
-	sc = scsi_host_find_tag(snic->shost, tag);
+	if (reserved)
+		return true;
 
-	/* Ignore Cmd that don't belong to Lun Reset device */
-	if (!sc || sc->device != lr_sdev)
-		goto skip_clean;
+	if (sc->device != iter_data->sdev)
+		return true;
 
+	io_lock = snic_io_lock_tag(snic, sc->request->tag);
+	spin_lock_irqsave(io_lock, flags);
 	rqi = (struct snic_req_info *) CMD_SP(sc);
-
 	if (!rqi)
 		goto skip_clean;
 
@@ -1784,7 +1787,7 @@ snic_dr_clean_single_req(struct snic *snic,
 	if (ret) {
 		SNIC_HOST_ERR(snic->shost,
 			      "clean_single_req_err:sc %p, tag %d abt failed. tm_tag %d flags 0x%llx\n",
-			      sc, tag, rqi->tm_tag, CMD_FLAGS(sc));
+			      sc, sc->request->tag, rqi->tm_tag, CMD_FLAGS(sc));
 
 		spin_lock_irqsave(io_lock, flags);
 		rqi = (struct snic_req_info *) CMD_SP(sc);
@@ -1795,7 +1798,7 @@ snic_dr_clean_single_req(struct snic *snic,
 		if (CMD_STATE(sc) == SNIC_IOREQ_ABTS_PENDING)
 			CMD_STATE(sc) = sv_state;
 
-		ret = 1;
+		iter_data->ret = 1;
 		goto skip_clean;
 	}
 
@@ -1821,56 +1824,49 @@ snic_dr_clean_single_req(struct snic *snic,
 	if (CMD_ABTS_STATUS(sc) == SNIC_INVALID_CODE) {
 		SNIC_HOST_ERR(snic->shost,
 			      "clean_single_req_err:sc %p tag %d abt still pending w/ fw, tm_tag %d flags 0x%llx\n",
-			      sc, tag, rqi->tm_tag, CMD_FLAGS(sc));
+			      sc, sc->request->tag, rqi->tm_tag, CMD_FLAGS(sc));
 
 		CMD_FLAGS(sc) |= SNIC_IO_ABTS_TERM_DONE;
-		ret = 1;
-
-		goto skip_clean;
-	}
-
-	CMD_STATE(sc) = SNIC_IOREQ_ABTS_COMPLETE;
-	CMD_SP(sc) = NULL;
-	spin_unlock_irqrestore(io_lock, flags);
-
-	snic_release_req_buf(snic, rqi, sc);
-
-	sc->result = (DID_ERROR << 16);
-	sc->scsi_done(sc);
+		iter_data->ret = 1;
+	} else {
+		CMD_STATE(sc) = SNIC_IOREQ_ABTS_COMPLETE;
+		CMD_SP(sc) = NULL;
+		spin_unlock_irqrestore(io_lock, flags);
 
-	ret = 0;
+		snic_release_req_buf(snic, rqi, sc);
 
-	return ret;
+		sc->result = (DID_ERROR << 16);
+		sc->scsi_done(sc);
+	}
+	return true;
 
 skip_clean:
 	spin_unlock_irqrestore(io_lock, flags);
-
-	return ret;
+	return true;
 } /* end of snic_dr_clean_single_req */
 
 static int
-snic_dr_clean_pending_req(struct snic *snic, struct scsi_cmnd *lr_sc)
+snic_dr_clean_pending_req(struct snic *snic, struct scsi_device *lr_sdev)
 {
-	struct scsi_device *lr_sdev = lr_sc->device;
-	u32 tag = 0;
 	int ret = FAILED;
+	struct snic_cmd_pending_iter_data iter_data = {
+		.snic = snic,
+		.sdev = lr_sdev,
+		.ret = 0,
+	};
 
-	for (tag = 0; tag < snic->max_tag_id; tag++) {
-		if (tag == snic_cmd_tag(lr_sc))
-			continue;
-
-		ret = snic_dr_clean_single_req(snic, tag, lr_sdev);
-		if (ret) {
-			SNIC_HOST_ERR(snic->shost, "clean_err:tag = %d\n", tag);
+	scsi_host_busy_iter(snic->shost,
+			    snic_dr_clean_single_req, &iter_data);
+	if (iter_data.ret) {
+		SNIC_HOST_ERR(snic->shost, "clean_err = %d\n", iter_data.ret);
 
-			goto clean_err;
-		}
+		goto clean_err;
 	}
 
 	schedule_timeout(msecs_to_jiffies(100));
 
 	/* Walk through all the cmds and check abts status. */
-	if (snic_is_abts_pending(snic, lr_sc)) {
+	if (snic_is_abts_pending(snic, lr_sdev)) {
 		ret = FAILED;
 
 		goto clean_err;
@@ -1957,7 +1953,7 @@ snic_dr_finish(struct snic *snic, struct scsi_cmnd *sc)
 	 * succeeds.
 	 */
 
-	ret = snic_dr_clean_pending_req(snic, sc);
+	ret = snic_dr_clean_pending_req(snic, sc->device);
 	if (ret) {
 		spin_lock_irqsave(io_lock, flags);
 		SNIC_SCSI_DBG(snic->shost,
@@ -2581,6 +2577,40 @@ snic_internal_abort_io(struct snic *snic, struct scsi_cmnd *sc, int tmf)
 	return ret;
 } /* end of snic_internal_abort_io */
 
+struct snic_tgt_scsi_abort_io_iter_data {
+	struct snic *snic;
+	struct snic_tgt *tgt;
+	int tmf;
+	int abt_cnt;
+};
+
+static bool
+snic_tgt_scsi_abort_io_iter(struct scsi_cmnd *sc, void *data, bool reserved)
+{
+	struct snic_tgt_scsi_abort_io_iter_data *iter_data = data;
+	struct snic_tgt *sc_tgt = NULL;
+	int ret;
+
+	if (reserved)
+		return true;
+
+	sc_tgt = starget_to_tgt(scsi_target(sc->device));
+	if (sc_tgt != iter_data->tgt)
+		return true;
+
+	ret = snic_internal_abort_io(iter_data->snic, sc, iter_data->tmf);
+	if (ret < 0) {
+		SNIC_HOST_ERR(iter_data->snic->shost,
+			      "tgt_abt_io: Tag %x, Failed w err = %d\n",
+			      snic_cmd_tag(sc), ret);
+		return true;
+	}
+
+	if (ret == SUCCESS)
+		iter_data->abt_cnt++;
+	return true;
+}
+
 /*
  * snic_tgt_scsi_abort_io : called by snic_tgt_del
  */
@@ -2588,11 +2618,10 @@ int
 snic_tgt_scsi_abort_io(struct snic_tgt *tgt)
 {
 	struct snic *snic = NULL;
-	struct scsi_cmnd *sc = NULL;
-	struct snic_tgt *sc_tgt = NULL;
-	spinlock_t *io_lock = NULL;
-	unsigned long flags;
-	int ret = 0, tag, abt_cnt = 0, tmf = 0;
+	struct snic_tgt_scsi_abort_io_iter_data iter_data = {
+		.tgt = tgt,
+		.abt_cnt = 0,
+	};
 
 	if (!tgt)
 		return -1;
@@ -2601,43 +2630,16 @@ snic_tgt_scsi_abort_io(struct snic_tgt *tgt)
 	SNIC_SCSI_DBG(snic->shost, "tgt_abt_io: Cleaning Pending IOs.\n");
 
 	if (tgt->tdata.typ == SNIC_TGT_DAS)
-		tmf = SNIC_ITMF_ABTS_TASK;
+		iter_data.tmf = SNIC_ITMF_ABTS_TASK;
 	else
-		tmf = SNIC_ITMF_ABTS_TASK_TERM;
-
-	for (tag = 0; tag < snic->max_tag_id; tag++) {
-		io_lock = snic_io_lock_tag(snic, tag);
-
-		spin_lock_irqsave(io_lock, flags);
-		sc = scsi_host_find_tag(snic->shost, tag);
-		if (!sc) {
-			spin_unlock_irqrestore(io_lock, flags);
+		iter_data.tmf = SNIC_ITMF_ABTS_TASK_TERM;
+	iter_data.snic = snic;
 
-			continue;
-		}
-
-		sc_tgt = starget_to_tgt(scsi_target(sc->device));
-		if (sc_tgt != tgt) {
-			spin_unlock_irqrestore(io_lock, flags);
-
-			continue;
-		}
-		spin_unlock_irqrestore(io_lock, flags);
-
-		ret = snic_internal_abort_io(snic, sc, tmf);
-		if (ret < 0) {
-			SNIC_HOST_ERR(snic->shost,
-				      "tgt_abt_io: Tag %x, Failed w err = %d\n",
-				      tag, ret);
-
-			continue;
-		}
-
-		if (ret == SUCCESS)
-			abt_cnt++;
-	}
+	scsi_host_busy_iter(snic->shost,
+			    snic_tgt_scsi_abort_io_iter, &iter_data);
 
-	SNIC_SCSI_DBG(snic->shost, "tgt_abt_io: abt_cnt = %d\n", abt_cnt);
+	SNIC_SCSI_DBG(snic->shost, "tgt_abt_io: abt_cnt = %d\n",
+		      iter_data.abt_cnt);
 
 	return 0;
 } /* end of snic_tgt_scsi_abort_io */
Signature · 1 line
2.16.4
#76 lore raw

RFC PATCH v3 31/41 mv_sas: kill mvsas_debug_issue_ssp_tmf()

In reply to Hannes Reinecke

Just a wrapper around mvs_exec_internal_tmf_task(), so call it directly and kill the wrapper.

---
 drivers/scsi/mvsas/mv_sas.c | 33 +++++++++++----------------------
 1 file changed, 11 insertions(+), 22 deletions(-)
drivers/scsi/mvsas/mv_sas.c +11 -22
index a920eced92ec..937c27777ab9 100644
@@ -1277,11 +1277,14 @@ static void mvs_tmf_timedout(struct timer_list *t)
 
 #define MVS_TASK_TIMEOUT 20
 static int mvs_exec_internal_tmf_task(struct domain_device *dev,
-			void *parameter, u32 para_len, struct mvs_tmf_task *tmf)
+				      u8 *lun, struct mvs_tmf_task *tmf)
 {
 	int res, retry;
 	struct sas_task *task = NULL;
 
+	if (!(dev->tproto & SAS_PROTOCOL_SSP))
+		return TMF_RESP_FUNC_ESUPP;
+
 	for (retry = 0; retry < 3; retry++) {
 		task = sas_alloc_slow_task(GFP_KERNEL);
 		if (!task)
@@ -1290,7 +1293,7 @@ static int mvs_exec_internal_tmf_task(struct domain_device *dev,
 		task->dev = dev;
 		task->task_proto = dev->tproto;
 
-		memcpy(&task->ssp_task, parameter, para_len);
+		memcpy(task->ssp_task.LUN, lun, 8);
 		task->task_done = mvs_task_done;
 
 		task->slow_task->timer.function = mvs_tmf_timedout;
@@ -1351,20 +1354,6 @@ static int mvs_exec_internal_tmf_task(struct domain_device *dev,
 	return res;
 }
 
-static int mvs_debug_issue_ssp_tmf(struct domain_device *dev,
-				u8 *lun, struct mvs_tmf_task *tmf)
-{
-	struct sas_ssp_task ssp_task;
-	if (!(dev->tproto & SAS_PROTOCOL_SSP))
-		return TMF_RESP_FUNC_ESUPP;
-
-	memcpy(ssp_task.LUN, lun, 8);
-
-	return mvs_exec_internal_tmf_task(dev, &ssp_task,
-				sizeof(ssp_task), tmf);
-}
-
-
 /*  Standard mandates link reset for ATA  (type 0)
     and hard reset for SSP (type 1) , only for RECOVERY */
 static int mvs_debug_I_T_nexus_reset(struct domain_device *dev)
@@ -1390,7 +1379,7 @@ int mvs_lu_reset(struct domain_device *dev, u8 *lun)
 
 	tmf_task.tmf = TMF_LU_RESET;
 	mvi_dev->dev_status = MVS_DEV_EH;
-	rc = mvs_debug_issue_ssp_tmf(dev, lun, &tmf_task);
+	rc = mvs_exec_internal_tmf_task(dev, lun, &tmf_task);
 	if (rc == TMF_RESP_FUNC_COMPLETE) {
 		spin_lock_irqsave(&mvi->lock, flags);
 		mvs_release_task(mvi, dev);
@@ -1447,7 +1436,7 @@ int mvs_query_task(struct sas_task *task)
 		tmf_task.tmf = TMF_QUERY_TASK;
 		tmf_task.tag_of_task_to_be_managed = cpu_to_le16(tag);
 
-		rc = mvs_debug_issue_ssp_tmf(dev, lun.scsi_lun, &tmf_task);
+		rc = mvs_exec_internal_tmf_task(dev, lun.scsi_lun, &tmf_task);
 		switch (rc) {
 		/* The task is still in Lun, release it then */
 		case TMF_RESP_FUNC_SUCC:
@@ -1502,7 +1491,7 @@ int mvs_abort_task(struct sas_task *task)
 		tmf_task.tmf = TMF_ABORT_TASK;
 		tmf_task.tag_of_task_to_be_managed = cpu_to_le16(tag);
 
-		rc = mvs_debug_issue_ssp_tmf(dev, lun.scsi_lun, &tmf_task);
+		rc = mvs_exec_internal_tmf_task(dev, lun.scsi_lun, &tmf_task);
 
 		/* if successful, clear the task and callback forwards.*/
 		if (rc == TMF_RESP_FUNC_COMPLETE) {
@@ -1545,7 +1534,7 @@ int mvs_abort_task_set(struct domain_device *dev, u8 *lun)
 	struct mvs_tmf_task tmf_task;
 
 	tmf_task.tmf = TMF_ABORT_TASK_SET;
-	rc = mvs_debug_issue_ssp_tmf(dev, lun, &tmf_task);
+	rc = mvs_exec_internal_tmf_task(dev, lun, &tmf_task);
 
 	return rc;
 }
@@ -1556,7 +1545,7 @@ int mvs_clear_aca(struct domain_device *dev, u8 *lun)
 	struct mvs_tmf_task tmf_task;
 
 	tmf_task.tmf = TMF_CLEAR_ACA;
-	rc = mvs_debug_issue_ssp_tmf(dev, lun, &tmf_task);
+	rc = mvs_exec_internal_tmf_task(dev, lun, &tmf_task);
 
 	return rc;
 }
@@ -1567,7 +1556,7 @@ int mvs_clear_task_set(struct domain_device *dev, u8 *lun)
 	struct mvs_tmf_task tmf_task;
 
 	tmf_task.tmf = TMF_CLEAR_TASK_SET;
-	rc = mvs_debug_issue_ssp_tmf(dev, lun, &tmf_task);
+	rc = mvs_exec_internal_tmf_task(dev, lun, &tmf_task);
 
 	return rc;
 }
Signature · 1 line
2.16.4
#77 lore raw

RFC PATCH v3 32/41 pm8001: kill pm8001_issue_ssp_tmf()

In reply to Hannes Reinecke

Just a wrapper around pm8001_exec_internal_tmf_task(), so call it directly and kill the wrapper.

---
 drivers/scsi/pm8001/pm8001_sas.c | 42 ++++++++++++++++------------------------
 1 file changed, 17 insertions(+), 25 deletions(-)
drivers/scsi/pm8001/pm8001_sas.c +17 -25
index b7cbc312843e..eb645e4a5940 100644
@@ -700,16 +700,15 @@ static void pm8001_tmf_timedout(struct timer_list *t)
 /**
   * pm8001_exec_internal_tmf_task - execute some task management commands.
   * @dev: the wanted device.
+  * @lun: the logical unit number of the device
   * @tmf: which task management wanted to be take.
-  * @para_len: para_len.
-  * @parameter: ssp task parameter.
   *
   * when errors or exception happened, we may want to do something, for example
   * abort the issued task which result in this execption, it is done by calling
   * this function, note it is also with the task execute interface.
   */
 static int pm8001_exec_internal_tmf_task(struct domain_device *dev,
-	void *parameter, u32 para_len, struct pm8001_tmf_task *tmf)
+	u8 *lun, struct pm8001_tmf_task *tmf)
 {
 	int res, retry;
 	struct sas_task *task = NULL;
@@ -717,6 +716,9 @@ static int pm8001_exec_internal_tmf_task(struct domain_device *dev,
 	struct pm8001_device *pm8001_dev = dev->lldd_dev;
 	DECLARE_COMPLETION_ONSTACK(completion_setstate);
 
+	if (!(dev->tproto & SAS_PROTOCOL_SSP))
+		return TMF_RESP_FUNC_ESUPP;
+
 	for (retry = 0; retry < 3; retry++) {
 		task = sas_alloc_slow_task(GFP_KERNEL);
 		if (!task)
@@ -724,7 +726,7 @@ static int pm8001_exec_internal_tmf_task(struct domain_device *dev,
 
 		task->dev = dev;
 		task->task_proto = dev->tproto;
-		memcpy(&task->ssp_task, parameter, para_len);
+		memcpy(task->ssp_task.LUN, lun, 8);
 		task->task_done = pm8001_task_done;
 		task->slow_task->timer.function = pm8001_tmf_timedout;
 		task->slow_task->timer.expires = jiffies + PM8001_TASK_TIMEOUT*HZ;
@@ -909,18 +911,6 @@ void pm8001_dev_gone(struct domain_device *dev)
 	pm8001_dev_gone_notify(dev);
 }
 
-static int pm8001_issue_ssp_tmf(struct domain_device *dev,
-	u8 *lun, struct pm8001_tmf_task *tmf)
-{
-	struct sas_ssp_task ssp_task;
-	if (!(dev->tproto & SAS_PROTOCOL_SSP))
-		return TMF_RESP_FUNC_ESUPP;
-
-	strncpy((u8 *)&ssp_task.LUN, lun, 8);
-	return pm8001_exec_internal_tmf_task(dev, &ssp_task, sizeof(ssp_task),
-		tmf);
-}
-
 /* retry commands by ha, by task and/or by device */
 void pm8001_open_reject_retry(
 	struct pm8001_hba_info *pm8001_ha,
@@ -1127,7 +1117,7 @@ int pm8001_lu_reset(struct domain_device *dev, u8 *lun)
 		wait_for_completion(&completion_setstate);
 	} else {
 		tmf_task.tmf = TMF_LU_RESET;
-		rc = pm8001_issue_ssp_tmf(dev, lun, &tmf_task);
+		rc = pm8001_exec_internal_tmf_task(dev, lun, &tmf_task);
 	}
 	/* If failed, fall-through I_T_Nexus reset */
 	PM8001_EH_DBG(pm8001_ha, pm8001_printk("for device[%x]:rc=%d\n",
@@ -1162,10 +1152,10 @@ int pm8001_query_task(struct sas_task *task)
 		for (i = 0; i < 16; i++)
 			printk(KERN_INFO "%02x ", cmnd->cmnd[i]);
 		printk(KERN_INFO "]\n");
-		tmf_task.tmf = 	TMF_QUERY_TASK;
+		tmf_task.tmf = TMF_QUERY_TASK;
 		tmf_task.tag_of_task_to_be_managed = tag;
 
-		rc = pm8001_issue_ssp_tmf(dev, lun.scsi_lun, &tmf_task);
+		rc = pm8001_exec_internal_tmf_task(dev, lun.scsi_lun, &tmf_task);
 		switch (rc) {
 		/* The task is still in Lun, release it then */
 		case TMF_RESP_FUNC_SUCC:
@@ -1225,9 +1215,11 @@ int pm8001_abort_task(struct sas_task *task)
 		int_to_scsilun(cmnd->device->lun, &lun);
 		tmf_task.tmf = TMF_ABORT_TASK;
 		tmf_task.tag_of_task_to_be_managed = tag;
-		rc = pm8001_issue_ssp_tmf(dev, lun.scsi_lun, &tmf_task);
-		pm8001_exec_internal_task_abort(pm8001_ha, pm8001_dev,
-			pm8001_dev->sas_device, 0, tag);
+		rc = pm8001_exec_internal_tmf_task(dev, lun.scsi_lun,
+						   &tmf_task);
+		if (rc == TMF_RESP_FUNC_SUCC)
+			pm8001_exec_internal_task_abort(pm8001_ha, pm8001_dev,
+				pm8001_dev->sas_device, 0, tag);
 	} else if (task->task_proto & SAS_PROTOCOL_SATA ||
 		task->task_proto & SAS_PROTOCOL_STP) {
 		if (pm8001_ha->chip_id == chip_8006) {
@@ -1337,7 +1329,7 @@ int pm8001_abort_task_set(struct domain_device *dev, u8 *lun)
 	struct pm8001_tmf_task tmf_task;
 
 	tmf_task.tmf = TMF_ABORT_TASK_SET;
-	return pm8001_issue_ssp_tmf(dev, lun, &tmf_task);
+	return pm8001_exec_internal_tmf_task(dev, lun, &tmf_task);
 }
 
 int pm8001_clear_aca(struct domain_device *dev, u8 *lun)
@@ -1345,7 +1337,7 @@ int pm8001_clear_aca(struct domain_device *dev, u8 *lun)
 	struct pm8001_tmf_task tmf_task;
 
 	tmf_task.tmf = TMF_CLEAR_ACA;
-	return pm8001_issue_ssp_tmf(dev, lun, &tmf_task);
+	return pm8001_exec_internal_tmf_task(dev, lun, &tmf_task);
 }
 
 int pm8001_clear_task_set(struct domain_device *dev, u8 *lun)
@@ -1358,6 +1350,6 @@ int pm8001_clear_task_set(struct domain_device *dev, u8 *lun)
 		pm8001_printk("I_T_L_Q clear task set[%x]\n",
 		pm8001_dev->device_id));
 	tmf_task.tmf = TMF_CLEAR_TASK_SET;
-	return pm8001_issue_ssp_tmf(dev, lun, &tmf_task);
+	return pm8001_exec_internal_tmf_task(dev, lun, &tmf_task);
 }
 
Signature · 1 line
2.16.4
#78 lore raw

RFC PATCH v3 33/41 pm8001: kill 'dev' argument from pm8001_exec_internal_task_abort()

In reply to Hannes Reinecke

'dev' is always pm8001_dev->sas_device, so we can kill it.

---
 drivers/scsi/pm8001/pm8001_sas.c | 31 ++++++++++++++++---------------
 1 file changed, 16 insertions(+), 15 deletions(-)
drivers/scsi/pm8001/pm8001_sas.c +16 -15
index eb645e4a5940..ae25a8b62fcc 100644
@@ -798,9 +798,10 @@ static int pm8001_exec_internal_tmf_task(struct domain_device *dev,
 
 static int
 pm8001_exec_internal_task_abort(struct pm8001_hba_info *pm8001_ha,
-	struct pm8001_device *pm8001_dev, struct domain_device *dev, u32 flag,
+	struct pm8001_device *pm8001_dev, u32 flag,
 	u32 task_tag)
 {
+	struct domain_device *dev = pm8001_dev->sas_device;
 	int res, retry;
 	u32 ccb_tag;
 	struct pm8001_ccb_info *ccb;
@@ -890,8 +891,8 @@ static void pm8001_dev_gone_notify(struct domain_device *dev)
 			pm8001_dev->device_id, pm8001_dev->dev_type));
 		if (pm8001_dev->running_req) {
 			spin_unlock_irqrestore(&pm8001_ha->lock, flags);
-			pm8001_exec_internal_task_abort(pm8001_ha, pm8001_dev ,
-				dev, 1, 0);
+			pm8001_exec_internal_task_abort(pm8001_ha,
+				pm8001_dev, 1, 0);
 			while (pm8001_dev->running_req)
 				msleep(20);
 			spin_lock_irqsave(&pm8001_ha->lock, flags);
@@ -1014,8 +1015,8 @@ int pm8001_I_T_nexus_reset(struct domain_device *dev)
 			goto out;
 		}
 		msleep(2000);
-		rc = pm8001_exec_internal_task_abort(pm8001_ha, pm8001_dev ,
-			dev, 1, 0);
+		rc = pm8001_exec_internal_task_abort(pm8001_ha,
+			pm8001_dev, 1, 0);
 		if (rc) {
 			PM8001_EH_DBG(pm8001_ha,
 			pm8001_printk("task abort failed %x\n"
@@ -1062,8 +1063,8 @@ int pm8001_I_T_nexus_event_handler(struct domain_device *dev)
 			goto out;
 		}
 		/* send internal ssp/sata/smp abort command to FW */
-		rc = pm8001_exec_internal_task_abort(pm8001_ha, pm8001_dev ,
-							dev, 1, 0);
+		rc = pm8001_exec_internal_task_abort(pm8001_ha, pm8001_dev,
+						     1, 0);
 		msleep(100);
 
 		/* deregister the target device */
@@ -1078,8 +1079,8 @@ int pm8001_I_T_nexus_event_handler(struct domain_device *dev)
 		wait_for_completion(&completion_setstate);
 	} else {
 		/* send internal ssp/sata/smp abort command to FW */
-		rc = pm8001_exec_internal_task_abort(pm8001_ha, pm8001_dev ,
-							dev, 1, 0);
+		rc = pm8001_exec_internal_task_abort(pm8001_ha, pm8001_dev,
+						     1, 0);
 		msleep(100);
 
 		/* deregister the target device */
@@ -1107,8 +1108,8 @@ int pm8001_lu_reset(struct domain_device *dev, u8 *lun)
 	DECLARE_COMPLETION_ONSTACK(completion_setstate);
 	if (dev_is_sata(dev)) {
 		struct sas_phy *phy = sas_get_local_phy(dev);
-		rc = pm8001_exec_internal_task_abort(pm8001_ha, pm8001_dev ,
-			dev, 1, 0);
+		rc = pm8001_exec_internal_task_abort(pm8001_ha, pm8001_dev,
+			1, 0);
 		rc = sas_phy_reset(phy, 1);
 		sas_put_local_phy(phy);
 		pm8001_dev->setds_completion = &completion_setstate;
@@ -1219,7 +1220,7 @@ int pm8001_abort_task(struct sas_task *task)
 						   &tmf_task);
 		if (rc == TMF_RESP_FUNC_SUCC)
 			pm8001_exec_internal_task_abort(pm8001_ha, pm8001_dev,
-				pm8001_dev->sas_device, 0, tag);
+				0, tag);
 	} else if (task->task_proto & SAS_PROTOCOL_SATA ||
 		task->task_proto & SAS_PROTOCOL_STP) {
 		if (pm8001_ha->chip_id == chip_8006) {
@@ -1288,7 +1289,7 @@ int pm8001_abort_task(struct sas_task *task)
 			 * going to free the task.
 			 */
 			ret = pm8001_exec_internal_task_abort(pm8001_ha,
-				pm8001_dev, pm8001_dev->sas_device, 1, tag);
+				pm8001_dev, 1, tag);
 			if (ret)
 				goto out;
 			ret = wait_for_completion_timeout(
@@ -1305,13 +1306,13 @@ int pm8001_abort_task(struct sas_task *task)
 			wait_for_completion(&completion);
 		} else {
 			rc = pm8001_exec_internal_task_abort(pm8001_ha,
-				pm8001_dev, pm8001_dev->sas_device, 0, tag);
+				pm8001_dev, 0, tag);
 		}
 		rc = TMF_RESP_FUNC_COMPLETE;
 	} else if (task->task_proto & SAS_PROTOCOL_SMP) {
 		/* SMP */
 		rc = pm8001_exec_internal_task_abort(pm8001_ha, pm8001_dev,
-			pm8001_dev->sas_device, 0, tag);
+			0, tag);
 
 	}
 out:
Signature · 1 line
2.16.4
#79 lore raw

RFC PATCH v3 34/41 pm8001: use libsas-provided domain devices for SATA

In reply to Hannes Reinecke

The pm8001 driver assumes that libsas does not provide any domain devices for SATA, which is actually not true.
So use the libsas-provided domain devices for SATA and don't allocate private ones.

---
 drivers/scsi/pm8001/pm8001_hwi.c | 14 +-------------
 drivers/scsi/pm8001/pm80xx_hwi.c | 15 +--------------
 2 files changed, 2 insertions(+), 27 deletions(-)
drivers/scsi/pm8001/pm8001_hwi.c +1 -13
index fb9848e1d481..1c1b87905bea 100644
@@ -1785,7 +1785,7 @@ static void pm8001_send_read_log(struct pm8001_hba_info *pm8001_ha,
 	struct pm8001_ccb_info *ccb;
 	struct sas_task *task = NULL;
 	struct host_to_dev_fis fis;
-	struct domain_device *dev;
+	struct domain_device *dev = pm8001_ha_dev->sas_device;
 	struct inbound_queue_table *circularQ;
 	u32 opc = OPC_INB_SATA_HOST_OPSTART;
 
@@ -1806,17 +1806,6 @@ static void pm8001_send_read_log(struct pm8001_hba_info *pm8001_ha,
 		return;
 	}
 
-	/* allocate domain device by ourselves as libsas
-	 * is not going to provide any
-	*/
-	dev = kzalloc(sizeof(struct domain_device), GFP_ATOMIC);
-	if (!dev) {
-		sas_free_task(task);
-		pm8001_tag_free(pm8001_ha, ccb_tag);
-		PM8001_FAIL_DBG(pm8001_ha,
-			pm8001_printk("Domain device cannot be allocated\n"));
-		return;
-	}
 	task->dev = dev;
 	task->dev->lldd_dev = pm8001_ha_dev;
 
@@ -1848,7 +1837,6 @@ static void pm8001_send_read_log(struct pm8001_hba_info *pm8001_ha,
 	if (res) {
 		sas_free_task(task);
 		pm8001_tag_free(pm8001_ha, ccb_tag);
-		kfree(dev);
 	}
 }
 
drivers/scsi/pm8001/pm80xx_hwi.c +1 -14
index 4d205ebaee87..e59e72b20b0c 100644
@@ -1809,7 +1809,7 @@ static void pm80xx_send_read_log(struct pm8001_hba_info *pm8001_ha,
 	struct pm8001_ccb_info *ccb;
 	struct sas_task *task = NULL;
 	struct host_to_dev_fis fis;
-	struct domain_device *dev;
+	struct domain_device *dev = pm8001_ha_dev->sas_device;
 	struct inbound_queue_table *circularQ;
 	u32 opc = OPC_INB_SATA_HOST_OPSTART;
 
@@ -1830,18 +1830,6 @@ static void pm80xx_send_read_log(struct pm8001_hba_info *pm8001_ha,
 		return;
 	}
 
-	/* allocate domain device by ourselves as libsas
-	 * is not going to provide any
-	*/
-	dev = kzalloc(sizeof(struct domain_device), GFP_ATOMIC);
-	if (!dev) {
-		sas_free_task(task);
-		pm8001_tag_free(pm8001_ha, ccb_tag);
-		PM8001_FAIL_DBG(pm8001_ha,
-			pm8001_printk("Domain device cannot be allocated\n"));
-		return;
-	}
-
 	task->dev = dev;
 	task->dev->lldd_dev = pm8001_ha_dev;
 
@@ -1875,7 +1863,6 @@ static void pm80xx_send_read_log(struct pm8001_hba_info *pm8001_ha,
 	if (res) {
 		sas_free_task(task);
 		pm8001_tag_free(pm8001_ha, ccb_tag);
-		kfree(dev);
 	}
 }
 
Signature · 1 line
2.16.4
#80 lore raw

RFC PATCH v3 35/41 libsas: add SCSI target pointer to struct domain_device

In reply to Hannes Reinecke

Add a pointer to the SCSI target which corresponds to the domain device.

---
 drivers/scsi/libsas/sas_scsi_host.c | 2 ++
 include/scsi/libsas.h               | 1 +
 2 files changed, 3 insertions(+)
drivers/scsi/libsas/sas_scsi_host.c +2 -0
index 9e0975e55c27..c5a430e3fa2d 100644
@@ -830,6 +830,7 @@ int sas_target_alloc(struct scsi_target *starget)
 
 	kref_get(&found_dev->kref);
 	starget->hostdata = found_dev;
+	found_dev->starget = starget;
 	return 0;
 }
 
@@ -919,6 +920,7 @@ void sas_target_destroy(struct scsi_target *starget)
 		return;
 
 	starget->hostdata = NULL;
+	found_dev->starget = NULL;
 	sas_put_device(found_dev);
 }
 
include/scsi/libsas.h +1 -0
index 4e2d61e8fb1e..2d9bc4882930 100644
@@ -175,6 +175,7 @@ struct domain_device {
 
 	struct domain_device *parent;
 	struct list_head siblings; /* devices on the same level */
+	struct scsi_target *starget; /* Corresponding SCSI target device */
 	struct asd_sas_port *port;        /* shortcut to root of the tree */
 	struct sas_phy *phy;
 
Signature · 1 line
2.16.4
#81 lore raw

RFC PATCH v3 36/41 scsi: libsas,hisi_sas,mvsas,pm8001: Allocate Scsi_cmd for slow task

In reply to Hannes Reinecke

From: John Garry <[email protected]>

Allocate a Scsi_cmd for SAS slow tasks, so they can be accounted for in the blk-mq layer.

---
 drivers/scsi/hisi_sas/hisi_sas_main.c | 56 ++++++++++++++++++++++----------
 drivers/scsi/libsas/sas_expander.c    |  7 ++--
 drivers/scsi/libsas/sas_init.c        | 61 +++++++++++++++++++++++++++++++----
 drivers/scsi/mvsas/mv_sas.c           |  5 ++-
 drivers/scsi/pm8001/pm8001_hwi.c      |  9 ++++--
 drivers/scsi/pm8001/pm8001_sas.c      | 39 ++++++++++++++--------
 drivers/scsi/pm8001/pm80xx_hwi.c      |  9 ++++--
 include/scsi/libsas.h                 |  8 +++--
 8 files changed, 148 insertions(+), 46 deletions(-)
drivers/scsi/hisi_sas/hisi_sas_main.c +39 -17
index 9a6deb21fe4d..991241ab87d1 100644
@@ -15,6 +15,7 @@ static int hisi_sas_debug_issue_ssp_tmf(struct domain_device *device,
 static int
 hisi_sas_internal_task_abort(struct hisi_hba *hisi_hba,
 			     struct domain_device *device,
+			     struct scsi_lun *lun,
 			     int abort_flag, int tag);
 static int hisi_sas_softreset_ata_disk(struct domain_device *device);
 static int hisi_sas_control_phy(struct asd_sas_phy *sas_phy, enum phy_func func,
@@ -1046,15 +1047,17 @@ static void hisi_sas_dev_gone(struct domain_device *device)
 	struct hisi_sas_device *sas_dev = device->lldd_dev;
 	struct hisi_hba *hisi_hba = dev_to_hisi_hba(device);
 	struct device *dev = hisi_hba->dev;
+	struct scsi_lun lun;
 	int ret = 0;
 
 	dev_info(dev, "dev[%d:%x] is gone\n",
 		 sas_dev->device_id, sas_dev->dev_type);
 
 	down(&hisi_hba->sem);
+	int_to_scsilun(0, &lun);
 	if (!test_bit(HISI_SAS_RESET_BIT, &hisi_hba->flags)) {
 		hisi_sas_internal_task_abort(hisi_hba, device,
-					     HISI_SAS_INT_ABT_DEV, 0);
+				&lun, HISI_SAS_INT_ABT_DEV, 0);
 
 		hisi_sas_dereg_device(hisi_hba, device);
 
@@ -1182,12 +1185,21 @@ static int hisi_sas_exec_internal_tmf_task(struct domain_device *device,
 {
 	struct hisi_sas_device *sas_dev = device->lldd_dev;
 	struct hisi_hba *hisi_hba = sas_dev->hisi_hba;
+	struct sas_ha_struct *sha = &hisi_hba->sha;
 	struct device *dev = hisi_hba->dev;
 	struct sas_task *task;
 	int res, retry;
 
 	for (retry = 0; retry < TASK_RETRY; retry++) {
-		task = sas_alloc_slow_task(GFP_KERNEL);
+		struct scsilun lun;
+
+		int_to_scsilun(0, &lun);
+		if (!dev_is_sata) {
+			struct sas_ssp_task ssp_task = parameter;
+
+			memcpy(lun.scsi_lun, ssp_task.LUN, 8);
+		}
+		task = sas_alloc_slow_task(sha, device, &lun, GFP_KERNEL);
 		if (!task)
 			return -ENOMEM;
 
@@ -1486,7 +1498,9 @@ static void hisi_sas_terminate_stp_reject(struct hisi_hba *hisi_hba)
 {
 	struct device *dev = hisi_hba->dev;
 	int port_no, rc, i;
+	struct scsi_lun lun;
 
+	int_to_scsilun(0, &lun);
 	for (i = 0; i < HISI_SAS_MAX_DEVICES; i++) {
 		struct hisi_sas_device *sas_dev = &hisi_hba->devices[i];
 		struct domain_device *device = sas_dev->sas_device;
@@ -1495,7 +1509,7 @@ static void hisi_sas_terminate_stp_reject(struct hisi_hba *hisi_hba)
 			continue;
 
 		rc = hisi_sas_internal_task_abort(hisi_hba, device,
-						  HISI_SAS_INT_ABT_DEV, 0);
+				&lun, HISI_SAS_INT_ABT_DEV, 0);
 		if (rc < 0)
 			dev_err(dev, "STP reject: abort dev failed %d\n", rc);
 	}
@@ -1647,7 +1661,7 @@ static int hisi_sas_abort_task(struct sas_task *task)
 						  &tmf_task);
 
 		rc2 = hisi_sas_internal_task_abort(hisi_hba, device,
-						   HISI_SAS_INT_ABT_CMD, tag);
+				&lun, HISI_SAS_INT_ABT_CMD, tag);
 		if (rc2 < 0) {
 			dev_err(dev, "abort task: internal abort (%d)\n", rc2);
 			return TMF_RESP_FUNC_FAILED;
@@ -1667,9 +1681,9 @@ static int hisi_sas_abort_task(struct sas_task *task)
 	} else if (task->task_proto & SAS_PROTOCOL_SATA ||
 		task->task_proto & SAS_PROTOCOL_STP) {
 		if (task->dev->dev_type == SAS_SATA_DEV) {
+			int_to_scsilun(0, &lun);
 			rc = hisi_sas_internal_task_abort(hisi_hba, device,
-							  HISI_SAS_INT_ABT_DEV,
-							  0);
+					&lun, HISI_SAS_INT_ABT_DEV, 0);
 			if (rc < 0) {
 				dev_err(dev, "abort task: internal abort failed\n");
 				goto out;
@@ -1683,8 +1697,9 @@ static int hisi_sas_abort_task(struct sas_task *task)
 		u32 tag = slot->idx;
 		struct hisi_sas_cq *cq = &hisi_hba->cq[slot->dlvry_queue];
 
+		int_to_scsilun(0, &lun);
 		rc = hisi_sas_internal_task_abort(hisi_hba, device,
-						  HISI_SAS_INT_ABT_CMD, tag);
+				&lun, HISI_SAS_INT_ABT_CMD, tag);
 		if (((rc < 0) || (rc == TMF_RESP_FUNC_FAILED)) &&
 					task->lldd_task) {
 			/*
@@ -1710,7 +1725,7 @@ static int hisi_sas_abort_task_set(struct domain_device *device, u8 *lun)
 	int rc;
 
 	rc = hisi_sas_internal_task_abort(hisi_hba, device,
-					  HISI_SAS_INT_ABT_DEV, 0);
+			(struct scsi_lun *)lun, HISI_SAS_INT_ABT_DEV, 0);
 	if (rc < 0) {
 		dev_err(dev, "abort task set: internal abort rc=%d\n", rc);
 		return TMF_RESP_FUNC_FAILED;
@@ -1798,10 +1813,12 @@ static int hisi_sas_I_T_nexus_reset(struct domain_device *device)
 {
 	struct hisi_hba *hisi_hba = dev_to_hisi_hba(device);
 	struct device *dev = hisi_hba->dev;
+	struct scsi_lun lun;
 	int rc;
 
+	int_to_scsilun(0, &lun);
 	rc = hisi_sas_internal_task_abort(hisi_hba, device,
-					  HISI_SAS_INT_ABT_DEV, 0);
+			&lun, HISI_SAS_INT_ABT_DEV, 0);
 	if (rc < 0) {
 		dev_err(dev, "I_T nexus reset: internal abort (%d)\n", rc);
 		return TMF_RESP_FUNC_FAILED;
@@ -1831,7 +1848,7 @@ static int hisi_sas_lu_reset(struct domain_device *device, u8 *lun)
 
 	/* Clear internal IO and then lu reset */
 	rc = hisi_sas_internal_task_abort(hisi_hba, device,
-					  HISI_SAS_INT_ABT_DEV, 0);
+			(struct scsi_lun *)lun, HISI_SAS_INT_ABT_DEV, 0);
 	if (rc < 0) {
 		dev_err(dev, "lu_reset: internal abort failed\n");
 		goto out;
@@ -2012,6 +2029,7 @@ hisi_sas_internal_abort_task_exec(struct hisi_hba *hisi_hba, int device_id,
  * abort command for single IO command or a device
  * @hisi_hba: host controller struct
  * @device: domain device
+ * @lun: logical unit number
  * @abort_flag: mode of operation, device or single IO
  * @tag: tag of IO to be aborted (only relevant to single
  *       IO mode)
@@ -2019,12 +2037,15 @@ hisi_sas_internal_abort_task_exec(struct hisi_hba *hisi_hba, int device_id,
  */
 static int
 _hisi_sas_internal_task_abort(struct hisi_hba *hisi_hba,
-			      struct domain_device *device, int abort_flag,
-			      int tag, struct hisi_sas_dq *dq)
+			      struct domain_device *device,
+			      struct scsi_lun *lun,
+			      int abort_flag, int tag,
+			      struct hisi_sas_dq *dq)
 {
-	struct sas_task *task;
 	struct hisi_sas_device *sas_dev = device->lldd_dev;
+	struct sas_ha_struct *sha = &hisi_hba->sha;
 	struct device *dev = hisi_hba->dev;
+	struct sas_task *task;
 	int res;
 
 	/*
@@ -2036,7 +2057,8 @@ _hisi_sas_internal_task_abort(struct hisi_hba *hisi_hba,
 	if (!hisi_hba->hw->prep_abort)
 		return TMF_RESP_FUNC_FAILED;
 
-	task = sas_alloc_slow_task(GFP_KERNEL);
+	task = sas_alloc_slow_task(sha, device,
+				   (struct scsi_lun *)lun, GFP_KERNEL);
 	if (!task)
 		return -ENOMEM;
 
@@ -2109,6 +2131,7 @@ _hisi_sas_internal_task_abort(struct hisi_hba *hisi_hba,
 static int
 hisi_sas_internal_task_abort(struct hisi_hba *hisi_hba,
 			     struct domain_device *device,
+			     struct scsi_lun *lun,
 			     int abort_flag, int tag)
 {
 	struct hisi_sas_slot *slot;
@@ -2121,7 +2144,7 @@ hisi_sas_internal_task_abort(struct hisi_hba *hisi_hba,
 		slot = &hisi_hba->slot_info[tag];
 		dq = &hisi_hba->dq[slot->dlvry_queue];
 		return _hisi_sas_internal_task_abort(hisi_hba, device,
-						     abort_flag, tag, dq);
+				lun, abort_flag, tag, dq);
 	case HISI_SAS_INT_ABT_DEV:
 		for (i = 0; i < hisi_hba->cq_nvecs; i++) {
 			struct hisi_sas_cq *cq = &hisi_hba->cq[i];
@@ -2131,8 +2154,7 @@ hisi_sas_internal_task_abort(struct hisi_hba *hisi_hba,
 				continue;
 			dq = &hisi_hba->dq[i];
 			rc = _hisi_sas_internal_task_abort(hisi_hba, device,
-							   abort_flag, tag,
-							   dq);
+					lun, abort_flag, tag, dq);
 			if (rc)
 				return rc;
 		}
drivers/scsi/libsas/sas_expander.c +5 -2
index ab671cdd4cfb..ec140bb2af1c 100644
@@ -56,9 +56,12 @@ static int smp_execute_task_sg(struct domain_device *dev,
 {
 	int res, retry;
 	struct sas_task *task = NULL;
+	struct sas_ha_struct *ha = dev->port->ha;
 	struct sas_internal *i =
-		to_sas_internal(dev->port->ha->core.shost->transportt);
+		to_sas_internal(ha->core.shost->transportt);
+	struct scsi_lun lun;
 
+	int_to_scsilun(0, &lun);
 	mutex_lock(&dev->ex_dev.cmd_mutex);
 	for (retry = 0; retry < 3; retry++) {
 		if (test_bit(SAS_DEV_GONE, &dev->state)) {
@@ -66,7 +69,7 @@ static int smp_execute_task_sg(struct domain_device *dev,
 			break;
 		}
 
-		task = sas_alloc_slow_task(GFP_KERNEL);
+		task = sas_alloc_slow_task(ha, dev, &lun, GFP_KERNEL);
 		if (!task) {
 			res = -ENOMEM;
 			break;
drivers/scsi/libsas/sas_init.c +54 -7

101 lines not on this page · raw message

drivers/scsi/mvsas/mv_sas.c +4 -1
index 937c27777ab9..0b647bf7dc0a 100644
@@ -1280,13 +1280,16 @@ static int mvs_exec_internal_tmf_task(struct domain_device *dev,
 				      u8 *lun, struct mvs_tmf_task *tmf)
 {
 	int res, retry;
+	struct sas_ha_struct *ha = dev->port->ha;
 	struct sas_task *task = NULL;
+	struct scsi_lun scsilun;
 
 	if (!(dev->tproto & SAS_PROTOCOL_SSP))
 		return TMF_RESP_FUNC_ESUPP;
 
+	memcpy(scsilun.scsi_lun, lun, 8);
 	for (retry = 0; retry < 3; retry++) {
-		task = sas_alloc_slow_task(GFP_KERNEL);
+		task = sas_alloc_slow_task(ha, dev, &scsilun, GFP_KERNEL);
 		if (!task)
 			return -ENOMEM;
 
drivers/scsi/pm8001/pm8001_hwi.c +7 -2
index 1c1b87905bea..62208e98508c 100644
@@ -1736,6 +1736,7 @@ static void pm8001_send_abort_all(struct pm8001_hba_info *pm8001_ha,
 	struct task_abort_req task_abort;
 	struct inbound_queue_table *circularQ;
 	u32 opc = OPC_INB_SATA_ABORT;
+	struct scsi_lun lun;
 	int ret;
 
 	if (!pm8001_ha_dev) {
@@ -1743,7 +1744,9 @@ static void pm8001_send_abort_all(struct pm8001_hba_info *pm8001_ha,
 		return;
 	}
 
-	task = sas_alloc_slow_task(GFP_ATOMIC);
+	int_to_scsilun(0, &lun);
+	task = sas_alloc_slow_task(pm8001_ha->sas, pm8001_ha_dev->sas_device,
+				   &lun, GFP_ATOMIC);
 
 	if (!task) {
 		PM8001_FAIL_DBG(pm8001_ha, pm8001_printk("cannot "
@@ -1788,8 +1791,10 @@ static void pm8001_send_read_log(struct pm8001_hba_info *pm8001_ha,
 	struct domain_device *dev = pm8001_ha_dev->sas_device;
 	struct inbound_queue_table *circularQ;
 	u32 opc = OPC_INB_SATA_HOST_OPSTART;
+	struct scsi_lun lun;;
 
-	task = sas_alloc_slow_task(GFP_ATOMIC);
+	int_to_scsilun(0, &lun);
+	task = sas_alloc_slow_task(pm8001_ha->sas, dev, &lun, GFP_ATOMIC);
 
 	if (!task) {
 		PM8001_FAIL_DBG(pm8001_ha,
drivers/scsi/pm8001/pm8001_sas.c +26 -13
index ae25a8b62fcc..6de448e07528 100644
@@ -720,7 +720,9 @@ static int pm8001_exec_internal_tmf_task(struct domain_device *dev,
 		return TMF_RESP_FUNC_ESUPP;
 
 	for (retry = 0; retry < 3; retry++) {
-		task = sas_alloc_slow_task(GFP_KERNEL);
+		task = sas_alloc_slow_task(pm8001_ha->sas,
+					   pm8001_dev->sas_device,
+					   (struct scsi_lun *)lun, GFP_KERNEL);
 		if (!task)
 			return -ENOMEM;
 
@@ -798,7 +800,7 @@ static int pm8001_exec_internal_tmf_task(struct domain_device *dev,
 
 static int
 pm8001_exec_internal_task_abort(struct pm8001_hba_info *pm8001_ha,
-	struct pm8001_device *pm8001_dev, u32 flag,
+	struct pm8001_device *pm8001_dev, u8 *lun, u32 flag,
 	u32 task_tag)
 {
 	struct domain_device *dev = pm8001_dev->sas_device;
@@ -808,7 +810,8 @@ pm8001_exec_internal_task_abort(struct pm8001_hba_info *pm8001_ha,
 	struct sas_task *task = NULL;
 
 	for (retry = 0; retry < 3; retry++) {
-		task = sas_alloc_slow_task(GFP_KERNEL);
+		task = sas_alloc_slow_task(pm8001_ha->sas, dev,
+					   (struct scsi_lun *)lun, GFP_KERNEL);
 		if (!task)
 			return -ENOMEM;
 
@@ -885,14 +888,16 @@ static void pm8001_dev_gone_notify(struct domain_device *dev)
 	spin_lock_irqsave(&pm8001_ha->lock, flags);
 	if (pm8001_dev) {
 		u32 device_id = pm8001_dev->device_id;
+		struct scsi_lun lun;
 
+		int_to_scsilun(0, &lun);
 		PM8001_DISC_DBG(pm8001_ha,
 			pm8001_printk("found dev[%d:%x] is gone.\n",
 			pm8001_dev->device_id, pm8001_dev->dev_type));
 		if (pm8001_dev->running_req) {
 			spin_unlock_irqrestore(&pm8001_ha->lock, flags);
 			pm8001_exec_internal_task_abort(pm8001_ha,
-				pm8001_dev, 1, 0);
+				pm8001_dev, lun.scsi_lun, 1, 0);
 			while (pm8001_dev->running_req)
 				msleep(20);
 			spin_lock_irqsave(&pm8001_ha->lock, flags);
@@ -1002,6 +1007,9 @@ int pm8001_I_T_nexus_reset(struct domain_device *dev)
 	phy = sas_get_local_phy(dev);
 
 	if (dev_is_sata(dev)) {
+		struct scsi_lun lun;
+
+		int_to_scsilun(0, &lun);
 		if (scsi_is_sas_phy_local(phy)) {
 			rc = 0;
 			goto out;
@@ -1016,7 +1024,7 @@ int pm8001_I_T_nexus_reset(struct domain_device *dev)
 		}
 		msleep(2000);
 		rc = pm8001_exec_internal_task_abort(pm8001_ha,
-			pm8001_dev, 1, 0);
+			pm8001_dev, lun.scsi_lun, 1, 0);
 		if (rc) {
 			PM8001_EH_DBG(pm8001_ha,
 			pm8001_printk("task abort failed %x\n"
@@ -1044,7 +1052,9 @@ int pm8001_I_T_nexus_event_handler(struct domain_device *dev)
 	struct pm8001_device *pm8001_dev;
 	struct pm8001_hba_info *pm8001_ha;
 	struct sas_phy *phy;
+	struct scsi_lun lun;
 
+	int_to_scsilun(0, &lun);
 	if (!dev || !dev->lldd_dev)
 		return -1;
 
@@ -1064,7 +1074,7 @@ int pm8001_I_T_nexus_event_handler(struct domain_device *dev)
 		}
 		/* send internal ssp/sata/smp abort command to FW */
 		rc = pm8001_exec_internal_task_abort(pm8001_ha, pm8001_dev,
-						     1, 0);
+						     lun.scsi_lun, 1, 0);
 		msleep(100);
 
 		/* deregister the target device */
@@ -1080,7 +1090,7 @@ int pm8001_I_T_nexus_event_handler(struct domain_device *dev)
 	} else {
 		/* send internal ssp/sata/smp abort command to FW */
 		rc = pm8001_exec_internal_task_abort(pm8001_ha, pm8001_dev,
-						     1, 0);
+						     lun.scsi_lun, 1, 0);
 		msleep(100);
 
 		/* deregister the target device */
@@ -1108,8 +1118,8 @@ int pm8001_lu_reset(struct domain_device *dev, u8 *lun)
 	DECLARE_COMPLETION_ONSTACK(completion_setstate);
 	if (dev_is_sata(dev)) {
 		struct sas_phy *phy = sas_get_local_phy(dev);
-		rc = pm8001_exec_internal_task_abort(pm8001_ha, pm8001_dev,
-			1, 0);
+		rc = pm8001_exec_internal_task_abort(pm8001_ha, pm8001_dev ,
+			lun, 1, 0);
 		rc = sas_phy_reset(phy, 1);
 		sas_put_local_phy(phy);
 		pm8001_dev->setds_completion = &completion_setstate;
@@ -1220,9 +1230,10 @@ int pm8001_abort_task(struct sas_task *task)
 						   &tmf_task);
 		if (rc == TMF_RESP_FUNC_SUCC)
 			pm8001_exec_internal_task_abort(pm8001_ha, pm8001_dev,
-				0, tag);
+				lun.scsi_lun, 0, tag);
 	} else if (task->task_proto & SAS_PROTOCOL_SATA ||
 		task->task_proto & SAS_PROTOCOL_STP) {
+		memset(lun.scsi_lun, 0, sizeof(lun.scsi_lun));
 		if (pm8001_ha->chip_id == chip_8006) {
 			DECLARE_COMPLETION_ONSTACK(completion_reset);
 			DECLARE_COMPLETION_ONSTACK(completion);
@@ -1289,7 +1300,7 @@ int pm8001_abort_task(struct sas_task *task)
 			 * going to free the task.
 			 */
 			ret = pm8001_exec_internal_task_abort(pm8001_ha,
-				pm8001_dev, 1, tag);
+				pm8001_dev, lun.scsi_lun, 1, tag);
 			if (ret)
 				goto out;
 			ret = wait_for_completion_timeout(
@@ -1306,13 +1317,15 @@ int pm8001_abort_task(struct sas_task *task)
 			wait_for_completion(&completion);
 		} else {
 			rc = pm8001_exec_internal_task_abort(pm8001_ha,
-				pm8001_dev, 0, tag);
+				pm8001_dev, lun.scsi_lun, 0, tag);
 		}
 		rc = TMF_RESP_FUNC_COMPLETE;
 	} else if (task->task_proto & SAS_PROTOCOL_SMP) {
 		/* SMP */
+
+		int_to_scsilun(0, &lun);
 		rc = pm8001_exec_internal_task_abort(pm8001_ha, pm8001_dev,
-			0, tag);
+			lun.scsi_lun, 0, tag);
 
 	}
 out:
drivers/scsi/pm8001/pm80xx_hwi.c +7 -2
index e59e72b20b0c..702609f10717 100644
@@ -1755,6 +1755,7 @@ static void pm80xx_send_abort_all(struct pm8001_hba_info *pm8001_ha,
 	struct task_abort_req task_abort;
 	struct inbound_queue_table *circularQ;
 	u32 opc = OPC_INB_SATA_ABORT;
+	struct scsi_lun lun;
 	int ret;
 
 	if (!pm8001_ha_dev) {
@@ -1762,7 +1763,9 @@ static void pm80xx_send_abort_all(struct pm8001_hba_info *pm8001_ha,
 		return;
 	}
 
-	task = sas_alloc_slow_task(GFP_ATOMIC);
+	int_to_scsilun(0, &lun);
+	task = sas_alloc_slow_task(pm8001_ha->sas,pm8001_ha_dev->sas_device,
+				   &lun, GFP_ATOMIC);
 
 	if (!task) {
 		PM8001_FAIL_DBG(pm8001_ha, pm8001_printk("cannot "
@@ -1812,8 +1815,10 @@ static void pm80xx_send_read_log(struct pm8001_hba_info *pm8001_ha,
 	struct domain_device *dev = pm8001_ha_dev->sas_device;
 	struct inbound_queue_table *circularQ;
 	u32 opc = OPC_INB_SATA_HOST_OPSTART;
+	struct scsi_lun lun;
 
-	task = sas_alloc_slow_task(GFP_ATOMIC);
+	int_to_scsilun(0, &lun);
+	task = sas_alloc_slow_task(pm8001_ha->sas, dev, &lun, GFP_ATOMIC);
 
 	if (!task) {
 		PM8001_FAIL_DBG(pm8001_ha,
include/scsi/libsas.h +6 -2
index 2d9bc4882930..c927228019c9 100644
@@ -349,7 +349,7 @@ struct asd_sas_phy {
 
 struct scsi_core {
 	struct Scsi_Host *shost;
-
+	struct scsi_device *shost_dev;
 };
 
 enum sas_ha_state {
@@ -609,6 +609,7 @@ struct sas_task_slow {
 	struct timer_list     timer;
 	struct completion     completion;
 	struct sas_task       *task;
+	struct scsi_cmnd      *scmd;
 };
 
 #define SAS_TASK_STATE_PENDING      1
@@ -618,7 +619,10 @@ struct sas_task_slow {
 #define SAS_TASK_AT_INITIATOR       16
 
 extern struct sas_task *sas_alloc_task(gfp_t flags);
-extern struct sas_task *sas_alloc_slow_task(gfp_t flags);
+extern struct sas_task *sas_alloc_slow_task(struct sas_ha_struct *ha,
+					    struct domain_device *dev,
+					    struct scsi_lun *lun,
+					    gfp_t flags);
 extern void sas_free_task(struct sas_task *task);
 
 struct sas_domain_function_template {
Signature · 1 line
2.16.4
#82 lore raw

Re: [PATCH RFC v3 36/41] scsi: libsas,hisi_sas,mvsas,pm8001: Allocate Scsi_cmd for slow task

Quoted 21 lines · Hannes Reinecke

k *sas_alloc_slow_task(gfp_t flags)

drivers/scsi/libsas/sas_init.c +11 -5
+struct sas_task *sas_alloc_slow_task(struct sas_ha_struct *ha,
+				     struct domain_device *dev,
+				     struct scsi_lun *lun, gfp_t flags)
  {
  	struct sas_task *task = sas_alloc_task(flags);
-	struct sas_task_slow *slow = kmalloc(sizeof(*slow), flags);
+	struct Scsi_Host *shost = ha->core.shost;
+	struct sas_task_slow *slow;
 
-	if (!task || !slow) {
-		if (task)
-			kmem_cache_free(sas_task_cache, task);
-		kfree(slow);
+	if (!task)
  		return NULL;
+
+	slow = kzalloc(sizeof(*slow), flags);
+	if (!slow)
+		goto out_err_slow;
+

drivers/scsi/libsas/sas_init.c struct sas_task *sas_alloc_task(gfp_t flags) · line 65

Quoted 10 lines
drivers/scsi/libsas/sas_init.c +10 -0
+	if (shost->nr_reserved_cmds) {
+		struct scsi_device *sdev;
+
+		if (dev && dev->starget) {
+			sdev = scsi_device_lookup_by_target(dev->starget,
+						    scsilun_to_int(lun));
+			if (!sdev)
+				goto out_err_scmd;
+		} else
+			sdev = ha->core.shost_dev;

Could we always just use the ha->core.shost_dev? I feel that it would make the code neater.

Thanks,
John

Quoted 5 lines
drivers/scsi/libsas/sas_init.c +4 -0
+		slow->scmd = scsi_get_reserved_cmd(sdev, DMA_NONE, false);
+		if (!slow->scmd)
+			goto out_err_scmd;
+		ASSIGN_SAS_TASK(slow->scmd, task);
  	}
#83 lore raw

Re: [PATCH RFC v3 36/41] scsi: libsas, hisi_sas, mvsas, pm8001: Allocate Scsi_cmd for slow task

In reply to Hannes Reinecke
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system. BTW, we also suggest to use '--base' option to specify the base tree in git format-patch, please…

Hi Hannes,

[FYI, it's a private test report for your RFC patch.]
[auto build test ERROR on scsi/for-next]
[also build test ERROR on next-20200501]
[cannot apply to mkp-scsi/for-next block/for-next linus/master v5.7-rc3] [if your patch is applied to the wrong git tree, please drop us a note to help improve the system. BTW, we also suggest to use '--base' option to specify the base tree in git format-patch, please see https://stackoverflow.com/a/37406982]

url:    https://github.com/0day-ci/linux/commits/Hannes-Reinecke/scsi-enable-reserved-commands-for-LLDDs/20200501-035639
base:   https://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi.git for-next
config: arm64-allyesconfig (attached as .config)
compiler: aarch64-linux-gcc (GCC) 9.3.0

reproduce:
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day GCC_VERSION=9.3.0 make.cross ARCH=arm64

If you fix the issue, kindly add following tag as appropriate

All errors (new ones prefixed by >>):

drivers/scsi/hisi_sas/hisi_sas_main.c: In function 'hisi_sas_exec_internal_tmf_task':

>> drivers/scsi/hisi_sas/hisi_sas_main.c:1194:18: error: storage size of 'lun' isn't known
    1194 |   struct scsilun lun;
         |                  ^~~
   drivers/scsi/hisi_sas/hisi_sas_main.c:1197:7: warning: the address of 'dev_is_sata' will always evaluate as 'true' [-Waddress]
    1197 |   if (!dev_is_sata) {
         |       ^
>> drivers/scsi/hisi_sas/hisi_sas_main.c:1198:35: error: invalid initializer
    1198 |    struct sas_ssp_task ssp_task = parameter;
         |                                   ^~~~~~~~~
   drivers/scsi/hisi_sas/hisi_sas_main.c:1194:18: warning: unused variable 'lun' [-Wunused-variable]
    1194 |   struct scsilun lun;
         |                  ^~~

vim +1194 drivers/scsi/hisi_sas/hisi_sas_main.c

1178
1179 #define TASK_TIMEOUT 20
1180 #define TASK_RETRY 3
1181 #define INTERNAL_ABORT_TIMEOUT 6
1182 static int hisi_sas_exec_internal_tmf_task(struct domain_device *device,
1183 void *parameter, u32 para_len,
1184 struct hisi_sas_tmf_task *tmf)
1185 {
1186 struct hisi_sas_device *sas_dev = device->lldd_dev;
1187 struct hisi_hba *hisi_hba = sas_dev->hisi_hba;
1188 struct sas_ha_struct *sha = &hisi_hba->sha;
1189 struct device *dev = hisi_hba->dev;
1190 struct sas_task *task;
1191 int res, retry;
1192
1193 for (retry = 0; retry < TASK_RETRY; retry++) {

> 1194			struct scsilun lun;
  1195	
  1196			int_to_scsilun(0, &lun);
  1197			if (!dev_is_sata) {
> 1198				struct sas_ssp_task ssp_task = parameter;

1199
1200 memcpy(lun.scsi_lun, ssp_task.LUN, 8);
1201 }
1202 task = sas_alloc_slow_task(sha, device, &lun, GFP_KERNEL);
1203 if (!task)
1204 return -ENOMEM;
1205
1206 task->dev = device;
1207 task->task_proto = device->tproto;
1208
1209 if (dev_is_sata(device)) {
1210 task->ata_task.device_control_reg_update = 1;
1211 memcpy(&task->ata_task.fis, parameter, para_len);
1212 } else {
1213 memcpy(&task->ssp_task, parameter, para_len);
1214 }
1215 task->task_done = hisi_sas_task_done;
1216
1217 task->slow_task->timer.function = hisi_sas_tmf_timedout;
1218 task->slow_task->timer.expires = jiffies + TASK_TIMEOUT * HZ;
1219 add_timer(&task->slow_task->timer);
1220
1221 res = hisi_sas_task_exec(task, GFP_KERNEL, 1, tmf);
1222
1223 if (res) {
1224 del_timer(&task->slow_task->timer);
1225 dev_err(dev, "abort tmf: executing internal task failed: %d\n",
1226 res);
1227 goto ex_err;
1228 }
1229
1230 wait_for_completion(&task->slow_task->completion);
1231 res = TMF_RESP_FUNC_FAILED;
1232 /* Even TMF timed out, return direct. */
1233 if ((task->task_state_flags & SAS_TASK_STATE_ABORTED)) {
1234 if (!(task->task_state_flags & SAS_TASK_STATE_DONE)) {
1235 struct hisi_sas_slot *slot = task->lldd_task;
1236
1237 dev_err(dev, "abort tmf: TMF task timeout and not done\n");
1238 if (slot) {
1239 struct hisi_sas_cq *cq =
1240 &hisi_hba->cq[slot->dlvry_queue];
1241 /*
1242 * sync irq to avoid free'ing task
1243 * before using task in IO completion
1244 */
1245 synchronize_irq(cq->irq_no);
1246 slot->task = NULL;
1247 }
1248
1249 goto ex_err;
1250 } else
1251 dev_err(dev, "abort tmf: TMF task timeout\n");
1252 }
1253
1254 if (task->task_status.resp == SAS_TASK_COMPLETE &&
1255 task->task_status.stat == TMF_RESP_FUNC_COMPLETE) {
1256 res = TMF_RESP_FUNC_COMPLETE;
1257 break;
1258 }
1259
1260 if (task->task_status.resp == SAS_TASK_COMPLETE &&
1261 task->task_status.stat == TMF_RESP_FUNC_SUCC) {
1262 res = TMF_RESP_FUNC_SUCC;
1263 break;
1264 }
1265
1266 if (task->task_status.resp == SAS_TASK_COMPLETE &&
1267 task->task_status.stat == SAS_DATA_UNDERRUN) {
1268 /* no error, but return the number of bytes of
1269 * underrun
1270 */
1271 dev_warn(dev, "abort tmf: task to dev %016llx resp: 0x%x sts 0x%x underrun\n",
1272 SAS_ADDR(device->sas_addr),
1273 task->task_status.resp,
1274 task->task_status.stat);
1275 res = task->task_status.residual;
1276 break;
1277 }
1278
1279 if (task->task_status.resp == SAS_TASK_COMPLETE &&
1280 task->task_status.stat == SAS_DATA_OVERRUN) {
1281 dev_warn(dev, "abort tmf: blocked task error\n");
1282 res = -EMSGSIZE;
1283 break;
1284 }
1285
1286 if (task->task_status.resp == SAS_TASK_COMPLETE &&
1287 task->task_status.stat == SAS_OPEN_REJECT) {
1288 dev_warn(dev, "abort tmf: open reject failed\n");
1289 res = -EIO;
1290 } else {
1291 dev_warn(dev, "abort tmf: task to dev %016llx resp: 0x%x status 0x%x\n",
1292 SAS_ADDR(device->sas_addr),
1293 task->task_status.resp,
1294 task->task_status.stat);
1295 }
1296 sas_free_task(task);
1297 task = NULL;
1298 }
1299 ex_err:
1300 if (retry == TASK_RETRY)
1301 dev_warn(dev, "abort tmf: executing internal task failed!\n");
1302 sas_free_task(task);
1303 return res;
1304 }
1305

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org

#84 lore raw

Re: [PATCH RFC v3 36/41] scsi: libsas, hisi_sas, mvsas, pm8001: Allocate Scsi_cmd for slow task

In reply to Hannes Reinecke
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system. BTW, we also suggest to use '--base' option to specify the base tree in git format-patch, please…

Hi Hannes,

[FYI, it's a private test report for your RFC patch.]
[auto build test WARNING on scsi/for-next]
[also build test WARNING on next-20200504]
[cannot apply to mkp-scsi/for-next block/for-next linus/master v5.7-rc4] [if your patch is applied to the wrong git tree, please drop us a note to help improve the system. BTW, we also suggest to use '--base' option to specify the base tree in git format-patch, please see https://stackoverflow.com/a/37406982]

url:    https://github.com/0day-ci/linux/commits/Hannes-Reinecke/scsi-enable-reserved-commands-for-LLDDs/20200501-035639
base:   https://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi.git for-next
config: x86_64-randconfig-a002-20200503 (attached as .config)
compiler: gcc-6 (Debian 6.3.0-18+deb9u1) 6.3.0 20170516
reproduce:
        # save the attached .config to linux build tree
        make ARCH=x86_64 

If you fix the issue, kindly add following tag as appropriate

All warnings (new ones prefixed by >>):

drivers/scsi/hisi_sas/hisi_sas_main.c: In function 'hisi_sas_exec_internal_tmf_task':
drivers/scsi/hisi_sas/hisi_sas_main.c:1194:18: error: storage size of 'lun' isn't known

      struct scsilun lun;
                     ^~~
   In file included from include/linux/ioport.h:13:0,
                    from include/linux/acpi.h:12,
                    from drivers/scsi/hisi_sas/hisi_sas.h:10,
                    from drivers/scsi/hisi_sas/hisi_sas_main.c:7:
   drivers/scsi/hisi_sas/hisi_sas_main.c:1197:7: warning: the address of 'dev_is_sata' will always evaluate as 'true' [-Waddress]
      if (!dev_is_sata) {
          ^
   include/linux/compiler.h:58:52: note: in definition of macro '__trace_if_var'
    #define __trace_if_var(cond) (__builtin_constant_p(cond) ? (cond) : __trace_if_value(cond))
                                                       ^~~~
>> drivers/scsi/hisi_sas/hisi_sas_main.c:1197:3: note: in expansion of macro 'if'
      if (!dev_is_sata) {
      ^~
   drivers/scsi/hisi_sas/hisi_sas_main.c:1197:7: warning: the address of 'dev_is_sata' will always evaluate as 'true' [-Waddress]
      if (!dev_is_sata) {
          ^
   include/linux/compiler.h:58:61: note: in definition of macro '__trace_if_var'
    #define __trace_if_var(cond) (__builtin_constant_p(cond) ? (cond) : __trace_if_value(cond))
                                                                ^~~~
>> drivers/scsi/hisi_sas/hisi_sas_main.c:1197:3: note: in expansion of macro 'if'
      if (!dev_is_sata) {
      ^~
   drivers/scsi/hisi_sas/hisi_sas_main.c:1197:7: warning: the address of 'dev_is_sata' will always evaluate as 'true' [-Waddress]
      if (!dev_is_sata) {
          ^
   include/linux/compiler.h:69:3: note: in definition of macro '__trace_if_value'
     (cond) ?     \
      ^~~~
   include/linux/compiler.h:56:28: note: in expansion of macro '__trace_if_var'
    #define if(cond, ...) if ( __trace_if_var( !!(cond , ## __VA_ARGS__) ) )
                               ^~~~~~~~~~~~~~
>> drivers/scsi/hisi_sas/hisi_sas_main.c:1197:3: note: in expansion of macro 'if'
      if (!dev_is_sata) {
      ^~
   drivers/scsi/hisi_sas/hisi_sas_main.c:1198:35: error: invalid initializer
       struct sas_ssp_task ssp_task = parameter;
                                      ^~~~~~~~~
   drivers/scsi/hisi_sas/hisi_sas_main.c:1194:18: warning: unused variable 'lun' [-Wunused-variable]
      struct scsilun lun;
                     ^~~

vim +/if +1197 drivers/scsi/hisi_sas/hisi_sas_main.c

1178
1179 #define TASK_TIMEOUT 20
1180 #define TASK_RETRY 3
1181 #define INTERNAL_ABORT_TIMEOUT 6
1182 static int hisi_sas_exec_internal_tmf_task(struct domain_device *device,
1183 void *parameter, u32 para_len,
1184 struct hisi_sas_tmf_task *tmf)
1185 {
1186 struct hisi_sas_device *sas_dev = device->lldd_dev;
1187 struct hisi_hba *hisi_hba = sas_dev->hisi_hba;
1188 struct sas_ha_struct *sha = &hisi_hba->sha;
1189 struct device *dev = hisi_hba->dev;
1190 struct sas_task *task;
1191 int res, retry;
1192
1193 for (retry = 0; retry < TASK_RETRY; retry++) {
1194 struct scsilun lun;
1195
1196 int_to_scsilun(0, &lun);

> 1197			if (!dev_is_sata) {

1198 struct sas_ssp_task ssp_task = parameter;
1199
1200 memcpy(lun.scsi_lun, ssp_task.LUN, 8);
1201 }
1202 task = sas_alloc_slow_task(sha, device, &lun, GFP_KERNEL);
1203 if (!task)
1204 return -ENOMEM;
1205
1206 task->dev = device;
1207 task->task_proto = device->tproto;
1208
1209 if (dev_is_sata(device)) {
1210 task->ata_task.device_control_reg_update = 1;
1211 memcpy(&task->ata_task.fis, parameter, para_len);
1212 } else {
1213 memcpy(&task->ssp_task, parameter, para_len);
1214 }
1215 task->task_done = hisi_sas_task_done;
1216
1217 task->slow_task->timer.function = hisi_sas_tmf_timedout;
1218 task->slow_task->timer.expires = jiffies + TASK_TIMEOUT * HZ;
1219 add_timer(&task->slow_task->timer);
1220
1221 res = hisi_sas_task_exec(task, GFP_KERNEL, 1, tmf);
1222
1223 if (res) {
1224 del_timer(&task->slow_task->timer);
1225 dev_err(dev, "abort tmf: executing internal task failed: %d\n",
1226 res);
1227 goto ex_err;
1228 }
1229
1230 wait_for_completion(&task->slow_task->completion);
1231 res = TMF_RESP_FUNC_FAILED;
1232 /* Even TMF timed out, return direct. */
1233 if ((task->task_state_flags & SAS_TASK_STATE_ABORTED)) {
1234 if (!(task->task_state_flags & SAS_TASK_STATE_DONE)) {
1235 struct hisi_sas_slot *slot = task->lldd_task;
1236
1237 dev_err(dev, "abort tmf: TMF task timeout and not done\n");
1238 if (slot) {
1239 struct hisi_sas_cq *cq =
1240 &hisi_hba->cq[slot->dlvry_queue];
1241 /*
1242 * sync irq to avoid free'ing task
1243 * before using task in IO completion
1244 */
1245 synchronize_irq(cq->irq_no);
1246 slot->task = NULL;
1247 }
1248
1249 goto ex_err;
1250 } else
1251 dev_err(dev, "abort tmf: TMF task timeout\n");
1252 }
1253
1254 if (task->task_status.resp == SAS_TASK_COMPLETE &&
1255 task->task_status.stat == TMF_RESP_FUNC_COMPLETE) {
1256 res = TMF_RESP_FUNC_COMPLETE;
1257 break;
1258 }
1259
1260 if (task->task_status.resp == SAS_TASK_COMPLETE &&
1261 task->task_status.stat == TMF_RESP_FUNC_SUCC) {
1262 res = TMF_RESP_FUNC_SUCC;
1263 break;
1264 }
1265
1266 if (task->task_status.resp == SAS_TASK_COMPLETE &&
1267 task->task_status.stat == SAS_DATA_UNDERRUN) {
1268 /* no error, but return the number of bytes of
1269 * underrun
1270 */
1271 dev_warn(dev, "abort tmf: task to dev %016llx resp: 0x%x sts 0x%x underrun\n",
1272 SAS_ADDR(device->sas_addr),
1273 task->task_status.resp,
1274 task->task_status.stat);
1275 res = task->task_status.residual;
1276 break;
1277 }
1278
1279 if (task->task_status.resp == SAS_TASK_COMPLETE &&
1280 task->task_status.stat == SAS_DATA_OVERRUN) {
1281 dev_warn(dev, "abort tmf: blocked task error\n");
1282 res = -EMSGSIZE;
1283 break;
1284 }
1285
1286 if (task->task_status.resp == SAS_TASK_COMPLETE &&
1287 task->task_status.stat == SAS_OPEN_REJECT) {
1288 dev_warn(dev, "abort tmf: open reject failed\n");
1289 res = -EIO;
1290 } else {
1291 dev_warn(dev, "abort tmf: task to dev %016llx resp: 0x%x status 0x%x\n",
1292 SAS_ADDR(device->sas_addr),
1293 task->task_status.resp,
1294 task->task_status.stat);
1295 }
1296 sas_free_task(task);
1297 task = NULL;
1298 }
1299 ex_err:
1300 if (retry == TASK_RETRY)
1301 dev_warn(dev, "abort tmf: executing internal task failed!\n");
1302 sas_free_task(task);
1303 return res;
1304 }
1305

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org

#85 lore raw

RFC PATCH v3 37/41 libsas: add tag to struct sas_task

In reply to Hannes Reinecke

All block layer commands now have a tag, so we should be storing it in the sas_task structure for easier lookup.

---
 drivers/scsi/libsas/sas_ata.c       | 4 ++++
 drivers/scsi/libsas/sas_init.c      | 2 ++
 drivers/scsi/libsas/sas_scsi_host.c | 2 +-
 include/scsi/libsas.h               | 2 ++
 4 files changed, 9 insertions(+), 1 deletion(-)
drivers/scsi/libsas/sas_ata.c +4 -0
index 5d716d388707..897007343b3d 100644
@@ -211,6 +211,10 @@ static unsigned int sas_ata_qc_issue(struct ata_queued_cmd *qc)
 
 	task->data_dir = qc->dma_dir;
 	task->scatter = qc->sg;
+	if (qc->scsicmd)
+		task->tag = qc->scsicmd->request->tag;
+	else
+		task->tag = qc->tag;
 	task->ata_task.retry_count = 1;
 	task->task_state_flags = SAS_TASK_STATE_PENDING;
 	qc->lldd_task = task;
drivers/scsi/libsas/sas_init.c +2 -0
index 5aa8593b88b5..0d32cb49d0af 100644
@@ -53,6 +53,7 @@ struct sas_task *sas_alloc_slow_task(struct sas_ha_struct *ha,
 	if (!slow)
 		goto out_err_slow;
 
+	task->tag = -1;
 	if (shost->nr_reserved_cmds) {
 		struct scsi_device *sdev;
 
@@ -66,6 +67,7 @@ struct sas_task *sas_alloc_slow_task(struct sas_ha_struct *ha,
 		slow->scmd = scsi_get_reserved_cmd(sdev, DMA_NONE, false);
 		if (!slow->scmd)
 			goto out_err_scmd;
+		task->tag = slow->scmd->request->tag;
 		ASSIGN_SAS_TASK(slow->scmd, task);
 	}
 
drivers/scsi/libsas/sas_scsi_host.c +1 -1
index c5a430e3fa2d..585e0df5fce2 100644
@@ -149,7 +149,7 @@ static struct sas_task *sas_create_task(struct scsi_cmnd *cmd,
 	memcpy(task->ssp_task.LUN, &lun.scsi_lun, 8);
 	task->ssp_task.task_attr = TASK_ATTR_SIMPLE;
 	task->ssp_task.cmd = cmd;
-
+	task->tag = cmd->request->tag;
 	task->scatter = scsi_sglist(cmd);
 	task->num_scatter = scsi_sg_count(cmd);
 	task->total_xfer_len = scsi_bufflen(cmd);
include/scsi/libsas.h +2 -0
index c927228019c9..af864f68b5cc 100644
@@ -594,6 +594,8 @@ struct sas_task {
 	u32    total_xfer_len;
 	u8     data_dir:2;	  /* Use PCI_DMA_... */
 
+	u32    tag;
+
 	struct task_status_struct task_status;
 	void   (*task_done)(struct sas_task *);
 
Signature · 1 line
2.16.4
#86 lore raw

Re: [PATCH RFC v3 37/41] libsas: add tag to struct sas_task

Quoted 2 lines · Hannes Reinecke

All block layer commands now have a tag, so we should be storing it in the sas_task structure for easier lookup.

This seems like a decent idea, to put the tag here, so that we don't need to do the lookup in the LLDD queuecommand.

However it feels safer to use a sas_task scsicmd->request->tag in the LLDD directly, if we can make that possible. More below.

Quoted 14 lines
---
  drivers/scsi/libsas/sas_ata.c       | 4 ++++
  drivers/scsi/libsas/sas_init.c      | 2 ++
  drivers/scsi/libsas/sas_scsi_host.c | 2 +-
  include/scsi/libsas.h               | 2 ++
  4 files changed, 9 insertions(+), 1 deletion(-)
drivers/scsi/libsas/sas_ata.c +0 -0
index 5d716d388707..897007343b3d 100644

drivers/scsi/libsas/sas_ata.c static unsigned int sas_ata_qc_issue(struct ata_queued_cmd *qc) · line 217

Quoted 7 lines
drivers/scsi/libsas/sas_ata.c +4 -0
@@ -211,6 +211,10 @@ static unsigned int sas_ata_qc_issue(struct ata_queued_cmd *qc)
 
  	task->data_dir = qc->dma_dir;
  	task->scatter = qc->sg;
+	if (qc->scsicmd)
+		task->tag = qc->scsicmd->request->tag;
+	else
+		task->tag = qc->tag;

I think that this tag comes from ata_sas_allocate_tag(), and would be managed from yet another bitmap for ATA commands tags. Maybe we can allocate a sas slow task here, instead of using this tag. Needs more checking...

Quoted 41 lines
drivers/scsi/libsas/sas_ata.c +0 -0
  	task->ata_task.retry_count = 1;
  	task->task_state_flags = SAS_TASK_STATE_PENDING;
  	qc->lldd_task = task;
drivers/scsi/libsas/sas_init.c +2 -0
index 5aa8593b88b5..0d32cb49d0af 100644
@@ -53,6 +53,7 @@ struct sas_task *sas_alloc_slow_task(struct sas_ha_struct *ha,
  	if (!slow)
  		goto out_err_slow;
 
+	task->tag = -1;
  	if (shost->nr_reserved_cmds) {
  		struct scsi_device *sdev;
 
@@ -66,6 +67,7 @@ struct sas_task *sas_alloc_slow_task(struct sas_ha_struct *ha,
  		slow->scmd = scsi_get_reserved_cmd(sdev, DMA_NONE, false);
  		if (!slow->scmd)
  			goto out_err_scmd;
+		task->tag = slow->scmd->request->tag;
  		ASSIGN_SAS_TASK(slow->scmd, task);
  	}
 
drivers/scsi/libsas/sas_scsi_host.c +1 -1
index c5a430e3fa2d..585e0df5fce2 100644
@@ -149,7 +149,7 @@ static struct sas_task *sas_create_task(struct scsi_cmnd *cmd,
  	memcpy(task->ssp_task.LUN, &lun.scsi_lun, 8);
  	task->ssp_task.task_attr = TASK_ATTR_SIMPLE;
  	task->ssp_task.cmd = cmd;
-
+	task->tag = cmd->request->tag;
  	task->scatter = scsi_sglist(cmd);
  	task->num_scatter = scsi_sg_count(cmd);
  	task->total_xfer_len = scsi_bufflen(cmd);
include/scsi/libsas.h +0 -0
index c927228019c9..af864f68b5cc 100644

include/scsi/libsas.h struct sas_task { · line 597

Quoted 4 lines
include/scsi/libsas.h +1 -0
@@ -594,6 +594,8 @@ struct sas_task {
  	u32    total_xfer_len;
  	u8     data_dir:2;	  /* Use PCI_DMA_... */
 
+	u32    tag;

unsigned, yet we assign it -1?

include/scsi/libsas.h struct sas_task { · line 601

Quoted 5 lines
include/scsi/libsas.h +1 -0
+
  	struct task_status_struct task_status;
  	void   (*task_done)(struct sas_task *);
 

Thanks!

#87 lore raw

Re: [PATCH RFC v3 37/41] libsas: add tag to struct sas_task

Quoted 17 lines · John Garry
Quoted 2 lines · Hannes Reinecke

All block layer commands now have a tag, so we should be storing it in the sas_task structure for easier lookup.

This seems like a decent idea, to put the tag here, so that we don't need to do the lookup in the LLDD queuecommand.

However it feels safer to use a sas_task scsicmd->request->tag in the LLDD directly, if we can make that possible. More below.

Quoted 9 lines

---
  drivers/scsi/libsas/sas_ata.c       | 4 ++++
  drivers/scsi/libsas/sas_init.c      | 2 ++
  drivers/scsi/libsas/sas_scsi_host.c | 2 +-
  include/scsi/libsas.h               | 2 ++

  4 files changed, 9 insertions(+), 1 deletion(-)
Quoted 15 lines

drivers/scsi/libsas/sas_ata.c static unsigned int sas_ata_qc_issue(struct · line 217

Quoted 12 lines
drivers/scsi/libsas/sas_ata.c +4 -0
index 5d716d388707..897007343b3d 100644
@@ -211,6 +211,10 @@ static unsigned int sas_ata_qc_issue(struct 
 ata_queued_cmd *qc)
       task->data_dir = qc->dma_dir;
       task->scatter = qc->sg;
+    if (qc->scsicmd)
+        task->tag = qc->scsicmd->request->tag;
+    else
+        task->tag = qc->tag;

I think that this tag comes from ata_sas_allocate_tag(), and would be managed from yet another bitmap for ATA commands tags. Maybe we can allocate a sas slow task here, instead of using this tag. Needs more checking...

That is actually not a bad idea.
non-scsi ATA commands are typically internal commands, so for them a slow task should be sufficient. Plus ultimately the tags have to come from the same pool than the normal FS I/O, as they still do touch the same hardware.

So indeed, well spotted. I'll need to revisit this and see if one can't be using slow tasks here.

Quoted 38 lines
Quoted 38 lines
drivers/scsi/libsas/sas_ata.c +0 -0
       task->ata_task.retry_count = 1;
       task->task_state_flags = SAS_TASK_STATE_PENDING;
       qc->lldd_task = task;
drivers/scsi/libsas/sas_init.c +2 -0
index 5aa8593b88b5..0d32cb49d0af 100644
@@ -53,6 +53,7 @@ struct sas_task *sas_alloc_slow_task(struct 
 sas_ha_struct *ha,
       if (!slow)
           goto out_err_slow;
+    task->tag = -1;
       if (shost->nr_reserved_cmds) {
           struct scsi_device *sdev;
@@ -66,6 +67,7 @@ struct sas_task *sas_alloc_slow_task(struct 
 sas_ha_struct *ha,
           slow->scmd = scsi_get_reserved_cmd(sdev, DMA_NONE, false);
           if (!slow->scmd)
               goto out_err_scmd;
+        task->tag = slow->scmd->request->tag;
           ASSIGN_SAS_TASK(slow->scmd, task);
       }
drivers/scsi/libsas/sas_scsi_host.c +1 -1
index c5a430e3fa2d..585e0df5fce2 100644
@@ -149,7 +149,7 @@ static struct sas_task *sas_create_task(struct 
 scsi_cmnd *cmd,
       memcpy(task->ssp_task.LUN, &lun.scsi_lun, 8);
       task->ssp_task.task_attr = TASK_ATTR_SIMPLE;
       task->ssp_task.cmd = cmd;
-
+    task->tag = cmd->request->tag;
       task->scatter = scsi_sglist(cmd);
       task->num_scatter = scsi_sg_count(cmd);

      task->total_xfer_len = scsi_bufflen(cmd);

include/scsi/libsas.h +0 -0
index c927228019c9..af864f68b5cc 100644
Quoted 9 lines
Quoted 6 lines
include/scsi/libsas.h +0 -0
@@ -594,6 +594,8 @@ struct sas_task {
       u32    total_xfer_len;

      u8     data_dir:2;      /* Use PCI_DMA_... */ +    u32    tag;

unsigned, yet we assign it -1?

Yeah, that's how the block layer does internally, too. Maybe we should export SCSI_NO_TAG and use it here.

Cheers,

Hannes

Signature · 4 lines
Dr. Hannes Reinecke            Teamlead Storage & Networking
[email protected]                               +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer
#88 lore raw

Re: [PATCH RFC v3 37/41] libsas: add tag to struct sas_task

Quoted 16 lines
Quoted 7 lines
Quoted 7 lines
drivers/scsi/libsas/sas_scsi_host.c +1 -1
-
+    task->tag = cmd->request->tag;
       task->scatter = scsi_sglist(cmd);
       task->num_scatter = scsi_sg_count(cmd);
       task->total_xfer_len = scsi_bufflen(cmd);
include/scsi/libsas.h +0 -0
index c927228019c9..af864f68b5cc 100644
Quoted 9 lines

include/scsi/libsas.h struct sas_task { · line 596

Quoted 6 lines
include/scsi/libsas.h +1 -0
@@ -594,6 +594,8 @@ struct sas_task {
       u32    total_xfer_len;
       u8     data_dir:2;      /* Use PCI_DMA_... */
+    u32    tag;

unsigned, yet we assign it -1?

Quoted 3 lines

Yeah, that's how the block layer does internally, too. Maybe we should export SCSI_NO_TAG and use it here.

I think it's better that the LLDD would not have to deal with "no tag" scenario (pm8001 driver has to handle it in this series). Rather libsas can handle that, and fail an allocation of a slow_task to the LLDD instead.

Thanks,
John

#89 lore raw

Re: [PATCH RFC v3 37/41] libsas: add tag to struct sas_task

Quoted 16 lines · John Garry
Quoted 16 lines
Quoted 16 lines
Quoted 11 lines

-
+    task->tag = cmd->request->tag;
       task->scatter = scsi_sglist(cmd);
       task->num_scatter = scsi_sg_count(cmd);
       task->total_xfer_len = scsi_bufflen(cmd);

include/scsi/libsas.h +0 -0
index c927228019c9..af864f68b5cc 100644
@@ -594,6 +594,8 @@ struct sas_task {
        u32    total_xfer_len;
Quoted 2 lines

       u8     data_dir:2;      /* Use PCI_DMA_... */ +    u32    tag;

unsigned, yet we assign it -1?

Quoted 6 lines
Quoted 3 lines

Yeah, that's how the block layer does internally, too. Maybe we should export SCSI_NO_TAG and use it here.

I think it's better that the LLDD would not have to deal with "no tag" scenario (pm8001 driver has to handle it in this series). Rather libsas can handle that, and fail an allocation of a slow_task to the LLDD instead.

I fully agree. The 'no tag' scenario should never happen with libsas with this patchset.
But for that to happen we need to:
- Ensure that even ATA devices on libsas always have a domain device (I think it's true nowadays, but we'll need to check).
- Add a host port to libsas, such that sas_alloc_target() will have a valid target/port upon lookup (and your patch won't be needed anymore)
- Move ATA non-SCSI commands over to using sas slow tasks. This shouldn't be much of a problem as really the only non-SCSI ATA command which will need to be sent from within an I/O stream is the 'read log' command, and that is handled internally by the drivers anyway.
All other commands are sent during device discovery or from out-of-band things like ioctl, so they should be fine using slow tasks.

Cheers,

Hannes

Signature · 4 lines
Dr. Hannes Reinecke            Teamlead Storage & Networking
[email protected]                               +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer
#90 lore raw

Re: [PATCH RFC v3 37/41] libsas: add tag to struct sas_task

Quoted 21 lines · Hannes Reinecke
Quoted 20 lines · John Garry
Quoted 19 lines
Quoted 7 lines
Quoted 7 lines

-
+    task->tag = cmd->request->tag;
       task->scatter = scsi_sglist(cmd);
       task->num_scatter = scsi_sg_count(cmd);
       task->total_xfer_len = scsi_bufflen(cmd);

include/scsi/libsas.h +0 -0
index c927228019c9..af864f68b5cc 100644
Quoted 9 lines
Quoted 6 lines
include/scsi/libsas.h +0 -0
@@ -594,6 +594,8 @@ struct sas_task {
        u32    total_xfer_len;

       u8     data_dir:2;      /* Use PCI_DMA_... */ +    u32    tag;

unsigned, yet we assign it -1?

Yeah, that's how the block layer does internally, too. Maybe we should export SCSI_NO_TAG and use it here.

Quoted 6 lines
Quoted 2 lines

I think it's better that the LLDD would not have to deal with "no tag" scenario (pm8001 driver has to handle it in this series). Rather libsas can handle that, and fail an allocation of a slow_task to the LLDD instead.

I fully agree. The 'no tag' scenario should never happen with libsas with this patchset. But for that to happen we need to:
- Ensure that even ATA devices on libsas always have a domain device (I think it's true nowadays, but we'll need to check).

They should do. See references to sas_ata_init(), which is at the 2x points of ATA dev discovery - root port and expander remote port.

Quoted 2 lines

- Add a host port to libsas, such that sas_alloc_target() will have a valid target/port upon lookup (and your patch won't be needed anymore)

I think that they should already have for ATA devices.

Quoted 3 lines

- Move ATA non-SCSI commands over to using sas slow tasks. This
shouldn't be much of a problem as really the only non-SCSI ATA command which will need to be sent from within an I/O stream is the 'read log' command, and that is handled internally by the drivers anyway.

ok

Quoted 2 lines

All other commands are sent during device discovery or from out-of-band things like ioctl, so they should be fine using slow tasks.

ok

Cheers,
John

#91 lore raw

RFC PATCH v3 38/41 scsi: hisi_sas: Use libsas slow task SCSI command

In reply to Hannes Reinecke

From: John Garry <[email protected]>

Now that a SCSI command can be allocated for a libsas slow tasks, make the task prep code use it.

---
 drivers/scsi/hisi_sas/hisi_sas_main.c  | 5 ++++-
 drivers/scsi/hisi_sas/hisi_sas_v3_hw.c | 5 +++--
 2 files changed, 7 insertions(+), 3 deletions(-)
drivers/scsi/hisi_sas/hisi_sas_main.c +4 -1
index 991241ab87d1..2aa8a4124cfb 100644
@@ -477,8 +477,10 @@ static int hisi_sas_task_prep(struct sas_task *task,
 			} else {
 				scsi_cmnd = task->uldd_task;
 			}
+		} else if (task->slow_task) {
+			scsi_cmnd = task->slow_task->scmd;
 		}
-		rc  = hisi_sas_slot_index_alloc(hisi_hba, scsi_cmnd);
+		rc = hisi_sas_slot_index_alloc(hisi_hba, scsi_cmnd);
 	}
 	if (rc < 0)
 		goto err_out_dif_dma_unmap;
@@ -2670,6 +2672,7 @@ int hisi_sas_probe(struct platform_device *pdev,
 	} else {
 		shost->can_queue = HISI_SAS_UNRESERVED_IPTT;
 		shost->cmd_per_lun = HISI_SAS_UNRESERVED_IPTT;
+		shost->nr_reserved_cmds = HISI_SAS_RESERVED_IPTT;
 	}
 
 	sha->sas_ha_name = DRV_NAME;
drivers/scsi/hisi_sas/hisi_sas_v3_hw.c +3 -2
index 374885aa8d77..9c691c9a09f1 100644
@@ -3238,8 +3238,9 @@ hisi_sas_v3_probe(struct pci_dev *pdev, const struct pci_device_id *id)
 	shost->max_lun = ~0;
 	shost->max_channel = 1;
 	shost->max_cmd_len = 16;
-	shost->can_queue = HISI_SAS_UNRESERVED_IPTT;
-	shost->cmd_per_lun = HISI_SAS_UNRESERVED_IPTT;
+	shost->can_queue = HISI_SAS_MAX_COMMANDS;
+	shost->cmd_per_lun = HISI_SAS_MAX_COMMANDS;
+	shost->nr_reserved_cmds = HISI_SAS_RESERVED_IPTT;
 
 	sha->sas_ha_name = DRV_NAME;
 	sha->dev = dev;
Signature · 1 line
2.16.4
#92 lore raw

RFC PATCH v3 39/41 hisi_sas: use task tag to reference the slot

In reply to Hannes Reinecke

Use the task task to reference the command slot and drop the internal slot bitmap.

---
 drivers/scsi/hisi_sas/hisi_sas_main.c | 61 +++--------------------------------
 1 file changed, 5 insertions(+), 56 deletions(-)
drivers/scsi/hisi_sas/hisi_sas_main.c +5 -56
index 2aa8a4124cfb..b982097a2398 100644
@@ -172,41 +172,6 @@ static void hisi_sas_slot_index_free(struct hisi_hba *hisi_hba, int slot_idx)
 	}
 }
 
-static void hisi_sas_slot_index_set(struct hisi_hba *hisi_hba, int slot_idx)
-{
-	void *bitmap = hisi_hba->slot_index_tags;
-
-	set_bit(slot_idx, bitmap);
-}
-
-static int hisi_sas_slot_index_alloc(struct hisi_hba *hisi_hba,
-				     struct scsi_cmnd *scsi_cmnd)
-{
-	int index;
-	void *bitmap = hisi_hba->slot_index_tags;
-
-	if (scsi_cmnd)
-		return scsi_cmnd->request->tag;
-
-	spin_lock(&hisi_hba->lock);
-	index = find_next_zero_bit(bitmap, hisi_hba->slot_index_count,
-				   hisi_hba->last_slot_index + 1);
-	if (index >= hisi_hba->slot_index_count) {
-		index = find_next_zero_bit(bitmap,
-				hisi_hba->slot_index_count,
-				HISI_SAS_UNRESERVED_IPTT);
-		if (index >= hisi_hba->slot_index_count) {
-			spin_unlock(&hisi_hba->lock);
-			return -SAS_QUEUE_FULL;
-		}
-	}
-	hisi_sas_slot_index_set(hisi_hba, index);
-	hisi_hba->last_slot_index = index;
-	spin_unlock(&hisi_hba->lock);
-
-	return index;
-}
-
 static void hisi_sas_slot_index_init(struct hisi_hba *hisi_hba)
 {
 	int i;
@@ -465,23 +430,9 @@ static int hisi_sas_task_prep(struct sas_task *task,
 
 	if (hisi_hba->hw->slot_index_alloc)
 		rc = hisi_hba->hw->slot_index_alloc(hisi_hba, device);
-	else {
-		struct scsi_cmnd *scsi_cmnd = NULL;
-
-		if (task->uldd_task) {
-			struct ata_queued_cmd *qc;
+	else
+		rc = (task->tag == -1) ? -EINVAL : task->tag;
 
-			if (dev_is_sata(device)) {
-				qc = task->uldd_task;
-				scsi_cmnd = qc->scsicmd;
-			} else {
-				scsi_cmnd = task->uldd_task;
-			}
-		} else if (task->slow_task) {
-			scsi_cmnd = task->slow_task->scmd;
-		}
-		rc = hisi_sas_slot_index_alloc(hisi_hba, scsi_cmnd);
-	}
 	if (rc < 0)
 		goto err_out_dif_dma_unmap;
 
@@ -1970,12 +1921,10 @@ hisi_sas_internal_abort_task_exec(struct hisi_hba *hisi_hba, int device_id,
 
 	port = to_hisi_sas_port(sas_port);
 
-	/* simply get a slot and send abort command */
-	rc = hisi_sas_slot_index_alloc(hisi_hba, NULL);
-	if (rc < 0)
-		goto err_out;
+	slot_idx = task->tag;
+	if (task->tag == -1)
+		return -EAGAIN;
 
-	slot_idx = rc;
 	slot = &hisi_hba->slot_info[slot_idx];
 
 	spin_lock(&dq->lock);
Signature · 1 line
2.16.4
#93 lore raw

RFC PATCH v3 40/41 mv_sas: use reserved tags and drop private tag allocation

In reply to Hannes Reinecke

Switch to use reserved tags. With that all tags are maintained by the block layer and we can drop the private tag allocation.

---
 drivers/scsi/mvsas/mv_init.c | 10 +----
 drivers/scsi/mvsas/mv_sas.c  | 98 +++++++++-----------------------------------
 drivers/scsi/mvsas/mv_sas.h  | 13 +++---
 3 files changed, 27 insertions(+), 94 deletions(-)
drivers/scsi/mvsas/mv_init.c +1 -9
index 5973eed94938..17dab5aa966a 100644
@@ -141,7 +141,6 @@ static void mvs_free(struct mvs_info *mvi)
 		scsi_host_put(mvi->shost);
 	list_for_each_entry(mwq, &mvi->wq_list, entry)
 		cancel_delayed_work(&mwq->work_q);
-	kfree(mvi->tags);
 	kfree(mvi);
 }
 
@@ -286,10 +285,6 @@ static int mvs_alloc(struct mvs_info *mvi, struct Scsi_Host *shost)
 			printk(KERN_DEBUG "failed to create dma pool %s.\n", pool_name);
 			goto err_out;
 	}
-	mvi->tags_num = slot_nr;
-
-	/* Initialize tags */
-	mvs_tag_init(mvi);
 	return 0;
 err_out:
 	return 1;
@@ -372,10 +367,6 @@ static struct mvs_info *mvs_pci_alloc(struct pci_dev *pdev,
 	mvi->sas = sha;
 	mvi->shost = shost;
 
-	mvi->tags = kzalloc(MVS_CHIP_SLOT_SZ>>3, GFP_KERNEL);
-	if (!mvi->tags)
-		goto err_out;
-
 	if (MVS_CHIP_DISP->chip_ioremap(mvi))
 		goto err_out;
 	if (!mvs_alloc(mvi, shost))
@@ -474,6 +465,7 @@ static void  mvs_post_sas_ha_init(struct Scsi_Host *shost,
 	else
 		can_queue = MVS_CHIP_SLOT_SZ;
 
+	shost->nr_reserved_cmds = 2;
 	shost->sg_tablesize = min_t(u16, SG_ALL, MVS_MAX_SG);
 	shost->can_queue = can_queue;
 	mvi->shost->cmd_per_lun = MVS_QUEUE_SIZE;
drivers/scsi/mvsas/mv_sas.c +20 -78

201 lines not on this page · raw message

drivers/scsi/mvsas/mv_sas.h +6 -7
index 327fdd5ee962..480a5129eff4 100644
@@ -83,6 +83,12 @@ enum dev_reset {
 	MVS_PHY_TUNE	= 2,
 };
 
+enum mvs_slot_complete_flags {
+	MVS_SLOT_CMPL_NONE = 0,
+	MVS_SLOT_CMPL_PHY_GONE = 1,
+	MVS_SLOT_CMPL_SLOT_RESET = 2,
+};
+
 struct mvs_info;
 struct mvs_prv_info;
 
@@ -370,8 +376,6 @@ struct mvs_info {
 	u32 chip_id;
 	const struct mvs_chip_info *chip;
 
-	int tags_num;
-	unsigned long *tags;
 	/* further per-slot information */
 	struct mvs_phy phy[MVS_MAX_PHYS];
 	struct mvs_port port[MVS_MAX_PHYS];
@@ -424,11 +428,6 @@ struct mvs_task_exec_info {
 
 /******************** function prototype *********************/
 void mvs_get_sas_addr(void *buf, u32 buflen);
-void mvs_tag_clear(struct mvs_info *mvi, u32 tag);
-void mvs_tag_free(struct mvs_info *mvi, u32 tag);
-void mvs_tag_set(struct mvs_info *mvi, unsigned int tag);
-int mvs_tag_alloc(struct mvs_info *mvi, u32 *tag_out);
-void mvs_tag_init(struct mvs_info *mvi);
 void mvs_iounmap(void __iomem *regs);
 int mvs_ioremap(struct mvs_info *mvi, int bar, int bar_ex);
 void mvs_phys_reset(struct mvs_info *mvi, u32 phy_mask, int hard);
Signature · 1 line
2.16.4
#94 lore raw

RFC PATCH v3 41/41 pm8001: use block-layer tags for ccb allocation

In reply to Hannes Reinecke

Always allocate a command from the block layer whenever we need a tag. With that we can drop the internal ccb bitmap and allow the block layer control tag allocation.

---
 drivers/scsi/pm8001/pm8001_hwi.c  | 110 ++++++++++++++++++--------------------
 drivers/scsi/pm8001/pm8001_init.c |  30 ++++++-----
 drivers/scsi/pm8001/pm8001_sas.c  | 106 ++++++++++++++----------------------
 drivers/scsi/pm8001/pm8001_sas.h  |  13 +++--
 drivers/scsi/pm8001/pm80xx_hwi.c  | 107 +++++++++++++++++-------------------
 5 files changed, 165 insertions(+), 201 deletions(-)
drivers/scsi/pm8001/pm8001_hwi.c +53 -57
index 62208e98508c..3b0dfeee8563 100644
@@ -1579,11 +1579,11 @@ void pm8001_work_fn(struct work_struct *work)
 				" done with event 0x%x resp 0x%x stat 0x%x but"
 				" aborted by upper layer!\n",
 				t, pw->handler, ts->resp, ts->stat));
-			pm8001_ccb_task_free(pm8001_ha, t, ccb, tag);
+			pm8001_ccb_task_free(pm8001_ha, t, ccb);
 			spin_unlock_irqrestore(&pm8001_ha->lock, flags);
 		} else {
 			spin_unlock_irqrestore(&t->task_state_lock, flags1);
-			pm8001_ccb_task_free(pm8001_ha, t, ccb, tag);
+			pm8001_ccb_task_free(pm8001_ha, t, ccb);
 			mb();/* in order to force CPU ordering */
 			spin_unlock_irqrestore(&pm8001_ha->lock, flags);
 			t->task_done(t);
@@ -1729,8 +1729,6 @@ int pm8001_handle_event(struct pm8001_hba_info *pm8001_ha, void *data,
 static void pm8001_send_abort_all(struct pm8001_hba_info *pm8001_ha,
 		struct pm8001_device *pm8001_ha_dev)
 {
-	int res;
-	u32 ccb_tag;
 	struct pm8001_ccb_info *ccb;
 	struct sas_task *task = NULL;
 	struct task_abort_req task_abort;
@@ -1756,13 +1754,16 @@ static void pm8001_send_abort_all(struct pm8001_hba_info *pm8001_ha,
 
 	task->task_done = pm8001_task_done;
 
-	res = pm8001_tag_alloc(pm8001_ha, &ccb_tag);
-	if (res)
+	if (task->tag == -1) {
+		sas_free_task(task);
+		PM8001_FAIL_DBG(pm8001_ha,
+			pm8001_printk("cannot allocate tag\n"));
 		return;
+	}
 
-	ccb = &pm8001_ha->ccb_info[ccb_tag];
+	ccb = &pm8001_ha->ccb_info[task->tag];
 	ccb->device = pm8001_ha_dev;
-	ccb->ccb_tag = ccb_tag;
+	ccb->ccb_tag = task->tag;
 	ccb->task = task;
 
 	circularQ = &pm8001_ha->inbnd_q_tbl[0];
@@ -1770,13 +1771,12 @@ static void pm8001_send_abort_all(struct pm8001_hba_info *pm8001_ha,
 	memset(&task_abort, 0, sizeof(task_abort));
 	task_abort.abort_all = cpu_to_le32(1);
 	task_abort.device_id = cpu_to_le32(pm8001_ha_dev->device_id);
-	task_abort.tag = cpu_to_le32(ccb_tag);
+	task_abort.tag = cpu_to_le32(task->tag);
 
 	ret = pm8001_mpi_build_cmd(pm8001_ha, circularQ, opc, &task_abort,
 			sizeof(task_abort), 0);
 	if (ret)
-		pm8001_tag_free(pm8001_ha, ccb_tag);
-
+		sas_free_task(task);
 }
 
 static void pm8001_send_read_log(struct pm8001_hba_info *pm8001_ha,
@@ -1784,7 +1784,6 @@ static void pm8001_send_read_log(struct pm8001_hba_info *pm8001_ha,
 {
 	struct sata_start_req sata_cmd;
 	int res;
-	u32 ccb_tag;
 	struct pm8001_ccb_info *ccb;
 	struct sas_task *task = NULL;
 	struct host_to_dev_fis fis;
@@ -1803,8 +1802,7 @@ static void pm8001_send_read_log(struct pm8001_hba_info *pm8001_ha,
 	}
 	task->task_done = pm8001_task_done;
 
-	res = pm8001_tag_alloc(pm8001_ha, &ccb_tag);
-	if (res) {
+	if (task->tag == -1) {
 		sas_free_task(task);
 		PM8001_FAIL_DBG(pm8001_ha,
 			pm8001_printk("cannot allocate tag !!!\n"));
@@ -1814,9 +1812,9 @@ static void pm8001_send_read_log(struct pm8001_hba_info *pm8001_ha,
 	task->dev = dev;
 	task->dev->lldd_dev = pm8001_ha_dev;
 
-	ccb = &pm8001_ha->ccb_info[ccb_tag];
+	ccb = &pm8001_ha->ccb_info[task->tag];
 	ccb->device = pm8001_ha_dev;
-	ccb->ccb_tag = ccb_tag;
+	ccb->ccb_tag = task->tag;
 	ccb->task = task;
 	pm8001_ha_dev->id |= NCQ_READ_LOG_FLAG;
 	pm8001_ha_dev->id |= NCQ_2ND_RLE_FLAG;
@@ -1832,17 +1830,15 @@ static void pm8001_send_read_log(struct pm8001_hba_info *pm8001_ha,
 	fis.lbal = 0x10;
 	fis.sector_count = 0x1;
 
-	sata_cmd.tag = cpu_to_le32(ccb_tag);
+	sata_cmd.tag = cpu_to_le32(task->tag);
 	sata_cmd.device_id = cpu_to_le32(pm8001_ha_dev->device_id);
 	sata_cmd.ncqtag_atap_dir_m |= ((0x1 << 7) | (0x5 << 9));
 	memcpy(&sata_cmd.sata_fis, &fis, sizeof(struct host_to_dev_fis));
 
 	res = pm8001_mpi_build_cmd(pm8001_ha, circularQ, opc, &sata_cmd,
 			sizeof(sata_cmd), 0);
-	if (res) {
+	if (res)
 		sas_free_task(task);
-		pm8001_tag_free(pm8001_ha, ccb_tag);
-	}
 }
 
 /**
@@ -2103,10 +2099,10 @@ mpi_ssp_completion(struct pm8001_hba_info *pm8001_ha , void *piomb)
 			" io_status 0x%x resp 0x%x "
 			"stat 0x%x but aborted by upper layer!\n",
 			t, status, ts->resp, ts->stat));
-		pm8001_ccb_task_free(pm8001_ha, t, ccb, tag);
+		pm8001_ccb_task_free(pm8001_ha, t, ccb);
 	} else {
 		spin_unlock_irqrestore(&t->task_state_lock, flags);
-		pm8001_ccb_task_free(pm8001_ha, t, ccb, tag);
+		pm8001_ccb_task_free(pm8001_ha, t, ccb);
 		mb();/* in order to force CPU ordering */
 		t->task_done(t);
 	}
@@ -2291,10 +2287,10 @@ static void mpi_ssp_event(struct pm8001_hba_info *pm8001_ha , void *piomb)
 			" event 0x%x resp 0x%x "
 			"stat 0x%x but aborted by upper layer!\n",
 			t, event, ts->resp, ts->stat));
-		pm8001_ccb_task_free(pm8001_ha, t, ccb, tag);
+		pm8001_ccb_task_free(pm8001_ha, t, ccb);
 	} else {
 		spin_unlock_irqrestore(&t->task_state_lock, flags);
-		pm8001_ccb_task_free(pm8001_ha, t, ccb, tag);
+		pm8001_ccb_task_free(pm8001_ha, t, ccb);
 		mb();/* in order to force CPU ordering */
 		t->task_done(t);
 	}
@@ -2423,8 +2419,6 @@ mpi_sata_completion(struct pm8001_hba_info *pm8001_ha, void *piomb)
 				/* clear bit for read log */
 				pm8001_dev->id = pm8001_dev->id & 0x7FFFFFFF;
 				pm8001_send_abort_all(pm8001_ha, pm8001_dev);
-				/* Free the tag */
-				pm8001_tag_free(pm8001_ha, tag);
 				sas_free_task(t);
 				return;
 			}
@@ -2534,7 +2528,7 @@ mpi_sata_completion(struct pm8001_hba_info *pm8001_ha, void *piomb)
 				IO_OPEN_CNX_ERROR_IT_NEXUS_LOSS);
 			ts->resp = SAS_TASK_UNDELIVERED;
 			ts->stat = SAS_QUEUE_FULL;
-			pm8001_ccb_task_free_done(pm8001_ha, t, ccb, tag);
+			pm8001_ccb_task_free_done(pm8001_ha, t, ccb);
 			return;
 		}
 		break;
@@ -2550,7 +2544,7 @@ mpi_sata_completion(struct pm8001_hba_info *pm8001_ha, void *piomb)
 				IO_OPEN_CNX_ERROR_IT_NEXUS_LOSS);
 			ts->resp = SAS_TASK_UNDELIVERED;
 			ts->stat = SAS_QUEUE_FULL;
-			pm8001_ccb_task_free_done(pm8001_ha, t, ccb, tag);
+			pm8001_ccb_task_free_done(pm8001_ha, t, ccb);
 			return;
 		}
 		break;
@@ -2574,7 +2568,7 @@ mpi_sata_completion(struct pm8001_hba_info *pm8001_ha, void *piomb)
 				IO_OPEN_CNX_ERROR_STP_RESOURCES_BUSY);
 			ts->resp = SAS_TASK_UNDELIVERED;
 			ts->stat = SAS_QUEUE_FULL;
-			pm8001_ccb_task_free_done(pm8001_ha, t, ccb, tag);
+			pm8001_ccb_task_free_done(pm8001_ha, t, ccb);
 			return;
 		}
 		break;
@@ -2637,7 +2631,7 @@ mpi_sata_completion(struct pm8001_hba_info *pm8001_ha, void *piomb)
 				    IO_DS_NON_OPERATIONAL);
 			ts->resp = SAS_TASK_UNDELIVERED;
 			ts->stat = SAS_QUEUE_FULL;
-			pm8001_ccb_task_free_done(pm8001_ha, t, ccb, tag);
+			pm8001_ccb_task_free_done(pm8001_ha, t, ccb);
 			return;
 		}
 		break;
@@ -2657,7 +2651,7 @@ mpi_sata_completion(struct pm8001_hba_info *pm8001_ha, void *piomb)
 				    IO_DS_IN_ERROR);
 			ts->resp = SAS_TASK_UNDELIVERED;
 			ts->stat = SAS_QUEUE_FULL;
-			pm8001_ccb_task_free_done(pm8001_ha, t, ccb, tag);
+			pm8001_ccb_task_free_done(pm8001_ha, t, ccb);
 			return;
 		}
 		break;
@@ -2686,10 +2680,10 @@ mpi_sata_completion(struct pm8001_hba_info *pm8001_ha, void *piomb)
 			pm8001_printk("task 0x%p done with io_status 0x%x"
 			" resp 0x%x stat 0x%x but aborted by upper layer!\n",
 			t, status, ts->resp, ts->stat));
-		pm8001_ccb_task_free(pm8001_ha, t, ccb, tag);
+		pm8001_ccb_task_free(pm8001_ha, t, ccb);
 	} else {
 		spin_unlock_irqrestore(&t->task_state_lock, flags);
-		pm8001_ccb_task_free_done(pm8001_ha, t, ccb, tag);
+		pm8001_ccb_task_free_done(pm8001_ha, t, ccb);
 	}
 }
 
@@ -2798,7 +2792,7 @@ static void mpi_sata_event(struct pm8001_hba_info *pm8001_ha , void *piomb)
 				IO_OPEN_CNX_ERROR_IT_NEXUS_LOSS);
 			ts->resp = SAS_TASK_COMPLETE;
 			ts->stat = SAS_QUEUE_FULL;
-			pm8001_ccb_task_free_done(pm8001_ha, t, ccb, tag);
+			pm8001_ccb_task_free_done(pm8001_ha, t, ccb);
 			return;
 		}
 		break;
@@ -2906,10 +2900,10 @@ static void mpi_sata_event(struct pm8001_hba_info *pm8001_ha , void *piomb)
 			pm8001_printk("task 0x%p done with io_status 0x%x"
 			" resp 0x%x stat 0x%x but aborted by upper layer!\n",
 			t, event, ts->resp, ts->stat));
-		pm8001_ccb_task_free(pm8001_ha, t, ccb, tag);
+		pm8001_ccb_task_free(pm8001_ha, t, ccb);
 	} else {
 		spin_unlock_irqrestore(&t->task_state_lock, flags);
-		pm8001_ccb_task_free_done(pm8001_ha, t, ccb, tag);
+		pm8001_ccb_task_free_done(pm8001_ha, t, ccb);
 	}
 }
 
@@ -3108,10 +3102,10 @@ mpi_smp_completion(struct pm8001_hba_info *pm8001_ha, void *piomb)
 			" io_status 0x%x resp 0x%x "
 			"stat 0x%x but aborted by upper layer!\n",
 			t, status, ts->resp, ts->stat));
-		pm8001_ccb_task_free(pm8001_ha, t, ccb, tag);
+		pm8001_ccb_task_free(pm8001_ha, t, ccb);
 	} else {
 		spin_unlock_irqrestore(&t->task_state_lock, flags);
-		pm8001_ccb_task_free(pm8001_ha, t, ccb, tag);
+		pm8001_ccb_task_free(pm8001_ha, t, ccb);
 		mb();/* in order to force CPU ordering */
 		t->task_done(t);
 	}
@@ -3791,11 +3785,10 @@ int pm8001_mpi_task_abort_resp(struct pm8001_hba_info *pm8001_ha, void *piomb)
 	t->task_state_flags &= ~SAS_TASK_AT_INITIATOR;
 	t->task_state_flags |= SAS_TASK_STATE_DONE;
 	spin_unlock_irqrestore(&t->task_state_lock, flags);
-	pm8001_ccb_task_free(pm8001_ha, t, ccb, tag);
+	pm8001_ccb_task_free(pm8001_ha, t, ccb);
 	mb();
 
 	if (pm8001_dev->id & NCQ_ABORT_ALL_FLAG) {
-		pm8001_tag_free(pm8001_ha, tag);
 		sas_free_task(t);
 		/* clear the flag */
 		pm8001_dev->id &= 0xBFFFFFFF;
@@ -4474,12 +4467,11 @@ static int pm8001_chip_sata_req(struct pm8001_hba_info *pm8001_ha,
 					pm8001_printk("task 0x%p resp 0x%x "
 					" stat 0x%x but aborted by upper layer "
 					"\n", task, ts->resp, ts->stat));
-				pm8001_ccb_task_free(pm8001_ha, task, ccb, tag);
+				pm8001_ccb_task_free(pm8001_ha, task, ccb);
 			} else {
 				spin_unlock_irqrestore(&task->task_state_lock,
 							flags);
-				pm8001_ccb_task_free_done(pm8001_ha, task,
-								ccb, tag);
+				pm8001_ccb_task_free_done(pm8001_ha, task, ccb);
 				return 0;
 			}
 		}
@@ -4570,9 +4562,9 @@ static int pm8001_chip_reg_dev_req(struct pm8001_hba_info *pm8001_ha,
 	circularQ = &pm8001_ha->inbnd_q_tbl[0];
 
 	memset(&payload, 0, sizeof(payload));
-	rc = pm8001_tag_alloc(pm8001_ha, &tag);
-	if (rc)
-		return rc;
+	tag = pm8001_tag_alloc(pm8001_ha, dev);
+	if (tag == -1)
+		return -SAS_QUEUE_FULL;
 	ccb = &pm8001_ha->ccb_info[tag];
 	ccb->device = pm8001_dev;
 	ccb->ccb_tag = tag;
@@ -4606,6 +4598,8 @@ static int pm8001_chip_reg_dev_req(struct pm8001_hba_info *pm8001_ha,
 		SAS_ADDR_SIZE);
 	rc = pm8001_mpi_build_cmd(pm8001_ha, circularQ, opc, &payload,
 			sizeof(payload), 0);
+	if (rc)
+		pm8001_tag_free(pm8001_ha, tag);
 	return rc;
 }
 
@@ -4789,10 +4783,10 @@ int pm8001_chip_get_nvmd_req(struct pm8001_hba_info *pm8001_ha,
 	fw_control_context->len = ioctl_payload->rd_length;
 	circularQ = &pm8001_ha->inbnd_q_tbl[0];
 	memset(&nvmd_req, 0, sizeof(nvmd_req));
-	rc = pm8001_tag_alloc(pm8001_ha, &tag);
-	if (rc) {
+	tag = pm8001_tag_alloc(pm8001_ha, NULL);
+	if (tag == -1) {
 		kfree(fw_control_context);
-		return rc;
+		return -SAS_QUEUE_FULL;
 	}
 	ccb = &pm8001_ha->ccb_info[tag];
 	ccb->ccb_tag = tag;
@@ -4885,8 +4879,8 @@ int pm8001_chip_set_nvmd_req(struct pm8001_hba_info *pm8001_ha,
 		&ioctl_payload->func_specific,
 		ioctl_payload->wr_length);
 	memset(&nvmd_req, 0, sizeof(nvmd_req));
-	rc = pm8001_tag_alloc(pm8001_ha, &tag);
-	if (rc) {
+	tag = pm8001_tag_alloc(pm8001_ha, NULL);
+	if (tag == -1) {
 		kfree(fw_control_context);
 		return -EBUSY;
 	}
@@ -5011,8 +5005,8 @@ pm8001_chip_fw_flash_update_req(struct pm8001_hba_info *pm8001_ha,
 	fw_control_context->virtAddr = buffer;
 	fw_control_context->phys_addr = phys_addr;
 	fw_control_context->len = fw_control->len;
-	rc = pm8001_tag_alloc(pm8001_ha, &tag);
-	if (rc) {
+	tag = pm8001_tag_alloc(pm8001_ha, NULL);
+	if (tag == -1) {
 		kfree(fw_control_context);
 		return -EBUSY;
 	}
@@ -5113,8 +5107,8 @@ pm8001_chip_set_dev_state_req(struct pm8001_hba_info *pm8001_ha,
 	u32 tag;
 	u32 opc = OPC_INB_SET_DEVICE_STATE;
 	memset(&payload, 0, sizeof(payload));
-	rc = pm8001_tag_alloc(pm8001_ha, &tag);
-	if (rc)
+	tag = pm8001_tag_alloc(pm8001_ha, pm8001_dev->sas_device);
+	if (tag == -1)
 		return -1;
 	ccb = &pm8001_ha->ccb_info[tag];
 	ccb->ccb_tag = tag;
@@ -5125,6 +5119,8 @@ pm8001_chip_set_dev_state_req(struct pm8001_hba_info *pm8001_ha,
 	payload.nds = cpu_to_le32(state);
 	rc = pm8001_mpi_build_cmd(pm8001_ha, circularQ, opc, &payload,
 			sizeof(payload), 0);
+	if (rc)
+		pm8001_tag_free(pm8001_ha, tag);
 	return rc;
 
 }
@@ -5139,8 +5135,8 @@ pm8001_chip_sas_re_initialization(struct pm8001_hba_info *pm8001_ha)
 	u32 tag;
 	u32 opc = OPC_INB_SAS_RE_INITIALIZE;
 	memset(&payload, 0, sizeof(payload));
-	rc = pm8001_tag_alloc(pm8001_ha, &tag);
-	if (rc)
+	tag = pm8001_tag_alloc(pm8001_ha, NULL);
+	if (tag == -1)
 		return -ENOMEM;
 	ccb = &pm8001_ha->ccb_info[tag];
 	ccb->ccb_tag = tag;
drivers/scsi/pm8001/pm8001_init.c +16 -14
index a8f5344fdfda..31b22169a477 100644
@@ -176,7 +176,6 @@ static void pm8001_free(struct pm8001_hba_info *pm8001_ha)
 	}
 	PM8001_CHIP_DISP->chip_iounmap(pm8001_ha);
 	flush_workqueue(pm8001_wq);
-	kfree(pm8001_ha->tags);
 	kfree(pm8001_ha);
 }
 
@@ -260,11 +259,10 @@ static u32 pm8001_request_irq(struct pm8001_hba_info *pm8001_ha);
  *
  */
 static int pm8001_alloc(struct pm8001_hba_info *pm8001_ha,
-			const struct pci_device_id *ent)
+			 const struct pci_device_id *ent)
 {
 	int i;
 	spin_lock_init(&pm8001_ha->lock);
-	spin_lock_init(&pm8001_ha->bitmap_lock);
 	PM8001_INIT_DBG(pm8001_ha,
 		pm8001_printk("pm8001_alloc: PHY:%x\n",
 				pm8001_ha->chip->n_phy));
@@ -276,9 +274,6 @@ static int pm8001_alloc(struct pm8001_hba_info *pm8001_ha,
 		INIT_LIST_HEAD(&pm8001_ha->port[i].list);
 	}
 
-	pm8001_ha->tags = kzalloc(PM8001_MAX_CCB, GFP_KERNEL);
-	if (!pm8001_ha->tags)
-		goto err_out;
 	/* MPI Memory region 1 for AAP Event Log for fw */
 	pm8001_ha->memoryMap.region[AAP1].num_elements = 1;
 	pm8001_ha->memoryMap.region[AAP1].element_size = PM8001_EVENT_LOG_SIZE;
@@ -378,7 +373,7 @@ static int pm8001_alloc(struct pm8001_hba_info *pm8001_ha,
 				PM8001_FAIL_DBG(pm8001_ha,
 					pm8001_printk("Mem%d alloc failed\n",
 					i));
-				goto err_out;
+				return false;
 		}
 	}
 
@@ -400,11 +395,7 @@ static int pm8001_alloc(struct pm8001_hba_info *pm8001_ha,
 		++pm8001_ha->tags_num;
 	}
 	pm8001_ha->flags = PM8001F_INIT_TIME;
-	/* Initialize tags */
-	pm8001_tag_init(pm8001_ha);
-	return 0;
-err_out:
-	return 1;
+	return true;
 }
 
 /**
@@ -589,7 +580,7 @@ static int pm8001_prep_sas_ha_init(struct Scsi_Host *shost,
  * @shost: scsi host which has been allocated outside
  * @chip_info: our ha struct.
  */
-static void  pm8001_post_sas_ha_init(struct Scsi_Host *shost,
+static bool  pm8001_post_sas_ha_init(struct Scsi_Host *shost,
 				     const struct pm8001_chip_info *chip_info)
 {
 	int i = 0;
@@ -610,6 +601,13 @@ static void  pm8001_post_sas_ha_init(struct Scsi_Host *shost,
 	sha->sas_addr = &pm8001_ha->sas_addr[0];
 	sha->num_phys = chip_info->n_phy;
 	sha->core.shost = shost;
+
+	pm8001_ha->host_dev = scsi_get_virtual_dev(pm8001_ha->shost, 0,
+						   PM8001_MAX_DEVICES);
+	if (!pm8001_ha->host_dev)
+		return false;
+
+	return true;
 }
 
 /**
@@ -1108,7 +1106,11 @@ static int pm8001_pci_probe(struct pci_dev *pdev,
 	if (pm8001_configure_phy_settings(pm8001_ha))
 		goto err_out_shost;
 
-	pm8001_post_sas_ha_init(shost, chip);
+	if (!pm8001_post_sas_ha_init(shost, chip)) {
+		PM8001_FAIL_DBG(pm8001_ha, pm8001_printk(
+					"pm8001_post_sas_ha_init failed\n"));
+		goto err_out_shost;
+	}
 	rc = sas_register_ha(SHOST_TO_SAS_HA(shost));
 	if (rc) {
 		PM8001_FAIL_DBG(pm8001_ha, pm8001_printk(
drivers/scsi/pm8001/pm8001_sas.c +40 -66

228 lines not on this page · raw message

drivers/scsi/pm8001/pm8001_sas.h +6 -7
index ae7ba9b3c4bc..b77fb1e1a167 100644
@@ -499,7 +499,6 @@ struct pm8001_hba_info {
 	struct list_head	list;
 	unsigned long		flags;
 	spinlock_t		lock;/* host-wide lock */
-	spinlock_t		bitmap_lock;
 	struct pci_dev		*pdev;/* our device */
 	struct device		*dev;
 	struct pm8001_hba_memspace io_mem[6];
@@ -531,11 +530,11 @@ struct pm8001_hba_info {
 	u8			sas_addr[SAS_ADDR_SIZE];
 	struct sas_ha_struct	*sas;/* SCSI/SAS glue */
 	struct Scsi_Host	*shost;
+	struct scsi_device	*host_dev;
 	u32			chip_id;
 	const struct pm8001_chip_info	*chip;
 	struct completion	*nvmd_completion;
 	int			tags_num;
-	unsigned long		*tags;
 	struct pm8001_phy	phy[PM8001_MAX_PHYS];
 	struct pm8001_port	port[PM8001_MAX_PHYS];
 	u32			id;
@@ -652,11 +651,12 @@ struct fw_control_ex {
 extern struct workqueue_struct *pm8001_wq;
 
 /******************** function prototype *********************/
-int pm8001_tag_alloc(struct pm8001_hba_info *pm8001_ha, u32 *tag_out);
+u32 pm8001_tag_alloc(struct pm8001_hba_info *pm8001_ha,
+		     struct domain_device *dev);
 void pm8001_tag_init(struct pm8001_hba_info *pm8001_ha);
 u32 pm8001_get_ncq_tag(struct sas_task *task, u32 *tag);
 void pm8001_ccb_task_free(struct pm8001_hba_info *pm8001_ha,
-	struct sas_task *task, struct pm8001_ccb_info *ccb, u32 ccb_idx);
+	struct sas_task *task, struct pm8001_ccb_info *ccb);
 int pm8001_phy_control(struct asd_sas_phy *sas_phy, enum phy_func func,
 	void *funcdata);
 void pm8001_scan_start(struct Scsi_Host *shost);
@@ -752,10 +752,9 @@ extern struct device_attribute *pm8001_host_attrs[];
 
 static inline void
 pm8001_ccb_task_free_done(struct pm8001_hba_info *pm8001_ha,
-			struct sas_task *task, struct pm8001_ccb_info *ccb,
-			u32 ccb_idx)
+			struct sas_task *task, struct pm8001_ccb_info *ccb)
 {
-	pm8001_ccb_task_free(pm8001_ha, task, ccb, ccb_idx);
+	pm8001_ccb_task_free(pm8001_ha, task, ccb);
 	smp_mb(); /*in order to force CPU ordering*/
 	spin_unlock(&pm8001_ha->lock);
 	task->task_done(task);
drivers/scsi/pm8001/pm80xx_hwi.c +50 -57

305 lines not on this page · raw message

Signature · 1 line
2.16.4
#95 lore raw

Re: [PATCH RFC v3 41/41] pm8001: use block-layer tags for ccb allocation

On 30/04/2020 14:19, Hannes Reinecke wrote:

Making this change is good.

Quoted 20 lines

/**
* pm8001_tag_alloc - allocate a empty tag for task used.
* @pm8001_ha: our hba struct

drivers/scsi/pm8001/pm8001_sas.c +5 -9
-  * @tag_out: the found empty tag .
+  * @dev: device from which the tag should be allocated or NULL
    */
-inline int pm8001_tag_alloc(struct pm8001_hba_info *pm8001_ha, u32 *tag_out)
+inline u32 pm8001_tag_alloc(struct pm8001_hba_info *pm8001_ha,
+			    struct domain_device *dev)
  {
-	unsigned int tag;
-	void *bitmap = pm8001_ha->tags;
-	unsigned long flags;
+	struct sas_task *task;
+	struct scsi_lun lun;
 
-	spin_lock_irqsave(&pm8001_ha->bitmap_lock, flags);
-	tag = find_first_zero_bit(bitmap, pm8001_ha->tags_num);
-	if (tag >= pm8001_ha->tags_num) {
-		spin_unlock_irqrestore(&pm8001_ha->bitmap_lock, flags);

drivers/scsi/pm8001/pm8001_sas.c static int pm8001_find_tag(struct sas_task *task, u32 *tag) · line 73

Quoted 10 lines
drivers/scsi/pm8001/pm8001_sas.c +3 -7
-		return -SAS_QUEUE_FULL;
-	}
-	set_bit(tag, bitmap);
-	spin_unlock_irqrestore(&pm8001_ha->bitmap_lock, flags);
-	*tag_out = tag;
-	return 0;
-}
+	int_to_scsilun(0, &lun);
+	task = sas_alloc_slow_task(pm8001_ha->sas, dev,
+				   &lun, GFP_KERNEL);

According to the current code in sas_alloc_slow_task(), we should now set pm8001 shost->nr_reserved_cmds for this to work. Or always call scsi_device_reserved_cmd() in sas_alloc_slow_task().

So even though the current code does not reserve commands, I would prefer if it did, as good practice.

Thanks,
John

Quoted 12 lines
drivers/scsi/pm8001/pm8001_sas.c +3 -5
+	if (!task)
+		return -1;
 
-void pm8001_tag_init(struct pm8001_hba_info *pm8001_ha)
-{
-	int i;
-	for (i = 0; i < pm8001_ha->tags_num; ++i)
-		pm8001_tag_free(pm8001_ha, i);
+	return task->tag;
  }
 
   /**
#96 lore raw
In reply to Hannes Reinecke

On 30/04/2020 14:18, Hannes Reinecke wrote:

Thanks for this.

Quoted 3 lines

Conversion of the SAS drivers hisi_sas, pm8001, and mv_sas are compile tested only; I'd be grateful if someone could give these patches a spin on that hardware, too.

So after some build fixups, I get this a NULL pointer deref:

[ 5.565899]  sas_find_dev_by_rphy+0x3c/0x104
[ 5.570182]  sas_target_alloc+0x18/0x84
[ 5.574029]  scsi_alloc_target+0x20c/0x304
[ 5.578136]  scsi_get_virtual_dev+0x44/0xec
[ 5.582331]  sas_register_ha+0xd0/0x258
[ 5.586178]  hisi_sas_probe+0x2ec/0x36c
[ 5.590024]  hisi_sas_v2_probe+0x34/0x64
[ 5.593958]  platform_drv_probe+0x4c/0xa0
[ 5.597978]  really_probe+0xd8/0x334
[ 5.601561]  driver_probe_device+0x58/0xe8
[ 5.605669]  device_driver_attach+0x68/0x70
[ 5.609864]  __driver_attach+0x9c/0xf8
[ 5.613622]  bus_for_each_dev+0x50/0xa0
[ 5.617468]  driver_attach+0x20/0x28
[ 5.621051]  bus_add_driver+0x148/0x1fc
[ 5.624897]  driver_register+0x6c/0x124
[ 5.628742]  __platform_driver_register+0x48/0x50
[ 5.633463]  hisi_sas_v2_driver_init+0x18/0x20
[ 5.637921]  do_one_initcall+0x50/0x194
[ 5.641767]  kernel_init_freeable+0x1e4/0x24c

And so we need this change:

commit 51f607bf91853026af102367d9e6666605cdec61 (HEAD)

Author: John Garry <[email protected]>
Date:   Fri May 1 12:30:32 2020 +0100

scsi: libsas: Don't attempt to find scsi host rphy in target alloc

It doesn't have one.

drivers/scsi/libsas/sas_scsi_host.c +9 -2
index 585e0df5fce2..f1a823d51044 100644
@@ -822,8 +825,15 @@ struct domain_device *sas_find_dev_by_rphy(struct sas_rphy *rphy)
 
 int sas_target_alloc(struct scsi_target *starget)
 {
-       struct sas_rphy *rphy = dev_to_rphy(starget->dev.parent);
-       struct domain_device *found_dev = sas_find_dev_by_rphy(rphy);
+       struct device *parent = starget->dev.parent;
+       struct sas_rphy *rphy;
+       struct domain_device *found_dev;
+
+       if (scsi_is_host_device(parent))
+               return 0;
+
+       rphy = dev_to_rphy(parent);
+       found_dev = sas_find_dev_by_rphy(rphy);
 
 

Cheers,

#97 lore raw
Quoted 55 lines · John Garry

On 30/04/2020 14:18, Hannes Reinecke wrote:

Thanks for this.

Quoted 3 lines

Conversion of the SAS drivers hisi_sas, pm8001, and mv_sas are compile tested only; I'd be grateful if someone could give these patches a spin on that hardware, too.

So after some build fixups, I get this a NULL pointer deref:

[ 5.565899]  sas_find_dev_by_rphy+0x3c/0x104
[ 5.570182]  sas_target_alloc+0x18/0x84
[ 5.574029]  scsi_alloc_target+0x20c/0x304
[ 5.578136]  scsi_get_virtual_dev+0x44/0xec
[ 5.582331]  sas_register_ha+0xd0/0x258
[ 5.586178]  hisi_sas_probe+0x2ec/0x36c
[ 5.590024]  hisi_sas_v2_probe+0x34/0x64
[ 5.593958]  platform_drv_probe+0x4c/0xa0
[ 5.597978]  really_probe+0xd8/0x334
[ 5.601561]  driver_probe_device+0x58/0xe8
[ 5.605669]  device_driver_attach+0x68/0x70
[ 5.609864]  __driver_attach+0x9c/0xf8
[ 5.613622]  bus_for_each_dev+0x50/0xa0
[ 5.617468]  driver_attach+0x20/0x28
[ 5.621051]  bus_add_driver+0x148/0x1fc
[ 5.624897]  driver_register+0x6c/0x124
[ 5.628742]  __platform_driver_register+0x48/0x50
[ 5.633463]  hisi_sas_v2_driver_init+0x18/0x20
[ 5.637921]  do_one_initcall+0x50/0x194
[ 5.641767]  kernel_init_freeable+0x1e4/0x24c

And so we need this change:

commit 51f607bf91853026af102367d9e6666605cdec61 (HEAD)
Author: John Garry <[email protected]> Date:   Fri May 1 12:30:32 2020 +0100

    scsi: libsas: Don't attempt to find scsi host rphy in target alloc

    It doesn't have one.

drivers/scsi/libsas/sas_scsi_host.c +1 -2
index 585e0df5fce2..f1a823d51044 100644
@@ -822,8 +825,15 @@ struct domain_device *sas_find_dev_by_rphy(struct 
 sas_rphy *rphy)
 
  int sas_target_alloc(struct scsi_target *starget)
  {
-       struct sas_rphy *rphy = dev_to_rphy(starget->dev.parent);
-       struct domain_device *found_dev = sas_find_dev_by_rphy(rphy);
+       struct device *parent = starget->dev.parent;

drivers/scsi/libsas/sas_scsi_host.c struct domain_device *sas_find_dev_by_rphy(struct · line 839

Quoted 10 lines
drivers/scsi/libsas/sas_scsi_host.c +8 -0
+       struct sas_rphy *rphy;
+       struct domain_device *found_dev;
+
+       if (scsi_is_host_device(parent))
+               return 0;
+
+       rphy = dev_to_rphy(parent);
+       found_dev = sas_find_dev_by_rphy(rphy);
 
 

Thank you, will be including it in the next round.

Cheers,

Hannes

Signature · 4 lines
Dr. Hannes Reinecke            Teamlead Storage & Networking
[email protected]                               +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer
#98 lore raw
In reply to Hannes Reinecke

Can we get the basic infrastructure sorted out with just say csiostor and virtio-scsi before we get into all the more complicated bits? A 40+ series gets close to impossible to review unless it is just all mechnical changes.

#99 lore raw
Quoted 5 lines · Christoph Hellwig

Can we get the basic infrastructure sorted out with just say csiostor and virtio-scsi before we get into all the more complicated bits? A 40+ series gets close to impossible to review unless it is just all mechnical changes.

Sure. This patchset was just an RFC to show where I would want to go. I'll be restricting it to the virtio and csiostor for the next round.

Cheers,

Hannes

Signature · 4 lines
Dr. Hannes Reinecke            Teamlead Storage & Networking
[email protected]                               +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer
#100 lore raw
In reply to Hannes Reinecke

git://git.kernel.org/pub/scm/linux/kernel/git/hare/scsi-devel.git reserved-tags.v3

As usual, comments and reviews are welcome.

I cloned your tree and ran some tests using hpsa.

I used 3 SATA HBA disks, 3 SAS HBA disks, 2 LOGICAL VOLUMES using HDDs, and 2 LOGICAL VOLUMES using SDDs for ioaccel path.

I have an IO stress test that does mkfs, mount/umount, fio, and fsck to the above volumes while doing sg_reset opeations in parallel.

The stress test has survived 2 full days of testing.

However, my insmod/rmmod test causes a kernel panic.

Not sure why yet. I had re-cloned yesterday an d the panic is still there.

Earlier kernel versions do not panic on rmmod.

--
2.16.4

#101 lore raw
Quoted 13 lines · On 6/5/20 5:32 PM, [email protected] wrote:

git://git.kernel.org/pub/scm/linux/kernel/git/hare/scsi-devel.git reserved-tags.v3

As usual, comments and reviews are welcome.

I cloned your tree and ran some tests using hpsa.

I used 3 SATA HBA disks, 3 SAS HBA disks, 2 LOGICAL VOLUMES using HDDs, and 2 LOGICAL VOLUMES using SDDs for ioaccel path.

I have an IO stress test that does mkfs, mount/umount, fio, and fsck to the above volumes while doing sg_reset opeations in parallel.

The stress test has survived 2 full days of testing.

Quoted 9 lines

However, my insmod/rmmod test causes a kernel panic.

Not sure why yet. I had re-cloned yesterday an d the panic is still there.

Earlier kernel versions do not panic on rmmod.

--
2.16.4

Ah, right. Of course.

Should be fixed with this patch:\

drivers/scsi/scsi_scan.c +1 -1
index fdc291c47b9b..87469908524c 100644
@@ -1885,9 +1885,9 @@ void scsi_forget_host(struct Scsi_Host *shost)
                goto restart;
        }
        /* Remove virtual device last, might be needed to send commands */
+       spin_unlock_irqrestore(shost->host_lock, flags);
        if (virtual_sdev)
                __scsi_remove_device(virtual_sdev);
-       spin_unlock_irqrestore(shost->host_lock, flags);
 }
 
 /**

Can you test with that?

Cheers,

Hannes

Signature · 4 lines
Dr. Hannes Reinecke            Teamlead Storage & Networking
[email protected]                               +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer
#102 lore raw

Subject: Re: [PATCH RFC v3 00/41] scsi: enable reserved commands for LLDDs

EXTERNAL EMAIL: Do not click links or open attachments unless you know the content is safe

Quoted 3 lines

However, my insmod/rmmod test causes a kernel panic.

Ah, right. Of course.

Should be fixed with this patch:\

Can you test with that?

Cheers,

Hannes
--
Dr. Hannes Reinecke Teamlead Storage & Networking
[email protected] +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer

Yes, I'll start today.

maintainer Tested-by Don Brace
#103 lore raw
In reply to Hannes Reinecke

Subject: Re: [PATCH RFC v3 00/41] scsi: enable reserved commands for LLDDs

Quoted 14 lines · On 6/5/20 5:32 PM, [email protected] wrote:

git://git.kernel.org/pub/scm/linux/kernel/git/hare/scsi-devel.git
reserved-tags.v3

As usual, comments and reviews are welcome.

I cloned your tree and ran some tests using hpsa.

I used 3 SATA HBA disks, 3 SAS HBA disks, 2 LOGICAL VOLUMES using HDDs, and 2 LOGICAL VOLUMES using SDDs for ioaccel path.

I have an IO stress test that does mkfs, mount/umount, fio, and fsck to the above volumes while doing sg_reset opeations in parallel.

The stress test has survived 2 full days of testing.

Quoted 12 lines

However, my insmod/rmmod test causes a kernel panic.

Not sure why yet. I had re-cloned yesterday an d the panic is still there.

Earlier kernel versions do not panic on rmmod.

--
2.16.4

Quoted 3 lines

Ah, right. Of course.
Should be fixed with this patch:\
Can you test with that?

---
Applied patch...
Ran 8000 iterations of my io stress script with 10K reset operations in parallel over 3 days. insmod/rmmod
hpsa_ioctl testing
controller FW flashing
kdump/kexec testing
reboot

Cheers,

Hannes
--
Dr. Hannes Reinecke Teamlead Storage & Networking
[email protected] +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer