Skip to contentSkip to conversation

scsi: Support to handle Intermittent errors

PATCH Superseded Muneendra 2 participants 16 messages Aug 5, 2020 – Aug 14, 2020 SCSI

v1 of 8 Superseded Latest is v8 →

lore
Share & tools
lore

Muneendra

author
#0 lore raw

cover scsi: Support to handle Intermittent errors

This patch adds a support to prevent retries of all the pending/inflight io's after an abort succeeds on a particular device when transport connectivity to the device is encountering intermittent errors.

Intermittent connectivity is a condition that can be detected by transport fabric notifications. A service can monitor the ELS notifications and take action on all the outstanding io's of a scsi device at that instant.

This feature is intended to be used when the device is part of a multipath environment. When the service detects the poor connectivity, the multipath path can be placed in a marginal path group and ignored further io operations.

After placing a path in the marginal path group,the daemon sets a bit in scmd->state for all the outstanding io's on that particular device with the new sysfs interface provided in this patch.This prevent retries of all the pending/inflight io's if an io hits a scsi timeout which inturn issues an abort.On Abort succeeds on a marginal path the io will be immediately retried on another active path.On abort fails then the things escalates to existing target reset sg interface recovery process.

Below is the interface provided to abort the io
echo 1 >> /sys/class/fc_transport/targetX\:Y\:Z/noretries_abort

The patches were cut against 5.9/scsi-queue tree

Muneendra (5):
scsi: Added a new macro in scsi_cmnd.h
scsi: Clear state bit SCMD_NORETRIES_ABORT of scsi_cmd before start
request
scsi: No retries on abort success
scsi: Added routine to set SCMD_NORETRIES_ABORT bit for outstanding io
on scsi_dev
scsi_transport_fc: Added a new sysfs attribute noretries_abort

 drivers/scsi/scsi_error.c        | 63 ++++++++++++++++++++++++++++++++++++++++
 drivers/scsi/scsi_lib.c          |  8 +++--
 drivers/scsi/scsi_priv.h         |  1 +
 drivers/scsi/scsi_transport_fc.c | 49 +++++++++++++++++++++++++++++--
 include/scsi/scsi_cmnd.h         |  3 ++
 5 files changed, 120 insertions(+), 4 deletions(-)
Signature · 1 line
1.8.3.1
author
#1 lore raw

PATCH 1/5 scsi: Added a new macro in scsi_cmnd.h

Added a new macro SCMD_NORETRIES_ABORT in scsi_cmnd.h

The SCMD_NORETRIES_ABORT macro defines the third bit postion in scmd->state

---
 include/scsi/scsi_cmnd.h | 3 +++
 1 file changed, 3 insertions(+)
include/scsi/scsi_cmnd.h +3 -0
index e76bac4..e1883fe 100644
@@ -64,6 +64,9 @@ struct scsi_pointer {
 /* for scmd->state */
 #define SCMD_STATE_COMPLETE	0
 #define SCMD_STATE_INFLIGHT	1
+#define SCMD_NORETRIES_ABORT	2 /* If this bit is set then there won't be any
+				   * retries of scmd on abort success
+				   */
 
 struct scsi_cmnd {
 	struct scsi_request req;
Signature · 1 line
1.8.3.1
Reviewed-by on v1 1/5
#2 lore raw

Re: [PATCH 1/5] scsi: Added a new macro in scsi_cmnd.h

Quoted 2 lines · Muneendra

Added a new macro SCMD_NORETRIES_ABORT in scsi_cmnd.h

It's not a macro, it's a definition.

Quoted 14 lines

The SCMD_NORETRIES_ABORT macro defines the third bit postion in scmd->state

---
  include/scsi/scsi_cmnd.h | 3 +++
  1 file changed, 3 insertions(+)
include/scsi/scsi_cmnd.h +0 -0
index e76bac4..e1883fe 100644

include/scsi/scsi_cmnd.h struct scsi_pointer { · line 72

Quoted 9 lines
include/scsi/scsi_cmnd.h +3 -0
@@ -64,6 +64,9 @@ struct scsi_pointer {
  /* for scmd->state */
  #define SCMD_STATE_COMPLETE	0
  #define SCMD_STATE_INFLIGHT	1
+#define SCMD_NORETRIES_ABORT	2 /* If this bit is set then there won't be any
+				   * retries of scmd on abort success
+				   */
 
  struct scsi_cmnd {
  	struct scsi_request req;

Other than that:

Cheers,

Hannes

Signature · 4 lines
Dr. Hannes Reinecke            Teamlead Storage & Networking
[email protected]                               +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer
author
#3 lore raw

PATCH 2/5 scsi: Clear state bit SCMD_NORETRIES_ABORT of scsi_cmd before start request

In reply to Muneendra

Clearing the SCMD_NORETRIES_ABORT bit in state flag of scsi_cmd if the block layer didn't complete the request due to a timeout injection so that the timeout handler will see it needs to escalate its own error recovery. Also clearing the SCMD_NORETRIES_ABORT bit in state flag before blk_mq_start_request.

---
 drivers/scsi/scsi_lib.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
drivers/scsi/scsi_lib.c +6 -2
index 27b52fc..3da6402 100644
@@ -1594,12 +1594,15 @@ static void scsi_mq_done(struct scsi_cmnd *cmd)
 
 	/*
 	 * If the block layer didn't complete the request due to a timeout
-	 * injection, scsi must clear its internal completed state so that the
+	 * injection, scsi must clear its internal completed state and
+	 * SCMD_NORETRIES_ABORT bit in state field  so that the
 	 * timeout handler will see it needs to escalate its own error
 	 * recovery.
 	 */
-	if (unlikely(!blk_mq_complete_request(cmd->request)))
+	if (unlikely(!blk_mq_complete_request(cmd->request))) {
 		clear_bit(SCMD_STATE_COMPLETE, &cmd->state);
+		clear_bit(SCMD_NORETRIES_ABORT, &cmd->state);
+	}
 }
 
 static void scsi_mq_put_budget(struct blk_mq_hw_ctx *hctx)
@@ -1652,6 +1655,7 @@ static blk_status_t scsi_queue_rq(struct blk_mq_hw_ctx *hctx,
 		req->rq_flags |= RQF_DONTPREP;
 	} else {
 		clear_bit(SCMD_STATE_COMPLETE, &cmd->state);
+		clear_bit(SCMD_NORETRIES_ABORT, &cmd->state);
 		blk_mq_start_request(req);
 	}
 
Signature · 1 line
1.8.3.1
Reviewed-by on v1 2/5
#4 lore raw

Re: [PATCH 2/5] scsi: Clear state bit SCMD_NORETRIES_ABORT of scsi_cmd before start request

Quoted 35 lines · Muneendra

Clearing the SCMD_NORETRIES_ABORT bit in state flag of scsi_cmd if the block layer didn't complete the request due to a timeout injection so that the timeout handler will see it needs to escalate its own error recovery. Also clearing the SCMD_NORETRIES_ABORT bit in state flag before blk_mq_start_request.

---
  drivers/scsi/scsi_lib.c | 8 ++++++--
  1 file changed, 6 insertions(+), 2 deletions(-)
drivers/scsi/scsi_lib.c +5 -2
index 27b52fc..3da6402 100644
@@ -1594,12 +1594,15 @@ static void scsi_mq_done(struct scsi_cmnd *cmd)
 
  	/*
  	 * If the block layer didn't complete the request due to a timeout
-	 * injection, scsi must clear its internal completed state so that the
+	 * injection, scsi must clear its internal completed state and
+	 * SCMD_NORETRIES_ABORT bit in state field  so that the
  	 * timeout handler will see it needs to escalate its own error
  	 * recovery.
  	 */
-	if (unlikely(!blk_mq_complete_request(cmd->request)))
+	if (unlikely(!blk_mq_complete_request(cmd->request))) {
  		clear_bit(SCMD_STATE_COMPLETE, &cmd->state);
+		clear_bit(SCMD_NORETRIES_ABORT, &cmd->state);
+	}
  }
 
  static void scsi_mq_put_budget(struct blk_mq_hw_ctx *hctx)

drivers/scsi/scsi_lib.c static blk_status_t scsi_queue_rq(struct blk_mq_hw_ctx *hctx, · line 1661

Quoted 7 lines
drivers/scsi/scsi_lib.c +1 -0
@@ -1652,6 +1655,7 @@ static blk_status_t scsi_queue_rq(struct blk_mq_hw_ctx *hctx,
  		req->rq_flags |= RQF_DONTPREP;
  	} else {
  		clear_bit(SCMD_STATE_COMPLETE, &cmd->state);
+		clear_bit(SCMD_NORETRIES_ABORT, &cmd->state);
  		blk_mq_start_request(req);
  	}
 

Cheers,

Hannes

Signature · 4 lines
Dr. Hannes Reinecke            Teamlead Storage & Networking
[email protected]                               +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer
author
#5 lore raw

PATCH 3/5 scsi: No retries on abort success

In reply to Muneendra

Made an additional check in scsi_noretry_cmd to verify whether user has decided not to do retries on abort(issued on scsi timeouts) success by checking the SCMD_NORETRIES_ABORT bit

If SCMD_NORETRIES_ABORT bit is set we are making sure there won't be any retries done on the same path and also setting the host byte as DID_TRANSPORT_FAILFAST so that the error can be propogated as recoverable transport error to the blk layers.

---
 drivers/scsi/scsi_error.c | 10 ++++++++++
 1 file changed, 10 insertions(+)
drivers/scsi/scsi_error.c +10 -0
index 927b1e6..3222496 100644
@@ -1749,6 +1749,16 @@ int scsi_noretry_cmd(struct scsi_cmnd *scmd)
 
 check_type:
 	/*
+	 * Check whether caller has decided not to do retries on
+	 * abort success by setting the SCMD_NORETRIES_ABORT bit
+	 */
+	if ((test_bit(SCMD_NORETRIES_ABORT, &scmd->state)) &&
+		(scmd->request->cmd_flags & REQ_FAILFAST_TRANSPORT)) {
+		set_host_byte(scmd, DID_TRANSPORT_FAILFAST);
+		return 1;
+	}
+
+	/*
 	 * assume caller has checked sense and determined
 	 * the check condition was retryable.
 	 */
Signature · 1 line
1.8.3.1
Reviewed-by on v1 3/5
#6 lore raw

Re: [PATCH 3/5] scsi: No retries on abort success

Quoted 26 lines · Muneendra

Made an additional check in scsi_noretry_cmd to verify whether user has decided not to do retries on abort(issued on scsi timeouts) success by checking the SCMD_NORETRIES_ABORT bit

If SCMD_NORETRIES_ABORT bit is set we are making sure there won't be any retries done on the same path and also setting the host byte as DID_TRANSPORT_FAILFAST so that the error can be propogated as recoverable transport error to the blk layers.

---
  drivers/scsi/scsi_error.c | 10 ++++++++++
  1 file changed, 10 insertions(+)
drivers/scsi/scsi_error.c +3 -0
index 927b1e6..3222496 100644
@@ -1749,6 +1749,16 @@ int scsi_noretry_cmd(struct scsi_cmnd *scmd)
 
  check_type:
  	/*
+	 * Check whether caller has decided not to do retries on
+	 * abort success by setting the SCMD_NORETRIES_ABORT bit
+	 */

drivers/scsi/scsi_error.c int scsi_noretry_cmd(struct scsi_cmnd *scmd) · line 1764

Quoted 10 lines
drivers/scsi/scsi_error.c +7 -0
+	if ((test_bit(SCMD_NORETRIES_ABORT, &scmd->state)) &&
+		(scmd->request->cmd_flags & REQ_FAILFAST_TRANSPORT)) {
+		set_host_byte(scmd, DID_TRANSPORT_FAILFAST);
+		return 1;
+	}
+
+	/*
  	 * assume caller has checked sense and determined
  	 * the check condition was retryable.
  	 */

_Actually_ DID_TRANSPORT_FAILFAST is just for transport aborted commands, so maybe we should use a different error code (or add a new one). But other than that:

Cheers,

Hannes

Signature · 4 lines
Dr. Hannes Reinecke            Teamlead Storage & Networking
[email protected]                               +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer
author
#7 lore raw

PATCH 4/5 scsi: Added routine to set SCMD_NORETRIES_ABORT bit for outstanding io on scsi_dev

In reply to Muneendra

Added a new routine scsi_set_noretries_abort_io_device()to set SCMD_NORETRIES_ABORT for all the inflight/pending IO's on a particular scsi device at that particular instant.

Export the symbol so the routine can be called by scsi_transport_fc.c

Added new function declaration scsi_set_noretries_abort_io_device in scsi_priv.h

---
 drivers/scsi/scsi_error.c | 53 +++++++++++++++++++++++++++++++++++++++++++++++
 drivers/scsi/scsi_priv.h  |  1 +
 2 files changed, 54 insertions(+)
drivers/scsi/scsi_error.c +53 -0
index 3222496..938d770 100644
@@ -271,6 +271,59 @@ void scsi_eh_scmd_add(struct scsi_cmnd *scmd)
 	call_rcu(&scmd->rcu, scsi_eh_inc_host_failed);
 }
 
+static bool
+scsi_set_noretries_abort_io(struct request *rq, void *priv, bool reserved)
+{
+	struct scsi_cmnd *scmd = blk_mq_rq_to_pdu(rq);
+	struct scsi_device *sdev = scmd->device;
+
+	/* only set SCMD_NORETRIES_ABORT on ios on a specific sdev */
+	if (sdev != priv)
+		return true;
+	/* we don't want this command reissued on abort success
+	 * so set SCMD_NORETRIES_ABORT bit to ensure it
+	 * won't get reissued
+	 */
+	if (READ_ONCE(rq->state) == MQ_RQ_IN_FLIGHT)
+		set_bit(SCMD_NORETRIES_ABORT, &scmd->state);
+	return true;
+}
+
+static int
+__scsi_set_noretries_abort_io_device(struct scsi_device *sdev)
+{
+
+	if (sdev->sdev_state != SDEV_RUNNING)
+		return -EINVAL;
+
+	if (blk_queue_init_done(sdev->request_queue)) {
+
+		blk_mq_quiesce_queue(sdev->request_queue);
+		blk_mq_tagset_busy_iter(&sdev->host->tag_set,
+				scsi_set_noretries_abort_io, sdev);
+		blk_mq_unquiesce_queue(sdev->request_queue);
+	}
+	return 0;
+}
+
+/*
+ * scsi_set_noretries_abort_io_device - set the SCMD_NORETRIES_ABORT
+ * bit for all the pending io's on a device
+ * @sdev:	scsi_device
+ */
+int
+scsi_set_noretries_abort_io_device(struct scsi_device *sdev)
+{
+	struct Scsi_Host *shost = sdev->host;
+	int ret  = -EINVAL;
+
+	mutex_lock(&shost->scan_mutex);
+	ret = __scsi_set_noretries_abort_io_device(sdev);
+	mutex_unlock(&shost->scan_mutex);
+	return ret;
+}
+EXPORT_SYMBOL(scsi_set_noretries_abort_io_device);
+
 /**
  * scsi_times_out - Timeout function for normal scsi commands.
  * @req:	request that is timing out.
drivers/scsi/scsi_priv.h +1 -0
index d12ada0..1bbffd3 100644
@@ -81,6 +81,7 @@ void scsi_eh_ready_devs(struct Scsi_Host *shost,
 int scsi_eh_get_sense(struct list_head *work_q,
 		      struct list_head *done_q);
 int scsi_noretry_cmd(struct scsi_cmnd *scmd);
+extern int scsi_set_noretries_abort_io_device(struct scsi_device *sdev);
 
 /* scsi_lib.c */
 extern int scsi_maybe_unblock_host(struct scsi_device *sdev);
Signature · 1 line
1.8.3.1
Reviewed-by on v1 4/5
#8 lore raw

Re: [PATCH 4/5] scsi: Added routine to set SCMD_NORETRIES_ABORT bit for outstanding io on scsi_dev

Quoted 85 lines · Muneendra

Added a new routine scsi_set_noretries_abort_io_device()to set SCMD_NORETRIES_ABORT for all the inflight/pending IO's on a particular scsi device at that particular instant.

Export the symbol so the routine can be called by scsi_transport_fc.c

Added new function declaration scsi_set_noretries_abort_io_device in scsi_priv.h

---
  drivers/scsi/scsi_error.c | 53 +++++++++++++++++++++++++++++++++++++++++++++++
  drivers/scsi/scsi_priv.h  |  1 +
  2 files changed, 54 insertions(+)
drivers/scsi/scsi_error.c +53 -0
index 3222496..938d770 100644
@@ -271,6 +271,59 @@ void scsi_eh_scmd_add(struct scsi_cmnd *scmd)
  	call_rcu(&scmd->rcu, scsi_eh_inc_host_failed);
  }
 
+static bool
+scsi_set_noretries_abort_io(struct request *rq, void *priv, bool reserved)
+{
+	struct scsi_cmnd *scmd = blk_mq_rq_to_pdu(rq);
+	struct scsi_device *sdev = scmd->device;
+
+	/* only set SCMD_NORETRIES_ABORT on ios on a specific sdev */
+	if (sdev != priv)
+		return true;
+	/* we don't want this command reissued on abort success
+	 * so set SCMD_NORETRIES_ABORT bit to ensure it
+	 * won't get reissued
+	 */
+	if (READ_ONCE(rq->state) == MQ_RQ_IN_FLIGHT)
+		set_bit(SCMD_NORETRIES_ABORT, &scmd->state);
+	return true;
+}
+
+static int
+__scsi_set_noretries_abort_io_device(struct scsi_device *sdev)
+{
+
+	if (sdev->sdev_state != SDEV_RUNNING)
+		return -EINVAL;
+
+	if (blk_queue_init_done(sdev->request_queue)) {
+
+		blk_mq_quiesce_queue(sdev->request_queue);
+		blk_mq_tagset_busy_iter(&sdev->host->tag_set,
+				scsi_set_noretries_abort_io, sdev);
+		blk_mq_unquiesce_queue(sdev->request_queue);
+	}
+	return 0;
+}
+
+/*
+ * scsi_set_noretries_abort_io_device - set the SCMD_NORETRIES_ABORT
+ * bit for all the pending io's on a device
+ * @sdev:	scsi_device
+ */
+int
+scsi_set_noretries_abort_io_device(struct scsi_device *sdev)
+{
+	struct Scsi_Host *shost = sdev->host;
+	int ret  = -EINVAL;
+
+	mutex_lock(&shost->scan_mutex);
+	ret = __scsi_set_noretries_abort_io_device(sdev);
+	mutex_unlock(&shost->scan_mutex);
+	return ret;
+}
+EXPORT_SYMBOL(scsi_set_noretries_abort_io_device);
+
  /**
   * scsi_times_out - Timeout function for normal scsi commands.
   * @req:	request that is timing out.
drivers/scsi/scsi_priv.h +0 -0
index d12ada0..1bbffd3 100644

drivers/scsi/scsi_priv.h void scsi_eh_ready_devs(struct Scsi_Host *shost, · line 87

Quoted 7 lines
drivers/scsi/scsi_priv.h +1 -0
@@ -81,6 +81,7 @@ void scsi_eh_ready_devs(struct Scsi_Host *shost,
  int scsi_eh_get_sense(struct list_head *work_q,
  		      struct list_head *done_q);
  int scsi_noretry_cmd(struct scsi_cmnd *scmd);
+extern int scsi_set_noretries_abort_io_device(struct scsi_device *sdev);
 
  /* scsi_lib.c */
  extern int scsi_maybe_unblock_host(struct scsi_device *sdev);

Cheers,

Hannes

Signature · 4 lines
Dr. Hannes Reinecke            Teamlead Storage & Networking
[email protected]                               +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer
author
#9 lore raw

PATCH 5/5 scsi_transport_fc: Added a new sysfs attribute noretries_abort

In reply to Muneendra

Added a new sysfs attribute noretries_abort under fc_transport/target*/

This interface will set SCMD_NORETRIES_ABORT bit in scmd->state for all the pending io's on the scsi device associated with target port.

Below is the interface provided to abort the io
echo 1 >> /sys/class/fc_transport/targetX\:Y\:Z/noretries_abort

---
 drivers/scsi/scsi_transport_fc.c | 49 ++++++++++++++++++++++++++++++++++++++--
 1 file changed, 47 insertions(+), 2 deletions(-)
drivers/scsi/scsi_transport_fc.c +47 -2
index 2732fa6..f7b00ae 100644
@@ -305,7 +305,7 @@ struct device_attribute device_attr_##_prefix##_##_name = 	\
  * Attribute counts pre object type...
  * Increase these values if you add attributes
  */
-#define FC_STARGET_NUM_ATTRS 	3
+#define FC_STARGET_NUM_ATTRS	4
 #define FC_RPORT_NUM_ATTRS	10
 #define FC_VPORT_NUM_ATTRS	9
 #define FC_HOST_NUM_ATTRS	29
@@ -994,6 +994,44 @@ static FC_DEVICE_ATTR(rport, fast_io_fail_tmo, S_IRUGO | S_IWUSR,
 /*
  * FC SCSI Target Attribute Management
  */
+static void scsi_target_set_noretries_abort(struct scsi_target *starget)
+{
+	struct scsi_device *sdev, *tmp;
+	struct Scsi_Host *shost = dev_to_shost(starget->dev.parent);
+	unsigned long flags;
+
+	spin_lock_irqsave(shost->host_lock, flags);
+	list_for_each_entry_safe(sdev, tmp, &starget->devices, same_target_siblings) {
+		if (sdev->sdev_state == SDEV_DEL)
+			continue;
+		if (scsi_device_get(sdev))
+			continue;
+
+		spin_unlock_irqrestore(shost->host_lock, flags);
+		scsi_set_noretries_abort_io_device(sdev);
+		spin_lock_irqsave(shost->host_lock, flags);
+		scsi_device_put(sdev);
+	}
+	spin_unlock_irqrestore(shost->host_lock, flags);
+}
+
+/*
+ * Sets  no retries on abort in scmd->state for all
+ * outstanding io of all the scsi_devs
+ * write 1 to set the bit for all outstanding io's
+ */
+static ssize_t fc_target_set_noretries_abort(struct device *dev,
+						struct device_attribute *attr,
+						const char *buf, size_t count)
+{
+	struct scsi_target *starget = transport_class_to_starget(dev);
+
+	scsi_target_set_noretries_abort(starget);
+	return count;
+}
+
+static FC_DEVICE_ATTR(starget, noretries_abort, 0200,
+		NULL, fc_target_set_noretries_abort);
 
 /*
  * Note: in the target show function we recognize when the remote
@@ -1036,6 +1074,13 @@ static FC_DEVICE_ATTR(starget, field, S_IRUGO,			\
 	if (i->f->show_starget_##field)					\
 		count++
 
+#define SETUP_PRIVATE_STARGET_ATTRIBUTE_RW(field)			\
+do {									\
+	i->private_starget_attrs[count] = device_attr_starget_##field; \
+	i->starget_attrs[count] = &i->private_starget_attrs[count];	\
+	count++;							\
+} while (0)
+
 #define SETUP_STARGET_ATTRIBUTE_RW(field)				\
 	i->private_starget_attrs[count] = device_attr_starget_##field; \
 	if (!i->f->set_starget_##field) {				\
@@ -2197,7 +2242,7 @@ struct scsi_transport_template *
 	SETUP_STARGET_ATTRIBUTE_RD(node_name);
 	SETUP_STARGET_ATTRIBUTE_RD(port_name);
 	SETUP_STARGET_ATTRIBUTE_RD(port_id);
-
+	SETUP_PRIVATE_STARGET_ATTRIBUTE_RW(noretries_abort);
 	BUG_ON(count > FC_STARGET_NUM_ATTRS);
 
 	i->starget_attrs[count] = NULL;
Signature · 1 line
1.8.3.1
#10 lore raw

Re: [PATCH 5/5] scsi_transport_fc: Added a new sysfs attribute noretries_abort

Quoted 87 lines · Muneendra

Added a new sysfs attribute noretries_abort under fc_transport/target*/

This interface will set SCMD_NORETRIES_ABORT bit in scmd->state for all the pending io's on the scsi device associated with target port.

Below is the interface provided to abort the io
echo 1 >> /sys/class/fc_transport/targetX\:Y\:Z/noretries_abort

---
  drivers/scsi/scsi_transport_fc.c | 49 ++++++++++++++++++++++++++++++++++++++--
  1 file changed, 47 insertions(+), 2 deletions(-)
drivers/scsi/scsi_transport_fc.c +46 -1
index 2732fa6..f7b00ae 100644
@@ -305,7 +305,7 @@ struct device_attribute device_attr_##_prefix##_##_name = 	\
   * Attribute counts pre object type...
   * Increase these values if you add attributes
   */
-#define FC_STARGET_NUM_ATTRS 	3
+#define FC_STARGET_NUM_ATTRS	4
  #define FC_RPORT_NUM_ATTRS	10
  #define FC_VPORT_NUM_ATTRS	9
  #define FC_HOST_NUM_ATTRS	29
@@ -994,6 +994,44 @@ static FC_DEVICE_ATTR(rport, fast_io_fail_tmo, S_IRUGO | S_IWUSR,
  /*
   * FC SCSI Target Attribute Management
   */
+static void scsi_target_set_noretries_abort(struct scsi_target *starget)
+{
+	struct scsi_device *sdev, *tmp;
+	struct Scsi_Host *shost = dev_to_shost(starget->dev.parent);
+	unsigned long flags;
+
+	spin_lock_irqsave(shost->host_lock, flags);
+	list_for_each_entry_safe(sdev, tmp, &starget->devices, same_target_siblings) {
+		if (sdev->sdev_state == SDEV_DEL)
+			continue;
+		if (scsi_device_get(sdev))
+			continue;
+
+		spin_unlock_irqrestore(shost->host_lock, flags);
+		scsi_set_noretries_abort_io_device(sdev);
+		spin_lock_irqsave(shost->host_lock, flags);
+		scsi_device_put(sdev);
+	}
+	spin_unlock_irqrestore(shost->host_lock, flags);
+}
+
+/*
+ * Sets  no retries on abort in scmd->state for all
+ * outstanding io of all the scsi_devs
+ * write 1 to set the bit for all outstanding io's
+ */
+static ssize_t fc_target_set_noretries_abort(struct device *dev,
+						struct device_attribute *attr,
+						const char *buf, size_t count)
+{
+	struct scsi_target *starget = transport_class_to_starget(dev);
+
+	scsi_target_set_noretries_abort(starget);
+	return count;
+}
+
+static FC_DEVICE_ATTR(starget, noretries_abort, 0200,
+		NULL, fc_target_set_noretries_abort);
 
  /*
   * Note: in the target show function we recognize when the remote
@@ -1036,6 +1074,13 @@ static FC_DEVICE_ATTR(starget, field, S_IRUGO,			\
  	if (i->f->show_starget_##field)					\
  		count++
 
+#define SETUP_PRIVATE_STARGET_ATTRIBUTE_RW(field)			\
+do {									\
+	i->private_starget_attrs[count] = device_attr_starget_##field; \
+	i->starget_attrs[count] = &i->private_starget_attrs[count];	\
+	count++;							\
+} while (0)
+
  #define SETUP_STARGET_ATTRIBUTE_RW(field)				\
  	i->private_starget_attrs[count] = device_attr_starget_##field; \
  	if (!i->f->set_starget_##field) {				\

drivers/scsi/scsi_transport_fc.c struct scsi_transport_template * · line 2248

Quoted 8 lines
drivers/scsi/scsi_transport_fc.c +1 -1
@@ -2197,7 +2242,7 @@ struct scsi_transport_template *
  	SETUP_STARGET_ATTRIBUTE_RD(node_name);
  	SETUP_STARGET_ATTRIBUTE_RD(port_name);
  	SETUP_STARGET_ATTRIBUTE_RD(port_id);
-
+	SETUP_PRIVATE_STARGET_ATTRIBUTE_RW(noretries_abort);
  	BUG_ON(count > FC_STARGET_NUM_ATTRS);
 
  	i->starget_attrs[count] = NULL;

Hmm. Wouldn't it make more sense to introduce a new port state 'marginal' for this? We might want/need to introduce additional error recovery mechanisms here, so having a new state might be easier in the long run ...

Additionally, from my understanding the FPIN events will be generated with a certain frequency. So we could model the new 'marginal' state similar to the dev_loss_tmo mechanism; start a timer whenever the 'marginal' state is being set, and clear the state back to 'running' if the state hasn't been refreshed within that timeframe. That would give us an automatic state reset back to running, and quite easy to implement from userland.

Cheers,

Hannes

Signature · 4 lines
Dr. Hannes Reinecke            Teamlead Storage & Networking
[email protected]                               +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer
#11 lore raw

RE: [PATCH 5/5] scsi_transport_fc: Added a new sysfs attribute noretries_abort

Hi Hannes,

Quoted 5 lines

Hmm. Wouldn't it make more sense to introduce a new port state 'marginal' for this? We might >want/need to introduce additional error recovery mechanisms here, so having a new state >might be easier in the long run ...

Quoted 7 lines

Additionally, from my understanding the FPIN events will be generated with a certain >frequency. So we could model the new 'marginal' state similar to the dev_loss_tmo >mechanism; start a timer whenever the 'marginal' state is being set, and clear the state back to >'running' if the state hasn't been refreshed within that timeframe.
That would give us an automatic state reset back to running, and quite easy to implement from >userland.

Thanks for the review.
I have a small doubt.
When the port state moves from marginal to running state does it mean we expect a traffic from the path ?

Regards,
Muneendra.

#12 lore raw

Re: [PATCH 5/5] scsi_transport_fc: Added a new sysfs attribute noretries_abort

Quoted 15 lines · Muneendra Kumar M

Hi Hannes,

Quoted 5 lines

Hmm. Wouldn't it make more sense to introduce a new port state 'marginal' for this? We might >want/need to introduce additional error recovery mechanisms here, so having a new state >might be easier in the long run ...

Quoted 7 lines

Additionally, from my understanding the FPIN events will be generated with a certain >frequency. So we could model the new 'marginal' state similar to the dev_loss_tmo >mechanism; start a timer whenever the 'marginal' state is being set, and clear the state back to >'running' if the state hasn't been refreshed within that timeframe.
That would give us an automatic state reset back to running, and quite easy to implement from >userland.

Quoted 5 lines

Thanks for the review.
I have a small doubt.
When the port state moves from marginal to running state does it mean we expect a traffic from the path ?

We don't expect traffic; rather we _allow_ traffic.
But moving to from marginal to running means that we didn't receive FPIN events, and the path should be considered healthy again.
So from that perspective it should be back to normal operations.

Cheers,

Hannes

Signature · 4 lines
Dr. Hannes Reinecke		           Kernel Storage Architect
[email protected]			                  +49 911 74053 688
SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), GF: Felix Imendörffer
#13 lore raw

RE: [PATCH 5/5] scsi_transport_fc: Added a new sysfs attribute noretries_abort

Hi Hannes,

Quoted 19 lines
Quoted 6 lines

Hmm. Wouldn't it make more sense to introduce a new port state 'marginal' for this? We might >want/need to introduce additional error recovery mechanisms here, so having a new state >might be easier in the long run
...

Quoted 7 lines

Additionally, from my understanding the FPIN events will be generated with a certain >frequency. So we could model the new 'marginal' state similar to the dev_loss_tmo >mechanism; start a timer whenever the 'marginal' state is being set, and clear the state back to >'running' if the state hasn't been refreshed within that timeframe. That would give us an automatic state reset back to running, and quite easy to implement from >userland.

Thanks for the review.
I have a small doubt.
When the port state moves from marginal to running state does it mean we expect a traffic from the path ?

Quoted 4 lines

We don't expect traffic; rather we _allow_ traffic.
But moving to from marginal to running means that we didn't receive FPIN events, and the path should be considered healthy again.
So from that perspective it should be back to normal operations.

But this could apply only to FPIN-Congestion. Only in this case FPIN-CN FPIN events will be generated with a certain frequency.
But for FPIN-Li this is not the case.
FPIN-LI is used to inform about marginal paths, which needs manual intervention to recover.
And for FPIN-LI the path should be re-enabled on any link bounce (portdisable followed by portenable) which would correlated to a cable/sfp change.
For now, however, we are addressing FPIN-LI primarily.

Regards,
Muneendra.

#14 lore raw

Re: [PATCH 5/5] scsi_transport_fc: Added a new sysfs attribute noretries_abort

Quoted 26 lines · Muneendra Kumar M

Hi Hannes,

Quoted 19 lines
Quoted 6 lines

Hmm. Wouldn't it make more sense to introduce a new port state 'marginal' for this? We might >want/need to introduce additional error recovery mechanisms here, so having a new state >might be easier in the long run
...

Quoted 7 lines

Additionally, from my understanding the FPIN events will be generated with a certain >frequency. So we could model the new 'marginal' state similar to the dev_loss_tmo >mechanism; start a timer whenever the 'marginal' state is being set, and clear the state back to >'running' if the state hasn't been refreshed within that timeframe. That would give us an automatic state reset back to running, and quite easy to implement from >userland.

Thanks for the review.
I have a small doubt.
When the port state moves from marginal to running state does it mean we expect a traffic from the path ?

Quoted 4 lines

We don't expect traffic; rather we _allow_ traffic.
But moving to from marginal to running means that we didn't receive FPIN events, and the path should be considered healthy again.
So from that perspective it should be back to normal operations.

Quoted 10 lines

But this could apply only to FPIN-Congestion. Only in this case FPIN-CN FPIN events will be generated with a certain frequency.
But for FPIN-Li this is not the case.
FPIN-LI is used to inform about marginal paths, which needs manual intervention to recover.
And for FPIN-LI the path should be re-enabled on any link bounce (portdisable followed by portenable) which would correlated to a cable/sfp change.
For now, however, we are addressing FPIN-LI primarily.

Ah. So that changes things slightly; I had hoped we can address things systematically, but with link integrity issues we don't have any other choice but to replace the cable (ie wait for user interaction).

But still I'm in favour of the 'marginal' state; that one could be set manually (or by an FPIN LI event), and would need to be reset either manually or by link reset.

And have the advantage of being easier to implement :-)

Cheers,

Hannes

Signature · 4 lines
Dr. Hannes Reinecke		           Kernel Storage Architect
[email protected]			                  +49 911 74053 688
SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), GF: Felix Imendörffer
#15 lore raw

RE: [PATCH 5/5] scsi_transport_fc: Added a new sysfs attribute noretries_abort

Hi Hannes,

Quoted 10 lines
Quoted 6 lines
Quoted 6 lines

Hmm. Wouldn't it make more sense to introduce a new port state 'marginal'
for this? We might >want/need to introduce additional error recovery mechanisms here, so having a new state >might be easier in the long run ...

Ah. So that changes things slightly; I had hoped we can address things systematically, but with link integrity issues we don't have any other choice but to replace the cable (ie wait for user interaction).

Quoted 5 lines

But still I'm in favour of the 'marginal' state; that one could be set manually (or by an FPIN LI event), and would need to be reset either manually or by link reset.

And have the advantage of being easier to implement :-)

Thanks for the review.
I will incorporate all your review comments and will add marginal state in my next version.

Regards,
Muneendra.