Skip to contentSkip to conversation

hisi_sas: Some misc updates

PATCH Applied John Garry 2 participants 9 messages Feb 24, 2022 – Mar 2, 2022

Subsystems (2)
lore
Share & tools
lore

John Garry

author
#0 lore raw

cover hisi_sas: Some misc updates

Hi Martin,

This series includes some updates as following:
- better management of freeing PCI vectors
- hopefully the following series can make it soon so we may remove this

    new code:
    https://lore.kernel.org/linux-pci/[email protected]/T/#md27a594ac53b8e05abb2f22ec350137b9023ff18

- Fix prot_mask module param permissions
- Change phyup timeout to handle some special SATA disks
- modify underflow handling to quickly identify faulty disks
- debugfs hardening

Please consider for 5.18 - hopefully not too late

Thanks!

Qi Liu (2):
scsi: hisi_sas: Free irq vectors in order for v3 HW
scsi: hisi_sas: Rename error labels in hisi_sas_v3_probe()

Xiang Chen (3):
scsi: hisi_sas: Change permission of parameter prot_mask
scsi: hisi_sas: Change hisi_sas_control_phy() phyup timeout
scsi: hisi_sas: Limit users changing debugfs BIST count value

Xingui Yang (1):
scsi: hisi_sas: Modify v3 HW SSP underflow error processing

 drivers/scsi/hisi_sas/hisi_sas.h       |   2 +-
 drivers/scsi/hisi_sas/hisi_sas_main.c  |   3 +-
 drivers/scsi/hisi_sas/hisi_sas_v3_hw.c | 127 +++++++++++++++++++------
 3 files changed, 100 insertions(+), 32 deletions(-)
Signature · 1 line
2.26.2
author
#1 lore raw

PATCH 1/6 scsi: hisi_sas: Change permission of parameter prot_mask

From: Xiang Chen <[email protected]>

Currently the permission of parameter prot_mask is 0x0, which means that the member does not appear in sysfs. Change it as other module parameters to 0444 for world-readable.

Fixes d6a9000b81be ("scsi: hisi_sas: Add support for DIF feature for v3 hw")
---
 drivers/scsi/hisi_sas/hisi_sas_v3_hw.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/scsi/hisi_sas/hisi_sas_v3_hw.c +1 -1
index a57f247481ed..29a566a19219 100644
@@ -530,7 +530,7 @@ MODULE_PARM_DESC(intr_conv, "interrupt converge enable (0-1)");
 
 /* permit overriding the host protection capabilities mask (EEDP/T10 PI) */
 static int prot_mask;
-module_param(prot_mask, int, 0);
+module_param(prot_mask, int, 0444);
 MODULE_PARM_DESC(prot_mask, " host protection capabilities mask, def=0x0 ");
 
 static void debugfs_work_handler_v3_hw(struct work_struct *work);
Signature · 1 line
2.26.2
author
#2 lore raw

PATCH 2/6 scsi: hisi_sas: Change hisi_sas_control_phy() phyup timeout

In reply to John Garry

From: Xiang Chen <[email protected]>

The time of phyup not only depends on the controller but also the type of disk connected. As an example, from experience, for some SATA disks the amount of time from reset/power-on to receive the D2H FIS for phyup can take upto and more than 10s sometimes. According to the specification of some SATA disks such as ST14000NM0018, the max time from power-on to ready is 30s.

Based on this the current timeout of phyup at 2s which is not enough. So set the value as HISI_SAS_WAIT_PHYUP_TIMEOUT (30s) in hisi_sas_control_phy().

For v3 hw there is a pre-existing workaround for a HW bug, being that we issue a link reset when the OOB occurs but the phyup does not. The current phyup timeout is HISI_SAS_WAIT_PHYUP_TIMEOUT. So if this does occur from when issuing a phy enable or similar via hisi_sas_control_phy(), the subsequent HW workaround linkreset processing calls hisi_sas_control_phy(), but this will pend the original phy reset timing out, so it is safe.

---
 drivers/scsi/hisi_sas/hisi_sas.h      | 2 +-
 drivers/scsi/hisi_sas/hisi_sas_main.c | 3 ++-
 2 files changed, 3 insertions(+), 2 deletions(-)
drivers/scsi/hisi_sas/hisi_sas.h +1 -1
index fe0c15bbfca9..99ceffad4bd9 100644
@@ -91,7 +91,7 @@
 
 #define HISI_SAS_PROT_MASK (HISI_SAS_DIF_PROT_MASK | HISI_SAS_DIX_PROT_MASK)
 
-#define HISI_SAS_WAIT_PHYUP_TIMEOUT	(20 * HZ)
+#define HISI_SAS_WAIT_PHYUP_TIMEOUT	(30 * HZ)
 #define HISI_SAS_CLEAR_ITCT_TIMEOUT	(20 * HZ)
 
 struct hisi_hba;
drivers/scsi/hisi_sas/hisi_sas_main.c +2 -1
index efedfb3332c3..cd8ec851e760 100644
@@ -1201,7 +1201,8 @@ static int hisi_sas_control_phy(struct asd_sas_phy *sas_phy, enum phy_func func,
 		goto out;
 	}
 
-	if (sts && !wait_for_completion_timeout(&completion, 2 * HZ)) {
+	if (sts && !wait_for_completion_timeout(&completion,
+		HISI_SAS_WAIT_PHYUP_TIMEOUT)) {
 		dev_warn(dev, "phy%d wait phyup timed out for func %d\n",
 			 phy_no, func);
 		if (phy->in_reset)
Signature · 1 line
2.26.2
author
#3 lore raw

PATCH 3/6 scsi: hisi_sas: Free irq vectors in order for v3 HW

In reply to John Garry

From: Qi Liu <[email protected]>

If the driver probe fails to request the channel IRQ or fatal IRQ, the driver will free the IRQ vectors before freeing the IRQs in free_irq(), and this will cause a kernel BUG like this:

------------[ cut here ]------------
kernel BUG at drivers/pci/msi.c:369!
Internal error: Oops - BUG: 0 [#1] PREEMPT SMP
Call trace:
   free_msi_irqs+0x118/0x13c
   pci_disable_msi+0xfc/0x120
   pci_free_irq_vectors+0x24/0x3c
   hisi_sas_v3_probe+0x360/0x9d0 [hisi_sas_v3_hw]
   local_pci_probe+0x44/0xb0
   work_for_cpu_fn+0x20/0x34
   process_one_work+0x1d0/0x340
   worker_thread+0x2e0/0x460
   kthread+0x180/0x190
   ret_from_fork+0x10/0x20
---[ end trace b88990335b610c11 ]---

So we use devm_add_action() to control the order in which we free the vectors.

---
 drivers/scsi/hisi_sas/hisi_sas_v3_hw.c | 16 +++++++++++-----
 1 file changed, 11 insertions(+), 5 deletions(-)
drivers/scsi/hisi_sas/hisi_sas_v3_hw.c +11 -5
index 29a566a19219..5b5557cab7ed 100644
@@ -2397,17 +2397,25 @@ static irqreturn_t cq_interrupt_v3_hw(int irq_no, void *p)
 	return IRQ_WAKE_THREAD;
 }
 
+static void hisi_sas_v3_free_vectors(void *data)
+{
+	struct pci_dev *pdev = data;
+
+	pci_free_irq_vectors(pdev);
+}
+
 static int interrupt_preinit_v3_hw(struct hisi_hba *hisi_hba)
 {
 	int vectors;
 	int max_msi = HISI_SAS_MSI_COUNT_V3_HW, min_msi;
 	struct Scsi_Host *shost = hisi_hba->shost;
+	struct pci_dev *pdev = hisi_hba->pci_dev;
 	struct irq_affinity desc = {
 		.pre_vectors = BASE_VECTORS_V3_HW,
 	};
 
 	min_msi = MIN_AFFINE_VECTORS_V3_HW;
-	vectors = pci_alloc_irq_vectors_affinity(hisi_hba->pci_dev,
+	vectors = pci_alloc_irq_vectors_affinity(pdev,
 						 min_msi, max_msi,
 						 PCI_IRQ_MSI |
 						 PCI_IRQ_AFFINITY,
@@ -2419,6 +2427,7 @@ static int interrupt_preinit_v3_hw(struct hisi_hba *hisi_hba)
 	hisi_hba->cq_nvecs = vectors - BASE_VECTORS_V3_HW;
 	shost->nr_hw_queues = hisi_hba->cq_nvecs;
 
+	devm_add_action(&pdev->dev, hisi_sas_v3_free_vectors, pdev);
 	return 0;
 }
 
@@ -4768,7 +4777,7 @@ hisi_sas_v3_probe(struct pci_dev *pdev, const struct pci_device_id *id)
 
 	rc = scsi_add_host(shost, dev);
 	if (rc)
-		goto err_out_free_irq_vectors;
+		goto err_out_debugfs;
 
 	rc = sas_register_ha(sha);
 	if (rc)
@@ -4799,8 +4808,6 @@ hisi_sas_v3_probe(struct pci_dev *pdev, const struct pci_device_id *id)
 	sas_unregister_ha(sha);
 err_out_register_ha:
 	scsi_remove_host(shost);
-err_out_free_irq_vectors:
-	pci_free_irq_vectors(pdev);
 err_out_debugfs:
 	debugfs_exit_v3_hw(hisi_hba);
 err_out_ha:
@@ -4824,7 +4831,6 @@ hisi_sas_v3_destroy_irqs(struct pci_dev *pdev, struct hisi_hba *hisi_hba)
 
 		devm_free_irq(&pdev->dev, pci_irq_vector(pdev, nr), cq);
 	}
-	pci_free_irq_vectors(pdev);
 }
 
 static void hisi_sas_v3_remove(struct pci_dev *pdev)
Signature · 1 line
2.26.2
author
#4 lore raw

PATCH 4/6 scsi: hisi_sas: Rename error labels in hisi_sas_v3_probe()

In reply to John Garry

From: Qi Liu <[email protected]>

To avoid doubt, rename the error labels to indicate the action they will take.

---
 drivers/scsi/hisi_sas/hisi_sas_v3_hw.c | 20 ++++++++++----------
 1 file changed, 10 insertions(+), 10 deletions(-)
drivers/scsi/hisi_sas/hisi_sas_v3_hw.c +10 -10
index 5b5557cab7ed..43d57b68c208 100644
@@ -4723,7 +4723,7 @@ hisi_sas_v3_probe(struct pci_dev *pdev, const struct pci_device_id *id)
 	if (!hisi_hba->regs) {
 		dev_err(dev, "cannot map register\n");
 		rc = -ENOMEM;
-		goto err_out_ha;
+		goto err_out_free_host;
 	}
 
 	phy_nr = port_nr = hisi_hba->n_phy;
@@ -4732,7 +4732,7 @@ hisi_sas_v3_probe(struct pci_dev *pdev, const struct pci_device_id *id)
 	arr_port = devm_kcalloc(dev, port_nr, sizeof(void *), GFP_KERNEL);
 	if (!arr_phy || !arr_port) {
 		rc = -ENOMEM;
-		goto err_out_ha;
+		goto err_out_free_host;
 	}
 
 	sha->sas_phy = arr_phy;
@@ -4773,19 +4773,19 @@ hisi_sas_v3_probe(struct pci_dev *pdev, const struct pci_device_id *id)
 
 	rc = interrupt_preinit_v3_hw(hisi_hba);
 	if (rc)
-		goto err_out_debugfs;
+		goto err_out_undo_debugfs;
 
 	rc = scsi_add_host(shost, dev);
 	if (rc)
-		goto err_out_debugfs;
+		goto err_out_undo_debugfs;
 
 	rc = sas_register_ha(sha);
 	if (rc)
-		goto err_out_register_ha;
+		goto err_out_remove_host;
 
 	rc = hisi_sas_v3_init(hisi_hba);
 	if (rc)
-		goto err_out_hw_init;
+		goto err_out_unregister_ha;
 
 	scsi_scan_host(shost);
 
@@ -4804,13 +4804,13 @@ hisi_sas_v3_probe(struct pci_dev *pdev, const struct pci_device_id *id)
 
 	return 0;
 
-err_out_hw_init:
+err_out_unregister_ha:
 	sas_unregister_ha(sha);
-err_out_register_ha:
+err_out_remove_host:
 	scsi_remove_host(shost);
-err_out_debugfs:
+err_out_undo_debugfs:
 	debugfs_exit_v3_hw(hisi_hba);
-err_out_ha:
+err_out_free_host:
 	hisi_sas_free(hisi_hba);
 	scsi_host_put(shost);
 err_out:
Signature · 1 line
2.26.2
author
#5 lore raw

PATCH 5/6 scsi: hisi_sas: Limit users changing debugfs BIST count value

In reply to John Garry

From: Xiang Chen <[email protected]>

Add a file operation for "cnt" file under bist directory, so users could only read "cnt" or clear "cnt" to zero, but cannot randomly modify.

---
 drivers/scsi/hisi_sas/hisi_sas_v3_hw.c | 52 +++++++++++++++++++++++++-
 1 file changed, 50 insertions(+), 2 deletions(-)
drivers/scsi/hisi_sas/hisi_sas_v3_hw.c +50 -2
index 43d57b68c208..ad3e2db0f520 100644
@@ -3975,6 +3975,54 @@ static const struct file_operations debugfs_bist_phy_v3_hw_fops = {
 	.owner = THIS_MODULE,
 };
 
+static ssize_t debugfs_bist_cnt_v3_hw_write(struct file *filp,
+					const char __user *buf,
+					size_t count, loff_t *ppos)
+{
+	struct seq_file *m = filp->private_data;
+	struct hisi_hba *hisi_hba = m->private;
+	unsigned int cnt;
+	int val;
+
+	if (hisi_hba->debugfs_bist_enable)
+		return -EPERM;
+
+	val = kstrtouint_from_user(buf, count, 0, &cnt);
+	if (val)
+		return val;
+
+	if (cnt)
+		return -EINVAL;
+
+	hisi_hba->debugfs_bist_cnt = 0;
+	return count;
+}
+
+static int debugfs_bist_cnt_v3_hw_show(struct seq_file *s, void *p)
+{
+	struct hisi_hba *hisi_hba = s->private;
+
+	seq_printf(s, "%u\n", hisi_hba->debugfs_bist_cnt);
+
+	return 0;
+}
+
+static int debugfs_bist_cnt_v3_hw_open(struct inode *inode,
+					  struct file *filp)
+{
+	return single_open(filp, debugfs_bist_cnt_v3_hw_show,
+			   inode->i_private);
+}
+
+static const struct file_operations debugfs_bist_cnt_v3_hw_ops = {
+	.open = debugfs_bist_cnt_v3_hw_open,
+	.read = seq_read,
+	.write = debugfs_bist_cnt_v3_hw_write,
+	.llseek = seq_lseek,
+	.release = single_release,
+	.owner = THIS_MODULE,
+};
+
 static const struct {
 	int		value;
 	char		*name;
@@ -4612,8 +4660,8 @@ static void debugfs_bist_init_v3_hw(struct hisi_hba *hisi_hba)
 	debugfs_create_file("phy_id", 0600, hisi_hba->debugfs_bist_dentry,
 			    hisi_hba, &debugfs_bist_phy_v3_hw_fops);
 
-	debugfs_create_u32("cnt", 0600, hisi_hba->debugfs_bist_dentry,
-			   &hisi_hba->debugfs_bist_cnt);
+	debugfs_create_file("cnt", 0600, hisi_hba->debugfs_bist_dentry,
+			    hisi_hba, &debugfs_bist_cnt_v3_hw_ops);
 
 	debugfs_create_file("loopback_mode", 0600,
 			    hisi_hba->debugfs_bist_dentry,
Signature · 1 line
2.26.2
author
#6 lore raw

PATCH 6/6 scsi: hisi_sas: Modify v3 HW SSP underflow error processing

In reply to John Garry

From: Xingui Yang <[email protected]>

In case of SSP underflow allow the response frame IU to be examined for setting the response stat value rather than always setting SAS_DATA_UNDERRUN.

This will mean that we call sas_ssp_task_response() in those scenarios and may send sense data to upper layer.

Such a condition would be for bad blocks were we just reporting an underflow error to upper layer, but now the sense data will tell immediately that the media is faulty.

---
 drivers/scsi/hisi_sas/hisi_sas_v3_hw.c | 39 +++++++++++++++++---------
 1 file changed, 26 insertions(+), 13 deletions(-)
drivers/scsi/hisi_sas/hisi_sas_v3_hw.c +26 -13
index ad3e2db0f520..914ae4e82f5e 100644
@@ -406,6 +406,8 @@
 #define CMPLT_HDR_ERROR_PHASE_MSK   (0xff << CMPLT_HDR_ERROR_PHASE_OFF)
 #define CMPLT_HDR_RSPNS_XFRD_OFF	10
 #define CMPLT_HDR_RSPNS_XFRD_MSK	(0x1 << CMPLT_HDR_RSPNS_XFRD_OFF)
+#define CMPLT_HDR_RSPNS_GOOD_OFF	11
+#define CMPLT_HDR_RSPNS_GOOD_MSK	(0x1 << CMPLT_HDR_RSPNS_GOOD_OFF)
 #define CMPLT_HDR_ERX_OFF		12
 #define CMPLT_HDR_ERX_MSK		(0x1 << CMPLT_HDR_ERX_OFF)
 #define CMPLT_HDR_ABORT_STAT_OFF	13
@@ -2140,7 +2142,7 @@ static irqreturn_t fatal_axi_int_v3_hw(int irq_no, void *p)
 	return IRQ_HANDLED;
 }
 
-static void
+static bool
 slot_err_v3_hw(struct hisi_hba *hisi_hba, struct sas_task *task,
 	       struct hisi_sas_slot *slot)
 {
@@ -2158,6 +2160,15 @@ slot_err_v3_hw(struct hisi_hba *hisi_hba, struct sas_task *task,
 	switch (task->task_proto) {
 	case SAS_PROTOCOL_SSP:
 		if (dma_rx_err_type & RX_DATA_LEN_UNDERFLOW_MSK) {
+			/*
+			 * If returned response frame is incorrect because of data underflow,
+			 * but I/O information has been written to the host memory, we examine
+			 * response IU.
+			 */
+			if (!(complete_hdr->dw0 & CMPLT_HDR_RSPNS_GOOD_MSK) &&
+				(complete_hdr->dw0 & CMPLT_HDR_RSPNS_XFRD_MSK))
+				return false;
+
 			ts->residual = trans_tx_fail_type;
 			ts->stat = SAS_DATA_UNDERRUN;
 		} else if (dw3 & CMPLT_HDR_IO_IN_TARGET_MSK) {
@@ -2189,6 +2200,7 @@ slot_err_v3_hw(struct hisi_hba *hisi_hba, struct sas_task *task,
 	default:
 		break;
 	}
+	return true;
 }
 
 static void slot_complete_v3_hw(struct hisi_hba *hisi_hba,
@@ -2262,19 +2274,20 @@ static void slot_complete_v3_hw(struct hisi_hba *hisi_hba,
 	if ((dw0 & CMPLT_HDR_CMPLT_MSK) == 0x3) {
 		u32 *error_info = hisi_sas_status_buf_addr_mem(slot);
 
-		slot_err_v3_hw(hisi_hba, task, slot);
-		if (ts->stat != SAS_DATA_UNDERRUN)
-			dev_info(dev, "erroneous completion iptt=%d task=%pK dev id=%d addr=%016llx CQ hdr: 0x%x 0x%x 0x%x 0x%x Error info: 0x%x 0x%x 0x%x 0x%x\n",
-				 slot->idx, task, sas_dev->device_id,
-				 SAS_ADDR(device->sas_addr),
-				 dw0, dw1, complete_hdr->act, dw3,
-				 error_info[0], error_info[1],
-				 error_info[2], error_info[3]);
-		if (unlikely(slot->abort)) {
-			sas_task_abort(task);
-			return;
+		if (slot_err_v3_hw(hisi_hba, task, slot)) {
+			if (ts->stat != SAS_DATA_UNDERRUN)
+				dev_info(dev, "erroneous completion iptt=%d task=%pK dev id=%d addr=%016llx CQ hdr: 0x%x 0x%x 0x%x 0x%x Error info: 0x%x 0x%x 0x%x 0x%x\n",
+					slot->idx, task, sas_dev->device_id,
+					SAS_ADDR(device->sas_addr),
+					dw0, dw1, complete_hdr->act, dw3,
+					error_info[0], error_info[1],
+					error_info[2], error_info[3]);
+			if (unlikely(slot->abort)) {
+				sas_task_abort(task);
+				return;
+			}
+			goto out;
 		}
-		goto out;
 	}
 
 	switch (task->task_proto) {
Signature · 1 line
2.26.2
#7 lore raw
In reply to John Garry

John,

Quoted 1 line

This series includes some updates as following:

Applied to 5.18/scsi-staging, thanks!

Signature · 1 line
Martin K. Petersen	Oracle Linux Engineering
#8 lore raw
In reply to John Garry
Quoted 11 lines · John Garry

This series includes some updates as following:
- better management of freeing PCI vectors
- hopefully the following series can make it soon so we may remove this

    new code:
    https://lore.kernel.org/linux-pci/[email protected]/T/#md27a594ac53b8e05abb2f22ec350137b9023ff18

- Fix prot_mask module param permissions
- Change phyup timeout to handle some special SATA disks
- modify underflow handling to quickly identify faulty disks
- debugfs hardening

[...]

Applied to 5.18/scsi-queue, thanks!

[1/6] scsi: hisi_sas: Change permission of parameter prot_mask

      https://git.kernel.org/mkp/scsi/c/c4e070457a93
[2/6] scsi: hisi_sas: Change hisi_sas_control_phy() phyup timeout
      https://git.kernel.org/mkp/scsi/c/512623de5239
[3/6] scsi: hisi_sas: Free irq vectors in order for v3 HW
      https://git.kernel.org/mkp/scsi/c/554fb72ee34f
[4/6] scsi: hisi_sas: Rename error labels in hisi_sas_v3_probe()
      https://git.kernel.org/mkp/scsi/c/86287065fac2
[5/6] scsi: hisi_sas: Limit users changing debugfs BIST count value
      https://git.kernel.org/mkp/scsi/c/286ce4c65fbd
[6/6] scsi: hisi_sas: Modify v3 HW SSP underflow error processing
      https://git.kernel.org/mkp/scsi/c/62413199cd6d
Signature · 1 line
Martin K. Petersen	Oracle Linux Engineering