Skip to contentSkip to conversation

smartpqi updates

PATCH Applied Don Brace 2 participants 9 messages Aug 27, 2024 – Aug 29, 2024

Subsystems (2)
lore
Share & tools
lore

Don Brace

author maintainer
#0 lore raw

cover smartpqi updates

These patches are based on Martin Petersen's 6.12/scsi-queue tree
https://git.kernel.org/pub/scm/linux/kernel/git/mkp/scsi.git
6.12/scsi-queue

There are two functional changes:
smartpqi-add-fw-log-to-kdump smartpqi-add-counter-for-parity-write-stream-requests

There are three minor bug fixes:
smartpqi-fix-stream-detection smartpqi-fix-rare-system-hang-during-LUN-reset smartpqi-fix-volume-size-updates

The other two patches add PCI-IDs for new controllers and change the driver version.

This set of changes consists of:
* smartpqi-add-fw-log-to-kdump

During a kdump, the driver tells the controller to copy its logging information to some pre-allocated buffers that can be analyzed later.

This is a "feature" driven capability and is backward compatible with existing controller FW.

This patch renames some prefixes for OFA (Online-Firmware Activation ofa_*) buffers to host_memory_*. So, not a lot of actual functional changes to smartpqi_init.c, mainly determining the memory size allocation.

We added a function to notify the controller to copy debug data into host memory before continuing kdump.

Most of the functional changes are in smartpqi_sis.c where the actual handshaking is done.

* smartpqi-fix-stream-detection

Correct some false write-stream detections. The data structure used to check for write-streams was not initialized to all 0's causing some false write stream detections. The driver sends down streamed requests to the raid engine instead of using AIO bypass for some extra performance. (Potential full-stripe write verses Read Modify Write).

False detections have not caused any data corruption. Found by internal testing. No known externally reported bugs.

* smartpqi-add-counter-for-parity-write-stream-requests

Adding some counters for raid_bypass and write streams. These two counters are related because write stream detection is only checked if an I/O request is eligible for bypass (AIO).

The bypass counter (raid_bypass_cnt) was moved into a common structure (pqi_raid_io_stats) and changed to type __percpu. The write stream counter is (write_stream_cnt) has been added to this same structure.

These counters are __percpu counters for performance. We added a sysfs entry to show the write stream count. The raid bypass counter sysfs entry already exists.

Useful for checking streaming writes. The change in the sysfs entry write_stream_cnt can be checked during AIO eligible write operations.

* smartpqi-add-new-controller-PCI-IDs

Adding support for new controller HW.
No functional changes.

* smartpqi-fix-rare-system-hang-during-LUN-reset

We found a rare race condition that can occur during a LUN reset. We were not emptying our internal queue completely.

There have been some rare conditions where our internal request queue has requests for multiple LUNs and a reset comes in for one of the LUNs. The driver waits for this internal queue to empty. We were only clearing out the requests for the LUN being reset so the request queue was never empty causing a hang.

The Fix:
For all requests in our internal request queue:
Complete requests with DID_RESET for queued requests for the device undergoing a reset. Complete requests with DID_REQUEUE for all other queued requests.

Found by internal testing. No known externally reported bugs.

* smartpqi-fix-volume-size-updates

The current code only checks for a size change if there is also a queue depth change. We are separating the check for queue depth and the size changes.

Found by internal testing. No known bugs were filed.

* smartpqi-update-version-to-2.1.30-031
No functional changes.

---

David Strahan (1):
smartpqi: add new controller PCI IDs

Don Brace (2):
smartpqi: fix volume size updates
smartpqi: update driver version to 2.1.30-031

Mahesh Rajashekhara (2):
smartpqi: correct stream detection
smartpqi: add counter for parity write stream requests

Murthy Bhat (2):
smartpqi: Add fw log to kdump
smartpqi: fix rare system hang during LUN reset

 drivers/scsi/smartpqi/smartpqi.h      |  39 ++-
 drivers/scsi/smartpqi/smartpqi_init.c | 352 +++++++++++++++++---------
 drivers/scsi/smartpqi/smartpqi_sis.c  |  60 +++++
 drivers/scsi/smartpqi/smartpqi_sis.h  |   3 +
 4 files changed, 322 insertions(+), 132 deletions(-)
Signature · 1 line
2.46.0.421.g159f2d50e7
author maintainer
#1 lore raw

PATCH 1/7 smartpqi: Add fw log to kdump

From: Murthy Bhat <[email protected]>

Add controller logs to kdump.

Driver allocates DMA memory and communicates this address to FW. In the event of system crash, host driver notifies the firmware about the crash and firmware posts all the necessary logs in the pre-allocated host buffer for firmware debugging.

Once firmware notifies the completion of the log uploading to the host memory and host continues with the OS crash dump saving.

This is a "feature" driven capability and is backward compatible with existing controller FW.

Rename some prefixes for OFA (Online-Firmware Activation ofa_*) buffers to host_memory_*. So, not a lot of actual functional changes to smartpqi_init.c, mainly determining the memory size allocation.

Added a function to notify the controller to copy debug data into host memory before continuing kdump.

Most of the functional changes are in smartpqi_sis.c where the actual handshaking is done.

---
 drivers/scsi/smartpqi/smartpqi.h      |  32 ++--
 drivers/scsi/smartpqi/smartpqi_init.c | 226 ++++++++++++++++----------
 drivers/scsi/smartpqi/smartpqi_sis.c  |  60 +++++++
 drivers/scsi/smartpqi/smartpqi_sis.h  |   3 +
 4 files changed, 221 insertions(+), 100 deletions(-)
drivers/scsi/smartpqi/smartpqi.h +21 -11
index 023fbce04e7a..f493006bee9d 100644
@@ -505,7 +505,7 @@ struct pqi_vendor_general_request {
 			__le64	buffer_address;
 			__le32	buffer_length;
 			u8	reserved[40];
-		} ofa_memory_allocation;
+		} host_memory_allocation;
 	} data;
 };
 
@@ -517,21 +517,30 @@ struct pqi_vendor_general_response {
 	u8	reserved[2];
 };
 
-#define PQI_VENDOR_GENERAL_CONFIG_TABLE_UPDATE	0
-#define PQI_VENDOR_GENERAL_HOST_MEMORY_UPDATE	1
+#define PQI_VENDOR_GENERAL_CONFIG_TABLE_UPDATE		0
+#define PQI_VENDOR_GENERAL_OFA_MEMORY_UPDATE		1
+#define PQI_VENDOR_GENERAL_CTRL_LOG_MEMORY_UPDATE	2
 
 #define PQI_OFA_VERSION			1
 #define PQI_OFA_SIGNATURE		"OFA_QRM"
-#define PQI_OFA_MAX_SG_DESCRIPTORS	64
+#define PQI_CTRL_LOG_VERSION		1
+#define PQI_CTRL_LOG_SIGNATURE		"FW_DATA"
+#define PQI_HOST_MAX_SG_DESCRIPTORS	64
 
-struct pqi_ofa_memory {
-	__le64	signature;	/* "OFA_QRM" */
+struct pqi_host_memory {
+	__le64	signature;	/* "OFA_QRM", "FW_DATA", etc. */
 	__le16	version;	/* version of this struct (1 = 1st version) */
 	u8	reserved[62];
 	__le32	bytes_allocated;	/* total allocated memory in bytes */
 	__le16	num_memory_descriptors;
 	u8	reserved1[2];
-	struct pqi_sg_descriptor sg_descriptor[PQI_OFA_MAX_SG_DESCRIPTORS];
+	struct pqi_sg_descriptor sg_descriptor[PQI_HOST_MAX_SG_DESCRIPTORS];
+};
+
+struct pqi_host_memory_descriptor {
+	struct pqi_host_memory *host_memory;
+	dma_addr_t		host_memory_dma_handle;
+	void			**host_chunk_virt_address;
 };
 
 struct pqi_aio_error_info {
@@ -867,7 +876,8 @@ struct pqi_config_table_firmware_features {
 #define PQI_FIRMWARE_FEATURE_FW_TRIAGE				17
 #define PQI_FIRMWARE_FEATURE_RPL_EXTENDED_FORMAT_4_5		18
 #define PQI_FIRMWARE_FEATURE_MULTI_LUN_DEVICE_SUPPORT           21
-#define PQI_FIRMWARE_FEATURE_MAXIMUM                            21
+#define PQI_FIRMWARE_FEATURE_CTRL_LOGGING			22
+#define PQI_FIRMWARE_FEATURE_MAXIMUM				22
 
 struct pqi_config_table_debug {
 	struct pqi_config_table_section_header header;
@@ -1357,6 +1367,7 @@ struct pqi_ctrl_info {
 	u8		firmware_triage_supported : 1;
 	u8		rpl_extended_format_4_5_supported : 1;
 	u8		multi_lun_device_supported : 1;
+	u8		ctrl_logging_supported : 1;
 	u8		enable_r1_writes : 1;
 	u8		enable_r5_writes : 1;
 	u8		enable_r6_writes : 1;
@@ -1398,13 +1409,12 @@ struct pqi_ctrl_info {
 	wait_queue_head_t block_requests_wait;
 
 	struct mutex	ofa_mutex;
-	struct pqi_ofa_memory *pqi_ofa_mem_virt_addr;
-	dma_addr_t	pqi_ofa_mem_dma_handle;
-	void		**pqi_ofa_chunk_virt_addr;
 	struct work_struct ofa_memory_alloc_work;
 	struct work_struct ofa_quiesce_work;
 	u32		ofa_bytes_requested;
 	u16		ofa_cancel_reason;
+	struct pqi_host_memory_descriptor ofa_memory;
+	struct pqi_host_memory_descriptor ctrl_log_memory;
 	enum pqi_ctrl_removal_state ctrl_removal_state;
 };
 
drivers/scsi/smartpqi/smartpqi_init.c +137 -89

409 lines not on this page · raw message

drivers/scsi/smartpqi/smartpqi_sis.c +60 -0

90 lines not on this page · raw message

drivers/scsi/smartpqi/smartpqi_sis.h +3 -0
index 0c97626d87d4..7e0eac3d07de 100644
@@ -31,6 +31,9 @@ u32 sis_read_driver_scratch(struct pqi_ctrl_info *ctrl_info);
 void sis_soft_reset(struct pqi_ctrl_info *ctrl_info);
 u32 sis_get_product_id(struct pqi_ctrl_info *ctrl_info);
 int sis_wait_for_fw_triage_completion(struct pqi_ctrl_info *ctrl_info);
+bool sis_is_ctrl_logging_supported(struct pqi_ctrl_info *ctrl_info);
+void sis_notify_kdump(struct pqi_ctrl_info *ctrl_info);
+int sis_wait_for_ctrl_logging_completion(struct pqi_ctrl_info *ctrl_info);
 
 extern unsigned int sis_ctrl_ready_timeout_secs;
 
Signature · 1 line
2.46.0.421.g159f2d50e7
author maintainer
#2 lore raw

PATCH 2/7 smartpqi: correct stream detection

In reply to Don Brace

From: Mahesh Rajashekhara <[email protected]>

Correct stream detection by initializing the structure pqi_scsi_dev_raid_map_data to 0s.

When the OS issues SCSI READ commands, the driver erroneously considers them as SCSI WRITES. If they are identified as sequential IOs, the driver then submits those requests via the RAID path instead of the AIO path.

The 'is_write' flag might be set for SCSI READ commands also. The driver may interpret SCSI READ commands as SCSI WRITE commands, resulting in IOs being submitted through the RAID path.

Note: This does not cause data corruption.

---
 drivers/scsi/smartpqi/smartpqi_init.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/scsi/smartpqi/smartpqi_init.c +1 -1
index 54f7fe843445..8e2e71ab49ae 100644
@@ -5942,7 +5942,7 @@ static bool pqi_is_parity_write_stream(struct pqi_ctrl_info *ctrl_info,
 	int rc;
 	struct pqi_scsi_dev *device;
 	struct pqi_stream_data *pqi_stream_data;
-	struct pqi_scsi_dev_raid_map_data rmd;
+	struct pqi_scsi_dev_raid_map_data rmd = { 0 };
 
 	if (!ctrl_info->enable_stream_detection)
 		return false;
Signature · 1 line
2.46.0.421.g159f2d50e7
author maintainer
#3 lore raw

PATCH 3/7 smartpqi: add counter for parity write stream requests

In reply to Don Brace

From: Mahesh Rajashekhara <[email protected]>

Add sysfs entry to check for write stream requests.

Move existing raid_bypass_cnt into a structure named pqi_raid_io_stats and add member write_stream_cnt. These two counters are related because write stream detection is only checked if an I/O request is eligible for bypass (AIO).

Example usage:

lsscsi
[15:1:0:0] disk Adaptec LOGICAL VOLUME 0129 /dev/sdae

cat /sys/block/sdae/device/ssd_smart_path_enabled 1
^
|
+---- NOTE: here bypass has been enabled on device sdae

To read the counter for parity write stream requests:

cat /sys/block/sdae/device/write_stream_cnt
0x60cd507

---
 drivers/scsi/smartpqi/smartpqi.h      |  7 ++-
 drivers/scsi/smartpqi/smartpqi_init.c | 62 +++++++++++++++++++++------
 2 files changed, 55 insertions(+), 14 deletions(-)
drivers/scsi/smartpqi/smartpqi.h +6 -1
index f493006bee9d..fae6db20a6e9 100644
@@ -1106,6 +1106,11 @@ struct pqi_tmf_work {
 	u8	scsi_opcode;
 };
 
+struct pqi_raid_io_stats {
+	u64	raid_bypass_cnt;
+	u64	write_stream_cnt;
+};
+
 struct pqi_scsi_dev {
 	int	devtype;		/* as reported by INQUIRY command */
 	u8	device_type;		/* as reported by */
@@ -1168,7 +1173,7 @@ struct pqi_scsi_dev {
 
 	struct pqi_stream_data stream_data[NUM_STREAMS_PER_LUN];
 	atomic_t scsi_cmds_outstanding[PQI_MAX_LUNS_PER_DEVICE];
-	u64 __percpu *raid_bypass_cnt;
+	struct pqi_raid_io_stats __percpu *raid_io_stats;
 
 	struct pqi_tmf_work tmf_work[PQI_MAX_LUNS_PER_DEVICE];
 };
drivers/scsi/smartpqi/smartpqi_init.c +49 -13

129 lines not on this page · raw message

Signature · 1 line
2.46.0.421.g159f2d50e7
author maintainer
#4 lore raw

PATCH 4/7 smartpqi: add new controller PCI IDs

In reply to Don Brace

From: David Strahan <[email protected]>

All PCI ID entries in Hex.

Add new cisco pci ids:

                                             VID  / DID  / SVID / SDID
                                             ----   ----   ----   ----
                                             9005   028f   1137   02fe
                                             9005   028f   1137   02ff
                                             9005   028f   1137   0300

Add new h3c pci ids:

                                             VID  / DID  / SVID / SDID
                                             ----   ----   ----   ----
                                             9005   028f   193d   0462
                                             9005   028f   193d   8462

Add new ieit pci ids:

                                             VID  / DID  / SVID / SDID
                                             ----   ----   ----   ----
                                             9005   028f   1ff9   00a3
---
 drivers/scsi/smartpqi/smartpqi_init.c | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)
drivers/scsi/smartpqi/smartpqi_init.c +24 -0
index 6a941735c982..46bef2cf95c4 100644
@@ -9548,6 +9548,10 @@ static const struct pci_device_id pqi_pci_id_table[] = {
 		PCI_DEVICE_SUB(PCI_VENDOR_ID_ADAPTEC2, 0x028f,
 			       0x152d, 0x8a37)
 	},
+	{
+		PCI_DEVICE_SUB(PCI_VENDOR_ID_ADAPTEC2, 0x028f,
+			       0x193d, 0x0462)
+	},
 	{
 		PCI_DEVICE_SUB(PCI_VENDOR_ID_ADAPTEC2, 0x028f,
 			       0x193d, 0x1104)
@@ -9588,6 +9592,10 @@ static const struct pci_device_id pqi_pci_id_table[] = {
 		PCI_DEVICE_SUB(PCI_VENDOR_ID_ADAPTEC2, 0x028f,
 			       0x193d, 0x8461)
 	},
+	{
+		PCI_DEVICE_SUB(PCI_VENDOR_ID_ADAPTEC2, 0x028f,
+			       0x193d, 0x8462)
+	},
 	{
 		PCI_DEVICE_SUB(PCI_VENDOR_ID_ADAPTEC2, 0x028f,
 			       0x193d, 0xc460)
@@ -10296,6 +10304,18 @@ static const struct pci_device_id pqi_pci_id_table[] = {
 		PCI_DEVICE_SUB(PCI_VENDOR_ID_ADAPTEC2, 0x028f,
 			       0x1137, 0x02fa)
 	},
+	{
+		PCI_DEVICE_SUB(PCI_VENDOR_ID_ADAPTEC2, 0x028f,
+			       0x1137, 0x02fe)
+	},
+	{
+		PCI_DEVICE_SUB(PCI_VENDOR_ID_ADAPTEC2, 0x028f,
+			       0x1137, 0x02ff)
+	},
+	{
+		PCI_DEVICE_SUB(PCI_VENDOR_ID_ADAPTEC2, 0x028f,
+			       0x1137, 0x0300)
+	},
 	{
 		PCI_DEVICE_SUB(PCI_VENDOR_ID_ADAPTEC2, 0x028f,
 			       0x1ff9, 0x0045)
@@ -10472,6 +10492,10 @@ static const struct pci_device_id pqi_pci_id_table[] = {
 		PCI_DEVICE_SUB(PCI_VENDOR_ID_ADAPTEC2, 0x028f,
 			       0x1f51, 0x1045)
 	},
+	{
+		PCI_DEVICE_SUB(PCI_VENDOR_ID_ADAPTEC2, 0x028f,
+			       0x1ff9, 0x00a3)
+	},
 	{
 		PCI_DEVICE_SUB(PCI_VENDOR_ID_ADAPTEC2, 0x028f,
 			       PCI_ANY_ID, PCI_ANY_ID)
Signature · 1 line
2.46.0.421.g159f2d50e7
author maintainer
#5 lore raw

PATCH 5/7 smartpqi: fix rare system hang during LUN reset

In reply to Don Brace

From: Murthy Bhat <[email protected]>

Correct a rare case where in a LUN reset occurs on a device and I/O requests for other devices persist in the driver's internal request queue.

Part of a LUN reset involves waiting for our internal request queue to empty before proceeding. The internal request queue contains requests not yet sent down to the controller.

We were clearing the requests queued for the LUN undergoing a reset, but not all of the queued requests. Causing a hang.

For all requests in our internal request queue:
Complete requests with DID_RESET for queued requests for the device undergoing a reset.

Complete requests with DID_REQUEUE for all other queued requests.

---
 drivers/scsi/smartpqi/smartpqi_init.c | 10 ++++------
 1 file changed, 4 insertions(+), 6 deletions(-)
drivers/scsi/smartpqi/smartpqi_init.c +4 -6
index 46bef2cf95c4..d1d117d5d08d 100644
@@ -6200,14 +6200,12 @@ static void pqi_fail_io_queued_for_device(struct pqi_ctrl_info *ctrl_info,
 					continue;
 
 				scsi_device = scmd->device->hostdata;
-				if (scsi_device != device)
-					continue;
-
-				if ((u8)scmd->device->lun != lun)
-					continue;
 
 				list_del(&io_request->request_list_entry);
-				set_host_byte(scmd, DID_RESET);
+				if (scsi_device == device && (u8)scmd->device->lun == lun)
+					set_host_byte(scmd, DID_RESET);
+				else
+					set_host_byte(scmd, DID_REQUEUE);
 				pqi_free_io_request(io_request);
 				scsi_dma_unmap(scmd);
 				pqi_scsi_done(scmd);
Signature · 1 line
2.46.0.421.g159f2d50e7
author maintainer
#6 lore raw

PATCH 6/7 smartpqi: fix volume size updates

In reply to Don Brace

Correct logical volume size changes by moving the check for a volume rescan outside of the check for a queue depth change.

---
 drivers/scsi/smartpqi/smartpqi_init.c | 22 ++++++++++++++--------
 1 file changed, 14 insertions(+), 8 deletions(-)
drivers/scsi/smartpqi/smartpqi_init.c +14 -8
index d1d117d5d08d..0dab30753f0a 100644
@@ -2303,17 +2303,23 @@ static void pqi_update_device_list(struct pqi_ctrl_info *ctrl_info,
 	 * queue depth, device size.
 	 */
 	list_for_each_entry(device, &ctrl_info->scsi_device_list, scsi_device_list_entry) {
+		/*
+		 * Check for queue depth change.
+		 */
 		if (device->sdev && device->queue_depth != device->advertised_queue_depth) {
 			device->advertised_queue_depth = device->queue_depth;
 			scsi_change_queue_depth(device->sdev, device->advertised_queue_depth);
-			spin_lock_irqsave(&ctrl_info->scsi_device_list_lock, flags);
-			if (pqi_volume_rescan_needed(device)) {
-				device->rescan = false;
-				spin_unlock_irqrestore(&ctrl_info->scsi_device_list_lock, flags);
-				scsi_rescan_device(device->sdev);
-			} else {
-				spin_unlock_irqrestore(&ctrl_info->scsi_device_list_lock, flags);
-			}
+		}
+		spin_lock_irqsave(&ctrl_info->scsi_device_list_lock, flags);
+		/*
+		 * Check for changes in the device, such as size.
+		 */
+		if (pqi_volume_rescan_needed(device)) {
+			device->rescan = false;
+			spin_unlock_irqrestore(&ctrl_info->scsi_device_list_lock, flags);
+			scsi_rescan_device(device->sdev);
+		} else {
+			spin_unlock_irqrestore(&ctrl_info->scsi_device_list_lock, flags);
 		}
 	}
 
Signature · 1 line
2.46.0.421.g159f2d50e7
author maintainer
#7 lore raw

PATCH 7/7 smartpqi: update driver version to 2.1.30-031

In reply to Don Brace

Update driver version to 2.1.30-031

---
 drivers/scsi/smartpqi/smartpqi_init.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
drivers/scsi/smartpqi/smartpqi_init.c +3 -3
index 0dab30753f0a..7fd5a8c813dc 100644
@@ -33,11 +33,11 @@
 #define BUILD_TIMESTAMP
 #endif
 
-#define DRIVER_VERSION		"2.1.28-025"
+#define DRIVER_VERSION		"2.1.30-031"
 #define DRIVER_MAJOR		2
 #define DRIVER_MINOR		1
-#define DRIVER_RELEASE		28
-#define DRIVER_REVISION		25
+#define DRIVER_RELEASE		30
+#define DRIVER_REVISION		31
 
 #define DRIVER_NAME		"Microchip SmartPQI Driver (v" \
 				DRIVER_VERSION BUILD_TIMESTAMP ")"
Signature · 1 line
2.46.0.421.g159f2d50e7