Skip to contentSkip to conversation

UFS Host Performance Booster (HPB v1.0) driver

PATCH Superseded [email protected] 6 participants 30 messages Apr 16, 2020 – May 2, 2020 SCSI

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

v2 cover UFS Host Performance Booster (HPB v1.0) driver

From: Bean Huo <[email protected]>

Hi,
I disclose my development/changes on UFS HPB (UFS Host Performance Booster) to the community, as this would enable more UFS developers to test and start an iterative review and update process.

The HPB is defined in Jedec Standard Universal Flash (UFS) Host Performance Booster (HPB) Extension Version 1.0, which is designed to improve read performance by utilizing the host side memory. Based on our testing, the HPB can increase the random read performance by up to about 46% in the random read.

The original HPB driver is from [1]. Based on it, I did optimizations, simplifications, fixed reliability issues, implemented HPB host control mode and make it much more readable. which's pushed to here [2] as V1.

To avoid touching the traditional SCSI core, the HPB driver in this series HPB patch chooses to develop under SCSI sub-system layer, and sits the same layer with UFSHCD. At the same time, to minimize changes to UFSHCD driver, the HPB driver submits its HPB READ BUFFER and HPB WRITE BUFFER requests to the scsi device->request_queueu to execute, rather than that directly go through raw UPIU request path.

v1--v2:
1. Rebased the patch on the [3] branch 5.8/scsi-queue
1. Optimized and simplified several functions
2. Add parameter read_threshold in HPB sysfs, by which the user can change read_threshold for the HPB host control mode
3. Add HPB memory limitation, let the user adjust its size according to the system memory capacity

[1] https://github.com/OpenMPDK/HPBDriver
[2] https://www.spinics.net/lists/kernel/msg3449471.html
[3] https://git.kernel.org/pub/scm/linux/kernel/git/mkp/scsi.git

Bean Huo (5):
scsi; ufs: add device descriptor for Host Performance Booster
scsi: ufs: make ufshcd_read_unit_desc_param() non-static func
scsi: ufs: add ufs_features parameter in structure ufs_dev_info
scsi: ufs: add unit and geometry parameters for HPB
scsi: ufs: UFS Host Performance Booster(HPB) driver

 drivers/scsi/ufs/Kconfig  |   62 +
 drivers/scsi/ufs/Makefile |    1 +
 drivers/scsi/ufs/ufs.h    |   15 +
 drivers/scsi/ufs/ufshcd.c |   66 +-
 drivers/scsi/ufs/ufshcd.h |   15 +-
 drivers/scsi/ufs/ufshpb.c | 3279 +++++++++++++++++++++++++++++++++++++
 drivers/scsi/ufs/ufshpb.h |  450 +++++
 7 files changed, 3881 insertions(+), 7 deletions(-)
 create mode 100644 drivers/scsi/ufs/ufshpb.c
 create mode 100644 drivers/scsi/ufs/ufshpb.h
Signature · 1 line
2.17.1
#1 lore raw

PATCH v2 1/5 scsi; ufs: add device descriptor for Host Performance Booster

From: Bean Huo <[email protected]>

Add HPB related attributes in devivce descriptor. HPB support is specified by Bit7 of bUFSFeatureSupport, HPB version is indicated by wHPBVersion and the HPB Control Mode is specified by bHPBControl in UFS device descriptor.

---
 drivers/scsi/ufs/ufs.h    |  6 ++++++
 drivers/scsi/ufs/ufshcd.c | 11 +++++++++++
 2 files changed, 17 insertions(+)
drivers/scsi/ufs/ufs.h +6 -0
index b7fec5c73688..53a5e263f7c8 100644
@@ -258,6 +258,8 @@ enum device_desc_param {
 	DEVICE_DESC_PARAM_PSA_MAX_DATA		= 0x25,
 	DEVICE_DESC_PARAM_PSA_TMT		= 0x29,
 	DEVICE_DESC_PARAM_PRDCT_REV		= 0x2A,
+	DEVICE_DESC_PARAM_HPB_VER		= 0x40,
+	DEVICE_DESC_PARAM_HPB_CTRL_MODE		= 0x42,
 };
 
 /* Interconnect descriptor parameters offsets in bytes*/
@@ -537,6 +539,10 @@ struct ufs_dev_info {
 	u8 *model;
 	u16 wspecversion;
 	u32 clk_gating_wait_us;
+	/* HPB Version */
+	u16 hpb_ver;
+	/* bHPBControl */
+	u8 hpb_control_mode;
 };
 
 /**
drivers/scsi/ufs/ufshcd.c +11 -0
index 698e8d20b4ba..de13d2333f1f 100644
@@ -6627,6 +6627,17 @@ static int ufs_get_device_desc(struct ufs_hba *hba)
 		goto out;
 	}
 
+	if (desc_buf[DEVICE_DESC_PARAM_UFS_FEAT] & 0x80) {
+		hba->dev_info.hpb_control_mode =
+			desc_buf[DEVICE_DESC_PARAM_HPB_CTRL_MODE];
+		hba->dev_info.hpb_ver =
+			(u16) (desc_buf[DEVICE_DESC_PARAM_HPB_VER] << 8) |
+			desc_buf[DEVICE_DESC_PARAM_HPB_VER + 1];
+		dev_info(hba->dev, "HPB Version: 0x%2x\n",
+			 hba->dev_info.hpb_ver);
+		dev_info(hba->dev, "HPB control mode: %d\n",
+			 hba->dev_info.hpb_control_mode);
+	}
 	/*
 	 * getting vendor (manufacturerID) and Bank Index in big endian
 	 * format
Signature · 1 line
2.17.1
#2 lore raw

Re: [PATCH v2 1/5] scsi; ufs: add device descriptor for Host Performance Booster

drivers/scsi/ufs/ufshcd.c static int ufs_get_device_desc(struct ufs_hba *hba) · line 6630

Quoted 1 line · On 4/16/20 1:31 PM, [email protected] wrote:
drivers/scsi/ufs/ufshcd.c +1 -0
+	if (desc_buf[DEVICE_DESC_PARAM_UFS_FEAT] & 0x80) {

Please introduce a symbolic name instead of using the number 0x80 directly.

drivers/scsi/ufs/ufshcd.c static int ufs_get_device_desc(struct ufs_hba *hba) · line 6635

Quoted 5 lines
drivers/scsi/ufs/ufshcd.c +5 -0
+		hba->dev_info.hpb_control_mode =
+			desc_buf[DEVICE_DESC_PARAM_HPB_CTRL_MODE];
+		hba->dev_info.hpb_ver =
+			(u16) (desc_buf[DEVICE_DESC_PARAM_HPB_VER] << 8) |
+			desc_buf[DEVICE_DESC_PARAM_HPB_VER + 1];

Please use get_unaligned_be16() instead of open-coding it.

Thanks,

Bart.

#3 lore raw

RE: [EXT] Re: [PATCH v2 1/5] scsi; ufs: add device descriptor for Host Performance Booster

Hi Bart
Thanks your review, I will take your suggestions in next version development. Thanks,

Bean

Quoted 18 lines · On 4/16/20 1:31 PM, [email protected] wrote:

drivers/scsi/ufs/ufshcd.c static int ufs_get_device_desc(struct ufs_hba *hba) · line 6630

Quoted 1 line
drivers/scsi/ufs/ufshcd.c +1 -0
+	if (desc_buf[DEVICE_DESC_PARAM_UFS_FEAT] & 0x80) {

Please introduce a symbolic name instead of using the number 0x80 directly.

drivers/scsi/ufs/ufshcd.c static int ufs_get_device_desc(struct ufs_hba *hba)

Quoted 4 lines
drivers/scsi/ufs/ufshcd.c +4 -0
+		hba->dev_info.hpb_control_mode =
+			desc_buf[DEVICE_DESC_PARAM_HPB_CTRL_MODE];
+		hba->dev_info.hpb_ver =
+			(u16) (desc_buf[DEVICE_DESC_PARAM_HPB_VER] << 8)

|

drivers/scsi/ufs/ufshcd.c static int ufs_get_device_desc(struct ufs_hba *hba) · line 6635

Quoted 1 line
drivers/scsi/ufs/ufshcd.c +1 -0
+			desc_buf[DEVICE_DESC_PARAM_HPB_VER + 1];

Please use get_unaligned_be16() instead of open-coding it.

Thanks,

Bart.

#4 lore raw

Re: [EXT] Re: [PATCH v2 1/5] scsi; ufs: add device descriptor for Host Performance Booster

Quoted 1 line · Bean Huo (beanhuo)

Thanks your review, I will take your suggestions in next version development.

Hi Bean,

Please make sure that the issues raised by Christoph are addressed before reposting this series.

Additionally, I think the following two call chains are deadlock-prone and should be addressed:

ufshcd_queuecommand()
-> ufshcd_comp_scsi_upiu()
-> ufshpb_prep_fn()

    -> schedule_work(&hpb->ufshpb_req_work)
      -> ufshpb_req_workq_fn()
        -> ufshpb_subregion_activate()
          -> ufshpb_subregion_l2p_load()
            -> ufshpb_l2p_load_req()
              -> blk_execute_rq_nowait()

ufshcd_queuecommand()
-> ufshcd_comp_scsi_upiu()
-> ufshpb_prep_fn()

    -> schedule_work(&hpb->ufshpb_req_work)
      -> ufshpb_req_workq_fn()
        -> ufshpb_subregion_activate()
        -> ufshpb_region_deactivate()
          -> ufshpb_execute_cmd()
            -> scsi_execute()

I don't know any other block or SCSI driver that calls blk_execute_rq*() or scsi_execute() from inside its .queue_rq() callback to queue requests onto the same request queue (the device mapper requeues requests onto a lower level request queue).

Thanks,

Bart.

#5 lore raw

PATCH v2 2/5 scsi: ufs: make ufshcd_read_unit_desc_param() non-static func

In reply to [email protected]

From: Bean Huo <[email protected]>

HPB driver needs to read unit descriptors through ufshcd_read_unit_desc_param(), make it an extern Function.

---
 drivers/scsi/ufs/ufshcd.c | 8 +++-----
 drivers/scsi/ufs/ufshcd.h | 5 ++++-
 2 files changed, 7 insertions(+), 6 deletions(-)
drivers/scsi/ufs/ufshcd.c +3 -5
index de13d2333f1f..83ed2879d930 100644
@@ -3326,11 +3326,9 @@ int ufshcd_read_string_desc(struct ufs_hba *hba, u8 desc_index,
  *
  * Return 0 in case of success, non-zero otherwise
  */
-static inline int ufshcd_read_unit_desc_param(struct ufs_hba *hba,
-					      int lun,
-					      enum unit_desc_param param_offset,
-					      u8 *param_read_buf,
-					      u32 param_size)
+int ufshcd_read_unit_desc_param(struct ufs_hba *hba, int lun,
+				enum unit_desc_param param_offset,
+				u8 *param_read_buf, u32 param_size)
 {
 	/*
 	 * Unit descriptors are only available for general purpose LUs (LUN id
drivers/scsi/ufs/ufshcd.h +4 -1
index 6ffc08ad85f6..7ce9cc2f10fe 100644
@@ -856,7 +856,10 @@ extern int ufshcd_dme_set_attr(struct ufs_hba *hba, u32 attr_sel,
 extern int ufshcd_dme_get_attr(struct ufs_hba *hba, u32 attr_sel,
 			       u32 *mib_val, u8 peer);
 extern int ufshcd_config_pwr_mode(struct ufs_hba *hba,
-			struct ufs_pa_layer_attr *desired_pwr_mode);
+				  struct ufs_pa_layer_attr *desired_pwr_mode);
+extern int ufshcd_read_unit_desc_param(struct ufs_hba *hba, int lun,
+				       enum unit_desc_param param_offset,
+				       u8 *param_read_buf, u32 param_size);
 
 /* UIC command interfaces for DME primitives */
 #define DME_LOCAL	0
Signature · 1 line
2.17.1
#6 lore raw

PATCH v2 3/5 scsi: ufs: add ufs_features parameter in structure ufs_dev_info

In reply to [email protected]

From: Bean Huo <[email protected]>

Make a copy of bUFSFeaturesSupport, name it ufs_features, add it to structure ufs_dev_info.

---
 drivers/scsi/ufs/ufs.h    | 2 ++
 drivers/scsi/ufs/ufshcd.c | 2 ++
 2 files changed, 4 insertions(+)
drivers/scsi/ufs/ufs.h +2 -0
index 53a5e263f7c8..1f2d4b4950b8 100644
@@ -543,6 +543,8 @@ struct ufs_dev_info {
 	u16 hpb_ver;
 	/* bHPBControl */
 	u8 hpb_control_mode;
+	/* bUFSFeaturesSupport */
+	u8 ufs_features;
 };
 
 /**
drivers/scsi/ufs/ufshcd.c +2 -0
index 83ed2879d930..1fe7ffc1a75a 100644
@@ -6625,6 +6625,8 @@ static int ufs_get_device_desc(struct ufs_hba *hba)
 		goto out;
 	}
 
+	dev_info->ufs_features = desc_buf[DEVICE_DESC_PARAM_UFS_FEAT];
+
 	if (desc_buf[DEVICE_DESC_PARAM_UFS_FEAT] & 0x80) {
 		hba->dev_info.hpb_control_mode =
 			desc_buf[DEVICE_DESC_PARAM_HPB_CTRL_MODE];
Signature · 1 line
2.17.1
#7 lore raw

Re: [PATCH v2 3/5] scsi: ufs: add ufs_features parameter in structure ufs_dev_info

Quoted 6 lines · On 4/16/20 1:31 PM, [email protected] wrote:
drivers/scsi/ufs/ufshcd.c +0 -0
index 83ed2879d930..1fe7ffc1a75a 100644

drivers/scsi/ufs/ufshcd.c static int ufs_get_device_desc(struct ufs_hba *hba) · line 6632

Quoted 8 lines
drivers/scsi/ufs/ufshcd.c +2 -0
@@ -6625,6 +6625,8 @@ static int ufs_get_device_desc(struct ufs_hba *hba)
  		goto out;
  	}
 
+	dev_info->ufs_features = desc_buf[DEVICE_DESC_PARAM_UFS_FEAT];
+
  	if (desc_buf[DEVICE_DESC_PARAM_UFS_FEAT] & 0x80) {
  		hba->dev_info.hpb_control_mode =
  			desc_buf[DEVICE_DESC_PARAM_HPB_CTRL_MODE];

Since this change is very closely related to the changes in patch 1/3, please merge this patch into patch 1/3.

Thanks,

Bart.

#8 lore raw

PATCH v2 4/5 scsi: ufs: add unit and geometry parameters for HPB

In reply to [email protected]

From: Bean Huo <[email protected]>

Add HPB related parameters introduced in Unit Descriptor and Geometry Descriptor.

---
 drivers/scsi/ufs/ufs.h | 7 +++++++
 1 file changed, 7 insertions(+)
drivers/scsi/ufs/ufs.h +7 -0
index 1f2d4b4950b8..6210e489d2ce 100644
@@ -219,6 +219,9 @@ enum unit_desc_param {
 	UNIT_DESC_PARAM_PHY_MEM_RSRC_CNT	= 0x18,
 	UNIT_DESC_PARAM_CTX_CAPABILITIES	= 0x20,
 	UNIT_DESC_PARAM_LARGE_UNIT_SIZE_M1	= 0x22,
+	UNIT_DESC_PARAM_HPB_MAX_ACTIVE_REGIONS	= 0x23,
+	UNIT_DESC_PARAM_HPB_PIN_REGION_START_OFFSET	= 0x25,
+	UNIT_DESC_PARAM_HPB_NUM_PIN_REGIONS             = 0x27,
 };
 
 /* Device descriptor parameters offsets in bytes*/
@@ -304,6 +307,10 @@ enum geometry_desc_param {
 	GEOMETRY_DESC_PARAM_ENM4_MAX_NUM_UNITS	= 0x3E,
 	GEOMETRY_DESC_PARAM_ENM4_CAP_ADJ_FCTR	= 0x42,
 	GEOMETRY_DESC_PARAM_OPT_LOG_BLK_SIZE	= 0x44,
+	GEOMETRY_DESC_PARAM_HPB_REGION_SIZE	= 0x48,
+	GEOMETRY_DESC_PARAM_HPB_NUMBER_LU	= 0x49,
+	GEOMETRY_DESC_PARAM_HPB_SUBREGION_SIZE  = 0x4A,
+	GEOMETRY_DESC_PARAM_HPB_MAX_ACTIVE_REGIONS = 0x4B,
 };
 
 /* Health descriptor parameters offsets in bytes*/
Signature · 1 line
2.17.1
#9 lore raw

PATCH v2 5/5 scsi: ufs: UFS Host Performance Booster(HPB) driver

In reply to [email protected]

From: Bean Huo <[email protected]>

This patch is to add support for the UFS Host Performance Booster (HPB v1.0), which is used to improve UFS read performance, especially for the random read.

NAND flash-based storage devices, including UFS, have mechanisms to translate logical addresses of requests to the corresponding physical addresses of the flash storage. Traditionally this L2P mapping data is loaded to the internal SRAM in the storage controller. When the capacity of storage is larger, a larger size of SRAM for the L2P map data is required. Since increased SRAM size affects the manufacturing cost significantly, it is not cost-effective to allocate all the amount of SRAM needed to keep all the Logical-address to Physical-address (L2P) map data. Therefore, L2P map data, which is required to identify the physical address for the requested IOs, can only be partially stored in SRAM from NAND flash. Due to this partial loading, accessing the flash address area where the L2P information for that address is not loaded in the SRAM can result in serious performance degradation.

The HPB is a software solution for the above problem, which uses the host’s system memory as a cache for the FTL L2P mapping table. It does not need additional hardware support from the host side. By using HPB, the L2P mapping table can be read from host memory and stored in host-side memory. while reading the operation, the corresponding L2P information will be sent to the UFS device along with the reading request. Since the L2P entry is provided in the read request, UFS device does not have to load L2P entry from flash memory. This will significantly improve random read performance.

---
 drivers/scsi/ufs/Kconfig  |   62 +
 drivers/scsi/ufs/Makefile |    1 +
 drivers/scsi/ufs/ufshcd.c |   55 +-
 drivers/scsi/ufs/ufshcd.h |   10 +
 drivers/scsi/ufs/ufshpb.c | 3279 +++++++++++++++++++++++++++++++++++++
 drivers/scsi/ufs/ufshpb.h |  450 +++++
 6 files changed, 3851 insertions(+), 6 deletions(-)
 create mode 100644 drivers/scsi/ufs/ufshpb.c
 create mode 100644 drivers/scsi/ufs/ufshpb.h
drivers/scsi/ufs/Kconfig +62 -0
index e2005aeddc2d..48704062861a 100644
@@ -160,3 +160,65 @@ config SCSI_UFS_BSG
 
 	  Select this if you need a bsg device node for your UFS controller.
 	  If unsure, say N.
+
+config SCSI_UFSHPB
+	bool "UFS Host Performance Booster (EXPERIMENTAL)"
+	depends on SCSI_UFSHCD
+	help
+	  NAND flash-based storage devices, including UFS, have mechanisms to
+	  translate logical addresses of the IO requests to the corresponding
+	  physical addresses of the flash storage. Traditionally, this L2P
+	  mapping data is loaded to the internal SRAM in the storage controller.
+	  When the capacity of storage is larger, a larger size of SRAM for the
+	  L2P map data is required. Since increased SRAM size affects the
+	  manufacturing cost significantly, it is not cost-effective to allocate
+	  all the amount of SRAM needed to keep all the Logical-address to
+	  Physical-address (L2P) map data. Therefore, L2P map data, which is
+	  required to identify the physical address for the requested IOs, can
+	  only be partially stored in SRAM from NAND flash. Due to this partial
+	  loading, accessing the flash address area where the L2P information
+	  for that address is not loaded in the SRAM can result in serious
+	  performance degradation.
+
+	  UFS Host Performance Booster (HPB) is a software solution for the
+	  above problem, which uses the host side system memory as a cache for
+	  the FTL L2P mapping table. It does not need additional hardware
+	  support from the host side. By using HPB, the L2P mapping table can be
+	  read from host memory and stored in host-side memory. while reading
+	  the operation, the corresponding L2P information will be sent to the
+	  UFS device along with the reading request. Since the L2P entry is
+	  provided in the read request, UFS device does not have to load L2P
+	  entry from flash memory to UFS internal SRAM. This will significantly
+	  improve the read performance.
+
+	  When selected, this feature will be built in the UFS driver.
+
+	  If in doubt, say N.
+
+config UFSHPB_MAX_MEM_SIZE
+	int "UFS HPB maximum memory size per controller (in MiB)"
+	depends on SCSI_UFSHPB
+	default 128
+	range 0 65536
+	help
+	  This parameter defines the maximum UFS HPB memory/cache size in the
+	  host system. The recommended HPB cache size by the UFS device can be
+	  calculated from bHPBRegionSize and wDeviceMaxActiveHPBRegions. The
+	  reference formula can be
+
+		(bHPBRegionSize(in KB) / 4KB) * 8 * wDeviceMaxActiveHPBRegions.
+
+	  The HPB cache in the host system is used to contain L2P mapping entry.
+	  If the allocated HPB cache size is lower than what calculated by the
+	  above formula, the use of HPB feature may provide lower performance
+	  advantage. But the system memory resource has the limitation, we can
+	  not let HPB driver allocate its cache at will according to the UFS
+	  device recommendation, so an appropriate size of the cache for HPB
+	  should be specified before you choose to use HPB, then please input a
+	  non-zero positive integer value.
+
+	  Nevertheless, if you want to leave this right to the HPB driver, and
+	  let the HPB driver allocate the HPB cache based on the recommendation
+	  of the UFS device. Just give 0 value to this parameter.
+
+	  Leave the default value if unsure.
drivers/scsi/ufs/Makefile +1 -0
index 94c6c5d7334b..24d9b69eab0c 100644
@@ -7,6 +7,7 @@ obj-$(CONFIG_SCSI_UFS_QCOM) += ufs-qcom.o
 obj-$(CONFIG_SCSI_UFSHCD) += ufshcd-core.o
 ufshcd-core-y				+= ufshcd.o ufs-sysfs.o
 ufshcd-core-$(CONFIG_SCSI_UFS_BSG)	+= ufs_bsg.o
+ufshcd-core-$(CONFIG_SCSI_UFSHPB) += ufshpb.o
 obj-$(CONFIG_SCSI_UFSHCD_PCI) += ufshcd-pci.o
 obj-$(CONFIG_SCSI_UFSHCD_PLATFORM) += ufshcd-pltfrm.o
 obj-$(CONFIG_SCSI_UFS_HISI) += ufs-hisi.o
drivers/scsi/ufs/ufshcd.c +49 -6

114 lines not on this page · raw message

drivers/scsi/ufs/ufshcd.h +10 -0
index 7ce9cc2f10fe..a78b7836b848 100644
@@ -70,6 +70,7 @@
 
 #include "ufs.h"
 #include "ufshci.h"
+#include "ufshpb.h"
 
 #define UFSHCD "ufshcd"
 #define UFSHCD_DRIVER_VERSION "0.2"
@@ -727,6 +728,15 @@ struct ufs_hba {
 
 	struct device		bsg_dev;
 	struct request_queue	*bsg_queue;
+
+#if defined(CONFIG_SCSI_UFSHPB)
+	enum UFSHPB_STATE ufshpb_state;
+	struct ufshpb_geo hpb_geo;
+	struct ufshpb_lu *ufshpb_lup[32];
+	struct delayed_work ufshpb_init_work;
+	struct work_struct ufshpb_eh_work;
+	struct scsi_device *sdev_ufs_lu[32];
+#endif
 };
 
 /* Returns true if clocks can be gated. Otherwise false */
drivers/scsi/ufs/ufshpb.c new file +3279 -0

2,792 lines not on this page · raw message

drivers/scsi/ufs/ufshpb.h new file +450 -0

450 lines not on this page · raw message

The rest of this diff is not shown. View the raw message.

Signature · 1 line
2.17.1

Part of the diff is not shown here. View the raw message.

#10 lore raw

Re: [PATCH v2 5/5] scsi: ufs: UFS Host Performance Booster(HPB) driver

Hi--

A few Kconfig changes for you to consider:

Quoted 62 lines · On 4/16/20 1:31 PM, [email protected] wrote:

From: Bean Huo <[email protected]>

This patch is to add support for the UFS Host Performance Booster (HPB v1.0), which is used to improve UFS read performance, especially for the random read.

NAND flash-based storage devices, including UFS, have mechanisms to translate logical addresses of requests to the corresponding physical addresses of the flash storage. Traditionally this L2P mapping data is loaded to the internal SRAM in the storage controller. When the capacity of storage is larger, a larger size of SRAM for the L2P map data is required. Since increased SRAM size affects the manufacturing cost significantly, it is not cost-effective to allocate all the amount of SRAM needed to keep all the Logical-address to Physical-address (L2P) map data. Therefore, L2P map data, which is required to identify the physical address for the requested IOs, can only be partially stored in SRAM from NAND flash. Due to this partial loading, accessing the flash address area where the L2P information for that address is not loaded in the SRAM can result in serious performance degradation.

The HPB is a software solution for the above problem, which uses the host’s system memory as a cache for the FTL L2P mapping table. It does not need additional hardware support from the host side. By using HPB, the L2P mapping table can be read from host memory and stored in host-side memory. while reading the operation, the corresponding L2P information will be sent to the UFS device along with the reading request. Since the L2P entry is provided in the read request, UFS device does not have to load L2P entry from flash memory. This will significantly improve random read performance.

---
 drivers/scsi/ufs/Kconfig  |   62 +
 drivers/scsi/ufs/Makefile |    1 +
 drivers/scsi/ufs/ufshcd.c |   55 +-
 drivers/scsi/ufs/ufshcd.h |   10 +
 drivers/scsi/ufs/ufshpb.c | 3279 +++++++++++++++++++++++++++++++++++++
 drivers/scsi/ufs/ufshpb.h |  450 +++++
 6 files changed, 3851 insertions(+), 6 deletions(-)
 create mode 100644 drivers/scsi/ufs/ufshpb.c
 create mode 100644 drivers/scsi/ufs/ufshpb.h
drivers/scsi/ufs/Kconfig +15 -0
index e2005aeddc2d..48704062861a 100644
@@ -160,3 +160,65 @@ config SCSI_UFS_BSG
 
 	  Select this if you need a bsg device node for your UFS controller.
 	  If unsure, say N.
+
+config SCSI_UFSHPB
+	bool "UFS Host Performance Booster (EXPERIMENTAL)"
+	depends on SCSI_UFSHCD
+	help
+	  NAND flash-based storage devices, including UFS, have mechanisms to
+	  translate logical addresses of the IO requests to the corresponding
+	  physical addresses of the flash storage. Traditionally, this L2P
+	  mapping data is loaded to the internal SRAM in the storage controller.
+	  When the capacity of storage is larger, a larger size of SRAM for the
+	  L2P map data is required. Since increased SRAM size affects the
+	  manufacturing cost significantly, it is not cost-effective to allocate
+	  all the amount of SRAM needed to keep all the Logical-address to
+	  Physical-address (L2P) map data. Therefore, L2P map data, which is
+	  required to identify the physical address for the requested IOs, can

drivers/scsi/ufs/Kconfig config SCSI_UFS_BSG · line 187

Quoted 10 lines
drivers/scsi/ufs/Kconfig +10 -0
+	  only be partially stored in SRAM from NAND flash. Due to this partial
+	  loading, accessing the flash address area where the L2P information
+	  for that address is not loaded in the SRAM can result in serious
+	  performance degradation.
+
+	  UFS Host Performance Booster (HPB) is a software solution for the
+	  above problem, which uses the host side system memory as a cache for
+	  the FTL L2P mapping table. It does not need additional hardware
+	  support from the host side. By using HPB, the L2P mapping table can be
+	  read from host memory and stored in host-side memory. while reading

^??^ When performing a

drivers/scsi/ufs/Kconfig config SCSI_UFS_BSG · line 188

Quoted 1 line
drivers/scsi/ufs/Kconfig +1 -0
+	  the operation, the corresponding L2P information will be sent to the

read operation, the

drivers/scsi/ufs/Kconfig config SCSI_UFS_BSG · line 189

Quoted 1 line
drivers/scsi/ufs/Kconfig +1 -0
+	  UFS device along with the reading request. Since the L2P entry is

the read request.

Quoted 12 lines
drivers/scsi/ufs/Kconfig +12 -0
+	  provided in the read request, UFS device does not have to load L2P
+	  entry from flash memory to UFS internal SRAM. This will significantly
+	  improve the read performance.
+
+	  When selected, this feature will be built in the UFS driver.
+
+	  If in doubt, say N.
+
+config UFSHPB_MAX_MEM_SIZE
+	int "UFS HPB maximum memory size per controller (in MiB)"
+	depends on SCSI_UFSHPB
+	default 128

drivers/scsi/ufs/Kconfig config SCSI_UFS_BSG · line 211

Quoted 10 lines
drivers/scsi/ufs/Kconfig +10 -0
+	range 0 65536
+	help
+	  This parameter defines the maximum UFS HPB memory/cache size in the
+	  host system. The recommended HPB cache size by the UFS device can be
+	  calculated from bHPBRegionSize and wDeviceMaxActiveHPBRegions. The
+	  reference formula can be
+
+		(bHPBRegionSize(in KB) / 4KB) * 8 * wDeviceMaxActiveHPBRegions.
+
+	  The HPB cache in the host system is used to contain L2P mapping entry.

entries.

drivers/scsi/ufs/Kconfig config SCSI_UFS_BSG · line 212

Quoted 1 line
drivers/scsi/ufs/Kconfig +1 -0
+	  If the allocated HPB cache size is lower than what calculated by the

that

drivers/scsi/ufs/Kconfig config SCSI_UFS_BSG · line 217

Quoted 5 lines
drivers/scsi/ufs/Kconfig +5 -0
+	  above formula, the use of HPB feature may provide lower performance
+	  advantage. But the system memory resource has the limitation, we can
+	  not let HPB driver allocate its cache at will according to the UFS
+	  device recommendation, so an appropriate size of the cache for HPB
+	  should be specified before you choose to use HPB, then please input a

then enter a

drivers/scsi/ufs/Kconfig config SCSI_UFS_BSG · line 220

Quoted 3 lines
drivers/scsi/ufs/Kconfig +3 -0
+	  non-zero positive integer value.
+
+	  Nevertheless, if you want to leave this right to the HPB driver, and

leave this to the HPB driver, and

drivers/scsi/ufs/Kconfig config SCSI_UFS_BSG · line 222

Quoted 2 lines
drivers/scsi/ufs/Kconfig +2 -0
+	  let the HPB driver allocate the HPB cache based on the recommendation
+	  of the UFS device. Just give 0 value to this parameter.

device, just

drivers/scsi/ufs/Kconfig config SCSI_UFS_BSG · line 224

Quoted 2 lines
drivers/scsi/ufs/Kconfig +2 -0
+
+	  Leave the default value if unsure.

gday.

Signature · 1 line
~Randy
#11 lore raw

Re: [PATCH v2 5/5] scsi: ufs: UFS Host Performance Booster(HPB) driver

In reply to [email protected]
Quoted 16 lines · On 4/16/20 1:31 PM, [email protected] wrote:

This patch is to add support for the UFS Host Performance Booster (HPB v1.0), which is used to improve UFS read performance, especially for the random read.

NAND flash-based storage devices, including UFS, have mechanisms to translate logical addresses of requests to the corresponding physical addresses of the flash storage. Traditionally this L2P mapping data is loaded to the internal SRAM in the storage controller. When the capacity of storage is larger, a larger size of SRAM for the L2P map data is required. Since increased SRAM size affects the manufacturing cost significantly, it is not cost-effective to allocate all the amount of SRAM needed to keep all the Logical-address to Physical-address (L2P) map data. Therefore, L2P map data, which is required to identify the physical address for the requested IOs, can only be partially stored in SRAM from NAND flash. Due to this partial loading, accessing the flash address area where the L2P information for that address is not loaded in the SRAM can result in serious performance degradation.

Quoted 8 lines

The HPB is a software solution for the above problem, which uses the host’s system memory as a cache for the FTL L2P mapping table. It does not need additional hardware support from the host side. By using HPB, the L2P mapping table can be read from host memory and stored in host-side memory. while reading the operation, the corresponding L2P information will be sent to the UFS device along with the reading request. Since the L2P entry is provided in the read request, UFS device does not have to load L2P entry from flash memory. This will significantly improve random read performance.

The above description is incomplete: what is missing is an explanation of how the L2P data is made persistent and also how it is stored on the device. Is an L2P update written to the device every time data on the device is modified or only during an orderly shutdown? In the former case, how big is the impact on write performance? In the latter case, does that mean that data is lost during an unorderly shutdown? Is HPB a feature that perhaps only works well for battery-powered devices?

Is the persistent L2P data overwritten in place or is new data appended as an update after existing data? In the former case, is all L2P data lost in case of a power failure? In the latter case, how is it guaranteed that enough free space is available to make the L2P data persistent?

What are the similarities and differences compared to the lightnvm framework that was added several years ago to the Linux kernel? Which of the code in this patch can be shared with the lightnvm framework?

Quoted 17 lines
drivers/scsi/ufs/Kconfig +17 -0
+config UFSHPB_MAX_MEM_SIZE
+	int "UFS HPB maximum memory size per controller (in MiB)"
+	depends on SCSI_UFSHPB
+	default 128
+	range 0 65536
+	help
+	  This parameter defines the maximum UFS HPB memory/cache size in the
+	  host system. The recommended HPB cache size by the UFS device can be
+	  calculated from bHPBRegionSize and wDeviceMaxActiveHPBRegions. The
+	  reference formula can be
+
+		(bHPBRegionSize(in KB) / 4KB) * 8 * wDeviceMaxActiveHPBRegions.
+
+	  The HPB cache in the host system is used to contain L2P mapping entry.
+	  If the allocated HPB cache size is lower than what calculated by the
+	  above formula, the use of HPB feature may provide lower performance
+	  advantage. But the system memory resource has the limitation, we can

drivers/scsi/ufs/Kconfig config SCSI_UFS_BSG · line 224

Quoted 10 lines
drivers/scsi/ufs/Kconfig +10 -0
+	  not let HPB driver allocate its cache at will according to the UFS
+	  device recommendation, so an appropriate size of the cache for HPB
+	  should be specified before you choose to use HPB, then please input a
+	  non-zero positive integer value.
+
+	  Nevertheless, if you want to leave this right to the HPB driver, and
+	  let the HPB driver allocate the HPB cache based on the recommendation
+	  of the UFS device. Just give 0 value to this parameter.
+
+	  Leave the default value if unsure.

Can this parameter be changed from a compile-time option into something that is configurable at runtime, e.g. a sysfs or configfs attribute?

drivers/scsi/ufs/ufshcd.c static int ufshcd_slave_configure(struct scsi_device *sdev) · line 4634

Quoted 4 lines
drivers/scsi/ufs/ufshcd.c +4 -0
+#if defined(CONFIG_SCSI_UFSHPB)
+	if (sdev->lun < hba->dev_info.max_lu_supported)
+		hba->sdev_ufs_lu[sdev->lun] = sdev;
+#endif

Isn't maintaining a LUN to scsi_device mapping something that is already done by the SCSI core? See also __scsi_device_lookup().

drivers/scsi/ufs/ufshcd.c ufshcd_transfer_rsp_status(struct ufs_hba *hba, struct ufshcd_lrb *lrbp) · line 4759

Quoted 9 lines
drivers/scsi/ufs/ufshcd.c +9 -0
+#if defined(CONFIG_SCSI_UFSHPB)
+			/*
+			 * HPB recommendations are provided in RESPONSE UPIU
+			 * packets of successfully completed commands, which
+			 * are commands terminated with GOOD status.
+			 */
+			if (scsi_status == SAM_STAT_GOOD)
+				ufshpb_rsp_handler(hba, lrbp);
+#endif

Please introduce helper functions such that no #if defined(CONFIG_SCSI_UFSHPB) statements are required in .c files. From Documentation/process/4.Coding.rst: "As a general rule, #ifdef use should be confined to header files whenever possible."

drivers/scsi/ufs/ufshcd.c static int ufs_get_device_desc(struct ufs_hba *hba) · line 6659

Quoted 7 lines
drivers/scsi/ufs/ufshcd.c +1 -2
@@ -6627,16 +6656,16 @@ static int ufs_get_device_desc(struct ufs_hba *hba)
 
  	dev_info->ufs_features = desc_buf[DEVICE_DESC_PARAM_UFS_FEAT];
 
-	if (desc_buf[DEVICE_DESC_PARAM_UFS_FEAT] & 0x80) {
-		hba->dev_info.hpb_control_mode =
+	if (dev_info->ufs_features & 0x80) {

The above change should be moved into the combination of patches 1/5 and 3/5, and the changes below probably too.

Quoted 13 lines
drivers/scsi/ufs/ufshcd.c +4 -3
+		dev_info->hpb_control_mode =
  			desc_buf[DEVICE_DESC_PARAM_HPB_CTRL_MODE];
-		hba->dev_info.hpb_ver =
+		dev_info->hpb_ver =
  			(u16) (desc_buf[DEVICE_DESC_PARAM_HPB_VER] << 8) |
  			desc_buf[DEVICE_DESC_PARAM_HPB_VER + 1];
  		dev_info(hba->dev, "HPB Version: 0x%2x\n",
-			 hba->dev_info.hpb_ver);
+			 dev_info->hpb_ver);
  		dev_info(hba->dev, "HPB control mode: %d\n",
-			 hba->dev_info.hpb_control_mode);
+			 dev_info->hpb_control_mode);
  	}

[ ... ]

Quoted 17 lines
drivers/scsi/ufs/ufshpb.h new file +16 -0
+/* BYTE SHIFT */
+#define ZERO_BYTE_SHIFT			0
+#define ONE_BYTE_SHIFT			8
+#define TWO_BYTE_SHIFT			16
+#define THREE_BYTE_SHIFT		24
+#define FOUR_BYTE_SHIFT			32
+#define FIVE_BYTE_SHIFT			40
+#define SIX_BYTE_SHIFT			48
+#define SEVEN_BYTE_SHIFT		56
 
+#define SHIFT_BYTE_0(num)		((num) << ZERO_BYTE_SHIFT)
+#define SHIFT_BYTE_1(num)		((num) << ONE_BYTE_SHIFT)
+#define SHIFT_BYTE_2(num)		((num) << TWO_BYTE_SHIFT)
+#define SHIFT_BYTE_3(num)		((num) << THREE_BYTE_SHIFT)
+#define SHIFT_BYTE_4(num)		((num) << FOUR_BYTE_SHIFT)
+#define SHIFT_BYTE_5(num)		((num) << FIVE_BYTE_SHIFT)
+#define SHIFT_BYTE_6(num)		((num) << SIX_BYTE_SHIFT)

drivers/scsi/ufs/ufshpb.h line 109

Quoted 10 lines
drivers/scsi/ufs/ufshpb.h new file +9 -0
+#define SHIFT_BYTE_7(num)		((num) << SEVEN_BYTE_SHIFT)
 
+#define GET_BYTE_0(num)			(((num) >> ZERO_BYTE_SHIFT) & 0xff)
+#define GET_BYTE_1(num)			(((num) >> ONE_BYTE_SHIFT) & 0xff)
+#define GET_BYTE_2(num)			(((num) >> TWO_BYTE_SHIFT) & 0xff)
+#define GET_BYTE_3(num)			(((num) >> THREE_BYTE_SHIFT) & 0xff)
+#define GET_BYTE_4(num)			(((num) >> FOUR_BYTE_SHIFT) & 0xff)
+#define GET_BYTE_5(num)			(((num) >> FIVE_BYTE_SHIFT) & 0xff)
+#define GET_BYTE_6(num)			(((num) >> SIX_BYTE_SHIFT) & 0xff)
+#define GET_BYTE_7(num)			(((num) >> SEVEN_BYTE_SHIFT) & 0xff)

Please remove the above 24 macro definitions and use get_unaligned_*() / put_unaligned_*() instead.

Thanks,

Bart.

#12 lore raw

RE: [EXT] Re: [PATCH v2 5/5] scsi: ufs: UFS Host Performance Booster(HPB) driver

Hi, Bart

Quoted 5 lines

What are the similarities and differences compared to the lightnvm framework that was added several years ago to the Linux kernel? Which of the code in this patch can be shared with the lightnvm framework?

I don’t fully comprehend Lightnvm, and studied it in recent days. it is not possible to deploy Lightnvm framework in UFS HPB.
UFS HPB has a similar idea with Nvme HMB, which both take host-side memory as device onboard flash to hold L2P map entries. But both are not the same, UFS HPB cache cannot be accessed by UFS device over UFS bus. Access and control of UFS HPB cache completely all depend on the host side UFS HPB driver.

HPB has two control modes: device control mode and host control mode. In the device Mode, HPB driver reads L2P map data only based on the device recommendations. However in the host control mode, it is much more flexible, HPB driver completely can initiate the L2P read actively. In host mode, HPB can limit read latency and achieve predictable read latency, since the host side can load the L2P map entries based on its read behavior and oncoming read. But current exist UFS subsystem, cannot make full use of this feature.

I don't know how do you think about adding a framework in SCSI, let FS can pass oncoming read length to UFS HPB, the based on that, HPB can load L2P entries in advance.

//Bean

#13 lore raw

RE: [PATCH v2 5/5] scsi: ufs: UFS Host Performance Booster(HPB) driver

In reply to Bart Van Assche

Hi Bart,

Quoted 3 lines

What are the similarities and differences compared to the lightnvm framework that was added several years ago to the Linux kernel? Which of the code in this patch can be shared with the lightnvm framework?

Simply put, unlike lightnvn, HPB is not host-managed FTL, But instead can be perceived as a cost-reduction effort. Its aim is not to move the fw to the host, but to control the iNAND cost by limiting the amount of its internal RAM. It is done by using the host memory to cache the L2P tables, and replace READ_10 that have only the lba, by an alternative command - HPB_READ, that have both the logical and physical addresses.

Using Lightnvm was considered in the past as possible framework for HPB, but was rejected by both Christoph & Mattias.

The HPB feature was NAKed by Christoph in its entirety, regardless of the driver design. Until this is not reversed, keep commenting this patch is counterproductive and confusing.

Should this decision is reversed, I think this patch should be re-posted as a RFC, And fragment its 5,000 lines or so into a set of reviewable patches.

Thanks,
Avri

#14 lore raw

RE: [EXT] RE: [PATCH v2 5/5] scsi: ufs: UFS Host Performance Booster(HPB) driver

Hi, Avri

Quoted 3 lines

Should this decision is reversed, I think this patch should be re-posted as a RFC, And fragment its 5,000 lines or so into a set of reviewable patches.

Ok, I will repost it as RFC, and also will fix several issues Bart raised. Thanks.

Bean

#15 lore raw

RE: [PATCH v2 5/5] scsi: ufs: UFS Host Performance Booster(HPB) driver

In reply to Avri Altman

One last word for the community members that are not into ufs day-to-day:

HPB implementation made its first public appearance around 2018 as part of Google's Pixel3 and some VIVO models. Since then, more and more mobile platforms are publically using HPB: Galaxy Note10,
Galaxy S20 and VIVO NEX3 (that is already using HPB2.0), some Xiaomi models etc.

On the other hand, HPB1.0 spec was just recently closed - not even as part of UFS3.1, but only after - about 2 months ago. The industry is rushing forward, we've seen this many times.

The fact is that HPB is here to stay - either as a horde of out-of-tree implementations, or as a standard acceptable mainline driver.

Thanks,
Avri

Quoted 29 lines

Hi Bart,

Quoted 3 lines

What are the similarities and differences compared to the lightnvm framework that was added several years ago to the Linux kernel? Which of the code in this patch can be shared with the lightnvm framework?

Simply put, unlike lightnvn, HPB is not host-managed FTL, But instead can be perceived as a cost-reduction effort.
Its aim is not to move the fw to the host, but to control the iNAND cost by limiting the amount of its internal RAM.
It is done by using the host memory to cache the L2P tables, and replace READ_10 that have only the lba,
by an alternative command - HPB_READ, that have both the logical and physical addresses.

Using Lightnvm was considered in the past as possible framework for HPB, but was rejected by both Christoph & Mattias.

The HPB feature was NAKed by Christoph in its entirety, regardless of the driver design.
Until this is not reversed, keep commenting this patch is counterproductive and confusing.

Should this decision is reversed, I think this patch should be re-posted as a RFC,
And fragment its 5,000 lines or so into a set of reviewable patches.

Thanks,
Avri

#16 lore raw

Re: [PATCH v2 5/5] scsi: ufs: UFS Host Performance Booster(HPB) driver

Quoted 11 lines · Avri Altman

One last word for the community members that are not into ufs day-to-day:

HPB implementation made its first public appearance around 2018 as part of Google's Pixel3 and some VIVO models. Since then, more and more mobile platforms are publically using HPB: Galaxy Note10,
Galaxy S20 and VIVO NEX3 (that is already using HPB2.0), some Xiaomi models etc.

On the other hand, HPB1.0 spec was just recently closed - not even as part of UFS3.1, but only after - about 2 months ago. The industry is rushing forward, we've seen this many times.

The fact is that HPB is here to stay - either as a horde of out-of-tree implementations, or as a standard acceptable mainline driver.

Hi Avri,

Thanks for the additional clarification. I think we need HPB support in the upstream kernel. A possible approach is to start a discussion about how to integrate HPB support in the upstream kernel and to defer posting new implementations until agreement about an approach has been achieved. Is there e.g. an alternative for avoiding deadlocks other than using the blk-mq reserved tag feature for the commands that manage the L2P tables?

Thanks,

Bart.

#17 lore raw

RE: [PATCH v2 5/5] scsi: ufs: UFS Host Performance Booster(HPB) driver

Quoted 21 lines · Avri Altman
Quoted 1 line

One last word for the community members that are not into ufs day-to-

day:

Quoted 2 lines

HPB implementation made its first public appearance around 2018 as part

of Google's Pixel3 and some VIVO models.

Quoted 1 line

Since then, more and more mobile platforms are publically using HPB:

Galaxy Note10,

Quoted 1 line

Galaxy S20 and VIVO NEX3 (that is already using HPB2.0), some Xiaomi

models etc.

Quoted 2 lines

On the other hand, HPB1.0 spec was just recently closed - not even as part

of UFS3.1,

Quoted 1 line

but only after - about 2 months ago. The industry is rushing forward, we've

seen this many times.

Quoted 2 lines

The fact is that HPB is here to stay - either as a horde of out-of-tree

implementations,

Quoted 1 line

or as a standard acceptable mainline driver.

Quoted 8 lines

Hi Avri,

Thanks for the additional clarification. I think we need HPB support in the upstream kernel. A possible approach is to start a discussion about how to integrate HPB support in the upstream kernel and to defer posting new implementations until agreement about an approach has been achieved. Is there e.g. an alternative for avoiding deadlocks other than using the blk-mq reserved tag feature for the commands that manage the L2P tables?

Ok then.
HPB support is comprised of 4 main duties:
1) Read the device HPB configuration
2) Attend the device's recommendations that are embedded in the sense buffer
3) L2P cache management - This entails sending 2 new scsi commands (opcodes were taken from the vendor pool):
a. HPB-READ-BUFFER - read L2P physical addresses for a subregion
b. HPB-WRITE-BUFFER - notify the device that a region is inactive (in host-managed mode)
4) Use HPB-READ: a 3rd new scsi command (again - uses the vendor pool) to perform read operation instead of READ10. HPB-READ carries both the logical and the physical addresses.

I will let Bean defend the Samsung approach of using a single LLD to attend all 4 duties.

Another approach might be to split those duties between 2 modules:
- A LLD that will perform the first 2 - those can be done only using ufs privet stuff, and
- another module in scsi mid-layer that will be responsible of L2P cache management, and HPB-READ command setup.
A framework to host the scsi mid-layer module can be the scsi device handler.

The scsi-device-handler infrastructure was added to the kernel mainly to facilitate multiple paths for storage devices. The HPB framework, although far from the original intention of the authors, might as well fit in.
In that sense, using READs and HPB_READs intermittently, can be perceived as a multi-path.

Scsi device handlers are also attached to a specific scsi_device (lun).
This can serve as the glue linking between the ufs LLD and the device handler which resides in the scsi level.

Device handlers comes with a rich and handy set of APIs & ops, which we can use to support HPB. Specifically we can use it to attach & activate the device handler,
only after the ufs driver verified that HPB is supported by both the platform and the device.

The 2 modules can communicate using the handler_data opaque pointer, and the handler’s set_params op-mode: which is an open protocol essentially, and we can use it to pass the sense buffer in its full or just a parsed version.

Being a scsi mid-layer module, it will not break anything while sending HPB-READ-BUFFER and HPB-WRITE-BUFFER as part of the L2P cache management duties.

Last but not least, the device handler is already hooked in the scsi command setup flow - scsi_setup_fs_cmnd(), So we get the hook into HPB-READ prep_fn for free.

Later on, we might want to export the L2P cache management logic to user-space.
Locating the L2P cache management in scsi mid-layer will enable us to do so, using the scsi-netlink or some other means.

Thanks,
Avri

#18 lore raw

Re: [PATCH v2 5/5] scsi: ufs: UFS Host Performance Booster(HPB) driver

Quoted 32 lines · Avri Altman
Quoted 1 line · Avri Altman

HPB support is comprised of 4 main duties:
1) Read the device HPB configuration
2) Attend the device's recommendations that are embedded in the sense buffer
3) L2P cache management - This entails sending 2 new scsi commands (opcodes were taken from the vendor pool):
a. HPB-READ-BUFFER - read L2P physical addresses for a subregion
b. HPB-WRITE-BUFFER - notify the device that a region is inactive (in host-managed mode)
4) Use HPB-READ: a 3rd new scsi command (again - uses the vendor pool) to perform read operation instead of READ10. HPB-READ carries both the logical and the physical addresses.

I will let Bean defend the Samsung approach of using a single LLD to attend all 4 duties.

Another approach might be to split those duties between 2 modules:
- A LLD that will perform the first 2 - those can be done only using ufs privet stuff, and
- another module in scsi mid-layer that will be responsible of L2P cache management, and HPB-READ command setup.
A framework to host the scsi mid-layer module can be the scsi device handler.

The scsi-device-handler infrastructure was added to the kernel mainly to facilitate multiple paths for storage devices. The HPB framework, although far from the original intention of the authors, might as well fit in.
In that sense, using READs and HPB_READs intermittently, can be perceived as a multi-path.

Scsi device handlers are also attached to a specific scsi_device (lun).
This can serve as the glue linking between the ufs LLD and the device handler which resides in the scsi level.

Device handlers comes with a rich and handy set of APIs & ops, which we can use to support HPB. Specifically we can use it to attach & activate the device handler,
only after the ufs driver verified that HPB is supported by both the platform and the device.

The 2 modules can communicate using the handler_data opaque pointer, and the handler’s set_params op-mode: which is an open protocol essentially, and we can use it to pass the sense buffer in its full or just a parsed version.

Quoted 8 lines

Being a scsi mid-layer module, it will not break anything while sending HPB-READ-BUFFER and HPB-WRITE-BUFFER as part of the L2P cache management duties.

Last but not least, the device handler is already hooked in the scsi command setup flow - scsi_setup_fs_cmnd(), So we get the hook into HPB-READ prep_fn for free.

Later on, we might want to export the L2P cache management logic to user-space.
Locating the L2P cache management in scsi mid-layer will enable us to do so, using the scsi-netlink or some other means.

Hi Avri,

I'm not sure that I agree that HPB can be perceived as multi-path. Anyway, the above approach sounds interesting to me. A few questions though:
- The only in-tree caller of scsi_dh_attach() I am aware of exists in the dm-mpath driver. I think that call is triggered by multipathd. I don't think that it is acceptable to require that multipathd is running to use the UFS HPB functionality. What is the plan for attaching the UFS device handler to UFS devices?
- Will preparing a SCSI command involve executing a SCSI command? If so, how will it be prevented that execution of that internally submitted SCSI command triggers a deadlock due to tag exhaustion?

Thanks,

Bart.

#19 lore raw

RE: [PATCH v2 5/5] scsi: ufs: UFS Host Performance Booster(HPB) driver

Quoted 67 lines · Avri Altman
Quoted 4 lines
Quoted 1 line · Avri Altman

HPB support is comprised of 4 main duties:
1) Read the device HPB configuration
2) Attend the device's recommendations that are embedded in the sense

buffer

Quoted 1 line

3) L2P cache management - This entails sending 2 new scsi commands

(opcodes were taken from the vendor pool):

Quoted 2 lines

a. HPB-READ-BUFFER - read L2P physical addresses for a subregion b. HPB-WRITE-BUFFER - notify the device that a region is inactive (in

host-managed mode)

Quoted 1 line

4) Use HPB-READ: a 3rd new scsi command (again - uses the vendor pool)

to perform read operation instead of READ10. HPB-READ carries both the logical and the physical addresses.

Quoted 2 lines

I will let Bean defend the Samsung approach of using a single LLD to attend

all 4 duties.

Quoted 3 lines

Another approach might be to split those duties between 2 modules:
- A LLD that will perform the first 2 - those can be done only using ufs

privet stuff, and

Quoted 1 line

- another module in scsi mid-layer that will be responsible of L2P cache

management,

Quoted 2 lines

and HPB-READ command setup.
A framework to host the scsi mid-layer module can be the scsi device

handler.

Quoted 2 lines

The scsi-device-handler infrastructure was added to the kernel mainly to

facilitate multiple paths for storage devices.

Quoted 1 line

The HPB framework, although far from the original intention of the

authors, might as well fit in.

Quoted 1 line

In that sense, using READs and HPB_READs intermittently, can be perceived

as a multi-path.

Quoted 3 lines

Scsi device handlers are also attached to a specific scsi_device (lun). This can serve as the glue linking between the ufs LLD and the device

handler which resides in the scsi level.

Quoted 2 lines

Device handlers comes with a rich and handy set of APIs & ops, which we

can use to support HPB.

Quoted 2 lines

Specifically we can use it to attach & activate the device handler, only after the ufs driver verified that HPB is supported by both the platform

and the device.

Quoted 3 lines

The 2 modules can communicate using the handler_data opaque pointer, and the handler’s set_params op-mode: which is an open protocol

essentially,

Quoted 4 lines

and we can use it to pass the sense buffer in its full or just a parsed version.

Being a scsi mid-layer module, it will not break anything while sending HPB-READ-BUFFER and HPB-WRITE-BUFFER as part of the L2P cache

management duties.

Quoted 2 lines

Last but not least, the device handler is already hooked in the scsi

command setup flow - scsi_setup_fs_cmnd(),

Quoted 3 lines

So we get the hook into HPB-READ prep_fn for free.

Later on, we might want to export the L2P cache management logic to

user-space.

Quoted 1 line

Locating the L2P cache management in scsi mid-layer will enable us to do

so, using the scsi-netlink or some other means.

Hi Avri,

Quoted 8 lines

I'm not sure that I agree that HPB can be perceived as multi-path. Anyway, the above approach sounds interesting to me. A few questions though:
- The only in-tree caller of scsi_dh_attach() I am aware of exists in the dm-mpath driver. I think that call is triggered by multipathd. I don't think that it is acceptable to require that multipathd is running to use the UFS HPB functionality. What is the plan for attaching the UFS device handler to UFS devices?

Right.
Device handlers are meant to be called as part of the device mapper multi-path code. We can’t do that – we need to attach & activate the device handler manually, only after the ufs driver verified that HPB is supported by both the platform and the device.

I was thinking to rely on the ufs's 2-phase boot:
The ufs boot process is essentially comprised of 2 parts: first a handshake with the device, and then, scsi scans and assign a scsi device to each lun. The latter, although running a-synchronically, is happening right after reading the device configuration - lun by lun.

By now we've read the device HPB configuration, and we are ready to attach a scsi device to our HPB luns. A perfect timing might be while scsi is performing its .slave_alloc() or .slave_configure().

It will require however, to include <scsi/scsi_dh.h> and "../scsi_priv.h" in ufshpb.c, Not sure if this is acceptable.

Quoted 3 lines

- Will preparing a SCSI command involve executing a SCSI command? If so, how will it be prevented that execution of that internally submitted SCSI command triggers a deadlock due to tag exhaustion?

Not sure what deadlock are you referring to. Executing HPB-READ-BUFFER is not part of the data path, And should not happen too often.
Anyway it can wait for a tag to become free. Same goes to HPB-WRITE-BUFFER.

Quoted 4 lines

Thanks,

Bart.

author
#20 lore raw

Re: [PATCH v2 5/5] scsi: ufs: UFS Host Performance Booster(HPB) driver

Quoted 150 lines · Avri Altman
Quoted 110 lines · Avri Altman
Quoted 6 lines
Quoted 1 line · Avri Altman

HPB support is comprised of 4 main duties:
1) Read the device HPB configuration
2) Attend the device's recommendations that are embedded in the sense

buffer

Quoted 2 lines

3) L2P cache management - This entails sending 2 new scsi commands

(opcodes were taken from the vendor pool):

Quoted 4 lines

a. HPB-READ-BUFFER - read L2P physical addresses for a
subregion
b. HPB-WRITE-BUFFER - notify the device that a region is
inactive (in

host-managed mode)

Quoted 2 lines

4) Use HPB-READ: a 3rd new scsi command (again - uses the vendor pool)

to perform read operation instead of READ10. HPB-READ carries both the
logical and the physical addresses.

Quoted 3 lines

I will let Bean defend the Samsung approach of using a single LLD to attend

all 4 duties.

Quoted 5 lines

Another approach might be to split those duties between 2 modules:
- A LLD that will perform the first 2 - those can be done only
using ufs

privet stuff, and

Quoted 2 lines

- another module in scsi mid-layer that will be responsible of
L2P cache

management,

Quoted 3 lines

and HPB-READ command setup.
A framework to host the scsi mid-layer module can be the scsi device

handler.

Quoted 3 lines

The scsi-device-handler infrastructure was added to the kernel mainly to

facilitate multiple paths for storage devices.

Quoted 2 lines

The HPB framework, although far from the original intention of the

authors, might as well fit in.

Quoted 2 lines

In that sense, using READs and HPB_READs intermittently, can be perceived

as a multi-path.

Quoted 5 lines

Scsi device handlers are also attached to a specific scsi_device (lun).
This can serve as the glue linking between the ufs LLD and the device

handler which resides in the scsi level.

Quoted 3 lines

Device handlers comes with a rich and handy set of APIs & ops, which we

can use to support HPB.

Quoted 4 lines

Specifically we can use it to attach & activate the device handler,
only after the ufs driver verified that HPB is supported by both the platform

and the device.

Quoted 4 lines

The 2 modules can communicate using the handler_data opaque pointer,
and the handler’s set_params op-mode: which is an open protocol

essentially,

Quoted 6 lines

and we can use it to pass the sense buffer in its full or just a parsed version.

Being a scsi mid-layer module, it will not break anything while sending
HPB-READ-BUFFER and HPB-WRITE-BUFFER as part of the L2P cache

management duties.

Quoted 3 lines

Last but not least, the device handler is already hooked in the scsi

command setup flow - scsi_setup_fs_cmnd(),

Quoted 4 lines

So we get the hook into HPB-READ prep_fn for free.

Later on, we might want to export the L2P cache management logic to

user-space.

Quoted 2 lines

Locating the L2P cache management in scsi mid-layer will enable us to do

so, using the scsi-netlink or some other means.

Hi Avri,

Quoted 11 lines

I'm not sure that I agree that HPB can be perceived as multi-path. Anyway, the above approach sounds interesting to me. A few questions
though:
- The only in-tree caller of scsi_dh_attach() I am aware of exists in
the dm-mpath driver. I think that call is triggered by
multipathd.
I don't think that it is acceptable to require that multipathd is running to use the UFS HPB functionality. What is the plan for attaching the UFS device handler to UFS devices?

Right.
Device handlers are meant to be called as part of the device mapper multi-path code. We can’t do that – we need to attach & activate the device handler manually, only after the ufs driver verified that HPB is
supported by both the platform and the device.

I was thinking to rely on the ufs's 2-phase boot:
The ufs boot process is essentially comprised of 2 parts: first a

handshake with the device, and then, scsi scans and assign a scsi device

to each lun. The latter, although running a-synchronically, is

happening right after reading the device configuration - lun by lun.

Quoted 5 lines

By now we've read the device HPB configuration, and we are ready to attach a scsi device to our HPB luns. A perfect timing might be while
scsi is performing its .slave_alloc() or .slave_configure().

hi, Avri
That means HPB memory allocation done in .scan_finished() ? and sd_init_command() needs to change as well, add a new request type REQ_OP_HPB_READ?

Bean

#21 lore raw

RE: [PATCH v2 5/5] scsi: ufs: UFS Host Performance Booster(HPB) driver

Hi Bean,

Quoted 8 lines
Quoted 5 lines

By now we've read the device HPB configuration, and we are ready to attach a scsi device to our HPB luns. A perfect timing might be while
scsi is performing its .slave_alloc() or .slave_configure().

hi, Avri
That means HPB memory allocation done in .scan_finished() ?

The specifics of this feature are yet to be determined.

Among those, yes - we need to discuss how to handle the memory allocation. Statically allocating the required cache for the entire max-active-subregions, Which may sum-up to a hundreds of MB, has its obvious downsize. We need to discuss this further.

Quoted 2 lines

and sd_init_command() needs to change as well, add a new request type REQ_OP_HPB_READ?

Again, this is an implementation issue.
We need to figure it out in the sequel.
E.g. we might want to make use of the combination of a valid handler and blk_op_is_private.

I think it would be more constructive, if we can decide first on the module layout, And figure out the other details as we go?

Can you provide the pros and cons for the Samsung approach - implementing all HPB functionalities using a single LLD?

Thanks,
Avri

Quoted 4 lines

Bean

author
#22 lore raw

Re: [PATCH v2 5/5] scsi: ufs: UFS Host Performance Booster(HPB) driver

Quoted 33 lines · Avri Altman

Hi Bean,

Quoted 9 lines
Quoted 6 lines

By now we've read the device HPB configuration, and we are ready to
attach a scsi device to our HPB luns. A perfect timing might be while
scsi is performing its .slave_alloc() or .slave_configure().

hi, Avri
That means HPB memory allocation done in .scan_finished() ?

The specifics of this feature are yet to be determined.

Among those, yes - we need to discuss how to handle the memory allocation.
Statically allocating the required cache for the entire max-active- subregions,
Which may sum-up to a hundreds of MB, has its obvious downsize. We need to discuss this further.

Quoted 2 lines

and sd_init_command() needs to change as well, add a new request type REQ_OP_HPB_READ?

Again, this is an implementation issue.
We need to figure it out in the sequel.
E.g. we might want to make use of the combination of a valid handler and blk_op_is_private.

I think it would be more constructive, if we can decide first on the module layout,
And figure out the other details as we go?

Quoted 6 lines

Can you provide the pros and cons for the Samsung approach - implementing all HPB functionalities using a single LLD?

Thanks,
Avri

Hi Avri

Samsung approach is a flat design and the HPB functions are embedded in the UFSHBA driver, looks ugly. Each LU has its own HPB cache, which statically allocated in HPB initialization stage. If one LU runs out of its HPB cache, it is impossbile to borrow HPB cache from its neighbour LU. Also, HPB requests are enqueued to the scsi_device request_queue and then fly back to SCSI layer. This unavoidably lengthens the latency of HPB entry update.
For the HPB host control mode, the predictability of HPB region activation of this design is lower, since HPB driver doesn't know which Region exactly should be activated in advance.

Regarding its pros, exactly I don't know, maybe it is relatively simple, work, and there are already customers who are using it now. also, we don't need to argue with SCSI layers and maintenance is easier.

To me, hierarchical design sounds good, and move the implementation of HPB manager module to SCSI layer is nice. but what is opinion of others? and which way they prefer. or they want us to continue current Samsung approach and solve its cons further.

thanks,

Bean

Quoted 6 lines
Quoted 4 lines

Bean

#23 lore raw

RE: [PATCH v2 5/5] scsi: ufs: UFS Host Performance Booster(HPB) driver

Hi Bean,

Quoted 4 lines

To me, hierarchical design sounds good, and move the implementation of HPB manager module to SCSI layer is nice. but what is opinion of others? and which way they prefer. or they want us to continue current Samsung approach and solve its cons further.

I can put an RFC together in few days.
My plan is to borrow as much as possible from Samsung driver,
But to move the L2P cache management and HPB-READ command setup to the scsi mid-layer. Also maybe elaborate the host-managed mode logic to some extent.
This will not be a full-fledged driver, but more of a skeleton -
The gory details can come after, once we'll agree on the general concept.

Let's wait few days more to allow some more people to comment. Ideally, this should be a joint effort of the ufs community, Given the size of this driver, and the industry's interest and focus.

Thanks,
Avri

author
#24 lore raw

Re: [PATCH v2 5/5] scsi: ufs: UFS Host Performance Booster(HPB) driver

In reply to Bart Van Assche
Quoted 69 lines · Bart Van Assche
Quoted 58 lines · Avri Altman
Quoted 1 line · Avri Altman

HPB support is comprised of 4 main duties:
1) Read the device HPB configuration
2) Attend the device's recommendations that are embedded in the sense buffer
3) L2P cache management - This entails sending 2 new scsi commands (opcodes were taken from the vendor pool):
a. HPB-READ-BUFFER - read L2P physical addresses for a
subregion
b. HPB-WRITE-BUFFER - notify the device that a region is
inactive (in host-managed mode)
4) Use HPB-READ: a 3rd new scsi command (again - uses the vendor pool) to perform read operation instead of READ10. HPB-READ carries both the logical and the physical addresses.

I will let Bean defend the Samsung approach of using a single LLD to attend all 4 duties.

Another approach might be to split those duties between 2 modules:
- A LLD that will perform the first 2 - those can be done only
using ufs privet stuff, and
- another module in scsi mid-layer that will be responsible of L2P
cache management,
and HPB-READ command setup.
A framework to host the scsi mid-layer module can be the scsi device handler.

The scsi-device-handler infrastructure was added to the kernel mainly to facilitate multiple paths for storage devices. The HPB framework, although far from the original intention of the authors, might as well fit in.
In that sense, using READs and HPB_READs intermittently, can be perceived as a multi-path.

Scsi device handlers are also attached to a specific scsi_device (lun).
This can serve as the glue linking between the ufs LLD and the device handler which resides in the scsi level.

Device handlers comes with a rich and handy set of APIs & ops, which we can use to support HPB.
Specifically we can use it to attach & activate the device handler, only after the ufs driver verified that HPB is supported by both the platform and the device.

The 2 modules can communicate using the handler_data opaque pointer,
and the handler’s set_params op-mode: which is an open protocol essentially,
and we can use it to pass the sense buffer in its full or just a parsed version.

Being a scsi mid-layer module, it will not break anything while sending
HPB-READ-BUFFER and HPB-WRITE-BUFFER as part of the L2P cache management duties.

Quoted 8 lines

Last but not least, the device handler is already hooked in the scsi command setup flow - scsi_setup_fs_cmnd(),
So we get the hook into HPB-READ prep_fn for free.

Later on, we might want to export the L2P cache management logic to user-space.
Locating the L2P cache management in scsi mid-layer will enable us to do so, using the scsi-netlink or some other means.

Quoted 8 lines

- Will preparing a SCSI command involve executing a SCSI command? If so,
how will it be prevented that execution of that internally
submitted
SCSI command triggers a deadlock due to tag exhaustion?

Thanks,

No, as for the HPB READ, it will replace SCSI READ in SCSI request executing path in case L2P entry hit in HPB cache. so it still uses the previously assigned tag.

As for the HPB BUFFER READ and HPB BUFFER WRITE, should beg for a new tag from hba->cmd_queue.

Bean

#25 lore raw

Re: [PATCH v2 5/5] scsi: ufs: UFS Host Performance Booster(HPB) driver

In reply to [email protected]
Quoted 38 lines · On 2020-04-16 13:31, [email protected] wrote:
drivers/scsi/ufs/ufshpb.c new file +38 -0
+static int ufshpb_execute_cmd(struct ufshpb_lu *hpb, unsigned char *cmd)
+{
+	struct scsi_sense_hdr sshdr;
+	struct scsi_device *sdp;
+	struct ufs_hba *hba;
+	int retries;
+	int ret = 0;
+
+	hba = hpb->hba;
+
+	sdp = hba->sdev_ufs_lu[hpb->lun];
+	if (!sdp) {
+		hpb_warn("%s UFSHPB cannot find scsi_device\n",  __func__);
+		return -ENODEV;
+	}
+
+	ret = scsi_device_get(sdp);
+	if (!ret && !scsi_device_online(sdp)) {
+		ret = -ENODEV;
+		scsi_device_put(sdp);
+		return ret;
+	}
+
+	for (retries = 3; retries > 0; --retries) {
+		ret = scsi_execute(sdp, cmd, DMA_NONE, NULL, 0, NULL, &sshdr,
+				   msecs_to_jiffies(30000), 3, 0, RQF_PM, NULL);
+		if (ret == 0)
+			break;
+	}
+
+	if (ret) {
+		if (driver_byte(ret) == DRIVER_SENSE &&
+		    scsi_sense_valid(&sshdr)) {
+			switch (sshdr.sense_key) {
+			case ILLEGAL_REQUEST:
+				hpb_err("ILLEGAL REQUEST asc=0x%x ascq=0x%x\n",
+					sshdr.asc, sshdr.ascq);
+				break;

drivers/scsi/ufs/ufshpb.c line 1098

Quoted 10 lines
drivers/scsi/ufs/ufshpb.c new file +10 -0
+			default:
+				hpb_err("Unknown return code = %x\n", ret);
+				break;
+			}
+		}
+	}
+	scsi_device_put(sdp);
+
+	return ret;
+}

If scsi_execute() would be changed into asynchronous SCSI command submission, can ufshpb_execute_cmd() be called from inside the UFS .queue_rq() callback instead of from workqueue context?

The scsi_device_get() call looks misplaced. I think that call should happen before schedule_work() is called.

Quoted 49 lines
drivers/scsi/ufs/ufshpb.c new file +49 -0
+static int ufshpb_l2p_load_req(struct ufshpb_lu *hpb, struct request_queue *q,
+			       struct ufshpb_map_req *map_req)
+{
+	struct scsi_request *scsi_rq;
+	unsigned char cmd[16] = { };
+	struct request *req;
+	struct bio *bio;
+	int alloc_len;
+	int ret;
+
+	bio = &map_req->bio;
+
+	ret = ufshpb_add_bio_page(hpb, q, bio, map_req->bvec, map_req->mctx);
+	if (ret) {
+		hpb_err("ufshpb_add_bio_page() failed with ret %d\n", ret);
+		return ret;
+	}
+
+	alloc_len = hpb->hba->hpb_geo.subregion_entry_sz;
+	/*
+	 * HPB Sub-Regions are equally sized except for the last one which is
+	 * smaller if the last hpb Region is not an integer multiple of
+	 * bHPBSubRegionSize.
+	 */
+	if (map_req->region == (hpb->lu_region_cnt - 1) &&
+	    map_req->subregion == (hpb->subregions_in_last_region - 1))
+		alloc_len = hpb->last_subregion_entry_size;
+
+	ufshpb_prepare_read_buf_cmd(cmd, map_req->region, map_req->subregion,
+				    alloc_len);
+	if (!map_req->req) {
+		map_req->req = blk_get_request(q, REQ_OP_SCSI_IN, 0);
+		if (IS_ERR(map_req->req))
+			return PTR_ERR(map_req->req);
+	}
+	req = map_req->req;
+	scsi_rq = scsi_req(req);
+
+	blk_rq_append_bio(req, &bio);
+
+	scsi_req_init(scsi_rq);
+
+	scsi_rq->cmd_len = COMMAND_SIZE(cmd[0]);
+	memcpy(scsi_rq->cmd, cmd, scsi_rq->cmd_len);
+	req->timeout = msecs_to_jiffies(30000);
+	req->end_io_data = (void *)map_req;
+
+	hpb_dbg(SCHEDULE_INFO, hpb, "ISSUE READ_BUFFER : (%d-%d) retry = %d\n",
+		map_req->region, map_req->subregion, map_req->retry_cnt);

drivers/scsi/ufs/ufshpb.c line 1208

Quoted 10 lines
drivers/scsi/ufs/ufshpb.c new file +10 -0
+	hpb_trace(hpb, "%s: I RB %d - %d", DRIVER_NAME, map_req->region,
+		  map_req->subregion);
+
+	blk_execute_rq_nowait(q, NULL, req, 1, ufshpb_l2p_load_req_done);
+	map_req->req_issue_t = ktime_to_ns(ktime_get());
+
+	atomic64_inc(&hpb->status.map_req_cnt);
+
+	return 0;
+}

Same question here: if 'req' would be submitted asynchronously, can ufshpb_l2p_load_req() be called directly from inside the UFS .queue_rq() callback instead of from workqueue context?

Thanks,

Bart.

author
#26 lore raw

Re: [PATCH v2 5/5] scsi: ufs: UFS Host Performance Booster(HPB) driver

Quoted 119 lines · Bart Van Assche
Quoted 38 lines · On 2020-04-16 13:31, [email protected] wrote:
drivers/scsi/ufs/ufshpb.c new file +38 -0
+static int ufshpb_execute_cmd(struct ufshpb_lu *hpb, unsigned char *cmd)
+{
+	struct scsi_sense_hdr sshdr;
+	struct scsi_device *sdp;
+	struct ufs_hba *hba;
+	int retries;
+	int ret = 0;
+
+	hba = hpb->hba;
+
+	sdp = hba->sdev_ufs_lu[hpb->lun];
+	if (!sdp) {
+		hpb_warn("%s UFSHPB cannot find scsi_device\n",  __func__);
+		return -ENODEV;
+	}
+
+	ret = scsi_device_get(sdp);
+	if (!ret && !scsi_device_online(sdp)) {
+		ret = -ENODEV;
+		scsi_device_put(sdp);
+		return ret;
+	}
+
+	for (retries = 3; retries > 0; --retries) {
+		ret = scsi_execute(sdp, cmd, DMA_NONE, NULL, 0, NULL, &sshdr,
+				   msecs_to_jiffies(30000), 3, 0, RQF_PM, NULL);
+		if (ret == 0)
+			break;
+	}
+
+	if (ret) {
+		if (driver_byte(ret) == DRIVER_SENSE &&
+		    scsi_sense_valid(&sshdr)) {
+			switch (sshdr.sense_key) {
+			case ILLEGAL_REQUEST:
+				hpb_err("ILLEGAL REQUEST asc=0x%x ascq=0x%x\n",
+					sshdr.asc, sshdr.ascq);
+				break;

drivers/scsi/ufs/ufshpb.c line 1098

Quoted 10 lines
drivers/scsi/ufs/ufshpb.c new file +10 -0
+			default:
+				hpb_err("Unknown return code = %x\n", ret);
+				break;
+			}
+		}
+	}
+	scsi_device_put(sdp);
+
+	return ret;
+}

If scsi_execute() would be changed into asynchronous SCSI command submission, can ufshpb_execute_cmd() be called from inside the UFS .queue_rq() callback instead of from workqueue context?

The scsi_device_get() call looks misplaced. I think that call should happen before schedule_work() is called.

Quoted 49 lines
drivers/scsi/ufs/ufshpb.c new file +49 -0
+static int ufshpb_l2p_load_req(struct ufshpb_lu *hpb, struct request_queue *q,
+			       struct ufshpb_map_req *map_req)
+{
+	struct scsi_request *scsi_rq;
+	unsigned char cmd[16] = { };
+	struct request *req;
+	struct bio *bio;
+	int alloc_len;
+	int ret;
+
+	bio = &map_req->bio;
+
+	ret = ufshpb_add_bio_page(hpb, q, bio, map_req->bvec, map_req->mctx);
+	if (ret) {
+		hpb_err("ufshpb_add_bio_page() failed with ret %d\n", ret);
+		return ret;
+	}
+
+	alloc_len = hpb->hba->hpb_geo.subregion_entry_sz;
+	/*
+	 * HPB Sub-Regions are equally sized except for the last one which is
+	 * smaller if the last hpb Region is not an integer multiple of
+	 * bHPBSubRegionSize.
+	 */
+	if (map_req->region == (hpb->lu_region_cnt - 1) &&
+	    map_req->subregion == (hpb->subregions_in_last_region - 1))
+		alloc_len = hpb->last_subregion_entry_size;
+
+	ufshpb_prepare_read_buf_cmd(cmd, map_req->region, map_req->subregion,
+				    alloc_len);
+	if (!map_req->req) {
+		map_req->req = blk_get_request(q, REQ_OP_SCSI_IN, 0);
+		if (IS_ERR(map_req->req))
+			return PTR_ERR(map_req->req);
+	}
+	req = map_req->req;
+	scsi_rq = scsi_req(req);
+
+	blk_rq_append_bio(req, &bio);
+
+	scsi_req_init(scsi_rq);
+
+	scsi_rq->cmd_len = COMMAND_SIZE(cmd[0]);
+	memcpy(scsi_rq->cmd, cmd, scsi_rq->cmd_len);
+	req->timeout = msecs_to_jiffies(30000);
+	req->end_io_data = (void *)map_req;
+
+	hpb_dbg(SCHEDULE_INFO, hpb, "ISSUE READ_BUFFER : (%d-%d) retry = %d\n",
+		map_req->region, map_req->subregion, map_req->retry_cnt);

drivers/scsi/ufs/ufshpb.c line 1208

Quoted 10 lines
drivers/scsi/ufs/ufshpb.c new file +10 -0
+	hpb_trace(hpb, "%s: I RB %d - %d", DRIVER_NAME, map_req->region,
+		  map_req->subregion);
+
+	blk_execute_rq_nowait(q, NULL, req, 1, ufshpb_l2p_load_req_done);
+	map_req->req_issue_t = ktime_to_ns(ktime_get());
+
+	atomic64_inc(&hpb->status.map_req_cnt);
+
+	return 0;
+}

Same question here: if 'req' would be submitted asynchronously, can ufshpb_l2p_load_req() be called directly from inside the UFS .queue_rq() callback instead of from workqueue context?

Quoted 3 lines

Thanks,

Bart.

Hi, Bart
Thanks very much.
If I understood the Christoph's the second question correctly. Enqueue HPB requests to the scsi_device->request_queueu, and fly back to SCSI schedule, it's unacceptable. I don't like this implementation way either. Also, the latency of the L2P table update is higher. I will change it in the RFC patch.

thanks,
Bean

#27 lore raw
In reply to [email protected]
Quoted 16 lines · On Thu, Apr 16, 2020 at 10:31:21PM +0200, [email protected] wrote:

From: Bean Huo <[email protected]>

Hi,
I disclose my development/changes on UFS HPB (UFS Host Performance Booster) to the community, as this would enable more UFS developers to test and start an iterative review and update process.

The HPB is defined in Jedec Standard Universal Flash (UFS) Host Performance Booster (HPB) Extension Version 1.0, which is designed to improve read performance by utilizing the host side memory. Based on our testing, the HPB can increase the random read performance by up to about 46% in the random read.

The original HPB driver is from [1]. Based on it, I did optimizations, simplifications, fixed reliability issues, implemented HPB host control mode and make it much more readable. which's pushed to here [2] as V1.

Quoted 6 lines

To avoid touching the traditional SCSI core, the HPB driver in this series HPB patch chooses to develop under SCSI sub-system layer, and sits the same layer with UFSHCD. At the same time, to minimize changes to UFSHCD driver, the HPB driver submits its HPB READ BUFFER and HPB WRITE BUFFER requests to the scsi device->request_queueu to execute, rather than that directly go through raw UPIU request path.

This feature is completley broken, and rather dangerous due to feeding "physical" addresses looked up by the host in. I do not think we should support something that broken in Linux.

Independent of that using two requests in the I/O path is not going to fly either. The whole thing seems like an exercise in benchmarketing.

#28 lore raw

Hi, Christoph
Thanks for your feedback.

Quoted 9 lines
Quoted 8 lines

To avoid touching the traditional SCSI core, the HPB driver in this series HPB patch chooses to develop under SCSI sub-system layer, and sits the same layer with UFSHCD. At the same time, to minimize changes to UFSHCD driver, the HPB driver submits its HPB READ BUFFER and HPB WRITE BUFFER requests to the scsi
device->request_queueu to execute, rather than that directly go device->through
raw UPIU request path.

Quoted 4 lines

This feature is completley broken, and rather dangerous due to feeding "physical" addresses looked up by the host in. I do not think we should support something that broken in Linux.

It Is not plain physical address, has been encrypted before loading from UFS to HPB memory, I think we don't worry about its safety.

Quoted 2 lines

Independent of that using two requests in the I/O path is not going to fly either. The whole thing seems like an exercise in benchmarketing.

I agree with you. This is my major concern. I have been thinking about HPB implementation in SCSI layer. That will let SCSI layer manage HPB by calling UFS helper interface.
If you don't consider UFS HPB is an idiot design, I want to change in another version. Firstly, we really want to hear your suggestion.
Thanks,

//Bean

#29 lore raw
Quoted 36 lines

Hi, Christoph
Thanks for your feedback.

Quoted 11 lines
Quoted 8 lines

To avoid touching the traditional SCSI core, the HPB driver in this series HPB patch chooses to develop under SCSI sub-system layer, and sits the same layer with UFSHCD. At the same time, to minimize changes to UFSHCD driver, the HPB driver submits its HPB READ BUFFER and HPB WRITE BUFFER requests to the scsi
device->request_queueu to execute, rather than that directly go device->through
raw UPIU request path.

This feature is completley broken, and rather dangerous due to feeding "physical" addresses looked up by the host in. I do not think we should

support

Quoted 2 lines

something that broken in Linux.

It Is not plain physical address, has been encrypted before loading from UFS to
HPB memory, I think we don't worry about its safety.

Quoted 1 line

Independent of that using two requests in the I/O path is not going to fly

either.

Quoted 1 line

The whole thing seems like an exercise in benchmarketing.

I agree with you. This is my major concern. I have been thinking about HPB implementation in SCSI layer.
That will let SCSI layer manage HPB by calling UFS helper interface. If you don't consider UFS HPB is an idiot design, I want to change in another version. Firstly, we really
want to hear your suggestion.
Thanks,

Quoted 1 line

//Bean

If indeed this will move forward, please publish your patches as a RFC, To allow competing approaches to be published as well.

Thanks,
Avri