UFS Host Performance Booster (HPB v1.0) driver
v2 cover UFS Host Performance Booster (HPB v1.0) driver
From: Bean Huo <[email protected]>
Hi,
I disclose my development/changes on UFS HPB (UFS Host Performance Booster) to the community, as this would enable more UFS developers to test and start an iterative review and update process.
The HPB is defined in Jedec Standard Universal Flash (UFS) Host Performance Booster (HPB) Extension Version 1.0, which is designed to improve read performance by utilizing the host side memory. Based on our testing, the HPB can increase the random read performance by up to about 46% in the random read.
The original HPB driver is from [1]. Based on it, I did optimizations, simplifications, fixed reliability issues, implemented HPB host control mode and make it much more readable. which's pushed to here [2] as V1.
To avoid touching the traditional SCSI core, the HPB driver in this series HPB patch chooses to develop under SCSI sub-system layer, and sits the same layer with UFSHCD. At the same time, to minimize changes to UFSHCD driver, the HPB driver submits its HPB READ BUFFER and HPB WRITE BUFFER requests to the scsi device->request_queueu to execute, rather than that directly go through raw UPIU request path.
v1--v2:
1. Rebased the patch on the [3] branch 5.8/scsi-queue
1. Optimized and simplified several functions
2. Add parameter read_threshold in HPB sysfs, by which the user can change read_threshold for the HPB host control mode
3. Add HPB memory limitation, let the user adjust its size according to the system memory capacity
[1] https://github.com/OpenMPDK/HPBDriver
[2] https://www.spinics.net/lists/kernel/msg3449471.html
[3] https://git.kernel.org/pub/scm/linux/kernel/git/mkp/scsi.git
Bean Huo (5):
scsi; ufs: add device descriptor for Host Performance Booster
scsi: ufs: make ufshcd_read_unit_desc_param() non-static func
scsi: ufs: add ufs_features parameter in structure ufs_dev_info
scsi: ufs: add unit and geometry parameters for HPB
scsi: ufs: UFS Host Performance Booster(HPB) driver
drivers/scsi/ufs/Kconfig | 62 + drivers/scsi/ufs/Makefile | 1 + drivers/scsi/ufs/ufs.h | 15 + drivers/scsi/ufs/ufshcd.c | 66 +- drivers/scsi/ufs/ufshcd.h | 15 +- drivers/scsi/ufs/ufshpb.c | 3279 +++++++++++++++++++++++++++++++++++++ drivers/scsi/ufs/ufshpb.h | 450 +++++ 7 files changed, 3881 insertions(+), 7 deletions(-) create mode 100644 drivers/scsi/ufs/ufshpb.c create mode 100644 drivers/scsi/ufs/ufshpb.h
Signature · 1 line
2.17.1
PATCH v2 1/5 scsi; ufs: add device descriptor for Host Performance Booster
From: Bean Huo <[email protected]>
Add HPB related attributes in devivce descriptor. HPB support is specified by Bit7 of bUFSFeatureSupport, HPB version is indicated by wHPBVersion and the HPB Control Mode is specified by bHPBControl in UFS device descriptor.
--- drivers/scsi/ufs/ufs.h | 6 ++++++ drivers/scsi/ufs/ufshcd.c | 11 +++++++++++ 2 files changed, 17 insertions(+)
drivers/scsi/ufs/ufs.h +6 -0
@@ -258,6 +258,8 @@ enum device_desc_param {DEVICE_DESC_PARAM_PSA_MAX_DATA =0x25 ,DEVICE_DESC_PARAM_PSA_TMT =0x29 ,DEVICE_DESC_PARAM_PRDCT_REV =0x2A , +DEVICE_DESC_PARAM_HPB_VER =0x40 , + DEVICE_DESC_PARAM_HPB_CTRL_MODE = 0x42, };/* Interconnect descriptor parameters offsets in bytes*/
@@ -537,6 +539,10 @@ struct ufs_dev_info {u8 *model;u16 wspecversion;u32 clk_gating_wait_us; +/* HPB Version */ +u16 hpb_ver; +/* bHPBControl */ +u8 hpb_control_mode; };/**
drivers/scsi/ufs/ufshcd.c +11 -0
@@ -6627,6 +6627,17 @@ static int ufs_get_device_desc(struct ufs_hba *hba) goto out; } +if (desc_buf[DEVICE_DESC_PARAM_UFS_FEAT ] &0x80 ) { + hba->dev_info.hpb_control_mode = + desc_buf[DEVICE_DESC_PARAM_HPB_CTRL_MODE ]; + hba->dev_info.hpb_ver = + (u16) (desc_buf[DEVICE_DESC_PARAM_HPB_VER ] <<8 ) | + desc_buf[DEVICE_DESC_PARAM_HPB_VER +1 ]; +dev_info (hba->dev,"HPB Version: 0x%2x\n" , + hba->dev_info.hpb_ver); +dev_info (hba->dev,"HPB control mode: %d\n" , + hba->dev_info.hpb_control_mode); + }/* * getting vendor (manufacturerID) and Bank Index in big endian * format
Signature · 1 line
2.17.1
Re: [PATCH v2 1/5] scsi; ufs: add device descriptor for Host Performance Booster
drivers/scsi/ufs/ufshcd.c static int ufs_get_device_desc(struct ufs_hba *hba) · line 6630
Quoted 1 line · On 4/16/20 1:31 PM, [email protected] wrote:
drivers/scsi/ufs/ufshcd.c +1
-0+if (desc_buf[DEVICE_DESC_PARAM_UFS_FEAT ] &0x80 ) {
Please introduce a symbolic name instead of using the number 0x80 directly.
drivers/scsi/ufs/ufshcd.c static int ufs_get_device_desc(struct ufs_hba *hba) · line 6635
Quoted 5 lines
drivers/scsi/ufs/ufshcd.c +5
-0+ hba->dev_info.hpb_control_mode = + desc_buf[DEVICE_DESC_PARAM_HPB_CTRL_MODE ]; + hba->dev_info.hpb_ver = + (u16) (desc_buf[DEVICE_DESC_PARAM_HPB_VER ] <<8 ) | + desc_buf[DEVICE_DESC_PARAM_HPB_VER +1 ];
Please use get_unaligned_be16() instead of open-coding it.
Thanks,
Bart.
RE: [EXT] Re: [PATCH v2 1/5] scsi; ufs: add device descriptor for Host Performance Booster
Hi Bart
Thanks your review, I will take your suggestions in next version development. Thanks,
Bean
Quoted 18 lines · On 4/16/20 1:31 PM, [email protected] wrote:
drivers/scsi/ufs/ufshcd.c static int ufs_get_device_desc(struct ufs_hba *hba) · line 6630
Quoted 1 line
drivers/scsi/ufs/ufshcd.c +1
-0+if (desc_buf[DEVICE_DESC_PARAM_UFS_FEAT ] &0x80 ) {Please introduce a symbolic name instead of using the number
0x80directly.drivers/scsi/ufs/ufshcd.c static int ufs_get_device_desc(struct ufs_hba *hba)
Quoted 4 lines
drivers/scsi/ufs/ufshcd.c +4
-0+ hba->dev_info.hpb_control_mode = + desc_buf[DEVICE_DESC_PARAM_HPB_CTRL_MODE ]; + hba->dev_info.hpb_ver = + (u16) (desc_buf[DEVICE_DESC_PARAM_HPB_VER ] <<8 )|
drivers/scsi/ufs/ufshcd.c static int ufs_get_device_desc(struct ufs_hba *hba) · line 6635
Quoted 1 line
drivers/scsi/ufs/ufshcd.c +1
-0+ desc_buf[DEVICE_DESC_PARAM_HPB_VER +1 ];Please use
get_unaligned_be16()instead of open-coding it.Thanks,
Bart.
Re: [EXT] Re: [PATCH v2 1/5] scsi; ufs: add device descriptor for Host Performance Booster
Quoted 1 line · Bean Huo (beanhuo)
Thanks your review, I will take your suggestions in next version development.
Hi Bean,
Please make sure that the issues raised by Christoph are addressed before reposting this series.
Additionally, I think the following two call chains are deadlock-prone and should be addressed:
ufshcd_queuecommand()
-> ufshcd_comp_scsi_upiu()
-> ufshpb_prep_fn()
-> schedule_work(&hpb->ufshpb_req_work)
-> ufshpb_req_workq_fn()
-> ufshpb_subregion_activate()
-> ufshpb_subregion_l2p_load()
-> ufshpb_l2p_load_req()
-> blk_execute_rq_nowait()ufshcd_queuecommand()
-> ufshcd_comp_scsi_upiu()
-> ufshpb_prep_fn()
-> schedule_work(&hpb->ufshpb_req_work)
-> ufshpb_req_workq_fn()
-> ufshpb_subregion_activate()
-> ufshpb_region_deactivate()
-> ufshpb_execute_cmd()
-> scsi_execute()I don't know any other block or SCSI driver that calls blk_execute_rq*() or scsi_execute() from inside its .queue_rq() callback to queue requests onto the same request queue (the device mapper requeues requests onto a lower level request queue).
Thanks,
Bart.
PATCH v2 2/5 scsi: ufs: make ufshcd_read_unit_desc_param() non-static func
In reply to [email protected]From: Bean Huo <[email protected]>
HPB driver needs to read unit descriptors through ufshcd_read_unit_desc_param(), make it an extern Function.
--- drivers/scsi/ufs/ufshcd.c | 8 +++----- drivers/scsi/ufs/ufshcd.h | 5 ++++- 2 files changed, 7 insertions(+), 6 deletions(-)
drivers/scsi/ufs/ufshcd.c +3 -5
@@ -3326,11 +3326,9 @@ int ufshcd_read_string_desc(struct ufs_hba *hba, u8 desc_index,* * Return 0 in case of success, non-zero otherwise */ -static inline int ufshcd_read_unit_desc_param (struct ufs_hba *hba, -int lun, -enum unit_desc_param param_offset, -u8 *param_read_buf, -u32 param_size) +int ufshcd_read_unit_desc_param (struct ufs_hba *hba,int lun, +enum unit_desc_param param_offset, +u8 *param_read_buf,u32 param_size) {/* * Unit descriptors are only available for general purpose LUs (LUN id
drivers/scsi/ufs/ufshcd.h +4 -1
@@ -856,7 +856,10 @@ extern int ufshcd_dme_set_attr(struct ufs_hba *hba, u32 attr_sel,extern int ufshcd_dme_get_attr (struct ufs_hba *hba,u32 attr_sel,u32 *mib_val,u8 peer);extern int ufshcd_config_pwr_mode (struct ufs_hba *hba, -struct ufs_pa_layer_attr *desired_pwr_mode); +struct ufs_pa_layer_attr *desired_pwr_mode); +extern int ufshcd_read_unit_desc_param (struct ufs_hba *hba,int lun, +enum unit_desc_param param_offset, +u8 *param_read_buf,u32 param_size);/* UIC command interfaces for DME primitives */ #define DME_LOCAL 0
Signature · 1 line
2.17.1
PATCH v2 3/5 scsi: ufs: add ufs_features parameter in structure ufs_dev_info
In reply to [email protected]From: Bean Huo <[email protected]>
Make a copy of bUFSFeaturesSupport, name it ufs_features, add it to structure ufs_dev_info.
--- drivers/scsi/ufs/ufs.h | 2 ++ drivers/scsi/ufs/ufshcd.c | 2 ++ 2 files changed, 4 insertions(+)
drivers/scsi/ufs/ufs.h +2 -0
@@ -543,6 +543,8 @@ struct ufs_dev_info {u16 hpb_ver;/* bHPBControl */ u8 hpb_control_mode; +/* bUFSFeaturesSupport */ +u8 ufs_features; };/**
drivers/scsi/ufs/ufshcd.c +2 -0
@@ -6625,6 +6625,8 @@ static int ufs_get_device_desc(struct ufs_hba *hba) goto out; } + dev_info->ufs_features = desc_buf[DEVICE_DESC_PARAM_UFS_FEAT ]; +if (desc_buf[DEVICE_DESC_PARAM_UFS_FEAT ] &0x80 ) { hba->dev_info.hpb_control_mode = desc_buf[DEVICE_DESC_PARAM_HPB_CTRL_MODE ];
Signature · 1 line
2.17.1
Re: [PATCH v2 3/5] scsi: ufs: add ufs_features parameter in structure ufs_dev_info
Quoted 6 lines · On 4/16/20 1:31 PM, [email protected] wrote:
drivers/scsi/ufs/ufshcd.c +0
-0
drivers/scsi/ufs/ufshcd.c static int ufs_get_device_desc(struct ufs_hba *hba) · line 6632
Quoted 8 lines
drivers/scsi/ufs/ufshcd.c +2
-0@@ -6625,6 +6625,8 @@ static int ufs_get_device_desc(struct ufs_hba *hba) goto out; } + dev_info->ufs_features = desc_buf[DEVICE_DESC_PARAM_UFS_FEAT ]; +if (desc_buf[DEVICE_DESC_PARAM_UFS_FEAT ] &0x80 ) { hba->dev_info.hpb_control_mode = desc_buf[DEVICE_DESC_PARAM_HPB_CTRL_MODE ];
Since this change is very closely related to the changes in patch 1/3, please merge this patch into patch 1/3.
Thanks,
Bart.
PATCH v2 4/5 scsi: ufs: add unit and geometry parameters for HPB
In reply to [email protected]From: Bean Huo <[email protected]>
Add HPB related parameters introduced in Unit Descriptor and Geometry Descriptor.
--- drivers/scsi/ufs/ufs.h | 7 +++++++ 1 file changed, 7 insertions(+)
drivers/scsi/ufs/ufs.h +7 -0
@@ -219,6 +219,9 @@ enum unit_desc_param {UNIT_DESC_PARAM_PHY_MEM_RSRC_CNT =0x18 ,UNIT_DESC_PARAM_CTX_CAPABILITIES =0x20 ,UNIT_DESC_PARAM_LARGE_UNIT_SIZE_M1 =0x22 , +UNIT_DESC_PARAM_HPB_MAX_ACTIVE_REGIONS =0x23 , +UNIT_DESC_PARAM_HPB_PIN_REGION_START_OFFSET =0x25 , + UNIT_DESC_PARAM_HPB_NUM_PIN_REGIONS = 0x27, };/* Device descriptor parameters offsets in bytes*/
@@ -304,6 +307,10 @@ enum geometry_desc_param {GEOMETRY_DESC_PARAM_ENM4_MAX_NUM_UNITS =0x3E ,GEOMETRY_DESC_PARAM_ENM4_CAP_ADJ_FCTR =0x42 ,GEOMETRY_DESC_PARAM_OPT_LOG_BLK_SIZE =0x44 , +GEOMETRY_DESC_PARAM_HPB_REGION_SIZE =0x48 , +GEOMETRY_DESC_PARAM_HPB_NUMBER_LU =0x49 , +GEOMETRY_DESC_PARAM_HPB_SUBREGION_SIZE =0x4A , + GEOMETRY_DESC_PARAM_HPB_MAX_ACTIVE_REGIONS = 0x4B, };/* Health descriptor parameters offsets in bytes*/
Signature · 1 line
2.17.1
PATCH v2 5/5 scsi: ufs: UFS Host Performance Booster(HPB) driver
In reply to [email protected]From: Bean Huo <[email protected]>
This patch is to add support for the UFS Host Performance Booster (HPB v1.0), which is used to improve UFS read performance, especially for the random read.
NAND flash-based storage devices, including UFS, have mechanisms to translate logical addresses of requests to the corresponding physical addresses of the flash storage. Traditionally this L2P mapping data is loaded to the internal SRAM in the storage controller. When the capacity of storage is larger, a larger size of SRAM for the L2P map data is required. Since increased SRAM size affects the manufacturing cost significantly, it is not cost-effective to allocate all the amount of SRAM needed to keep all the Logical-address to Physical-address (L2P) map data. Therefore, L2P map data, which is required to identify the physical address for the requested IOs, can only be partially stored in SRAM from NAND flash. Due to this partial loading, accessing the flash address area where the L2P information for that address is not loaded in the SRAM can result in serious performance degradation.
The HPB is a software solution for the above problem, which uses the host’s system memory as a cache for the FTL L2P mapping table. It does not need additional hardware support from the host side. By using HPB, the L2P mapping table can be read from host memory and stored in host-side memory. while reading the operation, the corresponding L2P information will be sent to the UFS device along with the reading request. Since the L2P entry is provided in the read request, UFS device does not have to load L2P entry from flash memory. This will significantly improve random read performance.
--- drivers/scsi/ufs/Kconfig | 62 + drivers/scsi/ufs/Makefile | 1 + drivers/scsi/ufs/ufshcd.c | 55 +- drivers/scsi/ufs/ufshcd.h | 10 + drivers/scsi/ufs/ufshpb.c | 3279 +++++++++++++++++++++++++++++++++++++ drivers/scsi/ufs/ufshpb.h | 450 +++++ 6 files changed, 3851 insertions(+), 6 deletions(-) create mode 100644 drivers/scsi/ufs/ufshpb.c create mode 100644 drivers/scsi/ufs/ufshpb.h
drivers/scsi/ufs/Kconfig +62 -0
@@ -160,3 +160,65 @@ config SCSI_UFS_BSG Select this if you need a bsg device node for your UFS controller. If unsure, say N. + +config SCSI_UFSHPB + bool "UFS Host Performance Booster (EXPERIMENTAL)" + depends on SCSI_UFSHCD + help + NAND flash-based storage devices, including UFS, have mechanisms to + translate logical addresses of the IO requests to the corresponding + physical addresses of the flash storage. Traditionally, this L2P + mapping data is loaded to the internal SRAM in the storage controller. + When the capacity of storage is larger, a larger size of SRAM for the + L2P map data is required. Since increased SRAM size affects the + manufacturing cost significantly, it is not cost-effective to allocate + all the amount of SRAM needed to keep all the Logical-address to + Physical-address (L2P) map data. Therefore, L2P map data, which is + required to identify the physical address for the requested IOs, can + only be partially stored in SRAM from NAND flash. Due to this partial + loading, accessing the flash address area where the L2P information + for that address is not loaded in the SRAM can result in serious + performance degradation. + + UFS Host Performance Booster (HPB) is a software solution for the + above problem, which uses the host side system memory as a cache for + the FTL L2P mapping table. It does not need additional hardware + support from the host side. By using HPB, the L2P mapping table can be + read from host memory and stored in host-side memory. while reading + the operation, the corresponding L2P information will be sent to the + UFS device along with the reading request. Since the L2P entry is + provided in the read request, UFS device does not have to load L2P + entry from flash memory to UFS internal SRAM. This will significantly + improve the read performance. + + When selected, this feature will be built in the UFS driver. + + If in doubt, say N. + +config UFSHPB_MAX_MEM_SIZE + int "UFS HPB maximum memory size per controller (in MiB)" + depends on SCSI_UFSHPB + default 128 + range 0 65536 + help + This parameter defines the maximum UFS HPB memory/cache size in the + host system. The recommended HPB cache size by the UFS device can be + calculated from bHPBRegionSize and wDeviceMaxActiveHPBRegions. The + reference formula can be + + (bHPBRegionSize(in KB) / 4KB) * 8 * wDeviceMaxActiveHPBRegions. + + The HPB cache in the host system is used to contain L2P mapping entry. + If the allocated HPB cache size is lower than what calculated by the + above formula, the use of HPB feature may provide lower performance + advantage. But the system memory resource has the limitation, we can + not let HPB driver allocate its cache at will according to the UFS + device recommendation, so an appropriate size of the cache for HPB + should be specified before you choose to use HPB, then please input a + non-zero positive integer value. + + Nevertheless, if you want to leave this right to the HPB driver, and + let the HPB driver allocate the HPB cache based on the recommendation + of the UFS device. Just give 0 value to this parameter. + + Leave the default value if unsure.
drivers/scsi/ufs/Makefile +1 -0
@@ -7,6 +7,7 @@ obj-$(CONFIG_SCSI_UFS_QCOM) += ufs-qcom.o obj-$(CONFIG_SCSI_UFSHCD) += ufshcd-core.o ufshcd-core-y += ufshcd.o ufs-sysfs.o ufshcd-core-$(CONFIG_SCSI_UFS_BSG) += ufs_bsg.o +ufshcd-core-$(CONFIG_SCSI_UFSHPB) += ufshpb.o obj-$(CONFIG_SCSI_UFSHCD_PCI) += ufshcd-pci.o obj-$(CONFIG_SCSI_UFSHCD_PLATFORM) += ufshcd-pltfrm.o obj-$(CONFIG_SCSI_UFS_HISI) += ufs-hisi.o
drivers/scsi/ufs/ufshcd.c +49 -6
114 lines not on this page · raw message
drivers/scsi/ufs/ufshcd.h +10 -0
@@ -70,6 +70,7 @@#include "ufs.h" #include "ufshci.h" +#include "ufshpb.h" #define UFSHCD "ufshcd"#define UFSHCD_DRIVER_VERSION "0.2"
@@ -727,6 +728,15 @@ struct ufs_hba {struct device bsg_dev;struct request_queue *bsg_queue; + +#if defined(CONFIG_SCSI_UFSHPB ) +enum UFSHPB_STATE ufshpb_state; +struct ufshpb_geo hpb_geo; +struct ufshpb_lu *ufshpb_lup[32 ]; +struct delayed_work ufshpb_init_work; +struct work_struct ufshpb_eh_work; +struct scsi_device *sdev_ufs_lu[32 ]; +#endif };/* Returns true if clocks can be gated. Otherwise false */
drivers/scsi/ufs/ufshpb.c new file +3279 -0
2,792 lines not on this page · raw message
drivers/scsi/ufs/ufshpb.h new file +450 -0
450 lines not on this page · raw message
The rest of this diff is not shown. View the raw message.
Signature · 1 line
2.17.1
Part of the diff is not shown here. View the raw message.
Re: [PATCH v2 5/5] scsi: ufs: UFS Host Performance Booster(HPB) driver
Hi--
A few Kconfig changes for you to consider:
Quoted 62 lines · On 4/16/20 1:31 PM, [email protected] wrote:
From: Bean Huo <[email protected]>
This patch is to add support for the UFS Host Performance Booster (HPB v1.0), which is used to improve UFS read performance, especially for the random read.
NAND flash-based storage devices, including UFS, have mechanisms to translate logical addresses of requests to the corresponding physical addresses of the flash storage. Traditionally this L2P mapping data is loaded to the internal SRAM in the storage controller. When the capacity of storage is larger, a larger size of SRAM for the L2P map data is required. Since increased SRAM size affects the manufacturing cost significantly, it is not cost-effective to allocate all the amount of SRAM needed to keep all the Logical-address to Physical-address (L2P) map data. Therefore, L2P map data, which is required to identify the physical address for the requested IOs, can only be partially stored in SRAM from NAND flash. Due to this partial loading, accessing the flash address area where the L2P information for that address is not loaded in the SRAM can result in serious performance degradation.
The HPB is a software solution for the above problem, which uses the host’s system memory as a cache for the FTL L2P mapping table. It does not need additional hardware support from the host side. By using HPB, the L2P mapping table can be read from host memory and stored in host-side memory. while reading the operation, the corresponding L2P information will be sent to the UFS device along with the reading request. Since the L2P entry is provided in the read request, UFS device does not have to load L2P entry from flash memory. This will significantly improve random read performance.
Signed-off-by Bean Huo <[email protected]>--- drivers/scsi/ufs/Kconfig | 62 + drivers/scsi/ufs/Makefile | 1 + drivers/scsi/ufs/ufshcd.c | 55 +- drivers/scsi/ufs/ufshcd.h | 10 + drivers/scsi/ufs/ufshpb.c | 3279 +++++++++++++++++++++++++++++++++++++ drivers/scsi/ufs/ufshpb.h | 450 +++++ 6 files changed, 3851 insertions(+), 6 deletions(-) create mode 100644 drivers/scsi/ufs/ufshpb.c create mode 100644 drivers/scsi/ufs/ufshpb.hdrivers/scsi/ufs/Kconfig +15
-0@@ -160,3 +160,65 @@ config SCSI_UFS_BSG Select this if you need a bsg device node for your UFS controller. If unsure, say N. + +config SCSI_UFSHPB + bool "UFS Host Performance Booster (EXPERIMENTAL)" + depends on SCSI_UFSHCD + help + NAND flash-based storage devices, including UFS, have mechanisms to + translate logical addresses of the IO requests to the corresponding + physical addresses of the flash storage. Traditionally, this L2P + mapping data is loaded to the internal SRAM in the storage controller. + When the capacity of storage is larger, a larger size of SRAM for the + L2P map data is required. Since increased SRAM size affects the + manufacturing cost significantly, it is not cost-effective to allocate + all the amount of SRAM needed to keep all the Logical-address to + Physical-address (L2P) map data. Therefore, L2P map data, which is + required to identify the physical address for the requested IOs, can
drivers/scsi/ufs/Kconfig config SCSI_UFS_BSG · line 187
Quoted 10 lines
drivers/scsi/ufs/Kconfig +10
-0+ only be partially stored in SRAM from NAND flash. Due to this partial + loading, accessing the flash address area where the L2P information + for that address is not loaded in the SRAM can result in serious + performance degradation. + + UFS Host Performance Booster (HPB) is a software solution for the + above problem, which uses the host side system memory as a cache for + the FTL L2P mapping table. It does not need additional hardware + support from the host side. By using HPB, the L2P mapping table can be + read from host memory and stored in host-side memory. while reading
^??^ When performing a
drivers/scsi/ufs/Kconfig config SCSI_UFS_BSG · line 188
Quoted 1 line
drivers/scsi/ufs/Kconfig +1
-0+ the operation, the corresponding L2P information will be sent to the
read operation, the
drivers/scsi/ufs/Kconfig config SCSI_UFS_BSG · line 189
Quoted 1 line
drivers/scsi/ufs/Kconfig +1
-0+ UFS device along with the reading request. Since the L2P entry is
the read request.
Quoted 12 lines
drivers/scsi/ufs/Kconfig +12
-0+ provided in the read request, UFS device does not have to load L2P + entry from flash memory to UFS internal SRAM. This will significantly + improve the read performance. + + When selected, this feature will be built in the UFS driver. + + If in doubt, say N. + +config UFSHPB_MAX_MEM_SIZE + int "UFS HPB maximum memory size per controller (in MiB)" + depends on SCSI_UFSHPB + default 128
drivers/scsi/ufs/Kconfig config SCSI_UFS_BSG · line 211
Quoted 10 lines
drivers/scsi/ufs/Kconfig +10
-0+ range 0 65536 + help + This parameter defines the maximum UFS HPB memory/cache size in the + host system. The recommended HPB cache size by the UFS device can be + calculated from bHPBRegionSize and wDeviceMaxActiveHPBRegions. The + reference formula can be + + (bHPBRegionSize(in KB) / 4KB) * 8 * wDeviceMaxActiveHPBRegions. + + The HPB cache in the host system is used to contain L2P mapping entry.
entries.
drivers/scsi/ufs/Kconfig config SCSI_UFS_BSG · line 212
Quoted 1 line
drivers/scsi/ufs/Kconfig +1
-0+ If the allocated HPB cache size is lower than what calculated by the
that
drivers/scsi/ufs/Kconfig config SCSI_UFS_BSG · line 217
Quoted 5 lines
drivers/scsi/ufs/Kconfig +5
-0+ above formula, the use of HPB feature may provide lower performance + advantage. But the system memory resource has the limitation, we can + not let HPB driver allocate its cache at will according to the UFS + device recommendation, so an appropriate size of the cache for HPB + should be specified before you choose to use HPB, then please input a
then enter a
drivers/scsi/ufs/Kconfig config SCSI_UFS_BSG · line 220
Quoted 3 lines
drivers/scsi/ufs/Kconfig +3
-0+ non-zero positive integer value. + + Nevertheless, if you want to leave this right to the HPB driver, and
leave this to the HPB driver, and
drivers/scsi/ufs/Kconfig config SCSI_UFS_BSG · line 222
Quoted 2 lines
drivers/scsi/ufs/Kconfig +2
-0+ let the HPB driver allocate the HPB cache based on the recommendation + of the UFS device. Just give 0 value to this parameter.
device, just
drivers/scsi/ufs/Kconfig config SCSI_UFS_BSG · line 224
Quoted 2 lines
drivers/scsi/ufs/Kconfig +2
-0+ + Leave the default value if unsure.
gday.
Signature · 1 line
~Randy
Re: [PATCH v2 5/5] scsi: ufs: UFS Host Performance Booster(HPB) driver
In reply to [email protected]Quoted 16 lines · On 4/16/20 1:31 PM, [email protected] wrote:
This patch is to add support for the UFS Host Performance Booster (HPB v1.0), which is used to improve UFS read performance, especially for the random read.
NAND flash-based storage devices, including UFS, have mechanisms to translate logical addresses of requests to the corresponding physical addresses of the flash storage. Traditionally this L2P mapping data is loaded to the internal SRAM in the storage controller. When the capacity of storage is larger, a larger size of SRAM for the L2P map data is required. Since increased SRAM size affects the manufacturing cost significantly, it is not cost-effective to allocate all the amount of SRAM needed to keep all the Logical-address to Physical-address (L2P) map data. Therefore, L2P map data, which is required to identify the physical address for the requested IOs, can only be partially stored in SRAM from NAND flash. Due to this partial loading, accessing the flash address area where the L2P information for that address is not loaded in the SRAM can result in serious performance degradation.
Quoted 8 lines
The HPB is a software solution for the above problem, which uses the host’s system memory as a cache for the FTL L2P mapping table. It does not need additional hardware support from the host side. By using HPB, the L2P mapping table can be read from host memory and stored in host-side memory. while reading the operation, the corresponding L2P information will be sent to the UFS device along with the reading request. Since the L2P entry is provided in the read request, UFS device does not have to load L2P entry from flash memory. This will significantly improve random read performance.
The above description is incomplete: what is missing is an explanation of how the L2P data is made persistent and also how it is stored on the device. Is an L2P update written to the device every time data on the device is modified or only during an orderly shutdown? In the former case, how big is the impact on write performance? In the latter case, does that mean that data is lost during an unorderly shutdown? Is HPB a feature that perhaps only works well for battery-powered devices?
Is the persistent L2P data overwritten in place or is new data appended as an update after existing data? In the former case, is all L2P data lost in case of a power failure? In the latter case, how is it guaranteed that enough free space is available to make the L2P data persistent?
What are the similarities and differences compared to the lightnvm framework that was added several years ago to the Linux kernel? Which of the code in this patch can be shared with the lightnvm framework?
Quoted 17 lines
drivers/scsi/ufs/Kconfig +17
-0+config UFSHPB_MAX_MEM_SIZE + int "UFS HPB maximum memory size per controller (in MiB)" + depends on SCSI_UFSHPB + default 128 + range 0 65536 + help + This parameter defines the maximum UFS HPB memory/cache size in the + host system. The recommended HPB cache size by the UFS device can be + calculated from bHPBRegionSize and wDeviceMaxActiveHPBRegions. The + reference formula can be + + (bHPBRegionSize(in KB) / 4KB) * 8 * wDeviceMaxActiveHPBRegions. + + The HPB cache in the host system is used to contain L2P mapping entry. + If the allocated HPB cache size is lower than what calculated by the + above formula, the use of HPB feature may provide lower performance + advantage. But the system memory resource has the limitation, we can
drivers/scsi/ufs/Kconfig config SCSI_UFS_BSG · line 224
Quoted 10 lines
drivers/scsi/ufs/Kconfig +10
-0+ not let HPB driver allocate its cache at will according to the UFS + device recommendation, so an appropriate size of the cache for HPB + should be specified before you choose to use HPB, then please input a + non-zero positive integer value. + + Nevertheless, if you want to leave this right to the HPB driver, and + let the HPB driver allocate the HPB cache based on the recommendation + of the UFS device. Just give 0 value to this parameter. + + Leave the default value if unsure.
Can this parameter be changed from a compile-time option into something that is configurable at runtime, e.g. a sysfs or configfs attribute?
drivers/scsi/ufs/ufshcd.c static int ufshcd_slave_configure(struct scsi_device *sdev) · line 4634
Quoted 4 lines
drivers/scsi/ufs/ufshcd.c +4
-0+#if defined(CONFIG_SCSI_UFSHPB ) +if (sdev->lun < hba->dev_info.max_lu_supported) + hba->sdev_ufs_lu[sdev->lun] = sdev; +#endif
Isn't maintaining a LUN to scsi_device mapping something that is already done by the SCSI core? See also __scsi_device_lookup().
drivers/scsi/ufs/ufshcd.c ufshcd_transfer_rsp_status(struct ufs_hba *hba, struct ufshcd_lrb *lrbp) · line 4759
Quoted 9 lines
drivers/scsi/ufs/ufshcd.c +9
-0+#if defined(CONFIG_SCSI_UFSHPB ) +/* +* HPB recommendations are provided in RESPONSE UPIU +* packets of successfully completed commands, which +* are commands terminated with GOOD status. +*/ +if (scsi_status ==SAM_STAT_GOOD ) +ufshpb_rsp_handler (hba, lrbp); +#endif
Please introduce helper functions such that no #if defined(CONFIG_SCSI_UFSHPB) statements are required in .c files. From Documentation/process/4.Coding.rst: "As a general rule, #ifdef use should be confined to header files whenever possible."
drivers/scsi/ufs/ufshcd.c static int ufs_get_device_desc(struct ufs_hba *hba) · line 6659
Quoted 7 lines
drivers/scsi/ufs/ufshcd.c +1
-2@@ -6627,16 +6656,16 @@ static int ufs_get_device_desc(struct ufs_hba *hba) dev_info->ufs_features = desc_buf[DEVICE_DESC_PARAM_UFS_FEAT ]; -if (desc_buf[DEVICE_DESC_PARAM_UFS_FEAT ] &0x80 ) { - hba->dev_info.hpb_control_mode = +if (dev_info->ufs_features &0x80 ) {
The above change should be moved into the combination of patches 1/5 and 3/5, and the changes below probably too.
Quoted 13 lines
drivers/scsi/ufs/ufshcd.c +4
-3+ dev_info->hpb_control_mode = desc_buf[DEVICE_DESC_PARAM_HPB_CTRL_MODE ]; - hba->dev_info.hpb_ver = + dev_info->hpb_ver = (u16) (desc_buf[DEVICE_DESC_PARAM_HPB_VER ] <<8 ) | desc_buf[DEVICE_DESC_PARAM_HPB_VER +1 ];dev_info (hba->dev,"HPB Version: 0x%2x\n" , - hba->dev_info.hpb_ver); + dev_info->hpb_ver);dev_info (hba->dev,"HPB control mode: %d\n" , - hba->dev_info.hpb_control_mode); + dev_info->hpb_control_mode); }
[ ... ]
Quoted 17 lines
drivers/scsi/ufs/ufshpb.h new file +16
-0+/* BYTE SHIFT */ +#define ZERO_BYTE_SHIFT 0 +#define ONE_BYTE_SHIFT 8 +#define TWO_BYTE_SHIFT 16 +#define THREE_BYTE_SHIFT 24 +#define FOUR_BYTE_SHIFT 32 +#define FIVE_BYTE_SHIFT 40 +#define SIX_BYTE_SHIFT 48 +#define SEVEN_BYTE_SHIFT 56 +#define SHIFT_BYTE_0 (num) ((num) << ZERO_BYTE_SHIFT) +#define SHIFT_BYTE_1 (num) ((num) << ONE_BYTE_SHIFT) +#define SHIFT_BYTE_2 (num) ((num) << TWO_BYTE_SHIFT) +#define SHIFT_BYTE_3 (num) ((num) << THREE_BYTE_SHIFT) +#define SHIFT_BYTE_4 (num) ((num) << FOUR_BYTE_SHIFT) +#define SHIFT_BYTE_5 (num) ((num) << FIVE_BYTE_SHIFT) +#define SHIFT_BYTE_6 (num) ((num) << SIX_BYTE_SHIFT)
drivers/scsi/ufs/ufshpb.h line 109
Quoted 10 lines
drivers/scsi/ufs/ufshpb.h new file +9
-0+#define SHIFT_BYTE_7 (num) ((num) << SEVEN_BYTE_SHIFT) +#define GET_BYTE_0 (num) (((num) >> ZERO_BYTE_SHIFT) & 0xff) +#define GET_BYTE_1 (num) (((num) >> ONE_BYTE_SHIFT) & 0xff) +#define GET_BYTE_2 (num) (((num) >> TWO_BYTE_SHIFT) & 0xff) +#define GET_BYTE_3 (num) (((num) >> THREE_BYTE_SHIFT) & 0xff) +#define GET_BYTE_4 (num) (((num) >> FOUR_BYTE_SHIFT) & 0xff) +#define GET_BYTE_5 (num) (((num) >> FIVE_BYTE_SHIFT) & 0xff) +#define GET_BYTE_6 (num) (((num) >> SIX_BYTE_SHIFT) & 0xff) +#define GET_BYTE_7 (num) (((num) >> SEVEN_BYTE_SHIFT) & 0xff)
Please remove the above 24 macro definitions and use get_unaligned_*() / put_unaligned_*() instead.
Thanks,
Bart.
RE: [EXT] Re: [PATCH v2 5/5] scsi: ufs: UFS Host Performance Booster(HPB) driver
Hi, Bart
Quoted 5 lines
What are the similarities and differences compared to the lightnvm framework that was added several years ago to the Linux kernel? Which of the code in this patch can be shared with the lightnvm framework?
I don’t fully comprehend Lightnvm, and studied it in recent days. it is not possible to deploy Lightnvm framework in UFS HPB.
UFS HPB has a similar idea with Nvme HMB, which both take host-side memory as device onboard flash to hold L2P map entries. But both are not the same, UFS HPB cache cannot be accessed by UFS device over UFS bus. Access and control of UFS HPB cache completely all depend on the host side UFS HPB driver.
HPB has two control modes: device control mode and host control mode. In the device Mode, HPB driver reads L2P map data only based on the device recommendations. However in the host control mode, it is much more flexible, HPB driver completely can initiate the L2P read actively. In host mode, HPB can limit read latency and achieve predictable read latency, since the host side can load the L2P map entries based on its read behavior and oncoming read. But current exist UFS subsystem, cannot make full use of this feature.
I don't know how do you think about adding a framework in SCSI, let FS can pass oncoming read length to UFS HPB, the based on that, HPB can load L2P entries in advance.
//Bean
RE: [PATCH v2 5/5] scsi: ufs: UFS Host Performance Booster(HPB) driver
In reply to Bart Van AsscheHi Bart,
Quoted 3 lines
What are the similarities and differences compared to the lightnvm framework that was added several years ago to the Linux kernel? Which of the code in this patch can be shared with the lightnvm framework?
Simply put, unlike lightnvn, HPB is not host-managed FTL, But instead can be perceived as a cost-reduction effort. Its aim is not to move the fw to the host, but to control the iNAND cost by limiting the amount of its internal RAM. It is done by using the host memory to cache the L2P tables, and replace READ_10 that have only the lba, by an alternative command - HPB_READ, that have both the logical and physical addresses.
Using Lightnvm was considered in the past as possible framework for HPB, but was rejected by both Christoph & Mattias.
The HPB feature was NAKed by Christoph in its entirety, regardless of the driver design. Until this is not reversed, keep commenting this patch is counterproductive and confusing.
Should this decision is reversed, I think this patch should be re-posted as a RFC, And fragment its 5,000 lines or so into a set of reviewable patches.
Thanks,
Avri
RE: [EXT] RE: [PATCH v2 5/5] scsi: ufs: UFS Host Performance Booster(HPB) driver
Hi, Avri
Quoted 3 lines
Should this decision is reversed, I think this patch should be re-posted as a RFC, And fragment its 5,000 lines or so into a set of reviewable patches.
Ok, I will repost it as RFC, and also will fix several issues Bart raised. Thanks.
Bean
RE: [PATCH v2 5/5] scsi: ufs: UFS Host Performance Booster(HPB) driver
In reply to Avri AltmanOne last word for the community members that are not into ufs day-to-day:
HPB implementation made its first public appearance around 2018 as part of Google's Pixel3 and some VIVO models. Since then, more and more mobile platforms are publically using HPB: Galaxy Note10,
Galaxy S20 and VIVO NEX3 (that is already using HPB2.0), some Xiaomi models etc.
On the other hand, HPB1.0 spec was just recently closed - not even as part of UFS3.1, but only after - about 2 months ago. The industry is rushing forward, we've seen this many times.
The fact is that HPB is here to stay - either as a horde of out-of-tree implementations, or as a standard acceptable mainline driver.
Thanks,
Avri
Quoted 29 lines
Hi Bart,
Quoted 3 lines
What are the similarities and differences compared to the lightnvm framework that was added several years ago to the Linux kernel? Which of the code in this patch can be shared with the lightnvm framework?
Simply put, unlike lightnvn, HPB is not host-managed FTL, But instead can be perceived as a cost-reduction effort.
Its aim is not to move the fw to the host, but to control the iNAND cost by limiting the amount of its internal RAM.
It is done by using the host memory to cache the L2P tables, and replaceREAD_10that have only the lba,
by an alternative command -HPB_READ, that have both the logical and physical addresses.Using Lightnvm was considered in the past as possible framework for HPB, but was rejected by both Christoph & Mattias.
The HPB feature was NAKed by Christoph in its entirety, regardless of the driver design.
Until this is not reversed, keep commenting this patch is counterproductive and confusing.Should this decision is reversed, I think this patch should be re-posted as a RFC,
And fragment its 5,000 lines or so into a set of reviewable patches.Thanks,
Avri
Re: [PATCH v2 5/5] scsi: ufs: UFS Host Performance Booster(HPB) driver
Quoted 11 lines · Avri Altman
One last word for the community members that are not into ufs day-to-day:
HPB implementation made its first public appearance around 2018 as part of Google's Pixel3 and some VIVO models. Since then, more and more mobile platforms are publically using HPB: Galaxy Note10,
Galaxy S20 and VIVO NEX3 (that is already using HPB2.0), some Xiaomi models etc.On the other hand, HPB1.0 spec was just recently closed - not even as part of UFS3.1, but only after - about 2 months ago. The industry is rushing forward, we've seen this many times.
The fact is that HPB is here to stay - either as a horde of out-of-tree implementations, or as a standard acceptable mainline driver.
Hi Avri,
Thanks for the additional clarification. I think we need HPB support in the upstream kernel. A possible approach is to start a discussion about how to integrate HPB support in the upstream kernel and to defer posting new implementations until agreement about an approach has been achieved. Is there e.g. an alternative for avoiding deadlocks other than using the blk-mq reserved tag feature for the commands that manage the L2P tables?
Thanks,
Bart.
RE: [PATCH v2 5/5] scsi: ufs: UFS Host Performance Booster(HPB) driver
Quoted 21 lines · Avri Altman
Quoted 1 line
One last word for the community members that are not into ufs day-to-
day:
Quoted 2 lines
HPB implementation made its first public appearance around 2018 as part
of Google's Pixel3 and some VIVO models.
Quoted 1 line
Since then, more and more mobile platforms are publically using HPB:
Galaxy Note10,
Quoted 1 line
Galaxy S20 and VIVO NEX3 (that is already using HPB2.0), some Xiaomi
models etc.
Quoted 2 lines
On the other hand, HPB1.0 spec was just recently closed - not even as part
of UFS3.1,
Quoted 1 line
but only after - about 2 months ago. The industry is rushing forward, we've
seen this many times.
Quoted 2 lines
The fact is that HPB is here to stay - either as a horde of out-of-tree
implementations,
Quoted 1 line
or as a standard acceptable mainline driver.
Quoted 8 lines
Hi Avri,
Thanks for the additional clarification. I think we need HPB support in the upstream kernel. A possible approach is to start a discussion about how to integrate HPB support in the upstream kernel and to defer posting new implementations until agreement about an approach has been achieved. Is there e.g. an alternative for avoiding deadlocks other than using the blk-mq reserved tag feature for the commands that manage the L2P tables?
Ok then.
HPB support is comprised of 4 main duties:
1) Read the device HPB configuration
2) Attend the device's recommendations that are embedded in the sense buffer
3) L2P cache management - This entails sending 2 new scsi commands (opcodes were taken from the vendor pool):
a. HPB-READ-BUFFER - read L2P physical addresses for a subregion
b. HPB-WRITE-BUFFER - notify the device that a region is inactive (in host-managed mode)
4) Use HPB-READ: a 3rd new scsi command (again - uses the vendor pool) to perform read operation instead of READ10. HPB-READ carries both the logical and the physical addresses.
I will let Bean defend the Samsung approach of using a single LLD to attend all 4 duties.
Another approach might be to split those duties between 2 modules:
- A LLD that will perform the first 2 - those can be done only using ufs privet stuff, and
- another module in scsi mid-layer that will be responsible of L2P cache management, and HPB-READ command setup.
A framework to host the scsi mid-layer module can be the scsi device handler.
The scsi-device-handler infrastructure was added to the kernel mainly to facilitate multiple paths for storage devices. The HPB framework, although far from the original intention of the authors, might as well fit in.
In that sense, using READs and HPB_READs intermittently, can be perceived as a multi-path.
Scsi device handlers are also attached to a specific scsi_device (lun).
This can serve as the glue linking between the ufs LLD and the device handler which resides in the scsi level.
Device handlers comes with a rich and handy set of APIs & ops, which we can use to support HPB. Specifically we can use it to attach & activate the device handler,
only after the ufs driver verified that HPB is supported by both the platform and the device.
The 2 modules can communicate using the handler_data opaque pointer, and the handler’s set_params op-mode: which is an open protocol essentially, and we can use it to pass the sense buffer in its full or just a parsed version.
Being a scsi mid-layer module, it will not break anything while sending HPB-READ-BUFFER and HPB-WRITE-BUFFER as part of the L2P cache management duties.
Last but not least, the device handler is already hooked in the scsi command setup flow - scsi_setup_fs_cmnd(), So we get the hook into HPB-READ prep_fn for free.
Later on, we might want to export the L2P cache management logic to user-space.
Locating the L2P cache management in scsi mid-layer will enable us to do so, using the scsi-netlink or some other means.
Thanks,
Avri
Re: [PATCH v2 5/5] scsi: ufs: UFS Host Performance Booster(HPB) driver
Quoted 32 lines · Avri Altman
Quoted 1 line · Avri Altman
HPB support is comprised of 4 main duties:
1) Read the device HPB configuration
2) Attend the device's recommendations that are embedded in the sense buffer
3) L2P cache management - This entails sending 2 new scsi commands (opcodes were taken from the vendor pool):
a. HPB-READ-BUFFER - read L2P physical addresses for a subregion
b. HPB-WRITE-BUFFER - notify the device that a region is inactive (in host-managed mode)
4) Use HPB-READ: a 3rd new scsi command (again - uses the vendor pool) to perform read operation instead of READ10. HPB-READ carries both the logical and the physical addresses.I will let Bean defend the Samsung approach of using a single LLD to attend all 4 duties.
Another approach might be to split those duties between 2 modules:
- A LLD that will perform the first 2 - those can be done only using ufs privet stuff, and
- another module in scsi mid-layer that will be responsible of L2P cache management, and HPB-READ command setup.
A framework to host the scsi mid-layer module can be the scsi device handler.The scsi-device-handler infrastructure was added to the kernel mainly to facilitate multiple paths for storage devices. The HPB framework, although far from the original intention of the authors, might as well fit in.
In that sense, using READs andHPB_READsintermittently, can be perceived as a multi-path.Scsi device handlers are also attached to a specific
scsi_device(lun).
This can serve as the glue linking between the ufs LLD and the device handler which resides in the scsi level.Device handlers comes with a rich and handy set of APIs & ops, which we can use to support HPB. Specifically we can use it to attach & activate the device handler,
only after the ufs driver verified that HPB is supported by both the platform and the device.The 2 modules can communicate using the
handler_dataopaque pointer, and the handler’sset_paramsop-mode: which is an open protocol essentially, and we can use it to pass the sense buffer in its full or just a parsed version.
Quoted 8 lines
Being a scsi mid-layer module, it will not break anything while sending HPB-READ-BUFFER and HPB-WRITE-BUFFER as part of the L2P cache management duties.
Last but not least, the device handler is already hooked in the scsi command setup flow -
scsi_setup_fs_cmnd(), So we get the hook into HPB-READprep_fnfor free.Later on, we might want to export the L2P cache management logic to user-space.
Locating the L2P cache management in scsi mid-layer will enable us to do so, using the scsi-netlink or some other means.
Hi Avri,
I'm not sure that I agree that HPB can be perceived as multi-path. Anyway, the above approach sounds interesting to me. A few questions though:
- The only in-tree caller of scsi_dh_attach() I am aware of exists in the dm-mpath driver. I think that call is triggered by multipathd. I don't think that it is acceptable to require that multipathd is running to use the UFS HPB functionality. What is the plan for attaching the UFS device handler to UFS devices?
- Will preparing a SCSI command involve executing a SCSI command? If so, how will it be prevented that execution of that internally submitted SCSI command triggers a deadlock due to tag exhaustion?
Thanks,
Bart.
RE: [PATCH v2 5/5] scsi: ufs: UFS Host Performance Booster(HPB) driver
Quoted 67 lines · Avri Altman
Quoted 4 lines
Quoted 1 line · Avri Altman
HPB support is comprised of 4 main duties:
1) Read the device HPB configuration
2) Attend the device's recommendations that are embedded in the sensebuffer
Quoted 1 line
3) L2P cache management - This entails sending 2 new scsi commands
(opcodes were taken from the vendor pool):
Quoted 2 lines
a. HPB-READ-BUFFER - read L2P physical addresses for a subregion b. HPB-WRITE-BUFFER - notify the device that a region is inactive (in
host-managed mode)
Quoted 1 line
4) Use HPB-READ: a 3rd new scsi command (again - uses the vendor pool)
to perform read operation instead of READ10. HPB-READ carries both the logical and the physical addresses.
Quoted 2 lines
I will let Bean defend the Samsung approach of using a single LLD to attend
all 4 duties.
Quoted 3 lines
Another approach might be to split those duties between 2 modules:
- A LLD that will perform the first 2 - those can be done only using ufsprivet stuff, and
Quoted 1 line
- another module in scsi mid-layer that will be responsible of L2P cache
management,
Quoted 2 lines
and HPB-READ command setup.
A framework to host the scsi mid-layer module can be the scsi devicehandler.
Quoted 2 lines
The scsi-device-handler infrastructure was added to the kernel mainly to
facilitate multiple paths for storage devices.
Quoted 1 line
The HPB framework, although far from the original intention of the
authors, might as well fit in.
Quoted 1 line
In that sense, using READs and
HPB_READsintermittently, can be perceivedas a multi-path.
Quoted 3 lines
Scsi device handlers are also attached to a specific
scsi_device(lun). This can serve as the glue linking between the ufs LLD and the devicehandler which resides in the scsi level.
Quoted 2 lines
Device handlers comes with a rich and handy set of APIs & ops, which we
can use to support HPB.
Quoted 2 lines
Specifically we can use it to attach & activate the device handler, only after the ufs driver verified that HPB is supported by both the platform
and the device.
Quoted 3 lines
The 2 modules can communicate using the
handler_dataopaque pointer, and the handler’sset_paramsop-mode: which is an open protocolessentially,
Quoted 4 lines
and we can use it to pass the sense buffer in its full or just a parsed version.
Being a scsi mid-layer module, it will not break anything while sending HPB-READ-BUFFER and HPB-WRITE-BUFFER as part of the L2P cache
management duties.
Quoted 2 lines
Last but not least, the device handler is already hooked in the scsi
command setup flow -
scsi_setup_fs_cmnd(),Quoted 3 lines
So we get the hook into HPB-READ
prep_fnfor free.Later on, we might want to export the L2P cache management logic to
user-space.
Quoted 1 line
Locating the L2P cache management in scsi mid-layer will enable us to do
so, using the scsi-netlink or some other means.
Hi Avri,
Quoted 8 lines
I'm not sure that I agree that HPB can be perceived as multi-path. Anyway, the above approach sounds interesting to me. A few questions though:
- The only in-tree caller ofscsi_dh_attach()I am aware of exists in the dm-mpath driver. I think that call is triggered by multipathd. I don't think that it is acceptable to require that multipathd is running to use the UFS HPB functionality. What is the plan for attaching the UFS device handler to UFS devices?
Right.
Device handlers are meant to be called as part of the device mapper multi-path code. We can’t do that – we need to attach & activate the device handler manually, only after the ufs driver verified that HPB is supported by both the platform and the device.
I was thinking to rely on the ufs's 2-phase boot:
The ufs boot process is essentially comprised of 2 parts: first a handshake with the device, and then, scsi scans and assign a scsi device to each lun. The latter, although running a-synchronically, is happening right after reading the device configuration - lun by lun.
By now we've read the device HPB configuration, and we are ready to attach a scsi device to our HPB luns. A perfect timing might be while scsi is performing its .slave_alloc() or .slave_configure().
It will require however, to include <scsi/scsi_dh.h> and "../scsi_priv.h" in ufshpb.c, Not sure if this is acceptable.
Quoted 3 lines
- Will preparing a SCSI command involve executing a SCSI command? If so, how will it be prevented that execution of that internally submitted SCSI command triggers a deadlock due to tag exhaustion?
Not sure what deadlock are you referring to. Executing HPB-READ-BUFFER is not part of the data path, And should not happen too often.
Anyway it can wait for a tag to become free. Same goes to HPB-WRITE-BUFFER.
Quoted 4 lines
Thanks,
Bart.
Re: [PATCH v2 5/5] scsi: ufs: UFS Host Performance Booster(HPB) driver
Quoted 150 lines · Avri Altman
Quoted 110 lines · Avri Altman
Quoted 6 lines
Quoted 1 line · Avri Altman
HPB support is comprised of 4 main duties:
1) Read the device HPB configuration
2) Attend the device's recommendations that are embedded in the sensebuffer
Quoted 2 lines
3) L2P cache management - This entails sending 2 new scsi commands
(opcodes were taken from the vendor pool):
Quoted 4 lines
a. HPB-READ-BUFFER - read L2P physical addresses for a
subregion
b. HPB-WRITE-BUFFER - notify the device that a region is
inactive (inhost-managed mode)
Quoted 2 lines
4) Use HPB-READ: a 3rd new scsi command (again - uses the vendor pool)
to perform read operation instead of READ10. HPB-READ carries both the
logical and the physical addresses.Quoted 3 lines
I will let Bean defend the Samsung approach of using a single LLD to attend
all 4 duties.
Quoted 5 lines
Another approach might be to split those duties between 2 modules:
- A LLD that will perform the first 2 - those can be done only
using ufsprivet stuff, and
Quoted 2 lines
- another module in scsi mid-layer that will be responsible of
L2P cachemanagement,
Quoted 3 lines
and HPB-READ command setup.
A framework to host the scsi mid-layer module can be the scsi devicehandler.
Quoted 3 lines
The scsi-device-handler infrastructure was added to the kernel mainly to
facilitate multiple paths for storage devices.
Quoted 2 lines
The HPB framework, although far from the original intention of the
authors, might as well fit in.
Quoted 2 lines
In that sense, using READs and
HPB_READsintermittently, can be perceivedas a multi-path.
Quoted 5 lines
Scsi device handlers are also attached to a specific
scsi_device(lun).
This can serve as the glue linking between the ufs LLD and the devicehandler which resides in the scsi level.
Quoted 3 lines
Device handlers comes with a rich and handy set of APIs & ops, which we
can use to support HPB.
Quoted 4 lines
Specifically we can use it to attach & activate the device handler,
only after the ufs driver verified that HPB is supported by both the platformand the device.
Quoted 4 lines
The 2 modules can communicate using the
handler_dataopaque pointer,
and the handler’sset_paramsop-mode: which is an open protocolessentially,
Quoted 6 lines
and we can use it to pass the sense buffer in its full or just a parsed version.
Being a scsi mid-layer module, it will not break anything while sending
HPB-READ-BUFFER and HPB-WRITE-BUFFER as part of the L2P cachemanagement duties.
Quoted 3 lines
Last but not least, the device handler is already hooked in the scsi
command setup flow -
scsi_setup_fs_cmnd(),Quoted 4 lines
So we get the hook into HPB-READ
prep_fnfor free.Later on, we might want to export the L2P cache management logic to
user-space.
Quoted 2 lines
Locating the L2P cache management in scsi mid-layer will enable us to do
so, using the scsi-netlink or some other means.
Hi Avri,
Quoted 11 lines
I'm not sure that I agree that HPB can be perceived as multi-path. Anyway, the above approach sounds interesting to me. A few questions
though:
- The only in-tree caller ofscsi_dh_attach()I am aware of exists in
the dm-mpath driver. I think that call is triggered by
multipathd.
I don't think that it is acceptable to require that multipathd is running to use the UFS HPB functionality. What is the plan for attaching the UFS device handler to UFS devices?Right.
Device handlers are meant to be called as part of the device mapper multi-path code. We can’t do that – we need to attach & activate the device handler manually, only after the ufs driver verified that HPB is
supported by both the platform and the device.I was thinking to rely on the ufs's 2-phase boot:
The ufs boot process is essentially comprised of 2 parts: first ahandshake with the device, and then, scsi scans and assign a scsi device
to each lun. The latter, although running a-synchronically, is
happening right after reading the device configuration - lun by lun.
Quoted 5 lines
By now we've read the device HPB configuration, and we are ready to attach a scsi device to our HPB luns. A perfect timing might be while
scsi is performing its .slave_alloc() or .slave_configure().
hi, Avri
That means HPB memory allocation done in .scan_finished() ? and sd_init_command() needs to change as well, add a new request type REQ_OP_HPB_READ?
Bean
RE: [PATCH v2 5/5] scsi: ufs: UFS Host Performance Booster(HPB) driver
Hi Bean,
Quoted 8 lines
Quoted 5 lines
By now we've read the device HPB configuration, and we are ready to attach a scsi device to our HPB luns. A perfect timing might be while
scsi is performing its .slave_alloc() or .slave_configure().hi, Avri
That means HPB memory allocation done in .scan_finished() ?
The specifics of this feature are yet to be determined.
Among those, yes - we need to discuss how to handle the memory allocation. Statically allocating the required cache for the entire max-active-subregions, Which may sum-up to a hundreds of MB, has its obvious downsize. We need to discuss this further.
Quoted 2 lines
and
sd_init_command()needs to change as well, add a new request typeREQ_OP_HPB_READ?
Again, this is an implementation issue.
We need to figure it out in the sequel.
E.g. we might want to make use of the combination of a valid handler and blk_op_is_private.
I think it would be more constructive, if we can decide first on the module layout, And figure out the other details as we go?
Can you provide the pros and cons for the Samsung approach - implementing all HPB functionalities using a single LLD?
Thanks,
Avri
Quoted 4 lines
Bean
Re: [PATCH v2 5/5] scsi: ufs: UFS Host Performance Booster(HPB) driver
Quoted 33 lines · Avri Altman
Hi Bean,
Quoted 9 lines
Quoted 6 lines
By now we've read the device HPB configuration, and we are ready to
attach a scsi device to our HPB luns. A perfect timing might be while
scsi is performing its .slave_alloc() or .slave_configure().hi, Avri
That means HPB memory allocation done in .scan_finished() ?The specifics of this feature are yet to be determined.
Among those, yes - we need to discuss how to handle the memory allocation.
Statically allocating the required cache for the entire max-active- subregions,
Which may sum-up to a hundreds of MB, has its obvious downsize. We need to discuss this further.Quoted 2 lines
and
sd_init_command()needs to change as well, add a new request typeREQ_OP_HPB_READ?Again, this is an implementation issue.
We need to figure it out in the sequel.
E.g. we might want to make use of the combination of a valid handler andblk_op_is_private.I think it would be more constructive, if we can decide first on the module layout,
And figure out the other details as we go?
Quoted 6 lines
Can you provide the pros and cons for the Samsung approach - implementing all HPB functionalities using a single LLD?
Thanks,
Avri
Hi Avri
Samsung approach is a flat design and the HPB functions are embedded in the UFSHBA driver, looks ugly. Each LU has its own HPB cache, which statically allocated in HPB initialization stage. If one LU runs out of its HPB cache, it is impossbile to borrow HPB cache from its neighbour LU. Also, HPB requests are enqueued to the scsi_device request_queue and then fly back to SCSI layer. This unavoidably lengthens the latency of HPB entry update.
For the HPB host control mode, the predictability of HPB region activation of this design is lower, since HPB driver doesn't know which Region exactly should be activated in advance.
Regarding its pros, exactly I don't know, maybe it is relatively simple, work, and there are already customers who are using it now. also, we don't need to argue with SCSI layers and maintenance is easier.
To me, hierarchical design sounds good, and move the implementation of HPB manager module to SCSI layer is nice. but what is opinion of others? and which way they prefer. or they want us to continue current Samsung approach and solve its cons further.
thanks,
Bean
Quoted 6 lines
Quoted 4 lines
Bean
RE: [PATCH v2 5/5] scsi: ufs: UFS Host Performance Booster(HPB) driver
Hi Bean,
Quoted 4 lines
To me, hierarchical design sounds good, and move the implementation of HPB manager module to SCSI layer is nice. but what is opinion of others? and which way they prefer. or they want us to continue current Samsung approach and solve its cons further.
I can put an RFC together in few days.
My plan is to borrow as much as possible from Samsung driver,
But to move the L2P cache management and HPB-READ command setup to the scsi mid-layer. Also maybe elaborate the host-managed mode logic to some extent.
This will not be a full-fledged driver, but more of a skeleton -
The gory details can come after, once we'll agree on the general concept.
Let's wait few days more to allow some more people to comment. Ideally, this should be a joint effort of the ufs community, Given the size of this driver, and the industry's interest and focus.
Thanks,
Avri
Re: [PATCH v2 5/5] scsi: ufs: UFS Host Performance Booster(HPB) driver
In reply to Bart Van AsscheQuoted 69 lines · Bart Van Assche
Quoted 58 lines · Avri Altman
Quoted 1 line · Avri Altman
HPB support is comprised of 4 main duties:
1) Read the device HPB configuration
2) Attend the device's recommendations that are embedded in the sense buffer
3) L2P cache management - This entails sending 2 new scsi commands (opcodes were taken from the vendor pool):
a. HPB-READ-BUFFER - read L2P physical addresses for a
subregion
b. HPB-WRITE-BUFFER - notify the device that a region is
inactive (in host-managed mode)
4) Use HPB-READ: a 3rd new scsi command (again - uses the vendor pool) to perform read operation instead of READ10. HPB-READ carries both the logical and the physical addresses.I will let Bean defend the Samsung approach of using a single LLD to attend all 4 duties.
Another approach might be to split those duties between 2 modules:
- A LLD that will perform the first 2 - those can be done only
using ufs privet stuff, and
- another module in scsi mid-layer that will be responsible of L2P
cache management,
and HPB-READ command setup.
A framework to host the scsi mid-layer module can be the scsi device handler.The scsi-device-handler infrastructure was added to the kernel mainly to facilitate multiple paths for storage devices. The HPB framework, although far from the original intention of the authors, might as well fit in.
In that sense, using READs andHPB_READsintermittently, can be perceived as a multi-path.Scsi device handlers are also attached to a specific
scsi_device(lun).
This can serve as the glue linking between the ufs LLD and the device handler which resides in the scsi level.Device handlers comes with a rich and handy set of APIs & ops, which we can use to support HPB.
Specifically we can use it to attach & activate the device handler, only after the ufs driver verified that HPB is supported by both the platform and the device.The 2 modules can communicate using the
handler_dataopaque pointer,
and the handler’sset_paramsop-mode: which is an open protocol essentially,
and we can use it to pass the sense buffer in its full or just a parsed version.Being a scsi mid-layer module, it will not break anything while sending
HPB-READ-BUFFER and HPB-WRITE-BUFFER as part of the L2P cache management duties.Quoted 8 lines
Last but not least, the device handler is already hooked in the scsi command setup flow -
scsi_setup_fs_cmnd(),
So we get the hook into HPB-READprep_fnfor free.Later on, we might want to export the L2P cache management logic to user-space.
Locating the L2P cache management in scsi mid-layer will enable us to do so, using the scsi-netlink or some other means.
Quoted 8 lines
- Will preparing a SCSI command involve executing a SCSI command? If so,
how will it be prevented that execution of that internally
submitted
SCSI command triggers a deadlock due to tag exhaustion?Thanks,
No, as for the HPB READ, it will replace SCSI READ in SCSI request executing path in case L2P entry hit in HPB cache. so it still uses the previously assigned tag.
As for the HPB BUFFER READ and HPB BUFFER WRITE, should beg for a new tag from hba->cmd_queue.
Bean
Re: [PATCH v2 5/5] scsi: ufs: UFS Host Performance Booster(HPB) driver
In reply to [email protected]Quoted 38 lines · On 2020-04-16 13:31, [email protected] wrote:
drivers/scsi/ufs/ufshpb.c new file +38
-0+static int ufshpb_execute_cmd (struct ufshpb_lu *hpb,unsigned char *cmd) +{ +struct scsi_sense_hdr sshdr; +struct scsi_device *sdp; +struct ufs_hba *hba; +int retries; +int ret =0 ; + + hba = hpb->hba; + + sdp = hba->sdev_ufs_lu[hpb->lun]; +if (!sdp) { +hpb_warn ("%s UFSHPB cannot find scsi_device\n" , __func__); +return -ENODEV ; + } + + ret =scsi_device_get (sdp); +if (!ret && !scsi_device_online (sdp)) { + ret = -ENODEV ; +scsi_device_put (sdp); +return ret; + } + +for (retries =3 ; retries >0 ; --retries) { + ret =scsi_execute (sdp, cmd,DMA_NONE ,NULL ,0 ,NULL , &sshdr, +msecs_to_jiffies (30000 ),3 ,0 ,RQF_PM ,NULL ); +if (ret ==0 ) +break ; + } + +if (ret) { +if (driver_byte (ret) ==DRIVER_SENSE && +scsi_sense_valid (&sshdr)) { +switch (sshdr.sense_key) { +case ILLEGAL_REQUEST : +hpb_err ("ILLEGAL REQUEST asc=0x%x ascq=0x%x\n" , + sshdr.asc, sshdr.ascq); +break ;
drivers/scsi/ufs/ufshpb.c line 1098
Quoted 10 lines
drivers/scsi/ufs/ufshpb.c new file +10
-0+default : +hpb_err ("Unknown return code = %x\n" , ret); +break ; + } + } + } +scsi_device_put (sdp); + +return ret; +}
If scsi_execute() would be changed into asynchronous SCSI command submission, can ufshpb_execute_cmd() be called from inside the UFS .queue_rq() callback instead of from workqueue context?
The scsi_device_get() call looks misplaced. I think that call should happen before schedule_work() is called.
Quoted 49 lines
drivers/scsi/ufs/ufshpb.c new file +49
-0+static int ufshpb_l2p_load_req (struct ufshpb_lu *hpb,struct request_queue *q, +struct ufshpb_map_req *map_req) +{ +struct scsi_request *scsi_rq; +unsigned char cmd[16 ] = { }; +struct request *req; +struct bio *bio; +int alloc_len; +int ret; + + bio = &map_req->bio; + + ret =ufshpb_add_bio_page (hpb, q, bio, map_req->bvec, map_req->mctx); +if (ret) { +hpb_err ("ufshpb_add_bio_page() failed with ret %d\n" , ret); +return ret; + } + + alloc_len = hpb->hba->hpb_geo.subregion_entry_sz; +/* +* HPB Sub-Regions are equally sized except for the last one which is +* smaller if the last hpb Region is not an integer multiple of +* bHPBSubRegionSize. +*/ +if (map_req->region == (hpb->lu_region_cnt -1 ) && + map_req->subregion == (hpb->subregions_in_last_region -1 )) + alloc_len = hpb->last_subregion_entry_size; + +ufshpb_prepare_read_buf_cmd (cmd, map_req->region, map_req->subregion, + alloc_len); +if (!map_req->req) { + map_req->req =blk_get_request (q,REQ_OP_SCSI_IN ,0 ); +if (IS_ERR (map_req->req)) +return PTR_ERR (map_req->req); + } + req = map_req->req; + scsi_rq =scsi_req (req); + +blk_rq_append_bio (req, &bio); + +scsi_req_init (scsi_rq); + + scsi_rq->cmd_len =COMMAND_SIZE (cmd[0 ]); +memcpy (scsi_rq->cmd, cmd, scsi_rq->cmd_len); + req->timeout =msecs_to_jiffies (30000 ); + req->end_io_data = (void *)map_req; + +hpb_dbg (SCHEDULE_INFO , hpb,"ISSUE READ_BUFFER : (%d-%d) retry = %d\n" , + map_req->region, map_req->subregion, map_req->retry_cnt);
drivers/scsi/ufs/ufshpb.c line 1208
Quoted 10 lines
drivers/scsi/ufs/ufshpb.c new file +10
-0+hpb_trace (hpb,"%s: I RB %d - %d" ,DRIVER_NAME , map_req->region, + map_req->subregion); + +blk_execute_rq_nowait (q,NULL , req,1 , ufshpb_l2p_load_req_done); + map_req->req_issue_t =ktime_to_ns (ktime_get ()); + +atomic64_inc (&hpb->status.map_req_cnt); + +return 0 ; +}
Same question here: if 'req' would be submitted asynchronously, can ufshpb_l2p_load_req() be called directly from inside the UFS .queue_rq() callback instead of from workqueue context?
Thanks,
Bart.
Re: [PATCH v2 5/5] scsi: ufs: UFS Host Performance Booster(HPB) driver
Quoted 119 lines · Bart Van Assche
Quoted 38 lines · On 2020-04-16 13:31, [email protected] wrote:
drivers/scsi/ufs/ufshpb.c new file +38
-0+static int ufshpb_execute_cmd (struct ufshpb_lu *hpb,unsigned char *cmd) +{ +struct scsi_sense_hdr sshdr; +struct scsi_device *sdp; +struct ufs_hba *hba; +int retries; +int ret =0 ; + + hba = hpb->hba; + + sdp = hba->sdev_ufs_lu[hpb->lun]; +if (!sdp) { +hpb_warn ("%s UFSHPB cannot find scsi_device\n" , __func__); +return -ENODEV ; + } + + ret =scsi_device_get (sdp); +if (!ret && !scsi_device_online (sdp)) { + ret = -ENODEV ; +scsi_device_put (sdp); +return ret; + } + +for (retries =3 ; retries >0 ; --retries) { + ret =scsi_execute (sdp, cmd,DMA_NONE ,NULL ,0 ,NULL , &sshdr, +msecs_to_jiffies (30000 ),3 ,0 ,RQF_PM ,NULL ); +if (ret ==0 ) +break ; + } + +if (ret) { +if (driver_byte (ret) ==DRIVER_SENSE && +scsi_sense_valid (&sshdr)) { +switch (sshdr.sense_key) { +case ILLEGAL_REQUEST : +hpb_err ("ILLEGAL REQUEST asc=0x%x ascq=0x%x\n" , + sshdr.asc, sshdr.ascq); +break ;drivers/scsi/ufs/ufshpb.c line 1098
Quoted 10 lines
drivers/scsi/ufs/ufshpb.c new file +10
-0+default : +hpb_err ("Unknown return code = %x\n" , ret); +break ; + } + } + } +scsi_device_put (sdp); + +return ret; +}If
scsi_execute()would be changed into asynchronous SCSI command submission, canufshpb_execute_cmd()be called from inside the UFS .queue_rq() callback instead of from workqueue context?The
scsi_device_get()call looks misplaced. I think that call should happen beforeschedule_work()is called.Quoted 49 lines
drivers/scsi/ufs/ufshpb.c new file +49
-0+static int ufshpb_l2p_load_req (struct ufshpb_lu *hpb,struct request_queue *q, +struct ufshpb_map_req *map_req) +{ +struct scsi_request *scsi_rq; +unsigned char cmd[16 ] = { }; +struct request *req; +struct bio *bio; +int alloc_len; +int ret; + + bio = &map_req->bio; + + ret =ufshpb_add_bio_page (hpb, q, bio, map_req->bvec, map_req->mctx); +if (ret) { +hpb_err ("ufshpb_add_bio_page() failed with ret %d\n" , ret); +return ret; + } + + alloc_len = hpb->hba->hpb_geo.subregion_entry_sz; +/* +* HPB Sub-Regions are equally sized except for the last one which is +* smaller if the last hpb Region is not an integer multiple of +* bHPBSubRegionSize. +*/ +if (map_req->region == (hpb->lu_region_cnt -1 ) && + map_req->subregion == (hpb->subregions_in_last_region -1 )) + alloc_len = hpb->last_subregion_entry_size; + +ufshpb_prepare_read_buf_cmd (cmd, map_req->region, map_req->subregion, + alloc_len); +if (!map_req->req) { + map_req->req =blk_get_request (q,REQ_OP_SCSI_IN ,0 ); +if (IS_ERR (map_req->req)) +return PTR_ERR (map_req->req); + } + req = map_req->req; + scsi_rq =scsi_req (req); + +blk_rq_append_bio (req, &bio); + +scsi_req_init (scsi_rq); + + scsi_rq->cmd_len =COMMAND_SIZE (cmd[0 ]); +memcpy (scsi_rq->cmd, cmd, scsi_rq->cmd_len); + req->timeout =msecs_to_jiffies (30000 ); + req->end_io_data = (void *)map_req; + +hpb_dbg (SCHEDULE_INFO , hpb,"ISSUE READ_BUFFER : (%d-%d) retry = %d\n" , + map_req->region, map_req->subregion, map_req->retry_cnt);drivers/scsi/ufs/ufshpb.c line 1208
Quoted 10 lines
drivers/scsi/ufs/ufshpb.c new file +10
-0+hpb_trace (hpb,"%s: I RB %d - %d" ,DRIVER_NAME , map_req->region, + map_req->subregion); + +blk_execute_rq_nowait (q,NULL , req,1 , ufshpb_l2p_load_req_done); + map_req->req_issue_t =ktime_to_ns (ktime_get ()); + +atomic64_inc (&hpb->status.map_req_cnt); + +return 0 ; +}Same question here: if 'req' would be submitted asynchronously, can
ufshpb_l2p_load_req()be called directly from inside the UFS .queue_rq() callback instead of from workqueue context?
Quoted 3 lines
Thanks,
Bart.
Hi, Bart
Thanks very much.
If I understood the Christoph's the second question correctly. Enqueue HPB requests to the scsi_device->request_queueu, and fly back to SCSI schedule, it's unacceptable. I don't like this implementation way either. Also, the latency of the L2P table update is higher. I will change it in the RFC patch.
thanks,
Bean
Quoted 16 lines · On Thu, Apr 16, 2020 at 10:31:21PM +0200, [email protected] wrote:
From: Bean Huo <[email protected]>
Hi,
I disclose my development/changes on UFS HPB (UFS Host Performance Booster) to the community, as this would enable more UFS developers to test and start an iterative review and update process.The HPB is defined in Jedec Standard Universal Flash (UFS) Host Performance Booster (HPB) Extension Version 1.0, which is designed to improve read performance by utilizing the host side memory. Based on our testing, the HPB can increase the random read performance by up to about 46% in the random read.
The original HPB driver is from [1]. Based on it, I did optimizations, simplifications, fixed reliability issues, implemented HPB host control mode and make it much more readable. which's pushed to here [2] as V1.
Quoted 6 lines
To avoid touching the traditional SCSI core, the HPB driver in this series HPB patch chooses to develop under SCSI sub-system layer, and sits the same layer with UFSHCD. At the same time, to minimize changes to UFSHCD driver, the HPB driver submits its HPB READ BUFFER and HPB WRITE BUFFER requests to the scsi
device->request_queueuto execute, rather than that directly go through raw UPIU request path.
This feature is completley broken, and rather dangerous due to feeding "physical" addresses looked up by the host in. I do not think we should support something that broken in Linux.
Independent of that using two requests in the I/O path is not going to fly either. The whole thing seems like an exercise in benchmarketing.
Hi, Christoph
Thanks for your feedback.
Quoted 9 lines
Quoted 8 lines
To avoid touching the traditional SCSI core, the HPB driver in this series HPB patch chooses to develop under SCSI sub-system layer, and sits the same layer with UFSHCD. At the same time, to minimize changes to UFSHCD driver, the HPB driver submits its HPB READ BUFFER and HPB WRITE BUFFER requests to the scsi
device->request_queueuto execute, rather than that directly godevice->through
raw UPIU request path.
Quoted 4 lines
This feature is completley broken, and rather dangerous due to feeding "physical" addresses looked up by the host in. I do not think we should support something that broken in Linux.
It Is not plain physical address, has been encrypted before loading from UFS to HPB memory, I think we don't worry about its safety.
Quoted 2 lines
Independent of that using two requests in the I/O path is not going to fly either. The whole thing seems like an exercise in benchmarketing.
I agree with you. This is my major concern. I have been thinking about HPB implementation in SCSI layer. That will let SCSI layer manage HPB by calling UFS helper interface.
If you don't consider UFS HPB is an idiot design, I want to change in another version. Firstly, we really want to hear your suggestion.
Thanks,
//Bean
Quoted 36 lines
Hi, Christoph
Thanks for your feedback.Quoted 11 lines
Quoted 8 lines
To avoid touching the traditional SCSI core, the HPB driver in this series HPB patch chooses to develop under SCSI sub-system layer, and sits the same layer with UFSHCD. At the same time, to minimize changes to UFSHCD driver, the HPB driver submits its HPB READ BUFFER and HPB WRITE BUFFER requests to the scsi
device->request_queueuto execute, rather than that directly godevice->through
raw UPIU request path.This feature is completley broken, and rather dangerous due to feeding "physical" addresses looked up by the host in. I do not think we should
support
Quoted 2 lines
something that broken in Linux.
It Is not plain physical address, has been encrypted before loading from UFS to
HPB memory, I think we don't worry about its safety.Quoted 1 line
Independent of that using two requests in the I/O path is not going to fly
either.
Quoted 1 line
The whole thing seems like an exercise in benchmarketing.
I agree with you. This is my major concern. I have been thinking about HPB implementation in SCSI layer.
That will let SCSI layer manage HPB by calling UFS helper interface. If you don't consider UFS HPB is an idiot design, I want to change in another version. Firstly, we really
want to hear your suggestion.
Thanks,
Quoted 1 line
//Bean
If indeed this will move forward, please publish your patches as a RFC, To allow competing approaches to be published as well.
Thanks,
Avri