scsi: ufs: Add Host Performance Booster Support
A message that is not in the archive (sent off-list or not yet received): CGME20200828070950epcms2p5470bd43374be18d184dd802da09e73c8@epcms2p5
v9 cover scsi: ufs: Add Host Performance Booster Support
Changelog:
v8 -> v9
1. Change sysfs initialization.
2. Change reading descriptor during HPB initialization
3. Fix problems commentted in Bart's review.
4. Change base commit from 5.9/scsi-queue to 5.10/scsi-queue.
v7 -> v8
Remove wrongly added tags.
v6 -> v7
1. Remove UFS feature layer.
2. Cleanup for sparse error.
v5 -> v6
Change base commit to b53293fa662e28ae0cdd40828dc641c09f133405
v4 -> v5
Delete unused macro define.
v3 -> v4
1. Cleanup.
v2 -> v3
1. Add checking input module parameter value.
2. Change base commit from 5.8/scsi-queue to 5.9/scsi-queue.
3. Cleanup for unused variables and label.
v1 -> v2
1. Change the full boilerplate text to SPDX style.
2. Adopt dynamic allocation for sub-region data structure.
3. Cleanup.
NAND flash memory-based storage devices use Flash Translation Layer (FTL) to translate logical addresses of I/O requests to corresponding flash memory addresses. Mobile storage devices typically have RAM with constrained size, thus lack in memory to keep the whole mapping table. Therefore, mapping tables are partially retrieved from NAND flash on demand, causing random-read performance degradation.
To improve random read performance, JESD220-3 (HPB v1.0) proposes HPB (Host Performance Booster) which uses host system memory as a cache for the FTL mapping table. By using HPB, FTL data can be read from host memory faster than from NAND flash memory.
The current version only supports the DCM (device control mode). This patch consists of 3 parts to support HPB feature.
1) HPB probe and initialization process
2) READ -> HPB READ using cached map information
3) L2P (logical to physical) map management
In the HPB probe and init process, the device information of the UFS is queried. After checking supported features, the data structure for the HPB is initialized according to the device information.
A read I/O in the active sub-region where the map is cached is changed to HPB READ by the HPB.
The HPB manages the L2P map using information received from the device. For active sub-region, the HPB caches through ufshpb_map request. For the in-active region, the HPB discards the L2P map. When a write I/O occurs in an active sub-region area, associated dirty bitmap checked as dirty for preventing stale read.
HPB is shown to have a performance improvement of 58 - 67% for random read workload. [1]
This series patches are based on the 5.9/scsi-queue branch.
[1]:
https://www.usenix.org/conference/hotstorage17/program/presentation/jeong
Daejun park (4):
scsi: ufs: Add HPB feature related parameters
scsi: ufs: Introduce HPB feature
scsi: ufs: L2P map management for HPB read
scsi: ufs: Prepare HPB read for cached sub-region
drivers/scsi/ufs/Kconfig | 9 + drivers/scsi/ufs/Makefile | 1 + drivers/scsi/ufs/ufs.h | 47 ++ drivers/scsi/ufs/ufshcd.c | 58 ++ drivers/scsi/ufs/ufshcd.h | 22 +- drivers/scsi/ufs/ufshpb.c | 1817 ++++++++++++++++++++++++++++++++++++++++ drivers/scsi/ufs/ufshpb.h | 231 +++++ 7 files changed, 2184 insertions(+), 1 deletion(-)
created mode 100644 drivers/scsi/ufs/ufshpb.c
created mode 100644 drivers/scsi/ufs/ufshpb.h
PATCH v9 1/4 scsi: ufs: Add HPB feature related parameters
This is a patch for parameters to be used for HPB feature.
--- drivers/scsi/ufs/ufs.h | 13 +++++++++++++ 1 file changed, 13 insertions(+)
drivers/scsi/ufs/ufs.h +13 -0
@@ -122,6 +122,7 @@ enum flag_idn {QUERY_FLAG_IDN_WB_EN =0x0E ,QUERY_FLAG_IDN_WB_BUFF_FLUSH_EN =0x0F ,QUERY_FLAG_IDN_WB_BUFF_FLUSH_DURING_HIBERN8 =0x10 , + QUERY_FLAG_IDN_HPB_RESET = 0x11, };/* Attribute idn for Query requests */
@@ -195,6 +196,9 @@ enum unit_desc_param {UNIT_DESC_PARAM_PHY_MEM_RSRC_CNT =0x18 ,UNIT_DESC_PARAM_CTX_CAPABILITIES =0x20 ,UNIT_DESC_PARAM_LARGE_UNIT_SIZE_M1 =0x22 , +UNIT_DESC_HPB_LU_MAX_ACTIVE_REGIONS =0x23 , +UNIT_DESC_HPB_LU_PIN_REGION_START_OFFSET =0x25 , +UNIT_DESC_HPB_LU_NUM_PIN_REGIONS =0x27 , UNIT_DESC_PARAM_WB_BUF_ALLOC_UNITS = 0x29, };
@@ -235,6 +239,8 @@ enum device_desc_param {DEVICE_DESC_PARAM_PSA_MAX_DATA =0x25 ,DEVICE_DESC_PARAM_PSA_TMT =0x29 ,DEVICE_DESC_PARAM_PRDCT_REV =0x2A , +DEVICE_DESC_PARAM_HPB_VER =0x40 , +DEVICE_DESC_PARAM_HPB_CONTROL =0x42 ,DEVICE_DESC_PARAM_EXT_UFS_FEATURE_SUP =0x4F ,DEVICE_DESC_PARAM_WB_PRESRV_USRSPC_EN =0x53 , DEVICE_DESC_PARAM_WB_TYPE = 0x54,
@@ -283,6 +289,10 @@ enum geometry_desc_param {GEOMETRY_DESC_PARAM_ENM4_MAX_NUM_UNITS =0x3E ,GEOMETRY_DESC_PARAM_ENM4_CAP_ADJ_FCTR =0x42 ,GEOMETRY_DESC_PARAM_OPT_LOG_BLK_SIZE =0x44 , +GEOMETRY_DESC_PARAM_HPB_REGION_SIZE =0x48 , +GEOMETRY_DESC_PARAM_HPB_NUMBER_LU =0x49 , +GEOMETRY_DESC_PARAM_HPB_SUBREGION_SIZE =0x4A , +GEOMETRY_DESC_PARAM_HPB_MAX_ACTIVE_REGS =0x4B ,GEOMETRY_DESC_PARAM_WB_MAX_ALLOC_UNITS =0x4F ,GEOMETRY_DESC_PARAM_WB_MAX_WB_LUNS =0x53 , GEOMETRY_DESC_PARAM_WB_BUFF_CAP_ADJ = 0x54,
@@ -327,8 +337,10 @@ enum {/* Possible values for dExtendedUFSFeaturesSupport */ enum { +UFS_DEV_HPB_SUPPORT =BIT (7 ),UFS_DEV_WRITE_BOOSTER_SUP =BIT (8 ), }; +#define UFS_DEV_HPB_SUPPORT_VERSION 0x310#define POWER_DESC_MAX_SIZE 0x62#define POWER_DESC_MAX_ACTV_ICC_LVLS 16
@@ -537,6 +549,7 @@ struct ufs_dev_info {u8 *model;u16 wspecversion;u32 clk_gating_wait_us; +u8 b_ufs_feature_sup;u32 d_ext_ufs_feature_sup;u8 b_wb_buffer_type;u32 d_wb_alloc_units;
Signature · 1 line
2.17.1
Re: [PATCH v9 1/4] scsi: ufs: Add HPB feature related parameters
Quoted 1 line · Daejun Park
This is a patch for parameters to be used for HPB feature.
PATCH v9 2/4 scsi: ufs: Introduce HPB feature
In reply to Daejun ParkThis is a patch for the HPB feature.
This patch adds HPB function calls to UFS core driver.
The mininum size of the memory pool used in the HPB is implemented as a module parameter, so that it can be configurable by the user.
To gurantee a minimum memory pool size of 4MB: ufshpb_host_map_kbytes=4096
--- drivers/scsi/ufs/Kconfig | 9 + drivers/scsi/ufs/Makefile | 1 + drivers/scsi/ufs/ufshcd.c | 58 ++++ drivers/scsi/ufs/ufshcd.h | 22 +- drivers/scsi/ufs/ufshpb.c | 596 ++++++++++++++++++++++++++++++++++++++ drivers/scsi/ufs/ufshpb.h | 176 +++++++++++ 6 files changed, 861 insertions(+), 1 deletion(-) create mode 100644 drivers/scsi/ufs/ufshpb.c create mode 100644 drivers/scsi/ufs/ufshpb.h
drivers/scsi/ufs/Kconfig +9 -0
@@ -182,3 +182,12 @@ config SCSI_UFS_CRYPTO Enabling this makes it possible for the kernel to use the crypto capabilities of the UFS device (if present) to perform crypto operations on data being transferred to/from the device. + +config SCSI_UFS_HPB + bool "Support UFS Host Performance Booster" + depends on SCSI_UFSHCD + help + The UFS HPB feature improves random read performance. It caches + L2P (logical to physical) map of UFS to host DRAM. The driver uses HPB + read command by piggybacking physical page number for bypassing FTL (flash + translation layer)'s L2P address translation.
drivers/scsi/ufs/Makefile +1 -0
@@ -11,6 +11,7 @@ obj-$(CONFIG_SCSI_UFSHCD) += ufshcd-core.o ufshcd-core-y += ufshcd.o ufs-sysfs.o ufshcd-core-$(CONFIG_SCSI_UFS_BSG) += ufs_bsg.o ufshcd-core-$(CONFIG_SCSI_UFS_CRYPTO) += ufshcd-crypto.o +ufshcd-core-$(CONFIG_SCSI_UFS_HPB) += ufshpb.o obj-$(CONFIG_SCSI_UFSHCD_PCI) += ufshcd-pci.o obj-$(CONFIG_SCSI_UFSHCD_PLATFORM) += ufshcd-pltfrm.o obj-$(CONFIG_SCSI_UFS_HISI) += ufs-hisi.o
drivers/scsi/ufs/ufshcd.c +58 -0
157 lines not on this page · raw message
drivers/scsi/ufs/ufshcd.h +21 -1
@@ -588,6 +588,24 @@ struct ufs_hba_variant_params {u16 hba_enable_delay_us;u32 wb_flush_threshold; }; +#ifdef CONFIG_SCSI_UFS_HPB +/** +* struct ufshpb_dev_info - UFSHPB device related info +* @num_lu: the number of user logical unit to check whether all lu finished +* initialization +* @rgn_size: device reported HPB region size +* @srgn_size: device reported HPB sub-region size +* @slave_conf_cnt: counter to check all lu finished initialization +* @hpb_disabled: flag to check if HPB is disabled +*/ +struct ufshpb_dev_info { +int num_lu; +int rgn_size; +int srgn_size; +atomic_t slave_conf_cnt; +bool hpb_disabled; +}; +#endif /** * struct ufs_hba - per adapter private structure
@@ -770,7 +788,9 @@ struct ufs_hba {bool wb_buf_flush_enabled;bool wb_enabled;struct delayed_work rpm_dev_flush_recheck_work; - +#ifdef CONFIG_SCSI_UFS_HPB +struct ufshpb_dev_info ufshpb_dev; +#endif #ifdef CONFIG_SCSI_UFS_CRYPTO union ufs_crypto_capabilities crypto_capabilities;union ufs_crypto_cap_entry *crypto_cap_array;
drivers/scsi/ufs/ufshpb.c new file +596 -0
596 lines not on this page · raw message
drivers/scsi/ufs/ufshpb.h new file +176 -0
176 lines not on this page · raw message
Signature · 1 line
2.17.1
kernel test robot
build botRe: [PATCH v9 2/4] scsi: ufs: Introduce HPB feature
Thank you for the patch! Yet something to improve:
Hi Daejun,
Thank you for the patch! Yet something to improve:
[auto build test ERROR on mkp-scsi/for-next]
[also build test ERROR on scsi/for-next v5.9-rc2 next-20200828] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/Daejun-Park/scsi-ufs-Add-Host-Performance-Booster-Support/20200828-155548 base: https://git.kernel.org/pub/scm/linux/kernel/git/mkp/scsi.git for-next config: m68k-allyesconfig (attached as .config) compiler: m68k-linux-gcc (GCC) 9.3.0
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-9.3.0 make.cross ARCH=m68k
If you fix the issue, kindly add following tag as appropriate
All errors (new ones prefixed by >>):
m68k-linux-ld: drivers/scsi/ufs/ufshpb.o: in function `ufshpb_init_lu_parameter.isra.0':
>> ufshpb.c:(.text.unlikely+0xce): undefined reference to `__udivdi3' >> m68k-linux-ld: ufshpb.c:(.text.unlikely+0x10c): undefined reference to `__udivdi3'
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org
kernel test robot
build botRe: [PATCH v9 2/4] scsi: ufs: Introduce HPB feature
In reply to Daejun ParkThank you for the patch! Yet something to improve:
Hi Daejun,
Thank you for the patch! Yet something to improve:
[auto build test ERROR on mkp-scsi/for-next]
[also build test ERROR on scsi/for-next v5.9-rc2 next-20200828] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/Daejun-Park/scsi-ufs-Add-Host-Performance-Booster-Support/20200828-155548 base: https://git.kernel.org/pub/scm/linux/kernel/git/mkp/scsi.git for-next config: mips-allyesconfig (attached as .config) compiler: mips-linux-gcc (GCC) 9.3.0
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-9.3.0 make.cross ARCH=mips
If you fix the issue, kindly add following tag as appropriate
All errors (new ones prefixed by >>):
arch/mips/kernel/head.o: in function `dtb_found':
(.ref.text+0xe0): relocation truncated to fit: R_MIPS_26 against start_kernel' init/main.o: in function set_reset_devices':
main.c:(.init.text+0x20): relocation truncated to fit: R_MIPS_26 against _mcount' main.c:(.init.text+0x30): relocation truncated to fit: R_MIPS_26 against __sanitizer_cov_trace_pc' init/main.o: in function `debug_kernel':
main.c:(.init.text+0x9c): relocation truncated to fit: R_MIPS_26 against _mcount' main.c:(.init.text+0xac): relocation truncated to fit: R_MIPS_26 against __sanitizer_cov_trace_pc' init/main.o: in function `quiet_kernel':
main.c:(.init.text+0x118): relocation truncated to fit: R_MIPS_26 against _mcount' main.c:(.init.text+0x128): relocation truncated to fit: R_MIPS_26 against __sanitizer_cov_trace_pc' init/main.o: in function `init_setup':
main.c:(.init.text+0x1a4): relocation truncated to fit: R_MIPS_26 against _mcount' main.c:(.init.text+0x1c8): relocation truncated to fit: R_MIPS_26 against __sanitizer_cov_trace_pc' main.c:(.init.text+0x1e8): relocation truncated to fit: R_MIPS_26 against `__sanitizer_cov_trace_pc' main.c:(.init.text+0x1fc): additional relocation overflows omitted from the output
mips-linux-ld: drivers/scsi/ufs/ufshpb.o: in function `ufshpb_init_lu_parameter.isra.0':
>> ufshpb.c:(.text.unlikely.ufshpb_init_lu_parameter.isra.0+0x14c): undefined reference to `__udivdi3' >> mips-linux-ld: ufshpb.c:(.text.unlikely.ufshpb_init_lu_parameter.isra.0+0x180): undefined reference to `__udivdi3'
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org
Re: [PATCH v9 2/4] scsi: ufs: Introduce HPB feature
In reply to Daejun ParkQuoted 16 lines · Daejun Park
drivers/scsi/ufs/ufshcd.h +8
-0@@ -588,6 +588,24 @@ struct ufs_hba_variant_params {u16 hba_enable_delay_us;u32 wb_flush_threshold; }; +#ifdef CONFIG_SCSI_UFS_HPB +/** +* struct ufshpb_dev_info - UFSHPB device related info +* @num_lu: the number of user logical unit to check whether all lu finished +* initialization +* @rgn_size: device reported HPB region size +* @srgn_size: device reported HPB sub-region size +* @slave_conf_cnt: counter to check all lu finished initialization
drivers/scsi/ufs/ufshcd.h struct ufs_hba_variant_params { · line 608
Quoted 10 lines
drivers/scsi/ufs/ufshcd.h +10
-0+* @hpb_disabled: flag to check if HPB is disabled +*/ +struct ufshpb_dev_info { +int num_lu; +int rgn_size; +int srgn_size; +atomic_t slave_conf_cnt; +bool hpb_disabled; +}; +#endif
Please insert a blank line above "#ifdef CONFIG_SCSI_UFS_HPB"
drivers/scsi/ufs/ufshcd.h struct ufs_hba_variant_params { · line 610
Quoted 2 lines
drivers/scsi/ufs/ufshcd.h +2
-0+/* HPB enabled lu list */ +static LIST_HEAD (lh_hpb_lu);
Is it necessary to maintain this list? Or in other words, is it possible to change all list_for_each_entry(hpb, &lh_hpb_lu, list_hpb_lu) calls into shost_for_each_device() calls?
drivers/scsi/ufs/ufshcd.h struct ufs_hba_variant_params {
Quoted 13 lines
drivers/scsi/ufs/ufshcd.h +13
-0+/* SYSFS functions */ +#define ufshpb_sysfs_attr_show_func (__name) \ +static ssize_t __name##_show(struct device *dev, \ + struct device_attribute *attr, char *buf) \ +{ \ + struct scsi_device *sdev = to_scsi_device(dev); \ + struct ufshpb_lu *hpb = sdev->hostdata; \ + if (!hpb) \ + return -ENOENT; \ + return snprintf(buf, PAGE_SIZE, "%d\n", \ + atomic_read(&hpb->stats.__name)); \ +} \ +static DEVICE_ATTR_RO(__name)
Please leave a blank line after declarations (between the "hpb" declaration and "if (!hpb)").
drivers/scsi/ufs/ufshcd.h
Quoted 8 lines
drivers/scsi/ufs/ufshcd.h +8
-0+#ifndef CONFIG_SCSI_UFS_HPB +[...] +static struct attribute *hpb_dev_attrs[] = {NULL ,}; +static struct attribute_group ufs_sysfs_hpb_stat_group = {.attrs = hpb_dev_attrs,}; +#else +[...] +extern struct attribute_group ufs_sysfs_hpb_stat_group; +#endif
Defining static variables or arrays in header files is questionable because the definition of these variables will be duplicated in each source file that header file is included in. Although the general rule for kernel code is to confine #ifdefs to header files, for ufs_sysfs_hpb_stat_group I think that it is better to surround its use with #ifndef CONFIG_SCSI_UFS_HPB / #endif instead of defining a dummy structure as static variable in a header file if HPB support is disabled.
Otherwise this patch looks good to me.
Thanks,
Bart.
Re: [PATCH v9 2/4] scsi: ufs: Introduce HPB feature
Hi, Bart
Quoted 20 lines
Quoted 20 lines
drivers/scsi/ufs/ufshcd.h +12
-0@@ -588,6 +588,24 @@ struct ufs_hba_variant_params {u16 hba_enable_delay_us;u32 wb_flush_threshold; }; +#ifdef CONFIG_SCSI_UFS_HPB +/** + * struct ufshpb_dev_info - UFSHPB device related info + * @num_lu: the number of user logical unit to check whether all lu finished + * initialization + * @rgn_size: device reported HPB region size + * @srgn_size: device reported HPB sub-region size + * @slave_conf_cnt: counter to check all lu finished initialization + * @hpb_disabled: flag to check if HPB is disabled + */ +struct ufshpb_dev_info { + int num_lu;
Quoted 8 lines
drivers/scsi/ufs/ufshcd.h struct ufs_hba_variant_params { · line 608
Quoted 6 lines
drivers/scsi/ufs/ufshcd.h +6
-0+int rgn_size; +int srgn_size; +atomic_t slave_conf_cnt; + bool hpb_disabled; +}; +#endif Please insert a blank line above "#ifdef
CONFIG_SCSI_UFS_HPB"
OK, I will insert a blank line.
Quoted 6 lines
drivers/scsi/ufs/ufshcd.h struct ufs_hba_variant_params { · line 610
Quoted 2 lines
drivers/scsi/ufs/ufshcd.h +2
-0+/* HPB enabled lu list */ +static LIST_HEAD (lh_hpb_lu);Is it necessary to maintain this list? Or in other words, is it possible to change all
list_for_each_entry(hpb, &lh_hpb_lu, list_hpb_lu)calls intoshost_for_each_device()calls?
OK, I will remove lh_hpb_lu.
Quoted 8 lines
Quoted 8 lines
drivers/scsi/ufs/ufshcd.h +8
-0+/* SYSFS functions */ +#define ufshpb_sysfs_attr_show_func (__name) \ +static ssize_t __name##_show(struct device *dev, \ + struct device_attribute *attr, char *buf) \ +{ \ + struct scsi_device *sdev = to_scsi_device(dev); \ + struct ufshpb_lu *hpb = sdev->hostdata; \ + if (!hpb) \
Quoted 8 lines
drivers/scsi/ufs/ufshcd.h
Quoted 5 lines
drivers/scsi/ufs/ufshcd.h +5
-0+return -ENOENT ; \ +return snprintf (buf,PAGE_SIZE ,"%d\n" , \ +atomic_read (&hpb->stats.__name)); \ +} \ +static DEVICE_ATTR_RO (__name)Please leave a blank line after declarations (between the "hpb" declaration and "if (!hpb)").
OK, I will add a blank line.
Quoted 9 lines
drivers/scsi/ufs/ufshcd.h
Quoted 8 lines
drivers/scsi/ufs/ufshcd.h +8
-0+#ifndef CONFIG_SCSI_UFS_HPB +[...] +static struct attribute *hpb_dev_attrs[] = {NULL ,}; +static struct attribute_group ufs_sysfs_hpb_stat_group = {.attrs = hpb_dev_attrs,}; +#else +[...] +extern struct attribute_group ufs_sysfs_hpb_stat_group; +#endif
Quoted 7 lines
Defining static variables or arrays in header files is questionable because the definition of these variables will be duplicated in each source file that header file is included in. Although the general rule for kernel code is to confine #ifdefs to header files, for
ufs_sysfs_hpb_stat_groupI think that it is better to surround its use with #ifndefCONFIG_SCSI_UFS_HPB/ #endif instead of defining a dummy structure as static variable in a header file if HPB support is disabled.
OK, I added #ifdef in ufshcd.c.
static const struct attribute_group *ufshcd_driver_groups[] = {
&ufs_sysfs_unit_descriptor_group, &ufs_sysfs_lun_attributes_group,
#ifdef CONFIG_SCSI_UFS_HPB
&ufs_sysfs_hpb_stat_group,
#endif
NULL,
};
Thanks,
Daejun
PATCH v9 3/4 scsi: ufs: L2P map management for HPB read
In reply to Daejun ParkThis is a patch for managing L2P map in HPB module.
The HPB divides logical addresses into several regions. A region consists of several sub-regions. The sub-region is a basic unit where L2P mapping is managed. The driver loads L2P mapping data of each sub-region. The loaded sub-region is called active-state. The HPB driver unloads L2P mapping data as region unit. The unloaded region is called inactive-state.
Sub-region/region candidates to be loaded and unloaded are delivered from the UFS device. The UFS device delivers the recommended active sub-region and inactivate region to the driver using sensedata.
The HPB module performs L2P mapping management on the host through the delivered information.
A pinned region is a pre-set regions on the UFS device that is always activate-state.
The data structure for map data request and L2P map uses mempool API, minimizing allocation overhead while avoiding static allocation.
The map_work manages active/inactive by 2 "to-do" lists. Each hpb lun maintains 2 "to-do" lists:
hpb->lh_inact_rgn - regions to be inactivated, and hpb->lh_act_srgn - subregions to be activated
Those lists are maintained on IO completion.
--- drivers/scsi/ufs/ufs.h | 34 ++ drivers/scsi/ufs/ufshpb.c | 996 +++++++++++++++++++++++++++++++++++++- drivers/scsi/ufs/ufshpb.h | 55 +++ 3 files changed, 1082 insertions(+), 3 deletions(-)
drivers/scsi/ufs/ufs.h +34 -0
@@ -472,6 +472,39 @@ struct utp_cmd_rsp {u8 sense_data[UFS_SENSE_SIZE ]; }; +struct ufshpb_active_field { +__be16 active_rgn; +__be16 active_srgn; +}; + +/** +* struct utp_hpb_rsp - Response UPIU structure +* @residual_transfer_count: Residual transfer count DW-3 +* @reserved1: Reserved double words DW-4 to DW-7 +* @sense_data_len: Sense data length DW-8 U16 +* @desc_type: Descriptor type of sense data +* @additional_len: Additional length of sense data +* @hpb_type: HPB operation type +* @reserved2: Reserved field +* @active_rgn_cnt: Active region count +* @inactive_rgn_cnt: Inactive region count +* @hpb_active_field: Recommended to read HPB region and subregion +* @hpb_inactive_field: To be inactivated HPB region and subregion +*/ +struct utp_hpb_rsp { +__be32 residual_transfer_count; +__be32 reserved1[4 ]; +__be16 sense_data_len; +u8 desc_type; +u8 additional_len; +u8 hpb_type; +u8 reserved2; +u8 active_rgn_cnt; +u8 inactive_rgn_cnt; +struct ufshpb_active_field hpb_active_field[2 ]; +__be16 hpb_inactive_field[2 ]; +}; +/** * struct utp_upiu_rsp - general upiu response structure * @header: UPIU header structure DW-0 to DW-2
@@ -482,6 +515,7 @@ struct utp_upiu_rsp {struct utp_upiu_header header;union {struct utp_cmd_rsp sr; +struct utp_hpb_rsp hr;struct utp_upiu_query qr; }; };
drivers/scsi/ufs/ufshpb.c +993 -3
1,132 lines not on this page · raw message
drivers/scsi/ufs/ufshpb.h +55 -0
83 lines not on this page · raw message
Signature · 1 line
2.17.1
Re: [PATCH v9 3/4] scsi: ufs: L2P map management for HPB read
Quoted 1 line · Daejun Park
+static unsigned int
ufshpb_host_map_kbytes= 1024;
A comment that explains where this value comes from would be welcome.
drivers/scsi/ufs/ufshpb.c void ufshpb_prep(struct ufs_hba *hba, struct ufshcd_lrb *lrbp) · line 118
Quoted 13 lines
drivers/scsi/ufs/ufshpb.c +13
-0+static struct ufshpb_req *ufshpb_get_map_req (struct ufshpb_lu *hpb, +struct ufshpb_subregion *srgn) +{ +struct ufshpb_req *map_req; +struct request *req; +struct bio *bio; + + map_req =kmem_cache_alloc (hpb->map_req_cache,GFP_KERNEL ); +if (!map_req) +return NULL ; + + req =blk_get_request (hpb->sdev_ufs_lu->request_queue, +REQ_OP_SCSI_IN ,BLK_MQ_REQ_PREEMPT );
Why BLK_MQ_REQ_PREEMPT? Since this code is only executed while medium access commands are processed and since none of these commands have the PREEMPT flag set I think that the PREEMPT flag should be left out. Otherwise there probably will be weird interactions with runtime suspended devices.
Is it acceptable that the above blk_get_request() call blocks if a UFS device has been runtime suspended? If not, consider using the flag BLK_MQ_REQ_NOWAIT instead.
drivers/scsi/ufs/ufshpb.c void ufshpb_prep(struct ufs_hba *hba, struct ufshcd_lrb *lrbp) · line 126
Quoted 5 lines
drivers/scsi/ufs/ufshpb.c +5
-0+ bio =bio_alloc (GFP_KERNEL , hpb->pages_per_srgn); +if (!bio) { +blk_put_request (req); + goto free_map_req; + }
If the blk_get_request() would be modified such that it doesn't wait, this call may have to be modified too (GFP_NOWAIT?).
drivers/scsi/ufs/ufshpb.c void ufshpb_prep(struct ufs_hba *hba, struct ufshcd_lrb *lrbp) · line 559
Quoted 3 lines
drivers/scsi/ufs/ufshpb.c +3
-0+if (rgn->rgn_state ==HPB_RGN_INACTIVE ) { +if (atomic_read (&lru_info->active_cnt) + == lru_info->max_lru_active_cnt) {
When splitting a line, please put comparison operators at the end of the line instead of at the start, e.g. as follows:
if (atomic_read(&lru_info->active_cnt) ==
lru_info->max_lru_active_cnt) {Review
Quoted 1 line
(quoted diff) +1
-0+ pool_size = DIV_ROUND_UP(ufshpb_host_map_kbytes * 1024, PAGE_SIZE);
Please use PAGE_ALIGN() to align to the next page boundary.
Thanks,
Bart.
Re: [PATCH v9 3/4] scsi: ufs: L2P map management for HPB read
Hi Bart,
Quoted 3 lines
Quoted 1 line
+static unsigned int
ufshpb_host_map_kbytes= 1024;A comment that explains where this value comes from would be welcome.
I will add a follows comment and change defalut value 1024 to 2048. "A cache size of 2MB can cache ppn in the 1GB range."
Quoted 10 lines
Quoted 10 lines
drivers/scsi/ufs/ufshpb.c +10
-0+static struct ufshpb_req *ufshpb_get_map_req (struct ufshpb_lu *hpb, +struct ufshpb_subregion *srgn) +{ +struct ufshpb_req *map_req; +struct request *req; +struct bio *bio; + + map_req =kmem_cache_alloc (hpb->map_req_cache,GFP_KERNEL ); +if (!map_req) +return NULL ;
Quoted 8 lines
drivers/scsi/ufs/ufshpb.c void ufshpb_prep(struct ufs_hba *hba, struct ufshcd_lrb *lrbp) · line 118
Quoted 3 lines
drivers/scsi/ufs/ufshpb.c +3
-0+ + req =blk_get_request (hpb->sdev_ufs_lu->request_queue, +REQ_OP_SCSI_IN ,BLK_MQ_REQ_PREEMPT );Why
BLK_MQ_REQ_PREEMPT? Since this code is only executed while medium access commands are processed and since none of these commands have the PREEMPT flag set I think that the PREEMPT flag should be left out. Otherwise there probably will be weird interactions with runtime suspended devices.
OK, I will remove BLK_MQ_REQ_PREEMPT flag.
Quoted 3 lines
Is it acceptable that the above
blk_get_request()call blocks if a UFS device has been runtime suspended? If not, consider using the flagBLK_MQ_REQ_NOWAITinstead.
The map worker don't issue map requests when the UFS device is in runtime suspend. So, I think BLK_MQ_REQ_NOWAIT flags is not needed.
Quoted 16 lines
drivers/scsi/ufs/ufshpb.c void ufshpb_prep(struct ufs_hba *hba, struct ufshcd_lrb *lrbp) · line 126
Quoted 5 lines
drivers/scsi/ufs/ufshpb.c +5
-0+ bio =bio_alloc (GFP_KERNEL , hpb->pages_per_srgn); +if (!bio) { +blk_put_request (req); + goto free_map_req; + }If the
blk_get_request()would be modified such that it doesn't wait, this call may have to be modified too (GFP_NOWAIT?).drivers/scsi/ufs/ufshpb.c void ufshpb_prep(struct ufs_hba *hba, struct ufshcd_lrb *lrbp) · line 559
Quoted 3 lines
drivers/scsi/ufs/ufshpb.c +3
-0+if (rgn->rgn_state ==HPB_RGN_INACTIVE ) { +if (atomic_read (&lru_info->active_cnt) + == lru_info->max_lru_active_cnt) {When splitting a line, please put comparison operators at the end of the line instead of at the start, e.g. as follows:
Quoted 2 lines
if (atomic_read(&lru_info->active_cnt) ==
lru_info->max_lru_active_cnt) {OK, I will change it.
Quoted 3 lines
Review
Quoted 1 line
(quoted diff) +1
-0+ pool_size = DIV_ROUND_UP(ufshpb_host_map_kbytes * 1024, PAGE_SIZE);Please use
PAGE_ALIGN()to align to the next page boundary.
OK, I will.
Thanks,
Daejun.
PATCH v9 4/4 scsi: ufs: Prepare HPB read for cached sub-region
In reply to Daejun ParkThis patch changes the read I/O to the HPB read I/O.
If the logical address of the read I/O belongs to active sub-region, the HPB driver modifies the read I/O command to HPB read. It modifies the UPIU command of UFS instead of modifying the existing SCSI command.
In the HPB version 1.0, the maximum read I/O size that can be converted to HPB read is 4KB.
The dirty map of the active sub-region prevents an incorrect HPB read that has stale physical page number which is updated by previous write I/O.
--- drivers/scsi/ufs/ufshpb.c | 231 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 231 insertions(+)
drivers/scsi/ufs/ufshpb.c +231 -0
@@ -33,6 +33,29 @@ bool ufshpb_is_allowed(struct ufs_hba *hba)return !(hba->ufshpb_dev.hpb_disabled); } +static int ufshpb_is_valid_srgn (struct ufshpb_region *rgn, +struct ufshpb_subregion *srgn) +{ +return rgn->rgn_state !=HPB_RGN_INACTIVE && + srgn->srgn_state ==HPB_SRGN_VALID ; +} + +static bool ufshpb_is_read_cmd (struct scsi_cmnd *cmd) +{ +return req_op (cmd->request) ==REQ_OP_READ ; +} + +static bool ufshpb_is_write_discard_cmd (struct scsi_cmnd *cmd) +{ +return op_is_write (req_op (cmd->request)) || +op_is_discard (req_op (cmd->request)); +} + +static bool ufshpb_is_support_chunk (int transfer_len) +{ +return transfer_len <=HPB_MULTI_CHUNK_HIGH ; +} +static bool ufshpb_is_general_lun (int lun) {return lun <UFS_UPIU_MAX_UNIT_NUM_ID ;
@@ -99,8 +122,216 @@ static void ufshpb_set_state(struct ufshpb_lu *hpb, int state)atomic_set (&hpb->hpb_state, state); } +static void ufshpb_set_ppn_dirty (struct ufshpb_lu *hpb,int rgn_idx, +int srgn_idx,int srgn_offset,int cnt) +{ +struct ufshpb_region *rgn; +struct ufshpb_subregion *srgn; +int set_bit_len; +int bitmap_len = hpb->entries_per_srgn; + +next_srgn: + rgn = hpb->rgn_tbl + rgn_idx; + srgn = rgn->srgn_tbl + srgn_idx; + +if ((srgn_offset + cnt) > bitmap_len) + set_bit_len = bitmap_len - srgn_offset; +else + set_bit_len = cnt; + +if (rgn->rgn_state !=HPB_RGN_INACTIVE && + srgn->srgn_state ==HPB_SRGN_VALID ) +bitmap_set (srgn->mctx->ppn_dirty, srgn_offset, set_bit_len); + + srgn_offset =0 ; +if (++srgn_idx == hpb->srgns_per_rgn) { + srgn_idx =0 ; + rgn_idx++; + } + + cnt -= set_bit_len; +if (cnt >0 ) + goto next_srgn; + +WARN_ON (cnt <0 ); +} + +static bool ufshpb_test_ppn_dirty (struct ufshpb_lu *hpb,int rgn_idx, +int srgn_idx,int srgn_offset,int cnt) +{ +struct ufshpb_region *rgn; +struct ufshpb_subregion *srgn; +int bitmap_len = hpb->entries_per_srgn; +int bit_len; + +next_srgn: + rgn = hpb->rgn_tbl + rgn_idx; + srgn = rgn->srgn_tbl + srgn_idx; + +if (!ufshpb_is_valid_srgn (rgn, srgn)) +return true; + +/* +* If the region state is active, mctx must be allocated. +* In this case, check whether the region is evicted or +* mctx allcation fail. +*/ +WARN_ON (!srgn->mctx); + +if ((srgn_offset + cnt) > bitmap_len) + bit_len = bitmap_len - srgn_offset; +else + bit_len = cnt; + +if (find_next_bit (srgn->mctx->ppn_dirty, + bit_len, srgn_offset) >= srgn_offset) +return true; + + srgn_offset =0 ; +if (++srgn_idx == hpb->srgns_per_rgn) { + srgn_idx =0 ; + rgn_idx++; + } + + cnt -= bit_len; +if (cnt >0 ) + goto next_srgn; + +return false; +} + +static u64 ufshpb_get_ppn (struct ufshpb_lu *hpb, +struct ufshpb_map_ctx *mctx,int pos,int *error) +{ +u64 *ppn_table; +struct page *page; +int index, offset; + + index = pos / (PAGE_SIZE /HPB_ENTRY_SIZE ); + offset = pos % (PAGE_SIZE /HPB_ENTRY_SIZE ); + + page = mctx->m_page[index]; +if (unlikely (!page)) { + *error = -ENOMEM ; +dev_err (&hpb->sdev_ufs_lu->sdev_dev, +"error. cannot find page in mctx\n" ); +return 0 ; + } + + ppn_table =page_address (page); +if (unlikely (!ppn_table)) { + *error = -ENOMEM ; +dev_err (&hpb->sdev_ufs_lu->sdev_dev, +"error. cannot get ppn_table\n" ); +return 0 ; + } + +return ppn_table[offset]; +} + +static void +ufshpb_get_pos_from_lpn (struct ufshpb_lu *hpb,unsigned long lpn,int *rgn_idx, +int *srgn_idx,int *offset) +{ +int rgn_offset; + + *rgn_idx = lpn >> hpb->entries_per_rgn_shift; + rgn_offset = lpn & hpb->entries_per_rgn_mask; + *srgn_idx = rgn_offset >> hpb->entries_per_srgn_shift; + *offset = rgn_offset & hpb->entries_per_srgn_mask; +} + +static void +ufshpb_set_hpb_read_to_upiu (struct ufshpb_lu *hpb,struct ufshcd_lrb *lrbp, +u32 lpn,u64 ppn,unsigned int transfer_len) +{ +unsigned char *cdb = lrbp->ucd_req_ptr->sc.cdb; + + cdb[0 ] =UFSHPB_READ ; + +put_unaligned_be64 (ppn, &cdb[6 ]); + cdb[14 ] = transfer_len; +} + +/* +* This function will set up HPB read command using host-side L2P map data. +* In HPB v1.0, maximum size of HPB read command is 4KB. +*/ void ufshpb_prep (struct ufs_hba *hba,struct ufshcd_lrb *lrbp) { +struct ufshpb_lu *hpb; +struct ufshpb_region *rgn; +struct ufshpb_subregion *srgn; +struct scsi_cmnd *cmd = lrbp->cmd; +u32 lpn; +u64 ppn; +unsigned long flags; +int transfer_len, rgn_idx, srgn_idx, srgn_offset; +int err =0 ; + + hpb =ufshpb_get_hpb_data (cmd); +if (!hpb) +return ; + +if (ufshpb_get_state (hpb) !=HPB_PRESENT ) { +dev_notice (&hpb->sdev_ufs_lu->sdev_dev, +"%s: ufshpb state is not PRESENT" , __func__); +return ; + } + +WARN_ON (hpb->lun != cmd->device->lun); +if (!ufshpb_is_write_discard_cmd (cmd) && + !ufshpb_is_read_cmd (cmd)) +return ; + + transfer_len =sectors_to_logical (cmd->device,blk_rq_sectors (cmd->request)); +if (unlikely (!transfer_len)) +return ; + + lpn =sectors_to_logical (cmd->device,blk_rq_pos (cmd->request)); +ufshpb_get_pos_from_lpn (hpb, lpn, &rgn_idx, &srgn_idx, &srgn_offset); + rgn = hpb->rgn_tbl + rgn_idx; + srgn = rgn->srgn_tbl + srgn_idx; + +/* If command type is WRITE or DISCARD, set bitmap as drity */ +if (ufshpb_is_write_discard_cmd (cmd)) { +spin_lock_irqsave (&hpb->hpb_state_lock, flags); +ufshpb_set_ppn_dirty (hpb, rgn_idx, srgn_idx, srgn_offset, + transfer_len); +spin_unlock_irqrestore (&hpb->hpb_state_lock, flags); +return ; + } + +WARN_ON (!ufshpb_is_read_cmd (cmd)); + +if (!ufshpb_is_support_chunk (transfer_len)) +return ; + +spin_lock_irqsave (&hpb->hpb_state_lock, flags); +if (ufshpb_test_ppn_dirty (hpb, rgn_idx, srgn_idx, srgn_offset, + transfer_len)) { +atomic_inc (&hpb->stats.miss_cnt); +spin_unlock_irqrestore (&hpb->hpb_state_lock, flags); +return ; + } + + ppn =ufshpb_get_ppn (hpb, srgn->mctx, srgn_offset, &err); +spin_unlock_irqrestore (&hpb->hpb_state_lock, flags); +if (unlikely (err)) { +/* +* In this case, the region state is active, +* but the ppn table is not allocated. +* Make sure that ppn table must be allocated on +* active state. +*/ +WARN_ON (true); +dev_err (hba->dev,"ufshpb_get_ppn failed. err %d\n" , err); +return ; + } + +ufshpb_set_hpb_read_to_upiu (hpb, lrbp, lpn, ppn, transfer_len); + +atomic_inc (&hpb->stats.hit_cnt); } static struct ufshpb_req *ufshpb_get_map_req(struct ufshpb_lu *hpb,
Signature · 1 line
2.17.1
Re: [PATCH v9 4/4] scsi: ufs: Prepare HPB read for cached sub-region
Quoted 1 line · Daejun Park
This patch changes the read I/O to the HPB read I/O.