mpt3sas: Added support for shared host tagset for cpuhotplug
Subsystems (2)
Sreekanth Reddy
maintainerPATCH 1/1 mpt3sas: Added support for shared host tagset for cpuhotplug
MPT Fusion adapters can steer completions to individual queues, and we now have support for shared host-wide tags.
So we can enable multiqueue support for MPT fusion adapters.
Once driver enable shared host-wide tags, cpu hotplug feature is also supported as it was enabled using below patchsets -
commit bf0beec060 ("blk-mq: drain I/O when all CPUs in a hctx are offline")
Currently the driver has provision to disable host-wide tags using "host_tagset_enable" module parameter.
Once we do not have any major performance regression using host-wide tags, we will drop the hand-crafted interrupt affinity settings.
Performance is also meeting the expectation - (used both none and mq-deadline scheduler)
24 Drive SSD on Aero with/without this patch can get 3.1M IOPs"
--- drivers/scsi/mpt3sas/mpt3sas_base.c | 50 ++++++++++++++++++---------- drivers/scsi/mpt3sas/mpt3sas_base.h | 1 + drivers/scsi/mpt3sas/mpt3sas_scsih.c | 42 ++++++++++++++++++++++- 3 files changed, 75 insertions(+), 18 deletions(-)
drivers/scsi/mpt3sas/mpt3sas_base.c +33 -17
@@ -3648,25 +3648,16 @@ _base_get_msix_index(struct MPT3SAS_ADAPTER *ioc,base_mod64 (atomic64_add_return (1 , &ioc->total_io_cnt), ioc->reply_queue_count) : 0; -return ioc->cpu_msix_table[raw_smp_processor_id ()]; -} +if (scmd && ioc->shost->nr_hw_queues >1 ) { +u32 tag =blk_mq_unique_tag (scmd->request); -/** -* _base_sdev_nr_inflight_request -get number of inflight requests -* of a request queue. -* @q: request_queue object -* -* returns number of inflight request of a request queue. -*/ -inline unsigned long -_base_sdev_nr_inflight_request (struct request_queue *q) -{ -struct blk_mq_hw_ctx *hctx = q->queue_hw_ctx[0 ]; +return blk_mq_unique_tag_to_hwq (tag) + + ioc->high_iops_queues; + } -return atomic_read (&hctx->nr_active); +return ioc->cpu_msix_table[raw_smp_processor_id ()]; } -/** * _base_get_high_iops_msix_index - get the msix index of * high iops queues
@@ -3686,7 +3677,8 @@ _base_get_high_iops_msix_index(struct MPT3SAS_ADAPTER *ioc,* reply queues in terms of batch count 16 when outstanding * IOs on the target device is >=8. */ -if (_base_sdev_nr_inflight_request (scmd->device->request_queue) > + +if (atomic_read (&scmd->device->device_busy) >MPT3SAS_DEVICE_HIGH_IOPS_DEPTH ) return base_mod64(( atomic64_add_return(1, &ioc->high_iops_outstanding) /
@@ -3739,8 +3731,23 @@ mpt3sas_base_get_smid_scsiio(struct MPT3SAS_ADAPTER *ioc, u8 cb_idx, struct scsi_cmnd *scmd) {struct scsiio_tracker *request =scsi_cmd_priv (scmd); -unsigned int tag = scmd->request->tag;u16 smid; +u32 tag, unique_tag; + + unique_tag =blk_mq_unique_tag (scmd->request); + tag =blk_mq_unique_tag_to_tag (unique_tag); + +/* +* store hw queue number corresponding to the tag, +* this hw queue number is used later to determine +* the unqiue_tag using below logic. This unique_tag +* is used to retrieve the scmd pointer corresponding +* to tag using scsi_host_find_tag() API, +* +* tag = smid - 1; +* unique_tag = ioc->io_queue_num[tag] << BLK_MQ_UNIQUE_TAG_BITS | tag; +*/ + ioc->io_queue_num[tag] =blk_mq_unique_tag_to_hwq (unique_tag); smid = tag +1 ; request->cb_idx = cb_idx;
@@ -3831,6 +3838,7 @@ mpt3sas_base_free_smid(struct MPT3SAS_ADAPTER *ioc, u16 smid)mpt3sas_base_clear_st (ioc, st);_base_recovery_check (ioc); + ioc->io_queue_num[smid -1 ] =0 ;return ; }
@@ -5362,6 +5370,9 @@ _base_release_memory_pools(struct MPT3SAS_ADAPTER *ioc)kfree (ioc->chain_lookup); ioc->chain_lookup =NULL ; } + +kfree (ioc->io_queue_num); + ioc->io_queue_num =NULL ; }/**
@@ -5772,6 +5783,11 @@ _base_allocate_memory_pools(struct MPT3SAS_ADAPTER *ioc)ioc_info (ioc,"internal(0x%p): depth(%d), start smid(%d)\n" , ioc->internal, ioc->internal_depth, ioc->internal_smid)); + + ioc->io_queue_num =kcalloc (ioc->scsiio_depth, +sizeof (u16),GFP_KERNEL ); +if (!ioc->io_queue_num) + goto out;/* * The number of NVMe page sized blocks needed is: * (((sg_tablesize * 8) - 1) / (page_size - 8)) + 1
drivers/scsi/mpt3sas/mpt3sas_base.h +1 -0
@@ -1439,6 +1439,7 @@ struct MPT3SAS_ADAPTER {spinlock_t scsi_lookup_lock;int pending_io_count;wait_queue_head_t reset_wq; +u16 *io_queue_num;/* PCIe SGL */ struct dma_pool *pcie_sgl_dma_pool;
drivers/scsi/mpt3sas/mpt3sas_scsih.c +41 -1
81 lines not on this page · raw message
Signature · 1 line
2.27.0
kernel test robot
build botI love your patch! Yet something to improve:
Hi Sreekanth,
I love your patch! Yet something to improve:
[auto build test ERROR on scsi/for-next]
[also build test ERROR on next-20210125]
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/Sreekanth-Reddy/mpt3sas-Added-support-for-shared-host-tagset-for-cpuhotplug/20210201-173312 base: https://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi.git for-next config: i386-randconfig-a005-20210201 (attached as .config) compiler: gcc-9 (Debian 9.3.0-15) 9.3.0 reproduce (this is a W=1 build): # https://github.com/0day-ci/linux/commit/97790904f55baf75095685beb4e2598850acbc42 git remote add linux-review https://github.com/0day-ci/linux git fetch --no-tags linux-review Sreekanth-Reddy/mpt3sas-Added-support-for-shared-host-tagset-for-cpuhotplug/20210201-173312 git checkout 97790904f55baf75095685beb4e2598850acbc42 # save the attached .config to linux build tree make W=1 ARCH=i386
If you fix the issue, kindly add following tag as appropriate
All errors (new ones prefixed by >>):
>> ld: drivers/scsi/mpt3sas/mpt3sas_scsih.o:drivers/scsi/mpt3sas/mpt3sas_scsih.c:172: multiple definition of `host_tagset_enable'; drivers/scsi/megaraid/megaraid_sas_base.o:drivers/scsi/megaraid/megaraid_sas_base.c:117: first defined here
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org
kernel test robot
build botI love your patch! Yet something to improve:
Hi Sreekanth,
I love your patch! Yet something to improve:
[auto build test ERROR on scsi/for-next]
[also build test ERROR on mkp-scsi/for-next next-20210125] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/Sreekanth-Reddy/mpt3sas-Added-support-for-shared-host-tagset-for-cpuhotplug/20210201-173312 base: https://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi.git for-next config: arm64-allyesconfig (attached as .config) compiler: aarch64-linux-gcc (GCC) 9.3.0
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# https://github.com/0day-ci/linux/commit/97790904f55baf75095685beb4e2598850acbc42
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Sreekanth-Reddy/mpt3sas-Added-support-for-shared-host-tagset-for-cpuhotplug/20210201-173312 git checkout 97790904f55baf75095685beb4e2598850acbc42
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-9.3.0 make.cross ARCH=arm64
If you fix the issue, kindly add following tag as appropriate
All errors (new ones prefixed by >>):
>> aarch64-linux-ld: drivers/scsi/mpt3sas/mpt3sas_scsih.o:(.data+0xb80): multiple definition of `host_tagset_enable'; drivers/scsi/megaraid/megaraid_sas_base.o:(.data+0x2280): first defined here
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org
Quoted 5 lines · kernel test robot
All errors (new ones prefixed by >>):
Quoted 1 line
Quoted 1 line
aarch64-linux-ld: drivers/scsi/mpt3sas/mpt3sas_scsih.o:(.data+0xb80): multiple definition of `host_tagset_enable'; drivers/scsi/megaraid/megaraid_sas_base.o:(.data+0x2280): first defined here
---
0-DAY CI Kernel Test Service, Intel Corporation
It would be good to fix this for megaraid sas also, i.e. make host_tagset_enable static?
Cheers,
John
Quoted 19 lines
-----Original Message-----
From: John Garry [mailto:john.garry(a)huawei.com]
Sent: Tuesday, February 9, 2021 1:45 PM
To: kernel test robot <[email protected]>; Sreekanth Reddy <[email protected]>; martin.petersen(a)oracle.com
Cc: kbuild-all(a)lists.01.org; linux-scsi(a)vger.kernel.org; sathya.prakash(a)broadcom.com; suganath- prabu.subramani(a)broadcom.com; kashyap.desai(a)broadcom.com
Subject: Re: [PATCH] mpt3sas: Added support for shared host tagset for cpuhotplugQuoted 7 lines · kernel test robot
All errors (new ones prefixed by >>):
Quoted 5 lines
Quoted 5 lines
aarch64-linux-ld:
drivers/scsi/mpt3sas/mpt3sas_scsih.o:(.data+0xb80): multiple definition of `host_tagset_enable'; drivers/scsi/megaraid/megaraid_sas_base.o:(.data+0x2280): first defined here
Quoted 6 lines
Quoted 2 lines
---
0-DAY CI Kernel Test Service, Intel CorporationIt would be good to fix this for megaraid sas also, i.e. make
host_tagset_enablestatic?
I will take care this in megaraid_sas.
Quoted 3 lines
Cheers,
John