scsi: mpi3mr: Introduce smp_affinity_enable module parameter
Subsystems (2)
v2 cover scsi: mpi3mr: Introduce smp_affinity_enable module parameter
I noticed that the Linux MegaRAID driver for SAS based RAID controllers has the same aforementioned module parameter. Despite the potential performance drawbacks, I suspect it would be useful with the Broadcom MPI3 Storage Controller Driver too, to respect the default IRQ affinity. Please let me know your thoughts.
Changes since v1 [1]
- Addressed WARN_ON() in pci_alloc_irq_vectors_affinity() due to affd != NULL when smp_affinity_enable = 0
- Avoid unnecessary complexity for a single queue
[1]: https://lore.kernel.org/lkml/[email protected]/
Aaron Tomlin (1):
scsi: mpi3mr: Introduce smp_affinity_enable module parameter
drivers/scsi/mpi3mr/mpi3mr.h | 1 + drivers/scsi/mpi3mr/mpi3mr_fw.c | 14 ++++++++++++-- drivers/scsi/mpi3mr/mpi3mr_os.c | 14 +++++++++++--- 3 files changed, 24 insertions(+), 5 deletions(-)
Signature · 1 line
2.49.0
RFC PATCH v2 1/1 scsi: mpi3mr: Introduce smp_affinity_enable module parameter
This patch introduces a new module parameter namely "smp_affinity_enable", to govern the application of system-wide IRQ affinity (with kernel boot-time parameter "irqaffinity") for MSI-X interrupts. By default, the default IRQ affinity mask will not be respected. Set smp_affinity_enable to 0 disables this behaviour. Consequently, preventing the auto-assignment of MSI-X IRQs.
--- drivers/scsi/mpi3mr/mpi3mr.h | 1 + drivers/scsi/mpi3mr/mpi3mr_fw.c | 14 ++++++++++++-- drivers/scsi/mpi3mr/mpi3mr_os.c | 14 +++++++++++--- 3 files changed, 24 insertions(+), 5 deletions(-)
drivers/scsi/mpi3mr/mpi3mr.h +1 -0
@@ -1378,6 +1378,7 @@ struct mpi3mr_ioc {u32 num_tb_segs;struct dma_pool *trace_buf_pool;struct segments *trace_buf; +bool smp_affinity_enable; };
drivers/scsi/mpi3mr/mpi3mr_fw.c +12 -2
@@ -22,6 +22,10 @@ static int poll_queues;module_param (poll_queues, int,0444 );MODULE_PARM_DESC (poll_queues,"Number of queues for io_uring poll mode. (Range 1 - 126)" ); +static int smp_affinity_enable =1 ; +module_param (smp_affinity_enable, int,0444 ); +MODULE_PARM_DESC (smp_affinity_enable,"SMP affinity feature enable/disable Default: enable(1)" ); +#if defined(writeq) && defined(CONFIG_64BIT )static inline void mpi3mr_writeq (__u64 b,volatile void __iomem *addr) {
@@ -821,6 +825,7 @@ static int mpi3mr_setup_isr(struct mpi3mr_ioc *mrioc, u8 setup_one)int retval;int i;struct irq_affinity desc = { .pre_vectors =1 , .post_vectors =1 }; +struct irq_affinity *descp = &desc;if (mrioc->is_intr_info_set)return 0 ;
@@ -852,10 +857,13 @@ static int mpi3mr_setup_isr(struct mpi3mr_ioc *mrioc, u8 setup_one) desc.post_vectors = mrioc->requested_poll_qcount; min_vec = desc.pre_vectors + desc.post_vectors; - irq_flags |=PCI_IRQ_AFFINITY |PCI_IRQ_ALL_TYPES ; +if (mrioc->smp_affinity_enable) + irq_flags |=PCI_IRQ_AFFINITY |PCI_IRQ_ALL_TYPES ; +else + descp =NULL ; retval =pci_alloc_irq_vectors_affinity (mrioc->pdev, - min_vec, max_vectors, irq_flags, &desc); + min_vec, max_vectors, irq_flags, descp);if (retval <0 ) { ioc_err(mrioc, "cannot allocate irq vectors, ret %d\n",
@@ -4233,6 +4241,8 @@ int mpi3mr_init_ioc(struct mpi3mr_ioc *mrioc) goto out_failed_noretry; } + mrioc->smp_affinity_enable = smp_affinity_enable ? true : false; + retval =mpi3mr_setup_isr (mrioc,1 );if (retval) { ioc_err(mrioc, "Failed to setup ISR error %d\n",
drivers/scsi/mpi3mr/mpi3mr_os.c +11 -3
@@ -4064,6 +4064,9 @@ static void mpi3mr_map_queues(struct Scsi_Host *shost)int i, qoff, offset;struct blk_mq_queue_map *map =NULL ; +if (shost->nr_hw_queues ==1 ) +return ; + offset = mrioc->op_reply_q_offset;for (i =0 , qoff =0 ; i <HCTX_MAX_TYPES ; i++) {
@@ -5422,8 +5425,6 @@ mpi3mr_probe(struct pci_dev *pdev, const struct pci_device_id *id) shost->max_channel =0 ; shost->max_id =0xFFFFFFFF ; - shost->host_tagset =1 ; -if (prot_mask >=0 )scsi_host_set_prot (shost, prot_mask);else {
@@ -5471,7 +5472,14 @@ mpi3mr_probe(struct pci_dev *pdev, const struct pci_device_id *id) goto init_ioc_failed; } - shost->nr_hw_queues = mrioc->num_op_reply_q; + shost->host_tagset =0 ; + shost->nr_hw_queues =1 ; + +if (mrioc->smp_affinity_enable) { + shost->nr_hw_queues = mrioc->num_op_reply_q; + shost->host_tagset =1 ; + } +if (mrioc->active_poll_qcount) shost->nr_maps =3 ;
Signature · 1 line
2.49.0
Quoted 7 lines
This patch introduces a new module parameter namely "
smp_affinity_enable", to govern the application of system-wide IRQ affinity (with kernel boot-time parameter "irqaffinity") for MSI-X interrupts. By default, the default IRQ affinity mask will not be respected. Setsmp_affinity_enableto 0 disables this behaviour. Consequently, preventing the auto-assignment of MSI-X IRQs.
Broadcom: Please review!
Signature · 1 line
Martin K. Petersen
Quoted 6 lines · Aaron Tomlin
This patch introduces a new module parameter namely "
smp_affinity_enable", to govern the application of system-wide IRQ affinity (with kernel boot-time parameter "irqaffinity") for MSI-X interrupts. By default, the default IRQ affinity mask will not be respected. Setsmp_affinity_enableto 0 disables this behaviour. Consequently, preventing the auto-assignment of MSI-X IRQs.
You have given no substantial motivation for this change.
On the cover letter you have, "I noticed that the Linux MegaRAID driver for SAS based RAID controllers has the same aforementioned module parameter ..." and " I suspect it would be useful ..."