IB/hfi1, rdmavt, qib: Driver updates for 12/18/2017
Subsystems (3)
cover IB/hfi1, rdmavt, qib: Driver updates for 12/18/2017
Hi Jason and Doug,
Here is another set of patches to land fdor 4.16. Just driver changes and a small change for the MAINTIANERS file. We are getting rid of that ipath email and ensuring Mike and I both receive direct mails for all our drivers instead.
https://github.com/ddalessa/kernel/tree/for-4.16
---
Dennis Dalessandro (2):
IB/hfi1: Check return value of strchr before using it
rdma: Update maintainer contact for Intel RDMA drivers
Kaike Wan (2):
IB/rdmavt: No need to cancel RNRNAK retry timer when it is running IB/rdmavt: Add trace for RNRNAK timer
Michael J. Ruhl (3):
IB/{rdmavt,hfi1,qib}: Self determine driver name IB/{rdmavt,hfi1,qib}: Remove get_card_name() downcall IB/{hfi1,qib}: Fix a concurrency issue with device name in logging
Mike Marciniszyn (2):
IB/rdmavt: Use correct numa node for SRQ allocation IB/rdmavt: Allocate CQ memory on the correct node
Patel Jay P (1):
IB/hfi1: Destroy link_wq workqueue after free_irq()
Sebastian Sanchez (1):
IB/hfi1: Fix infinite loop in 8051 command error path
MAINTAINERS | 4 + drivers/infiniband/hw/hfi1/chip.c | 98 +++++++++++++------------------ drivers/infiniband/hw/hfi1/chip.h | 2 - drivers/infiniband/hw/hfi1/driver.c | 16 ----- drivers/infiniband/hw/hfi1/firmware.c | 64 +++++--------------- drivers/infiniband/hw/hfi1/hfi.h | 27 +++++---- drivers/infiniband/hw/hfi1/init.c | 33 ++++++++-- drivers/infiniband/hw/hfi1/verbs.c | 10 ++- drivers/infiniband/hw/qib/qib.h | 8 +-- drivers/infiniband/hw/qib/qib_driver.c | 16 ----- drivers/infiniband/hw/qib/qib_eeprom.c | 3 - drivers/infiniband/hw/qib/qib_init.c | 2 + drivers/infiniband/hw/qib/qib_verbs.c | 2 - drivers/infiniband/sw/rdmavt/cq.c | 10 ++- drivers/infiniband/sw/rdmavt/qp.c | 9 +-- drivers/infiniband/sw/rdmavt/srq.c | 16 +++-- drivers/infiniband/sw/rdmavt/trace.h | 4 + drivers/infiniband/sw/rdmavt/trace_qp.h | 42 +++++++++++++ drivers/infiniband/sw/rdmavt/vt.c | 1 drivers/infiniband/sw/rdmavt/vt.h | 6 +- include/rdma/rdma_vt.h | 31 ++++++++-- 21 files changed, 204 insertions(+), 200 deletions(-)
--
-Denny
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 20171219034753.2126.78386.stgit-9QXIwq+3FY+1XWohqUldA0EOCMrvLtNR@public.gmane.org
PATCH 1/11 IB/hfi1: Destroy link_wq workqueue after free_irq()
From: Patel Jay P <[email protected]>
A sporadic crash occurs when handle_8051_interrupt handler is invoked while doing rmmod. Actually, handler is invoked after all workqueue related resources are freed which results into crash.
Call Trace: queue_work_on+0x27/0x40 handle_8051_interrupt+0x417/0x710 [hfi1] ? handle_dcc_err+0x212/0x660 [hfi1] ? check_preempt_wakeup+0x119/0x250 ? tracing_is_on+0x15/0x30 ? tracing_record_taskinfo_skip+0x1e/0x40 ? radix_tree_next_chunk+0x10b/0x2e0 ? __slab_free+0x9b/0x2c0 interrupt_clear_down+0x43/0x120 [hfi1] is_dc_int+0x2f/0xa0 [hfi1] general_interrupt+0x18c/0x1f0 [hfi1] __free_irq+0x1b3/0x2d0 free_irq+0x35/0x70 pci_free_irq+0x1c/0x30 clean_up_interrupts+0x53/0xf0 [hfi1] hfi1_start_cleanup+0x122/0x190 [hfi1] postinit_cleanup+0x1d/0x280 [hfi1] remove_one+0x233/0x250 [hfi1] pci_device_remove+0x39/0xc0
When kernel is built with CONFIG_DEBUG_SHIRQ config flag, an extra call to IRQ handler is made from _free_irq() function. The driver should be prepared for this fake call.
Adding a mechanism which detects whether handler is invoked after disabling interrupts. hfi_intr_mask field is added to hfi1_devdata structure which is replica of interrupt mask register of hfi device. The field is updated while writing a value to register.
Destroying link_wq workqueue after calling free_irq. This will make sure that if interrupt handler is invoked before or while calling free_irq then workqueue is destroyed after interrupt is handled.
--- drivers/infiniband/hw/hfi1/chip.c | 8 +++++++- drivers/infiniband/hw/hfi1/hfi.h | 4 ++++ drivers/infiniband/hw/hfi1/init.c | 31 ++++++++++++++++++++++--------- 3 files changed, 33 insertions(+), 10 deletions(-)
drivers/infiniband/hw/hfi1/chip.c +7 -1
@@ -8224,6 +8224,8 @@ static irqreturn_t general_interrupt(int irq, void *data)/* only clear if anything is set */ if (regs[i])write_csr (dd,CCE_INT_CLEAR + (8 * i), regs[i]); + + regs[i] &= dd->hfi_intr_mask[i]; }/* phase 2: call the appropriate handler */
@@ -12942,12 +12944,15 @@ void set_intr_state(struct hfi1_devdata *dd, u32 enable)u64 mask =get_int_mask (dd, i);write_csr (dd,CCE_INT_MASK + (8 * i), mask); + dd->hfi_intr_mask[i] = mask; }init_qsfp_int (dd); }else { -for (i =0 ; i <CCE_NUM_INT_CSRS ; i++) +for (i =0 ; i <CCE_NUM_INT_CSRS ; i++) {write_csr (dd,CCE_INT_MASK + (8 * i),0ull ); + dd->hfi_intr_mask[i] =0ull ; + } } }
@@ -14773,6 +14778,7 @@ void hfi1_start_cleanup(struct hfi1_devdata *dd)free_cntrs (dd);free_rcverr (dd);clean_up_interrupts (dd); +clean_up_workqueues (dd);finish_chip_resources (dd); }
drivers/infiniband/hw/hfi1/hfi.h +4 -0
@@ -1188,6 +1188,9 @@ struct hfi1_devdata {/* INTx information */ u32 requested_intx_irq;/* did we request one? */ +/* copy of interrupt mask register */ +u64 hfi_intr_mask[CCE_NUM_INT_CSRS ]; +/* general interrupt: mask of handled interrupts */ u64 gi_mask[CCE_NUM_INT_CSRS ];
@@ -1993,6 +1996,7 @@ static inline void flush_wc(void)int kdeth_process_eager (struct hfi1_packet *packet);int process_receive_invalid (struct hfi1_packet *packet);void seqfile_dump_rcd (struct seq_file *s,struct hfi1_ctxtdata *rcd); +void clean_up_workqueues (struct hfi1_devdata *dd);/* global module parameter variables */ extern unsigned int hfi1_max_mtu;
drivers/infiniband/hw/hfi1/init.c +22 -9
@@ -823,6 +823,28 @@ static int create_workqueues(struct hfi1_devdata *dd) }/** +* clean_up_workqueues - destroys hfi1_wq and link_wq workqueues +* @dd: the hfi1_ib device +*/ +void clean_up_workqueues (struct hfi1_devdata *dd) +{ +int pidx; +struct hfi1_pportdata *ppd; + +for (pidx =0 ; pidx < dd->num_pports; ++pidx) { + ppd = dd->pport + pidx; +if (ppd->hfi1_wq) { +destroy_workqueue (ppd->hfi1_wq); + ppd->hfi1_wq =NULL ; + } +if (ppd->link_wq) { +destroy_workqueue (ppd->link_wq); + ppd->link_wq =NULL ; + } + } +} + +/** * hfi1_init - do the actual initialization sequence on the chip * @dd: the hfi1_ib device * @reinit: re-initializing, so don't allocate new memory
@@ -1102,15 +1124,6 @@ static void shutdown_device(struct hfi1_devdata *dd)* We can't count on interrupts since we are stopping. */ hfi1_quiet_serdes (ppd); - -if (ppd->hfi1_wq) { -destroy_workqueue (ppd->hfi1_wq); - ppd->hfi1_wq =NULL ; - } -if (ppd->link_wq) { -destroy_workqueue (ppd->link_wq); - ppd->link_wq =NULL ; - } }sdma_exit (dd); }
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 20171219035612.2126.10447.stgit-9QXIwq+3FY+1XWohqUldA0EOCMrvLtNR@public.gmane.org
Re: [PATCH for-next 01/11] IB/hfi1: Destroy link_wq workqueue after free_irq()
Quoted 3 lines · Dennis Dalessandro
When kernel is built with
CONFIG_DEBUG_SHIRQconfig flag, an extra call to IRQ handler is made from_free_irq()function. The driver should be prepared for this fake call.
It is not a 'fake call' it is call designed to test what happens during an IRQ handler racing with free.
Quoted 4 lines
Adding a mechanism which detects whether handler is invoked after disabling interrupts.
hfi_intr_maskfield is added tohfi1_devdatastructure which is replica of interrupt mask register of hfi device. The field is updated while writing a value to register.
And this is not the typical solution.
Explain why adding a non-atomic variable to a concurrancy doesn't just create races and a mess??
free_irq() is a synchronous fence, It returns once all the running IRQ handlers have exited and guarantees they will not be called again.
You are supposed to call it before destroying anything that the IRQ handler could be using.
re-ordering desstruction order is the typical solution.
Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 20171219205754.GE14814-uk2M96/[email protected]
RE: [PATCH for-next 01/11] IB/hfi1: Destroy link_wq workqueue after free_irq()
Quoted 10 lines
-----Original Message-----
From: Jason Gunthorpe [mailto:jgg-uk2M96/[email protected]]
Sent: Tuesday, December 19, 2017 3:58 PM
To: Dalessandro, Dennis <[email protected]>
Cc: [email protected]; [email protected]; Ruhl, Michael J <[email protected]>; Patel, Jay P <[email protected]>; Sanchez, Sebastian <[email protected]>
Subject: Re: [PATCH for-next 01/11] IB/hfi1: Destroylink_wqworkqueue afterfree_irq()
Quoted 8 lines
Quoted 3 lines · Dennis Dalessandro
When kernel is built with
CONFIG_DEBUG_SHIRQconfig flag, an extra call to IRQ handler is made from_free_irq()function. The driver should be prepared for this fake call.It is not a 'fake call' it is call designed to test what happens during an IRQ handler racing with free.
Hi Jason,
After further review it appears that our patch:
commit 05cb18fda926ddce299280bd86cbc9d491306f28
IB/hfi1: Update HFI to use the latest PCI API
has exposed an architecture issue in how we do our driver remove.
Our current path does not adequately deal with the race condition of getting an IRQ during the removal.
To address, we could submit a patch to revert the above commit.
After re-architecting the driver remove path, we would re-submit the PCI API patch.
Note: reverting the PCI API patch will not make the crash addressed in:
IB/hfi1: Destroy link_wq workqueue after free_irq()
issue go away, but will make it more difficult for the issue to occur.
Thanks,
Mike
Quoted 14 lines
Quoted 5 lines
Adding a mechanism which detects whether handler is invoked after disabling interrupts.
hfi_intr_maskfield is added tohfi1_devdatastructure which is replica of interrupt mask register of hfi device. The field is updated while writing a value to register.And this is not the typical solution.
Explain why adding a non-atomic variable to a concurrancy doesn't just create races and a mess??
free_irq()is a synchronous fence, It returns once all the running IRQ handlers have exited and guarantees they will not be called again.
Quoted 6 lines
You are supposed to call it before destroying anything that the IRQ handler could be using.
re-ordering desstruction order is the typical solution.
Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 14063C7AD467DE4B82DEDB5C278E86639F0E3917-AtyAts71sc88Ug9VwtkbtrfspsVTdybXVpNB7YpNyf8@public.gmane.org
Re: [PATCH for-next 01/11] IB/hfi1: Destroy link_wq workqueue after free_irq()
Quoted 5 lines · Michael J
Note: reverting the PCI API patch will not make the crash addressed in:
IB/hfi1: Destroy
link_wqworkqueue afterfree_irq()issue go away, but will make it more difficult for the issue to occur.
It is up to you.
Device removal races should be fixed, but it is often unlikely they would impact any user.
So, abstractly, reverting a racy situation to another racy situation doesn't seem like a worthwhile win to me.
But I do not want to merge this patch which is just making the race harder to detect with our debug tools for finding racing..
Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 20171220211112.GG22908-uk2M96/[email protected]
RE: [PATCH for-next 01/11] IB/hfi1: Destroy link_wq workqueue after free_irq()
Quoted 21 lines
-----Original Message-----
From: [email protected] [mailto:linux-rdma- [email protected]] On Behalf Of Jason Gunthorpe
Sent: Wednesday, December 20, 2017 4:11 PM
To: Ruhl, Michael J <[email protected]>
Cc: Dalessandro, Dennis <[email protected]>; [email protected]; [email protected]; Patel, Jay P <[email protected]>; Sanchez, Sebastian <[email protected]>
Subject: Re: [PATCH for-next 01/11] IB/hfi1: Destroylink_wqworkqueue afterfree_irq()Quoted 5 lines · Michael J
Note: reverting the PCI API patch will not make the crash addressed in:
IB/hfi1: Destroy
link_wqworkqueue afterfree_irq()issue go away, but will make it more difficult for the issue to occur.
It is up to you.
Quoted 8 lines
Device removal races should be fixed, but it is often unlikely they would impact any user.
So, abstractly, reverting a racy situation to another racy situation doesn't seem like a worthwhile win to me.
But I do not want to merge this patch which is just making the race harder to detect with our debug tools for finding racing..
Ok. Will work toward getting the race condition addressed more completely.
Mike
Quoted 5 lines
Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH 2/11 IB/hfi1: Check return value of strchr before using it
In reply to a message not in the archiveThe call to strchr in our counter initialization does not check the return value before attempting to use the pointer. In theory this should not happen given the way the code is structured but do the smart thing and check the value anyway to harden the code.
--- drivers/infiniband/hw/hfi1/verbs.c | 6 ++++++ 1 files changed, 6 insertions(+), 0 deletions(-)
drivers/infiniband/hw/hfi1/verbs.c +6 -0
@@ -1733,6 +1733,12 @@ static int init_cntr_names(const char *names_in,for (i =0 ; i < n; i++) { q[i] = p; p =strchr (p,'\n' ); +if (!p) { + *num_cntrs =0 ; + *cntr_names =NULL ; +kfree (names_out); +return -EINVAL ; + } *p++ ='\0' ; }
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 20171219035621.2126.23093.stgit-9QXIwq+3FY+1XWohqUldA0EOCMrvLtNR@public.gmane.org
Re: [PATCH for-next 02/11] IB/hfi1: Check return value of strchr before using it
Quoted 4 lines · Dennis Dalessandro
The call to strchr in our counter initialization does not check the return value before attempting to use the pointer. In theory this should not happen given the way the code is structured but do the smart thing and check the value anyway to harden the code.
The smartest way is to get rid of the whole "\n"<->"\0" logic and copy/paste mlx5 implementation which does the same thing but statically and much safer than here.
Thanks
A message that is not in the archive (sent off-list or not yet received): 20171220082555.GN2942-U/[email protected]
Re: [PATCH for-next 02/11] IB/hfi1: Check return value of strchr before using it
Quoted 6 lines · Leon Romanovsky
Quoted 4 lines · Dennis Dalessandro
The call to strchr in our counter initialization does not check the return value before attempting to use the pointer. In theory this should not happen given the way the code is structured but do the smart thing and check the value anyway to harden the code.
Quoted 6 lines
The smartest way is to get rid of the whole "\n"<->"\0" logic and copy/paste mlx5 implementation which does the same thing but statically and much safer than here.
Thanks
Not sure I'd agree. Is there something unsafe about the code here? The hole is plugged. Changing the entire implementation for a copy/paste job doesn't seem like a good thing to me.
-Denny
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): f5849e2b-c8cd-b93b-f32f-f423bff9ae31-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org
Re: [PATCH for-next 02/11] IB/hfi1: Check return value of strchr before using it
Quoted 11 lines · Dennis Dalessandro
Quoted 10 lines · Leon Romanovsky
Quoted 4 lines · Dennis Dalessandro
The call to strchr in our counter initialization does not check the return value before attempting to use the pointer. In theory this should not happen given the way the code is structured but do the smart thing and check the value anyway to harden the code.
The smartest way is to get rid of the whole "\n"<->"\0" logic and copy/paste mlx5 implementation which does the same thing but statically and much safer than here.
Quoted 6 lines
Quoted 2 lines
Thanks
Not sure I'd agree. Is there something unsafe about the code here? The hole is plugged. Changing the entire implementation for a copy/paste job doesn't seem like a good thing to me.
The names are static and can't be changed. IMHO, the whole implementation is overkill.
Thanks
Quoted 2 lines
-Denny
A message that is not in the archive (sent off-list or not yet received): 20180103152721.GT10145-U/[email protected]
Re: [PATCH for-next 02/11] IB/hfi1: Check return value of strchr before using it
Quoted 15 lines · Leon Romanovsky
Quoted 14 lines · Dennis Dalessandro
Quoted 12 lines · Leon Romanovsky
Quoted 4 lines · Dennis Dalessandro
The call to strchr in our counter initialization does not check the return value before attempting to use the pointer. In theory this should not happen given the way the code is structured but do the smart thing and check the value anyway to harden the code.
The smartest way is to get rid of the whole "\n"<->"\0" logic and copy/paste mlx5 implementation which does the same thing but statically and much safer than here.
Thanks
Quoted 6 lines
Quoted 3 lines
Not sure I'd agree. Is there something unsafe about the code here? The hole is plugged. Changing the entire implementation for a copy/paste job doesn't seem like a good thing to me.
The names are static and can't be changed. IMHO, the whole implementation is overkill.
Now that I do agree with. It is certainly an area that needs improved upon.
-Denny
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 4555c08f-a568-48ea-e183-2d49ebd36c7c-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org
Re: [PATCH for-next 02/11] IB/hfi1: Check return value of strchr before using it
Quoted 16 lines · Dennis Dalessandro
Quoted 15 lines · Leon Romanovsky
Quoted 14 lines · Dennis Dalessandro
Quoted 12 lines · Leon Romanovsky
Quoted 4 lines · Dennis Dalessandro
The call to strchr in our counter initialization does not check the return value before attempting to use the pointer. In theory this should not happen given the way the code is structured but do the smart thing and check the value anyway to harden the code.
The smartest way is to get rid of the whole "\n"<->"\0" logic and copy/paste mlx5 implementation which does the same thing but statically and much safer than here.
Thanks
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
Not sure I'd agree. Is there something unsafe about the code here? The hole is plugged. Changing the entire implementation for a copy/paste job doesn't seem like a good thing to me.
The names are static and can't be changed. IMHO, the whole implementation is overkill.
Now that I do agree with. It is certainly an area that needs improved upon.
As there is a better solution on the table, and, as you point out in the original patch, this shouldn't happen, I'm dropping this patch and will wait for the better solution to come along.
Signature · 3 lines
Doug Ledford <[email protected]> GPG KeyID: B826A3330E572FDD Key fingerprint = AE6B 1BDA 122B 23B4 265B 1274 B826 A333 0E57 2FDD
PATCH 3/11 IB/rdmavt: No need to cancel RNRNAK retry timer when it is running
In reply to a message not in the archiveFrom: Kaike Wan <[email protected]>
When the rdmavt's RNRNAK timer is fired, it tries to cancel the timer by calling hrtimer_try_to_cancel(), which always returns -1 because the timer is currently running. This patch removes this useless call.
--- drivers/infiniband/sw/rdmavt/qp.c | 4 +--- 1 files changed, 1 insertions(+), 3 deletions(-)
drivers/infiniband/sw/rdmavt/qp.c +1 -3
@@ -2110,10 +2110,8 @@ static int rvt_stop_rnr_timer(struct rvt_qp *qp)lockdep_assert_held (&qp->s_lock);/* Remove QP from rnr timer */ -if (qp->s_flags &RVT_S_WAIT_RNR ) { +if (qp->s_flags &RVT_S_WAIT_RNR ) qp->s_flags &= ~RVT_S_WAIT_RNR ; - rval =hrtimer_try_to_cancel (&qp->s_rnr_timer); - }return rval; }
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH 4/11 IB/{rdmavt, hfi1, qib}: Self determine driver name
In reply to a message not in the archiveFrom: Michael J. Ruhl <[email protected]>
Currently the HFI and QIB drivers allow the IB core to assign a unit number to the driver name string.
If multiple devices exist in a system, there is a possibility that the device unit number and the IB core number will be mismatched.
Fix by using the driver defined unit number to generate the device name.
--- drivers/infiniband/hw/hfi1/init.c | 2 ++ drivers/infiniband/hw/hfi1/verbs.c | 3 --- drivers/infiniband/hw/qib/qib_init.c | 2 ++ drivers/infiniband/hw/qib/qib_verbs.c | 1 - include/rdma/rdma_vt.h | 13 +++++++++++++ 5 files changed, 17 insertions(+), 4 deletions(-)
drivers/infiniband/hw/hfi1/init.c +2 -0
@@ -1285,6 +1285,8 @@ struct hfi1_devdata *hfi1_alloc_devdata(struct pci_dev *pdev, size_t extra) "Could not allocate unit ID: error %d\n", -ret); goto bail; } +rvt_set_ibdev_name (&dd->verbs_dev.rdi,"%s_%d" ,class_name (), dd->unit); +/* * Initialize all locks for the device. This needs to be as early as * possible so locks are usable.
drivers/infiniband/hw/hfi1/verbs.c +0 -3
@@ -1850,7 +1850,6 @@ int hfi1_register_ib_device(struct hfi1_devdata *dd)struct hfi1_ibport *ibp = &ppd->ibport_data;unsigned i;int ret; -size_t lcpysz =IB_DEVICE_NAME_MAX ;for (i =0 ; i < dd->num_pports; i++)init_ibport (ppd + i);
@@ -1878,8 +1877,6 @@ int hfi1_register_ib_device(struct hfi1_devdata *dd)*/ if (!ib_hfi1_sys_image_guid) ib_hfi1_sys_image_guid = ibdev->node_guid; - lcpysz =strlcpy (ibdev->name,class_name (), lcpysz); -strlcpy (ibdev->name + lcpysz,"_%d" ,IB_DEVICE_NAME_MAX - lcpysz); ibdev->owner =THIS_MODULE ; ibdev->phys_port_cnt = dd->num_pports; ibdev->dev.parent = &dd->pcidev->dev;
drivers/infiniband/hw/qib/qib_init.c +2 -0
@@ -1119,6 +1119,8 @@ struct qib_devdata *qib_alloc_devdata(struct pci_dev *pdev, size_t extra) "Could not allocate unit ID: error %d\n", -ret); goto bail; } +rvt_set_ibdev_name (&dd->verbs_dev.rdi,"%s%d" ,"qib" , dd->unit); + dd->int_counter =alloc_percpu (u64);if (!dd->int_counter) { ret = -ENOMEM ;
drivers/infiniband/hw/qib/qib_verbs.c +0 -1
@@ -1571,7 +1571,6 @@ int qib_register_ib_device(struct qib_devdata *dd)if (!ib_qib_sys_image_guid) ib_qib_sys_image_guid = ppd->guid; -strlcpy (ibdev->name,"qib%d" ,IB_DEVICE_NAME_MAX ); ibdev->owner =THIS_MODULE ; ibdev->node_guid = ppd->guid; ibdev->phys_port_cnt = dd->num_pports;
include/rdma/rdma_vt.h +13 -0
@@ -419,6 +419,19 @@ struct rvt_dev_info { }; +/** +* rvt_set_ibdev_name - Craft an IB device name from client info +* @rdi: pointer to the client rvt_dev_info structure +* @name: client specific name +* @unit: client specific unit number. +*/ +static inline void rvt_set_ibdev_name (struct rvt_dev_info *rdi, +const char *fmt,const char *name, +const int unit) +{ +snprintf (rdi->ibdev.name,sizeof (rdi->ibdev.name), fmt, name, unit); +} +static inline struct rvt_pd *ibpd_to_rvtpd (struct ib_pd *ibpd) { return container_of(ibpd, struct rvt_pd, ibpd);
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 20171219035635.2126.59763.stgit-9QXIwq+3FY+1XWohqUldA0EOCMrvLtNR@public.gmane.org
Re: [PATCH for-next 04/11] IB/{rdmavt, hfi1, qib}: Self determine driver name
Quoted 7 lines · Dennis Dalessandro
From: Michael J. Ruhl <[email protected]>
Currently the HFI and QIB drivers allow the IB core to assign a unit number to the driver name string.
If multiple devices exist in a system, there is a possibility that the device unit number and the IB core number will be mismatched.
This is much better than the last try at this. thanks
Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH 5/11 IB/{rdmavt, hfi1, qib}: Remove get_card_name() downcall
In reply to a message not in the archiveFrom: Michael J. Ruhl <[email protected]>
rdmavt has a down call to client drivers to retrieve a crafted card name.
This name should be the IB defined name.
Rather than craft the name each time it is needed, simply retrieve the IB allocated name from the IB device.
Update the function name to reflect its application.
Clean up driver code to match this change.
--- drivers/infiniband/hw/hfi1/driver.c | 8 -------- drivers/infiniband/hw/hfi1/hfi.h | 1 - drivers/infiniband/hw/hfi1/verbs.c | 1 - drivers/infiniband/hw/qib/qib.h | 1 - drivers/infiniband/hw/qib/qib_driver.c | 8 -------- drivers/infiniband/hw/qib/qib_verbs.c | 1 - drivers/infiniband/sw/rdmavt/trace.h | 4 ++-- drivers/infiniband/sw/rdmavt/vt.c | 1 - drivers/infiniband/sw/rdmavt/vt.h | 6 +++--- include/rdma/rdma_vt.h | 18 +++++++++++------- 10 files changed, 16 insertions(+), 33 deletions(-)
drivers/infiniband/hw/hfi1/driver.c +0 -8
@@ -167,14 +167,6 @@ static int hfi1_caps_get(char *buffer, const struct kernel_param *kp)return iname; } -const char *get_card_name (struct rvt_dev_info *rdi) -{ - struct hfi1_ibdev *ibdev = container_of(rdi, struct hfi1_ibdev, rdi); -struct hfi1_devdata *dd =container_of (ibdev, - struct hfi1_devdata, verbs_dev); -return get_unit_name (dd->unit); -} -struct pci_dev *get_pci_dev (struct rvt_dev_info *rdi) { struct hfi1_ibdev *ibdev = container_of(rdi, struct hfi1_ibdev, rdi);
drivers/infiniband/hw/hfi1/hfi.h +0 -1
@@ -1976,7 +1976,6 @@ int get_platform_config_field(struct hfi1_devdata *dd, u32 *data, u32 len);const char *get_unit_name (int unit); -const char *get_card_name (struct rvt_dev_info *rdi);struct pci_dev *get_pci_dev (struct rvt_dev_info *rdi);/*
drivers/infiniband/hw/hfi1/verbs.c +0 -1
@@ -1896,7 +1896,6 @@ int hfi1_register_ib_device(struct hfi1_devdata *dd)* Fill in rvt info object. */ dd->verbs_dev.rdi.driver_f.port_callback = hfi1_create_port_files; - dd->verbs_dev.rdi.driver_f.get_card_name = get_card_name; dd->verbs_dev.rdi.driver_f.get_pci_dev = get_pci_dev; dd->verbs_dev.rdi.driver_f.check_ah = hfi1_check_ah; dd->verbs_dev.rdi.driver_f.notify_new_ah = hfi1_notify_new_ah;
drivers/infiniband/hw/qib/qib.h +0 -1
@@ -1429,7 +1429,6 @@ int qib_pcie_ddinit(struct qib_devdata *, struct pci_dev *,dma_addr_t qib_map_page (struct pci_dev *,struct page *,unsigned long ,size_t ,int );const char *qib_get_unit_name (int unit); -const char *qib_get_card_name (struct rvt_dev_info *rdi);struct pci_dev *qib_get_pci_dev (struct rvt_dev_info *rdi);/*
drivers/infiniband/hw/qib/qib_driver.c +0 -8
@@ -89,14 +89,6 @@return iname; } -const char *qib_get_card_name (struct rvt_dev_info *rdi) -{ - struct qib_ibdev *ibdev = container_of(rdi, struct qib_ibdev, rdi); -struct qib_devdata *dd =container_of (ibdev, - struct qib_devdata, verbs_dev); -return qib_get_unit_name (dd->unit); -} -struct pci_dev *qib_get_pci_dev (struct rvt_dev_info *rdi) { struct qib_ibdev *ibdev = container_of(rdi, struct qib_ibdev, rdi);
drivers/infiniband/hw/qib/qib_verbs.c +0 -1
@@ -1585,7 +1585,6 @@ int qib_register_ib_device(struct qib_devdata *dd)* Fill in rvt info object. */ dd->verbs_dev.rdi.driver_f.port_callback = qib_create_port_files; - dd->verbs_dev.rdi.driver_f.get_card_name = qib_get_card_name; dd->verbs_dev.rdi.driver_f.get_pci_dev = qib_get_pci_dev; dd->verbs_dev.rdi.driver_f.check_ah = qib_check_ah; dd->verbs_dev.rdi.driver_f.check_send_wqe = qib_check_send_wqe;
drivers/infiniband/sw/rdmavt/trace.h +2 -2
@@ -45,8 +45,8 @@* */ -#define RDI_DEV_ENTRY (rdi) __string(dev, rdi->driver_f.get_card_name(rdi)) -#define RDI_DEV_ASSIGN (rdi) __assign_str(dev, rdi->driver_f.get_card_name(rdi)) +#define RDI_DEV_ENTRY (rdi) __string(dev, rvt_get_ibdev_name(rdi)) +#define RDI_DEV_ASSIGN (rdi) __assign_str(dev, rvt_get_ibdev_name(rdi))#include "trace_rvt.h" #include "trace_qp.h"
drivers/infiniband/sw/rdmavt/vt.c +0 -1
@@ -413,7 +413,6 @@ static noinline int check_support(struct rvt_dev_info *rdi, int verb)* required for rdmavt to function. */ if ((!rdi->driver_f.port_callback) || - (!rdi->driver_f.get_card_name) || (!rdi->driver_f.get_pci_dev))return -EINVAL ;break ;
drivers/infiniband/sw/rdmavt/vt.h +3 -3
@@ -63,19 +63,19 @@#define rvt_pr_info (rdi, fmt, ...) \ __rvt_pr_info(rdi->driver_f.get_pci_dev(rdi), \ - rdi->driver_f.get_card_name(rdi), \ + rvt_get_ibdev_name(rdi), \ fmt, \ ##__VA_ARGS__)#define rvt_pr_warn (rdi, fmt, ...) \ __rvt_pr_warn(rdi->driver_f.get_pci_dev(rdi), \ - rdi->driver_f.get_card_name(rdi), \ + rvt_get_ibdev_name(rdi), \ fmt, \ ##__VA_ARGS__)#define rvt_pr_err (rdi, fmt, ...) \ __rvt_pr_err(rdi->driver_f.get_pci_dev(rdi), \ - rdi->driver_f.get_card_name(rdi), \ + rvt_get_ibdev_name(rdi), \ fmt, \ ##__VA_ARGS__)
include/rdma/rdma_vt.h +11 -7
@@ -228,13 +228,6 @@ struct rvt_driver_provided {int (*port_callback)(struct ib_device *,u8 ,struct kobject *);/* -* Returns a string to represent the device for which is being -* registered. This is primarily used for error and debug messages on -* the console. -*/ -const char * (*get_card_name)(struct rvt_dev_info *rdi); - -/* * Returns a pointer to the undelying hardware's PCI device. This is * used to display information as to what hardware is being referenced * in an output message
@@ -432,6 +425,17 @@ static inline void rvt_set_ibdev_name(struct rvt_dev_info *rdi,snprintf (rdi->ibdev.name,sizeof (rdi->ibdev.name), fmt, name, unit); } +/** +* rvt_get_ibdev_name - return the IB name +* @rdi: rdmavt device +* +* Return the registered name of the device. +*/ +static inline const char *rvt_get_ibdev_name (const struct rvt_dev_info *rdi) +{ +return rdi->ibdev.name; +} +static inline struct rvt_pd *ibpd_to_rvtpd (struct ib_pd *ibpd) { return container_of(ibpd, struct rvt_pd, ibpd);
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH 6/11 IB/rdmavt: Use correct numa node for SRQ allocation
In reply to a message not in the archiveFrom: Mike Marciniszyn <[email protected]>
Normal receive queue allocation ensures that kernel receive queues are allocated on the local numa node. Shared receive queues do not behave the same way.
Ensure that kernel shared receive queues are allocated on the device local node.
--- drivers/infiniband/sw/rdmavt/srq.c | 16 +++++++++------- 1 files changed, 9 insertions(+), 7 deletions(-)
drivers/infiniband/sw/rdmavt/srq.c +9 -7
@@ -90,7 +90,7 @@ struct ib_srq *rvt_create_srq(struct ib_pd *ibpd, srq_init_attr->attr.max_wr > dev->dparms.props.max_srq_wr)return ERR_PTR (-EINVAL ); - srq =kmalloc (sizeof (*srq),GFP_KERNEL ); + srq =kzalloc_node (sizeof (*srq),GFP_KERNEL , dev->dparms.node);if (!srq)return ERR_PTR (-ENOMEM );
@@ -101,7 +101,10 @@ struct ib_srq *rvt_create_srq(struct ib_pd *ibpd, srq->rq.max_sge = srq_init_attr->attr.max_sge; sz =sizeof (struct ib_sge ) * srq->rq.max_sge +sizeof (struct rvt_rwqe ); - srq->rq.wq =vmalloc_user (sizeof (struct rvt_rwq ) + srq->rq.size * sz); + srq->rq.wq = udata ? +vmalloc_user (sizeof (struct rvt_rwq ) + srq->rq.size * sz) : +vzalloc_node (sizeof (struct rvt_rwq ) + srq->rq.size * sz, + dev->dparms.node);if (!srq->rq.wq) { ret =ERR_PTR (-ENOMEM ); goto bail_srq;
@@ -129,16 +132,12 @@ struct ib_srq *rvt_create_srq(struct ib_pd *ibpd, ret =ERR_PTR (err); goto bail_ip; } - }else { - srq->ip =NULL ; }/* * ib_create_srq() will initialize srq->ibsrq. */ spin_lock_init (&srq->rq.lock); - srq->rq.wq->head =0 ; - srq->rq.wq->tail =0 ; srq->limit = srq_init_attr->attr.srq_limit;spin_lock (&dev->n_srqs_lock);
@@ -200,7 +199,10 @@ int rvt_modify_srq(struct ib_srq *ibsrq, struct ib_srq_attr *attr, sz =sizeof (struct rvt_rwqe ) + srq->rq.max_sge *sizeof (struct ib_sge ); size = attr->max_wr +1 ; - wq =vmalloc_user (sizeof (struct rvt_rwq ) + size * sz); + wq = udata ? +vmalloc_user (sizeof (struct rvt_rwq ) + size * sz) : +vzalloc_node (sizeof (struct rvt_rwq ) + size * sz, + dev->dparms.node);if (!wq)return -ENOMEM ;
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 20171219035649.2126.1625.stgit-9QXIwq+3FY+1XWohqUldA0EOCMrvLtNR@public.gmane.org
Re: [PATCH for-next 06/11] IB/rdmavt: Use correct numa node for SRQ allocation
Quoted 9 lines · Dennis Dalessandro
From: Mike Marciniszyn <[email protected]>
Normal receive queue allocation ensures that kernel receive queues are allocated on the local numa node. Shared receive queues do not behave the same way.
Ensure that kernel shared receive queues are allocated on the device local node.
Quoted 6 lines
Reviewed-by Sebastian Sanchez <[email protected]>Signed-off-by Mike Marciniszyn <[email protected]>Signed-off-by Dennis Dalessandro <[email protected]>--- drivers/infiniband/sw/rdmavt/srq.c | 16 +++++++++------- 1 files changed, 9 insertions(+), 7 deletions(-)
Are you aware of "__GFP_THISNODE" flag?
Thanks
A message that is not in the archive (sent off-list or not yet received): 20171220081720.GM2942-U/[email protected]
Re: [PATCH for-next 06/11] IB/rdmavt: Use correct numa node for SRQ allocation
Quoted 10 lines · Leon Romanovsky
Quoted 9 lines · Dennis Dalessandro
From: Mike Marciniszyn <[email protected]>
Normal receive queue allocation ensures that kernel receive queues are allocated on the local numa node. Shared receive queues do not behave the same way.
Ensure that kernel shared receive queues are allocated on the device local node.
Quoted 8 lines
Quoted 6 lines
Reviewed-by Sebastian Sanchez <[email protected]>Signed-off-by Mike Marciniszyn <[email protected]>Signed-off-by Dennis Dalessandro <[email protected]>--- drivers/infiniband/sw/rdmavt/srq.c | 16 +++++++++------- 1 files changed, 9 insertions(+), 7 deletions(-)Are you aware of "
__GFP_THISNODE" flag?
It is not relevant to this patch, sorry.
Quoted 2 lines
Thanks
PATCH 7/11 IB/hfi1: Fix infinite loop in 8051 command error path
In reply to a message not in the archiveFrom: Sebastian Sanchez <[email protected]>
When an 8051 command times out, the entire DC block is restarted. During the restart, the host interface version bit is set, which calls do_8051_command() recursively. The host version bit needs to be set before the link moves into polling, so the host version bit can be set in set_local_link_attributes() instead. Thus, the 8051 command functions can be simplied as a non-locking version (dd->dc8051_lock) of those functions are no longer needed.
--- drivers/infiniband/hw/hfi1/chip.c | 85 ++++++++++++--------------------- drivers/infiniband/hw/hfi1/chip.h | 2 - drivers/infiniband/hw/hfi1/firmware.c | 64 ++++++------------------- 3 files changed, 49 insertions(+), 102 deletions(-)
drivers/infiniband/hw/hfi1/chip.c +32 -53
@@ -6518,11 +6518,12 @@ static void _dc_start(struct hfi1_devdata *dd)if (!dd->dc_shutdown)return ; -/* -* Take the 8051 out of reset, wait until 8051 is ready, and set host -* version bit. -*/ -release_and_wait_ready_8051_firmware (dd); +/* Take the 8051 out of reset */ +write_csr (dd,DC_DC8051_CFG_RST ,0ull ); +/* Wait until 8051 is ready */ +if (wait_fm_ready (dd,TIMEOUT_8051_START )) +dd_dev_err (dd,"%s: timeout starting 8051 firmware\n" , + __func__);/* Take away reset for LCB and RX FPE (set in lcb_shutdown). */ write_csr (dd,DCC_CFG_RESET ,0x10 );
@@ -8566,23 +8567,30 @@ int write_lcb_csr(struct hfi1_devdata *dd, u32 addr, u64 data) }/* -* If the 8051 is in reset mode (dd->dc_shutdown == 1), this function -* will still continue executing. -* * Returns: * < 0 = Linux error, not able to get access * > 0 = 8051 command RETURN_CODE */ -static int _do_8051_command (struct hfi1_devdata *dd,u32 type,u64 in_data, -u64 *out_data) +static int do_8051_command ( +struct hfi1_devdata *dd, +u32 type, +u64 in_data, +u64 *out_data) {u64 reg, completed;int return_code;unsigned long timeout; -lockdep_assert_held (&dd->dc8051_lock);hfi1_cdbg (DC8051 ,"type %d, data 0x%012llx" , type, in_data); +mutex_lock (&dd->dc8051_lock); + +/* We can't send any commands to the 8051 if it's in reset */ +if (dd->dc_shutdown) { + return_code = -ENODEV ; + goto fail; + } +/* * If an 8051 host command timed out previously, then the 8051 is * stuck.
@@ -8683,29 +8691,6 @@ static int _do_8051_command(struct hfi1_devdata *dd, u32 type, u64 in_data,write_csr (dd,DC_DC8051_CFG_HOST_CMD_0 ,0 ); fail: -return return_code; -} - -/* -* Returns: -* < 0 = Linux error, not able to get access -* > 0 = 8051 command RETURN_CODE -*/ -static int do_8051_command (struct hfi1_devdata *dd,u32 type,u64 in_data, -u64 *out_data) -{ -int return_code; - -mutex_lock (&dd->dc8051_lock); -/* We can't send any commands to the 8051 if it's in reset */ -if (dd->dc_shutdown) { - return_code = -ENODEV ; - goto fail; - } - - return_code =_do_8051_command (dd, type, in_data, out_data); - -fail:mutex_unlock (&dd->dc8051_lock);return return_code; }
@@ -8715,17 +8700,16 @@ static int set_physical_link_state(struct hfi1_devdata *dd, u64 state)return do_8051_command (dd,HCMD_CHANGE_PHY_STATE , state,NULL ); } -static int _load_8051_config (struct hfi1_devdata *dd,u8 field_id, -u8 lane_id,u32 config_data) +int load_8051_config (struct hfi1_devdata *dd,u8 field_id, +u8 lane_id,u32 config_data) {u64 data;int ret; -lockdep_assert_held (&dd->dc8051_lock); data = (u64 )field_id <<LOAD_DATA_FIELD_ID_SHIFT | (u64 )lane_id <<LOAD_DATA_LANE_ID_SHIFT | (u64 )config_data <<LOAD_DATA_DATA_SHIFT ; - ret =_do_8051_command (dd,HCMD_LOAD_CONFIG_DATA , data,NULL ); + ret =do_8051_command (dd,HCMD_LOAD_CONFIG_DATA , data,NULL );if (ret !=HCMD_SUCCESS ) { dd_dev_err(dd, "load 8051 config: field id %d, lane %d, err %d\n",
@@ -8734,18 +8718,6 @@ static int _load_8051_config(struct hfi1_devdata *dd, u8 field_id,return ret; } -int load_8051_config (struct hfi1_devdata *dd,u8 field_id, -u8 lane_id,u32 config_data) -{ -int return_code; - -mutex_lock (&dd->dc8051_lock); - return_code =_load_8051_config (dd, field_id, lane_id, config_data); -mutex_unlock (&dd->dc8051_lock); - -return return_code; -} -/* * Read the 8051 firmware "registers". Use the RAM directly. Always * set the result, even on error.
@@ -8861,14 +8833,13 @@ int write_host_interface_version(struct hfi1_devdata *dd, u8 version)u32 frame;u32 mask; -lockdep_assert_held (&dd->dc8051_lock); mask = (HOST_INTERFACE_VERSION_MASK <<HOST_INTERFACE_VERSION_SHIFT );read_8051_config (dd,RESERVED_REGISTERS ,GENERAL_CONFIG , &frame);/* Clear, then set field */ frame &= ~mask; frame |= ((u32 )version <<HOST_INTERFACE_VERSION_SHIFT ); -return _load_8051_config (dd,RESERVED_REGISTERS ,GENERAL_CONFIG , - frame); +return load_8051_config (dd,RESERVED_REGISTERS ,GENERAL_CONFIG , + frame); } void read_misc_status(struct hfi1_devdata *dd, u8 *ver_major, u8 *ver_minor,
@@ -9272,6 +9243,14 @@ static int set_local_link_attributes(struct hfi1_pportdata *ppd)if (ret !=HCMD_SUCCESS ) goto set_local_link_attributes_fail; + ret =write_host_interface_version (dd,HOST_INTERFACE_VERSION ); +if (ret !=HCMD_SUCCESS ) { +dd_dev_err (dd, +"Failed to set host interface version, return 0x%x\n" , + ret); + goto set_local_link_attributes_fail; + } +/* * DC supports continuous updates. */
drivers/infiniband/hw/hfi1/chip.h +1 -1
@@ -508,6 +508,7 @@#define DOWN_REMOTE_REASON_SHIFT 16#define DOWN_REMOTE_REASON_MASK 0xff +#define HOST_INTERFACE_VERSION 1#define HOST_INTERFACE_VERSION_SHIFT 16#define HOST_INTERFACE_VERSION_MASK 0xff
@@ -713,7 +714,6 @@ void read_misc_status(struct hfi1_devdata *dd, u8 *ver_major, u8 *ver_minor, u8 *ver_patch);int write_host_interface_version (struct hfi1_devdata *dd,u8 version);void read_guid (struct hfi1_devdata *dd); -int release_and_wait_ready_8051_firmware (struct hfi1_devdata *dd);int wait_fm_ready (struct hfi1_devdata *dd,u32 mstimeout);void set_link_down_reason (struct hfi1_pportdata *ppd,u8 lcl_reason,u8 neigh_reason,u8 rem_reason);
drivers/infiniband/hw/hfi1/firmware.c +16 -48
91 lines not on this page · raw message
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 20171219035657.2126.88651.stgit-9QXIwq+3FY+1XWohqUldA0EOCMrvLtNR@public.gmane.org
Re: [PATCH for-next 07/11] IB/hfi1: Fix infinite loop in 8051 command error path
Quoted 50 lines · Dennis Dalessandro
From: Sebastian Sanchez <[email protected]>
When an 8051 command times out, the entire DC block is restarted. During the restart, the host interface version bit is set, which calls
do_8051_command()recursively. The host version bit needs to be set before the link moves into polling, so the host version bit can be set inset_local_link_attributes()instead. Thus, the 8051 command functions can be simplied as a non-locking version (dd->dc8051_lock) of those functions are no longer needed.Fixes 9be6a5d788b0 ("IB/hfi1: Prevent LNI out of sync by resetting host interface version")Reviewed-by Michael J. Ruhl <[email protected]>Signed-off-by Sebastian Sanchez <[email protected]>Signed-off-by Dennis Dalessandro <[email protected]>--- drivers/infiniband/hw/hfi1/chip.c | 85 ++++++++++++--------------------- drivers/infiniband/hw/hfi1/chip.h | 2 - drivers/infiniband/hw/hfi1/firmware.c | 64 ++++++------------------- 3 files changed, 49 insertions(+), 102 deletions(-)drivers/infiniband/hw/hfi1/chip.c +6
-8@@ -6518,11 +6518,12 @@ static void _dc_start(struct hfi1_devdata *dd)if (!dd->dc_shutdown)return ; -/* -* Take the 8051 out of reset, wait until 8051 is ready, and set host -* version bit. -*/ -release_and_wait_ready_8051_firmware (dd); +/* Take the 8051 out of reset */ +write_csr (dd,DC_DC8051_CFG_RST ,0ull ); +/* Wait until 8051 is ready */ +if (wait_fm_ready (dd,TIMEOUT_8051_START )) +dd_dev_err (dd,"%s: timeout starting 8051 firmware\n" , + __func__);/* Take away reset for LCB and RX FPE (set in lcb_shutdown). */ write_csr (dd,DCC_CFG_RESET ,0x10 );@@ -8566,23 +8567,30 @@ int write_lcb_csr(struct hfi1_devdata *dd, u32 addr, u64 data) }/* -* If the 8051 is in reset mode (dd->dc_shutdown == 1), this function -* will still continue executing. -* * Returns:
drivers/infiniband/hw/hfi1/chip.c int write_lcb_csr(struct hfi1_devdata *dd, u32 addr, u64 data) · line 8578
Quoted 10 lines
drivers/infiniband/hw/hfi1/chip.c +5
-2* < 0 = Linux error, not able to get access * > 0 = 8051 command RETURN_CODE */ -static int _do_8051_command (struct hfi1_devdata *dd,u32 type,u64 in_data, -u64 *out_data) +static int do_8051_command ( +struct hfi1_devdata *dd, +u32 type, +u64 in_data, +u64 *out_data)
What did you try to say by this change ? :)
Thanks
A message that is not in the archive (sent off-list or not yet received): 20171220080854.GL2942-U/[email protected]
RE: [PATCH for-next 07/11] IB/hfi1: Fix infinite loop in 8051 command error path
Quoted 1 line
What did you try to say by this change ? :)
Are you referring to the parameters for do_8051_command() or something else?
Thank you,
Sebastian
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 5CDA63463B33C94CA80846587415F0772829387D-8oqHQFITsIGkrb+BlOpmy7fspsVTdybXVpNB7YpNyf8@public.gmane.org
Re: [PATCH for-next 07/11] IB/hfi1: Fix infinite loop in 8051 command error path
Quoted 3 lines · Sebastian
Quoted 1 line
What did you try to say by this change ? :)
Are you referring to the parameters for
do_8051_command()or something else?
I guess he means why did you randomly reformat the whitespace around _do_8051_command just for removing a '_', and why did you reformat it to a non-kernel-standard formatting?
Have you discovered clang-format's editor integration yet? Makes these issues go away..
Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 20171220181244.GD22908-uk2M96/[email protected]
RE: [PATCH for-next 07/11] IB/hfi1: Fix infinite loop in 8051 command error path
Quoted 1 line
I guess he means why did you randomly reformat the whitespace around
_do_8051_commandjust for >removing a '_', and why did you reformat it to a non-kernel-standard formatting?
An updated patch will be resubmitted.
Thank you,
Sebastian
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH 8/11 IB/rdmavt: Allocate CQ memory on the correct node
In reply to a message not in the archiveFrom: Mike Marciniszyn <[email protected]>
CQ allocation does not ensure that completion queue entries and the completion queue structure are allocated on the correct numa node.
Fix by allocating the rvt_cq and kernel CQ entries on the device node, leaving the user CQ entries on the default local node. Also ensure CQ resizes use the correct allocator when extending a CQ.
--- drivers/infiniband/sw/rdmavt/cq.c | 10 +++++++--- 1 files changed, 7 insertions(+), 3 deletions(-)
drivers/infiniband/sw/rdmavt/cq.c +7 -3
@@ -198,7 +198,7 @@ struct ib_cq *rvt_create_cq(struct ib_device *ibdev,return ERR_PTR (-EINVAL );/* Allocate the completion queue structure. */ - cq =kzalloc (sizeof (*cq),GFP_KERNEL ); + cq =kzalloc_node (sizeof (*cq),GFP_KERNEL , rdi->dparms.node);if (!cq)return ERR_PTR (-ENOMEM );
@@ -214,7 +214,9 @@ struct ib_cq *rvt_create_cq(struct ib_device *ibdev, sz +=sizeof (struct ib_uverbs_wc ) * (entries +1 ); else sz += sizeof(struct ib_wc) * (entries + 1); - wc =vmalloc_user (sz); + wc = udata ? +vmalloc_user (sz) : +vzalloc_node (sz, rdi->dparms.node);if (!wc) { ret =ERR_PTR (-ENOMEM ); goto bail_cq;
@@ -369,7 +371,9 @@ int rvt_resize_cq(struct ib_cq *ibcq, int cqe, struct ib_udata *udata) sz +=sizeof (struct ib_uverbs_wc ) * (cqe +1 ); else sz += sizeof(struct ib_wc) * (cqe + 1); - wc =vmalloc_user (sz); + wc = udata ? +vmalloc_user (sz) : +vzalloc_node (sz, rdi->dparms.node);if (!wc)return -ENOMEM ;
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH 9/11 rdma: Update maintainer contact for Intel RDMA drivers
In reply to a message not in the archiveEnsure both Mike and I are listed as maintainer contacts for Intel's qib, hfi1, and rdmavt drivers.
--- MAINTAINERS | 4 +++- 1 files changed, 3 insertions(+), 1 deletions(-)
MAINTAINERS +3 -1
@@ -11053,7 +11053,8 @@ S: Maintained F: drivers/firmware/qemu_fw_cfg.c QIB DRIVER -M: Mike Marciniszyn <[email protected]> +M: Dennis Dalessandro <[email protected]> +M: Mike Marciniszyn <[email protected]> L: [email protected] S: Supported F: drivers/infiniband/hw/qib/
@@ -11351,6 +11352,7 @@ F: drivers/net/ethernet/rdc/r6040.c RDMAVT - RDMA verbs software M: Dennis Dalessandro <[email protected]> +M: Mike Marciniszyn <[email protected]> L: [email protected] S: Supported F: drivers/infiniband/sw/rdmavt
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 20171219035711.2126.47130.stgit-9QXIwq+3FY+1XWohqUldA0EOCMrvLtNR@public.gmane.org
Re: [PATCH for-next 09/11] rdma: Update maintainer contact for Intel RDMA drivers
Quoted 12 lines · Dennis Dalessandro
Ensure both Mike and I are listed as maintainer contacts for Intel's qib, hfi1, and rdmavt drivers.
Reviewed-by Mike Marciniszyn <[email protected]>Signed-off-by Dennis Dalessandro <[email protected]>MAINTAINERS | 4 +++- 1 files changed, 3 insertions(+), 1 deletions(-)MAINTAINERS +1
-0+++ b/MAINTAINERS
MAINTAINERS S: Maintained · line 11060
Quoted 9 lines
MAINTAINERS +2
-1@@ -11053,7 +11053,8 @@ S: Maintained F: drivers/firmware/qemu_fw_cfg.c QIB DRIVER -M: Mike Marciniszyn <[email protected]> +M: Dennis Dalessandro <[email protected]> +M: Mike Marciniszyn <[email protected]> L: [email protected] S: Supported F: drivers/infiniband/hw/qib/
Please send an update for rdma-core as well
Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH for-next 09/11] rdma: Update maintainer contact for Intel RDMA drivers
In reply to a message not in the archiveQuoted 5 lines · Dennis Dalessandro
Ensure both Mike and I are listed as maintainer contacts for Intel's qib, hfi1, and rdmavt drivers.
Reviewed-by Mike Marciniszyn <[email protected]>Signed-off-by Dennis Dalessandro <[email protected]>
I took this one to for-next, others in the series had comments.
Thanks,
Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH 10/11 IB/{hfi1, qib}: Fix a concurrency issue with device name in logging
In reply to a message not in the archiveFrom: Michael J. Ruhl <[email protected]>
The get_unit_name() function crafts a string based on the device name and the device unit number. It then stores this in a static variable.
This has concurrency issues as can be seen with this log:
hfi1 0000:02:00.0: hfi1_1: read_idle_message: read idle message 0x203 hfi1 0000:01:00.0: hfi1_1: read_idle_message: read idle message 0x203
The PCI device ID (0000:02:00.0 vs. 0000:01:00.0) is correct for the message, but the device string hfi1_1 is incorrect (it should be hfi1_0 for the second log message).
Remove get_unit_name() function.
Instead, use the rvt accessor rvt_get_ibdev_name() to get the IB name string.
Clean up any hfi1_early_xx calls that can now use the new path.
QIB has the same (qib_get_unit_name()) issue. Updating as necessary.
Remove qib_get_unit_name() function.
Update log message that has redundant device name.
--- drivers/infiniband/hw/hfi1/chip.c | 5 ++--- drivers/infiniband/hw/hfi1/driver.c | 8 -------- drivers/infiniband/hw/hfi1/hfi.h | 22 ++++++++++++---------- drivers/infiniband/hw/qib/qib.h | 7 +++---- drivers/infiniband/hw/qib/qib_driver.c | 8 -------- drivers/infiniband/hw/qib/qib_eeprom.c | 3 +-- 6 files changed, 18 insertions(+), 35 deletions(-)
drivers/infiniband/hw/hfi1/chip.c +2 -3
@@ -14929,9 +14929,8 @@ struct hfi1_devdata *hfi1_init_dd(struct pci_dev *pdev,if (num_vls <HFI1_MIN_VLS_SUPPORTED || num_vls >HFI1_MAX_VLS_SUPPORTED ) { -hfi1_early_err (&pdev->dev, -"Invalid num_vls %u, using %u VLs\n" , - num_vls,HFI1_MAX_VLS_SUPPORTED ); +dd_dev_err (dd,"Invalid num_vls %u, using %u VLs\n" , + num_vls,HFI1_MAX_VLS_SUPPORTED ); num_vls =HFI1_MAX_VLS_SUPPORTED ; } ppd->vls_supported = num_vls;
drivers/infiniband/hw/hfi1/driver.c +0 -8
@@ -159,14 +159,6 @@ static int hfi1_caps_get(char *buffer, const struct kernel_param *kp)return scnprintf (buffer,PAGE_SIZE ,"0x%lx" , cap_mask); } -const char *get_unit_name (int unit) -{ -static char iname[16 ]; - -snprintf (iname,sizeof (iname),DRIVER_NAME "_%u" , unit); -return iname; -} -struct pci_dev *get_pci_dev (struct rvt_dev_info *rdi) { struct hfi1_ibdev *ibdev = container_of(rdi, struct hfi1_ibdev, rdi);
drivers/infiniband/hw/hfi1/hfi.h +12 -10
@@ -1975,7 +1975,6 @@ int get_platform_config_field(struct hfi1_devdata *dd, table_type, int table_index, int field_index, u32 *data, u32 len); -const char *get_unit_name (int unit);struct pci_dev *get_pci_dev (struct rvt_dev_info *rdi);/*
@@ -2126,39 +2125,42 @@ static inline u64 hfi1_pkt_base_sdma_integrity(struct hfi1_devdata *dd)#define dd_dev_emerg (dd, fmt, ...) \ dev_emerg(&(dd)->pcidev->dev, "%s: " fmt, \ - get_unit_name((dd)->unit), ##__VA_ARGS__) + rvt_get_ibdev_name(&(dd)->verbs_dev.rdi), ##__VA_ARGS__)#define dd_dev_err (dd, fmt, ...) \ dev_err(&(dd)->pcidev->dev, "%s: " fmt, \ - get_unit_name((dd)->unit), ##__VA_ARGS__) + rvt_get_ibdev_name(&(dd)->verbs_dev.rdi), ##__VA_ARGS__)#define dd_dev_err_ratelimited (dd, fmt, ...) \ dev_err_ratelimited(&(dd)->pcidev->dev, "%s: " fmt, \ - get_unit_name((dd)->unit), ##__VA_ARGS__) + rvt_get_ibdev_name(&(dd)->verbs_dev.rdi), \ + ##__VA_ARGS__)#define dd_dev_warn (dd, fmt, ...) \ dev_warn(&(dd)->pcidev->dev, "%s: " fmt, \ - get_unit_name((dd)->unit), ##__VA_ARGS__) + rvt_get_ibdev_name(&(dd)->verbs_dev.rdi), ##__VA_ARGS__)#define dd_dev_warn_ratelimited (dd, fmt, ...) \ dev_warn_ratelimited(&(dd)->pcidev->dev, "%s: " fmt, \ - get_unit_name((dd)->unit), ##__VA_ARGS__) + rvt_get_ibdev_name(&(dd)->verbs_dev.rdi), \ + ##__VA_ARGS__)#define dd_dev_info (dd, fmt, ...) \ dev_info(&(dd)->pcidev->dev, "%s: " fmt, \ - get_unit_name((dd)->unit), ##__VA_ARGS__) + rvt_get_ibdev_name(&(dd)->verbs_dev.rdi), ##__VA_ARGS__)#define dd_dev_info_ratelimited (dd, fmt, ...) \ dev_info_ratelimited(&(dd)->pcidev->dev, "%s: " fmt, \ - get_unit_name((dd)->unit), ##__VA_ARGS__) + rvt_get_ibdev_name(&(dd)->verbs_dev.rdi), \ + ##__VA_ARGS__)#define dd_dev_dbg (dd, fmt, ...) \ dev_dbg(&(dd)->pcidev->dev, "%s: " fmt, \ - get_unit_name((dd)->unit), ##__VA_ARGS__) + rvt_get_ibdev_name(&(dd)->verbs_dev.rdi), ##__VA_ARGS__)#define hfi1_dev_porterr (dd, port, fmt, ...) \ dev_err(&(dd)->pcidev->dev, "%s: port %u: " fmt, \ - get_unit_name((dd)->unit), (port), ##__VA_ARGS__) + rvt_get_ibdev_name(&(dd)->verbs_dev.rdi), (port), ##__VA_ARGS__)/* * this is used for formatting hw error messages...
drivers/infiniband/hw/qib/qib.h +3 -4
@@ -1428,7 +1428,6 @@ int qib_pcie_ddinit(struct qib_devdata *, struct pci_dev *,*/ dma_addr_t qib_map_page (struct pci_dev *,struct page *,unsigned long ,size_t ,int ); -const char *qib_get_unit_name (int unit);struct pci_dev *qib_get_pci_dev (struct rvt_dev_info *rdi);/*
@@ -1487,15 +1486,15 @@ static inline void qib_flush_wc(void)#define qib_dev_err (dd, fmt, ...) \ dev_err(&(dd)->pcidev->dev, "%s: " fmt, \ - qib_get_unit_name((dd)->unit), ##__VA_ARGS__) + rvt_get_ibdev_name(&(dd)->verbs_dev.rdi), ##__VA_ARGS__)#define qib_dev_warn (dd, fmt, ...) \ dev_warn(&(dd)->pcidev->dev, "%s: " fmt, \ - qib_get_unit_name((dd)->unit), ##__VA_ARGS__) + rvt_get_ibdev_name(&(dd)->verbs_dev.rdi), ##__VA_ARGS__)#define qib_dev_porterr (dd, port, fmt, ...) \ dev_err(&(dd)->pcidev->dev, "%s: IB%u:%u " fmt, \ - qib_get_unit_name((dd)->unit), (dd)->unit, (port), \ + rvt_get_ibdev_name(&(dd)->verbs_dev.rdi), (dd)->unit, (port), \ ##__VA_ARGS__)#define qib_devinfo (pcidev, fmt, ...) \
drivers/infiniband/hw/qib/qib_driver.c +0 -8
@@ -81,14 +81,6 @@struct qlogic_ib_stats qib_stats; -const char *qib_get_unit_name (int unit) -{ -static char iname[16 ]; - -snprintf (iname,sizeof (iname),"infinipath%u" , unit); -return iname; -} -struct pci_dev *qib_get_pci_dev (struct rvt_dev_info *rdi) { struct qib_ibdev *ibdev = container_of(rdi, struct qib_ibdev, rdi);
drivers/infiniband/hw/qib/qib_eeprom.c +1 -2
@@ -163,8 +163,7 @@ void qib_get_eeprom_info(struct qib_devdata *dd)if (bguid[6 ] ==0xff ) {if (bguid[5 ] ==0xff ) {qib_dev_err (dd, -"Can't set %s GUID from base, wraps to OUI!\n" , -qib_get_unit_name (t)); +"Can't set GUID from base, wraps to OUI!\n" ); dd->base_guid =0 ; goto bail; }
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH 11/11 IB/rdmavt: Add trace for RNRNAK timer
In reply to a message not in the archiveFrom: Kaike Wan <[email protected]>
This patch adds static trace for RNRNAK timer. Currently the output from hrtimer static trace only shows the addresses of hrtimers in the system and there is no easy way to correlate an RNRNAK timer with its entries in the hrtimer trace. This patch adds the correlation among a QP, its RNRNAK timer, and its entries in the hrtimer trace. This correlation will be enormously helpful when debugging RNRNAK related issues. In addition, this patch cleans up rvt_stop_rnr_timer() to be void while here.
--- drivers/infiniband/sw/rdmavt/qp.c | 11 ++++---- drivers/infiniband/sw/rdmavt/trace_qp.h | 42 +++++++++++++++++++++++++++++++ 2 files changed, 48 insertions(+), 5 deletions(-)
drivers/infiniband/sw/rdmavt/qp.c +6 -5
@@ -2075,6 +2075,7 @@ void rvt_add_rnr_timer(struct rvt_qp *qp, u32 aeth)lockdep_assert_held (&qp->s_lock); qp->s_flags |=RVT_S_WAIT_RNR ; to =rvt_aeth_to_usec (aeth); +trace_rvt_rnrnak_add (qp, to);hrtimer_start (&qp->s_rnr_timer,ns_to_ktime (1000 * to),HRTIMER_MODE_REL ); }
@@ -2104,15 +2105,14 @@ void rvt_stop_rc_timers(struct rvt_qp *qp)* stop an rnr timer and return if the timer * had been pending. */ -static int rvt_stop_rnr_timer (struct rvt_qp *qp) +static void rvt_stop_rnr_timer (struct rvt_qp *qp) { -int rval =0 ; -lockdep_assert_held (&qp->s_lock);/* Remove QP from rnr timer */ -if (qp->s_flags &RVT_S_WAIT_RNR ) +if (qp->s_flags &RVT_S_WAIT_RNR ) { qp->s_flags &= ~RVT_S_WAIT_RNR ; -return rval; +trace_rvt_rnrnak_stop (qp,0 ); + } }/**
@@ -2165,6 +2165,7 @@ enum hrtimer_restart rvt_rc_rnr_retry(struct hrtimer *t)spin_lock_irqsave (&qp->s_lock, flags);rvt_stop_rnr_timer (qp); +trace_rvt_rnrnak_timeout (qp,0 ); rdi->driver_f.schedule_send (qp);spin_unlock_irqrestore (&qp->s_lock, flags);return HRTIMER_NORESTART ;
drivers/infiniband/sw/rdmavt/trace_qp.h +42 -0
48 lines not on this page · raw message
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 48 lines · Dennis Dalessandro
Hi Jason and Doug,
Here is another set of patches to land fdor 4.16. Just driver changes and a small change for the MAINTIANERS file. We are getting rid of that ipath email and ensuring Mike and I both receive direct mails for all our drivers instead.
https://github.com/ddalessa/kernel/tree/for-4.16
---
Dennis Dalessandro (2):
IB/hfi1: Check return value of strchr before using it
rdma: Update maintainer contact for Intel RDMA driversKaike Wan (2):
IB/rdmavt: No need to cancel RNRNAK retry timer when it is running IB/rdmavt: Add trace for RNRNAK timerMichael J. Ruhl (3):
IB/{rdmavt,hfi1,qib}: Self determine driver name IB/{rdmavt,hfi1,qib}: Removeget_card_name()downcall IB/{hfi1,qib}: Fix a concurrency issue with device name in loggingMike Marciniszyn (2):
IB/rdmavt: Use correct numa node for SRQ allocation IB/rdmavt: Allocate CQ memory on the correct nodePatel Jay P (1):
IB/hfi1: Destroylink_wqworkqueue afterfree_irq()Sebastian Sanchez (1):
IB/hfi1: Fix infinite loop in 8051 command error pathMAINTAINERS | 4 + drivers/infiniband/hw/hfi1/chip.c | 98 +++++++++++++------------------ drivers/infiniband/hw/hfi1/chip.h | 2 - drivers/infiniband/hw/hfi1/driver.c | 16 ----- drivers/infiniband/hw/hfi1/firmware.c | 64 +++++--------------- drivers/infiniband/hw/hfi1/hfi.h | 27 +++++---- drivers/infiniband/hw/hfi1/init.c | 33 ++++++++-- drivers/infiniband/hw/hfi1/verbs.c | 10 ++- drivers/infiniband/hw/qib/qib.h | 8 +-- drivers/infiniband/hw/qib/qib_driver.c | 16 ----- drivers/infiniband/hw/qib/qib_eeprom.c | 3 - drivers/infiniband/hw/qib/qib_init.c | 2 + drivers/infiniband/hw/qib/qib_verbs.c | 2 - drivers/infiniband/sw/rdmavt/cq.c | 10 ++-
Quoted 8 lines
drivers/infiniband/sw/rdmavt/qp.c | 9 +-- drivers/infiniband/sw/rdmavt/srq.c | 16 +++-- drivers/infiniband/sw/rdmavt/trace.h | 4 + drivers/infiniband/sw/rdmavt/trace_qp.h | 42 +++++++++++++ drivers/infiniband/sw/rdmavt/vt.c | 1 drivers/infiniband/sw/rdmavt/vt.h | 6 +- include/rdma/rdma_vt.h | 31 ++++++++-- 21 files changed, 204 insertions(+), 200 deletions(-)
Hi Denny,
I went through this series. Patch 1 had already been dropped. I dropped patch 2 as deferred in lieu of a better patch. Jason had already taken patch 9. Patch 8 was the only one with outstanding comments, and it was just a formatting issue. So, I took patches 3-8 and 10-11, fixed up the formatting in patch 7 myself, and these are now applied. Thanks!
Signature · 3 lines
Doug Ledford <[email protected]> GPG KeyID: B826A3330E572FDD Key fingerprint = AE6B 1BDA 122B 23B4 265B 1274 B826 A333 0E57 2FDD