IB/hfi1: Don't cast parameter in bit operations
Subsystems (2)
PATCH 1/1 IB/hfi1: Don't cast parameter in bit operations
While in this particular case it would not be a (critical) issue, the pattern itself is bad and error prone in case somebody blindly copies to their code.
Don't cast parameter to unsigned long pointer in the bit operations. Instead copy to a local variable on stack of a proper type and use.
--- drivers/infiniband/hw/hfi1/chip.c | 29 ++++++++++++++--------------- 1 file changed, 14 insertions(+), 15 deletions(-)
drivers/infiniband/hw/hfi1/chip.c +14 -15
@@ -8286,34 +8286,33 @@ static void is_interrupt(struct hfi1_devdata *dd, unsigned int source)irqreturn_t general_interrupt (int irq,void *data) {struct hfi1_devdata *dd = data; -u64 regs[CCE_NUM_INT_CSRS ]; +DECLARE_BITMAP (pending,CCE_NUM_INT_CSRS *64 ); +u64 value;u32 bit;int i; -irqreturn_t handled =IRQ_NONE ;this_cpu_inc (*dd->int_counter);/* phase 1: scan and clear all handled interrupts */ for (i =0 ; i <CCE_NUM_INT_CSRS ; i++) { -if (dd->gi_mask[i] ==0 ) { - regs[i] =0 ;/* used later */ -continue ; - } - regs[i] =read_csr (dd,CCE_INT_STATUS + (8 * i)) & - dd->gi_mask[i]; +if (dd->gi_mask[i] ==0 ) + value =0 ;/* used later */ +else + value =read_csr (dd,CCE_INT_STATUS + (8 * i)) & dd->gi_mask[i]; + +/* save for further use */ +bitmap_from_u64 (&pending[BITS_TO_LONGS (i *64 )], value); +/* only clear if anything is set */ -if (regs[i]) -write_csr (dd,CCE_INT_CLEAR + (8 * i), regs[i]); +if (value) +write_csr (dd,CCE_INT_CLEAR + (8 * i), value); }/* phase 2: call the appropriate handler */ -for_each_set_bit (bit, (unsigned long *)®s[0 ], -CCE_NUM_INT_CSRS *64 ) { +for_each_set_bit (bit, pending,CCE_NUM_INT_CSRS *64 )is_interrupt (dd, bit); - handled =IRQ_HANDLED ; - } -return handled; +return IRQ_RETVAL (!bitmap_empty (pending,CCE_NUM_INT_CSRS *64 )); }irqreturn_t sdma_interrupt (int irq,void *data)
Signature · 1 line
2.34.1
From: Andy Shevchenko
Quoted 5 lines
Sent: 23 February 2022 18:54
While in this particular case it would not be a (critical) issue, the pattern itself is bad and error prone in case somebody blindly copies to their code.
It is horribly wrong on BE systems.
Quoted 38 lines
Don't cast parameter to unsigned long pointer in the bit operations. Instead copy to a local variable on stack of a proper type and use.
Fixes 7724105686e7 ("IB/hfi1: add driver files")Signed-off-by Andy Shevchenko <[email protected]>--- drivers/infiniband/hw/hfi1/chip.c | 29 ++++++++++++++--------------- 1 file changed, 14 insertions(+), 15 deletions(-)drivers/infiniband/hw/hfi1/chip.c +6
-8@@ -8286,34 +8286,33 @@ static void is_interrupt(struct hfi1_devdata *dd, unsigned int source)irqreturn_t general_interrupt (int irq,void *data) {struct hfi1_devdata *dd = data; -u64 regs[CCE_NUM_INT_CSRS ]; +DECLARE_BITMAP (pending,CCE_NUM_INT_CSRS *64 ); +u64 value;u32 bit;int i; -irqreturn_t handled =IRQ_NONE ;this_cpu_inc (*dd->int_counter);/* phase 1: scan and clear all handled interrupts */ for (i =0 ; i <CCE_NUM_INT_CSRS ; i++) { -if (dd->gi_mask[i] ==0 ) { - regs[i] =0 ;/* used later */ -continue ; - } - regs[i] =read_csr (dd,CCE_INT_STATUS + (8 * i)) & - dd->gi_mask[i]; +if (dd->gi_mask[i] ==0 ) + value =0 ;/* used later */ +else + value =read_csr (dd,CCE_INT_STATUS + (8 * i)) & dd->gi_mask[i];
drivers/infiniband/hw/hfi1/chip.c static void is_interrupt(struct hfi1_devdata *dd, unsigned int source) · line 8309
Quoted 10 lines
drivers/infiniband/hw/hfi1/chip.c +6
-2+ +/* save for further use */ +bitmap_from_u64 (&pending[BITS_TO_LONGS (i *64 )], value); +/* only clear if anything is set */ -if (regs[i]) -write_csr (dd,CCE_INT_CLEAR + (8 * i), regs[i]); +if (value) +write_csr (dd,CCE_INT_CLEAR + (8 * i), value); }
I think I'd leave all that alone.
drivers/infiniband/hw/hfi1/chip.c static void is_interrupt(struct hfi1_devdata *dd, unsigned int source) · line 8312
Quoted 4 lines
drivers/infiniband/hw/hfi1/chip.c +1
-2/* phase 2: call the appropriate handler */ -for_each_set_bit (bit, (unsigned long *)®s[0 ], -CCE_NUM_INT_CSRS *64 ) { +for_each_set_bit (bit, pending,CCE_NUM_INT_CSRS *64 )
And do something else for that loop instead.
drivers/infiniband/hw/hfi1/chip.c static void is_interrupt(struct hfi1_devdata *dd, unsigned int source) · line 8315
Quoted 6 lines
drivers/infiniband/hw/hfi1/chip.c +1
-3is_interrupt (dd, bit); - handled =IRQ_HANDLED ; - } -return handled; +return IRQ_RETVAL (!bitmap_empty (pending,CCE_NUM_INT_CSRS *64 ));
You really don't want to scan the bitmap again.
Actually, of the face of it, you could merge the two loops. Provided you clear the status bit before calling the relevant handler I expect it will all work.
David
Quoted 5 lines
drivers/infiniband/hw/hfi1/chip.c +0
-1}irqreturn_t sdma_interrupt (int irq,void *data) --2.34.1
-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK Registration No: 1397386 (Wales)
Quoted 8 lines · David Laight
From: Andy Shevchenko
Quoted 5 lines
Sent: 23 February 2022 18:54
While in this particular case it would not be a (critical) issue, the pattern itself is bad and error prone in case somebody blindly copies to their code.
It is horribly wrong on BE systems.
You mean the pattern? Yes, it has three issues regarding to endianess and potential out of boundary access.
...
Quoted 4 lines
drivers/infiniband/hw/hfi1/chip.c static void is_interrupt(struct hfi1_devdata *dd, unsigned int source) · line 8315
Quoted 2 lines
drivers/infiniband/hw/hfi1/chip.c +1
-1-return handled; +return IRQ_RETVAL (!bitmap_empty (pending,CCE_NUM_INT_CSRS *64 ));You really don't want to scan the bitmap again.
Either way it wastes cycles, the outcome depends on the actual distribution of the interrupts across the bitmap. If it gathered closer to the beginning of the bitmap, my code wins, otherwise the original ones.
Quoted 3 lines
Actually, of the face of it, you could merge the two loops. Provided you clear the status bit before calling the relevant handler I expect it will all work.
True. I will consider that for v2.
Signature · 2 lines
With Best Regards, Andy Shevchenko
From: 'Andy Shevchenko'
Quoted 6 lines
Sent: 23 February 2022 22:30
Quoted 3 lines · David Laight
From: Andy Shevchenko
Quoted 2 lines
Sent: 23 February 2022 18:54
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines
While in this particular case it would not be a (critical) issue, the pattern itself is bad and error prone in case somebody blindly copies to their code.
It is horribly wrong on BE systems.
You mean the pattern? Yes, it has three issues regarding to endianess and potential out of boundary access.
Never mind the misaligned page-boundary-crossing locked access.
Quoted 10 lines
...
Quoted 4 lines
drivers/infiniband/hw/hfi1/chip.c static void is_interrupt(struct hfi1_devdata *dd, unsigned int source) · line 8315
Quoted 2 lines
drivers/infiniband/hw/hfi1/chip.c +1
-1-return handled; +return IRQ_RETVAL (!bitmap_empty (pending,CCE_NUM_INT_CSRS *64 ));You really don't want to scan the bitmap again.
Either way it wastes cycles, the outcome depends on the actual distribution of the interrupts across the bitmap. If it gathered closer to the beginning of the bitmap, my code wins, otherwise the original ones.
The loop in bitmap_empty() will kill you - even if the first word in non-zero.
Or just 'or' together the 'value' written to clear the pending interrupts in the first loop.
Or just return IRQ_HANDLED ;-)
Depending on exactly how the interrupt system works on you hardware it is perfectly possible to get another ISR entry for an IRQ bit you just cleared.
Which can generate a 'spurious interrupt' message when IRQ_HANDLED isn't returned (maybe not in Linux...)
It is easiest to see how that can happen with a level sensitive interrupt request.
The write to clear the pending register can get delayed (posted bus write) long enough for the cpu to have actually exited the ISR.
So the IRQ line is still set and the ISR re-entered.
But no pending bits are now set.
Put enough PCIe bridges in a system and overload PCIe links and you might get the same to happen for MSI-X.
Especially since there will be additional delays on the device itself converting the internal IRQ into the required PCIe write.
David
-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK Registration No: 1397386 (Wales)
Quoted 5 lines · David Laight
From: 'Andy Shevchenko'
Quoted 4 lines
Sent: 23 February 2022 22:30
Quoted 2 lines · David Laight
From: Andy Shevchenko
Quoted 1 line
Sent: 23 February 2022 18:54
...
Quoted 5 lines
Quoted 3 lines
Either way it wastes cycles, the outcome depends on the actual distribution of the interrupts across the bitmap. If it gathered closer to the beginning of the bitmap, my code wins, otherwise the original ones.
The loop in
bitmap_empty()will kill you - even if the first word in non-zero.
What loop? Did you really look into implementation of bitmap_empty()?
Signature · 2 lines
With Best Regards, Andy Shevchenko
Dennis Dalessandro
maintainerQuoted 24 lines · Andy Shevchenko
Quoted 8 lines · David Laight
From: Andy Shevchenko
Quoted 5 lines
Sent: 23 February 2022 18:54
While in this particular case it would not be a (critical) issue, the pattern itself is bad and error prone in case somebody blindly copies to their code.
It is horribly wrong on BE systems.
You mean the pattern? Yes, it has three issues regarding to endianess and potential out of boundary access.
...
Quoted 4 lines
drivers/infiniband/hw/hfi1/chip.c static void is_interrupt(struct hfi1_devdata *dd, unsigned int source) · line 8315
Quoted 2 lines
drivers/infiniband/hw/hfi1/chip.c +1
-1-return handled; +return IRQ_RETVAL (!bitmap_empty (pending,CCE_NUM_INT_CSRS *64 ));You really don't want to scan the bitmap again.
Either way it wastes cycles, the outcome depends on the actual distribution of the interrupts across the bitmap. If it gathered closer to the beginning of the bitmap, my code wins, otherwise the original ones.
Quoted 5 lines
Quoted 3 lines
Actually, of the face of it, you could merge the two loops. Provided you clear the status bit before calling the relevant handler I expect it will all work.
True. I will consider that for v2.
Will wait for a v2 patch and I'll test it. We are very sensitive to performance changes.
-Denny
Dennis Dalessandro
maintainerQuoted 8 lines · David Laight
From: Andy Shevchenko
Quoted 5 lines
Sent: 23 February 2022 18:54
While in this particular case it would not be a (critical) issue, the pattern itself is bad and error prone in case somebody blindly copies to their code.
It is horribly wrong on BE systems.
Note that this driver has in Kconfig: depends on X86_64. However it's a good point about not wanting anyone to blindly copy.
-Denny