RDMA/irdma: Fix FRWR IRDMAQPSQ_PBLADDR field population
Subsystems (2)
PATCH 1/1 RDMA/irdma: Fix FRWR IRDMAQPSQ_PBLADDR field population
Previously, the IRDMAQPSQ_PBLADDR FRWR WQE field was being populated using a mask that was right-shifted, whereas it should have been the address being right-shifted.
This ended up working fine in many cases because the lowermost 12 bits are always zero (the addresses are page aligned) and the uppermost 12 bits are usually zero too, but it could break for DMA addresses that are wider than 52 bits.
Fix this by right-shifting the address instead of the mask, similar to how it was done in the original i40iw driver.
Tested with NFS over RDMA with some added debug logs to ensure it was hitting both the direct (single page) path as well as multi page (< 512).
--- drivers/infiniband/hw/irdma/ctrl.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/infiniband/hw/irdma/ctrl.c +1 -1
@@ -1727,7 +1727,7 @@ int irdma_sc_mr_fast_register(struct irdma_sc_qp *qp, info->first_pm_pbl_index >> 16);set_64bit_val (wqe,8 ,FIELD_PREP (IRDMAQPSQ_FIRSTPMPBLIDXHI , temp) | -FIELD_PREP (IRDMAQPSQ_PBLADDR >>IRDMA_HW_PAGE_SHIFT , info->reg_addr_pa)); +FIELD_PREP (IRDMAQPSQ_PBLADDR , info->reg_addr_pa >>IRDMA_HW_PAGE_SHIFT ));set_64bit_val (wqe,16 , info->total_len |FIELD_PREP (IRDMAQPSQ_FIRSTPMPBLIDXLO , info->first_pm_pbl_index));
Signature · 1 line
2.56.0.385.gd3acb90ef8-goog
[email protected]
AI reviewSashiko has reviewed this patch and found no issues. It looks great!
Quoted 12 lines
Previously, the
IRDMAQPSQ_PBLADDRFRWR WQE field was being populated using a mask that was right-shifted, whereas it should have been the address being right-shifted.This ended up working fine in many cases because the lowermost 12 bits are always zero (the addresses are page aligned) and the uppermost 12 bits are usually zero too, but it could break for DMA addresses that are wider than 52 bits.
Fix this by right-shifting the address instead of the mask, similar to how it was done in the original i40iw driver.
Quoted 6 lines
Tested with NFS over RDMA with some added debug logs to ensure it was hitting both the direct (single page) path as well as multi page (< 512).
Fixes 3f49d6842569 ("RDMA/irdma: Implement HW Admin Queue OPs")Signed-off-by Jacob Moroni <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Leon Romanovsky
AppliedQuoted 10 lines · Jacob Moroni
Previously, the
IRDMAQPSQ_PBLADDRFRWR WQE field was being populated using a mask that was right-shifted, whereas it should have been the address being right-shifted.This ended up working fine in many cases because the lowermost 12 bits are always zero (the addresses are page aligned) and the uppermost 12 bits are usually zero too, but it could break for DMA addresses that are wider than 52 bits.
[...]
Applied, thanks!
[1/1] RDMA/irdma: Fix FRWR IRDMAQPSQ_PBLADDR field population
https://git.kernel.org/rdma/rdma/c/59df57262e6fa9
Best regards,
Signature · 1 line
Leon Romanovsky <[email protected]>
Am 15.10.26 um 20:44 schrieb Leon Romanovsky:
Quoted 11 lines · Jacob Moroni
Quoted 9 lines
Previously, the
IRDMAQPSQ_PBLADDRFRWR WQE field was being populated using a mask that was right-shifted, whereas it should have been the address being right-shifted.This ended up working fine in many cases because the lowermost 12 bits are always zero (the addresses are page aligned) and the uppermost 12 bits are usually zero too, but it could break for DMA addresses that are wider than 52 bits.
Quoted 6 lines
Quoted 1 line
[...]
Applied, thanks!
[1/1] RDMA/irdma: Fix FRWR
IRDMAQPSQ_PBLADDRfield populationhttps://git.kernel.org/rdma/rdma/c/59df57262e6fa9
My initial tests with this resolved the smbdirect related data corruption.
So it replaces this with success:
https://lore.kernel.org/linux-rdma/[email protected]/
Thanks!
metze
Thanks for verifying. There's still another fix for PBLE_LEVEL_2 on the way as well.
I should have it out shortly.
Thanks,
Jake