ntb_transport: Fix bug with max_mw_size parameter
Subsystems (2)
Logan Gunthorpe
maintainerPATCH v2 1/2 ntb_transport: Fix bug with max_mw_size parameter
When using the max_mw_size parameter of ntb_transport to limit the size of the Memory windows, communication cannot be established and the queues freeze.
This is because the mw_size that's reported to the peer is correctly limited but the size used locally is not. So the MW is initialized with a buffer smaller than the window but the TX side is using the full window. This means the TX side will be writing to a region of the window that points nowhere.
This is easily fixed by applying the same limit to tx_size in ntb_transport_init_queue().
--- drivers/ntb/ntb_transport.c | 3 +++ 1 file changed, 3 insertions(+)
drivers/ntb/ntb_transport.c +3 -0
@@ -1003,6 +1003,9 @@ static int ntb_transport_init_queue(struct ntb_transport_ctx *nt, mw_base = nt->mw_vec[mw_num].phys_addr; mw_size = nt->mw_vec[mw_num].phys_size; +if (max_mw_size && mw_size > max_mw_size) + mw_size = max_mw_size; + tx_size = (unsigned int )mw_size / num_qps_mw; qp_offset = tx_size * (qp_num / mw_count);
Signature · 1 line
2.11.0
Logan Gunthorpe
maintainerPATCH v2 2/2 ntb_hw_switchtec: Check for alignment of the buffer in mw_set_trans()
With Switchtec hardware, the buffer used for a memory window must be aligned to its size (the hardware only replaces the lower bits). In certain circumstances dma_alloc_coherent() will not provide a buffer that adheres to this requirement like when using the CMA and CONFIG_CMA_ALIGNMENT is set lower than the buffer size.
When we get an unaligned buffer mw_set_trans() should return an error. We also log an error so we know the cause of the problem.
---
v2: Use IS_ALIGNED macro as suggested by Allen
drivers/ntb/hw/mscc/ntb_hw_switchtec.c | 13 +++++++++++++ 1 file changed, 13 insertions(+)
drivers/ntb/hw/mscc/ntb_hw_switchtec.c +13 -0
@@ -320,6 +320,19 @@ static int switchtec_ntb_mw_set_trans(struct ntb_dev *ntb, int pidx, int widx,if (xlate_pos <12 )return -EINVAL ; +if (!IS_ALIGNED (addr,BIT_ULL (xlate_pos))) { +/* +* In certain circumstances we can get a buffer that is +* not aligned to its size. (Most of the time +* dma_alloc_coherent ensures this). This can happen when +* using large buffers allocated by the CMA +* (see CMA_CONFIG_ALIGNMENT) +*/ +dev_err (&sndev->stdev->dev, +"ERROR: Memory window address is not aligned to it's size!\n" ); +return -EINVAL ; + } + rc =switchtec_ntb_part_op (sndev, ctl,NTB_CTRL_PART_OP_LOCK ,NTB_CTRL_PART_STATUS_LOCKED );if (rc)
--
2.11.0
Re: [PATCH v2 2/2] ntb_hw_switchtec: Check for alignment of the buffer in mw_set_trans()
Quoted 9 lines · Logan Gunthorpe
With Switchtec hardware, the buffer used for a memory window must be aligned to its size (the hardware only replaces the lower bits). In certain circumstances
dma_alloc_coherent()will not provide a buffer that adheres to this requirement like when using the CMA andCONFIG_CMA_ALIGNMENTis set lower than the buffer size.When we get an unaligned buffer
mw_set_trans()should return an error. We also log an error so we know the cause of the problem.
Applied to ntb-next.
Thanks,
Jon
Quoted 42 lines
Signed-off-by Logan Gunthorpe <[email protected]>Cc Jon Mason <[email protected]>---
v2: Use
IS_ALIGNEDmacro as suggested by Allendrivers/ntb/hw/mscc/ntb_hw_switchtec.c | 13 +++++++++++++ 1 file changed, 13 insertions(+)drivers/ntb/hw/mscc/ntb_hw_switchtec.c +13
-1@@ -320,6 +320,19 @@ static int switchtec_ntb_mw_set_trans(struct ntb_dev *ntb, int pidx, int widx,if (xlate_pos <12 )return -EINVAL ; +if (!IS_ALIGNED (addr,BIT_ULL (xlate_pos))) { +/* +* In certain circumstances we can get a buffer that is +* not aligned to its size. (Most of the time +* dma_alloc_coherent ensures this). This can happen when +* using large buffers allocated by the CMA +* (see CMA_CONFIG_ALIGNMENT) +*/ +dev_err (&sndev->stdev->dev, +"ERROR: Memory window address is not aligned to it's size!\n" ); +return -EINVAL ; + } + rc =switchtec_ntb_part_op (sndev, ctl,NTB_CTRL_PART_OP_LOCK ,NTB_CTRL_PART_STATUS_LOCKED );if (rc) --2.11.0
Signature · 5 lines
You received this message because you are subscribed to the Google Groups "linux-ntb" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/linux-ntb/20171218182506.5219-2-logang%40deltatee.com. For more options, visit https://groups.google.com/d/optout.
Quoted 4 lines · Logan Gunthorpe
When using the
max_mw_sizeparameter ofntb_transportto limit the size of the Memory windows, communication cannot be established and the queues freeze.
Quoted 8 lines
This is because the
mw_sizethat's reported to the peer is correctly limited but the size used locally is not. So the MW is initialized with a buffer smaller than the window but the TX side is using the full window. This means the TX side will be writing to a region of the window that points nowhere.This is easily fixed by applying the same limit to
tx_sizeinntb_transport_init_queue().
Applied to ntb-next.
Thanks,
Jon
Quoted 33 lines
Fixes e26a5843f7f5 ("NTB: Split ntb_hw_intel and ntb_transport drivers")Signed-off-by Logan Gunthorpe <[email protected]>Acked-by Allen Hubbe <[email protected]>Cc Jon Mason <[email protected]>Cc Dave Jiang <[email protected]>--- drivers/ntb/ntb_transport.c | 3 +++ 1 file changed, 3 insertions(+)drivers/ntb/ntb_transport.c +3
-0@@ -1003,6 +1003,9 @@ static int ntb_transport_init_queue(struct ntb_transport_ctx *nt, mw_base = nt->mw_vec[mw_num].phys_addr; mw_size = nt->mw_vec[mw_num].phys_size; +if (max_mw_size && mw_size > max_mw_size) + mw_size = max_mw_size; + tx_size = (unsigned int )mw_size / num_qps_mw; qp_offset = tx_size * (qp_num / mw_count);Signature · 8 lines
2.11.0 -- You received this message because you are subscribed to the Google Groups "linux-ntb" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/linux-ntb/20171218182506.5219-1-logang%40deltatee.com. For more options, visit https://groups.google.com/d/optout.