Skip to contentSkip to conversation

Fix breakage caused by the NTB multi-port patchset

PATCH In review Logan Gunthorpe 3 participants 12 messages Jan 9, 2019 – Feb 11, 2019

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

v3 cover Fix breakage caused by the NTB multi-port patchset

Hey,

I'm resending this because I've recently found out that the change we made to use the NTB struct device in DMA allocations is wrong and needs to be reverted. Turns out that, when running with an IOMMU, dma_alloc_coherent() will always fail if you pass it the NTB struct device. This is because the device has not been assigned an IOMMU group and the Intel IOMMU at least expect the devices to be on the PCI bus and be able to find a proper bus-dev-fn number through a struct pci device. Therefore, we must revert the change and I've changed patch 2 to do this and remove the no longer necessary DMA mask adjustments.

I'm not sure if we can get past the impass in getting this series merged:
I still maintain every patch in this series is necessary to fix a regression and there's no way to add port numbers to switchtec in the crosslink configuration so it can't be fixed in the other way that was suggested.

Logan

--

Changes since v2:
- Rebased on v5.0-rc1
- Modify Patch 2 to revert back to using the PCI struct device instead of the NTB struct device in DMA calls
- Collected Allen's Acks
- Collected Alexander's Tested-By

Changes since v1:
- Rebased onto ntb-next (there was a minor conflict in a recent change to the intel driver)
- Collected Dave's Ack

--

Logan Gunthorpe (8):
NTB: ntb_tool: reading the link file should not end in a NULL byte
NTB: Revert the change to use the NTB device dev for DMA allocations
NTB: Fix the default port and peer numbers for legacy drivers
NTB: ntb_pingpong: Choose doorbells based on port number
NTB: perf: Don't require one more memory window than number of peers
NTB: perf: Fix support for hardware that doesn't have port numbers
NTB: perf: Fix race condition when run with ntb_test
NTB: ntb_test: Fix bug when counting remote files

 drivers/ntb/hw/amd/ntb_hw_amd.c         |  4 ----
 drivers/ntb/hw/idt/ntb_hw_idt.c         |  6 -----
 drivers/ntb/hw/intel/ntb_hw_gen1.c      |  4 ----
 drivers/ntb/ntb.c                       |  9 ++------
 drivers/ntb/test/ntb_perf.c             | 29 +++++++++++++++++++------
 drivers/ntb/test/ntb_pingpong.c         | 14 +++++-------
 drivers/ntb/test/ntb_tool.c             |  9 ++++----
 tools/testing/selftests/ntb/ntb_test.sh |  2 +-
 8 files changed, 35 insertions(+), 42 deletions(-)

--
2.19.0

#1 lore raw

PATCH v3 1/8 NTB: ntb_tool: reading the link file should not end in a NULL byte

When running ntb_test this warning is issued:

./ntb_test.sh: line 200: warning: command substitution: ignored null byte in input

This is caused by the kernel returning one more byte than is necessary when reading the link file.

Reduce the number of bytes read back to 2 as it was before the commit that regressed this.

Fixes 7f46c8b3a552 ("NTB: ntb_tool: Add full multi-port NTB API support")
---
 drivers/ntb/test/ntb_tool.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
drivers/ntb/test/ntb_tool.c +1 -2
index d592c0ffbd19..ec5cf095cdb9 100644
@@ -504,7 +504,7 @@ static ssize_t tool_peer_link_read(struct file *filep, char __user *ubuf,
 	buf[1] = '\n';
 	buf[2] = '\0';
 
-	return simple_read_from_buffer(ubuf, size, offp, buf, 3);
+	return simple_read_from_buffer(ubuf, size, offp, buf, 2);
 }
 
 static TOOL_FOPS_RDWR(tool_peer_link_fops,
@@ -1690,4 +1690,3 @@ static void __exit tool_exit(void)
 	debugfs_remove_recursive(tool_dbgfs_topdir);
 }
 module_exit(tool_exit);
-
Signature · 1 line
2.19.0
#2 lore raw

PATCH v3 2/8 NTB: Revert the change to use the NTB device dev for DMA allocations

In reply to Logan Gunthorpe

Commit 417cf39cfea9 ("NTB: Set dma mask and dma coherent mask to NTB devices") started using the NTB device for DMA allocations which was turns out was wrong. If the IOMMU is enabled, such alloctanions will always fail with messages such as:

DMAR: Allocating domain for 0000:02:00.1 failed

This is because the IOMMU has not setup the device for such use.

Change the tools back to using the PCI device for allocations seeing it doesn't make sense to add an IOMMU group for the non-physical NTB device. Also remove the code that sets the DMA mask as it no longer makes sense to do this.

Fixes 7f46c8b3a552 ("NTB: ntb_tool: Add full multi-port NTB API support")
---
 drivers/ntb/hw/amd/ntb_hw_amd.c    | 4 ----
 drivers/ntb/hw/idt/ntb_hw_idt.c    | 6 ------
 drivers/ntb/hw/intel/ntb_hw_gen1.c | 4 ----
 drivers/ntb/ntb.c                  | 1 -
 drivers/ntb/test/ntb_perf.c        | 7 +++----
 drivers/ntb/test/ntb_tool.c        | 6 +++---
 6 files changed, 6 insertions(+), 22 deletions(-)
drivers/ntb/hw/amd/ntb_hw_amd.c +0 -4
index efb214fc545a..16c10dfe0919 100644
@@ -1020,10 +1020,6 @@ static int amd_ntb_init_pci(struct amd_ntb_dev *ndev,
 			goto err_dma_mask;
 		dev_warn(&pdev->dev, "Cannot DMA consistent highmem\n");
 	}
-	rc = dma_coerce_mask_and_coherent(&ndev->ntb.dev,
-					  dma_get_mask(&pdev->dev));
-	if (rc)
-		goto err_dma_mask;
 
 	ndev->self_mmio = pci_iomap(pdev, 0, 0);
 	if (!ndev->self_mmio) {
drivers/ntb/hw/idt/ntb_hw_idt.c +0 -6
index 1dede87dd54f..30c29183f515 100644
@@ -2667,12 +2667,6 @@ static int idt_init_pci(struct idt_ntb_dev *ndev)
 		dev_warn(&pdev->dev,
 			"Cannot set consistent DMA highmem bit mask\n");
 	}
-	ret = dma_coerce_mask_and_coherent(&ndev->ntb.dev,
-					   dma_get_mask(&pdev->dev));
-	if (ret != 0) {
-		dev_err(&pdev->dev, "Failed to set NTB device DMA bit mask\n");
-		return ret;
-	}
 
 	/*
 	 * Enable the device advanced error reporting. It's not critical to
drivers/ntb/hw/intel/ntb_hw_gen1.c +0 -4
index 2ad263f708da..9128b32a562f 100644
@@ -1770,10 +1770,6 @@ static int intel_ntb_init_pci(struct intel_ntb_dev *ndev, struct pci_dev *pdev)
 			goto err_dma_mask;
 		dev_warn(&pdev->dev, "Cannot DMA consistent highmem\n");
 	}
-	rc = dma_coerce_mask_and_coherent(&ndev->ntb.dev,
-					  dma_get_mask(&pdev->dev));
-	if (rc)
-		goto err_dma_mask;
 
 	ndev->self_mmio = pci_iomap(pdev, 0, 0);
 	if (!ndev->self_mmio) {
drivers/ntb/ntb.c +0 -1
index 2581ab724c34..dc82be2dcf47 100644
@@ -315,4 +315,3 @@ static void __exit ntb_driver_exit(void)
 	bus_unregister(&ntb_bus);
 }
 module_exit(ntb_driver_exit);
-
drivers/ntb/test/ntb_perf.c +3 -4
index 2a9d6b0d1f19..9174085ddc13 100644
@@ -559,7 +559,7 @@ static void perf_free_inbuf(struct perf_peer *peer)
 		return;
 
 	(void)ntb_mw_clear_trans(peer->perf->ntb, peer->pidx, peer->gidx);
-	dma_free_coherent(&peer->perf->ntb->dev, peer->inbuf_size,
+	dma_free_coherent(&peer->perf->ntb->pdev->dev, peer->inbuf_size,
 			  peer->inbuf, peer->inbuf_xlat);
 	peer->inbuf = NULL;
 }
@@ -588,8 +588,8 @@ static int perf_setup_inbuf(struct perf_peer *peer)
 
 	perf_free_inbuf(peer);
 
-	peer->inbuf = dma_alloc_coherent(&perf->ntb->dev, peer->inbuf_size,
-					 &peer->inbuf_xlat, GFP_KERNEL);
+	peer->inbuf = dma_alloc_coherent(&perf->ntb->pdev->dev,
+			peer->inbuf_size, &peer->inbuf_xlat, GFP_KERNEL);
 	if (!peer->inbuf) {
 		dev_err(&perf->ntb->dev, "Failed to alloc inbuf of %pa\n",
 			&peer->inbuf_size);
@@ -1512,4 +1512,3 @@ static void __exit perf_exit(void)
 	destroy_workqueue(perf_wq);
 }
 module_exit(perf_exit);
-
drivers/ntb/test/ntb_tool.c +3 -3
index ec5cf095cdb9..311d6ab8d016 100644
@@ -590,7 +590,7 @@ static int tool_setup_mw(struct tool_ctx *tc, int pidx, int widx,
 	inmw->size = min_t(resource_size_t, req_size, size);
 	inmw->size = round_up(inmw->size, addr_align);
 	inmw->size = round_up(inmw->size, size_align);
-	inmw->mm_base = dma_alloc_coherent(&tc->ntb->dev, inmw->size,
+	inmw->mm_base = dma_alloc_coherent(&tc->ntb->pdev->dev, inmw->size,
 					   &inmw->dma_base, GFP_KERNEL);
 	if (!inmw->mm_base)
 		return -ENOMEM;
@@ -612,7 +612,7 @@ static int tool_setup_mw(struct tool_ctx *tc, int pidx, int widx,
 	return 0;
 
 err_free_dma:
-	dma_free_coherent(&tc->ntb->dev, inmw->size, inmw->mm_base,
+	dma_free_coherent(&tc->ntb->pdev->dev, inmw->size, inmw->mm_base,
 			  inmw->dma_base);
 	inmw->mm_base = NULL;
 	inmw->dma_base = 0;
@@ -629,7 +629,7 @@ static void tool_free_mw(struct tool_ctx *tc, int pidx, int widx)
 
 	if (inmw->mm_base != NULL) {
 		ntb_mw_clear_trans(tc->ntb, pidx, widx);
-		dma_free_coherent(&tc->ntb->dev, inmw->size,
+		dma_free_coherent(&tc->ntb->pdev->dev, inmw->size,
 				  inmw->mm_base, inmw->dma_base);
 	}
 
Signature · 1 line
2.19.0
#3 lore raw

PATCH v3 3/8 NTB: Fix the default port and peer numbers for legacy drivers

In reply to Logan Gunthorpe

When the commit adding ntb_default_port_number() and ntb_default_peer_port_number() entered the kernel there was no users of it so it was impossible to tell what the API needed.

When a user finally landed a year later (ntb_pingpong) there were more NTB topologies were created and no consideration was considered to how other drivers had changed.

Now that there is a user it can be fixed to provide a sensible default for the legacy drivers that do not implement ntb_{peer_}port_number(). Seeing ntb_pingpong doesn't check error codes returning EINVAL was also not sensible.

Patches for ntb_pingpong and ntb_perf follow (which are broken otherwise) to support hardware that doesn't have port numbers. This is important not only to not break support with existing drivers but for the cross link topology which, due to its perfect symmetry, cannot assign unique port numbers to each side.

Fixes 1e5301196a88 ("NTB: Add indexed ports NTB API")
---
 drivers/ntb/ntb.c | 8 ++------
 1 file changed, 2 insertions(+), 6 deletions(-)
drivers/ntb/ntb.c +2 -6
index dc82be2dcf47..f8f75a504a58 100644
@@ -214,10 +214,8 @@ int ntb_default_port_number(struct ntb_dev *ntb)
 	case NTB_TOPO_B2B_DSD:
 		return NTB_PORT_SEC_DSD;
 	default:
-		break;
+		return 0;
 	}
-
-	return -EINVAL;
 }
 EXPORT_SYMBOL(ntb_default_port_number);
 
@@ -240,10 +238,8 @@ int ntb_default_peer_port_number(struct ntb_dev *ntb, int pidx)
 	case NTB_TOPO_B2B_DSD:
 		return NTB_PORT_PRI_USD;
 	default:
-		break;
+		return 0;
 	}
-
-	return -EINVAL;
 }
 EXPORT_SYMBOL(ntb_default_peer_port_number);
 
Signature · 1 line
2.19.0
#4 lore raw

PATCH v3 4/8 NTB: ntb_pingpong: Choose doorbells based on port number

In reply to Logan Gunthorpe

This commit fixes pingpong support for existing drivers that do not implement ntb_default_port_number() and ntb_default_peer_port_number(). This is required for hardware (like the crosslink topology of switchtec) which cannot assign reasonable port numbers to each port due to its perfect symmetry.

Instead of picking the doorbell to use based on the the index of the peer, we use the peer's port number. This is a bit clearer and easier to understand.

Fixes c7aeb0afdcc2 ("NTB: ntb_pp: Add full multi-port NTB API support")
---
 drivers/ntb/test/ntb_pingpong.c | 14 ++++++--------
 1 file changed, 6 insertions(+), 8 deletions(-)
drivers/ntb/test/ntb_pingpong.c +6 -8
index 65865e460ab8..18d00eec7b02 100644
@@ -121,15 +121,14 @@ static int pp_find_next_peer(struct pp_ctx *pp)
 	link = ntb_link_is_up(pp->ntb, NULL, NULL);
 
 	/* Find next available peer */
-	if (link & pp->nmask) {
+	if (link & pp->nmask)
 		pidx = __ffs64(link & pp->nmask);
-		out_db = BIT_ULL(pidx + 1);
-	} else if (link & pp->pmask) {
+	else if (link & pp->pmask)
 		pidx = __ffs64(link & pp->pmask);
-		out_db = BIT_ULL(pidx);
-	} else {
+	else
 		return -ENODEV;
-	}
+
+	out_db = BIT_ULL(ntb_peer_port_number(pp->ntb, pidx));
 
 	spin_lock(&pp->lock);
 	pp->out_pidx = pidx;
@@ -303,7 +302,7 @@ static void pp_init_flds(struct pp_ctx *pp)
 			break;
 	}
 
-	pp->in_db = BIT_ULL(pidx);
+	pp->in_db = BIT_ULL(lport);
 	pp->pmask = GENMASK_ULL(pidx, 0) >> 1;
 	pp->nmask = GENMASK_ULL(pcnt - 1, pidx);
 
@@ -435,4 +434,3 @@ static void __exit pp_exit(void)
 	debugfs_remove_recursive(pp_dbgfs_topdir);
 }
 module_exit(pp_exit);
-
Signature · 1 line
2.19.0
#5 lore raw

PATCH v3 5/8 NTB: perf: Don't require one more memory window than number of peers

In reply to Logan Gunthorpe

ntb_perf should not require more than one memory window per peer. This was probably an off-by-one error.

Fixes 5648e56d03fa ("NTB: ntb_perf: Add full multi-port NTB API support")
---
 drivers/ntb/test/ntb_perf.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/ntb/test/ntb_perf.c +1 -1
index 9174085ddc13..bd0474e62a40 100644
@@ -655,7 +655,7 @@ static int perf_init_service(struct perf_ctx *perf)
 {
 	u64 mask;
 
-	if (ntb_peer_mw_count(perf->ntb) < perf->pcnt + 1) {
+	if (ntb_peer_mw_count(perf->ntb) < perf->pcnt) {
 		dev_err(&perf->ntb->dev, "Not enough memory windows\n");
 		return -EINVAL;
 	}
Signature · 1 line
2.19.0
#6 lore raw

PATCH v3 6/8 NTB: perf: Fix support for hardware that doesn't have port numbers

In reply to Logan Gunthorpe

Legacy drivers do not have port numbers (but is reliably only two ports) and was broken by the recent commit that added mult-port support to ntb_perf. This is especially important to support the cross link topology which is perfectly symmetric and cannot assign unique port numbers easily.

Hardware that returns zero for both the local port and the peer should just always use gidx=0 for the only peer.

Fixes 5648e56d03fa ("NTB: ntb_perf: Add full multi-port NTB API support")
---
 drivers/ntb/test/ntb_perf.c | 10 ++++++++++
 1 file changed, 10 insertions(+)
drivers/ntb/test/ntb_perf.c +10 -0
index bd0474e62a40..7165f7ab8e35 100644
@@ -1417,6 +1417,16 @@ static int perf_init_peers(struct perf_ctx *perf)
 	if (perf->gidx == -1)
 		perf->gidx = pidx;
 
+	/*
+	 * Hardware with only two ports may not have unique port
+	 * numbers. In this case, the gidxs should all be zero.
+	 */
+	if (perf->pcnt == 1 &&  ntb_port_number(perf->ntb) == 0 &&
+	    ntb_peer_port_number(perf->ntb, 0) == 0) {
+		perf->gidx = 0;
+		perf->peers[0].gidx = 0;
+	}
+
 	for (pidx = 0; pidx < perf->pcnt; pidx++) {
 		ret = perf_setup_peer_mw(&perf->peers[pidx]);
 		if (ret)
Signature · 1 line
2.19.0
#7 lore raw

PATCH v3 7/8 NTB: perf: Fix race condition when run with ntb_test

In reply to Logan Gunthorpe

When running ntb_test, the script tries to run the ntb_perf test immediately after probing the modules. Since adding multi-port support, this fails seeing the new initialization procedure in ntb_perf can not complete instantly.

To fix this we add a completion which is waited on when a test is started. In this way, run can be written any time after the module is loaded and it will wait for the initialization to complete instead of sending an error.

Fixes 5648e56d03fa ("NTB: ntb_perf: Add full multi-port NTB API support")
---
 drivers/ntb/test/ntb_perf.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)
drivers/ntb/test/ntb_perf.c +8 -2
index 7165f7ab8e35..2c11e513cf78 100644
@@ -158,6 +158,8 @@ struct perf_peer {
 	/* NTB connection setup service */
 	struct work_struct	service;
 	unsigned long		sts;
+
+	struct completion init_comp;
 };
 #define to_peer_service(__work) \
 	container_of(__work, struct perf_peer, service)
@@ -549,6 +551,7 @@ static int perf_setup_outbuf(struct perf_peer *peer)
 
 	/* Initialization is finally done */
 	set_bit(PERF_STS_DONE, &peer->sts);
+	complete_all(&peer->init_comp);
 
 	return 0;
 }
@@ -639,6 +642,7 @@ static void perf_service_work(struct work_struct *work)
 		perf_setup_outbuf(peer);
 
 	if (test_and_clear_bit(PERF_CMD_CLEAR, &peer->sts)) {
+		init_completion(&peer->init_comp);
 		clear_bit(PERF_STS_DONE, &peer->sts);
 		if (test_bit(0, &peer->perf->busy_flag) &&
 		    peer == peer->perf->test_peer) {
@@ -1046,8 +1050,9 @@ static int perf_submit_test(struct perf_peer *peer)
 	struct perf_thread *pthr;
 	int tidx, ret;
 
-	if (!test_bit(PERF_STS_DONE, &peer->sts))
-		return -ENOLINK;
+	ret = wait_for_completion_interruptible(&peer->init_comp);
+	if (ret < 0)
+		return ret;
 
 	if (test_and_set_bit_lock(0, &perf->busy_flag))
 		return -EBUSY;
@@ -1413,6 +1418,7 @@ static int perf_init_peers(struct perf_ctx *perf)
 			peer->gidx = pidx;
 		}
 		INIT_WORK(&peer->service, perf_service_work);
+		init_completion(&peer->init_comp);
 	}
 	if (perf->gidx == -1)
 		perf->gidx = pidx;
Signature · 1 line
2.19.0
#8 lore raw

PATCH v3 8/8 NTB: ntb_test: Fix bug when counting remote files

In reply to Logan Gunthorpe

When remote files are counted in get_files_count, without using SSH, the code returns 0 because there is a colon prepended to $LOC. $VPATH should have been used instead of $LOC.

Fixes 06bd0407d06c ("NTB: ntb_test: Update ntb_tool Scratchpad tests")
---
 tools/testing/selftests/ntb/ntb_test.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
tools/testing/selftests/ntb/ntb_test.sh +1 -1
index 08cbfbbc7029..17ca36403d04 100755
@@ -250,7 +250,7 @@ function get_files_count()
 	split_remote $LOC
 
 	if [[ "$REMOTE" == "" ]]; then
-		echo $(ls -1 "$LOC"/${NAME}* 2>/dev/null | wc -l)
+		echo $(ls -1 "$VPATH"/${NAME}* 2>/dev/null | wc -l)
 	else
 		echo $(ssh "$REMOTE" "ls -1 \"$VPATH\"/${NAME}* | \
 		       wc -l" 2> /dev/null)
Signature · 1 line
2.19.0
maintainer
#9 lore raw
In reply to Logan Gunthorpe
Quoted 12 lines · Logan Gunthorpe

Hey,

I'm resending this because I've recently found out that the change we made to use the NTB struct device in DMA allocations is wrong and needs to be reverted. Turns out that, when running with an IOMMU, dma_alloc_coherent() will always fail if you pass it the NTB struct device. This is because the device has not been assigned an IOMMU group and the Intel IOMMU at least expect the devices to be on the PCI bus and be able to find a proper bus-dev-fn number through a struct pci device. Therefore, we must revert the change and I've changed patch 2 to do this and remove the no longer necessary DMA mask adjustments.

For the revert, I think we should cc stable as well.

Quoted 48 lines

I'm not sure if we can get past the impass in getting this series merged:
I still maintain every patch in this series is necessary to fix a regression and there's no way to add port numbers to switchtec in the crosslink configuration so it can't be fixed in the other way that was suggested.

Logan

--

Changes since v2:
- Rebased on v5.0-rc1
- Modify Patch 2 to revert back to using the PCI struct device instead of the NTB struct device in DMA calls
- Collected Allen's Acks
- Collected Alexander's Tested-By

Changes since v1:
- Rebased onto ntb-next (there was a minor conflict in a recent change to the intel driver)
- Collected Dave's Ack

--

Logan Gunthorpe (8):
NTB: ntb_tool: reading the link file should not end in a NULL byte
NTB: Revert the change to use the NTB device dev for DMA allocations
NTB: Fix the default port and peer numbers for legacy drivers
NTB: ntb_pingpong: Choose doorbells based on port number
NTB: perf: Don't require one more memory window than number of peers
NTB: perf: Fix support for hardware that doesn't have port numbers
NTB: perf: Fix race condition when run with ntb_test
NTB: ntb_test: Fix bug when counting remote files

 drivers/ntb/hw/amd/ntb_hw_amd.c         |  4 ----
 drivers/ntb/hw/idt/ntb_hw_idt.c         |  6 -----
 drivers/ntb/hw/intel/ntb_hw_gen1.c      |  4 ----
 drivers/ntb/ntb.c                       |  9 ++------
 drivers/ntb/test/ntb_perf.c             | 29 +++++++++++++++++++------
 drivers/ntb/test/ntb_pingpong.c         | 14 +++++-------
 drivers/ntb/test/ntb_tool.c             |  9 ++++----
 tools/testing/selftests/ntb/ntb_test.sh |  2 +-
 8 files changed, 35 insertions(+), 42 deletions(-)

--
2.19.0

#10 lore raw
Quoted 11 lines · Dave Jiang
Quoted 10 lines

I'm resending this because I've recently found out that the change we made to use the NTB struct device in DMA allocations is wrong and needs to be reverted. Turns out that, when running with an IOMMU, dma_alloc_coherent() will always fail if you pass it the NTB struct device. This is because the device has not been assigned an IOMMU group and the Intel IOMMU at least expect the devices to be on the PCI bus and be able to find a proper bus-dev-fn number through a struct pci device. Therefore, we must revert the change and I've changed patch 2 to do this and remove the no longer necessary DMA mask adjustments.

Quoted 1 line

For the revert, I think we should cc stable as well.

I agree it should go into stable too. But I thought the fixes tag would make that happen...

Logan

maintainer
#11 lore raw
In reply to Logan Gunthorpe
Quoted 13 lines · Logan Gunthorpe

Hey,

I'm resending this because I've recently found out that the change we made to use the NTB struct device in DMA allocations is wrong and needs to be reverted. Turns out that, when running with an IOMMU, dma_alloc_coherent() will always fail if you pass it the NTB struct device. This is because the device has not been assigned an IOMMU group and the Intel IOMMU at least expect the devices to be on the PCI bus and be able to find a proper bus-dev-fn number through a struct pci device. Therefore, we must revert the change and I've changed patch 2 to do this and remove the no longer necessary DMA mask adjustments.

Quoted 5 lines

I'm not sure if we can get past the impass in getting this series merged:
I still maintain every patch in this series is necessary to fix a regression and there's no way to add port numbers to switchtec in the crosslink configuration so it can't be fixed in the other way that was suggested.

Given the need for this to get in (as it does fix actual problems) and the absence of comments on v3, I'm adding this to the ntb-next branch. If there is anyone that has anything more to say on this, please do so immediately. Otherwise, I think this probably needs to be moved to the ntb branch and go into 5.1-rc1.

Thoughts?

Thanks,
Jon

Quoted 41 lines

Logan

--

Changes since v2:
- Rebased on v5.0-rc1
- Modify Patch 2 to revert back to using the PCI struct device instead of the NTB struct device in DMA calls
- Collected Allen's Acks
- Collected Alexander's Tested-By

Changes since v1:
- Rebased onto ntb-next (there was a minor conflict in a recent change to the intel driver)
- Collected Dave's Ack

--

Logan Gunthorpe (8):
NTB: ntb_tool: reading the link file should not end in a NULL byte
NTB: Revert the change to use the NTB device dev for DMA allocations
NTB: Fix the default port and peer numbers for legacy drivers
NTB: ntb_pingpong: Choose doorbells based on port number
NTB: perf: Don't require one more memory window than number of peers
NTB: perf: Fix support for hardware that doesn't have port numbers
NTB: perf: Fix race condition when run with ntb_test
NTB: ntb_test: Fix bug when counting remote files

 drivers/ntb/hw/amd/ntb_hw_amd.c         |  4 ----
 drivers/ntb/hw/idt/ntb_hw_idt.c         |  6 -----
 drivers/ntb/hw/intel/ntb_hw_gen1.c      |  4 ----
 drivers/ntb/ntb.c                       |  9 ++------
 drivers/ntb/test/ntb_perf.c             | 29 +++++++++++++++++++------
 drivers/ntb/test/ntb_pingpong.c         | 14 +++++-------
 drivers/ntb/test/ntb_tool.c             |  9 ++++----
 tools/testing/selftests/ntb/ntb_test.sh |  2 +-
 8 files changed, 35 insertions(+), 42 deletions(-)

--
2.19.0