Skip to contentSkip to conversation

cxl/port: Fix missing port lock in cxl_dport_remove()

PATCH In review Terry Bowman 4 participants 7 messages Jun 8 – Jun 9

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 cxl/port: Fix missing port lock in cxl_dport_remove()

xa_erase() in cxl_dport_remove() runs without the port device lock, creating a race with any caller that does xa_load() on port->dports and then dereferences the returned dport pointer. A concurrent cxl_dport_remove() can erase and free the dport between the xa_load() and the caller acquiring the port lock, causing a use-after-free.

For non-root ports the port lock is already held by the caller on two paths:

1. Driver unbind: devres_release_all() is called from __device_release_driver() which holds port->dev.mutex.

2. Dynamic endpoint removal: cxl_detach_ep() takes the port lock before calling del_dports() -> del_dport() -> devres_release_group(), which synchronously runs cxl_dport_remove().

Use cond_cxl_root_lock/unlock(), which only acquires the port lock when the port is a root port and the lock is therefore not already held. This matches the pattern used in __devm_cxl_add_dport() for the same reason.

The write-side fix to cxl_dport_remove() is necessary but not sufficient. Callers that obtain a dport pointer via cxl_mem_find_port() use a lockless xa_load() and must not dereference that pointer until a lock that excludes free_dport()/kfree() is held.

For root ports, dport_to_host() returns uport_dev, so all three devres actions (free_dport, cxl_dport_remove, cxl_dport_unlink) are registered on uport_dev. __device_release_driver() holds uport_dev->mutex for the full teardown sequence including kfree(dport). Holding uport_dev->mutex on the read side therefore excludes concurrent dport freeing.

Fix rcd_pcie_cap_emit() by passing NULL to cxl_mem_find_port() to avoid capturing a lockless dport pointer, then re-fetching dport inside the uport_dev guard via cxl_find_dport_by_dev(). The previous guard on root->dev was wrong: cxl_dport_remove() releases root->dev before free_dport() runs, so root->dev does not protect against concurrent kfree(dport).

Fix cxl_mem_probe() similarly: pass NULL to cxl_mem_find_port(), then re-fetch dport inside scoped_guard(device, &parent_port->dev) for the VH path, and re-fetch again inside scoped_guard(device, uport_dev) for the RCH path. This closes both the TOCTOU window between the lockless xa_load() and the guard acquisition, and the window between the two sequential guards in the RCH path where a concurrent surprise removal could free dport before devm_cxl_add_endpoint() dereferences it.

Reported-by Sashiko
Fixes 391785859e7e ("cxl/port: Move dport tracking to an xarray")
---
 drivers/cxl/core/port.c | 10 +++++++
 drivers/cxl/mem.c       | 65 +++++++++++++++++++++++++++++++----------
 drivers/cxl/pci.c       | 17 +++++++----
 3 files changed, 72 insertions(+), 20 deletions(-)
drivers/cxl/core/port.c +10 -0
index c5aacd7054f1..0b8f144596e8 100644
@@ -1092,8 +1092,18 @@ static void cxl_dport_remove(void *data)
 	struct cxl_dport *dport = data;
 	struct cxl_port *port = dport->port;
 
+	/*
+	 * For non-root ports the port lock is already held by the caller
+	 * via devres_release_all() during driver unbind, which holds
+	 * port->dev.mutex throughout.  Acquiring it again unconditionally
+	 * would deadlock.  Use cond_cxl_root_lock() which only acquires
+	 * when the port is a root port and the lock is therefore not yet
+	 * held.
+	 */
+	cond_cxl_root_lock(port);
 	port->nr_dports--;
 	xa_erase(&port->dports, (unsigned long) dport->dport_dev);
+	cond_cxl_root_unlock(port);
 	put_device(dport->dport_dev);
 }
 
drivers/cxl/mem.c +50 -15

88 lines not on this page · raw message

drivers/cxl/pci.c +12 -5
index bace662dc988..710a62a66429 100644
@@ -708,10 +708,10 @@ static ssize_t rcd_pcie_cap_emit(struct device *dev, u16 offset, char *buf, size
 {
 	struct cxl_dev_state *cxlds = dev_get_drvdata(dev);
 	struct cxl_memdev *cxlmd = cxlds->cxlmd;
-	struct device *root_dev;
 	struct cxl_dport *dport;
+	struct device *root_dev;
 	struct cxl_port *root __free(put_cxl_port) =
-		cxl_mem_find_port(cxlmd, &dport);
+		cxl_mem_find_port(cxlmd, NULL);
 
 	if (!root)
 		return -ENXIO;
@@ -720,13 +720,20 @@ static ssize_t rcd_pcie_cap_emit(struct device *dev, u16 offset, char *buf, size
 	if (!root_dev)
 		return -ENXIO;
 
-	if (!dport->regs.rcd_pcie_cap)
-		return -ENXIO;
-
 	guard(device)(root_dev);
 	if (!root_dev->driver)
 		return -ENXIO;
 
+	/*
+	 * Fetch dport under uport_dev lock to protect against concurrent
+	 * hotplug removal. uport_dev->mutex is held for the entire devres
+	 * teardown sequence including free_dport(), so holding it here
+	 * excludes concurrent kfree(dport).
+	 */
+	dport = cxl_find_dport_by_dev(root, cxlmd->dev.parent->parent);
+	if (!dport || !dport->regs.rcd_pcie_cap)
+		return -ENXIO;
+
 	switch (width) {
 	case 2:
 		return sysfs_emit(buf, "%#x\n",
Signature · 1 line
2.34.1
#1 lore raw
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:

Pre-existing issues:
- [High] Incomplete fix for use-after-free vulnerabilities. The patch fixes use-after-free races in cxl_mem_probe() and rcd_pcie_cap_emit() by avoiding the dereference of locklessly-obtained dport pointers. However, it missed identical vulnerabilities in cxl_pci_setup_regs() and cxl_handle_rdport_errors(). Both functions use cxl_pci_find_port() to locklessly capture a dport pointer (via xa_load) and then immediately dereference it without holding a device lock (such as uport_dev->mutex) that would exclude concurrent devres teardown. A concurrent surprise removal could cause cxl_dport_remove() and free_dport() to free the dport structure while these functions are dereferencing it, leading to a Use-After-Free.
- [High] Use-After-Free in the error path of devm_cxl_add_nvdimm() due to a cleanup ordering violation. The function holds the device lock via guard(device)(&cxl_nvb->dev). On the err_alloc failure path, it manually calls put_device(&cxl_nvb->dev). If this is the last reference to the device, the object is immediately freed. Upon returning, the guard goes out of scope and calls device_unlock(&cxl_nvb->dev) on the already-freed memory, resulting in a use-after-free.
--

commit 643083e519043c77cd0a30288dae3823f932ab9f
Author: Terry Bowman <[email protected]>

cxl/port: Fix missing port lock in cxl_dport_remove()

This commit addresses a use-after-free race condition in cxl_dport_remove() by properly managing the port lock for root ports. It also updates callers to re-fetch the dport pointer under the appropriate locks.

Quoted 4 lines
drivers/cxl/mem.c +0 -0
index ab88eaa31d1db..419ae58acce81 100644

[ ... ]

Quoted 1 line
drivers/cxl/mem.c +0 -0
@@ -133,21 +132,57 @@ static int cxl_mem_probe(struct device *dev)

[ ... ]

Quoted 15 lines
drivers/cxl/mem.c +15 -0
+	if (rch) {
+		struct device *uport_dev = parent_port->uport_dev;
+
+		scoped_guard(device, uport_dev) {
+			if (!uport_dev->driver) {
+				dev_err(dev, "CXL port topology %s not enabled\n",
+					dev_name(uport_dev));
+				return -ENXIO;
+			}
+			/*
+			 * Re-fetch dport under uport_dev lock.  uport_dev->mutex
+			 * is held for the full devres teardown sequence including
+			 * free_dport()/kfree(), so this excludes concurrent
+			 * hotplug removal through the entire dereference.
+			 */

drivers/cxl/mem.c static int cxl_mem_probe(struct device *dev) · line 176

Quoted 10 lines
drivers/cxl/mem.c +9 -0
+			dport = cxl_find_dport_by_dev(parent_port, cxlmd->dev.parent->parent);
+			if (!dport) {
+				dev_err(dev, "CXL RCH dport not found\n");
+				return -ENXIO;
+			}
+			rc = devm_cxl_add_endpoint(uport_dev, cxlmd, dport);
+			if (rc)
+				return rc;
+		}
 	}

[Severity: High]
This is a pre-existing issue, but while this patch fixes use-after-free races here and in rcd_pcie_cap_emit() by avoiding the dereference of locklessly-obtained dport pointers, could similar vulnerabilities still exist in other functions?

For example, in drivers/cxl/core/pci.c:cxl_pci_setup_regs():

    rc = cxl_rcrb_get_comp_regs(pdev, map, dport);
    ...
    rc = cxl_dport_map_rcd_linkcap(pdev, dport);

And in drivers/cxl/core/ras_rch.c:cxl_handle_rdport_errors():

    if (!cxl_rch_get_aer_info(dport->regs.dport_aer, &aer_regs))

Both functions appear to use cxl_pci_find_port() to locklessly capture a dport pointer (via xa_load) and dereference it without holding a device lock (like uport_dev->mutex) that would exclude concurrent devres teardown. Could a concurrent surprise removal cause cxl_dport_remove() and free_dport() to free the dport structure while these functions are dereferencing it?

[Severity: High]
This is a pre-existing issue, but I noticed a potential use-after-free in the error path of devm_cxl_add_nvdimm(), which is called during cxl_mem_probe().

In drivers/cxl/core/pmem.c:devm_cxl_add_nvdimm():

    guard(device)(&cxl_nvb->dev);
    ...
    err_alloc:
        cxlmd->cxl_nvb = NULL;
        cxlmd->cxl_nvd = NULL;
        put_device(&cxl_nvb->dev);
    
        return rc;

Since guard() releases the device lock when returning, and the error path manually calls put_device(&cxl_nvb->dev) before returning, wouldn't device_unlock() be called on the already-freed memory if this was the last reference, causing a use-after-free and lockdep crash?

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
maintainer
#2 lore raw
In reply to Terry Bowman
Quoted 163 lines · Terry Bowman

xa_erase() in cxl_dport_remove() runs without the port device lock, creating a race with any caller that does xa_load() on port->dports and then dereferences the returned dport pointer. A concurrent cxl_dport_remove() can erase and free the dport between the xa_load() and the caller acquiring the port lock, causing a use-after-free.

For non-root ports the port lock is already held by the caller on two paths:

1. Driver unbind: devres_release_all() is called from __device_release_driver() which holds port->dev.mutex.

2. Dynamic endpoint removal: cxl_detach_ep() takes the port lock before calling del_dports() -> del_dport() -> devres_release_group(), which synchronously runs cxl_dport_remove().

Use cond_cxl_root_lock/unlock(), which only acquires the port lock when the port is a root port and the lock is therefore not already held. This matches the pattern used in __devm_cxl_add_dport() for the same reason.

The write-side fix to cxl_dport_remove() is necessary but not sufficient. Callers that obtain a dport pointer via cxl_mem_find_port() use a lockless xa_load() and must not dereference that pointer until a lock that excludes free_dport()/kfree() is held.

For root ports, dport_to_host() returns uport_dev, so all three devres actions (free_dport, cxl_dport_remove, cxl_dport_unlink) are registered on uport_dev. __device_release_driver() holds uport_dev->mutex for the full teardown sequence including kfree(dport). Holding uport_dev->mutex on the read side therefore excludes concurrent dport freeing.

Fix rcd_pcie_cap_emit() by passing NULL to cxl_mem_find_port() to avoid capturing a lockless dport pointer, then re-fetching dport inside the uport_dev guard via cxl_find_dport_by_dev(). The previous guard on root->dev was wrong: cxl_dport_remove() releases root->dev before free_dport() runs, so root->dev does not protect against concurrent kfree(dport).

Fix cxl_mem_probe() similarly: pass NULL to cxl_mem_find_port(), then re-fetch dport inside scoped_guard(device, &parent_port->dev) for the VH path, and re-fetch again inside scoped_guard(device, uport_dev) for the RCH path. This closes both the TOCTOU window between the lockless xa_load() and the guard acquisition, and the window between the two sequential guards in the RCH path where a concurrent surprise removal could free dport before devm_cxl_add_endpoint() dereferences it.

Reported-by Sashiko
Fixes 391785859e7e ("cxl/port: Move dport tracking to an xarray")
---
 drivers/cxl/core/port.c | 10 +++++++
 drivers/cxl/mem.c       | 65 +++++++++++++++++++++++++++++++----------
 drivers/cxl/pci.c       | 17 +++++++----
 3 files changed, 72 insertions(+), 20 deletions(-)
drivers/cxl/core/port.c +10 -0
index c5aacd7054f1..0b8f144596e8 100644
@@ -1092,8 +1092,18 @@ static void cxl_dport_remove(void *data)
 	struct cxl_dport *dport = data;
 	struct cxl_port *port = dport->port;
 
+	/*
+	 * For non-root ports the port lock is already held by the caller
+	 * via devres_release_all() during driver unbind, which holds
+	 * port->dev.mutex throughout.  Acquiring it again unconditionally
+	 * would deadlock.  Use cond_cxl_root_lock() which only acquires
+	 * when the port is a root port and the lock is therefore not yet
+	 * held.
+	 */
+	cond_cxl_root_lock(port);
 	port->nr_dports--;
 	xa_erase(&port->dports, (unsigned long) dport->dport_dev);
+	cond_cxl_root_unlock(port);
 	put_device(dport->dport_dev);
 }
 
drivers/cxl/mem.c +40 -15
index fcffe24dcb42..345b56f215ff 100644
@@ -70,9 +70,9 @@ static int cxl_mem_probe(struct device *dev)
 	struct cxl_memdev *cxlmd = to_cxl_memdev(dev);
 	struct cxl_memdev_state *mds = to_cxl_memdev_state(cxlmd->cxlds);
 	struct cxl_dev_state *cxlds = cxlmd->cxlds;
-	struct device *endpoint_parent;
 	struct cxl_dport *dport;
 	struct dentry *dentry;
+	bool rch = false;
 	int rc;
 
 	if (!cxlds->media_ready)
@@ -107,8 +107,7 @@ static int cxl_mem_probe(struct device *dev)
 	if (rc)
 		return rc;
 
-	struct cxl_port *parent_port __free(put_cxl_port) =
-		cxl_mem_find_port(cxlmd, &dport);
+	struct cxl_port *parent_port __free(put_cxl_port) = cxl_mem_find_port(cxlmd, NULL);
 	if (!parent_port) {
 		dev_err(dev, "CXL port topology not found\n");
 		return -ENXIO;
@@ -123,21 +122,57 @@ static int cxl_mem_probe(struct device *dev)
 		}
 	}
 
-	if (dport->rch)
-		endpoint_parent = parent_port->uport_dev;
-	else
-		endpoint_parent = &parent_port->dev;
-
-	scoped_guard(device, endpoint_parent) {
-		if (!endpoint_parent->driver) {
-			dev_err(dev, "CXL port topology %s not enabled\n",
-				dev_name(endpoint_parent));
+	scoped_guard(device, &parent_port->dev) {
+		/*
+		 * Re-fetch dport under the port lock to close the TOCTOU
+		 * window between cxl_mem_find_port()'s lockless xa_load() and
+		 * this guard acquisition.  A concurrent surprise removal can
+		 * free the dport in that window.
+		 */
+		dport = cxl_find_dport_by_dev(parent_port, cxlmd->dev.parent->parent);
+		if (!dport) {
+			dev_err(dev, "CXL port topology %s not found\n",
+				dev_name(&parent_port->dev));
 			return -ENXIO;
 		}
+		rch = dport->rch;
+
+		if (!rch) {
+			if (!parent_port->dev.driver) {
+				dev_err(dev, "CXL port topology %s not enabled\n",
+					dev_name(&parent_port->dev));
+				return -ENXIO;
+			}
+			rc = devm_cxl_add_endpoint(&parent_port->dev, cxlmd, dport);
+			if (rc)
+				return rc;
+		}
+	}
 
-		rc = devm_cxl_add_endpoint(endpoint_parent, cxlmd, dport);
-		if (rc)
-			return rc;
+	if (rch) {
+		struct device *uport_dev = parent_port->uport_dev;
+
+		scoped_guard(device, uport_dev) {
+			if (!uport_dev->driver) {
+				dev_err(dev, "CXL port topology %s not enabled\n",
+					dev_name(uport_dev));
+				return -ENXIO;
+			}
+			/*
+			 * Re-fetch dport under uport_dev lock.  uport_dev->mutex
+			 * is held for the full devres teardown sequence including
+			 * free_dport()/kfree(), so this excludes concurrent
+			 * hotplug removal through the entire dereference.

drivers/cxl/mem.c static int cxl_mem_probe(struct device *dev) · line 175

Quoted 10 lines
drivers/cxl/mem.c +10 -0
+			 */
+			dport = cxl_find_dport_by_dev(parent_port, cxlmd->dev.parent->parent);
+			if (!dport) {
+				dev_err(dev, "CXL RCH dport not found\n");
+				return -ENXIO;
+			}
+			rc = devm_cxl_add_endpoint(uport_dev, cxlmd, dport);
+			if (rc)
+				return rc;
+		}

Still reviewing the patch, but thoughts on moving the two new big blocks to a helper function?

DJ

Quoted 44 lines
drivers/cxl/mem.c +0 -0
 	}
 
 	if (cxlmd->attach) {
drivers/cxl/pci.c +12 -5
index bace662dc988..710a62a66429 100644
@@ -708,10 +708,10 @@ static ssize_t rcd_pcie_cap_emit(struct device *dev, u16 offset, char *buf, size
 {
 	struct cxl_dev_state *cxlds = dev_get_drvdata(dev);
 	struct cxl_memdev *cxlmd = cxlds->cxlmd;
-	struct device *root_dev;
 	struct cxl_dport *dport;
+	struct device *root_dev;
 	struct cxl_port *root __free(put_cxl_port) =
-		cxl_mem_find_port(cxlmd, &dport);
+		cxl_mem_find_port(cxlmd, NULL);
 
 	if (!root)
 		return -ENXIO;
@@ -720,13 +720,20 @@ static ssize_t rcd_pcie_cap_emit(struct device *dev, u16 offset, char *buf, size
 	if (!root_dev)
 		return -ENXIO;
 
-	if (!dport->regs.rcd_pcie_cap)
-		return -ENXIO;
-
 	guard(device)(root_dev);
 	if (!root_dev->driver)
 		return -ENXIO;
 
+	/*
+	 * Fetch dport under uport_dev lock to protect against concurrent
+	 * hotplug removal. uport_dev->mutex is held for the entire devres
+	 * teardown sequence including free_dport(), so holding it here
+	 * excludes concurrent kfree(dport).
+	 */
+	dport = cxl_find_dport_by_dev(root, cxlmd->dev.parent->parent);
+	if (!dport || !dport->regs.rcd_pcie_cap)
+		return -ENXIO;
+
 	switch (width) {
 	case 2:
 		return sysfs_emit(buf, "%#x\n",
#3 lore raw
Quoted 24 lines · Dave Jiang
Quoted 21 lines · Terry Bowman

xa_erase() in cxl_dport_remove() runs without the port device lock, creating a race with any caller that does xa_load() on port->dports and then dereferences the returned dport pointer. A concurrent cxl_dport_remove() can erase and free the dport between the xa_load() and the caller acquiring the port lock, causing a use-after-free.

For non-root ports the port lock is already held by the caller on two paths:

1. Driver unbind: devres_release_all() is called from __device_release_driver() which holds port->dev.mutex.

2. Dynamic endpoint removal: cxl_detach_ep() takes the port lock before calling del_dports() -> del_dport() -> devres_release_group(), which synchronously runs cxl_dport_remove().

Use cond_cxl_root_lock/unlock(), which only acquires the port lock when the port is a root port and the lock is therefore not already held. This matches the pattern used in __devm_cxl_add_dport() for the same reason.

Quoted 5 lines
Quoted 5 lines

The write-side fix to cxl_dport_remove() is necessary but not sufficient. Callers that obtain a dport pointer via cxl_mem_find_port() use a lockless xa_load() and must not dereference that pointer until a lock that excludes free_dport()/kfree() is held.

Hi Terry,

I think the mechanism is right, cond_cxl_root_lock() in cxl_dport_remove() is a no-op for non-root ports and a real qcquire only for root dports, so no deadlock.

But this only cover the 2 cxl_mem_find_port() callers. The sibling cxl_pci_find_port() has similar lockless xa_load() plus deref, and those callers aren't fied.

Could you either extend the same fix in this series?
I'm happy to send a follow-up for other readers if that's easier for you.

Quoted 40 lines
Quoted 40 lines

For root ports, dport_to_host() returns uport_dev, so all three devres actions (free_dport, cxl_dport_remove, cxl_dport_unlink) are registered on uport_dev. __device_release_driver() holds uport_dev->mutex for the full teardown sequence including kfree(dport). Holding uport_dev->mutex on the read side therefore excludes concurrent dport freeing.

Fix rcd_pcie_cap_emit() by passing NULL to cxl_mem_find_port() to avoid capturing a lockless dport pointer, then re-fetching dport inside the uport_dev guard via cxl_find_dport_by_dev(). The previous guard on root->dev was wrong: cxl_dport_remove() releases root->dev before free_dport() runs, so root->dev does not protect against concurrent kfree(dport).

Fix cxl_mem_probe() similarly: pass NULL to cxl_mem_find_port(), then re-fetch dport inside scoped_guard(device, &parent_port->dev) for the VH path, and re-fetch again inside scoped_guard(device, uport_dev) for the RCH path. This closes both the TOCTOU window between the lockless xa_load() and the guard acquisition, and the window between the two sequential guards in the RCH path where a concurrent surprise removal could free dport before devm_cxl_add_endpoint() dereferences it.

Reported-by Sashiko
Fixes 391785859e7e ("cxl/port: Move dport tracking to an xarray")
---
 drivers/cxl/core/port.c | 10 +++++++
 drivers/cxl/mem.c       | 65 +++++++++++++++++++++++++++++++----------
 drivers/cxl/pci.c       | 17 +++++++----
 3 files changed, 72 insertions(+), 20 deletions(-)
drivers/cxl/core/port.c +0 -0
index c5aacd7054f1..0b8f144596e8 100644
@@ -1092,8 +1092,18 @@ static void cxl_dport_remove(void *data)
 	struct cxl_dport *dport = data;
 	struct cxl_port *port = dport->port;
 
Quoted 12 lines

drivers/cxl/core/port.c static void cxl_dport_remove(void *data) · line 1106

Quoted 12 lines
drivers/cxl/core/port.c +10 -0
+	/*
+	 * For non-root ports the port lock is already held by the caller
+	 * via devres_release_all() during driver unbind, which holds
+	 * port->dev.mutex throughout.  Acquiring it again unconditionally
+	 * would deadlock.  Use cond_cxl_root_lock() which only acquires
+	 * when the port is a root port and the lock is therefore not yet
+	 * held.
+	 */
+	cond_cxl_root_lock(port);
 	port->nr_dports--;
 	xa_erase(&port->dports, (unsigned long) dport->dport_dev);
+	cond_cxl_root_unlock(port);

In the comment above, maybe worth adding some contents about this is also what makes the RCH reads safe. It's no obvious for me.

Best regards,
Richard Cheng.

Quoted 146 lines
Quoted 85 lines
drivers/cxl/core/port.c +0 -0
 	put_device(dport->dport_dev);
 }
 
drivers/cxl/mem.c +40 -15
index fcffe24dcb42..345b56f215ff 100644
@@ -70,9 +70,9 @@ static int cxl_mem_probe(struct device *dev)
 	struct cxl_memdev *cxlmd = to_cxl_memdev(dev);
 	struct cxl_memdev_state *mds = to_cxl_memdev_state(cxlmd->cxlds);
 	struct cxl_dev_state *cxlds = cxlmd->cxlds;
-	struct device *endpoint_parent;
 	struct cxl_dport *dport;
 	struct dentry *dentry;
+	bool rch = false;
 	int rc;
 
 	if (!cxlds->media_ready)
@@ -107,8 +107,7 @@ static int cxl_mem_probe(struct device *dev)
 	if (rc)
 		return rc;
 
-	struct cxl_port *parent_port __free(put_cxl_port) =
-		cxl_mem_find_port(cxlmd, &dport);
+	struct cxl_port *parent_port __free(put_cxl_port) = cxl_mem_find_port(cxlmd, NULL);
 	if (!parent_port) {
 		dev_err(dev, "CXL port topology not found\n");
 		return -ENXIO;
@@ -123,21 +122,57 @@ static int cxl_mem_probe(struct device *dev)
 		}
 	}
 
-	if (dport->rch)
-		endpoint_parent = parent_port->uport_dev;
-	else
-		endpoint_parent = &parent_port->dev;
-
-	scoped_guard(device, endpoint_parent) {
-		if (!endpoint_parent->driver) {
-			dev_err(dev, "CXL port topology %s not enabled\n",
-				dev_name(endpoint_parent));
+	scoped_guard(device, &parent_port->dev) {
+		/*
+		 * Re-fetch dport under the port lock to close the TOCTOU
+		 * window between cxl_mem_find_port()'s lockless xa_load() and
+		 * this guard acquisition.  A concurrent surprise removal can
+		 * free the dport in that window.
+		 */
+		dport = cxl_find_dport_by_dev(parent_port, cxlmd->dev.parent->parent);
+		if (!dport) {
+			dev_err(dev, "CXL port topology %s not found\n",
+				dev_name(&parent_port->dev));
 			return -ENXIO;
 		}
+		rch = dport->rch;
+
+		if (!rch) {
+			if (!parent_port->dev.driver) {
+				dev_err(dev, "CXL port topology %s not enabled\n",
+					dev_name(&parent_port->dev));
+				return -ENXIO;
+			}
+			rc = devm_cxl_add_endpoint(&parent_port->dev, cxlmd, dport);
+			if (rc)
+				return rc;
+		}
+	}
 
-		rc = devm_cxl_add_endpoint(endpoint_parent, cxlmd, dport);
-		if (rc)
-			return rc;
+	if (rch) {
+		struct device *uport_dev = parent_port->uport_dev;
+
+		scoped_guard(device, uport_dev) {
+			if (!uport_dev->driver) {
+				dev_err(dev, "CXL port topology %s not enabled\n",
+					dev_name(uport_dev));
+				return -ENXIO;
+			}
+			/*
+			 * Re-fetch dport under uport_dev lock.  uport_dev->mutex
+			 * is held for the full devres teardown sequence including
+			 * free_dport()/kfree(), so this excludes concurrent
+			 * hotplug removal through the entire dereference.

drivers/cxl/mem.c static int cxl_mem_probe(struct device *dev) · line 175

Quoted 10 lines
drivers/cxl/mem.c +10 -0
+			 */
+			dport = cxl_find_dport_by_dev(parent_port, cxlmd->dev.parent->parent);
+			if (!dport) {
+				dev_err(dev, "CXL RCH dport not found\n");
+				return -ENXIO;
+			}
+			rc = devm_cxl_add_endpoint(uport_dev, cxlmd, dport);
+			if (rc)
+				return rc;
+		}

Still reviewing the patch, but thoughts on moving the two new big blocks to a helper function?

DJ

Quoted 44 lines
drivers/cxl/mem.c +0 -0
 	}
 
 	if (cxlmd->attach) {
drivers/cxl/pci.c +12 -5
index bace662dc988..710a62a66429 100644
@@ -708,10 +708,10 @@ static ssize_t rcd_pcie_cap_emit(struct device *dev, u16 offset, char *buf, size
 {
 	struct cxl_dev_state *cxlds = dev_get_drvdata(dev);
 	struct cxl_memdev *cxlmd = cxlds->cxlmd;
-	struct device *root_dev;
 	struct cxl_dport *dport;
+	struct device *root_dev;
 	struct cxl_port *root __free(put_cxl_port) =
-		cxl_mem_find_port(cxlmd, &dport);
+		cxl_mem_find_port(cxlmd, NULL);
 
 	if (!root)
 		return -ENXIO;
@@ -720,13 +720,20 @@ static ssize_t rcd_pcie_cap_emit(struct device *dev, u16 offset, char *buf, size
 	if (!root_dev)
 		return -ENXIO;
 
-	if (!dport->regs.rcd_pcie_cap)
-		return -ENXIO;
-
 	guard(device)(root_dev);
 	if (!root_dev->driver)
 		return -ENXIO;
 
+	/*
+	 * Fetch dport under uport_dev lock to protect against concurrent
+	 * hotplug removal. uport_dev->mutex is held for the entire devres
+	 * teardown sequence including free_dport(), so holding it here
+	 * excludes concurrent kfree(dport).
+	 */
+	dport = cxl_find_dport_by_dev(root, cxlmd->dev.parent->parent);
+	if (!dport || !dport->regs.rcd_pcie_cap)
+		return -ENXIO;
+
 	switch (width) {
 	case 2:
 		return sysfs_emit(buf, "%#x\n",
#4 lore raw
Quoted 37 lines · Richard Cheng
Quoted 24 lines · Dave Jiang
Quoted 21 lines · Terry Bowman

xa_erase() in cxl_dport_remove() runs without the port device lock, creating a race with any caller that does xa_load() on port->dports and then dereferences the returned dport pointer. A concurrent cxl_dport_remove() can erase and free the dport between the xa_load() and the caller acquiring the port lock, causing a use-after-free.

For non-root ports the port lock is already held by the caller on two paths:

1. Driver unbind: devres_release_all() is called from __device_release_driver() which holds port->dev.mutex.

2. Dynamic endpoint removal: cxl_detach_ep() takes the port lock before calling del_dports() -> del_dport() -> devres_release_group(), which synchronously runs cxl_dport_remove().

Use cond_cxl_root_lock/unlock(), which only acquires the port lock when the port is a root port and the lock is therefore not already held. This matches the pattern used in __devm_cxl_add_dport() for the same reason.

Quoted 5 lines
Quoted 5 lines

The write-side fix to cxl_dport_remove() is necessary but not sufficient. Callers that obtain a dport pointer via cxl_mem_find_port() use a lockless xa_load() and must not dereference that pointer until a lock that excludes free_dport()/kfree() is held.

Hi Terry,

I think the mechanism is right, cond_cxl_root_lock() in cxl_dport_remove() is a no-op for non-root ports and a real qcquire only for root dports, so no deadlock.

Quoted 7 lines

But this only cover the 2 cxl_mem_find_port() callers. The sibling cxl_pci_find_port() has similar lockless xa_load() plus deref, and those callers aren't fied.

Could you either extend the same fix in this series?
I'm happy to send a follow-up for other readers if that's easier for you.

Youre right there are other callsites requiring similar changes. You're follow-up would helpful. It will allow me to return to the error handling series. Thanks.

Quoted 56 lines
Quoted 40 lines
Quoted 40 lines

For root ports, dport_to_host() returns uport_dev, so all three devres actions (free_dport, cxl_dport_remove, cxl_dport_unlink) are registered on uport_dev. __device_release_driver() holds uport_dev->mutex for the full teardown sequence including kfree(dport). Holding uport_dev->mutex on the read side therefore excludes concurrent dport freeing.

Fix rcd_pcie_cap_emit() by passing NULL to cxl_mem_find_port() to avoid capturing a lockless dport pointer, then re-fetching dport inside the uport_dev guard via cxl_find_dport_by_dev(). The previous guard on root->dev was wrong: cxl_dport_remove() releases root->dev before free_dport() runs, so root->dev does not protect against concurrent kfree(dport).

Fix cxl_mem_probe() similarly: pass NULL to cxl_mem_find_port(), then re-fetch dport inside scoped_guard(device, &parent_port->dev) for the VH path, and re-fetch again inside scoped_guard(device, uport_dev) for the RCH path. This closes both the TOCTOU window between the lockless xa_load() and the guard acquisition, and the window between the two sequential guards in the RCH path where a concurrent surprise removal could free dport before devm_cxl_add_endpoint() dereferences it.

Reported-by Sashiko
Fixes 391785859e7e ("cxl/port: Move dport tracking to an xarray")
---
 drivers/cxl/core/port.c | 10 +++++++
 drivers/cxl/mem.c       | 65 +++++++++++++++++++++++++++++++----------
 drivers/cxl/pci.c       | 17 +++++++----
 3 files changed, 72 insertions(+), 20 deletions(-)
drivers/cxl/core/port.c +0 -0
index c5aacd7054f1..0b8f144596e8 100644
@@ -1092,8 +1092,18 @@ static void cxl_dport_remove(void *data)
 	struct cxl_dport *dport = data;
 	struct cxl_port *port = dport->port;
 
Quoted 12 lines

drivers/cxl/core/port.c static void cxl_dport_remove(void *data) · line 1106

Quoted 12 lines
drivers/cxl/core/port.c +10 -0
+	/*
+	 * For non-root ports the port lock is already held by the caller
+	 * via devres_release_all() during driver unbind, which holds
+	 * port->dev.mutex throughout.  Acquiring it again unconditionally
+	 * would deadlock.  Use cond_cxl_root_lock() which only acquires
+	 * when the port is a root port and the lock is therefore not yet
+	 * held.
+	 */
+	cond_cxl_root_lock(port);
 	port->nr_dports--;
 	xa_erase(&port->dports, (unsigned long) dport->dport_dev);
+	cond_cxl_root_unlock(port);

In the comment above, maybe worth adding some contents about this is also what makes the RCH reads safe. It's no obvious for me.

Quoted 3 lines

Best regards,
Richard Cheng.

For RCH/RCD case, the cond_cxl_root_lock() only synchronizes accesses to the xarray. The RP in the RCH/RCD context is a SW construct. We need to hold the port->uport_dev->mutex to prevent kfree().

- Terry

Quoted 146 lines
Quoted 146 lines
Quoted 85 lines
drivers/cxl/core/port.c +0 -0
 	put_device(dport->dport_dev);
 }
 
drivers/cxl/mem.c +40 -15
index fcffe24dcb42..345b56f215ff 100644
@@ -70,9 +70,9 @@ static int cxl_mem_probe(struct device *dev)
 	struct cxl_memdev *cxlmd = to_cxl_memdev(dev);
 	struct cxl_memdev_state *mds = to_cxl_memdev_state(cxlmd->cxlds);
 	struct cxl_dev_state *cxlds = cxlmd->cxlds;
-	struct device *endpoint_parent;
 	struct cxl_dport *dport;
 	struct dentry *dentry;
+	bool rch = false;
 	int rc;
 
 	if (!cxlds->media_ready)
@@ -107,8 +107,7 @@ static int cxl_mem_probe(struct device *dev)
 	if (rc)
 		return rc;
 
-	struct cxl_port *parent_port __free(put_cxl_port) =
-		cxl_mem_find_port(cxlmd, &dport);
+	struct cxl_port *parent_port __free(put_cxl_port) = cxl_mem_find_port(cxlmd, NULL);
 	if (!parent_port) {
 		dev_err(dev, "CXL port topology not found\n");
 		return -ENXIO;
@@ -123,21 +122,57 @@ static int cxl_mem_probe(struct device *dev)
 		}
 	}
 
-	if (dport->rch)
-		endpoint_parent = parent_port->uport_dev;
-	else
-		endpoint_parent = &parent_port->dev;
-
-	scoped_guard(device, endpoint_parent) {
-		if (!endpoint_parent->driver) {
-			dev_err(dev, "CXL port topology %s not enabled\n",
-				dev_name(endpoint_parent));
+	scoped_guard(device, &parent_port->dev) {
+		/*
+		 * Re-fetch dport under the port lock to close the TOCTOU
+		 * window between cxl_mem_find_port()'s lockless xa_load() and
+		 * this guard acquisition.  A concurrent surprise removal can
+		 * free the dport in that window.
+		 */
+		dport = cxl_find_dport_by_dev(parent_port, cxlmd->dev.parent->parent);
+		if (!dport) {
+			dev_err(dev, "CXL port topology %s not found\n",
+				dev_name(&parent_port->dev));
 			return -ENXIO;
 		}
+		rch = dport->rch;
+
+		if (!rch) {
+			if (!parent_port->dev.driver) {
+				dev_err(dev, "CXL port topology %s not enabled\n",
+					dev_name(&parent_port->dev));
+				return -ENXIO;
+			}
+			rc = devm_cxl_add_endpoint(&parent_port->dev, cxlmd, dport);
+			if (rc)
+				return rc;
+		}
+	}
 
-		rc = devm_cxl_add_endpoint(endpoint_parent, cxlmd, dport);
-		if (rc)
-			return rc;
+	if (rch) {
+		struct device *uport_dev = parent_port->uport_dev;
+
+		scoped_guard(device, uport_dev) {
+			if (!uport_dev->driver) {
+				dev_err(dev, "CXL port topology %s not enabled\n",
+					dev_name(uport_dev));
+				return -ENXIO;
+			}
+			/*
+			 * Re-fetch dport under uport_dev lock.  uport_dev->mutex
+			 * is held for the full devres teardown sequence including
+			 * free_dport()/kfree(), so this excludes concurrent
+			 * hotplug removal through the entire dereference.

drivers/cxl/mem.c static int cxl_mem_probe(struct device *dev) · line 175

Quoted 10 lines
drivers/cxl/mem.c +10 -0
+			 */
+			dport = cxl_find_dport_by_dev(parent_port, cxlmd->dev.parent->parent);
+			if (!dport) {
+				dev_err(dev, "CXL RCH dport not found\n");
+				return -ENXIO;
+			}
+			rc = devm_cxl_add_endpoint(uport_dev, cxlmd, dport);
+			if (rc)
+				return rc;
+		}

Still reviewing the patch, but thoughts on moving the two new big blocks to a helper function?

DJ

Quoted 44 lines
drivers/cxl/mem.c +0 -0
 	}
 
 	if (cxlmd->attach) {
drivers/cxl/pci.c +12 -5
index bace662dc988..710a62a66429 100644
@@ -708,10 +708,10 @@ static ssize_t rcd_pcie_cap_emit(struct device *dev, u16 offset, char *buf, size
 {
 	struct cxl_dev_state *cxlds = dev_get_drvdata(dev);
 	struct cxl_memdev *cxlmd = cxlds->cxlmd;
-	struct device *root_dev;
 	struct cxl_dport *dport;
+	struct device *root_dev;
 	struct cxl_port *root __free(put_cxl_port) =
-		cxl_mem_find_port(cxlmd, &dport);
+		cxl_mem_find_port(cxlmd, NULL);
 
 	if (!root)
 		return -ENXIO;
@@ -720,13 +720,20 @@ static ssize_t rcd_pcie_cap_emit(struct device *dev, u16 offset, char *buf, size
 	if (!root_dev)
 		return -ENXIO;
 
-	if (!dport->regs.rcd_pcie_cap)
-		return -ENXIO;
-
 	guard(device)(root_dev);
 	if (!root_dev->driver)
 		return -ENXIO;
 
+	/*
+	 * Fetch dport under uport_dev lock to protect against concurrent
+	 * hotplug removal. uport_dev->mutex is held for the entire devres
+	 * teardown sequence including free_dport(), so holding it here
+	 * excludes concurrent kfree(dport).
+	 */
+	dport = cxl_find_dport_by_dev(root, cxlmd->dev.parent->parent);
+	if (!dport || !dport->regs.rcd_pcie_cap)
+		return -ENXIO;
+
 	switch (width) {
 	case 2:
 		return sysfs_emit(buf, "%#x\n",
maintainer
#5 lore raw
Quoted 42 lines · Terry
Quoted 41 lines · Richard Cheng
Quoted 24 lines · Dave Jiang
Quoted 21 lines · Terry Bowman

xa_erase() in cxl_dport_remove() runs without the port device lock, creating a race with any caller that does xa_load() on port->dports and then dereferences the returned dport pointer. A concurrent cxl_dport_remove() can erase and free the dport between the xa_load() and the caller acquiring the port lock, causing a use-after-free.

For non-root ports the port lock is already held by the caller on two paths:

1. Driver unbind: devres_release_all() is called from __device_release_driver() which holds port->dev.mutex.

2. Dynamic endpoint removal: cxl_detach_ep() takes the port lock before calling del_dports() -> del_dport() -> devres_release_group(), which synchronously runs cxl_dport_remove().

Use cond_cxl_root_lock/unlock(), which only acquires the port lock when the port is a root port and the lock is therefore not already held. This matches the pattern used in __devm_cxl_add_dport() for the same reason.

Quoted 5 lines
Quoted 5 lines

The write-side fix to cxl_dport_remove() is necessary but not sufficient. Callers that obtain a dport pointer via cxl_mem_find_port() use a lockless xa_load() and must not dereference that pointer until a lock that excludes free_dport()/kfree() is held.

Hi Terry,

I think the mechanism is right, cond_cxl_root_lock() in cxl_dport_remove() is a no-op for non-root ports and a real qcquire only for root dports, so no deadlock.

But this only cover the 2 cxl_mem_find_port() callers. The sibling cxl_pci_find_port() has similar lockless xa_load() plus deref, and those callers aren't fied.

Quoted 7 lines
Quoted 3 lines

Could you either extend the same fix in this series?
I'm happy to send a follow-up for other readers if that's easier for you.

Youre right there are other callsites requiring similar changes. You're follow-up would helpful. It will allow me to return to the error handling series. Thanks.

Richard,
Do you mind just bundle Terry's fix with your series and we can apply it all together? Thanks!

DJ

Quoted 215 lines
Quoted 57 lines
Quoted 41 lines
Quoted 41 lines

For root ports, dport_to_host() returns uport_dev, so all three devres actions (free_dport, cxl_dport_remove, cxl_dport_unlink) are registered on uport_dev. __device_release_driver() holds uport_dev->mutex for the full teardown sequence including kfree(dport). Holding uport_dev->mutex on the read side therefore excludes concurrent dport freeing.

Fix rcd_pcie_cap_emit() by passing NULL to cxl_mem_find_port() to avoid capturing a lockless dport pointer, then re-fetching dport inside the uport_dev guard via cxl_find_dport_by_dev(). The previous guard on root->dev was wrong: cxl_dport_remove() releases root->dev before free_dport() runs, so root->dev does not protect against concurrent kfree(dport).

Fix cxl_mem_probe() similarly: pass NULL to cxl_mem_find_port(), then re-fetch dport inside scoped_guard(device, &parent_port->dev) for the VH path, and re-fetch again inside scoped_guard(device, uport_dev) for the RCH path. This closes both the TOCTOU window between the lockless xa_load() and the guard acquisition, and the window between the two sequential guards in the RCH path where a concurrent surprise removal could free dport before devm_cxl_add_endpoint() dereferences it.

Reported-by Sashiko
Fixes 391785859e7e ("cxl/port: Move dport tracking to an xarray")
---
 drivers/cxl/core/port.c | 10 +++++++
 drivers/cxl/mem.c       | 65 +++++++++++++++++++++++++++++++----------
 drivers/cxl/pci.c       | 17 +++++++----
 3 files changed, 72 insertions(+), 20 deletions(-)
drivers/cxl/core/port.c +0 -0
index c5aacd7054f1..0b8f144596e8 100644
@@ -1092,8 +1092,18 @@ static void cxl_dport_remove(void *data)
 	struct cxl_dport *dport = data;
 	struct cxl_port *port = dport->port;
 
Quoted 12 lines

drivers/cxl/core/port.c static void cxl_dport_remove(void *data) · line 1106

Quoted 12 lines
drivers/cxl/core/port.c +10 -0
+	/*
+	 * For non-root ports the port lock is already held by the caller
+	 * via devres_release_all() during driver unbind, which holds
+	 * port->dev.mutex throughout.  Acquiring it again unconditionally
+	 * would deadlock.  Use cond_cxl_root_lock() which only acquires
+	 * when the port is a root port and the lock is therefore not yet
+	 * held.
+	 */
+	cond_cxl_root_lock(port);
 	port->nr_dports--;
 	xa_erase(&port->dports, (unsigned long) dport->dport_dev);
+	cond_cxl_root_unlock(port);

In the comment above, maybe worth adding some contents about this is also what makes the RCH reads safe. It's no obvious for me.

Quoted 3 lines

Best regards,
Richard Cheng.

For RCH/RCD case, the cond_cxl_root_lock() only synchronizes accesses to the xarray. The RP in the RCH/RCD context is a SW construct. We need to hold the port->uport_dev->mutex to prevent kfree().

- Terry

Quoted 146 lines
Quoted 146 lines
Quoted 85 lines
drivers/cxl/core/port.c +0 -0
 	put_device(dport->dport_dev);
 }
 
drivers/cxl/mem.c +40 -15
index fcffe24dcb42..345b56f215ff 100644
@@ -70,9 +70,9 @@ static int cxl_mem_probe(struct device *dev)
 	struct cxl_memdev *cxlmd = to_cxl_memdev(dev);
 	struct cxl_memdev_state *mds = to_cxl_memdev_state(cxlmd->cxlds);
 	struct cxl_dev_state *cxlds = cxlmd->cxlds;
-	struct device *endpoint_parent;
 	struct cxl_dport *dport;
 	struct dentry *dentry;
+	bool rch = false;
 	int rc;
 
 	if (!cxlds->media_ready)
@@ -107,8 +107,7 @@ static int cxl_mem_probe(struct device *dev)
 	if (rc)
 		return rc;
 
-	struct cxl_port *parent_port __free(put_cxl_port) =
-		cxl_mem_find_port(cxlmd, &dport);
+	struct cxl_port *parent_port __free(put_cxl_port) = cxl_mem_find_port(cxlmd, NULL);
 	if (!parent_port) {
 		dev_err(dev, "CXL port topology not found\n");
 		return -ENXIO;
@@ -123,21 +122,57 @@ static int cxl_mem_probe(struct device *dev)
 		}
 	}
 
-	if (dport->rch)
-		endpoint_parent = parent_port->uport_dev;
-	else
-		endpoint_parent = &parent_port->dev;
-
-	scoped_guard(device, endpoint_parent) {
-		if (!endpoint_parent->driver) {
-			dev_err(dev, "CXL port topology %s not enabled\n",
-				dev_name(endpoint_parent));
+	scoped_guard(device, &parent_port->dev) {
+		/*
+		 * Re-fetch dport under the port lock to close the TOCTOU
+		 * window between cxl_mem_find_port()'s lockless xa_load() and
+		 * this guard acquisition.  A concurrent surprise removal can
+		 * free the dport in that window.
+		 */
+		dport = cxl_find_dport_by_dev(parent_port, cxlmd->dev.parent->parent);
+		if (!dport) {
+			dev_err(dev, "CXL port topology %s not found\n",
+				dev_name(&parent_port->dev));
 			return -ENXIO;
 		}
+		rch = dport->rch;
+
+		if (!rch) {
+			if (!parent_port->dev.driver) {
+				dev_err(dev, "CXL port topology %s not enabled\n",
+					dev_name(&parent_port->dev));
+				return -ENXIO;
+			}
+			rc = devm_cxl_add_endpoint(&parent_port->dev, cxlmd, dport);
+			if (rc)
+				return rc;
+		}
+	}
 
-		rc = devm_cxl_add_endpoint(endpoint_parent, cxlmd, dport);
-		if (rc)
-			return rc;
+	if (rch) {
+		struct device *uport_dev = parent_port->uport_dev;
+
+		scoped_guard(device, uport_dev) {
+			if (!uport_dev->driver) {
+				dev_err(dev, "CXL port topology %s not enabled\n",
+					dev_name(uport_dev));
+				return -ENXIO;
+			}
+			/*
+			 * Re-fetch dport under uport_dev lock.  uport_dev->mutex
+			 * is held for the full devres teardown sequence including
+			 * free_dport()/kfree(), so this excludes concurrent
+			 * hotplug removal through the entire dereference.

drivers/cxl/mem.c static int cxl_mem_probe(struct device *dev) · line 175

Quoted 10 lines
drivers/cxl/mem.c +10 -0
+			 */
+			dport = cxl_find_dport_by_dev(parent_port, cxlmd->dev.parent->parent);
+			if (!dport) {
+				dev_err(dev, "CXL RCH dport not found\n");
+				return -ENXIO;
+			}
+			rc = devm_cxl_add_endpoint(uport_dev, cxlmd, dport);
+			if (rc)
+				return rc;
+		}

Still reviewing the patch, but thoughts on moving the two new big blocks to a helper function?

DJ

Quoted 44 lines
drivers/cxl/mem.c +0 -0
 	}
 
 	if (cxlmd->attach) {
drivers/cxl/pci.c +12 -5
index bace662dc988..710a62a66429 100644
@@ -708,10 +708,10 @@ static ssize_t rcd_pcie_cap_emit(struct device *dev, u16 offset, char *buf, size
 {
 	struct cxl_dev_state *cxlds = dev_get_drvdata(dev);
 	struct cxl_memdev *cxlmd = cxlds->cxlmd;
-	struct device *root_dev;
 	struct cxl_dport *dport;
+	struct device *root_dev;
 	struct cxl_port *root __free(put_cxl_port) =
-		cxl_mem_find_port(cxlmd, &dport);
+		cxl_mem_find_port(cxlmd, NULL);
 
 	if (!root)
 		return -ENXIO;
@@ -720,13 +720,20 @@ static ssize_t rcd_pcie_cap_emit(struct device *dev, u16 offset, char *buf, size
 	if (!root_dev)
 		return -ENXIO;
 
-	if (!dport->regs.rcd_pcie_cap)
-		return -ENXIO;
-
 	guard(device)(root_dev);
 	if (!root_dev->driver)
 		return -ENXIO;
 
+	/*
+	 * Fetch dport under uport_dev lock to protect against concurrent
+	 * hotplug removal. uport_dev->mutex is held for the entire devres
+	 * teardown sequence including free_dport(), so holding it here
+	 * excludes concurrent kfree(dport).
+	 */
+	dport = cxl_find_dport_by_dev(root, cxlmd->dev.parent->parent);
+	if (!dport || !dport->regs.rcd_pcie_cap)
+		return -ENXIO;
+
 	switch (width) {
 	case 2:
 		return sysfs_emit(buf, "%#x\n",
#6 lore raw
In reply to Dave Jiang
Quoted 179 lines · Dave Jiang
Quoted 163 lines · Terry Bowman

xa_erase() in cxl_dport_remove() runs without the port device lock, creating a race with any caller that does xa_load() on port->dports and then dereferences the returned dport pointer. A concurrent cxl_dport_remove() can erase and free the dport between the xa_load() and the caller acquiring the port lock, causing a use-after-free.

For non-root ports the port lock is already held by the caller on two paths:

1. Driver unbind: devres_release_all() is called from __device_release_driver() which holds port->dev.mutex.

2. Dynamic endpoint removal: cxl_detach_ep() takes the port lock before calling del_dports() -> del_dport() -> devres_release_group(), which synchronously runs cxl_dport_remove().

Use cond_cxl_root_lock/unlock(), which only acquires the port lock when the port is a root port and the lock is therefore not already held. This matches the pattern used in __devm_cxl_add_dport() for the same reason.

The write-side fix to cxl_dport_remove() is necessary but not sufficient. Callers that obtain a dport pointer via cxl_mem_find_port() use a lockless xa_load() and must not dereference that pointer until a lock that excludes free_dport()/kfree() is held.

For root ports, dport_to_host() returns uport_dev, so all three devres actions (free_dport, cxl_dport_remove, cxl_dport_unlink) are registered on uport_dev. __device_release_driver() holds uport_dev->mutex for the full teardown sequence including kfree(dport). Holding uport_dev->mutex on the read side therefore excludes concurrent dport freeing.

Fix rcd_pcie_cap_emit() by passing NULL to cxl_mem_find_port() to avoid capturing a lockless dport pointer, then re-fetching dport inside the uport_dev guard via cxl_find_dport_by_dev(). The previous guard on root->dev was wrong: cxl_dport_remove() releases root->dev before free_dport() runs, so root->dev does not protect against concurrent kfree(dport).

Fix cxl_mem_probe() similarly: pass NULL to cxl_mem_find_port(), then re-fetch dport inside scoped_guard(device, &parent_port->dev) for the VH path, and re-fetch again inside scoped_guard(device, uport_dev) for the RCH path. This closes both the TOCTOU window between the lockless xa_load() and the guard acquisition, and the window between the two sequential guards in the RCH path where a concurrent surprise removal could free dport before devm_cxl_add_endpoint() dereferences it.

Reported-by Sashiko
Fixes 391785859e7e ("cxl/port: Move dport tracking to an xarray")
---
 drivers/cxl/core/port.c | 10 +++++++
 drivers/cxl/mem.c       | 65 +++++++++++++++++++++++++++++++----------
 drivers/cxl/pci.c       | 17 +++++++----
 3 files changed, 72 insertions(+), 20 deletions(-)
drivers/cxl/core/port.c +10 -0
index c5aacd7054f1..0b8f144596e8 100644
@@ -1092,8 +1092,18 @@ static void cxl_dport_remove(void *data)
 	struct cxl_dport *dport = data;
 	struct cxl_port *port = dport->port;
 
+	/*
+	 * For non-root ports the port lock is already held by the caller
+	 * via devres_release_all() during driver unbind, which holds
+	 * port->dev.mutex throughout.  Acquiring it again unconditionally
+	 * would deadlock.  Use cond_cxl_root_lock() which only acquires
+	 * when the port is a root port and the lock is therefore not yet
+	 * held.
+	 */
+	cond_cxl_root_lock(port);
 	port->nr_dports--;
 	xa_erase(&port->dports, (unsigned long) dport->dport_dev);
+	cond_cxl_root_unlock(port);
 	put_device(dport->dport_dev);
 }
 
drivers/cxl/mem.c +40 -15
index fcffe24dcb42..345b56f215ff 100644
@@ -70,9 +70,9 @@ static int cxl_mem_probe(struct device *dev)
 	struct cxl_memdev *cxlmd = to_cxl_memdev(dev);
 	struct cxl_memdev_state *mds = to_cxl_memdev_state(cxlmd->cxlds);
 	struct cxl_dev_state *cxlds = cxlmd->cxlds;
-	struct device *endpoint_parent;
 	struct cxl_dport *dport;
 	struct dentry *dentry;
+	bool rch = false;
 	int rc;
 
 	if (!cxlds->media_ready)
@@ -107,8 +107,7 @@ static int cxl_mem_probe(struct device *dev)
 	if (rc)
 		return rc;
 
-	struct cxl_port *parent_port __free(put_cxl_port) =
-		cxl_mem_find_port(cxlmd, &dport);
+	struct cxl_port *parent_port __free(put_cxl_port) = cxl_mem_find_port(cxlmd, NULL);
 	if (!parent_port) {
 		dev_err(dev, "CXL port topology not found\n");
 		return -ENXIO;
@@ -123,21 +122,57 @@ static int cxl_mem_probe(struct device *dev)
 		}
 	}
 
-	if (dport->rch)
-		endpoint_parent = parent_port->uport_dev;
-	else
-		endpoint_parent = &parent_port->dev;
-
-	scoped_guard(device, endpoint_parent) {
-		if (!endpoint_parent->driver) {
-			dev_err(dev, "CXL port topology %s not enabled\n",
-				dev_name(endpoint_parent));
+	scoped_guard(device, &parent_port->dev) {
+		/*
+		 * Re-fetch dport under the port lock to close the TOCTOU
+		 * window between cxl_mem_find_port()'s lockless xa_load() and
+		 * this guard acquisition.  A concurrent surprise removal can
+		 * free the dport in that window.
+		 */
+		dport = cxl_find_dport_by_dev(parent_port, cxlmd->dev.parent->parent);
+		if (!dport) {
+			dev_err(dev, "CXL port topology %s not found\n",
+				dev_name(&parent_port->dev));
 			return -ENXIO;
 		}
+		rch = dport->rch;
+
+		if (!rch) {
+			if (!parent_port->dev.driver) {
+				dev_err(dev, "CXL port topology %s not enabled\n",
+					dev_name(&parent_port->dev));
+				return -ENXIO;
+			}
+			rc = devm_cxl_add_endpoint(&parent_port->dev, cxlmd, dport);
+			if (rc)
+				return rc;
+		}
+	}
 
-		rc = devm_cxl_add_endpoint(endpoint_parent, cxlmd, dport);
-		if (rc)
-			return rc;
+	if (rch) {
+		struct device *uport_dev = parent_port->uport_dev;
+
+		scoped_guard(device, uport_dev) {
+			if (!uport_dev->driver) {
+				dev_err(dev, "CXL port topology %s not enabled\n",
+					dev_name(uport_dev));
+				return -ENXIO;
+			}
+			/*
+			 * Re-fetch dport under uport_dev lock.  uport_dev->mutex
+			 * is held for the full devres teardown sequence including
+			 * free_dport()/kfree(), so this excludes concurrent
+			 * hotplug removal through the entire dereference.

drivers/cxl/mem.c static int cxl_mem_probe(struct device *dev) · line 175

Quoted 10 lines
drivers/cxl/mem.c +10 -0
+			 */
+			dport = cxl_find_dport_by_dev(parent_port, cxlmd->dev.parent->parent);
+			if (!dport) {
+				dev_err(dev, "CXL RCH dport not found\n");
+				return -ENXIO;
+			}
+			rc = devm_cxl_add_endpoint(uport_dev, cxlmd, dport);
+			if (rc)
+				return rc;
+		}

Still reviewing the patch, but thoughts on moving the two new big blocks to a helper function?

Quoted 2 lines

DJ

Hi Dave,

Yes, good point. That needs to be done.

- Terry