Skip to contentSkip to conversation

dmaengine: idxd: Do not enable user type Work Queue without Shared Virtual Addressing

PATCH Applied Fenghua Yu 4 participants 4 messages Oct 14, 2022 – Oct 19, 2022

Subsystems (3)
lore
Share & tools
lore

Fenghua Yu

author
#0 lore raw

PATCH v2 1/1 dmaengine: idxd: Do not enable user type Work Queue without Shared Virtual Addressing

When the idxd_user_drv driver is bound to a Work Queue (WQ) device without IOMMU or with IOMMU Passthrough without Shared Virtual Addressing (SVA), the application gains direct access to physical memory via the device by programming physical address to a submitted descriptor. This allows direct userspace read and write access to arbitrary physical memory. This is inconsistent with the security goals of a good kernel API.

Unlike vfio_pci driver, the IDXD char device driver does not provide any ways to pin user pages and translate the address from user VA to IOVA or PA without IOMMU SVA. Therefore the application has no way to instruct the device to perform DMA function. This makes the char device not usable for normal application usage.

Since user type WQ without SVA cannot be used for normal application usage and presents the security issue, bind idxd_user_drv driver and enable user type WQ only when SVA is enabled (i.e. user PASID is enabled).

Fixes 448c3de8ac83 ("dmaengine: idxd: create user driver for wq 'device'")

---
v2:
- Update changlog per Dave Hansen's comments

 drivers/dma/idxd/cdev.c   | 18 ++++++++++++++++++
 include/uapi/linux/idxd.h |  1 +
 2 files changed, 19 insertions(+)
drivers/dma/idxd/cdev.c +18 -0
index c2808fd081d6..a9b96b18772f 100644
@@ -312,6 +312,24 @@ static int idxd_user_drv_probe(struct idxd_dev *idxd_dev)
 	if (idxd->state != IDXD_DEV_ENABLED)
 		return -ENXIO;
 
+	/*
+	 * User type WQ is enabled only when SVA is enabled for two reasons:
+	 *   - If no IOMMU or IOMMU Passthrough without SVA, userspace
+	 *     can directly access physical address through the WQ.
+	 *   - The IDXD cdev driver does not provide any ways to pin
+	 *     user pages and translate the address from user VA to IOVA or
+	 *     PA without IOMMU SVA. Therefore the application has no way
+	 *     to instruct the device to perform DMA function. This makes
+	 *     the cdev not usable for normal application usage.
+	 */
+	if (!device_user_pasid_enabled(idxd)) {
+		idxd->cmd_status = IDXD_SCMD_WQ_USER_NO_IOMMU;
+		dev_dbg(&idxd->pdev->dev,
+			"User type WQ cannot be enabled without SVA.\n");
+
+		return -EOPNOTSUPP;
+	}
+
 	mutex_lock(&wq->wq_lock);
 	wq->type = IDXD_WQT_USER;
 	rc = drv_enable_wq(wq);
include/uapi/linux/idxd.h +1 -0
index 095299c75828..2b9e7feba3f3 100644
@@ -29,6 +29,7 @@ enum idxd_scmd_stat {
 	IDXD_SCMD_WQ_NO_SIZE = 0x800e0000,
 	IDXD_SCMD_WQ_NO_PRIV = 0x800f0000,
 	IDXD_SCMD_WQ_IRQ_ERR = 0x80100000,
+	IDXD_SCMD_WQ_USER_NO_IOMMU = 0x80110000,
 };
 
 #define IDXD_SCMD_SOFTERR_MASK	0x80000000
Signature · 1 line
2.32.0
Reviewed-by on v2 1/1
#1 lore raw
Quoted 18 lines · Fenghua Yu

When the idxd_user_drv driver is bound to a Work Queue (WQ) device without IOMMU or with IOMMU Passthrough without Shared Virtual Addressing (SVA), the application gains direct access to physical memory via the device by programming physical address to a submitted descriptor. This allows direct userspace read and write access to arbitrary physical memory. This is inconsistent with the security goals of a good kernel API.

Unlike vfio_pci driver, the IDXD char device driver does not provide any ways to pin user pages and translate the address from user VA to IOVA or PA without IOMMU SVA. Therefore the application has no way to instruct the device to perform DMA function. This makes the char device not usable for normal application usage.

Since user type WQ without SVA cannot be used for normal application usage and presents the security issue, bind idxd_user_drv driver and enable user type WQ only when SVA is enabled (i.e. user PASID is enabled).

Quoted 5 lines
Fixes 448c3de8ac83 ("dmaengine: idxd: create user driver for wq 'device'")
Quoted 52 lines

---
v2:
- Update changlog per Dave Hansen's comments

 drivers/dma/idxd/cdev.c   | 18 ++++++++++++++++++
 include/uapi/linux/idxd.h |  1 +
 2 files changed, 19 insertions(+)
drivers/dma/idxd/cdev.c +18 -0
index c2808fd081d6..a9b96b18772f 100644
@@ -312,6 +312,24 @@ static int idxd_user_drv_probe(struct idxd_dev *idxd_dev)
 	if (idxd->state != IDXD_DEV_ENABLED)
 		return -ENXIO;
 
+	/*
+	 * User type WQ is enabled only when SVA is enabled for two reasons:
+	 *   - If no IOMMU or IOMMU Passthrough without SVA, userspace
+	 *     can directly access physical address through the WQ.
+	 *   - The IDXD cdev driver does not provide any ways to pin
+	 *     user pages and translate the address from user VA to IOVA or
+	 *     PA without IOMMU SVA. Therefore the application has no way
+	 *     to instruct the device to perform DMA function. This makes
+	 *     the cdev not usable for normal application usage.
+	 */
+	if (!device_user_pasid_enabled(idxd)) {
+		idxd->cmd_status = IDXD_SCMD_WQ_USER_NO_IOMMU;
+		dev_dbg(&idxd->pdev->dev,
+			"User type WQ cannot be enabled without SVA.\n");
+
+		return -EOPNOTSUPP;
+	}
+
 	mutex_lock(&wq->wq_lock);
 	wq->type = IDXD_WQT_USER;
 	rc = drv_enable_wq(wq);
include/uapi/linux/idxd.h +1 -0
index 095299c75828..2b9e7feba3f3 100644
@@ -29,6 +29,7 @@ enum idxd_scmd_stat {
 	IDXD_SCMD_WQ_NO_SIZE = 0x800e0000,
 	IDXD_SCMD_WQ_NO_PRIV = 0x800f0000,
 	IDXD_SCMD_WQ_IRQ_ERR = 0x80100000,
+	IDXD_SCMD_WQ_USER_NO_IOMMU = 0x80110000,
 };
 
 #define IDXD_SCMD_SOFTERR_MASK	0x80000000
Signature · 1 line
2.32.0
#2 lore raw
In reply to Fenghua Yu

drivers/dma/idxd/cdev.c static int idxd_user_drv_probe(struct idxd_dev *idxd_dev) · line 317

Quoted 7 lines

From: Fenghua Yu <[email protected]>
Sent: Saturday, October 15, 2022 6:26 AM

(quoted diff) +2 -0
+	/*
+	 * User type WQ is enabled only when SVA is enabled for two

reasons:

drivers/dma/idxd/cdev.c +1 -0
+	 *   - If no IOMMU or IOMMU Passthrough without SVA, userspace

This statement is kind of misleading. Even if IOMMU is in DMA mode user type WQ still doesn't work w/o SVA.

drivers/dma/idxd/cdev.c static int idxd_user_drv_probe(struct idxd_dev *idxd_dev) · line 324

Quoted 7 lines
drivers/dma/idxd/cdev.c +7 -0
+	 *     can directly access physical address through the WQ.
+	 *   - The IDXD cdev driver does not provide any ways to pin
+	 *     user pages and translate the address from user VA to IOVA or
+	 *     PA without IOMMU SVA. Therefore the application has no way
+	 *     to instruct the device to perform DMA function. This makes
+	 *     the cdev not usable for normal application usage.
+	 */

It could be simply stated as "SVA is the only secure/reliable way for the device to access user space memory"

drivers/dma/idxd/cdev.c static int idxd_user_drv_probe(struct idxd_dev *idxd_dev) · line 326

Quoted 2 lines
drivers/dma/idxd/cdev.c +2 -0
+	if (!device_user_pasid_enabled(idxd)) {
+		idxd->cmd_status = IDXD_SCMD_WQ_USER_NO_IOMMU;

be specific i.e. IDXD_SCMD_WQ_USER_NO_SVA

drivers/dma/idxd/cdev.c static int idxd_user_drv_probe(struct idxd_dev *idxd_dev) · line 332

Quoted 7 lines
drivers/dma/idxd/cdev.c +6 -0
+		dev_dbg(&idxd->pdev->dev,
+			"User type WQ cannot be enabled without SVA.\n");
+
+		return -EOPNOTSUPP;
+	}
+

with above change the check on pasid_enabled should be removed from idxd_cdev_open() and idxd_cdev_release(). They should always work with SVA enabled.

maintainer Applied
#3 lore raw
In reply to Fenghua Yu
Quoted 14 lines · Fenghua Yu

When the idxd_user_drv driver is bound to a Work Queue (WQ) device without IOMMU or with IOMMU Passthrough without Shared Virtual Addressing (SVA), the application gains direct access to physical memory via the device by programming physical address to a submitted descriptor. This allows direct userspace read and write access to arbitrary physical memory. This is inconsistent with the security goals of a good kernel API.

Unlike vfio_pci driver, the IDXD char device driver does not provide any ways to pin user pages and translate the address from user VA to IOVA or PA without IOMMU SVA. Therefore the application has no way to instruct the device to perform DMA function. This makes the char device not usable for normal application usage.

Quoted 3 lines

Since user type WQ without SVA cannot be used for normal application usage and presents the security issue, bind idxd_user_drv driver and enable user type WQ only when SVA is enabled (i.e. user PASID is enabled).

Applied, thanks

Signature · 1 line
~Vinod