Skip to contentSkip to conversation

iommu/amd: SVA Support (Part 4) - SVA and IOPF

PATCH Superseded Vasant Hegde 1 participant 11 messages Aug 23, 2023

Subsystems (4)

v1 of 8 Superseded Latest is v8 →

lore
Share & tools
lore

Vasant Hegde

author maintainer
#0 lore raw

cover iommu/amd: SVA Support (Part 4) - SVA and IOPF

This is part 4 of the 4-part series to introduce Share Virtual Address (SVA) support for devices, which can support PCI ATS, PASID and PRI capabilities. These devices are referred to as SVA-capable devices in this series.

Patch series organization
-------------------------

This series is organized as following:

* Patch 1 - 2:
Rename / add support to enable/disable features

* Patch 3 - 4:
Introduce SVA support

* Patch 5 - 10:
Add IOPF support

This patch series is based on top of SVA Part 3 [1].
[1] https://lore.kernel.org/linux-iommu/[email protected]/

This is also available at github :
https://github.com/AMDESE/linux/tree/iommu_sva_part4_v1_v6.5_rc1

Thank you,
Vasant / Suravee

Suravee Suthikulpanit (5):
iommu/amd: Add support to enable/disable SVA feature iommu/amd: Move PPR-related functions into ppr.c iommu/amd: Define per-IOMMU iopf_queue iommu/amd: Add support for page response iommu/amd: Introduce logic to enable/disable IOPF

Vasant Hegde (2):
iommu/amd: Rename amd_iommu_v2_supported() as amd_iommu_sva_supported() iommu/amd: Initial SVA support for AMD IOMMU

Wei Huang (3):
iommu/amd: Add support for enabling/disabling IOMMU features iommu/amd: Add support for add/remove device for IOPF iommu/amd: Add IO page fault notifier handler

 drivers/iommu/amd/Kconfig           |   1 +
 drivers/iommu/amd/Makefile          |   2 +-
 drivers/iommu/amd/amd_iommu.h       |  41 ++-
 drivers/iommu/amd/amd_iommu_types.h |  22 ++
 drivers/iommu/amd/init.c            |  72 ++----
 drivers/iommu/amd/iommu.c           | 170 ++++++++-----
 drivers/iommu/amd/ppr.c             | 377 ++++++++++++++++++++++++++++
 drivers/iommu/amd/sva.c             | 328 ++++++++++++++++++++++++
 8 files changed, 894 insertions(+), 119 deletions(-)
 create mode 100644 drivers/iommu/amd/ppr.c
 create mode 100644 drivers/iommu/amd/sva.c
Signature · 1 line
2.31.1
author maintainer
#1 lore raw

PATCH 1/10 dma: Add SDXI device support

From: Wei Huang <[email protected]>

Smart Data Accelerator Interface (SDXI) DMA controller provides DMA capabilities, plus other advanced data processing features, that can offload the work for CPUs under different address spaces. This patch adds SDXI device driver support.

---
 drivers/dma/Kconfig        |   2 +
 drivers/dma/Makefile       |   1 +
 drivers/dma/sdxi/Kconfig   |  11 +
 drivers/dma/sdxi/Makefile  |   3 +
 drivers/dma/sdxi/core.c    | 428 +++++++++++++++++++++++++++++
 drivers/dma/sdxi/dma.c     | 441 ++++++++++++++++++++++++++++++
 drivers/dma/sdxi/dma.h     |  23 ++
 drivers/dma/sdxi/ioctl.c   | 440 ++++++++++++++++++++++++++++++
 drivers/dma/sdxi/ioctl.h   |  49 ++++
 drivers/dma/sdxi/pci.c     | 533 +++++++++++++++++++++++++++++++++++++
 drivers/dma/sdxi/pci.h     | 130 +++++++++
 drivers/dma/sdxi/process.c | 285 ++++++++++++++++++++
 drivers/dma/sdxi/process.h |  71 +++++
 drivers/dma/sdxi/sdxi.h    | 414 ++++++++++++++++++++++++++++
 drivers/dma/sdxi/sq.c      | 295 ++++++++++++++++++++
 drivers/dma/sdxi/sq.h      |  67 +++++
 drivers/dma/sdxi/trace.c   |  10 +
 drivers/dma/sdxi/trace.h   | 115 ++++++++
 include/linux/pci_ids.h    |   2 +
 include/uapi/linux/sdxi.h  |  70 +++++
 20 files changed, 3390 insertions(+)
 create mode 100644 drivers/dma/sdxi/Kconfig
 create mode 100644 drivers/dma/sdxi/Makefile
 create mode 100644 drivers/dma/sdxi/core.c
 create mode 100644 drivers/dma/sdxi/dma.c
 create mode 100644 drivers/dma/sdxi/dma.h
 create mode 100644 drivers/dma/sdxi/ioctl.c
 create mode 100644 drivers/dma/sdxi/ioctl.h
 create mode 100644 drivers/dma/sdxi/pci.c
 create mode 100644 drivers/dma/sdxi/pci.h
 create mode 100644 drivers/dma/sdxi/process.c
 create mode 100644 drivers/dma/sdxi/process.h
 create mode 100644 drivers/dma/sdxi/sdxi.h
 create mode 100644 drivers/dma/sdxi/sq.c
 create mode 100644 drivers/dma/sdxi/sq.h
 create mode 100644 drivers/dma/sdxi/trace.c
 create mode 100644 drivers/dma/sdxi/trace.h
 create mode 100644 include/uapi/linux/sdxi.h
drivers/dma/Kconfig +2 -0
index 644c188d6a11..731d320499e7 100644
@@ -775,6 +775,8 @@ source "drivers/dma/fsl-dpaa2-qdma/Kconfig"
 
 source "drivers/dma/lgm/Kconfig"
 
+source "drivers/dma/sdxi/Kconfig"
+
 # clients
 comment "DMA Clients"
 	depends on DMA_ENGINE
drivers/dma/Makefile +1 -0
index a4fd1ce29510..43c57226fd3b 100644
@@ -81,6 +81,7 @@ obj-$(CONFIG_XGENE_DMA) += xgene-dma.o
 obj-$(CONFIG_ST_FDMA) += st_fdma.o
 obj-$(CONFIG_FSL_DPAA2_QDMA) += fsl-dpaa2-qdma/
 obj-$(CONFIG_INTEL_LDMA) += lgm/
+obj-$(CONFIG_SDXI) += sdxi/
 
 obj-y += mediatek/
 obj-y += qcom/
drivers/dma/sdxi/Kconfig new file +11 -0
new file mode 100644
index 000000000000..10494042b20c
@@ -0,0 +1,11 @@
+config SDXI
+	tristate "SDXI support"
+	default m
+	select DMA_ENGINE
+	select DMA_VIRTUAL_CHANNELS
+	select AMD_IOMMU_V2
+	help
+	  Enable support for Smart Data Accelerator Interface (SDXI) DMA
+	  controller. This controller provides DMA capabilities, plus other
+	  advanced data processing features, that can offload the work for
+	  CPUs.
drivers/dma/sdxi/Makefile new file +3 -0
new file mode 100644
index 000000000000..dcb81f574a77
@@ -0,0 +1,3 @@
+obj-$(CONFIG_SDXI)	+= sdxi.o
+
+sdxi-objs := pci.o core.o sq.o dma.o ioctl.o process.o trace.o
drivers/dma/sdxi/core.c new file +428 -0

428 lines not on this page · raw message

drivers/dma/sdxi/dma.c new file +441 -0

441 lines not on this page · raw message

drivers/dma/sdxi/dma.h new file +23 -0
new file mode 100644
index 000000000000..950d94af868a
@@ -0,0 +1,23 @@
+/*
+ * DMA engine support
+ *
+ * Copyright (C) 2022 AMD, Inc. All rights reserved.
+ *
+ * Author: Wei Huang <[email protected]>
+ * Author: Sanjay R Mehta <[email protected]>
+ *
+ */
+
+#include <linux/dma-mapping.h>
+#include <linux/dmaengine.h>
+
+#include "../dma/dmaengine.h"
+#include "../dma/virt-dma.h"
+
+#include "sdxi.h"
+#include "sq.h"
+
+#ifndef __SDXI_DMA_H
+#define __SDXI_DMA_H
+
+#endif /* __SDXI_DMA_H */
drivers/dma/sdxi/ioctl.c new file +440 -0

440 lines not on this page · raw message

drivers/dma/sdxi/ioctl.h new file +49 -0

49 lines not on this page · raw message

drivers/dma/sdxi/pci.c new file +533 -0

533 lines not on this page · raw message

drivers/dma/sdxi/pci.h new file +130 -0

130 lines not on this page · raw message

drivers/dma/sdxi/process.c new file +285 -0

285 lines not on this page · raw message

drivers/dma/sdxi/process.h new file +71 -0

71 lines not on this page · raw message

drivers/dma/sdxi/sdxi.h new file +414 -0

414 lines not on this page · raw message

drivers/dma/sdxi/sq.c new file +295 -0

157 lines not on this page · raw message

drivers/dma/sdxi/sq.h new file +67 -0

67 lines not on this page · raw message

drivers/dma/sdxi/trace.c new file +10 -0
new file mode 100644
index 000000000000..11cc3a3da140

Lines of this file are not shown. View the raw message.

drivers/dma/sdxi/trace.h new file +115 -0

115 lines not on this page · raw message

include/linux/pci_ids.h +2 -0
index 2dc75df1437f..a0314d65255c 100644

Lines of this file are not shown. View the raw message.

include/uapi/linux/sdxi.h new file +70 -0

70 lines not on this page · raw message

The rest of this diff is not shown. View the raw message.

Signature · 1 line
2.31.1

Part of the diff is not shown here. View the raw message.

author maintainer
#2 lore raw

PATCH 1/10 iommu/amd: Rename amd_iommu_v2_supported() as amd_iommu_sva_supported()

In reply to Vasant Hegde

To reflect its usage. No functional changes intended.

---
 drivers/iommu/amd/amd_iommu.h | 2 +-
 drivers/iommu/amd/init.c      | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
drivers/iommu/amd/amd_iommu.h +1 -1
index 92fba9bda3d2..f236d540c917 100644
@@ -41,7 +41,7 @@ extern int amd_iommu_guest_ir;
 extern enum io_pgtable_fmt amd_iommu_pgtable;
 extern int amd_iommu_gpt_level;
 
-bool amd_iommu_v2_supported(void);
+bool amd_iommu_sva_supported(void);
 struct amd_iommu *get_amd_iommu(unsigned int idx);
 u8 amd_iommu_pc_get_max_banks(unsigned int idx);
 bool amd_iommu_pc_supported(void);
drivers/iommu/amd/init.c +1 -1
index 64bcf3df37ee..3820ae738088 100644
@@ -3652,7 +3652,7 @@ __setup("ivrs_ioapic",		parse_ivrs_ioapic);
 __setup("ivrs_hpet",		parse_ivrs_hpet);
 __setup("ivrs_acpihid",		parse_ivrs_acpihid);
 
-bool amd_iommu_v2_supported(void)
+bool amd_iommu_sva_supported(void)
 {
 	/* CPU page table size should match IOMMU guest page table size */
 	if (cpu_feature_enabled(X86_FEATURE_LA57) &&
Signature · 1 line
2.31.1
author maintainer
#3 lore raw

PATCH 2/10 dma/sdxi: Switch SDXI virtual address support from iommu_v2 to SVA

In reply to Vasant Hegde

From: Wei Huang <[email protected]>

Current SDXI uses iommu_v2 driver for user space's address support. However this driver is AMD IOMMU specific. The upstream Linux creates a new interface, called Shared Virtual Address (SVA). SVA has been supported by different IOMMU devices. AMD is also adding support for SVA in the IOMMU driver. This patch switches SDXI's virtual address support from iommu_v2 to SVA.

---
 drivers/dma/sdxi/Makefile             |   2 +-
 drivers/dma/sdxi/core.c               |   2 +-
 drivers/dma/sdxi/ioctl.c              |   6 +-
 drivers/dma/sdxi/ioctl.h              |   2 +-
 drivers/dma/sdxi/pci.c                |   8 +-
 drivers/dma/sdxi/process.c            | 285 --------------------------
 drivers/dma/sdxi/sva.c                | 187 +++++++++++++++++
 drivers/dma/sdxi/{process.h => sva.h} |  47 +----
 8 files changed, 208 insertions(+), 331 deletions(-)
 delete mode 100644 drivers/dma/sdxi/process.c
 create mode 100644 drivers/dma/sdxi/sva.c
 rename drivers/dma/sdxi/{process.h => sva.h} (50%)
drivers/dma/sdxi/Makefile +1 -1
index dcb81f574a77..31c37e2357de 100644
@@ -1,3 +1,3 @@
 obj-$(CONFIG_SDXI)	+= sdxi.o
 
-sdxi-objs := pci.o core.o sq.o dma.o ioctl.o process.o trace.o
+sdxi-objs := pci.o core.o sq.o dma.o ioctl.o trace.o sva.o
drivers/dma/sdxi/core.c +1 -1
index 187e2e0c660c..d3a2d984e1d9 100644
@@ -18,7 +18,7 @@
 #include "sdxi.h"
 #include "pci.h"
 #include "sq.h"
-#include "process.h"
+#include "sva.h"
 
 #define CREATE_TRACE_POINTS
 #include "trace.h"
drivers/dma/sdxi/ioctl.c +3 -3
index 6b926795dc03..4355099a9584 100644
@@ -18,7 +18,7 @@
 #include <uapi/linux/sdxi.h>
 
 #include "ioctl.h"
-#include "process.h"
+#include "sva.h"
 #include "sdxi.h"
 #include "sq.h"
 
@@ -166,7 +166,7 @@ static int sdxi_ioctl_create_ctxt(struct file *filep, struct sdxi_process *p,
 	mutex_lock(&p->mutex);
 
 	p->ctxt = ctxt;
-	err = sdxi_bind_process_to_device(p);
+	err = sdxi_sva_bind_process(p);
 	if (err) {
 		err = -ESRCH;
 		goto err_bind_process;
@@ -203,7 +203,7 @@ static int sdxi_ioctl_close_ctxt(struct file *filep, struct sdxi_process *p,
 	mutex_lock(&p->mutex);
 
 	if (args->ctxt_id == p->ctxt->id) {
-		sdxi_unbind_process_to_device(p);
+		sdxi_sva_unbind_process(p);
 		//sdxi_working_ctxt_exit(p->ctxt);
 	}
 
drivers/dma/sdxi/ioctl.h +1 -1
index dedafec37673..3f84f7a498fa 100644
@@ -17,7 +17,7 @@
 #include <linux/workqueue.h>
 
 #include "sdxi.h"
-#include "process.h"
+#include "sva.h"
 
 #define SDXI_IOCTL_MAJOR_VER 0
 #define SDXI_IOCTL_MINOR_VER 2
drivers/dma/sdxi/pci.c +4 -4
index db0ba908e4ea..d51c190a3cb7 100644
@@ -15,7 +15,7 @@
 
 #include "sdxi.h"
 #include "pci.h"
-#include "process.h"
+#include "sva.h"
 
 LIST_HEAD(sdxi_device_list);
 
@@ -435,7 +435,7 @@ static int sdxi_pci_probe(struct pci_dev *pdev,
 	if (ret)
 		goto err_dev_init;
 
-	ret = sdxi_iommu_device_init(sdxi);
+	ret = sdxi_sva_device_init(sdxi);
 	if (ret)
 		goto err_iommu_init;
 
@@ -446,7 +446,7 @@ static int sdxi_pci_probe(struct pci_dev *pdev,
 	return 0;
 
 err_pci_enable:
-	sdxi_iommu_device_exit(sdxi);
+	sdxi_sva_device_exit(sdxi);
 err_iommu_init:
 	sdxi_device_exit(sdxi);
 err_dev_init:
@@ -461,7 +461,7 @@ static void sdxi_pci_remove(struct pci_dev *pdev)
 	struct sdxi_dev *sdxi = pci_get_drvdata(pdev);
 
 	sdxi_pci_disable(sdxi);
-	sdxi_iommu_device_exit(sdxi);
+	sdxi_sva_device_exit(sdxi);
 	sdxi_device_exit(sdxi);
 	sdxi_pci_exit(sdxi);
 	sdxi_device_free(sdxi);
drivers/dma/sdxi/process.c deleted +0 -285

285 lines not on this page · raw message

drivers/dma/sdxi/sva.c new file +187 -0

187 lines not on this page · raw message

drivers/dma/sdxi/sva.h renamed from drivers/dma/sdxi/process.h +11 -36

68 lines not on this page · raw message

Signature · 1 line
2.31.1
author maintainer
#4 lore raw

PATCH 2/10 iommu/amd: Add support for enabling/disabling IOMMU features

In reply to Vasant Hegde

From: Wei Huang <[email protected]>

Add support for struct iommu_ops.dev_{enable/disable}_feat. Please note that the empty feature switches will be populated by subsequent patches.

---
 drivers/iommu/amd/iommu.c | 28 ++++++++++++++++++++++++++++
 1 file changed, 28 insertions(+)
drivers/iommu/amd/iommu.c +28 -0
index 15fce04ac59b..8fd647c0d657 100644
@@ -2563,6 +2563,32 @@ static bool amd_iommu_enforce_cache_coherency(struct iommu_domain *domain)
 	return true;
 }
 
+static int amd_iommu_dev_enable_feature(struct device *dev,
+					enum iommu_dev_features feat)
+{
+	int ret;
+
+	switch (feat) {
+	default:
+		ret = -EINVAL;
+		break;
+	}
+	return ret;
+}
+
+static int amd_iommu_dev_disable_feature(struct device *dev,
+					 enum iommu_dev_features feat)
+{
+	int ret;
+
+	switch (feat) {
+	default:
+		ret = -EINVAL;
+		break;
+	}
+	return ret;
+}
+
 const struct iommu_ops amd_iommu_ops = {
 	.capable = amd_iommu_capable,
 	.domain_alloc = amd_iommu_domain_alloc,
@@ -2574,6 +2600,8 @@ const struct iommu_ops amd_iommu_ops = {
 	.is_attach_deferred = amd_iommu_is_attach_deferred,
 	.pgsize_bitmap	= AMD_IOMMU_PGSIZES,
 	.def_domain_type = amd_iommu_def_domain_type,
+	.dev_enable_feat = amd_iommu_dev_enable_feature,
+	.dev_disable_feat = amd_iommu_dev_disable_feature,
 	.default_domain_ops = &(const struct iommu_domain_ops) {
 		.attach_dev	= amd_iommu_attach_device,
 		.map_pages	= amd_iommu_map_pages,
Signature · 1 line
2.31.1
author maintainer
#5 lore raw

PATCH 3/10 dma/sdxi: Disable event log register read temporarily

In reply to Vasant Hegde

From: Wei Huang <[email protected]>

It seems the driver is having issues during MMIO register read when error log is triggered. This can potentially be FW issue. Let us disable it for now.

---
 drivers/dma/sdxi/pci.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/dma/sdxi/pci.c +2 -2
index d51c190a3cb7..a7c3aaa3ff12 100644
@@ -81,14 +81,14 @@ static void sdxi_do_cmd_complete(unsigned long data)
 
 static irqreturn_t sdxi_irq_thread(int irq, void *data)
 {
-	struct sdxi_dev *sdxi = (struct sdxi_dev *)data;
+	/*struct sdxi_dev *sdxi = (struct sdxi_dev *)data;
 	u64 status = reg_read64(sdxi->ctrl_regs + ERR_LOG_STS_OFFSET);
 
 	while (status & ERR_LOG_STATUS_MASK) {
 		printk(KERN_ERR "read status\n");
 		sdxi_handle_err(sdxi);
 		status = reg_read64(sdxi->ctrl_regs + ERR_LOG_STS_OFFSET);
-	}
+		}*/
 
 	/* sdxi_do_cmd_complete((ulong)&sdxi->tdata); */
 
Signature · 1 line
2.31.1
author maintainer
#6 lore raw

PATCH 3/10 iommu/amd: Initial SVA support for AMD IOMMU

In reply to Vasant Hegde

This includes :
- Add support to allocate SVA domain. It also sets 'set_dev_pasid' to bind PASID to device and make necessary updates to DTE.

- Add iommu_ops.remove_dev_pasid support. It will unbind PASID from device and makes necessary update to DTE.

- PASID management
xarray to keep track the PASIDs
struct amd_sva_pasid
It will contain mm struct, mmu notifier, protection domain and list of devices within IOMMU group using this pasid.
struct amd_sva_dev
This contains device list and pointer to iommu_dev_data. So that we can track pasid to device binding.

- mmu notifier for iotlb invalidation

- Add IOMMU_SVA as dependency to AMD_IOMMU driver

---
 drivers/iommu/amd/Kconfig           |   1 +
 drivers/iommu/amd/Makefile          |   2 +-
 drivers/iommu/amd/amd_iommu.h       |   5 +
 drivers/iommu/amd/amd_iommu_types.h |  10 +
 drivers/iommu/amd/iommu.c           |  10 +
 drivers/iommu/amd/sva.c             | 299 ++++++++++++++++++++++++++++
 6 files changed, 326 insertions(+), 1 deletion(-)
 create mode 100644 drivers/iommu/amd/sva.c
drivers/iommu/amd/Kconfig +1 -0
index 75132ae861a2..cabf4ccde1ed 100644
@@ -10,6 +10,7 @@ config AMD_IOMMU
 	select IOMMU_API
 	select IOMMU_IOVA
 	select IOMMU_IO_PGTABLE
+	select IOMMU_SVA
 	depends on X86_64 && PCI && ACPI && HAVE_CMPXCHG_DOUBLE
 	help
 	  With this option you can enable support for AMD IOMMU hardware in
drivers/iommu/amd/Makefile +1 -1
index f454fbb1569e..848c89fa5238 100644
@@ -1,3 +1,3 @@
 # SPDX-License-Identifier: GPL-2.0-only
-obj-$(CONFIG_AMD_IOMMU) += iommu.o init.o quirks.o io_pgtable.o io_pgtable_v2.o
+obj-$(CONFIG_AMD_IOMMU) += iommu.o init.o quirks.o io_pgtable.o io_pgtable_v2.o sva.o
 obj-$(CONFIG_AMD_IOMMU_DEBUGFS) += debugfs.o
drivers/iommu/amd/amd_iommu.h +5 -0
index f236d540c917..6fdad7a54781 100644
@@ -41,7 +41,12 @@ extern int amd_iommu_guest_ir;
 extern enum io_pgtable_fmt amd_iommu_pgtable;
 extern int amd_iommu_gpt_level;
 
+/* SVA */
 bool amd_iommu_sva_supported(void);
+int amd_iommu_set_dev_pasid(struct iommu_domain *domain,
+			    struct device *dev, ioasid_t pasid);
+void amd_iommu_remove_dev_pasid(struct device *dev, ioasid_t pasid);
+
 struct amd_iommu *get_amd_iommu(unsigned int idx);
 u8 amd_iommu_pc_get_max_banks(unsigned int idx);
 bool amd_iommu_pc_supported(void);
drivers/iommu/amd/amd_iommu_types.h +10 -0
index b1a54d8c7506..88b9e41b6fbc 100644
@@ -922,6 +922,16 @@ static inline int get_hpet_devid(int id)
 	return -EINVAL;
 }
 
+static inline struct protection_domain *amd_iommu_get_pdomain(struct device *dev)
+{
+	struct iommu_dev_data *dev_data = dev_iommu_priv_get(dev);
+
+	if (!dev_data || !dev_data->domain)
+		return NULL;
+
+	return dev_data->domain;
+}
+
 enum amd_iommu_intr_mode_type {
 	AMD_IOMMU_GUEST_IR_LEGACY,
 
drivers/iommu/amd/iommu.c +10 -0
index 8fd647c0d657..46bb1bfdc322 100644
@@ -76,6 +76,7 @@ struct iommu_cmd {
 struct kmem_cache *amd_iommu_irq_cache;
 
 static void detach_device(struct device *dev);
+static void amd_iommu_domain_free(struct iommu_domain *dom);
 
 static void set_dte_entry(struct amd_iommu *iommu, u16 devid,
 			  struct gcr3_tbl_info *gcr3_info,
@@ -2187,6 +2188,11 @@ static int protection_domain_init_v2(struct protection_domain *pdom)
 	return 0;
 }
 
+static const struct iommu_domain_ops amd_svm_domain_ops = {
+	.set_dev_pasid = amd_iommu_set_dev_pasid,
+	.free	       = amd_iommu_domain_free
+};
+
 static struct protection_domain *protection_domain_alloc(unsigned int type)
 {
 	struct io_pgtable_ops *pgtbl_ops;
@@ -2221,6 +2227,9 @@ static struct protection_domain *protection_domain_alloc(unsigned int type)
 	case IOMMU_DOMAIN_UNMANAGED:
 		pgtable = AMD_IOMMU_V1;
 		break;
+	case IOMMU_DOMAIN_SVA:
+		domain->domain.ops = &amd_svm_domain_ops;
+		return domain;
 	default:
 		goto out_err;
 	}
@@ -2602,6 +2611,7 @@ const struct iommu_ops amd_iommu_ops = {
 	.def_domain_type = amd_iommu_def_domain_type,
 	.dev_enable_feat = amd_iommu_dev_enable_feature,
 	.dev_disable_feat = amd_iommu_dev_disable_feature,
+	.remove_dev_pasid = amd_iommu_remove_dev_pasid,
 	.default_domain_ops = &(const struct iommu_domain_ops) {
 		.attach_dev	= amd_iommu_attach_device,
 		.map_pages	= amd_iommu_map_pages,
drivers/iommu/amd/sva.c new file +299 -0

299 lines not on this page · raw message

Signature · 1 line
2.31.1
author maintainer
#7 lore raw

PATCH 4/10 iommu/amd: Add support to enable/disable SVA feature

In reply to Vasant Hegde

From: Suravee Suthikulpanit <[email protected]>

Add support to enable/disable SVA feature. It will setup v2 page table mode with PASID and enables SVA.

---
 drivers/iommu/amd/amd_iommu.h |  4 +++
 drivers/iommu/amd/iommu.c     | 63 +++++++++++++++++++++++++++++++++++
 drivers/iommu/amd/sva.c       | 29 ++++++++++++++++
 3 files changed, 96 insertions(+)
drivers/iommu/amd/amd_iommu.h +4 -0
index 6fdad7a54781..3f7ddf19ed72 100644
@@ -46,6 +46,10 @@ bool amd_iommu_sva_supported(void);
 int amd_iommu_set_dev_pasid(struct iommu_domain *domain,
 			    struct device *dev, ioasid_t pasid);
 void amd_iommu_remove_dev_pasid(struct device *dev, ioasid_t pasid);
+int amd_iommu_sva_enable(struct device *dev);
+int amd_iommu_sva_disable(struct device *dev);
+int amd_iommu_sva_gcr3_init(struct iommu_dev_data *dev_data, int pasids);
+int amd_iommu_sva_gcr3_uninit(struct iommu_dev_data *dev_data);
 
 struct amd_iommu *get_amd_iommu(unsigned int idx);
 u8 amd_iommu_pc_get_max_banks(unsigned int idx);
drivers/iommu/amd/iommu.c +63 -0

81 lines not on this page · raw message

drivers/iommu/amd/sva.c +29 -0
index c7c7e7cb5414..b4cc5b854753 100644
@@ -297,3 +297,32 @@ void amd_iommu_remove_dev_pasid(struct device *dev, ioasid_t pasid)
 		break;
 	}
 }
+
+int amd_iommu_sva_enable(struct device *dev)
+{
+	struct pci_dev *pdev = dev_is_pci(dev) ? to_pci_dev(dev) : NULL;
+	struct amd_iommu *iommu = get_amd_iommu_from_dev(dev);
+	struct iommu_dev_data *dev_data = dev_iommu_priv_get(dev);
+
+	if (!pdev || !iommu || !dev_data)
+		return -EINVAL;
+
+	if (!amd_iommu_sva_supported())
+		return -ENODEV;
+
+	if (!dev_data->pasid_enabled)
+		return -EINVAL;
+
+	return amd_iommu_sva_gcr3_init(dev_data, dev->iommu->max_pasids);
+}
+
+int amd_iommu_sva_disable(struct device *dev)
+{
+	struct amd_iommu *iommu = get_amd_iommu_from_dev(dev);
+	struct iommu_dev_data *dev_data = dev_iommu_priv_get(dev);
+
+	if (!iommu || !dev_data)
+		return -EINVAL;
+
+	return amd_iommu_sva_gcr3_uninit(dev_data);
+}
Signature · 1 line
2.31.1
author maintainer
#8 lore raw

PATCH 4/10 sdxi: Fix for vma->vm_flags

In reply to Vasant Hegde
---
 drivers/dma/sdxi/ioctl.c | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)
drivers/dma/sdxi/ioctl.c +10 -4
index 4355099a9584..9f7c7dc5014a 100644
@@ -43,8 +43,11 @@ static int sdxi_ctxt_doorbell_mmap(struct sdxi_process *process,
 
 	address = ctxt->db_base;
 
-	vma->vm_flags |= VM_IO | VM_DONTCOPY | VM_DONTEXPAND | VM_NORESERVE |
-		VM_DONTDUMP | VM_PFNMAP;
+	vm_flags_init(vma, VM_IO | VM_DONTCOPY | VM_DONTEXPAND | VM_NORESERVE |
+		      VM_DONTDUMP | VM_PFNMAP);
+
+	//vma->vm_flags |= VM_IO | VM_DONTCOPY | VM_DONTEXPAND | VM_NORESERVE |
+	//	VM_DONTDUMP | VM_PFNMAP;
 
 	vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot);
 
@@ -83,8 +86,11 @@ static int sdxi_ctxt_struct_mmap(struct sdxi_process *process,
 		return -EINVAL;
 	}
 
-	vma->vm_flags |= VM_IO | VM_DONTCOPY | VM_DONTEXPAND
-		| VM_NORESERVE | VM_DONTDUMP | VM_PFNMAP;
+	vm_flags_init(vma, VM_IO | VM_DONTCOPY | VM_DONTEXPAND
+		      | VM_NORESERVE | VM_DONTDUMP | VM_PFNMAP);
+
+	//vma->vm_flags |= VM_IO | VM_DONTCOPY | VM_DONTEXPAND
+	//	| VM_NORESERVE | VM_DONTDUMP | VM_PFNMAP;
 
 	//vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot);
 
Signature · 1 line
2.31.1
author maintainer
#9 lore raw

PATCH 5/10 iommu/amd: Move PPR-related functions into ppr.c

In reply to Vasant Hegde

From: Suravee Suthikulpanit <[email protected]>

In preparation to subsequent PPR-related patches, and also remove static declaration for certain helper functions so that it can be reused in other files.

Also rename below functions:
alloc_ppr_log -> amd_iommu_alloc_ppr_log
iommu_enable_ppr_log -> amd_iommu_enable_ppr_log
free_ppr_log -> amd_iommu_free_ppr_log
iommu_poll_ppr_log -> amd_iommu_poll_ppr_log

---
 drivers/iommu/amd/Makefile    |   2 +-
 drivers/iommu/amd/amd_iommu.h |  17 +++++-
 drivers/iommu/amd/init.c      |  65 +++-----------------
 drivers/iommu/amd/iommu.c     |  55 +----------------
 drivers/iommu/amd/ppr.c       | 112 ++++++++++++++++++++++++++++++++++
 5 files changed, 137 insertions(+), 114 deletions(-)
 create mode 100644 drivers/iommu/amd/ppr.c
drivers/iommu/amd/Makefile +1 -1
index 848c89fa5238..a6601d885c5b 100644
@@ -1,3 +1,3 @@
 # SPDX-License-Identifier: GPL-2.0-only
-obj-$(CONFIG_AMD_IOMMU) += iommu.o init.o quirks.o io_pgtable.o io_pgtable_v2.o sva.o
+obj-$(CONFIG_AMD_IOMMU) += iommu.o init.o quirks.o io_pgtable.o io_pgtable_v2.o sva.o ppr.o
 obj-$(CONFIG_AMD_IOMMU_DEBUGFS) += debugfs.o
drivers/iommu/amd/amd_iommu.h +14 -3
index 3f7ddf19ed72..31e6cb3ba5d6 100644
@@ -17,6 +17,9 @@ irqreturn_t amd_iommu_int_thread_pprlog(int irq, void *data);
 irqreturn_t amd_iommu_int_thread_galog(int irq, void *data);
 irqreturn_t amd_iommu_int_handler(int irq, void *data);
 void amd_iommu_apply_erratum_63(struct amd_iommu *iommu, u16 devid);
+void amd_iommu_restart_log(struct amd_iommu *iommu, const char *evt_type,
+			   u8 cntrl_intr, u8 cntrl_log,
+			   u32 status_run_mask, u32 status_overflow_mask);
 void amd_iommu_restart_event_logging(struct amd_iommu *iommu);
 void amd_iommu_restart_ga_log(struct amd_iommu *iommu);
 void amd_iommu_restart_ppr_log(struct amd_iommu *iommu);
@@ -24,6 +27,9 @@ int amd_iommu_init_devices(void);
 void amd_iommu_uninit_devices(void);
 void amd_iommu_init_notifier(void);
 void amd_iommu_set_rlookup_table(struct amd_iommu *iommu, u16 devid);
+void iommu_feature_enable(struct amd_iommu *iommu, u8 bit);
+void *__init iommu_alloc_4k_pages(struct amd_iommu *iommu,
+				  gfp_t gfp, size_t size);
 
 #ifdef CONFIG_AMD_IOMMU_DEBUGFS
 void amd_iommu_debugfs_setup(struct amd_iommu *iommu);
@@ -69,6 +75,14 @@ int amd_iommu_set_gcr3(struct iommu_dev_data *dev_data,
 		       u32 pasid, unsigned long gcr3);
 int amd_iommu_clear_gcr3(struct iommu_dev_data *dev_data, u32 pasid);
 
+/* PPR */
+int __init amd_iommu_alloc_ppr_log(struct amd_iommu *iommu);
+void __init amd_iommu_free_ppr_log(struct amd_iommu *iommu);
+void amd_iommu_enable_ppr_log(struct amd_iommu *iommu);
+void amd_iommu_poll_ppr_log(struct amd_iommu *iommu);
+int amd_iommu_complete_ppr(struct pci_dev *pdev, u32 pasid,
+			   int status, int tag);
+
 int amd_iommu_flush_page(struct protection_domain *domain, u32 pasid, u64 address);
 void amd_iommu_update_and_flush_device_table(struct protection_domain *domain);
 void amd_iommu_domain_update(struct protection_domain *domain);
@@ -89,9 +103,6 @@ static inline int amd_iommu_create_irq_domain(struct amd_iommu *iommu)
 #define PPR_INVALID			0x1
 #define PPR_FAILURE			0xf
 
-int amd_iommu_complete_ppr(struct pci_dev *pdev, u32 pasid,
-			   int status, int tag);
-
 static inline bool is_rd890_iommu(struct pci_dev *pdev)
 {
 	return (pdev->vendor == PCI_VENDOR_ID_ATI) &&
drivers/iommu/amd/init.c +9 -56

113 lines not on this page · raw message

drivers/iommu/amd/iommu.c +1 -54

67 lines not on this page · raw message

drivers/iommu/amd/ppr.c new file +112 -0

112 lines not on this page · raw message

Signature · 1 line
2.31.1
author maintainer
#10 lore raw
In reply to Vasant Hegde

Hi All,

Please ignore this series. Looks like I did mess up while generating patchset and accidentally included some test patches! I will resend the series.

-Vasant

Quoted 58 lines · Vasant Hegde

This is part 4 of the 4-part series to introduce Share Virtual Address (SVA) support for devices, which can support PCI ATS, PASID and PRI capabilities. These devices are referred to as SVA-capable devices in this series.

Patch series organization
-------------------------

This series is organized as following:

* Patch 1 - 2:
Rename / add support to enable/disable features

* Patch 3 - 4:
Introduce SVA support

* Patch 5 - 10:
Add IOPF support

This patch series is based on top of SVA Part 3 [1].
[1] https://lore.kernel.org/linux-iommu/[email protected]/

This is also available at github :
https://github.com/AMDESE/linux/tree/iommu_sva_part4_v1_v6.5_rc1

Thank you,
Vasant / Suravee

Suravee Suthikulpanit (5):
iommu/amd: Add support to enable/disable SVA feature iommu/amd: Move PPR-related functions into ppr.c iommu/amd: Define per-IOMMU iopf_queue iommu/amd: Add support for page response iommu/amd: Introduce logic to enable/disable IOPF

Vasant Hegde (2):
iommu/amd: Rename amd_iommu_v2_supported() as amd_iommu_sva_supported() iommu/amd: Initial SVA support for AMD IOMMU

Wei Huang (3):
iommu/amd: Add support for enabling/disabling IOMMU features iommu/amd: Add support for add/remove device for IOPF iommu/amd: Add IO page fault notifier handler

 drivers/iommu/amd/Kconfig           |   1 +
 drivers/iommu/amd/Makefile          |   2 +-
 drivers/iommu/amd/amd_iommu.h       |  41 ++-
 drivers/iommu/amd/amd_iommu_types.h |  22 ++
 drivers/iommu/amd/init.c            |  72 ++----
 drivers/iommu/amd/iommu.c           | 170 ++++++++-----
 drivers/iommu/amd/ppr.c             | 377 ++++++++++++++++++++++++++++
 drivers/iommu/amd/sva.c             | 328 ++++++++++++++++++++++++
 8 files changed, 894 insertions(+), 119 deletions(-)
 create mode 100644 drivers/iommu/amd/ppr.c
 create mode 100644 drivers/iommu/amd/sva.c