Skip to contentSkip to conversation

iommu/riscv: Add irqbypass support

RFC PATCH Superseded Andrew Jones 5 participants 36 messages Nov 14, 2024 – Dec 5, 2024

Subsystems (12)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

cover iommu/riscv: Add irqbypass support

Platforms with implementations of the RISC-V IOMMU and AIA support directly delivering MSIs of devices assigned to guests to the VCPUs. VFIO and KVM have a framework to enable such support, which is called irqbypass. This series enables irqbypass for devices assigned to KVM guests when all VCPUs are allocated guest interrupt files. The RISC-V IOMMU and AIA also support MRIFs (memory-resident interrupt files), but support for those will be posted as a follow-on to this series.

The patches are organized as follows:
1-3: Prepare to create an MSI remapping irqdomain parented by the IMSIC 4-6: Prepare the IOMMU driver for VFIO domains which will use a single
stage of translation, but G-stage instead of the first stage
7-8: Add support to the IOMMU driver for creating irqdomains 9-10: Prepare KVM to enable irqbypass
11-13: Add irqbypass support to the IOMMU driver and KVM 14-15: Enable VFIO by default

This series is also available here [1] and may be tested with QEMU recent enough to have the IOMMU model.

Based on linux-next commit 28955f4fa282 ("Add linux-next specific files for 20241112")

[1] https://github.com/jones-drew/linux/commits/riscv/iommu-irqbypass-rfc-v1/

Thanks,
drew

Andrew Jones (10):
irqchip/riscv-imsic: Use hierarchy to reach irq_set_affinity genirq/msi: Provide DOMAIN_BUS_MSI_REMAP irqchip/riscv-imsic: Add support for DOMAIN_BUS_MSI_REMAP iommu/riscv: Move definitions to iommu.h iommu/riscv: Add IRQ domain for interrupt remapping
RISC-V: KVM: Add irqbypass skeleton
RISC-V: Define irqbypass vcpu_info
iommu/riscv: Add guest file irqbypass support
RISC-V: KVM: Add guest file irqbypass support
RISC-V: defconfig: Add VFIO modules

Tomasz Jeznach (3):
iommu/riscv: report iommu capabilities
RISC-V: KVM: Enable KVM_VFIO interfaces on RISC-V arch
vfio: enable IOMMU_TYPE1 for RISC-V

Zong Li (2):
iommu/riscv: use data structure instead of individual values iommu/riscv: support GSCID and GVMA invalidation command

 arch/riscv/configs/defconfig               |   2 +
 arch/riscv/include/asm/irq.h               |   9 +
 arch/riscv/include/asm/kvm_host.h          |   3 +
 arch/riscv/kvm/Kconfig                     |   3 +
 arch/riscv/kvm/aia_imsic.c                 | 136 +++++++-
 arch/riscv/kvm/vm.c                        |  60 ++++
 drivers/iommu/riscv/Makefile               |   2 +-
 drivers/iommu/riscv/iommu-bits.h           |  11 +
 drivers/iommu/riscv/iommu-ir.c             | 360 +++++++++++++++++++++
 drivers/iommu/riscv/iommu.c                | 183 ++++++-----
 drivers/iommu/riscv/iommu.h                |  78 +++++
 drivers/irqchip/irq-riscv-imsic-platform.c |  18 +-
 drivers/vfio/Kconfig                       |   2 +-
 include/linux/irqdomain_defs.h             |   1 +
 14 files changed, 776 insertions(+), 92 deletions(-)
 create mode 100644 drivers/iommu/riscv/iommu-ir.c
Signature · 1 line
2.47.0
#1 lore raw

RFC PATCH 1/15 irqchip/riscv-imsic: Use hierarchy to reach irq_set_affinity

In order to support IRQ domains which reside between the leaf domains and IMSIC, put the IMSIC implementation of irq_set_affinity into its chip.

---
 drivers/irqchip/irq-riscv-imsic-platform.c | 16 +++++++++-------
 1 file changed, 9 insertions(+), 7 deletions(-)
drivers/irqchip/irq-riscv-imsic-platform.c +9 -7
index c708780e8760..5d7c30ad8855 100644
@@ -96,9 +96,8 @@ static int imsic_irq_set_affinity(struct irq_data *d, const struct cpumask *mask
 				  bool force)
 {
 	struct imsic_vector *old_vec, *new_vec;
-	struct irq_data *pd = d->parent_data;
 
-	old_vec = irq_data_get_irq_chip_data(pd);
+	old_vec = irq_data_get_irq_chip_data(d);
 	if (WARN_ON(!old_vec))
 		return -ENOENT;
 
@@ -116,13 +115,13 @@ static int imsic_irq_set_affinity(struct irq_data *d, const struct cpumask *mask
 		return -ENOSPC;
 
 	/* Point device to the new vector */
-	imsic_msi_update_msg(d, new_vec);
+	imsic_msi_update_msg(irq_get_irq_data(d->irq), new_vec);
 
 	/* Update irq descriptors with the new vector */
-	pd->chip_data = new_vec;
+	d->chip_data = new_vec;
 
-	/* Update effective affinity of parent irq data */
-	irq_data_update_effective_affinity(pd, cpumask_of(new_vec->cpu));
+	/* Update effective affinity */
+	irq_data_update_effective_affinity(d, cpumask_of(new_vec->cpu));
 
 	/* Move state of the old vector to the new vector */
 	imsic_vector_move(old_vec, new_vec);
@@ -135,6 +134,9 @@ static struct irq_chip imsic_irq_base_chip = {
 	.name			= "IMSIC",
 	.irq_mask		= imsic_irq_mask,
 	.irq_unmask		= imsic_irq_unmask,
+#ifdef CONFIG_SMP
+	.irq_set_affinity	= imsic_irq_set_affinity,
+#endif
 	.irq_retrigger		= imsic_irq_retrigger,
 	.irq_compose_msi_msg	= imsic_irq_compose_msg,
 	.flags			= IRQCHIP_SKIP_SET_WAKE |
@@ -245,7 +247,7 @@ static bool imsic_init_dev_msi_info(struct device *dev,
 		if (WARN_ON_ONCE(domain != real_parent))
 			return false;
 #ifdef CONFIG_SMP
-		info->chip->irq_set_affinity = imsic_irq_set_affinity;
+		info->chip->irq_set_affinity = irq_chip_set_affinity_parent;
 #endif
 		break;
 	default:
Signature · 1 line
2.47.0
#2 lore raw

Re: [RFC PATCH 01/15] irqchip/riscv-imsic: Use hierarchy to reach irq_set_affinity

Quoted 12 lines · Andrew Jones
(quoted diff) +1 -2
@@ -96,9 +96,8 @@ static int imsic_irq_set_affinity(struct irq_data *d, const struct cpumask *mask
 				  bool force)
 {
 	struct imsic_vector *old_vec, *new_vec;
-	struct irq_data *pd = d->parent_data;
 
-	old_vec = irq_data_get_irq_chip_data(pd);
+	old_vec = irq_data_get_irq_chip_data(d);
 	if (WARN_ON(!old_vec))
 		return -ENOENT;
 

Review static int imsic_irq_set_affinity(struct irq_data *d, const struct cpumask *mask · line 118

Quoted 5 lines
(quoted diff) +1 -1
@@ -116,13 +115,13 @@ static int imsic_irq_set_affinity(struct irq_data *d, const struct cpumask *mask
 		return -ENOSPC;
 
 	/* Point device to the new vector */
-	imsic_msi_update_msg(d, new_vec);
+	imsic_msi_update_msg(irq_get_irq_data(d->irq), new_vec);

This looks more than fishy. See below.

Review static bool imsic_init_dev_msi_info(struct device *dev, · line 250

Quoted 6 lines
(quoted diff) +1 -1
@@ -245,7 +247,7 @@ static bool imsic_init_dev_msi_info(struct device *dev,
 		if (WARN_ON_ONCE(domain != real_parent))
 			return false;
 #ifdef CONFIG_SMP
-		info->chip->irq_set_affinity = imsic_irq_set_affinity;
+		info->chip->irq_set_affinity = irq_chip_set_affinity_parent;

This should use msi_domain_set_affinity(), which does the right thing:

1) It invokes the irq_set_affinity() callback of the parent domain

2) It composes the message via the hierarchy

3) It writes the message with the msi_write_msg() callback of the top level domain

Sorry, I missed that when reviewing the original IMSIC MSI support.

The whole IMSIC MSI support can be moved over to MSI LIB which makes all of this indirection go away and your intermediate domain will just fit in.

Uncompiled patch below. If that works, it needs to be split up properly.

Note, this removes the setup of the irq_retrigger callback, but that's fine because on hierarchical domains irq_chip_retrigger_hierarchy() is invoked anyway. See try_retrigger().

Thanks,

tglx

---
 drivers/irqchip/Kconfig                    |    1 
 drivers/irqchip/irq-gic-v2m.c              |    1 
 drivers/irqchip/irq-imx-mu-msi.c           |    1 
 drivers/irqchip/irq-msi-lib.c              |   11 +-
 drivers/irqchip/irq-mvebu-gicp.c           |    1 
 drivers/irqchip/irq-mvebu-odmi.c           |    1 
 drivers/irqchip/irq-mvebu-sei.c            |    1 
 drivers/irqchip/irq-riscv-imsic-platform.c |  131 +----------------------------
 include/linux/msi.h                        |   11 ++
 9 files changed, 32 insertions(+), 127 deletions(-)
drivers/irqchip/Kconfig +1 -0
@@ -587,6 +587,7 @@ config RISCV_IMSIC
 	select IRQ_DOMAIN_HIERARCHY
 	select GENERIC_IRQ_MATRIX_ALLOCATOR
 	select GENERIC_MSI_IRQ
+	select IRQ_MSI_LIB
 
 config RISCV_IMSIC_PCI
 	bool
drivers/irqchip/irq-gic-v2m.c +1 -0
@@ -255,6 +255,7 @@ static void __init gicv2m_teardown(void)
 static struct msi_parent_ops gicv2m_msi_parent_ops = {
 	.supported_flags	= GICV2M_MSI_FLAGS_SUPPORTED,
 	.required_flags		= GICV2M_MSI_FLAGS_REQUIRED,
+	.chip_flags		= MSI_CHIP_FLAG_SET_EOI | MSI_CHIP_FLAG_SET_ACK,
 	.bus_select_token	= DOMAIN_BUS_NEXUS,
 	.bus_select_mask	= MATCH_PCI_MSI | MATCH_PLATFORM_MSI,
 	.prefix			= "GICv2m-",
drivers/irqchip/irq-imx-mu-msi.c +1 -0
@@ -214,6 +214,7 @@ static void imx_mu_msi_irq_handler(struc
 static const struct msi_parent_ops imx_mu_msi_parent_ops = {
 	.supported_flags	= IMX_MU_MSI_FLAGS_SUPPORTED,
 	.required_flags		= IMX_MU_MSI_FLAGS_REQUIRED,
+	.chip_flags		= MSI_CHIP_FLAG_SET_EOI | MSI_CHIP_FLAG_SET_ACK,
 	.bus_select_token       = DOMAIN_BUS_NEXUS,
 	.bus_select_mask	= MATCH_PLATFORM_MSI,
 	.prefix			= "MU-MSI-",
drivers/irqchip/irq-msi-lib.c +6 -5
@@ -28,6 +28,7 @@ bool msi_lib_init_dev_msi_info(struct de
 			       struct msi_domain_info *info)
 {
 	const struct msi_parent_ops *pops = real_parent->msi_parent_ops;
+	struct irq_chip *chip = info->chip;
 	u32 required_flags;
 
 	/* Parent ops available? */
@@ -92,10 +93,10 @@ bool msi_lib_init_dev_msi_info(struct de
 	info->flags			|= required_flags;
 
 	/* Chip updates for all child bus types */
-	if (!info->chip->irq_eoi)
-		info->chip->irq_eoi	= irq_chip_eoi_parent;
-	if (!info->chip->irq_ack)
-		info->chip->irq_ack	= irq_chip_ack_parent;
+	if (!chip->irq_eoi && (pops->chip_flags & MSI_CHIP_FLAG_SET_EOI))
+		chip->irq_eoi		= irq_chip_eoi_parent;
+	if (!chip->irq_ack && (pops->chip_flags & MSI_CHIP_FLAG_SET_ACK))
+		chip->irq_ack		= irq_chip_ack_parent;
 
 	/*
 	 * The device MSI domain can never have a set affinity callback. It
@@ -105,7 +106,7 @@ bool msi_lib_init_dev_msi_info(struct de
 	 * device MSI domain aside of mask/unmask which is provided e.g. by
 	 * PCI/MSI device domains.
 	 */
-	info->chip->irq_set_affinity	= msi_domain_set_affinity;
+	chip->irq_set_affinity		= msi_domain_set_affinity;
 	return true;
 }
 EXPORT_SYMBOL_GPL(msi_lib_init_dev_msi_info);
drivers/irqchip/irq-mvebu-gicp.c +1 -0
@@ -161,6 +161,7 @@ static const struct irq_domain_ops gicp_
 static const struct msi_parent_ops gicp_msi_parent_ops = {
 	.supported_flags	= GICP_MSI_FLAGS_SUPPORTED,
 	.required_flags		= GICP_MSI_FLAGS_REQUIRED,
+	.chip_flags		= MSI_CHIP_FLAG_SET_EOI | MSI_CHIP_FLAG_SET_ACK,
 	.bus_select_token       = DOMAIN_BUS_GENERIC_MSI,
 	.bus_select_mask	= MATCH_PLATFORM_MSI,
 	.prefix			= "GICP-",
drivers/irqchip/irq-mvebu-odmi.c +1 -0
@@ -157,6 +157,7 @@ static const struct irq_domain_ops odmi_
 static const struct msi_parent_ops odmi_msi_parent_ops = {
 	.supported_flags	= ODMI_MSI_FLAGS_SUPPORTED,
 	.required_flags		= ODMI_MSI_FLAGS_REQUIRED,
+	.chip_flags		= MSI_CHIP_FLAG_SET_EOI | MSI_CHIP_FLAG_SET_ACK,
 	.bus_select_token	= DOMAIN_BUS_GENERIC_MSI,
 	.bus_select_mask	= MATCH_PLATFORM_MSI,
 	.prefix			= "ODMI-",
drivers/irqchip/irq-mvebu-sei.c +1 -0
@@ -356,6 +356,7 @@ static void mvebu_sei_reset(struct mvebu
 static const struct msi_parent_ops sei_msi_parent_ops = {
 	.supported_flags	= SEI_MSI_FLAGS_SUPPORTED,
 	.required_flags		= SEI_MSI_FLAGS_REQUIRED,
+	.chip_flags		= MSI_CHIP_FLAG_SET_EOI | MSI_CHIP_FLAG_SET_ACK,
 	.bus_select_mask	= MATCH_PLATFORM_MSI,
 	.bus_select_token	= DOMAIN_BUS_GENERIC_MSI,
 	.prefix			= "SEI-",
drivers/irqchip/irq-riscv-imsic-platform.c +9 -122

180 lines not on this page · raw message

include/linux/msi.h +11 -0
@@ -558,11 +558,21 @@ enum {
 	MSI_FLAG_NO_AFFINITY		= (1 << 21),
 };
 
+/*
+ * Flags for msi_parent_ops::chip_flags
+ */
+enum {
+	MSI_CHIP_FLAG_SET_EOI		= (1 << 0),
+	MSI_CHIP_FLAG_SET_ACK		= (1 << 1),
+};
+
 /**
  * struct msi_parent_ops - MSI parent domain callbacks and configuration info
  *
  * @supported_flags:	Required: The supported MSI flags of the parent domain
  * @required_flags:	Optional: The required MSI flags of the parent MSI domain
+ * @chip_flags:		Optional: Select MSI chip callbacks to update with defaults
+ *			in msi_lib_init_dev_msi_info().
  * @bus_select_token:	Optional: The bus token of the real parent domain for
  *			irq_domain::select()
  * @bus_select_mask:	Optional: A mask of supported BUS_DOMAINs for
@@ -575,6 +585,7 @@ enum {
 struct msi_parent_ops {
 	u32		supported_flags;
 	u32		required_flags;
+	u32		chip_flags;
 	u32		bus_select_token;
 	u32		bus_select_mask;
 	const char	*prefix;
#3 lore raw

Re: [RFC PATCH 01/15] irqchip/riscv-imsic: Use hierarchy to reach irq_set_affinity

Quoted 37 lines · Thomas Gleixner
Quoted 12 lines · Andrew Jones
drivers/irqchip/irq-riscv-imsic-platform.c +1 -2
@@ -96,9 +96,8 @@ static int imsic_irq_set_affinity(struct irq_data *d, const struct cpumask *mask
 				  bool force)
 {
 	struct imsic_vector *old_vec, *new_vec;
-	struct irq_data *pd = d->parent_data;
 
-	old_vec = irq_data_get_irq_chip_data(pd);
+	old_vec = irq_data_get_irq_chip_data(d);
 	if (WARN_ON(!old_vec))
 		return -ENOENT;
 

drivers/irqchip/irq-riscv-imsic-platform.c static int imsic_irq_set_affinity(struct irq_data *d, const struct cpumask *mask · line 118

Quoted 5 lines
drivers/irqchip/irq-riscv-imsic-platform.c +1 -1
@@ -116,13 +115,13 @@ static int imsic_irq_set_affinity(struct irq_data *d, const struct cpumask *mask
 		return -ENOSPC;
 
 	/* Point device to the new vector */
-	imsic_msi_update_msg(d, new_vec);
+	imsic_msi_update_msg(irq_get_irq_data(d->irq), new_vec);

This looks more than fishy. See below.

Review static bool imsic_init_dev_msi_info(struct device *dev, · line 250

Quoted 6 lines
(quoted diff) +1 -1
@@ -245,7 +247,7 @@ static bool imsic_init_dev_msi_info(struct device *dev,
 		if (WARN_ON_ONCE(domain != real_parent))
 			return false;
 #ifdef CONFIG_SMP
-		info->chip->irq_set_affinity = imsic_irq_set_affinity;
+		info->chip->irq_set_affinity = irq_chip_set_affinity_parent;

This should use msi_domain_set_affinity(), which does the right thing:

1) It invokes the irq_set_affinity() callback of the parent domain

2) It composes the message via the hierarchy

3) It writes the message with the msi_write_msg() callback of the top level domain

Quoted 7 lines

Sorry, I missed that when reviewing the original IMSIC MSI support.

The whole IMSIC MSI support can be moved over to MSI LIB which makes all of this indirection go away and your intermediate domain will just fit in.

Uncompiled patch below. If that works, it needs to be split up properly.

Thanks Thomas. I gave your patch below a go, but we now fail to have an msi domain set up when probing devices which go through aplic_msi_setup(), resulting in an immediate NULL deference in msi_create_device_irq_domain(). I'll look closer tomorrow.

Thanks,
drew

Quoted 334 lines

Note, this removes the setup of the irq_retrigger callback, but that's fine because on hierarchical domains irq_chip_retrigger_hierarchy() is invoked anyway. See try_retrigger().

Thanks,

tglx

---
 drivers/irqchip/Kconfig                    |    1 
 drivers/irqchip/irq-gic-v2m.c              |    1 
 drivers/irqchip/irq-imx-mu-msi.c           |    1 
 drivers/irqchip/irq-msi-lib.c              |   11 +-
 drivers/irqchip/irq-mvebu-gicp.c           |    1 
 drivers/irqchip/irq-mvebu-odmi.c           |    1 
 drivers/irqchip/irq-mvebu-sei.c            |    1 
 drivers/irqchip/irq-riscv-imsic-platform.c |  131 +----------------------------
 include/linux/msi.h                        |   11 ++
 9 files changed, 32 insertions(+), 127 deletions(-)
drivers/irqchip/Kconfig +1 -0
@@ -587,6 +587,7 @@ config RISCV_IMSIC
 	select IRQ_DOMAIN_HIERARCHY
 	select GENERIC_IRQ_MATRIX_ALLOCATOR
 	select GENERIC_MSI_IRQ
+	select IRQ_MSI_LIB
 
 config RISCV_IMSIC_PCI
 	bool
drivers/irqchip/irq-gic-v2m.c +1 -0
@@ -255,6 +255,7 @@ static void __init gicv2m_teardown(void)
 static struct msi_parent_ops gicv2m_msi_parent_ops = {
 	.supported_flags	= GICV2M_MSI_FLAGS_SUPPORTED,
 	.required_flags		= GICV2M_MSI_FLAGS_REQUIRED,
+	.chip_flags		= MSI_CHIP_FLAG_SET_EOI | MSI_CHIP_FLAG_SET_ACK,
 	.bus_select_token	= DOMAIN_BUS_NEXUS,
 	.bus_select_mask	= MATCH_PCI_MSI | MATCH_PLATFORM_MSI,
 	.prefix			= "GICv2m-",
drivers/irqchip/irq-imx-mu-msi.c +1 -0
@@ -214,6 +214,7 @@ static void imx_mu_msi_irq_handler(struc
 static const struct msi_parent_ops imx_mu_msi_parent_ops = {
 	.supported_flags	= IMX_MU_MSI_FLAGS_SUPPORTED,
 	.required_flags		= IMX_MU_MSI_FLAGS_REQUIRED,
+	.chip_flags		= MSI_CHIP_FLAG_SET_EOI | MSI_CHIP_FLAG_SET_ACK,
 	.bus_select_token       = DOMAIN_BUS_NEXUS,
 	.bus_select_mask	= MATCH_PLATFORM_MSI,
 	.prefix			= "MU-MSI-",
drivers/irqchip/irq-msi-lib.c +6 -5
@@ -28,6 +28,7 @@ bool msi_lib_init_dev_msi_info(struct de
 			       struct msi_domain_info *info)
 {
 	const struct msi_parent_ops *pops = real_parent->msi_parent_ops;
+	struct irq_chip *chip = info->chip;
 	u32 required_flags;
 
 	/* Parent ops available? */
@@ -92,10 +93,10 @@ bool msi_lib_init_dev_msi_info(struct de
 	info->flags			|= required_flags;
 
 	/* Chip updates for all child bus types */
-	if (!info->chip->irq_eoi)
-		info->chip->irq_eoi	= irq_chip_eoi_parent;
-	if (!info->chip->irq_ack)
-		info->chip->irq_ack	= irq_chip_ack_parent;
+	if (!chip->irq_eoi && (pops->chip_flags & MSI_CHIP_FLAG_SET_EOI))
+		chip->irq_eoi		= irq_chip_eoi_parent;
+	if (!chip->irq_ack && (pops->chip_flags & MSI_CHIP_FLAG_SET_ACK))
+		chip->irq_ack		= irq_chip_ack_parent;
 
 	/*
 	 * The device MSI domain can never have a set affinity callback. It
@@ -105,7 +106,7 @@ bool msi_lib_init_dev_msi_info(struct de
 	 * device MSI domain aside of mask/unmask which is provided e.g. by
 	 * PCI/MSI device domains.
 	 */
-	info->chip->irq_set_affinity	= msi_domain_set_affinity;
+	chip->irq_set_affinity		= msi_domain_set_affinity;
 	return true;
 }
 EXPORT_SYMBOL_GPL(msi_lib_init_dev_msi_info);
drivers/irqchip/irq-mvebu-gicp.c +1 -0
@@ -161,6 +161,7 @@ static const struct irq_domain_ops gicp_
 static const struct msi_parent_ops gicp_msi_parent_ops = {
 	.supported_flags	= GICP_MSI_FLAGS_SUPPORTED,
 	.required_flags		= GICP_MSI_FLAGS_REQUIRED,
+	.chip_flags		= MSI_CHIP_FLAG_SET_EOI | MSI_CHIP_FLAG_SET_ACK,
 	.bus_select_token       = DOMAIN_BUS_GENERIC_MSI,
 	.bus_select_mask	= MATCH_PLATFORM_MSI,
 	.prefix			= "GICP-",
drivers/irqchip/irq-mvebu-odmi.c +1 -0
@@ -157,6 +157,7 @@ static const struct irq_domain_ops odmi_
 static const struct msi_parent_ops odmi_msi_parent_ops = {
 	.supported_flags	= ODMI_MSI_FLAGS_SUPPORTED,
 	.required_flags		= ODMI_MSI_FLAGS_REQUIRED,
+	.chip_flags		= MSI_CHIP_FLAG_SET_EOI | MSI_CHIP_FLAG_SET_ACK,
 	.bus_select_token	= DOMAIN_BUS_GENERIC_MSI,
 	.bus_select_mask	= MATCH_PLATFORM_MSI,
 	.prefix			= "ODMI-",
drivers/irqchip/irq-mvebu-sei.c +1 -0
@@ -356,6 +356,7 @@ static void mvebu_sei_reset(struct mvebu
 static const struct msi_parent_ops sei_msi_parent_ops = {
 	.supported_flags	= SEI_MSI_FLAGS_SUPPORTED,
 	.required_flags		= SEI_MSI_FLAGS_REQUIRED,
+	.chip_flags		= MSI_CHIP_FLAG_SET_EOI | MSI_CHIP_FLAG_SET_ACK,
 	.bus_select_mask	= MATCH_PLATFORM_MSI,
 	.bus_select_token	= DOMAIN_BUS_GENERIC_MSI,
 	.prefix			= "SEI-",
drivers/irqchip/irq-riscv-imsic-platform.c +9 -122
@@ -21,6 +21,7 @@
 #include <linux/smp.h>
 
 #include "irq-riscv-imsic-state.h"
+#include "irq-msi-lib.h"
 
 static bool imsic_cpu_page_phys(unsigned int cpu, unsigned int guest_index,
 				phys_addr_t *out_msi_pa)
@@ -84,19 +85,10 @@ static void imsic_irq_compose_msg(struct
 }
 
 #ifdef CONFIG_SMP
-static void imsic_msi_update_msg(struct irq_data *d, struct imsic_vector *vec)
-{
-	struct msi_msg msg = { };
-
-	imsic_irq_compose_vector_msg(vec, &msg);
-	irq_data_get_irq_chip(d)->irq_write_msi_msg(d, &msg);
-}
-
 static int imsic_irq_set_affinity(struct irq_data *d, const struct cpumask *mask_val,
 				  bool force)
 {
 	struct imsic_vector *old_vec, *new_vec;
-	struct irq_data *pd = d->parent_data;
 
 	old_vec = irq_data_get_irq_chip_data(pd);
 	if (WARN_ON(!old_vec))
@@ -115,14 +107,11 @@ static int imsic_irq_set_affinity(struct
 	if (!new_vec)
 		return -ENOSPC;
 
-	/* Point device to the new vector */
-	imsic_msi_update_msg(d, new_vec);
-
 	/* Update irq descriptors with the new vector */
-	pd->chip_data = new_vec;
+	d->chip_data = new_vec;
 
 	/* Update effective affinity of parent irq data */
-	irq_data_update_effective_affinity(pd, cpumask_of(new_vec->cpu));
+	irq_data_update_effective_affinity(d, cpumask_of(new_vec->cpu));
 
 	/* Move state of the old vector to the new vector */
 	imsic_vector_move(old_vec, new_vec);
@@ -137,6 +126,9 @@ static struct irq_chip imsic_irq_base_ch
 	.irq_unmask		= imsic_irq_unmask,
 	.irq_retrigger		= imsic_irq_retrigger,
 	.irq_compose_msi_msg	= imsic_irq_compose_msg,
+#ifdef CONFIG_SMP
+	.irq_set_affinity	= imsic_irq_set_affinity,
+#endif
 	.flags			= IRQCHIP_SKIP_SET_WAKE |
 				  IRQCHIP_MASK_ON_SUSPEND,
 };
@@ -172,22 +164,6 @@ static void imsic_irq_domain_free(struct
 	irq_domain_free_irqs_parent(domain, virq, nr_irqs);
 }
 
-static int imsic_irq_domain_select(struct irq_domain *domain, struct irq_fwspec *fwspec,
-				   enum irq_domain_bus_token bus_token)
-{
-	const struct msi_parent_ops *ops = domain->msi_parent_ops;
-	u32 busmask = BIT(bus_token);
-
-	if (fwspec->fwnode != domain->fwnode || fwspec->param_count != 0)
-		return 0;
-
-	/* Handle pure domain searches */
-	if (bus_token == ops->bus_select_token)
-		return 1;
-
-	return !!(ops->bus_select_mask & busmask);
-}
-
 #ifdef CONFIG_GENERIC_IRQ_DEBUGFS
 static void imsic_irq_debug_show(struct seq_file *m, struct irq_domain *d,
 				 struct irq_data *irqd, int ind)
@@ -210,104 +186,15 @@ static const struct irq_domain_ops imsic
 #endif
 };
 
-#ifdef CONFIG_RISCV_IMSIC_PCI
-
-static void imsic_pci_mask_irq(struct irq_data *d)
-{
-	pci_msi_mask_irq(d);
-	irq_chip_mask_parent(d);
-}
-
-static void imsic_pci_unmask_irq(struct irq_data *d)
-{
-	irq_chip_unmask_parent(d);
-	pci_msi_unmask_irq(d);
-}
-
-#define MATCH_PCI_MSI		BIT(DOMAIN_BUS_PCI_MSI)
-
-#else
-
-#define MATCH_PCI_MSI		0
-
-#endif
-
-static bool imsic_init_dev_msi_info(struct device *dev,
-				    struct irq_domain *domain,
-				    struct irq_domain *real_parent,
-				    struct msi_domain_info *info)
-{
-	const struct msi_parent_ops *pops = real_parent->msi_parent_ops;
-
-	/* MSI parent domain specific settings */
-	switch (real_parent->bus_token) {
-	case DOMAIN_BUS_NEXUS:
-		if (WARN_ON_ONCE(domain != real_parent))
-			return false;
-#ifdef CONFIG_SMP
-		info->chip->irq_set_affinity = imsic_irq_set_affinity;
-#endif
-		break;
-	default:
-		WARN_ON_ONCE(1);
-		return false;
-	}
-
-	/* Is the target supported? */
-	switch (info->bus_token) {
-#ifdef CONFIG_RISCV_IMSIC_PCI
-	case DOMAIN_BUS_PCI_DEVICE_MSI:
-	case DOMAIN_BUS_PCI_DEVICE_MSIX:
-		info->chip->irq_mask = imsic_pci_mask_irq;
-		info->chip->irq_unmask = imsic_pci_unmask_irq;
-		break;
-#endif
-	case DOMAIN_BUS_DEVICE_MSI:
-		/*
-		 * Per-device MSI should never have any MSI feature bits
-		 * set. It's sole purpose is to create a dumb interrupt
-		 * chip which has a device specific irq_write_msi_msg()
-		 * callback.
-		 */
-		if (WARN_ON_ONCE(info->flags))
-			return false;
-
-		/* Core managed MSI descriptors */
-		info->flags |= MSI_FLAG_ALLOC_SIMPLE_MSI_DESCS |
-			       MSI_FLAG_FREE_MSI_DESCS;
-		break;
-	case DOMAIN_BUS_WIRED_TO_MSI:
-		break;
-	default:
-		WARN_ON_ONCE(1);
-		return false;
-	}
-
-	/* Use hierarchial chip operations re-trigger */
-	info->chip->irq_retrigger = irq_chip_retrigger_hierarchy;
-
-	/*
-	 * Mask out the domain specific MSI feature flags which are not
-	 * supported by the real parent.
-	 */
-	info->flags &= pops->supported_flags;
-
-	/* Enforce the required flags */
-	info->flags |= pops->required_flags;
-
-	return true;
-}
-
-#define MATCH_PLATFORM_MSI		BIT(DOMAIN_BUS_PLATFORM_MSI)
-
 static const struct msi_parent_ops imsic_msi_parent_ops = {
 	.supported_flags	= MSI_GENERIC_FLAGS_MASK |
 				  MSI_FLAG_PCI_MSIX,
 	.required_flags		= MSI_FLAG_USE_DEF_DOM_OPS |
-				  MSI_FLAG_USE_DEF_CHIP_OPS,
+				  MSI_FLAG_USE_DEF_CHIP_OPS |
+				  MSI_FLAG_PCI_MSI_MASK_PARENT,
 	.bus_select_token	= DOMAIN_BUS_NEXUS,
 	.bus_select_mask	= MATCH_PCI_MSI | MATCH_PLATFORM_MSI,
-	.init_dev_msi_info	= imsic_init_dev_msi_info,
+	.init_dev_msi_info	= msi_lib_init_dev_msi_info,
 };
 
 int imsic_irqdomain_init(void)
include/linux/msi.h +11 -0
@@ -558,11 +558,21 @@ enum {
 	MSI_FLAG_NO_AFFINITY		= (1 << 21),
 };
 
+/*
+ * Flags for msi_parent_ops::chip_flags
+ */
+enum {
+	MSI_CHIP_FLAG_SET_EOI		= (1 << 0),
+	MSI_CHIP_FLAG_SET_ACK		= (1 << 1),
+};
+
 /**
  * struct msi_parent_ops - MSI parent domain callbacks and configuration info
  *
  * @supported_flags:	Required: The supported MSI flags of the parent domain
  * @required_flags:	Optional: The required MSI flags of the parent MSI domain
+ * @chip_flags:		Optional: Select MSI chip callbacks to update with defaults
+ *			in msi_lib_init_dev_msi_info().
  * @bus_select_token:	Optional: The bus token of the real parent domain for
  *			irq_domain::select()
  * @bus_select_mask:	Optional: A mask of supported BUS_DOMAINs for
@@ -575,6 +585,7 @@ enum {
 struct msi_parent_ops {
 	u32		supported_flags;
 	u32		required_flags;
+	u32		chip_flags;
 	u32		bus_select_token;
 	u32		bus_select_mask;
 	const char	*prefix;
#4 lore raw

Re: [RFC PATCH 01/15] irqchip/riscv-imsic: Use hierarchy to reach irq_set_affinity

Quoted 6 lines · Andrew Jones
Quoted 5 lines · Thomas Gleixner

On Thu, Nov 14 2024 at 17:18, Andrew Jones wrote:
The whole IMSIC MSI support can be moved over to MSI LIB which makes all of this indirection go away and your intermediate domain will just fit in.

Quoted 6 lines
Quoted 1 line

Uncompiled patch below. If that works, it needs to be split up properly.

Thanks Thomas. I gave your patch below a go, but we now fail to have an msi domain set up when probing devices which go through aplic_msi_setup(), resulting in an immediate NULL deference in msi_create_device_irq_domain(). I'll look closer tomorrow.

Duh! I forgot to update the .select callback. I don't know how you fixed that compile fail up. Delta patch below.

Thanks,

tglx
---

drivers/irqchip/irq-riscv-imsic-platform.c +1 -1
@@ -180,7 +180,7 @@ static void imsic_irq_debug_show(struct
 static const struct irq_domain_ops imsic_base_domain_ops = {
 	.alloc		= imsic_irq_domain_alloc,
 	.free		= imsic_irq_domain_free,
-	.select		= imsic_irq_domain_select,
+	.select		= msi_lib_irq_domain_select,
 #ifdef CONFIG_GENERIC_IRQ_DEBUGFS
 	.debug_show	= imsic_irq_debug_show,
 #endif
#5 lore raw

Re: [RFC PATCH 01/15] irqchip/riscv-imsic: Use hierarchy to reach irq_set_affinity

Quoted 24 lines · Thomas Gleixner
Quoted 6 lines · Andrew Jones
Quoted 5 lines · Thomas Gleixner

On Thu, Nov 14 2024 at 17:18, Andrew Jones wrote:
The whole IMSIC MSI support can be moved over to MSI LIB which makes all of this indirection go away and your intermediate domain will just fit in.

Quoted 6 lines
Quoted 1 line

Uncompiled patch below. If that works, it needs to be split up properly.

Thanks Thomas. I gave your patch below a go, but we now fail to have an msi domain set up when probing devices which go through aplic_msi_setup(), resulting in an immediate NULL deference in msi_create_device_irq_domain(). I'll look closer tomorrow.

Duh! I forgot to update the .select callback. I don't know how you fixed that compile fail up. Delta patch below.

Thanks,

tglx
---

drivers/irqchip/irq-riscv-imsic-platform.c static void imsic_irq_debug_show(struct · line 186

Quoted 8 lines
drivers/irqchip/irq-riscv-imsic-platform.c +1 -1
@@ -180,7 +180,7 @@ static void imsic_irq_debug_show(struct
 static const struct irq_domain_ops imsic_base_domain_ops = {
 	.alloc		= imsic_irq_domain_alloc,
 	.free		= imsic_irq_domain_free,
-	.select		= imsic_irq_domain_select,
+	.select		= msi_lib_irq_domain_select,
 #ifdef CONFIG_GENERIC_IRQ_DEBUGFS
 	.debug_show	= imsic_irq_debug_show,
 #endif

Hi Thomas,

With this select fix and replacing patch 03/15 of this series with the following diff, this irqbypass PoC still works.

Based on what Anup said, I kept imsic_msi_update_msg(), which means I kept this entire patch (01/15) as is. Anup is working on a series to fix the non-atomic MSI message writes to the device and will likely pick this patch up along with your changes to convert IMSIC to msi-lib.

I'd like to know your opinion on patch 02/15 of this series and the diff below. afaict, x86 does something similar with the DOMAIN_BUS_DMAR and DOMAIN_BUS_AMDVI tokens in x86_init_dev_msi_info().

Thanks,
drew

drivers/irqchip/irq-msi-lib.c +4 -4
index 51464c6257f3..cc18516a4e82 100644
@@ -36,14 +36,14 @@ bool msi_lib_init_dev_msi_info(struct device *dev, struct irq_domain *domain,
                return false;
 
        /*
-        * MSI parent domain specific settings. For now there is only the
-        * root parent domain, e.g. NEXUS, acting as a MSI parent, but it is
-        * possible to stack MSI parents. See x86 vector -> irq remapping
+        * MSI parent domain specific settings. There may be only the root
+        * parent domain, e.g. NEXUS, acting as a MSI parent, or there may
+        * be stacked MSI parents, typically used for remapping.
         */
        if (domain->bus_token == pops->bus_select_token) {
                if (WARN_ON_ONCE(domain != real_parent))
                        return false;
-       } else {
+       } else if (real_parent->bus_token != DOMAIN_BUS_MSI_REMAP) {
                WARN_ON_ONCE(1);
                return false;
        }
maintainer
#6 lore raw

Re: [RFC PATCH 01/15] irqchip/riscv-imsic: Use hierarchy to reach irq_set_affinity

In reply to Thomas Gleixner
Quoted 44 lines · Thomas Gleixner
Quoted 12 lines · Andrew Jones
drivers/irqchip/irq-riscv-imsic-platform.c +1 -2
@@ -96,9 +96,8 @@ static int imsic_irq_set_affinity(struct irq_data *d, const struct cpumask *mask
                                bool force)
 {
      struct imsic_vector *old_vec, *new_vec;
-     struct irq_data *pd = d->parent_data;
 
-     old_vec = irq_data_get_irq_chip_data(pd);
+     old_vec = irq_data_get_irq_chip_data(d);
      if (WARN_ON(!old_vec))
              return -ENOENT;
 

drivers/irqchip/irq-riscv-imsic-platform.c static int imsic_irq_set_affinity(struct irq_data *d, const struct cpumask *mask · line 118

Quoted 5 lines
drivers/irqchip/irq-riscv-imsic-platform.c +1 -1
@@ -116,13 +115,13 @@ static int imsic_irq_set_affinity(struct irq_data *d, const struct cpumask *mask
              return -ENOSPC;
 
      /* Point device to the new vector */
-     imsic_msi_update_msg(d, new_vec);
+     imsic_msi_update_msg(irq_get_irq_data(d->irq), new_vec);

This looks more than fishy. See below.

Review static bool imsic_init_dev_msi_info(struct device *dev, · line 250

Quoted 6 lines
(quoted diff) +1 -1
@@ -245,7 +247,7 @@ static bool imsic_init_dev_msi_info(struct device *dev,
              if (WARN_ON_ONCE(domain != real_parent))
                      return false;
 #ifdef CONFIG_SMP
-             info->chip->irq_set_affinity = imsic_irq_set_affinity;
+             info->chip->irq_set_affinity = irq_chip_set_affinity_parent;

This should use msi_domain_set_affinity(), which does the right thing:

1) It invokes the irq_set_affinity() callback of the parent domain

2) It composes the message via the hierarchy

3) It writes the message with the msi_write_msg() callback of the top level domain

Sorry, I missed that when reviewing the original IMSIC MSI support.

The whole IMSIC MSI support can be moved over to MSI LIB which makes all of this indirection go away and your intermediate domain will just fit in.

Quoted 5 lines

Uncompiled patch below. If that works, it needs to be split up properly.

Note, this removes the setup of the irq_retrigger callback, but that's fine because on hierarchical domains irq_chip_retrigger_hierarchy() is invoked anyway. See try_retrigger().

The IMSIC driver was merged one kernel release before common MSI LIB was merged.

We should definitely update the IMSIC driver to use MSI LIB, I will try your suggested changes (below) and post a separate series.

Thanks,
Anup

Quoted 330 lines

Thanks,

tglx

---
 drivers/irqchip/Kconfig                    |    1
 drivers/irqchip/irq-gic-v2m.c              |    1
 drivers/irqchip/irq-imx-mu-msi.c           |    1
 drivers/irqchip/irq-msi-lib.c              |   11 +-
 drivers/irqchip/irq-mvebu-gicp.c           |    1
 drivers/irqchip/irq-mvebu-odmi.c           |    1
 drivers/irqchip/irq-mvebu-sei.c            |    1
 drivers/irqchip/irq-riscv-imsic-platform.c |  131 +----------------------------
 include/linux/msi.h                        |   11 ++
 9 files changed, 32 insertions(+), 127 deletions(-)
drivers/irqchip/Kconfig +1 -0
@@ -587,6 +587,7 @@ config RISCV_IMSIC
        select IRQ_DOMAIN_HIERARCHY
        select GENERIC_IRQ_MATRIX_ALLOCATOR
        select GENERIC_MSI_IRQ
+       select IRQ_MSI_LIB
 
 config RISCV_IMSIC_PCI
        bool
drivers/irqchip/irq-gic-v2m.c +1 -0
@@ -255,6 +255,7 @@ static void __init gicv2m_teardown(void)
 static struct msi_parent_ops gicv2m_msi_parent_ops = {
        .supported_flags        = GICV2M_MSI_FLAGS_SUPPORTED,
        .required_flags         = GICV2M_MSI_FLAGS_REQUIRED,
+       .chip_flags             = MSI_CHIP_FLAG_SET_EOI | MSI_CHIP_FLAG_SET_ACK,
        .bus_select_token       = DOMAIN_BUS_NEXUS,
        .bus_select_mask        = MATCH_PCI_MSI | MATCH_PLATFORM_MSI,
        .prefix                 = "GICv2m-",
drivers/irqchip/irq-imx-mu-msi.c +1 -0
@@ -214,6 +214,7 @@ static void imx_mu_msi_irq_handler(struc
 static const struct msi_parent_ops imx_mu_msi_parent_ops = {
        .supported_flags        = IMX_MU_MSI_FLAGS_SUPPORTED,
        .required_flags         = IMX_MU_MSI_FLAGS_REQUIRED,
+       .chip_flags             = MSI_CHIP_FLAG_SET_EOI | MSI_CHIP_FLAG_SET_ACK,
        .bus_select_token       = DOMAIN_BUS_NEXUS,
        .bus_select_mask        = MATCH_PLATFORM_MSI,
        .prefix                 = "MU-MSI-",
drivers/irqchip/irq-msi-lib.c +6 -5
@@ -28,6 +28,7 @@ bool msi_lib_init_dev_msi_info(struct de
                               struct msi_domain_info *info)
 {
        const struct msi_parent_ops *pops = real_parent->msi_parent_ops;
+       struct irq_chip *chip = info->chip;
        u32 required_flags;
 
        /* Parent ops available? */
@@ -92,10 +93,10 @@ bool msi_lib_init_dev_msi_info(struct de
        info->flags                     |= required_flags;
 
        /* Chip updates for all child bus types */
-       if (!info->chip->irq_eoi)
-               info->chip->irq_eoi     = irq_chip_eoi_parent;
-       if (!info->chip->irq_ack)
-               info->chip->irq_ack     = irq_chip_ack_parent;
+       if (!chip->irq_eoi && (pops->chip_flags & MSI_CHIP_FLAG_SET_EOI))
+               chip->irq_eoi           = irq_chip_eoi_parent;
+       if (!chip->irq_ack && (pops->chip_flags & MSI_CHIP_FLAG_SET_ACK))
+               chip->irq_ack           = irq_chip_ack_parent;
 
        /*
         * The device MSI domain can never have a set affinity callback. It
@@ -105,7 +106,7 @@ bool msi_lib_init_dev_msi_info(struct de
         * device MSI domain aside of mask/unmask which is provided e.g. by
         * PCI/MSI device domains.
         */
-       info->chip->irq_set_affinity    = msi_domain_set_affinity;
+       chip->irq_set_affinity          = msi_domain_set_affinity;
        return true;
 }
 EXPORT_SYMBOL_GPL(msi_lib_init_dev_msi_info);
drivers/irqchip/irq-mvebu-gicp.c +1 -0
@@ -161,6 +161,7 @@ static const struct irq_domain_ops gicp_
 static const struct msi_parent_ops gicp_msi_parent_ops = {
        .supported_flags        = GICP_MSI_FLAGS_SUPPORTED,
        .required_flags         = GICP_MSI_FLAGS_REQUIRED,
+       .chip_flags             = MSI_CHIP_FLAG_SET_EOI | MSI_CHIP_FLAG_SET_ACK,
        .bus_select_token       = DOMAIN_BUS_GENERIC_MSI,
        .bus_select_mask        = MATCH_PLATFORM_MSI,
        .prefix                 = "GICP-",
drivers/irqchip/irq-mvebu-odmi.c +1 -0
@@ -157,6 +157,7 @@ static const struct irq_domain_ops odmi_
 static const struct msi_parent_ops odmi_msi_parent_ops = {
        .supported_flags        = ODMI_MSI_FLAGS_SUPPORTED,
        .required_flags         = ODMI_MSI_FLAGS_REQUIRED,
+       .chip_flags             = MSI_CHIP_FLAG_SET_EOI | MSI_CHIP_FLAG_SET_ACK,
        .bus_select_token       = DOMAIN_BUS_GENERIC_MSI,
        .bus_select_mask        = MATCH_PLATFORM_MSI,
        .prefix                 = "ODMI-",
drivers/irqchip/irq-mvebu-sei.c +1 -0
@@ -356,6 +356,7 @@ static void mvebu_sei_reset(struct mvebu
 static const struct msi_parent_ops sei_msi_parent_ops = {
        .supported_flags        = SEI_MSI_FLAGS_SUPPORTED,
        .required_flags         = SEI_MSI_FLAGS_REQUIRED,
+       .chip_flags             = MSI_CHIP_FLAG_SET_EOI | MSI_CHIP_FLAG_SET_ACK,
        .bus_select_mask        = MATCH_PLATFORM_MSI,
        .bus_select_token       = DOMAIN_BUS_GENERIC_MSI,
        .prefix                 = "SEI-",
drivers/irqchip/irq-riscv-imsic-platform.c +9 -122
@@ -21,6 +21,7 @@
 #include <linux/smp.h>
 
 #include "irq-riscv-imsic-state.h"
+#include "irq-msi-lib.h"
 
 static bool imsic_cpu_page_phys(unsigned int cpu, unsigned int guest_index,
                                phys_addr_t *out_msi_pa)
@@ -84,19 +85,10 @@ static void imsic_irq_compose_msg(struct
 }
 
 #ifdef CONFIG_SMP
-static void imsic_msi_update_msg(struct irq_data *d, struct imsic_vector *vec)
-{
-       struct msi_msg msg = { };
-
-       imsic_irq_compose_vector_msg(vec, &msg);
-       irq_data_get_irq_chip(d)->irq_write_msi_msg(d, &msg);
-}
-
 static int imsic_irq_set_affinity(struct irq_data *d, const struct cpumask *mask_val,
                                  bool force)
 {
        struct imsic_vector *old_vec, *new_vec;
-       struct irq_data *pd = d->parent_data;
 
        old_vec = irq_data_get_irq_chip_data(pd);
        if (WARN_ON(!old_vec))
@@ -115,14 +107,11 @@ static int imsic_irq_set_affinity(struct
        if (!new_vec)
                return -ENOSPC;
 
-       /* Point device to the new vector */
-       imsic_msi_update_msg(d, new_vec);
-
        /* Update irq descriptors with the new vector */
-       pd->chip_data = new_vec;
+       d->chip_data = new_vec;
 
        /* Update effective affinity of parent irq data */
-       irq_data_update_effective_affinity(pd, cpumask_of(new_vec->cpu));
+       irq_data_update_effective_affinity(d, cpumask_of(new_vec->cpu));
 
        /* Move state of the old vector to the new vector */
        imsic_vector_move(old_vec, new_vec);
@@ -137,6 +126,9 @@ static struct irq_chip imsic_irq_base_ch
        .irq_unmask             = imsic_irq_unmask,
        .irq_retrigger          = imsic_irq_retrigger,
        .irq_compose_msi_msg    = imsic_irq_compose_msg,
+#ifdef CONFIG_SMP
+       .irq_set_affinity       = imsic_irq_set_affinity,
+#endif
        .flags                  = IRQCHIP_SKIP_SET_WAKE |
                                  IRQCHIP_MASK_ON_SUSPEND,
 };
@@ -172,22 +164,6 @@ static void imsic_irq_domain_free(struct
        irq_domain_free_irqs_parent(domain, virq, nr_irqs);
 }
 
-static int imsic_irq_domain_select(struct irq_domain *domain, struct irq_fwspec *fwspec,
-                                  enum irq_domain_bus_token bus_token)
-{
-       const struct msi_parent_ops *ops = domain->msi_parent_ops;
-       u32 busmask = BIT(bus_token);
-
-       if (fwspec->fwnode != domain->fwnode || fwspec->param_count != 0)
-               return 0;
-
-       /* Handle pure domain searches */
-       if (bus_token == ops->bus_select_token)
-               return 1;
-
-       return !!(ops->bus_select_mask & busmask);
-}
-
 #ifdef CONFIG_GENERIC_IRQ_DEBUGFS
 static void imsic_irq_debug_show(struct seq_file *m, struct irq_domain *d,
                                 struct irq_data *irqd, int ind)
@@ -210,104 +186,15 @@ static const struct irq_domain_ops imsic
 #endif
 };
 
-#ifdef CONFIG_RISCV_IMSIC_PCI
-
-static void imsic_pci_mask_irq(struct irq_data *d)
-{
-       pci_msi_mask_irq(d);
-       irq_chip_mask_parent(d);
-}
-
-static void imsic_pci_unmask_irq(struct irq_data *d)
-{
-       irq_chip_unmask_parent(d);
-       pci_msi_unmask_irq(d);
-}
-
-#define MATCH_PCI_MSI          BIT(DOMAIN_BUS_PCI_MSI)
-
-#else
-
-#define MATCH_PCI_MSI          0
-
-#endif
-
-static bool imsic_init_dev_msi_info(struct device *dev,
-                                   struct irq_domain *domain,
-                                   struct irq_domain *real_parent,
-                                   struct msi_domain_info *info)
-{
-       const struct msi_parent_ops *pops = real_parent->msi_parent_ops;
-
-       /* MSI parent domain specific settings */
-       switch (real_parent->bus_token) {
-       case DOMAIN_BUS_NEXUS:
-               if (WARN_ON_ONCE(domain != real_parent))
-                       return false;
-#ifdef CONFIG_SMP
-               info->chip->irq_set_affinity = imsic_irq_set_affinity;
-#endif
-               break;
-       default:
-               WARN_ON_ONCE(1);
-               return false;
-       }
-
-       /* Is the target supported? */
-       switch (info->bus_token) {
-#ifdef CONFIG_RISCV_IMSIC_PCI
-       case DOMAIN_BUS_PCI_DEVICE_MSI:
-       case DOMAIN_BUS_PCI_DEVICE_MSIX:
-               info->chip->irq_mask = imsic_pci_mask_irq;
-               info->chip->irq_unmask = imsic_pci_unmask_irq;
-               break;
-#endif
-       case DOMAIN_BUS_DEVICE_MSI:
-               /*
-                * Per-device MSI should never have any MSI feature bits
-                * set. It's sole purpose is to create a dumb interrupt
-                * chip which has a device specific irq_write_msi_msg()
-                * callback.
-                */
-               if (WARN_ON_ONCE(info->flags))
-                       return false;
-
-               /* Core managed MSI descriptors */
-               info->flags |= MSI_FLAG_ALLOC_SIMPLE_MSI_DESCS |
-                              MSI_FLAG_FREE_MSI_DESCS;
-               break;
-       case DOMAIN_BUS_WIRED_TO_MSI:
-               break;
-       default:
-               WARN_ON_ONCE(1);
-               return false;
-       }
-
-       /* Use hierarchial chip operations re-trigger */
-       info->chip->irq_retrigger = irq_chip_retrigger_hierarchy;
-
-       /*
-        * Mask out the domain specific MSI feature flags which are not
-        * supported by the real parent.
-        */
-       info->flags &= pops->supported_flags;
-
-       /* Enforce the required flags */
-       info->flags |= pops->required_flags;
-
-       return true;
-}
-
-#define MATCH_PLATFORM_MSI             BIT(DOMAIN_BUS_PLATFORM_MSI)
-
 static const struct msi_parent_ops imsic_msi_parent_ops = {
        .supported_flags        = MSI_GENERIC_FLAGS_MASK |
                                  MSI_FLAG_PCI_MSIX,
        .required_flags         = MSI_FLAG_USE_DEF_DOM_OPS |
-                                 MSI_FLAG_USE_DEF_CHIP_OPS,
+                                 MSI_FLAG_USE_DEF_CHIP_OPS |
+                                 MSI_FLAG_PCI_MSI_MASK_PARENT,
        .bus_select_token       = DOMAIN_BUS_NEXUS,
        .bus_select_mask        = MATCH_PCI_MSI | MATCH_PLATFORM_MSI,
-       .init_dev_msi_info      = imsic_init_dev_msi_info,
+       .init_dev_msi_info      = msi_lib_init_dev_msi_info,
 };
 
 int imsic_irqdomain_init(void)
include/linux/msi.h +11 -0
@@ -558,11 +558,21 @@ enum {
        MSI_FLAG_NO_AFFINITY            = (1 << 21),
 };
 
+/*
+ * Flags for msi_parent_ops::chip_flags
+ */
+enum {
+       MSI_CHIP_FLAG_SET_EOI           = (1 << 0),
+       MSI_CHIP_FLAG_SET_ACK           = (1 << 1),
+};
+
 /**
  * struct msi_parent_ops - MSI parent domain callbacks and configuration info
  *
  * @supported_flags:   Required: The supported MSI flags of the parent domain
  * @required_flags:    Optional: The required MSI flags of the parent MSI domain
+ * @chip_flags:                Optional: Select MSI chip callbacks to update with defaults
+ *                     in msi_lib_init_dev_msi_info().
  * @bus_select_token:  Optional: The bus token of the real parent domain for
  *                     irq_domain::select()
  * @bus_select_mask:   Optional: A mask of supported BUS_DOMAINs for
@@ -575,6 +585,7 @@ enum {
 struct msi_parent_ops {
        u32             supported_flags;
        u32             required_flags;
+       u32             chip_flags;
        u32             bus_select_token;
        u32             bus_select_mask;
        const char      *prefix;
#7 lore raw

Re: [RFC PATCH 01/15] irqchip/riscv-imsic: Use hierarchy to reach irq_set_affinity

Quoted 7 lines · Anup Patel
Quoted 6 lines · Thomas Gleixner

Sorry, I missed that when reviewing the original IMSIC MSI support.

The whole IMSIC MSI support can be moved over to MSI LIB which makes all of this indirection go away and your intermediate domain will just fit in.

Quoted 8 lines
Quoted 5 lines

Uncompiled patch below. If that works, it needs to be split up properly.

Note, this removes the setup of the irq_retrigger callback, but that's fine because on hierarchical domains irq_chip_retrigger_hierarchy() is invoked anyway. See try_retrigger().

The IMSIC driver was merged one kernel release before common MSI LIB was merged.

Ah indeed.

Quoted 2 lines

We should definitely update the IMSIC driver to use MSI LIB, I will try your suggested changes (below) and post a separate series.

Pick up the delta patch I gave Andrew...

#8 lore raw

Re: [RFC PATCH 01/15] irqchip/riscv-imsic: Use hierarchy to reach irq_set_affinity

Quoted 17 lines · Thomas Gleixner
Quoted 7 lines · Anup Patel
Quoted 6 lines · Thomas Gleixner

Sorry, I missed that when reviewing the original IMSIC MSI support.

The whole IMSIC MSI support can be moved over to MSI LIB which makes all of this indirection go away and your intermediate domain will just fit in.

Quoted 8 lines
Quoted 5 lines

Uncompiled patch below. If that works, it needs to be split up properly.

Note, this removes the setup of the irq_retrigger callback, but that's fine because on hierarchical domains irq_chip_retrigger_hierarchy() is invoked anyway. See try_retrigger().

The IMSIC driver was merged one kernel release before common MSI LIB was merged.

Quoted 6 lines

Ah indeed.

Quoted 2 lines

We should definitely update the IMSIC driver to use MSI LIB, I will try your suggested changes (below) and post a separate series.

Pick up the delta patch I gave Andrew...

As I was looking at something else MSI related I had a look at imsic_irq_set_affinity() again.

It's actually required to have the message write in that function and not afterwards as you invoke imsic_vector_move() from that function.

That's obviously not true for the remap case as that will not change the message address/data pair because the remap table entry is immutable - at least I assume so for my mental sanity sake :)

But that brings me to a related question. How is this supposed to work with non-atomic message updates? PCI/MSI does not necessarily provide masking, and the write of the address/data pair is done in bits and pieces. So you can end up with an intermediate state seen by the device which ends up somewhere in interrupt nirvana space.

See the dance in msi_set_affinity() and commit 6f1a4891a592 ("x86/apic/msi: Plug non-maskable MSI affinity race") for further explanation.

The way how the IMSIC driver works seems to be pretty much the same as the x86 APIC mess:

@address is the physical address of the per CPU MSI target address and @data is the vector ID on that CPU.

So the non-atomic update in case of non-maskable MSI suffers from the same problem. It works most of the time, but if it doesn't you might stare at the occasionally lost interrupt and the stale device in disbelief for quite a while :)

I might be missing something which magically prevent that though :)

Thanks,

tglx

maintainer
#9 lore raw

Re: [RFC PATCH 01/15] irqchip/riscv-imsic: Use hierarchy to reach irq_set_affinity

Quoted 49 lines · Thomas Gleixner
Quoted 17 lines · Thomas Gleixner
Quoted 7 lines · Anup Patel
Quoted 6 lines · Thomas Gleixner

Sorry, I missed that when reviewing the original IMSIC MSI support.

The whole IMSIC MSI support can be moved over to MSI LIB which makes all of this indirection go away and your intermediate domain will just fit in.

Quoted 8 lines
Quoted 5 lines

Uncompiled patch below. If that works, it needs to be split up properly.

Note, this removes the setup of the irq_retrigger callback, but that's fine because on hierarchical domains irq_chip_retrigger_hierarchy() is invoked anyway. See try_retrigger().

The IMSIC driver was merged one kernel release before common MSI LIB was merged.

Quoted 6 lines

Ah indeed.

Quoted 2 lines

We should definitely update the IMSIC driver to use MSI LIB, I will try your suggested changes (below) and post a separate series.

Pick up the delta patch I gave Andrew...

As I was looking at something else MSI related I had a look at imsic_irq_set_affinity() again.

It's actually required to have the message write in that function and not afterwards as you invoke imsic_vector_move() from that function.

That's obviously not true for the remap case as that will not change the message address/data pair because the remap table entry is immutable - at least I assume so for my mental sanity sake :)

But that brings me to a related question. How is this supposed to work with non-atomic message updates? PCI/MSI does not necessarily provide masking, and the write of the address/data pair is done in bits and pieces. So you can end up with an intermediate state seen by the device which ends up somewhere in interrupt nirvana space.

See the dance in msi_set_affinity() and commit 6f1a4891a592 ("x86/apic/msi: Plug non-maskable MSI affinity race") for further explanation.

The way how the IMSIC driver works seems to be pretty much the same as the x86 APIC mess:

Quoted 7 lines

@address is the physical address of the per CPU MSI target address and @data is the vector ID on that CPU.

So the non-atomic update in case of non-maskable MSI suffers from the same problem. It works most of the time, but if it doesn't you might stare at the occasionally lost interrupt and the stale device in disbelief for quite a while :)

Yes, we have the same challenges as x86 APIC when changing MSI affinity.

Quoted 3 lines

I might be missing something which magically prevent that though :)

Your understanding is correct. In fact, the IMSIC msi_set_affinity() handling is inspired from x86 APIC approach due to similarity in the overall MSI controller.

The high-level idea of imsic_irq_set_affinity() is as follows:

1) Allocate new_vector (new CPU IMSIC address + new ID on that CPU)

2) Update the MSI address and data programmed in the device based on new_vector (see imsic_msi_update_msg())

3) At this point the device points to the new_vector but old_vector (old CPU IMSIC address + old ID on that CPU) is still enabled and we might have received MSI on old_vector while we were busy setting up a new_vector for the device. To address this, we call imsic_vector_move().

4) The imsic_vector_move() marks the old_vector as being moved and schedules a lazy timer on the old CPU.

5) The lazy timer expires on the old CPU and results in __imsic_local_sync() being called on the old CPU.

6) If there was a pending MSI on the old vector then the __imsic_local_sync() function injects an MSI to the new_vector using an MMIO write.

It is very unlikely that an MSI from device will be dropped (unless I am missing something) but the unsolved issue is that handling of in-flight MSI received on the old_vector during the MSI re-programming is delayed which may have side effects on the device driver side.

I believe in the future RISC-V AIA v2.0 (whenever that happens) will address the gaps in AIA v1.0 (like this one).

Regards,
Anup

#10 lore raw

Re: [RFC PATCH 01/15] irqchip/riscv-imsic: Use hierarchy to reach irq_set_affinity

Quoted 64 lines · Anup Patel
Quoted 23 lines · Thomas Gleixner

As I was looking at something else MSI related I had a look at imsic_irq_set_affinity() again.

It's actually required to have the message write in that function and not afterwards as you invoke imsic_vector_move() from that function.

That's obviously not true for the remap case as that will not change the message address/data pair because the remap table entry is immutable - at least I assume so for my mental sanity sake :)

But that brings me to a related question. How is this supposed to work with non-atomic message updates? PCI/MSI does not necessarily provide masking, and the write of the address/data pair is done in bits and pieces. So you can end up with an intermediate state seen by the device which ends up somewhere in interrupt nirvana space.

See the dance in msi_set_affinity() and commit 6f1a4891a592 ("x86/apic/msi: Plug non-maskable MSI affinity race") for further explanation.

The way how the IMSIC driver works seems to be pretty much the same as the x86 APIC mess:

Quoted 7 lines

@address is the physical address of the per CPU MSI target address and @data is the vector ID on that CPU.

So the non-atomic update in case of non-maskable MSI suffers from the same problem. It works most of the time, but if it doesn't you might stare at the occasionally lost interrupt and the stale device in disbelief for quite a while :)

Yes, we have the same challenges as x86 APIC when changing MSI affinity.

Quoted 3 lines

I might be missing something which magically prevent that though :)

Your understanding is correct. In fact, the IMSIC msi_set_affinity() handling is inspired from x86 APIC approach due to similarity in the overall MSI controller.

The high-level idea of imsic_irq_set_affinity() is as follows:

1) Allocate new_vector (new CPU IMSIC address + new ID on that CPU)

2) Update the MSI address and data programmed in the device based on new_vector (see imsic_msi_update_msg())

3) At this point the device points to the new_vector but old_vector (old CPU IMSIC address + old ID on that CPU) is still enabled and we might have received MSI on old_vector while we were busy setting up a new_vector for the device. To address this, we call imsic_vector_move().

4) The imsic_vector_move() marks the old_vector as being moved and schedules a lazy timer on the old CPU.

5) The lazy timer expires on the old CPU and results in __imsic_local_sync() being called on the old CPU.

6) If there was a pending MSI on the old vector then the __imsic_local_sync() function injects an MSI to the new_vector using an MMIO write.

Quoted 5 lines

It is very unlikely that an MSI from device will be dropped (unless I am missing something) but the unsolved issue is that handling of in-flight MSI received on the old_vector during the MSI re-programming is delayed which may have side effects on the device driver side.

Interrupt delivery can be delayed for tons of other reasons.

But yes, you are missing something which is worse than a jiffie delay:

CPU Device
msi_update_msg()

     compose()
     write()
       write(msg->address_lo); [1]
       write(msg->address_hi); [2]
                                        Raises interrupt [3]
       write(msg->data);       [4]

[2] can be ignored as it should not change (otherwise 32bit only devices would not work).

Lets assume that the original message was:

     addr_lo = 0x1000, data = 0x10    (CPU1, vector 0x10)

The new message is

     addr_lo = 0x2000, data = 0x20    (CPU2, vector 0x20)

After [2] the device sees:

     addr_lo = 0x2000, data = 0x10    (CPU2, vector 0x10)

The interrupt raised in [3] will end up on CPU2 at vector 0x10 which might be not in use or used by some other device. In any case the interrupt is not reaching the real device handler and you can't see that interrupt as pending in CPU1.

That's why x86 in case of non-remapped interrupts has this for the two situations:

1) old->data == new->data

write_msg(new);

The next interrupt either arrives on the old CPU or on the new CPU depending on timing. There is no intermediate state because the vector (data) is the same on both CPUs.

2) old->data != new->data

tmp_msg.addr = old_msg.addr
tmp_msg.data = new_msg.data

write_msg(tmp_msg);

So after that write the device might raise the interrupt on CPU1 and vector 0x20.

The next step is

write_msg(new);

which changes the destination CPU to CPU2.

So depending what the device has observed the interrupt might end up on

CPU1 vector 0x10 (old)
CPU1 vector 0x20 (tmp)
CPU2 vector 0x20 (new)

CPU1 vector 0x20 (tmp) is then checked in the pending register and the interrupt is retriggered if pending.

That requires to move the interrupt from actual interrupt context on the old target CPU. It allows to evaluate the old target CPUs pending register with local interrupts disabled, which obviously does not work remote.

I don't see a way how that can work remote with the IMSIC either even if you can easily access the pending state of the remote CPU:

CPU0 CPU1 Device
set_affinity()
write_msg(tmp)

    write(addr); // CPU1
    write(data); // vector 0x20
                                                        raise IRQ
                                handle vector 0x20
                                (spurious or other device)

    check_pending(CPU1, 0x20) == false -> Interrupt is lost

Remapped interrupts do not have that issue because the table update is atomic vs. a concurrently raised interrupt, so that it either ends up on the old or on the new destination. The device message does not change as it always targets the table entry, which is immutable after setup.

That's why x86 has this special msi affinity setter for the non remapped case. For the remapped case it's just using the hierarchy default which ends up at the remap domain.

So the hierarchies look like this:

1) non-remap

PCI/MSI device domain
irq_chip.irq_set_affinity = msi_set_affinity;
VECTOR domain

2) remap

      PCI/MSI device domain
         irq_chip.irq_set_affinity = msi_domain_set_affinity;
      REMAP domain
         irq_chip.irq_set_affinity = remap_set_affinity;
      VECTOR domain

The special case of msi_set_affinity() is not involved in the remap case and solely utilized for the non-remap horrors. The remap case does not require the interrupt to be moved in interrupt context either because there is no intermediate step and pending register check on the old CPU involved.

The vector cleanup of the old CPU always happens when the interrupt arrives on the new target for the first time independent of remapping mode. That's required for both cases to take care of the case where the interrupt is raised on the old vector (cpu1, 0x10) before the write happens and is pending in CPU1. So after desc::lock is released and interrupts are reenabled the interrupt is handled on CPU1. The next one is guaranteed to arrive on CPU2 and that triggers the clean up of the CPU1 vector, which releases it for reuse in the matrix allocator.

I think you should model it in a similar way instead of trying to artificially reuse imsic_irq_set_affinity() for the remap case, especially when you decide to close the non-maskable MSI hole described above, which I recommend to do :)

You still can utilize msi-lib and just have your private implementation of init_dev_msi_info() as a wrapper around the library similar to mbi_init_dev_msi_info() and its_init_dev_msi_info(). That wrapper would just handle the non-remap case to set info::chip:irq_set_affinity to the magic non-remap function.

Hope that helps.

Quoted 2 lines

I believe in the future RISC-V AIA v2.0 (whenever that happens) will address the gaps in AIA v1.0 (like this one).

If that is strictly translation table based, yes.

Thanks,

tglx

#11 lore raw

RFC PATCH 2/15 genirq/msi: Provide DOMAIN_BUS_MSI_REMAP

In reply to Andrew Jones

Provide a domain bus token for the upcoming support for the RISC-V IOMMU interrupt remapping domain, which needs to be distinguished from NEXUS domains. The new token name is generic, as the only information that needs to be conveyed is that the IRQ domain will remap MSIs, i.e. there's nothing RISC-V specific to convey.

---
 include/linux/irqdomain_defs.h | 1 +
 1 file changed, 1 insertion(+)
include/linux/irqdomain_defs.h +1 -0
index 36653e2ee1c9..676eca8147ae 100644
@@ -27,6 +27,7 @@ enum irq_domain_bus_token {
 	DOMAIN_BUS_AMDVI,
 	DOMAIN_BUS_DEVICE_MSI,
 	DOMAIN_BUS_WIRED_TO_MSI,
+	DOMAIN_BUS_MSI_REMAP,
 };
 
 #endif /* _LINUX_IRQDOMAIN_DEFS_H */
Signature · 1 line
2.47.0
#12 lore raw

RFC PATCH 3/15 irqchip/riscv-imsic: Add support for DOMAIN_BUS_MSI_REMAP

In reply to Andrew Jones

Unlike a child of an MSI NEXUS domain, a child of an MSI_REMAP domain will not invoke init_dev_msi_info() with 'domain' equal to 'msi_parent_domain'. This is because the MSI_REMAP domain implements init_dev_msi_info() with msi_parent_init_dev_msi_info(), which makes 'domain' point to the NEXUS (IMSIC) domain, while keeping 'msi_parent_domain' pointing to itself. The rest of the IMSIC init_dev_msi_info() implementation works for MSI_REMAP domains, though, so there's nothing to do to add support except accept the token.

---
 drivers/irqchip/irq-riscv-imsic-platform.c | 2 ++
 1 file changed, 2 insertions(+)
drivers/irqchip/irq-riscv-imsic-platform.c +2 -0
index 5d7c30ad8855..6a7d7fefda6a 100644
@@ -246,6 +246,8 @@ static bool imsic_init_dev_msi_info(struct device *dev,
 	case DOMAIN_BUS_NEXUS:
 		if (WARN_ON_ONCE(domain != real_parent))
 			return false;
+		fallthrough;
+	case DOMAIN_BUS_MSI_REMAP:
 #ifdef CONFIG_SMP
 		info->chip->irq_set_affinity = irq_chip_set_affinity_parent;
 #endif
Signature · 1 line
2.47.0
#13 lore raw

RFC PATCH 4/15 iommu/riscv: report iommu capabilities

In reply to Andrew Jones

From: Tomasz Jeznach <[email protected]>

Report RISC-V IOMMU capabilities required by VFIO subsystem to enable PCIe device assignment.

---
 drivers/iommu/riscv/iommu.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)
drivers/iommu/riscv/iommu.c +12 -0
index 8a05def774bd..3fe4ceba8dd3 100644
@@ -1462,6 +1462,17 @@ static struct iommu_group *riscv_iommu_device_group(struct device *dev)
 	return generic_device_group(dev);
 }
 
+static bool riscv_iommu_capable(struct device *dev, enum iommu_cap cap)
+{
+	switch (cap) {
+	case IOMMU_CAP_CACHE_COHERENCY:
+	case IOMMU_CAP_DEFERRED_FLUSH:
+		return true;
+	default:
+		return false;
+	}
+}
+
 static int riscv_iommu_of_xlate(struct device *dev, const struct of_phandle_args *args)
 {
 	return iommu_fwspec_add_ids(dev, args->args, 1);
@@ -1526,6 +1537,7 @@ static void riscv_iommu_release_device(struct device *dev)
 static const struct iommu_ops riscv_iommu_ops = {
 	.pgsize_bitmap = SZ_4K,
 	.of_xlate = riscv_iommu_of_xlate,
+	.capable = riscv_iommu_capable,
 	.identity_domain = &riscv_iommu_identity_domain,
 	.blocked_domain = &riscv_iommu_blocking_domain,
 	.release_domain = &riscv_iommu_blocking_domain,
Signature · 1 line
2.47.0
reviewer
#14 lore raw

Re: [RFC PATCH 04/15] iommu/riscv: report iommu capabilities

Quoted 4 lines · Andrew Jones

From: Tomasz Jeznach <[email protected]>

Report RISC-V IOMMU capabilities required by VFIO subsystem to enable PCIe device assignment.

IOMMU_CAP_DEFERRED_FLUSH has nothing at all to do with VFIO. As far as I can tell from what's queued, riscv_iommu_unmap_pages() isn't really implementing the full optimisation to get the most out of it either.

I guess IOMMU_CAP_CACHE_COHERENCY falls out of the assumption of a coherent IOMMU and lack of PBMT support making everything implicitly IOMMU_CACHE all the time whether you want it or not, but clarifying that might be nice (especially since there's some chance that something will eventually come along to break it...)

Thanks,
Robin.

Quoted 36 lines
---
  drivers/iommu/riscv/iommu.c | 12 ++++++++++++
  1 file changed, 12 insertions(+)
drivers/iommu/riscv/iommu.c +12 -0
index 8a05def774bd..3fe4ceba8dd3 100644
@@ -1462,6 +1462,17 @@ static struct iommu_group *riscv_iommu_device_group(struct device *dev)
  	return generic_device_group(dev);
  }
 
+static bool riscv_iommu_capable(struct device *dev, enum iommu_cap cap)
+{
+	switch (cap) {
+	case IOMMU_CAP_CACHE_COHERENCY:
+	case IOMMU_CAP_DEFERRED_FLUSH:
+		return true;
+	default:
+		return false;
+	}
+}
+
  static int riscv_iommu_of_xlate(struct device *dev, const struct of_phandle_args *args)
  {
  	return iommu_fwspec_add_ids(dev, args->args, 1);
@@ -1526,6 +1537,7 @@ static void riscv_iommu_release_device(struct device *dev)
  static const struct iommu_ops riscv_iommu_ops = {
  	.pgsize_bitmap = SZ_4K,
  	.of_xlate = riscv_iommu_of_xlate,
+	.capable = riscv_iommu_capable,
  	.identity_domain = &riscv_iommu_identity_domain,
  	.blocked_domain = &riscv_iommu_blocking_domain,
  	.release_domain = &riscv_iommu_blocking_domain,
#15 lore raw

Re: [RFC PATCH 04/15] iommu/riscv: report iommu capabilities

Quoted 9 lines · Robin Murphy
Quoted 4 lines · Andrew Jones

From: Tomasz Jeznach <[email protected]>

Report RISC-V IOMMU capabilities required by VFIO subsystem to enable PCIe device assignment.

IOMMU_CAP_DEFERRED_FLUSH has nothing at all to do with VFIO. As far as I can tell from what's queued, riscv_iommu_unmap_pages() isn't really implementing the full optimisation to get the most out of it either.

Thanks, Robin. I'll drop this cap for the next version.

Quoted 6 lines

I guess IOMMU_CAP_CACHE_COHERENCY falls out of the assumption of a coherent IOMMU and lack of PBMT support making everything implicitly IOMMU_CACHE all the time whether you want it or not, but clarifying that might be nice (especially since there's some chance that something will eventually come along to break it...)

Yes, riscv selects ARCH_DMA_DEFAULT_COHERENT and the riscv IOMMU hardware descriptions don't provide any way to say otherwise. I can put a comment above the IOMMU_CAP_CACHE_COHERENCY case which states "The RISC-V IOMMU is always DMA cache coherent", or did you have something else in mind?

Thanks,
drew

Quoted 39 lines

Thanks,
Robin.

Quoted 35 lines
---
  drivers/iommu/riscv/iommu.c | 12 ++++++++++++
  1 file changed, 12 insertions(+)
drivers/iommu/riscv/iommu.c +12 -0
index 8a05def774bd..3fe4ceba8dd3 100644
@@ -1462,6 +1462,17 @@ static struct iommu_group *riscv_iommu_device_group(struct device *dev)
  	return generic_device_group(dev);
  }
+static bool riscv_iommu_capable(struct device *dev, enum iommu_cap cap)
+{
+	switch (cap) {
+	case IOMMU_CAP_CACHE_COHERENCY:
+	case IOMMU_CAP_DEFERRED_FLUSH:
+		return true;
+	default:
+		return false;
+	}
+}
+
  static int riscv_iommu_of_xlate(struct device *dev, const struct of_phandle_args *args)
  {
  	return iommu_fwspec_add_ids(dev, args->args, 1);
@@ -1526,6 +1537,7 @@ static void riscv_iommu_release_device(struct device *dev)
  static const struct iommu_ops riscv_iommu_ops = {
  	.pgsize_bitmap = SZ_4K,
  	.of_xlate = riscv_iommu_of_xlate,
+	.capable = riscv_iommu_capable,
  	.identity_domain = &riscv_iommu_identity_domain,
  	.blocked_domain = &riscv_iommu_blocking_domain,
  	.release_domain = &riscv_iommu_blocking_domain,
#16 lore raw

RFC PATCH 5/15 iommu/riscv: use data structure instead of individual values

In reply to Andrew Jones

From: Zong Li <[email protected]>

The parameter will be increased when we need to set up more bit fields in the device context. Use a data structure to wrap them up.

---
 drivers/iommu/riscv/iommu.c | 31 +++++++++++++++++++------------
 1 file changed, 19 insertions(+), 12 deletions(-)
drivers/iommu/riscv/iommu.c +19 -12
index 3fe4ceba8dd3..9d7945dc3c24 100644
@@ -1001,7 +1001,7 @@ static void riscv_iommu_iotlb_inval(struct riscv_iommu_domain *domain,
  * interim translation faults.
  */
 static void riscv_iommu_iodir_update(struct riscv_iommu_device *iommu,
-				     struct device *dev, u64 fsc, u64 ta)
+				     struct device *dev, struct riscv_iommu_dc *new_dc)
 {
 	struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev);
 	struct riscv_iommu_dc *dc;
@@ -1035,10 +1035,10 @@ static void riscv_iommu_iodir_update(struct riscv_iommu_device *iommu,
 	for (i = 0; i < fwspec->num_ids; i++) {
 		dc = riscv_iommu_get_dc(iommu, fwspec->ids[i]);
 		tc = READ_ONCE(dc->tc);
-		tc |= ta & RISCV_IOMMU_DC_TC_V;
+		tc |= new_dc->ta & RISCV_IOMMU_DC_TC_V;
 
-		WRITE_ONCE(dc->fsc, fsc);
-		WRITE_ONCE(dc->ta, ta & RISCV_IOMMU_PC_TA_PSCID);
+		WRITE_ONCE(dc->fsc, new_dc->fsc);
+		WRITE_ONCE(dc->ta, new_dc->ta & RISCV_IOMMU_PC_TA_PSCID);
 		/* Update device context, write TC.V as the last step. */
 		dma_wmb();
 		WRITE_ONCE(dc->tc, tc);
@@ -1315,20 +1315,20 @@ static int riscv_iommu_attach_paging_domain(struct iommu_domain *iommu_domain,
 	struct riscv_iommu_domain *domain = iommu_domain_to_riscv(iommu_domain);
 	struct riscv_iommu_device *iommu = dev_to_iommu(dev);
 	struct riscv_iommu_info *info = dev_iommu_priv_get(dev);
-	u64 fsc, ta;
+	struct riscv_iommu_dc dc = {0};
 
 	if (!riscv_iommu_pt_supported(iommu, domain->pgd_mode))
 		return -ENODEV;
 
-	fsc = FIELD_PREP(RISCV_IOMMU_PC_FSC_MODE, domain->pgd_mode) |
-	      FIELD_PREP(RISCV_IOMMU_PC_FSC_PPN, virt_to_pfn(domain->pgd_root));
-	ta = FIELD_PREP(RISCV_IOMMU_PC_TA_PSCID, domain->pscid) |
-	     RISCV_IOMMU_PC_TA_V;
+	dc.fsc = FIELD_PREP(RISCV_IOMMU_PC_FSC_MODE, domain->pgd_mode) |
+		 FIELD_PREP(RISCV_IOMMU_PC_FSC_PPN, virt_to_pfn(domain->pgd_root));
+	dc.ta = FIELD_PREP(RISCV_IOMMU_PC_TA_PSCID, domain->pscid) |
+			   RISCV_IOMMU_PC_TA_V;
 
 	if (riscv_iommu_bond_link(domain, dev))
 		return -ENOMEM;
 
-	riscv_iommu_iodir_update(iommu, dev, fsc, ta);
+	riscv_iommu_iodir_update(iommu, dev, &dc);
 	riscv_iommu_bond_unlink(info->domain, dev);
 	info->domain = domain;
 
@@ -1419,9 +1419,12 @@ static int riscv_iommu_attach_blocking_domain(struct iommu_domain *iommu_domain,
 {
 	struct riscv_iommu_device *iommu = dev_to_iommu(dev);
 	struct riscv_iommu_info *info = dev_iommu_priv_get(dev);
+	struct riscv_iommu_dc dc = {0};
+
+	dc.fsc = RISCV_IOMMU_FSC_BARE;
 
 	/* Make device context invalid, translation requests will fault w/ #258 */
-	riscv_iommu_iodir_update(iommu, dev, RISCV_IOMMU_FSC_BARE, 0);
+	riscv_iommu_iodir_update(iommu, dev, &dc);
 	riscv_iommu_bond_unlink(info->domain, dev);
 	info->domain = NULL;
 
@@ -1440,8 +1443,12 @@ static int riscv_iommu_attach_identity_domain(struct iommu_domain *iommu_domain,
 {
 	struct riscv_iommu_device *iommu = dev_to_iommu(dev);
 	struct riscv_iommu_info *info = dev_iommu_priv_get(dev);
+	struct riscv_iommu_dc dc = {0};
+
+	dc.fsc = RISCV_IOMMU_FSC_BARE;
+	dc.ta = RISCV_IOMMU_PC_TA_V;
 
-	riscv_iommu_iodir_update(iommu, dev, RISCV_IOMMU_FSC_BARE, RISCV_IOMMU_PC_TA_V);
+	riscv_iommu_iodir_update(iommu, dev, &dc);
 	riscv_iommu_bond_unlink(info->domain, dev);
 	info->domain = NULL;
 
Signature · 1 line
2.47.0
#17 lore raw

RFC PATCH 6/15 iommu/riscv: support GSCID and GVMA invalidation command

In reply to Andrew Jones

From: Zong Li <[email protected]>

This patch adds a ID Allocator for GSCID and a wrap for setting up GSCID in IOTLB invalidation command.

Set up iohgatp to enable second stage table and flush stage-2 table if the GSCID is set.

The GSCID of domain should be freed when release domain. GSCID will be allocated for parent domain in nested IOMMU process.

---
 drivers/iommu/riscv/iommu-bits.h |  7 +++++++
 drivers/iommu/riscv/iommu.c      | 32 ++++++++++++++++++++++++++------
 2 files changed, 33 insertions(+), 6 deletions(-)
drivers/iommu/riscv/iommu-bits.h +7 -0
index 98daf0e1a306..d72b982cf9bf 100644
@@ -715,6 +715,13 @@ static inline void riscv_iommu_cmd_inval_vma(struct riscv_iommu_command *cmd)
 	cmd->dword1 = 0;
 }
 
+static inline void riscv_iommu_cmd_inval_gvma(struct riscv_iommu_command *cmd)
+{
+	cmd->dword0 = FIELD_PREP(RISCV_IOMMU_CMD_OPCODE, RISCV_IOMMU_CMD_IOTINVAL_OPCODE) |
+		      FIELD_PREP(RISCV_IOMMU_CMD_FUNC, RISCV_IOMMU_CMD_IOTINVAL_FUNC_GVMA);
+	cmd->dword1 = 0;
+}
+
 static inline void riscv_iommu_cmd_inval_set_addr(struct riscv_iommu_command *cmd,
 						  u64 addr)
 {
drivers/iommu/riscv/iommu.c +26 -6
index 9d7945dc3c24..ef38a1bb3eca 100644
@@ -45,6 +45,10 @@
 static DEFINE_IDA(riscv_iommu_pscids);
 #define RISCV_IOMMU_MAX_PSCID		(BIT(20) - 1)
 
+/* IOMMU GSCID allocation namespace. */
+static DEFINE_IDA(riscv_iommu_gscids);
+#define RISCV_IOMMU_MAX_GSCID		(BIT(16) - 1)
+
 /* Device resource-managed allocations */
 struct riscv_iommu_devres {
 	void *addr;
@@ -801,6 +805,7 @@ struct riscv_iommu_domain {
 	struct list_head bonds;
 	spinlock_t lock;		/* protect bonds list updates. */
 	int pscid;
+	int gscid;
 	bool amo_enabled;
 	int numa_node;
 	unsigned int pgd_mode;
@@ -954,15 +959,20 @@ static void riscv_iommu_iotlb_inval(struct riscv_iommu_domain *domain,
 
 		/*
 		 * IOTLB invalidation request can be safely omitted if already sent
-		 * to the IOMMU for the same PSCID, and with domain->bonds list
+		 * to the IOMMU for the same PSCID/GSCID, and with domain->bonds list
 		 * arranged based on the device's IOMMU, it's sufficient to check
 		 * last device the invalidation was sent to.
 		 */
 		if (iommu == prev)
 			continue;
 
-		riscv_iommu_cmd_inval_vma(&cmd);
-		riscv_iommu_cmd_inval_set_pscid(&cmd, domain->pscid);
+		if (domain->gscid) {
+			riscv_iommu_cmd_inval_gvma(&cmd);
+			riscv_iommu_cmd_inval_set_gscid(&cmd, domain->gscid);
+		} else {
+			riscv_iommu_cmd_inval_vma(&cmd);
+			riscv_iommu_cmd_inval_set_pscid(&cmd, domain->pscid);
+		}
 		if (len && len < RISCV_IOMMU_IOTLB_INVAL_LIMIT) {
 			for (iova = start; iova < end; iova += PAGE_SIZE) {
 				riscv_iommu_cmd_inval_set_addr(&cmd, iova);
@@ -1039,6 +1049,7 @@ static void riscv_iommu_iodir_update(struct riscv_iommu_device *iommu,
 
 		WRITE_ONCE(dc->fsc, new_dc->fsc);
 		WRITE_ONCE(dc->ta, new_dc->ta & RISCV_IOMMU_PC_TA_PSCID);
+		WRITE_ONCE(dc->iohgatp, new_dc->iohgatp);
 		/* Update device context, write TC.V as the last step. */
 		dma_wmb();
 		WRITE_ONCE(dc->tc, tc);
@@ -1287,8 +1298,10 @@ static void riscv_iommu_free_paging_domain(struct iommu_domain *iommu_domain)
 
 	WARN_ON(!list_empty(&domain->bonds));
 
-	if ((int)domain->pscid > 0)
+	if (domain->pscid > 0)
 		ida_free(&riscv_iommu_pscids, domain->pscid);
+	if (domain->gscid > 0)
+		ida_free(&riscv_iommu_gscids, domain->gscid);
 
 	riscv_iommu_pte_free(domain, _io_pte_entry(pfn, _PAGE_TABLE), NULL);
 	kfree(domain);
@@ -1320,8 +1333,15 @@ static int riscv_iommu_attach_paging_domain(struct iommu_domain *iommu_domain,
 	if (!riscv_iommu_pt_supported(iommu, domain->pgd_mode))
 		return -ENODEV;
 
-	dc.fsc = FIELD_PREP(RISCV_IOMMU_PC_FSC_MODE, domain->pgd_mode) |
-		 FIELD_PREP(RISCV_IOMMU_PC_FSC_PPN, virt_to_pfn(domain->pgd_root));
+	if (domain->gscid) {
+		dc.iohgatp = FIELD_PREP(RISCV_IOMMU_DC_IOHGATP_MODE, domain->pgd_mode) |
+			     FIELD_PREP(RISCV_IOMMU_DC_IOHGATP_GSCID, domain->gscid) |
+			     FIELD_PREP(RISCV_IOMMU_DC_IOHGATP_PPN, virt_to_pfn(domain->pgd_root));
+	} else {
+		dc.fsc = FIELD_PREP(RISCV_IOMMU_PC_FSC_MODE, domain->pgd_mode) |
+			 FIELD_PREP(RISCV_IOMMU_PC_FSC_PPN, virt_to_pfn(domain->pgd_root));
+	}
+
 	dc.ta = FIELD_PREP(RISCV_IOMMU_PC_TA_PSCID, domain->pscid) |
 			   RISCV_IOMMU_PC_TA_V;
 
Signature · 1 line
2.47.0
#18 lore raw

RFC PATCH 7/15 iommu/riscv: Move definitions to iommu.h

In reply to Andrew Jones

In order to add the interrupt remapping support in a separate file, share definitions through the header, as well as making some functions public.

---
 drivers/iommu/riscv/iommu-bits.h |  4 ++
 drivers/iommu/riscv/iommu.c      | 71 ++++----------------------------
 drivers/iommu/riscv/iommu.h      | 54 ++++++++++++++++++++++++
 3 files changed, 67 insertions(+), 62 deletions(-)
drivers/iommu/riscv/iommu-bits.h +4 -0
index d72b982cf9bf..d3d98dbed709 100644
@@ -36,6 +36,10 @@
 #define RISCV_IOMMU_ATP_PPN_FIELD	GENMASK_ULL(43, 0)
 #define RISCV_IOMMU_ATP_MODE_FIELD	GENMASK_ULL(63, 60)
 
+/* RISC-V IOMMU PPN <> PHYS address conversions, PHYS <=> PPN[53:10] */
+#define riscv_iommu_phys_to_ppn(pa)	(((pa) >> 2) & (((1ULL << 44) - 1) << 10))
+#define riscv_iommu_ppn_to_phys(pn)	(((pn) << 2) & (((1ULL << 44) - 1) << 12))
+
 /* 5.3 IOMMU Capabilities (64bits) */
 #define RISCV_IOMMU_REG_CAPABILITIES		0x0000
 #define RISCV_IOMMU_CAPABILITIES_VERSION	GENMASK_ULL(7, 0)
drivers/iommu/riscv/iommu.c +9 -62

131 lines not on this page · raw message

drivers/iommu/riscv/iommu.h +54 -0

71 lines not on this page · raw message

Signature · 1 line
2.47.0
#19 lore raw

RFC PATCH 8/15 iommu/riscv: Add IRQ domain for interrupt remapping

In reply to Andrew Jones

This is just a skeleton. Until irq_set_vcpu_affinity() is implemented the IRQ domain doesn't serve any purpose.

---
 drivers/iommu/riscv/Makefile   |   2 +-
 drivers/iommu/riscv/iommu-ir.c | 209 +++++++++++++++++++++++++++++++++
 drivers/iommu/riscv/iommu.c    |  43 ++++++-
 drivers/iommu/riscv/iommu.h    |  21 ++++
 4 files changed, 270 insertions(+), 5 deletions(-)
 create mode 100644 drivers/iommu/riscv/iommu-ir.c
drivers/iommu/riscv/Makefile +1 -1
index f54c9ed17d41..8420dd1776cb 100644
@@ -1,3 +1,3 @@
 # SPDX-License-Identifier: GPL-2.0-only
-obj-$(CONFIG_RISCV_IOMMU) += iommu.o iommu-platform.o
+obj-$(CONFIG_RISCV_IOMMU) += iommu.o iommu-ir.o iommu-platform.o
 obj-$(CONFIG_RISCV_IOMMU_PCI) += iommu-pci.o
drivers/iommu/riscv/iommu-ir.c new file +209 -0

209 lines not on this page · raw message

drivers/iommu/riscv/iommu.c +39 -4

102 lines not on this page · raw message

drivers/iommu/riscv/iommu.h +21 -0
index dd538b19fbb7..6ce71095781c 100644
@@ -23,6 +23,12 @@
 #define RISCV_IOMMU_DDTP_TIMEOUT	10000000
 #define RISCV_IOMMU_IOTINVAL_TIMEOUT	90000000
 
+struct riscv_iommu_msiptp_state {
+	u64 msiptp;
+	u64 msi_addr_mask;
+	u64 msi_addr_pattern;
+};
+
 /* This struct contains protection domain specific IOMMU driver data. */
 struct riscv_iommu_domain {
 	struct iommu_domain domain;
@@ -34,6 +40,13 @@ struct riscv_iommu_domain {
 	int numa_node;
 	unsigned int pgd_mode;
 	unsigned long *pgd_root;
+	u32 group_index_bits;
+	u32 group_index_shift;
+	int msi_order;
+	struct riscv_iommu_msipte *msi_root;
+	spinlock_t msi_lock;
+	struct riscv_iommu_msiptp_state msiptp;
+	struct irq_domain *irqdomain;
 };
 
 /* Private IOMMU data for managed devices, dev_iommu_priv_* */
@@ -119,6 +132,14 @@ void riscv_iommu_cmd_send(struct riscv_iommu_device *iommu,
 void riscv_iommu_cmd_sync(struct riscv_iommu_device *iommu,
 			  unsigned int timeout_us);
 
+int riscv_iommu_irq_domain_create(struct riscv_iommu_domain *domain,
+				  struct device *dev);
+void riscv_iommu_irq_domain_remove(struct riscv_iommu_domain *domain);
+void riscv_iommu_irq_domain_unlink(struct riscv_iommu_domain *domain,
+				   struct device *dev);
+void riscv_iommu_ir_get_resv_regions(struct device *dev,
+				     struct list_head *head);
+
 #define riscv_iommu_readl(iommu, addr) \
 	readl_relaxed((iommu)->reg + (addr))
 
Signature · 1 line
2.47.0
#20 lore raw

Re: [RFC PATCH 08/15] iommu/riscv: Add IRQ domain for interrupt remapping

drivers/iommu/riscv/iommu.c static int riscv_iommu_attach_paging_domain(struct iommu_domain *iommu_domain, · line 1290

Quoted 13 lines · Andrew Jones
drivers/iommu/riscv/iommu.c +5 -0
@@ -1276,10 +1279,30 @@ static int riscv_iommu_attach_paging_domain(struct iommu_domain *iommu_domain,
 	struct riscv_iommu_device *iommu = dev_to_iommu(dev);
 	struct riscv_iommu_info *info = dev_iommu_priv_get(dev);
 	struct riscv_iommu_dc dc = {0};
+	int ret;
 
 	if (!riscv_iommu_pt_supported(iommu, domain->pgd_mode))
 		return -ENODEV;
 
+	if (riscv_iommu_bond_link(domain, dev))
+		return -ENOMEM;
+
+	if (iommu_domain->type == IOMMU_DOMAIN_UNMANAGED) {

Drivers should not be making tests like this.

Quoted 4 lines
drivers/iommu/riscv/iommu.c +4 -0
+		domain->gscid = ida_alloc_range(&riscv_iommu_gscids, 1,
+						RISCV_IOMMU_MAX_GSCID, GFP_KERNEL);
+		if (domain->gscid < 0) {
+			riscv_iommu_bond_unlink(domain, dev);

drivers/iommu/riscv/iommu.c static int riscv_iommu_attach_paging_domain(struct iommu_domain *iommu_domain, · line 1304

Quoted 10 lines
drivers/iommu/riscv/iommu.c +10 -0
+			return -ENOMEM;
+		}
+
+		ret = riscv_iommu_irq_domain_create(domain, dev);
+		if (ret) {
+			riscv_iommu_bond_unlink(domain, dev);
+			ida_free(&riscv_iommu_gscids, domain->gscid);
+			return ret;
+		}
+	}

What are you trying to do? Make something behave different for VFIO? That isn't OK, we are trying to remove all the hacky VFIO special cases in drivers.

What is the HW issue here? It is very very strange (and probably not going to work right) that the irq domains change when domain attachment changes.

The IRQ setup should really be fixed before any device drivers probe onto the device.

Jason

#21 lore raw

Re: [RFC PATCH 08/15] iommu/riscv: Add IRQ domain for interrupt remapping

Quoted 36 lines · Jason Gunthorpe

drivers/iommu/riscv/iommu.c static int riscv_iommu_attach_paging_domain(struct iommu_domain *iommu_domain, · line 1290

Quoted 13 lines · Andrew Jones
drivers/iommu/riscv/iommu.c +5 -0
@@ -1276,10 +1279,30 @@ static int riscv_iommu_attach_paging_domain(struct iommu_domain *iommu_domain,
 	struct riscv_iommu_device *iommu = dev_to_iommu(dev);
 	struct riscv_iommu_info *info = dev_iommu_priv_get(dev);
 	struct riscv_iommu_dc dc = {0};
+	int ret;
 
 	if (!riscv_iommu_pt_supported(iommu, domain->pgd_mode))
 		return -ENODEV;
 
+	if (riscv_iommu_bond_link(domain, dev))
+		return -ENOMEM;
+
+	if (iommu_domain->type == IOMMU_DOMAIN_UNMANAGED) {

Drivers should not be making tests like this.

Quoted 4 lines
drivers/iommu/riscv/iommu.c +4 -0
+		domain->gscid = ida_alloc_range(&riscv_iommu_gscids, 1,
+						RISCV_IOMMU_MAX_GSCID, GFP_KERNEL);
+		if (domain->gscid < 0) {
+			riscv_iommu_bond_unlink(domain, dev);

drivers/iommu/riscv/iommu.c static int riscv_iommu_attach_paging_domain(struct iommu_domain *iommu_domain, · line 1304

Quoted 10 lines
drivers/iommu/riscv/iommu.c +10 -0
+			return -ENOMEM;
+		}
+
+		ret = riscv_iommu_irq_domain_create(domain, dev);
+		if (ret) {
+			riscv_iommu_bond_unlink(domain, dev);
+			ida_free(&riscv_iommu_gscids, domain->gscid);
+			return ret;
+		}
+	}

What are you trying to do? Make something behave different for VFIO? That isn't OK, we are trying to remove all the hacky VFIO special cases in drivers.

Quoted 6 lines

What is the HW issue here? It is very very strange (and probably not going to work right) that the irq domains change when domain attachment changes.

The IRQ setup should really be fixed before any device drivers probe onto the device.

I can't disagree with the statement that this looks hacky, but considering a VFIO domain needs to use the g-stage for its single-stage translation and a paging domain for the host would use s-stage, then it seems we need to identify the VFIO domains for their special treatment. Is there an example of converting VFIO special casing in other drivers to something cleaner that you can point me at?

The IRQ domain will only be useful for device assignment, as that's when an MSI translation will be needed. I can't think of any problems that could arise from only creating the IRQ domain when probing assigned devices, but I could certainly be missing something. Do you have some potential problems in mind?

Thanks,
drew

#22 lore raw

Re: [RFC PATCH 08/15] iommu/riscv: Add IRQ domain for interrupt remapping

Quoted 41 lines · Andrew Jones
Quoted 40 lines · Jason Gunthorpe

drivers/iommu/riscv/iommu.c static int riscv_iommu_attach_paging_domain(struct iommu_domain *iommu_domain, · line 1290

Quoted 13 lines · Andrew Jones
drivers/iommu/riscv/iommu.c +5 -0
@@ -1276,10 +1279,30 @@ static int riscv_iommu_attach_paging_domain(struct iommu_domain *iommu_domain,
 	struct riscv_iommu_device *iommu = dev_to_iommu(dev);
 	struct riscv_iommu_info *info = dev_iommu_priv_get(dev);
 	struct riscv_iommu_dc dc = {0};
+	int ret;
 
 	if (!riscv_iommu_pt_supported(iommu, domain->pgd_mode))
 		return -ENODEV;
 
+	if (riscv_iommu_bond_link(domain, dev))
+		return -ENOMEM;
+
+	if (iommu_domain->type == IOMMU_DOMAIN_UNMANAGED) {

Drivers should not be making tests like this.

Quoted 4 lines
drivers/iommu/riscv/iommu.c +4 -0
+		domain->gscid = ida_alloc_range(&riscv_iommu_gscids, 1,
+						RISCV_IOMMU_MAX_GSCID, GFP_KERNEL);
+		if (domain->gscid < 0) {
+			riscv_iommu_bond_unlink(domain, dev);

drivers/iommu/riscv/iommu.c static int riscv_iommu_attach_paging_domain(struct iommu_domain *iommu_domain, · line 1304

Quoted 10 lines
drivers/iommu/riscv/iommu.c +10 -0
+			return -ENOMEM;
+		}
+
+		ret = riscv_iommu_irq_domain_create(domain, dev);
+		if (ret) {
+			riscv_iommu_bond_unlink(domain, dev);
+			ida_free(&riscv_iommu_gscids, domain->gscid);
+			return ret;
+		}
+	}

What are you trying to do? Make something behave different for VFIO? That isn't OK, we are trying to remove all the hacky VFIO special cases in drivers.

What is the HW issue here? It is very very strange (and probably not going to work right) that the irq domains change when domain attachment changes.

Quoted 7 lines
Quoted 2 lines

The IRQ setup should really be fixed before any device drivers probe onto the device.

I can't disagree with the statement that this looks hacky, but considering a VFIO domain needs to use the g-stage for its single-stage translation and a paging domain for the host would use s-stage, then it seems we need to identify the VFIO domains for their special treatment.

This is the wrong thinking entirely. There is no such thing as a "VFIO domain".

Default VFIO created domains should act excatly the same as a DMA API domain.

If you want your system to have irq remapping, then it should be on by default and DMA API gets remapping too. There would need to be a very strong reason not to do that in order to make something special for riscv. If so you'd need to add some kind of flag to select it.

Until you reach nested translation there is no "need" for VFIO to use any particular stage. The design is that default VFIO uses the same stage as the DMA API because it is doing the same basic default translation function.

Nested translation has a control to select the stage, and you can then force the g-stage for VFIO users at that point.

Regardless, you must not use UNMANAGED as some indication of VFIO, that is not what it means, that is not what it is for.

Quoted 2 lines

Is there an example of converting VFIO special casing in other drivers to something cleaner that you can point me at?

Nobody has had an issue where they want interrupt remapping on/off depending on VFIO. I think that is inherently wrong.

Quoted 5 lines

The IRQ domain will only be useful for device assignment, as that's when an MSI translation will be needed. I can't think of any problems that could arise from only creating the IRQ domain when probing assigned devices, but I could certainly be missing something. Do you have some potential problems in mind?

I'm not an expert in the interrupt subsystem, but my understanding was we expect the interrupt domains/etc to be static once a device driver is probed. Changing things during iommu domain attach is after drivers are probed. I don't really expect it to work correctly in all corner cases.

VFIO is allowed to change the translation as it operates and we expect that interrupts are not disturbed.

Jason

#23 lore raw

Re: [RFC PATCH 08/15] iommu/riscv: Add IRQ domain for interrupt remapping

Quoted 61 lines · Jason Gunthorpe
Quoted 41 lines · Andrew Jones
Quoted 40 lines · Jason Gunthorpe

drivers/iommu/riscv/iommu.c static int riscv_iommu_attach_paging_domain(struct iommu_domain *iommu_domain, · line 1290

Quoted 13 lines · Andrew Jones
drivers/iommu/riscv/iommu.c +5 -0
@@ -1276,10 +1279,30 @@ static int riscv_iommu_attach_paging_domain(struct iommu_domain *iommu_domain,
 	struct riscv_iommu_device *iommu = dev_to_iommu(dev);
 	struct riscv_iommu_info *info = dev_iommu_priv_get(dev);
 	struct riscv_iommu_dc dc = {0};
+	int ret;
 
 	if (!riscv_iommu_pt_supported(iommu, domain->pgd_mode))
 		return -ENODEV;
 
+	if (riscv_iommu_bond_link(domain, dev))
+		return -ENOMEM;
+
+	if (iommu_domain->type == IOMMU_DOMAIN_UNMANAGED) {

Drivers should not be making tests like this.

Quoted 4 lines
drivers/iommu/riscv/iommu.c +4 -0
+		domain->gscid = ida_alloc_range(&riscv_iommu_gscids, 1,
+						RISCV_IOMMU_MAX_GSCID, GFP_KERNEL);
+		if (domain->gscid < 0) {
+			riscv_iommu_bond_unlink(domain, dev);

drivers/iommu/riscv/iommu.c static int riscv_iommu_attach_paging_domain(struct iommu_domain *iommu_domain, · line 1304

Quoted 10 lines
drivers/iommu/riscv/iommu.c +10 -0
+			return -ENOMEM;
+		}
+
+		ret = riscv_iommu_irq_domain_create(domain, dev);
+		if (ret) {
+			riscv_iommu_bond_unlink(domain, dev);
+			ida_free(&riscv_iommu_gscids, domain->gscid);
+			return ret;
+		}
+	}

What are you trying to do? Make something behave different for VFIO? That isn't OK, we are trying to remove all the hacky VFIO special cases in drivers.

What is the HW issue here? It is very very strange (and probably not going to work right) that the irq domains change when domain attachment changes.

Quoted 7 lines
Quoted 2 lines

The IRQ setup should really be fixed before any device drivers probe onto the device.

I can't disagree with the statement that this looks hacky, but considering a VFIO domain needs to use the g-stage for its single-stage translation and a paging domain for the host would use s-stage, then it seems we need to identify the VFIO domains for their special treatment.

This is the wrong thinking entirely. There is no such thing as a "VFIO domain".

Default VFIO created domains should act excatly the same as a DMA API domain.

If you want your system to have irq remapping, then it should be on by default and DMA API gets remapping too. There would need to be a very strong reason not to do that in order to make something special for riscv. If so you'd need to add some kind of flag to select it.

Quoted 4 lines

Until you reach nested translation there is no "need" for VFIO to use any particular stage. The design is that default VFIO uses the same stage as the DMA API because it is doing the same basic default translation function.

The RISC-V IOMMU needs to use g-stage for device assignment, if we also want to enable irqbypass, because the IOMMU is specified to only look at the MSI table when g-stage is in use. This is actually another reason the irq domain only makes sense for device assignment.

Quoted 3 lines

Nested translation has a control to select the stage, and you can then force the g-stage for VFIO users at that point.

We could force riscv device assignment to always be nested, and when not providing an iommu to the guest, it will still be single-stage, but g-stage, but I don't think that's currently possible with VFIO, is it?

Quoted 16 lines

Regardless, you must not use UNMANAGED as some indication of VFIO, that is not what it means, that is not what it is for.

Quoted 2 lines

Is there an example of converting VFIO special casing in other drivers to something cleaner that you can point me at?

Nobody has had an issue where they want interrupt remapping on/off depending on VFIO. I think that is inherently wrong.

Quoted 5 lines

The IRQ domain will only be useful for device assignment, as that's when an MSI translation will be needed. I can't think of any problems that could arise from only creating the IRQ domain when probing assigned devices, but I could certainly be missing something. Do you have some potential problems in mind?

Quoted 5 lines

I'm not an expert in the interrupt subsystem, but my understanding was we expect the interrupt domains/etc to be static once a device driver is probed. Changing things during iommu domain attach is after drivers are probed. I don't really expect it to work correctly in all corner cases.

With VFIO the iommu domain attach comes after an unbind/bind, so the new driver is probed. I think that's a safe time. However, if there could be cases where the attach does not follow an unbind/bind, then I agree that wouldn't be safe. I'll consider always creating an IRQ domain, even if it won't provide any additional functionality unless the device is assigned.

Quoted 4 lines

VFIO is allowed to change the translation as it operates and we expect that interrupts are not disturbed.

The IRQ domain stays the same during operation, the only changes are the mappings from what the guest believes are its s-mode interrupt files to the hypervisor selected guest interrupt files, and these changes are made possible by the IRQ domain's vcpu-affinity support.

Thanks,
drew

#24 lore raw

Re: [RFC PATCH 08/15] iommu/riscv: Add IRQ domain for interrupt remapping

Quoted 16 lines · Andrew Jones
Quoted 11 lines

This is the wrong thinking entirely. There is no such thing as a "VFIO domain".

Default VFIO created domains should act excatly the same as a DMA API domain.

If you want your system to have irq remapping, then it should be on by default and DMA API gets remapping too. There would need to be a very strong reason not to do that in order to make something special for riscv. If so you'd need to add some kind of flag to select it.

Quoted 4 lines

Until you reach nested translation there is no "need" for VFIO to use any particular stage. The design is that default VFIO uses the same stage as the DMA API because it is doing the same basic default translation function.

Quoted 5 lines

The RISC-V IOMMU needs to use g-stage for device assignment, if we also want to enable irqbypass, because the IOMMU is specified to only look at the MSI table when g-stage is in use. This is actually another reason the irq domain only makes sense for device assignment.

Most HW has enablable interrupt remapping and typically Linux just turns it always on.

Is there a reason the DMA API shouldn't use this translation mode too? That seems to be the main issue here, you are trying to avoid interrupt remapping for DMA API and use it only for VFIO, and haven't explained why we need such complexity. Just use it always?

Quoted 7 lines
Quoted 2 lines

Nested translation has a control to select the stage, and you can then force the g-stage for VFIO users at that point.

We could force riscv device assignment to always be nested, and when not providing an iommu to the guest, it will still be single-stage, but g-stage, but I don't think that's currently possible with VFIO, is it?

That isn't what I mean, I mean you should not be forcing the kind of domain being created until you get to special cases like nested.

Default VFIO should work the same as the DMA API.

Quoted 6 lines
Quoted 6 lines
Quoted 5 lines

The IRQ domain will only be useful for device assignment, as that's when an MSI translation will be needed. I can't think of any problems that could arise from only creating the IRQ domain when probing assigned devices, but I could certainly be missing something. Do you have some potential problems in mind?

Quoted 8 lines
Quoted 5 lines

I'm not an expert in the interrupt subsystem, but my understanding was we expect the interrupt domains/etc to be static once a device driver is probed. Changing things during iommu domain attach is after drivers are probed. I don't really expect it to work correctly in all corner cases.

With VFIO the iommu domain attach comes after an unbind/bind, so the new driver is probed.

That's the opposite of what I mean. The irq domain should be setup *before* VFIO binds to the driver.

Changing the active irq_domain while VFIO is already probed to the device is highly unlikely to work right in all cases.

Quoted 3 lines

I think that's a safe time. However, if there
could be cases where the attach does not follow an unbind/bind, then I agree that wouldn't be safe.

These cases exist.

Quoted 3 lines

I'll consider always creating an IRQ
domain, even if it won't provide any additional functionality unless the device is assigned.

That isn't ideal, the translation under the IRQs shouldn't really be changing as the translation under the IOMMU changes.

Further, VFIO assumes iommu_group_has_isolated_msi(), ie IRQ_DOMAIN_FLAG_ISOLATED_MSI, is fixed while it is is bound. Will that be true if the iommu is flapping all about? What will you do when VFIO has it attached to a blocked domain?

It just doesn't make sense to change something so fundamental as the interrupt path on an iommu domain attachement. :\

Quoted 7 lines
Quoted 2 lines

VFIO is allowed to change the translation as it operates and we expect that interrupts are not disturbed.

The IRQ domain stays the same during operation, the only changes are the mappings from what the guest believes are its s-mode interrupt files to the hypervisor selected guest interrupt files, and these changes are made possible by the IRQ domain's vcpu-affinity support.

That is only the case when talking about kvm, this all still has to work fully for non-kvm VFIO uses cases too.

Jason

#25 lore raw

Re: [RFC PATCH 08/15] iommu/riscv: Add IRQ domain for interrupt remapping

Quoted 24 lines · Jason Gunthorpe
Quoted 16 lines · Andrew Jones
Quoted 11 lines

This is the wrong thinking entirely. There is no such thing as a "VFIO domain".

Default VFIO created domains should act excatly the same as a DMA API domain.

If you want your system to have irq remapping, then it should be on by default and DMA API gets remapping too. There would need to be a very strong reason not to do that in order to make something special for riscv. If so you'd need to add some kind of flag to select it.

Quoted 4 lines

Until you reach nested translation there is no "need" for VFIO to use any particular stage. The design is that default VFIO uses the same stage as the DMA API because it is doing the same basic default translation function.

Quoted 5 lines

The RISC-V IOMMU needs to use g-stage for device assignment, if we also want to enable irqbypass, because the IOMMU is specified to only look at the MSI table when g-stage is in use. This is actually another reason the irq domain only makes sense for device assignment.

Quoted 7 lines

Most HW has enablable interrupt remapping and typically Linux just turns it always on.

Is there a reason the DMA API shouldn't use this translation mode too? That seems to be the main issue here, you are trying to avoid interrupt remapping for DMA API and use it only for VFIO, and haven't explained why we need such complexity. Just use it always?

The reason is that the RISC-V IOMMU only checks the MSI table, i.e. enables its support for MSI remapping, when the g-stage (second-stage) page table is in use. However, the expected virtual memory scheme for an OS to use for DMA would be to have s-stage (first-stage) in use and the g-stage set to 'Bare' (not in use). OIOW, it doesn't appear the spec authors expected MSI remapping to be enabled for the host DMA use case. That does make some sense, since it's actually not necessary. For the host DMA use case, providing mappings for each s-mode interrupt file which the device is allowed to write to in the s-stage page table sufficiently enables MSIs to be delivered.

Quoted 9 lines
Quoted 8 lines
Quoted 3 lines

Nested translation has a control to select the stage, and you can then force the g-stage for VFIO users at that point.

We could force riscv device assignment to always be nested, and when not providing an iommu to the guest, it will still be single-stage, but g-stage, but I don't think that's currently possible with VFIO, is it?

Quoted 4 lines

That isn't what I mean, I mean you should not be forcing the kind of domain being created until you get to special cases like nested.

Default VFIO should work the same as the DMA API.

If "default VFIO" means VFIO without irqbypass, then it would work the same as the DMA API, assuming all mappings for all necessary s-mode interrupt files are created (something the DMA API needs as well). However, VFIO would also need 'vfio_iommu_type1.allow_unsafe_interrupts=1' to be set for this no-irqbypass configuration.

Quoted 26 lines
Quoted 7 lines
Quoted 7 lines
Quoted 6 lines

The IRQ domain will only be useful for device assignment, as that's when an MSI translation will be needed. I can't think of any problems that could arise from only creating the IRQ domain when probing assigned devices, but I could certainly be missing something. Do you have some potential problems in mind?

Quoted 8 lines
Quoted 5 lines

I'm not an expert in the interrupt subsystem, but my understanding was we expect the interrupt domains/etc to be static once a device driver is probed. Changing things during iommu domain attach is after drivers are probed. I don't really expect it to work correctly in all corner cases.

With VFIO the iommu domain attach comes after an unbind/bind, so the new driver is probed.

That's the opposite of what I mean. The irq domain should be setup *before* VFIO binds to the driver.

Changing the active irq_domain while VFIO is already probed to the device is highly unlikely to work right in all cases.

Quoted 3 lines

I think that's a safe time. However, if there
could be cases where the attach does not follow an unbind/bind, then I agree that wouldn't be safe.

Quoted 8 lines

These cases exist.

Quoted 3 lines

I'll consider always creating an IRQ
domain, even if it won't provide any additional functionality unless the device is assigned.

That isn't ideal, the translation under the IRQs shouldn't really be changing as the translation under the IOMMU changes.

Unless the device is assigned to a guest, then the IRQ domain wouldn't do anything at all (it'd just sit between the device and the device's old MSI parent domain), but it also wouldn't come and go, risking issues with anything sensitive to changes in the IRQ domain hierarchy.

Quoted 8 lines

Further, VFIO assumes iommu_group_has_isolated_msi(), ie IRQ_DOMAIN_FLAG_ISOLATED_MSI, is fixed while it is is bound. Will that be true if the iommu is flapping all about? What will you do when VFIO has it attached to a blocked domain?

It just doesn't make sense to change something so fundamental as the interrupt path on an iommu domain attachement. :\

Yes, it does appear I should be doing this at iommu device probe time instead. It won't provide any additional functionality to use cases which aren't assigning devices to guests, but it also won't hurt, and it should avoid the risks you point out.

Quoted 4 lines
Quoted 4 lines
Quoted 3 lines

VFIO is allowed to change the translation as it operates and we expect that interrupts are not disturbed.

Quoted 8 lines
Quoted 4 lines

The IRQ domain stays the same during operation, the only changes are the mappings from what the guest believes are its s-mode interrupt files to the hypervisor selected guest interrupt files, and these changes are made possible by the IRQ domain's vcpu-affinity support.

That is only the case when talking about kvm, this all still has to work fully for non-kvm VFIO uses cases too.

I'll rework the irq domain creation to be at iommu device probe time for the next version.

Thanks,
drew

#26 lore raw

Re: [RFC PATCH 08/15] iommu/riscv: Add IRQ domain for interrupt remapping

Quoted 5 lines · Andrew Jones

The reason is that the RISC-V IOMMU only checks the MSI table, i.e. enables its support for MSI remapping, when the g-stage (second-stage) page table is in use. However, the expected virtual memory scheme for an OS to use for DMA would be to have s-stage (first-stage) in use and the g-stage set to 'Bare' (not in use).

That isn't really a technical reason.

Quoted 6 lines

OIOW, it doesn't appear the spec authors expected MSI remapping to be enabled for the host DMA use case. That does make some sense, since it's actually not necessary. For the host DMA use case, providing mappings for each s-mode interrupt file which the device is allowed to write to in the s-stage page table sufficiently enables MSIs to be delivered.

Well, that seems to be the main problem here. You are grappling with a spec design that doesn't match the SW expecations. Since it has deviated from what everyone else has done you now have extra challenges to resolve in some way.

Just always using interrupt remapping if the HW is capable of interrupt remapping and ignoring the spec "expectation" is a nice a simple way to make things work with existing Linux.

Quoted 5 lines

If "default VFIO" means VFIO without irqbypass, then it would work the same as the DMA API, assuming all mappings for all necessary s-mode interrupt files are created (something the DMA API needs as well). However, VFIO would also need 'vfio_iommu_type1.allow_unsafe_interrupts=1' to be set for this no-irqbypass configuration.

Which isn't what anyone wants, you need to make the DMA API domain be fully functional so that VFIO works.

Quoted 7 lines
Quoted 2 lines

That isn't ideal, the translation under the IRQs shouldn't really be changing as the translation under the IOMMU changes.

Unless the device is assigned to a guest, then the IRQ domain wouldn't do anything at all (it'd just sit between the device and the device's old MSI parent domain), but it also wouldn't come and go, risking issues with anything sensitive to changes in the IRQ domain hierarchy.

VFIO isn't restricted to such a simple use model. You have to support all the generality, which includes fully supporting changing the iommu translation on the fly.

Quoted 5 lines
Quoted 5 lines

Further, VFIO assumes iommu_group_has_isolated_msi(), ie IRQ_DOMAIN_FLAG_ISOLATED_MSI, is fixed while it is is bound. Will that be true if the iommu is flapping all about? What will you do when VFIO has it attached to a blocked domain?

Quoted 7 lines
Quoted 2 lines

It just doesn't make sense to change something so fundamental as the interrupt path on an iommu domain attachement. :\

Yes, it does appear I should be doing this at iommu device probe time instead. It won't provide any additional functionality to use cases which aren't assigning devices to guests, but it also won't hurt, and it should avoid the risks you point out.

Even if you statically create the domain you can't change the value of IRQ_DOMAIN_FLAG_ISOLATED_MSI depending on what is currently attached to the IOMMU.

What you are trying to do is not supported by the software stack right now. You need to make much bigger, more intrusive changes, if you really want to make interrupt remapping dynamic.

Jason

#27 lore raw

Re: [RFC PATCH 08/15] iommu/riscv: Add IRQ domain for interrupt remapping

Quoted 60 lines · Jason Gunthorpe
Quoted 5 lines · Andrew Jones

The reason is that the RISC-V IOMMU only checks the MSI table, i.e. enables its support for MSI remapping, when the g-stage (second-stage) page table is in use. However, the expected virtual memory scheme for an OS to use for DMA would be to have s-stage (first-stage) in use and the g-stage set to 'Bare' (not in use).

That isn't really a technical reason.

Quoted 6 lines

OIOW, it doesn't appear the spec authors expected MSI remapping to be enabled for the host DMA use case. That does make some sense, since it's actually not necessary. For the host DMA use case, providing mappings for each s-mode interrupt file which the device is allowed to write to in the s-stage page table sufficiently enables MSIs to be delivered.

Well, that seems to be the main problem here. You are grappling with a spec design that doesn't match the SW expecations. Since it has deviated from what everyone else has done you now have extra challenges to resolve in some way.

Just always using interrupt remapping if the HW is capable of interrupt remapping and ignoring the spec "expectation" is a nice a simple way to make things work with existing Linux.

Quoted 5 lines

If "default VFIO" means VFIO without irqbypass, then it would work the same as the DMA API, assuming all mappings for all necessary s-mode interrupt files are created (something the DMA API needs as well). However, VFIO would also need 'vfio_iommu_type1.allow_unsafe_interrupts=1' to be set for this no-irqbypass configuration.

Which isn't what anyone wants, you need to make the DMA API domain be fully functional so that VFIO works.

Quoted 7 lines
Quoted 2 lines

That isn't ideal, the translation under the IRQs shouldn't really be changing as the translation under the IOMMU changes.

Unless the device is assigned to a guest, then the IRQ domain wouldn't do anything at all (it'd just sit between the device and the device's old MSI parent domain), but it also wouldn't come and go, risking issues with anything sensitive to changes in the IRQ domain hierarchy.

VFIO isn't restricted to such a simple use model. You have to support all the generality, which includes fully supporting changing the iommu translation on the fly.

Quoted 5 lines
Quoted 5 lines

Further, VFIO assumes iommu_group_has_isolated_msi(), ie IRQ_DOMAIN_FLAG_ISOLATED_MSI, is fixed while it is is bound. Will that be true if the iommu is flapping all about? What will you do when VFIO has it attached to a blocked domain?

Quoted 7 lines
Quoted 2 lines

It just doesn't make sense to change something so fundamental as the interrupt path on an iommu domain attachement. :\

Yes, it does appear I should be doing this at iommu device probe time instead. It won't provide any additional functionality to use cases which aren't assigning devices to guests, but it also won't hurt, and it should avoid the risks you point out.

Quoted 8 lines

Even if you statically create the domain you can't change the value of IRQ_DOMAIN_FLAG_ISOLATED_MSI depending on what is currently attached to the IOMMU.

What you are trying to do is not supported by the software stack right now. You need to make much bigger, more intrusive changes, if you really want to make interrupt remapping dynamic.

Let the fun begin. I'll look into this more. It also looks like I need to collect some test cases to ensure I can support all use cases with whatever I propose next. Pointers for those would be welcome.

Thanks,
drew

#28 lore raw

Re: [RFC PATCH 08/15] iommu/riscv: Add IRQ domain for interrupt remapping

Quoted 8 lines · Andrew Jones
Quoted 4 lines

What you are trying to do is not supported by the software stack right now. You need to make much bigger, more intrusive changes, if you really want to make interrupt remapping dynamic.

Let the fun begin. I'll look into this more. It also looks like I need to collect some test cases to ensure I can support all use cases with whatever I propose next. Pointers for those would be welcome.

Sorry, I don't really have anything.. But iommufd allows changing the translation at will and we expect this to happen in normal VMM scenarios. So blocking, paging, nesting are all expected to be dynamically selectable and non-disruptive to interrupts.

So, you can't decide if remapping is enabled or not for a device based only on the domain attachment.

I think you'd need to create a way for VFIO to request dynamic interrupt remapping be enabled for the device very, very early in it's process and that would remain fixed while VFIO is using the device.

The dynamic state of interrupt remapping would constrain what iommu attachment configurations are permitted.

Jason

#29 lore raw

RFC PATCH 9/15 RISC-V: KVM: Enable KVM_VFIO interfaces on RISC-V arch

In reply to Andrew Jones

From: Tomasz Jeznach <[email protected]>

Enable KVM/VFIO support on RISC-V architecture.

---
 arch/riscv/kvm/Kconfig | 2 ++
 1 file changed, 2 insertions(+)
arch/riscv/kvm/Kconfig +2 -0
index 0c3cbb0915ff..333d95da8ebe 100644
@@ -29,10 +29,12 @@ config KVM
 	select KVM_GENERIC_DIRTYLOG_READ_PROTECT
 	select KVM_GENERIC_HARDWARE_ENABLING
 	select KVM_MMIO
+	select KVM_VFIO
 	select KVM_XFER_TO_GUEST_WORK
 	select KVM_GENERIC_MMU_NOTIFIER
 	select SCHED_INFO
 	select GUEST_PERF_EVENTS if PERF_EVENTS
+	select SRCU
 	help
 	  Support hosting virtualized guest machines.
 
Signature · 1 line
2.47.0
#30 lore raw

RFC PATCH 10/15 RISC-V: KVM: Add irqbypass skeleton

In reply to Andrew Jones

Add all the functions needed to wire up irqbypass support. kvm_arch_update_irqfd_routing() is left unimplemented, so return false from kvm_arch_has_irq_bypass().

---
 arch/riscv/include/asm/kvm_host.h |  3 ++
 arch/riscv/kvm/Kconfig            |  1 +
 arch/riscv/kvm/aia_imsic.c        |  6 ++++
 arch/riscv/kvm/vm.c               | 60 +++++++++++++++++++++++++++++++
 4 files changed, 70 insertions(+)
arch/riscv/include/asm/kvm_host.h +3 -0
index 35eab6e0f4ae..fef7422939f6 100644
@@ -114,6 +114,9 @@ struct kvm_arch {
 
 	/* AIA Guest/VM context */
 	struct kvm_aia aia;
+
+#define __KVM_HAVE_ARCH_ASSIGNED_DEVICE
+	atomic_t assigned_device_count;
 };
 
 struct kvm_cpu_trap {
arch/riscv/kvm/Kconfig +1 -0
index 333d95da8ebe..9a4feb1e671d 100644
@@ -21,6 +21,7 @@ config KVM
 	tristate "Kernel-based Virtual Machine (KVM) support (EXPERIMENTAL)"
 	depends on RISCV_SBI && MMU
 	select HAVE_KVM_IRQCHIP
+	select HAVE_KVM_IRQ_BYPASS
 	select HAVE_KVM_IRQ_ROUTING
 	select HAVE_KVM_MSI
 	select HAVE_KVM_VCPU_ASYNC_IOCTL
arch/riscv/kvm/aia_imsic.c +6 -0
index a8085cd8215e..64b1f3713dd5 100644
@@ -727,6 +727,12 @@ void kvm_riscv_vcpu_aia_imsic_release(struct kvm_vcpu *vcpu)
 	kvm_riscv_aia_free_hgei(old_vsfile_cpu, old_vsfile_hgei);
 }
 
+int kvm_arch_update_irqfd_routing(struct kvm *kvm, unsigned int host_irq,
+				  uint32_t guest_irq, bool set)
+{
+	return -ENXIO;
+}
+
 int kvm_riscv_vcpu_aia_imsic_update(struct kvm_vcpu *vcpu)
 {
 	unsigned long flags;
arch/riscv/kvm/vm.c +60 -0

72 lines not on this page · raw message

Signature · 1 line
2.47.0
#31 lore raw

RFC PATCH 11/15 RISC-V: Define irqbypass vcpu_info

In reply to Andrew Jones

The vcpu_info parameter to irq_set_vcpu_affinity() effectively defines an arch specific IOMMU <=> hypervisor protocol. Provide a definition for the RISCV IOMMU.

---
 arch/riscv/include/asm/irq.h | 9 +++++++++
 1 file changed, 9 insertions(+)
arch/riscv/include/asm/irq.h +9 -0
index 7b038f3b7cb0..8588667cbb5f 100644
@@ -23,6 +23,15 @@ void riscv_set_intc_hwnode_fn(struct fwnode_handle *(*fn)(void));
 
 struct fwnode_handle *riscv_get_intc_hwnode(void);
 
+struct riscv_iommu_vcpu_info {
+	u64 msi_addr_pattern;
+	u64 msi_addr_mask;
+	u32 group_index_bits;
+	u32 group_index_shift;
+	u64 gpa;
+	u64 hpa;
+};
+
 #ifdef CONFIG_ACPI
 
 enum riscv_irqchip_type {
Signature · 1 line
2.47.0
#32 lore raw

RFC PATCH 12/15 iommu/riscv: Add guest file irqbypass support

In reply to Andrew Jones

Implement irq_set_vcpu_affinity() in the RISCV IOMMU driver. irq_set_vcpu_affinity() is the channel from a hypervisor to the IOMMU needed to ensure that assigned devices which direct MSIs to guest IMSIC addresses will have those MSI writes redirected to their corresponding guest interrupt files.

---
 drivers/iommu/riscv/iommu-ir.c | 151 +++++++++++++++++++++++++++++++++
 drivers/iommu/riscv/iommu.c    |   4 +-
 drivers/iommu/riscv/iommu.h    |   3 +
 3 files changed, 156 insertions(+), 2 deletions(-)
drivers/iommu/riscv/iommu-ir.c +151 -0
index c177e064b205..958270450ec1 100644
@@ -7,6 +7,8 @@
 #include <linux/irqdomain.h>
 #include <linux/msi.h>
 
+#include <asm/irq.h>
+
 #include "../iommu-pages.h"
 #include "iommu.h"
 
@@ -52,10 +54,159 @@ static size_t riscv_iommu_ir_nr_msiptes(struct riscv_iommu_domain *domain)
 	return max_idx + 1;
 }
 
+static void riscv_iommu_ir_msitbl_inval(struct riscv_iommu_domain *domain,
+					struct riscv_iommu_msipte *pte)
+{
+	struct riscv_iommu_bond *bond;
+	struct riscv_iommu_device *iommu, *prev;
+	struct riscv_iommu_command cmd;
+	u64 addr;
+
+	addr = pfn_to_phys(FIELD_GET(RISCV_IOMMU_MSIPTE_PPN, pte->pte));
+	riscv_iommu_cmd_inval_gvma(&cmd);
+	riscv_iommu_cmd_inval_set_addr(&cmd, addr);
+
+	/* Like riscv_iommu_iotlb_inval(), synchronize with riscv_iommu_bond_link() */
+	smp_mb();
+
+	rcu_read_lock();
+
+	prev = NULL;
+	list_for_each_entry_rcu(bond, &domain->bonds, list) {
+		iommu = dev_to_iommu(bond->dev);
+
+		if (iommu == prev)
+			continue;
+
+		riscv_iommu_cmd_send(iommu, &cmd);
+		riscv_iommu_cmd_sync(iommu, RISCV_IOMMU_IOTINVAL_TIMEOUT);
+		prev = iommu;
+	}
+
+	rcu_read_unlock();
+}
+
+static void riscv_iommu_ir_msitbl_update(struct riscv_iommu_domain *domain,
+					 struct riscv_iommu_msiptp_state *msiptp)
+{
+	struct riscv_iommu_bond *bond;
+	struct riscv_iommu_device *iommu, *prev;
+	struct riscv_iommu_command cmd;
+	struct iommu_fwspec *fwspec;
+	struct riscv_iommu_dc *dc;
+	int i;
+
+	/* Like riscv_iommu_ir_msitbl_inval(), synchronize with riscv_iommu_bond_link() */
+	smp_mb();
+	rcu_read_lock();
+
+	prev = NULL;
+	list_for_each_entry_rcu(bond, &domain->bonds, list) {
+		iommu = dev_to_iommu(bond->dev);
+		fwspec = dev_iommu_fwspec_get(bond->dev);
+
+		for (i = 0; i < fwspec->num_ids; i++) {
+			dc = riscv_iommu_get_dc(iommu, fwspec->ids[i]);
+			WRITE_ONCE(dc->msiptp, msiptp->msiptp);
+			WRITE_ONCE(dc->msi_addr_mask, msiptp->msi_addr_mask);
+			WRITE_ONCE(dc->msi_addr_pattern, msiptp->msi_addr_pattern);
+		}
+
+		dma_wmb();
+
+		if (iommu == prev)
+			continue;
+
+		riscv_iommu_cmd_inval_gvma(&cmd);
+		riscv_iommu_cmd_send(iommu, &cmd);
+		riscv_iommu_cmd_sync(iommu, RISCV_IOMMU_IOTINVAL_TIMEOUT);
+		prev = iommu;
+	}
+
+	rcu_read_unlock();
+}
+
+static int riscv_iommu_ir_msitbl_init(struct riscv_iommu_domain *domain,
+				      struct riscv_iommu_vcpu_info *vcpu_info)
+{
+	domain->msiptp.msi_addr_pattern = vcpu_info->msi_addr_pattern;
+	domain->msiptp.msi_addr_mask = vcpu_info->msi_addr_mask;
+	domain->group_index_bits = vcpu_info->group_index_bits;
+	domain->group_index_shift = vcpu_info->group_index_shift;
+
+	if (riscv_iommu_ir_nr_msiptes(domain) * sizeof(*domain->msi_root) > PAGE_SIZE * 2)
+		return -ENOMEM;
+
+	domain->msiptp.msiptp = virt_to_pfn(domain->msi_root) |
+				FIELD_PREP(RISCV_IOMMU_DC_MSIPTP_MODE,
+					   RISCV_IOMMU_DC_MSIPTP_MODE_FLAT);
+
+	riscv_iommu_ir_msitbl_update(domain, &domain->msiptp);
+
+	return 0;
+}
+
+static int riscv_iommu_irq_set_vcpu_affinity(struct irq_data *data, void *info)
+{
+	struct riscv_iommu_vcpu_info *vcpu_info = info;
+	struct riscv_iommu_domain *domain = data->domain->host_data;
+	struct riscv_iommu_msipte *pte;
+	int ret = -EINVAL;
+	u64 pteval;
+
+	if (WARN_ON(domain->domain.type != IOMMU_DOMAIN_UNMANAGED))
+		return ret;
+
+	spin_lock(&domain->msi_lock);
+
+	if (!domain->msiptp.msiptp) {
+		if (WARN_ON(!vcpu_info))
+			goto out_unlock;
+
+		ret = riscv_iommu_ir_msitbl_init(domain, vcpu_info);
+		if (ret)
+			goto out_unlock;
+	} else if (!vcpu_info) {
+		/*
+		 * Nothing to do here since we don't track host_irq <=> msipte mappings
+		 * nor reference count the ptes. If we did do that tracking then we would
+		 * decrement the reference count of the pte for the host_irq and possibly
+		 * clear its valid bit if it was the last one mapped.
+		 */
+		ret = 0;
+		goto out_unlock;
+	} else if (WARN_ON(vcpu_info->msi_addr_pattern != domain->msiptp.msi_addr_pattern ||
+			   vcpu_info->msi_addr_mask != domain->msiptp.msi_addr_mask ||
+			   vcpu_info->group_index_bits != domain->group_index_bits ||
+			   vcpu_info->group_index_shift != domain->group_index_shift)) {
+		goto out_unlock;
+	}
+
+	pte = riscv_iommu_ir_get_msipte(domain, vcpu_info->gpa);
+	if (!pte)
+		goto out_unlock;
+
+	pteval = FIELD_PREP(RISCV_IOMMU_MSIPTE_M, 3) |
+		 riscv_iommu_phys_to_ppn(vcpu_info->hpa) |
+		 FIELD_PREP(RISCV_IOMMU_MSIPTE_V, 1);
+
+	if (pte->pte != pteval) {
+		pte->pte = pteval;
+		riscv_iommu_ir_msitbl_inval(domain, pte);
+	}
+
+	ret = 0;
+
+out_unlock:
+	spin_unlock(&domain->msi_lock);
+	return ret;
+}
+
 static struct irq_chip riscv_iommu_irq_chip = {
 	.name			= "IOMMU-IR",
 	.irq_mask		= irq_chip_mask_parent,
 	.irq_unmask		= irq_chip_unmask_parent,
+	.irq_set_vcpu_affinity	= riscv_iommu_irq_set_vcpu_affinity,
 };
 
 static int riscv_iommu_irq_domain_alloc_irqs(struct irq_domain *irqdomain,
drivers/iommu/riscv/iommu.c +2 -2
index c4a47b21c58f..46ac228ba7ac 100644
@@ -544,8 +544,8 @@ static irqreturn_t riscv_iommu_fltq_process(int irq, void *data)
 }
 
 /* Lookup and initialize device context info structure. */
-static struct riscv_iommu_dc *riscv_iommu_get_dc(struct riscv_iommu_device *iommu,
-						 unsigned int devid)
+struct riscv_iommu_dc *riscv_iommu_get_dc(struct riscv_iommu_device *iommu,
+					  unsigned int devid)
 {
 	const bool base_format = !(iommu->caps & RISCV_IOMMU_CAPABILITIES_MSI_FLAT);
 	unsigned int depth;
drivers/iommu/riscv/iommu.h +3 -0
index 6ce71095781c..2ca76edf48f5 100644
@@ -127,6 +127,9 @@ struct riscv_iommu_bond {
 int riscv_iommu_init(struct riscv_iommu_device *iommu);
 void riscv_iommu_remove(struct riscv_iommu_device *iommu);
 
+struct riscv_iommu_dc *riscv_iommu_get_dc(struct riscv_iommu_device *iommu,
+					  unsigned int devid);
+
 void riscv_iommu_cmd_send(struct riscv_iommu_device *iommu,
 			  struct riscv_iommu_command *cmd);
 void riscv_iommu_cmd_sync(struct riscv_iommu_device *iommu,
Signature · 1 line
2.47.0
#33 lore raw

RFC PATCH 13/15 RISC-V: KVM: Add guest file irqbypass support

In reply to Andrew Jones

Implement kvm_arch_update_irqfd_routing() which makes irq_set_vcpu_affinity() calls whenever the assigned device updates its target addresses and whenever the hypervisor has migrated a VCPU to another CPU (which requires changing the guest interrupt file).

---
 arch/riscv/kvm/aia_imsic.c | 132 ++++++++++++++++++++++++++++++++++++-
 arch/riscv/kvm/vm.c        |   2 +-
 2 files changed, 130 insertions(+), 4 deletions(-)
arch/riscv/kvm/aia_imsic.c +129 -3
index 64b1f3713dd5..6a7c23e25f79 100644
@@ -11,11 +11,13 @@
 #include <linux/bitmap.h>
 #include <linux/irqchip/riscv-imsic.h>
 #include <linux/kvm_host.h>
+#include <linux/kvm_irqfd.h>
 #include <linux/math.h>
 #include <linux/spinlock.h>
 #include <linux/swab.h>
 #include <kvm/iodev.h>
 #include <asm/csr.h>
+#include <asm/irq.h>
 
 #define IMSIC_MAX_EIX	(IMSIC_MAX_ID / BITS_PER_TYPE(u64))
 
@@ -676,6 +678,14 @@ static void imsic_swfile_update(struct kvm_vcpu *vcpu,
 	imsic_swfile_extirq_update(vcpu);
 }
 
+static u64 kvm_riscv_aia_msi_addr_mask(struct kvm_aia *aia)
+{
+	u64 group_mask = BIT(aia->nr_group_bits) - 1;
+
+	return (group_mask << (aia->nr_group_shift - IMSIC_MMIO_PAGE_SHIFT)) |
+	       (BIT(aia->nr_hart_bits + aia->nr_guest_bits) - 1);
+}
+
 void kvm_riscv_vcpu_aia_imsic_release(struct kvm_vcpu *vcpu)
 {
 	unsigned long flags;
@@ -730,7 +740,120 @@ void kvm_riscv_vcpu_aia_imsic_release(struct kvm_vcpu *vcpu)
 int kvm_arch_update_irqfd_routing(struct kvm *kvm, unsigned int host_irq,
 				  uint32_t guest_irq, bool set)
 {
-	return -ENXIO;
+	struct irq_data *irqdata = irq_get_irq_data(host_irq);
+	struct kvm_irq_routing_table *irq_rt;
+	struct kvm_vcpu *vcpu;
+	unsigned long tmp, flags;
+	int idx, ret = -ENXIO;
+
+	if (!set)
+		return irq_set_vcpu_affinity(host_irq, NULL);
+
+	idx = srcu_read_lock(&kvm->irq_srcu);
+	irq_rt = srcu_dereference(kvm->irq_routing, &kvm->irq_srcu);
+	if (guest_irq >= irq_rt->nr_rt_entries ||
+	    hlist_empty(&irq_rt->map[guest_irq])) {
+		pr_warn_once("no route for guest_irq %u/%u (broken user space?)\n",
+			     guest_irq, irq_rt->nr_rt_entries);
+		goto out;
+	}
+
+	kvm_for_each_vcpu(tmp, vcpu, kvm) {
+		struct imsic *imsic = vcpu->arch.aia_context.imsic_state;
+		gpa_t ippn = vcpu->arch.aia_context.imsic_addr >> IMSIC_MMIO_PAGE_SHIFT;
+		struct kvm_aia *aia = &kvm->arch.aia;
+		struct kvm_kernel_irq_routing_entry *e;
+
+		hlist_for_each_entry(e, &irq_rt->map[guest_irq], link) {
+			struct msi_msg msg[2] = {
+			{
+				.address_hi = e->msi.address_hi,
+				.address_lo = e->msi.address_lo,
+				.data = e->msi.data,
+			},
+			};
+			struct riscv_iommu_vcpu_info vcpu_info = {
+				.msi_addr_mask = kvm_riscv_aia_msi_addr_mask(aia),
+				.group_index_bits = aia->nr_group_bits,
+				.group_index_shift = aia->nr_group_shift,
+			};
+			gpa_t target, tppn;
+
+			if (e->type != KVM_IRQ_ROUTING_MSI)
+				continue;
+
+			target = ((gpa_t)e->msi.address_hi << 32) | e->msi.address_lo;
+			tppn = target >> IMSIC_MMIO_PAGE_SHIFT;
+
+			WARN_ON(target & (IMSIC_MMIO_PAGE_SZ - 1));
+
+			if (ippn != tppn)
+				continue;
+
+			vcpu_info.msi_addr_pattern = tppn & ~vcpu_info.msi_addr_mask;
+			vcpu_info.gpa = target;
+
+			read_lock_irqsave(&imsic->vsfile_lock, flags);
+
+			if (WARN_ON_ONCE(imsic->vsfile_cpu < 0)) {
+				read_unlock_irqrestore(&imsic->vsfile_lock, flags);
+				goto out;
+			}
+
+			vcpu_info.hpa = imsic->vsfile_pa;
+
+			ret = irq_set_vcpu_affinity(host_irq, &vcpu_info);
+			if (ret) {
+				read_unlock_irqrestore(&imsic->vsfile_lock, flags);
+				goto out;
+			}
+
+			irq_data_get_irq_chip(irqdata)->irq_write_msi_msg(irqdata, msg);
+
+			read_unlock_irqrestore(&imsic->vsfile_lock, flags);
+		}
+	}
+
+	ret = 0;
+out:
+	srcu_read_unlock(&kvm->irq_srcu, idx);
+	return ret;
+}
+
+static int kvm_riscv_vcpu_irq_update(struct kvm_vcpu *vcpu)
+{
+	struct kvm *kvm = vcpu->kvm;
+	struct imsic *imsic = vcpu->arch.aia_context.imsic_state;
+	struct kvm_aia *aia = &kvm->arch.aia;
+	u64 mask = kvm_riscv_aia_msi_addr_mask(aia);
+	u64 target = vcpu->arch.aia_context.imsic_addr;
+	struct riscv_iommu_vcpu_info vcpu_info = {
+		.msi_addr_pattern = (target >> IMSIC_MMIO_PAGE_SHIFT) & ~mask,
+		.msi_addr_mask = mask,
+		.group_index_bits = aia->nr_group_bits,
+		.group_index_shift = aia->nr_group_shift,
+		.gpa = target,
+		.hpa = imsic->vsfile_pa,
+	};
+	struct kvm_kernel_irqfd *irqfd;
+	int host_irq, ret;
+
+	spin_lock_irq(&kvm->irqfds.lock);
+
+	list_for_each_entry(irqfd, &kvm->irqfds.items, list) {
+		if (!irqfd->producer)
+			continue;
+		host_irq = irqfd->producer->irq;
+		ret = irq_set_vcpu_affinity(host_irq, &vcpu_info);
+		if (ret) {
+			spin_unlock_irq(&kvm->irqfds.lock);
+			return ret;
+		}
+	}
+
+	spin_unlock_irq(&kvm->irqfds.lock);
+
+	return 0;
 }
 
 int kvm_riscv_vcpu_aia_imsic_update(struct kvm_vcpu *vcpu)
@@ -797,14 +920,17 @@ int kvm_riscv_vcpu_aia_imsic_update(struct kvm_vcpu *vcpu)
 	if (ret)
 		goto fail_free_vsfile_hgei;
 
-	/* TODO: Update the IOMMU mapping ??? */
-
 	/* Update new IMSIC VS-file details in IMSIC context */
 	write_lock_irqsave(&imsic->vsfile_lock, flags);
+
 	imsic->vsfile_hgei = new_vsfile_hgei;
 	imsic->vsfile_cpu = vcpu->cpu;
 	imsic->vsfile_va = new_vsfile_va;
 	imsic->vsfile_pa = new_vsfile_pa;
+
+	/* Update the IOMMU mapping */
+	kvm_riscv_vcpu_irq_update(vcpu);
+
 	write_unlock_irqrestore(&imsic->vsfile_lock, flags);
 
 	/*
arch/riscv/kvm/vm.c +1 -1
index 9c5837518c1a..5f697d9a37da 100644
@@ -78,7 +78,7 @@ EXPORT_SYMBOL_GPL(kvm_arch_has_assigned_device);
 
 bool kvm_arch_has_irq_bypass(void)
 {
-	return false;
+	return true;
 }
 
 int kvm_arch_irq_bypass_add_producer(struct irq_bypass_consumer *cons,
Signature · 1 line
2.47.0
#34 lore raw

RFC PATCH 14/15 vfio: enable IOMMU_TYPE1 for RISC-V

In reply to Andrew Jones

From: Tomasz Jeznach <[email protected]>

Enable VFIO support on RISC-V architecture.

---
 drivers/vfio/Kconfig | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/vfio/Kconfig +1 -1
index ceae52fd7586..ad62205b4e45 100644
@@ -39,7 +39,7 @@ config VFIO_GROUP
 
 config VFIO_CONTAINER
 	bool "Support for the VFIO container /dev/vfio/vfio"
-	select VFIO_IOMMU_TYPE1 if MMU && (X86 || S390 || ARM || ARM64)
+	select VFIO_IOMMU_TYPE1 if MMU && (X86 || S390 || ARM || ARM64 || RISCV)
 	depends on VFIO_GROUP
 	default y
 	help
Signature · 1 line
2.47.0
#35 lore raw

RFC PATCH 15/15 RISC-V: defconfig: Add VFIO modules

In reply to Andrew Jones

Add the VFIO modules to the defconfig to complement KVM now that there is IOMMU support.

---
 arch/riscv/configs/defconfig | 2 ++
 1 file changed, 2 insertions(+)
arch/riscv/configs/defconfig +2 -0
index b4a37345703e..10fc9d84a28c 100644
@@ -242,6 +242,8 @@ CONFIG_RTC_DRV_SUN6I=y
 CONFIG_DMADEVICES=y
 CONFIG_DMA_SUN6I=m
 CONFIG_DW_AXI_DMAC=y
+CONFIG_VFIO=m
+CONFIG_VFIO_PCI=m
 CONFIG_VIRTIO_PCI=y
 CONFIG_VIRTIO_BALLOON=y
 CONFIG_VIRTIO_INPUT=y
Signature · 1 line
2.47.0