Skip to contentSkip to conversation

Add platform check for subdevice irq domain

RFC PATCH In review Lu Baolu 4 participants 12 messages Jan 14, 2021 – Jan 17, 2021

Subsystems (8)
x86 KVM PCI Driver core +4 more subsystems
lore
Share & tools
lore

Lu Baolu

author maintainer
#0 lore raw

v3 cover Add platform check for subdevice irq domain

Hi,

Learnt from the discussions in this thread:

https://lore.kernel.org/linux-pci/160408357912.912050.17005584526266191420.stgit@djiang5-desk3.ch.intel.com/

The device IMS (Interrupt Message Storage) should not be enabled in any virtualization environments unless there is a HYPERCALL domain which makes the changes in the message store monitored by the hypervisor.

As the initial step, we allow the IMS to be enabled only if we are running on the bare metal. It's easy to enable IMS in the virtualization environments if above preconditions are met in the future.

This series is only for comments purpose. We will include it in the Intel IMS implementation later once we reach a consensus.

Change log:
v2->v3:
- v2:
https://lore.kernel.org/linux-pci/[email protected]/
- Add all identified heuristics so far.

v1->v2:
- v1:
https://lore.kernel.org/linux-pci/[email protected]/
- Rename probably_on_bare_metal() with on_bare_metal();
- Some vendors might use the same name for both bare metal and virtual environment. Before we add vendor specific code to distinguish between them, let's return false in on_bare_metal(). This won't introduce any regression. The only impact is that the coming new platform msi feature won't be supported until the vendor specific code is provided.

Best regards,
baolu

Lu Baolu (2):
iommu: Add capability IOMMU_CAP_VIOMMU
platform-msi: Add platform check for subdevice irq domain

 arch/x86/pci/common.c        | 71 ++++++++++++++++++++++++++++++++++++
 drivers/base/platform-msi.c  |  8 ++++
 drivers/iommu/intel/iommu.c  | 20 ++++++++++
 drivers/iommu/virtio-iommu.c |  9 +++++
 include/linux/iommu.h        |  1 +
 include/linux/msi.h          |  1 +
 6 files changed, 110 insertions(+)
Signature · 6 lines
2.25.1

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
author maintainer
#1 lore raw

RFC PATCH v3 1/2 iommu: Add capability IOMMU_CAP_VIOMMU

Some vendor IOMMU drivers are able to declare that it is running in a VM context. This is very valuable for the features that only want to be supported on bare metal. Add a capability bit so that it could be used.

---
 drivers/iommu/intel/iommu.c  | 20 ++++++++++++++++++++
 drivers/iommu/virtio-iommu.c |  9 +++++++++
 include/linux/iommu.h        |  1 +
 3 files changed, 30 insertions(+)
drivers/iommu/intel/iommu.c +20 -0
index cb205a04fe4c..8eb022d0e8aa 100644
@@ -5738,12 +5738,32 @@ static inline bool nested_mode_support(void)
 	return ret;
 }
 
+static inline bool caching_mode_enabled(void)
+{
+	struct dmar_drhd_unit *drhd;
+	struct intel_iommu *iommu;
+	bool ret = false;
+
+	rcu_read_lock();
+	for_each_active_iommu(iommu, drhd) {
+		if (cap_caching_mode(iommu->cap)) {
+			ret = true;
+			break;
+		}
+	}
+	rcu_read_unlock();
+
+	return ret;
+}
+
 static bool intel_iommu_capable(enum iommu_cap cap)
 {
 	if (cap == IOMMU_CAP_CACHE_COHERENCY)
 		return domain_update_iommu_snooping(NULL) == 1;
 	if (cap == IOMMU_CAP_INTR_REMAP)
 		return irq_remapping_enabled == 1;
+	if (cap == IOMMU_CAP_VIOMMU)
+		return caching_mode_enabled();
 
 	return false;
 }
drivers/iommu/virtio-iommu.c +9 -0
index 2bfdd5734844..719793e103db 100644
@@ -931,7 +931,16 @@ static int viommu_of_xlate(struct device *dev, struct of_phandle_args *args)
 	return iommu_fwspec_add_ids(dev, args->args, 1);
 }
 
+static bool viommu_capable(enum iommu_cap cap)
+{
+	if (cap == IOMMU_CAP_VIOMMU)
+		return true;
+
+	return false;
+}
+
 static struct iommu_ops viommu_ops = {
+	.capable		= viommu_capable,
 	.domain_alloc		= viommu_domain_alloc,
 	.domain_free		= viommu_domain_free,
 	.attach_dev		= viommu_attach_dev,
include/linux/iommu.h +1 -0
index b95a6f8db6ff..1d24be667a03 100644
@@ -94,6 +94,7 @@ enum iommu_cap {
 					   transactions */
 	IOMMU_CAP_INTR_REMAP,		/* IOMMU supports interrupt isolation */
 	IOMMU_CAP_NOEXEC,		/* IOMMU_NOEXEC flag */
+	IOMMU_CAP_VIOMMU,		/* IOMMU can declar running in a VM */
 };
 
 /*
Signature · 6 lines
2.25.1

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
#2 lore raw

Re: [RFC PATCH v3 1/2] iommu: Add capability IOMMU_CAP_VIOMMU

Quoted 3 lines · Lu Baolu

Some vendor IOMMU drivers are able to declare that it is running in a VM context. This is very valuable for the features that only want to be supported on bare metal. Add a capability bit so that it could be used.

And how is it used? Who and how will set it?

Quoted 13 lines
---
 drivers/iommu/intel/iommu.c  | 20 ++++++++++++++++++++
 drivers/iommu/virtio-iommu.c |  9 +++++++++
 include/linux/iommu.h        |  1 +
 3 files changed, 30 insertions(+)
drivers/iommu/intel/iommu.c +0 -0
index cb205a04fe4c..8eb022d0e8aa 100644

drivers/iommu/intel/iommu.c static inline bool nested_mode_support(void) · line 5742

Quoted 5 lines
drivers/iommu/intel/iommu.c +2 -0
@@ -5738,12 +5738,32 @@ static inline bool nested_mode_support(void)
 	return ret;
 }
 
+static inline bool caching_mode_enabled(void)
+{

Kernel coding style is not in favour of inline functions in *.c files.

Quoted 61 lines
drivers/iommu/intel/iommu.c +18 -0
+	struct dmar_drhd_unit *drhd;
+	struct intel_iommu *iommu;
+	bool ret = false;
+
+	rcu_read_lock();
+	for_each_active_iommu(iommu, drhd) {
+		if (cap_caching_mode(iommu->cap)) {
+			ret = true;
+			break;
+		}
+	}
+	rcu_read_unlock();
+
+	return ret;
+}
+
 static bool intel_iommu_capable(enum iommu_cap cap)
 {
 	if (cap == IOMMU_CAP_CACHE_COHERENCY)
 		return domain_update_iommu_snooping(NULL) == 1;
 	if (cap == IOMMU_CAP_INTR_REMAP)
 		return irq_remapping_enabled == 1;
+	if (cap == IOMMU_CAP_VIOMMU)
+		return caching_mode_enabled();
 
 	return false;
 }
drivers/iommu/virtio-iommu.c +9 -0
index 2bfdd5734844..719793e103db 100644
@@ -931,7 +931,16 @@ static int viommu_of_xlate(struct device *dev, struct of_phandle_args *args)
 	return iommu_fwspec_add_ids(dev, args->args, 1);
 }
 
+static bool viommu_capable(enum iommu_cap cap)
+{
+	if (cap == IOMMU_CAP_VIOMMU)
+		return true;
+
+	return false;
+}
+
 static struct iommu_ops viommu_ops = {
+	.capable		= viommu_capable,
 	.domain_alloc		= viommu_domain_alloc,
 	.domain_free		= viommu_domain_free,
 	.attach_dev		= viommu_attach_dev,
include/linux/iommu.h +1 -1
index b95a6f8db6ff..1d24be667a03 100644
@@ -94,6 +94,7 @@ enum iommu_cap {
 					   transactions */
 	IOMMU_CAP_INTR_REMAP,		/* IOMMU supports interrupt isolation */
 	IOMMU_CAP_NOEXEC,		/* IOMMU_NOEXEC flag */
+	IOMMU_CAP_VIOMMU,		/* IOMMU can declar running in a VM */
 };
 
 /*
--
Quoted 2 lines

2.25.1

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu

author maintainer
#3 lore raw

Re: [RFC PATCH v3 1/2] iommu: Add capability IOMMU_CAP_VIOMMU

Hi Leon,

Quoted 6 lines · Leon Romanovsky
Quoted 3 lines · Lu Baolu

Some vendor IOMMU drivers are able to declare that it is running in a VM context. This is very valuable for the features that only want to be supported on bare metal. Add a capability bit so that it could be used.

And how is it used? Who and how will set it?

Use the existing iommu_capable(). I should add more descriptions about who and how to use it.

Quoted 17 lines
Quoted 17 lines
---
  drivers/iommu/intel/iommu.c  | 20 ++++++++++++++++++++
  drivers/iommu/virtio-iommu.c |  9 +++++++++
  include/linux/iommu.h        |  1 +
  3 files changed, 30 insertions(+)
drivers/iommu/intel/iommu.c +0 -0
index cb205a04fe4c..8eb022d0e8aa 100644
@@ -5738,12 +5738,32 @@ static inline bool nested_mode_support(void)
  	return ret;
  }
 
Quoted 4 lines

drivers/iommu/intel/iommu.c static inline bool nested_mode_support(void) · line 5742

Quoted 2 lines
drivers/iommu/intel/iommu.c +2 -0
+static inline bool caching_mode_enabled(void)
+{

Kernel coding style is not in favour of inline functions in *.c files.

Yes, agreed.

Best regards,
baolu

Quoted 60 lines
Quoted 60 lines
drivers/iommu/intel/iommu.c +18 -0
+	struct dmar_drhd_unit *drhd;
+	struct intel_iommu *iommu;
+	bool ret = false;
+
+	rcu_read_lock();
+	for_each_active_iommu(iommu, drhd) {
+		if (cap_caching_mode(iommu->cap)) {
+			ret = true;
+			break;
+		}
+	}
+	rcu_read_unlock();
+
+	return ret;
+}
+
  static bool intel_iommu_capable(enum iommu_cap cap)
  {
  	if (cap == IOMMU_CAP_CACHE_COHERENCY)
  		return domain_update_iommu_snooping(NULL) == 1;
  	if (cap == IOMMU_CAP_INTR_REMAP)
  		return irq_remapping_enabled == 1;
+	if (cap == IOMMU_CAP_VIOMMU)
+		return caching_mode_enabled();
 
  	return false;
  }
drivers/iommu/virtio-iommu.c +9 -0
index 2bfdd5734844..719793e103db 100644
@@ -931,7 +931,16 @@ static int viommu_of_xlate(struct device *dev, struct of_phandle_args *args)
  	return iommu_fwspec_add_ids(dev, args->args, 1);
  }
 
+static bool viommu_capable(enum iommu_cap cap)
+{
+	if (cap == IOMMU_CAP_VIOMMU)
+		return true;
+
+	return false;
+}
+
  static struct iommu_ops viommu_ops = {
+	.capable		= viommu_capable,
  	.domain_alloc		= viommu_domain_alloc,
  	.domain_free		= viommu_domain_free,
  	.attach_dev		= viommu_attach_dev,
include/linux/iommu.h +1 -0
index b95a6f8db6ff..1d24be667a03 100644
@@ -94,6 +94,7 @@ enum iommu_cap {
  					   transactions */
  	IOMMU_CAP_INTR_REMAP,		/* IOMMU supports interrupt isolation */
  	IOMMU_CAP_NOEXEC,		/* IOMMU_NOEXEC flag */
+	IOMMU_CAP_VIOMMU,		/* IOMMU can declar running in a VM */
  };
 
Quoted 4 lines
Quoted 4 lines
include/linux/iommu.h +0 -1
  /*
--

2.25.1

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu

#4 lore raw

Re: [RFC PATCH v3 1/2] iommu: Add capability IOMMU_CAP_VIOMMU

Quoted 4 lines · Lu Baolu

Hi Leon,

Quoted 1 line · Leon Romanovsky

On Thu, Jan 14, 2021 at 09:30:02AM +0800, Lu Baolu wrote:

Quoted 8 lines
Quoted 5 lines
Quoted 3 lines

Some vendor IOMMU drivers are able to declare that it is running in a VM context. This is very valuable for the features that only want to be supported on bare metal. Add a capability bit so that it could be used.

And how is it used? Who and how will set it?

Use the existing iommu_capable(). I should add more descriptions about who and how to use it.

I want to see the code that sets this capability.

Thanks
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu

author maintainer
#5 lore raw

Re: [RFC PATCH v3 1/2] iommu: Add capability IOMMU_CAP_VIOMMU

Hi,

Quoted 9 lines · Leon Romanovsky
Quoted 8 lines · Lu Baolu

Hi Leon,

Quoted 5 lines · Leon Romanovsky
Quoted 3 lines · Lu Baolu

Some vendor IOMMU drivers are able to declare that it is running in a VM context. This is very valuable for the features that only want to be supported on bare metal. Add a capability bit so that it could be used.

Quoted 6 lines
Quoted 4 lines
Quoted 1 line

And how is it used? Who and how will set it?

Use the existing iommu_capable(). I should add more descriptions about who and how to use it.

I want to see the code that sets this capability.

Currently we have Intel VT-d and the virt-iommu setting this capability.

static bool intel_iommu_capable(enum iommu_cap cap) {
if (cap == IOMMU_CAP_CACHE_COHERENCY)
return domain_update_iommu_snooping(NULL) == 1;
if (cap == IOMMU_CAP_INTR_REMAP)
return irq_remapping_enabled == 1;
+ if (cap == IOMMU_CAP_VIOMMU)
+ return caching_mode_enabled();

return false;
}

And,

+static bool viommu_capable(enum iommu_cap cap)
+{
+ if (cap == IOMMU_CAP_VIOMMU)
+ return true;
+
+ return false;
+}

Best regards,
baolu
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu

#6 lore raw

Re: [RFC PATCH v3 1/2] iommu: Add capability IOMMU_CAP_VIOMMU

Quoted 35 lines · Lu Baolu

Hi,

Quoted 9 lines · Leon Romanovsky
Quoted 8 lines · Lu Baolu

Hi Leon,

Quoted 5 lines · Leon Romanovsky
Quoted 3 lines · Lu Baolu

Some vendor IOMMU drivers are able to declare that it is running in a VM context. This is very valuable for the features that only want to be supported on bare metal. Add a capability bit so that it could be used.

Quoted 6 lines
Quoted 4 lines
Quoted 1 line

And how is it used? Who and how will set it?

Use the existing iommu_capable(). I should add more descriptions about who and how to use it.

I want to see the code that sets this capability.

Currently we have Intel VT-d and the virt-iommu setting this capability.

static bool intel_iommu_capable(enum iommu_cap cap) {

drivers/iommu/intel/iommu.c +2 -0
 	if (cap == IOMMU_CAP_CACHE_COHERENCY)
 		return domain_update_iommu_snooping(NULL) == 1;
 	if (cap == IOMMU_CAP_INTR_REMAP)
 		return irq_remapping_enabled == 1;
+	if (cap == IOMMU_CAP_VIOMMU)
+		return caching_mode_enabled();
 
 	return false;
 }

And,

drivers/iommu/virtio-iommu.c static int viommu_of_xlate(struct device *dev, struct of_phandle_args *args) · line 940

Quoted 7 lines
drivers/iommu/virtio-iommu.c +7 -0
+static bool viommu_capable(enum iommu_cap cap)
+{
+	if (cap == IOMMU_CAP_VIOMMU)
+		return true;
+
+	return false;
+}

These two functions are reading this cap and not setting. Where can I see code that does "cap = IOMMU_CAP_VIOMMU" and not "=="?

Quoted 3 lines

Best regards,
baolu

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu

author maintainer
#7 lore raw

Re: [RFC PATCH v3 1/2] iommu: Add capability IOMMU_CAP_VIOMMU

Hi Leon,

Quoted 36 lines · Leon Romanovsky
Quoted 35 lines · Lu Baolu

Hi,

Quoted 9 lines · Leon Romanovsky
Quoted 8 lines · Lu Baolu

Hi Leon,

Quoted 5 lines · Leon Romanovsky
Quoted 3 lines · Lu Baolu

Some vendor IOMMU drivers are able to declare that it is running in a VM context. This is very valuable for the features that only want to be supported on bare metal. Add a capability bit so that it could be used.

Quoted 6 lines
Quoted 4 lines
Quoted 1 line

And how is it used? Who and how will set it?

Use the existing iommu_capable(). I should add more descriptions about who and how to use it.

I want to see the code that sets this capability.

Currently we have Intel VT-d and the virt-iommu setting this capability.

  static bool intel_iommu_capable(enum iommu_cap cap)
  {
  	if (cap == IOMMU_CAP_CACHE_COHERENCY)
  		return domain_update_iommu_snooping(NULL) == 1;
  	if (cap == IOMMU_CAP_INTR_REMAP)
  		return irq_remapping_enabled == 1;
drivers/iommu/intel/iommu.c +2 -0
+	if (cap == IOMMU_CAP_VIOMMU)
+		return caching_mode_enabled();
 
  	return false;
  }

And,

Quoted 10 lines

drivers/iommu/virtio-iommu.c static int viommu_of_xlate(struct device *dev, struct of_phandle_args *args) · line 940

Quoted 7 lines
drivers/iommu/virtio-iommu.c +7 -0
+static bool viommu_capable(enum iommu_cap cap)
+{
+	if (cap == IOMMU_CAP_VIOMMU)
+		return true;
+
+	return false;
+}

These two functions are reading this cap and not setting. Where can I see code that does "cap = IOMMU_CAP_VIOMMU" and not "=="?

The iommu_capable() is a generic IOMMU interface to query IOMMU capabilities. It takes @bus and @cap as input, and calls the callback of vendor iommu. If the vendor iommu driver supports the specific capability, it returns true. Otherwise, it returns false.

bool iommu_capable(struct bus_type *bus, enum iommu_cap cap)

{
        if (!bus->iommu_ops || !bus->iommu_ops->capable)
                return false;

        return bus->iommu_ops->capable(cap);
}

EXPORT_SYMBOL_GPL(iommu_capable);

In the vendor iommu's callback, it checks the capability and returns a value according to its capability, just as showed above.

Best regards,
baolu
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu

#8 lore raw

Re: [RFC PATCH v3 1/2] iommu: Add capability IOMMU_CAP_VIOMMU

Quoted 62 lines · Lu Baolu

Hi Leon,

Quoted 36 lines · Leon Romanovsky
Quoted 35 lines · Lu Baolu

Hi,

Quoted 9 lines · Leon Romanovsky
Quoted 8 lines · Lu Baolu

Hi Leon,

Quoted 5 lines · Leon Romanovsky
Quoted 3 lines · Lu Baolu

Some vendor IOMMU drivers are able to declare that it is running in a VM context. This is very valuable for the features that only want to be supported on bare metal. Add a capability bit so that it could be used.

Quoted 6 lines
Quoted 4 lines
Quoted 1 line

And how is it used? Who and how will set it?

Use the existing iommu_capable(). I should add more descriptions about who and how to use it.

I want to see the code that sets this capability.

Currently we have Intel VT-d and the virt-iommu setting this capability.

  static bool intel_iommu_capable(enum iommu_cap cap)
  {
  	if (cap == IOMMU_CAP_CACHE_COHERENCY)
  		return domain_update_iommu_snooping(NULL) == 1;
  	if (cap == IOMMU_CAP_INTR_REMAP)
  		return irq_remapping_enabled == 1;
(quoted diff) +2 -0
+	if (cap == IOMMU_CAP_VIOMMU)
+		return caching_mode_enabled();
 
  	return false;
  }

And,

Quoted 10 lines

Review

Quoted 7 lines
(quoted diff) +7 -0
+static bool viommu_capable(enum iommu_cap cap)
+{
+	if (cap == IOMMU_CAP_VIOMMU)
+		return true;
+
+	return false;
+}

These two functions are reading this cap and not setting. Where can I see code that does "cap = IOMMU_CAP_VIOMMU" and not "=="?

The iommu_capable() is a generic IOMMU interface to query IOMMU capabilities. It takes @bus and @cap as input, and calls the callback of vendor iommu. If the vendor iommu driver supports the specific capability, it returns true. Otherwise, it returns false.

bool iommu_capable(struct bus_type *bus, enum iommu_cap cap)

{
        if (!bus->iommu_ops || !bus->iommu_ops->capable)
                return false;

        return bus->iommu_ops->capable(cap);
}
Quoted 4 lines

EXPORT_SYMBOL_GPL(iommu_capable);

In the vendor iommu's callback, it checks the capability and returns a value according to its capability, just as showed above.

Ohh, sorry.
I missed "iommu_capable(dev->bus, IOMMU_CAP_VIOMMU)" from second patch.

Thanks

Quoted 3 lines

Best regards,
baolu

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu

#9 lore raw

Re: [RFC PATCH v3 1/2] iommu: Add capability IOMMU_CAP_VIOMMU

In reply to Lu Baolu
Quoted 35 lines · Lu Baolu

Some vendor IOMMU drivers are able to declare that it is running in a VM context. This is very valuable for the features that only want to be supported on bare metal. Add a capability bit so that it could be used.

---
  drivers/iommu/intel/iommu.c  | 20 ++++++++++++++++++++
  drivers/iommu/virtio-iommu.c |  9 +++++++++
  include/linux/iommu.h        |  1 +
  3 files changed, 30 insertions(+)
drivers/iommu/intel/iommu.c +16 -0
index cb205a04fe4c..8eb022d0e8aa 100644
@@ -5738,12 +5738,32 @@ static inline bool nested_mode_support(void)
  	return ret;
  }
 
+static inline bool caching_mode_enabled(void)
+{
+	struct dmar_drhd_unit *drhd;
+	struct intel_iommu *iommu;
+	bool ret = false;
+
+	rcu_read_lock();
+	for_each_active_iommu(iommu, drhd) {
+		if (cap_caching_mode(iommu->cap)) {
+			ret = true;
+			break;
+		}
+	}
+	rcu_read_unlock();
+
+	return ret;

drivers/iommu/intel/iommu.c static inline bool nested_mode_support(void) · line 5766

Quoted 10 lines
drivers/iommu/intel/iommu.c +4 -0
+}
+
  static bool intel_iommu_capable(enum iommu_cap cap)
  {
  	if (cap == IOMMU_CAP_CACHE_COHERENCY)
  		return domain_update_iommu_snooping(NULL) == 1;
  	if (cap == IOMMU_CAP_INTR_REMAP)
  		return irq_remapping_enabled == 1;
+	if (cap == IOMMU_CAP_VIOMMU)
+		return caching_mode_enabled();

This part I don't understand. Does it mean Intel IOMMU can't be used in VM without caching mode?

Thanks

Quoted 29 lines
drivers/iommu/intel/iommu.c +0 -0
  	return false;
  }
drivers/iommu/virtio-iommu.c +9 -0
index 2bfdd5734844..719793e103db 100644
@@ -931,7 +931,16 @@ static int viommu_of_xlate(struct device *dev, struct of_phandle_args *args)
  	return iommu_fwspec_add_ids(dev, args->args, 1);
  }
 
+static bool viommu_capable(enum iommu_cap cap)
+{
+	if (cap == IOMMU_CAP_VIOMMU)
+		return true;
+
+	return false;
+}
+
  static struct iommu_ops viommu_ops = {
+	.capable		= viommu_capable,
  	.domain_alloc		= viommu_domain_alloc,
  	.domain_free		= viommu_domain_free,
  	.attach_dev		= viommu_attach_dev,
include/linux/iommu.h +0 -0
index b95a6f8db6ff..1d24be667a03 100644

include/linux/iommu.h enum iommu_cap { · line 100

Quoted 7 lines
include/linux/iommu.h +1 -0
@@ -94,6 +94,7 @@ enum iommu_cap {
  					   transactions */
  	IOMMU_CAP_INTR_REMAP,		/* IOMMU supports interrupt isolation */
  	IOMMU_CAP_NOEXEC,		/* IOMMU_NOEXEC flag */
+	IOMMU_CAP_VIOMMU,		/* IOMMU can declar running in a VM */
  };
 
  /*

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu

author maintainer
#10 lore raw

RFC PATCH v3 2/2 platform-msi: Add platform check for subdevice irq domain

In reply to Lu Baolu

The pci_subdevice_msi_create_irq_domain() should fail if the underlying platform is not able to support IMS (Interrupt Message Storage). Otherwise, the isolation of interrupt is not guaranteed.

For x86, IMS is only supported on bare metal for now. We could enable it in the virtualization environments in the future if interrupt HYPERCALL domain is supported or the hardware has the capability of interrupt isolation for subdevices.

---
 arch/x86/pci/common.c       | 71 +++++++++++++++++++++++++++++++++++++
 drivers/base/platform-msi.c |  8 +++++
 include/linux/msi.h         |  1 +
 3 files changed, 80 insertions(+)
arch/x86/pci/common.c +71 -0
index 3507f456fcd0..9deb826fb242 100644
@@ -12,6 +12,7 @@
 #include <linux/init.h>
 #include <linux/dmi.h>
 #include <linux/slab.h>
+#include <linux/iommu.h>
 
 #include <asm/acpi.h>
 #include <asm/segment.h>
@@ -724,3 +725,73 @@ struct pci_dev *pci_real_dma_dev(struct pci_dev *dev)
 	return dev;
 }
 #endif
+
+/*
+ * We want to figure out which context we are running in. But the hardware
+ * does not introduce a reliable way (instruction, CPUID leaf, MSR, whatever)
+ * which can be manipulated by the VMM to let the OS figure out where it runs.
+ * So we go with the below probably on_bare_metal() function as a replacement
+ * for definitely on_bare_metal() to go forward only for the very simple reason
+ * that this is the only option we have.
+ */
+static const char * const vmm_vendor_name[] = {
+	"QEMU", "Bochs", "KVM", "Xen", "VMware", "VMW", "VMware Inc.",
+	"innotek GmbH", "Oracle Corporation", "Parallels", "BHYVE"
+};
+
+static void read_type0_virtual_machine(const struct dmi_header *dm, void *p)
+{
+	u8 *data = (u8 *)dm + 0x13;
+
+	/* BIOS Information (Type 0) */
+	if (dm->type != 0 || dm->length < 0x14)
+		return;
+
+	/* Bit 4 of BIOS Characteristics Extension Byte 2*/
+	if (*data & BIT(4))
+		*((bool *)p) = true;
+}
+
+static bool smbios_virtual_machine(void)
+{
+	bool bit_present = false;
+
+	dmi_walk(read_type0_virtual_machine, &bit_present);
+
+	return bit_present;
+}
+
+static bool on_bare_metal(struct device *dev)
+{
+	int i;
+
+	if (boot_cpu_has(X86_FEATURE_HYPERVISOR))
+		return false;
+
+	if (smbios_virtual_machine())
+		return false;
+
+	if (iommu_capable(dev->bus, IOMMU_CAP_VIOMMU))
+		return false;
+
+	for (i = 0; i < ARRAY_SIZE(vmm_vendor_name); i++)
+		if (dmi_match(DMI_SYS_VENDOR, vmm_vendor_name[i]))
+			return false;
+
+	pr_info("System running on bare metal, report to bugzilla.kernel.org if not the case.");
+
+	return true;
+}
+
+bool arch_support_pci_device_ims(struct pci_dev *pdev)
+{
+	/*
+	 * When we are running in a VMM context, the device IMS could only be
+	 * enabled when the underlying hardware supports interrupt isolation
+	 * of the subdevice, or any mechanism (trap, hypercall) is added so
+	 * that changes in the interrupt message store could be managed by the
+	 * VMM. For now, we only support the device IMS when we are running on
+	 * the bare metal.
+	 */
+	return on_bare_metal(&pdev->dev);
+}
drivers/base/platform-msi.c +8 -0
index 8432a1bf4e28..88e5fe4dae67 100644
@@ -512,6 +512,11 @@ struct irq_domain *device_msi_create_irq_domain(struct fwnode_handle *fn,
 #ifdef CONFIG_PCI
 #include <linux/pci.h>
 
+bool __weak arch_support_pci_device_ims(struct pci_dev *pdev)
+{
+	return false;
+}
+
 /**
  * pci_subdevice_msi_create_irq_domain - Create an irq domain for subdevices
  * @pdev:	Pointer to PCI device for which the subdevice domain is created
@@ -523,6 +528,9 @@ struct irq_domain *pci_subdevice_msi_create_irq_domain(struct pci_dev *pdev,
 	struct irq_domain *domain, *pdev_msi;
 	struct fwnode_handle *fn;
 
+	if (!arch_support_pci_device_ims(pdev))
+		return NULL;
+
 	/*
 	 * Retrieve the MSI domain of the underlying PCI device's MSI
 	 * domain. The PCI device domain's parent domain is also the parent
include/linux/msi.h +1 -0
index f319d7c6a4ef..6fda81c4b859 100644
@@ -478,6 +478,7 @@ struct irq_domain *device_msi_create_irq_domain(struct fwnode_handle *fn,
 						struct irq_domain *parent);
 
 # ifdef CONFIG_PCI
+bool arch_support_pci_device_ims(struct pci_dev *pdev);
 struct irq_domain *pci_subdevice_msi_create_irq_domain(struct pci_dev *pdev,
 						       struct msi_domain_info *info);
 # endif
Signature · 6 lines
2.25.1

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
#11 lore raw

RE: [RFC PATCH v3 2/2] platform-msi: Add platform check for subdevice irq domain

Quoted 93 lines

From: Lu Baolu
Sent: Thursday, January 14, 2021 9:30 AM

The pci_subdevice_msi_create_irq_domain() should fail if the underlying platform is not able to support IMS (Interrupt Message Storage). Otherwise, the isolation of interrupt is not guaranteed.

For x86, IMS is only supported on bare metal for now. We could enable it in the virtualization environments in the future if interrupt HYPERCALL domain is supported or the hardware has the capability of interrupt isolation for subdevices.

pci/[email protected]/

pci/[email protected]/

pci/[email protected]/

---
 arch/x86/pci/common.c       | 71
+++++++++++++++++++++++++++++++++++++
 drivers/base/platform-msi.c |  8 +++++
 include/linux/msi.h         |  1 +
 3 files changed, 80 insertions(+)
arch/x86/pci/common.c +6 -0
index 3507f456fcd0..9deb826fb242 100644
@@ -12,6 +12,7 @@
 #include <linux/init.h>
 #include <linux/dmi.h>
 #include <linux/slab.h>
+#include <linux/iommu.h>
 
 #include <asm/acpi.h>
 #include <asm/segment.h>
@@ -724,3 +725,73 @@ struct pci_dev *pci_real_dma_dev(struct pci_dev
 *dev)
 	return dev;
 }
 #endif
+
+/*
+ * We want to figure out which context we are running in. But the hardware
+ * does not introduce a reliable way (instruction, CPUID leaf, MSR, whatever)
+ * which can be manipulated by the VMM to let the OS figure out where it

runs.
+ * So we go with the below probably on_bare_metal() function as a replacement
+ * for definitely on_bare_metal() to go forward only for the very simple reason
+ * that this is the only option we have.
+ */

arch/x86/pci/common.c +6 -0
+static const char * const vmm_vendor_name[] = {
+	"QEMU", "Bochs", "KVM", "Xen", "VMware", "VMW", "VMware Inc.",
+	"innotek GmbH", "Oracle Corporation", "Parallels", "BHYVE"
+};
+
+static void read_type0_virtual_machine(const struct dmi_header *dm, void

*p)

arch/x86/pci/common.c +27 -0
+{
+	u8 *data = (u8 *)dm + 0x13;
+
+	/* BIOS Information (Type 0) */
+	if (dm->type != 0 || dm->length < 0x14)
+		return;
+
+	/* Bit 4 of BIOS Characteristics Extension Byte 2*/
+	if (*data & BIT(4))
+		*((bool *)p) = true;
+}
+
+static bool smbios_virtual_machine(void)
+{
+	bool bit_present = false;
+
+	dmi_walk(read_type0_virtual_machine, &bit_present);
+
+	return bit_present;
+}
+
+static bool on_bare_metal(struct device *dev)
+{
+	int i;
+
+	if (boot_cpu_has(X86_FEATURE_HYPERVISOR))
+		return false;

arch/x86/pci/common.c struct pci_dev *pci_real_dma_dev(struct pci_dev *dev) · line 779

Quoted 10 lines
arch/x86/pci/common.c +10 -0
+
+	if (smbios_virtual_machine())
+		return false;
+
+	if (iommu_capable(dev->bus, IOMMU_CAP_VIOMMU))
+		return false;
+
+	for (i = 0; i < ARRAY_SIZE(vmm_vendor_name); i++)
+		if (dmi_match(DMI_SYS_VENDOR, vmm_vendor_name[i]))
+			return false;

Thinking more I wonder whether this check is actually useful here. As Leon and David commented, the same vendor name can be used both for VM and bare metal instances. It implies that both bare metal and VM might be misinterpreted with this check. This might not be what we want originally - find heuristics to indicate a VM environment and tolerate misinterpreting VM as bare metal in corner cases (but not vice versa).

Thomas?

Thanks
Kevin

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu