Skip to contentSkip to conversation

Add support for ACPI device in RMRR

PATCH In review FelixCuioc 2 participants 5 messages Oct 10, 2020 – Oct 13, 2020

Subsystems (2)
lore
Share & tools
lore

FelixCuioc

author
#0 lore raw

v3 cover Add support for ACPI device in RMRR

BIOS allocate reserved memory ranges that may be DMA targets. BIOS may report each such reserved memory region through the RMRR structures,along with the devices that requires access to the specified reserved memory region.

The purpose of this series is to achieve ACPI device in RMRR access reserved memory.Therefore,it is necessary to increase the analysis of acpi device in RMRR and establish a mapping for this device.

The first patch adds interfaces for detecting ACPI device in RMRR and in order to distinguish it from pci device, some interface functions are modified.

The second patch adds support for probing ACPI device in RMRR. In probe_acpi_namespace_devices(),add support for direct mapping of ACPI device and add support for physical node of acpi device to be NULL.

v2->v3:
- Add the blank line between functions.
- Make dmar_acpi_insert_dev_scope() bool,change the 1/0 to true/false and add a comment explaining.
- Delete unused initialization.
- if dmar_acpi_insert_dev_scope() always returns zero,will not call dmar_rmrr_add_acpi_dev().
- Use a proper error code.
- Use if(!pdev).
- Use goto unlock instead of mutex_unlock().

FelixCuioc (2):
iommu/vt-d:Add support for detecting ACPI device in RMRR iommu/vt-d:Add support for probing ACPI device in RMRR

 drivers/iommu/intel/dmar.c  | 76 +++++++++++++++++++++----------------
 drivers/iommu/intel/iommu.c | 52 ++++++++++++++++++++++++-
 drivers/iommu/iommu.c       |  6 +++
 include/linux/dmar.h        | 12 +++++-
 include/linux/iommu.h       |  2 +
 5 files changed, 113 insertions(+), 35 deletions(-)
Signature · 6 lines
2.17.1

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
author
#1 lore raw

PATCH v3 1/2 iommu/vt-d:Add support for detecting ACPI device in RMRR

Some ACPI devices need to issue dma requests to access the reserved memory area.BIOS uses the device scope type ACPI_NAMESPACE_DEVICE in RMRR to report these ACPI devices. This patch add support for detecting ACPI devices in RMRR.

---
 drivers/iommu/intel/dmar.c  | 76 +++++++++++++++++++++----------------
 drivers/iommu/intel/iommu.c | 23 ++++++++++-
 include/linux/dmar.h        | 12 +++++-
 3 files changed, 76 insertions(+), 35 deletions(-)
drivers/iommu/intel/dmar.c +44 -32
index 54d790f75824..5ebe317280a6 100644
@@ -215,7 +215,7 @@ static bool dmar_match_pci_path(struct dmar_pci_notify_info *info, int bus,
 }
 
 /* Return: > 0 if match found, 0 if no match found, < 0 if error happens */
-int dmar_insert_dev_scope(struct dmar_pci_notify_info *info,
+int dmar_pci_insert_dev_scope(struct dmar_pci_notify_info *info,
 			  void *start, void*end, u16 segment,
 			  struct dmar_dev_scope *devices,
 			  int devices_cnt)
@@ -304,7 +304,7 @@ static int dmar_pci_bus_add_dev(struct dmar_pci_notify_info *info)
 
 		drhd = container_of(dmaru->hdr,
 				    struct acpi_dmar_hardware_unit, header);
-		ret = dmar_insert_dev_scope(info, (void *)(drhd + 1),
+		ret = dmar_pci_insert_dev_scope(info, (void *)(drhd + 1),
 				((void *)drhd) + drhd->header.length,
 				dmaru->segment,
 				dmaru->devices, dmaru->devices_cnt);
@@ -697,47 +697,59 @@ dmar_find_matched_drhd_unit(struct pci_dev *dev)
 	return dmaru;
 }
 
-static void __init dmar_acpi_insert_dev_scope(u8 device_number,
-					      struct acpi_device *adev)
+/* Return: > 0 if match found, 0 if no match found */
+bool dmar_acpi_insert_dev_scope(u8 device_number,
+				struct acpi_device *adev,
+				void *start, void *end,
+				struct dmar_dev_scope *devices,
+				int devices_cnt)
 {
-	struct dmar_drhd_unit *dmaru;
-	struct acpi_dmar_hardware_unit *drhd;
 	struct acpi_dmar_device_scope *scope;
 	struct device *tmp;
 	int i;
 	struct acpi_dmar_pci_path *path;
 
+	for (; start < end; start += scope->length) {
+		scope = start;
+		if (scope->entry_type != ACPI_DMAR_SCOPE_TYPE_NAMESPACE)
+			continue;
+		if (scope->enumeration_id != device_number)
+			continue;
+		path = (void *)(scope + 1);
+		for_each_dev_scope(devices, devices_cnt, i, tmp)
+			if (tmp == NULL) {
+				devices[i].bus = scope->bus;
+				devices[i].devfn = PCI_DEVFN(path->device, path->function);
+				rcu_assign_pointer(devices[i].dev,
+						   get_device(&adev->dev));
+				return true;
+			}
+		WARN_ON(i >= devices_cnt);
+	}
+	return false;
+}
+
+static int dmar_acpi_bus_add_dev(u8 device_number, struct acpi_device *adev)
+{
+	struct dmar_drhd_unit *dmaru;
+	struct acpi_dmar_hardware_unit *drhd;
+	int ret;
+
 	for_each_drhd_unit(dmaru) {
 		drhd = container_of(dmaru->hdr,
 				    struct acpi_dmar_hardware_unit,
 				    header);
+		ret = dmar_acpi_insert_dev_scope(device_number, adev, (void *)(drhd+1),
+						((void *)drhd)+drhd->header.length,
+						dmaru->devices, dmaru->devices_cnt);
+		if (ret)
+			break;
+	}
+	if (ret > 0)
+		ret = dmar_rmrr_add_acpi_dev(device_number, adev);
 
-		for (scope = (void *)(drhd + 1);
-		     (unsigned long)scope < ((unsigned long)drhd) + drhd->header.length;
-		     scope = ((void *)scope) + scope->length) {
-			if (scope->entry_type != ACPI_DMAR_SCOPE_TYPE_NAMESPACE)
-				continue;
-			if (scope->enumeration_id != device_number)
-				continue;
+	return ret;
 
-			path = (void *)(scope + 1);
-			pr_info("ACPI device \"%s\" under DMAR at %llx as %02x:%02x.%d\n",
-				dev_name(&adev->dev), dmaru->reg_base_addr,
-				scope->bus, path->device, path->function);
-			for_each_dev_scope(dmaru->devices, dmaru->devices_cnt, i, tmp)
-				if (tmp == NULL) {
-					dmaru->devices[i].bus = scope->bus;
-					dmaru->devices[i].devfn = PCI_DEVFN(path->device,
-									    path->function);
-					rcu_assign_pointer(dmaru->devices[i].dev,
-							   get_device(&adev->dev));
-					return;
-				}
-			BUG_ON(i >= dmaru->devices_cnt);
-		}
-	}
-	pr_warn("No IOMMU scope found for ANDD enumeration ID %d (%s)\n",
-		device_number, dev_name(&adev->dev));
 }
 
 static int __init dmar_acpi_dev_scope_init(void)
@@ -766,7 +778,7 @@ static int __init dmar_acpi_dev_scope_init(void)
 				       andd->device_name);
 				continue;
 			}
-			dmar_acpi_insert_dev_scope(andd->device_number, adev);
+			dmar_acpi_bus_add_dev(andd->device_number, adev);
 		}
 	}
 	return 0;
drivers/iommu/intel/iommu.c +21 -2
index 2485441445cf..594b2f29c043 100644
@@ -4553,6 +4553,25 @@ int dmar_find_matched_atsr_unit(struct pci_dev *dev)
 	return ret;
 }
 
+int dmar_rmrr_add_acpi_dev(u8 device_number, struct acpi_device *adev)
+{
+	int ret;
+	struct dmar_rmrr_unit *rmrru;
+	struct acpi_dmar_reserved_memory *rmrr;
+
+	list_for_each_entry(rmrru, &dmar_rmrr_units, list) {
+		rmrr = container_of(rmrru->hdr,
+				struct acpi_dmar_reserved_memory,
+				header);
+		ret = dmar_acpi_insert_dev_scope(device_number, adev, (void *)(rmrr + 1),
+						((void *)rmrr) + rmrr->header.length,
+						rmrru->devices, rmrru->devices_cnt);
+		if (ret)
+			break;
+	}
+	return 0;
+}
+
 int dmar_iommu_notify_scope_dev(struct dmar_pci_notify_info *info)
 {
 	int ret;
@@ -4568,7 +4587,7 @@ int dmar_iommu_notify_scope_dev(struct dmar_pci_notify_info *info)
 		rmrr = container_of(rmrru->hdr,
 				    struct acpi_dmar_reserved_memory, header);
 		if (info->event == BUS_NOTIFY_ADD_DEVICE) {
-			ret = dmar_insert_dev_scope(info, (void *)(rmrr + 1),
+			ret = dmar_pci_insert_dev_scope(info, (void *)(rmrr + 1),
 				((void *)rmrr) + rmrr->header.length,
 				rmrr->segment, rmrru->devices,
 				rmrru->devices_cnt);
@@ -4586,7 +4605,7 @@ int dmar_iommu_notify_scope_dev(struct dmar_pci_notify_info *info)
 
 		atsr = container_of(atsru->hdr, struct acpi_dmar_atsr, header);
 		if (info->event == BUS_NOTIFY_ADD_DEVICE) {
-			ret = dmar_insert_dev_scope(info, (void *)(atsr + 1),
+			ret = dmar_pci_insert_dev_scope(info, (void *)(atsr + 1),
 					(void *)atsr + atsr->header.length,
 					atsr->segment, atsru->devices,
 					atsru->devices_cnt);
include/linux/dmar.h +11 -1
index 65565820328a..d0981d35d3c9 100644
@@ -113,10 +113,14 @@ extern int dmar_parse_dev_scope(void *start, void *end, int *cnt,
 				struct dmar_dev_scope **devices, u16 segment);
 extern void *dmar_alloc_dev_scope(void *start, void *end, int *cnt);
 extern void dmar_free_dev_scope(struct dmar_dev_scope **devices, int *cnt);
-extern int dmar_insert_dev_scope(struct dmar_pci_notify_info *info,
+extern int dmar_pci_insert_dev_scope(struct dmar_pci_notify_info *info,
 				 void *start, void*end, u16 segment,
 				 struct dmar_dev_scope *devices,
 				 int devices_cnt);
+extern bool dmar_acpi_insert_dev_scope(u8 device_number,
+				struct acpi_device *adev, void *start, void *end,
+				struct dmar_dev_scope *devices, int devices_cnt);
+
 extern int dmar_remove_dev_scope(struct dmar_pci_notify_info *info,
 				 u16 segment, struct dmar_dev_scope *devices,
 				 int count);
@@ -140,6 +144,7 @@ extern int dmar_parse_one_atsr(struct acpi_dmar_header *header, void *arg);
 extern int dmar_check_one_atsr(struct acpi_dmar_header *hdr, void *arg);
 extern int dmar_release_one_atsr(struct acpi_dmar_header *hdr, void *arg);
 extern int dmar_iommu_hotplug(struct dmar_drhd_unit *dmaru, bool insert);
+extern int dmar_rmrr_add_acpi_dev(u8 device_number, struct acpi_device *adev);
 extern int dmar_iommu_notify_scope_dev(struct dmar_pci_notify_info *info);
 #else /* !CONFIG_INTEL_IOMMU: */
 static inline int intel_iommu_init(void) { return -ENODEV; }
@@ -150,6 +155,11 @@ static inline void intel_iommu_shutdown(void) { }
 #define	dmar_check_one_atsr		dmar_res_noop
 #define	dmar_release_one_atsr		dmar_res_noop
 
+static inline int dmar_rmrr_add_acpi_dev(u8 device_number, struct acpi_device *adev)
+{
+	return 0;
+}
+
 static inline int dmar_iommu_notify_scope_dev(struct dmar_pci_notify_info *info)
 {
 	return 0;
Signature · 6 lines
2.17.1

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
author
#2 lore raw

PATCH v3 2/2 iommu/vt-d:Add support for probing ACPI device in RMRR

In reply to FelixCuioc

After acpi device in RMRR is detected,it is necessary to establish a mapping for these devices.
In acpi_device_create_direct_mappings(),create a mapping for the acpi device in RMRR.
Add a helper to achieve the acpi namespace device can access the RMRR region.

---
 drivers/iommu/intel/iommu.c | 29 +++++++++++++++++++++++++++++
 drivers/iommu/iommu.c       |  6 ++++++
 include/linux/iommu.h       |  2 ++
 3 files changed, 37 insertions(+)
drivers/iommu/intel/iommu.c +29 -0
index 594b2f29c043..a0fe867aa1f5 100644
@@ -4844,6 +4844,21 @@ static int __init platform_optin_force_iommu(void)
 	return 1;
 }
 
+static int acpi_device_create_direct_mappings(struct device *pn_dev, struct device *acpi_device)
+{
+	struct iommu_group *group;
+
+	acpi_device->bus->iommu_ops = &intel_iommu_ops;
+	group = iommu_group_get(pn_dev);
+	if (!group) {
+		pr_warn("ACPI name space devices create direct mappings wrong!\n");
+		return -EINVAL;
+	}
+	__acpi_device_create_direct_mappings(group, acpi_device);
+
+	return 0;
+}
+
 static int __init probe_acpi_namespace_devices(void)
 {
 	struct dmar_drhd_unit *drhd;
@@ -4858,6 +4873,7 @@ static int __init probe_acpi_namespace_devices(void)
 			struct acpi_device_physical_node *pn;
 			struct iommu_group *group;
 			struct acpi_device *adev;
+			struct device *pn_dev = NULL;
 
 			if (dev->bus != &acpi_bus_type)
 				continue;
@@ -4868,6 +4884,7 @@ static int __init probe_acpi_namespace_devices(void)
 					    &adev->physical_node_list, node) {
 				group = iommu_group_get(pn->dev);
 				if (group) {
+					pn_dev = pn->dev;
 					iommu_group_put(group);
 					continue;
 				}
@@ -4876,7 +4893,19 @@ static int __init probe_acpi_namespace_devices(void)
 				ret = iommu_probe_device(pn->dev);
 				if (ret)
 					break;
+				pn_dev = pn->dev;
+			}
+			if (!pn_dev) {
+				dev->bus->iommu_ops = &intel_iommu_ops;
+				ret = iommu_probe_device(dev);
+				if (ret) {
+					pr_err("acpi_device probe fail! ret:%d\n", ret);
+					goto unlock;
+				}
+				goto unlock;
 			}
+			ret = acpi_device_create_direct_mappings(pn_dev, dev);
+unlock:
 			mutex_unlock(&adev->physical_node_lock);
 
 			if (ret)
drivers/iommu/iommu.c +6 -0
index 6d847027d35e..ffd470297ea6 100644
@@ -779,6 +779,12 @@ static bool iommu_is_attach_deferred(struct iommu_domain *domain,
 	return false;
 }
 
+void  __acpi_device_create_direct_mappings(struct iommu_group *group, struct device *acpi_device)
+{
+	iommu_create_device_direct_mappings(group, acpi_device);
+}
+EXPORT_SYMBOL_GPL(__acpi_device_create_direct_mappings);
+
 /**
  * iommu_group_add_device - add a device to an iommu group
  * @group: the group into which to add the device (reference should be held)
include/linux/iommu.h +2 -0
index 82876f682367..bfc46e538c29 100644
@@ -516,6 +516,8 @@ extern void iommu_domain_window_disable(struct iommu_domain *domain, u32 wnd_nr)
 
 extern int report_iommu_fault(struct iommu_domain *domain, struct device *dev,
 			      unsigned long iova, int flags);
+extern void __acpi_device_create_direct_mappings(struct iommu_group *group,
+						 struct device *acpi_device);
 
 static inline void iommu_flush_iotlb_all(struct iommu_domain *domain)
 {
Signature · 6 lines
2.17.1

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
maintainer
#3 lore raw
In reply to FelixCuioc

Hi Felix,

Quoted 10 lines · FelixCuioc

BIOS allocate reserved memory ranges that may be DMA targets. BIOS may report each such reserved memory region through the RMRR structures,along with the devices that requires access to the specified reserved memory region.

The purpose of this series is to achieve ACPI device in RMRR access reserved memory.Therefore,it is necessary to increase the analysis of acpi device in RMRR and establish a mapping for this device.

Quoted 8 lines

The first patch adds interfaces for detecting ACPI device in RMRR and in order to distinguish it from pci device, some interface functions are modified.

The second patch adds support for probing ACPI device in RMRR. In probe_acpi_namespace_devices(),add support for direct mapping of ACPI device and add support for physical node of acpi device to be NULL.

Thanks for your patches. As I explained in the previous reply, RMRRs were added as work around for certain legacy device and we have been working hard to fix those legacy devices so that RMRR are no longer needed. Any new use case of RMRR is not encouraged.

By the way, I guess the problem you are facing can still be handled well under current RMRR mechanism by simple putting the device in the ACPI/ANDD table. It's worth trying.

Best regards,
baolu

Quoted 24 lines

v2->v3:
- Add the blank line between functions.
- Make dmar_acpi_insert_dev_scope() bool,change the 1/0 to true/false and add a comment explaining.
- Delete unused initialization.
- if dmar_acpi_insert_dev_scope() always returns zero,will not call dmar_rmrr_add_acpi_dev().
- Use a proper error code.
- Use if(!pdev).
- Use goto unlock instead of mutex_unlock().

FelixCuioc (2):
iommu/vt-d:Add support for detecting ACPI device in RMRR iommu/vt-d:Add support for probing ACPI device in RMRR

  drivers/iommu/intel/dmar.c  | 76 +++++++++++++++++++++----------------
  drivers/iommu/intel/iommu.c | 52 ++++++++++++++++++++++++-
  drivers/iommu/iommu.c       |  6 +++
  include/linux/dmar.h        | 12 +++++-
  include/linux/iommu.h       |  2 +
  5 files changed, 113 insertions(+), 35 deletions(-)
#4 lore raw

hi baolu,

Quoted 3 lines

By the way, I guess the problem you are facing can still be handled well under current RMRR mechanism by simple putting the device in the ACPI/ANDD table. It's worth trying.

I understand what you mean is that just put the device in the ANDD table and don't use RMRR, right?

But this can't solve my problem. Can you explain it in detail how to solve my problem?

In addition, the latest VT-D spec still has support for the use of acpi device under RMRR and ANDD.

From the perpective of supporting spec , the driver should support the case we submitted.

Best regards

Felixcui-oc

________________________________
发件人: Lu Baolu <[email protected]>
发送时间: 2020年10月12日 10:31:40
收件人: FelixCui-oc; Joerg Roedel; [email protected]; [email protected]; David Woodhouse; Dan Carpenter; [email protected] 抄送: [email protected]; CobeChen-oc; RaymondPang-oc; Tony W Wang-oc
主题: Re: [PATCH v3 0/2] Add support for ACPI device in RMRR

Hi Felix,

Quoted 10 lines · FelixCuioc

BIOS allocate reserved memory ranges that may be DMA targets. BIOS may report each such reserved memory region through the RMRR structures,along with the devices that requires access to the specified reserved memory region.

The purpose of this series is to achieve ACPI device in RMRR access reserved memory.Therefore,it is necessary to increase the analysis of acpi device in RMRR and establish a mapping for this device.

Quoted 8 lines

The first patch adds interfaces for detecting ACPI device in RMRR and in order to distinguish it from pci device, some interface functions are modified.

The second patch adds support for probing ACPI device in RMRR. In probe_acpi_namespace_devices(),add support for direct mapping of ACPI device and add support for physical node of acpi device to be NULL.

Thanks for your patches. As I explained in the previous reply, RMRRs were added as work around for certain legacy device and we have been working hard to fix those legacy devices so that RMRR are no longer needed. Any new use case of RMRR is not encouraged.

By the way, I guess the problem you are facing can still be handled well under current RMRR mechanism by simple putting the device in the ACPI/ANDD table. It's worth trying.

Best regards,
baolu

Quoted 17 lines

v2->v3:
- Add the blank line between functions.
- Make dmar_acpi_insert_dev_scope() bool,change the 1/0 to true/false and add a comment explaining.
- Delete unused initialization.
- if dmar_acpi_insert_dev_scope() always returns zero,will not call dmar_rmrr_add_acpi_dev().
- Use a proper error code.
- Use if(!pdev).
- Use goto unlock instead of mutex_unlock().

FelixCuioc (2):
iommu/vt-d:Add support for detecting ACPI device in RMRR iommu/vt-d:Add support for probing ACPI device in RMRR

Quoted 7 lines
  drivers/iommu/intel/dmar.c  | 76 +++++++++++++++++++++----------------
  drivers/iommu/intel/iommu.c | 52 ++++++++++++++++++++++++-
  drivers/iommu/iommu.c       |  6 +++
  include/linux/dmar.h        | 12 +++++-
  include/linux/iommu.h       |  2 +
  5 files changed, 113 insertions(+), 35 deletions(-)

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu