Skip to contentSkip to conversation

iommu/arm: Add Renesas IPMMU-VMSA support + Linux's iommu_fwspec

PATCH Superseded Oleksandr Tyshchenko 5 participants 41 messages Aug 20, 2019 – Sep 11, 2019

v3 of 6 Superseded Latest is v6 →

lore
Share & tools
lore
#0 lore raw

v3 cover iommu/arm: Add Renesas IPMMU-VMSA support + Linux's iommu_fwspec

From: Oleksandr Tyshchenko <[email protected]>

The purpose of this patch series is to add IPMMU-VMSA support to Xen on ARM.

Besides new IOMMU driver, this series contains "iommu_fwspec" support and new API iommu_add_dt_device() for adding DT device to IOMMU and many other things.

The IPMMU-VMSA is VMSA-compatible I/O Memory Management Unit (IOMMU) which provides address translation and access protection functionalities to processing units and interconnect networks.

Please note, this driver is supposed to work only with newest Gen3 SoCs revisions which IPMMU hardware supports stage 2 translation table format and is able to use CPU's P2M table as is if one is 3-level page table (up to 40 bit IPA).

----------
This driver is based on Linux's IPMMU-VMSA driver from Renesas BSP:
https://git.kernel.org/pub/scm/linux/kernel/git/horms/renesas-bsp.git/tree/drivers/iommu/ipmmu-vmsa.c?h=v4.14.75-ltsi/rcar-3.9.6
and Xen's SMMU driver:
xen/drivers/passthrough/arm/smmu.c

Although Xen driver has a lot in common with Linux driver, it is not a "direct ported" copy and should be treated as such.

The major differences compare to the Linux driver are:

1. Stage 1/Stage 2 translation. Linux driver supports Stage 1 translation only (with Stage 1 translation table format). It manages page table by itself. But Xen driver supports Stage 2 translation (with Stage 2 translation table format) to be able to share the P2M with the CPU. Stage 1 translation is always bypassed in Xen driver.

So, Xen driver is supposed to be used with newest Gen3 SoC revisions only (H3 ES3.0, M3-W+, etc.) which IPMMU H/W supports stage 2 translation table format.

2. AArch64 support. Linux driver uses VMSAv8-32 mode, while Xen driver enables Armv8 VMSAv8-64 mode to cover up to 40 bit input address.

3. Context bank (sets of page table) usage. In Xen, each context bank is mapped to one Xen domain. So, all devices being pass throughed to the same Xen domain share the same context bank.

4. IPMMU device tracking. In Xen, all IOMMU devices are managed by single driver instance. So, driver uses global list to keep track of registered IPMMU devices.

----------
Series was tested on Gen3 H3 ES3.0 based boards using current staging (248faa6 xen/arm: add reserved-memory regions to the dom0 memory node) in a system with several DMA masters being assigned to different guest domains. Guest domain reboot, destroy/create are functional.

Please note, the series depends on Julien's patch (on review):
https://lists.xenproject.org/archives/html/xen-devel/2019-08/msg01924.html

You can find the whole series (+ Julien's patch) here:
repo: https://github.com/otyshchenko1/xen.git branch: ipmmu_upstream3

You can find previous discussions here:
[V1] https://lists.xenproject.org/archives/html/xen-devel/2019-06/msg01755.html
[V2] https://lists.xenproject.org/archives/html/xen-devel/2019-08/msg00253.html

Oleksandr Tyshchenko (8):
iommu/arm: Add iommu_helpers.c file to keep common for IOMMUs stuff iommu/arm: Add ability to handle deferred probing request xen/common: Introduce _xrealloc function
xen/common: Introduce xrealloc_flex_struct() helper macros iommu/arm: Add lightweight iommu_fwspec support
iommu: Add of_xlate callback
iommu/arm: Introduce iommu_add_dt_device API
iommu/arm: Add Renesas IPMMU-VMSA support

 xen/arch/arm/domain_build.c                 |   11 +
 xen/arch/arm/platforms/Kconfig              |    1 +
 xen/common/xmalloc_tlsf.c                   |   45 +
 xen/drivers/passthrough/Kconfig             |   13 +
 xen/drivers/passthrough/arm/Makefile        |    3 +-
 xen/drivers/passthrough/arm/iommu.c         |  114 ++-
 xen/drivers/passthrough/arm/iommu_fwspec.c  |   94 ++
 xen/drivers/passthrough/arm/iommu_helpers.c |   84 ++
 xen/drivers/passthrough/arm/ipmmu-vmsa.c    | 1359 +++++++++++++++++++++++++++
 xen/drivers/passthrough/arm/smmu.c          |   48 +-
 xen/include/asm-arm/device.h                |    7 +-
 xen/include/asm-arm/iommu.h                 |   18 +
 xen/include/asm-arm/iommu_fwspec.h          |   68 ++
 xen/include/xen/device_tree.h               |    7 +
 xen/include/xen/iommu.h                     |   10 +
 xen/include/xen/xmalloc.h                   |   13 +
 16 files changed, 1843 insertions(+), 52 deletions(-)
 create mode 100644 xen/drivers/passthrough/arm/iommu_fwspec.c
 create mode 100644 xen/drivers/passthrough/arm/iommu_helpers.c
 create mode 100644 xen/drivers/passthrough/arm/ipmmu-vmsa.c
 create mode 100644 xen/include/asm-arm/iommu_fwspec.h
Signature · 7 lines
2.7.4


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#1 lore raw

PATCH v3 1/8 iommu/arm: Add iommu_helpers.c file to keep common for IOMMUs stuff

From: Oleksandr Tyshchenko <[email protected]>

Introduce a separate file to keep various helpers which could be used by more than one IOMMU driver in order not to duplicate code.

The first candidates to be moved to the new file are SMMU driver's "map_page/unmap_page" callbacks. There callbacks neither contain any SMMU specific info nor perform any SMMU specific actions and are going to be the same across all IOMMU drivers which H/W IP shares P2M with the CPU like SMMU does.

So, move callbacks to iommu_helpers.c for the upcoming IPMMU driver to be able to re-use them.

---
Changes V2 -> V3:
- put headers in alphabetical order
- retained Copyright from the SMMU code

---
 xen/drivers/passthrough/arm/Makefile        |  2 +-
 xen/drivers/passthrough/arm/iommu_helpers.c | 84 +++++++++++++++++++++++++++++
 xen/drivers/passthrough/arm/smmu.c          | 48 +----------------
 xen/include/asm-arm/iommu.h                 |  7 +++
 4 files changed, 94 insertions(+), 47 deletions(-)
 create mode 100644 xen/drivers/passthrough/arm/iommu_helpers.c
xen/drivers/passthrough/arm/Makefile +1 -1
index b3efcfd..4abb87a 100644
@@ -1,2 +1,2 @@
-obj-y += iommu.o
+obj-y += iommu.o iommu_helpers.o
 obj-$(CONFIG_ARM_SMMU) += smmu.o
xen/drivers/passthrough/arm/iommu_helpers.c new file +84 -0

84 lines not on this page · raw message

xen/drivers/passthrough/arm/smmu.c +2 -46

60 lines not on this page · raw message

xen/include/asm-arm/iommu.h +7 -0
index 904c9ae..20d865e 100644
@@ -26,6 +26,13 @@ struct arch_iommu
 const struct iommu_ops *iommu_get_ops(void);
 void iommu_set_ops(const struct iommu_ops *ops);
 
+/* mapping helpers */
+int __must_check arm_iommu_map_page(struct domain *d, dfn_t dfn, mfn_t mfn,
+                                    unsigned int flags,
+                                    unsigned int *flush_flags);
+int __must_check arm_iommu_unmap_page(struct domain *d, dfn_t dfn,
+                                      unsigned int *flush_flags);
+
 #endif /* __ARCH_ARM_IOMMU_H__ */
 
 /*
Signature · 7 lines
2.7.4


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Acked-by on v3 1/8
#2 lore raw

Re: [Xen-devel] [PATCH V3 1/8] iommu/arm: Add iommu_helpers.c file to keep common for IOMMUs stuff

Hi Oleksandr,

Quoted 7 lines · Oleksandr Tyshchenko

From: Oleksandr Tyshchenko <[email protected]>

Introduce a separate file to keep various helpers which could be used by more than one IOMMU driver in order not to duplicate code.

The first candidates to be moved to the new file are SMMU driver's "map_page/unmap_page" callbacks. There callbacks neither contain any

s/There/These/ I think.

Quoted 9 lines

SMMU specific info nor perform any SMMU specific actions and are going to be the same across all IOMMU drivers which H/W IP shares P2M with the CPU like SMMU does.

So, move callbacks to iommu_helpers.c for the upcoming IPMMU driver to be able to re-use them.

[...]

xen/include/asm-arm/iommu.h struct arch_iommu · line 29

Quoted 10 lines

static __init const struct arm_smmu_device *find_smmu(const struct device *dev)

xen/include/asm-arm/iommu.h +1 -0
index 904c9ae..20d865e 100644
@@ -26,6 +26,13 @@ struct arch_iommu
  const struct iommu_ops *iommu_get_ops(void);
  void iommu_set_ops(const struct iommu_ops *ops);
 
+/* mapping helpers */

I would specify this only works when the PT is shared between the IOMMU and CPU.

With the two requests addressed:

xen/include/asm-arm/iommu.h struct arch_iommu · line 38

Quoted 10 lines
xen/include/asm-arm/iommu.h +6 -0
+int __must_check arm_iommu_map_page(struct domain *d, dfn_t dfn, mfn_t mfn,
+                                    unsigned int flags,
+                                    unsigned int *flush_flags);
+int __must_check arm_iommu_unmap_page(struct domain *d, dfn_t dfn,
+                                      unsigned int *flush_flags);
+
  #endif /* __ARCH_ARM_IOMMU_H__ */
 
  /*

Cheers,

Signature · 6 lines
Julien Grall

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
author
#3 lore raw

Re: [Xen-devel] [PATCH V3 1/8] iommu/arm: Add iommu_helpers.c file to keep common for IOMMUs stuff

Quoted 1 line · Julien Grall

Hi Oleksandr,

Hi, Julien

Quoted 11 lines · Oleksandr Tyshchenko
Quoted 7 lines

From: Oleksandr Tyshchenko <[email protected]>

Introduce a separate file to keep various helpers which could be used by more than one IOMMU driver in order not to duplicate code.

The first candidates to be moved to the new file are SMMU driver's "map_page/unmap_page" callbacks. There callbacks neither contain any

s/There/These/ I think.

ok

Quoted 12 lines
Quoted 10 lines

  static __init const struct arm_smmu_device *find_smmu(const struct device *dev)

xen/include/asm-arm/iommu.h +0 -0
index 904c9ae..20d865e 100644
@@ -26,6 +26,13 @@ struct arch_iommu
   const struct iommu_ops *iommu_get_ops(void);
   void iommu_set_ops(const struct iommu_ops *ops);

  +/* mapping helpers */

I would specify this only works when the PT is shared between the IOMMU and CPU.

ok

Quoted 5 lines

With the two requests addressed:

Thank you

Signature · 9 lines
Regards,

Oleksandr Tyshchenko


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#4 lore raw

PATCH v3 2/8 iommu/arm: Add ability to handle deferred probing request

In reply to Oleksandr Tyshchenko

From: Oleksandr Tyshchenko <[email protected]>

This patch adds minimal required support to General IOMMU framework to be able to handle a case when IOMMU driver requesting deferred probing for a device.

In order not to pull Linux's error code (-EPROBE_DEFER) to Xen we have chosen -EAGAIN to be used for indicating that device probing is deferred.

This is needed for the upcoming IPMMU driver which may request deferred probing depending on what device will be probed the first (there is some dependency between these devices, Root device must be registered before Cache devices. If not the case, driver will deny further Cache device probes until Root device is registered). As we can't guarantee a fixed pre-defined order for the device nodes in DT, we need to be ready for the situation where devices being probed in "any" order.

While here, order the headers alphabetically.

---
Changes V2 -> V3:
- removed deferred_probe field from struct dt_device_node, re-used domain_list instead
- documented domain_list usage
- added ASSERT to check that np->domain_list is empty before re-using it
- put deferred_probe_list to init section
- used more strict logic regarding processing devices in the deferred list
- added more comments to code
- put headers in alphabetical order

---
 xen/drivers/passthrough/arm/iommu.c | 59 ++++++++++++++++++++++++++++++++++---
 xen/include/asm-arm/device.h        |  6 +++-
 xen/include/xen/device_tree.h       |  7 +++++
 3 files changed, 67 insertions(+), 5 deletions(-)
xen/drivers/passthrough/arm/iommu.c +55 -4
index f219de9..72a30e0 100644
@@ -15,11 +15,20 @@
  * GNU General Public License for more details.
  */
 
-#include <xen/lib.h>
-#include <xen/iommu.h>
 #include <xen/device_tree.h>
+#include <xen/iommu.h>
+#include <xen/lib.h>
+
 #include <asm/device.h>
 
+/*
+ * Deferred probe list is used to keep track of devices for which driver
+ * requested deferred probing (returned -EAGAIN).
+ *
+ * We re-use device's domain_list to link the device in the deferred list.
+ */
+static __initdata LIST_HEAD(deferred_probe_list);
+
 static const struct iommu_ops *iommu_ops;
 
 const struct iommu_ops *iommu_get_ops(void)
@@ -42,7 +51,7 @@ void __init iommu_set_ops(const struct iommu_ops *ops)
 
 int __init iommu_hardware_setup(void)
 {
-    struct dt_device_node *np;
+    struct dt_device_node *np, *tmp;
     int rc;
     unsigned int num_iommus = 0;
 
@@ -51,6 +60,17 @@ int __init iommu_hardware_setup(void)
         rc = device_init(np, DEVICE_IOMMU, NULL);
         if ( !rc )
             num_iommus++;
+        else if ( rc == -EAGAIN )
+        {
+            /* We expect nobody uses domain_list at such early stage. */
+            ASSERT(list_empty(&np->domain_list));
+
+            /*
+             * Driver requested deferred probing, so add this device to
+             * the deferred list for further processing.
+             */
+            list_add(&np->domain_list, &deferred_probe_list);
+        }
         /*
          * Ignore the following error codes:
          *   - EBADF: Indicate the current not is not an IOMMU
@@ -61,7 +81,38 @@ int __init iommu_hardware_setup(void)
             return rc;
     }
 
-    return ( num_iommus > 0 ) ? 0 : -ENODEV;
+    /* Return immediately if there are no initialized devices. */
+    if ( !num_iommus )
+        return ( list_empty(&deferred_probe_list) ) ? -ENODEV : -EAGAIN;
+
+    rc = 0;
+
+    /*
+     * Process devices in the deferred list if it is not empty.
+     * Check that at least one device is initialized at each loop, otherwise
+     * we may get an infinite loop. Also stop processing if we got an error
+     * other than -EAGAIN.
+     */
+    while ( !list_empty(&deferred_probe_list) && num_iommus )
+    {
+        num_iommus = 0;
+
+        list_for_each_entry_safe ( np, tmp, &deferred_probe_list, domain_list )
+        {
+            rc = device_init(np, DEVICE_IOMMU, NULL);
+            if ( !rc )
+            {
+                num_iommus++;
+
+                /* Remove initialized device from the deferred list. */
+                list_del_init(&np->domain_list);
+            }
+            else if ( rc != -EAGAIN )
+                return rc;
+        }
+    }
+
+    return rc;
 }
 
 void __hwdom_init arch_iommu_check_autotranslated_hwdom(struct domain *d)
xen/include/asm-arm/device.h +5 -1
index 63a0f36..ee1c3bc 100644
@@ -44,7 +44,11 @@ struct device_desc {
     enum device_class class;
     /* List of devices supported by this driver */
     const struct dt_device_match *dt_match;
-    /* Device initialization */
+    /*
+     * Device initialization.
+     *
+     * -EAGAIN is used to indicate that device probing is deferred.
+     */
     int (*init)(struct dt_device_node *dev, const void *data);
 };
 
xen/include/xen/device_tree.h +7 -0
index 9a7a8f2..3702e9b 100644
@@ -92,6 +92,13 @@ struct dt_device_node {
 
     /* IOMMU specific fields */
     bool is_protected;
+    /*
+     * The main purpose of this list node is to link the structure in the list
+     * of devices assigned to domain.
+     *
+     * Boot code (iommu_hardware_setup) re-uses this list to link the structure
+     * in the list of devices for which driver requested deferred probing.
+     */
     struct list_head domain_list;
 
     struct device dev;
Signature · 7 lines
2.7.4


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#5 lore raw

Re: [Xen-devel] [PATCH V3 2/8] iommu/arm: Add ability to handle deferred probing request

Hi Oleksandr,

The code looks code, few comments below.

Quoted 19 lines · Oleksandr Tyshchenko

From: Oleksandr Tyshchenko <[email protected]>

This patch adds minimal required support to General IOMMU framework to be able to handle a case when IOMMU driver requesting deferred probing for a device.

In order not to pull Linux's error code (-EPROBE_DEFER) to Xen we have chosen -EAGAIN to be used for indicating that device probing is deferred.

This is needed for the upcoming IPMMU driver which may request deferred probing depending on what device will be probed the first (there is some dependency between these devices, Root device must be registered before Cache devices. If not the case, driver will deny further Cache device probes until Root device is registered). As we can't guarantee a fixed pre-defined order for the device nodes in DT, we need to be ready for the situation where devices being probed in "any" order.

Quoted 1 line

While here, order the headers alphabetically.

It is common to clean code you modify in the same patch if they are not complex, but this is not the case here... Indeed, the headers are not touched. So I would prefer this to be in a separate patch unless it breaks the compilation without it.

Quoted 59 lines

---
Changes V2 -> V3:
- removed deferred_probe field from struct dt_device_node, re-used domain_list instead
- documented domain_list usage
- added ASSERT to check that np->domain_list is empty before re-using it
- put deferred_probe_list to init section
- used more strict logic regarding processing devices in the deferred list
- added more comments to code
- put headers in alphabetical order

---
  xen/drivers/passthrough/arm/iommu.c | 59 ++++++++++++++++++++++++++++++++++---
  xen/include/asm-arm/device.h        |  6 +++-
  xen/include/xen/device_tree.h       |  7 +++++
  3 files changed, 67 insertions(+), 5 deletions(-)
xen/drivers/passthrough/arm/iommu.c +12 -3
index f219de9..72a30e0 100644
@@ -15,11 +15,20 @@
   * GNU General Public License for more details.
   */
 
-#include <xen/lib.h>
-#include <xen/iommu.h>
  #include <xen/device_tree.h>
+#include <xen/iommu.h>
+#include <xen/lib.h>
+
  #include <asm/device.h>
 
+/*
+ * Deferred probe list is used to keep track of devices for which driver
+ * requested deferred probing (returned -EAGAIN).
+ *
+ * We re-use device's domain_list to link the device in the deferred list.
+ */
+static __initdata LIST_HEAD(deferred_probe_list);
+
  static const struct iommu_ops *iommu_ops;
 
  const struct iommu_ops *iommu_get_ops(void)
@@ -42,7 +51,7 @@ void __init iommu_set_ops(const struct iommu_ops *ops)
 
  int __init iommu_hardware_setup(void)
  {
-    struct dt_device_node *np;
+    struct dt_device_node *np, *tmp;
      int rc;
      unsigned int num_iommus = 0;
 

xen/drivers/passthrough/arm/iommu.c int __init iommu_hardware_setup(void) · line 65

Quoted 6 lines
xen/drivers/passthrough/arm/iommu.c +3 -0
@@ -51,6 +60,17 @@ int __init iommu_hardware_setup(void)
          rc = device_init(np, DEVICE_IOMMU, NULL);
          if ( !rc )
              num_iommus++;
+        else if ( rc == -EAGAIN )
+        {
+            /* We expect nobody uses domain_list at such early stage. */

AFAICT, this comment is only an English version of the next line. It would be best if you explain why domain_list is re-used here.

Quoted 12 lines
xen/drivers/passthrough/arm/iommu.c +8 -0
+            ASSERT(list_empty(&np->domain_list));
+
+            /*
+             * Driver requested deferred probing, so add this device to
+             * the deferred list for further processing.
+             */
+            list_add(&np->domain_list, &deferred_probe_list);
+        }
          /*
           * Ignore the following error codes:
           *   - EBADF: Indicate the current not is not an IOMMU

xen/drivers/passthrough/arm/iommu.c int __init iommu_hardware_setup(void) · line 86

Quoted 7 lines
xen/drivers/passthrough/arm/iommu.c +3 -1
@@ -61,7 +81,38 @@ int __init iommu_hardware_setup(void)
              return rc;
      }
 
-    return ( num_iommus > 0 ) ? 0 : -ENODEV;
+    /* Return immediately if there are no initialized devices. */
+    if ( !num_iommus )
+        return ( list_empty(&deferred_probe_list) ) ? -ENODEV : -EAGAIN;

NIT: Do you need the outer ()?

Quoted 58 lines
xen/drivers/passthrough/arm/iommu.c +29 -0
+
+    rc = 0;
+
+    /*
+     * Process devices in the deferred list if it is not empty.
+     * Check that at least one device is initialized at each loop, otherwise
+     * we may get an infinite loop. Also stop processing if we got an error
+     * other than -EAGAIN.
+     */
+    while ( !list_empty(&deferred_probe_list) && num_iommus )
+    {
+        num_iommus = 0;
+
+        list_for_each_entry_safe ( np, tmp, &deferred_probe_list, domain_list )
+        {
+            rc = device_init(np, DEVICE_IOMMU, NULL);
+            if ( !rc )
+            {
+                num_iommus++;
+
+                /* Remove initialized device from the deferred list. */
+                list_del_init(&np->domain_list);
+            }
+            else if ( rc != -EAGAIN )
+                return rc;
+        }
+    }
+
+    return rc;
  }
 
  void __hwdom_init arch_iommu_check_autotranslated_hwdom(struct domain *d)
xen/include/asm-arm/device.h +5 -1
index 63a0f36..ee1c3bc 100644
@@ -44,7 +44,11 @@ struct device_desc {
      enum device_class class;
      /* List of devices supported by this driver */
      const struct dt_device_match *dt_match;
-    /* Device initialization */
+    /*
+     * Device initialization.
+     *
+     * -EAGAIN is used to indicate that device probing is deferred.
+     */
      int (*init)(struct dt_device_node *dev, const void *data);
  };
 
xen/include/xen/device_tree.h +0 -0
index 9a7a8f2..3702e9b 100644
@@ -92,6 +92,13 @@ struct dt_device_node {
 
      /* IOMMU specific fields */
      bool is_protected;

xen/include/xen/device_tree.h struct dt_device_node { · line 104

Quoted 10 lines
xen/include/xen/device_tree.h +7 -0
+    /*
+     * The main purpose of this list node is to link the structure in the list
+     * of devices assigned to domain.
+     *
+     * Boot code (iommu_hardware_setup) re-uses this list to link the structure
+     * in the list of devices for which driver requested deferred probing.
+     */
      struct list_head domain_list;
 
      struct device dev;

Cheers,

Signature · 6 lines
Julien Grall

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
author
#6 lore raw

Re: [Xen-devel] [PATCH V3 2/8] iommu/arm: Add ability to handle deferred probing request

Quoted 1 line · Julien Grall

Hi Oleksandr,

Hi, Julien

Quoted 23 lines

The code looks code, few comments below.

Quoted 19 lines · Oleksandr Tyshchenko

From: Oleksandr Tyshchenko <[email protected]>

This patch adds minimal required support to General IOMMU framework to be able to handle a case when IOMMU driver requesting deferred probing for a device.

In order not to pull Linux's error code (-EPROBE_DEFER) to Xen we have chosen -EAGAIN to be used for indicating that device probing is deferred.

This is needed for the upcoming IPMMU driver which may request deferred probing depending on what device will be probed the first (there is some dependency between these devices, Root device must be registered before Cache devices. If not the case, driver will deny further Cache device probes until Root device is registered). As we can't guarantee a fixed pre-defined order for the device nodes in DT, we need to be ready for the situation where devices being probed in "any" order.

Quoted 3 lines
Quoted 1 line

While here, order the headers alphabetically.

It is common to clean code you modify in the same patch if they are not complex, but this is not the case here... Indeed, the headers are not touched. So I would prefer this to be in a separate patch unless it breaks the compilation without it.

Well, will make this change in a separate patch.

Quoted 60 lines
Quoted 60 lines

---
Changes V2 -> V3:
     - removed deferred_probe field from struct dt_device_node, re-used domain_list instead
     - documented domain_list usage
     - added ASSERT to check that np->domain_list is empty before re-using it
     - put deferred_probe_list to init section
     - used more strict logic regarding processing devices in the deferred list
     - added more comments to code
     - put headers in alphabetical order
---
  xen/drivers/passthrough/arm/iommu.c | 59
++++++++++++++++++++++++++++++++++---
  xen/include/asm-arm/device.h        |  6 +++- xen/include/xen/device_tree.h       |  7 +++++

  3 files changed, 67 insertions(+), 5 deletions(-)
xen/drivers/passthrough/arm/iommu.c +0 -0
index f219de9..72a30e0 100644
@@ -15,11 +15,20 @@
    * GNU General Public License for more details.
    */

  -#include <xen/lib.h>
-#include <xen/iommu.h>
  #include <xen/device_tree.h>

xen/drivers/passthrough/arm/iommu.c +3 -0
+#include <xen/iommu.h>
+#include <xen/lib.h>
+

  #include <asm/device.h>
  +/*
+ * Deferred probe list is used to keep track of devices for which driver
+ * requested deferred probing (returned -EAGAIN).
+ *
+ * We re-use device's domain_list to link the device in the deferred list.
+ */

xen/drivers/passthrough/arm/iommu.c +2 -0
+static __initdata LIST_HEAD(deferred_probe_list);
+

  static const struct iommu_ops *iommu_ops;
    const struct iommu_ops *iommu_get_ops(void)

xen/drivers/passthrough/arm/iommu.c +1 -1
@@ -42,7 +51,7 @@ void __init iommu_set_ops(const struct iommu_ops *ops)
     int __init iommu_hardware_setup(void)
   {
-    struct dt_device_node *np;
+    struct dt_device_node *np, *tmp;
       int rc;
       unsigned int num_iommus = 0;
   @@ -51,6 +60,17 @@ int __init iommu_hardware_setup(void)
           rc = device_init(np, DEVICE_IOMMU, NULL);
Quoted 8 lines
Quoted 6 lines

          if ( !rc )
              num_iommus++;
+        else if ( rc == -EAGAIN )
+        {
+            /* We expect nobody uses domain_list at such early stage. */

AFAICT, this comment is only an English version of the next line. It would be best if you explain why domain_list is re-used here.

Will do.

Quoted 15 lines
Quoted 15 lines

+ ASSERT(list_empty(&np->domain_list));
+
+            /*
+             * Driver requested deferred probing, so add this device to +             * the deferred list for further processing.
+             */
+            list_add(&np->domain_list, &deferred_probe_list); +        }
          /*
           * Ignore the following error codes:
           *   - EBADF: Indicate the current not is not an IOMMU

xen/drivers/passthrough/arm/iommu.c +0 -0
@@ -61,7 +81,38 @@ int __init iommu_hardware_setup(void)
               return rc;
Quoted 8 lines

xen/drivers/passthrough/arm/iommu.c int __init iommu_hardware_setup(void)

Quoted 6 lines
xen/drivers/passthrough/arm/iommu.c +3 -1
       }
   -    return ( num_iommus > 0 ) ? 0 : -ENODEV;
+    /* Return immediately if there are no initialized devices. */
+    if ( !num_iommus )
+        return ( list_empty(&deferred_probe_list) ) ? -ENODEV : 
-EAGAIN;

NIT: Do you need the outer ()?

No, I don't.

Signature · 9 lines
Regards,

Oleksandr Tyshchenko


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#7 lore raw

PATCH v3 3/8 xen/common: Introduce _xrealloc function

In reply to Oleksandr Tyshchenko

From: Oleksandr Tyshchenko <[email protected]>

This patch introduces type-unsafe function which besides re-allocation handles the following corner cases:
1. if requested size is zero, it will behave like xfree
2. if incoming pointer is not valid (NULL or ZERO_BLOCK_PTR), it will behave like xmalloc

If both pointer and size are valid the function will re-allocate and copy only if requested size doesn't fit in already allocated space.

Subsequent patch will add type-safe helper macros.

CC Andrew Cooper <[email protected]>
CC George Dunlap <[email protected]>
CC Ian Jackson <[email protected]>
CC Jan Beulich <[email protected]>
CC Julien Grall <[email protected]>
CC Konrad Rzeszutek Wilk <[email protected]>
CC Stefano Stabellini <[email protected]>
CC Tim Deegan <[email protected]>
CC Wei Liu <[email protected]>

---
Changes since RFC:
- behave like xmalloc if incoming pointer is ZERO_BLOCK_PTR or NULL
- return ZERO_BLOCK_PTR after xfree if requested size is zero
- add patch description
- use allocator internals to recognize current size of the incoming pointer
- do not re-allocate and copy if requested size fits in already allocated space

...

Original patch was initially posted by Sameer Goel:
https://lists.xen.org/archives/html/xen-devel/2017-06/msg00858.html

This could be considered as another attempt to add it:
https://www.mail-archive.com/[email protected]/msg21335.html

[As it was previously discussed with Julien in IRC]

The reason for this patch to be an RFC is that patch itself is not completely correct and I don't fully understand what/how should be done for this patch to be accepted. Or whether community even wants this to go in. So, to avoid bike shedding, the first target is to collect feedback.

For everyone who wants more details why this is needed and where used, please see next patch of this thread:
"iommu/arm: Add lightweight iommu_fwspec support"

In a nutshell, the upcoming "iommu_fwspec" support on ARM is going to use xrealloc to expand an array for device IDs. We really want to have "iommu_fwspec" support which will give us a generic abstract way to add new device to the IOMMU based on the generic IOMMU DT binding.

---
 xen/common/xmalloc_tlsf.c | 45 +++++++++++++++++++++++++++++++++++++++++++++
 xen/include/xen/xmalloc.h |  1 +
 2 files changed, 46 insertions(+)
xen/common/xmalloc_tlsf.c +45 -0
index e98ad65..eecae2e 100644
@@ -598,6 +598,51 @@ void *_xzalloc(unsigned long size, unsigned long align)
     return p ? memset(p, 0, size) : p;
 }
 
+void *_xrealloc(void *ptr, unsigned long size, unsigned long align)
+{
+    unsigned long curr_size, tmp_size;
+    void *p;
+
+    if ( !size )
+    {
+        xfree(ptr);
+        return ZERO_BLOCK_PTR;
+    }
+
+    if ( ptr == NULL || ptr == ZERO_BLOCK_PTR )
+        return _xmalloc(size, align);
+
+    if ( !((unsigned long)ptr & (PAGE_SIZE - 1)) )
+        curr_size = PFN_ORDER(virt_to_page(ptr)) << PAGE_SHIFT;
+    else
+    {
+        struct bhdr *b;
+        b = (struct bhdr *)((char *)ptr - BHDR_OVERHEAD);
+        curr_size = b->size & BLOCK_SIZE_MASK;
+    }
+
+    ASSERT((align & (align - 1)) == 0);
+    if ( align < MEM_ALIGN )
+        align = MEM_ALIGN;
+    tmp_size = size + align - MEM_ALIGN;
+
+    if ( tmp_size < PAGE_SIZE )
+        tmp_size = ( tmp_size < MIN_BLOCK_SIZE ) ? MIN_BLOCK_SIZE :
+            ROUNDUP_SIZE(tmp_size);
+
+    if ( tmp_size <= curr_size ) /* fits in current block */
+        return ptr;
+
+    p = _xmalloc(size, align);
+    if ( p )
+    {
+        memcpy(p, ptr, min(curr_size, size));
+        xfree(ptr);
+    }
+
+    return p;
+}
+
 void xfree(void *p)
 {
     struct bhdr *b;
xen/include/xen/xmalloc.h +1 -0
index f075d2d..831152f 100644
@@ -51,6 +51,7 @@ extern void xfree(void *);
 /* Underlying functions */
 extern void *_xmalloc(unsigned long size, unsigned long align);
 extern void *_xzalloc(unsigned long size, unsigned long align);
+extern void *_xrealloc(void *ptr, unsigned long size, unsigned long align);
 
 static inline void *_xmalloc_array(
     unsigned long size, unsigned long align, unsigned long num)
Signature · 7 lines
2.7.4


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#8 lore raw

Re: [Xen-devel] [PATCH V3 3/8] xen/common: Introduce _xrealloc function

Quoted 95 lines

-----Original Message-----
From: Xen-devel <[email protected]> On Behalf Of Oleksandr Tyshchenko
Sent: 20 August 2019 19:10
To: [email protected]
Cc: [email protected]; Wei Liu <[email protected]>; Konrad Rzeszutek Wilk <[email protected]>; George Dunlap <[email protected]>; Andrew Cooper <[email protected]>; Ian Jackson <[email protected]>; Tim (Xen.org) <[email protected]>; Oleksandr Tyshchenko <[email protected]>; [email protected]; Jan Beulich <[email protected]>; [email protected]
Subject: [Xen-devel] [PATCH V3 3/8] xen/common: Introduce _xrealloc function

From: Oleksandr Tyshchenko <[email protected]>

This patch introduces type-unsafe function which besides re-allocation handles the following corner cases:
1. if requested size is zero, it will behave like xfree
2. if incoming pointer is not valid (NULL or ZERO_BLOCK_PTR), it will behave like xmalloc

If both pointer and size are valid the function will re-allocate and copy only if requested size doesn't fit in already allocated space.

Subsequent patch will add type-safe helper macros.

CC Andrew Cooper <[email protected]>
CC George Dunlap <[email protected]>
CC Ian Jackson <[email protected]>
CC Jan Beulich <[email protected]>
CC Julien Grall <[email protected]>
CC Konrad Rzeszutek Wilk <[email protected]>
CC Stefano Stabellini <[email protected]>
CC Tim Deegan <[email protected]>
CC Wei Liu <[email protected]>

---
Changes since RFC:
- behave like xmalloc if incoming pointer is ZERO_BLOCK_PTR or NULL
- return ZERO_BLOCK_PTR after xfree if requested size is zero
- add patch description
- use allocator internals to recognize current size of the incoming pointer
- do not re-allocate and copy if requested size fits in already allocated space

...

Original patch was initially posted by Sameer Goel:
https://lists.xen.org/archives/html/xen-devel/2017-06/msg00858.html

This could be considered as another attempt to add it:
https://www.mail-archive.com/[email protected]/msg21335.html

[As it was previously discussed with Julien in IRC]

The reason for this patch to be an RFC is that patch itself is not completely correct and I don't fully understand what/how should be done for this patch to be accepted. Or whether community even wants this to go in. So, to avoid bike shedding, the first target is to collect feedback.

For everyone who wants more details why this is needed and where used, please see next patch of this thread:
"iommu/arm: Add lightweight iommu_fwspec support"

In a nutshell, the upcoming "iommu_fwspec" support on ARM is going to use xrealloc to expand an array for device IDs. We really want to have "iommu_fwspec" support which will give us a generic abstract way to add new device to the IOMMU based on the generic IOMMU DT binding.

---
 xen/common/xmalloc_tlsf.c | 45 +++++++++++++++++++++++++++++++++++++++++++++
 xen/include/xen/xmalloc.h |  1 +
 2 files changed, 46 insertions(+)
xen/common/xmalloc_tlsf.c +12 -0
index e98ad65..eecae2e 100644
@@ -598,6 +598,51 @@ void *_xzalloc(unsigned long size, unsigned long align)
     return p ? memset(p, 0, size) : p;
 }
 
+void *_xrealloc(void *ptr, unsigned long size, unsigned long align)
+{
+    unsigned long curr_size, tmp_size;
+    void *p;
+
+    if ( !size )
+    {
+        xfree(ptr);
+        return ZERO_BLOCK_PTR;
+    }
+
+    if ( ptr == NULL || ptr == ZERO_BLOCK_PTR )

xen/common/xmalloc_tlsf.c void *_xzalloc(unsigned long size, unsigned long align) · line 622

Quoted 10 lines
xen/common/xmalloc_tlsf.c +10 -0
+        return _xmalloc(size, align);
+
+    if ( !((unsigned long)ptr & (PAGE_SIZE - 1)) )
+        curr_size = PFN_ORDER(virt_to_page(ptr)) << PAGE_SHIFT;
+    else
+    {
+        struct bhdr *b;
+        b = (struct bhdr *)((char *)ptr - BHDR_OVERHEAD);
+        curr_size = b->size & BLOCK_SIZE_MASK;
+    }

That seconds clause is not going to give you the block size if the previous allocation had alignment padding. You'll need to check the FREE_BLOCK bit to tell whether it's a real block header or the 'fake' alignment header and then maybe walk backwards onto the real header. See the code in xfree(). You should also check whether the new requested alignment is compatible with the existing block alignment

Paul

Quoted 42 lines
xen/common/xmalloc_tlsf.c +23 -0
+
+    ASSERT((align & (align - 1)) == 0);
+    if ( align < MEM_ALIGN )
+        align = MEM_ALIGN;
+    tmp_size = size + align - MEM_ALIGN;
+
+    if ( tmp_size < PAGE_SIZE )
+        tmp_size = ( tmp_size < MIN_BLOCK_SIZE ) ? MIN_BLOCK_SIZE :
+            ROUNDUP_SIZE(tmp_size);
+
+    if ( tmp_size <= curr_size ) /* fits in current block */
+        return ptr;
+
+    p = _xmalloc(size, align);
+    if ( p )
+    {
+        memcpy(p, ptr, min(curr_size, size));
+        xfree(ptr);
+    }
+
+    return p;
+}
+
 void xfree(void *p)
 {
     struct bhdr *b;
xen/include/xen/xmalloc.h +1 -1
index f075d2d..831152f 100644
@@ -51,6 +51,7 @@ extern void xfree(void *);
 /* Underlying functions */
 extern void *_xmalloc(unsigned long size, unsigned long align);
 extern void *_xzalloc(unsigned long size, unsigned long align);
+extern void *_xrealloc(void *ptr, unsigned long size, unsigned long align);
 
 static inline void *_xmalloc_array(
     unsigned long size, unsigned long align, unsigned long num)
--

2.7.4

Quoted 4 lines

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

author
#9 lore raw

Re: [Xen-devel] [PATCH V3 3/8] xen/common: Introduce _xrealloc function

On 21.08.19 11:09, Paul Durrant wrote:

Hi, Paul

Quoted 17 lines
Quoted 17 lines
  }
(quoted diff) +15 -0
+void *_xrealloc(void *ptr, unsigned long size, unsigned long align)
+{
+    unsigned long curr_size, tmp_size;
+    void *p;
+
+    if ( !size )
+    {
+        xfree(ptr);
+        return ZERO_BLOCK_PTR;
+    }
+
+    if ( ptr == NULL || ptr == ZERO_BLOCK_PTR )
+        return _xmalloc(size, align);
+
+    if ( !((unsigned long)ptr & (PAGE_SIZE - 1)) )
Quoted 8 lines

Review

Quoted 7 lines
(quoted diff) +7 -0
+        curr_size = PFN_ORDER(virt_to_page(ptr)) << PAGE_SHIFT;
+    else
+    {
+        struct bhdr *b;
+        b = (struct bhdr *)((char *)ptr - BHDR_OVERHEAD);
+        curr_size = b->size & BLOCK_SIZE_MASK;
+    }

That seconds clause is not going to give you the block size if the previous allocation had alignment padding. You'll need to check the FREE_BLOCK bit to tell whether it's a real block header or the 'fake' alignment header and then maybe walk backwards onto the real header. See the code in xfree().

Indeed. Thank you for the pointer.

Quoted 1 line

You should also check whether the new requested alignment is compatible with the existing block alignment

I am wondering:

In case when we use only type-safe construction for the basic allocation (xmalloc) and type-safe construction for the re-allocation (xrealloc_flex_struct) over the same pointer of the correct type, will we get a possible alignment incompatibility?

This is how I see it:

xen/common/xmalloc_tlsf.c +0 -0
index eecae2e..f90f453 100644
@@ -616,8 +616,15 @@ void *_xrealloc(void *ptr, unsigned long size, unsigned long align)

         curr_size = PFN_ORDER(virt_to_page(ptr)) << PAGE_SHIFT; else
     {
-        struct bhdr *b;
-        b = (struct bhdr *)((char *)ptr - BHDR_OVERHEAD);
+        struct bhdr *b = (struct bhdr *)((char *)ptr - BHDR_OVERHEAD); +
+        if ( b->size & FREE_BLOCK )
+        {
+            p = (char *)ptr - (b->size & ~FREE_BLOCK);
+            b = (struct bhdr *)((char *)p - BHDR_OVERHEAD); +            ASSERT(!(b->size & FREE_BLOCK));
+        }
+
         curr_size = b->size & BLOCK_SIZE_MASK;
     }

@@ -630,8 +637,8 @@ void *_xrealloc(void *ptr, unsigned long size, unsigned long align) tmp_size = ( tmp_size < MIN_BLOCK_SIZE ) ? MIN_BLOCK_SIZE :
             ROUNDUP_SIZE(tmp_size);

-    if ( tmp_size <= curr_size ) /* fits in current block */
-        return ptr;
+    if ( tmp_size <= curr_size && ((unsigned long)ptr & (align - 1)) == 0 ) +        return ptr; /* fits in current block */

     p = _xmalloc(size, align);
     if ( p )

(END)

Did I get your point correct?

Signature · 9 lines
Regards,

Oleksandr Tyshchenko


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#10 lore raw

Re: [Xen-devel] [PATCH V3 3/8] xen/common: Introduce _xrealloc function

Quoted 51 lines

-----Original Message-----
From: Oleksandr <[email protected]>
Sent: 21 August 2019 15:36
To: Paul Durrant <[email protected]>; [email protected]
Cc: [email protected]; Wei Liu <[email protected]>; Konrad Rzeszutek Wilk <[email protected]>; George Dunlap <[email protected]>; Andrew Cooper <[email protected]>; Ian Jackson <[email protected]>; Tim (Xen.org) <[email protected]>; Oleksandr Tyshchenko <[email protected]>; [email protected]; Jan Beulich <[email protected]>; [email protected]
Subject: Re: [Xen-devel] [PATCH V3 3/8] xen/common: Introduce _xrealloc function

On 21.08.19 11:09, Paul Durrant wrote:

Hi, Paul

Quoted 17 lines
Quoted 17 lines
  }
(quoted diff) +15 -0
+void *_xrealloc(void *ptr, unsigned long size, unsigned long align)
+{
+    unsigned long curr_size, tmp_size;
+    void *p;
+
+    if ( !size )
+    {
+        xfree(ptr);
+        return ZERO_BLOCK_PTR;
+    }
+
+    if ( ptr == NULL || ptr == ZERO_BLOCK_PTR )
+        return _xmalloc(size, align);
+
+    if ( !((unsigned long)ptr & (PAGE_SIZE - 1)) )
Quoted 8 lines

Review

Quoted 7 lines
(quoted diff) +7 -0
+        curr_size = PFN_ORDER(virt_to_page(ptr)) << PAGE_SHIFT;
+    else
+    {
+        struct bhdr *b;
+        b = (struct bhdr *)((char *)ptr - BHDR_OVERHEAD);
+        curr_size = b->size & BLOCK_SIZE_MASK;
+    }

That seconds clause is not going to give you the block size if the previous allocation had alignment

padding. You'll need to check the FREE_BLOCK bit to tell whether it's a real block header or the 'fake' alignment header and then maybe walk backwards onto the real header. See the code in xfree().

Indeed. Thank you for the pointer.

Quoted 1 line

You should also check whether the new requested alignment is compatible with the existing block

alignment

Quoted 6 lines

I am wondering:

In case when we use only type-safe construction for the basic allocation (xmalloc) and type-safe construction for the re-allocation (xrealloc_flex_struct) over the same pointer of the correct type, will we get a possible alignment incompatibility?

You shouldn't but you're trusting that no-one will want to call the function directly with a higher alignment.

Quoted 15 lines

This is how I see it:

xen/common/xmalloc_tlsf.c +0 -2
index eecae2e..f90f453 100644
@@ -616,8 +616,15 @@ void *_xrealloc(void *ptr, unsigned long size,
 unsigned long align)
          curr_size = PFN_ORDER(virt_to_page(ptr)) << PAGE_SHIFT;
      else
      {
-        struct bhdr *b;
-        b = (struct bhdr *)((char *)ptr - BHDR_OVERHEAD);

xen/common/xmalloc_tlsf.c void *_xrealloc(void *ptr, unsigned long size, · line 629

Quoted 10 lines
xen/common/xmalloc_tlsf.c +9 -0
+        struct bhdr *b = (struct bhdr *)((char *)ptr - BHDR_OVERHEAD);
+
+        if ( b->size & FREE_BLOCK )
+        {
+            p = (char *)ptr - (b->size & ~FREE_BLOCK);
+            b = (struct bhdr *)((char *)p - BHDR_OVERHEAD);
+            ASSERT(!(b->size & FREE_BLOCK));
+        }
+
          curr_size = b->size & BLOCK_SIZE_MASK;

Yes, that looks ok for getting the size right...

xen/common/xmalloc_tlsf.c void *_xrealloc(void *ptr, unsigned long size, · line 642

Quoted 11 lines
xen/common/xmalloc_tlsf.c +2 -2
      }
@@ -630,8 +637,8 @@ void *_xrealloc(void *ptr, unsigned long size,
 unsigned long align)
          tmp_size = ( tmp_size < MIN_BLOCK_SIZE ) ? MIN_BLOCK_SIZE :
              ROUNDUP_SIZE(tmp_size);
 
-    if ( tmp_size <= curr_size ) /* fits in current block */
-        return ptr;
+    if ( tmp_size <= curr_size && ((unsigned long)ptr & (align - 1)) == 0 )
+        return ptr; /* fits in current block */

...and that should deal with the alignment too (although you may want to mention the alignment part in the comment too)

Quoted 9 lines
xen/common/xmalloc_tlsf.c +0 -0
      p = _xmalloc(size, align);
      if ( p )

(END)

Did I get your point correct?

Yes, with those changes I think it will look ok.

Paul

Quoted 5 lines

--
Regards,

Oleksandr Tyshchenko

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

author
#11 lore raw

Re: [Xen-devel] [PATCH V3 3/8] xen/common: Introduce _xrealloc function

On 21.08.19 18:47, Paul Durrant wrote:

Hi

Quoted 5 lines
Quoted 5 lines

You should also check whether the new requested alignment is compatible with the existing block alignment

Quoted 7 lines
Quoted 6 lines

I am wondering:

In case when we use only type-safe construction for the basic allocation (xmalloc) and type-safe construction for the re-allocation (xrealloc_flex_struct) over the same pointer of the correct type, will we get a possible alignment incompatibility?

You shouldn't but you're trusting that no-one will want to call the function directly with a higher alignment.

Yes, it is best to be on the safe side.

Quoted 43 lines
Quoted 15 lines

This is how I see it:

xen/common/xmalloc_tlsf.c +0 -2
index eecae2e..f90f453 100644
@@ -616,8 +616,15 @@ void *_xrealloc(void *ptr, unsigned long size,
 unsigned long align)
           curr_size = PFN_ORDER(virt_to_page(ptr)) << PAGE_SHIFT;
       else
       {
-        struct bhdr *b;
-        b = (struct bhdr *)((char *)ptr - BHDR_OVERHEAD);

xen/common/xmalloc_tlsf.c void *_xrealloc(void *ptr, unsigned long size, · line 629

Quoted 10 lines
xen/common/xmalloc_tlsf.c +9 -0
+        struct bhdr *b = (struct bhdr *)((char *)ptr - BHDR_OVERHEAD);
+
+        if ( b->size & FREE_BLOCK )
+        {
+            p = (char *)ptr - (b->size & ~FREE_BLOCK);
+            b = (struct bhdr *)((char *)p - BHDR_OVERHEAD);
+            ASSERT(!(b->size & FREE_BLOCK));
+        }
+
           curr_size = b->size & BLOCK_SIZE_MASK;

Yes, that looks ok for getting the size right...

xen/common/xmalloc_tlsf.c void *_xrealloc(void *ptr, unsigned long size, · line 642

Quoted 11 lines
xen/common/xmalloc_tlsf.c +2 -2
       }
@@ -630,8 +637,8 @@ void *_xrealloc(void *ptr, unsigned long size,
 unsigned long align)
           tmp_size = ( tmp_size < MIN_BLOCK_SIZE ) ? MIN_BLOCK_SIZE :
               ROUNDUP_SIZE(tmp_size);
 
-    if ( tmp_size <= curr_size ) /* fits in current block */
-        return ptr;
+    if ( tmp_size <= curr_size && ((unsigned long)ptr & (align - 1)) == 0 )
+        return ptr; /* fits in current block */

...and that should deal with the alignment too (although you may want to mention the alignment part in the comment too)

Quoted 3 lines
xen/common/xmalloc_tlsf.c +0 -0
       p = _xmalloc(size, align);
       if ( p )
Quoted 6 lines
Quoted 5 lines

(END)

Did I get your point correct?

Yes, with those changes I think it will look ok.

Thank you.

Signature · 9 lines
Regards,

Oleksandr Tyshchenko


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#12 lore raw

PATCH v3 4/8 xen/common: Introduce xrealloc_flex_struct() helper macros

In reply to Oleksandr Tyshchenko

From: Oleksandr Tyshchenko <[email protected]>

This patch introduces type-safe helper macros to re-allocate space for a structure with a flexible array of typed objects.

For example, if we need to re-size an array with a single element:

   struct arrlen
   {
      size_t len;
      int data[1];
   };

We can use the proposed macros in the following way:

   new_ptr = realloc_flex_struct(old_ptr, struct arrlen, data, num_elem);

Subsequent patch will use this macros.

Also, while here, introduce xmalloc_flex_struct() to allocate space for a structure with a flexible array of typed objects.

CC Andrew Cooper <[email protected]>
CC George Dunlap <[email protected]>
CC Ian Jackson <[email protected]>
CC Jan Beulich <[email protected]>
CC Julien Grall <[email protected]>
CC Konrad Rzeszutek Wilk <[email protected]>
CC Stefano Stabellini <[email protected]>
CC Tim Deegan <[email protected]>
CC Wei Liu <[email protected]>
---
 xen/include/xen/xmalloc.h | 12 ++++++++++++
 1 file changed, 12 insertions(+)
xen/include/xen/xmalloc.h +12 -0
index 831152f..2eb88a8 100644
@@ -35,6 +35,18 @@
 #define xzalloc_array(_type, _num) \
     ((_type *)_xzalloc_array(sizeof(_type), __alignof__(_type), _num))
 
+/* Re-allocate space for a structure with a flexible array of typed objects. */
+#define xrealloc_flex_struct(_ptr, _type, _field, _len) ({                   \
+    /* type checking: make sure that incoming pointer is of correct type */  \
+    (void)((typeof(_ptr)) 0 == (_type *) 0);                                 \
+    ((_type *)_xrealloc(_ptr, offsetof(_type, _field[_len]),                 \
+                        __alignof__(_type)));                                \
+})
+
+/* Allocate space for a structure with a flexible array of typed objects. */
+#define xmalloc_flex_struct(_type, _field, _len) \
+    ((_type *)_xmalloc(offsetof(_type, _field[_len]), __alignof__(_type)))
+
 /* Allocate untyped storage. */
 #define xmalloc_bytes(_bytes) _xmalloc(_bytes, SMP_CACHE_BYTES)
 #define xzalloc_bytes(_bytes) _xzalloc(_bytes, SMP_CACHE_BYTES)
Signature · 7 lines
2.7.4


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#13 lore raw

Re: [Xen-devel] [PATCH V3 4/8] xen/common: Introduce xrealloc_flex_struct() helper macros

xen/include/xen/xmalloc.h line 39

Quoted 8 lines · Oleksandr Tyshchenko
xen/include/xen/xmalloc.h +2 -0
@@ -35,6 +35,18 @@
  #define xzalloc_array(_type, _num) \
      ((_type *)_xzalloc_array(sizeof(_type), __alignof__(_type), _num))
 
+/* Re-allocate space for a structure with a flexible array of typed objects. */
+#define xrealloc_flex_struct(_ptr, _type, _field, _len) ({                   \

May I ask that you don't extend the bad use of leading underscores here?

xen/include/xen/xmalloc.h line 40

Quoted 1 line
xen/include/xen/xmalloc.h +1 -0
+    /* type checking: make sure that incoming pointer is of correct type */  \

Comment style (should start with upper case char)

xen/include/xen/xmalloc.h line 41

Quoted 1 line
xen/include/xen/xmalloc.h +1 -0
+    (void)((typeof(_ptr)) 0 == (_type *) 0);                                 \

Stray blanks before 0. And why not simply "(void)((ptr) == (type *)0)"? (You'd need to avoid the double evaluation of ptr, yes.)

xen/include/xen/xmalloc.h line 43

Quoted 2 lines
xen/include/xen/xmalloc.h +2 -0
+    ((_type *)_xrealloc(_ptr, offsetof(_type, _field[_len]),                 \
+                        __alignof__(_type)));                                \

Unnecessary pair of outermost parentheses.

xen/include/xen/xmalloc.h line 48

Quoted 5 lines
xen/include/xen/xmalloc.h +5 -0
+})
+
+/* Allocate space for a structure with a flexible array of typed objects. */
+#define xmalloc_flex_struct(_type, _field, _len) \
+    ((_type *)_xmalloc(offsetof(_type, _field[_len]), __alignof__(_type)))

I think this wants to go ahead of its re-alloc counterpart.

In both cases I'd also like to suggest using "nr" instead of "len", as something called "length" can be mistaken to represent the overall length of the resulting object, rather than the number of array elements.

Jan

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

author
#14 lore raw

Re: [Xen-devel] [PATCH V3 4/8] xen/common: Introduce xrealloc_flex_struct() helper macros

On 27.08.19 16:28, Jan Beulich wrote:

Hi Jan

Quoted 11 lines · Oleksandr Tyshchenko
Quoted 8 lines
xen/include/xen/xmalloc.h +0 -0
@@ -35,6 +35,18 @@
   #define xzalloc_array(_type, _num) \
       ((_type *)_xzalloc_array(sizeof(_type), __alignof__(_type), _num))

  +/* Re-allocate space for a structure with a flexible array of typed objects. */ +#define xrealloc_flex_struct(_ptr, _type, _field, _len)
({                   \

May I ask that you don't extend the bad use of leading underscores here?

Yes, sure.

Quoted 5 lines
Quoted 3 lines

+    /* type checking: make sure that incoming pointer is of correct type */  \

Comment style (should start with upper case char)

ok

Quoted 6 lines
Quoted 3 lines

+    (void)((typeof(_ptr)) 0 == (_type *)
0);                                 \

Stray blanks before 0. And why not simply "(void)((ptr) == (type *)0)"? (You'd need to avoid the double evaluation of ptr, yes.)

ok

Quoted 6 lines
Quoted 4 lines

+    ((_type *)_xrealloc(_ptr, offsetof(_type, _field[_len]),                 \
+ __alignof__(_type)));                                \

Unnecessary pair of outermost parentheses.

ok

Quoted 10 lines
Quoted 8 lines
xen/include/xen/xmalloc.h +3 -0
+})
+
+/* Allocate space for a structure with a flexible array of typed 

objects. */
+#define xmalloc_flex_struct(_type, _field, _len) \
+    ((_type *)_xmalloc(offsetof(_type, _field[_len]), __alignof__(_type)))

I think this wants to go ahead of its re-alloc counterpart.

ok

Quoted 5 lines

In both cases I'd also like to suggest using "nr" instead of "len", as something called "length" can be mistaken to represent the overall length of the resulting object, rather than the number of array elements.

sounds reasonable

Thank you.

xen/include/xen/xmalloc.h +0 -0
index 831152f..3eec10b 100644
@@ -35,6 +35,18 @@

 #define xzalloc_array(_type, _num) \
     ((_type *)_xzalloc_array(sizeof(_type), __alignof__(_type), _num))

+/* Allocate space for a structure with a flexible array of typed objects. */ +#define xmalloc_flex_struct(type, field, nr) \
+    ((type *)_xmalloc(offsetof(type, field[nr]), __alignof__(type))) +
+/* Re-allocate space for a structure with a flexible array of typed objects. */ +#define xrealloc_flex_struct(ptr, type, field, nr) ({                        \
+    typeof(*(ptr)) *ptr_ = (ptr);                                            \
+    /* Type checking: make sure that incoming pointer is of correct type */  \
+    (void)((ptr) == (type *)0);                                              \
+    (type *)_xrealloc(ptr_, offsetof(type, field[nr]), __alignof__(type));   \
+})
+
 /* Allocate untyped storage. */
 #define xmalloc_bytes(_bytes) _xmalloc(_bytes, SMP_CACHE_BYTES) #define xzalloc_bytes(_bytes) _xzalloc(_bytes, SMP_CACHE_BYTES) (END)

Signature · 9 lines
Regards,

Oleksandr Tyshchenko


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#15 lore raw

Re: [Xen-devel] [PATCH V3 4/8] xen/common: Introduce xrealloc_flex_struct() helper macros

Quoted 23 lines · Oleksandr
xen/include/xen/xmalloc.h +1 -0
@@ -35,6 +35,18 @@
  #define xzalloc_array(_type, _num) \
      ((_type *)_xzalloc_array(sizeof(_type), __alignof__(_type), _num))
 
+/* Allocate space for a structure with a flexible array of typed 

objects. */

xen/include/xen/xmalloc.h +4 -0
+#define xmalloc_flex_struct(type, field, nr) \
+    ((type *)_xmalloc(offsetof(type, field[nr]), __alignof__(type)))
+
+/* Re-allocate space for a structure with a flexible array of typed 

objects. */
+#define xrealloc_flex_struct(ptr, type, field, nr)
({                        \
+    typeof(*(ptr)) *ptr_ =
(ptr);                                            \
+    /* Type checking: make sure that incoming pointer is of correct type */  \
+    (void)((ptr) == (type
*)0);                                              \
+    (type *)_xrealloc(ptr_, offsetof(type, field[nr]), __alignof__(type));   \

xen/include/xen/xmalloc.h

Quoted 2 lines
xen/include/xen/xmalloc.h +2 -0
+})
+

What about

#define xrealloc_flex_struct(ptr, field, nr)                         \
    (typeof(ptr))_xrealloc(ptr, offsetof(typeof(*(ptr)), field[nr]), \
                           __alignof__(typeof(*(ptr))))

?

Jan

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

author
#16 lore raw

Re: [Xen-devel] [PATCH V3 4/8] xen/common: Introduce xrealloc_flex_struct() helper macros

On 29.08.19 10:21, Jan Beulich wrote:

Hi Jan

Quoted 28 lines · Oleksandr
Quoted 23 lines
xen/include/xen/xmalloc.h +1 -0
@@ -35,6 +35,18 @@
   #define xzalloc_array(_type, _num) \
       ((_type *)_xzalloc_array(sizeof(_type), __alignof__(_type), _num))
 
+/* Allocate space for a structure with a flexible array of typed

objects. */

xen/include/xen/xmalloc.h +4 -0
+#define xmalloc_flex_struct(type, field, nr) \
+    ((type *)_xmalloc(offsetof(type, field[nr]), __alignof__(type)))
+
+/* Re-allocate space for a structure with a flexible array of typed

objects. */
+#define xrealloc_flex_struct(ptr, type, field, nr)
({                        \
+    typeof(*(ptr)) *ptr_ =
(ptr);                                            \
+    /* Type checking: make sure that incoming pointer is of correct type */  \
+    (void)((ptr) == (type
*)0);                                              \
+    (type *)_xrealloc(ptr_, offsetof(type, field[nr]), __alignof__(type));   \

xen/include/xen/xmalloc.h

Quoted 2 lines
xen/include/xen/xmalloc.h +2 -0
+})
+

What about

Quoted 5 lines
#define xrealloc_flex_struct(ptr, field, nr)                         \
     (typeof(ptr))_xrealloc(ptr, offsetof(typeof(*(ptr)), field[nr]), \
                            __alignof__(typeof(*(ptr))))

?

Even better. And works...

Thank you.

Signature · 9 lines
Regards,

Oleksandr Tyshchenko


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#17 lore raw

PATCH v3 5/8 iommu/arm: Add lightweight iommu_fwspec support

In reply to Oleksandr Tyshchenko

From: Oleksandr Tyshchenko <[email protected]>

We need to have some abstract way to add new device to the IOMMU based on the generic IOMMU DT bindings [1] which can be used for both DT (right now) and ACPI (in future).

For that reason we can borrow the idea used in Linux these days called "iommu_fwspec". Having this in, it will be possible to configure IOMMU master interfaces of the device (device IDs) from a single common place and avoid keeping almost identical look-up implementations in each IOMMU driver.

There is no need to port the whole implementation of "iommu_fwspec" to Xen, we could, probably, end up with a much simpler solution, some "stripped down" version which fits our requirements.

So, this patch adds the following:
1. A common structure "iommu_fwspec" to hold the the per-device firmware data
2. New member "iommu_fwspec" of struct device
3. Functions/helpers to deal with "dev->iommu_fwspec"

It should be noted that in comparing with original "iommu_fwspec" Xen's variant doesn't contain some fields, which are not really needed at the moment (ops, flag) and "iommu_fwnode" field was replaced by "iommu_dev" to avoid porting a lot of code (to support "fwnode_handle") with little benefit.

Subsequent patches will use of that support.

[1] https://www.kernel.org/doc/Documentation/devicetree/bindings/iommu/iommu.txt

---
Changes V2 -> V3:
- added Copyright from Linux
- ordered the headers alphabetically
- removed check for not a NULL before calling xfree()
- used unsigned for variables which can't be negative
- removed #include <asm/iommu_fwspec.h> from iommu.h
- added check to iommu_fwspec_init() to not allow overriding
- clarified comments in code
- modified iommu_fwspec_add_ids() to use type-safe xrealloc_flex_struct()

---
 xen/drivers/passthrough/arm/Makefile       |  2 +-
 xen/drivers/passthrough/arm/iommu_fwspec.c | 94 ++++++++++++++++++++++++++++++
 xen/include/asm-arm/device.h               |  1 +
 xen/include/asm-arm/iommu_fwspec.h         | 68 +++++++++++++++++++++
 4 files changed, 164 insertions(+), 1 deletion(-)
 create mode 100644 xen/drivers/passthrough/arm/iommu_fwspec.c
 create mode 100644 xen/include/asm-arm/iommu_fwspec.h
xen/drivers/passthrough/arm/Makefile +1 -1
index 4abb87a..5fbad45 100644
@@ -1,2 +1,2 @@
-obj-y += iommu.o iommu_helpers.o
+obj-y += iommu.o iommu_helpers.o iommu_fwspec.o
 obj-$(CONFIG_ARM_SMMU) += smmu.o
xen/drivers/passthrough/arm/iommu_fwspec.c new file +94 -0

94 lines not on this page · raw message

xen/include/asm-arm/device.h +1 -0
index ee1c3bc..ee7cff2 100644
@@ -18,6 +18,7 @@ struct device
     struct dt_device_node *of_node; /* Used by drivers imported from Linux */
 #endif
     struct dev_archdata archdata;
+    struct iommu_fwspec *iommu_fwspec; /* per-device IOMMU instance data */
 };
 
 typedef struct device device_t;
xen/include/asm-arm/iommu_fwspec.h new file +68 -0

68 lines not on this page · raw message

Signature · 7 lines
2.7.4


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#18 lore raw

Re: [Xen-devel] [PATCH V3 5/8] iommu/arm: Add lightweight iommu_fwspec support

Hi Oleksandr,

Quoted 16 lines · Oleksandr Tyshchenko

From: Oleksandr Tyshchenko <[email protected]>

We need to have some abstract way to add new device to the IOMMU based on the generic IOMMU DT bindings [1] which can be used for both DT (right now) and ACPI (in future).

For that reason we can borrow the idea used in Linux these days called "iommu_fwspec". Having this in, it will be possible to configure IOMMU master interfaces of the device (device IDs) from a single common place and avoid keeping almost identical look-up implementations in each IOMMU driver.

There is no need to port the whole implementation of "iommu_fwspec" to Xen, we could, probably, end up with a much simpler solution, some "stripped down" version which fits our requirements.

Quoted 7 lines

So, this patch adds the following:
1. A common structure "iommu_fwspec" to hold the the per-device firmware data
2. New member "iommu_fwspec" of struct device
3. Functions/helpers to deal with "dev->iommu_fwspec"

It should be noted that in comparing with original "iommu_fwspec"

s/comparing with/comparison of/
s/original/the Linux/ or "the original".

Quoted 4 lines

Xen's variant doesn't contain some fields, which are not really needed at the moment (ops, flag) and "iommu_fwnode" field was replaced by "iommu_dev" to avoid porting a lot of code (to support "fwnode_handle") with little benefit.

It would be good to mention which version of Linux this is based on. So if there is a critical bug in Linux we can port it.

The rest of the patch looks good to me.

Cheers,

Signature · 6 lines
Julien Grall

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
author
#19 lore raw

Re: [Xen-devel] [PATCH V3 5/8] iommu/arm: Add lightweight iommu_fwspec support

Quoted 1 line · Julien Grall

Hi Oleksandr,

Hi, Julien

Quoted 24 lines · Oleksandr Tyshchenko
Quoted 22 lines

From: Oleksandr Tyshchenko <[email protected]>

We need to have some abstract way to add new device to the IOMMU based on the generic IOMMU DT bindings [1] which can be used for both DT (right now) and ACPI (in future).

For that reason we can borrow the idea used in Linux these days called "iommu_fwspec". Having this in, it will be possible to configure IOMMU master interfaces of the device (device IDs) from a single common place and avoid keeping almost identical look-up implementations in each IOMMU driver.

There is no need to port the whole implementation of "iommu_fwspec" to Xen, we could, probably, end up with a much simpler solution, some "stripped down" version which fits our requirements.

So, this patch adds the following:
1. A common structure "iommu_fwspec" to hold the the per-device firmware data
2. New member "iommu_fwspec" of struct device
3. Functions/helpers to deal with "dev->iommu_fwspec"

Quoted 4 lines
Quoted 1 line

It should be noted that in comparing with original "iommu_fwspec"

s/comparing with/comparison of/
s/original/the Linux/ or "the original".

ok

Quoted 8 lines
Quoted 5 lines

Xen's variant doesn't contain some fields, which are not really needed at the moment (ops, flag) and "iommu_fwnode" field was replaced by "iommu_dev" to avoid porting a lot of code (to support "fwnode_handle") with little benefit.

It would be good to mention which version of Linux this is based on. So if there is a critical bug in Linux we can port it.

It was based on current Linux master. I will mention exact version.

Signature · 9 lines
Regards,

Oleksandr Tyshchenko


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#20 lore raw

PATCH v3 6/8 iommu: Add of_xlate callback

In reply to Oleksandr Tyshchenko

From: Oleksandr Tyshchenko <[email protected]>

According to the generic IOMMU DT bindings [1] the context of required properties for IOMMU device/master node (#iommu-cells, iommus) depends on many factors and is really driver depended thing.

We need some way to provide the driver with DT IOMMU specifier which describes the IOMMU master interfaces of that device (device IDs, etc) to let it decide how to interpret them. For that reason we can borrow Linux's "of_xlate" callback.

All IOMMU drivers which support generic IOMMU DT bindings should use this callback.

Subsequent patches will use of that support.

[1] https://www.kernel.org/doc/Documentation/devicetree/bindings/iommu/iommu.txt

---
 xen/include/xen/iommu.h | 10 ++++++++++
 1 file changed, 10 insertions(+)
xen/include/xen/iommu.h +10 -0
index 48f8748..79b86ac 100644
@@ -240,6 +240,16 @@ struct iommu_ops {
     int __must_check (*iotlb_flush_all)(struct domain *d);
     int (*get_reserved_device_memory)(iommu_grdm_t *, void *);
     void (*dump_p2m_table)(struct domain *d);
+
+#ifdef CONFIG_HAS_DEVICE_TREE
+    /*
+     * All IOMMU drivers which support generic IOMMU DT bindings should use
+     * this callback. This is a way for the framework to provide the driver
+     * with DT IOMMU specifier which describes the IOMMU master interfaces of
+     * that device (device IDs, etc).
+     */
+    int (*of_xlate)(device_t *dev, struct dt_phandle_args *args);
+#endif
 };
 
 #include <asm/iommu.h>
Signature · 7 lines
2.7.4


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#21 lore raw

Re: [Xen-devel] [PATCH V3 6/8] iommu: Add of_xlate callback

Quoted 7 lines · Oleksandr Tyshchenko
xen/include/xen/iommu.h +1 -0
@@ -240,6 +240,16 @@ struct iommu_ops {
      int __must_check (*iotlb_flush_all)(struct domain *d);
      int (*get_reserved_device_memory)(iommu_grdm_t *, void *);
      void (*dump_p2m_table)(struct domain *d);
+

xen/include/xen/iommu.h struct iommu_ops { · line 253

Quoted 10 lines
xen/include/xen/iommu.h +9 -0
+#ifdef CONFIG_HAS_DEVICE_TREE
+    /*
+     * All IOMMU drivers which support generic IOMMU DT bindings should use
+     * this callback. This is a way for the framework to provide the driver
+     * with DT IOMMU specifier which describes the IOMMU master interfaces of
+     * that device (device IDs, etc).
+     */
+    int (*of_xlate)(device_t *dev, struct dt_phandle_args *args);
+#endif
  };

I think this should be folded into the first patch using it. Doing so may also answer my question of what "of" here stands for.

Jan

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

author
#22 lore raw

Re: [Xen-devel] [PATCH V3 6/8] iommu: Add of_xlate callback

Hi Jan

Quoted 16 lines · Oleksandr Tyshchenko
Quoted 15 lines
xen/include/xen/iommu.h +4 -0
@@ -240,6 +240,16 @@ struct iommu_ops {
       int __must_check (*iotlb_flush_all)(struct domain *d);
       int (*get_reserved_device_memory)(iommu_grdm_t *, void *);
       void (*dump_p2m_table)(struct domain *d);
+
+#ifdef CONFIG_HAS_DEVICE_TREE
+    /*
+     * All IOMMU drivers which support generic IOMMU DT bindings 

should use
+     * this callback. This is a way for the framework to provide the driver
+     * with DT IOMMU specifier which describes the IOMMU master interfaces of

Quoted 8 lines
Quoted 5 lines

+     * that device (device IDs, etc).
+     */
+    int (*of_xlate)(device_t *dev, struct dt_phandle_args *args); +#endif
  };

I think this should be folded into the first patch using it. Doing so may also answer my question of what "of" here stands for.

In Linux, code related to the device-tree is prefixed with "of_". But, in Xen, code related to the device-tree is prefixed with "dt_" [1]. Looks like, I should have named it "dt_xlate" to be consistent. There was a preference to introduce callback in a separate patch [2]. Anyway, shall I fold it?

[1] https://github.com/xen-project/xen/blob/master/xen/include/xen/device_tree.h
[2] https://lists.xenproject.org/archives/html/xen-devel/2019-08/msg01393.html

Signature · 9 lines
Regards,

Oleksandr Tyshchenko


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#23 lore raw

Re: [Xen-devel] [PATCH V3 6/8] iommu: Add of_xlate callback

Quoted 1 line · Oleksandr

There was a preference to introduce callback in a separate patch [2]. Anyway, shall I fold it?

Hmm, I disagree with Julien here. I don't think we should have unused hooks in the tree, not even intermediately.

Jan

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#24 lore raw

Re: [Xen-devel] [PATCH V3 6/8] iommu: Add of_xlate callback

Hi,

Quoted 5 lines · Jan Beulich
Quoted 1 line · Oleksandr

There was a preference to introduce callback in a separate patch [2]. Anyway, shall I fold it?

Hmm, I disagree with Julien here. I don't think we should have unused hooks in the tree, not even intermediately.

I have stated my preference before, but this is not a strong one nor a must. If this is your condition to get the series accepted, then Oleksandr should follow your requested.

Cheers,

Signature · 6 lines
Julien Grall

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
author
#25 lore raw

Re: [Xen-devel] [PATCH V3 6/8] iommu: Add of_xlate callback

Quoted 1 line · Julien Grall

Hi,

Hi, all.

Quoted 10 lines · Jan Beulich
Quoted 5 lines · Oleksandr
Quoted 1 line

There was a preference to introduce callback in a separate patch [2]. Anyway, shall I fold it?

Hmm, I disagree with Julien here. I don't think we should have unused hooks in the tree, not even intermediately.

I have stated my preference before, but this is not a strong one nor a must. If this is your condition to get the series accepted, then Oleksandr should follow your requested.

Will fold it as requested. And will rename callback to "dt_xlate".

Quoted 3 lines

Cheers,

Signature · 9 lines
Regards,

Oleksandr Tyshchenko


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#26 lore raw

PATCH v3 7/8 iommu/arm: Introduce iommu_add_dt_device API

In reply to Oleksandr Tyshchenko

From: Oleksandr Tyshchenko <[email protected]>

This patch adds new iommu_add_dt_device API for adding DT device to the IOMMU using generic IOMMU DT bindings [1] and previously added "iommu_fwspec" support and "add_device/of_xlate" callbacks.

New function does the following:
- Parse the DT bindings according to the specification
- Provide DT IOMMU specifier which describes the IOMMU master interfaces of that device (device IDs, etc) to the driver
- Add master device to the IOMMU if latter is present and available

The additional benefit here is to avoid to go through the whole DT multiple times in IOMMU driver trying to locate master devices which belong to each IOMMU device being probed.

[1] https://www.kernel.org/doc/Documentation/devicetree/bindings/iommu/iommu.txt

---
Changes V2 -> V3:
- clarified patch description
- clarified comments in code
- modified to provide DT IOMMU specifier to the driver using "of_xlate" callback
- documented function usage
- modified to return an error if ops is not present/implemented,
- added ability to return a possitive value to indicate that device doesn't need to be protected
- removed check for the "iommu" property presence
in the common code
- included <asm/iommu_fwspec.h> directly

---
 xen/arch/arm/domain_build.c         | 11 ++++++++
 xen/drivers/passthrough/arm/iommu.c | 55 +++++++++++++++++++++++++++++++++++++
 xen/include/asm-arm/iommu.h         | 11 ++++++++
 3 files changed, 77 insertions(+)
xen/arch/arm/domain_build.c +11 -0
index e79d4e2..159ea6a 100644
@@ -1240,6 +1240,7 @@ static int __init map_device_children(struct domain *d,
 
 /*
  * For a given device node:
+ *  - Try to call iommu_add_dt_device to protect the device by an IOMMU
  *  - Give permission to the guest to manage IRQ and MMIO range
  *  - Retrieve the IRQ configuration (i.e edge/level) from device tree
  * When the device is not marked for guest passthrough:
@@ -1257,6 +1258,16 @@ static int __init handle_device(struct domain *d, struct dt_device_node *dev,
     u64 addr, size;
     bool need_mapping = !dt_device_for_passthrough(dev);
 
+    dt_dprintk("%s add to iommu\n", dt_node_full_name(dev));
+
+    res = iommu_add_dt_device(dev);
+    if ( res < 0 )
+    {
+        printk(XENLOG_ERR "Failed to add %s to the IOMMU\n",
+               dt_node_full_name(dev));
+        return res;
+    }
+
     nirq = dt_number_of_irq(dev);
     naddr = dt_number_of_address(dev);
 
xen/drivers/passthrough/arm/iommu.c +55 -0

64 lines not on this page · raw message

xen/include/asm-arm/iommu.h +11 -0
index 20d865e..e75359c 100644
@@ -26,6 +26,17 @@ struct arch_iommu
 const struct iommu_ops *iommu_get_ops(void);
 void iommu_set_ops(const struct iommu_ops *ops);
 
+/*
+ * Helper to add master device to the IOMMU using generic IOMMU DT bindings.
+ *
+ * Return values:
+ *  0 : device is protected by an IOMMU
+ * <0 : device is not protected by an IOMMU, but must be (error condition)
+ * >0 : device doesn't need to be protected by an IOMMU
+ *      (IOMMU is not enabled/present or device is not connected to it).
+ */
+int iommu_add_dt_device(struct dt_device_node *np);
+
 /* mapping helpers */
 int __must_check arm_iommu_map_page(struct domain *d, dfn_t dfn, mfn_t mfn,
                                     unsigned int flags,
Signature · 7 lines
2.7.4


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#27 lore raw

Re: [Xen-devel] [PATCH V3 7/8] iommu/arm: Introduce iommu_add_dt_device API

Hi Oleksandr,

Quoted 12 lines · Oleksandr Tyshchenko

From: Oleksandr Tyshchenko <[email protected]>

This patch adds new iommu_add_dt_device API for adding DT device to the IOMMU using generic IOMMU DT bindings [1] and previously added "iommu_fwspec" support and "add_device/of_xlate" callbacks.

New function does the following:
- Parse the DT bindings according to the specification
- Provide DT IOMMU specifier which describes the IOMMU master interfaces of that device (device IDs, etc) to the driver
- Add master device to the IOMMU if latter is present and available

Quoted 3 lines

The additional benefit here is to avoid to go through the whole DT multiple times in IOMMU driver trying to locate master devices which belong to each IOMMU device being probed.

So the commit title/message describes the new function iommu_add_dt_device, but not the main important thing i.e. "Why is it called when building dom0".

While I agree the new function is the big part of the function what matter is we need to register device using the generic IOMMU bindings before assigning the device to a domain. The split is to keep separate "add" and "assign". The later can be called from dom0.

Quoted 38 lines

[1] https://www.kernel.org/doc/Documentation/devicetree/bindings/iommu/iommu.txt

---
Changes V2 -> V3:
- clarified patch description
- clarified comments in code
- modified to provide DT IOMMU specifier to the driver using "of_xlate" callback
- documented function usage
- modified to return an error if ops is not present/implemented,
- added ability to return a possitive value to indicate that device doesn't need to be protected
- removed check for the "iommu" property presence
in the common code
- included <asm/iommu_fwspec.h> directly

---
  xen/arch/arm/domain_build.c         | 11 ++++++++
  xen/drivers/passthrough/arm/iommu.c | 55 +++++++++++++++++++++++++++++++++++++
  xen/include/asm-arm/iommu.h         | 11 ++++++++
  3 files changed, 77 insertions(+)
xen/arch/arm/domain_build.c +1 -0
index e79d4e2..159ea6a 100644
@@ -1240,6 +1240,7 @@ static int __init map_device_children(struct domain *d,
 
  /*
   * For a given device node:
+ *  - Try to call iommu_add_dt_device to protect the device by an IOMMU
   *  - Give permission to the guest to manage IRQ and MMIO range
   *  - Retrieve the IRQ configuration (i.e edge/level) from device tree
   * When the device is not marked for guest passthrough:

xen/arch/arm/domain_build.c static int __init handle_device(struct domain *d, struct dt_device_node *dev, · line 1261

Quoted 4 lines
xen/arch/arm/domain_build.c +1 -0
@@ -1257,6 +1258,16 @@ static int __init handle_device(struct domain *d, struct dt_device_node *dev,
      u64 addr, size;
      bool need_mapping = !dt_device_for_passthrough(dev);
 
+    dt_dprintk("%s add to iommu\n", dt_node_full_name(dev));

This message is slightly confusing. You are not adding the device, you are trying to. So how about "Check if %s is behind an IOMMU and add it".

Quoted 30 lines
xen/arch/arm/domain_build.c +9 -0
+
+    res = iommu_add_dt_device(dev);
+    if ( res < 0 )
+    {
+        printk(XENLOG_ERR "Failed to add %s to the IOMMU\n",
+               dt_node_full_name(dev));
+        return res;
+    }
+
      nirq = dt_number_of_irq(dev);
      naddr = dt_number_of_address(dev);
 
xen/drivers/passthrough/arm/iommu.c +3 -0
index 72a30e0..47e4bc6 100644
@@ -20,6 +20,7 @@
  #include <xen/lib.h>
 
  #include <asm/device.h>
+#include <asm/iommu_fwspec.h>
 
  /*
   * Deferred probe list is used to keep track of devices for which driver
@@ -139,3 +140,57 @@ int arch_iommu_populate_page_table(struct domain *d)
  void __hwdom_init arch_iommu_hwdom_init(struct domain *d)
  {
  }
+
+int __init iommu_add_dt_device(struct dt_device_node *np)

xen/drivers/passthrough/arm/iommu.c int arch_iommu_populate_page_table(struct domain *d) · line 154

Quoted 10 lines
xen/drivers/passthrough/arm/iommu.c +10 -0
+{
+    const struct iommu_ops *ops = iommu_get_ops();
+    struct dt_phandle_args iommu_spec;
+    struct device *dev = dt_to_dev(np);
+    int rc = 1, index = 0;
+
+    if ( !iommu_enabled )
+        return 1;
+
+    if ( !ops || !ops->add_device || !ops->of_xlate )

The SMMU does not implement of_xlate(). It is actually only mandatory if you are using the generic bindings. So I would only check ops->of_xlate if "iommus" exists.

xen/drivers/passthrough/arm/iommu.c int arch_iommu_populate_page_table(struct domain *d) · line 160

Quoted 5 lines
xen/drivers/passthrough/arm/iommu.c +5 -0
+        return -EINVAL;
+
+    if ( dev_iommu_fwspec_get(dev) )
+        return -EEXIST; > +
+    /* According Documentation/devicetree/bindings/iommu/iommu.txt from Linux */

s/According/According to/

Quoted 4 lines
xen/drivers/passthrough/arm/iommu.c +4 -0
+    while ( !dt_parse_phandle_with_args(np, "iommus", "#iommu-cells",
+                                        index, &iommu_spec) )
+    {
+        if ( !dt_device_is_available(iommu_spec.np) )

xen/drivers/passthrough/arm/iommu.c int arch_iommu_populate_page_table(struct domain *d) · line 174

Quoted 10 lines
xen/drivers/passthrough/arm/iommu.c +10 -0
+            break;
+
+        rc = iommu_fwspec_init(dev, &iommu_spec.np->dev);
+        if ( rc )
+            break;
+
+        /*
+         * Provide DT IOMMU specifier which describes the IOMMU master
+         * interfaces of that device (device IDs, etc) to the driver.
+         * The driver's responsibility is to decide how to interpret them.

NIT: "The driver is responsible to decide...".

xen/drivers/passthrough/arm/iommu.c int arch_iommu_populate_page_table(struct domain *d) · line 175

Quoted 1 line
xen/drivers/passthrough/arm/iommu.c +1 -0
+         * It should also initialize/verify that device.

What do you mean? of_xlate should mostly translate the specifier to fwspec.

xen/drivers/passthrough/arm/iommu.c int arch_iommu_populate_page_table(struct domain *d) · line 187

Quoted 12 lines
xen/drivers/passthrough/arm/iommu.c +12 -0
+         */
+        rc = ops->of_xlate(dev, &iommu_spec);
+        if ( rc )
+            break;
+
+        index++;
+    }
+
+    /*
+     * Add master device to the IOMMU if latter is present and available.
+     * The driver's responsibility is to check whether that device
+     * was initialized/verified before and mark that device as protected.

I don't understand the last sentence. For me, "device" refers to what's pointed by "dev". But the IOMMU driver is not responsible for initializing the device.

Quoted 22 lines
xen/drivers/passthrough/arm/iommu.c +9 -0
+     */
+    if ( !rc )
+        rc = ops->add_device(0, dev);
+
+    if ( rc < 0 )
+        iommu_fwspec_free(dev);
+
+    return rc;
+}
xen/include/asm-arm/iommu.h +4 -0
index 20d865e..e75359c 100644
@@ -26,6 +26,17 @@ struct arch_iommu
  const struct iommu_ops *iommu_get_ops(void);
  void iommu_set_ops(const struct iommu_ops *ops);
 
+/*
+ * Helper to add master device to the IOMMU using generic IOMMU DT bindings.
+ *
+ * Return values:

xen/include/asm-arm/iommu.h struct arch_iommu · line 42

Quoted 10 lines
xen/include/asm-arm/iommu.h +7 -0
+ *  0 : device is protected by an IOMMU
+ * <0 : device is not protected by an IOMMU, but must be (error condition)
+ * >0 : device doesn't need to be protected by an IOMMU
+ *      (IOMMU is not enabled/present or device is not connected to it).
+ */
+int iommu_add_dt_device(struct dt_device_node *np);
+
  /* mapping helpers */
  int __must_check arm_iommu_map_page(struct domain *d, dfn_t dfn, mfn_t mfn,
                                      unsigned int flags,

Cheers,

Signature · 6 lines
Julien Grall

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
author
#28 lore raw

Re: [Xen-devel] [PATCH V3 7/8] iommu/arm: Introduce iommu_add_dt_device API

Quoted 1 line · Julien Grall

Hi Oleksandr,

Hi Julien

Quoted 15 lines · Oleksandr Tyshchenko
Quoted 12 lines

From: Oleksandr Tyshchenko <[email protected]>

This patch adds new iommu_add_dt_device API for adding DT device to the IOMMU using generic IOMMU DT bindings [1] and previously added "iommu_fwspec" support and "add_device/of_xlate" callbacks.

New function does the following:
- Parse the DT bindings according to the specification
- Provide DT IOMMU specifier which describes the IOMMU master interfaces of that device (device IDs, etc) to the driver
- Add master device to the IOMMU if latter is present and available

Quoted 7 lines
Quoted 3 lines

The additional benefit here is to avoid to go through the whole DT multiple times in IOMMU driver trying to locate master devices which belong to each IOMMU device being probed.

So the commit title/message describes the new function iommu_add_dt_device, but not the main important thing i.e. "Why is it called when building dom0".

While I agree the new function is the big part of the function what matter is we need to register device using the generic IOMMU bindings before assigning the device to a domain. The split is to keep separate "add" and "assign". The later can be called from dom0.

Good point. I will update description.

Quoted 39 lines
Quoted 39 lines

[1] https://www.kernel.org/doc/Documentation/devicetree/bindings/iommu/iommu.txt

---
Changes V2 -> V3:
     - clarified patch description
     - clarified comments in code
     - modified to provide DT IOMMU specifier to the driver using "of_xlate" callback
     - documented function usage
     - modified to return an error if ops is not present/implemented,
     - added ability to return a possitive value to indicate that device doesn't need to be protected
     - removed check for the "iommu" property presence
       in the common code
     - included <asm/iommu_fwspec.h> directly
---
  xen/arch/arm/domain_build.c         | 11 ++++++++ xen/drivers/passthrough/arm/iommu.c | 55
+++++++++++++++++++++++++++++++++++++
  xen/include/asm-arm/iommu.h         | 11 ++++++++

  3 files changed, 77 insertions(+)
xen/arch/arm/domain_build.c +1 -0
index e79d4e2..159ea6a 100644
@@ -1240,6 +1240,7 @@ static int __init map_device_children(struct 
 domain *d,
     /*
    * For a given device node:
+ *  - Try to call iommu_add_dt_device to protect the device by an IOMMU
    *  - Give permission to the guest to manage IRQ and MMIO range
    *  - Retrieve the IRQ configuration (i.e edge/level) from device tree
Quoted 8 lines

xen/arch/arm/domain_build.c static int __init handle_device(struct domain · line 1261

Quoted 6 lines
xen/arch/arm/domain_build.c +0 -0
    * When the device is not marked for guest passthrough:
@@ -1257,6 +1258,16 @@ static int __init handle_device(struct domain 
 *d, struct dt_device_node *dev,
       u64 addr, size;
       bool need_mapping = !dt_device_for_passthrough(dev);
   +    dt_dprintk("%s add to iommu\n", dt_node_full_name(dev));

This message is slightly confusing. You are not adding the device, you are trying to. So how about "Check if %s is behind an IOMMU and add it".

Sounds reasonable.

Quoted 34 lines
Quoted 34 lines
xen/arch/arm/domain_build.c +9 -0
+
+    res = iommu_add_dt_device(dev);
+    if ( res < 0 )
+    {
+        printk(XENLOG_ERR "Failed to add %s to the IOMMU\n",
+               dt_node_full_name(dev));
+        return res;
+    }
+
       nirq = dt_number_of_irq(dev);
       naddr = dt_number_of_address(dev);

  diff --git a/xen/drivers/passthrough/arm/iommu.c b/xen/drivers/passthrough/arm/iommu.c index 72a30e0..47e4bc6 100644

xen/drivers/passthrough/arm/iommu.c +1 -0
@@ -20,6 +20,7 @@
   #include <xen/lib.h>
     #include <asm/device.h>
+#include <asm/iommu_fwspec.h>
     /*
    * Deferred probe list is used to keep track of devices for which driver
@@ -139,3 +140,57 @@ int arch_iommu_populate_page_table(struct domain 
 *d)
   void __hwdom_init arch_iommu_hwdom_init(struct domain *d)
   {

  }

xen/drivers/passthrough/arm/iommu.c +6 -0
+
+int __init iommu_add_dt_device(struct dt_device_node *np)
+{
+    const struct iommu_ops *ops = iommu_get_ops();
+    struct dt_phandle_args iommu_spec;
+    struct device *dev = dt_to_dev(np);
Quoted 8 lines

xen/drivers/passthrough/arm/iommu.c int arch_iommu_populate_page_table(struct domain *d) · line 154

Quoted 6 lines

+    int rc = 1, index = 0;

xen/drivers/passthrough/arm/iommu.c +5 -0
+
+    if ( !iommu_enabled )
+        return 1;
+
+    if ( !ops || !ops->add_device || !ops->of_xlate )

The SMMU does not implement of_xlate(). It is actually only mandatory if you are using the generic bindings. So I would only check ops->of_xlate if "iommus" exists.

Agree. Will do.

Quoted 10 lines
Quoted 8 lines

+        return -EINVAL;
+
+    if ( dev_iommu_fwspec_get(dev) )
+        return -EEXIST; > +
+    /* According Documentation/devicetree/bindings/iommu/iommu.txt from Linux */

s/According/According to/

ok

Quoted 11 lines
Quoted 11 lines

+    while ( !dt_parse_phandle_with_args(np, "iommus", "#iommu-cells", +                                        index, &iommu_spec) ) +    {
+        if ( !dt_device_is_available(iommu_spec.np) )
+            break;
+
+        rc = iommu_fwspec_init(dev, &iommu_spec.np->dev); +        if ( rc )
+            break;

Quoted 8 lines
Quoted 6 lines

+
+        /*
+         * Provide DT IOMMU specifier which describes the IOMMU master +         * interfaces of that device (device IDs, etc) to the driver. +         * The driver's responsibility is to decide how to interpret them.

NIT: "The driver is responsible to decide...".

ok

Quoted 4 lines
Quoted 2 lines

+         * It should also initialize/verify that device.

What do you mean? of_xlate should mostly translate the specifier to fwspec.

Yes. Saying "initialize/verify that device" I meant to verify passed DT IOMMU specifier and initialize driver's private data for this device (iommu_priv). But, this is obvious.

I will remove confusing word "initialize" or even the whole sentence.

Quoted 18 lines
Quoted 12 lines

+         */
+        rc = ops->of_xlate(dev, &iommu_spec);
+        if ( rc )
+            break;

xen/drivers/passthrough/arm/iommu.c +6 -0
+
+        index++;
+    }
+
+    /*
+     * Add master device to the IOMMU if latter is present and 
Quoted 4 lines

available.
+     * The driver's responsibility is to check whether that device +     * was initialized/verified before and mark that device as protected.

I don't understand the last sentence. For me, "device" refers to what's pointed by "dev". But the IOMMU driver is not responsible for initializing the device.

Yes. The same as for comment above. I will remove confusing word "initialize".

Signature · 9 lines
Regards,

Oleksandr Tyshchenko


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
author
#29 lore raw

Re: [Xen-devel] [PATCH V3 7/8] iommu/arm: Introduce iommu_add_dt_device API

Hi, Julien

Quoted 31 lines
Quoted 22 lines
Quoted 22 lines

  diff --git a/xen/drivers/passthrough/arm/iommu.c b/xen/drivers/passthrough/arm/iommu.c index 72a30e0..47e4bc6 100644

xen/drivers/passthrough/arm/iommu.c +1 -0
@@ -20,6 +20,7 @@
   #include <xen/lib.h>
     #include <asm/device.h>
+#include <asm/iommu_fwspec.h>
     /*
    * Deferred probe list is used to keep track of devices for which driver
@@ -139,3 +140,57 @@ int arch_iommu_populate_page_table(struct 
 domain *d)
   void __hwdom_init arch_iommu_hwdom_init(struct domain *d)
   {

  }

xen/drivers/passthrough/arm/iommu.c +6 -0
+
+int __init iommu_add_dt_device(struct dt_device_node *np)
+{
+    const struct iommu_ops *ops = iommu_get_ops();
+    struct dt_phandle_args iommu_spec;
+    struct device *dev = dt_to_dev(np);
Quoted 8 lines

xen/drivers/passthrough/arm/iommu.c int arch_iommu_populate_page_table(struct domain *d) · line 154

Quoted 6 lines

+    int rc = 1, index = 0;

xen/drivers/passthrough/arm/iommu.c +5 -0
+
+    if ( !iommu_enabled )
+        return 1;
+
+    if ( !ops || !ops->add_device || !ops->of_xlate )

The SMMU does not implement of_xlate(). It is actually only mandatory if you are using the generic bindings. So I would only check ops->of_xlate if "iommus" exists.

Quoted 1 line

Agree. Will do.

Just to clarify.

What about "ops->add_device", shall I check it if "iommus" exists as well?

Signature · 9 lines
Regards,

Oleksandr Tyshchenko


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#30 lore raw

Re: [Xen-devel] [PATCH V3 7/8] iommu/arm: Introduce iommu_add_dt_device API

Quoted 2 lines · Oleksandr

Hi, Julien

Hi,

Quoted 31 lines
Quoted 31 lines
Quoted 30 lines
Quoted 18 lines

  diff --git a/xen/drivers/passthrough/arm/iommu.c b/xen/drivers/passthrough/arm/iommu.c index 72a30e0..47e4bc6 100644

xen/drivers/passthrough/arm/iommu.c +1 -0
@@ -20,6 +20,7 @@
   #include <xen/lib.h>
     #include <asm/device.h>
+#include <asm/iommu_fwspec.h>
     /*
    * Deferred probe list is used to keep track of devices for which driver
@@ -139,3 +140,57 @@ int arch_iommu_populate_page_table(struct 
 domain *d)
   void __hwdom_init arch_iommu_hwdom_init(struct domain *d)
   {

  }

xen/drivers/passthrough/arm/iommu.c +2 -0
+
+int __init iommu_add_dt_device(struct dt_device_node *np)

xen/drivers/passthrough/arm/iommu.c int arch_iommu_populate_page_table(struct domain *d) · line 154

Quoted 10 lines
xen/drivers/passthrough/arm/iommu.c +10 -0
+{
+    const struct iommu_ops *ops = iommu_get_ops();
+    struct dt_phandle_args iommu_spec;
+    struct device *dev = dt_to_dev(np);
+    int rc = 1, index = 0;
+
+    if ( !iommu_enabled )
+        return 1;
+
+    if ( !ops || !ops->add_device || !ops->of_xlate )

The SMMU does not implement of_xlate(). It is actually only mandatory if you are using the generic bindings. So I would only check ops->of_xlate if "iommus" exists.

Quoted 6 lines
Quoted 1 line

Agree. Will do.

Just to clarify.

What about "ops->add_device", shall I check it if "iommus" exists as well?

Yes. Somehow I thought add_device was implemented for the SMMU driver, but I got confused with the Linux IOMMU ops.

Cheers,

Signature · 6 lines
Julien Grall

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#31 lore raw

PATCH v3 8/8 iommu/arm: Add Renesas IPMMU-VMSA support

In reply to Oleksandr Tyshchenko

From: Oleksandr Tyshchenko <[email protected]>

The IPMMU-VMSA is VMSA-compatible I/O Memory Management Unit (IOMMU) which provides address translation and access protection functionalities to processing units and interconnect networks.

Please note, current driver is supposed to work only with newest Gen3 SoCs revisions which IPMMU hardware supports stage 2 translation table format and is able to use CPU's P2M table as is if one is 3-level page table (up to 40 bit IPA).

The major differences compare to the Linux driver are:

1. Stage 1/Stage 2 translation. Linux driver supports Stage 1 translation only (with Stage 1 translation table format). It manages page table by itself. But Xen driver supports Stage 2 translation (with Stage 2 translation table format) to be able to share the P2M with the CPU. Stage 1 translation is always bypassed in Xen driver.

So, Xen driver is supposed to be used with newest R-Car Gen3 SoC revisions only (H3 ES3.0, M3-W+, etc.) which IPMMU H/W supports stage 2 translation table format.

2. AArch64 support. Linux driver uses VMSAv8-32 mode, while Xen driver enables Armv8 VMSAv8-64 mode to cover up to 40 bit input address.

3. Context bank (sets of page table) usage. In Xen, each context bank is mapped to one Xen domain. So, all devices being pass throughed to the same Xen domain share the same context bank.

4. IPMMU device tracking. In Xen, all IOMMU devices are managed by single driver instance. So, driver uses global list to keep track of registered IPMMU devices.

CC Julien Grall <[email protected]>
CC Yoshihiro Shimoda <[email protected]>

---
Changes V2 -> V3:
- added Copyright from Linux
- included <xen/iommu.h> and <asm/iommu_fwspec.h> directly
- removed pointless spin_lock from ipmmu_iommu_domain_teardown()
- changed "Gen3" to "R-Car Gen3", "M3 ES3.0" to "M3-W+"
- changed RCAR_PRODUCT_M3 to RCAR_PRODUCT_M3W
- clarified IMTTBCR_TSZ1_MASK
- removed TLB flush from ipmmu_domain_irq() until clarified the purpose
- returned -ENODEV in ipmmu_init() if P2M sharing or P2M IPA size is not supported to show that IOMMU device is not usable in Xen
- implemented new callback "of_xlate" to parse DT IOMMU specifier and add device IDs to "iommu_fwspec"
- reworked "add_device" callback
- updated patch description

Changes V1 -> V2:
- rewrited driver to use iommu_fwspec
- removed DT parsing code for micro-TLBs
- removed struct ipmmu_vmsa_master_cfg, dev_archdata macro
- added ipmmu_find_mmu_by_dev(), various helpers to access fwspec->iommu_priv
- implemented new callback "add_device" to add master device to IPMMU
- removed ipmmu_protect_masters()
- removed code to locate Root device in the first place, used EAGAIN to request deferred probing
- used printk_once for the system wide error messages in ipmmu_init() which don't need to be shown for every device being probed
- removed map_page/unmap_page implementation, reused them from iommu_helpers.c
- used %pd for printing domaid id
- performed various cosmetic fixes
- changed u32 -> uint32_t, u64 -> uint64_t,
unsigned int -> uint32_t where needed
- clarified TODOs
- clafiried supported SoC versions in config IPMMU_VMSA, set default to "n"
- updated comments in code, provided more accurate description, added new comments where needed
- updated patch description by providing differences between Linux/Xen implementations
- removed fields for cache snoop transaction when configuring IMTTBCR (update from Renesas BSP)

---
 xen/arch/arm/platforms/Kconfig           |    1 +
 xen/drivers/passthrough/Kconfig          |   13 +
 xen/drivers/passthrough/arm/Makefile     |    1 +
 xen/drivers/passthrough/arm/ipmmu-vmsa.c | 1359 ++++++++++++++++++++++++++++++
 4 files changed, 1374 insertions(+)
 create mode 100644 xen/drivers/passthrough/arm/ipmmu-vmsa.c
xen/arch/arm/platforms/Kconfig +1 -0
index bc0e9cd..c93a6b2 100644
@@ -25,6 +25,7 @@ config RCAR3
 	bool "Renesas RCar3 support"
 	depends on ARM_64
 	select HAS_SCIF
+	select IPMMU_VMSA
 	---help---
 	Enable all the required drivers for Renesas RCar3
 
xen/drivers/passthrough/Kconfig +13 -0
index a3c0649..47eadb4 100644
@@ -12,4 +12,17 @@ config ARM_SMMU
 
 	  Say Y here if your SoC includes an IOMMU device implementing the
 	  ARM SMMU architecture.
+
+config IPMMU_VMSA
+	bool "Renesas IPMMU-VMSA found in R-Car Gen3 SoCs"
+	default n
+	depends on ARM_64
+	---help---
+	  Support for implementations of the Renesas IPMMU-VMSA found
+	  in R-Car Gen3 SoCs.
+
+	  Say Y here if you are using newest R-Car Gen3 SoCs revisions
+	  (H3 ES3.0, M3-W+, etc) which IPMMU hardware supports stage 2
+	  translation table format and is able to use CPU's P2M table as is.
+
 endif
xen/drivers/passthrough/arm/Makefile +1 -0
index 5fbad45..fcd918e 100644
@@ -1,2 +1,3 @@
 obj-y += iommu.o iommu_helpers.o iommu_fwspec.o
 obj-$(CONFIG_ARM_SMMU) += smmu.o
+obj-$(CONFIG_IPMMU_VMSA) += ipmmu-vmsa.o
xen/drivers/passthrough/arm/ipmmu-vmsa.c new file +1359 -0

1,359 lines not on this page · raw message

Signature · 7 lines
2.7.4


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#32 lore raw

Re: [Xen-devel] [PATCH V3 8/8] iommu/arm: Add Renesas IPMMU-VMSA support

Hi Oleksandr-san,

Quoted 10 lines

From: Oleksandr Tyshchenko, Sent: Wednesday, August 21, 2019 3:10 AM

From: Oleksandr Tyshchenko <[email protected]>

The IPMMU-VMSA is VMSA-compatible I/O Memory Management Unit (IOMMU) which provides address translation and access protection functionalities to processing units and interconnect networks.

Please note, current driver is supposed to work only with newest Gen3 SoCs revisions which IPMMU hardware supports stage 2 translation

This is still "Gen3", so that please replace it with "R-Car Gen3"...

Quoted 22 lines

table format and is able to use CPU's P2M table as is if one is 3-level page table (up to 40 bit IPA).

The major differences compare to the Linux driver are:

1. Stage 1/Stage 2 translation. Linux driver supports Stage 1 translation only (with Stage 1 translation table format). It manages page table by itself. But Xen driver supports Stage 2 translation (with Stage 2 translation table format) to be able to share the P2M with the CPU. Stage 1 translation is always bypassed in Xen driver.

So, Xen driver is supposed to be used with newest R-Car Gen3 SoC revisions only (H3 ES3.0, M3-W+, etc.) which IPMMU H/W supports stage 2 translation table format.

2. AArch64 support. Linux driver uses VMSAv8-32 mode, while Xen driver enables Armv8 VMSAv8-64 mode to cover up to 40 bit input address.

3. Context bank (sets of page table) usage. In Xen, each context bank is mapped to one Xen domain. So, all devices being pass throughed to the same Xen domain share the same context bank.

Quoted 7 lines

4. IPMMU device tracking. In Xen, all IOMMU devices are managed by single driver instance. So, driver uses global list to keep track of registered IPMMU devices.

CC Julien Grall <[email protected]>
CC Yoshihiro Shimoda <[email protected]>

About this hardware handling, this patch seems good to me. But, since I'm not familiar about Xen passthrough framework, I think I cannot add my Reviewed-by tag into this patch. What do you think?

Best regards,
Yoshihiro Shimoda

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

author
#33 lore raw

Re: [Xen-devel] [PATCH V3 8/8] iommu/arm: Add Renesas IPMMU-VMSA support

Quoted 1 line · Yoshihiro Shimoda

Hi Oleksandr-san,

Hi Shimoda-san.

Quoted 5 lines
Quoted 5 lines

From: Oleksandr Tyshchenko, Sent: Wednesday, August 21, 2019 3:10 AM

From: Oleksandr Tyshchenko <[email protected]>

Quoted 7 lines
Quoted 6 lines

The IPMMU-VMSA is VMSA-compatible I/O Memory Management Unit (IOMMU) which provides address translation and access protection functionalities to processing units and interconnect networks.

Please note, current driver is supposed to work only with newest Gen3 SoCs revisions which IPMMU hardware supports stage 2 translation

This is still "Gen3", so that please replace it with "R-Car Gen3"...

Will do.

Quoted 27 lines
Quoted 27 lines

table format and is able to use CPU's P2M table as is if one is 3-level page table (up to 40 bit IPA).

The major differences compare to the Linux driver are:

1. Stage 1/Stage 2 translation. Linux driver supports Stage 1 translation only (with Stage 1 translation table format). It manages page table by itself. But Xen driver supports Stage 2 translation (with Stage 2 translation table format) to be able to share the P2M with the CPU. Stage 1 translation is always bypassed in Xen driver.

So, Xen driver is supposed to be used with newest R-Car Gen3 SoC revisions only (H3 ES3.0, M3-W+, etc.) which IPMMU H/W supports stage 2 translation table format.

2. AArch64 support. Linux driver uses VMSAv8-32 mode, while Xen driver enables Armv8 VMSAv8-64 mode to cover up to 40 bit input address.

3. Context bank (sets of page table) usage. In Xen, each context bank is mapped to one Xen domain. So, all devices being pass throughed to the same Xen domain share the same context bank.

4. IPMMU device tracking. In Xen, all IOMMU devices are managed by single driver instance. So, driver uses global list to keep track of registered IPMMU devices.

Quoted 6 lines
Quoted 3 lines
CC Julien Grall <[email protected]>
CC Yoshihiro Shimoda <[email protected]>

About this hardware handling, this patch seems good to me. But, since I'm not familiar about Xen passthrough framework, I think I cannot add my Reviewed-by tag into this patch. What do you think?

I am not completely sure regarding that, but I have seen cases when the people give their R-b not for the entire patch, but for a part of it.

Especially, when patch touches many sub-systems/archs. Would you mind if I specify what your R-b covers? -> [for the IPMMU H/W bits]

Thank you for the review.

Quoted 4 lines

Best regards,
Yoshihiro Shimoda

Signature · 9 lines
Regards,

Oleksandr Tyshchenko


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#34 lore raw

Re: [Xen-devel] [PATCH V3 8/8] iommu/arm: Add Renesas IPMMU-VMSA support

Hi Oleksandr-san,

Quoted 1 line

From: Oleksandr, Sent: Thursday, August 29, 2019 7:56 PM

<snip>

Quoted 9 lines
Quoted 3 lines

About this hardware handling, this patch seems good to me. But, since I'm not familiar about Xen passthrough framework, I think I cannot add my Reviewed-by tag into this patch. What do you think?

I am not completely sure regarding that, but I have seen cases when the people give their R-b not for the entire patch, but for a part of it.

Especially, when patch touches many sub-systems/archs. Would you mind if I specify what your R-b covers? -> [for the IPMMU H/W bits]

I got it. When you submit v4 patch, I'll review and send such a tag.

Best regards,
Yoshihiro Shimoda

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

author
#35 lore raw

Re: [Xen-devel] [PATCH V3 8/8] iommu/arm: Add Renesas IPMMU-VMSA support

Quoted 1 line · Yoshihiro Shimoda

Hi Oleksandr-san,

Hi, Shimoda-san

Quoted 6 lines
Quoted 2 lines

From: Oleksandr, Sent: Thursday, August 29, 2019 7:56 PM

<snip>

Quoted 3 lines
Quoted 3 lines

About this hardware handling, this patch seems good to me. But, since I'm not familiar about Xen passthrough framework, I think I cannot add my Reviewed-by tag into this patch. What do you think?

Quoted 6 lines
Quoted 5 lines

I am not completely sure regarding that, but I have seen cases when the people give their R-b not for the entire patch, but for a part of it.

Especially, when patch touches many sub-systems/archs. Would you mind if I specify what your R-b covers? -> [for the IPMMU H/W bits]

I got it. When you submit v4 patch, I'll review and send such a tag.

Thank you.

Signature · 9 lines
Regards,

Oleksandr Tyshchenko


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
author
#36 lore raw

Re: [Xen-devel] [PATCH V3 8/8] iommu/arm: Add Renesas IPMMU-VMSA support

In reply to Oleksandr Tyshchenko

Hi, all

Quoted 29 lines
xen/drivers/passthrough/arm/ipmmu-vmsa.c new file +29 -0
+
+static __init int ipmmu_init(struct dt_device_node *node, const void *data)
+{
+    int ret;
+
+    /*
+     * Even if the device can't be initialized, we don't want to give
+     * the IPMMU device to dom0.
+     */
+    dt_device_set_used_by(node, DOMID_XEN);
+
+    if ( !iommu_hap_pt_share )
+    {
+        printk_once(XENLOG_ERR "ipmmu: P2M table must always be shared between the CPU and the IPMMU\n");
+        return -EINVAL;
+    }
+
+    if ( !ipmmu_stage2_supported() )
+    {
+        printk_once(XENLOG_ERR "ipmmu: P2M sharing is not supported in current SoC revision\n");
+        return -ENODEV;
+    }
+    else
+    {
+        /*
+         * As 4-level translation table is not supported in IPMMU, we need
+         * to check IPA size used for P2M table beforehand to be sure it is
+         * 3-level and the IPMMU will be able to use it.
+         *

xen/drivers/passthrough/arm/ipmmu-vmsa.c line 1332

Quoted 10 lines
xen/drivers/passthrough/arm/ipmmu-vmsa.c new file +10 -0
+         * TODO: First initialize the IOMMU and gather the requirements and
+         * then initialize the P2M. In the P2M code, take into the account
+         * the IOMMU requirements and restrict "pa_range" if necessary.
+         */
+        if ( IPMMU_MAX_P2M_IPA_BITS < p2m_ipa_bits )
+        {
+            printk_once(XENLOG_ERR "ipmmu: P2M IPA size is not supported (P2M=%u IPMMU=%u)!\n",
+                        p2m_ipa_bits, IPMMU_MAX_P2M_IPA_BITS);
+            return -ENODEV;
+        }

There is a patch in ML which is intended to address this TODO:

https://lists.xenproject.org/archives/html/xen-devel/2019-08/msg02237.html

Signature · 9 lines
Regards,

Oleksandr Tyshchenko


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#37 lore raw

Re: [Xen-devel] [PATCH V3 8/8] iommu/arm: Add Renesas IPMMU-VMSA support

In reply to Oleksandr Tyshchenko

Hi Oleksandr,

xen/arch/arm/platforms/Kconfig config RCAR3 · line 28

Quoted 9 lines · Oleksandr Tyshchenko
xen/arch/arm/platforms/Kconfig +1 -0
index bc0e9cd..c93a6b2 100644
@@ -25,6 +25,7 @@ config RCAR3
  	bool "Renesas RCar3 support"
  	depends on ARM_64
  	select HAS_SCIF
+	select IPMMU_VMSA

As discussed previously, I think the IPMMU driver should be merged as tech preview for a couple of release to allow more users to test before we mark it as supported.

Based on this, I would not advise to select IPMMU_VMSA by default as user may not want to use tech preview code by default. Instead I would only select if EXPERT is set.

Quoted 25 lines
xen/arch/arm/platforms/Kconfig +0 -0
  	---help---
  	Enable all the required drivers for Renesas RCar3
 
xen/drivers/passthrough/Kconfig +13 -0
index a3c0649..47eadb4 100644
@@ -12,4 +12,17 @@ config ARM_SMMU
 
  	  Say Y here if your SoC includes an IOMMU device implementing the
  	  ARM SMMU architecture.
+
+config IPMMU_VMSA
+	bool "Renesas IPMMU-VMSA found in R-Car Gen3 SoCs"
+	default n
+	depends on ARM_64
+	---help---
+	  Support for implementations of the Renesas IPMMU-VMSA found
+	  in R-Car Gen3 SoCs.
+
+	  Say Y here if you are using newest R-Car Gen3 SoCs revisions
+	  (H3 ES3.0, M3-W+, etc) which IPMMU hardware supports stage 2
+	  translation table format and is able to use CPU's P2M table as is.
+
  endif

[...]

xen/drivers/passthrough/Kconfig

Quoted 4 lines
xen/drivers/passthrough/Kconfig +4 -0
+    /* Wait until the Root device has been registered for sure. */
+    if ( !mmu->root )
+    {
+        dev_err(&node->dev, "Root IPMMU hasn't been registered yet\n");

This is a bit odd to throw an error if we are going to defer the probe.

Quoted 3 lines
xen/drivers/passthrough/Kconfig +3 -0
+        ret = -EAGAIN;
+        goto out;
+    }

[...]

Quoted 4 lines
xen/drivers/passthrough/Kconfig +4 -0
+static __init bool ipmmu_stage2_supported(void)
+{
+    struct dt_device_node *np;
+    uint64_t addr, size;

xen/drivers/passthrough/Kconfig

Quoted 10 lines
xen/drivers/passthrough/Kconfig +10 -0
+    void __iomem *base;
+    uint32_t product, cut;
+    static enum
+    {
+        UNKNOWN,
+        SUPPORTED,
+        NOTSUPPORTED
+    } stage2_supported = UNKNOWN;
+
+    /* Use the flag to avoid checking for the compatibility more then once. */

There are only one IOMMU root that will always be initialized first. So can't you move this code in the root IOMMU path?

Quoted 69 lines
xen/drivers/passthrough/Kconfig +69 -0
+    switch ( stage2_supported )
+    {
+    case SUPPORTED:
+        return true;
+
+    case NOTSUPPORTED:
+        return false;
+
+    case UNKNOWN:
+    default:
+        stage2_supported = NOTSUPPORTED;
+        break;
+    }
+
+    np = dt_find_compatible_node(NULL, NULL, "renesas,prr");
+    if ( !np )
+    {
+        printk(XENLOG_ERR "ipmmu: Failed to find PRR node\n");
+        return false;
+    }
+
+    if ( dt_device_get_address(np, 0, &addr, &size) )
+    {
+        printk(XENLOG_ERR "ipmmu: Failed to get PRR MMIO\n");
+        return false;
+    }
+
+    base = ioremap_nocache(addr, size);
+    if ( !base )
+    {
+        printk(XENLOG_ERR "ipmmu: Failed to ioremap PRR MMIO\n");
+        return false;
+    }
+
+    product = readl(base);
+    cut = product & RCAR_CUT_MASK;
+    product &= RCAR_PRODUCT_MASK;
+
+    switch ( product )
+    {
+    case RCAR_PRODUCT_H3:
+    case RCAR_PRODUCT_M3W:
+        if ( cut >= RCAR_CUT_VER30 )
+            stage2_supported = SUPPORTED;
+        break;
+
+    case RCAR_PRODUCT_M3N:
+        stage2_supported = SUPPORTED;
+        break;
+
+    default:
+        printk(XENLOG_ERR "ipmmu: Unsupported SoC version\n");
+        break;
+    }
+
+    iounmap(base);
+
+    return stage2_supported == SUPPORTED;
+}
+
+static const struct dt_device_match ipmmu_dt_match[] __initconst =
+{
+    DT_MATCH_COMPATIBLE("renesas,ipmmu-r8a7795"),
+    DT_MATCH_COMPATIBLE("renesas,ipmmu-r8a77965"),
+    DT_MATCH_COMPATIBLE("renesas,ipmmu-r8a7796"),
+    { /* sentinel */ },
+};
+
+static __init int ipmmu_init(struct dt_device_node *node, const void *data)

xen/drivers/passthrough/Kconfig

Quoted 10 lines
xen/drivers/passthrough/Kconfig +10 -0
+{
+    int ret;
+
+    /*
+     * Even if the device can't be initialized, we don't want to give
+     * the IPMMU device to dom0.
+     */
+    dt_device_set_used_by(node, DOMID_XEN);
+
+    if ( !iommu_hap_pt_share )

iommu_hap_pt_share will soon be hardcoded to true on Arm (see [1]). Even without the patch, the value of iommu_hap_pt_share should be ignored as done by the SMMU.

xen/drivers/passthrough/Kconfig

Quoted 9 lines
xen/drivers/passthrough/Kconfig +9 -0
+    {
+        printk_once(XENLOG_ERR "ipmmu: P2M table must always be shared between the CPU and the IPMMU\n");
+        return -EINVAL;
+    }
+
+    if ( !ipmmu_stage2_supported() )
+    {
+        printk_once(XENLOG_ERR "ipmmu: P2M sharing is not supported in current SoC revision\n");
+        return -ENODEV;

The if part is returning so...

xen/drivers/passthrough/Kconfig

Quoted 2 lines
xen/drivers/passthrough/Kconfig +2 -0
+    }
+    else

... the else part is not necessary removing one layer of indentation.

Quoted 34 lines
xen/drivers/passthrough/Kconfig +33 -0
+    {
+        /*
+         * As 4-level translation table is not supported in IPMMU, we need
+         * to check IPA size used for P2M table beforehand to be sure it is
+         * 3-level and the IPMMU will be able to use it.
+         *
+         * TODO: First initialize the IOMMU and gather the requirements and
+         * then initialize the P2M. In the P2M code, take into the account
+         * the IOMMU requirements and restrict "pa_range" if necessary.
+         */
+        if ( IPMMU_MAX_P2M_IPA_BITS < p2m_ipa_bits )
+        {
+            printk_once(XENLOG_ERR "ipmmu: P2M IPA size is not supported (P2M=%u IPMMU=%u)!\n",
+                        p2m_ipa_bits, IPMMU_MAX_P2M_IPA_BITS);
+            return -ENODEV;
+        }
+    }
+
+    ret = ipmmu_probe(node);
+    if ( ret )
+    {
+        dev_err(&node->dev, "Failed to init IPMMU (%d)\n", ret);
+        return ret;
+    }
+
+    iommu_set_ops(&ipmmu_iommu_ops);
+
+    return 0;
+}
+
+DT_DEVICE_START(ipmmu, "Renesas IPMMU-VMSA", DEVICE_IOMMU)
+    .dt_match = ipmmu_dt_match,
+    .init = ipmmu_init,

xen/drivers/passthrough/Kconfig

Quoted 10 lines
xen/drivers/passthrough/Kconfig +10 -0
+DT_DEVICE_END
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */

[1] <[email protected]>

Signature · 6 lines
Julien Grall

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
author
#38 lore raw

Re: [Xen-devel] [PATCH V3 8/8] iommu/arm: Add Renesas IPMMU-VMSA support

Quoted 1 line · Julien Grall

Hi Oleksandr,

Hi Julien

Quoted 13 lines · Oleksandr Tyshchenko
Quoted 9 lines
xen/arch/arm/platforms/Kconfig +0 -0
index bc0e9cd..c93a6b2 100644
@@ -25,6 +25,7 @@ config RCAR3

      bool "Renesas RCar3 support"
      depends on ARM_64
      select HAS_SCIF
+    select IPMMU_VMSA

As discussed previously, I think the IPMMU driver should be merged as tech preview for a couple of release to allow more users to test before we mark it as supported.

Yes

Quoted 2 lines

Based on this, I would not advise to select IPMMU_VMSA by default as user may not want to use tech preview code by default. Instead I would only select if EXPERT is set.

Agree. Will do.

Quoted 13 lines

xen/drivers/passthrough/Kconfig config ARM_SMMU · line 17

Quoted 13 lines

      ---help---
      Enable all the required drivers for Renesas RCar3
  diff --git a/xen/drivers/passthrough/Kconfig b/xen/drivers/passthrough/Kconfig index a3c0649..47eadb4 100644

xen/drivers/passthrough/Kconfig +0 -0
@@ -12,4 +12,17 @@ config ARM_SMMU

          Say Y here if your SoC includes an IOMMU device implementing the ARM SMMU architecture.

xen/drivers/passthrough/Kconfig +3 -0
+
+config IPMMU_VMSA
+    bool "Renesas IPMMU-VMSA found in R-Car Gen3 SoCs"

Shall I add EXPERT here also?

bool "Renesas IPMMU-VMSA found in R-Car Gen3 SoCs" if EXPERT = "y"

Quoted 15 lines
Quoted 12 lines

+    default n
+    depends on ARM_64
+    ---help---
+      Support for implementations of the Renesas IPMMU-VMSA found +      in R-Car Gen3 SoCs.
+
+      Say Y here if you are using newest R-Car Gen3 SoCs revisions +      (H3 ES3.0, M3-W+, etc) which IPMMU hardware supports stage 2 +      translation table format and is able to use CPU's P2M table as is.
+
  endif

[...]

Quoted 6 lines
Quoted 4 lines

+    /* Wait until the Root device has been registered for sure. */ +    if ( !mmu->root )
+    {
+        dev_err(&node->dev, "Root IPMMU hasn't been registered yet\n");

This is a bit odd to throw an error if we are going to defer the probe.

Agree. Will remove.

Quoted 10 lines
Quoted 10 lines
xen/drivers/passthrough/Kconfig +9 -0
+static __init bool ipmmu_stage2_supported(void)
+{
+    struct dt_device_node *np;
+    uint64_t addr, size;
+    void __iomem *base;
+    uint32_t product, cut;
+    static enum
+    {
+        UNKNOWN,
Quoted 8 lines
Quoted 6 lines

+        SUPPORTED,
+        NOTSUPPORTED
+    } stage2_supported = UNKNOWN;
+
+    /* Use the flag to avoid checking for the compatibility more then once. */

There are only one IOMMU root that will always be initialized first. So can't you move this code in the root IOMMU path?

Actually, I can. Good point.

Quoted 76 lines
Quoted 76 lines

+    switch ( stage2_supported )
+    {
+    case SUPPORTED:
+        return true;

xen/drivers/passthrough/Kconfig +65 -0
+
+    case NOTSUPPORTED:
+        return false;
+
+    case UNKNOWN:
+    default:
+        stage2_supported = NOTSUPPORTED;
+        break;
+    }
+
+    np = dt_find_compatible_node(NULL, NULL, "renesas,prr");
+    if ( !np )
+    {
+        printk(XENLOG_ERR "ipmmu: Failed to find PRR node\n");
+        return false;
+    }
+
+    if ( dt_device_get_address(np, 0, &addr, &size) )
+    {
+        printk(XENLOG_ERR "ipmmu: Failed to get PRR MMIO\n");
+        return false;
+    }
+
+    base = ioremap_nocache(addr, size);
+    if ( !base )
+    {
+        printk(XENLOG_ERR "ipmmu: Failed to ioremap PRR MMIO\n");
+        return false;
+    }
+
+    product = readl(base);
+    cut = product & RCAR_CUT_MASK;
+    product &= RCAR_PRODUCT_MASK;
+
+    switch ( product )
+    {
+    case RCAR_PRODUCT_H3:
+    case RCAR_PRODUCT_M3W:
+        if ( cut >= RCAR_CUT_VER30 )
+            stage2_supported = SUPPORTED;
+        break;
+
+    case RCAR_PRODUCT_M3N:
+        stage2_supported = SUPPORTED;
+        break;
+
+    default:
+        printk(XENLOG_ERR "ipmmu: Unsupported SoC version\n");
+        break;
+    }
+
+    iounmap(base);
+
+    return stage2_supported == SUPPORTED;
+}
+
+static const struct dt_device_match ipmmu_dt_match[] __initconst =
+{
+    DT_MATCH_COMPATIBLE("renesas,ipmmu-r8a7795"),
+    DT_MATCH_COMPATIBLE("renesas,ipmmu-r8a77965"),
+    DT_MATCH_COMPATIBLE("renesas,ipmmu-r8a7796"),
+    { /* sentinel */ },
+};
+
+static __init int ipmmu_init(struct dt_device_node *node, const void 

*data)

xen/drivers/passthrough/Kconfig +4 -0
+{
+    int ret;
+
+    /*
Quoted 8 lines
Quoted 6 lines

+     * Even if the device can't be initialized, we don't want to give +     * the IPMMU device to dom0.
+     */
+    dt_device_set_used_by(node, DOMID_XEN);
+
+    if ( !iommu_hap_pt_share )

iommu_hap_pt_share will soon be hardcoded to true on Arm (see [1]). Even without the patch, the value of iommu_hap_pt_share should be ignored as done by the SMMU.

I got it. Will ignore.

Quoted 14 lines
Quoted 13 lines

+    {
+        printk_once(XENLOG_ERR "ipmmu: P2M table must always be shared between the CPU and the IPMMU\n");
+        return -EINVAL;
+    }
+
+    if ( !ipmmu_stage2_supported() )
+    {
+        printk_once(XENLOG_ERR "ipmmu: P2M sharing is not supported in current SoC revision\n");
+        return -ENODEV;

Quoted 6 lines

The if part is returning so...

Quoted 2 lines

+    }
+    else

... the else part is not necessary removing one layer of indentation.

ok

Signature · 9 lines
Regards,

Oleksandr Tyshchenko


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#39 lore raw

Re: [Xen-devel] [PATCH V3 8/8] iommu/arm: Add Renesas IPMMU-VMSA support

Hi,

Quoted 15 lines · Oleksandr
Quoted 4 lines · Julien Grall
Quoted 4 lines

      ---help---
      Enable all the required drivers for Renesas RCar3
  diff --git a/xen/drivers/passthrough/Kconfig b/xen/drivers/passthrough/Kconfig index a3c0649..47eadb4 100644

Quoted 8 lines

xen/drivers/passthrough/Kconfig config ARM_SMMU · line 17

Quoted 8 lines
xen/drivers/passthrough/Kconfig +0 -0
@@ -12,4 +12,17 @@ config ARM_SMMU

          Say Y here if your SoC includes an IOMMU device implementing the ARM SMMU architecture.

xen/drivers/passthrough/Kconfig +3 -0
+
+config IPMMU_VMSA
+    bool "Renesas IPMMU-VMSA found in R-Car Gen3 SoCs"

Shall I add EXPERT here also?

Good question. Probably, but then I am wondering if the EXPERT = "y" would actually be sufficient.

Cheers,

Signature · 6 lines
Julien Grall

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
author
#40 lore raw

Re: [Xen-devel] [PATCH V3 8/8] iommu/arm: Add Renesas IPMMU-VMSA support

Quoted 1 line · Julien Grall

Hi,

Hi Julien

Quoted 18 lines · Oleksandr
Quoted 7 lines · Julien Grall
Quoted 6 lines
Quoted 6 lines

      ---help---
      Enable all the required drivers for Renesas RCar3
  diff --git a/xen/drivers/passthrough/Kconfig b/xen/drivers/passthrough/Kconfig index a3c0649..47eadb4 100644

xen/drivers/passthrough/Kconfig +0 -0
Quoted 8 lines
Quoted 6 lines

xen/drivers/passthrough/Kconfig config ARM_SMMU · line 17

Quoted 6 lines
xen/drivers/passthrough/Kconfig +0 -0
@@ -12,4 +12,17 @@ config ARM_SMMU

          Say Y here if your SoC includes an IOMMU device implementing the ARM SMMU architecture.

xen/drivers/passthrough/Kconfig +3 -0
+
+config IPMMU_VMSA
+    bool "Renesas IPMMU-VMSA found in R-Car Gen3 SoCs"

Shall I add EXPERT here also?

Quoted 1 line

Good question. Probably, but then I am wondering if the EXPERT = "y" would actually be sufficient.

Should be in both places, here and where you pointed (config RCAR3).

Signature · 9 lines
Regards,

Oleksandr Tyshchenko


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel