Skip to contentSkip to conversation

iommu/arm: Add Renesas IPMMU-VMSA support + Linux's iommu_fwspec

PATCH In review Oleksandr Tyshchenko 4 participants 23 messages Sep 26, 2019 – Oct 1, 2019

lore
Share & tools
lore
#0 lore raw

v6 cover iommu/arm: Add Renesas IPMMU-VMSA support + Linux's iommu_fwspec

From: Oleksandr Tyshchenko <[email protected]>

The purpose of this patch series is to add IPMMU-VMSA support to Xen on ARM.

Besides new IOMMU driver, this series contains "iommu_fwspec" support and new API iommu_add_dt_device() for adding DT device to IOMMU and many other things.

The IPMMU-VMSA is VMSA-compatible I/O Memory Management Unit (IOMMU) which provides address translation and access protection functionalities to processing units and interconnect networks.

Please note, this driver is supposed to work only with newest R-Car Gen3 SoCs revisions which IPMMU hardware supports stage 2 translation table format and is able to use CPU's P2M table as is if one is 3-level page table (up to 40 bit IPA).

----------
This driver is based on Linux's IPMMU-VMSA driver from Renesas BSP:
https://git.kernel.org/pub/scm/linux/kernel/git/horms/renesas-bsp.git/tree/drivers/iommu/ipmmu-vmsa.c?h=v4.14.75-ltsi/rcar-3.9.6
and Xen's SMMU driver:
xen/drivers/passthrough/arm/smmu.c

Although Xen driver has a lot in common with Linux driver, it is not a "direct ported" copy and should be treated as such.

The major differences compare to the Linux driver are:

1. Stage 1/Stage 2 translation. Linux driver supports Stage 1 translation only (with Stage 1 translation table format). It manages page table by itself. But Xen driver supports Stage 2 translation (with Stage 2 translation table format) to be able to share the P2M with the CPU. Stage 1 translation is always bypassed in Xen driver.

So, Xen driver is supposed to be used with newest R-Car Gen3 SoC revisions only (H3 ES3.0, M3-W+, etc.) which IPMMU H/W supports stage 2 translation table format.

2. AArch64 support. Linux driver uses VMSAv8-32 mode, while Xen driver enables Armv8 VMSAv8-64 mode to cover up to 40 bit input address.

3. Context bank (sets of page table) usage. In Xen, each context bank is mapped to one Xen domain. So, all devices being pass throughed to the same Xen domain share the same context bank.

4. IPMMU device tracking. In Xen, all IPMMU devices are managed by single driver instance. So, driver uses global list to keep track of registered devices.

----------
Series was tested on R-Car Gen3 H3 ES3.0/M3N ES1.0 based boards using current staging (babde47 introduce a 'passthrough' configuration option to xl.cfg...) in a system with several DMA masters being assigned to different guest domains. Guest domain reboot, destroy/create are functional.

You can find the whole series (and [1]) here:
repo: https://github.com/otyshchenko1/xen.git branch: ipmmu_upstream6

You can find previous discussions here:
[V1] https://lists.xenproject.org/archives/html/xen-devel/2019-06/msg01755.html
[V2] https://lists.xenproject.org/archives/html/xen-devel/2019-08/msg00253.html
[V3] https://lists.xenproject.org/archives/html/xen-devel/2019-08/msg01948.html
[V4] https://lists.xenproject.org/archives/html/xen-devel/2019-09/msg01255.html
[V5] https://lists.xenproject.org/archives/html/xen-devel/2019-09/msg02282.html

[1] Also, please note, there is a patch in ML which is intended to address the main TODO in the IPMMU-VMSA driver:
https://lists.xenproject.org/archives/html/xen-devel/2019-09/msg02304.html

For convenience:
1. iommu/arm: Add iommu_helpers.c file to keep common for IOMMUs stuff A-b
2. iommu/arm: Add ability to handle deferred probing request R-b
3. xen/common: Introduce _xrealloc function
-none-
4. xen/common: Introduce xrealloc_flex_struct() helper macros R-b
5. iommu/arm: Add lightweight iommu_fwspec support
A-b
6. iommu: Order the headers alphabetically in device_tree.c A-b
7. iommu/arm: Introduce iommu_add_dt_device API
-none-
8. iommu/arm: Add Renesas IPMMU-VMSA support
A-b/R-b

Oleksandr Tyshchenko (8):
iommu/arm: Add iommu_helpers.c file to keep common for IOMMUs stuff iommu/arm: Add ability to handle deferred probing request xen/common: Introduce _xrealloc function
xen/common: Introduce xrealloc_flex_struct() helper macros iommu/arm: Add lightweight iommu_fwspec support
iommu: Order the headers alphabetically in device_tree.c iommu/arm: Introduce iommu_add_dt_device API
iommu/arm: Add Renesas IPMMU-VMSA support

 xen/arch/arm/domain_build.c                 |   25 +-
 xen/arch/arm/platforms/Kconfig              |    1 +
 xen/common/xmalloc_tlsf.c                   |  114 ++-
 xen/drivers/passthrough/Kconfig             |   13 +
 xen/drivers/passthrough/arm/Makefile        |    3 +-
 xen/drivers/passthrough/arm/iommu.c         |   56 +-
 xen/drivers/passthrough/arm/iommu_fwspec.c  |   97 ++
 xen/drivers/passthrough/arm/iommu_helpers.c |   84 ++
 xen/drivers/passthrough/arm/ipmmu-vmsa.c    | 1333 +++++++++++++++++++++++++++
 xen/drivers/passthrough/arm/smmu.c          |   48 +-
 xen/drivers/passthrough/device_tree.c       |   83 +-
 xen/include/asm-arm/device.h                |    7 +-
 xen/include/asm-arm/iommu.h                 |   10 +
 xen/include/asm-arm/iommu_fwspec.h          |   68 ++
 xen/include/xen/device_tree.h               |    7 +
 xen/include/xen/iommu.h                     |   21 +
 xen/include/xen/xmalloc.h                   |   13 +
 17 files changed, 1907 insertions(+), 76 deletions(-)
 create mode 100644 xen/drivers/passthrough/arm/iommu_fwspec.c
 create mode 100644 xen/drivers/passthrough/arm/iommu_helpers.c
 create mode 100644 xen/drivers/passthrough/arm/ipmmu-vmsa.c
 create mode 100644 xen/include/asm-arm/iommu_fwspec.h
Signature · 7 lines
2.7.4


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#1 lore raw

PATCH v6 1/8 iommu/arm: Add iommu_helpers.c file to keep common for IOMMUs stuff

From: Oleksandr Tyshchenko <[email protected]>

Introduce a separate file to keep various helpers which could be used by more than one IOMMU driver in order not to duplicate code.

The first candidates to be moved to the new file are SMMU driver's "map_page/unmap_page" callbacks. These callbacks neither contain any SMMU specific info nor perform any SMMU specific actions and are going to be the same across all IOMMU drivers which H/W IP shares P2M with the CPU like SMMU does.

So, move callbacks to iommu_helpers.c for the upcoming IPMMU driver to be able to re-use them.

---
Changes V3 -> V4:
- added Julien's A-b
- fixed Grammatical error
- clarified comment in a header

Changes V2 -> V3:
- put headers in alphabetical order
- retained Copyright from the SMMU code

---
 xen/drivers/passthrough/arm/Makefile        |  2 +-
 xen/drivers/passthrough/arm/iommu_helpers.c | 84 +++++++++++++++++++++++++++++
 xen/drivers/passthrough/arm/smmu.c          | 48 +----------------
 xen/include/asm-arm/iommu.h                 | 10 ++++
 4 files changed, 97 insertions(+), 47 deletions(-)
 create mode 100644 xen/drivers/passthrough/arm/iommu_helpers.c
xen/drivers/passthrough/arm/Makefile +1 -1
index b3efcfd..4abb87a 100644
@@ -1,2 +1,2 @@
-obj-y += iommu.o
+obj-y += iommu.o iommu_helpers.o
 obj-$(CONFIG_ARM_SMMU) += smmu.o
xen/drivers/passthrough/arm/iommu_helpers.c new file +84 -0

84 lines not on this page · raw message

xen/drivers/passthrough/arm/smmu.c +2 -46

60 lines not on this page · raw message

xen/include/asm-arm/iommu.h +10 -0
index 77a94b2..937edc8 100644
@@ -23,6 +23,16 @@ struct arch_iommu
 const struct iommu_ops *iommu_get_ops(void);
 void iommu_set_ops(const struct iommu_ops *ops);
 
+/*
+ * The mapping helpers below should only be used if P2M Table is shared
+ * between the CPU and the IOMMU.
+ */
+int __must_check arm_iommu_map_page(struct domain *d, dfn_t dfn, mfn_t mfn,
+                                    unsigned int flags,
+                                    unsigned int *flush_flags);
+int __must_check arm_iommu_unmap_page(struct domain *d, dfn_t dfn,
+                                      unsigned int *flush_flags);
+
 #endif /* __ARCH_ARM_IOMMU_H__ */
 
 /*
Signature · 7 lines
2.7.4


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#2 lore raw

PATCH v6 2/8 iommu/arm: Add ability to handle deferred probing request

In reply to Oleksandr Tyshchenko

From: Oleksandr Tyshchenko <[email protected]>

This patch adds minimal required support to General IOMMU framework to be able to handle a case when IOMMU driver requesting deferred probing for a device.

In order not to pull Linux's error code (-EPROBE_DEFER) to Xen we have chosen -EAGAIN to be used for indicating that device probing is deferred.

This is needed for the upcoming IPMMU driver which may request deferred probing depending on what device will be probed the first (there is some dependency between these devices, Root device must be registered before Cache devices. If not the case, driver will deny further Cache device probes until Root device is registered). As we can't guarantee a fixed pre-defined order for the device nodes in DT, we need to be ready for the situation where devices being probed in "any" order.

---
Changes V4 -> V5:
- added Julien's R-b
- fixed Grammatical error

Changes V3 -> V4:
- moved changes related to the "headers ordering" to a separate patch
- added explaination why domain_list is re-used
- removed the unnecessary pair of outermost parentheses when checking the list_empty

Changes V2 -> V3:
- removed deferred_probe field from struct dt_device_node, re-used domain_list instead
- documented domain_list usage
- added ASSERT to check that np->domain_list is empty before re-using it
- put deferred_probe_list to init section
- used more strict logic regarding processing devices in the deferred list
- added more comments to code
- put headers in alphabetical order

---
 xen/drivers/passthrough/arm/iommu.c | 56 +++++++++++++++++++++++++++++++++++--
 xen/include/asm-arm/device.h        |  6 +++-
 xen/include/xen/device_tree.h       |  7 +++++
 3 files changed, 66 insertions(+), 3 deletions(-)
xen/drivers/passthrough/arm/iommu.c +54 -2
index 2678ef5..76e1152 100644
@@ -21,6 +21,12 @@
 
 #include <asm/device.h>
 
+/*
+ * Deferred probe list is used to keep track of devices for which driver
+ * requested deferred probing (returned -EAGAIN).
+ */
+static __initdata LIST_HEAD(deferred_probe_list);
+
 static const struct iommu_ops *iommu_ops;
 
 const struct iommu_ops *iommu_get_ops(void)
@@ -43,7 +49,7 @@ void __init iommu_set_ops(const struct iommu_ops *ops)
 
 int __init iommu_hardware_setup(void)
 {
-    struct dt_device_node *np;
+    struct dt_device_node *np, *tmp;
     int rc;
     unsigned int num_iommus = 0;
 
@@ -52,6 +58,21 @@ int __init iommu_hardware_setup(void)
         rc = device_init(np, DEVICE_IOMMU, NULL);
         if ( !rc )
             num_iommus++;
+        else if ( rc == -EAGAIN )
+        {
+            /*
+             * Nobody should use device's domain_list at such early stage,
+             * so we can re-use it to link the device in the deferred list to
+             * avoid introducing extra list_head field in struct dt_device_node.
+             */
+            ASSERT(list_empty(&np->domain_list));
+
+            /*
+             * Driver requested deferred probing, so add this device to
+             * the deferred list for further processing.
+             */
+            list_add(&np->domain_list, &deferred_probe_list);
+        }
         /*
          * Ignore the following error codes:
          *   - EBADF: Indicate the current is not an IOMMU
@@ -62,7 +83,38 @@ int __init iommu_hardware_setup(void)
             return rc;
     }
 
-    return ( num_iommus > 0 ) ? 0 : -ENODEV;
+    /* Return immediately if there are no initialized devices. */
+    if ( !num_iommus )
+        return list_empty(&deferred_probe_list) ? -ENODEV : -EAGAIN;
+
+    rc = 0;
+
+    /*
+     * Process devices in the deferred list if it is not empty.
+     * Check that at least one device is initialized at each loop, otherwise
+     * we may get an infinite loop. Also stop processing if we got an error
+     * other than -EAGAIN.
+     */
+    while ( !list_empty(&deferred_probe_list) && num_iommus )
+    {
+        num_iommus = 0;
+
+        list_for_each_entry_safe ( np, tmp, &deferred_probe_list, domain_list )
+        {
+            rc = device_init(np, DEVICE_IOMMU, NULL);
+            if ( !rc )
+            {
+                num_iommus++;
+
+                /* Remove initialized device from the deferred list. */
+                list_del_init(&np->domain_list);
+            }
+            else if ( rc != -EAGAIN )
+                return rc;
+        }
+    }
+
+    return rc;
 }
 
 void __hwdom_init arch_iommu_check_autotranslated_hwdom(struct domain *d)
xen/include/asm-arm/device.h +5 -1
index 63a0f36..ee1c3bc 100644
@@ -44,7 +44,11 @@ struct device_desc {
     enum device_class class;
     /* List of devices supported by this driver */
     const struct dt_device_match *dt_match;
-    /* Device initialization */
+    /*
+     * Device initialization.
+     *
+     * -EAGAIN is used to indicate that device probing is deferred.
+     */
     int (*init)(struct dt_device_node *dev, const void *data);
 };
 
xen/include/xen/device_tree.h +7 -0
index 9a7a8f2..f2ad22b 100644
@@ -92,6 +92,13 @@ struct dt_device_node {
 
     /* IOMMU specific fields */
     bool is_protected;
+    /*
+     * The main purpose of this list is to link the structure in the list
+     * of devices assigned to domain.
+     *
+     * Boot code (iommu_hardware_setup) re-uses this list to link the structure
+     * in the list of devices for which driver requested deferred probing.
+     */
     struct list_head domain_list;
 
     struct device dev;
Signature · 7 lines
2.7.4


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#3 lore raw

PATCH v6 3/8 xen/common: Introduce _xrealloc function

In reply to Oleksandr Tyshchenko

From: Oleksandr Tyshchenko <[email protected]>

This patch introduces type-unsafe function which besides re-allocation handles the following corner cases:
1. if requested size is zero, it will behave like xfree
2. if incoming pointer is not valid (NULL or ZERO_BLOCK_PTR), it will behave like xmalloc

If both pointer and size are valid the function will re-allocate and copy only if requested size and alignment don't fit in already allocated space.

Subsequent patch will add type-safe helper macros.

CC Andrew Cooper <[email protected]>
CC George Dunlap <[email protected]>
CC Ian Jackson <[email protected]>
CC Jan Beulich <[email protected]>
CC Julien Grall <[email protected]>
CC Konrad Rzeszutek Wilk <[email protected]>
CC Stefano Stabellini <[email protected]>
CC Tim Deegan <[email protected]>
CC Wei Liu <[email protected]>
CC Paul Durrant <[email protected]>

---
Changes V5 -> V6:
- change uint32_t to unsigned int for "pad" in add_padding()
- change back to _xmalloc
- move tmp_size declaration/calculation to the corresponding block
- use "!" over "== 0"

Changes V4 -> V5:
- avoid possible truncation with allocations of 4GiB or above
- introduce helper functions add(strip)_padding to avoid duplicating the code
- omit the unnecessary casts, change u32 to uint32_t when moving the code
- use _xzalloc instead of _xmalloc to get the tail portion zeroed
- update pointer according to the requsted alignment
- compared against "size" instead of "tmp_size" for the allocations above PAGE_SIZE

Changes V3 -> V4:
- add check for the alignment compatibility
- properly detect current size (take into the account a possible fake alignment header)
- update comment in code/patch description

Changes RFC -> V3:
- behave like xmalloc if incoming pointer is ZERO_BLOCK_PTR or NULL
- return ZERO_BLOCK_PTR after xfree if requested size is zero
- add patch description
- use allocator internals to recognize current size of the incoming pointer
- do not re-allocate and copy if requested size fits in already allocated space

...

Original patch was initially posted by Sameer Goel:
https://lists.xen.org/archives/html/xen-devel/2017-06/msg00858.html

This could be considered as another attempt to add it:
https://www.mail-archive.com/[email protected]/msg21335.html

[As it was previously discussed with Julien in IRC]

The reason for this patch to be an RFC is that patch itself is not completely correct and I don't fully understand what/how should be done for this patch to be accepted. Or whether community even wants this to go in. So, to avoid bike shedding, the first target is to collect feedback.

For everyone who wants more details why this is needed and where used, please see next patch of this thread:
"iommu/arm: Add lightweight iommu_fwspec support"

In a nutshell, the upcoming "iommu_fwspec" support on ARM is going to use xrealloc to expand an array for device IDs. We really want to have "iommu_fwspec" support which will give us a generic abstract way to add new device to the IOMMU based on the generic IOMMU DT binding.

---
 xen/common/xmalloc_tlsf.c | 114 ++++++++++++++++++++++++++++++++++++++--------
 xen/include/xen/xmalloc.h |   1 +
 2 files changed, 97 insertions(+), 18 deletions(-)
xen/common/xmalloc_tlsf.c +96 -18
index 1e8d72d..7ab2b3b 100644
@@ -549,10 +549,40 @@ static void tlsf_init(void)
  * xmalloc()
  */
 
+static void *strip_padding(void *p)
+{
+    struct bhdr *b = p - BHDR_OVERHEAD;
+
+    if ( b->size & FREE_BLOCK )
+    {
+        p -= b->size & ~FREE_BLOCK;
+        b = p - BHDR_OVERHEAD;
+        ASSERT(!(b->size & FREE_BLOCK));
+    }
+
+    return p;
+}
+
+static void *add_padding(void *p, unsigned long align)
+{
+    unsigned int pad;
+
+    if ( (pad = -(long)p & (align - 1)) != 0 )
+    {
+        void *q = p + pad;
+        struct bhdr *b = q - BHDR_OVERHEAD;
+
+        ASSERT(q > p);
+        b->size = pad | FREE_BLOCK;
+        p = q;
+    }
+
+    return p;
+}
+
 void *_xmalloc(unsigned long size, unsigned long align)
 {
     void *p = NULL;
-    u32 pad;
 
     ASSERT(!in_irq());
 
@@ -573,14 +603,7 @@ void *_xmalloc(unsigned long size, unsigned long align)
         return xmalloc_whole_pages(size - align + MEM_ALIGN, align);
 
     /* Add alignment padding. */
-    if ( (pad = -(long)p & (align - 1)) != 0 )
-    {
-        char *q = (char *)p + pad;
-        struct bhdr *b = (struct bhdr *)(q - BHDR_OVERHEAD);
-        ASSERT(q > (char *)p);
-        b->size = pad | FREE_BLOCK;
-        p = q;
-    }
+    p = add_padding(p, align);
 
     ASSERT(((unsigned long)p & (align - 1)) == 0);
     return p;
@@ -593,10 +616,71 @@ void *_xzalloc(unsigned long size, unsigned long align)
     return p ? memset(p, 0, size) : p;
 }
 
-void xfree(void *p)
+void *_xrealloc(void *ptr, unsigned long size, unsigned long align)
 {
-    struct bhdr *b;
+    unsigned long curr_size;
+    void *p;
+
+    if ( !size )
+    {
+        xfree(ptr);
+        return ZERO_BLOCK_PTR;
+    }
 
+    if ( ptr == NULL || ptr == ZERO_BLOCK_PTR )
+        return _xmalloc(size, align);
+
+    ASSERT(!(align & (align - 1)));
+    if ( align < MEM_ALIGN )
+        align = MEM_ALIGN;
+
+    if ( !((unsigned long)ptr & (PAGE_SIZE - 1)) )
+    {
+        curr_size = (unsigned long)PFN_ORDER(virt_to_page(ptr)) << PAGE_SHIFT;
+
+        if ( size <= curr_size && !((unsigned long)ptr & (align - 1)) )
+            return ptr;
+    }
+    else
+    {
+        unsigned long tmp_size;
+        struct bhdr *b;
+
+        tmp_size = size + align - MEM_ALIGN;
+
+        if ( tmp_size < PAGE_SIZE )
+            tmp_size = (tmp_size < MIN_BLOCK_SIZE) ? MIN_BLOCK_SIZE :
+                ROUNDUP_SIZE(tmp_size);
+
+        /* Strip alignment padding. */
+        p = strip_padding(ptr);
+
+        b = p - BHDR_OVERHEAD;
+        curr_size = b->size & BLOCK_SIZE_MASK;
+
+        if ( tmp_size <= curr_size )
+        {
+            /* Add alignment padding. */
+            p = add_padding(p, align);
+
+            ASSERT(!((unsigned long)p & (align - 1)));
+
+            return p;
+        }
+    }
+
+    p = _xmalloc(size, align);
+    if ( p )
+    {
+        memcpy(p, ptr, min(curr_size, size));
+        xfree(ptr);
+    }
+
+    return p;
+}
+
+void xfree(void *p)
+{
     if ( p == NULL || p == ZERO_BLOCK_PTR )
         return;
 
@@ -621,13 +705,7 @@ void xfree(void *p)
     }
 
     /* Strip alignment padding. */
-    b = (struct bhdr *)((char *)p - BHDR_OVERHEAD);
-    if ( b->size & FREE_BLOCK )
-    {
-        p = (char *)p - (b->size & ~FREE_BLOCK);
-        b = (struct bhdr *)((char *)p - BHDR_OVERHEAD);
-        ASSERT(!(b->size & FREE_BLOCK));
-    }
+    p = strip_padding(p);
 
     xmem_pool_free(p, xenpool);
 }
xen/include/xen/xmalloc.h +1 -0
index f075d2d..831152f 100644
@@ -51,6 +51,7 @@ extern void xfree(void *);
 /* Underlying functions */
 extern void *_xmalloc(unsigned long size, unsigned long align);
 extern void *_xzalloc(unsigned long size, unsigned long align);
+extern void *_xrealloc(void *ptr, unsigned long size, unsigned long align);
 
 static inline void *_xmalloc_array(
     unsigned long size, unsigned long align, unsigned long num)
Signature · 7 lines
2.7.4


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Reviewed-by on v6 3/8
#4 lore raw

Re: [Xen-devel] [PATCH V6 3/8] xen/common: Introduce _xrealloc function

Quoted 8 lines · Oleksandr Tyshchenko

From: Oleksandr Tyshchenko <[email protected]>

This patch introduces type-unsafe function which besides re-allocation handles the following corner cases:
1. if requested size is zero, it will behave like xfree
2. if incoming pointer is not valid (NULL or ZERO_BLOCK_PTR), it will behave like xmalloc

Quoted 7 lines

If both pointer and size are valid the function will re-allocate and copy only if requested size and alignment don't fit in already allocated space.

Subsequent patch will add type-safe helper macros.

preferably with two more cosmetics addressed (can surely be done while committing):

xen/common/xmalloc_tlsf.c static void tlsf_init(void) · line 554

Quoted 9 lines
xen/common/xmalloc_tlsf.c +3 -0
@@ -549,10 +549,40 @@ static void tlsf_init(void)
  * xmalloc()
  */
 
+static void *strip_padding(void *p)
+{
+    struct bhdr *b = p - BHDR_OVERHEAD;

Looks like this can (and hence should) be pointer to const.

Quoted 25 lines
xen/common/xmalloc_tlsf.c +17 -2
@@ -593,10 +616,71 @@ void *_xzalloc(unsigned long size, unsigned long align)
     return p ? memset(p, 0, size) : p;
 }
 
-void xfree(void *p)
+void *_xrealloc(void *ptr, unsigned long size, unsigned long align)
 {
-    struct bhdr *b;
+    unsigned long curr_size;
+    void *p;
+
+    if ( !size )
+    {
+        xfree(ptr);
+        return ZERO_BLOCK_PTR;
+    }
 
+    if ( ptr == NULL || ptr == ZERO_BLOCK_PTR )
+        return _xmalloc(size, align);
+
+    ASSERT(!(align & (align - 1)));
+    if ( align < MEM_ALIGN )
+        align = MEM_ALIGN;
+
+    if ( !((unsigned long)ptr & (PAGE_SIZE - 1)) )

xen/common/xmalloc_tlsf.c void *_xzalloc(unsigned long size, unsigned long align) · line 647

Quoted 10 lines
xen/common/xmalloc_tlsf.c +10 -0
+    {
+        curr_size = (unsigned long)PFN_ORDER(virt_to_page(ptr)) << PAGE_SHIFT;
+
+        if ( size <= curr_size && !((unsigned long)ptr & (align - 1)) )
+            return ptr;
+    }
+    else
+    {
+        unsigned long tmp_size;
+        struct bhdr *b;

Same here.

xen/common/xmalloc_tlsf.c void *_xzalloc(unsigned long size, unsigned long align) · line 649

Quoted 1 line
xen/common/xmalloc_tlsf.c +1 -0
+        tmp_size = size + align - MEM_ALIGN;

This could also be the initializer of the variable.

Jan

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

author
#5 lore raw

Re: [Xen-devel] [PATCH V6 3/8] xen/common: Introduce _xrealloc function

On 26.09.19 15:19, Jan Beulich wrote:

Hi, Jan

Quoted 9 lines · Oleksandr Tyshchenko
Quoted 8 lines

From: Oleksandr Tyshchenko <[email protected]>

This patch introduces type-unsafe function which besides re-allocation handles the following corner cases:
1. if requested size is zero, it will behave like xfree
2. if incoming pointer is not valid (NULL or ZERO_BLOCK_PTR), it will behave like xmalloc

Quoted 8 lines
Quoted 7 lines

If both pointer and size are valid the function will re-allocate and copy only if requested size and alignment don't fit in already allocated space.

Subsequent patch will add type-safe helper macros.

Thank you!

Quoted 9 lines

preferably with two more cosmetics addressed (can surely be done while committing):

Quoted 6 lines
xen/common/xmalloc_tlsf.c +0 -0
@@ -549,10 +549,40 @@ static void tlsf_init(void)
   * xmalloc()
   */
 
Quoted 4 lines

xen/common/xmalloc_tlsf.c static void tlsf_init(void) · line 554

Quoted 3 lines
xen/common/xmalloc_tlsf.c +3 -0
+static void *strip_padding(void *p)
+{
+    struct bhdr *b = p - BHDR_OVERHEAD;

Looks like this can (and hence should) be pointer to const.

agree

Quoted 30 lines
Quoted 30 lines
xen/common/xmalloc_tlsf.c +20 -2
@@ -593,10 +616,71 @@ void *_xzalloc(unsigned long size, unsigned long align)
      return p ? memset(p, 0, size) : p;
  }
 
-void xfree(void *p)
+void *_xrealloc(void *ptr, unsigned long size, unsigned long align)
  {
-    struct bhdr *b;
+    unsigned long curr_size;
+    void *p;
+
+    if ( !size )
+    {
+        xfree(ptr);
+        return ZERO_BLOCK_PTR;
+    }
 
+    if ( ptr == NULL || ptr == ZERO_BLOCK_PTR )
+        return _xmalloc(size, align);
+
+    ASSERT(!(align & (align - 1)));
+    if ( align < MEM_ALIGN )
+        align = MEM_ALIGN;
+
+    if ( !((unsigned long)ptr & (PAGE_SIZE - 1)) )
+    {
+        curr_size = (unsigned long)PFN_ORDER(virt_to_page(ptr)) << PAGE_SHIFT;
+
Quoted 8 lines

xen/common/xmalloc_tlsf.c void *_xzalloc(unsigned long size, unsigned long align) · line 647

Quoted 7 lines
xen/common/xmalloc_tlsf.c +7 -0
+        if ( size <= curr_size && !((unsigned long)ptr & (align - 1)) )
+            return ptr;
+    }
+    else
+    {
+        unsigned long tmp_size;
+        struct bhdr *b;

Same here.

agree

Quoted 3 lines

xen/common/xmalloc_tlsf.c void *_xzalloc(unsigned long size, unsigned long align) · line 649

Quoted 2 lines
xen/common/xmalloc_tlsf.c +1 -0
+        tmp_size = size + align - MEM_ALIGN;

This could also be the initializer of the variable.

agree

Signature · 9 lines
Regards,

Oleksandr Tyshchenko


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#6 lore raw

PATCH v6 4/8 xen/common: Introduce xrealloc_flex_struct() helper macros

In reply to Oleksandr Tyshchenko

From: Oleksandr Tyshchenko <[email protected]>

This patch introduces type-safe helper macros to re-allocate space for a structure with a flexible array of typed objects.

For example, if we need to re-size the "data" array:

   struct arrlen
   {
      size_t len;
      int data[];
   };

We can use the proposed macros in the following way:

   new_ptr = realloc_flex_struct(old_ptr, data, nr_elem);

where nr_elem is the desired number of elements.

Subsequent patch will use this macros.

Also, while here, introduce xmalloc(xzalloc)_flex_struct() to allocate space for a structure with a flexible array of typed objects.

CC Andrew Cooper <[email protected]>
CC George Dunlap <[email protected]>
CC Ian Jackson <[email protected]>
CC Julien Grall <[email protected]>
CC Konrad Rzeszutek Wilk <[email protected]>
CC Stefano Stabellini <[email protected]>
CC Tim Deegan <[email protected]>
CC Wei Liu <[email protected]>

---
Changes V4 -> V5:
- clarified patch description (data[] instead of data[1])
- introduced xzalloc_flex_struct()
- added Jan's R-b
- added missing parentheses around the entire constructs

Changes V3 -> V4:
- clarified patch description
- modified to not use leading underscores
- removed unnecessary pair of outermost parentheses
- modified to use "nr" instead of "len"
- placed xmalloc_flex_struct before xrealloc_flex_struct
- simplified xrealloc_flex_struct macros

---
 xen/include/xen/xmalloc.h | 12 ++++++++++++
 1 file changed, 12 insertions(+)
xen/include/xen/xmalloc.h +12 -0
index 831152f..f515cee 100644
@@ -35,6 +35,18 @@
 #define xzalloc_array(_type, _num) \
     ((_type *)_xzalloc_array(sizeof(_type), __alignof__(_type), _num))
 
+/* Allocate space for a structure with a flexible array of typed objects. */
+#define xzalloc_flex_struct(type, field, nr) \
+    ((type *)_xzalloc(offsetof(type, field[nr]), __alignof__(type)))
+
+#define xmalloc_flex_struct(type, field, nr) \
+    ((type *)_xmalloc(offsetof(type, field[nr]), __alignof__(type)))
+
+/* Re-allocate space for a structure with a flexible array of typed objects. */
+#define xrealloc_flex_struct(ptr, field, nr)                           \
+    ((typeof(ptr))_xrealloc(ptr, offsetof(typeof(*(ptr)), field[nr]),  \
+                            __alignof__(typeof(*(ptr)))))
+
 /* Allocate untyped storage. */
 #define xmalloc_bytes(_bytes) _xmalloc(_bytes, SMP_CACHE_BYTES)
 #define xzalloc_bytes(_bytes) _xzalloc(_bytes, SMP_CACHE_BYTES)
Signature · 7 lines
2.7.4


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#7 lore raw

PATCH v6 5/8 iommu/arm: Add lightweight iommu_fwspec support

In reply to Oleksandr Tyshchenko

From: Oleksandr Tyshchenko <[email protected]>

We need to have some abstract way to add new device to the IOMMU based on the generic IOMMU DT bindings [1] which can be used for both DT (right now) and ACPI (in future).

For that reason we can borrow the idea used in Linux these days called "iommu_fwspec". Having this in, it will be possible to configure IOMMU master interfaces of the device (device IDs) from a single common place and avoid keeping almost identical look-up implementations in each IOMMU driver.

There is no need to port the whole implementation of "iommu_fwspec" to Xen, we could, probably, end up with a much simpler solution, some "stripped down" version which fits our requirements.

So, this patch adds the following:
1. A common structure "iommu_fwspec" to hold the the per-device firmware data
2. New member "iommu_fwspec" of struct device
3. Functions/helpers to deal with "dev->iommu_fwspec"

It should be noted that in comparison of the original "iommu_fwspec" Xen's variant doesn't contain some fields, which are not really needed at the moment (ops, flag) and "iommu_fwnode" field was replaced by "iommu_dev" to avoid porting a lot of code (to support "fwnode_handle") with little benefit.

The "iommu_fwspec" support is based on the Linux's commit:
f74c2bb98776e2de508f4d607cd519873065118e "Linux 5.3-rc8"

Subsequent patches will use of that support.

[1] https://www.kernel.org/doc/Documentation/devicetree/bindings/iommu/iommu.txt

---
Changes V5 -> V6:
- added Julien's A-b

Changes V4 -> V5:
- modified struct iommu_fwspec to use flexible array (uint32_t ids[])
- modified to use xzalloc_flex_struct() on the first allocation
- added "const" to uint32_t *ids in iommu_fwspec_add_ids()

Changes V3 -> V4:
- modified iommu_fwspec_add_ids() to use new implementation of xrealloc_flex_struct()
- mentioned exact Linux version we are based on
- fixed Grammatical error

Changes V2 -> V3:
- added Copyright from Linux
- ordered the headers alphabetically
- removed check for not a NULL before calling xfree()
- used unsigned for variables which can't be negative
- removed #include <asm/iommu_fwspec.h> from iommu.h
- added check to iommu_fwspec_init() to not allow overriding
- clarified comments in code
- modified iommu_fwspec_add_ids() to use type-safe xrealloc_flex_struct()

---
 xen/drivers/passthrough/arm/Makefile       |  2 +-
 xen/drivers/passthrough/arm/iommu_fwspec.c | 97 ++++++++++++++++++++++++++++++
 xen/include/asm-arm/device.h               |  1 +
 xen/include/asm-arm/iommu_fwspec.h         | 68 +++++++++++++++++++++
 4 files changed, 167 insertions(+), 1 deletion(-)
 create mode 100644 xen/drivers/passthrough/arm/iommu_fwspec.c
 create mode 100644 xen/include/asm-arm/iommu_fwspec.h
xen/drivers/passthrough/arm/Makefile +1 -1
index 4abb87a..5fbad45 100644
@@ -1,2 +1,2 @@
-obj-y += iommu.o iommu_helpers.o
+obj-y += iommu.o iommu_helpers.o iommu_fwspec.o
 obj-$(CONFIG_ARM_SMMU) += smmu.o
xen/drivers/passthrough/arm/iommu_fwspec.c new file +97 -0

97 lines not on this page · raw message

xen/include/asm-arm/device.h +1 -0
index ee1c3bc..ee7cff2 100644
@@ -18,6 +18,7 @@ struct device
     struct dt_device_node *of_node; /* Used by drivers imported from Linux */
 #endif
     struct dev_archdata archdata;
+    struct iommu_fwspec *iommu_fwspec; /* per-device IOMMU instance data */
 };
 
 typedef struct device device_t;
xen/include/asm-arm/iommu_fwspec.h new file +68 -0

68 lines not on this page · raw message

Signature · 7 lines
2.7.4


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#8 lore raw

PATCH v6 6/8 iommu: Order the headers alphabetically in device_tree.c

In reply to Oleksandr Tyshchenko

From: Oleksandr Tyshchenko <[email protected]>

Clean up the code a bit by putting the headers in alphabetical order.

---
Changes new -> V6:
- added Julien's A-b

---
 xen/drivers/passthrough/device_tree.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
xen/drivers/passthrough/device_tree.c +3 -3
index ea9fd54..921a6e5 100644
@@ -15,11 +15,11 @@
  * GNU General Public License for more details.
  */
 
-#include <xen/lib.h>
-#include <xen/sched.h>
+#include <xen/device_tree.h>
 #include <xen/guest_access.h>
 #include <xen/iommu.h>
-#include <xen/device_tree.h>
+#include <xen/lib.h>
+#include <xen/sched.h>
 #include <xsm/xsm.h>
 
 static spinlock_t dtdevs_lock = SPIN_LOCK_UNLOCKED;
Signature · 7 lines
2.7.4


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#9 lore raw

PATCH v6 7/8 iommu/arm: Introduce iommu_add_dt_device API

In reply to Oleksandr Tyshchenko

From: Oleksandr Tyshchenko <[email protected]>

The main puprose of this patch is to add a way to register DT device (which is behind the IOMMU) using the generic IOMMU DT bindings [1] before assigning that device to a domain.

So, this patch adds new "iommu_add_dt_device" API for adding DT device to the IOMMU using generic IOMMU DT bindings and previously added "iommu_fwspec" support. As devices can be assigned to the hardware domain and other domains this function is called from two places: handle_device() and iommu_do_dt_domctl().

Besides that, this patch adds new "dt_xlate" callback (borrowed from Linux "of_xlate") for providing the driver with DT IOMMU specifier which describes the IOMMU master interfaces of that device (device IDs, etc). According to the generic IOMMU DT bindings the context of required properties for IOMMU device/master node (#iommu-cells, iommus) depends on many factors and is really driver depended thing.

Please note, all IOMMU drivers which support generic IOMMU DT bindings should use "dt_xlate" and "add_device" callbacks.

[1] https://www.kernel.org/doc/Documentation/devicetree/bindings/iommu/iommu.txt

---
Changes V5 -> V6:
- added check for the function's return value in both cases
- removed extra check for dt_device_is_protected() from iommu_do_dt_domctl()

Changes V4 -> V5:
- added "const" to struct dt_phandle_args *args in dt_xlate
- moved iommu_add_dt_device() to xen/passthrough/device_tree.c
- modified logic, don't try to add "all" devices to the IOMMU when constructing Dom0, but only devices which are going to be assigned to hwdom
- updated iommu_do_dt_domctl() to call iommu_add_dt_device()
- clarified patch description
- removed "__init" from iommu_add_dt_device()

Changes V3 -> V4:
- squashed with "iommu: Add of_xlate callback" patch
- renamed "of_xlate" to "dt_xlate"
- reworked patch description
- clarified comments in code, removed confusing word "initialize device", etc
- updated debug message in handle_device()
- modified to check ops->of_xlate and ops->add_device only if "iommus" property is exists

Changes V2 -> V3:
- clarified patch description
- clarified comments in code
- modified to provide DT IOMMU specifier to the driver using "of_xlate" callback
- documented function usage
- modified to return an error if ops is not present/implemented,
- added ability to return a possitive value to indicate that device doesn't need to be protected
- removed check for the "iommu" property presence
in the common code
- included <asm/iommu_fwspec.h> directly

---
 xen/arch/arm/domain_build.c           | 25 +++++++++---
 xen/drivers/passthrough/device_tree.c | 77 +++++++++++++++++++++++++++++++++++
 xen/include/xen/iommu.h               | 21 ++++++++++
 3 files changed, 118 insertions(+), 5 deletions(-)
xen/arch/arm/domain_build.c +20 -5
index a0fee1e..b84a448 100644
@@ -1243,6 +1243,7 @@ static int __init map_device_children(struct domain *d,
  *  - Give permission to the guest to manage IRQ and MMIO range
  *  - Retrieve the IRQ configuration (i.e edge/level) from device tree
  * When the device is not marked for guest passthrough:
+ *  - Try to call iommu_add_dt_device to protect the device by an IOMMU
  *  - Assign the device to the guest if it's protected by an IOMMU
  *  - Map the IRQs and iomem regions to DOM0
  */
@@ -1263,16 +1264,30 @@ static int __init handle_device(struct domain *d, struct dt_device_node *dev,
     dt_dprintk("%s passthrough = %d nirq = %d naddr = %u\n",
                dt_node_full_name(dev), need_mapping, nirq, naddr);
 
-    if ( dt_device_is_protected(dev) && need_mapping )
+    if ( need_mapping )
     {
-        dt_dprintk("%s setup iommu\n", dt_node_full_name(dev));
-        res = iommu_assign_dt_device(d, dev);
-        if ( res )
+        dt_dprintk("Check if %s is behind the IOMMU and add it\n",
+                   dt_node_full_name(dev));
+
+        res = iommu_add_dt_device(dev);
+        if ( res < 0 )
         {
-            printk(XENLOG_ERR "Failed to setup the IOMMU for %s\n",
+            printk(XENLOG_ERR "Failed to add %s to the IOMMU\n",
                    dt_node_full_name(dev));
             return res;
         }
+
+        if ( dt_device_is_protected(dev) )
+        {
+            dt_dprintk("%s setup iommu\n", dt_node_full_name(dev));
+            res = iommu_assign_dt_device(d, dev);
+            if ( res )
+            {
+                printk(XENLOG_ERR "Failed to setup the IOMMU for %s\n",
+                       dt_node_full_name(dev));
+                return res;
+            }
+        }
     }
 
     /* Give permission and map IRQs */
xen/drivers/passthrough/device_tree.c +77 -0

95 lines not on this page · raw message

xen/include/xen/iommu.h +21 -0
index 7c3003f..974bd3f 100644
@@ -182,6 +182,17 @@ int iommu_deassign_dt_device(struct domain *d, struct dt_device_node *dev);
 int iommu_dt_domain_init(struct domain *d);
 int iommu_release_dt_devices(struct domain *d);
 
+/*
+ * Helper to add master device to the IOMMU using generic IOMMU DT bindings.
+ *
+ * Return values:
+ *  0 : device is protected by an IOMMU
+ * <0 : device is not protected by an IOMMU, but must be (error condition)
+ * >0 : device doesn't need to be protected by an IOMMU
+ *      (IOMMU is not enabled/present or device is not connected to it).
+ */
+int iommu_add_dt_device(struct dt_device_node *np);
+
 int iommu_do_dt_domctl(struct xen_domctl *, struct domain *,
                        XEN_GUEST_HANDLE_PARAM(xen_domctl_t));
 
@@ -250,6 +261,16 @@ struct iommu_ops {
     int __must_check (*iotlb_flush_all)(struct domain *d);
     int (*get_reserved_device_memory)(iommu_grdm_t *, void *);
     void (*dump_p2m_table)(struct domain *d);
+
+#ifdef CONFIG_HAS_DEVICE_TREE
+    /*
+     * All IOMMU drivers which support generic IOMMU DT bindings should use
+     * this callback. This is a way for the framework to provide the driver
+     * with DT IOMMU specifier which describes the IOMMU master interfaces of
+     * that device (device IDs, etc).
+     */
+    int (*dt_xlate)(device_t *dev, const struct dt_phandle_args *args);
+#endif
 };
 
 #include <asm/iommu.h>
Signature · 7 lines
2.7.4


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Acked-by on v6 7/8
#10 lore raw

Re: [Xen-devel] [PATCH V6 7/8] iommu/arm: Introduce iommu_add_dt_device API

Hi,

Quoted 19 lines · Oleksandr Tyshchenko

From: Oleksandr Tyshchenko <[email protected]>

The main puprose of this patch is to add a way to register DT device (which is behind the IOMMU) using the generic IOMMU DT bindings [1] before assigning that device to a domain.

So, this patch adds new "iommu_add_dt_device" API for adding DT device to the IOMMU using generic IOMMU DT bindings and previously added "iommu_fwspec" support. As devices can be assigned to the hardware domain and other domains this function is called from two places: handle_device() and iommu_do_dt_domctl().

Besides that, this patch adds new "dt_xlate" callback (borrowed from Linux "of_xlate") for providing the driver with DT IOMMU specifier which describes the IOMMU master interfaces of that device (device IDs, etc). According to the generic IOMMU DT bindings the context of required properties for IOMMU device/master node (#iommu-cells, iommus) depends on many factors and is really driver depended thing.

Quoted 6 lines

Please note, all IOMMU drivers which support generic IOMMU DT bindings should use "dt_xlate" and "add_device" callbacks.

[1] https://www.kernel.org/doc/Documentation/devicetree/bindings/iommu/iommu.txt

Cheers,

Quoted 237 lines
CC Julien Grall <[email protected]>
CC Jan Beulich <[email protected]>

---
Changes V5 -> V6:
- added check for the function's return value in both cases
- removed extra check for dt_device_is_protected() from iommu_do_dt_domctl()

Changes V4 -> V5:
- added "const" to struct dt_phandle_args *args in dt_xlate
- moved iommu_add_dt_device() to xen/passthrough/device_tree.c
- modified logic, don't try to add "all" devices to the IOMMU when constructing Dom0, but only devices which are going to be assigned to hwdom
- updated iommu_do_dt_domctl() to call iommu_add_dt_device()
- clarified patch description
- removed "__init" from iommu_add_dt_device()

Changes V3 -> V4:
- squashed with "iommu: Add of_xlate callback" patch
- renamed "of_xlate" to "dt_xlate"
- reworked patch description
- clarified comments in code, removed confusing word "initialize device", etc
- updated debug message in handle_device()
- modified to check ops->of_xlate and ops->add_device only if "iommus" property is exists

Changes V2 -> V3:
- clarified patch description
- clarified comments in code
- modified to provide DT IOMMU specifier to the driver using "of_xlate" callback
- documented function usage
- modified to return an error if ops is not present/implemented,
- added ability to return a possitive value to indicate that device doesn't need to be protected
- removed check for the "iommu" property presence
in the common code
- included <asm/iommu_fwspec.h> directly

---
  xen/arch/arm/domain_build.c           | 25 +++++++++---
  xen/drivers/passthrough/device_tree.c | 77 +++++++++++++++++++++++++++++++++++
  xen/include/xen/iommu.h               | 21 ++++++++++
  3 files changed, 118 insertions(+), 5 deletions(-)
xen/arch/arm/domain_build.c +20 -5
index a0fee1e..b84a448 100644
@@ -1243,6 +1243,7 @@ static int __init map_device_children(struct domain *d,
   *  - Give permission to the guest to manage IRQ and MMIO range
   *  - Retrieve the IRQ configuration (i.e edge/level) from device tree
   * When the device is not marked for guest passthrough:
+ *  - Try to call iommu_add_dt_device to protect the device by an IOMMU
   *  - Assign the device to the guest if it's protected by an IOMMU
   *  - Map the IRQs and iomem regions to DOM0
   */
@@ -1263,16 +1264,30 @@ static int __init handle_device(struct domain *d, struct dt_device_node *dev,
      dt_dprintk("%s passthrough = %d nirq = %d naddr = %u\n",
                 dt_node_full_name(dev), need_mapping, nirq, naddr);
 
-    if ( dt_device_is_protected(dev) && need_mapping )
+    if ( need_mapping )
      {
-        dt_dprintk("%s setup iommu\n", dt_node_full_name(dev));
-        res = iommu_assign_dt_device(d, dev);
-        if ( res )
+        dt_dprintk("Check if %s is behind the IOMMU and add it\n",
+                   dt_node_full_name(dev));
+
+        res = iommu_add_dt_device(dev);
+        if ( res < 0 )
          {
-            printk(XENLOG_ERR "Failed to setup the IOMMU for %s\n",
+            printk(XENLOG_ERR "Failed to add %s to the IOMMU\n",
                     dt_node_full_name(dev));
              return res;
          }
+
+        if ( dt_device_is_protected(dev) )
+        {
+            dt_dprintk("%s setup iommu\n", dt_node_full_name(dev));
+            res = iommu_assign_dt_device(d, dev);
+            if ( res )
+            {
+                printk(XENLOG_ERR "Failed to setup the IOMMU for %s\n",
+                       dt_node_full_name(dev));
+                return res;
+            }
+        }
      }
 
      /* Give permission and map IRQs */
xen/drivers/passthrough/device_tree.c +77 -0
index 921a6e5..cc900ba 100644
@@ -22,6 +22,8 @@
  #include <xen/sched.h>
  #include <xsm/xsm.h>
 
+#include <asm/iommu_fwspec.h>
+
  static spinlock_t dtdevs_lock = SPIN_LOCK_UNLOCKED;
 
  int iommu_assign_dt_device(struct domain *d, struct dt_device_node *dev)
@@ -125,6 +127,68 @@ int iommu_release_dt_devices(struct domain *d)
      return 0;
  }
 
+int iommu_add_dt_device(struct dt_device_node *np)
+{
+    const struct iommu_ops *ops = iommu_get_ops();
+    struct dt_phandle_args iommu_spec;
+    struct device *dev = dt_to_dev(np);
+    int rc = 1, index = 0;
+
+    if ( !iommu_enabled )
+        return 1;
+
+    if ( !ops )
+        return -EINVAL;
+
+    if ( dev_iommu_fwspec_get(dev) )
+        return -EEXIST;
+
+    /*
+     * According to the Documentation/devicetree/bindings/iommu/iommu.txt
+     * from Linux.
+     */
+    while ( !dt_parse_phandle_with_args(np, "iommus", "#iommu-cells",
+                                        index, &iommu_spec) )
+    {
+        /*
+         * The driver which supports generic IOMMU DT bindings must have
+         * these callback implemented.
+         */
+        if ( !ops->add_device || !ops->dt_xlate )
+            return -EINVAL;
+
+        if ( !dt_device_is_available(iommu_spec.np) )
+            break;
+
+        rc = iommu_fwspec_init(dev, &iommu_spec.np->dev);
+        if ( rc )
+            break;
+
+        /*
+         * Provide DT IOMMU specifier which describes the IOMMU master
+         * interfaces of that device (device IDs, etc) to the driver.
+         * The driver is responsible to decide how to interpret them.
+         */
+        rc = ops->dt_xlate(dev, &iommu_spec);
+        if ( rc )
+            break;
+
+        index++;
+    }
+
+    /*
+     * Add master device to the IOMMU if latter is present and available.
+     * The driver is responsible to mark that device as protected.
+     */
+    if ( !rc )
+        rc = ops->add_device(0, dev);
+
+    if ( rc < 0 )
+        iommu_fwspec_free(dev);
+
+    return rc;
+}
+
  int iommu_do_dt_domctl(struct xen_domctl *domctl, struct domain *d,
                         XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
  {
@@ -166,6 +230,19 @@ int iommu_do_dt_domctl(struct xen_domctl *domctl, struct domain *d,
              break;
          }
 
+        ret = iommu_add_dt_device(dev);
+        /*
+         * Ignore "-EEXIST" error code as it would mean that the device is
+         * already added to the IOMMU (positive result). Such happens after
+         * re-creating guest domain.
+         */
+        if ( ret < 0 && ret != -EEXIST )
+        {
+            printk(XENLOG_G_ERR "Failed to add %s to the IOMMU\n",
+                   dt_node_full_name(dev));
+            break;
+        }
+
          ret = iommu_assign_dt_device(d, dev);
 
          if ( ret )
xen/include/xen/iommu.h +21 -0
index 7c3003f..974bd3f 100644
@@ -182,6 +182,17 @@ int iommu_deassign_dt_device(struct domain *d, struct dt_device_node *dev);
  int iommu_dt_domain_init(struct domain *d);
  int iommu_release_dt_devices(struct domain *d);
 
+/*
+ * Helper to add master device to the IOMMU using generic IOMMU DT bindings.
+ *
+ * Return values:
+ *  0 : device is protected by an IOMMU
+ * <0 : device is not protected by an IOMMU, but must be (error condition)
+ * >0 : device doesn't need to be protected by an IOMMU
+ *      (IOMMU is not enabled/present or device is not connected to it).
+ */
+int iommu_add_dt_device(struct dt_device_node *np);
+
  int iommu_do_dt_domctl(struct xen_domctl *, struct domain *,
                         XEN_GUEST_HANDLE_PARAM(xen_domctl_t));
 
@@ -250,6 +261,16 @@ struct iommu_ops {
      int __must_check (*iotlb_flush_all)(struct domain *d);
      int (*get_reserved_device_memory)(iommu_grdm_t *, void *);
      void (*dump_p2m_table)(struct domain *d);
+
+#ifdef CONFIG_HAS_DEVICE_TREE
+    /*
+     * All IOMMU drivers which support generic IOMMU DT bindings should use
+     * this callback. This is a way for the framework to provide the driver
+     * with DT IOMMU specifier which describes the IOMMU master interfaces of
+     * that device (device IDs, etc).
+     */
+    int (*dt_xlate)(device_t *dev, const struct dt_phandle_args *args);
+#endif
  };
 
  #include <asm/iommu.h>
Signature · 6 lines
Julien Grall

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
author
#11 lore raw

Re: [Xen-devel] [PATCH V6 7/8] iommu/arm: Introduce iommu_add_dt_device API

Quoted 1 line · Julien Grall

Hi,

Hi Julien

Quoted 21 lines · Oleksandr Tyshchenko
Quoted 19 lines

From: Oleksandr Tyshchenko <[email protected]>

The main puprose of this patch is to add a way to register DT device (which is behind the IOMMU) using the generic IOMMU DT bindings [1] before assigning that device to a domain.

So, this patch adds new "iommu_add_dt_device" API for adding DT device to the IOMMU using generic IOMMU DT bindings and previously added "iommu_fwspec" support. As devices can be assigned to the hardware domain and other domains this function is called from two places: handle_device() and iommu_do_dt_domctl().

Besides that, this patch adds new "dt_xlate" callback (borrowed from Linux "of_xlate") for providing the driver with DT IOMMU specifier which describes the IOMMU master interfaces of that device (device IDs, etc). According to the generic IOMMU DT bindings the context of required properties for IOMMU device/master node (#iommu-cells, iommus) depends on many factors and is really driver depended thing.

Quoted 8 lines
Quoted 6 lines

Please note, all IOMMU drivers which support generic IOMMU DT bindings should use "dt_xlate" and "add_device" callbacks.

[1] https://www.kernel.org/doc/Documentation/devicetree/bindings/iommu/iommu.txt

Thank you!

Signature · 9 lines
Regards,

Oleksandr Tyshchenko


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Acked-by
#12 lore raw

Re: [Xen-devel] [PATCH V6 7/8] iommu/arm: Introduce iommu_add_dt_device API

In reply to Julien Grall
Quoted 20 lines · Julien Grall
Quoted 19 lines · Oleksandr Tyshchenko

From: Oleksandr Tyshchenko <[email protected]>

The main puprose of this patch is to add a way to register DT device (which is behind the IOMMU) using the generic IOMMU DT bindings [1] before assigning that device to a domain.

So, this patch adds new "iommu_add_dt_device" API for adding DT device to the IOMMU using generic IOMMU DT bindings and previously added "iommu_fwspec" support. As devices can be assigned to the hardware domain and other domains this function is called from two places: handle_device() and iommu_do_dt_domctl().

Besides that, this patch adds new "dt_xlate" callback (borrowed from Linux "of_xlate") for providing the driver with DT IOMMU specifier which describes the IOMMU master interfaces of that device (device IDs, etc). According to the generic IOMMU DT bindings the context of required properties for IOMMU device/master node (#iommu-cells, iommus) depends on many factors and is really driver depended thing.

Quoted 8 lines
Quoted 6 lines

Please note, all IOMMU drivers which support generic IOMMU DT bindings should use "dt_xlate" and "add_device" callbacks.

[1] https://www.kernel.org/doc/Documentation/devicetree/bindings/iommu/iommu.txt

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#13 lore raw

PATCH v6 8/8 iommu/arm: Add Renesas IPMMU-VMSA support

In reply to Oleksandr Tyshchenko

From: Oleksandr Tyshchenko <[email protected]>

The IPMMU-VMSA is VMSA-compatible I/O Memory Management Unit (IOMMU) which provides address translation and access protection functionalities to processing units and interconnect networks.

Please note, current driver is supposed to work only with newest R-Car Gen3 SoCs revisions which IPMMU hardware supports stage 2 translation table format and is able to use CPU's P2M table as is if one is 3-level page table (up to 40 bit IPA).

The major differences compare to the Linux driver are:

1. Stage 1/Stage 2 translation. Linux driver supports Stage 1 translation only (with Stage 1 translation table format). It manages page table by itself. But Xen driver supports Stage 2 translation (with Stage 2 translation table format) to be able to share the P2M with the CPU. Stage 1 translation is always bypassed in Xen driver.

So, Xen driver is supposed to be used with newest R-Car Gen3 SoC revisions only (H3 ES3.0, M3-W+, etc.) which IPMMU H/W supports stage 2 translation table format.

2. AArch64 support. Linux driver uses VMSAv8-32 mode, while Xen driver enables Armv8 VMSAv8-64 mode to cover up to 40 bit input address.

3. Context bank (sets of page table) usage. In Xen, each context bank is mapped to one Xen domain. So, all devices being pass throughed to the same Xen domain share the same context bank.

4. IPMMU device tracking. In Xen, all IOMMU devices are managed by single driver instance. So, driver uses global list to keep track of registered IPMMU devices.

---
Changes V4 -> V5:
- added R-b/A-b
- added "const" to struct dt_phandle_args *spec in ipmmu_dt_xlate()
- added "const" to struct dt_phandle_args *args in ipmmu_init_platform_device()

Changes V3 -> V4:
- changed "Gen3" to "R-Car Gen3" in commit description
- renamed "of_xlate" to "dt_xlate"
- removed check for the iommu_hap_pt_share flag in ipmmu_init()
- modified to be able to select IPMMU_VMSA option only if EXPERT is set
- removed an error message when deferring the probe
- moved check for the P2M sharing possibility to the Root IPMMU path

Changes V2 -> V3:
- added Copyright from Linux
- included <xen/iommu.h> and <asm/iommu_fwspec.h> directly
- removed pointless spin_lock from ipmmu_iommu_domain_teardown()
- changed "Gen3" to "R-Car Gen3", "M3 ES3.0" to "M3-W+"
- changed RCAR_PRODUCT_M3 to RCAR_PRODUCT_M3W
- clarified IMTTBCR_TSZ1_MASK
- removed TLB flush from ipmmu_domain_irq() until clarified the purpose
- returned -ENODEV in ipmmu_init() if P2M sharing or P2M IPA size is not supported to show that IOMMU device is not usable in Xen
- implemented new callback "of_xlate" to parse DT IOMMU specifier and add device IDs to "iommu_fwspec"
- reworked "add_device" callback
- updated patch description

Changes V1 -> V2:
- rewrited driver to use iommu_fwspec
- removed DT parsing code for micro-TLBs
- removed struct ipmmu_vmsa_master_cfg, dev_archdata macro
- added ipmmu_find_mmu_by_dev(), various helpers to access fwspec->iommu_priv
- implemented new callback "add_device" to add master device to IPMMU
- removed ipmmu_protect_masters()
- removed code to locate Root device in the first place, used EAGAIN to request deferred probing
- used printk_once for the system wide error messages in ipmmu_init() which don't need to be shown for every device being probed
- removed map_page/unmap_page implementation, reused them from iommu_helpers.c
- used %pd for printing domaid id
- performed various cosmetic fixes
- changed u32 -> uint32_t, u64 -> uint64_t,
unsigned int -> uint32_t where needed
- clarified TODOs
- clafiried supported SoC versions in config IPMMU_VMSA, set default to "n"
- updated comments in code, provided more accurate description, added new comments where needed
- updated patch description by providing differences between Linux/Xen implementations
- removed fields for cache snoop transaction when configuring IMTTBCR (update from Renesas BSP)

---
 xen/arch/arm/platforms/Kconfig           |    1 +
 xen/drivers/passthrough/Kconfig          |   13 +
 xen/drivers/passthrough/arm/Makefile     |    1 +
 xen/drivers/passthrough/arm/ipmmu-vmsa.c | 1333 ++++++++++++++++++++++++++++++
 4 files changed, 1348 insertions(+)
 create mode 100644 xen/drivers/passthrough/arm/ipmmu-vmsa.c
xen/arch/arm/platforms/Kconfig +1 -0
index bc0e9cd..4bb7319 100644
@@ -25,6 +25,7 @@ config RCAR3
 	bool "Renesas RCar3 support"
 	depends on ARM_64
 	select HAS_SCIF
+	select IPMMU_VMSA if EXPERT
 	---help---
 	Enable all the required drivers for Renesas RCar3
 
xen/drivers/passthrough/Kconfig +13 -0
index 61f9446..89e44b3 100644
@@ -12,6 +12,19 @@ config ARM_SMMU
 
 	  Say Y here if your SoC includes an IOMMU device implementing the
 	  ARM SMMU architecture.
+
+config IPMMU_VMSA
+	bool "Renesas IPMMU-VMSA found in R-Car Gen3 SoCs" if EXPERT = "y"
+	default n
+	depends on ARM_64
+	---help---
+	  Support for implementations of the Renesas IPMMU-VMSA found
+	  in R-Car Gen3 SoCs.
+
+	  Say Y here if you are using newest R-Car Gen3 SoCs revisions
+	  (H3 ES3.0, M3-W+, etc) which IPMMU hardware supports stage 2
+	  translation table format and is able to use CPU's P2M table as is.
+
 endif
 
 config IOMMU_FORCE_PT_SHARE
xen/drivers/passthrough/arm/Makefile +1 -0
index 5fbad45..fcd918e 100644
@@ -1,2 +1,3 @@
 obj-y += iommu.o iommu_helpers.o iommu_fwspec.o
 obj-$(CONFIG_ARM_SMMU) += smmu.o
+obj-$(CONFIG_IPMMU_VMSA) += ipmmu-vmsa.o
xen/drivers/passthrough/arm/ipmmu-vmsa.c new file +1333 -0

1,333 lines not on this page · raw message

Signature · 7 lines
2.7.4


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#14 lore raw

Re: [Xen-devel] [PATCH V6 8/8] iommu/arm: Add Renesas IPMMU-VMSA support

xen/drivers/passthrough/Kconfig config ARM_SMMU · line 18

Quoted 10 lines · Oleksandr Tyshchenko
xen/drivers/passthrough/Kconfig +4 -0
@@ -12,6 +12,19 @@ config ARM_SMMU
 
 	  Say Y here if your SoC includes an IOMMU device implementing the
 	  ARM SMMU architecture.
+
+config IPMMU_VMSA
+	bool "Renesas IPMMU-VMSA found in R-Car Gen3 SoCs" if EXPERT = "y"
+	default n

I would find it nice, btw, if such unnecessary lines were omitted (can likely again be done while committing). For some reason I wasn't Cc-ed on this patch, so I didn't notice so far.

Jan

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

author
#15 lore raw

Re: [Xen-devel] [PATCH V6 8/8] iommu/arm: Add Renesas IPMMU-VMSA support

On 26.09.19 15:22, Jan Beulich wrote:

Hi, Jan

Quoted 5 lines · Oleksandr Tyshchenko
Quoted 4 lines
xen/drivers/passthrough/Kconfig +0 -0
@@ -12,6 +12,19 @@ config ARM_SMMU
 
Quoted 8 lines

xen/drivers/passthrough/Kconfig config ARM_SMMU · line 18

Quoted 6 lines
xen/drivers/passthrough/Kconfig +4 -0
  	  Say Y here if your SoC includes an IOMMU device implementing the
  	  ARM SMMU architecture.
+
+config IPMMU_VMSA
+	bool "Renesas IPMMU-VMSA found in R-Car Gen3 SoCs" if EXPERT = "y"
+	default n

I would find it nice, btw, if such unnecessary lines were omitted (can likely again be done while committing).

Yes (assuming you are speaking about omitting "default n").

Quoted 2 lines

For some reason I
wasn't Cc-ed on this patch, so I didn't notice so far.

Sorry for that, I should have added you.

Signature · 9 lines
Regards,

Oleksandr Tyshchenko


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#16 lore raw
In reply to Oleksandr Tyshchenko

Hi,

Quoted 9 lines · Oleksandr Tyshchenko

Oleksandr Tyshchenko (8):
iommu/arm: Add iommu_helpers.c file to keep common for IOMMUs stuff iommu/arm: Add ability to handle deferred probing request xen/common: Introduce _xrealloc function
xen/common: Introduce xrealloc_flex_struct() helper macros iommu/arm: Add lightweight iommu_fwspec support
iommu: Order the headers alphabetically in device_tree.c iommu/arm: Introduce iommu_add_dt_device API
iommu/arm: Add Renesas IPMMU-VMSA support

This series is now merged.

Cheers,

Signature · 6 lines
Julien Grall

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
author
#17 lore raw
Quoted 1 line · Julien Grall

Hi,

Hi Julien

Quoted 12 lines · Oleksandr Tyshchenko
Quoted 9 lines

Oleksandr Tyshchenko (8):
   iommu/arm: Add iommu_helpers.c file to keep common for IOMMUs stuff iommu/arm: Add ability to handle deferred probing request xen/common: Introduce _xrealloc function
   xen/common: Introduce xrealloc_flex_struct() helper macros iommu/arm: Add lightweight iommu_fwspec support
   iommu: Order the headers alphabetically in device_tree.c iommu/arm: Introduce iommu_add_dt_device API
   iommu/arm: Add Renesas IPMMU-VMSA support

Quoted 1 line

This series is now merged.

Thank you!

Julien, what do you think regarding the following:

https://lists.xenproject.org/archives/html/xen-devel/2019-09/msg02304.html

this one is intended to address "the main TODO" in the IPMMU-VMSA driver and it would be really nice if

it could go in too... (I will be able to resolve any issues with it today/tomorrow if still present)

Signature · 9 lines
Regards,

Oleksandr Tyshchenko


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#18 lore raw
Quoted 18 lines · Oleksandr
Quoted 1 line · Julien Grall

Hi,

Hi Julien

Quoted 12 lines · Oleksandr Tyshchenko
Quoted 9 lines

Oleksandr Tyshchenko (8):
   iommu/arm: Add iommu_helpers.c file to keep common for IOMMUs stuff iommu/arm: Add ability to handle deferred probing request xen/common: Introduce _xrealloc function
   xen/common: Introduce xrealloc_flex_struct() helper macros iommu/arm: Add lightweight iommu_fwspec support
   iommu: Order the headers alphabetically in device_tree.c iommu/arm: Introduce iommu_add_dt_device API
   iommu/arm: Add Renesas IPMMU-VMSA support

Quoted 3 lines
Quoted 1 line

This series is now merged.

Thank you!

I just wanted to provide early feedback that this series causes problems with the legacy mmu-masters binding:

(XEN) Failed to add /amba/dma@fd500000 to the IOMMU (XEN) Failed to add /amba/dma@fd510000 to the IOMMU (XEN) Failed to add /amba/dma@fd520000 to the IOMMU (XEN) Failed to add /amba/dma@fd530000 to the IOMMU (XEN) Failed to add /amba/dma@fd540000 to the IOMMU (XEN) Failed to add /amba/dma@fd550000 to the IOMMU (XEN) Failed to add /amba/dma@fd560000 to the IOMMU (XEN) Failed to add /amba/dma@fd570000 to the IOMMU (XEN) Failed to add /amba/dma@ffa80000 to the IOMMU (XEN) Failed to add /amba/dma@ffa90000 to the IOMMU (XEN) Failed to add /amba/dma@ffaa0000 to the IOMMU (XEN) Failed to add /amba/dma@ffab0000 to the IOMMU (XEN) Failed to add /amba/dma@ffac0000 to the IOMMU (XEN) Failed to add /amba/dma@ffad0000 to the IOMMU (XEN) Failed to add /amba/dma@ffae0000 to the IOMMU (XEN) Failed to add /amba/dma@ffaf0000 to the IOMMU (XEN) Failed to add /amba/nand@ff100000 to the IOMMU (XEN) Failed to add /amba/ethernet@ff0b0000 to the IOMMU (XEN) Failed to add /amba/ethernet@ff0c0000 to the IOMMU (XEN) Failed to add /amba/ethernet@ff0d0000 to the IOMMU (XEN) Failed to add /amba/spi@ff0f0000 to the IOMMU (XEN) Failed to add /amba/sdhci@ff160000 to the IOMMU (XEN) Failed to add /amba/sdhci@ff170000 to the IOMMU (XEN) Failed to add /amba/usb@fe200000 to the IOMMU (XEN) Failed to add /amba/usb@fe300000 to the IOMMU

Booting fails and also even forcing to continue devices are unusable. I haven't managed to investigate further.

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#19 lore raw

Hi Stefano

Sorry for the possible format issues.

сб, 28 сент. 2019 г., 01:50 Stefano Stabellini <[email protected]>:

Quoted 20 lines · Oleksandr
Quoted 10 lines · Julien Grall
Quoted 1 line

Hi,

Hi Julien

Quoted 4 lines · Oleksandr Tyshchenko
Quoted 2 lines

Oleksandr Tyshchenko (8):
iommu/arm: Add iommu_helpers.c file to keep common for IOMMUs

stuff

Quoted 8 lines
Quoted 8 lines
Quoted 7 lines

iommu/arm: Add ability to handle deferred probing request xen/common: Introduce _xrealloc function xen/common: Introduce xrealloc_flex_struct() helper macros iommu/arm: Add lightweight iommu_fwspec support
iommu: Order the headers alphabetically in device_tree.c iommu/arm: Introduce iommu_add_dt_device API iommu/arm: Add Renesas IPMMU-VMSA support

Quoted 7 lines
Quoted 3 lines
Quoted 1 line

This series is now merged.

Thank you!

I just wanted to provide early feedback that this series causes problems with the legacy mmu-masters binding:

This series was developed in a way to add new functionality, but not to brake existing (legacy bindings). Probably, I missed something important. iommu_add_dt_device() could return an error (I assume, this is what you are facing) if the device node in DT contains "iommus" property (I mean, uses new bindings), but the IOMMU driver doesn't implement required callbacks yet. Do the device nodes in your DT contain "iommus" property? And to which domain these devices (in your log) are going to be assigned (dom0 or other domains?).

Quoted 26 lines

(XEN) Failed to add /amba/dma@fd500000 to the IOMMU (XEN) Failed to add /amba/dma@fd510000 to the IOMMU (XEN) Failed to add /amba/dma@fd520000 to the IOMMU (XEN) Failed to add /amba/dma@fd530000 to the IOMMU (XEN) Failed to add /amba/dma@fd540000 to the IOMMU (XEN) Failed to add /amba/dma@fd550000 to the IOMMU (XEN) Failed to add /amba/dma@fd560000 to the IOMMU (XEN) Failed to add /amba/dma@fd570000 to the IOMMU (XEN) Failed to add /amba/dma@ffa80000 to the IOMMU (XEN) Failed to add /amba/dma@ffa90000 to the IOMMU (XEN) Failed to add /amba/dma@ffaa0000 to the IOMMU (XEN) Failed to add /amba/dma@ffab0000 to the IOMMU (XEN) Failed to add /amba/dma@ffac0000 to the IOMMU (XEN) Failed to add /amba/dma@ffad0000 to the IOMMU (XEN) Failed to add /amba/dma@ffae0000 to the IOMMU (XEN) Failed to add /amba/dma@ffaf0000 to the IOMMU (XEN) Failed to add /amba/nand@ff100000 to the IOMMU (XEN) Failed to add /amba/ethernet@ff0b0000 to the IOMMU (XEN) Failed to add /amba/ethernet@ff0c0000 to the IOMMU (XEN) Failed to add /amba/ethernet@ff0d0000 to the IOMMU (XEN) Failed to add /amba/spi@ff0f0000 to the IOMMU (XEN) Failed to add /amba/sdhci@ff160000 to the IOMMU (XEN) Failed to add /amba/sdhci@ff170000 to the IOMMU (XEN) Failed to add /amba/usb@fe200000 to the IOMMU (XEN) Failed to add /amba/usb@fe300000 to the IOMMU

Quoted 2 lines

Booting fails and also even forcing to continue devices are unusable. I haven't managed to investigate further.

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#20 lore raw
Quoted 31 lines · Oleksandr Tyshchenko

сб, 28 сент. 2019 г., 01:50 Stefano Stabellini <[email protected] <mailto:[email protected]>>:

On Thu, 26 Sep 2019, Oleksandr wrote:
> On 26.09.19 17:56, Julien Grall wrote:
> > Hi,
>
> Hi Julien
>
>
> >
> > On 9/26/19 12:20 PM, Oleksandr Tyshchenko wrote:
> > > Oleksandr Tyshchenko (8):
> > >    iommu/arm: Add iommu_helpers.c file to keep common for
IOMMUs stuff
> > >    iommu/arm: Add ability to handle deferred probing request > > >    xen/common: Introduce _xrealloc function
> > >    xen/common: Introduce xrealloc_flex_struct() helper macros > > >    iommu/arm: Add lightweight iommu_fwspec support > > >    iommu: Order the headers alphabetically in device_tree.c > > >    iommu/arm: Introduce iommu_add_dt_device API
> > >    iommu/arm: Add Renesas IPMMU-VMSA support
> >
> > This series is now merged.
>
> Thank you!

I just wanted to provide early feedback that this series causes problems with the legacy mmu-masters binding:

Quoted 8 lines

This series was developed in a way to add new functionality, but not to brake existing (legacy bindings). Probably, I missed something important. iommu_add_dt_device() could return an error (I assume, this is what you are facing) if the device node in DT contains "iommus" property (I mean, uses new bindings), but the IOMMU driver doesn't implement required callbacks yet. Do the device nodes in your DT contain "iommus" property? And to which domain these devices (in your log) are going to be assigned (dom0 or other domains?).

Looking at the bindings, I don't think it is legit to have a node using both legacy and generic binding together. If this is what happens, then I would consider it as unsupported.

Cheers,

Signature · 5 lines
Julien Grall
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#21 lore raw
Quoted 41 lines · Julien Grall
Quoted 31 lines · Oleksandr Tyshchenko

сб, 28 сент. 2019 г., 01:50 Stefano Stabellini <[email protected] <mailto:[email protected]>>:

On Thu, 26 Sep 2019, Oleksandr wrote:
> On 26.09.19 17:56, Julien Grall wrote:
> > Hi,
>
> Hi Julien
>
>
> >
> > On 9/26/19 12:20 PM, Oleksandr Tyshchenko wrote:
> > > Oleksandr Tyshchenko (8):
> > >    iommu/arm: Add iommu_helpers.c file to keep common for
IOMMUs stuff
> > >    iommu/arm: Add ability to handle deferred probing request > > >    xen/common: Introduce _xrealloc function
> > >    xen/common: Introduce xrealloc_flex_struct() helper macros > > >    iommu/arm: Add lightweight iommu_fwspec support > > >    iommu: Order the headers alphabetically in device_tree.c > > >    iommu/arm: Introduce iommu_add_dt_device API
> > >    iommu/arm: Add Renesas IPMMU-VMSA support
> >
> > This series is now merged.
>
> Thank you!

I just wanted to provide early feedback that this series causes problems with the legacy mmu-masters binding:

Quoted 8 lines

This series was developed in a way to add new functionality, but not to brake existing (legacy bindings). Probably, I missed something important. iommu_add_dt_device() could return an error (I assume, this is what you are facing) if the device node in DT contains "iommus" property (I mean, uses new bindings), but the IOMMU driver doesn't implement required callbacks yet. Do the device nodes in your DT contain "iommus" property? And to which domain these devices (in your log) are going to be assigned (dom0 or other domains?).

Quoted 3 lines

Looking at the bindings, I don't think it is legit to have a node using both legacy and generic binding together. If this is what happens, then I would consider it as unsupported.

I have just sent a fix for this.

The issue is that some Xilinx device trees expose both the legacy and generic bindings, however, only one set of bindings is supposed to be used, either the legacy or the generic bindings (not both!). Linux solves this problem by probing for the existence of "mmu-masters" (the legacy bindings property) very early on and disabling the generic bindings if "mmu-masters" is present.

Something like that would work for Xen too, but I chatted with Julien and came up with something simpler. (Also keeping in mind that a new colleague of mine has just started working on generic bindings support for the ARM SMMU driver in Xen so this issue will go away soon). See:

https://marc.info/?l=xen-devel&m=156987707614744

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

author
#22 lore raw

On 30.09.19 23:58, Stefano Stabellini wrote:

Hi Stefano

Quoted 41 lines · Julien Grall
Quoted 40 lines · Oleksandr Tyshchenko
Quoted 39 lines

сб, 28 сент. 2019 г., 01:50 Stefano Stabellini <[email protected] <mailto:[email protected]>>:

On Thu, 26 Sep 2019, Oleksandr wrote:
> On 26.09.19 17:56, Julien Grall wrote:
> > Hi,
>
> Hi Julien
>
>
> >
> > On 9/26/19 12:20 PM, Oleksandr Tyshchenko wrote:
> > > Oleksandr Tyshchenko (8):
> > >    iommu/arm: Add iommu_helpers.c file to keep common for
IOMMUs stuff
> > >    iommu/arm: Add ability to handle deferred probing request > > >    xen/common: Introduce _xrealloc function
> > >    xen/common: Introduce xrealloc_flex_struct() helper macros > > >    iommu/arm: Add lightweight iommu_fwspec support > > >    iommu: Order the headers alphabetically in device_tree.c > > >    iommu/arm: Introduce iommu_add_dt_device API
> > >    iommu/arm: Add Renesas IPMMU-VMSA support
> >
> > This series is now merged.
>
> Thank you!

I just wanted to provide early feedback that this series causes problems with the legacy mmu-masters binding:

This series was developed in a way to add new functionality, but not to brake existing (legacy bindings). Probably, I missed something important. iommu_add_dt_device() could return an error (I assume, this is what you are facing) if the device node in DT contains "iommus" property (I mean, uses new bindings), but the IOMMU driver doesn't implement required callbacks yet. Do the device nodes in your DT contain "iommus" property? And to which domain these devices (in your log) are going to be assigned (dom0 or other domains?).

Quoted 8 lines
Quoted 3 lines

Looking at the bindings, I don't think it is legit to have a node using both legacy and generic binding together. If this is what happens, then I would consider it as unsupported.

I have just sent a fix for this.

The issue is that some Xilinx device trees expose both the legacy and generic bindings, however, only one set of bindings is supposed to be used, either the legacy or the generic bindings (not both!).

I expected this could be a reason.

Quoted 11 lines

Linux
solves this problem by probing for the existence of "mmu-masters" (the legacy bindings property) very early on and disabling the generic bindings if "mmu-masters" is present.

Something like that would work for Xen too, but I chatted with Julien and came up with something simpler. (Also keeping in mind that a new colleague of mine has just started working on generic bindings support for the ARM SMMU driver in Xen so this issue will go away soon). See:

https://marc.info/?l=xen-devel&m=156987707614744

Sounds good to me.

Signature · 9 lines
Regards,

Oleksandr Tyshchenko


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel