iommu/arm: Add Renesas IPMMU-VMSA support + Linux's iommu_fwspec
v5 cover iommu/arm: Add Renesas IPMMU-VMSA support + Linux's iommu_fwspec
From: Oleksandr Tyshchenko <[email protected]>
The purpose of this patch series is to add IPMMU-VMSA support to Xen on ARM.
Besides new IOMMU driver, this series contains "iommu_fwspec" support and new API iommu_add_dt_device() for adding DT device to IOMMU and many other things.
The IPMMU-VMSA is VMSA-compatible I/O Memory Management Unit (IOMMU) which provides address translation and access protection functionalities to processing units and interconnect networks.
Please note, this driver is supposed to work only with newest R-Car Gen3 SoCs revisions which IPMMU hardware supports stage 2 translation table format and is able to use CPU's P2M table as is if one is 3-level page table (up to 40 bit IPA).
----------
This driver is based on Linux's IPMMU-VMSA driver from Renesas BSP:
https://git.kernel.org/pub/scm/linux/kernel/git/horms/renesas-bsp.git/tree/drivers/iommu/ipmmu-vmsa.c?h=v4.14.75-ltsi/rcar-3.9.6
and Xen's SMMU driver:xen/drivers/passthrough/arm/smmu.c
Although Xen driver has a lot in common with Linux driver, it is not a "direct ported" copy and should be treated as such.
The major differences compare to the Linux driver are:
1. Stage 1/Stage 2 translation. Linux driver supports Stage 1 translation only (with Stage 1 translation table format). It manages page table by itself. But Xen driver supports Stage 2 translation (with Stage 2 translation table format) to be able to share the P2M with the CPU. Stage 1 translation is always bypassed in Xen driver.
So, Xen driver is supposed to be used with newest R-Car Gen3 SoC revisions only (H3 ES3.0, M3-W+, etc.) which IPMMU H/W supports stage 2 translation table format.
2. AArch64 support. Linux driver uses VMSAv8-32 mode, while Xen driver enables Armv8 VMSAv8-64 mode to cover up to 40 bit input address.
3. Context bank (sets of page table) usage. In Xen, each context bank is mapped to one Xen domain. So, all devices being pass throughed to the same Xen domain share the same context bank.
4. IPMMU device tracking. In Xen, all IPMMU devices are managed by single driver instance. So, driver uses global list to keep track of registered devices.
----------
Series was tested on R-Car Gen3 H3 ES3.0/M3N ES1.0 based boards using current staging (df29d03 libxc/x86: avoid certain overflows in CPUID APIC ID adjustments) in a system with several DMA masters being assigned to different guest domains. Guest domain reboot, destroy/create are functional.
You can find the whole series here:
repo: https://github.com/otyshchenko1/xen.git branch: ipmmu_upstream5
You can find previous discussions here:
[V1] https://lists.xenproject.org/archives/html/xen-devel/2019-06/msg01755.html
[V2] https://lists.xenproject.org/archives/html/xen-devel/2019-08/msg00253.html
[V3] https://lists.xenproject.org/archives/html/xen-devel/2019-08/msg01948.html
[V4] https://lists.xenproject.org/archives/html/xen-devel/2019-09/msg01255.html
Also, please note, there is a patch in ML which is intended to address the main TODO in the IPMMU-VMSA driver (I am about to send non-RFC version):
https://lists.xenproject.org/archives/html/xen-devel/2019-09/msg00973.html
For convenience:
1. iommu/arm: Add iommu_helpers.c file to keep common for IOMMUs stuff A-b
2. iommu/arm: Add ability to handle deferred probing request R-b
3. xen/common: Introduce _xrealloc function
-none-
4. xen/common: Introduce xrealloc_flex_struct() helper macros R-b
5. iommu/arm: Add lightweight iommu_fwspec support
-none-
6. iommu: Order the headers alphabetically in device_tree.c new
7. iommu/arm: Introduce iommu_add_dt_device API
-none-
8. iommu/arm: Add Renesas IPMMU-VMSA support
A-b/R-b
Oleksandr Tyshchenko (8):
iommu/arm: Add iommu_helpers.c file to keep common for IOMMUs stuff iommu/arm: Add ability to handle deferred probing request xen/common: Introduce _xrealloc function
xen/common: Introduce xrealloc_flex_struct() helper macros iommu/arm: Add lightweight iommu_fwspec support
iommu: Order the headers alphabetically in device_tree.c iommu/arm: Introduce iommu_add_dt_device API
iommu/arm: Add Renesas IPMMU-VMSA support
xen/arch/arm/domain_build.c | 22 +- xen/arch/arm/platforms/Kconfig | 1 + xen/common/xmalloc_tlsf.c | 113 ++- xen/drivers/passthrough/Kconfig | 13 + xen/drivers/passthrough/arm/Makefile | 3 +- xen/drivers/passthrough/arm/iommu.c | 56 +- xen/drivers/passthrough/arm/iommu_fwspec.c | 97 ++ xen/drivers/passthrough/arm/iommu_helpers.c | 84 ++ xen/drivers/passthrough/arm/ipmmu-vmsa.c | 1333 +++++++++++++++++++++++++++ xen/drivers/passthrough/arm/smmu.c | 48 +- xen/drivers/passthrough/device_tree.c | 77 +- xen/include/asm-arm/device.h | 7 +- xen/include/asm-arm/iommu.h | 10 + xen/include/asm-arm/iommu_fwspec.h | 68 ++ xen/include/xen/device_tree.h | 7 + xen/include/xen/iommu.h | 21 + xen/include/xen/xmalloc.h | 13 + 17 files changed, 1895 insertions(+), 78 deletions(-) create mode 100644 xen/drivers/passthrough/arm/iommu_fwspec.c create mode 100644 xen/drivers/passthrough/arm/iommu_helpers.c create mode 100644 xen/drivers/passthrough/arm/ipmmu-vmsa.c create mode 100644 xen/include/asm-arm/iommu_fwspec.h
Signature · 7 lines
2.7.4 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v5 1/8 iommu/arm: Add iommu_helpers.c file to keep common for IOMMUs stuff
From: Oleksandr Tyshchenko <[email protected]>
Introduce a separate file to keep various helpers which could be used by more than one IOMMU driver in order not to duplicate code.
The first candidates to be moved to the new file are SMMU driver's "map_page/unmap_page" callbacks. These callbacks neither contain any SMMU specific info nor perform any SMMU specific actions and are going to be the same across all IOMMU drivers which H/W IP shares P2M with the CPU like SMMU does.
So, move callbacks to iommu_helpers.c for the upcoming IPMMU driver to be able to re-use them.
---
Changes V3 -> V4:
- added Julien's A-b
- fixed Grammatical error
- clarified comment in a header
Changes V2 -> V3:
- put headers in alphabetical order
- retained Copyright from the SMMU code
--- xen/drivers/passthrough/arm/Makefile | 2 +- xen/drivers/passthrough/arm/iommu_helpers.c | 84 +++++++++++++++++++++++++++++ xen/drivers/passthrough/arm/smmu.c | 48 +---------------- xen/include/asm-arm/iommu.h | 10 ++++ 4 files changed, 97 insertions(+), 47 deletions(-) create mode 100644 xen/drivers/passthrough/arm/iommu_helpers.c
xen/drivers/passthrough/arm/Makefile +1 -1
@@ -1,2 +1,2 @@ -obj-y += iommu.o +obj-y += iommu.o iommu_helpers.o obj-$(CONFIG_ARM_SMMU) += smmu.o
xen/drivers/passthrough/arm/iommu_helpers.c new file +84 -0
84 lines not on this page · raw message
xen/drivers/passthrough/arm/smmu.c +2 -46
60 lines not on this page · raw message
xen/include/asm-arm/iommu.h +10 -0
@@ -26,6 +26,16 @@ struct arch_iommuconst struct iommu_ops *iommu_get_ops (void );void iommu_set_ops (const struct iommu_ops *ops); +/* +* The mapping helpers below should only be used if P2M Table is shared +* between the CPU and the IOMMU. +*/ +int __must_check arm_iommu_map_page (struct domain *d,dfn_t dfn,mfn_t mfn, +unsigned int flags, +unsigned int *flush_flags); +int __must_check arm_iommu_unmap_page (struct domain *d,dfn_t dfn, +unsigned int *flush_flags); +#endif /* __ARCH_ARM_IOMMU_H__ */ /*
Signature · 7 lines
2.7.4 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v5 2/8 iommu/arm: Add ability to handle deferred probing request
In reply to Oleksandr TyshchenkoFrom: Oleksandr Tyshchenko <[email protected]>
This patch adds minimal required support to General IOMMU framework to be able to handle a case when IOMMU driver requesting deferred probing for a device.
In order not to pull Linux's error code (-EPROBE_DEFER) to Xen we have chosen -EAGAIN to be used for indicating that device probing is deferred.
This is needed for the upcoming IPMMU driver which may request deferred probing depending on what device will be probed the first (there is some dependency between these devices, Root device must be registered before Cache devices. If not the case, driver will deny further Cache device probes until Root device is registered). As we can't guarantee a fixed pre-defined order for the device nodes in DT, we need to be ready for the situation where devices being probed in "any" order.
---
Changes V4 -> V5:
- added Julien's R-b
- fixed Grammatical error
Changes V3 -> V4:
- moved changes related to the "headers ordering" to a separate patch
- added explaination why domain_list is re-used
- removed the unnecessary pair of outermost parentheses when checking the list_empty
Changes V2 -> V3:
- removed deferred_probe field from struct dt_device_node, re-used domain_list instead
- documented domain_list usage
- added ASSERT to check that np->domain_list is empty before re-using it
- put deferred_probe_list to init section
- used more strict logic regarding processing devices in the deferred list
- added more comments to code
- put headers in alphabetical order
--- xen/drivers/passthrough/arm/iommu.c | 56 +++++++++++++++++++++++++++++++++++-- xen/include/asm-arm/device.h | 6 +++- xen/include/xen/device_tree.h | 7 +++++ 3 files changed, 66 insertions(+), 3 deletions(-)
xen/drivers/passthrough/arm/iommu.c +54 -2
@@ -21,6 +21,12 @@#include <asm/device.h> +/* +* Deferred probe list is used to keep track of devices for which driver +* requested deferred probing (returned -EAGAIN). +*/ +static __initdata LIST_HEAD (deferred_probe_list); +static const struct iommu_ops *iommu_ops;const struct iommu_ops *iommu_get_ops (void )
@@ -43,7 +49,7 @@ void __init iommu_set_ops(const struct iommu_ops *ops)int __init iommu_hardware_setup (void ) { -struct dt_device_node *np; +struct dt_device_node *np, *tmp;int rc;unsigned int num_iommus =0 ;
@@ -52,6 +58,21 @@ int __init iommu_hardware_setup(void) rc =device_init (np,DEVICE_IOMMU ,NULL );if ( !rc ) num_iommus++; +else if ( rc == -EAGAIN ) + { +/* +* Nobody should use device's domain_list at such early stage, +* so we can re-use it to link the device in the deferred list to +* avoid introducing extra list_head field in struct dt_device_node. +*/ +ASSERT (list_empty (&np->domain_list)); + +/* +* Driver requested deferred probing, so add this device to +* the deferred list for further processing. +*/ +list_add (&np->domain_list, &deferred_probe_list); + }/* * Ignore the following error codes: * - EBADF: Indicate the current is not an IOMMU
@@ -62,7 +83,38 @@ int __init iommu_hardware_setup(void)return rc; } -return ( num_iommus >0 ) ?0 : -ENODEV ; +/* Return immediately if there are no initialized devices. */ +if ( !num_iommus ) +return list_empty (&deferred_probe_list) ? -ENODEV : -EAGAIN ; + + rc =0 ; + +/* +* Process devices in the deferred list if it is not empty. +* Check that at least one device is initialized at each loop, otherwise +* we may get an infinite loop. Also stop processing if we got an error +* other than -EAGAIN. +*/ +while ( !list_empty (&deferred_probe_list) && num_iommus ) + { + num_iommus =0 ; + + list_for_each_entry_safe ( np, tmp, &deferred_probe_list, domain_list ) + { + rc =device_init (np,DEVICE_IOMMU ,NULL ); +if ( !rc ) + { + num_iommus++; + +/* Remove initialized device from the deferred list. */ +list_del_init (&np->domain_list); + } +else if ( rc != -EAGAIN ) +return rc; + } + } + +return rc; } void __hwdom_init arch_iommu_check_autotranslated_hwdom(struct domain *d)
xen/include/asm-arm/device.h +5 -1
@@ -44,7 +44,11 @@ struct device_desc {enum device_class class;/* List of devices supported by this driver */ const struct dt_device_match *dt_match; -/* Device initialization */ +/* +* Device initialization. +* +* -EAGAIN is used to indicate that device probing is deferred. +*/ int (*init)(struct dt_device_node *dev,const void *data); };
xen/include/xen/device_tree.h +7 -0
@@ -92,6 +92,13 @@ struct dt_device_node {/* IOMMU specific fields */ bool is_protected; +/* +* The main purpose of this list is to link the structure in the list +* of devices assigned to domain. +* +* Boot code (iommu_hardware_setup) re-uses this list to link the structure +* in the list of devices for which driver requested deferred probing. +*/ struct list_head domain_list;struct device dev;
Signature · 7 lines
2.7.4 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v5 3/8 xen/common: Introduce _xrealloc function
In reply to Oleksandr TyshchenkoFrom: Oleksandr Tyshchenko <[email protected]>
This patch introduces type-unsafe function which besides re-allocation handles the following corner cases:
1. if requested size is zero, it will behave like xfree
2. if incoming pointer is not valid (NULL or ZERO_BLOCK_PTR), it will behave like xmalloc
If both pointer and size are valid the function will re-allocate and copy only if requested size and alignment don't fit in already allocated space.
Subsequent patch will add type-safe helper macros.
---
Changes V4 -> V5:
- avoid possible truncation with allocations of 4GiB or above
- introduce helper functions add(strip)_padding to avoid duplicating the code
- omit the unnecessary casts, change u32 to uint32_t when moving the code
- use _xzalloc instead of _xmalloc to get the tail portion zeroed
- update pointer according to the requsted alignment
- compared against "size" instead of "tmp_size" for the allocations above PAGE_SIZE
Changes V3 -> V4:
- add check for the alignment compatibility
- properly detect current size (take into the account a possible fake alignment header)
- update comment in code/patch description
Changes RFC -> V3:
- behave like xmalloc if incoming pointer is ZERO_BLOCK_PTR or NULL
- return ZERO_BLOCK_PTR after xfree if requested size is zero
- add patch description
- use allocator internals to recognize current size of the incoming pointer
- do not re-allocate and copy if requested size fits in already allocated space
...
Original patch was initially posted by Sameer Goel:
https://lists.xen.org/archives/html/xen-devel/2017-06/msg00858.html
This could be considered as another attempt to add it:
https://www.mail-archive.com/[email protected]/msg21335.html
[As it was previously discussed with Julien in IRC]
The reason for this patch to be an RFC is that patch itself is not completely correct and I don't fully understand what/how should be done for this patch to be accepted. Or whether community even wants this to go in. So, to avoid bike shedding, the first target is to collect feedback.
For everyone who wants more details why this is needed and where used, please see next patch of this thread:
"iommu/arm: Add lightweight iommu_fwspec support"
In a nutshell, the upcoming "iommu_fwspec" support on ARM is going to use xrealloc to expand an array for device IDs. We really want to have "iommu_fwspec" support which will give us a generic abstract way to add new device to the IOMMU based on the generic IOMMU DT binding.
--- xen/common/xmalloc_tlsf.c | 113 ++++++++++++++++++++++++++++++++++++++-------- xen/include/xen/xmalloc.h | 1 + 2 files changed, 96 insertions(+), 18 deletions(-)
xen/common/xmalloc_tlsf.c +95 -18
@@ -554,10 +554,40 @@ static void tlsf_init(void)#define ZERO_BLOCK_PTR ((void *)-1L)#endif +static void *strip_padding (void *p) +{ +struct bhdr *b = p -BHDR_OVERHEAD ; + +if ( b->size &FREE_BLOCK ) + { + p -= b->size & ~FREE_BLOCK ; + b = p -BHDR_OVERHEAD ; +ASSERT (!(b->size &FREE_BLOCK )); + } + +return p; +} + +static void *add_padding (void *p,unsigned long align) +{ +uint32_t pad; + +if ( (pad = -(long )p & (align -1 )) !=0 ) + { +void *q = p + pad; +struct bhdr *b = q -BHDR_OVERHEAD ; + +ASSERT (q > p); + b->size = pad |FREE_BLOCK ; + p = q; + } + +return p; +} +void *_xmalloc (unsigned long size,unsigned long align) {void *p =NULL ; -u32 pad;ASSERT (!in_irq ());
@@ -578,14 +608,7 @@ void *_xmalloc(unsigned long size, unsigned long align)return xmalloc_whole_pages (size - align +MEM_ALIGN , align);/* Add alignment padding. */ -if ( (pad = -(long )p & (align -1 )) !=0 ) - { -char *q = (char *)p + pad; -struct bhdr *b = (struct bhdr *)(q -BHDR_OVERHEAD ); -ASSERT (q > (char *)p); - b->size = pad |FREE_BLOCK ; - p = q; - } + p =add_padding (p, align);ASSERT (((unsigned long )p & (align -1 )) ==0 );return p;
@@ -598,10 +621,70 @@ void *_xzalloc(unsigned long size, unsigned long align)return p ?memset (p,0 , size) : p; } -void xfree (void *p) +void *_xrealloc (void *ptr,unsigned long size,unsigned long align) { -struct bhdr *b; +unsigned long curr_size, tmp_size; +void *p; + +if ( !size ) + { +xfree (ptr); +return ZERO_BLOCK_PTR ; + } +if ( ptr ==NULL || ptr ==ZERO_BLOCK_PTR ) +return _xmalloc (size, align); + +ASSERT ((align & (align -1 )) ==0 ); +if ( align <MEM_ALIGN ) + align =MEM_ALIGN ; + + tmp_size = size + align -MEM_ALIGN ; + +if ( tmp_size <PAGE_SIZE ) + tmp_size = (tmp_size <MIN_BLOCK_SIZE ) ?MIN_BLOCK_SIZE : +ROUNDUP_SIZE (tmp_size); + +if ( !((unsigned long )ptr & (PAGE_SIZE -1 )) ) + { + curr_size = (unsigned long )PFN_ORDER (virt_to_page (ptr)) <<PAGE_SHIFT ; + +if ( size <= curr_size && ((unsigned long )ptr & (align -1 )) ==0 ) +return ptr; + } +else + { +struct bhdr *b; + +/* Strip alignment padding. */ + p =strip_padding (ptr); + + b = p -BHDR_OVERHEAD ; + curr_size = b->size &BLOCK_SIZE_MASK ; + +if ( tmp_size <= curr_size ) + { +/* Add alignment padding. */ + p =add_padding (p, align); + +ASSERT (((unsigned long )p & (align -1 )) ==0 ); + +return p; + } + } + + p =_xzalloc (size, align); +if ( p ) + { +memcpy (p, ptr,min (curr_size, size)); +xfree (ptr); + } + +return p; +} + +void xfree (void *p) +{if ( p ==NULL || p ==ZERO_BLOCK_PTR )return ;
@@ -626,13 +709,7 @@ void xfree(void *p) }/* Strip alignment padding. */ - b = (struct bhdr *)((char *)p -BHDR_OVERHEAD ); -if ( b->size &FREE_BLOCK ) - { - p = (char *)p - (b->size & ~FREE_BLOCK ); - b = (struct bhdr *)((char *)p -BHDR_OVERHEAD ); -ASSERT (!(b->size &FREE_BLOCK )); - } + p =strip_padding (p);xmem_pool_free (p, xenpool); }
xen/include/xen/xmalloc.h +1 -0
@@ -51,6 +51,7 @@ extern void xfree(void *);/* Underlying functions */ extern void *_xmalloc (unsigned long size,unsigned long align);extern void *_xzalloc (unsigned long size,unsigned long align); +extern void *_xrealloc (void *ptr,unsigned long size,unsigned long align);static inline void *_xmalloc_array (unsigned long size,unsigned long align,unsigned long num)
Signature · 7 lines
2.7.4 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH V5 3/8] xen/common: Introduce _xrealloc function
Quoted 8 lines · Oleksandr Tyshchenko
Changes V4 -> V5:
- avoid possible truncation with allocations of 4GiB or above
- introduce helper functions add(strip)_padding to avoid duplicating the code
- omit the unnecessary casts, change u32 touint32_twhen moving the code
- use_xzallocinstead of_xmallocto get the tail portion zeroed
I'm sorry, but no, this is wasteful: You write the initialized portion of the block twice this way, when once is fully sufficient (but see below).
Quoted 13 lines
xen/common/xmalloc_tlsf.c +7
-0@@ -554,10 +554,40 @@ static void tlsf_init(void)#define ZERO_BLOCK_PTR ((void *)-1L)#endif +static void *strip_padding (void *p) +{ +struct bhdr *b = p -BHDR_OVERHEAD ; + +if ( b->size &FREE_BLOCK ) + { + p -= b->size & ~FREE_BLOCK ;
xen/common/xmalloc_tlsf.c static void tlsf_init(void) · line 573
Quoted 10 lines
xen/common/xmalloc_tlsf.c +10
-0+ b = p -BHDR_OVERHEAD ; +ASSERT (!(b->size &FREE_BLOCK )); + } + +return p; +} + +static void *add_padding (void *p,unsigned long align) +{ +uint32_t pad;
A fixed width type is inappropriate here anyway - unsigned int would suffice. Judging from the incoming parameters, unsigned long would be more future proof; alternatively the "align" parameter could be "unsigned int", since we don't allow such huge allocations anyway (but I agree that adjusting this doesn't really belong in the patch here).
Quoted 9 lines
xen/common/xmalloc_tlsf.c +2
-2@@ -598,10 +621,70 @@ void *_xzalloc(unsigned long size, unsigned long align)return p ?memset (p,0 , size) : p; } -void xfree (void *p) +void *_xrealloc (void *ptr,unsigned long size,unsigned long align) { -struct bhdr *b; +unsigned long curr_size, tmp_size;
xen/common/xmalloc_tlsf.c void *_xzalloc(unsigned long size, unsigned long align) · line 636
Quoted 10 lines
xen/common/xmalloc_tlsf.c +9
-0+void *p; + +if ( !size ) + { +xfree (ptr); +return ZERO_BLOCK_PTR ; + } +if ( ptr ==NULL || ptr ==ZERO_BLOCK_PTR ) +return _xmalloc (size, align);
This is inconsistent - you use _xzalloc() further down (and too aggressively imo, as said). Here use of that function would then be indicated. However, ...
Quoted 6 lines
xen/common/xmalloc_tlsf.c +6
-0+ASSERT ((align & (align -1 )) ==0 ); +if ( align <MEM_ALIGN ) + align =MEM_ALIGN ; + + tmp_size = size + align -MEM_ALIGN ; +
xen/common/xmalloc_tlsf.c void *_xzalloc(unsigned long size, unsigned long align) · line 653
Quoted 10 lines
xen/common/xmalloc_tlsf.c +10
-0+if ( tmp_size <PAGE_SIZE ) + tmp_size = (tmp_size <MIN_BLOCK_SIZE ) ?MIN_BLOCK_SIZE : +ROUNDUP_SIZE (tmp_size); + +if ( !((unsigned long )ptr & (PAGE_SIZE -1 )) ) + { + curr_size = (unsigned long )PFN_ORDER (virt_to_page (ptr)) <<PAGE_SHIFT ; + +if ( size <= curr_size && ((unsigned long )ptr & (align -1 )) ==0 ) +return ptr;
... I only now realize that in a case like this one you can't really zero-fill the tail portion that's extending beyond the original request. Hence I think the just-in-case zero filling would better be dropped again (and the _xmalloc() use above is fine).
Note that with the fix done here you don't need tmp_size anymore outside of ...
xen/common/xmalloc_tlsf.c void *_xzalloc(unsigned long size, unsigned long align) · line 656
Quoted 3 lines
xen/common/xmalloc_tlsf.c +3
-0+ } +else + {
... this block. Please move its calculation (and declaration) here.
Quoted 4 lines
xen/common/xmalloc_tlsf.c +4
-0+struct bhdr *b; + +/* Strip alignment padding. */ + p =strip_padding (ptr);
xen/common/xmalloc_tlsf.c void *_xzalloc(unsigned long size, unsigned long align) · line 670
Quoted 10 lines
xen/common/xmalloc_tlsf.c +10
-0+ + b = p -BHDR_OVERHEAD ; + curr_size = b->size &BLOCK_SIZE_MASK ; + +if ( tmp_size <= curr_size ) + { +/* Add alignment padding. */ + p =add_padding (p, align); + +ASSERT (((unsigned long )p & (align -1 )) ==0 );
Since another rev is going to be needed anyway - here and above please prefer ! over == 0.
Jan
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH V5 3/8] xen/common: Introduce _xrealloc function
On 24.09.19 18:51, Jan Beulich wrote:
Hi, Jan
Quoted 9 lines · Oleksandr Tyshchenko
Quoted 8 lines
Changes V4 -> V5:
- avoid possible truncation with allocations of 4GiB or above
- introduce helper functions add(strip)_padding to avoid duplicating the code
- omit the unnecessary casts, change u32 touint32_twhen moving the code
- use_xzallocinstead of_xmallocto get the tail portion zeroed
Quoted 3 lines
I'm sorry, but no, this is wasteful: You write the initialized portion of the block twice this way, when once is fully sufficient (but see below).
Indeed, not effectively.
Quoted 21 lines
Quoted 21 lines
xen/common/xmalloc_tlsf.c +15
-0@@ -554,10 +554,40 @@ static void tlsf_init(void)#define ZERO_BLOCK_PTR ((void *)-1L)#endif +static void *strip_padding (void *p) +{ +struct bhdr *b = p -BHDR_OVERHEAD ; + +if ( b->size &FREE_BLOCK ) + { + p -= b->size & ~FREE_BLOCK ; + b = p -BHDR_OVERHEAD ; +ASSERT (!(b->size &FREE_BLOCK )); + } + +return p; +} + +static void *add_padding (void *p,unsigned long align)
Quoted 8 lines
xen/common/xmalloc_tlsf.c static void tlsf_init(void) · line 573
Quoted 2 lines
xen/common/xmalloc_tlsf.c +2
-0+{ +uint32_t pad;A fixed width type is inappropriate here anyway - unsigned int would suffice. Judging from the incoming parameters, unsigned long would be more future proof; alternatively the "align" parameter could be "unsigned int", since we don't allow such huge allocations anyway (but I agree that adjusting this doesn't really belong in the patch here).
Will change to unsigned int.
Quoted 17 lines
Quoted 17 lines
xen/common/xmalloc_tlsf.c +9
-2@@ -598,10 +621,70 @@ void *_xzalloc(unsigned long size, unsigned long align)return p ?memset (p,0 , size) : p; } -void xfree (void *p) +void *_xrealloc (void *ptr,unsigned long size,unsigned long align) { -struct bhdr *b; +unsigned long curr_size, tmp_size; +void *p; + +if ( !size ) + { +xfree (ptr); +return ZERO_BLOCK_PTR ; + }
Quoted 4 lines
xen/common/xmalloc_tlsf.c void *_xzalloc(unsigned long size, unsigned long align) · line 636
Quoted 2 lines
xen/common/xmalloc_tlsf.c +2
-0+if ( ptr ==NULL || ptr ==ZERO_BLOCK_PTR ) +return _xmalloc (size, align);This is inconsistent - you use
_xzalloc()further down (and too aggressively imo, as said).
Indeed. I missed that.
Quoted 15 lines
Here use of that function would then
be indicated. However, ...Quoted 12 lines
xen/common/xmalloc_tlsf.c +12
-0+ASSERT ((align & (align -1 )) ==0 ); +if ( align <MEM_ALIGN ) + align =MEM_ALIGN ; + + tmp_size = size + align -MEM_ALIGN ; + +if ( tmp_size <PAGE_SIZE ) + tmp_size = (tmp_size <MIN_BLOCK_SIZE ) ?MIN_BLOCK_SIZE : +ROUNDUP_SIZE (tmp_size); + +if ( !((unsigned long )ptr & (PAGE_SIZE -1 )) ) + {
Quoted 8 lines
xen/common/xmalloc_tlsf.c void *_xzalloc(unsigned long size, unsigned long align) · line 653
Quoted 4 lines
xen/common/xmalloc_tlsf.c +4
-0+ curr_size = (unsigned long )PFN_ORDER (virt_to_page (ptr)) <<PAGE_SHIFT ; + +if ( size <= curr_size && ((unsigned long )ptr & (align -1 )) ==0 ) +return ptr;... I only now realize that in a case like this one you can't really zero-fill the tail portion that's extending beyond the original request. Hence I think the just-in-case zero filling would better be dropped again (and the
_xmalloc()use above is fine).
Got it.
Quoted 7 lines
Note that with the fix done here you don't need
tmp_sizeanymore outside of ...xen/common/xmalloc_tlsf.c void *_xzalloc(unsigned long size, unsigned long align) · line 656
Quoted 3 lines
xen/common/xmalloc_tlsf.c +3
-0+ } +else + {... this block. Please move its calculation (and declaration) here.
Agree. Will move.
Quoted 8 lines
Quoted 8 lines
xen/common/xmalloc_tlsf.c +8
-0+struct bhdr *b; + +/* Strip alignment padding. */ + p =strip_padding (ptr); + + b = p -BHDR_OVERHEAD ; + curr_size = b->size &BLOCK_SIZE_MASK ; +
Quoted 8 lines
xen/common/xmalloc_tlsf.c void *_xzalloc(unsigned long size, unsigned long align) · line 670
Quoted 6 lines
xen/common/xmalloc_tlsf.c +6
-0+if ( tmp_size <= curr_size ) + { +/* Add alignment padding. */ + p =add_padding (p, align); + +ASSERT (((unsigned long )p & (align -1 )) ==0 );Since another rev is going to be needed anyway - here and above please prefer ! over == 0.
ok, will do.
Signature · 9 lines
Regards, Oleksandr Tyshchenko _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v5 4/8 xen/common: Introduce xrealloc_flex_struct() helper macros
In reply to Oleksandr TyshchenkoFrom: Oleksandr Tyshchenko <[email protected]>
This patch introduces type-safe helper macros to re-allocate space for a structure with a flexible array of typed objects.
For example, if we need to re-size the "data" array:
struct arrlen
{
size_t len;
int data[];
};We can use the proposed macros in the following way:
new_ptr = realloc_flex_struct(old_ptr, data, nr_elem);
where nr_elem is the desired number of elements.
Subsequent patch will use this macros.
Also, while here, introduce xmalloc(xzalloc)_flex_struct() to allocate space for a structure with a flexible array of typed objects.
---
Changes V4 -> V5:
- clarified patch description (data[] instead of data[1])
- introduced xzalloc_flex_struct()
- added Jan's R-b
- added missing parentheses around the entire constructs
Changes V3 -> V4:
- clarified patch description
- modified to not use leading underscores
- removed unnecessary pair of outermost parentheses
- modified to use "nr" instead of "len"
- placed xmalloc_flex_struct before xrealloc_flex_struct
- simplified xrealloc_flex_struct macros
--- xen/include/xen/xmalloc.h | 12 ++++++++++++ 1 file changed, 12 insertions(+)
xen/include/xen/xmalloc.h +12 -0
@@ -35,6 +35,18 @@#define xzalloc_array (_type, _num) \ ((_type *)_xzalloc_array(sizeof(_type), __alignof__(_type), _num)) +/* Allocate space for a structure with a flexible array of typed objects. */ +#define xzalloc_flex_struct (type, field, nr) \ + ((type *)_xzalloc(offsetof(type, field[nr]), __alignof__(type))) + +#define xmalloc_flex_struct (type, field, nr) \ + ((type *)_xmalloc(offsetof(type, field[nr]), __alignof__(type))) + +/* Re-allocate space for a structure with a flexible array of typed objects. */ +#define xrealloc_flex_struct (ptr, field, nr) \ + ((typeof(ptr))_xrealloc(ptr, offsetof(typeof(*(ptr)), field[nr]), \ + __alignof__(typeof(*(ptr))))) +/* Allocate untyped storage. */ #define xmalloc_bytes (_bytes) _xmalloc(_bytes, SMP_CACHE_BYTES)#define xzalloc_bytes (_bytes) _xzalloc(_bytes, SMP_CACHE_BYTES)
Signature · 7 lines
2.7.4 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v5 5/8 iommu/arm: Add lightweight iommu_fwspec support
In reply to Oleksandr TyshchenkoFrom: Oleksandr Tyshchenko <[email protected]>
We need to have some abstract way to add new device to the IOMMU based on the generic IOMMU DT bindings [1] which can be used for both DT (right now) and ACPI (in future).
For that reason we can borrow the idea used in Linux these days called "iommu_fwspec". Having this in, it will be possible to configure IOMMU master interfaces of the device (device IDs) from a single common place and avoid keeping almost identical look-up implementations in each IOMMU driver.
There is no need to port the whole implementation of "iommu_fwspec" to Xen, we could, probably, end up with a much simpler solution, some "stripped down" version which fits our requirements.
So, this patch adds the following:
1. A common structure "iommu_fwspec" to hold the the per-device firmware data
2. New member "iommu_fwspec" of struct device
3. Functions/helpers to deal with "dev->iommu_fwspec"
It should be noted that in comparison of the original "iommu_fwspec" Xen's variant doesn't contain some fields, which are not really needed at the moment (ops, flag) and "iommu_fwnode" field was replaced by "iommu_dev" to avoid porting a lot of code (to support "fwnode_handle") with little benefit.
The "iommu_fwspec" support is based on the Linux's commit:f74c2bb98776e2de508f4d607cd519873065118e "Linux 5.3-rc8"
Subsequent patches will use of that support.
[1] https://www.kernel.org/doc/Documentation/devicetree/bindings/iommu/iommu.txt
---
Changes V4 -> V5:
- modified struct iommu_fwspec to use flexible array (uint32_t ids[])
- modified to use xzalloc_flex_struct() on the first allocation
- added "const" to uint32_t *ids in iommu_fwspec_add_ids()
Changes V3 -> V4:
- modified iommu_fwspec_add_ids() to use new implementation of xrealloc_flex_struct()
- mentioned exact Linux version we are based on
- fixed Grammatical error
Changes V2 -> V3:
- added Copyright from Linux
- ordered the headers alphabetically
- removed check for not a NULL before calling xfree()
- used unsigned for variables which can't be negative
- removed #include <asm/iommu_fwspec.h> from iommu.h
- added check to iommu_fwspec_init() to not allow overriding
- clarified comments in code
- modified iommu_fwspec_add_ids() to use type-safe xrealloc_flex_struct()
--- xen/drivers/passthrough/arm/Makefile | 2 +- xen/drivers/passthrough/arm/iommu_fwspec.c | 97 ++++++++++++++++++++++++++++++ xen/include/asm-arm/device.h | 1 + xen/include/asm-arm/iommu_fwspec.h | 68 +++++++++++++++++++++ 4 files changed, 167 insertions(+), 1 deletion(-) create mode 100644 xen/drivers/passthrough/arm/iommu_fwspec.c create mode 100644 xen/include/asm-arm/iommu_fwspec.h
xen/drivers/passthrough/arm/Makefile +1 -1
@@ -1,2 +1,2 @@ -obj-y += iommu.o iommu_helpers.o +obj-y += iommu.o iommu_helpers.o iommu_fwspec.o obj-$(CONFIG_ARM_SMMU) += smmu.o
xen/drivers/passthrough/arm/iommu_fwspec.c new file +97 -0
97 lines not on this page · raw message
xen/include/asm-arm/device.h +1 -0
@@ -18,6 +18,7 @@ struct devicestruct dt_device_node *of_node;/* Used by drivers imported from Linux */ #endif struct dev_archdata archdata; +struct iommu_fwspec *iommu_fwspec;/* per-device IOMMU instance data */ };typedef struct device device_t ;
xen/include/asm-arm/iommu_fwspec.h new file +68 -0
68 lines not on this page · raw message
Signature · 7 lines
2.7.4 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH V5 5/8] iommu/arm: Add lightweight iommu_fwspec support
Hi Oleksandr,
Quoted 28 lines · Oleksandr Tyshchenko
From: Oleksandr Tyshchenko <[email protected]>
We need to have some abstract way to add new device to the IOMMU based on the generic IOMMU DT bindings [1] which can be used for both DT (right now) and ACPI (in future).
For that reason we can borrow the idea used in Linux these days called "
iommu_fwspec". Having this in, it will be possible to configure IOMMU master interfaces of the device (device IDs) from a single common place and avoid keeping almost identical look-up implementations in each IOMMU driver.There is no need to port the whole implementation of "
iommu_fwspec" to Xen, we could, probably, end up with a much simpler solution, some "stripped down" version which fits our requirements.So, this patch adds the following:
1. A common structure "iommu_fwspec" to hold the the per-device firmware data
2. New member "iommu_fwspec" of struct device
3. Functions/helpers to deal with "dev->iommu_fwspec"It should be noted that in comparison of the original "
iommu_fwspec" Xen's variant doesn't contain some fields, which are not really needed at the moment (ops, flag) and "iommu_fwnode" field was replaced by "iommu_dev" to avoid porting a lot of code (to support "fwnode_handle") with little benefit.
Quoted 8 lines
The "
iommu_fwspec" support is based on the Linux's commit:f74c2bb98776e2de508f4d607cd519873065118e"Linux 5.3-rc8"Subsequent patches will use of that support.
[1] https://www.kernel.org/doc/Documentation/devicetree/bindings/iommu/iommu.txt
Signed-off-by Oleksandr Tyshchenko <[email protected]>
Cheers,
Signature · 6 lines
Julien Grall _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v5 6/8 iommu: Order the headers alphabetically in device_tree.c
In reply to Oleksandr TyshchenkoFrom: Oleksandr Tyshchenko <[email protected]>
Clean up the code a bit by putting the headers in alphabetical order.
--- xen/drivers/passthrough/device_tree.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
xen/drivers/passthrough/device_tree.c +3 -3
@@ -15,11 +15,11 @@* GNU General Public License for more details. */ -#include <xen/lib.h> -#include <xen/sched.h> +#include <xen/device_tree.h> #include <xen/guest_access.h> #include <xen/iommu.h> -#include <xen/device_tree.h> +#include <xen/lib.h> +#include <xen/sched.h> #include <xsm/xsm.h> static spinlock_t dtdevs_lock =SPIN_LOCK_UNLOCKED ;
Signature · 7 lines
2.7.4 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH V5 6/8] iommu: Order the headers alphabetically in device_tree.c
Hi,
Quoted 5 lines · Oleksandr Tyshchenko
From: Oleksandr Tyshchenko <[email protected]>
Clean up the code a bit by putting the headers in alphabetical order.
Signed-off-by Oleksandr Tyshchenko <[email protected]>
Cheers,
Quoted 24 lines
--- xen/drivers/passthrough/device_tree.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)xen/drivers/passthrough/device_tree.c +3
-3@@ -15,11 +15,11 @@* GNU General Public License for more details. */ -#include <xen/lib.h> -#include <xen/sched.h> +#include <xen/device_tree.h> #include <xen/guest_access.h> #include <xen/iommu.h> -#include <xen/device_tree.h> +#include <xen/lib.h> +#include <xen/sched.h> #include <xsm/xsm.h> static spinlock_t dtdevs_lock =SPIN_LOCK_UNLOCKED ;
Signature · 6 lines
Julien Grall _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v5 7/8 iommu/arm: Introduce iommu_add_dt_device API
In reply to Oleksandr TyshchenkoFrom: Oleksandr Tyshchenko <[email protected]>
The main puprose of this patch is to add a way to register DT device (which is behind the IOMMU) using the generic IOMMU DT bindings [1] before assigning that device to a domain.
So, this patch adds new "iommu_add_dt_device" API for adding DT device to the IOMMU using generic IOMMU DT bindings and previously added "iommu_fwspec" support. As devices can be assigned to the hardware domain and other domains this function is called from two places: handle_device() and iommu_do_dt_domctl().
Besides that, this patch adds new "dt_xlate" callback (borrowed from Linux "of_xlate") for providing the driver with DT IOMMU specifier which describes the IOMMU master interfaces of that device (device IDs, etc). According to the generic IOMMU DT bindings the context of required properties for IOMMU device/master node (#iommu-cells, iommus) depends on many factors and is really driver depended thing.
Please note, all IOMMU drivers which support generic IOMMU DT bindings should use "dt_xlate" and "add_device" callbacks.
[1] https://www.kernel.org/doc/Documentation/devicetree/bindings/iommu/iommu.txt
---
Changes V4 -> V5:
- added "const" to struct dt_phandle_args *args in dt_xlate
- moved iommu_add_dt_device() to xen/passthrough/device_tree.c
- modified logic, don't try to add "all" devices to the IOMMU when constructing Dom0, but only devices which are going to be assigned to hwdom
- updated iommu_do_dt_domctl() to call iommu_add_dt_device()
- clarified patch description
- removed "__init" from iommu_add_dt_device()
Changes V3 -> V4:
- squashed with "iommu: Add of_xlate callback" patch
- renamed "of_xlate" to "dt_xlate"
- reworked patch description
- clarified comments in code, removed confusing word "initialize device", etc
- updated debug message in handle_device()
- modified to check ops->of_xlate and ops->add_device only if "iommus" property is exists
Changes V2 -> V3:
- clarified patch description
- clarified comments in code
- modified to provide DT IOMMU specifier to the driver using "of_xlate" callback
- documented function usage
- modified to return an error if ops is not present/implemented,
- added ability to return a possitive value to indicate that device doesn't need to be protected
- removed check for the "iommu" property presence
in the common code
- included <asm/iommu_fwspec.h> directly
--- xen/arch/arm/domain_build.c | 22 +++++++---- xen/drivers/passthrough/device_tree.c | 71 +++++++++++++++++++++++++++++++++++ xen/include/xen/iommu.h | 21 +++++++++++ 3 files changed, 107 insertions(+), 7 deletions(-)
xen/arch/arm/domain_build.c +15 -7
@@ -1243,6 +1243,7 @@ static int __init map_device_children(struct domain *d,* - Give permission to the guest to manage IRQ and MMIO range * - Retrieve the IRQ configuration (i.e edge/level) from device tree * When the device is not marked for guest passthrough: +* - Try to call iommu_add_dt_device to protect the device by an IOMMU * - Assign the device to the guest if it's protected by an IOMMU * - Map the IRQs and iomem regions to DOM0 */
@@ -1263,15 +1264,22 @@ static int __init handle_device(struct domain *d, struct dt_device_node *dev,dt_dprintk ("%s passthrough = %d nirq = %d naddr = %u\n" ,dt_node_full_name (dev), need_mapping, nirq, naddr); -if (dt_device_is_protected (dev) && need_mapping ) +if ( need_mapping ) { -dt_dprintk ("%s setup iommu\n" ,dt_node_full_name (dev)); - res =iommu_assign_dt_device (d, dev); -if ( res ) - { -printk (XENLOG_ERR "Failed to setup the IOMMU for %s\n" , +dt_dprintk ("Check if %s is behind the IOMMU and add it\n" ,dt_node_full_name (dev)); -return res; + +iommu_add_dt_device (dev); +if (dt_device_is_protected (dev) ) + { +dt_dprintk ("%s setup iommu\n" ,dt_node_full_name (dev)); + res =iommu_assign_dt_device (d, dev); +if ( res ) + { +printk (XENLOG_ERR "Failed to setup the IOMMU for %s\n" , +dt_node_full_name (dev)); +return res; + } } }
xen/drivers/passthrough/device_tree.c +71 -0
89 lines not on this page · raw message
xen/include/xen/iommu.h +21 -0
@@ -186,6 +186,17 @@ int iommu_deassign_dt_device(struct domain *d, struct dt_device_node *dev);int iommu_dt_domain_init (struct domain *d);int iommu_release_dt_devices (struct domain *d); +/* +* Helper to add master device to the IOMMU using generic IOMMU DT bindings. +* +* Return values: +* 0 : device is protected by an IOMMU +* <0 : device is not protected by an IOMMU, but must be (error condition) +* >0 : device doesn't need to be protected by an IOMMU +* (IOMMU is not enabled/present or device is not connected to it). +*/ +int iommu_add_dt_device (struct dt_device_node *np); +int iommu_do_dt_domctl (struct xen_domctl *,struct domain *,XEN_GUEST_HANDLE_PARAM (xen_domctl_t ));
@@ -254,6 +265,16 @@ struct iommu_ops {int __must_check (*iotlb_flush_all)(struct domain *d);int (*get_reserved_device_memory)(iommu_grdm_t *,void *);void (*dump_p2m_table)(struct domain *d); + +#ifdef CONFIG_HAS_DEVICE_TREE +/* +* All IOMMU drivers which support generic IOMMU DT bindings should use +* this callback. This is a way for the framework to provide the driver +* with DT IOMMU specifier which describes the IOMMU master interfaces of +* that device (device IDs, etc). +*/ +int (*dt_xlate)(device_t *dev,const struct dt_phandle_args *args); +#endif };#include <asm/iommu.h>
Signature · 7 lines
2.7.4 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH V5 7/8] iommu/arm: Introduce iommu_add_dt_device API
Hi,
Quoted 7 lines · Oleksandr Tyshchenko
xen/arch/arm/domain_build.c +1
-1@@ -1263,15 +1264,22 @@ static int __init handle_device(struct domain *d, struct dt_device_node *dev,dt_dprintk ("%s passthrough = %d nirq = %d naddr = %u\n" ,dt_node_full_name (dev), need_mapping, nirq, naddr); -if (dt_device_is_protected (dev) && need_mapping ) +if ( need_mapping ) {
xen/arch/arm/domain_build.c static int __init handle_device(struct domain *d, struct dt_device_node *dev, · line 1272
Quoted 10 lines
xen/arch/arm/domain_build.c +3
-6-dt_dprintk ("%s setup iommu\n" ,dt_node_full_name (dev)); - res =iommu_assign_dt_device (d, dev); -if ( res ) - { -printk (XENLOG_ERR "Failed to setup the IOMMU for %s\n" , +dt_dprintk ("Check if %s is behind the IOMMU and add it\n" ,dt_node_full_name (dev)); -return res; + +iommu_add_dt_device (dev);
Return value should always be checked or explain why this is not done.
[...]
xen/drivers/passthrough/device_tree.c int iommu_do_dt_domctl(struct xen_domctl *domctl, struct domain *d, · line 244
Quoted 8 lines
xen/drivers/passthrough/device_tree.c +1
-0int iommu_do_dt_domctl (struct xen_domctl *domctl,struct domain *d,XEN_GUEST_HANDLE_PARAM (xen_domctl_t) u_domctl) {@@ -177,6 +241,13 @@ int iommu_do_dt_domctl(struct xen_domctl *domctl, struct domain *d,break ; } +iommu_add_dt_device (dev);
Same here.
xen/drivers/passthrough/device_tree.c int iommu_do_dt_domctl(struct xen_domctl *domctl, struct domain *d, · line 249
Quoted 5 lines
xen/drivers/passthrough/device_tree.c +5
-0+if ( !dt_device_is_protected (dev) ) + { + ret = -EINVAL ; +break ; + }
This is already checked in iommu_assign_dt_device().
Cheers,
Signature · 6 lines
Julien Grall _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH V5 7/8] iommu/arm: Introduce iommu_add_dt_device API
Quoted 1 line · Julien Grall
Hi,
Hi Julien
Quoted 13 lines · Oleksandr Tyshchenko
Quoted 11 lines
xen/arch/arm/domain_build.c +1
-4@@ -1263,15 +1264,22 @@ static int __init handle_device(struct domain *d, struct dt_device_node *dev, dt_dprintk("%s passthrough = %d nirq = %d naddr = %u\n", dt_node_full_name(dev), need_mapping, nirq, naddr); - if ( dt_device_is_protected(dev) && need_mapping ) + if ( need_mapping ) { - dt_dprintk("%s setup iommu\n", dt_node_full_name(dev)); - res = iommu_assign_dt_device(d, dev); - if ( res ) - {
Quoted 8 lines
xen/arch/arm/domain_build.c static int __init handle_device(struct domain *d, struct dt_device_node *dev, · line 1272
Quoted 6 lines
xen/arch/arm/domain_build.c +3
-2- printk(XENLOG_ERR "Failed to setup the IOMMU for %s\n", + dt_dprintk("Check if %s is behind the IOMMU and add it\n", dt_node_full_name(dev)); - return res; + + iommu_add_dt_device(dev);Return value should always be checked or explain why this is not done.
Yes, I will add a check. The positive result for us is non-negative (either "device is protected" or "device doesn't need to be protected").
Quoted 4 lines
[...]
Quoted 10 lines
Review int iommu_do_dt_domctl(struct xen_domctl · line 244
Quoted 8 lines
int
iommu_do_dt_domctl(struct xen_domctl *domctl, struct domain *d, XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl){(quoted diff) +0
-0@@ -177,6 +241,13 @@ int iommu_do_dt_domctl(struct xen_domctl *domctl, struct domain *d, break; } + iommu_add_dt_device(dev);Same here.
Yes, I think, we don't need to check for return value, because the only one positive result "here" is the fact that "device is protected" (which is checked below).
What is more, if we add a check for the return value to be strictly 0, we will get an error after guest's reboot (since iommu_add_dt_device() will return -EEXIST).
So, I will add a comment explaining why we don't check. What do you think?
Quoted 8 lines
xen/drivers/passthrough/device_tree.c int iommu_do_dt_domctl(struct xen_domctl *domctl, struct domain *d, · line 249
Quoted 6 lines
xen/drivers/passthrough/device_tree.c +5
-0+ if ( !dt_device_is_protected(dev) ) + { + ret = -EINVAL; + break; + }This is already checked in
iommu_assign_dt_device().
Will drop here.
Signature · 9 lines
Regards, Oleksandr Tyshchenko _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH V5 7/8] iommu/arm: Introduce iommu_add_dt_device API
Hi Oleksandr,
Quoted 47 lines · Oleksandr
Quoted 1 line · Julien Grall
Hi,
Hi Julien
Quoted 13 lines · Oleksandr Tyshchenko
Quoted 11 lines
xen/arch/arm/domain_build.c +1
-4@@ -1263,15 +1264,22 @@ static int __init handle_device(struct domain *d, struct dt_device_node *dev, dt_dprintk("%s passthrough = %d nirq = %d naddr = %u\n", dt_node_full_name(dev), need_mapping, nirq, naddr); - if ( dt_device_is_protected(dev) && need_mapping ) + if ( need_mapping ) { - dt_dprintk("%s setup iommu\n", dt_node_full_name(dev)); - res = iommu_assign_dt_device(d, dev); - if ( res ) - {Quoted 8 lines
xen/arch/arm/domain_build.c static int __init handle_device(struct domain *d, struct dt_device_node *dev, · line 1272
Quoted 6 lines
xen/arch/arm/domain_build.c +3
-2- printk(XENLOG_ERR "Failed to setup the IOMMU for %s\n", + dt_dprintk("Check if %s is behind the IOMMU and add it\n", dt_node_full_name(dev)); - return res; + + iommu_add_dt_device(dev);Return value should always be checked or explain why this is not done.
Yes, I will add a check. The positive result for us is non-negative (either "device is protected" or "device doesn't need to be protected").
Quoted 4 lines
[...]
Quoted 10 lines
Review int iommu_do_dt_domctl(struct xen_domctl · line 244
Quoted 8 lines
int
iommu_do_dt_domctl(struct xen_domctl *domctl, struct domain *d, XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl){(quoted diff) +0
-0@@ -177,6 +241,13 @@ int iommu_do_dt_domctl(struct xen_domctl *domctl, struct domain *d, break; } + iommu_add_dt_device(dev);Same here.
Quoted 5 lines
Yes, I think, we don't need to check for return value, because the only one positive result "here" is the fact that "device is protected" (which is checked below).
What is more, if we add a check for the return value to be strictly 0, we will get an error after guest's reboot (since
iommu_add_dt_device()will return-EEXIST).So, I will add a comment explaining why we don't check. What do you think?
Why don't you do the following code?
if ( ret < 0 && ret != -EEXIST )
This would allow the code to return the corrrect error to the upper layer. A suitable comment on top explaing the check would also be useful.
Cheers,
Signature · 6 lines
Julien Grall _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH V5 7/8] iommu/arm: Introduce iommu_add_dt_device API
Quoted 1 line · Julien Grall
Hi Oleksandr,
Hi Julien.
Quoted 20 lines
Quoted 20 lines
Quoted 5 lines
[...]
Quoted 10 lines
Review int iommu_do_dt_domctl(struct xen_domctl · line 244
Quoted 8 lines
int
iommu_do_dt_domctl(struct xen_domctl *domctl, struct domain *d, XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
{(quoted diff) +0
-0@@ -177,6 +241,13 @@ int iommu_do_dt_domctl(struct xen_domctl *domctl, struct domain *d, break; } + iommu_add_dt_device(dev);Same here.
Yes, I think, we don't need to check for return value, because the only one positive result "here" is the fact that "device is protected" (which is checked below).
What is more, if we add a check for the return value to be strictly 0, we will get an error after guest's reboot (since
iommu_add_dt_device()will return-EEXIST).
Quoted 7 lines
Quoted 1 line
So, I will add a comment explaining why we don't check. What do you think?
Why don't you do the following code?
if ( ret < 0 && ret !=
-EEXIST)This would allow the code to return the corrrect error to the upper layer. A suitable comment on top explaing the check would also be useful.
Being honest, I was thinking about the similar, but rejected this. I thought, all what we wanted to know "here" was whether the particular device protected or not. But, I agree now, the upper layer should be informed about the exact error reason.
Signature · 9 lines
Regards, Oleksandr Tyshchenko _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v5 8/8 iommu/arm: Add Renesas IPMMU-VMSA support
In reply to Oleksandr TyshchenkoFrom: Oleksandr Tyshchenko <[email protected]>
The IPMMU-VMSA is VMSA-compatible I/O Memory Management Unit (IOMMU) which provides address translation and access protection functionalities to processing units and interconnect networks.
Please note, current driver is supposed to work only with newest R-Car Gen3 SoCs revisions which IPMMU hardware supports stage 2 translation table format and is able to use CPU's P2M table as is if one is 3-level page table (up to 40 bit IPA).
The major differences compare to the Linux driver are:
1. Stage 1/Stage 2 translation. Linux driver supports Stage 1 translation only (with Stage 1 translation table format). It manages page table by itself. But Xen driver supports Stage 2 translation (with Stage 2 translation table format) to be able to share the P2M with the CPU. Stage 1 translation is always bypassed in Xen driver.
So, Xen driver is supposed to be used with newest R-Car Gen3 SoC revisions only (H3 ES3.0, M3-W+, etc.) which IPMMU H/W supports stage 2 translation table format.
2. AArch64 support. Linux driver uses VMSAv8-32 mode, while Xen driver enables Armv8 VMSAv8-64 mode to cover up to 40 bit input address.
3. Context bank (sets of page table) usage. In Xen, each context bank is mapped to one Xen domain. So, all devices being pass throughed to the same Xen domain share the same context bank.
4. IPMMU device tracking. In Xen, all IOMMU devices are managed by single driver instance. So, driver uses global list to keep track of registered IPMMU devices.
---
Changes V4 -> V5:
- added R-b/A-b
- added "const" to struct dt_phandle_args *spec in ipmmu_dt_xlate()
- added "const" to struct dt_phandle_args *args in ipmmu_init_platform_device()
Changes V3 -> V4:
- changed "Gen3" to "R-Car Gen3" in commit description
- renamed "of_xlate" to "dt_xlate"
- removed check for the iommu_hap_pt_share flag in ipmmu_init()
- modified to be able to select IPMMU_VMSA option only if EXPERT is set
- removed an error message when deferring the probe
- moved check for the P2M sharing possibility to the Root IPMMU path
Changes V2 -> V3:
- added Copyright from Linux
- included <xen/iommu.h> and <asm/iommu_fwspec.h> directly
- removed pointless spin_lock from ipmmu_iommu_domain_teardown()
- changed "Gen3" to "R-Car Gen3", "M3 ES3.0" to "M3-W+"
- changed RCAR_PRODUCT_M3 to RCAR_PRODUCT_M3W
- clarified IMTTBCR_TSZ1_MASK
- removed TLB flush from ipmmu_domain_irq() until clarified the purpose
- returned -ENODEV in ipmmu_init() if P2M sharing or P2M IPA size is not supported to show that IOMMU device is not usable in Xen
- implemented new callback "of_xlate" to parse DT IOMMU specifier and add device IDs to "iommu_fwspec"
- reworked "add_device" callback
- updated patch description
Changes V1 -> V2:
- rewrited driver to use iommu_fwspec
- removed DT parsing code for micro-TLBs
- removed struct ipmmu_vmsa_master_cfg, dev_archdata macro
- added ipmmu_find_mmu_by_dev(), various helpers to access fwspec->iommu_priv
- implemented new callback "add_device" to add master device to IPMMU
- removed ipmmu_protect_masters()
- removed code to locate Root device in the first place, used EAGAIN to request deferred probing
- used printk_once for the system wide error messages in ipmmu_init() which don't need to be shown for every device being probed
- removed map_page/unmap_page implementation, reused them from iommu_helpers.c
- used %pd for printing domaid id
- performed various cosmetic fixes
- changed u32 -> uint32_t, u64 -> uint64_t,
unsigned int -> uint32_t where needed
- clarified TODOs
- clafiried supported SoC versions in config IPMMU_VMSA, set default to "n"
- updated comments in code, provided more accurate description, added new comments where needed
- updated patch description by providing differences between Linux/Xen implementations
- removed fields for cache snoop transaction when configuring IMTTBCR (update from Renesas BSP)
--- xen/arch/arm/platforms/Kconfig | 1 + xen/drivers/passthrough/Kconfig | 13 + xen/drivers/passthrough/arm/Makefile | 1 + xen/drivers/passthrough/arm/ipmmu-vmsa.c | 1333 ++++++++++++++++++++++++++++++ 4 files changed, 1348 insertions(+) create mode 100644 xen/drivers/passthrough/arm/ipmmu-vmsa.c
xen/arch/arm/platforms/Kconfig +1 -0
@@ -25,6 +25,7 @@ config RCAR3 bool "Renesas RCar3 support" depends on ARM_64 select HAS_SCIF + select IPMMU_VMSA if EXPERT ---help--- Enable all the required drivers for Renesas RCar3
xen/drivers/passthrough/Kconfig +13 -0
@@ -12,4 +12,17 @@ config ARM_SMMU Say Y here if your SoC includes an IOMMU device implementing the ARM SMMU architecture. + +config IPMMU_VMSA + bool "Renesas IPMMU-VMSA found in R-Car Gen3 SoCs" if EXPERT = "y" + default n + depends on ARM_64 + ---help--- + Support for implementations of the Renesas IPMMU-VMSA found + in R-Car Gen3 SoCs. + + Say Y here if you are using newest R-Car Gen3 SoCs revisions + (H3 ES3.0, M3-W+, etc) which IPMMU hardware supports stage 2 + translation table format and is able to use CPU's P2M table as is. + endif
xen/drivers/passthrough/arm/Makefile +1 -0
@@ -1,2 +1,3 @@ obj-y += iommu.o iommu_helpers.o iommu_fwspec.o obj-$(CONFIG_ARM_SMMU) += smmu.o +obj-$(CONFIG_IPMMU_VMSA) += ipmmu-vmsa.o
xen/drivers/passthrough/arm/ipmmu-vmsa.c new file +1333 -0
1,333 lines not on this page · raw message
Signature · 7 lines
2.7.4 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel