xen/arm: fix "xen_add_mach_to_phys_entry: cannot add" problem
Subsystems (3)
v2 cover xen/arm: fix "xen_add_mach_to_phys_entry: cannot add" problem
Hi all,
Xen support in Linux for ARM and ARM64 suffers from lack of support for multiple mfn to pfn mappings: whenever a frontend grants the same page multiple times to the backend, the mfn to pfn accounting in arch/arm/xen/p2m.c fails. The issue has become critical since v3.15, when xen-netfront/xen-netback switched from grant copies to grant mappings, therefore causing the issue to happen much more often.
Fixing the mfn to pfn accounting in p2m.c is difficult and expensive, therefore we are looking for alternative solutions. One idea is avoiding mfn to pfn conversions altogether. The only code path that needs them is swiotlb-xen:unmap_page (and single_for_cpu and single_for_device).
To avoid mfn to pfn conversions we rely on a second p2m mapping done by Xen (a separate patch series will be sent for Xen). In Linux we use it to perform the cache maintenance operations without mfns conversions.
Changes in v2:
- introduce XENFEAT_grant_map_11;
- remeber the ptep corresponding to scratch pages so that we don't need to calculate it again every time;
- do not acutally unmap the page on xen_mm32_unmap;
- properly account preempt_enable/disable;
- do not check for mfn in xen_add_phys_to_mach_entry.
Stefano Stabellini (3):
xen/arm: introduce XENFEAT_grant_map_11 xen/arm: reimplement xen_dma_unmap_page & friends xen/arm: remove mach_to_phys rbtree
arch/arm/include/asm/xen/page-coherent.h | 25 ++-- arch/arm/include/asm/xen/page.h | 9 -- arch/arm/xen/Makefile | 2 +- arch/arm/xen/enlighten.c | 6 + arch/arm/xen/mm32.c | 202 ++++++++++++++++++++++++++++++ arch/arm/xen/p2m.c | 66 +--------- include/xen/interface/features.h | 3 + 7 files changed, 220 insertions(+), 93 deletions(-) create mode 100644 arch/arm/xen/mm32.c
PATCH v2 1/3 xen/arm: introduce XENFEAT_grant_map_11
The flag tells us that the hypervisor maps a grant page to guest physical address == machine address of the page in addition to the normal grant mapping address. It is needed to properly issue cache maintenance operation at the completion of a DMA operation involving a foreign grant.
--- arch/arm/xen/enlighten.c | 6 ++++++ include/xen/interface/features.h | 3 +++ 2 files changed, 9 insertions(+)
arch/arm/xen/enlighten.c +6 -0
@@ -262,6 +262,12 @@ static int __init xen_guest_init(void) xen_domain_type =XEN_HVM_DOMAIN ;xen_setup_features (); + +if (!xen_feature (XENFEAT_grant_map_11)) { +pr_warn ("Please upgrade your Xen.\n" +"If your platform has any non-coherent DMA devices, they won't work properly.\n" ); + } +if (xen_feature (XENFEAT_dom0)) xen_start_info->flags |=SIF_INITDOMAIN |SIF_PRIVILEGED ;else
include/xen/interface/features.h +3 -0
@@ -53,6 +53,9 @@/* operation as Dom0 is supported */ #define XENFEAT_dom0 11 +/* Xen also maps grant references at pfn = mfn */ +#define XENFEAT_grant_map_11 12 +#define XENFEAT_NR_SUBMAPS 1#endif /* __XEN_PUBLIC_FEATURES_H__ */
Signature · 1 line
1.7.10.4
PATCH v2 1/3 xen/arm: introduce XENFEAT_grant_map_11
Quoted 17 lines · Stefano Stabellini
The flag tells us that the hypervisor maps a grant page to guest physical address == machine address of the page in addition to the normal grant mapping address. It is needed to properly issue cache maintenance operation at the completion of a DMA operation involving a foreign grant.
Signed-off-by Stefano Stabellini <[email protected]>--- arch/arm/xen/enlighten.c | 6 ++++++ include/xen/interface/features.h | 3 +++ 2 files changed, 9 insertions(+)arch/arm/xen/enlighten.c +0
-0
arch/arm/xen/enlighten.c static int __init xen_guest_init(void) · line 269
Quoted 8 lines
arch/arm/xen/enlighten.c +5
-0@@ -262,6 +262,12 @@ static int __init xen_guest_init(void) xen_domain_type =XEN_HVM_DOMAIN ;xen_setup_features (); + +if (!xen_feature (XENFEAT_grant_map_11)) { +pr_warn ("Please upgrade your Xen.\n" +"If your platform has any non-coherent DMA devices, they won't work properly.\n" ); + }
Unfortunately this isn't quite complete. On a system where all devices are behind an SMMU then we would want to be able to disable the 1:1 workaround, which in turn would imply disabling this feature flag too (since it is no longer necessary and also impossible to implement in that case).
Ian.
PATCH v2 1/3 xen/arm: introduce XENFEAT_grant_map_11
Quoted 20 lines · Ian Campbell
Quoted 19 lines · Stefano Stabellini
The flag tells us that the hypervisor maps a grant page to guest physical address == machine address of the page in addition to the normal grant mapping address. It is needed to properly issue cache maintenance operation at the completion of a DMA operation involving a foreign grant.
Signed-off-by Stefano Stabellini <[email protected]>--- arch/arm/xen/enlighten.c | 6 ++++++ include/xen/interface/features.h | 3 +++ 2 files changed, 9 insertions(+)arch/arm/xen/enlighten.c +0
-0@@ -262,6 +262,12 @@ static int __init xen_guest_init(void) xen_domain_type =XEN_HVM_DOMAIN ;
Quoted 12 lines
arch/arm/xen/enlighten.c static int __init xen_guest_init(void) · line 269
Quoted 6 lines
arch/arm/xen/enlighten.c +5
-0xen_setup_features (); + +if (!xen_feature (XENFEAT_grant_map_11)) { +pr_warn ("Please upgrade your Xen.\n" +"If your platform has any non-coherent DMA devices, they won't work properly.\n" ); + }Unfortunately this isn't quite complete. On a system where all devices are behind an SMMU then we would want to be able to disable the 1:1 workaround, which in turn would imply disabling this feature flag too (since it is no longer necessary and also impossible to implement in that case).
That is true, but in such a system we would have to tell the kernel that DMAing is safe, so this will turn into:
if (!xen_feature(XENFEAT_grant_map_11) &&
!xen_feature(XENFEAT_safe_dma)) {
pr_warn("Please upgrade your Xen.\n"
"If your platform has any non-coherent DMA devices, they won't work properly.\n");
}PATCH v2 1/3 xen/arm: introduce XENFEAT_grant_map_11
Quoted 34 lines · Stefano Stabellini
Quoted 20 lines · Ian Campbell
Quoted 19 lines · Stefano Stabellini
The flag tells us that the hypervisor maps a grant page to guest physical address == machine address of the page in addition to the normal grant mapping address. It is needed to properly issue cache maintenance operation at the completion of a DMA operation involving a foreign grant.
Signed-off-by Stefano Stabellini <[email protected]>--- arch/arm/xen/enlighten.c | 6 ++++++ include/xen/interface/features.h | 3 +++ 2 files changed, 9 insertions(+)arch/arm/xen/enlighten.c +0
-0@@ -262,6 +262,12 @@ static int __init xen_guest_init(void) xen_domain_type =XEN_HVM_DOMAIN ;Quoted 12 lines
arch/arm/xen/enlighten.c static int __init xen_guest_init(void) · line 269
Quoted 6 lines
arch/arm/xen/enlighten.c +5
-0xen_setup_features (); + +if (!xen_feature (XENFEAT_grant_map_11)) { +pr_warn ("Please upgrade your Xen.\n" +"If your platform has any non-coherent DMA devices, they won't work properly.\n" ); + }Unfortunately this isn't quite complete. On a system where all devices are behind an SMMU then we would want to be able to disable the 1:1 workaround, which in turn would imply disabling this feature flag too (since it is no longer necessary and also impossible to implement in that case).
Quoted 2 lines
That is true, but in such a system we would have to tell the kernel that DMAing is safe, so this will turn into:
Oh right, yes. Good then ;-)
Quoted 6 lines
if (!xen_feature(XENFEAT_grant_map_11) &&
!xen_feature(XENFEAT_safe_dma)) { pr_warn("Please upgrade your Xen.\n" "If your platform has any non-coherent DMA devices, they won't work properly.\n"); }
PATCH v2 1/3 xen/arm: introduce XENFEAT_grant_map_11
Hi Ian and Stefano,
Quoted 35 lines · Ian Campbell
Quoted 34 lines · Stefano Stabellini
Quoted 32 lines · Ian Campbell
Quoted 17 lines · Stefano Stabellini
The flag tells us that the hypervisor maps a grant page to guest physical address == machine address of the page in addition to the normal grant mapping address. It is needed to properly issue cache maintenance operation at the completion of a DMA operation involving a foreign grant.
Signed-off-by Stefano Stabellini <[email protected]>--- arch/arm/xen/enlighten.c | 6 ++++++ include/xen/interface/features.h | 3 +++ 2 files changed, 9 insertions(+)arch/arm/xen/enlighten.c +0
-0arch/arm/xen/enlighten.c static int __init xen_guest_init(void) · line 269
Quoted 8 lines
arch/arm/xen/enlighten.c +5
-0@@ -262,6 +262,12 @@ static int __init xen_guest_init(void) xen_domain_type =XEN_HVM_DOMAIN ;xen_setup_features (); + +if (!xen_feature (XENFEAT_grant_map_11)) { +pr_warn ("Please upgrade your Xen.\n" +"If your platform has any non-coherent DMA devices, they won't work properly.\n" ); + }Unfortunately this isn't quite complete. On a system where all devices are behind an SMMU then we would want to be able to disable the 1:1 workaround, which in turn would imply disabling this feature flag too (since it is no longer necessary and also impossible to implement in that case).
Quoted 4 lines
Quoted 2 lines
That is true, but in such a system we would have to tell the kernel that DMAing is safe, so this will turn into:
Oh right, yes. Good then ;-)
FWIW, I've sent a patch series a couple ago to avoid using swiotlb when the device is protected (see https://patches.linaro.org/25070/).
I should take time to rework properly and send a new version.
Regards,
Signature · 1 line
Julien Grall
PATCH v2 1/3 xen/arm: introduce XENFEAT_grant_map_11
In reply to Stefano StabelliniQuoted 5 lines · Stefano Stabellini
The flag tells us that the hypervisor maps a grant page to guest physical address == machine address of the page in addition to the normal grant mapping address. It is needed to properly issue cache maintenance operation at the completion of a DMA operation involving a foreign grant.
[...]
include/xen/interface/features.h line 57
Quoted 2 lines
include/xen/interface/features.h +2
-0+/* Xen also maps grant references at pfn = mfn */ +#define XENFEAT_grant_map_11 12
I keep reading this as "grant map eleven". I think you've used this abbreviation else where so you should probably keep it as-is.
I might have picked XENFEAT_grant_map_identity.
David
PATCH v2 2/3 xen/arm: reimplement xen_dma_unmap_page & friends
In reply to Stefano Stabellinixen_dma_unmap_page, xen_dma_sync_single_for_cpu and xen_dma_sync_single_for_device are currently implemented by calling into the corresponding generic ARM implementation of these functions. In order to do this, firstly the dma_addr_t handle, that on Xen is a machine address, needs to be translated into a physical address. The operation is expensive and inaccurate, given that a single machine address can correspond to multiple physical addresses in one domain, because the same page can be granted multiple times by the frontend.
To avoid this problem, we introduce a Xen specific implementation of xen_dma_unmap_page, xen_dma_sync_single_for_cpu and xen_dma_sync_single_for_device, that can operate on machine addresses directly.
The new implementation relies on the fact that the hypervisor creates a second p2m mapping of any grant pages at physical address == machine address of the page for dom0. Therefore we can access memory at physical address == dma_addr_r handle and perform the cache flushing there. Some cache maintenance operations require a virtual address. Instead of using ioremap_cache, that is not safe in interrupt context, we allocate a per-cpu PAGE_KERNEL scratch page and we manually update the pte for it.
arm64 doesn't need cache maintenance operations on unmap for now.
---
Changes in v2:
- check for XENFEAT_grant_map_11;
- remeber the ptep corresponding to scratch pages so that we don't need to calculate it again every time;
- do not acutally unmap the page on xen_mm32_unmap;
- properly account preempt_enable/disable.
--- arch/arm/include/asm/xen/page-coherent.h | 25 ++-- arch/arm/xen/Makefile | 2 +- arch/arm/xen/mm32.c | 202 ++++++++++++++++++++++++++++++ 3 files changed, 210 insertions(+), 19 deletions(-) create mode 100644 arch/arm/xen/mm32.c
arch/arm/include/asm/xen/page-coherent.h +7 -18
@@ -26,25 +26,14 @@ static inline void xen_dma_map_page(struct device *hwdev, struct page *page, __generic_dma_ops(hwdev)->map_page(hwdev, page, offset, size, dir, attrs); } -static inline void xen_dma_unmap_page (struct device *hwdev,dma_addr_t handle, +void xen_dma_unmap_page (struct device *hwdev,dma_addr_t handle,size_t size,enum dma_data_direction dir, -struct dma_attrs *attrs) -{ -if (__generic_dma_ops (hwdev)->unmap_page) -__generic_dma_ops (hwdev)->unmap_page (hwdev, handle, size, dir, attrs); -} +struct dma_attrs *attrs); -static inline void xen_dma_sync_single_for_cpu (struct device *hwdev, -dma_addr_t handle,size_t size,enum dma_data_direction dir) -{ -if (__generic_dma_ops (hwdev)->sync_single_for_cpu) -__generic_dma_ops (hwdev)->sync_single_for_cpu (hwdev, handle, size, dir); -} +void xen_dma_sync_single_for_cpu (struct device *hwdev, +dma_addr_t handle,size_t size,enum dma_data_direction dir); + +void xen_dma_sync_single_for_device (struct device *hwdev, +dma_addr_t handle,size_t size,enum dma_data_direction dir); -static inline void xen_dma_sync_single_for_device (struct device *hwdev, -dma_addr_t handle,size_t size,enum dma_data_direction dir) -{ -if (__generic_dma_ops (hwdev)->sync_single_for_device) -__generic_dma_ops (hwdev)->sync_single_for_device (hwdev, handle, size, dir); -}#endif /* _ASM_ARM_XEN_PAGE_COHERENT_H */
arch/arm/xen/Makefile +1 -1
@@ -1 +1 @@ -obj-y := enlighten.o hypercall.o grant-table.o p2m.o mm.o +obj-y := enlighten.o hypercall.o grant-table.o p2m.o mm.o mm32.o
arch/arm/xen/mm32.c new file +202 -0
202 lines not on this page · raw message
Signature · 1 line
1.7.10.4
PATCH v2 3/3 xen/arm: remove mach_to_phys rbtree
In reply to Stefano StabelliniRemove the rbtree used to keep track of machine to physical mappings:
the frontend can grant the same page multiple times, leading to errors inserting or removing entries from the mach_to_phys tree.
Linux only needed to know the physical address corresponding to a given machine address in swiotlb-xen. Now that swiotlb-xen can call the xen_dma_* functions passing the machine address directly, we can remove it.
---
Changes in v2:
- do not check for mfn in xen_add_phys_to_mach_entry.
--- arch/arm/include/asm/xen/page.h | 9 ------ arch/arm/xen/p2m.c | 66 +-------------------------------------- 2 files changed, 1 insertion(+), 74 deletions(-)
arch/arm/include/asm/xen/page.h +0 -9
@@ -33,7 +33,6 @@ typedef struct xpaddr {#define INVALID_P2M_ENTRY (~0UL)unsigned long __pfn_to_mfn (unsigned long pfn); -unsigned long __mfn_to_pfn (unsigned long mfn);extern struct rb_root phys_to_mach;static inline unsigned long pfn_to_mfn (unsigned long pfn)
@@ -51,14 +50,6 @@ static inline unsigned long pfn_to_mfn(unsigned long pfn)static inline unsigned long mfn_to_pfn (unsigned long mfn) { -unsigned long pfn; - -if (phys_to_mach.rb_node !=NULL ) { - pfn =__mfn_to_pfn (mfn); -if (pfn !=INVALID_P2M_ENTRY ) -return pfn; - } -return mfn; }
arch/arm/xen/p2m.c +1 -65
102 lines not on this page · raw message
Signature · 1 line
1.7.10.4
v2 cover xen/arm: fix "xen_add_mach_to_phys_entry: cannot add" problem
In reply to Stefano StabelliniQuoted 8 lines · Stefano Stabellini
Hi all,
Xen support in Linux for ARM and ARM64 suffers from lack of support for multiple mfn to pfn mappings: whenever a frontend grants the same page multiple times to the backend, the mfn to pfn accounting inarch/arm/xen/p2m.cfails. The issue has become critical since v3.15, when xen-netfront/xen-netback switched from grant copies to grant mappings, therefore causing the issue to happen much more often.
Quoted 4 lines
Fixing the mfn to pfn accounting in
p2m.cis difficult and expensive, therefore we are looking for alternative solutions. One idea is avoiding mfn to pfn conversions altogether. The only code path that needs them is swiotlb-xen:unmap_page (andsingle_for_cpuandsingle_for_device).
I seem to have lost track of that patch? Or is it in now in the kernel?
Could you include the git commit id or URL for it in the cover letter please?
Thanks
Quoted 34 lines
To avoid mfn to pfn conversions we rely on a second p2m mapping done by Xen (a separate patch series will be sent for Xen). In Linux we use it to perform the cache maintenance operations without mfns conversions.
Changes in v2:
- introduceXENFEAT_grant_map_11;
- remeber the ptep corresponding to scratch pages so that we don't need to calculate it again every time;
- do not acutally unmap the page onxen_mm32_unmap;
- properly account preempt_enable/disable;
- do not check for mfn inxen_add_phys_to_mach_entry.Stefano Stabellini (3):
xen/arm: introduceXENFEAT_grant_map_11xen/arm: reimplementxen_dma_unmap_page& friends xen/arm: removemach_to_physrbtreearch/arm/include/asm/xen/page-coherent.h | 25 ++-- arch/arm/include/asm/xen/page.h | 9 -- arch/arm/xen/Makefile | 2 +- arch/arm/xen/enlighten.c | 6 + arch/arm/xen/mm32.c | 202 ++++++++++++++++++++++++++++++ arch/arm/xen/p2m.c | 66 +--------- include/xen/interface/features.h | 3 + 7 files changed, 220 insertions(+), 93 deletions(-) create mode 100644 arch/arm/xen/mm32.c_______________________________________________
Xen-devel mailing list
Xen-devel at lists.xen.org
http://lists.xen.org/xen-devel
v2 cover xen/arm: fix "xen_add_mach_to_phys_entry: cannot add" problem
Quoted 9 lines · Konrad Rzeszutek Wilk
Quoted 8 lines · Stefano Stabellini
Hi all,
Xen support in Linux for ARM and ARM64 suffers from lack of support for multiple mfn to pfn mappings: whenever a frontend grants the same page multiple times to the backend, the mfn to pfn accounting inarch/arm/xen/p2m.cfails. The issue has become critical since v3.15, when xen-netfront/xen-netback switched from grant copies to grant mappings, therefore causing the issue to happen much more often.
Quoted 8 lines
Quoted 4 lines
Fixing the mfn to pfn accounting in
p2m.cis difficult and expensive, therefore we are looking for alternative solutions. One idea is avoiding mfn to pfn conversions altogether. The only code path that needs them is swiotlb-xen:unmap_page (andsingle_for_cpuandsingle_for_device).I seem to have lost track of that patch? Or is it in now in the kernel?
Could you include the git commit id or URL for it in the cover letter please?
I take that you are asking the id of the commit that introduced the function calls that need mfn to pfn conversions in swiotlb-xen, right? The commit that introduced them is:
commit 6cf054636261ca5c88f3c2984058d51f927b8a2e
Author: Stefano Stabellini <[email protected]> Date: Fri Oct 25 10:33:25 2013 +0000
swiotlb-xen: use xen_dma_map/unmap_page, xen_dma_sync_single_for_cpu/device
xen_dma_unmap_page on x86 doesn't do anything but on ARM performs important cache invalidate operations for non-dma-coherent devices.
Quoted 36 lines
Thanks
Quoted 34 lines
To avoid mfn to pfn conversions we rely on a second p2m mapping done by Xen (a separate patch series will be sent for Xen). In Linux we use it to perform the cache maintenance operations without mfns conversions.
Changes in v2:
- introduceXENFEAT_grant_map_11;
- remeber the ptep corresponding to scratch pages so that we don't need to calculate it again every time;
- do not acutally unmap the page onxen_mm32_unmap;
- properly account preempt_enable/disable;
- do not check for mfn inxen_add_phys_to_mach_entry.Stefano Stabellini (3):
xen/arm: introduceXENFEAT_grant_map_11xen/arm: reimplementxen_dma_unmap_page& friends xen/arm: removemach_to_physrbtreearch/arm/include/asm/xen/page-coherent.h | 25 ++-- arch/arm/include/asm/xen/page.h | 9 -- arch/arm/xen/Makefile | 2 +- arch/arm/xen/enlighten.c | 6 + arch/arm/xen/mm32.c | 202 ++++++++++++++++++++++++++++++ arch/arm/xen/p2m.c | 66 +--------- include/xen/interface/features.h | 3 + 7 files changed, 220 insertions(+), 93 deletions(-) create mode 100644 arch/arm/xen/mm32.c_______________________________________________
Xen-devel mailing list
Xen-devel at lists.xen.org
http://lists.xen.org/xen-devel
v2 cover xen/arm: fix "xen_add_mach_to_phys_entry: cannot add" problem
Quoted 15 lines · Stefano Stabellini
Quoted 14 lines · Konrad Rzeszutek Wilk
Quoted 12 lines · Stefano Stabellini
Hi all,
Xen support in Linux for ARM and ARM64 suffers from lack of support for multiple mfn to pfn mappings: whenever a frontend grants the same page multiple times to the backend, the mfn to pfn accounting inarch/arm/xen/p2m.cfails. The issue has become critical since v3.15, when xen-netfront/xen-netback switched from grant copies to grant mappings, therefore causing the issue to happen much more often.Fixing the mfn to pfn accounting in
p2m.cis difficult and expensive, therefore we are looking for alternative solutions. One idea is avoiding mfn to pfn conversions altogether. The only code path that needs them is swiotlb-xen:unmap_page (andsingle_for_cpuandsingle_for_device).
Quoted 6 lines
Quoted 3 lines
I seem to have lost track of that patch? Or is it in now in the kernel?
Could you include the git commit id or URL for it in the cover letter please?
I take that you are asking the id of the commit that introduced the function calls that need mfn to pfn conversions in swiotlb-xen, right?
Earlier you said: "avoiding mfn to pfn conversions. The only code path that needs them is .." So I suspected (it seems wrongly) that there was some SWIOTLB patch that would do this and I had missed it.
But it seems that this soo fresh you hadn't yet posted the SWIOTLB changes to use this new API - so I hadn't missed them :-)
Quoted 49 lines
The commit that introduced them is:
commit
6cf054636261ca5c88f3c2984058d51f927b8a2eAuthor: Stefano Stabellini <[email protected]> Date: Fri Oct 25 10:33:25 2013 +0000swiotlb-xen: use xen_dma_map/unmap_page, xen_dma_sync_single_for_cpu/device
xen_dma_unmap_pageon x86 doesn't do anything but on ARM performs important cache invalidate operations for non-dma-coherent devices.Quoted 36 lines
Thanks
Quoted 34 lines
To avoid mfn to pfn conversions we rely on a second p2m mapping done by Xen (a separate patch series will be sent for Xen). In Linux we use it to perform the cache maintenance operations without mfns conversions.
Changes in v2:
- introduceXENFEAT_grant_map_11;
- remeber the ptep corresponding to scratch pages so that we don't need to calculate it again every time;
- do not acutally unmap the page onxen_mm32_unmap;
- properly account preempt_enable/disable;
- do not check for mfn inxen_add_phys_to_mach_entry.Stefano Stabellini (3):
xen/arm: introduceXENFEAT_grant_map_11xen/arm: reimplementxen_dma_unmap_page& friends xen/arm: removemach_to_physrbtreearch/arm/include/asm/xen/page-coherent.h | 25 ++-- arch/arm/include/asm/xen/page.h | 9 -- arch/arm/xen/Makefile | 2 +- arch/arm/xen/enlighten.c | 6 + arch/arm/xen/mm32.c | 202 ++++++++++++++++++++++++++++++ arch/arm/xen/p2m.c | 66 +--------- include/xen/interface/features.h | 3 + 7 files changed, 220 insertions(+), 93 deletions(-) create mode 100644 arch/arm/xen/mm32.c_______________________________________________
Xen-devel mailing list
Xen-devel at lists.xen.org
http://lists.xen.org/xen-devel
v2 cover xen/arm: fix "xen_add_mach_to_phys_entry: cannot add" problem
Quoted 23 lines · Konrad Rzeszutek Wilk
Quoted 15 lines · Stefano Stabellini
Quoted 14 lines · Konrad Rzeszutek Wilk
Quoted 12 lines · Stefano Stabellini
Hi all,
Xen support in Linux for ARM and ARM64 suffers from lack of support for multiple mfn to pfn mappings: whenever a frontend grants the same page multiple times to the backend, the mfn to pfn accounting inarch/arm/xen/p2m.cfails. The issue has become critical since v3.15, when xen-netfront/xen-netback switched from grant copies to grant mappings, therefore causing the issue to happen much more often.Fixing the mfn to pfn accounting in
p2m.cis difficult and expensive, therefore we are looking for alternative solutions. One idea is avoiding mfn to pfn conversions altogether. The only code path that needs them is swiotlb-xen:unmap_page (andsingle_for_cpuandsingle_for_device).Quoted 6 lines
Quoted 3 lines
I seem to have lost track of that patch? Or is it in now in the kernel?
Could you include the git commit id or URL for it in the cover letter please?
I take that you are asking the id of the commit that introduced the function calls that need mfn to pfn conversions in swiotlb-xen, right?
Quoted 6 lines
Earlier you said: "avoiding mfn to pfn conversions. The only code path that needs them is .." So I suspected (it seems wrongly) that there was some SWIOTLB patch that would do this and I had missed it.
But it seems that this soo fresh you hadn't yet posted the SWIOTLB changes to use this new API - so I hadn't missed them :-)
I think you didn't notice them because they don't do anything at all on x86 :-)
If you give a look at swiotlb-xen, you'll find that xen_bus_to_phys is called in xen_unmap_single and xen_swiotlb_sync_single. The returned physical address is passed to xen_dma_unmap_page and xen_dma_sync_single_for_cpu.
xen_dma_unmap_page and xen_dma_sync_single_for_cpu have an empty implementation on x86 (see arch/x86/include/asm/xen/page-coherent.h) but on ARM they perform important cache maintenance operations.
These are the only place where we need mfn to pfn conversions.
Quoted 50 lines
Quoted 49 lines
The commit that introduced them is:
commit
6cf054636261ca5c88f3c2984058d51f927b8a2eAuthor: Stefano Stabellini <[email protected]> Date: Fri Oct 25 10:33:25 2013 +0000swiotlb-xen: use xen_dma_map/unmap_page, xen_dma_sync_single_for_cpu/device
xen_dma_unmap_pageon x86 doesn't do anything but on ARM performs important cache invalidate operations for non-dma-coherent devices.Quoted 36 lines
Thanks
Quoted 34 lines
To avoid mfn to pfn conversions we rely on a second p2m mapping done by Xen (a separate patch series will be sent for Xen). In Linux we use it to perform the cache maintenance operations without mfns conversions.
Changes in v2:
- introduceXENFEAT_grant_map_11;
- remeber the ptep corresponding to scratch pages so that we don't need to calculate it again every time;
- do not acutally unmap the page onxen_mm32_unmap;
- properly account preempt_enable/disable;
- do not check for mfn inxen_add_phys_to_mach_entry.Stefano Stabellini (3):
xen/arm: introduceXENFEAT_grant_map_11xen/arm: reimplementxen_dma_unmap_page& friends xen/arm: removemach_to_physrbtreearch/arm/include/asm/xen/page-coherent.h | 25 ++-- arch/arm/include/asm/xen/page.h | 9 -- arch/arm/xen/Makefile | 2 +- arch/arm/xen/enlighten.c | 6 + arch/arm/xen/mm32.c | 202 ++++++++++++++++++++++++++++++ arch/arm/xen/p2m.c | 66 +--------- include/xen/interface/features.h | 3 + 7 files changed, 220 insertions(+), 93 deletions(-) create mode 100644 arch/arm/xen/mm32.c_______________________________________________
Xen-devel mailing list
Xen-devel at lists.xen.org
http://lists.xen.org/xen-devel
v2 cover xen/arm: fix "xen_add_mach_to_phys_entry: cannot add" problem
In reply to Stefano Stabellini2014-07-08 17:40 GMT+02:00 Stefano Stabellini
<[email protected]>:
Quoted 32 lines
Hi all,
Xen support in Linux for ARM and ARM64 suffers from lack of support for multiple mfn to pfn mappings: whenever a frontend grants the same page multiple times to the backend, the mfn to pfn accounting inarch/arm/xen/p2m.cfails. The issue has become critical since v3.15, when xen-netfront/xen-netback switched from grant copies to grant mappings, therefore causing the issue to happen much more often.Fixing the mfn to pfn accounting in
p2m.cis difficult and expensive, therefore we are looking for alternative solutions. One idea is avoiding mfn to pfn conversions altogether. The only code path that needs them is swiotlb-xen:unmap_page (andsingle_for_cpuandsingle_for_device).To avoid mfn to pfn conversions we rely on a second p2m mapping done by Xen (a separate patch series will be sent for Xen). In Linux we use it to perform the cache maintenance operations without mfns conversions.
Changes in v2:
- introduceXENFEAT_grant_map_11;
- remeber the ptep corresponding to scratch pages so that we don't need to calculate it again every time;
- do not acutally unmap the page onxen_mm32_unmap;
- properly account preempt_enable/disable;
- do not check for mfn inxen_add_phys_to_mach_entry.Stefano Stabellini (3):
xen/arm: introduceXENFEAT_grant_map_11xen/arm: reimplementxen_dma_unmap_page& friends xen/arm: removemach_to_physrbtree
Quoted 9 lines
arch/arm/include/asm/xen/page-coherent.h | 25 ++-- arch/arm/include/asm/xen/page.h | 9 -- arch/arm/xen/Makefile | 2 +- arch/arm/xen/enlighten.c | 6 + arch/arm/xen/mm32.c | 202 ++++++++++++++++++++++++++++++ arch/arm/xen/p2m.c | 66 +--------- include/xen/interface/features.h | 3 + 7 files changed, 220 insertions(+), 93 deletions(-) create mode 100644 arch/arm/xen/mm32.c
Stress-tested with network I/O, disk I/O for hours without any problem. Thanks.