xen/arm: fix "xen_add_mach_to_phys_entry: cannot add" problem

Hi all,
Xen support in Linux for ARM and ARM64 suffers from lack of support for multiple mfn to pfn mappings: whenever a frontend grants the same page multiple times to the backend, the mfn to pfn accounting in arch/arm/xen/p2m.c fails. The issue has become critical since v3.15, when xen-netfront/xen-netback switched from grant copies to grant mappings, therefore causing the issue to happen much more often.

Fixing the mfn to pfn accounting in p2m.c is difficult and expensive, therefore we are looking for alternative solutions. One idea is avoiding mfn to pfn conversions altogether. The only code path that needs them is swiotlb-xen:unmap_page (and single_for_cpu and single_for_device).

To avoid mfn to pfn conversions we rely on a second p2m mapping done by Xen, see the attached patch. In Linux we use it to perform the cache maintenance operations without mfns conversions.

Stefano Stabellini (2):
xen/arm: reimplement xen_dma_unmap_page & friends xen/arm: remove mach_to_phys rbtree

 arch/arm/include/asm/xen/page-coherent.h |   25 ++--
 arch/arm/include/asm/xen/page.h          |    9 --
 arch/arm/xen/Makefile                    |    2 +-
 arch/arm/xen/mm32.c                      |  202 ++++++++++++++++++++++++++++++
 arch/arm/xen/p2m.c                       |   64 +---------
 5 files changed, 211 insertions(+), 91 deletions(-)
 create mode 100644 arch/arm/xen/mm32.c

-------------- next part --------------

index eac8a70..95519c5 100644
@@ -34,7 +34,7 @@ static inline int replace_grant_supported(void)
      (i < max_nr_grant_frames)) ? 0 : (d->arch.grant_table_gpfn[i]))
 
 #define gnttab_need_iommu_mapping(d)                    \
-    (is_domain_direct_mapped(d) && need_iommu(d))
+    (is_domain_direct_mapped(d))
 
 #endif /* __ASM_GRANT_TABLE_H__ */
 /*

xen_dma_unmap_page, xen_dma_sync_single_for_cpu and xen_dma_sync_single_for_device are currently implemented by calling into the corresponding generic ARM implementation of these functions. In order to do this, firstly the dma_addr_t handle, that on Xen is a machine address, needs to be translated into a physical address. The operation is expensive and inaccurate, given that a single machine address can correspond to multiple physical addresses in one domain, because the same page can be granted multiple times by the frontend.

To avoid this problem, we introduce a Xen specific implementation of xen_dma_unmap_page, xen_dma_sync_single_for_cpu and xen_dma_sync_single_for_device, that can operate on machine addresses directly.

The new implementation relies on the fact that the hypervisor creates a second p2m mapping of any grant pages at physical address == machine address of the page for dom0. Therefore we can access memory at physical address == dma_addr_r handle and perform the cache flushing there. Some cache maintenance operations require a virtual address. Instead of using ioremap_cache, that is not safe in interrupt context, we allocate a per-cpu PAGE_KERNEL scratch page and we manually update the pte for it.

arm64 doesn't need cache maintenance operations on unmap for now.

---
 arch/arm/include/asm/xen/page-coherent.h |   25 ++--
 arch/arm/xen/Makefile                    |    2 +-
 arch/arm/xen/mm32.c                      |  202 ++++++++++++++++++++++++++++++
 3 files changed, 210 insertions(+), 19 deletions(-)
 create mode 100644 arch/arm/xen/mm32.c

index 1109017..e8275ea 100644
@@ -26,25 +26,14 @@ static inline void xen_dma_map_page(struct device *hwdev, struct page *page,
 	__generic_dma_ops(hwdev)->map_page(hwdev, page, offset, size, dir, attrs);
 }
 
-static inline void xen_dma_unmap_page(struct device *hwdev, dma_addr_t handle,
+void xen_dma_unmap_page(struct device *hwdev, dma_addr_t handle,
 		size_t size, enum dma_data_direction dir,
-		struct dma_attrs *attrs)
-{
-	if (__generic_dma_ops(hwdev)->unmap_page)
-		__generic_dma_ops(hwdev)->unmap_page(hwdev, handle, size, dir, attrs);
-}
+		struct dma_attrs *attrs);
 
-static inline void xen_dma_sync_single_for_cpu(struct device *hwdev,
-		dma_addr_t handle, size_t size, enum dma_data_direction dir)
-{
-	if (__generic_dma_ops(hwdev)->sync_single_for_cpu)
-		__generic_dma_ops(hwdev)->sync_single_for_cpu(hwdev, handle, size, dir);
-}
+void xen_dma_sync_single_for_cpu(struct device *hwdev,
+		dma_addr_t handle, size_t size, enum dma_data_direction dir);
+
+void xen_dma_sync_single_for_device(struct device *hwdev,
+		dma_addr_t handle, size_t size, enum dma_data_direction dir);
 
-static inline void xen_dma_sync_single_for_device(struct device *hwdev,
-		dma_addr_t handle, size_t size, enum dma_data_direction dir)
-{
-	if (__generic_dma_ops(hwdev)->sync_single_for_device)
-		__generic_dma_ops(hwdev)->sync_single_for_device(hwdev, handle, size, dir);
-}
 #endif /* _ASM_ARM_XEN_PAGE_COHERENT_H */

index 1296952..1f85bfe 100644
@@ -1 +1 @@
-obj-y		:= enlighten.o hypercall.o grant-table.o p2m.o mm.o
+obj-y		:= enlighten.o hypercall.o grant-table.o p2m.o mm.o mm32.o

1.7.10.4

Remove the rbtree used to keep track of machine to physical mappings:
the frontend can grant the same page multiple times, leading to errors inserting or removing entries from the mach_to_phys tree.

Linux only needed to know the physical address corresponding to a given machine address in swiotlb-xen. Now that swiotlb-xen can call the xen_dma_* functions passing the machine address directly, we can remove it.

---
 arch/arm/include/asm/xen/page.h |    9 ------
 arch/arm/xen/p2m.c              |   64 +--------------------------------------
 2 files changed, 1 insertion(+), 72 deletions(-)

index ded062f..135c24a 100644
@@ -33,7 +33,6 @@ typedef struct xpaddr {
 #define INVALID_P2M_ENTRY      (~0UL)
 
 unsigned long __pfn_to_mfn(unsigned long pfn);
-unsigned long __mfn_to_pfn(unsigned long mfn);
 extern struct rb_root phys_to_mach;
 
 static inline unsigned long pfn_to_mfn(unsigned long pfn)

@@ -51,14 +50,6 @@ static inline unsigned long pfn_to_mfn(unsigned long pfn)
 
 static inline unsigned long mfn_to_pfn(unsigned long mfn)
 {
-	unsigned long pfn;
-
-	if (phys_to_mach.rb_node != NULL) {
-		pfn = __mfn_to_pfn(mfn);
-		if (pfn != INVALID_P2M_ENTRY)
-			return pfn;
-	}
-
 	return mfn;
 }
 

1.7.10.4

Hi Stefano,

(Drop Linux ARM as my comment is only Xen specific and avoid spamming the Linux ML)

Julien Grall

Hi Stefano,

I tried this series on midway with a FreeBSD guest. I got those errors in DOM0 log:

xen_add_phys_to_mach_entry: cannot add pfn=0x002d99330003cdc7 -> mfn=0x00000001002d9933: pfn=0x002d99330003e3ea -> mfn=0x00000001002d9933 already exists
xen_add_phys_to_mach_entry: cannot add pfn=0x002d99330003d47a -> mfn=0x00000001002d9933: pfn=0x002d99330003e3ea -> mfn=0x00000001002d9933 already exists

It looks like you forgot to modify xen_add_phys_to_mach_entry.

Regards,

Julien Grall

This will only work with PIPT caches (or certain types of VIPT that also have alias detection). Do all the CPUs that Xen runs on have these type of cache?

David

Given that Xen only runs on v7 or newer I think we should be safe.

v7 (at least, didn't check/don't care about v6) mandates PIPT for data caches, yes.

Ian.