fix swiotlb-xen for RPi4
Subsystems (4)
Stefano Stabellini
maintainerv3 cover fix swiotlb-xen for RPi4
Hi all,
This series is a collection of fixes to get Linux running on the RPi4 as dom0. Conceptually there are only two significant changes:
- make sure not to call virt_to_page on vmalloc virt addresses (patch #1)
- use phys_to_dma and dma_to_phys to translate phys to/from dma addresses (all other patches)
I addressed all comments by Christoph to v2 of the series except from the one about merging the precursor "add struct device *" patches. I can always merge them together at any time as needed.
Boris gave his Reviewed-by to the whole series v2. I added his Reviewed-by to all patches, including the ones with small cosmetic fixes, except for patch #8 #9 #10 because they are either new or changed significantly in this version of the series.
I retained Roman and Corey's Tested-by.
Cheers,
Stefano
git://git.kernel.org/pub/scm/linux/kernel/git/sstabellini/xen.git fix-rpi4-v3
Boris Ostrovsky (1):
swiotlb-xen: use vmalloc_to_page on vmalloc virt addresses
Stefano Stabellini (10):
swiotlb-xen: remove start_dma_addr
swiotlb-xen: add struct device * parameter to xen_phys_to_bus
swiotlb-xen: add struct device * parameter to xen_bus_to_phys
swiotlb-xen: add struct device * parameter to xen_dma_sync_for_cpu
swiotlb-xen: add struct device * parameter to xen_dma_sync_for_device
swiotlb-xen: add struct device * parameter to is_xen_swiotlb_buffer
swiotlb-xen: remove XEN_PFN_PHYS
swiotlb-xen: introduce phys_to_dma/dma_to_phys translations xen/arm: introduce phys/dma translations in xen_dma_sync_for_* xen/arm: call dma_to_phys on the dma_addr_t parameter of dma_cache_maint
arch/arm/xen/mm.c | 34 +++++++++++++++---------------- drivers/xen/swiotlb-xen.c | 119 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++------------------------------------------- include/xen/page.h | 1 - include/xen/swiotlb-xen.h | 8 ++++---- 4 files changed, 93 insertions(+), 69 deletions(-)
Stefano Stabellini
maintainerPATCH v3 1/11 swiotlb-xen: use vmalloc_to_page on vmalloc virt addresses
From: Boris Ostrovsky <[email protected]>
xen_alloc_coherent_pages might return pages for which virt_to_phys and virt_to_page don't work, e.g. ioremap'ed pages.
So in xen_swiotlb_free_coherent we can't assume that virt_to_page works. Instead add a is_vmalloc_addr check and use vmalloc_to_page on vmalloc virt addresses.
This patch fixes the following crash at boot on RPi4 (the underlying issue is not RPi4 specific):
https://marc.info/?l=xen-devel&m=158862573216800
---
Changes in v3:
- style improvements
Changes in v2:
- update commit message
--- drivers/xen/swiotlb-xen.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)
drivers/xen/swiotlb-xen.c +7 -1
@@ -335,6 +335,7 @@ xen_swiotlb_free_coherent(struct device *hwdev, size_t size, void *vaddr,int order =get_order (size);phys_addr_t phys;u64 dma_mask =DMA_BIT_MASK (32 ); +struct page *page;if (hwdev && hwdev->coherent_dma_mask) dma_mask = hwdev->coherent_dma_mask;
@@ -346,9 +347,14 @@ xen_swiotlb_free_coherent(struct device *hwdev, size_t size, void *vaddr,/* Convert the size to actually allocated. */ size =1UL << (order +XEN_PAGE_SHIFT ); +if (is_vmalloc_addr (vaddr)) + page =vmalloc_to_page (vaddr); +else + page =virt_to_page (vaddr); +if (!WARN_ON ((dev_addr + size -1 > dma_mask) ||range_straddles_page_boundary (phys, size)) && -TestClearPageXenRemapped (virt_to_page (vaddr))) +TestClearPageXenRemapped (page))xen_destroy_contiguous_region (phys, order);xen_free_coherent_pages (hwdev, size, vaddr, (dma_addr_t )phys, attrs);
Signature · 1 line
2.17.1
Stefano Stabellini
maintainerPATCH v3 2/11 swiotlb-xen: remove start_dma_addr
In reply to Stefano StabelliniFrom: Stefano Stabellini <[email protected]>
It is not strictly needed. Call virt_to_phys on xen_io_tlb_start instead. It will be useful not to have a start_dma_addr around with the next patches.
Note that virt_to_phys is not the same as xen_virt_to_bus but actually it is used to compared again __pa(xen_io_tlb_start) as passed to swiotlb_init_with_tbl, so virt_to_phys is actually what we want.
---
Changes in v2:
- update commit message
--- drivers/xen/swiotlb-xen.c | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-)
drivers/xen/swiotlb-xen.c +2 -5
@@ -52,8 +52,6 @@ static unsigned long xen_io_tlb_nslabs;* Quick lookup value of the bus address of the IOTLB. */ -static u64 start_dma_addr; -/* * Both of these functions should avoid XEN_PFN_PHYS because phys_addr_t * can be 32bit when dma_addr_t is 64bit leading to a loss in
@@ -241,7 +239,6 @@ int __ref xen_swiotlb_init(int verbose, bool early) m_ret =XEN_SWIOTLB_EFIXUP ; goto error; } - start_dma_addr =xen_virt_to_bus (xen_io_tlb_start);if (early) {if (swiotlb_init_with_tbl (xen_io_tlb_start, xen_io_tlb_nslabs, verbose))
@@ -392,8 +389,8 @@ static dma_addr_t xen_swiotlb_map_page(struct device *dev, struct page *page,*/ trace_swiotlb_bounced (dev, dev_addr, size, swiotlb_force); - map =swiotlb_tbl_map_single (dev, start_dma_addr, phys, - size, size, dir, attrs); + map =swiotlb_tbl_map_single (dev,virt_to_phys (xen_io_tlb_start), + phys, size, size, dir, attrs);if (map == (phys_addr_t )DMA_MAPPING_ERROR )return DMA_MAPPING_ERROR ;
Signature · 1 line
2.17.1
Stefano Stabellini
maintainerPATCH v3 3/11 swiotlb-xen: add struct device * parameter to xen_phys_to_bus
In reply to Stefano StabelliniFrom: Stefano Stabellini <[email protected]>
No functional changes. The parameter is unused in this patch but will be used by next patches.
---
Changes in v3:
- add whitespace in title
- improve commit message
--- drivers/xen/swiotlb-xen.c | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-)
drivers/xen/swiotlb-xen.c +7 -7
@@ -57,7 +57,7 @@ static unsigned long xen_io_tlb_nslabs;* can be 32bit when dma_addr_t is 64bit leading to a loss in * information if the shift is done before casting to 64bit. */ -static inline dma_addr_t xen_phys_to_bus (phys_addr_t paddr) +static inline dma_addr_t xen_phys_to_bus (struct device *dev,phys_addr_t paddr) {unsigned long bfn =pfn_to_bfn (XEN_PFN_DOWN (paddr));dma_addr_t dma = (dma_addr_t )bfn <<XEN_PAGE_SHIFT ;
@@ -78,9 +78,9 @@ static inline phys_addr_t xen_bus_to_phys(dma_addr_t baddr)return paddr; } -static inline dma_addr_t xen_virt_to_bus (void *address) +static inline dma_addr_t xen_virt_to_bus (struct device *dev,void *address) { -return xen_phys_to_bus (virt_to_phys (address)); +return xen_phys_to_bus (dev,virt_to_phys (address)); }static inline int range_straddles_page_boundary (phys_addr_t p,size_t size)
@@ -309,7 +309,7 @@ xen_swiotlb_alloc_coherent(struct device *hwdev, size_t size,* Do not use virt_to_phys(ret) because on ARM it doesn't correspond * to *dma_handle. */ phys = *dma_handle; - dev_addr =xen_phys_to_bus (phys); + dev_addr =xen_phys_to_bus (hwdev, phys);if (((dev_addr + size -1 <= dma_mask)) && !range_straddles_page_boundary (phys, size)) *dma_handle = dev_addr;
@@ -370,7 +370,7 @@ static dma_addr_t xen_swiotlb_map_page(struct device *dev, struct page *page, unsigned long attrs) {phys_addr_t map, phys =page_to_phys (page) + offset; -dma_addr_t dev_addr =xen_phys_to_bus (phys); +dma_addr_t dev_addr =xen_phys_to_bus (dev, phys);BUG_ON (dir ==DMA_NONE );/*
@@ -395,7 +395,7 @@ static dma_addr_t xen_swiotlb_map_page(struct device *dev, struct page *page,return DMA_MAPPING_ERROR ; phys = map; - dev_addr =xen_phys_to_bus (map); + dev_addr =xen_phys_to_bus (dev, map);/* * Ensure that the address returned is DMA'ble
@@ -539,7 +539,7 @@ xen_swiotlb_sync_sg_for_device(struct device *dev, struct scatterlist *sgl,static int xen_swiotlb_dma_supported (struct device *hwdev,u64 mask) { -return xen_virt_to_bus (xen_io_tlb_end -1 ) <= mask; +return xen_virt_to_bus (hwdev, xen_io_tlb_end -1 ) <= mask; }const struct dma_map_ops xen_swiotlb_dma_ops = {
Signature · 1 line
2.17.1
Stefano Stabellini
maintainerPATCH v3 4/11 swiotlb-xen: add struct device * parameter to xen_bus_to_phys
In reply to Stefano StabelliniFrom: Stefano Stabellini <[email protected]>
No functional changes. The parameter is unused in this patch but will be used by next patches.
---
Changes in v3:
- add whitespace in title
- improve commit message
--- drivers/xen/swiotlb-xen.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
drivers/xen/swiotlb-xen.c +5 -5
@@ -67,7 +67,7 @@ static inline dma_addr_t xen_phys_to_bus(struct device *dev, phys_addr_t paddr)return dma; } -static inline phys_addr_t xen_bus_to_phys (dma_addr_t baddr) +static inline phys_addr_t xen_bus_to_phys (struct device *dev,dma_addr_t baddr) {unsigned long xen_pfn =bfn_to_pfn (XEN_PFN_DOWN (baddr));dma_addr_t dma = (dma_addr_t )xen_pfn <<XEN_PAGE_SHIFT ;
@@ -339,7 +339,7 @@ xen_swiotlb_free_coherent(struct device *hwdev, size_t size, void *vaddr,/* do not use virt_to_phys because on ARM it doesn't return you the * physical address */ - phys =xen_bus_to_phys (dev_addr); + phys =xen_bus_to_phys (hwdev, dev_addr);/* Convert the size to actually allocated. */ size =1UL << (order +XEN_PAGE_SHIFT );
@@ -423,7 +423,7 @@ static dma_addr_t xen_swiotlb_map_page(struct device *dev, struct page *page,static void xen_swiotlb_unmap_page (struct device *hwdev,dma_addr_t dev_addr,size_t size,enum dma_data_direction dir,unsigned long attrs) { -phys_addr_t paddr =xen_bus_to_phys (dev_addr); +phys_addr_t paddr =xen_bus_to_phys (hwdev, dev_addr);BUG_ON (dir ==DMA_NONE );
@@ -439,7 +439,7 @@ static void xen_swiotlb_sync_single_for_cpu(struct device *dev, dma_addr_t dma_addr, size_t size, enum dma_data_direction dir) { - phys_addr_t paddr = xen_bus_to_phys(dma_addr); + phys_addr_t paddr = xen_bus_to_phys(dev, dma_addr);if (!dev_is_dma_coherent (dev))xen_dma_sync_for_cpu (dma_addr, paddr, size, dir);
@@ -452,7 +452,7 @@ static void xen_swiotlb_sync_single_for_device(struct device *dev, dma_addr_t dma_addr, size_t size, enum dma_data_direction dir) { - phys_addr_t paddr = xen_bus_to_phys(dma_addr); + phys_addr_t paddr = xen_bus_to_phys(dev, dma_addr);if (is_xen_swiotlb_buffer (dma_addr))swiotlb_tbl_sync_single (dev, paddr, size, dir,SYNC_FOR_DEVICE );
Signature · 1 line
2.17.1
Stefano Stabellini
maintainerPATCH v3 5/11 swiotlb-xen: add struct device * parameter to xen_dma_sync_for_cpu
In reply to Stefano StabelliniFrom: Stefano Stabellini <[email protected]>
No functional changes. The parameter is unused in this patch but will be used by next patches.
---
Changes in v3:
- add whitespace in title
- improve commit message
--- arch/arm/xen/mm.c | 5 +++-- drivers/xen/swiotlb-xen.c | 4 ++-- include/xen/swiotlb-xen.h | 5 +++-- 3 files changed, 8 insertions(+), 6 deletions(-)
arch/arm/xen/mm.c +3 -2
@@ -71,8 +71,9 @@ static void dma_cache_maint(dma_addr_t handle, size_t size, u32 op)* pfn_valid returns true the pages is local and we can use the native * dma-direct functions, otherwise we call the Xen specific version. */ -void xen_dma_sync_for_cpu (dma_addr_t handle,phys_addr_t paddr,size_t size, -enum dma_data_direction dir) +void xen_dma_sync_for_cpu (struct device *dev,dma_addr_t handle, +phys_addr_t paddr,size_t size, +enum dma_data_direction dir) {if (pfn_valid (PFN_DOWN (handle)))arch_sync_dma_for_cpu (paddr, size, dir);
drivers/xen/swiotlb-xen.c +2 -2
@@ -428,7 +428,7 @@ static void xen_swiotlb_unmap_page(struct device *hwdev, dma_addr_t dev_addr,BUG_ON (dir ==DMA_NONE );if (!dev_is_dma_coherent (hwdev) && !(attrs &DMA_ATTR_SKIP_CPU_SYNC )) -xen_dma_sync_for_cpu (dev_addr, paddr, size, dir); +xen_dma_sync_for_cpu (hwdev, dev_addr, paddr, size, dir);/* NOTE: We use dev_addr here, not paddr! */ if (is_xen_swiotlb_buffer (dev_addr))
@@ -442,7 +442,7 @@ xen_swiotlb_sync_single_for_cpu(struct device *dev, dma_addr_t dma_addr,phys_addr_t paddr =xen_bus_to_phys (dev, dma_addr);if (!dev_is_dma_coherent (dev)) -xen_dma_sync_for_cpu (dma_addr, paddr, size, dir); +xen_dma_sync_for_cpu (dev, dma_addr, paddr, size, dir);if (is_xen_swiotlb_buffer (dma_addr))swiotlb_tbl_sync_single (dev, paddr, size, dir,SYNC_FOR_CPU );
include/xen/swiotlb-xen.h +3 -2
@@ -4,8 +4,9 @@#include <linux/swiotlb.h> -void xen_dma_sync_for_cpu (dma_addr_t handle,phys_addr_t paddr,size_t size, -enum dma_data_direction dir); +void xen_dma_sync_for_cpu (struct device *dev,dma_addr_t handle, +phys_addr_t paddr,size_t size, +enum dma_data_direction dir);void xen_dma_sync_for_device (dma_addr_t handle,phys_addr_t paddr,size_t size,enum dma_data_direction dir);
Signature · 1 line
2.17.1
Stefano Stabellini
maintainerPATCH v3 6/11 swiotlb-xen: add struct device * parameter to xen_dma_sync_for_device
In reply to Stefano StabelliniFrom: Stefano Stabellini <[email protected]>
No functional changes. The parameter is unused in this patch but will be used by next patches.
---
Changes in v3:
- add whitespace in title
- improve commit message
--- arch/arm/xen/mm.c | 5 +++-- drivers/xen/swiotlb-xen.c | 4 ++-- include/xen/swiotlb-xen.h | 5 +++-- 3 files changed, 8 insertions(+), 6 deletions(-)
arch/arm/xen/mm.c +3 -2
@@ -81,8 +81,9 @@ void xen_dma_sync_for_cpu(struct device *dev, dma_addr_t handle,dma_cache_maint (handle, size,GNTTAB_CACHE_INVAL ); } -void xen_dma_sync_for_device (dma_addr_t handle,phys_addr_t paddr,size_t size, -enum dma_data_direction dir) +void xen_dma_sync_for_device (struct device *dev,dma_addr_t handle, +phys_addr_t paddr,size_t size, +enum dma_data_direction dir) {if (pfn_valid (PFN_DOWN (handle)))arch_sync_dma_for_device (paddr, size, dir);
drivers/xen/swiotlb-xen.c +2 -2
@@ -408,7 +408,7 @@ static dma_addr_t xen_swiotlb_map_page(struct device *dev, struct page *page, done:if (!dev_is_dma_coherent (dev) && !(attrs &DMA_ATTR_SKIP_CPU_SYNC )) -xen_dma_sync_for_device (dev_addr, phys, size, dir); +xen_dma_sync_for_device (dev, dev_addr, phys, size, dir);return dev_addr; }
@@ -458,7 +458,7 @@ xen_swiotlb_sync_single_for_device(struct device *dev, dma_addr_t dma_addr,swiotlb_tbl_sync_single (dev, paddr, size, dir,SYNC_FOR_DEVICE );if (!dev_is_dma_coherent (dev)) -xen_dma_sync_for_device (dma_addr, paddr, size, dir); +xen_dma_sync_for_device (dev, dma_addr, paddr, size, dir); }/*
include/xen/swiotlb-xen.h +3 -2
@@ -7,8 +7,9 @@void xen_dma_sync_for_cpu (struct device *dev,dma_addr_t handle,phys_addr_t paddr,size_t size,enum dma_data_direction dir); -void xen_dma_sync_for_device (dma_addr_t handle,phys_addr_t paddr,size_t size, -enum dma_data_direction dir); +void xen_dma_sync_for_device (struct device *dev,dma_addr_t handle, +phys_addr_t paddr,size_t size, +enum dma_data_direction dir);extern int xen_swiotlb_init (int verbose,bool early);extern const struct dma_map_ops xen_swiotlb_dma_ops;
Signature · 1 line
2.17.1
Stefano Stabellini
maintainerPATCH v3 7/11 swiotlb-xen: add struct device * parameter to is_xen_swiotlb_buffer
In reply to Stefano StabelliniFrom: Stefano Stabellini <[email protected]>
No functional changes. The parameter is unused in this patch but will be used by next patches.
---
Changes in v3:
- add whitespace in title
- improve commit message
--- drivers/xen/swiotlb-xen.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
drivers/xen/swiotlb-xen.c +4 -4
@@ -97,7 +97,7 @@ static inline int range_straddles_page_boundary(phys_addr_t p, size_t size)return 0 ; } -static int is_xen_swiotlb_buffer (dma_addr_t dma_addr) +static int is_xen_swiotlb_buffer (struct device *dev,dma_addr_t dma_addr) {unsigned long bfn =XEN_PFN_DOWN (dma_addr);unsigned long xen_pfn =bfn_to_local_pfn (bfn);
@@ -431,7 +431,7 @@ static void xen_swiotlb_unmap_page(struct device *hwdev, dma_addr_t dev_addr,xen_dma_sync_for_cpu (hwdev, dev_addr, paddr, size, dir);/* NOTE: We use dev_addr here, not paddr! */ -if (is_xen_swiotlb_buffer (dev_addr)) +if (is_xen_swiotlb_buffer (hwdev, dev_addr))swiotlb_tbl_unmap_single (hwdev, paddr, size, size, dir, attrs); }
@@ -444,7 +444,7 @@ xen_swiotlb_sync_single_for_cpu(struct device *dev, dma_addr_t dma_addr,if (!dev_is_dma_coherent (dev))xen_dma_sync_for_cpu (dev, dma_addr, paddr, size, dir); -if (is_xen_swiotlb_buffer (dma_addr)) +if (is_xen_swiotlb_buffer (dev, dma_addr))swiotlb_tbl_sync_single (dev, paddr, size, dir,SYNC_FOR_CPU ); }
@@ -454,7 +454,7 @@ xen_swiotlb_sync_single_for_device(struct device *dev, dma_addr_t dma_addr, {phys_addr_t paddr =xen_bus_to_phys (dev, dma_addr); -if (is_xen_swiotlb_buffer (dma_addr)) +if (is_xen_swiotlb_buffer (dev, dma_addr))swiotlb_tbl_sync_single (dev, paddr, size, dir,SYNC_FOR_DEVICE );if (!dev_is_dma_coherent (dev))
Signature · 1 line
2.17.1
Stefano Stabellini
maintainerPATCH v3 8/11 swiotlb-xen: remove XEN_PFN_PHYS
In reply to Stefano StabelliniFrom: Stefano Stabellini <[email protected]>
XEN_PFN_PHYS is only used in one place in swiotlb-xen making things more complex than need to be.
Remove the definition of XEN_PFN_PHYS and open code the cast in the one place where it is needed.
--- drivers/xen/swiotlb-xen.c | 7 +------ include/xen/page.h | 1 - 2 files changed, 1 insertion(+), 7 deletions(-)
drivers/xen/swiotlb-xen.c +1 -6
@@ -52,11 +52,6 @@ static unsigned long xen_io_tlb_nslabs;* Quick lookup value of the bus address of the IOTLB. */ -/* -* Both of these functions should avoid XEN_PFN_PHYS because phys_addr_t -* can be 32bit when dma_addr_t is 64bit leading to a loss in -* information if the shift is done before casting to 64bit. -*/ static inline dma_addr_t xen_phys_to_bus (struct device *dev,phys_addr_t paddr) {unsigned long bfn =pfn_to_bfn (XEN_PFN_DOWN (paddr));
@@ -101,7 +96,7 @@ static int is_xen_swiotlb_buffer(struct device *dev, dma_addr_t dma_addr) {unsigned long bfn =XEN_PFN_DOWN (dma_addr);unsigned long xen_pfn =bfn_to_local_pfn (bfn); -phys_addr_t paddr =XEN_PFN_PHYS (xen_pfn); +phys_addr_t paddr = (phys_addr_t )xen_pfn <<XEN_PAGE_SHIFT ;/* If the address is outside our domain, it CAN * have the same virtual address as another address
include/xen/page.h +0 -1
@@ -24,7 +24,6 @@#define XEN_PFN_DOWN (x) ((x) >> XEN_PAGE_SHIFT)#define XEN_PFN_UP (x) (((x) + XEN_PAGE_SIZE-1) >> XEN_PAGE_SHIFT) -#define XEN_PFN_PHYS (x) ((phys_addr_t)(x) << XEN_PAGE_SHIFT)#include <asm/xen/page.h>
Signature · 1 line
2.17.1
Re: [PATCH v3 08/11] swiotlb-xen: remove XEN_PFN_PHYS
Quoted 9 lines · Stefano Stabellini
From: Stefano Stabellini <[email protected]>
XEN_PFN_PHYSis only used in one place in swiotlb-xen making things more complex than need to be.Remove the definition of
XEN_PFN_PHYSand open code the cast in the one place where it is needed.Signed-off-by Stefano Stabellini <[email protected]>
Juergen
Stefano Stabellini
maintainerPATCH v3 9/11 swiotlb-xen: introduce phys_to_dma/dma_to_phys translations
In reply to Stefano StabelliniFrom: Stefano Stabellini <[email protected]>
With some devices physical addresses are different than dma addresses. To be able to deal with these cases, we need to call phys_to_dma on physical addresses (including machine addresses in Xen terminology) before returning them from xen_swiotlb_alloc_coherent and xen_swiotlb_map_page.
We also need to convert dma addresses back to physical addresses using dma_to_phys in xen_swiotlb_free_coherent and xen_swiotlb_unmap_page if we want to do any operations on them.
Call dma_to_phys in is_xen_swiotlb_buffer.
Introduce xen_phys_to_dma and call phys_to_dma in its implementation. Introduce xen_dma_to_phys and call dma_to_phys in its implementation. Call xen_phys_to_dma/xen_dma_to_phys instead of xen_phys_to_bus/xen_bus_to_phys through swiotlb-xen.c.
Everything is taken care of by these changes except for xen_swiotlb_alloc_coherent and xen_swiotlb_free_coherent, which need a few explicit phys_to_dma/dma_to_phys calls.
---
Changes in v2:
- improve commit message
Changes in v3:
- add xen_phys_to_dma and xen_dma_to_phys in this patch
--- drivers/xen/swiotlb-xen.c | 53 +++++++++++++++++++++++---------------- 1 file changed, 32 insertions(+), 21 deletions(-)
drivers/xen/swiotlb-xen.c +32 -21
@@ -52,30 +52,39 @@ static unsigned long xen_io_tlb_nslabs;* Quick lookup value of the bus address of the IOTLB. */ -static inline dma_addr_t xen_phys_to_bus (struct device *dev,phys_addr_t paddr) +static inline phys_addr_t xen_phys_to_bus (struct device *dev,phys_addr_t paddr) {unsigned long bfn =pfn_to_bfn (XEN_PFN_DOWN (paddr)); -dma_addr_t dma = (dma_addr_t )bfn <<XEN_PAGE_SHIFT ; +phys_addr_t baddr = (phys_addr_t )bfn <<XEN_PAGE_SHIFT ; - dma |= paddr & ~XEN_PAGE_MASK ; + baddr |= paddr & ~XEN_PAGE_MASK ; +return baddr; +} -return dma; +static inline dma_addr_t xen_phys_to_dma (struct device *dev,phys_addr_t paddr) +{ +return phys_to_dma (dev,xen_phys_to_bus (dev, paddr)); } -static inline phys_addr_t xen_bus_to_phys (struct device *dev,dma_addr_t baddr) +static inline phys_addr_t xen_bus_to_phys (struct device *dev, +phys_addr_t baddr) {unsigned long xen_pfn =bfn_to_pfn (XEN_PFN_DOWN (baddr)); -dma_addr_t dma = (dma_addr_t )xen_pfn <<XEN_PAGE_SHIFT ; -phys_addr_t paddr = dma; - - paddr |= baddr & ~XEN_PAGE_MASK ; +phys_addr_t paddr = (xen_pfn <<XEN_PAGE_SHIFT ) | + (baddr & ~XEN_PAGE_MASK );return paddr; } +static inline phys_addr_t xen_dma_to_phys (struct device *dev, +dma_addr_t dma_addr) +{ +return xen_bus_to_phys (dev,dma_to_phys (dev, dma_addr)); +} +static inline dma_addr_t xen_virt_to_bus (struct device *dev,void *address) { -return xen_phys_to_bus (dev,virt_to_phys (address)); +return xen_phys_to_dma (dev,virt_to_phys (address)); }static inline int range_straddles_page_boundary (phys_addr_t p,size_t size)
@@ -94,7 +103,7 @@ static inline int range_straddles_page_boundary(phys_addr_t p, size_t size)static int is_xen_swiotlb_buffer (struct device *dev,dma_addr_t dma_addr) { -unsigned long bfn =XEN_PFN_DOWN (dma_addr); +unsigned long bfn =XEN_PFN_DOWN (dma_to_phys (dev, dma_addr));unsigned long xen_pfn =bfn_to_local_pfn (bfn);phys_addr_t paddr = (phys_addr_t )xen_pfn <<XEN_PAGE_SHIFT ;
@@ -299,12 +308,12 @@ xen_swiotlb_alloc_coherent(struct device *hwdev, size_t size,if (hwdev && hwdev->coherent_dma_mask) dma_mask = hwdev->coherent_dma_mask; -/* At this point dma_handle is the physical address, next we are +/* At this point dma_handle is the dma address, next we are * going to set it to the machine address. * Do not use virt_to_phys(ret) because on ARM it doesn't correspond * to *dma_handle. */ - phys = *dma_handle; - dev_addr =xen_phys_to_bus (hwdev, phys); + phys =dma_to_phys (hwdev, *dma_handle); + dev_addr =xen_phys_to_dma (hwdev, phys);if (((dev_addr + size -1 <= dma_mask)) && !range_straddles_page_boundary (phys, size)) *dma_handle = dev_addr;
@@ -314,6 +323,7 @@ xen_swiotlb_alloc_coherent(struct device *hwdev, size_t size,xen_free_coherent_pages (hwdev, size, ret, (dma_addr_t )phys, attrs);return NULL ; } + *dma_handle =phys_to_dma (hwdev, *dma_handle);SetPageXenRemapped (virt_to_page (ret)); }memset (ret,0 , size);
@@ -334,7 +344,7 @@ xen_swiotlb_free_coherent(struct device *hwdev, size_t size, void *vaddr,/* do not use virt_to_phys because on ARM it doesn't return you the * physical address */ - phys =xen_bus_to_phys (hwdev, dev_addr); + phys =xen_dma_to_phys (hwdev, dev_addr);/* Convert the size to actually allocated. */ size =1UL << (order +XEN_PAGE_SHIFT );
@@ -349,7 +359,8 @@ xen_swiotlb_free_coherent(struct device *hwdev, size_t size, void *vaddr, TestClearPageXenRemapped(page))xen_destroy_contiguous_region (phys, order); -xen_free_coherent_pages (hwdev, size, vaddr, (dma_addr_t )phys, attrs); +xen_free_coherent_pages (hwdev, size, vaddr,phys_to_dma (hwdev, phys), + attrs); }/*
@@ -365,7 +376,7 @@ static dma_addr_t xen_swiotlb_map_page(struct device *dev, struct page *page, unsigned long attrs) {phys_addr_t map, phys =page_to_phys (page) + offset; -dma_addr_t dev_addr =xen_phys_to_bus (dev, phys); +dma_addr_t dev_addr =xen_phys_to_dma (dev, phys);BUG_ON (dir ==DMA_NONE );/*
@@ -390,7 +401,7 @@ static dma_addr_t xen_swiotlb_map_page(struct device *dev, struct page *page,return DMA_MAPPING_ERROR ; phys = map; - dev_addr =xen_phys_to_bus (dev, map); + dev_addr =xen_phys_to_dma (dev, map);/* * Ensure that the address returned is DMA'ble
@@ -418,7 +429,7 @@ static dma_addr_t xen_swiotlb_map_page(struct device *dev, struct page *page,static void xen_swiotlb_unmap_page (struct device *hwdev,dma_addr_t dev_addr,size_t size,enum dma_data_direction dir,unsigned long attrs) { -phys_addr_t paddr =xen_bus_to_phys (hwdev, dev_addr); +phys_addr_t paddr =xen_dma_to_phys (hwdev, dev_addr);BUG_ON (dir ==DMA_NONE );
@@ -434,7 +445,7 @@ static void xen_swiotlb_sync_single_for_cpu(struct device *dev, dma_addr_t dma_addr, size_t size, enum dma_data_direction dir) { - phys_addr_t paddr = xen_bus_to_phys(dev, dma_addr); + phys_addr_t paddr = xen_dma_to_phys(dev, dma_addr);if (!dev_is_dma_coherent (dev))xen_dma_sync_for_cpu (dev, dma_addr, paddr, size, dir);
@@ -447,7 +458,7 @@ static void xen_swiotlb_sync_single_for_device(struct device *dev, dma_addr_t dma_addr, size_t size, enum dma_data_direction dir) { - phys_addr_t paddr = xen_bus_to_phys(dev, dma_addr); + phys_addr_t paddr = xen_dma_to_phys(dev, dma_addr);if (is_xen_swiotlb_buffer (dev, dma_addr))swiotlb_tbl_sync_single (dev, paddr, size, dir,SYNC_FOR_DEVICE );
Signature · 1 line
2.17.1
Re: [PATCH v3 09/11] swiotlb-xen: introduce phys_to_dma/dma_to_phys translations
Quoted 18 lines · Stefano Stabellini
From: Stefano Stabellini <[email protected]>
With some devices physical addresses are different than dma addresses. To be able to deal with these cases, we need to call
phys_to_dmaon physical addresses (including machine addresses in Xen terminology) before returning them fromxen_swiotlb_alloc_coherentandxen_swiotlb_map_page.We also need to convert dma addresses back to physical addresses using
dma_to_physinxen_swiotlb_free_coherentandxen_swiotlb_unmap_pageif we want to do any operations on them.Call
dma_to_physinis_xen_swiotlb_buffer.
Introducexen_phys_to_dmaand callphys_to_dmain its implementation. Introducexen_dma_to_physand calldma_to_physin its implementation. Call xen_phys_to_dma/xen_dma_to_phys instead of xen_phys_to_bus/xen_bus_to_phys throughswiotlb-xen.c.
Quoted 7 lines
Everything is taken care of by these changes except for
xen_swiotlb_alloc_coherentandxen_swiotlb_free_coherent, which need a few explicit phys_to_dma/dma_to_phys calls.Signed-off-by Stefano Stabellini <[email protected]>Tested-by Corey Minyard <[email protected]>Tested-by Roman Shaposhnik <[email protected]>
Juergen
Stefano Stabellini
maintainerPATCH v3 10/11 xen/arm: introduce phys/dma translations in xen_dma_sync_for_*
In reply to Stefano StabelliniFrom: Stefano Stabellini <[email protected]>
xen_dma_sync_for_cpu, xen_dma_sync_for_device, xen_arch_need_swiotlb are getting called passing dma addresses. On some platforms dma addresses could be different from physical addresses. Before doing any operations on these addresses we need to convert them back to physical addresses using dma_to_phys.
Move the arch_sync_dma_for_cpu and arch_sync_dma_for_device calls from xen_dma_sync_for_cpu/device to swiotlb-xen.c, and add a call dma_to_phys to do address translations there.
dma_cache_maint is fixed by the next patch.
---
Changes in v2:
- improve commit message
- don't use pfn_valid
Changes in v3:
- move arch_sync_dma_for_cpu/device calls to swiotlb-xen.c
--- arch/arm/xen/mm.c | 17 ++++++----------- drivers/xen/swiotlb-xen.c | 32 ++++++++++++++++++++++++-------- include/xen/swiotlb-xen.h | 6 ++---- 3 files changed, 32 insertions(+), 23 deletions(-)
arch/arm/xen/mm.c +6 -11
@@ -1,5 +1,6 @@// SPDX-License-Identifier: GPL-2.0-only #include <linux/cpu.h> +#include <linux/dma-direct.h> #include <linux/dma-noncoherent.h> #include <linux/gfp.h> #include <linux/highmem.h>
@@ -72,22 +73,16 @@ static void dma_cache_maint(dma_addr_t handle, size_t size, u32 op)* dma-direct functions, otherwise we call the Xen specific version. */ void xen_dma_sync_for_cpu (struct device *dev,dma_addr_t handle, -phys_addr_t paddr,size_t size, -enum dma_data_direction dir) +size_t size,enum dma_data_direction dir) { -if (pfn_valid (PFN_DOWN (handle))) -arch_sync_dma_for_cpu (paddr, size, dir); -else if (dir !=DMA_TO_DEVICE ) +if (dir !=DMA_TO_DEVICE )dma_cache_maint (handle, size,GNTTAB_CACHE_INVAL ); }void xen_dma_sync_for_device (struct device *dev,dma_addr_t handle, -phys_addr_t paddr,size_t size, -enum dma_data_direction dir) +size_t size,enum dma_data_direction dir) { -if (pfn_valid (PFN_DOWN (handle))) -arch_sync_dma_for_device (paddr, size, dir); -else if (dir ==DMA_FROM_DEVICE ) +if (dir ==DMA_FROM_DEVICE )dma_cache_maint (handle, size,GNTTAB_CACHE_INVAL );else dma_cache_maint (handle, size,GNTTAB_CACHE_CLEAN );
@@ -98,7 +93,7 @@ bool xen_arch_need_swiotlb(struct device *dev, dma_addr_t dev_addr) {unsigned int xen_pfn =XEN_PFN_DOWN (phys); -unsigned int bfn =XEN_PFN_DOWN (dev_addr); +unsigned int bfn =XEN_PFN_DOWN (dma_to_phys (dev, dev_addr));/* * The swiotlb buffer should be used if
drivers/xen/swiotlb-xen.c +24 -8
@@ -413,8 +413,12 @@ static dma_addr_t xen_swiotlb_map_page(struct device *dev, struct page *page, } done: -if (!dev_is_dma_coherent (dev) && !(attrs &DMA_ATTR_SKIP_CPU_SYNC )) -xen_dma_sync_for_device (dev, dev_addr, phys, size, dir); +if (!dev_is_dma_coherent (dev) && !(attrs &DMA_ATTR_SKIP_CPU_SYNC )) { +if (pfn_valid (PFN_DOWN (dma_to_phys (dev, dev_addr)))) +arch_sync_dma_for_device (phys, size, dir); +else +xen_dma_sync_for_device (dev, dev_addr, size, dir); + }return dev_addr; }
@@ -433,8 +437,12 @@ static void xen_swiotlb_unmap_page(struct device *hwdev, dma_addr_t dev_addr,BUG_ON (dir ==DMA_NONE ); -if (!dev_is_dma_coherent (hwdev) && !(attrs &DMA_ATTR_SKIP_CPU_SYNC )) -xen_dma_sync_for_cpu (hwdev, dev_addr, paddr, size, dir); +if (!dev_is_dma_coherent (hwdev) && !(attrs &DMA_ATTR_SKIP_CPU_SYNC )) { +if (pfn_valid (PFN_DOWN (dma_to_phys (hwdev, dev_addr)))) +arch_sync_dma_for_cpu (paddr, size, dir); +else +xen_dma_sync_for_cpu (hwdev, dev_addr, size, dir); + }/* NOTE: We use dev_addr here, not paddr! */ if (is_xen_swiotlb_buffer (hwdev, dev_addr))
@@ -447,8 +455,12 @@ xen_swiotlb_sync_single_for_cpu(struct device *dev, dma_addr_t dma_addr, {phys_addr_t paddr =xen_dma_to_phys (dev, dma_addr); -if (!dev_is_dma_coherent (dev)) -xen_dma_sync_for_cpu (dev, dma_addr, paddr, size, dir); +if (!dev_is_dma_coherent (dev)) { +if (pfn_valid (PFN_DOWN (dma_to_phys (dev, dma_addr)))) +arch_sync_dma_for_cpu (paddr, size, dir); +else +xen_dma_sync_for_cpu (dev, dma_addr, size, dir); + }if (is_xen_swiotlb_buffer (dev, dma_addr))swiotlb_tbl_sync_single (dev, paddr, size, dir,SYNC_FOR_CPU );
@@ -463,8 +475,12 @@ xen_swiotlb_sync_single_for_device(struct device *dev, dma_addr_t dma_addr,if (is_xen_swiotlb_buffer (dev, dma_addr))swiotlb_tbl_sync_single (dev, paddr, size, dir,SYNC_FOR_DEVICE ); -if (!dev_is_dma_coherent (dev)) -xen_dma_sync_for_device (dev, dma_addr, paddr, size, dir); +if (!dev_is_dma_coherent (dev)) { +if (pfn_valid (PFN_DOWN (dma_to_phys (dev, dma_addr)))) +arch_sync_dma_for_device (paddr, size, dir); +else +xen_dma_sync_for_device (dev, dma_addr, size, dir); + } }/*
include/xen/swiotlb-xen.h +2 -4
@@ -5,11 +5,9 @@#include <linux/swiotlb.h> void xen_dma_sync_for_cpu (struct device *dev,dma_addr_t handle, -phys_addr_t paddr,size_t size, -enum dma_data_direction dir); +size_t size,enum dma_data_direction dir);void xen_dma_sync_for_device (struct device *dev,dma_addr_t handle, -phys_addr_t paddr,size_t size, -enum dma_data_direction dir); +size_t size,enum dma_data_direction dir);extern int xen_swiotlb_init (int verbose,bool early);extern const struct dma_map_ops xen_swiotlb_dma_ops;
Signature · 1 line
2.17.1
Re: [PATCH v3 10/11] xen/arm: introduce phys/dma translations in xen_dma_sync_for_*
Quoted 12 lines · Stefano Stabellini
From: Stefano Stabellini <[email protected]>
xen_dma_sync_for_cpu,xen_dma_sync_for_device,xen_arch_need_swiotlbare getting called passing dma addresses. On some platforms dma addresses could be different from physical addresses. Before doing any operations on these addresses we need to convert them back to physical addresses usingdma_to_phys.Move the
arch_sync_dma_for_cpuandarch_sync_dma_for_devicecalls from xen_dma_sync_for_cpu/device toswiotlb-xen.c, and add a calldma_to_physto do address translations there.
Quoted 5 lines
dma_cache_maintis fixed by the next patch.Signed-off-by Stefano Stabellini <[email protected]>Tested-by Corey Minyard <[email protected]>Tested-by Roman Shaposhnik <[email protected]>
Juergen
Stefano Stabellini
maintainerPATCH v3 11/11 xen/arm: call dma_to_phys on the dma_addr_t parameter of dma_cache_maint
In reply to Stefano StabelliniFrom: Stefano Stabellini <[email protected]>
dma_cache_maint is getting called passing a dma address which could be different from a physical address.
Add a struct device* parameter to dma_cache_maint.
Translate the dma_addr_t parameter of dma_cache_maint by calling dma_to_phys. Do it for the first page and all the following pages, in case of multipage handling.
---
Changes in v2:
- improve commit message
--- arch/arm/xen/mm.c | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-)
arch/arm/xen/mm.c +9 -6
@@ -43,15 +43,18 @@ unsigned long xen_get_swiotlb_free_pages(unsigned int order)static bool hypercall_cflush = false;/* buffers in highmem or foreign pages cannot cross page boundaries */ -static void dma_cache_maint (dma_addr_t handle,size_t size,u32 op) +static void dma_cache_maint (struct device *dev,dma_addr_t handle, +size_t size,u32 op) {struct gnttab_cache_flush cflush; - cflush.a.dev_bus_addr = handle &XEN_PAGE_MASK ; cflush.offset =xen_offset_in_page (handle); cflush.op = op; + handle &=XEN_PAGE_MASK ; do { + cflush.a.dev_bus_addr = dma_to_phys(dev, handle); + if (size + cflush.offset > XEN_PAGE_SIZE) cflush.length = XEN_PAGE_SIZE - cflush.offset; else
@@ -60,7 +63,7 @@ static void dma_cache_maint(dma_addr_t handle, size_t size, u32 op)HYPERVISOR_grant_table_op (GNTTABOP_cache_flush, &cflush,1 ); cflush.offset =0 ; - cflush.a.dev_bus_addr += cflush.length; + handle += cflush.length; size -= cflush.length; }while (size); }
@@ -76,16 +79,16 @@ void xen_dma_sync_for_cpu(struct device *dev, dma_addr_t handle, size_t size, enum dma_data_direction dir) {if (dir !=DMA_TO_DEVICE ) -dma_cache_maint (handle, size,GNTTAB_CACHE_INVAL ); +dma_cache_maint (dev, handle, size,GNTTAB_CACHE_INVAL ); }void xen_dma_sync_for_device (struct device *dev,dma_addr_t handle,size_t size,enum dma_data_direction dir) {if (dir ==DMA_FROM_DEVICE ) -dma_cache_maint (handle, size,GNTTAB_CACHE_INVAL ); +dma_cache_maint (dev, handle, size,GNTTAB_CACHE_INVAL );else -dma_cache_maint (handle, size,GNTTAB_CACHE_CLEAN ); +dma_cache_maint (dev, handle, size,GNTTAB_CACHE_CLEAN ); } bool xen_arch_need_swiotlb(struct device *dev,
Signature · 1 line
2.17.1
Jürgen Groß
maintainerQuoted 16 lines · Stefano Stabellini
Hi all,
This series is a collection of fixes to get Linux running on the RPi4 as dom0. Conceptually there are only two significant changes:
- make sure not to call
virt_to_pageon vmalloc virt addresses (patch #1)
- usephys_to_dmaanddma_to_physto translate phys to/from dma addresses (all other patches)I addressed all comments by Christoph to v2 of the series except from the one about merging the precursor "add struct device *" patches. I can always merge them together at any time as needed.
Quoted 6 lines
Boris gave his Reviewed-by to the whole series v2. I added his Reviewed-by to all patches, including the ones with small cosmetic fixes, except for patch #8 #9 #10 because they are either new or changed significantly in this version of the series.
I retained Roman and Corey's Tested-by.
Series pushed to: xen/tip.git for-linus-5.9
Juergen
Stefano Stabellini
maintainerQuoted 17 lines · Jürgen Groß
Quoted 16 lines · Stefano Stabellini
Hi all,
This series is a collection of fixes to get Linux running on the RPi4 as dom0. Conceptually there are only two significant changes:
- make sure not to call
virt_to_pageon vmalloc virt addresses (patch #1)
- usephys_to_dmaanddma_to_physto translate phys to/from dma addresses (all other patches)I addressed all comments by Christoph to v2 of the series except from the one about merging the precursor "add struct device *" patches. I can always merge them together at any time as needed.
Quoted 8 lines
Quoted 6 lines
Boris gave his Reviewed-by to the whole series v2. I added his Reviewed-by to all patches, including the ones with small cosmetic fixes, except for patch #8 #9 #10 because they are either new or changed significantly in this version of the series.
I retained Roman and Corey's Tested-by.
Series pushed to: xen/tip.git for-linus-5.9
Fantastic, thank you!