Skip to content

RFC Improve swiotlb performance by using physical addresses

Alexander Duyck lkml 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 RFC Latest7 patches 21 repliesIn review

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/7swiotlb: Instead of tracking the end of the swiotlb region just calculate itlib/swiotlb.c+12 -133 replies
2/7swiotlb: Make io_tlb_start a physical address instead of a virtual addresslib/swiotlb.c+31 -306 replies
3/7swiotlb: Make io_tlb_overflow_buffer a physical addresslib/swiotlb.c+34 -270 replies
4/7swiotlb: Return physical addresses when calling swiotlb_tbl_map_singledrivers/xen/swiotlb-xen.c include/linux/swiotlb.h lib/swiotlb.c+56 -500 replies
5/7swiotlb: Use physical addresses for swiotlb_tbl_unmap_singledrivers/xen/swiotlb-xen.c include/linux/swiotlb.h lib/swiotlb.c+22 -200 replies
6/7swiotlb: Use physical addresses instead of virtual in swiotlb_tbl_sync_singledrivers/xen/swiotlb-xen.c include/linux/swiotlb.h lib/swiotlb.c+12 -120 replies
7/7swiotlb: Do not export swiotlb_bounce since there are no external consumersinclude/linux/swiotlb.h lib/swiotlb.c+13 -200 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.

Read the v1 thread

Cover letter [RFC PATCH 0/7] Improve swiotlb performance by using physical addresses

While working on 10Gb/s routing performance I found a significant amount of time was being spent in the swiotlb DMA handler. Further digging found that a significant amount of this was due to the fact that virtual to physical address translation and calling the function that did it. It accounted for nearly 60% of the total overhead.

This patch set works to resolve that by changing the io_tlb_start address and io_tlb_overflow_buffer address from virtual addresses to physical addresses. By doing this, devices that are not making use of bounce buffers can significantly reduce their overhead. In addition I followed through with the cleanup to the point that the only functions that really require the virtual address for the dma buffer are the init, free, and bounce functions.

When running a routing throughput test using small packets I saw roughly a 5% increase in packets rates after applying these patches. This appears to match up with the CPU overhead reduction I was tracking via perf.

Before:
Results 10.29Mps
# Overhead Symbol

# ........ ...........................................................................................................
#
     1.97%  [k] __phys_addr                                                                                   
            |              
            |--24.97%-- swiotlb_sync_single
            |          
            |--16.55%-- is_swiotlb_buffer
            |          
            |--11.25%-- unmap_single
            |          
             --2.71%-- swiotlb_dma_mapping_error
     1.66%  [k] swiotlb_sync_single                                                                                 
     1.45%  [k] is_swiotlb_buffer                                     
     0.53%  [k] unmap_single                                                                                     
     0.52%  [k] swiotlb_map_page                                      
     0.47%  [k] swiotlb_sync_single_for_device                                                                 
     0.43%  [k] swiotlb_sync_single_for_cpu                           
     0.42%  [k] swiotlb_dma_mapping_error                                                               
     0.34%  [k] swiotlb_unmap_page                

After:
Results 10.99Mps
# Overhead Symbol

# ........ ...........................................................................................................
#
     0.50%  [k] swiotlb_map_page                                                                                        
     0.50%  [k] swiotlb_sync_single                                                                                     
     0.36%  [k] swiotlb_sync_single_for_cpu                                                                             
     0.35%  [k] swiotlb_sync_single_for_device                                                                          
     0.25%  [k] swiotlb_unmap_page                                                                                      
     0.17%  [k] swiotlb_dma_mapping_error  

---

Alexander Duyck (7):
swiotlb: Do not export swiotlb_bounce since there are no external consumers
swiotlb: Use physical addresses instead of virtual in swiotlb_tbl_sync_single
swiotlb: Use physical addresses for swiotlb_tbl_unmap_single
swiotlb: Return physical addresses when calling swiotlb_tbl_map_single
swiotlb: Make io_tlb_overflow_buffer a physical address
swiotlb: Make io_tlb_start a physical address instead of a virtual address
swiotlb: Instead of tracking the end of the swiotlb region just calculate it

 drivers/xen/swiotlb-xen.c |   25 ++---
 include/linux/swiotlb.h   |   20 ++--
 lib/swiotlb.c             |  247 +++++++++++++++++++++++----------------------
 3 files changed, 150 insertions(+), 142 deletions(-)