Skip to contentSkip to conversation

swiotlb: Add child io tlb mem support

RFC PATCH In review Tianyu Lan 2 participants 5 messages May 26, 2022 – May 27, 2022

Subsystems (5)
lore
Share & tools
lore

Tianyu Lan

author
#0 lore raw

v3 cover swiotlb: Add child io tlb mem support

From: Tianyu Lan <[email protected]>

Traditionally swiotlb was not performance critical because it was only used for slow devices. But in some setups, like TDX/SEV confidential guests, all IO has to go through swiotlb. Currently swiotlb only has a single lock. Under high IO load with multiple CPUs this can lead to significant lock contention on the swiotlb lock.

This patch adds child IO TLB mem support to resolve spinlock overhead among device's queues. Each device may allocate IO tlb mem and setup child IO TLB mem according to queue number. The number child IO tlb mem maybe set up equal with device queue number and this helps to resolve swiotlb spinlock overhead among devices and queues.

introduces IO TLB Block concepts and swiotlb_device_allocate() API to allocate per-device swiotlb bounce buffer. The new API Accepts queue number as the number of child IO TLB mem to set up device's IO TLB mem.

Patch 2 calls new allocation function in the netvsc driver to resolve global spin lock issue.

Tianyu Lan (2):
swiotlb: Add Child IO TLB mem support
net: netvsc: Allocate per-device swiotlb bounce buffer for netvsc

 drivers/net/hyperv/netvsc.c |  10 ++
 include/linux/swiotlb.h     |  38 +++++
 kernel/dma/swiotlb.c        | 299 ++++++++++++++++++++++++++++++++++--
 3 files changed, 334 insertions(+), 13 deletions(-)
Signature · 6 lines
2.25.1

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
author
#1 lore raw

RFC PATCH v3 1/2 swiotlb: Add Child IO TLB mem support

From: Tianyu Lan <[email protected]>

Traditionally swiotlb was not performance critical because it was only used for slow devices. But in some setups, like TDX/SEV confidential guests, all IO has to go through swiotlb. Currently swiotlb only has a single lock. Under high IO load with multiple CPUs this can lead to significant lock contention on the swiotlb lock.

This patch adds child IO TLB mem support to resolve spinlock overhead among device's queues. Each device may allocate IO tlb mem and setup child IO TLB mem according to queue number. Swiotlb code allocates bounce buffer among child IO tlb mem iterately.

Introduce IO TLB Block unit(2MB) concepts to allocate big bounce buffer from default pool for devices. IO TLB segment(256k) is too small for device bounce buffer.

---
 include/linux/swiotlb.h |  38 +++++
 kernel/dma/swiotlb.c    | 304 ++++++++++++++++++++++++++++++++++++++--
 2 files changed, 329 insertions(+), 13 deletions(-)
include/linux/swiotlb.h +38 -0
index 7ed35dd3de6e..a48a9d64e3c3 100644
@@ -31,6 +31,14 @@ struct scatterlist;
 #define IO_TLB_SHIFT 11
 #define IO_TLB_SIZE (1 << IO_TLB_SHIFT)
 
+/*
+ * IO TLB BLOCK UNIT as device bounce buffer allocation unit.
+ * This allows device allocates bounce buffer from default io
+ * tlb pool.
+ */
+#define IO_TLB_BLOCKSIZE   (8 * IO_TLB_SEGSIZE)
+#define IO_TLB_BLOCK_UNIT  (IO_TLB_BLOCKSIZE << IO_TLB_SHIFT)
+
 /* default to 64MB */
 #define IO_TLB_DEFAULT_SIZE (64UL<<20)
 
@@ -89,6 +97,11 @@ extern enum swiotlb_force swiotlb_force;
  * @late_alloc:	%true if allocated using the page allocator
  * @force_bounce: %true if swiotlb bouncing is forced
  * @for_alloc:  %true if the pool is used for memory allocation
+ * @num_child:  The child io tlb mem number in the pool.
+ * @child_nslot:The number of IO TLB slot in the child IO TLB mem.
+ * @child_nblock:The number of IO TLB block in the child IO TLB mem.
+ * @child_start:The child index to start searching in the next round.
+ * @block_start:The block index to start searching in the next round.
  */
 struct io_tlb_mem {
 	phys_addr_t start;
@@ -102,6 +115,16 @@ struct io_tlb_mem {
 	bool late_alloc;
 	bool force_bounce;
 	bool for_alloc;
+	unsigned int num_child;
+	unsigned int child_nslot;
+	unsigned int child_nblock;
+	unsigned int child_start;
+	unsigned int block_index;
+	struct io_tlb_mem *child;
+	struct io_tlb_mem *parent;
+	struct io_tlb_block {
+		unsigned int list;
+	} *block;
 	struct io_tlb_slot {
 		phys_addr_t orig_addr;
 		size_t alloc_size;
@@ -130,6 +153,10 @@ unsigned int swiotlb_max_segment(void);
 size_t swiotlb_max_mapping_size(struct device *dev);
 bool is_swiotlb_active(struct device *dev);
 void __init swiotlb_adjust_size(unsigned long size);
+int swiotlb_device_allocate(struct device *dev,
+			    unsigned int area_num,
+			    unsigned long size);
+void swiotlb_device_free(struct device *dev);
 #else
 static inline void swiotlb_init(bool addressing_limited, unsigned int flags)
 {
@@ -162,6 +189,17 @@ static inline bool is_swiotlb_active(struct device *dev)
 static inline void swiotlb_adjust_size(unsigned long size)
 {
 }
+
+void swiotlb_device_free(struct device *dev)
+{
+}
+
+int swiotlb_device_allocate(struct device *dev,
+			    unsigned int area_num,
+			    unsigned long size)
+{
+	return -ENOMEM;
+}
 #endif /* CONFIG_SWIOTLB */
 
 extern void swiotlb_print_info(void);
kernel/dma/swiotlb.c +291 -13

404 lines not on this page · raw message

Signature · 6 lines
2.25.1

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
author
#2 lore raw

RFC PATCH v3 2/2 net: netvsc: Allocate per-device swiotlb bounce buffer for netvsc

In reply to Tianyu Lan

From: Tianyu Lan <[email protected]>

Netvsc driver allocates device io tlb mem via calling swiotlb_device_ allocate() and set child io tlb mem number according to device queue number. Child io tlb mem may reduce overhead of single spin lock in device io tlb mem among multi device queues.

---
 drivers/net/hyperv/netvsc.c | 10 ++++++++++
 1 file changed, 10 insertions(+)
drivers/net/hyperv/netvsc.c +10 -0
index 9442f751ad3a..26a8f8f84fc4 100644
@@ -23,6 +23,7 @@
 
 #include <asm/sync_bitops.h>
 #include <asm/mshyperv.h>
+#include <linux/swiotlb.h>
 
 #include "hyperv_net.h"
 #include "netvsc_trace.h"
@@ -98,6 +99,7 @@ static void netvsc_subchan_work(struct work_struct *w)
 	struct netvsc_device *nvdev =
 		container_of(w, struct netvsc_device, subchan_work);
 	struct rndis_device *rdev;
+	struct hv_device *hdev;
 	int i, ret;
 
 	/* Avoid deadlock with device removal already under RTNL */
@@ -108,6 +110,9 @@ static void netvsc_subchan_work(struct work_struct *w)
 
 	rdev = nvdev->extension;
 	if (rdev) {
+		hdev = ((struct net_device_context *)
+			netdev_priv(rdev->ndev))->device_ctx;
+
 		ret = rndis_set_subchannel(rdev->ndev, nvdev, NULL);
 		if (ret == 0) {
 			netif_device_attach(rdev->ndev);
@@ -119,6 +124,10 @@ static void netvsc_subchan_work(struct work_struct *w)
 			nvdev->max_chn = 1;
 			nvdev->num_chn = 1;
 		}
+
+		/* Allocate boucne buffer.*/
+		swiotlb_device_allocate(&hdev->device, nvdev->num_chn,
+				10 * IO_TLB_BLOCK_UNIT);
 	}
 
 	rtnl_unlock();
@@ -769,6 +778,7 @@ void netvsc_device_remove(struct hv_device *device)
 
 	/* Release all resources */
 	free_netvsc_device_rcu(net_device);
+	swiotlb_device_free(&device->device);
 }
 
 #define RING_AVAIL_PERCENT_HIWATER 20
Signature · 1 line
2.25.1
#3 lore raw

RE: [RFC PATCH V3 2/2] net: netvsc: Allocate per-device swiotlb bounce buffer for netvsc

drivers/net/hyperv/netvsc.c static void netvsc_subchan_work(struct work_struct · line 132

Quoted 13 lines

From: Tianyu Lan <[email protected]>
Sent: Thursday, May 26, 2022 5:01 AM
...

drivers/net/hyperv/netvsc.c +4 -0
@@ -119,6 +124,10 @@ static void netvsc_subchan_work(struct work_struct
 *w)
 			nvdev->max_chn = 1;
 			nvdev->num_chn = 1;
 		}
+
+		/* Allocate boucne buffer.*/
+		swiotlb_device_allocate(&hdev->device, nvdev->num_chn,
+				10 * IO_TLB_BLOCK_UNIT);
 	}

Looks like swiotlb_device_allocate() is not called if the netvsc device has only 1 primary channel and no sub-schannel, e.g. in the case of single-vCPU VM?
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu

author
#4 lore raw

Re: [RFC PATCH V3 2/2] net: netvsc: Allocate per-device swiotlb bounce buffer for netvsc

Quoted 9 lines · Dexuan Cui
Quoted 9 lines

From: Tianyu Lan <[email protected]>
Sent: Thursday, May 26, 2022 5:01 AM
...

drivers/net/hyperv/netvsc.c +1 -0
@@ -119,6 +124,10 @@ static void netvsc_subchan_work(struct work_struct
 *w)
  			nvdev->max_chn = 1;
  			nvdev->num_chn = 1;
  		}
+
Quoted 8 lines

drivers/net/hyperv/netvsc.c static void netvsc_subchan_work(struct work_struct *w) · line 131

Quoted 4 lines
drivers/net/hyperv/netvsc.c +3 -0
+		/* Allocate boucne buffer.*/
+		swiotlb_device_allocate(&hdev->device, nvdev->num_chn,
+				10 * IO_TLB_BLOCK_UNIT);
  	}

Looks like swiotlb_device_allocate() is not called if the netvsc device has only 1 primary channel and no sub-schannel, e.g. in the case of single-vCPU VM?

When there is only sinlge,there seems not to be much performance
penalty. But you are right, we should keep the same behavior when single CPU and multi CPU. Will update in the next version.

Thanks.

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu