Skip to contentSkip to conversation

xsk: Support UMEM chunk_size > PAGE_SIZE

PATCH Superseded Kal Conley 4 participants 28 messages Apr 6, 2023 – Apr 21, 2023

Subsystems (7)

v3 of 6 Superseded Latest is v6 →

lore
Share & tools
lore

Kal Conley

author
#0 lore raw

v3 cover xsk: Support UMEM chunk_size > PAGE_SIZE

The main purpose of this patchset is to add AF_XDP support for UMEM chunk sizes > PAGE_SIZE. This is enabled for UMEMs backed by HugeTLB pages.

Changes since v2:
* Related fixes/improvements included with v2 have been removed. These changes have all been resubmitted as standalone patchsets.
* Minimize uses of #ifdef CONFIG_HUGETLB_PAGE.
* Improve AF_XDP documentation.
* Update benchmark table in commit message.

Changes since v1:
* Add many fixes/improvements to the XSK selftests.
* Add check for unaligned descriptors that overrun UMEM.
* Fix compile errors when CONFIG_HUGETLB_PAGE is not set.
* Fix incorrect use of _Static_assert.
* Update AF_XDP documentation.
* Rename unaligned 9K frame size test.
* Make xp_check_dma_contiguity less conservative.
* Add more information to benchmark table.

Thanks to Magnus Karlsson for all his support!

Happy easter!

Kal Conley (3):
xsk: Support UMEM chunk_size > PAGE_SIZE
selftests: xsk: Use hugepages when umem->frame_size > PAGE_SIZE
selftests: xsk: Add tests for 8K and 9K frame sizes

 Documentation/networking/af_xdp.rst      | 36 ++++++++++-------
 include/net/xdp_sock.h                   |  1 +
 include/net/xdp_sock_drv.h               | 12 ++++++
 include/net/xsk_buff_pool.h              |  3 +-
 net/xdp/xdp_umem.c                       | 51 ++++++++++++++++++++----
 net/xdp/xsk_buff_pool.c                  | 28 ++++++++-----
 tools/testing/selftests/bpf/xskxceiver.c | 27 ++++++++++++-
 tools/testing/selftests/bpf/xskxceiver.h |  2 +
 8 files changed, 127 insertions(+), 33 deletions(-)
Signature · 1 line
2.39.2
author
#1 lore raw

PATCH v3 1/3 xsk: Support UMEM chunk_size > PAGE_SIZE

Add core AF_XDP support for chunk sizes larger than PAGE_SIZE. This enables sending/receiving jumbo ethernet frames up to the theoretical maxiumum of 64 KiB. For chunk sizes > PAGE_SIZE, the UMEM is required to consist of HugeTLB VMAs (and be hugepage aligned). Initially, only SKB mode is usable pending future driver work.

For consistency, check for HugeTLB pages during UMEM registration. This implies that hugepages are required for XDP_COPY mode despite DMA not being used. This restriction is desirable since it ensures user software can take advantage of future driver support.

Even in HugeTLB mode, continue to do page accounting using order-0 (4 KiB) pages. This minimizes the size of this change and reduces the risk of impacting other code. Taking full advantage of hugepages for accounting should improve XDP performance in the general case.

No significant change in RX/TX performance was observed with this patch. A few data points are reproduced below:

Machine : Dell PowerEdge R940
CPU : Intel(R) Xeon(R) Platinum 8168 CPU @ 2.70GHz
NIC : MT27700 Family [ConnectX-4]

+-----+------+------+-------+--------+--------+--------+
|     |      |      | chunk | packet | rxdrop | rxdrop |
|     | mode |  mtu |  size |   size | (Mpps) | (Gbps) |
+-----+------+------+-------+--------+--------+--------+
| old |   -z | 3498 |  4000 |    320 |   16.0 |   40.9 |
| new |   -z | 3498 |  4000 |    320 |   16.0 |   40.9 |
+-----+------+------+-------+--------+--------+--------+
| old |   -z | 3498 |  4096 |    320 |   16.4 |   42.1 |
| new |   -z | 3498 |  4096 |    320 |   16.5 |   42.2 |
+-----+------+------+-------+--------+--------+--------+
| new |   -c | 3498 | 10240 |    320 |    6.1 |   15.6 |
+-----+------+------+-------+--------+--------+--------+
| new |   -S | 9000 | 10240 |   9000 |   0.37 |   26.9 |
+-----+------+------+-------+--------+--------+--------+
---
 Documentation/networking/af_xdp.rst | 36 ++++++++++++--------
 include/net/xdp_sock.h              |  1 +
 include/net/xdp_sock_drv.h          | 12 +++++++
 include/net/xsk_buff_pool.h         |  3 +-
 net/xdp/xdp_umem.c                  | 51 ++++++++++++++++++++++++-----
 net/xdp/xsk_buff_pool.c             | 28 +++++++++++-----
 6 files changed, 99 insertions(+), 32 deletions(-)
Documentation/networking/af_xdp.rst +22 -14
index 247c6c4127e9..ea65cd882af6 100644
@@ -105,12 +105,13 @@ with AF_XDP". It can be found at https://lwn.net/Articles/750845/.
 UMEM
 ----
 
-UMEM is a region of virtual contiguous memory, divided into
-equal-sized frames. An UMEM is associated to a netdev and a specific
-queue id of that netdev. It is created and configured (chunk size,
-headroom, start address and size) by using the XDP_UMEM_REG setsockopt
-system call. A UMEM is bound to a netdev and queue id, via the bind()
-system call.
+UMEM is a region of virtual contiguous memory divided into equal-sized
+frames. This is the area that contains all the buffers that packets can
+reside in. A UMEM is associated with a netdev and a specific queue id of
+that netdev. It is created and configured (start address, size,
+chunk size, and headroom) by using the XDP_UMEM_REG setsockopt system
+call. A UMEM is bound to a netdev and queue id via the bind() system
+call.
 
 An AF_XDP is socket linked to a single UMEM, but one UMEM can have
 multiple AF_XDP sockets. To share an UMEM created via one socket A,
@@ -418,14 +419,21 @@ negatively impact performance.
 XDP_UMEM_REG setsockopt
 -----------------------
 
-This setsockopt registers a UMEM to a socket. This is the area that
-contain all the buffers that packet can reside in. The call takes a
-pointer to the beginning of this area and the size of it. Moreover, it
-also has parameter called chunk_size that is the size that the UMEM is
-divided into. It can only be 2K or 4K at the moment. If you have an
-UMEM area that is 128K and a chunk size of 2K, this means that you
-will be able to hold a maximum of 128K / 2K = 64 packets in your UMEM
-area and that your largest packet size can be 2K.
+This setsockopt registers a UMEM to a socket. The call takes a pointer
+to the beginning of this area and the size of it. Moreover, there is a
+parameter called chunk_size that is the size that the UMEM is divided
+into. The chunk size limits the maximum packet size that can be sent or
+received. For example, if you have a UMEM area that is 128K and a chunk
+size of 2K, then you will be able to hold a maximum of 128K / 2K = 64
+packets in your UMEM. In this case, the maximum packet size will be 2K.
+
+Valid chunk sizes range from 2K to 64K. However, in aligned mode, the
+chunk size must also be a power of two. Additionally, the chunk size
+must not exceed the size of a page (usually 4K). This limitation is
+relaxed for UMEM areas allocated with HugeTLB pages, in which case
+chunk sizes up to 64K are allowed. Note, this only works with hugepages
+allocated from the kernel's persistent pool. Using Transparent Huge
+Pages (THP) has no effect on the maximum chunk size.
 
 There is also an option to set the headroom of each single buffer in
 the UMEM. If you set this to N bytes, it means that the packet will
include/net/xdp_sock.h +1 -0
index e96a1151ec75..086fcf065656 100644
@@ -30,6 +30,7 @@ struct xdp_umem {
 	u8 flags;
 	bool zc;
 	struct page **pgs;
+	u32 page_size;
 	int id;
 	struct list_head xsk_dma_list;
 	struct work_struct work;
include/net/xdp_sock_drv.h +12 -0
index 9c0d860609ba..83fba3060c9a 100644
@@ -12,6 +12,18 @@
 #define XDP_UMEM_MIN_CHUNK_SHIFT 11
 #define XDP_UMEM_MIN_CHUNK_SIZE (1 << XDP_UMEM_MIN_CHUNK_SHIFT)
 
+static_assert(XDP_UMEM_MIN_CHUNK_SIZE <= PAGE_SIZE);
+
+/* Allow chunk sizes up to the maximum size of an ethernet frame (64 KiB).
+ * Larger chunks are not guaranteed to fit in a single SKB.
+ */
+#ifdef CONFIG_HUGETLB_PAGE
+#define XDP_UMEM_MAX_CHUNK_SHIFT min(16, HPAGE_SHIFT)
+#else
+#define XDP_UMEM_MAX_CHUNK_SHIFT min(16, PAGE_SHIFT)
+#endif
+#define XDP_UMEM_MAX_CHUNK_SIZE (1 << XDP_UMEM_MAX_CHUNK_SHIFT)
+
 #ifdef CONFIG_XDP_SOCKETS
 
 void xsk_tx_completed(struct xsk_buff_pool *pool, u32 nb_entries);
include/net/xsk_buff_pool.h +2 -1
index d318c769b445..6560d053ab88 100644
@@ -75,6 +75,7 @@ struct xsk_buff_pool {
 	u32 chunk_size;
 	u32 chunk_shift;
 	u32 frame_len;
+	u32 page_size;
 	u8 cached_need_wakeup;
 	bool uses_need_wakeup;
 	bool dma_need_sync;
@@ -175,7 +176,7 @@ static inline void xp_dma_sync_for_device(struct xsk_buff_pool *pool,
 static inline bool xp_desc_crosses_non_contig_pg(struct xsk_buff_pool *pool,
 						 u64 addr, u32 len)
 {
-	bool cross_pg = (addr & (PAGE_SIZE - 1)) + len > PAGE_SIZE;
+	bool cross_pg = (addr & (pool->page_size - 1)) + len > pool->page_size;
 
 	if (likely(!cross_pg))
 		return false;
net/xdp/xdp_umem.c +43 -8

78 lines not on this page · raw message

net/xdp/xsk_buff_pool.c +19 -9
index b2df1e0f8153..f9e083fa5e6d 100644
@@ -80,9 +80,10 @@ struct xsk_buff_pool *xp_create_and_assign_umem(struct xdp_sock *xs,
 	pool->headroom = umem->headroom;
 	pool->chunk_size = umem->chunk_size;
 	pool->chunk_shift = ffs(umem->chunk_size) - 1;
-	pool->unaligned = unaligned;
 	pool->frame_len = umem->chunk_size - umem->headroom -
 		XDP_PACKET_HEADROOM;
+	pool->page_size = umem->page_size;
+	pool->unaligned = unaligned;
 	pool->umem = umem;
 	pool->addrs = umem->addrs;
 	INIT_LIST_HEAD(&pool->free_list);
@@ -369,16 +370,25 @@ void xp_dma_unmap(struct xsk_buff_pool *pool, unsigned long attrs)
 }
 EXPORT_SYMBOL(xp_dma_unmap);
 
-static void xp_check_dma_contiguity(struct xsk_dma_map *dma_map)
+/* HugeTLB pools consider contiguity at hugepage granularity only. Hence, all
+ * order-0 pages within a hugepage have the same contiguity value.
+ */
+static void xp_check_dma_contiguity(struct xsk_dma_map *dma_map, u32 page_size)
 {
-	u32 i;
+	u32 stride = page_size >> PAGE_SHIFT; /* in order-0 pages */
+	u32 i, j;
 
-	for (i = 0; i < dma_map->dma_pages_cnt - 1; i++) {
-		if (dma_map->dma_pages[i] + PAGE_SIZE == dma_map->dma_pages[i + 1])
-			dma_map->dma_pages[i] |= XSK_NEXT_PG_CONTIG_MASK;
-		else
-			dma_map->dma_pages[i] &= ~XSK_NEXT_PG_CONTIG_MASK;
+	for (i = 0; i + stride < dma_map->dma_pages_cnt;) {
+		if (dma_map->dma_pages[i] + page_size == dma_map->dma_pages[i + stride]) {
+			for (j = 0; j < stride; i++, j++)
+				dma_map->dma_pages[i] |= XSK_NEXT_PG_CONTIG_MASK;
+		} else {
+			for (j = 0; j < stride; i++, j++)
+				dma_map->dma_pages[i] &= ~XSK_NEXT_PG_CONTIG_MASK;
+		}
 	}
+	for (; i < dma_map->dma_pages_cnt; i++)
+		dma_map->dma_pages[i] &= ~XSK_NEXT_PG_CONTIG_MASK;
 }
 
 static int xp_init_dma_info(struct xsk_buff_pool *pool, struct xsk_dma_map *dma_map)
@@ -441,7 +451,7 @@ int xp_dma_map(struct xsk_buff_pool *pool, struct device *dev,
 	}
 
 	if (pool->unaligned)
-		xp_check_dma_contiguity(dma_map);
+		xp_check_dma_contiguity(dma_map, pool->page_size);
 
 	err = xp_init_dma_info(pool, dma_map);
 	if (err) {
Signature · 1 line
2.39.2
#2 lore raw
Quoted 5 lines · Kal Conley

Add core AF_XDP support for chunk sizes larger than PAGE_SIZE. This enables sending/receiving jumbo ethernet frames up to the theoretical maxiumum of 64 KiB. For chunk sizes > PAGE_SIZE, the UMEM is required to consist of HugeTLB VMAs (and be hugepage aligned). Initially, only SKB mode is usable pending future driver work.

Hmm, interesting. So how does this interact with XDP multibuf?

-Toke

#3 lore raw
Quoted 9 lines · Toke Høiland-Jørgensen
Quoted 5 lines · Kal Conley

Add core AF_XDP support for chunk sizes larger than PAGE_SIZE. This enables sending/receiving jumbo ethernet frames up to the theoretical maxiumum of 64 KiB. For chunk sizes > PAGE_SIZE, the UMEM is required to consist of HugeTLB VMAs (and be hugepage aligned). Initially, only SKB mode is usable pending future driver work.

Hmm, interesting. So how does this interact with XDP multibuf?

To me it currently does not interact with mbuf in any way as it is enabled only for skb mode which linearizes the skb from what i see.

I'd like to hear more about Kal's use case - Kal do you use AF_XDP in SKB mode on your side?

Quoted 3 lines

-Toke

#4 lore raw
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines

Add core AF_XDP support for chunk sizes larger than PAGE_SIZE. This enables sending/receiving jumbo ethernet frames up to the theoretical maxiumum of 64 KiB. For chunk sizes > PAGE_SIZE, the UMEM is required to consist of HugeTLB VMAs (and be hugepage aligned). Initially, only SKB mode is usable pending future driver work.

Quoted 7 lines
Quoted 1 line

Hmm, interesting. So how does this interact with XDP multibuf?

To me it currently does not interact with mbuf in any way as it is enabled only for skb mode which linearizes the skb from what i see.

I'd like to hear more about Kal's use case - Kal do you use AF_XDP in SKB mode on your side?

Our use-case is to receive jumbo Ethernet frames up to 9000 bytes with AF_XDP in zero-copy mode. This patchset is a step in this direction. At the very least, it lets you test out the feature in SKB mode pending future driver support. Currently, XDP multi-buffer does not support AF_XDP at all. It could support it in theory, but I think it would need some UAPI design work and a bit of implementation work.

Also, I think that the approach taken in this patchset has some advantages over XDP multi-buffer:
(1) It should be possible to achieve higher performance
(a) because the packet data is kept together
(b) because you need to acquire and validate less descriptors
and touch the queue pointers less often.
(2) It is a nicer user-space API.
(a) Since the packet data is all available in one linear
buffer. This may even be a requirement to avoid an extra copy if the data must be handed off contiguously to other code.

The disadvantage of this patchset is requiring the user to allocate HugeTLB pages which is an extra complication.

I am not sure if this patchset would need to interact with XDP multi-buffer at all directly. Does anyone have anything to add here?

What other intermediate steps are needed to get to ZC? I think drivers would already be able to support this now?

Kal

#5 lore raw
Quoted 32 lines · Kal Cutter Conley
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines

Add core AF_XDP support for chunk sizes larger than PAGE_SIZE. This enables sending/receiving jumbo ethernet frames up to the theoretical maxiumum of 64 KiB. For chunk sizes > PAGE_SIZE, the UMEM is required to consist of HugeTLB VMAs (and be hugepage aligned). Initially, only SKB mode is usable pending future driver work.

Quoted 7 lines
Quoted 1 line

Hmm, interesting. So how does this interact with XDP multibuf?

To me it currently does not interact with mbuf in any way as it is enabled only for skb mode which linearizes the skb from what i see.

I'd like to hear more about Kal's use case - Kal do you use AF_XDP in SKB mode on your side?

Our use-case is to receive jumbo Ethernet frames up to 9000 bytes with AF_XDP in zero-copy mode. This patchset is a step in this direction. At the very least, it lets you test out the feature in SKB mode pending future driver support. Currently, XDP multi-buffer does not support AF_XDP at all. It could support it in theory, but I think it would need some UAPI design work and a bit of implementation work.

Also, I think that the approach taken in this patchset has some advantages over XDP multi-buffer:
(1) It should be possible to achieve higher performance
(a) because the packet data is kept together
(b) because you need to acquire and validate less descriptors
and touch the queue pointers less often.
(2) It is a nicer user-space API.
(a) Since the packet data is all available in one linear
buffer. This may even be a requirement to avoid an extra copy if the data must be handed off contiguously to other code.

Quoted 5 lines

The disadvantage of this patchset is requiring the user to allocate HugeTLB pages which is an extra complication.

I am not sure if this patchset would need to interact with XDP multi-buffer at all directly. Does anyone have anything to add here?

Well, I'm mostly concerned with having two different operation and configuration modes for the same thing. We'll probably need to support multibuf for AF_XDP anyway for the non-ZC path, which means we'll need to create a UAPI for that in any case. And having two APIs is just going to be more complexity to handle at both the documentation and maintenance level.

It *might* be worth it to do this if the performance benefit is really compelling, but, well, you'd need to implement both and compare directly to know that for sure :)

-Toke

#6 lore raw
Quoted 41 lines · Toke Høiland-Jørgensen
Quoted 32 lines · Kal Cutter Conley
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines

Add core AF_XDP support for chunk sizes larger than PAGE_SIZE. This enables sending/receiving jumbo ethernet frames up to the theoretical maxiumum of 64 KiB. For chunk sizes > PAGE_SIZE, the UMEM is required to consist of HugeTLB VMAs (and be hugepage aligned). Initially, only SKB mode is usable pending future driver work.

Quoted 7 lines
Quoted 1 line

Hmm, interesting. So how does this interact with XDP multibuf?

To me it currently does not interact with mbuf in any way as it is enabled only for skb mode which linearizes the skb from what i see.

I'd like to hear more about Kal's use case - Kal do you use AF_XDP in SKB mode on your side?

Our use-case is to receive jumbo Ethernet frames up to 9000 bytes with AF_XDP in zero-copy mode. This patchset is a step in this direction. At the very least, it lets you test out the feature in SKB mode pending future driver support. Currently, XDP multi-buffer does not support AF_XDP at all. It could support it in theory, but I think it would need some UAPI design work and a bit of implementation work.

Also, I think that the approach taken in this patchset has some advantages over XDP multi-buffer:
(1) It should be possible to achieve higher performance
(a) because the packet data is kept together
(b) because you need to acquire and validate less descriptors
and touch the queue pointers less often.
(2) It is a nicer user-space API.
(a) Since the packet data is all available in one linear
buffer. This may even be a requirement to avoid an extra copy if the data must be handed off contiguously to other code.

Quoted 5 lines

The disadvantage of this patchset is requiring the user to allocate HugeTLB pages which is an extra complication.

I am not sure if this patchset would need to interact with XDP multi-buffer at all directly. Does anyone have anything to add here?

Quoted 6 lines

Well, I'm mostly concerned with having two different operation and configuration modes for the same thing. We'll probably need to support multibuf for AF_XDP anyway for the non-ZC path, which means we'll need to create a UAPI for that in any case. And having two APIs is just going to be more complexity to handle at both the documentation and maintenance level.

One does not replace the other. We need them both, unfortunately. Multi-buff is great for e.g., stitching together different headers with the same data. Point to different buffers for the header in each packet but the same piece of data in all of them. This will never be solved with Kal's approach. We just need multi-buffer support for this. BTW, we are close to posting multi-buff support for AF_XDP. Just hang in there a little while longer while the last glitches are fixed. We have to stage it in two patch sets as it will be too long otherwise. First one will only contain improvements to the xsk selftests framework so that multi-buffer tests can be supported. The second one will be the core code and the actual multi-buffer tests. As for what Kal's patches are good for, please see below.

Quoted 3 lines

It *might* be worth it to do this if the performance benefit is really compelling, but, well, you'd need to implement both and compare directly to know that for sure :)

The performance benefit is compelling. As I wrote in a mail to a post by Kal, there are users out there that state that this feature (for zero-copy mode nota bene) is a must for them to be able to use AF_XDP instead of DPDK style user-mode drivers. They have really tough latency requirements.

Quoted 2 lines

-Toke

#7 lore raw
Quoted 49 lines · Magnus Karlsson
Quoted 41 lines · Toke Høiland-Jørgensen
Quoted 32 lines · Kal Cutter Conley
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines

Add core AF_XDP support for chunk sizes larger than PAGE_SIZE. This enables sending/receiving jumbo ethernet frames up to the theoretical maxiumum of 64 KiB. For chunk sizes > PAGE_SIZE, the UMEM is required to consist of HugeTLB VMAs (and be hugepage aligned). Initially, only SKB mode is usable pending future driver work.

Quoted 7 lines
Quoted 1 line

Hmm, interesting. So how does this interact with XDP multibuf?

To me it currently does not interact with mbuf in any way as it is enabled only for skb mode which linearizes the skb from what i see.

I'd like to hear more about Kal's use case - Kal do you use AF_XDP in SKB mode on your side?

Our use-case is to receive jumbo Ethernet frames up to 9000 bytes with AF_XDP in zero-copy mode. This patchset is a step in this direction. At the very least, it lets you test out the feature in SKB mode pending future driver support. Currently, XDP multi-buffer does not support AF_XDP at all. It could support it in theory, but I think it would need some UAPI design work and a bit of implementation work.

Also, I think that the approach taken in this patchset has some advantages over XDP multi-buffer:
(1) It should be possible to achieve higher performance
(a) because the packet data is kept together
(b) because you need to acquire and validate less descriptors
and touch the queue pointers less often.
(2) It is a nicer user-space API.
(a) Since the packet data is all available in one linear
buffer. This may even be a requirement to avoid an extra copy if the data must be handed off contiguously to other code.

Quoted 5 lines

The disadvantage of this patchset is requiring the user to allocate HugeTLB pages which is an extra complication.

I am not sure if this patchset would need to interact with XDP multi-buffer at all directly. Does anyone have anything to add here?

Quoted 6 lines

Well, I'm mostly concerned with having two different operation and configuration modes for the same thing. We'll probably need to support multibuf for AF_XDP anyway for the non-ZC path, which means we'll need to create a UAPI for that in any case. And having two APIs is just going to be more complexity to handle at both the documentation and maintenance level.

Quoted 11 lines

One does not replace the other. We need them both, unfortunately. Multi-buff is great for e.g., stitching together different headers with the same data. Point to different buffers for the header in each packet but the same piece of data in all of them. This will never be solved with Kal's approach. We just need multi-buffer support for this. BTW, we are close to posting multi-buff support for AF_XDP. Just hang in there a little while longer while the last glitches are fixed. We have to stage it in two patch sets as it will be too long otherwise. First one will only contain improvements to the xsk selftests framework so that multi-buffer tests can be supported. The second one will be the core code and the actual multi-buffer tests.

Alright, sounds good!

Quoted 11 lines

As for what Kal's patches are good for, please see below.

Quoted 3 lines

It *might* be worth it to do this if the performance benefit is really compelling, but, well, you'd need to implement both and compare directly to know that for sure :)

The performance benefit is compelling. As I wrote in a mail to a post by Kal, there are users out there that state that this feature (for zero-copy mode nota bene) is a must for them to be able to use AF_XDP instead of DPDK style user-mode drivers. They have really tough latency requirements.

Hmm, okay, looking forward to seeing the benchmark results, then! :)

-Toke

#8 lore raw
In reply to Toke Høiland-Jørgensen
Quoted 7 lines

Well, I'm mostly concerned with having two different operation and configuration modes for the same thing. We'll probably need to support multibuf for AF_XDP anyway for the non-ZC path, which means we'll need to create a UAPI for that in any case. And having two APIs is just going to be more complexity to handle at both the documentation and maintenance level.

I don't know if I would call this another "API". This patchset doesn't change the semantics of anything. It only lifts the chunk size restriction when hugepages are used. Furthermore, the changes here are quite small and easy to understand. The four sentences added to the documentation shouldn't be too concerning either. :-)

In 30 years when everyone finally migrates to page sizes >= 64K the maintenance burden will drop to zero. Knock wood. :-)

Quoted 4 lines

It *might* be worth it to do this if the performance benefit is really compelling, but, well, you'd need to implement both and compare directly to know that for sure :)

What about use-cases that require incoming packet data to be contiguous? Without larger chunk sizes, the user is forced to allocate extra space per packet and copy the data. This defeats the purpose of ZC.

#9 lore raw
Quoted 8 lines · Kal Cutter Conley
Quoted 7 lines

Well, I'm mostly concerned with having two different operation and configuration modes for the same thing. We'll probably need to support multibuf for AF_XDP anyway for the non-ZC path, which means we'll need to create a UAPI for that in any case. And having two APIs is just going to be more complexity to handle at both the documentation and maintenance level.

Quoted 5 lines

I don't know if I would call this another "API". This patchset doesn't change the semantics of anything. It only lifts the chunk size restriction when hugepages are used. Furthermore, the changes here are quite small and easy to understand. The four sentences added to the documentation shouldn't be too concerning either. :-)

Well, you mentioned yourself that:

Quoted 2 lines

The disadvantage of this patchset is requiring the user to allocate HugeTLB pages which is an extra complication.

In addition, presumably when using this mode, the other XDP actions (XDP_PASS, XDP_REDIRECT to other targets) would stop working unless we add special handling for that in the kernel? We'll definitely need to handle that somehow...

Quoted 2 lines

In 30 years when everyone finally migrates to page sizes >= 64K the maintenance burden will drop to zero. Knock wood. :-)

Haha, right, but let's make sure we have something that is consistent in the intervening decades, shall we? ;)

Quoted 8 lines
Quoted 3 lines

It *might* be worth it to do this if the performance benefit is really compelling, but, well, you'd need to implement both and compare directly to know that for sure :)

What about use-cases that require incoming packet data to be contiguous? Without larger chunk sizes, the user is forced to allocate extra space per packet and copy the data. This defeats the purpose of ZC.

What use cases would that be, exactly? Presumably this is also a performance issue? Which goes back to me asking for benchmarks :)

-Toke

#10 lore raw
Quoted 4 lines

Well, you mentioned yourself that:

Quoted 2 lines

The disadvantage of this patchset is requiring the user to allocate HugeTLB pages which is an extra complication.

It's a small extra complication *for the user*. However, users that need this feature are willing to allocate hugepages. We are one such user. For us, having to deal with packets split into disjoint buffers (from the XDP multi-buffer paradigm) is a significantly more annoying complication than allocating hugepages (particularly on the RX side).

#11 lore raw
Quoted 10 lines · Kal Cutter Conley
Quoted 4 lines

Well, you mentioned yourself that:

Quoted 2 lines

The disadvantage of this patchset is requiring the user to allocate HugeTLB pages which is an extra complication.

It's a small extra complication *for the user*. However, users that need this feature are willing to allocate hugepages. We are one such user. For us, having to deal with packets split into disjoint buffers (from the XDP multi-buffer paradigm) is a significantly more annoying complication than allocating hugepages (particularly on the RX side).

"More annoying" is not a great argument, though. You're basically saying "please complicate your code so I don't have to complicate mine". And since kernel API is essentially frozen forever, adding more of them carries a pretty high cost, which is why kernel developers tend not to be easily swayed by convenience arguments (if all you want is a more convenient API, just build one on top of the kernel primitives and wrap it into a library).

So you'll need to come up with either (1) a use case that you *can't* solve without this new API (with specifics as to why that is the case), or (2) a compelling performance benchmark showing the complexity is worth it. Magnus indicated he would be able to produce the latter, in which case I'm happy to be persuaded by the numbers.

In any case, however, the behaviour needs to be consistent wrt the rest of XDP, so it's not as simple as just increasing the limit (as I mentioned in my previous email).

-Toke

#12 lore raw
Quoted 7 lines

"More annoying" is not a great argument, though. You're basically saying "please complicate your code so I don't have to complicate mine". And since kernel API is essentially frozen forever, adding more of them carries a pretty high cost, which is why kernel developers tend not to be easily swayed by convenience arguments (if all you want is a more convenient API, just build one on top of the kernel primitives and wrap it into a library).

I was trying to make a fair comparison from the user's perspective between having to allocate huge pages and deal with discontiguous buffers. That was all.

I think the "your code" distinction is a bit harsh. The kernel is a community project. Why isn't it "our" code? I am trying to add a feature that I think is generally useful to people. The kernel only exists to serve its users. I believe I am doing more good than harm sending these patches.

#13 lore raw
Quoted 8 lines · Kal Cutter Conley
Quoted 7 lines

"More annoying" is not a great argument, though. You're basically saying "please complicate your code so I don't have to complicate mine". And since kernel API is essentially frozen forever, adding more of them carries a pretty high cost, which is why kernel developers tend not to be easily swayed by convenience arguments (if all you want is a more convenient API, just build one on top of the kernel primitives and wrap it into a library).

Quoted 8 lines

I was trying to make a fair comparison from the user's perspective between having to allocate huge pages and deal with discontiguous buffers. That was all.

I think the "your code" distinction is a bit harsh. The kernel is a community project. Why isn't it "our" code? I am trying to add a feature that I think is generally useful to people. The kernel only exists to serve its users.

Oh, I'm sorry if that came across as harsh, that was not my intention! I was certainly not trying to make a "you vs us" distinction; I was just trying to explain why making changes on the kernel side carries a higher cost than an equivalent (or even slightly more complex) change on the userspace side, because of the UAPI consideration.

Quoted 1 line

I believe I am doing more good than harm sending these patches.

I don't think so! You've certainly sparked a discussion, that is good :)

-Toke

#14 lore raw
Quoted 17 lines
Quoted 8 lines
Quoted 7 lines

"More annoying" is not a great argument, though. You're basically saying "please complicate your code so I don't have to complicate mine". And since kernel API is essentially frozen forever, adding more of them carries a pretty high cost, which is why kernel developers tend not to be easily swayed by convenience arguments (if all you want is a more convenient API, just build one on top of the kernel primitives and wrap it into a library).

Quoted 8 lines

I was trying to make a fair comparison from the user's perspective between having to allocate huge pages and deal with discontiguous buffers. That was all.

I think the "your code" distinction is a bit harsh. The kernel is a community project. Why isn't it "our" code? I am trying to add a feature that I think is generally useful to people. The kernel only exists to serve its users.

Quoted 5 lines

Oh, I'm sorry if that came across as harsh, that was not my intention! I was certainly not trying to make a "you vs us" distinction; I was just trying to explain why making changes on the kernel side carries a higher cost than an equivalent (or even slightly more complex) change on the userspace side, because of the UAPI consideration.

No problem! I agree. I am just somewhat confused by the "slightly more complex" view of the situation. Currently, packets > 4K are not supported _at all_ with AF_XDP. So this patchset adds something that is not possible _at all_ today. We have been patiently waiting since 2018 for AF_XDP jumbo packet support. It seems that interest in this feature from maintainers has been... lacking. :-) Now, if I understand the situation correctly, you are asking for benchmarks to compare this implementation with AF_XDP multi-buffer which doesn't exist yet? I am glad it's being worked on but until there is a patchset, AF_XDP multi-buffer is still VAPORWARE. :-)

Now in our case, we are primarily interested in throughput and reducing total system load (so we have more compute budget for other tasks). The faster we can receive packets the better. The packets we need to receive are (almost) all 8000-9000 bytes... Using AF_XDP multi-buffer would mean either (1) allocating extra space per packet and copying the data to linearize it or (2) rewriting a significant amount of code to handle segmented packets efficiently. If you want benchmarks for (1) just use xdpsock and compare -z with -c. I think that is a good approximation...

Hopefully, the HFT crowd can save this patchset in the end. :-)

-Kal

#15 lore raw
In reply to Toke Høiland-Jørgensen
Quoted 22 lines · Toke Høiland-Jørgensen
Quoted 10 lines · Kal Cutter Conley
Quoted 4 lines

Well, you mentioned yourself that:

Quoted 2 lines

The disadvantage of this patchset is requiring the user to allocate HugeTLB pages which is an extra complication.

It's a small extra complication *for the user*. However, users that need this feature are willing to allocate hugepages. We are one such user. For us, having to deal with packets split into disjoint buffers (from the XDP multi-buffer paradigm) is a significantly more annoying complication than allocating hugepages (particularly on the RX side).

"More annoying" is not a great argument, though. You're basically saying "please complicate your code so I don't have to complicate mine". And since kernel API is essentially frozen forever, adding more of them carries a pretty high cost, which is why kernel developers tend not to be easily swayed by convenience arguments (if all you want is a more convenient API, just build one on top of the kernel primitives and wrap it into a library).

Quoted 5 lines

So you'll need to come up with either (1) a use case that you *can't* solve without this new API (with specifics as to why that is the case), or (2) a compelling performance benchmark showing the complexity is worth it. Magnus indicated he would be able to produce the latter, in which case I'm happy to be persuaded by the numbers.

We will measure it and get back to you. Would be good with some numbers.

Quoted 6 lines

In any case, however, the behaviour needs to be consistent wrt the rest of XDP, so it's not as simple as just increasing the limit (as I mentioned in my previous email).

-Toke

#16 lore raw
Quoted 23 lines · Magnus Karlsson
Quoted 22 lines · Toke Høiland-Jørgensen
Quoted 10 lines · Kal Cutter Conley
Quoted 4 lines

Well, you mentioned yourself that:

Quoted 2 lines

The disadvantage of this patchset is requiring the user to allocate HugeTLB pages which is an extra complication.

It's a small extra complication *for the user*. However, users that need this feature are willing to allocate hugepages. We are one such user. For us, having to deal with packets split into disjoint buffers (from the XDP multi-buffer paradigm) is a significantly more annoying complication than allocating hugepages (particularly on the RX side).

"More annoying" is not a great argument, though. You're basically saying "please complicate your code so I don't have to complicate mine". And since kernel API is essentially frozen forever, adding more of them carries a pretty high cost, which is why kernel developers tend not to be easily swayed by convenience arguments (if all you want is a more convenient API, just build one on top of the kernel primitives and wrap it into a library).

Quoted 8 lines
Quoted 5 lines

So you'll need to come up with either (1) a use case that you *can't* solve without this new API (with specifics as to why that is the case), or (2) a compelling performance benchmark showing the complexity is worth it. Magnus indicated he would be able to produce the latter, in which case I'm happy to be persuaded by the numbers.

We will measure it and get back to you. Would be good with some numbers.

Sounds good, thanks! :)

-Toke

#17 lore raw
Quoted 7 lines
Quoted 2 lines

We will measure it and get back to you. Would be good with some numbers.

Sounds good, thanks! :)

-Toke

+1. Thanks a lot for doing this! :-)

#18 lore raw
In reply to Toke Høiland-Jørgensen
Quoted 4 lines

In addition, presumably when using this mode, the other XDP actions (XDP_PASS, XDP_REDIRECT to other targets) would stop working unless we add special handling for that in the kernel? We'll definitely need to handle that somehow...

I am not familiar with all the details here. Do you know a reason why these cases would stop working / why special handling would be needed? For example, if I have a UMEM that uses hugepages and XDP_PASS is returned, then the data is just copied into an SKB right? SKBs can also be created directly from hugepages AFAIK. So I don't understand what the issue would be. Can someone explain this concern?

#19 lore raw
Quoted 11 lines · Kal Cutter Conley
Quoted 4 lines

In addition, presumably when using this mode, the other XDP actions (XDP_PASS, XDP_REDIRECT to other targets) would stop working unless we add special handling for that in the kernel? We'll definitely need to handle that somehow...

I am not familiar with all the details here. Do you know a reason why these cases would stop working / why special handling would be needed? For example, if I have a UMEM that uses hugepages and XDP_PASS is returned, then the data is just copied into an SKB right? SKBs can also be created directly from hugepages AFAIK. So I don't understand what the issue would be. Can someone explain this concern?

Well, I was asking :) It may well be that the SKB path just works; did you test this? Pretty sure XDP_REDIRECT to another device won't, though?

-Toke

#20 lore raw
Quoted 12 lines
Quoted 11 lines
Quoted 4 lines

In addition, presumably when using this mode, the other XDP actions (XDP_PASS, XDP_REDIRECT to other targets) would stop working unless we add special handling for that in the kernel? We'll definitely need to handle that somehow...

I am not familiar with all the details here. Do you know a reason why these cases would stop working / why special handling would be needed? For example, if I have a UMEM that uses hugepages and XDP_PASS is returned, then the data is just copied into an SKB right? SKBs can also be created directly from hugepages AFAIK. So I don't understand what the issue would be. Can someone explain this concern?

Quoted 3 lines

Well, I was asking :) It may well be that the SKB path just works; did you test this? Pretty sure XDP_REDIRECT to another device won't, though?

I was also asking :-)

I tested that the SKB path is usable today with this patch. Specifically, sending and receiving large jumbo packets with AF_XDP and that a non-multi-buffer XDP program could access the whole packet. I have not specifically tested XDP_REDIRECT to another device or anything with ZC since that is not possible without driver support.

My feeling is, there wouldn't be non-trivial issues here since this patchset changes nothing except allowing the maximum chunk size to be larger. The driver either supports larger MTUs with XDP enabled or it doesn't. If it doesn't, the frames are dropped anyway. Also, chunk size mismatches between two XSKs (e.g. with XDP_REDIRECT) would be something supported or not supported irrespective of this patchset.

#21 lore raw

On Tue, Apr 18, 2023 at 01:12:00PM +0200, Kal Cutter Conley wrote:

Hi there,

Quoted 12 lines
Quoted 12 lines
Quoted 11 lines
Quoted 4 lines

In addition, presumably when using this mode, the other XDP actions (XDP_PASS, XDP_REDIRECT to other targets) would stop working unless we add special handling for that in the kernel? We'll definitely need to handle that somehow...

I am not familiar with all the details here. Do you know a reason why these cases would stop working / why special handling would be needed? For example, if I have a UMEM that uses hugepages and XDP_PASS is returned, then the data is just copied into an SKB right? SKBs can also be created directly from hugepages AFAIK. So I don't understand what the issue would be. Can someone explain this concern?

Quoted 2 lines
Quoted 2 lines

Well, I was asking :) It may well be that the SKB path just works; did you test this? Pretty sure XDP_REDIRECT to another device won't, though?

for XDP_PASS we have to allocate a new buffer and copy the contents from current xdp_buff that was backed by xsk_buff_pool and give the current one back to pool. I am not sure if __napi_alloc_skb() is always capable of handling len > PAGE_SIZE - i believe there might a particular combination of settings that allows it, but if not we should have a fallback path that would iterate over data and copy this to a certain (linear + frags) parts. This implies non-zero effort that is needed for jumbo frames ZC support.

I can certainly test this out and play with it - maybe this just works, I didn't check yet. Even if it does, then we need some kind of temporary mechanism that will forbid loading ZC jumbo frames due to what Toke brought up.

Quoted 10 lines
Quoted 1 line

I was also asking :-)

I tested that the SKB path is usable today with this patch. Specifically, sending and receiving large jumbo packets with AF_XDP and that a non-multi-buffer XDP program could access the whole packet. I have not specifically tested XDP_REDIRECT to another device or anything with ZC since that is not possible without driver support.

Quoted 6 lines

My feeling is, there wouldn't be non-trivial issues here since this patchset changes nothing except allowing the maximum chunk size to be larger. The driver either supports larger MTUs with XDP enabled or it doesn't. If it doesn't, the frames are dropped anyway. Also, chunk size mismatches between two XSKs (e.g. with XDP_REDIRECT) would be something supported or not supported irrespective of this patchset.

Here is the comparison between multi-buffer and jumbo frames that I did for ZC ice driver. Configured MTU was 8192 as this is the frame size for aligned mode when working with huge pages. I am presenting plain numbers over here from xdpsock.

Mbuf, packet size = 8192 - XDP_PACKET_HEADROOM
885,705pps - rxdrop frame_size=4096
806,307pps - l2fwd frame_size=4096
877,989pps - rxdrop frame_size=2048
773,331pps - l2fwd frame_size=2048

Jumbo, packet size = 8192 - XDP_PACKET_HEADROOM
893,530pps - rxdrop frame_size=8192
841,860pps - l2fwd frame_size=8192

Kal might say that multi-buffer numbers are imaginary as these patches were never shown to the public ;) but now that we have extensive test suite I am fixing some last issues that stand out, so we are asking for some more patience over here... overall i was expecting that they will be much worse when compared to jumbo frames, but then again i believe this implementation is not ideal and can be improved. Nevertheless, jumbo frames support has its value.

#22 lore raw
Quoted 27 lines · Maciej Fijalkowski

On Tue, Apr 18, 2023 at 01:12:00PM +0200, Kal Cutter Conley wrote:

Hi there,

Quoted 12 lines
Quoted 12 lines
Quoted 11 lines
Quoted 4 lines

In addition, presumably when using this mode, the other XDP actions (XDP_PASS, XDP_REDIRECT to other targets) would stop working unless we add special handling for that in the kernel? We'll definitely need to handle that somehow...

I am not familiar with all the details here. Do you know a reason why these cases would stop working / why special handling would be needed? For example, if I have a UMEM that uses hugepages and XDP_PASS is returned, then the data is just copied into an SKB right? SKBs can also be created directly from hugepages AFAIK. So I don't understand what the issue would be. Can someone explain this concern?

Quoted 2 lines
Quoted 2 lines

Well, I was asking :) It may well be that the SKB path just works; did you test this? Pretty sure XDP_REDIRECT to another device won't, though?

for XDP_PASS we have to allocate a new buffer and copy the contents from current xdp_buff that was backed by xsk_buff_pool and give the current one back to pool. I am not sure if __napi_alloc_skb() is always capable of handling len > PAGE_SIZE - i believe there might a particular combination of settings that allows it, but if not we should have a fallback path that would iterate over data and copy this to a certain (linear + frags) parts. This implies non-zero effort that is needed for jumbo frames ZC support.

Quoted 4 lines

I can certainly test this out and play with it - maybe this just works, I didn't check yet. Even if it does, then we need some kind of temporary mechanism that will forbid loading ZC jumbo frames due to what Toke brought up.

Yeah, this was exactly the kind of thing I was worried about (same for XDP_REDIRECT). Thanks for fleshing it out a bit :)

Quoted 32 lines
Quoted 10 lines
Quoted 1 line

I was also asking :-)

I tested that the SKB path is usable today with this patch. Specifically, sending and receiving large jumbo packets with AF_XDP and that a non-multi-buffer XDP program could access the whole packet. I have not specifically tested XDP_REDIRECT to another device or anything with ZC since that is not possible without driver support.

Quoted 6 lines

My feeling is, there wouldn't be non-trivial issues here since this patchset changes nothing except allowing the maximum chunk size to be larger. The driver either supports larger MTUs with XDP enabled or it doesn't. If it doesn't, the frames are dropped anyway. Also, chunk size mismatches between two XSKs (e.g. with XDP_REDIRECT) would be something supported or not supported irrespective of this patchset.

Here is the comparison between multi-buffer and jumbo frames that I did for ZC ice driver. Configured MTU was 8192 as this is the frame size for aligned mode when working with huge pages. I am presenting plain numbers over here from xdpsock.

Mbuf, packet size = 8192 - XDP_PACKET_HEADROOM
885,705pps - rxdrop frame_size=4096
806,307pps - l2fwd frame_size=4096
877,989pps - rxdrop frame_size=2048
773,331pps - l2fwd frame_size=2048

Jumbo, packet size = 8192 - XDP_PACKET_HEADROOM
893,530pps - rxdrop frame_size=8192
841,860pps - l2fwd frame_size=8192

Quoted 7 lines

Kal might say that multi-buffer numbers are imaginary as these patches were never shown to the public ;) but now that we have extensive test suite I am fixing some last issues that stand out, so we are asking for some more patience over here... overall i was expecting that they will be much worse when compared to jumbo frames, but then again i believe this implementation is not ideal and can be improved. Nevertheless, jumbo frames support has its value.

Thank you for doing these! Okay, so that's between 1-4% improvement (vs the 4k frags). I dunno, I wouldn't consider that a slam dunk; would depend on the additional complexity if it is worth it to do both, IMO...

-Toke

#23 lore raw
Quoted 78 lines · Toke Høiland-Jørgensen
Quoted 27 lines · Maciej Fijalkowski

On Tue, Apr 18, 2023 at 01:12:00PM +0200, Kal Cutter Conley wrote:

Hi there,

Quoted 12 lines
Quoted 12 lines
Quoted 11 lines
Quoted 4 lines

In addition, presumably when using this mode, the other XDP actions (XDP_PASS, XDP_REDIRECT to other targets) would stop working unless we add special handling for that in the kernel? We'll definitely need to handle that somehow...

I am not familiar with all the details here. Do you know a reason why these cases would stop working / why special handling would be needed? For example, if I have a UMEM that uses hugepages and XDP_PASS is returned, then the data is just copied into an SKB right? SKBs can also be created directly from hugepages AFAIK. So I don't understand what the issue would be. Can someone explain this concern?

Quoted 2 lines
Quoted 2 lines

Well, I was asking :) It may well be that the SKB path just works; did you test this? Pretty sure XDP_REDIRECT to another device won't, though?

for XDP_PASS we have to allocate a new buffer and copy the contents from current xdp_buff that was backed by xsk_buff_pool and give the current one back to pool. I am not sure if __napi_alloc_skb() is always capable of handling len > PAGE_SIZE - i believe there might a particular combination of settings that allows it, but if not we should have a fallback path that would iterate over data and copy this to a certain (linear + frags) parts. This implies non-zero effort that is needed for jumbo frames ZC support.

Quoted 4 lines

I can certainly test this out and play with it - maybe this just works, I didn't check yet. Even if it does, then we need some kind of temporary mechanism that will forbid loading ZC jumbo frames due to what Toke brought up.

Yeah, this was exactly the kind of thing I was worried about (same for XDP_REDIRECT). Thanks for fleshing it out a bit :)

Quoted 32 lines
Quoted 10 lines
Quoted 1 line

I was also asking :-)

I tested that the SKB path is usable today with this patch. Specifically, sending and receiving large jumbo packets with AF_XDP and that a non-multi-buffer XDP program could access the whole packet. I have not specifically tested XDP_REDIRECT to another device or anything with ZC since that is not possible without driver support.

Quoted 6 lines

My feeling is, there wouldn't be non-trivial issues here since this patchset changes nothing except allowing the maximum chunk size to be larger. The driver either supports larger MTUs with XDP enabled or it doesn't. If it doesn't, the frames are dropped anyway. Also, chunk size mismatches between two XSKs (e.g. with XDP_REDIRECT) would be something supported or not supported irrespective of this patchset.

Here is the comparison between multi-buffer and jumbo frames that I did for ZC ice driver. Configured MTU was 8192 as this is the frame size for aligned mode when working with huge pages. I am presenting plain numbers over here from xdpsock.

Mbuf, packet size = 8192 - XDP_PACKET_HEADROOM
885,705pps - rxdrop frame_size=4096
806,307pps - l2fwd frame_size=4096
877,989pps - rxdrop frame_size=2048
773,331pps - l2fwd frame_size=2048

Jumbo, packet size = 8192 - XDP_PACKET_HEADROOM
893,530pps - rxdrop frame_size=8192
841,860pps - l2fwd frame_size=8192

Quoted 7 lines

Kal might say that multi-buffer numbers are imaginary as these patches were never shown to the public ;) but now that we have extensive test suite I am fixing some last issues that stand out, so we are asking for some more patience over here... overall i was expecting that they will be much worse when compared to jumbo frames, but then again i believe this implementation is not ideal and can be improved. Nevertheless, jumbo frames support has its value.

Quoted 3 lines

Thank you for doing these! Okay, so that's between 1-4% improvement (vs the 4k frags). I dunno, I wouldn't consider that a slam dunk; would depend on the additional complexity if it is worth it to do both, IMO...

If we are using 4K frags, the worst case is that we have to use 3 frags to cover a 9K packet. Interpolating between the results above would mean somewhere in the 1 - 6% range of improvement with jumbo frames. Something that is not covered by these tests at all is the overhead of an abstraction layer for dealing with multiple buffers. I believe many applications would choose to have one to hide the fact that there are multiple buffers. So I think these improvement numbers are on the lower side.

But agree that we should factor in the complexity of covering the cases you have brought up and see if it is worth it.

Quoted 2 lines

-Toke

#24 lore raw
In reply to Maciej Fijalkowski
Quoted 20 lines · Maciej Fijalkowski

On Tue, Apr 18, 2023 at 01:12:00PM +0200, Kal Cutter Conley wrote:

Hi there,

Quoted 12 lines
Quoted 12 lines
Quoted 11 lines
Quoted 4 lines

In addition, presumably when using this mode, the other XDP actions (XDP_PASS, XDP_REDIRECT to other targets) would stop working unless we add special handling for that in the kernel? We'll definitely need to handle that somehow...

I am not familiar with all the details here. Do you know a reason why these cases would stop working / why special handling would be needed? For example, if I have a UMEM that uses hugepages and XDP_PASS is returned, then the data is just copied into an SKB right? SKBs can also be created directly from hugepages AFAIK. So I don't understand what the issue would be. Can someone explain this concern?

Quoted 2 lines
Quoted 2 lines

Well, I was asking :) It may well be that the SKB path just works; did you test this? Pretty sure XDP_REDIRECT to another device won't, though?

Quoted 7 lines

for XDP_PASS we have to allocate a new buffer and copy the contents from current xdp_buff that was backed by xsk_buff_pool and give the current one back to pool. I am not sure if __napi_alloc_skb() is always capable of handling len > PAGE_SIZE - i believe there might a particular combination of settings that allows it, but if not we should have a fallback path that would iterate over data and copy this to a certain (linear + frags) parts. This implies non-zero effort that is needed for jumbo frames ZC support.

Thinking aloud, could not our multi-buffer work help with this? Sounds quite similar to operations that we have to do in that patch set. And if so, would it not be prudent to get the multi-buffer support in there first, then implement these things on top of that? What do you think?

Quoted 44 lines

I can certainly test this out and play with it - maybe this just works, I didn't check yet. Even if it does, then we need some kind of temporary mechanism that will forbid loading ZC jumbo frames due to what Toke brought up.

Quoted 10 lines
Quoted 1 line

I was also asking :-)

I tested that the SKB path is usable today with this patch. Specifically, sending and receiving large jumbo packets with AF_XDP and that a non-multi-buffer XDP program could access the whole packet. I have not specifically tested XDP_REDIRECT to another device or anything with ZC since that is not possible without driver support.

Quoted 6 lines

My feeling is, there wouldn't be non-trivial issues here since this patchset changes nothing except allowing the maximum chunk size to be larger. The driver either supports larger MTUs with XDP enabled or it doesn't. If it doesn't, the frames are dropped anyway. Also, chunk size mismatches between two XSKs (e.g. with XDP_REDIRECT) would be something supported or not supported irrespective of this patchset.

Here is the comparison between multi-buffer and jumbo frames that I did for ZC ice driver. Configured MTU was 8192 as this is the frame size for aligned mode when working with huge pages. I am presenting plain numbers over here from xdpsock.

Mbuf, packet size = 8192 - XDP_PACKET_HEADROOM
885,705pps - rxdrop frame_size=4096
806,307pps - l2fwd frame_size=4096
877,989pps - rxdrop frame_size=2048
773,331pps - l2fwd frame_size=2048

Jumbo, packet size = 8192 - XDP_PACKET_HEADROOM
893,530pps - rxdrop frame_size=8192
841,860pps - l2fwd frame_size=8192

Kal might say that multi-buffer numbers are imaginary as these patches were never shown to the public ;) but now that we have extensive test suite I am fixing some last issues that stand out, so we are asking for some more patience over here... overall i was expecting that they will be much worse when compared to jumbo frames, but then again i believe this implementation is not ideal and can be improved. Nevertheless, jumbo frames support has its value.

#25 lore raw
In reply to Maciej Fijalkowski
Quoted 11 lines

Here is the comparison between multi-buffer and jumbo frames that I did for ZC ice driver. Configured MTU was 8192 as this is the frame size for aligned mode when working with huge pages. I am presenting plain numbers over here from xdpsock.

Mbuf, packet size = 8192 - XDP_PACKET_HEADROOM
885,705pps - rxdrop frame_size=4096
806,307pps - l2fwd frame_size=4096
877,989pps - rxdrop frame_size=2048
773,331pps - l2fwd frame_size=2048

Quoted 3 lines

Jumbo, packet size = 8192 - XDP_PACKET_HEADROOM
893,530pps - rxdrop frame_size=8192
841,860pps - l2fwd frame_size=8192

Thanks so much for sharing these initial results! Do you have similar measurements for ~9000 byte packets in unaligned mode? We typically receive packets larger than 8192 bytes.

Quoted 8 lines

Kal might say that multi-buffer numbers are imaginary as these patches were never shown to the public ;) but now that we have extensive test suite I am fixing some last issues that stand out, so we are asking for some more patience over here... overall i was expecting that they will be much worse when compared to jumbo frames, but then again i believe this implementation is not ideal and can be improved. Nevertheless, jumbo frames support has its value.

You made me chuckle ;-) Any measurements people can provide are helpful, even if they must be taken with a grain of salt. ;-). How much of your test suite can be upstreamed in the future? My assumption was the difference should be measurable, at least you have confirmed that. :-)

author
#26 lore raw

PATCH v3 2/3 selftests: xsk: Use hugepages when umem->frame_size > PAGE_SIZE

In reply to Kal Conley

HugeTLB UMEMs now support chunk_size > PAGE_SIZE. Set MAP_HUGETLB when frame_size > PAGE_SIZE for future tests.

---
 tools/testing/selftests/bpf/xskxceiver.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
tools/testing/selftests/bpf/xskxceiver.c +1 -1
index 5a9691e942de..7eccf57a0ccc 100644
@@ -1289,7 +1289,7 @@ static void thread_common_ops(struct test_spec *test, struct ifobject *ifobject)
 	void *bufs;
 	int ret;
 
-	if (ifobject->umem->unaligned_mode)
+	if (ifobject->umem->frame_size > sysconf(_SC_PAGESIZE) || ifobject->umem->unaligned_mode)
 		mmap_flags |= MAP_HUGETLB;
 
 	if (ifobject->shared_umem)
Signature · 1 line
2.39.2
author
#27 lore raw

PATCH v3 3/3 selftests: xsk: Add tests for 8K and 9K frame sizes

In reply to Kal Conley

Add tests:
- RUN_TO_COMPLETION_8K_FRAME_SIZE: frame_size=8192 (aligned)
- UNALIGNED_9K_FRAME_SIZE: frame_size=9000 (unaligned)

---
 tools/testing/selftests/bpf/xskxceiver.c | 25 ++++++++++++++++++++++++
 tools/testing/selftests/bpf/xskxceiver.h |  2 ++
 2 files changed, 27 insertions(+)
tools/testing/selftests/bpf/xskxceiver.c +25 -0
index 7eccf57a0ccc..86797de7fc50 100644
@@ -1841,6 +1841,17 @@ static void run_pkt_test(struct test_spec *test, enum test_mode mode, enum test_
 		pkt_stream_replace(test, DEFAULT_PKT_CNT, PKT_SIZE);
 		testapp_validate_traffic(test);
 		break;
+	case TEST_TYPE_RUN_TO_COMPLETION_8K_FRAME:
+		if (!hugepages_present(test->ifobj_tx)) {
+			ksft_test_result_skip("No 2M huge pages present.\n");
+			return;
+		}
+		test_spec_set_name(test, "RUN_TO_COMPLETION_8K_FRAME_SIZE");
+		test->ifobj_tx->umem->frame_size = 8192;
+		test->ifobj_rx->umem->frame_size = 8192;
+		pkt_stream_replace(test, DEFAULT_PKT_CNT, PKT_SIZE);
+		testapp_validate_traffic(test);
+		break;
 	case TEST_TYPE_RX_POLL:
 		test->ifobj_rx->use_poll = true;
 		test_spec_set_name(test, "POLL_RX");
@@ -1904,6 +1915,20 @@ static void run_pkt_test(struct test_spec *test, enum test_mode mode, enum test_
 		if (!testapp_unaligned(test))
 			return;
 		break;
+	case TEST_TYPE_UNALIGNED_9K_FRAME:
+		if (!hugepages_present(test->ifobj_tx)) {
+			ksft_test_result_skip("No 2M huge pages present.\n");
+			return;
+		}
+		test_spec_set_name(test, "UNALIGNED_9K_FRAME_SIZE");
+		test->ifobj_tx->umem->frame_size = 9000;
+		test->ifobj_rx->umem->frame_size = 9000;
+		test->ifobj_tx->umem->unaligned_mode = true;
+		test->ifobj_rx->umem->unaligned_mode = true;
+		pkt_stream_replace(test, DEFAULT_PKT_CNT, PKT_SIZE);
+		test->ifobj_rx->pkt_stream->use_addr_for_fill = true;
+		testapp_validate_traffic(test);
+		break;
 	case TEST_TYPE_HEADROOM:
 		testapp_headroom(test);
 		break;
tools/testing/selftests/bpf/xskxceiver.h +2 -0
index 919327807a4e..7f52f737f5e9 100644
@@ -69,12 +69,14 @@ enum test_mode {
 enum test_type {
 	TEST_TYPE_RUN_TO_COMPLETION,
 	TEST_TYPE_RUN_TO_COMPLETION_2K_FRAME,
+	TEST_TYPE_RUN_TO_COMPLETION_8K_FRAME,
 	TEST_TYPE_RUN_TO_COMPLETION_SINGLE_PKT,
 	TEST_TYPE_RX_POLL,
 	TEST_TYPE_TX_POLL,
 	TEST_TYPE_POLL_RXQ_TMOUT,
 	TEST_TYPE_POLL_TXQ_TMOUT,
 	TEST_TYPE_UNALIGNED,
+	TEST_TYPE_UNALIGNED_9K_FRAME,
 	TEST_TYPE_ALIGNED_INV_DESC,
 	TEST_TYPE_ALIGNED_INV_DESC_2K_FRAME,
 	TEST_TYPE_UNALIGNED_INV_DESC,
Signature · 1 line
2.39.2