xsk: Support UMEM chunk_size > PAGE_SIZE
Subsystems (7)
v3 cover xsk: Support UMEM chunk_size > PAGE_SIZE
The main purpose of this patchset is to add AF_XDP support for UMEM chunk sizes > PAGE_SIZE. This is enabled for UMEMs backed by HugeTLB pages.
Changes since v2:
* Related fixes/improvements included with v2 have been removed. These changes have all been resubmitted as standalone patchsets.
* Minimize uses of #ifdef CONFIG_HUGETLB_PAGE.
* Improve AF_XDP documentation.
* Update benchmark table in commit message.
Changes since v1:
* Add many fixes/improvements to the XSK selftests.
* Add check for unaligned descriptors that overrun UMEM.
* Fix compile errors when CONFIG_HUGETLB_PAGE is not set.
* Fix incorrect use of _Static_assert.
* Update AF_XDP documentation.
* Rename unaligned 9K frame size test.
* Make xp_check_dma_contiguity less conservative.
* Add more information to benchmark table.
Thanks to Magnus Karlsson for all his support!
Happy easter!
Kal Conley (3):
xsk: Support UMEM chunk_size > PAGE_SIZE
selftests: xsk: Use hugepages when umem->frame_size > PAGE_SIZE
selftests: xsk: Add tests for 8K and 9K frame sizes
Documentation/networking/af_xdp.rst | 36 ++++++++++------- include/net/xdp_sock.h | 1 + include/net/xdp_sock_drv.h | 12 ++++++ include/net/xsk_buff_pool.h | 3 +- net/xdp/xdp_umem.c | 51 ++++++++++++++++++++---- net/xdp/xsk_buff_pool.c | 28 ++++++++----- tools/testing/selftests/bpf/xskxceiver.c | 27 ++++++++++++- tools/testing/selftests/bpf/xskxceiver.h | 2 + 8 files changed, 127 insertions(+), 33 deletions(-)
Signature · 1 line
2.39.2
PATCH v3 1/3 xsk: Support UMEM chunk_size > PAGE_SIZE
Add core AF_XDP support for chunk sizes larger than PAGE_SIZE. This enables sending/receiving jumbo ethernet frames up to the theoretical maxiumum of 64 KiB. For chunk sizes > PAGE_SIZE, the UMEM is required to consist of HugeTLB VMAs (and be hugepage aligned). Initially, only SKB mode is usable pending future driver work.
For consistency, check for HugeTLB pages during UMEM registration. This implies that hugepages are required for XDP_COPY mode despite DMA not being used. This restriction is desirable since it ensures user software can take advantage of future driver support.
Even in HugeTLB mode, continue to do page accounting using order-0 (4 KiB) pages. This minimizes the size of this change and reduces the risk of impacting other code. Taking full advantage of hugepages for accounting should improve XDP performance in the general case.
No significant change in RX/TX performance was observed with this patch. A few data points are reproduced below:
Machine : Dell PowerEdge R940
CPU : Intel(R) Xeon(R) Platinum 8168 CPU @ 2.70GHz
NIC : MT27700 Family [ConnectX-4]
+-----+------+------+-------+--------+--------+--------+ | | | | chunk | packet | rxdrop | rxdrop | | | mode | mtu | size | size | (Mpps) | (Gbps) | +-----+------+------+-------+--------+--------+--------+ | old | -z | 3498 | 4000 | 320 | 16.0 | 40.9 | | new | -z | 3498 | 4000 | 320 | 16.0 | 40.9 | +-----+------+------+-------+--------+--------+--------+ | old | -z | 3498 | 4096 | 320 | 16.4 | 42.1 | | new | -z | 3498 | 4096 | 320 | 16.5 | 42.2 | +-----+------+------+-------+--------+--------+--------+ | new | -c | 3498 | 10240 | 320 | 6.1 | 15.6 | +-----+------+------+-------+--------+--------+--------+ | new | -S | 9000 | 10240 | 9000 | 0.37 | 26.9 | +-----+------+------+-------+--------+--------+--------+
--- Documentation/networking/af_xdp.rst | 36 ++++++++++++-------- include/net/xdp_sock.h | 1 + include/net/xdp_sock_drv.h | 12 +++++++ include/net/xsk_buff_pool.h | 3 +- net/xdp/xdp_umem.c | 51 ++++++++++++++++++++++++----- net/xdp/xsk_buff_pool.c | 28 +++++++++++----- 6 files changed, 99 insertions(+), 32 deletions(-)
Documentation/networking/af_xdp.rst +22 -14
@@ -105,12 +105,13 @@ with AF_XDP". It can be found at https://lwn.net/Articles/750845/. UMEM ---- -UMEM is a region of virtual contiguous memory, divided into -equal-sized frames. An UMEM is associated to a netdev and a specific -queue id of that netdev. It is created and configured (chunk size, -headroom, start address and size) by using the XDP_UMEM_REG setsockopt -system call. A UMEM is bound to a netdev and queue id, via the bind() -system call. +UMEM is a region of virtual contiguous memory divided into equal-sized +frames. This is the area that contains all the buffers that packets can +reside in. A UMEM is associated with a netdev and a specific queue id of +that netdev. It is created and configured (start address, size, +chunk size, and headroom) by using the XDP_UMEM_REG setsockopt system +call. A UMEM is bound to a netdev and queue id via the bind() system +call. An AF_XDP is socket linked to a single UMEM, but one UMEM can have multiple AF_XDP sockets. To share an UMEM created via one socket A,
@@ -418,14 +419,21 @@ negatively impact performance. XDP_UMEM_REG setsockopt ----------------------- -This setsockopt registers a UMEM to a socket. This is the area that -contain all the buffers that packet can reside in. The call takes a -pointer to the beginning of this area and the size of it. Moreover, it -also has parameter called chunk_size that is the size that the UMEM is -divided into. It can only be 2K or 4K at the moment. If you have an -UMEM area that is 128K and a chunk size of 2K, this means that you -will be able to hold a maximum of 128K / 2K = 64 packets in your UMEM -area and that your largest packet size can be 2K. +This setsockopt registers a UMEM to a socket. The call takes a pointer +to the beginning of this area and the size of it. Moreover, there is a +parameter called chunk_size that is the size that the UMEM is divided +into. The chunk size limits the maximum packet size that can be sent or +received. For example, if you have a UMEM area that is 128K and a chunk +size of 2K, then you will be able to hold a maximum of 128K / 2K = 64 +packets in your UMEM. In this case, the maximum packet size will be 2K. + +Valid chunk sizes range from 2K to 64K. However, in aligned mode, the +chunk size must also be a power of two. Additionally, the chunk size +must not exceed the size of a page (usually 4K). This limitation is +relaxed for UMEM areas allocated with HugeTLB pages, in which case +chunk sizes up to 64K are allowed. Note, this only works with hugepages +allocated from the kernel's persistent pool. Using Transparent Huge +Pages (THP) has no effect on the maximum chunk size. There is also an option to set the headroom of each single buffer in the UMEM. If you set this to N bytes, it means that the packet will
include/net/xdp_sock.h +1 -0
@@ -30,6 +30,7 @@ struct xdp_umem {u8 flags;bool zc;struct page **pgs; +u32 page_size;int id;struct list_head xsk_dma_list;struct work_struct work;
include/net/xdp_sock_drv.h +12 -0
@@ -12,6 +12,18 @@#define XDP_UMEM_MIN_CHUNK_SHIFT 11#define XDP_UMEM_MIN_CHUNK_SIZE (1 << XDP_UMEM_MIN_CHUNK_SHIFT) +static_assert (XDP_UMEM_MIN_CHUNK_SIZE <=PAGE_SIZE ); + +/* Allow chunk sizes up to the maximum size of an ethernet frame (64 KiB). +* Larger chunks are not guaranteed to fit in a single SKB. +*/ +#ifdef CONFIG_HUGETLB_PAGE +#define XDP_UMEM_MAX_CHUNK_SHIFT min(16, HPAGE_SHIFT) +#else +#define XDP_UMEM_MAX_CHUNK_SHIFT min(16, PAGE_SHIFT) +#endif +#define XDP_UMEM_MAX_CHUNK_SIZE (1 << XDP_UMEM_MAX_CHUNK_SHIFT) +#ifdef CONFIG_XDP_SOCKETS void xsk_tx_completed (struct xsk_buff_pool *pool,u32 nb_entries);
include/net/xsk_buff_pool.h +2 -1
@@ -75,6 +75,7 @@ struct xsk_buff_pool {u32 chunk_size;u32 chunk_shift;u32 frame_len; +u32 page_size;u8 cached_need_wakeup;bool uses_need_wakeup;bool dma_need_sync;
@@ -175,7 +176,7 @@ static inline void xp_dma_sync_for_device(struct xsk_buff_pool *pool,static inline bool xp_desc_crosses_non_contig_pg (struct xsk_buff_pool *pool,u64 addr,u32 len) { -bool cross_pg = (addr & (PAGE_SIZE -1 )) + len >PAGE_SIZE ; +bool cross_pg = (addr & (pool->page_size -1 )) + len > pool->page_size;if (likely (!cross_pg))return false;
net/xdp/xdp_umem.c +43 -8
78 lines not on this page · raw message
net/xdp/xsk_buff_pool.c +19 -9
@@ -80,9 +80,10 @@ struct xsk_buff_pool *xp_create_and_assign_umem(struct xdp_sock *xs, pool->headroom = umem->headroom; pool->chunk_size = umem->chunk_size; pool->chunk_shift =ffs (umem->chunk_size) -1 ; - pool->unaligned = unaligned; pool->frame_len = umem->chunk_size - umem->headroom -XDP_PACKET_HEADROOM ; + pool->page_size = umem->page_size; + pool->unaligned = unaligned; pool->umem = umem; pool->addrs = umem->addrs;INIT_LIST_HEAD (&pool->free_list);
@@ -369,16 +370,25 @@ void xp_dma_unmap(struct xsk_buff_pool *pool, unsigned long attrs) }EXPORT_SYMBOL (xp_dma_unmap); -static void xp_check_dma_contiguity (struct xsk_dma_map *dma_map) +/* HugeTLB pools consider contiguity at hugepage granularity only. Hence, all +* order-0 pages within a hugepage have the same contiguity value. +*/ +static void xp_check_dma_contiguity (struct xsk_dma_map *dma_map,u32 page_size) { -u32 i; +u32 stride = page_size >>PAGE_SHIFT ;/* in order-0 pages */ +u32 i, j; -for (i =0 ; i < dma_map->dma_pages_cnt -1 ; i++) { -if (dma_map->dma_pages[i] +PAGE_SIZE == dma_map->dma_pages[i +1 ]) - dma_map->dma_pages[i] |=XSK_NEXT_PG_CONTIG_MASK ; -else - dma_map->dma_pages[i] &= ~XSK_NEXT_PG_CONTIG_MASK ; +for (i =0 ; i + stride < dma_map->dma_pages_cnt;) { +if (dma_map->dma_pages[i] + page_size == dma_map->dma_pages[i + stride]) { +for (j =0 ; j < stride; i++, j++) + dma_map->dma_pages[i] |=XSK_NEXT_PG_CONTIG_MASK ; + }else { +for (j =0 ; j < stride; i++, j++) + dma_map->dma_pages[i] &= ~XSK_NEXT_PG_CONTIG_MASK ; + } } +for (; i < dma_map->dma_pages_cnt; i++) + dma_map->dma_pages[i] &= ~XSK_NEXT_PG_CONTIG_MASK ; }static int xp_init_dma_info (struct xsk_buff_pool *pool,struct xsk_dma_map *dma_map)
@@ -441,7 +451,7 @@ int xp_dma_map(struct xsk_buff_pool *pool, struct device *dev, }if (pool->unaligned) -xp_check_dma_contiguity (dma_map); +xp_check_dma_contiguity (dma_map, pool->page_size); err =xp_init_dma_info (pool, dma_map);if (err) {
Signature · 1 line
2.39.2
Quoted 5 lines · Kal Conley
Add core
AF_XDPsupport for chunk sizes larger thanPAGE_SIZE. This enables sending/receiving jumbo ethernet frames up to the theoretical maxiumum of 64 KiB. For chunk sizes >PAGE_SIZE, the UMEM is required to consist of HugeTLB VMAs (and be hugepage aligned). Initially, only SKB mode is usable pending future driver work.
Hmm, interesting. So how does this interact with XDP multibuf?
-Toke
Maciej Fijalkowski
maintainerQuoted 9 lines · Toke Høiland-Jørgensen
Quoted 5 lines · Kal Conley
Add core
AF_XDPsupport for chunk sizes larger thanPAGE_SIZE. This enables sending/receiving jumbo ethernet frames up to the theoretical maxiumum of 64 KiB. For chunk sizes >PAGE_SIZE, the UMEM is required to consist of HugeTLB VMAs (and be hugepage aligned). Initially, only SKB mode is usable pending future driver work.Hmm, interesting. So how does this interact with XDP multibuf?
To me it currently does not interact with mbuf in any way as it is enabled only for skb mode which linearizes the skb from what i see.
I'd like to hear more about Kal's use case - Kal do you use AF_XDP in SKB mode on your side?
Quoted 3 lines
-Toke
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
Add core
AF_XDPsupport for chunk sizes larger thanPAGE_SIZE. This enables sending/receiving jumbo ethernet frames up to the theoretical maxiumum of 64 KiB. For chunk sizes >PAGE_SIZE, the UMEM is required to consist of HugeTLB VMAs (and be hugepage aligned). Initially, only SKB mode is usable pending future driver work.
Quoted 7 lines
Quoted 1 line
Hmm, interesting. So how does this interact with XDP multibuf?
To me it currently does not interact with mbuf in any way as it is enabled only for skb mode which linearizes the skb from what i see.
I'd like to hear more about Kal's use case - Kal do you use
AF_XDPin SKB mode on your side?
Our use-case is to receive jumbo Ethernet frames up to 9000 bytes with AF_XDP in zero-copy mode. This patchset is a step in this direction. At the very least, it lets you test out the feature in SKB mode pending future driver support. Currently, XDP multi-buffer does not support AF_XDP at all. It could support it in theory, but I think it would need some UAPI design work and a bit of implementation work.
Also, I think that the approach taken in this patchset has some advantages over XDP multi-buffer:
(1) It should be possible to achieve higher performance
(a) because the packet data is kept together
(b) because you need to acquire and validate less descriptors
and touch the queue pointers less often.
(2) It is a nicer user-space API.
(a) Since the packet data is all available in one linear
buffer. This may even be a requirement to avoid an extra copy if the data must be handed off contiguously to other code.
The disadvantage of this patchset is requiring the user to allocate HugeTLB pages which is an extra complication.
I am not sure if this patchset would need to interact with XDP multi-buffer at all directly. Does anyone have anything to add here?
What other intermediate steps are needed to get to ZC? I think drivers would already be able to support this now?
Kal
Quoted 32 lines · Kal Cutter Conley
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
Add core
AF_XDPsupport for chunk sizes larger thanPAGE_SIZE. This enables sending/receiving jumbo ethernet frames up to the theoretical maxiumum of 64 KiB. For chunk sizes >PAGE_SIZE, the UMEM is required to consist of HugeTLB VMAs (and be hugepage aligned). Initially, only SKB mode is usable pending future driver work.Quoted 7 lines
Quoted 1 line
Hmm, interesting. So how does this interact with XDP multibuf?
To me it currently does not interact with mbuf in any way as it is enabled only for skb mode which linearizes the skb from what i see.
I'd like to hear more about Kal's use case - Kal do you use
AF_XDPin SKB mode on your side?Our use-case is to receive jumbo Ethernet frames up to 9000 bytes with
AF_XDPin zero-copy mode. This patchset is a step in this direction. At the very least, it lets you test out the feature in SKB mode pending future driver support. Currently, XDP multi-buffer does not supportAF_XDPat all. It could support it in theory, but I think it would need some UAPI design work and a bit of implementation work.Also, I think that the approach taken in this patchset has some advantages over XDP multi-buffer:
(1) It should be possible to achieve higher performance
(a) because the packet data is kept together
(b) because you need to acquire and validate less descriptors
and touch the queue pointers less often.
(2) It is a nicer user-space API.
(a) Since the packet data is all available in one linear
buffer. This may even be a requirement to avoid an extra copy if the data must be handed off contiguously to other code.
Quoted 5 lines
The disadvantage of this patchset is requiring the user to allocate HugeTLB pages which is an extra complication.
I am not sure if this patchset would need to interact with XDP multi-buffer at all directly. Does anyone have anything to add here?
Well, I'm mostly concerned with having two different operation and configuration modes for the same thing. We'll probably need to support multibuf for AF_XDP anyway for the non-ZC path, which means we'll need to create a UAPI for that in any case. And having two APIs is just going to be more complexity to handle at both the documentation and maintenance level.
It *might* be worth it to do this if the performance benefit is really compelling, but, well, you'd need to implement both and compare directly to know that for sure :)
-Toke
Quoted 41 lines · Toke Høiland-Jørgensen
Quoted 32 lines · Kal Cutter Conley
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
Add core
AF_XDPsupport for chunk sizes larger thanPAGE_SIZE. This enables sending/receiving jumbo ethernet frames up to the theoretical maxiumum of 64 KiB. For chunk sizes >PAGE_SIZE, the UMEM is required to consist of HugeTLB VMAs (and be hugepage aligned). Initially, only SKB mode is usable pending future driver work.Quoted 7 lines
Quoted 1 line
Hmm, interesting. So how does this interact with XDP multibuf?
To me it currently does not interact with mbuf in any way as it is enabled only for skb mode which linearizes the skb from what i see.
I'd like to hear more about Kal's use case - Kal do you use
AF_XDPin SKB mode on your side?Our use-case is to receive jumbo Ethernet frames up to 9000 bytes with
AF_XDPin zero-copy mode. This patchset is a step in this direction. At the very least, it lets you test out the feature in SKB mode pending future driver support. Currently, XDP multi-buffer does not supportAF_XDPat all. It could support it in theory, but I think it would need some UAPI design work and a bit of implementation work.Also, I think that the approach taken in this patchset has some advantages over XDP multi-buffer:
(1) It should be possible to achieve higher performance
(a) because the packet data is kept together
(b) because you need to acquire and validate less descriptors
and touch the queue pointers less often.
(2) It is a nicer user-space API.
(a) Since the packet data is all available in one linear
buffer. This may even be a requirement to avoid an extra copy if the data must be handed off contiguously to other code.Quoted 5 lines
The disadvantage of this patchset is requiring the user to allocate HugeTLB pages which is an extra complication.
I am not sure if this patchset would need to interact with XDP multi-buffer at all directly. Does anyone have anything to add here?
Quoted 6 lines
Well, I'm mostly concerned with having two different operation and configuration modes for the same thing. We'll probably need to support multibuf for
AF_XDPanyway for the non-ZC path, which means we'll need to create a UAPI for that in any case. And having two APIs is just going to be more complexity to handle at both the documentation and maintenance level.
One does not replace the other. We need them both, unfortunately. Multi-buff is great for e.g., stitching together different headers with the same data. Point to different buffers for the header in each packet but the same piece of data in all of them. This will never be solved with Kal's approach. We just need multi-buffer support for this. BTW, we are close to posting multi-buff support for AF_XDP. Just hang in there a little while longer while the last glitches are fixed. We have to stage it in two patch sets as it will be too long otherwise. First one will only contain improvements to the xsk selftests framework so that multi-buffer tests can be supported. The second one will be the core code and the actual multi-buffer tests. As for what Kal's patches are good for, please see below.
Quoted 3 lines
It *might* be worth it to do this if the performance benefit is really compelling, but, well, you'd need to implement both and compare directly to know that for sure :)
The performance benefit is compelling. As I wrote in a mail to a post by Kal, there are users out there that state that this feature (for zero-copy mode nota bene) is a must for them to be able to use AF_XDP instead of DPDK style user-mode drivers. They have really tough latency requirements.
Quoted 2 lines
-Toke
Quoted 49 lines · Magnus Karlsson
Quoted 41 lines · Toke Høiland-Jørgensen
Quoted 32 lines · Kal Cutter Conley
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
Add core
AF_XDPsupport for chunk sizes larger thanPAGE_SIZE. This enables sending/receiving jumbo ethernet frames up to the theoretical maxiumum of 64 KiB. For chunk sizes >PAGE_SIZE, the UMEM is required to consist of HugeTLB VMAs (and be hugepage aligned). Initially, only SKB mode is usable pending future driver work.Quoted 7 lines
Quoted 1 line
Hmm, interesting. So how does this interact with XDP multibuf?
To me it currently does not interact with mbuf in any way as it is enabled only for skb mode which linearizes the skb from what i see.
I'd like to hear more about Kal's use case - Kal do you use
AF_XDPin SKB mode on your side?Our use-case is to receive jumbo Ethernet frames up to 9000 bytes with
AF_XDPin zero-copy mode. This patchset is a step in this direction. At the very least, it lets you test out the feature in SKB mode pending future driver support. Currently, XDP multi-buffer does not supportAF_XDPat all. It could support it in theory, but I think it would need some UAPI design work and a bit of implementation work.Also, I think that the approach taken in this patchset has some advantages over XDP multi-buffer:
(1) It should be possible to achieve higher performance
(a) because the packet data is kept together
(b) because you need to acquire and validate less descriptors
and touch the queue pointers less often.
(2) It is a nicer user-space API.
(a) Since the packet data is all available in one linear
buffer. This may even be a requirement to avoid an extra copy if the data must be handed off contiguously to other code.Quoted 5 lines
The disadvantage of this patchset is requiring the user to allocate HugeTLB pages which is an extra complication.
I am not sure if this patchset would need to interact with XDP multi-buffer at all directly. Does anyone have anything to add here?
Quoted 6 lines
Well, I'm mostly concerned with having two different operation and configuration modes for the same thing. We'll probably need to support multibuf for
AF_XDPanyway for the non-ZC path, which means we'll need to create a UAPI for that in any case. And having two APIs is just going to be more complexity to handle at both the documentation and maintenance level.
Quoted 11 lines
One does not replace the other. We need them both, unfortunately. Multi-buff is great for e.g., stitching together different headers with the same data. Point to different buffers for the header in each packet but the same piece of data in all of them. This will never be solved with Kal's approach. We just need multi-buffer support for this. BTW, we are close to posting multi-buff support for
AF_XDP. Just hang in there a little while longer while the last glitches are fixed. We have to stage it in two patch sets as it will be too long otherwise. First one will only contain improvements to the xsk selftests framework so that multi-buffer tests can be supported. The second one will be the core code and the actual multi-buffer tests.
Alright, sounds good!
Quoted 11 lines
As for what Kal's patches are good for, please see below.
Quoted 3 lines
It *might* be worth it to do this if the performance benefit is really compelling, but, well, you'd need to implement both and compare directly to know that for sure :)
The performance benefit is compelling. As I wrote in a mail to a post by Kal, there are users out there that state that this feature (for zero-copy mode nota bene) is a must for them to be able to use
AF_XDPinstead of DPDK style user-mode drivers. They have really tough latency requirements.
Hmm, okay, looking forward to seeing the benchmark results, then! :)
-Toke
Quoted 7 lines
Well, I'm mostly concerned with having two different operation and configuration modes for the same thing. We'll probably need to support multibuf for
AF_XDPanyway for the non-ZC path, which means we'll need to create a UAPI for that in any case. And having two APIs is just going to be more complexity to handle at both the documentation and maintenance level.
I don't know if I would call this another "API". This patchset doesn't change the semantics of anything. It only lifts the chunk size restriction when hugepages are used. Furthermore, the changes here are quite small and easy to understand. The four sentences added to the documentation shouldn't be too concerning either. :-)
In 30 years when everyone finally migrates to page sizes >= 64K the maintenance burden will drop to zero. Knock wood. :-)
Quoted 4 lines
It *might* be worth it to do this if the performance benefit is really compelling, but, well, you'd need to implement both and compare directly to know that for sure :)
What about use-cases that require incoming packet data to be contiguous? Without larger chunk sizes, the user is forced to allocate extra space per packet and copy the data. This defeats the purpose of ZC.
Quoted 8 lines · Kal Cutter Conley
Quoted 7 lines
Well, I'm mostly concerned with having two different operation and configuration modes for the same thing. We'll probably need to support multibuf for
AF_XDPanyway for the non-ZC path, which means we'll need to create a UAPI for that in any case. And having two APIs is just going to be more complexity to handle at both the documentation and maintenance level.
Quoted 5 lines
I don't know if I would call this another "API". This patchset doesn't change the semantics of anything. It only lifts the chunk size restriction when hugepages are used. Furthermore, the changes here are quite small and easy to understand. The four sentences added to the documentation shouldn't be too concerning either. :-)
Well, you mentioned yourself that:
Quoted 2 lines
The disadvantage of this patchset is requiring the user to allocate HugeTLB pages which is an extra complication.
In addition, presumably when using this mode, the other XDP actions (XDP_PASS, XDP_REDIRECT to other targets) would stop working unless we add special handling for that in the kernel? We'll definitely need to handle that somehow...
Quoted 2 lines
In 30 years when everyone finally migrates to page sizes >= 64K the maintenance burden will drop to zero. Knock wood. :-)
Haha, right, but let's make sure we have something that is consistent in the intervening decades, shall we? ;)
Quoted 8 lines
Quoted 3 lines
It *might* be worth it to do this if the performance benefit is really compelling, but, well, you'd need to implement both and compare directly to know that for sure :)
What about use-cases that require incoming packet data to be contiguous? Without larger chunk sizes, the user is forced to allocate extra space per packet and copy the data. This defeats the purpose of ZC.
What use cases would that be, exactly? Presumably this is also a performance issue? Which goes back to me asking for benchmarks :)
-Toke
Quoted 4 lines
Well, you mentioned yourself that:
Quoted 2 lines
The disadvantage of this patchset is requiring the user to allocate HugeTLB pages which is an extra complication.
It's a small extra complication *for the user*. However, users that need this feature are willing to allocate hugepages. We are one such user. For us, having to deal with packets split into disjoint buffers (from the XDP multi-buffer paradigm) is a significantly more annoying complication than allocating hugepages (particularly on the RX side).
Quoted 10 lines · Kal Cutter Conley
Quoted 4 lines
Well, you mentioned yourself that:
Quoted 2 lines
The disadvantage of this patchset is requiring the user to allocate HugeTLB pages which is an extra complication.
It's a small extra complication *for the user*. However, users that need this feature are willing to allocate hugepages. We are one such user. For us, having to deal with packets split into disjoint buffers (from the XDP multi-buffer paradigm) is a significantly more annoying complication than allocating hugepages (particularly on the RX side).
"More annoying" is not a great argument, though. You're basically saying "please complicate your code so I don't have to complicate mine". And since kernel API is essentially frozen forever, adding more of them carries a pretty high cost, which is why kernel developers tend not to be easily swayed by convenience arguments (if all you want is a more convenient API, just build one on top of the kernel primitives and wrap it into a library).
So you'll need to come up with either (1) a use case that you *can't* solve without this new API (with specifics as to why that is the case), or (2) a compelling performance benchmark showing the complexity is worth it. Magnus indicated he would be able to produce the latter, in which case I'm happy to be persuaded by the numbers.
In any case, however, the behaviour needs to be consistent wrt the rest of XDP, so it's not as simple as just increasing the limit (as I mentioned in my previous email).
-Toke
Quoted 7 lines
"More annoying" is not a great argument, though. You're basically saying "please complicate your code so I don't have to complicate mine". And since kernel API is essentially frozen forever, adding more of them carries a pretty high cost, which is why kernel developers tend not to be easily swayed by convenience arguments (if all you want is a more convenient API, just build one on top of the kernel primitives and wrap it into a library).
I was trying to make a fair comparison from the user's perspective between having to allocate huge pages and deal with discontiguous buffers. That was all.
I think the "your code" distinction is a bit harsh. The kernel is a community project. Why isn't it "our" code? I am trying to add a feature that I think is generally useful to people. The kernel only exists to serve its users. I believe I am doing more good than harm sending these patches.
Quoted 8 lines · Kal Cutter Conley
Quoted 7 lines
"More annoying" is not a great argument, though. You're basically saying "please complicate your code so I don't have to complicate mine". And since kernel API is essentially frozen forever, adding more of them carries a pretty high cost, which is why kernel developers tend not to be easily swayed by convenience arguments (if all you want is a more convenient API, just build one on top of the kernel primitives and wrap it into a library).
Quoted 8 lines
I was trying to make a fair comparison from the user's perspective between having to allocate huge pages and deal with discontiguous buffers. That was all.
I think the "your code" distinction is a bit harsh. The kernel is a community project. Why isn't it "our" code? I am trying to add a feature that I think is generally useful to people. The kernel only exists to serve its users.
Oh, I'm sorry if that came across as harsh, that was not my intention! I was certainly not trying to make a "you vs us" distinction; I was just trying to explain why making changes on the kernel side carries a higher cost than an equivalent (or even slightly more complex) change on the userspace side, because of the UAPI consideration.
Quoted 1 line
I believe I am doing more good than harm sending these patches.
I don't think so! You've certainly sparked a discussion, that is good :)
-Toke
Quoted 17 lines
Quoted 8 lines
Quoted 7 lines
"More annoying" is not a great argument, though. You're basically saying "please complicate your code so I don't have to complicate mine". And since kernel API is essentially frozen forever, adding more of them carries a pretty high cost, which is why kernel developers tend not to be easily swayed by convenience arguments (if all you want is a more convenient API, just build one on top of the kernel primitives and wrap it into a library).
Quoted 8 lines
I was trying to make a fair comparison from the user's perspective between having to allocate huge pages and deal with discontiguous buffers. That was all.
I think the "your code" distinction is a bit harsh. The kernel is a community project. Why isn't it "our" code? I am trying to add a feature that I think is generally useful to people. The kernel only exists to serve its users.
Quoted 5 lines
Oh, I'm sorry if that came across as harsh, that was not my intention! I was certainly not trying to make a "you vs us" distinction; I was just trying to explain why making changes on the kernel side carries a higher cost than an equivalent (or even slightly more complex) change on the userspace side, because of the UAPI consideration.
No problem! I agree. I am just somewhat confused by the "slightly more complex" view of the situation. Currently, packets > 4K are not supported _at all_ with AF_XDP. So this patchset adds something that is not possible _at all_ today. We have been patiently waiting since 2018 for AF_XDP jumbo packet support. It seems that interest in this feature from maintainers has been... lacking. :-) Now, if I understand the situation correctly, you are asking for benchmarks to compare this implementation with AF_XDP multi-buffer which doesn't exist yet? I am glad it's being worked on but until there is a patchset, AF_XDP multi-buffer is still VAPORWARE. :-)
Now in our case, we are primarily interested in throughput and reducing total system load (so we have more compute budget for other tasks). The faster we can receive packets the better. The packets we need to receive are (almost) all 8000-9000 bytes... Using AF_XDP multi-buffer would mean either (1) allocating extra space per packet and copying the data to linearize it or (2) rewriting a significant amount of code to handle segmented packets efficiently. If you want benchmarks for (1) just use xdpsock and compare -z with -c. I think that is a good approximation...
Hopefully, the HFT crowd can save this patchset in the end. :-)
-Kal
Quoted 22 lines · Toke Høiland-Jørgensen
Quoted 10 lines · Kal Cutter Conley
Quoted 4 lines
Well, you mentioned yourself that:
Quoted 2 lines
The disadvantage of this patchset is requiring the user to allocate HugeTLB pages which is an extra complication.
It's a small extra complication *for the user*. However, users that need this feature are willing to allocate hugepages. We are one such user. For us, having to deal with packets split into disjoint buffers (from the XDP multi-buffer paradigm) is a significantly more annoying complication than allocating hugepages (particularly on the RX side).
"More annoying" is not a great argument, though. You're basically saying "please complicate your code so I don't have to complicate mine". And since kernel API is essentially frozen forever, adding more of them carries a pretty high cost, which is why kernel developers tend not to be easily swayed by convenience arguments (if all you want is a more convenient API, just build one on top of the kernel primitives and wrap it into a library).
Quoted 5 lines
So you'll need to come up with either (1) a use case that you *can't* solve without this new API (with specifics as to why that is the case), or (2) a compelling performance benchmark showing the complexity is worth it. Magnus indicated he would be able to produce the latter, in which case I'm happy to be persuaded by the numbers.
We will measure it and get back to you. Would be good with some numbers.
Quoted 6 lines
In any case, however, the behaviour needs to be consistent wrt the rest of XDP, so it's not as simple as just increasing the limit (as I mentioned in my previous email).
-Toke
Quoted 23 lines · Magnus Karlsson
Quoted 22 lines · Toke Høiland-Jørgensen
Quoted 10 lines · Kal Cutter Conley
Quoted 4 lines
Well, you mentioned yourself that:
Quoted 2 lines
The disadvantage of this patchset is requiring the user to allocate HugeTLB pages which is an extra complication.
It's a small extra complication *for the user*. However, users that need this feature are willing to allocate hugepages. We are one such user. For us, having to deal with packets split into disjoint buffers (from the XDP multi-buffer paradigm) is a significantly more annoying complication than allocating hugepages (particularly on the RX side).
"More annoying" is not a great argument, though. You're basically saying "please complicate your code so I don't have to complicate mine". And since kernel API is essentially frozen forever, adding more of them carries a pretty high cost, which is why kernel developers tend not to be easily swayed by convenience arguments (if all you want is a more convenient API, just build one on top of the kernel primitives and wrap it into a library).
Quoted 8 lines
Quoted 5 lines
So you'll need to come up with either (1) a use case that you *can't* solve without this new API (with specifics as to why that is the case), or (2) a compelling performance benchmark showing the complexity is worth it. Magnus indicated he would be able to produce the latter, in which case I'm happy to be persuaded by the numbers.
We will measure it and get back to you. Would be good with some numbers.
Sounds good, thanks! :)
-Toke
Quoted 7 lines
Quoted 2 lines
We will measure it and get back to you. Would be good with some numbers.
Sounds good, thanks! :)
-Toke
+1. Thanks a lot for doing this! :-)
Quoted 4 lines
In addition, presumably when using this mode, the other XDP actions (
XDP_PASS,XDP_REDIRECTto other targets) would stop working unless we add special handling for that in the kernel? We'll definitely need to handle that somehow...
I am not familiar with all the details here. Do you know a reason why these cases would stop working / why special handling would be needed? For example, if I have a UMEM that uses hugepages and XDP_PASS is returned, then the data is just copied into an SKB right? SKBs can also be created directly from hugepages AFAIK. So I don't understand what the issue would be. Can someone explain this concern?
Quoted 11 lines · Kal Cutter Conley
Quoted 4 lines
In addition, presumably when using this mode, the other XDP actions (
XDP_PASS,XDP_REDIRECTto other targets) would stop working unless we add special handling for that in the kernel? We'll definitely need to handle that somehow...I am not familiar with all the details here. Do you know a reason why these cases would stop working / why special handling would be needed? For example, if I have a UMEM that uses hugepages and
XDP_PASSis returned, then the data is just copied into an SKB right? SKBs can also be created directly from hugepages AFAIK. So I don't understand what the issue would be. Can someone explain this concern?
Well, I was asking :) It may well be that the SKB path just works; did you test this? Pretty sure XDP_REDIRECT to another device won't, though?
-Toke
Quoted 12 lines
Quoted 11 lines
Quoted 4 lines
In addition, presumably when using this mode, the other XDP actions (
XDP_PASS,XDP_REDIRECTto other targets) would stop working unless we add special handling for that in the kernel? We'll definitely need to handle that somehow...I am not familiar with all the details here. Do you know a reason why these cases would stop working / why special handling would be needed? For example, if I have a UMEM that uses hugepages and
XDP_PASSis returned, then the data is just copied into an SKB right? SKBs can also be created directly from hugepages AFAIK. So I don't understand what the issue would be. Can someone explain this concern?
Quoted 3 lines
Well, I was asking :) It may well be that the SKB path just works; did you test this? Pretty sure
XDP_REDIRECTto another device won't, though?
I was also asking :-)
I tested that the SKB path is usable today with this patch. Specifically, sending and receiving large jumbo packets with AF_XDP and that a non-multi-buffer XDP program could access the whole packet. I have not specifically tested XDP_REDIRECT to another device or anything with ZC since that is not possible without driver support.
My feeling is, there wouldn't be non-trivial issues here since this patchset changes nothing except allowing the maximum chunk size to be larger. The driver either supports larger MTUs with XDP enabled or it doesn't. If it doesn't, the frames are dropped anyway. Also, chunk size mismatches between two XSKs (e.g. with XDP_REDIRECT) would be something supported or not supported irrespective of this patchset.
Maciej Fijalkowski
maintainerOn Tue, Apr 18, 2023 at 01:12:00PM +0200, Kal Cutter Conley wrote:
Hi there,
Quoted 12 lines
Quoted 12 lines
Quoted 11 lines
Quoted 4 lines
In addition, presumably when using this mode, the other XDP actions (
XDP_PASS,XDP_REDIRECTto other targets) would stop working unless we add special handling for that in the kernel? We'll definitely need to handle that somehow...I am not familiar with all the details here. Do you know a reason why these cases would stop working / why special handling would be needed? For example, if I have a UMEM that uses hugepages and
XDP_PASSis returned, then the data is just copied into an SKB right? SKBs can also be created directly from hugepages AFAIK. So I don't understand what the issue would be. Can someone explain this concern?
Quoted 2 lines
Quoted 2 lines
Well, I was asking :) It may well be that the SKB path just works; did you test this? Pretty sure
XDP_REDIRECTto another device won't, though?
for XDP_PASS we have to allocate a new buffer and copy the contents from current xdp_buff that was backed by xsk_buff_pool and give the current one back to pool. I am not sure if __napi_alloc_skb() is always capable of handling len > PAGE_SIZE - i believe there might a particular combination of settings that allows it, but if not we should have a fallback path that would iterate over data and copy this to a certain (linear + frags) parts. This implies non-zero effort that is needed for jumbo frames ZC support.
I can certainly test this out and play with it - maybe this just works, I didn't check yet. Even if it does, then we need some kind of temporary mechanism that will forbid loading ZC jumbo frames due to what Toke brought up.
Quoted 10 lines
Quoted 1 line
I was also asking :-)
I tested that the SKB path is usable today with this patch. Specifically, sending and receiving large jumbo packets with
AF_XDPand that a non-multi-buffer XDP program could access the whole packet. I have not specifically testedXDP_REDIRECTto another device or anything with ZC since that is not possible without driver support.
Quoted 6 lines
My feeling is, there wouldn't be non-trivial issues here since this patchset changes nothing except allowing the maximum chunk size to be larger. The driver either supports larger MTUs with XDP enabled or it doesn't. If it doesn't, the frames are dropped anyway. Also, chunk size mismatches between two XSKs (e.g. with
XDP_REDIRECT) would be something supported or not supported irrespective of this patchset.
Here is the comparison between multi-buffer and jumbo frames that I did for ZC ice driver. Configured MTU was 8192 as this is the frame size for aligned mode when working with huge pages. I am presenting plain numbers over here from xdpsock.
Mbuf, packet size = 8192 - XDP_PACKET_HEADROOM
885,705pps - rxdrop frame_size=4096
806,307pps - l2fwd frame_size=4096
877,989pps - rxdrop frame_size=2048
773,331pps - l2fwd frame_size=2048
Jumbo, packet size = 8192 - XDP_PACKET_HEADROOM
893,530pps - rxdrop frame_size=8192
841,860pps - l2fwd frame_size=8192
Kal might say that multi-buffer numbers are imaginary as these patches were never shown to the public ;) but now that we have extensive test suite I am fixing some last issues that stand out, so we are asking for some more patience over here... overall i was expecting that they will be much worse when compared to jumbo frames, but then again i believe this implementation is not ideal and can be improved. Nevertheless, jumbo frames support has its value.
Quoted 27 lines · Maciej Fijalkowski
On Tue, Apr 18, 2023 at 01:12:00PM +0200, Kal Cutter Conley wrote:
Hi there,
Quoted 12 lines
Quoted 12 lines
Quoted 11 lines
Quoted 4 lines
In addition, presumably when using this mode, the other XDP actions (
XDP_PASS,XDP_REDIRECTto other targets) would stop working unless we add special handling for that in the kernel? We'll definitely need to handle that somehow...I am not familiar with all the details here. Do you know a reason why these cases would stop working / why special handling would be needed? For example, if I have a UMEM that uses hugepages and
XDP_PASSis returned, then the data is just copied into an SKB right? SKBs can also be created directly from hugepages AFAIK. So I don't understand what the issue would be. Can someone explain this concern?Quoted 2 lines
Quoted 2 lines
Well, I was asking :) It may well be that the SKB path just works; did you test this? Pretty sure
XDP_REDIRECTto another device won't, though?for
XDP_PASSwe have to allocate a new buffer and copy the contents from currentxdp_buffthat was backed byxsk_buff_pooland give the current one back to pool. I am not sure if__napi_alloc_skb()is always capable of handling len >PAGE_SIZE- i believe there might a particular combination of settings that allows it, but if not we should have a fallback path that would iterate over data and copy this to a certain (linear + frags) parts. This implies non-zero effort that is needed for jumbo frames ZC support.
Quoted 4 lines
I can certainly test this out and play with it - maybe this just works, I didn't check yet. Even if it does, then we need some kind of temporary mechanism that will forbid loading ZC jumbo frames due to what Toke brought up.
Yeah, this was exactly the kind of thing I was worried about (same for XDP_REDIRECT). Thanks for fleshing it out a bit :)
Quoted 32 lines
Quoted 10 lines
Quoted 1 line
I was also asking :-)
I tested that the SKB path is usable today with this patch. Specifically, sending and receiving large jumbo packets with
AF_XDPand that a non-multi-buffer XDP program could access the whole packet. I have not specifically testedXDP_REDIRECTto another device or anything with ZC since that is not possible without driver support.Quoted 6 lines
My feeling is, there wouldn't be non-trivial issues here since this patchset changes nothing except allowing the maximum chunk size to be larger. The driver either supports larger MTUs with XDP enabled or it doesn't. If it doesn't, the frames are dropped anyway. Also, chunk size mismatches between two XSKs (e.g. with
XDP_REDIRECT) would be something supported or not supported irrespective of this patchset.Here is the comparison between multi-buffer and jumbo frames that I did for ZC ice driver. Configured MTU was 8192 as this is the frame size for aligned mode when working with huge pages. I am presenting plain numbers over here from xdpsock.
Mbuf, packet size = 8192 -
XDP_PACKET_HEADROOM
885,705pps - rxdrop frame_size=4096
806,307pps - l2fwd frame_size=4096
877,989pps - rxdrop frame_size=2048
773,331pps - l2fwd frame_size=2048Jumbo, packet size = 8192 -
XDP_PACKET_HEADROOM
893,530pps - rxdrop frame_size=8192
841,860pps - l2fwd frame_size=8192
Quoted 7 lines
Kal might say that multi-buffer numbers are imaginary as these patches were never shown to the public ;) but now that we have extensive test suite I am fixing some last issues that stand out, so we are asking for some more patience over here... overall i was expecting that they will be much worse when compared to jumbo frames, but then again i believe this implementation is not ideal and can be improved. Nevertheless, jumbo frames support has its value.
Thank you for doing these! Okay, so that's between 1-4% improvement (vs the 4k frags). I dunno, I wouldn't consider that a slam dunk; would depend on the additional complexity if it is worth it to do both, IMO...
-Toke
Quoted 78 lines · Toke Høiland-Jørgensen
Quoted 27 lines · Maciej Fijalkowski
On Tue, Apr 18, 2023 at 01:12:00PM +0200, Kal Cutter Conley wrote:
Hi there,
Quoted 12 lines
Quoted 12 lines
Quoted 11 lines
Quoted 4 lines
In addition, presumably when using this mode, the other XDP actions (
XDP_PASS,XDP_REDIRECTto other targets) would stop working unless we add special handling for that in the kernel? We'll definitely need to handle that somehow...I am not familiar with all the details here. Do you know a reason why these cases would stop working / why special handling would be needed? For example, if I have a UMEM that uses hugepages and
XDP_PASSis returned, then the data is just copied into an SKB right? SKBs can also be created directly from hugepages AFAIK. So I don't understand what the issue would be. Can someone explain this concern?Quoted 2 lines
Quoted 2 lines
Well, I was asking :) It may well be that the SKB path just works; did you test this? Pretty sure
XDP_REDIRECTto another device won't, though?for
XDP_PASSwe have to allocate a new buffer and copy the contents from currentxdp_buffthat was backed byxsk_buff_pooland give the current one back to pool. I am not sure if__napi_alloc_skb()is always capable of handling len >PAGE_SIZE- i believe there might a particular combination of settings that allows it, but if not we should have a fallback path that would iterate over data and copy this to a certain (linear + frags) parts. This implies non-zero effort that is needed for jumbo frames ZC support.Quoted 4 lines
I can certainly test this out and play with it - maybe this just works, I didn't check yet. Even if it does, then we need some kind of temporary mechanism that will forbid loading ZC jumbo frames due to what Toke brought up.
Yeah, this was exactly the kind of thing I was worried about (same for
XDP_REDIRECT). Thanks for fleshing it out a bit :)Quoted 32 lines
Quoted 10 lines
Quoted 1 line
I was also asking :-)
I tested that the SKB path is usable today with this patch. Specifically, sending and receiving large jumbo packets with
AF_XDPand that a non-multi-buffer XDP program could access the whole packet. I have not specifically testedXDP_REDIRECTto another device or anything with ZC since that is not possible without driver support.Quoted 6 lines
My feeling is, there wouldn't be non-trivial issues here since this patchset changes nothing except allowing the maximum chunk size to be larger. The driver either supports larger MTUs with XDP enabled or it doesn't. If it doesn't, the frames are dropped anyway. Also, chunk size mismatches between two XSKs (e.g. with
XDP_REDIRECT) would be something supported or not supported irrespective of this patchset.Here is the comparison between multi-buffer and jumbo frames that I did for ZC ice driver. Configured MTU was 8192 as this is the frame size for aligned mode when working with huge pages. I am presenting plain numbers over here from xdpsock.
Mbuf, packet size = 8192 -
XDP_PACKET_HEADROOM
885,705pps - rxdrop frame_size=4096
806,307pps - l2fwd frame_size=4096
877,989pps - rxdrop frame_size=2048
773,331pps - l2fwd frame_size=2048Jumbo, packet size = 8192 -
XDP_PACKET_HEADROOM
893,530pps - rxdrop frame_size=8192
841,860pps - l2fwd frame_size=8192Quoted 7 lines
Kal might say that multi-buffer numbers are imaginary as these patches were never shown to the public ;) but now that we have extensive test suite I am fixing some last issues that stand out, so we are asking for some more patience over here... overall i was expecting that they will be much worse when compared to jumbo frames, but then again i believe this implementation is not ideal and can be improved. Nevertheless, jumbo frames support has its value.
Quoted 3 lines
Thank you for doing these! Okay, so that's between 1-4% improvement (vs the 4k frags). I dunno, I wouldn't consider that a slam dunk; would depend on the additional complexity if it is worth it to do both, IMO...
If we are using 4K frags, the worst case is that we have to use 3 frags to cover a 9K packet. Interpolating between the results above would mean somewhere in the 1 - 6% range of improvement with jumbo frames. Something that is not covered by these tests at all is the overhead of an abstraction layer for dealing with multiple buffers. I believe many applications would choose to have one to hide the fact that there are multiple buffers. So I think these improvement numbers are on the lower side.
But agree that we should factor in the complexity of covering the cases you have brought up and see if it is worth it.
Quoted 2 lines
-Toke
Quoted 20 lines · Maciej Fijalkowski
On Tue, Apr 18, 2023 at 01:12:00PM +0200, Kal Cutter Conley wrote:
Hi there,
Quoted 12 lines
Quoted 12 lines
Quoted 11 lines
Quoted 4 lines
In addition, presumably when using this mode, the other XDP actions (
XDP_PASS,XDP_REDIRECTto other targets) would stop working unless we add special handling for that in the kernel? We'll definitely need to handle that somehow...I am not familiar with all the details here. Do you know a reason why these cases would stop working / why special handling would be needed? For example, if I have a UMEM that uses hugepages and
XDP_PASSis returned, then the data is just copied into an SKB right? SKBs can also be created directly from hugepages AFAIK. So I don't understand what the issue would be. Can someone explain this concern?Quoted 2 lines
Quoted 2 lines
Well, I was asking :) It may well be that the SKB path just works; did you test this? Pretty sure
XDP_REDIRECTto another device won't, though?
Quoted 7 lines
for
XDP_PASSwe have to allocate a new buffer and copy the contents from currentxdp_buffthat was backed byxsk_buff_pooland give the current one back to pool. I am not sure if__napi_alloc_skb()is always capable of handling len >PAGE_SIZE- i believe there might a particular combination of settings that allows it, but if not we should have a fallback path that would iterate over data and copy this to a certain (linear + frags) parts. This implies non-zero effort that is needed for jumbo frames ZC support.
Thinking aloud, could not our multi-buffer work help with this? Sounds quite similar to operations that we have to do in that patch set. And if so, would it not be prudent to get the multi-buffer support in there first, then implement these things on top of that? What do you think?
Quoted 44 lines
I can certainly test this out and play with it - maybe this just works, I didn't check yet. Even if it does, then we need some kind of temporary mechanism that will forbid loading ZC jumbo frames due to what Toke brought up.
Quoted 10 lines
Quoted 1 line
I was also asking :-)
I tested that the SKB path is usable today with this patch. Specifically, sending and receiving large jumbo packets with
AF_XDPand that a non-multi-buffer XDP program could access the whole packet. I have not specifically testedXDP_REDIRECTto another device or anything with ZC since that is not possible without driver support.Quoted 6 lines
My feeling is, there wouldn't be non-trivial issues here since this patchset changes nothing except allowing the maximum chunk size to be larger. The driver either supports larger MTUs with XDP enabled or it doesn't. If it doesn't, the frames are dropped anyway. Also, chunk size mismatches between two XSKs (e.g. with
XDP_REDIRECT) would be something supported or not supported irrespective of this patchset.Here is the comparison between multi-buffer and jumbo frames that I did for ZC ice driver. Configured MTU was 8192 as this is the frame size for aligned mode when working with huge pages. I am presenting plain numbers over here from xdpsock.
Mbuf, packet size = 8192 -
XDP_PACKET_HEADROOM
885,705pps - rxdrop frame_size=4096
806,307pps - l2fwd frame_size=4096
877,989pps - rxdrop frame_size=2048
773,331pps - l2fwd frame_size=2048Jumbo, packet size = 8192 -
XDP_PACKET_HEADROOM
893,530pps - rxdrop frame_size=8192
841,860pps - l2fwd frame_size=8192Kal might say that multi-buffer numbers are imaginary as these patches were never shown to the public ;) but now that we have extensive test suite I am fixing some last issues that stand out, so we are asking for some more patience over here... overall i was expecting that they will be much worse when compared to jumbo frames, but then again i believe this implementation is not ideal and can be improved. Nevertheless, jumbo frames support has its value.
Quoted 11 lines
Here is the comparison between multi-buffer and jumbo frames that I did for ZC ice driver. Configured MTU was 8192 as this is the frame size for aligned mode when working with huge pages. I am presenting plain numbers over here from xdpsock.
Mbuf, packet size = 8192 -
XDP_PACKET_HEADROOM
885,705pps - rxdrop frame_size=4096
806,307pps - l2fwd frame_size=4096
877,989pps - rxdrop frame_size=2048
773,331pps - l2fwd frame_size=2048
Quoted 3 lines
Jumbo, packet size = 8192 -
XDP_PACKET_HEADROOM
893,530pps - rxdrop frame_size=8192
841,860pps - l2fwd frame_size=8192
Thanks so much for sharing these initial results! Do you have similar measurements for ~9000 byte packets in unaligned mode? We typically receive packets larger than 8192 bytes.
Quoted 8 lines
Kal might say that multi-buffer numbers are imaginary as these patches were never shown to the public ;) but now that we have extensive test suite I am fixing some last issues that stand out, so we are asking for some more patience over here... overall i was expecting that they will be much worse when compared to jumbo frames, but then again i believe this implementation is not ideal and can be improved. Nevertheless, jumbo frames support has its value.
You made me chuckle ;-) Any measurements people can provide are helpful, even if they must be taken with a grain of salt. ;-). How much of your test suite can be upstreamed in the future? My assumption was the difference should be measurable, at least you have confirmed that. :-)
PATCH v3 2/3 selftests: xsk: Use hugepages when umem->frame_size > PAGE_SIZE
In reply to Kal ConleyHugeTLB UMEMs now support chunk_size > PAGE_SIZE. Set MAP_HUGETLB when frame_size > PAGE_SIZE for future tests.
--- tools/testing/selftests/bpf/xskxceiver.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/testing/selftests/bpf/xskxceiver.c +1 -1
@@ -1289,7 +1289,7 @@ static void thread_common_ops(struct test_spec *test, struct ifobject *ifobject)void *bufs;int ret; -if (ifobject->umem->unaligned_mode) +if (ifobject->umem->frame_size >sysconf (_SC_PAGESIZE) || ifobject->umem->unaligned_mode) mmap_flags |=MAP_HUGETLB ;if (ifobject->shared_umem)
Signature · 1 line
2.39.2
PATCH v3 3/3 selftests: xsk: Add tests for 8K and 9K frame sizes
In reply to Kal ConleyAdd tests:
- RUN_TO_COMPLETION_8K_FRAME_SIZE: frame_size=8192 (aligned)
- UNALIGNED_9K_FRAME_SIZE: frame_size=9000 (unaligned)
--- tools/testing/selftests/bpf/xskxceiver.c | 25 ++++++++++++++++++++++++ tools/testing/selftests/bpf/xskxceiver.h | 2 ++ 2 files changed, 27 insertions(+)
tools/testing/selftests/bpf/xskxceiver.c +25 -0
@@ -1841,6 +1841,17 @@ static void run_pkt_test(struct test_spec *test, enum test_mode mode, enum test_pkt_stream_replace (test,DEFAULT_PKT_CNT ,PKT_SIZE );testapp_validate_traffic (test);break ; +case TEST_TYPE_RUN_TO_COMPLETION_8K_FRAME : +if (!hugepages_present (test->ifobj_tx)) { +ksft_test_result_skip ("No 2M huge pages present.\n" ); +return ; + } +test_spec_set_name (test,"RUN_TO_COMPLETION_8K_FRAME_SIZE" ); + test->ifobj_tx->umem->frame_size =8192 ; + test->ifobj_rx->umem->frame_size =8192 ; +pkt_stream_replace (test,DEFAULT_PKT_CNT ,PKT_SIZE ); +testapp_validate_traffic (test); +break ;case TEST_TYPE_RX_POLL : test->ifobj_rx->use_poll = true;test_spec_set_name (test,"POLL_RX" );
@@ -1904,6 +1915,20 @@ static void run_pkt_test(struct test_spec *test, enum test_mode mode, enum test_if (!testapp_unaligned (test))return ;break ; +case TEST_TYPE_UNALIGNED_9K_FRAME : +if (!hugepages_present (test->ifobj_tx)) { +ksft_test_result_skip ("No 2M huge pages present.\n" ); +return ; + } +test_spec_set_name (test,"UNALIGNED_9K_FRAME_SIZE" ); + test->ifobj_tx->umem->frame_size =9000 ; + test->ifobj_rx->umem->frame_size =9000 ; + test->ifobj_tx->umem->unaligned_mode = true; + test->ifobj_rx->umem->unaligned_mode = true; +pkt_stream_replace (test,DEFAULT_PKT_CNT ,PKT_SIZE ); + test->ifobj_rx->pkt_stream->use_addr_for_fill = true; +testapp_validate_traffic (test); +break ;case TEST_TYPE_HEADROOM :testapp_headroom (test);break ;
tools/testing/selftests/bpf/xskxceiver.h +2 -0
@@ -69,12 +69,14 @@ enum test_mode {enum test_type {TEST_TYPE_RUN_TO_COMPLETION ,TEST_TYPE_RUN_TO_COMPLETION_2K_FRAME , +TEST_TYPE_RUN_TO_COMPLETION_8K_FRAME ,TEST_TYPE_RUN_TO_COMPLETION_SINGLE_PKT ,TEST_TYPE_RX_POLL ,TEST_TYPE_TX_POLL ,TEST_TYPE_POLL_RXQ_TMOUT ,TEST_TYPE_POLL_TXQ_TMOUT ,TEST_TYPE_UNALIGNED , +TEST_TYPE_UNALIGNED_9K_FRAME ,TEST_TYPE_ALIGNED_INV_DESC ,TEST_TYPE_ALIGNED_INV_DESC_2K_FRAME ,TEST_TYPE_UNALIGNED_INV_DESC ,
Signature · 1 line
2.39.2