net: atlantic: Add XDP support
Subsystems (4)
v4 cover net: atlantic: Add XDP support
This patchset is to make atlantic to support multi-buffer XDP.
The first patch implement control plane of xdp.
The aq_xdp(), callback of .xdp_bpf is added.
The second patch implements data plane of xdp.XDP_TX, XDP_DROP, and XDP_PASS is supported. __aq_ring_xdp_clean() is added to receive and execute xdp program. aq_nic_xmit_xdpf() is added to send packet by XDP.
The third patch implements callback of .ndo_xdp_xmit. aq_xdp_xmit() is added to send redirected packets and it internally calls aq_nic_xmit_xdpf().
Memory model is MEM_TYPE_PAGE_ORDER0 so it doesn't reuse rx page when XDP_TX, XDP_PASS, XDP_REDIRECT.
Default the maximum rx frame size is 2K.
If xdp is attached, size is changed to about 3K.
It can be reused when XDP_DROP, and XDP_ABORTED.
Atlantic driver has AQ_CFG_RX_PAGEORDER option and it will be always 0 if xdp is attached.
LRO will be disabled if XDP program supports only single buffer.
AQC chip supports 32 multi-queues and 8 vectors(irq). There are two options.
1. under 8 cores and maximum 4 tx queues per core.
2. under 4 cores and maximum 8 tx queues per core.
Like other drivers, these tx queues can be used only for XDP_TX, XDP_REDIRECT queue. If so, no tx_lock is needed.
But this patchset doesn't use this strategy because getting hardware tx queue index cost is too high.
So, tx_lock is used in the aq_nic_xmit_xdpf().
single-core, single queue, 80% cpu utilization.
30.75% bpf_prog_xxx_xdp_prog_tx [k] bpf_prog_xxx_xdp_prog_tx 10.35% [kernel] [k] aq_hw_read_reg <---------- here 4.38% [kernel] [k] get_page_from_freelist
single-core, 8 queues, 100% cpu utilization, half PPS.
45.56% [kernel] [k] aq_hw_read_reg <---------- here 17.58% bpf_prog_xxx_xdp_prog_tx [k] bpf_prog_xxx_xdp_prog_tx 4.72% [kernel] [k] hw_atl_b0_hw_ring_rx_receive
Performance result(64 Byte)
1. XDP_TX
a. xdp_geieric, single core
- 2.5Mpps, 100% cpu
b. xdp_driver, single core
- 4.5Mpps, 80% cpu
c. xdp_generic, 8 core(hyper thread)
- 6.3Mpps, 5~10% cpu
d. xdp_driver, 8 core(hyper thread)
- 6.3Mpps, 5% cpu
2. XDP_REDIRECT
a. xdp_generic, single core
- 2.3Mpps
b. xdp_driver, single core
- 4.5Mpps
v4:
- Fix compile warning
v3:
- Change wrong PPS performance result 40% -> 80% in single core(Intel i3-12100)
- Separate aq_nic_map_xdp() from aq_nic_map_skb()
- Drop multi buffer packets if single buffer XDP is attached
- Disable LRO when single buffer XDP is attached
- Use xdp_get_{frame/buff}_len()
v2:
- Do not use inline in C file
Taehee Yoo (3):
net: atlantic: Implement xdp control plane
net: atlantic: Implement xdp data plane
net: atlantic: Implement .ndo_xdp_xmit handler
.../net/ethernet/aquantia/atlantic/aq_cfg.h | 1 + .../ethernet/aquantia/atlantic/aq_ethtool.c | 8 + .../net/ethernet/aquantia/atlantic/aq_main.c | 87 ++++ .../net/ethernet/aquantia/atlantic/aq_main.h | 2 + .../net/ethernet/aquantia/atlantic/aq_nic.c | 137 ++++++ .../net/ethernet/aquantia/atlantic/aq_nic.h | 5 + .../net/ethernet/aquantia/atlantic/aq_ring.c | 415 ++++++++++++++++-- .../net/ethernet/aquantia/atlantic/aq_ring.h | 17 + .../net/ethernet/aquantia/atlantic/aq_vec.c | 23 +- .../net/ethernet/aquantia/atlantic/aq_vec.h | 6 + .../aquantia/atlantic/hw_atl/hw_atl_a0.c | 6 +- .../aquantia/atlantic/hw_atl/hw_atl_b0.c | 10 +- 12 files changed, 675 insertions(+), 42 deletions(-)
Signature · 1 line
2.17.1
PATCH v4 1/3 net: atlantic: Implement xdp control plane
aq_xdp() is a xdp setup callback function for Atlantic driver. When XDP is attached or detached, the device will be restarted because it uses different maximum frame sizes.
If XDP is disabled, 2K RX frame size is used and alloc_skb() is used too. So, It can reuse the next half of that page(flip like the ixgbe). Or, if a reference count is 1(no other user) that page can be fully reused.
ethtool -S <interface name> | grep Page
Queue[0] PageFlips: 0 <-- use half of page
Queue[0] PageReuses: 7 <-- reuse full page
But if XDP is enabled, it uses 3K frame size.
So, the Flip strategy can not be applied.
And reusing a full page is possible only when XDP_DROP, XDP_ABORT.
Also, It supports xdp fragment feature.
mtu can be 16K if xdp prog supports xdp fragment. if not, mtu can not exceed 3K - ETH_HLEN - ETH_FCS.
And a static key is added and It will be used to call the xdp_clean handler in ->poll(). data plane implementation will be contained the followed patch.
---
v4:
- No changed
v3:
- Disable LRO when single buffer XDP is attached
v2:
- No changed
.../net/ethernet/aquantia/atlantic/aq_cfg.h | 1 + .../net/ethernet/aquantia/atlantic/aq_main.c | 86 +++++++++++++++++++ .../net/ethernet/aquantia/atlantic/aq_main.h | 2 + .../net/ethernet/aquantia/atlantic/aq_nic.h | 3 + .../net/ethernet/aquantia/atlantic/aq_ring.c | 57 +++++++----- .../net/ethernet/aquantia/atlantic/aq_ring.h | 9 ++ .../net/ethernet/aquantia/atlantic/aq_vec.c | 23 +++-- .../net/ethernet/aquantia/atlantic/aq_vec.h | 6 ++ .../aquantia/atlantic/hw_atl/hw_atl_a0.c | 6 +- .../aquantia/atlantic/hw_atl/hw_atl_b0.c | 10 +-- 10 files changed, 169 insertions(+), 34 deletions(-)
drivers/net/ethernet/aquantia/atlantic/aq_cfg.h +1 -0
@@ -32,6 +32,7 @@#define AQ_CFG_TX_FRAME_MAX (16U * 1024U)#define AQ_CFG_RX_FRAME_MAX (2U * 1024U) +#define AQ_CFG_XDP_FRAME_MAX (3U * 1024U)#define AQ_CFG_TX_CLEAN_BUDGET 256U
drivers/net/ethernet/aquantia/atlantic/aq_main.c +86 -0
134 lines not on this page · raw message
drivers/net/ethernet/aquantia/atlantic/aq_main.h +2 -0
@@ -12,6 +12,8 @@#include "aq_common.h" #include "aq_nic.h" +DECLARE_STATIC_KEY_FALSE (aq_xdp_locking_key); +void aq_ndev_schedule_work (struct work_struct *work);struct net_device *aq_ndev_alloc (void );
drivers/net/ethernet/aquantia/atlantic/aq_nic.h +3 -0
@@ -11,6 +11,8 @@#define AQ_NIC_H #include <linux/ethtool.h> +#include <net/xdp.h> +#include <linux/bpf.h> #include "aq_common.h" #include "aq_rss.h"
@@ -128,6 +130,7 @@ struct aq_nic_s {struct aq_vec_s *aq_vec[AQ_CFG_VECS_MAX ];struct aq_ring_s *aq_ring_tx[AQ_HW_QUEUES_MAX ];struct aq_hw_s *aq_hw; +struct bpf_prog *xdp_prog;struct net_device *ndev;unsigned int aq_vecs;unsigned int packet_filter;
drivers/net/ethernet/aquantia/atlantic/aq_ring.c +37 -20
148 lines not on this page · raw message
drivers/net/ethernet/aquantia/atlantic/aq_ring.h +9 -0
@@ -11,6 +11,9 @@#define AQ_RING_H #include "aq_common.h" +#include "aq_vec.h" + +#define AQ_XDP_HEADROOM ALIGN(max(NET_SKB_PAD, XDP_PACKET_HEADROOM), 8)struct page ;struct aq_nic_cfg_s ;
@@ -51,6 +54,7 @@ struct __packed aq_ring_buff_s {struct {dma_addr_t pa_eop;struct sk_buff *skb; +struct xdp_frame *xdpf; };/* TxC */ struct {
@@ -133,9 +137,13 @@ struct aq_ring_s {unsigned int dx_size;/* TX or RX descriptor size, */ /* stored here for fater math */ unsigned int page_order; +u16 page_offset; +u16 frame_max;union aq_ring_stats_s stats;dma_addr_t dx_ring_pa; +struct bpf_prog *xdp_prog;enum atl_ring_type ring_type; +struct xdp_rxq_info xdp_rxq; };struct aq_ring_param_s {
@@ -175,6 +183,7 @@ struct aq_ring_s *aq_ring_rx_alloc(struct aq_ring_s *self,struct aq_nic_s *aq_nic, unsigned int idx, struct aq_nic_cfg_s *aq_nic_cfg); +int aq_ring_init (struct aq_ring_s *self,const enum atl_ring_type ring_type);void aq_ring_rx_deinit (struct aq_ring_s *self);void aq_ring_free (struct aq_ring_s *self);
drivers/net/ethernet/aquantia/atlantic/aq_vec.c +17 -6
@@ -10,11 +10,6 @@*/ #include "aq_vec.h" -#include "aq_nic.h" -#include "aq_ring.h" -#include "aq_hw.h" - -#include <linux/netdevice.h> struct aq_vec_s {const struct aq_hw_ops *aq_hw_ops;
@@ -153,9 +148,23 @@ int aq_vec_ring_alloc(struct aq_vec_s *self, struct aq_nic_s *aq_nic,aq_nic_set_tx_ring (aq_nic, idx_ring, ring); +if (xdp_rxq_info_reg (&self->ring[i][AQ_VEC_RX_ID ].xdp_rxq, + aq_nic->ndev, idx, + self->napi.napi_id) <0 ) { + err = -ENOMEM ; + goto err_exit; + } +if (xdp_rxq_info_reg_mem_model (&self->ring[i][AQ_VEC_RX_ID ].xdp_rxq, +MEM_TYPE_PAGE_ORDER0 ,NULL ) <0 ) { +xdp_rxq_info_unreg (&self->ring[i][AQ_VEC_RX_ID ].xdp_rxq); + err = -ENOMEM ; + goto err_exit; + } + ring =aq_ring_rx_alloc (&self->ring[i][AQ_VEC_RX_ID ], aq_nic, idx_ring, aq_nic_cfg);if (!ring) { +xdp_rxq_info_unreg (&self->ring[i][AQ_VEC_RX_ID ].xdp_rxq); err = -ENOMEM ; goto err_exit; }
@@ -300,8 +309,10 @@ void aq_vec_ring_free(struct aq_vec_s *self)for (i =0U , ring = self->ring[0 ]; self->tx_rings > i; ++i, ring = self->ring[i]) {aq_ring_free (&ring[AQ_VEC_TX_ID ]); -if (i < self->rx_rings) +if (i < self->rx_rings) { +xdp_rxq_info_unreg (&ring[AQ_VEC_RX_ID ].xdp_rxq);aq_ring_free (&ring[AQ_VEC_RX_ID ]); + } } self->tx_rings =0 ;
drivers/net/ethernet/aquantia/atlantic/aq_vec.h +6 -0
@@ -13,7 +13,13 @@#define AQ_VEC_H #include "aq_common.h" +#include "aq_nic.h" +#include "aq_ring.h" +#include "aq_hw.h" +#include <linux/irqreturn.h> +#include <linux/filter.h> +#include <linux/netdevice.h> struct aq_hw_s ;struct aq_hw_ops ;
drivers/net/ethernet/aquantia/atlantic/hw_atl/hw_atl_a0.c +3 -3
@@ -531,7 +531,7 @@ static int hw_atl_a0_hw_ring_rx_init(struct aq_hw_s *self,hw_atl_rdm_rx_desc_len_set (self, aq_ring->size /8U , aq_ring->idx);hw_atl_rdm_rx_desc_data_buff_size_set (self, -AQ_CFG_RX_FRAME_MAX /1024U , + aq_ring->frame_max /1024U , aq_ring->idx);hw_atl_rdm_rx_desc_head_buff_size_set (self,0U , aq_ring->idx);
@@ -706,9 +706,9 @@ static int hw_atl_a0_hw_ring_rx_receive(struct aq_hw_s *self,if (HW_ATL_A0_RXD_WB_STAT2_EOP & rxd_wb->status) { buff->len = rxd_wb->pkt_len % -AQ_CFG_RX_FRAME_MAX ; + ring->frame_max; buff->len = buff->len ? - buff->len :AQ_CFG_RX_FRAME_MAX ; + buff->len : ring->frame_max; buff->next =0U ; buff->is_eop =1U ; }else {
drivers/net/ethernet/aquantia/atlantic/hw_atl/hw_atl_b0.c +5 -5
@@ -766,7 +766,7 @@ int hw_atl_b0_hw_ring_rx_init(struct aq_hw_s *self, struct aq_ring_s *aq_ring,hw_atl_rdm_rx_desc_len_set (self, aq_ring->size /8U , aq_ring->idx);hw_atl_rdm_rx_desc_data_buff_size_set (self, -AQ_CFG_RX_FRAME_MAX /1024U , + aq_ring->frame_max /1024U , aq_ring->idx);hw_atl_rdm_rx_desc_head_buff_size_set (self,0U , aq_ring->idx);
@@ -969,15 +969,15 @@ int hw_atl_b0_hw_ring_rx_receive(struct aq_hw_s *self, struct aq_ring_s *ring) rxd_wb->status);if (HW_ATL_B0_RXD_WB_STAT2_EOP & rxd_wb->status) { buff->len = rxd_wb->pkt_len % -AQ_CFG_RX_FRAME_MAX ; + ring->frame_max; buff->len = buff->len ? - buff->len :AQ_CFG_RX_FRAME_MAX ; + buff->len : ring->frame_max; buff->next =0U ; buff->is_eop =1U ; }else { buff->len = - rxd_wb->pkt_len >AQ_CFG_RX_FRAME_MAX ? -AQ_CFG_RX_FRAME_MAX : rxd_wb->pkt_len; + rxd_wb->pkt_len > ring->frame_max ? + ring->frame_max : rxd_wb->pkt_len;if (buff->is_lro) {/* LRO */
Signature · 1 line
2.17.1
PATCH v4 2/3 net: atlantic: Implement xdp data plane
In reply to Taehee YooIt supports XDP_PASS, XDP_DROP and multi buffer.
The new function aq_nic_xmit_xdpf() is used to send packet with xdp_frame and internally it calls aq_nic_map_xdp().
AQC chip supports 32 multi-queues and 8 vectors(irq). there are two option
1. under 8 cores and 4 tx queues per core.
2. under 4 cores and 8 tx queues per core.
Like ixgbe, these tx queues can be used only for XDP_TX, XDP_REDIRECT queue. If so, no tx_lock is needed.
But this patchset doesn't use this strategy because getting hardware tx queue index cost is too high.
So, tx_lock is used in the aq_nic_xmit_xdpf().
single-core, single queue, 80% cpu utilization.
30.75% bpf_prog_xxx_xdp_prog_tx [k] bpf_prog_xxx_xdp_prog_tx 10.35% [kernel] [k] aq_hw_read_reg <---------- here 4.38% [kernel] [k] get_page_from_freelist
single-core, 8 queues, 100% cpu utilization, half PPS.
45.56% [kernel] [k] aq_hw_read_reg <---------- here 17.58% bpf_prog_xxx_xdp_prog_tx [k] bpf_prog_xxx_xdp_prog_tx 4.72% [kernel] [k] hw_atl_b0_hw_ring_rx_receive
The new function __aq_ring_xdp_clean() is a xdp rx handler and this is called only when XDP is attached.
---
v4:
- Fix compile warning
v3:
- Change wrong PPS performance result 40% -> 80% in single core(Intel i3-12100)
- Separate aq_nic_map_xdp() from aq_nic_map_skb()
- Drop multi buffer packets if single buffer XDP is attached
- Use xdp_get_{frame/buff}_len()
v2:
- Do not use inline in C file
.../ethernet/aquantia/atlantic/aq_ethtool.c | 8 + .../net/ethernet/aquantia/atlantic/aq_nic.c | 137 +++++++ .../net/ethernet/aquantia/atlantic/aq_nic.h | 2 + .../net/ethernet/aquantia/atlantic/aq_ring.c | 335 +++++++++++++++++- .../net/ethernet/aquantia/atlantic/aq_ring.h | 6 + 5 files changed, 480 insertions(+), 8 deletions(-)
drivers/net/ethernet/aquantia/atlantic/aq_ethtool.c +8 -0
@@ -97,6 +97,14 @@ static const char * const aq_ethtool_queue_rx_stat_names[] = {"%sQueue[%d] AllocFails" ,"%sQueue[%d] SkbAllocFails" ,"%sQueue[%d] Polls" , +"%sQueue[%d] PageFlips" , +"%sQueue[%d] PageReuses" , +"%sQueue[%d] XdpAbort" , +"%sQueue[%d] XdpDrop" , +"%sQueue[%d] XdpPass" , +"%sQueue[%d] XdpTx" , +"%sQueue[%d] XdpInvalid" , + "%sQueue[%d] XdpRedirect", }; static const char * const aq_ethtool_queue_tx_stat_names[] = {
drivers/net/ethernet/aquantia/atlantic/aq_nic.c +137 -0
155 lines not on this page · raw message
drivers/net/ethernet/aquantia/atlantic/aq_nic.h +2 -0
@@ -180,6 +180,8 @@ void aq_nic_ndev_free(struct aq_nic_s *self);int aq_nic_start (struct aq_nic_s *self);unsigned int aq_nic_map_skb (struct aq_nic_s *self,struct sk_buff *skb,struct aq_ring_s *ring); +int aq_nic_xmit_xdpf (struct aq_nic_s *aq_nic,struct aq_ring_s *tx_ring, +struct xdp_frame *xdpf);int aq_nic_xmit (struct aq_nic_s *self,struct sk_buff *skb);int aq_nic_get_regs (struct aq_nic_s *self,struct ethtool_regs *regs,void *p);int aq_nic_get_regs_count (struct aq_nic_s *self);
drivers/net/ethernet/aquantia/atlantic/aq_ring.c +327 -8
371 lines not on this page · raw message
drivers/net/ethernet/aquantia/atlantic/aq_ring.h +6 -0
@@ -105,6 +105,12 @@ struct aq_ring_stats_rx_s {u64 pg_losts;u64 pg_flips;u64 pg_reuses; +u64 xdp_aborted; +u64 xdp_drop; +u64 xdp_pass; +u64 xdp_tx; +u64 xdp_invalid; +u64 xdp_redirect; };struct aq_ring_stats_tx_s {
Signature · 1 line
2.17.1
PATCH v4 3/3 net: atlantic: Implement .ndo_xdp_xmit handler
In reply to Taehee Yooaq_xdp_xmit() is the callback function of .ndo_xdp_xmit. It internally calls aq_nic_xmit_xdpf() to send packet.
---
v4:
- No changed
v3:
- No changed
v2:
- No changed
.../net/ethernet/aquantia/atlantic/aq_main.c | 1 + .../net/ethernet/aquantia/atlantic/aq_ring.c | 23 +++++++++++++++++++ .../net/ethernet/aquantia/atlantic/aq_ring.h | 2 ++ 3 files changed, 26 insertions(+)
drivers/net/ethernet/aquantia/atlantic/aq_main.c +1 -0
@@ -508,6 +508,7 @@ static const struct net_device_ops aq_ndev_ops = { .ndo_vlan_rx_kill_vid = aq_ndo_vlan_rx_kill_vid, .ndo_setup_tc = aq_ndo_setup_tc, .ndo_bpf = aq_xdp, + .ndo_xdp_xmit = aq_xdp_xmit, };static int __init aq_ndev_init_module (void )
drivers/net/ethernet/aquantia/atlantic/aq_ring.c +23 -0
@@ -401,6 +401,29 @@ static void aq_rx_checksum(struct aq_ring_s *self,__skb_incr_checksum_unnecessary (skb); } +int aq_xdp_xmit (struct net_device *dev,int num_frames, +struct xdp_frame **frames,u32 flags) +{ +struct aq_nic_s *aq_nic =netdev_priv (dev); +unsigned int vec, i, drop =0 ; +int cpu =smp_processor_id (); +struct aq_nic_cfg_s *aq_cfg; +struct aq_ring_s *ring; + + aq_cfg =aq_nic_get_cfg (aq_nic); + vec = cpu % aq_cfg->vecs; + ring = aq_nic->aq_ring_tx[AQ_NIC_CFG_TCVEC2RING (aq_cfg,0 , vec)]; + +for (i =0 ; i < num_frames; i++) { +struct xdp_frame *xdpf = frames[i]; + +if (aq_nic_xmit_xdpf (aq_nic, ring, xdpf) ==NETDEV_TX_BUSY ) + drop++; + } + +return num_frames - drop; +} + static struct sk_buff *aq_xdp_run_prog(struct aq_nic_s *aq_nic, struct xdp_buff *xdp, struct aq_ring_s *rx_ring,
drivers/net/ethernet/aquantia/atlantic/aq_ring.h +2 -0
@@ -197,6 +197,8 @@ void aq_ring_update_queue_state(struct aq_ring_s *ring);void aq_ring_queue_wake (struct aq_ring_s *ring);void aq_ring_queue_stop (struct aq_ring_s *ring);bool aq_ring_tx_clean (struct aq_ring_s *self); +int aq_xdp_xmit (struct net_device *dev,int num_frames, +struct xdp_frame **frames,u32 flags); int aq_ring_rx_clean(struct aq_ring_s *self, struct napi_struct *napi, int *work_done,
Signature · 1 line
2.17.1
Paolo Abeni
maintainerQuoted 85 lines · Taehee Yoo
This patchset is to make atlantic to support multi-buffer XDP.
The first patch implement control plane of xdp.
Theaq_xdp(), callback of .xdp_bpf is added.The second patch implements data plane of xdp.
XDP_TX,XDP_DROP, andXDP_PASSis supported.__aq_ring_xdp_clean()is added to receive and execute xdp program.aq_nic_xmit_xdpf()is added to send packet by XDP.The third patch implements callback of .ndo_xdp_xmit.
aq_xdp_xmit()is added to send redirected packets and it internally callsaq_nic_xmit_xdpf().Memory model is
MEM_TYPE_PAGE_ORDER0so it doesn't reuse rx page whenXDP_TX,XDP_PASS,XDP_REDIRECT.Default the maximum rx frame size is 2K.
If xdp is attached, size is changed to about 3K.
It can be reused whenXDP_DROP, andXDP_ABORTED.Atlantic driver has
AQ_CFG_RX_PAGEORDERoption and it will be always 0 if xdp is attached.LRO will be disabled if XDP program supports only single buffer.
AQC chip supports 32 multi-queues and 8 vectors(irq). There are two options.
1. under 8 cores and maximum 4 tx queues per core.
2. under 4 cores and maximum 8 tx queues per core.Like other drivers, these tx queues can be used only for
XDP_TX,XDP_REDIRECTqueue. If so, notx_lockis needed.
But this patchset doesn't use this strategy because getting hardware tx queue index cost is too high.
So,tx_lockis used in theaq_nic_xmit_xdpf().single-core, single queue, 80% cpu utilization.
30.75% bpf_prog_xxx_xdp_prog_tx [k] bpf_prog_xxx_xdp_prog_tx 10.35% [kernel] [k] aq_hw_read_reg <---------- here 4.38% [kernel] [k] get_page_from_freelistsingle-core, 8 queues, 100% cpu utilization, half PPS.
45.56% [kernel] [k] aq_hw_read_reg <---------- here 17.58% bpf_prog_xxx_xdp_prog_tx [k] bpf_prog_xxx_xdp_prog_tx 4.72% [kernel] [k] hw_atl_b0_hw_ring_rx_receivePerformance result(64 Byte)
1.XDP_TX
a.xdp_geieric, single core
- 2.5Mpps, 100% cpu
b.xdp_driver, single core
- 4.5Mpps, 80% cpu
c.xdp_generic, 8 core(hyper thread)
- 6.3Mpps, 5~10% cpu
d.xdp_driver, 8 core(hyper thread)
- 6.3Mpps, 5% cpu2.
XDP_REDIRECT
a.xdp_generic, single core
- 2.3Mpps
b.xdp_driver, single core
- 4.5Mppsv4:
- Fix compile warningv3:
- Change wrong PPS performance result 40% -> 80% in single core(Intel i3-12100)
- Separateaq_nic_map_xdp()fromaq_nic_map_skb()
- Drop multi buffer packets if single buffer XDP is attached
- Disable LRO when single buffer XDP is attached
- Use xdp_get_{frame/buff}_len()v2:
- Do not use inline in C fileTaehee Yoo (3):
net: atlantic: Implement xdp control plane
net: atlantic: Implement xdp data plane
net: atlantic: Implement .ndo_xdp_xmit handler
Quoted 14 lines
.../net/ethernet/aquantia/atlantic/aq_cfg.h | 1 + .../ethernet/aquantia/atlantic/aq_ethtool.c | 8 + .../net/ethernet/aquantia/atlantic/aq_main.c | 87 ++++ .../net/ethernet/aquantia/atlantic/aq_main.h | 2 + .../net/ethernet/aquantia/atlantic/aq_nic.c | 137 ++++++ .../net/ethernet/aquantia/atlantic/aq_nic.h | 5 + .../net/ethernet/aquantia/atlantic/aq_ring.c | 415 ++++++++++++++++-- .../net/ethernet/aquantia/atlantic/aq_ring.h | 17 + .../net/ethernet/aquantia/atlantic/aq_vec.c | 23 +- .../net/ethernet/aquantia/atlantic/aq_vec.h | 6 + .../aquantia/atlantic/hw_atl/hw_atl_a0.c | 6 +- .../aquantia/atlantic/hw_atl/hw_atl_b0.c | 10 +- 12 files changed, 675 insertions(+), 42 deletions(-)
@Igor: this should address you concerns on v2, could you please have a look?
Thanks!
Paolo
Quoted 5 lines
v4:
- Fix compile warning
Quoted 7 lines
v3:
- Change wrong PPS performance result 40% -> 80% in single core(Intel i3-12100)
- Separateaq_nic_map_xdp()fromaq_nic_map_skb()
- Drop multi buffer packets if single buffer XDP is attached
- Disable LRO when single buffer XDP is attached
- Use xdp_get_{frame/buff}_len()
Hi Taehee, thanks for taking care of that!
A small notice about the selection of 3K packet size for XDP.
Its a kind of compromise I think, because with common 1.4K MTU we'll get wasted 2K bytes minimum per packet.
I was thinking it would be possible to reuse the existing page flipping technique together with higher page_order, to keep default 2K fragment size. E.g.
( 256(xdp_head)+2K(pkt frag) ) x 3 (flips) = ~7K
Meaning we can allocate 8K (page_order=1) pages, and fit three xdp packets into each, wasting only 1K per three packets.
But its just kind of an idea for future optimization.
Regards,
Igor
2022. 4. 13. 오후 4:52에 Igor Russkikh 이(가) 쓴 글:
Hi Igor,
Thank you so much for your review!
Quoted 21 lines
Quoted 5 lines
v4:
- Fix compile warningQuoted 7 lines
v3:
- Change wrong PPS performance result 40% -> 80% in singlecore(Intel i3-12100)- Separate
aq_nic_map_xdp()fromaq_nic_map_skb()
- Drop multi buffer packets if single buffer XDP is attached
- Disable LRO when single buffer XDP is attached
- Use xdp_get_{frame/buff}_len()Hi Taehee, thanks for taking care of that!
Reviewed-by Igor Russkikh <[email protected]>A small notice about the selection of 3K packet size for XDP.
Its a kind of compromise I think, because with common 1.4K MTU we'll get wasted 2K bytes minimum per packet.
Quoted 9 lines
I was thinking it would be possible to reuse the existing page flipping technique together with higher
page_order, to keep default 2K fragment size. E.g.
( 256(xdp_head)+2K(pkt frag) ) x 3 (flips) = ~7KMeaning we can allocate 8K (page_order=1) pages, and fit three xdp packets into each, wasting only 1K per three packets.
But its just kind of an idea for future optimization.
Yes, I fully agree with your idea.
When I developed an initial version of this patchset, I simply tried that idea.
I expected to reduce CPU utilization(not for memory optimization), but there is no difference because page_ref_{inc/dec}() cost is too high. So, if we tried to switch from MEM_TYPE_PAGE_ORDER0 to MEM_TYPE_PAGE_SHARED, I think we should use a littie bit different flipping strategy like ixgbe. If so, we would achieve memory optimization and CPU optimization.
Thanks a lot,
Taehee Yoo
Quoted 2 lines
Regards,
Igor