Introduce AF_XDP PMD
Subsystems (2)
cover Introduce AF_XDP PMD
Overview
========
This patchset adds a new PMD driver for AF_XDP which is a proposed faster version of AF_PACKET interface in Linux, see below links [1] [2] for details of AF_XDP introduction:
AF_XDP roadmap
==============
- AF_XDP is included in upstream kernel since 4.18, and AF_XDP support in libbpf has been merged in bpf-next/master. [3]
- Now i40e and ixgbe drivers have supported zero copy mode.
Change logs
===========
changes vs RFC sent by Qi last Aug:
- Re-work base on AF_XDP's interface changes since the new libbpf has provided higher-level APIs that hide many of the details of the AP_XDP uapi. After the rework, it helps to reduce 300+ lines of code.
- multi-queues is not supported due to current libbpf doesn't support multi-sockets on a single umem.
- No extra steps to load xdp program manually, since the current behavior of libbpf would load a default xdp program when user calls xsk_socket__create. userspace application only needs to handle the cleanup.
How to try
==========
1. take the latest bpf-next/master, build kernel and replace your host kernel with it.
make sure you turn on XDP sockets when compiling
Networking support -->
Networking options -->
[ * ] XDP sockets
2. build libbpf in tools/lib/bpf, and copy the libbpf.a and libbpf.so to /usr/lib64
cd tools/lib/bpf
make
3. extra steps to build dpdk
Add below lines to drivers/net/af_xdp/Makefile
CFLAGS += -I/<your linux src root>/tools/include CFLAGS += -I/<your linux src root>/tools/lib/bpf
add below line to config/common_linuxapp to enble the config for af_xdp pmd
CONFIG_RTE_LIBRTE_PMD_AF_XDP=y
4. start testpmd
./build/app/testpmd -c 0xc -n 4 --vdev eth_af_xdp,iface=enp59s0f0,queue=0 -- -i --rxq=1 --txq=1
in this case, default xdp program will be loaded and linked to queue 0 of enp59s0f0, network traffics travel to queue 0 will be redirected to af_xdp socket.
[1] https://lwn.net/Articles/750845/
[2] https://fosdem.org/2018/schedule/event/af_xdp/
[3] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git/commit/?id=143bdc2e27b44d2559596424bfb017d578be33eb
Xiaolong Ye (6):
net/af_xdp: introduce AF_XDP PMD driver lib/mbuf: enable parse flags when create mempool lib/mempool: allow page size aligned mempool net/af_xdp: use mbuf mempool for buffer management net/af_xdp: enable zero copy
app/testpmd: add mempool flags parameter
MAINTAINERS | 6 + app/test-pmd/parameters.c | 12 + app/test-pmd/testpmd.c | 17 +- app/test-pmd/testpmd.h | 1 + config/common_base | 5 + doc/guides/nics/af_xdp.rst | 43 + doc/guides/rel_notes/release_18_11.rst | 7 + drivers/net/Makefile | 1 + drivers/net/af_xdp/Makefile | 31 + drivers/net/af_xdp/meson.build | 7 + drivers/net/af_xdp/rte_eth_af_xdp.c | 988 ++++++++++++++++++ drivers/net/af_xdp/rte_pmd_af_xdp_version.map | 4 + lib/librte_mbuf/rte_mbuf.c | 15 +- lib/librte_mbuf/rte_mbuf.h | 8 +- lib/librte_mempool/rte_mempool.c | 3 + lib/librte_mempool/rte_mempool.h | 1 + mk/rte.app.mk | 1 + 17 files changed, 1139 insertions(+), 11 deletions(-) create mode 100644 doc/guides/nics/af_xdp.rst create mode 100644 drivers/net/af_xdp/Makefile create mode 100644 drivers/net/af_xdp/meson.build create mode 100644 drivers/net/af_xdp/rte_eth_af_xdp.c create mode 100644 drivers/net/af_xdp/rte_pmd_af_xdp_version.map
Signature · 1 line
2.17.1
PATCH 1/6 net/af_xdp: introduce AF_XDP PMD driver
Add a new PMD driver for AF_XDP which is a proposed faster version of AF_PACKET interface in Linux. More info about AF_XDP, please refer to [1] [2].
This is the vanilla version PMD which just uses a raw buffer registered as the umem.
[1] https://fosdem.org/2018/schedule/event/af_xdp/
[2] https://lwn.net/Articles/745934/
--- MAINTAINERS | 6 + config/common_base | 5 + doc/guides/nics/af_xdp.rst | 43 + doc/guides/rel_notes/release_18_11.rst | 7 + drivers/net/Makefile | 1 + drivers/net/af_xdp/Makefile | 31 + drivers/net/af_xdp/meson.build | 7 + drivers/net/af_xdp/rte_eth_af_xdp.c | 903 ++++++++++++++++++ drivers/net/af_xdp/rte_pmd_af_xdp_version.map | 4 + mk/rte.app.mk | 1 + 10 files changed, 1008 insertions(+) create mode 100644 doc/guides/nics/af_xdp.rst create mode 100644 drivers/net/af_xdp/Makefile create mode 100644 drivers/net/af_xdp/meson.build create mode 100644 drivers/net/af_xdp/rte_eth_af_xdp.c create mode 100644 drivers/net/af_xdp/rte_pmd_af_xdp_version.map
MAINTAINERS +6 -0
@@ -469,6 +469,12 @@ M: John W. Linville <[email protected]> F: drivers/net/af_packet/ F: doc/guides/nics/features/afpacket.ini +Linux AF_XDP +M: Xiaolong Ye <[email protected]> +M: Qi Zhang <[email protected]> +F: drivers/net/af_xdp/ +F: doc/guides/nics/features/af_xdp.rst + Amazon ENA M: Marcin Wojtas <[email protected]> M: Michal Krawczyk <[email protected]>
config/common_base +5 -0
@@ -416,6 +416,11 @@ CONFIG_RTE_LIBRTE_VMXNET3_DEBUG_TX_FREE=n # CONFIG_RTE_LIBRTE_PMD_AF_PACKET=n +# +# Compile software PMD backed by AF_XDP sockets (Linux only) +# +CONFIG_RTE_LIBRTE_PMD_AF_XDP=n + # # Compile link bonding PMD library #
doc/guides/nics/af_xdp.rst new file +43 -0
43 lines not on this page · raw message
doc/guides/rel_notes/release_18_11.rst +7 -0
@@ -229,6 +229,13 @@ New Features The AESNI MB PMD has been updated with additional support for the AES-GCM algorithm. +* **Added the AF_XDP PMD.** + + Added a Linux-specific PMD driver for AF_XDP, it can create the AF_XDP socket + and bind it to a specific netdev queue, it allows a DPDK application to send + and receive raw packets through the socket which would bypass the kernel + network stack to achieve high performance packet processing. + * **Added NXP CAAM JR PMD.** Added the new caam job ring driver for NXP platforms. See the
drivers/net/Makefile +1 -0
@@ -9,6 +9,7 @@ ifeq ($(CONFIG_RTE_LIBRTE_THUNDERX_NICVF_PMD),d) endif DIRS-$(CONFIG_RTE_LIBRTE_PMD_AF_PACKET) += af_packet +DIRS-$(CONFIG_RTE_LIBRTE_PMD_AF_XDP) += af_xdp DIRS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += ark DIRS-$(CONFIG_RTE_LIBRTE_ATLANTIC_PMD) += atlantic DIRS-$(CONFIG_RTE_LIBRTE_AVF_PMD) += avf
drivers/net/af_xdp/Makefile new file +31 -0
@@ -0,0 +1,31 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Intel Corporation + +include $(RTE_SDK)/mk/rte.vars.mk + +# +# library name +# +LIB = librte_pmd_af_xdp.a + +EXPORT_MAP := rte_pmd_af_xdp_version.map + +LIBABIVER := 1 + + +CFLAGS += -O3 +# below line should be removed +CFLAGS += -I/root/yexl/shared_mks0/linux/tools/include +CFLAGS += -I/root/yexl/shared_mks0/linux/tools/lib/bpf + +CFLAGS += $(WERROR_FLAGS) +LDLIBS += -lrte_eal -lrte_mbuf -lrte_mempool -lrte_ring +LDLIBS += -lrte_ethdev -lrte_net -lrte_kvargs +LDLIBS += -lrte_bus_vdev + +# +# all source are stored in SRCS-y +# +SRCS-$(CONFIG_RTE_LIBRTE_PMD_AF_XDP) += rte_eth_af_xdp.c + +include $(RTE_SDK)/mk/rte.lib.mk
drivers/net/af_xdp/meson.build new file +7 -0
@@ -0,0 +1,7 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Intel Corporation + +if host_machine.system() != 'linux' + build = false +endif +sources = files('rte_eth_af_xdp.c')
drivers/net/af_xdp/rte_eth_af_xdp.c new file +903 -0
903 lines not on this page · raw message
drivers/net/af_xdp/rte_pmd_af_xdp_version.map new file +4 -0
@@ -0,0 +1,4 @@ +DPDK_2.0 { + + local: *; +};
mk/rte.app.mk +1 -0
@@ -143,6 +143,7 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_DPAA2_MEMPOOL) += -lrte_mempool_dpaa2 endif _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AF_PACKET) += -lrte_pmd_af_packet +_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AF_XDP) += -lrte_pmd_af_xdp -lelf -lbpf _LDLIBS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += -lrte_pmd_ark _LDLIBS-$(CONFIG_RTE_LIBRTE_ATLANTIC_PMD) += -lrte_pmd_atlantic _LDLIBS-$(CONFIG_RTE_LIBRTE_AVF_PMD) += -lrte_pmd_avf
Signature · 1 line
2.17.1
Re: [PATCH v1 1/6] net/af_xdp: introduce AF_XDP PMD driver
Quoted 32 lines · Xiaolong Ye
Add a new PMD driver for
AF_XDPwhich is a proposed faster version ofAF_PACKETinterface in Linux. More info aboutAF_XDP, please refer to
[1]
[2].This is the vanilla version PMD which just uses a raw buffer registered as
the umem.[1] https://fosdem.org/2018/schedule/event/af_xdp/
[2] https://lwn.net/Articles/745934/Signed-off-by Xiaolong Ye <[email protected]>--- MAINTAINERS | 6 + config/common_base | 5 + doc/guides/nics/af_xdp.rst | 43 + doc/guides/rel_notes/release_18_11.rst | 7 + drivers/net/Makefile | 1 + drivers/net/af_xdp/Makefile | 31 + drivers/net/af_xdp/meson.build | 7 + drivers/net/af_xdp/rte_eth_af_xdp.c | 903 ++++++++++++++++++ drivers/net/af_xdp/rte_pmd_af_xdp_version.map | 4 + mk/rte.app.mk | 1 + 10 files changed, 1008 insertions(+) create mode 100644 doc/guides/nics/af_xdp.rst create mode 100644 drivers/net/af_xdp/Makefile create mode 100644 drivers/net/af_xdp/meson.build create mode 100644 drivers/net/af_xdp/rte_eth_af_xdp.c create mode 100644 drivers/net/af_xdp/rte_pmd_af_xdp_version.map
<..>
drivers/net/Makefile ifeq ($(CONFIG_RTE_LIBRTE_THUNDERX_NICVF_PMD),d) · line 15
Quoted 10 lines
drivers/net/Makefile +1
-0@@ -9,6 +9,7 @@ ifeq ($(CONFIG_RTE_LIBRTE_THUNDERX_NICVF_PMD),d) endif DIRS-$(CONFIG_RTE_LIBRTE_PMD_AF_PACKET) += af_packet +DIRS-$(CONFIG_RTE_LIBRTE_PMD_AF_XDP) += af_xdp DIRS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += ark DIRS-$(CONFIG_RTE_LIBRTE_ATLANTIC_PMD) += atlantic DIRS-$(CONFIG_RTE_LIBRTE_AVF_PMD) += avf
You need a similar change in drivers/net/meson.build
Quoted 16 lines
drivers/net/af_xdp/Makefile new file +9
-0@@ -0,0 +1,31 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Intel Corporation + +include $(RTE_SDK)/mk/rte.vars.mk + +# +# library name +# +LIB = librte_pmd_af_xdp.a
drivers/net/af_xdp/Makefile line 19
Quoted 10 lines
drivers/net/af_xdp/Makefile new file +10
-0+ +EXPORT_MAP := rte_pmd_af_xdp_version.map + +LIBABIVER := 1 + + +CFLAGS += -O3 +# below line should be removed +CFLAGS += -I/root/yexl/shared_mks0/linux/tools/include +CFLAGS += -I/root/yexl/shared_mks0/linux/tools/lib/bpf
Leftovers?
Quoted 18 lines
drivers/net/af_xdp/Makefile new file +11
-0+CFLAGS += $(WERROR_FLAGS) +LDLIBS += -lrte_eal -lrte_mbuf -lrte_mempool -lrte_ring +LDLIBS += -lrte_ethdev -lrte_net -lrte_kvargs +LDLIBS += -lrte_bus_vdev + +# +# all source are stored in SRCS-y +# +SRCS-$(CONFIG_RTE_LIBRTE_PMD_AF_XDP) += rte_eth_af_xdp.c + +include $(RTE_SDK)/mk/rte.lib.mkdrivers/net/af_xdp/meson.build new file +0
-0
drivers/net/af_xdp/meson.build line 7
Quoted 7 lines
drivers/net/af_xdp/meson.build new file +7
-0@@ -0,0 +1,7 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Intel Corporation + +if host_machine.system() != 'linux' + build = false +endif +sources = files('rte_eth_af_xdp.c')
You need to add a dependency() on libbpf, and given upstream doesn't ship a pkg-config file yet a fallback to find_library, so that the pmd can be linked against it.
Check other PMDs to see how dependencies are handled, like drivers/net/pcap/meson.build
<..>
drivers/net/af_xdp/rte_pmd_af_xdp_version.map line 1
Quoted 8 lines
drivers/net/af_xdp/rte_pmd_af_xdp_version.map new file +1
-0@@ -0,0 +1,4 @@ +DPDK_2.0 {
2.0 is a bit old :-)
Quoted 8 lines
drivers/net/af_xdp/rte_pmd_af_xdp_version.map new file +3
-0+ + local: *; +};mk/rte.app.mk +0
-0
mk/rte.app.mk _LDLIBS-$(CONFIG_RTE_LIBRTE_DPAA2_MEMPOOL) += · line 147
Quoted 6 lines
mk/rte.app.mk +1
-2@@ -143,6 +143,7 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_DPAA2_MEMPOOL) += -lrte_mempool_dpaa2 endif _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AF_PACKET) += -lrte_pmd_af_packet +_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AF_XDP) += -lrte_pmd_af_xdp -lelf -lbpf
Are symbols from libelf being used by the PMD?
Quoted 3 lines
mk/rte.app.mk +0
-0_LDLIBS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += -lrte_pmd_ark _LDLIBS-$(CONFIG_RTE_LIBRTE_ATLANTIC_PMD) += -lrte_pmd_atlantic _LDLIBS-$(CONFIG_RTE_LIBRTE_AVF_PMD) += -lrte_pmd_avf
Signature · 2 lines
Kind regards, Luca Boccassi
Re: [PATCH v1 1/6] net/af_xdp: introduce AF_XDP PMD driver
Hi, Luca
Thanks for your review.
Quoted 40 lines · Luca Boccassi
Quoted 32 lines · Xiaolong Ye
Add a new PMD driver for
AF_XDPwhich is a proposed faster version ofAF_PACKETinterface in Linux. More info aboutAF_XDP, please refer to
[1]
[2].This is the vanilla version PMD which just uses a raw buffer registered as
the umem.[1] https://fosdem.org/2018/schedule/event/af_xdp/
[2] https://lwn.net/Articles/745934/Signed-off-by Xiaolong Ye <[email protected]>--- MAINTAINERS | 6 + config/common_base | 5 + doc/guides/nics/af_xdp.rst | 43 + doc/guides/rel_notes/release_18_11.rst | 7 + drivers/net/Makefile | 1 + drivers/net/af_xdp/Makefile | 31 + drivers/net/af_xdp/meson.build | 7 + drivers/net/af_xdp/rte_eth_af_xdp.c | 903 ++++++++++++++++++ drivers/net/af_xdp/rte_pmd_af_xdp_version.map | 4 + mk/rte.app.mk | 1 + 10 files changed, 1008 insertions(+) create mode 100644 doc/guides/nics/af_xdp.rst create mode 100644 drivers/net/af_xdp/Makefile create mode 100644 drivers/net/af_xdp/meson.build create mode 100644 drivers/net/af_xdp/rte_eth_af_xdp.c create mode 100644 drivers/net/af_xdp/rte_pmd_af_xdp_version.map<..>
Quoted 5 lines
drivers/net/Makefile +0
-0@@ -9,6 +9,7 @@ ifeq ($(CONFIG_RTE_LIBRTE_THUNDERX_NICVF_PMD),d) endif
Quoted 7 lines
drivers/net/Makefile ifeq ($(CONFIG_RTE_LIBRTE_THUNDERX_NICVF_PMD),d) · line 15
Quoted 5 lines
drivers/net/Makefile +1
-0DIRS-$(CONFIG_RTE_LIBRTE_PMD_AF_PACKET) += af_packet +DIRS-$(CONFIG_RTE_LIBRTE_PMD_AF_XDP) += af_xdp DIRS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += ark DIRS-$(CONFIG_RTE_LIBRTE_ATLANTIC_PMD) += atlantic DIRS-$(CONFIG_RTE_LIBRTE_AVF_PMD) += avfYou need a similar change in
drivers/net/meson.build
will do.
Quoted 21 lines
Quoted 21 lines
drivers/net/af_xdp/Makefile new file +13
-0@@ -0,0 +1,31 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Intel Corporation + +include $(RTE_SDK)/mk/rte.vars.mk + +# +# library name +# +LIB = librte_pmd_af_xdp.a + +EXPORT_MAP := rte_pmd_af_xdp_version.map + +LIBABIVER := 1
Quoted 9 lines
drivers/net/af_xdp/Makefile line 19
Quoted 6 lines
drivers/net/af_xdp/Makefile new file +6
-0+ + +CFLAGS += -O3 +# below line should be removed +CFLAGS += -I/root/yexl/shared_mks0/linux/tools/include +CFLAGS += -I/root/yexl/shared_mks0/linux/tools/lib/bpfLeftovers?
Yes, will remove in next version.
Quoted 23 lines
Quoted 23 lines
drivers/net/af_xdp/Makefile new file +11
-0+CFLAGS += $(WERROR_FLAGS) +LDLIBS += -lrte_eal -lrte_mbuf -lrte_mempool -lrte_ring +LDLIBS += -lrte_ethdev -lrte_net -lrte_kvargs +LDLIBS += -lrte_bus_vdev + +# +# all source are stored in SRCS-y +# +SRCS-$(CONFIG_RTE_LIBRTE_PMD_AF_XDP) += rte_eth_af_xdp.c + +include $(RTE_SDK)/mk/rte.lib.mkdrivers/net/af_xdp/meson.build new file +5
-0@@ -0,0 +1,7 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Intel Corporation + +if host_machine.system() != 'linux' + build = false
Quoted 9 lines
drivers/net/af_xdp/meson.build line 7
Quoted 2 lines
drivers/net/af_xdp/meson.build new file +2
-0+endif +sources = files('rte_eth_af_xdp.c')You need to add a
dependency()on libbpf, and given upstream doesn't ship a pkg-config file yet a fallback tofind_library, so that the pmd can be linked against it.
Check other PMDs to see how dependencies are handled, likedrivers/net/pcap/meson.build
will check the example to see how to handle it correctly.
Quoted 12 lines
<..>
drivers/net/af_xdp/rte_pmd_af_xdp_version.map line 1
Quoted 8 lines
drivers/net/af_xdp/rte_pmd_af_xdp_version.map new file +1
-0@@ -0,0 +1,4 @@ +DPDK_2.0 {2.0 is a bit old :-)
will change it.
Quoted 12 lines
Quoted 12 lines
drivers/net/af_xdp/rte_pmd_af_xdp_version.map new file +3
-0+ + local: *; +};mk/rte.app.mk +0
-1@@ -143,6 +143,7 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_DPAA2_MEMPOOL) += -lrte_mempool_dpaa2 endif
Quoted 5 lines
mk/rte.app.mk _LDLIBS-$(CONFIG_RTE_LIBRTE_DPAA2_MEMPOOL) += · line 147
Quoted 3 lines
mk/rte.app.mk +1
-1_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AF_PACKET) += -lrte_pmd_af_packet +_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AF_XDP) += -lrte_pmd_af_xdp -lelf -lbpfAre symbols from libelf being used by the PMD?
Hmm, it is a leftover of RFC, libelf is no longer needed in this version, will remove it in next version.
Thanks,
Xiaolong
Quoted 8 lines
Quoted 4 lines
mk/rte.app.mk +0
-0_LDLIBS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += -lrte_pmd_ark _LDLIBS-$(CONFIG_RTE_LIBRTE_ATLANTIC_PMD) += -lrte_pmd_atlantic _LDLIBS-$(CONFIG_RTE_LIBRTE_AVF_PMD) += -lrte_pmd_avfSignature · 2 lines
Kind regards, Luca Boccassi
Re: [PATCH v1 1/6] net/af_xdp: introduce AF_XDP PMD driver
Quoted 9 lines · Ye Xiaolong
Quoted 5 lines
Review
Quoted 3 lines
_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AF_PACKET) += -lrte_pmd_af_packet
(quoted diff) +1
-1+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AF_XDP) += -lrte_pmd_af_xdp -lelf -lbpfAre symbols from libelf being used by the PMD?
Hmm, it is a leftover of RFC, libelf is no longer needed in this version, will remove it in next version.
Correction, libelf is needed for libbpf, so we still need to keep it.
Thanks,
Xiaolong
Quoted 10 lines
Thanks,
XiaolongQuoted 8 lines
Quoted 4 lines
_LDLIBS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += -lrte_pmd_ark _LDLIBS-$(CONFIG_RTE_LIBRTE_ATLANTIC_PMD) += -lrte_pmd_atlantic _LDLIBS-$(CONFIG_RTE_LIBRTE_AVF_PMD) += -lrte_pmd_avfSignature · 2 lines
Kind regards, Luca Boccassi
Re: [PATCH v1 1/6] net/af_xdp: introduce AF_XDP PMD driver
Quoted 9 lines · Ye Xiaolong
Quoted 8 lines · Ye Xiaolong
Quoted 7 lines
Review
Quoted 5 lines
_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AF_PACKET) +=
(quoted diff) +1
-3-lrte_pmd_af_packet +_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AF_XDP) += -lrte_pmd_af_xdp -lelf -lbpfAre symbols from libelf being used by the PMD?
Quoted 7 lines
Quoted 4 lines
Hmm, it is a leftover of RFC, libelf is no longer needed in this version, will
remove it in next version.Correction, libelf is needed for libbpf, so we still need to keep it.
If libbpf needs libelf for its internal usage, it should be linked against it itself. Unless symbols from libelf are used in static functions defined in libbpf's public headers. Is this the case?
Signature · 2 lines
Kind regards, Luca Boccassi
Re: [PATCH v1 1/6] net/af_xdp: introduce AF_XDP PMD driver
Quoted 15 lines · Luca Boccassi
Quoted 14 lines · Ye Xiaolong
Quoted 12 lines · Ye Xiaolong
Quoted 7 lines
Review
Quoted 5 lines
_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AF_PACKET) +=
(quoted diff) +1
-3-lrte_pmd_af_packet +_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AF_XDP) += -lrte_pmd_af_xdp -lelf -lbpfAre symbols from libelf being used by the PMD?
Hmm, it is a leftover of RFC, libelf is no longer needed in this version, will
remove it in next version.
Quoted 7 lines
Quoted 2 lines
Correction, libelf is needed for libbpf, so we still need to keep it.
If libbpf needs libelf for its internal usage, it should be linked against it itself. Unless symbols from libelf are used in static functions defined in libbpf's public headers. Is this the case?
Yes, that's the case. without the libelf, it would have build error as below, and these symbols are used in static functions of libbpf.
/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/libbpf.so: undefined reference to elf_nextscn' /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/libbpf.so: undefined reference to elf_rawdata' /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/libbpf.so: undefined reference to elf_memory' /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/libbpf.so: undefined reference to gelf_getrel' /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/libbpf.so: undefined reference to elf_strptr' /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/libbpf.so: undefined reference to elf_end' /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/libbpf.so: undefined reference to elf_getscn' /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/libbpf.so: undefined reference to elf_begin' /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/libbpf.so: undefined reference to gelf_getsym' /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/libbpf.so: undefined reference to elf_version' /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/libbpf.so: undefined reference to gelf_getehdr' /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/libbpf.so: undefined reference to elf_getdata' /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/libbpf.so: undefined reference to `gelf_getshdr'
Thanks,
Xiaolong
Quoted 3 lines
Signature · 2 lines
Kind regards, Luca Boccassi
Re: [PATCH v1 1/6] net/af_xdp: introduce AF_XDP PMD driver
Quoted 56 lines · Ye Xiaolong
Quoted 16 lines · Luca Boccassi
Quoted 15 lines · Ye Xiaolong
Quoted 13 lines · Ye Xiaolong
Quoted 7 lines
Review
Quoted 5 lines
_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AF_PACKET) +=
(quoted diff) +1
-3-lrte_pmd_af_packet +_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AF_XDP) += -lrte_pmd_af_xdp -lelf -lbpfAre symbols from libelf being used by the PMD?
Hmm, it is a leftover of RFC, libelf is no longer needed in this
version, will
remove it in next version.Quoted 7 lines
Quoted 2 lines
Correction, libelf is needed for libbpf, so we still need to keep it.
If libbpf needs libelf for its internal usage, it should be linked against it itself. Unless symbols from libelf are used in static functions defined in libbpf's public headers. Is this the case?
Yes, that's the case. without the libelf, it would have build error as below,
and these symbols are used in static functions of libbpf.
/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/libbpf.so:
undefined reference to `elf_nextscn'/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/libbpf.so:
undefined reference to `elf_rawdata'/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/libbpf.so:
undefined reference to `elf_memory'/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/libbpf.so:
undefined reference to `gelf_getrel'/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/libbpf.so:
undefined reference to `elf_strptr'/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/libbpf.so:
undefined reference to `elf_end'/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/libbpf.so:
undefined reference to `elf_getscn'/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/libbpf.so:
undefined reference to `elf_begin'/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/libbpf.so:
undefined reference to `gelf_getsym'/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/libbpf.so:
undefined reference to `elf_version'/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/libbpf.so:
undefined reference to `gelf_getehdr'/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/libbpf.so:
undefined reference to `elf_getdata'/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/libbpf.so:
undefined reference to `gelf_getshdr'
Quoted 2 lines
Thanks,
Xiaolong
Looking at that list, at least the very first symbol is not used by a public header, but internally in libbpf:
$ grep -r elf_nextscn
libbpf.c: while ((scn = elf_nextscn(elf, scn)) != NULL) {https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git/tree/tools/lib/bpf/libbpf.c#n793
None of those symbols are used from the public headers:
$ grep elf_ bpf.h libbpf.h btf.h $
Therefore, this is an omission in libbpf's Makefile, as it must link against libelf. Please raise a ticket or send a patch upstream, and remove the -lelf from DPDK's makefiles.
Signature · 2 lines
Kind regards, Luca Boccassi
Re: [PATCH v1 1/6] net/af_xdp: introduce AF_XDP PMD driver
Quoted 68 lines · Luca Boccassi
Quoted 56 lines · Ye Xiaolong
Quoted 16 lines · Luca Boccassi
Quoted 15 lines · Ye Xiaolong
Quoted 13 lines · Ye Xiaolong
Quoted 7 lines
Review
Quoted 5 lines
_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AF_PACKET) +=
(quoted diff) +1
-3-lrte_pmd_af_packet +_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AF_XDP) += -lrte_pmd_af_xdp -lelf -lbpfAre symbols from libelf being used by the PMD?
Hmm, it is a leftover of RFC, libelf is no longer needed in this
version, will
remove it in next version.Quoted 7 lines
Quoted 2 lines
Correction, libelf is needed for libbpf, so we still need to keep it.
If libbpf needs libelf for its internal usage, it should be linked against it itself. Unless symbols from libelf are used in static functions defined in libbpf's public headers. Is this the case?
Yes, that's the case. without the libelf, it would have build error as below,
and these symbols are used in static functions of libbpf.
/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/libbpf.so:
undefined reference to `elf_nextscn'/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/libbpf.so:
undefined reference to `elf_rawdata'/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/libbpf.so:
undefined reference to `elf_memory'/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/libbpf.so:
undefined reference to `gelf_getrel'/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/libbpf.so:
undefined reference to `elf_strptr'/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/libbpf.so:
undefined reference to `elf_end'/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/libbpf.so:
undefined reference to `elf_getscn'/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/libbpf.so:
undefined reference to `elf_begin'/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/libbpf.so:
undefined reference to `gelf_getsym'/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/libbpf.so:
undefined reference to `elf_version'/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/libbpf.so:
undefined reference to `gelf_getehdr'/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/libbpf.so:
undefined reference to `elf_getdata'/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/libbpf.so:
undefined reference to `gelf_getshdr'Quoted 2 lines
Thanks,
XiaolongLooking at that list, at least the very first symbol is not used by a public header, but internally in libbpf:
$ grep -r elf_nextscn libbpf.c: while ((scn = elf_nextscn(elf, scn)) != NULL) {https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git/tree/tools/lib/bpf/libbpf.c#n793
Quoted 8 lines
None of those symbols are used from the public headers:
$ grep elf_ bpf.h libbpf.h btf.h $Therefore, this is an omission in libbpf's
Makefile, as it must link against libelf. Please raise a ticket or send a patch upstream, and remove the -lelf from DPDK's makefiles.
Hi, Magnus, could you help handle this?
Thanks,
Xiaolong
Quoted 4 lines
Signature · 3 lines
-- Kind regards, Luca Boccassi
Re: [PATCH v1 1/6] net/af_xdp: introduce AF_XDP PMD driver
In reply to Luca BoccassiQuoted 68 lines · Luca Boccassi
Quoted 56 lines · Ye Xiaolong
Quoted 16 lines · Luca Boccassi
Quoted 15 lines · Ye Xiaolong
Quoted 13 lines · Ye Xiaolong
Quoted 7 lines
Review
Quoted 5 lines
_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AF_PACKET) +=
(quoted diff) +1
-3-lrte_pmd_af_packet +_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AF_XDP) += -lrte_pmd_af_xdp -lelf -lbpfAre symbols from libelf being used by the PMD?
Hmm, it is a leftover of RFC, libelf is no longer needed in this
version, will
remove it in next version.Quoted 7 lines
Quoted 2 lines
Correction, libelf is needed for libbpf, so we still need to keep it.
If libbpf needs libelf for its internal usage, it should be linked against it itself. Unless symbols from libelf are used in static functions defined in libbpf's public headers. Is this the case?
Yes, that's the case. without the libelf, it would have build error as below,
and these symbols are used in static functions of libbpf.
/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/libbpf.so:
undefined reference to `elf_nextscn'/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/libbpf.so:
undefined reference to `elf_rawdata'/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/libbpf.so:
undefined reference to `elf_memory'/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/libbpf.so:
undefined reference to `gelf_getrel'/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/libbpf.so:
undefined reference to `elf_strptr'/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/libbpf.so:
undefined reference to `elf_end'/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/libbpf.so:
undefined reference to `elf_getscn'/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/libbpf.so:
undefined reference to `elf_begin'/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/libbpf.so:
undefined reference to `gelf_getsym'/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/libbpf.so:
undefined reference to `elf_version'/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/libbpf.so:
undefined reference to `gelf_getehdr'/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/libbpf.so:
undefined reference to `elf_getdata'/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/libbpf.so:
undefined reference to `gelf_getshdr'Quoted 2 lines
Thanks,
XiaolongLooking at that list, at least the very first symbol is not used by a public header, but internally in libbpf:
$ grep -r elf_nextscn libbpf.c: while ((scn = elf_nextscn(elf, scn)) != NULL) {https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git/tree/tools/lib/bpf/libbpf.c#n793
Quoted 8 lines
None of those symbols are used from the public headers:
$ grep elf_ bpf.h libbpf.h btf.h $Therefore, this is an omission in libbpf's
Makefile, as it must link against libelf. Please raise a ticket or send a patch upstream, and remove the -lelf from DPDK's makefiles.
As it may need sometime for kernel community to handle the libbpf's Makefile, We may still need the -lelf for af_xdp pmd currently, I'll remove it later after libbpf correct to link against libelf. Is it acceptable?
Thanks,
Xiaolong
Quoted 4 lines
Signature · 3 lines
-- Kind regards, Luca Boccassi
Re: [PATCH v1 1/6] net/af_xdp: introduce AF_XDP PMD driver
Quoted 97 lines · Ye Xiaolong
Quoted 88 lines · Luca Boccassi
Quoted 73 lines · Ye Xiaolong
Quoted 20 lines · Luca Boccassi
Quoted 15 lines · Ye Xiaolong
Quoted 8 lines · Ye Xiaolong
Quoted 7 lines
Review
Quoted 5 lines
_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AF_PACKET) +=
(quoted diff) +1
-3-lrte_pmd_af_packet +_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AF_XDP) += -lrte_pmd_af_xdp -lelf -lbpfAre symbols from libelf being used by the PMD?
Quoted 5 lines
Hmm, it is a leftover of RFC, libelf is no longer needed in this
version, will
remove it in next version.Quoted 3 lines
Correction, libelf is needed for libbpf, so we still need to keep
it.Quoted 6 lines
If libbpf needs libelf for its internal usage, it should be linked
against it itself. Unless symbols from libelf are used in static
functions defined in libbpf's public headers. Is this the case?Yes, that's the case. without the libelf, it would have build error
as below,
and these symbols are used in static functions of libbpf.
/usr/lib/gcc/x86_64-redhat-
linux/4.8.5/../../../../lib64/libbpf.so:
undefined reference to `elf_nextscn'/usr/lib/gcc/x86_64-redhat-
linux/4.8.5/../../../../lib64/libbpf.so:
undefined reference to `elf_rawdata'/usr/lib/gcc/x86_64-redhat-
linux/4.8.5/../../../../lib64/libbpf.so:
undefined reference to `elf_memory'/usr/lib/gcc/x86_64-redhat-
linux/4.8.5/../../../../lib64/libbpf.so:
undefined reference to `gelf_getrel'/usr/lib/gcc/x86_64-redhat-
linux/4.8.5/../../../../lib64/libbpf.so:
undefined reference to `elf_strptr'/usr/lib/gcc/x86_64-redhat-
linux/4.8.5/../../../../lib64/libbpf.so:
undefined reference to `elf_end'/usr/lib/gcc/x86_64-redhat-
linux/4.8.5/../../../../lib64/libbpf.so:
undefined reference to `elf_getscn'/usr/lib/gcc/x86_64-redhat-
linux/4.8.5/../../../../lib64/libbpf.so:
undefined reference to `elf_begin'/usr/lib/gcc/x86_64-redhat-
linux/4.8.5/../../../../lib64/libbpf.so:
undefined reference to `gelf_getsym'/usr/lib/gcc/x86_64-redhat-
linux/4.8.5/../../../../lib64/libbpf.so:
undefined reference to `elf_version'/usr/lib/gcc/x86_64-redhat-
linux/4.8.5/../../../../lib64/libbpf.so:
undefined reference to `gelf_getehdr'/usr/lib/gcc/x86_64-redhat-
linux/4.8.5/../../../../lib64/libbpf.so:
undefined reference to `elf_getdata'/usr/lib/gcc/x86_64-redhat-
linux/4.8.5/../../../../lib64/libbpf.so:
undefined reference to `gelf_getshdr'Quoted 2 lines
Thanks,
XiaolongLooking at that list, at least the very first symbol is not used by a
public header, but internally in libbpf:$ grep -r elf_nextscn libbpf.c: while ((scn = elf_nextscn(elf, scn)) != NULL) {https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git/tree/tools/lib/bpf/libbpf.c#n793
None of those symbols are used from the public headers:
Quoted 7 lines
$ grep elf_ bpf.h libbpf.h btf.h $Therefore, this is an omission in libbpf's
Makefile, as it must link
against libelf. Please raise a ticket or send a patch upstream, and remove the -lelf from DPDK's makefiles.
Quoted 8 lines
As it may need sometime for kernel community to handle the libbpf's
Makefile,
We may still need the -lelf foraf_xdppmd currently, I'll remove it later after
libbpf correct to link against libelf. Is it acceptable?Thanks,
Xiaolong
Isn't the final version not out yet anyway till May? Can the fix be included before the release?
Signature · 2 lines
Kind regards, Luca Boccassi
Re: [PATCH v1 1/6] net/af_xdp: introduce AF_XDP PMD driver
Quoted 104 lines · Luca Boccassi
Quoted 103 lines · Ye Xiaolong
Quoted 88 lines · Luca Boccassi
Quoted 73 lines · Ye Xiaolong
Quoted 20 lines · Luca Boccassi
Quoted 15 lines · Ye Xiaolong
Quoted 8 lines · Ye Xiaolong
Quoted 7 lines
Review
Quoted 5 lines
_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AF_PACKET) +=
(quoted diff) +1
-3-lrte_pmd_af_packet +_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AF_XDP) += -lrte_pmd_af_xdp -lelf -lbpfAre symbols from libelf being used by the PMD?
Quoted 5 lines
Hmm, it is a leftover of RFC, libelf is no longer needed in this
version, will
remove it in next version.Quoted 3 lines
Correction, libelf is needed for libbpf, so we still need to keep
it.Quoted 6 lines
If libbpf needs libelf for its internal usage, it should be linked
against it itself. Unless symbols from libelf are used in static
functions defined in libbpf's public headers. Is this the case?Yes, that's the case. without the libelf, it would have build error
as below,
and these symbols are used in static functions of libbpf.
/usr/lib/gcc/x86_64-redhat-
linux/4.8.5/../../../../lib64/libbpf.so:
undefined reference to `elf_nextscn'/usr/lib/gcc/x86_64-redhat-
linux/4.8.5/../../../../lib64/libbpf.so:
undefined reference to `elf_rawdata'/usr/lib/gcc/x86_64-redhat-
linux/4.8.5/../../../../lib64/libbpf.so:
undefined reference to `elf_memory'/usr/lib/gcc/x86_64-redhat-
linux/4.8.5/../../../../lib64/libbpf.so:
undefined reference to `gelf_getrel'/usr/lib/gcc/x86_64-redhat-
linux/4.8.5/../../../../lib64/libbpf.so:
undefined reference to `elf_strptr'/usr/lib/gcc/x86_64-redhat-
linux/4.8.5/../../../../lib64/libbpf.so:
undefined reference to `elf_end'/usr/lib/gcc/x86_64-redhat-
linux/4.8.5/../../../../lib64/libbpf.so:
undefined reference to `elf_getscn'/usr/lib/gcc/x86_64-redhat-
linux/4.8.5/../../../../lib64/libbpf.so:
undefined reference to `elf_begin'/usr/lib/gcc/x86_64-redhat-
linux/4.8.5/../../../../lib64/libbpf.so:
undefined reference to `gelf_getsym'/usr/lib/gcc/x86_64-redhat-
linux/4.8.5/../../../../lib64/libbpf.so:
undefined reference to `elf_version'/usr/lib/gcc/x86_64-redhat-
linux/4.8.5/../../../../lib64/libbpf.so:
undefined reference to `gelf_getehdr'/usr/lib/gcc/x86_64-redhat-
linux/4.8.5/../../../../lib64/libbpf.so:
undefined reference to `elf_getdata'/usr/lib/gcc/x86_64-redhat-
linux/4.8.5/../../../../lib64/libbpf.so:
undefined reference to `gelf_getshdr'Quoted 2 lines
Thanks,
XiaolongLooking at that list, at least the very first symbol is not used by a
public header, but internally in libbpf:$ grep -r elf_nextscn libbpf.c: while ((scn = elf_nextscn(elf, scn)) != NULL) {https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git/tree/tools/lib/bpf/libbpf.c#n793
None of those symbols are used from the public headers:
Quoted 7 lines
$ grep elf_ bpf.h libbpf.h btf.h $Therefore, this is an omission in libbpf's
Makefile, as it must link
against libelf. Please raise a ticket or send a patch upstream, and remove the -lelf from DPDK's makefiles.As it may need sometime for kernel community to handle the libbpf's
Makefile,
We may still need the -lelf foraf_xdppmd currently, I'll remove it later after
libbpf correct to link against libelf. Is it acceptable?
Quoted 5 lines
Quoted 2 lines
Thanks,
XiaolongIsn't the final version not out yet anyway till May? Can the fix be included before the release?
I think so.
Thanks,
Xiaolong
Quoted 4 lines
Signature · 3 lines
-- Kind regards, Luca Boccassi
Re: [PATCH v1 1/6] net/af_xdp: introduce AF_XDP PMD driver
In reply to Xiaolong Yedrivers/net/af_xdp/rte_eth_af_xdp.c line 433
Quoted 2 lines · On Fri, 1 Mar 2019 16:09:42 +0800 Xiaolong Ye <[email protected]> wrote:
drivers/net/af_xdp/rte_eth_af_xdp.c new file +2
-0+if (umem->buffer) +free (umem->buffer);
Minor nit: you don't need to check for NULL free() already handles this.
Re: [PATCH v1 1/6] net/af_xdp: introduce AF_XDP PMD driver
Hi,
Quoted 7 lines · Stephen Hemminger
drivers/net/af_xdp/rte_eth_af_xdp.c line 433
Quoted 2 lines · On Fri, 1 Mar 2019 16:09:42 +0800 Xiaolong Ye <[email protected]> wrote:
drivers/net/af_xdp/rte_eth_af_xdp.c new file +2
-0+if (umem->buffer) +free (umem->buffer);Minor nit: you don't need to check for NULL
free()already handles this.
Thanks for the suggestion, will change accordingly.
Re: [PATCH v1 1/6] net/af_xdp: introduce AF_XDP PMD driver
In reply to Xiaolong Yedrivers/net/af_xdp/rte_eth_af_xdp.c line 830
Quoted 13 lines · On Fri, 1 Mar 2019 16:09:42 +0800 Xiaolong Ye <[email protected]> wrote:
drivers/net/af_xdp/rte_eth_af_xdp.c new file +13
-0+ +static int +rte_pmd_af_xdp_probe (struct rte_vdev_device *dev) +{ +struct rte_kvargs *kvlist; +char *if_name =NULL ; +int queue_idx =ETH_AF_XDP_DFLT_QUEUE_IDX ; +struct rte_eth_dev *eth_dev; +const char *name; +int ret; + +RTE_LOG (INFO ,PMD ,"Initializing pmd_af_packet for %s\n" , +rte_vdev_device_name (dev));
The PMD log type is being phased out. I plan to mark it as deprecated. All new drivers must use their own log types (see other every other device driver for how to do this).
Re: [PATCH v1 1/6] net/af_xdp: introduce AF_XDP PMD driver
Hi,
Quoted 12 lines · Stephen Hemminger
Quoted 9 lines · On Fri, 1 Mar 2019 16:09:42 +0800 Xiaolong Ye <[email protected]> wrote:
drivers/net/af_xdp/rte_eth_af_xdp.c new file +9
-0+ +static int +rte_pmd_af_xdp_probe (struct rte_vdev_device *dev) +{ +struct rte_kvargs *kvlist; +char *if_name =NULL ; +int queue_idx =ETH_AF_XDP_DFLT_QUEUE_IDX ; +struct rte_eth_dev *eth_dev; +const char *name;
Quoted 8 lines
drivers/net/af_xdp/rte_eth_af_xdp.c line 830
Quoted 4 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +4
-0+int ret; + +RTE_LOG (INFO ,PMD ,"Initializing pmd_af_packet for %s\n" , +rte_vdev_device_name (dev));The PMD log type is being phased out. I plan to mark it as deprecated. All new drivers must use their own log types (see other every other device driver for how to do this).
Thanks for the feedback, will do.
Re: [PATCH v1 1/6] net/af_xdp: introduce AF_XDP PMD driver
In reply to Xiaolong YeQuoted 17 lines · Xiaolong Ye
doc/guides/rel_notes/release_18_11.rst +0
-0@@ -229,6 +229,13 @@ New Features The AESNI MB PMD has been updated with additional support for theAES-GCM
algorithm.doc/guides/rel_notes/release_18_11.rst +3
-0+* **Added the AF_XDP PMD.** + + Added a Linux-specific PMD driver for AF_XDP, it can create the AF_XDPsocket
+ and bind it to a specific netdev queue, it allows a DPDK application to send
doc/guides/rel_notes/release_18_11.rst
Quoted 3 lines
doc/guides/rel_notes/release_18_11.rst +3
-0+ and receive raw packets through the socket which would bypass the kernel + network stack to achieve high performance packet processing. +
Should be in 19.05 release notes.
(quoted diff) +0 -0
drivers/net/af_xdp/rte_eth_af_xdp.c
Quoted 7 lines
b/drivers/net/af_xdp/rte_eth_af_xdp.c
new file mode 100644
index 000000000..6de769650drivers/net/af_xdp/rte_eth_af_xdp.c new file +0
-0
+struct pkt_rx_queue {
drivers/net/af_xdp/rte_eth_af_xdp.c
Quoted 9 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +8
-0+struct xsk_ring_cons rx; +struct xsk_umem_info *umem; +struct xsk_socket *xsk; +struct rte_mempool *mb_pool; + +unsigned long rx_pkts; +unsigned long rx_bytes; +unsigned long rx_dropped;
ethdev stats are uint64_t, why declare those internal stats as unsigned long ?
+
drivers/net/af_xdp/rte_eth_af_xdp.c
Quoted 11 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+struct pkt_tx_queue *pair; +uint16_t queue_idx; +}; + +struct pkt_tx_queue { +struct xsk_ring_prod tx; + +unsigned long tx_pkts; +unsigned long err_pkts; +unsigned long tx_bytes;
Idem.
+
drivers/net/af_xdp/rte_eth_af_xdp.c
Quoted 4 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +3
-0+struct pkt_rx_queue *pair; +uint16_t queue_idx; +};
[snip]
+static int
Quoted 7 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +6
-0+eth_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats) +{ + struct pmd_internals *internals = dev->data->dev_private; +struct xdp_statistics xdp_stats; +struct pkt_rx_queue *rxq; +socklen_t optlen;
drivers/net/af_xdp/rte_eth_af_xdp.c
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+int i; + + optlen =sizeof (struct xdp_statistics ); +for (i =0 ; i < dev->data->nb_rx_queues; i++) { + rxq = &internals->rx_queues[i]; + stats->q_ipackets[i] = internals->rx_queues[i].rx_pkts; + stats->q_ibytes[i] = internals->rx_queues[i].rx_bytes; + + stats->q_opackets[i] = internals->tx_queues[i].tx_pkts; + stats->q_errors[i] = internals->tx_queues[i].err_pkts;
q_errors[] are for reception errors, see the patchset I sent:
http://mails.dpdk.org/archives/dev/2019-March/125703.html
If you want per queue stats, use xstats.
You can still account those errors in the global stats->oerrors below.
+ stats->q_obytes[i] = internals->tx_queues[i].tx_bytes;
drivers/net/af_xdp/rte_eth_af_xdp.c
Quoted 12 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +5
-0+ + stats->ipackets += stats->q_ipackets[i]; + stats->ibytes += stats->q_ibytes[i]; + stats->imissed += internals->rx_queues[i].rx_dropped; + getsockopt(xsk_socket__fd(rxq->xsk), SOL_XDP,
XDP_STATISTICS,drivers/net/af_xdp/rte_eth_af_xdp.c new file +5
-0+ &xdp_stats, &optlen); + stats->imissed += xdp_stats.rx_dropped; + + stats->opackets += stats->q_opackets[i]; + stats->oerrors += stats->q_errors[i];
- stats->oerrors += stats->q_errors[i];
+ stats->oerrors += internals->tx_queues[i].err_pkts;
Quoted 8 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +5
-0+ stats->obytes += stats->q_obytes[i]; + } + +return 0 ; +}
Signature · 1 line
David Marchand
Re: [PATCH v1 1/6] net/af_xdp: introduce AF_XDP PMD driver
Hi, David
Thanks for your review.
Quoted 12 lines · David Marchand
Quoted 10 lines · Xiaolong Ye
doc/guides/rel_notes/release_18_11.rst +0
-0@@ -229,6 +229,13 @@ New Features The AESNI MB PMD has been updated with additional support for theAES-GCM
algorithm.
Quoted 12 lines
doc/guides/rel_notes/release_18_11.rst
Quoted 10 lines
doc/guides/rel_notes/release_18_11.rst +3
-0+* **Added the AF_XDP PMD.** + + Added a Linux-specific PMD driver for AF_XDP, it can create the AF_XDPsocket
+ and bind it to a specific netdev queue, it allows a DPDK application to senddoc/guides/rel_notes/release_18_11.rst +3
-0+ and receive raw packets through the socket which would bypass the kernel + network stack to achieve high performance packet processing. +Should be in 19.05 release notes.
Ooops, my bad, will swith to 19.05 in next version.
Quoted 22 lines
(quoted diff) +0
-0drivers/net/af_xdp/rte_eth_af_xdp.c
Quoted 7 lines
b/drivers/net/af_xdp/rte_eth_af_xdp.c
new file mode 100644
index 000000000..6de769650drivers/net/af_xdp/rte_eth_af_xdp.c new file +0
-0+struct
pkt_rx_queue{drivers/net/af_xdp/rte_eth_af_xdp.c
Quoted 9 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +8
-0+struct xsk_ring_cons rx; +struct xsk_umem_info *umem; +struct xsk_socket *xsk; +struct rte_mempool *mb_pool; + +unsigned long rx_pkts; +unsigned long rx_bytes; +unsigned long rx_dropped;
Quoted 2 lines
ethdev stats are
uint64_t, why declare those internal stats as unsigned long ?
You are right, should use uint64_t instead.
Quoted 15 lines
+
drivers/net/af_xdp/rte_eth_af_xdp.c
Quoted 11 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+struct pkt_tx_queue *pair; +uint16_t queue_idx; +}; + +struct pkt_tx_queue { +struct xsk_ring_prod tx; + +unsigned long tx_pkts; +unsigned long err_pkts; +unsigned long tx_bytes;Idem.
Will change to uint64_t.
Quoted 48 lines
+
drivers/net/af_xdp/rte_eth_af_xdp.c
Quoted 4 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +3
-0+struct pkt_rx_queue *pair; +uint16_t queue_idx; +};[snip]
+static int
Quoted 7 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +6
-0+eth_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats) +{ + struct pmd_internals *internals = dev->data->dev_private; +struct xdp_statistics xdp_stats; +struct pkt_rx_queue *rxq; +socklen_t optlen;drivers/net/af_xdp/rte_eth_af_xdp.c
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+int i; + + optlen =sizeof (struct xdp_statistics ); +for (i =0 ; i < dev->data->nb_rx_queues; i++) { + rxq = &internals->rx_queues[i]; + stats->q_ipackets[i] = internals->rx_queues[i].rx_pkts; + stats->q_ibytes[i] = internals->rx_queues[i].rx_bytes; + + stats->q_opackets[i] = internals->tx_queues[i].tx_pkts; + stats->q_errors[i] = internals->tx_queues[i].err_pkts;q_errors[] are for reception errors, see the patchset I sent:
http://mails.dpdk.org/archives/dev/2019-March/125703.htmlIf you want per queue stats, use xstats.
You can still account those errors in the globalstats->oerrorsbelow.+ stats->q_obytes[i] = internals->tx_queues[i].tx_bytes;
Quoted 12 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +5
-0+ + stats->ipackets += stats->q_ipackets[i]; + stats->ibytes += stats->q_ibytes[i]; + stats->imissed += internals->rx_queues[i].rx_dropped; + getsockopt(xsk_socket__fd(rxq->xsk), SOL_XDP,
XDP_STATISTICS,drivers/net/af_xdp/rte_eth_af_xdp.c new file +5
-0+ &xdp_stats, &optlen); + stats->imissed += xdp_stats.rx_dropped; + + stats->opackets += stats->q_opackets[i]; + stats->oerrors += stats->q_errors[i];
Review
Quoted 2 lines
(quoted diff) +1
-1- stats->oerrors += stats->q_errors[i]; + stats->oerrors += internals->tx_queues[i].err_pkts;
will correct according to your patch.
Thanks,
Xiaolong
Quoted 12 lines
Quoted 9 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +5
-0+ stats->obytes += stats->q_obytes[i]; + } + +return 0 ; +}Signature · 1 line
David Marchand
Re: [PATCH v1 1/6] net/af_xdp: introduce AF_XDP PMD driver
In reply to Xiaolong YeQuoted 7 lines · Xiaolong Ye
Add a new PMD driver for
AF_XDPwhich is a proposed faster version ofAF_PACKETinterface in Linux. More info aboutAF_XDP, please refer to [1] [2].This is the vanilla version PMD which just uses a raw buffer registered as the umem.
Quoted 8 lines
[1] https://fosdem.org/2018/schedule/event/af_xdp/
[2] https://lwn.net/Articles/745934/Signed-off-by Xiaolong Ye <[email protected]>--- MAINTAINERS | 6 + config/common_base | 5 + doc/guides/nics/af_xdp.rst | 43 +
Can you please add new .rst file to index file, doc/guides/nics/index.rst ?
Quoted 1 line
doc/guides/rel_notes/release_18_11.rst | 7 +
Please switch to latest release notes.
Quoted 6 lines
drivers/net/Makefile | 1 + drivers/net/af_xdp/Makefile | 31 + drivers/net/af_xdp/meson.build | 7 + drivers/net/af_xdp/rte_eth_af_xdp.c | 903 ++++++++++++++++++ drivers/net/af_xdp/rte_pmd_af_xdp_version.map | 4 + mk/rte.app.mk | 1 +
Can you please add .ini file too?
<...>
config/common_base CONFIG_RTE_LIBRTE_VMXNET3_DEBUG_TX_FREE=n · line 423
Quoted 9 lines
config/common_base +5
-0@@ -416,6 +416,11 @@ CONFIG_RTE_LIBRTE_VMXNET3_DEBUG_TX_FREE=n # CONFIG_RTE_LIBRTE_PMD_AF_PACKET=n +# +# Compile software PMD backed by AF_XDP sockets (Linux only) +# +CONFIG_RTE_LIBRTE_PMD_AF_XDP=n +
Why it is not enabled in linux config (config/common_linuxapp)? Is it because of the external library dependencies?
I guess there is a requirement to the specific Linux kernel version, can it be possible to detect it in Makefile and enable/disable according this information?
<...>
doc/guides/nics/af_xdp.rst line 32
Quoted 6 lines
doc/guides/nics/af_xdp.rst new file +6
-0+Prerequisites +------------- + +This is a Linux-specific PMD, thus the following prerequisites apply: + +* A Linux Kernel with XDP sockets configuration enabled;
Can you please give more details of what exact vanilla kernel version?
Quoted 1 line
+* libbpf with latest
af_xdpsupport installed
Is there a specific version of libbpf for this?
I can see in makefile, libelf is also linked, is it a dependency?
<...>
Quoted 8 lines
drivers/net/af_xdp/Makefile new file +7
-0@@ -0,0 +1,31 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Intel Corporation + +include $(RTE_SDK)/mk/rte.vars.mk + +# +# library name
drivers/net/af_xdp/Makefile line 17
Quoted 10 lines
drivers/net/af_xdp/Makefile new file +10
-0+# +LIB = librte_pmd_af_xdp.a + +EXPORT_MAP := rte_pmd_af_xdp_version.map + +LIBABIVER := 1 + + +CFLAGS += -O3 +# below line should be removed
+1
drivers/net/af_xdp/Makefile line 24
Quoted 7 lines
drivers/net/af_xdp/Makefile new file +7
-0+CFLAGS += -I/root/yexl/shared_mks0/linux/tools/include +CFLAGS += -I/root/yexl/shared_mks0/linux/tools/lib/bpf + +CFLAGS += $(WERROR_FLAGS) +LDLIBS += -lrte_eal -lrte_mbuf -lrte_mempool -lrte_ring +LDLIBS += -lrte_ethdev -lrte_net -lrte_kvargs +LDLIBS += -lrte_bus_vdev
Dependent libraries should be linked here.
<...>
drivers/net/af_xdp/rte_eth_af_xdp.c line 15
Quoted 5 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +5
-0+ +#include <linux/if_ether.h> +#include <linux/if_xdp.h> +#include <linux/if_link.h> +#include <asm/barrier.h>
Getting an build error for this [1], can there be any include path param missing?
[1]drivers/net/af_xdp/rte_eth_af_xdp.c:15:10: fatal error: asm/barrier.h: No such file or directory
<...>
drivers/net/af_xdp/rte_eth_af_xdp.c line 324
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+static void +eth_dev_info (struct rte_eth_dev *dev,struct rte_eth_dev_info *dev_info) +{ +struct pmd_internals *internals = dev->data->dev_private; + + dev_info->if_index = internals->if_index; + dev_info->max_mac_addrs =1 ; + dev_info->max_rx_pktlen = (uint32_t )ETH_FRAME_LEN ; + dev_info->max_rx_queues =1 ; + dev_info->max_tx_queues =1 ;
'ETH_AF_XDP_MAX_QUEUE_PAIRS' is '16' but you are forcing the max Rx/Tx queue number to be '1', intentional?
Quoted 15 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +15
-0+ dev_info->min_rx_bufsize =0 ; + + dev_info->default_rxportconf.nb_queues =1 ; + dev_info->default_txportconf.nb_queues =1 ; + dev_info->default_rxportconf.ring_size =ETH_AF_XDP_DFLT_NUM_DESCS ; + dev_info->default_txportconf.ring_size =ETH_AF_XDP_DFLT_NUM_DESCS ; +} + +static int +eth_stats_get (struct rte_eth_dev *dev,struct rte_eth_stats *stats) +{ +struct pmd_internals *internals = dev->data->dev_private; +struct xdp_statistics xdp_stats; +struct pkt_rx_queue *rxq; +socklen_t optlen;
drivers/net/af_xdp/rte_eth_af_xdp.c line 349
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+int i; + + optlen =sizeof (struct xdp_statistics ); +for (i =0 ; i < dev->data->nb_rx_queues; i++) { + rxq = &internals->rx_queues[i]; + stats->q_ipackets[i] = internals->rx_queues[i].rx_pkts; + stats->q_ibytes[i] = internals->rx_queues[i].rx_bytes; + + stats->q_opackets[i] = internals->tx_queues[i].tx_pkts; + stats->q_errors[i] = internals->tx_queues[i].err_pkts;
There is a patch from David, which points the 'q_errors' is for Rx only:
https://patches.dpdk.org/cover/50783/
<...>
drivers/net/af_xdp/rte_eth_af_xdp.c line 435
Quoted 6 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +6
-0+static void xdp_umem_destroy (struct xsk_umem_info *umem) +{ +if (umem->buffer) +free (umem->buffer); + +free (umem);
Should we set freed pointers to 'null'?
Should free 'umem->buf_ring' before freeing 'umem'?
<...>
Quoted 9 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +9
-0+static int +eth_rx_queue_setup (struct rte_eth_dev *dev, +uint16_t rx_queue_id, +uint16_t nb_rx_desc, + unsigned int socket_id __rte_unused, + const struct rte_eth_rxconf *rx_conf __rte_unused, +struct rte_mempool *mb_pool) +{ +struct pmd_internals *internals = dev->data->dev_private;
drivers/net/af_xdp/rte_eth_af_xdp.c line 579
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+unsigned int buf_size, data_size; +struct pkt_rx_queue *rxq; +int ret =0 ; + +if (mb_pool ==NULL ) { +RTE_LOG (ERR ,PMD , +"Invalid mb_pool\n" ); + ret = -EINVAL ; + goto err; + }
if 'mb_pool' is 'null', it will crash in 'rte_eth_rx_queue_setup()' before coming here, I think we can drop this check.
drivers/net/af_xdp/rte_eth_af_xdp.c line 586
Quoted 7 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +7
-0+ +if (dev->data->nb_rx_queues <= rx_queue_id) { +RTE_LOG (ERR ,PMD , +"Invalid rx queue id: %d\n" , rx_queue_id); + ret = -EINVAL ; + goto err; + }
This check already done in 'rte_eth_rx_queue_setup()' shouldn't need to be done here.
<...>
Quoted 4 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +4
-0+static int +eth_tx_queue_setup(struct rte_eth_dev *dev, + uint16_t tx_queue_id, + uint16_t nb_tx_desc,
drivers/net/af_xdp/rte_eth_af_xdp.c line 636
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+ unsigned int socket_id __rte_unused, + const struct rte_eth_txconf *tx_conf __rte_unused) +{ + struct pmd_internals *internals = dev->data->dev_private; +struct pkt_tx_queue *txq; + +if (dev->data->nb_tx_queues <= tx_queue_id) { +RTE_LOG (ERR ,PMD ,"Invalid tx queue id: %d\n" , tx_queue_id); +return -EINVAL ; + }
Can skip the check, same as above.
drivers/net/af_xdp/rte_eth_af_xdp.c line 639
Quoted 3 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +3
-0+ +RTE_LOG (WARNING ,PMD ,"tx queue setup size=%d will be skipped\n" , + nb_tx_desc);
Why setup will be skipped?
Quoted 9 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +9
-0+ txq = &internals->tx_queues[tx_queue_id]; + + dev->data->tx_queues[tx_queue_id] = txq; +return 0 ; +} + +static int +eth_dev_mtu_set (struct rte_eth_dev *dev,uint16_t mtu) +{
drivers/net/af_xdp/rte_eth_af_xdp.c line 658
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+struct pmd_internals *internals = dev->data->dev_private; +struct ifreq ifr = { .ifr_mtu = mtu }; +int ret; +int s; + + s =socket (PF_INET ,SOCK_DGRAM ,0 ); +if (s <0 ) +return -EINVAL ; + +snprintf (ifr.ifr_name,IFNAMSIZ ,"%s" , internals->if_name);
Can you please prefer strlcpy?
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+ ret =ioctl (s,SIOCSIFMTU , &ifr); +close (s); + +if (ret <0 ) +return -EINVAL ; + +return 0 ; +} + +static void
drivers/net/af_xdp/rte_eth_af_xdp.c line 678
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+eth_dev_change_flags(char *if_name, uint32_t flags, uint32_t mask) +{ +struct ifreq ifr; +int s; + + s =socket (PF_INET ,SOCK_DGRAM ,0 ); +if (s <0 ) +return ; + +snprintf (ifr.ifr_name,IFNAMSIZ ,"%s" , if_name);
Can you please prefer strlcpy?
<...>
drivers/net/af_xdp/rte_eth_af_xdp.c line 723
Quoted 2 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +2
-0+ +static struct rte_vdev_driver pmd_af_xdp_drv;
Do we need this forward decleration?
drivers/net/af_xdp/rte_eth_af_xdp.c line 735
Quoted 12 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +12
-0+ +static void +parse_parameters (struct rte_kvargs *kvlist, +char **if_name, +int *queue_idx) +{ +struct rte_kvargs_pair *pair =NULL ; +unsigned int k_idx; + +for (k_idx =0 ; k_idx < kvlist->count; k_idx++) { + pair = &kvlist->pairs[k_idx]; +if (strstr (pair->key,ETH_AF_XDP_IFACE_ARG ))
It is better to use 'rte_kvargs_process()' instead of accessing the 'kvargs' internals.
Quoted 8 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +8
-0+ *if_name = pair->value; + else if (strstr(pair->key, ETH_AF_XDP_QUEUE_IDX_ARG)) + *queue_idx = atoi(pair->value); + } +} + +static int +get_iface_info(const char *if_name,
drivers/net/af_xdp/rte_eth_af_xdp.c line 753
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+struct ether_addr *eth_addr, + int *if_index) +{ + struct ifreq ifr; +int sock =socket (AF_INET ,SOCK_DGRAM ,IPPROTO_IP ); + +if (sock <0 ) +return -1 ; + +strcpy (ifr.ifr_name, if_name);
Please prefer strlcpy.
drivers/net/af_xdp/rte_eth_af_xdp.c line 760
Quoted 7 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +7
-0+if (ioctl (sock,SIOCGIFINDEX , &ifr)) + goto error; + +if (ioctl (sock,SIOCGIFHWADDR , &ifr)) + goto error; + +memcpy (eth_addr, ifr.ifr_hwaddr.sa_data,6 );
Can use 'ether_addr_copy()'
Quoted 18 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +18
-0+ +close (sock); + *if_index =if_nametoindex (if_name); +return 0 ; + +error: +close (sock); +return -1 ; +} + +static int +init_internals (struct rte_vdev_device *dev, +const char *if_name, +int queue_idx) +{ +const char *name =rte_vdev_device_name (dev); +struct rte_eth_dev *eth_dev =NULL ; +const unsigned int numa_node = dev->device.numa_node;
drivers/net/af_xdp/rte_eth_af_xdp.c line 788
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+struct pmd_internals *internals =NULL ; +int ret; +int i; + + internals =rte_zmalloc_socket (name,sizeof (*internals),0 , numa_node); +if (!internals) +return -ENOMEM ; + + internals->queue_idx = queue_idx; +strcpy (internals->if_name, if_name);
prefer 'strlcpy' please
Quoted 13 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +13
-0+ +for (i =0 ; i <ETH_AF_XDP_MAX_QUEUE_PAIRS ; i++) { + internals->tx_queues[i].pair = &internals->rx_queues[i]; + internals->rx_queues[i].pair = &internals->tx_queues[i]; + } + + ret =get_iface_info (if_name, &internals->eth_addr, + &internals->if_index); +if (ret) + goto err; + + eth_dev =rte_eth_vdev_allocate (dev,0 ); +if (!eth_dev)
drivers/net/af_xdp/rte_eth_af_xdp.c line 811
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+ goto err; + + eth_dev->data->dev_private = internals; + eth_dev->data->dev_link = pmd_link; + eth_dev->data->mac_addrs = &internals->eth_addr; + eth_dev->dev_ops = &ops; + eth_dev->rx_pkt_burst = eth_af_xdp_rx; + eth_dev->tx_pkt_burst = eth_af_xdp_tx; + +rte_eth_dev_probing_finish (eth_dev);
What do you think moving this call into 'rte_pmd_af_xdp_probe' function if 'init_internals' returns sucess instead of setting here?
drivers/net/af_xdp/rte_eth_af_xdp.c line 824
Quoted 13 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +13
-0+return 0 ; + +err: +rte_free (internals); +return -1 ; +} + +static int +rte_pmd_af_xdp_probe (struct rte_vdev_device *dev) +{ +struct rte_kvargs *kvlist; +char *if_name =NULL ; +int queue_idx =ETH_AF_XDP_DFLT_QUEUE_IDX ;
This 'queue_idx' is for interface queue to pass to xsk_* API, also we have same variable name 'queue_idx' that we use for DPDK queue index, they get confused easily, what do you think rename this one something like 'xsk_queue_idx'?
Quoted 13 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +13
-0+struct rte_eth_dev *eth_dev; +const char *name; +int ret; + +RTE_LOG (INFO ,PMD ,"Initializing pmd_af_packet for %s\n" , +rte_vdev_device_name (dev)); + + name =rte_vdev_device_name (dev); +if (rte_eal_process_type () ==RTE_PROC_SECONDARY && +strlen (rte_vdev_device_args (dev)) ==0 ) { + eth_dev =rte_eth_dev_attach_secondary (name); +if (!eth_dev) { +RTE_LOG (ERR ,PMD ,"Failed to probe %s\n" , name);
drivers/net/af_xdp/rte_eth_af_xdp.c line 847
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+return -EINVAL ; + } + eth_dev->dev_ops = &ops; +rte_eth_dev_probing_finish (eth_dev); + } + + kvlist =rte_kvargs_parse (rte_vdev_device_args (dev), valid_arguments); +if (!kvlist) { +RTE_LOG (ERR ,PMD , +"Invalid kvargs\n" );
No need to break the line.
drivers/net/af_xdp/rte_eth_af_xdp.c line 855
Quoted 8 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +8
-0+return -EINVAL ; + } + +if (dev->device.numa_node ==SOCKET_ID_ANY ) + dev->device.numa_node =rte_socket_id (); + +parse_parameters (kvlist, &if_name, + &queue_idx);
Same, no need to break the line.
Quoted 23 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +23
-0+ + ret =init_internals (dev, if_name, queue_idx); + +rte_kvargs_free (kvlist); + +return ret; +} + +static int +rte_pmd_af_xdp_remove (struct rte_vdev_device *dev) +{ +struct rte_eth_dev *eth_dev =NULL ; +struct pmd_internals *internals; + +RTE_LOG (INFO ,PMD ,"Removing AF_XDP ethdev on numa socket %u\n" , +rte_socket_id ()); + +if (!dev) +return -1 ; + +/* find the ethdev entry */ + eth_dev =rte_eth_dev_allocated (rte_vdev_device_name (dev)); +if (!eth_dev)
drivers/net/af_xdp/rte_eth_af_xdp.c line 888
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+return -1 ; + + internals = eth_dev->data->dev_private; + +rte_ring_free (internals->umem->buf_ring); +rte_free (internals->umem->buffer); +rte_free (internals->umem); +rte_free (internals); + +rte_eth_dev_release_port (eth_dev);
This frees the 'eth_dev->data->dev_private', (internals), it can be problem to try to free same pointer twice.
drivers/net/af_xdp/rte_eth_af_xdp.c line 900
Quoted 12 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +12
-0+ + +return 0 ; +} + +static struct rte_vdev_driver pmd_af_xdp_drv = { + .probe = rte_pmd_af_xdp_probe, + .remove = rte_pmd_af_xdp_remove, +}; + +RTE_PMD_REGISTER_VDEV (net_af_xdp, pmd_af_xdp_drv); +RTE_PMD_REGISTER_ALIAS (net_af_xdp, eth_af_xdp);
No need to create the alias, it is for backward compatibility.
drivers/net/af_xdp/rte_pmd_af_xdp_version.map line 1
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +3
-0+RTE_PMD_REGISTER_PARAM_STRING (net_af_xdp, +"iface=<string> " +"queue=<int> " );drivers/net/af_xdp/rte_pmd_af_xdp_version.map new file +1
-0@@ -0,0 +1,4 @@ +DPDK_2.0 {
Use release version please.
mk/rte.app.mk _LDLIBS-$(CONFIG_RTE_LIBRTE_DPAA2_MEMPOOL) += -lrte_mempool_dpaa2 · line 146
Quoted 12 lines
drivers/net/af_xdp/rte_pmd_af_xdp_version.map new file +3
-0+ + local: *; +};mk/rte.app.mk +1
-0@@ -143,6 +143,7 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_DPAA2_MEMPOOL) += -lrte_mempool_dpaa2 endif _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AF_PACKET) += -lrte_pmd_af_packet +_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AF_XDP) += -lrte_pmd_af_xdp -lelf -lbpf
For which API libelf is required?
Re: [PATCH v1 1/6] net/af_xdp: introduce AF_XDP PMD driver
Hi, Ferruh
Thanks for your review.
Quoted 11 lines · Ferruh Yigit
Quoted 10 lines · Xiaolong Ye
Add a new PMD driver for
AF_XDPwhich is a proposed faster version ofAF_PACKETinterface in Linux. More info aboutAF_XDP, please refer to [1] [2].This is the vanilla version PMD which just uses a raw buffer registered as the umem.
[1] https://fosdem.org/2018/schedule/event/af_xdp/
[2] https://lwn.net/Articles/745934/
Quoted 7 lines
Quoted 5 lines
Signed-off-by Xiaolong Ye <[email protected]>--- MAINTAINERS | 6 + config/common_base | 5 + doc/guides/nics/af_xdp.rst | 43 +Can you please add new .rst file to index file,
doc/guides/nics/index.rst?
Got it, will do in next version.
Quoted 4 lines
Quoted 2 lines
doc/guides/rel_notes/release_18_11.rst | 7 +Please switch to latest release notes.
My bad, will switch to 19.05..
Quoted 9 lines
Quoted 7 lines
drivers/net/Makefile | 1 + drivers/net/af_xdp/Makefile | 31 + drivers/net/af_xdp/meson.build | 7 + drivers/net/af_xdp/rte_eth_af_xdp.c | 903 ++++++++++++++++++ drivers/net/af_xdp/rte_pmd_af_xdp_version.map | 4 + mk/rte.app.mk | 1 +Can you please add .ini file too?
will do.
Quoted 7 lines
<...>
Quoted 4 lines
config/common_base +0
-0@@ -416,6 +416,11 @@ CONFIG_RTE_LIBRTE_VMXNET3_DEBUG_TX_FREE=n # CONFIG_RTE_LIBRTE_PMD_AF_PACKET=n
Quoted 8 lines
config/common_base CONFIG_RTE_LIBRTE_VMXNET3_DEBUG_TX_FREE=n · line 423
Quoted 5 lines
config/common_base +5
-0+# +# Compile software PMD backed by AF_XDP sockets (Linux only) +# +CONFIG_RTE_LIBRTE_PMD_AF_XDP=n +Why it is not enabled in linux config (config/common_linuxapp)? Is it because of the external library dependencies?
Yes, af_xdp pmd is dependent on libbpf which is not included in any linux distribution yet.
Quoted 1 line
I guess there is a requirement to the specific Linux kernel version, can it be
libbpf should be included in kernel 5.1 release.
Quoted 2 lines
possible to detect it in
Makefileand enable/disable according this information?
Ok, I'll investigate how to do it.
Quoted 10 lines
<...>
doc/guides/nics/af_xdp.rst line 32
Quoted 6 lines
doc/guides/nics/af_xdp.rst new file +6
-0+Prerequisites +------------- + +This is a Linux-specific PMD, thus the following prerequisites apply: + +* A Linux Kernel with XDP sockets configuration enabled;Can you please give more details of what exact vanilla kernel version?
Do you mean I should write more details about AF_XDP in kernel in this introduction document?
Quoted 4 lines
Quoted 2 lines
+* libbpf with latest
af_xdpsupport installedIs there a specific version of libbpf for this?
I'm not aware that there is specific version number for libbpf, it's part of linux kernel src code.
Quoted 1 line
I can see in makefile, libelf is also linked, is it a dependency?
libelf is a leftover of RFC, will delete it in next version.
Quoted 24 lines
<...>
Quoted 8 lines
drivers/net/af_xdp/Makefile new file +7
-0@@ -0,0 +1,31 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Intel Corporation + +include $(RTE_SDK)/mk/rte.vars.mk + +# +# library namedrivers/net/af_xdp/Makefile line 17
Quoted 10 lines
drivers/net/af_xdp/Makefile new file +10
-0+# +LIB = librte_pmd_af_xdp.a + +EXPORT_MAP := rte_pmd_af_xdp_version.map + +LIBABIVER := 1 + + +CFLAGS += -O3 +# below line should be removed+1
Quoted 9 lines
drivers/net/af_xdp/Makefile line 24
Quoted 7 lines
drivers/net/af_xdp/Makefile new file +7
-0+CFLAGS += -I/root/yexl/shared_mks0/linux/tools/include +CFLAGS += -I/root/yexl/shared_mks0/linux/tools/lib/bpf + +CFLAGS += $(WERROR_FLAGS) +LDLIBS += -lrte_eal -lrte_mbuf -lrte_mempool -lrte_ring +LDLIBS += -lrte_ethdev -lrte_net -lrte_kvargs +LDLIBS += -lrte_bus_vdevDependent libraries should be linked here.
Ok, I'll add libbpf here.
Quoted 11 lines
<...>
drivers/net/af_xdp/rte_eth_af_xdp.c line 15
Quoted 5 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +5
-0+ +#include <linux/if_ether.h> +#include <linux/if_xdp.h> +#include <linux/if_link.h> +#include <asm/barrier.h> Getting an build error for this [1], can there be any include path param missing?
Quoted 3 lines
[1]
drivers/net/af_xdp/rte_eth_af_xdp.c:15:10: fatal error:asm/barrier.h: No such file or directory
Yes, it need something like
CFLAGS += -I/root/yexl/shared_mks0/linux/tools/include
as in above Makefile currently.
Quoted 16 lines
<...>
drivers/net/af_xdp/rte_eth_af_xdp.c line 324
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+static void +eth_dev_info (struct rte_eth_dev *dev,struct rte_eth_dev_info *dev_info) +{ +struct pmd_internals *internals = dev->data->dev_private; + + dev_info->if_index = internals->if_index; + dev_info->max_mac_addrs =1 ; + dev_info->max_rx_pktlen = (uint32_t )ETH_FRAME_LEN ; + dev_info->max_rx_queues =1 ; + dev_info->max_tx_queues =1 ;'
ETH_AF_XDP_MAX_QUEUE_PAIRS' is '16' but you are forcing the max Rx/Tx queue number to be '1', intentional?
Yes, current implementation is single queue only, we plan to support muli-queues in futher.
Quoted 21 lines
Quoted 21 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +20
-0+ dev_info->min_rx_bufsize =0 ; + + dev_info->default_rxportconf.nb_queues =1 ; + dev_info->default_txportconf.nb_queues =1 ; + dev_info->default_rxportconf.ring_size =ETH_AF_XDP_DFLT_NUM_DESCS ; + dev_info->default_txportconf.ring_size =ETH_AF_XDP_DFLT_NUM_DESCS ; +} + +static int +eth_stats_get (struct rte_eth_dev *dev,struct rte_eth_stats *stats) +{ +struct pmd_internals *internals = dev->data->dev_private; +struct xdp_statistics xdp_stats; +struct pkt_rx_queue *rxq; +socklen_t optlen; +int i; + + optlen =sizeof (struct xdp_statistics ); +for (i =0 ; i < dev->data->nb_rx_queues; i++) { + rxq = &internals->rx_queues[i];
Quoted 8 lines
drivers/net/af_xdp/rte_eth_af_xdp.c line 349
Quoted 5 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +5
-0+ stats->q_ipackets[i] = internals->rx_queues[i].rx_pkts; + stats->q_ibytes[i] = internals->rx_queues[i].rx_bytes; + + stats->q_opackets[i] = internals->tx_queues[i].tx_pkts; + stats->q_errors[i] = internals->tx_queues[i].err_pkts;There is a patch from David, which points the '
q_errors' is for Rx only:
https://patches.dpdk.org/cover/50783/
Yes, I got the same comment from David, will change it accordingly.
Quoted 11 lines
<...>
drivers/net/af_xdp/rte_eth_af_xdp.c line 435
Quoted 6 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +6
-0+static void xdp_umem_destroy (struct xsk_umem_info *umem) +{ +if (umem->buffer) +free (umem->buffer); + +free (umem);Should we set freed pointers to 'null'?
will do.
Quoted 2 lines
Should free '
umem->buf_ring' before freeing 'umem'?
Good catch, will add free buf_ring.
Quoted 17 lines
<...>
Quoted 14 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +14
-0+static int +eth_rx_queue_setup (struct rte_eth_dev *dev, +uint16_t rx_queue_id, +uint16_t nb_rx_desc, + unsigned int socket_id __rte_unused, + const struct rte_eth_rxconf *rx_conf __rte_unused, +struct rte_mempool *mb_pool) +{ +struct pmd_internals *internals = dev->data->dev_private; +unsigned int buf_size, data_size; +struct pkt_rx_queue *rxq; +int ret =0 ; + +if (mb_pool ==NULL ) {
Quoted 8 lines
drivers/net/af_xdp/rte_eth_af_xdp.c line 579
Quoted 5 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +5
-0+RTE_LOG (ERR ,PMD , +"Invalid mb_pool\n" ); + ret = -EINVAL ; + goto err; + }if '
mb_pool' is 'null', it will crash in 'rte_eth_rx_queue_setup()' before coming here, I think we can drop this check.
Agree, it's a redundant check, will remove.
Quoted 11 lines
drivers/net/af_xdp/rte_eth_af_xdp.c line 586
Quoted 8 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +7
-0+ +if (dev->data->nb_rx_queues <= rx_queue_id) { +RTE_LOG (ERR ,PMD , +"Invalid rx queue id: %d\n" , rx_queue_id); + ret = -EINVAL ; + goto err; + }This check already done in '
rte_eth_rx_queue_setup()' shouldn't need to be done here.
will remove.
Quoted 18 lines
<...>
Quoted 4 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +4
-0+static int +eth_tx_queue_setup(struct rte_eth_dev *dev, + uint16_t tx_queue_id, + uint16_t nb_tx_desc,drivers/net/af_xdp/rte_eth_af_xdp.c line 636
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+ unsigned int socket_id __rte_unused, + const struct rte_eth_txconf *tx_conf __rte_unused) +{ + struct pmd_internals *internals = dev->data->dev_private; +struct pkt_tx_queue *txq; + +if (dev->data->nb_tx_queues <= tx_queue_id) { +RTE_LOG (ERR ,PMD ,"Invalid tx queue id: %d\n" , tx_queue_id); +return -EINVAL ; + }Can skip the check, same as above.
Got it.
Quoted 6 lines
drivers/net/af_xdp/rte_eth_af_xdp.c line 639
Quoted 4 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +3
-0+ +RTE_LOG (WARNING ,PMD ,"tx queue setup size=%d will be skipped\n" , + nb_tx_desc);Why setup will be skipped?
leftover of RFC, will remove.
Quoted 14 lines
Quoted 14 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +13
-0+ txq = &internals->tx_queues[tx_queue_id]; + + dev->data->tx_queues[tx_queue_id] = txq; +return 0 ; +} + +static int +eth_dev_mtu_set (struct rte_eth_dev *dev,uint16_t mtu) +{ +struct pmd_internals *internals = dev->data->dev_private; +struct ifreq ifr = { .ifr_mtu = mtu }; +int ret; +int s;
Quoted 8 lines
drivers/net/af_xdp/rte_eth_af_xdp.c line 658
Quoted 6 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +6
-0+ + s =socket (PF_INET ,SOCK_DGRAM ,0 ); +if (s <0 ) +return -EINVAL ; + +snprintf (ifr.ifr_name,IFNAMSIZ ,"%s" , internals->if_name);Can you please prefer strlcpy?
Sure.
Quoted 15 lines
Quoted 15 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +14
-0+ ret =ioctl (s,SIOCSIFMTU , &ifr); +close (s); + +if (ret <0 ) +return -EINVAL ; + +return 0 ; +} + +static void +eth_dev_change_flags (char *if_name,uint32_t flags,uint32_t mask) +{ +struct ifreq ifr; +int s;
Quoted 8 lines
drivers/net/af_xdp/rte_eth_af_xdp.c line 678
Quoted 6 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +6
-0+ + s =socket (PF_INET ,SOCK_DGRAM ,0 ); +if (s <0 ) +return ; + +snprintf (ifr.ifr_name,IFNAMSIZ ,"%s" , if_name);Can you please prefer strlcpy?
Sure.
Quoted 7 lines
<...>
drivers/net/af_xdp/rte_eth_af_xdp.c line 723
Quoted 2 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +2
-0+ +static struct rte_vdev_driver pmd_af_xdp_drv;Do we need this forward decleration?
Part of af_xdp pmd is refering to af_packet pmd, this is a simple copy from that driver, and as you point out, it should be unnecessary, will remove it.
Quoted 16 lines
drivers/net/af_xdp/rte_eth_af_xdp.c line 735
Quoted 13 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +12
-0+ +static void +parse_parameters (struct rte_kvargs *kvlist, +char **if_name, +int *queue_idx) +{ +struct rte_kvargs_pair *pair =NULL ; +unsigned int k_idx; + +for (k_idx =0 ; k_idx < kvlist->count; k_idx++) { + pair = &kvlist->pairs[k_idx]; +if (strstr (pair->key,ETH_AF_XDP_IFACE_ARG ))It is better to use '
rte_kvargs_process()' instead of accessing the 'kvargs' internals.
Will do.
Quoted 13 lines
Quoted 13 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +12
-0+ *if_name = pair->value; + else if (strstr(pair->key, ETH_AF_XDP_QUEUE_IDX_ARG)) + *queue_idx = atoi(pair->value); + } +} + +static int +get_iface_info (const char *if_name, +struct ether_addr *eth_addr, +int *if_index) +{ +struct ifreq ifr;
Quoted 8 lines
drivers/net/af_xdp/rte_eth_af_xdp.c line 753
Quoted 6 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +6
-0+int sock =socket (AF_INET ,SOCK_DGRAM ,IPPROTO_IP ); + +if (sock <0 ) +return -1 ; + +strcpy (ifr.ifr_name, if_name);Please prefer strlcpy.
Sure.
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c line 760
Quoted 8 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +7
-0+if (ioctl (sock,SIOCGIFINDEX , &ifr)) + goto error; + +if (ioctl (sock,SIOCGIFHWADDR , &ifr)) + goto error; + +memcpy (eth_addr, ifr.ifr_hwaddr.sa_data,6 );Can use '
ether_addr_copy()'
Got it.
Quoted 23 lines
Quoted 23 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +22
-0+ +close (sock); + *if_index =if_nametoindex (if_name); +return 0 ; + +error: +close (sock); +return -1 ; +} + +static int +init_internals (struct rte_vdev_device *dev, +const char *if_name, +int queue_idx) +{ +const char *name =rte_vdev_device_name (dev); +struct rte_eth_dev *eth_dev =NULL ; +const unsigned int numa_node = dev->device.numa_node; +struct pmd_internals *internals =NULL ; +int ret; +int i; +
Quoted 8 lines
drivers/net/af_xdp/rte_eth_af_xdp.c line 788
Quoted 6 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +6
-0+ internals =rte_zmalloc_socket (name,sizeof (*internals),0 , numa_node); +if (!internals) +return -ENOMEM ; + + internals->queue_idx = queue_idx; +strcpy (internals->if_name, if_name);prefer 'strlcpy' please
Got it.
Quoted 19 lines
Quoted 19 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +18
-0+ +for (i =0 ; i <ETH_AF_XDP_MAX_QUEUE_PAIRS ; i++) { + internals->tx_queues[i].pair = &internals->rx_queues[i]; + internals->rx_queues[i].pair = &internals->tx_queues[i]; + } + + ret =get_iface_info (if_name, &internals->eth_addr, + &internals->if_index); +if (ret) + goto err; + + eth_dev =rte_eth_vdev_allocate (dev,0 ); +if (!eth_dev) + goto err; + + eth_dev->data->dev_private = internals; + eth_dev->data->dev_link = pmd_link; + eth_dev->data->mac_addrs = &internals->eth_addr;
Quoted 8 lines
drivers/net/af_xdp/rte_eth_af_xdp.c line 811
Quoted 5 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +5
-0+ eth_dev->dev_ops = &ops; + eth_dev->rx_pkt_burst = eth_af_xdp_rx; + eth_dev->tx_pkt_burst = eth_af_xdp_tx; + +rte_eth_dev_probing_finish (eth_dev);What do you think moving this call into '
rte_pmd_af_xdp_probe' function if 'init_internals' returns sucess instead of setting here?
Sounds better, will do.
Quoted 10 lines
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +9
-0+return 0 ; + +err: +rte_free (internals); +return -1 ; +} + +static int +rte_pmd_af_xdp_probe (struct rte_vdev_device *dev)
Quoted 8 lines
drivers/net/af_xdp/rte_eth_af_xdp.c line 824
Quoted 4 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +4
-0+{ +struct rte_kvargs *kvlist; +char *if_name =NULL ; +int queue_idx =ETH_AF_XDP_DFLT_QUEUE_IDX ;This '
queue_idx' is for interface queue to pass to xsk_* API, also we have same variable name 'queue_idx' that we use for DPDK queue index, they get confused easily, what do you think rename this one something like 'xsk_queue_idx'?
Agree, xsk_queue_idx is a better name, will adopt.
Quoted 18 lines
Quoted 18 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +17
-0+struct rte_eth_dev *eth_dev; +const char *name; +int ret; + +RTE_LOG (INFO ,PMD ,"Initializing pmd_af_packet for %s\n" , +rte_vdev_device_name (dev)); + + name =rte_vdev_device_name (dev); +if (rte_eal_process_type () ==RTE_PROC_SECONDARY && +strlen (rte_vdev_device_args (dev)) ==0 ) { + eth_dev =rte_eth_dev_attach_secondary (name); +if (!eth_dev) { +RTE_LOG (ERR ,PMD ,"Failed to probe %s\n" , name); +return -EINVAL ; + } + eth_dev->dev_ops = &ops; +rte_eth_dev_probing_finish (eth_dev);
Quoted 8 lines
drivers/net/af_xdp/rte_eth_af_xdp.c line 847
Quoted 6 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +6
-0+ } + + kvlist =rte_kvargs_parse (rte_vdev_device_args (dev), valid_arguments); +if (!kvlist) { +RTE_LOG (ERR ,PMD , +"Invalid kvargs\n" );No need to break the line.
Got it.
Quoted 11 lines
drivers/net/af_xdp/rte_eth_af_xdp.c line 855
Quoted 9 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +8
-0+return -EINVAL ; + } + +if (dev->device.numa_node ==SOCKET_ID_ANY ) + dev->device.numa_node =rte_socket_id (); + +parse_parameters (kvlist, &if_name, + &queue_idx);Same, no need to break the line.
Got it.
Quoted 29 lines
Quoted 29 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +28
-0+ + ret =init_internals (dev, if_name, queue_idx); + +rte_kvargs_free (kvlist); + +return ret; +} + +static int +rte_pmd_af_xdp_remove (struct rte_vdev_device *dev) +{ +struct rte_eth_dev *eth_dev =NULL ; +struct pmd_internals *internals; + +RTE_LOG (INFO ,PMD ,"Removing AF_XDP ethdev on numa socket %u\n" , +rte_socket_id ()); + +if (!dev) +return -1 ; + +/* find the ethdev entry */ + eth_dev =rte_eth_dev_allocated (rte_vdev_device_name (dev)); +if (!eth_dev) +return -1 ; + + internals = eth_dev->data->dev_private; + +rte_ring_free (internals->umem->buf_ring);
Quoted 8 lines
drivers/net/af_xdp/rte_eth_af_xdp.c line 888
Quoted 5 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +5
-0+rte_free (internals->umem->buffer); +rte_free (internals->umem); +rte_free (internals); + +rte_eth_dev_release_port (eth_dev);This frees the '
eth_dev->data->dev_private', (internals), it can be problem to try to free same pointer twice.
Thanks for pointing out, will remove the rte_free(internals) to avoid a double free.
Quoted 15 lines
drivers/net/af_xdp/rte_eth_af_xdp.c line 900
Quoted 13 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +12
-0+ + +return 0 ; +} + +static struct rte_vdev_driver pmd_af_xdp_drv = { + .probe = rte_pmd_af_xdp_probe, + .remove = rte_pmd_af_xdp_remove, +}; + +RTE_PMD_REGISTER_VDEV (net_af_xdp, pmd_af_xdp_drv); +RTE_PMD_REGISTER_ALIAS (net_af_xdp, eth_af_xdp);No need to create the alias, it is for backward compatibility.
Got it.
Quoted 13 lines
drivers/net/af_xdp/rte_pmd_af_xdp_version.map line 1
Quoted 11 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +3
-0+RTE_PMD_REGISTER_PARAM_STRING (net_af_xdp, +"iface=<string> " +"queue=<int> " );drivers/net/af_xdp/rte_pmd_af_xdp_version.map new file +1
-0@@ -0,0 +1,4 @@ +DPDK_2.0 {Use release version please.
Got it.
Quoted 11 lines
Quoted 11 lines
drivers/net/af_xdp/rte_pmd_af_xdp_version.map new file +3
-0+ + local: *; +};mk/rte.app.mk +0
-0@@ -143,6 +143,7 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_DPAA2_MEMPOOL) += -lrte_mempool_dpaa2 endif
Quoted 4 lines
mk/rte.app.mk _LDLIBS-$(CONFIG_RTE_LIBRTE_DPAA2_MEMPOOL) += -lrte_mempool_dpaa2 · line 146
Quoted 2 lines
mk/rte.app.mk +1
-0_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AF_PACKET) += -lrte_pmd_af_packet +_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AF_XDP) += -lrte_pmd_af_xdp -lelf -lbpfFor which API libelf is required?
It is a leftover and will be removed.
Thanks,
Xiaolong
Quoted 1 line
Re: [PATCH v1 1/6] net/af_xdp: introduce AF_XDP PMD driver
Quoted 12 lines
Quoted 11 lines
<...>
doc/guides/nics/af_xdp.rst line 32
Quoted 6 lines
doc/guides/nics/af_xdp.rst new file +6
-0+Prerequisites +------------- + +This is a Linux-specific PMD, thus the following prerequisites apply: + +* A Linux Kernel with XDP sockets configuration enabled;Can you please give more details of what exact vanilla kernel version?
Quoted 2 lines
Do you mean I should write more details about
AF_XDPin kernel in this introduction document?
I think it is good to document the exact version information instead of saying "Linux Kernel with af_xdp".
Quoted 6 lines
Quoted 3 lines
Quoted 1 line
+* libbpf with latest
af_xdpsupport installedIs there a specific version of libbpf for this?
I'm not aware that there is specific version number for libbpf, it's part of linux kernel src code.
If it is coming with Linux kernel, which version of Linux kernel?
Quoted 14 lines
Quoted 10 lines
<...>
drivers/net/af_xdp/rte_eth_af_xdp.c line 15
Quoted 5 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +5
-0+ +#include <linux/if_ether.h> +#include <linux/if_xdp.h> +#include <linux/if_link.h> +#include <asm/barrier.h> Getting an build error for this [1], can there be any include path param missing?
Quoted 3 lines
[1]
drivers/net/af_xdp/rte_eth_af_xdp.c:15:10: fatal error:asm/barrier.h: No such file or directory
Quoted 5 lines
Yes, it need something like
CFLAGS += -I/root/yexl/shared_mks0/linux/tools/include
as in above
Makefilecurrently.
I see, assuming you will be booting up with that kernel, can something like below work:
CFLAGS += -I/lib/modules/$(shell uname -r)/build/tools/include/
Quoted 16 lines
Quoted 15 lines
<...>
drivers/net/af_xdp/rte_eth_af_xdp.c line 324
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+static void +eth_dev_info (struct rte_eth_dev *dev,struct rte_eth_dev_info *dev_info) +{ +struct pmd_internals *internals = dev->data->dev_private; + + dev_info->if_index = internals->if_index; + dev_info->max_mac_addrs =1 ; + dev_info->max_rx_pktlen = (uint32_t )ETH_FRAME_LEN ; + dev_info->max_rx_queues =1 ; + dev_info->max_tx_queues =1 ;'
ETH_AF_XDP_MAX_QUEUE_PAIRS' is '16' but you are forcing the max Rx/Tx queue number to be '1', intentional?
Quoted 2 lines
Yes, current implementation is single queue only, we plan to support muli-queues in futher.
OK, Can you please document this information?
Re: [PATCH v1 1/6] net/af_xdp: introduce AF_XDP PMD driver
Quoted 12 lines · Ferruh Yigit
Quoted 12 lines
Quoted 11 lines
<...>
doc/guides/nics/af_xdp.rst line 32
Quoted 6 lines
doc/guides/nics/af_xdp.rst new file +6
-0+Prerequisites +------------- + +This is a Linux-specific PMD, thus the following prerequisites apply: + +* A Linux Kernel with XDP sockets configuration enabled;Can you please give more details of what exact vanilla kernel version?
Quoted 5 lines
Quoted 2 lines
Do you mean I should write more details about
AF_XDPin kernel in this introduction document?I think it is good to document the exact version information instead of saying "Linux Kernel with
af_xdp".
Get your point now, will add the exact kernel info.
Quoted 9 lines
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
+* libbpf with latest
af_xdpsupport installedIs there a specific version of libbpf for this?
I'm not aware that there is specific version number for libbpf, it's part of linux kernel src code.
If it is coming with Linux kernel, which version of Linux kernel?
Will add the kernel version info.
Quoted 17 lines
Quoted 17 lines
Quoted 11 lines
<...>
drivers/net/af_xdp/rte_eth_af_xdp.c line 15
Quoted 5 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +5
-0+ +#include <linux/if_ether.h> +#include <linux/if_xdp.h> +#include <linux/if_link.h> +#include <asm/barrier.h> Getting an build error for this [1], can there be any include path param missing?
Quoted 3 lines
[1]
drivers/net/af_xdp/rte_eth_af_xdp.c:15:10: fatal error:asm/barrier.h: No such file or directoryYes, it need something like
Quoted 9 lines
Quoted 3 lines
CFLAGS += -I/root/yexl/shared_mks0/linux/tools/include
as in above
Makefilecurrently.I see, assuming you will be booting up with that kernel, can something like below work:
CFLAGS += -I/lib/modules/$(shell uname -r)/build/tools/include/
I'll have a try.
Quoted 16 lines
Quoted 16 lines
Quoted 15 lines
<...>
drivers/net/af_xdp/rte_eth_af_xdp.c line 324
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+static void +eth_dev_info (struct rte_eth_dev *dev,struct rte_eth_dev_info *dev_info) +{ +struct pmd_internals *internals = dev->data->dev_private; + + dev_info->if_index = internals->if_index; + dev_info->max_mac_addrs =1 ; + dev_info->max_rx_pktlen = (uint32_t )ETH_FRAME_LEN ; + dev_info->max_rx_queues =1 ; + dev_info->max_tx_queues =1 ;'
ETH_AF_XDP_MAX_QUEUE_PAIRS' is '16' but you are forcing the max Rx/Tx queue number to be '1', intentional?
Quoted 4 lines
Quoted 2 lines
Yes, current implementation is single queue only, we plan to support muli-queues in futher.
OK, Can you please document this information?
Sure.
Thanks,
Xiaolong
Quoted 1 line
Re: [PATCH v1 1/6] net/af_xdp: introduce AF_XDP PMD driver
In reply to Ye XiaolongQuoted 3 lines · Ye Xiaolong
Quoted 1 line
I can see in makefile, libelf is also linked, is it a dependency?
libelf is a leftover of RFC, will delete it in next version.
Correction, libbpf depends on libelf, so I still need to keep it.
Thanks,
Xiaolong
PATCH 2/6 lib/mbuf: enable parse flags when create mempool
In reply to Xiaolong YeThis give the option that applicaiton can configure each memory chunk's size precisely. (by MEMPOOL_F_NO_SPREAD).
--- lib/librte_mbuf/rte_mbuf.c | 15 ++++++++++++--- lib/librte_mbuf/rte_mbuf.h | 8 +++++++- 2 files changed, 19 insertions(+), 4 deletions(-)
lib/librte_mbuf/rte_mbuf.c +12 -3
@@ -110,7 +110,7 @@ rte_pktmbuf_init(struct rte_mempool *mp,struct rte_mempool *rte_pktmbuf_pool_create_by_ops (const char *name,unsigned int n,unsigned int cache_size,uint16_t priv_size,uint16_t data_room_size, -int socket_id,const char *ops_name) +unsigned int flags,int socket_id,const char *ops_name) {struct rte_mempool *mp;struct rte_pktmbuf_pool_private mbp_priv;
@@ -130,7 +130,7 @@ rte_pktmbuf_pool_create_by_ops(const char *name, unsigned int n, mbp_priv.mbuf_priv_size = priv_size; mp =rte_mempool_create_empty (name, n, elt_size, cache_size, -sizeof (struct rte_pktmbuf_pool_private ), socket_id,0 ); +sizeof (struct rte_pktmbuf_pool_private ), socket_id, flags);if (mp ==NULL )return NULL ;
@@ -164,9 +164,18 @@ rte_pktmbuf_pool_create(const char *name, unsigned int n, int socket_id) {return rte_pktmbuf_pool_create_by_ops (name, n, cache_size, priv_size, - data_room_size, socket_id,NULL ); + data_room_size,0 , socket_id,NULL ); } +/* helper to create a mbuf pool with NO_SPREAD */ +struct rte_mempool * +rte_pktmbuf_pool_create_with_flags (const char *name,unsigned int n, +unsigned int cache_size,uint16_t priv_size,uint16_t data_room_size, +unsigned int flags,int socket_id) +{ +return rte_pktmbuf_pool_create_by_ops (name, n, cache_size, priv_size, + data_room_size, flags, socket_id,NULL ); +}/* do some sanity checks on a mbuf: panic if it fails */ void rte_mbuf_sanity_check (const struct rte_mbuf *m,int is_header)
lib/librte_mbuf/rte_mbuf.h +7 -1
@@ -1266,6 +1266,12 @@ rte_pktmbuf_pool_create(const char *name, unsigned n, unsigned cache_size, uint16_t priv_size, uint16_t data_room_size, int socket_id); +struct rte_mempool * +rte_pktmbuf_pool_create_with_flags (const char *name,unsigned int n, +unsigned int cache_size,uint16_t priv_size,uint16_t data_room_size, +unsigned int flags,int socket_id); + +/** * Create a mbuf pool with a given mempool ops name *
@@ -1306,7 +1312,7 @@ rte_pktmbuf_pool_create(const char *name, unsigned n,struct rte_mempool *rte_pktmbuf_pool_create_by_ops (const char *name,unsigned int n,unsigned int cache_size,uint16_t priv_size,uint16_t data_room_size, -int socket_id,const char *ops_name); +unsigned int flags,int socket_id,const char *ops_name);/** * Get the data room size of mbufs stored in a pktmbuf_pool
Signature · 1 line
2.17.1
Re: [PATCH v1 2/6] lib/mbuf: enable parse flags when create mempool
Quoted 6 lines · Xiaolong Ye
This give the option that applicaiton can configure each memory chunk's size precisely. (by
MEMPOOL_F_NO_SPREAD).Signed-off-by Qi Zhang <[email protected]>Signed-off-by Xiaolong Ye <[email protected]>
---
Quoted 17 lines
lib/librte_mbuf/rte_mbuf.c | 15 ++++++++++++--- lib/librte_mbuf/rte_mbuf.h | 8 +++++++- 2 files changed, 19 insertions(+), 4 deletions(-)lib/librte_mbuf/rte_mbuf.c +1
-1@@ -110,7 +110,7 @@ rte_pktmbuf_init(struct rte_mempool *mp,struct rte_mempool *rte_pktmbuf_pool_create_by_ops (const char *name,unsigned int n,unsigned int cache_size,uint16_t priv_size,uint16_t data_room_size, -int socket_id,const char *ops_name) +unsigned int flags,int socket_id,const char *ops_name) {struct rte_mempool *mp;
Quoted 2 lines
struct rte_pktmbuf_pool_private mbp_priv;
You can't do that, rte_pktmbuf_pool_create_by_ops is exposed to the user apps and part of the ABI.
You must define a new internal fonction that takes this flag, keeps the existing interface and add your new experimental api.
Signature · 1 line
David Marchand
Re: [PATCH v1 2/6] lib/mbuf: enable parse flags when create mempool
Hi David,
Thanks for you comments.
Quoted 28 lines · David Marchand
Quoted 6 lines · Xiaolong Ye
This give the option that applicaiton can configure each memory chunk's size precisely. (by
MEMPOOL_F_NO_SPREAD).Signed-off-by Qi Zhang <[email protected]>Signed-off-by Xiaolong Ye <[email protected]>Cc maintainer---
Quoted 16 lines
lib/librte_mbuf/rte_mbuf.c | 15 ++++++++++++--- lib/librte_mbuf/rte_mbuf.h | 8 +++++++- 2 files changed, 19 insertions(+), 4 deletions(-)lib/librte_mbuf/rte_mbuf.c +1
-1@@ -110,7 +110,7 @@ rte_pktmbuf_init(struct rte_mempool *mp,struct rte_mempool *rte_pktmbuf_pool_create_by_ops (const char *name,unsigned int n,unsigned int cache_size,uint16_t priv_size,uint16_t data_room_size, -int socket_id,const char *ops_name) +unsigned int flags,int socket_id,const char *ops_name) {
Quoted 9 lines
Quoted 3 lines
lib/librte_mbuf/rte_mbuf.c +0
-0struct rte_mempool *mp;struct rte_pktmbuf_pool_private mbp_priv;You can't do that,
rte_pktmbuf_pool_create_by_opsis exposed to the user apps and part of the ABI.
You must define a new internal fonction that takes this flag, keeps the existing interface and add your new experimental api.
In this case, if I define a new function that takes the flag, it seems would have a lot of duplicated code with rte_pktmbuf_pool_create_by_ops, do you have any suggestions for better handling?
Thanks,
Xiaolong
Quoted 3 lines
Signature · 2 lines
-- David Marchand
PATCH 3/6 lib/mempool: allow page size aligned mempool
In reply to Xiaolong YeAllow create a mempool with page size aligned base address.
--- lib/librte_mempool/rte_mempool.c | 3 +++ lib/librte_mempool/rte_mempool.h | 1 + 2 files changed, 4 insertions(+)
lib/librte_mempool/rte_mempool.c +3 -0
@@ -543,6 +543,9 @@ rte_mempool_populate_default(struct rte_mempool *mp)if (try_contig) flags |=RTE_MEMZONE_IOVA_CONTIG ; +if (mp->flags &MEMPOOL_F_PAGE_ALIGN ) + align =getpagesize (); + mz =rte_memzone_reserve_aligned (mz_name, mem_size, mp->socket_id, flags, align);
lib/librte_mempool/rte_mempool.h +1 -0
@@ -264,6 +264,7 @@ struct rte_mempool {#define MEMPOOL_F_POOL_CREATED 0x0010/**< Internal: pool is created. */ #define MEMPOOL_F_NO_IOVA_CONTIG 0x0020/**< Don't need IOVA contiguous objs. */ #define MEMPOOL_F_NO_PHYS_CONTIG MEMPOOL_F_NO_IOVA_CONTIG/* deprecated */ +#define MEMPOOL_F_PAGE_ALIGN 0x0040/**< Chunk's base address is page aligned */ /** * @internal When debug is enabled, store some statistics.
Signature · 1 line
2.17.1
PATCH 4/6 net/af_xdp: use mbuf mempool for buffer management
In reply to Xiaolong YeNow, af_xdp registered memory buffer is managed by rte_mempool. mbuf be allocated from rte_mempool can be convert to xdp_desc's address and vice versa.
--- drivers/net/af_xdp/rte_eth_af_xdp.c | 121 +++++++++++++++++----------- 1 file changed, 75 insertions(+), 46 deletions(-)
drivers/net/af_xdp/rte_eth_af_xdp.c +75 -46
@@ -41,7 +41,11 @@#define ETH_AF_XDP_FRAME_SIZE XSK_UMEM__DEFAULT_FRAME_SIZE#define ETH_AF_XDP_NUM_BUFFERS 4096 -#define ETH_AF_XDP_DATA_HEADROOM 0 +/* mempool hdrobj size (64 bytes) + sizeof(struct rte_mbuf) (128 bytes) */ +#define ETH_AF_XDP_MBUF_OVERHEAD 192 +/* data start from offset 320 (192 + 128) bytes */ +#define ETH_AF_XDP_DATA_HEADROOM \ + (ETH_AF_XDP_MBUF_OVERHEAD + RTE_PKTMBUF_HEADROOM)#define ETH_AF_XDP_DFLT_NUM_DESCS XSK_RING_CONS__DEFAULT_NUM_DESCS#define ETH_AF_XDP_DFLT_QUEUE_IDX 0
@@ -54,7 +58,7 @@ struct xsk_umem_info {struct xsk_ring_prod fq;struct xsk_ring_cons cq;struct xsk_umem *umem; -struct rte_ring *buf_ring; +struct rte_mempool *mb_pool;void *buffer; };
@@ -108,12 +112,32 @@ static struct rte_eth_link pmd_link = { .link_autoneg = ETH_LINK_AUTONEG }; +static inline struct rte_mbuf * +addr_to_mbuf (struct xsk_umem_info *umem,uint64_t addr) +{ +uint64_t offset = (addr /ETH_AF_XDP_FRAME_SIZE * +ETH_AF_XDP_FRAME_SIZE ); +struct rte_mbuf *mbuf = (struct rte_mbuf *)((uint64_t )umem->buffer + + offset +ETH_AF_XDP_MBUF_OVERHEAD - +sizeof (struct rte_mbuf )); + mbuf->data_off = addr - offset -ETH_AF_XDP_MBUF_OVERHEAD ; +return mbuf; +} + +static inline uint64_t +mbuf_to_addr (struct xsk_umem_info *umem,struct rte_mbuf *mbuf) +{ +return (uint64_t )mbuf->buf_addr + mbuf->data_off - + (uint64_t )umem->buffer; +} +static inline int reserve_fill_queue (struct xsk_umem_info *umem,int reserve_size) {struct xsk_ring_prod *fq = &umem->fq; +struct rte_mbuf *mbuf;uint32_t idx; -void *addr =NULL ; +uint64_t addr;int i, ret =0 ; ret =xsk_ring_prod__reserve (fq, reserve_size, &idx);
@@ -123,8 +147,9 @@ reserve_fill_queue(struct xsk_umem_info *umem, int reserve_size) }for (i =0 ; i < reserve_size; i++) { -rte_ring_dequeue (umem->buf_ring, &addr); - *xsk_ring_prod__fill_addr (fq, idx++) = (uint64_t )addr; + mbuf =rte_pktmbuf_alloc (umem->mb_pool); + addr =mbuf_to_addr (umem, mbuf); + *xsk_ring_prod__fill_addr (fq, idx++) = addr; }xsk_ring_prod__submit (fq, reserve_size);
@@ -172,7 +197,7 @@ eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) }else { dropped++; } -rte_ring_enqueue (umem->buf_ring, (void *)addr); +rte_pktmbuf_free (addr_to_mbuf (umem, addr)); }xsk_ring_cons__release (rx, rcvd);
@@ -195,9 +220,8 @@ static void pull_umem_cq(struct xsk_umem_info *umem, int size) n =xsk_ring_cons__peek (cq, size, &idx_cq);if (n >0 ) {for (i =0 ; i < n; i++) { - addr = *xsk_ring_cons__comp_addr (cq, - idx_cq++); -rte_ring_enqueue (umem->buf_ring, (void *)addr); + addr = *xsk_ring_cons__comp_addr (cq, idx_cq++); +rte_pktmbuf_free (addr_to_mbuf (umem, addr)); }xsk_ring_cons__release (cq, n);
@@ -233,7 +257,7 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)struct pkt_tx_queue *txq = queue;struct xsk_umem_info *umem = txq->pair->umem;struct rte_mbuf *mbuf; -void *addrs[ETH_AF_XDP_TX_BATCH_SIZE ]; +struct rte_mbuf *mbuf_to_tx;unsigned long tx_bytes =0 ;int i, valid =0 ;uint32_t idx_tx;
@@ -243,11 +267,6 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)pull_umem_cq (umem, nb_pkts); - nb_pkts =rte_ring_dequeue_bulk (umem->buf_ring, addrs, - nb_pkts,NULL ); -if (!nb_pkts) -return 0 ; -if (xsk_ring_prod__reserve (&txq->tx, nb_pkts, &idx_tx) != nb_pkts)return 0 ;
@@ -260,7 +279,12 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) desc =xsk_ring_prod__tx_desc (&txq->tx, idx_tx + i); mbuf = bufs[i];if (mbuf->pkt_len <= buf_len) { - desc->addr = (uint64_t )addrs[valid]; + mbuf_to_tx =rte_pktmbuf_alloc (umem->mb_pool); +if (!mbuf_to_tx) { +rte_pktmbuf_free (mbuf); +continue ; + } + desc->addr =mbuf_to_addr (umem, mbuf_to_tx); desc->len = mbuf->pkt_len; pkt =xsk_umem__get_data (umem->buffer, desc->addr);
@@ -276,10 +300,6 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)kick_tx (txq); -if (valid < nb_pkts) -rte_ring_enqueue_bulk (umem->buf_ring, &addrs[valid], - nb_pkts - valid,NULL ); - txq->err_pkts += nb_pkts - valid; txq->tx_pkts += valid; txq->tx_bytes += tx_bytes;
@@ -429,12 +449,28 @@ eth_link_update(struct rte_eth_dev *dev __rte_unused,static void xdp_umem_destroy (struct xsk_umem_info *umem) { -if (umem->buffer) -free (umem->buffer); +if (umem->mb_pool) +rte_mempool_free (umem->mb_pool);free (umem); } +static inline uint64_t get_base_addr (struct rte_mempool *mp) +{ +struct rte_mempool_memhdr *memhdr; + + memhdr =STAILQ_FIRST (&mp->mem_list); +return (uint64_t )(memhdr->addr); +} + +static inline uint64_t get_len (struct rte_mempool *mp) +{ +struct rte_mempool_memhdr *memhdr; + + memhdr =STAILQ_FIRST (&mp->mem_list); +return (uint64_t )(memhdr->len); +} +static struct xsk_umem_info *xdp_umem_configure (void ) {struct xsk_umem_info *umem;
@@ -443,10 +479,9 @@ static struct xsk_umem_info *xdp_umem_configure(void) .comp_size = ETH_AF_XDP_DFLT_NUM_DESCS, .frame_size = ETH_AF_XDP_FRAME_SIZE, .frame_headroom = ETH_AF_XDP_DATA_HEADROOM }; -void *bufs =NULL ; -char ring_name[0x100 ]; +void *base_addr =NULL ; +char pool_name[0x100 ];int ret; -uint64_t i; umem =calloc (1 ,sizeof (*umem));if (!umem) {
@@ -454,28 +489,23 @@ static struct xsk_umem_info *xdp_umem_configure(void)return NULL ; } -snprintf (ring_name,0x100 ,"af_xdp_ring" ); - umem->buf_ring =rte_ring_create (ring_name, -ETH_AF_XDP_NUM_BUFFERS , -SOCKET_ID_ANY , -0x0 ); -if (!umem->buf_ring) { +snprintf (pool_name,0x100 ,"af_xdp_ring" ); + umem->mb_pool =rte_pktmbuf_pool_create_with_flags (pool_name, +ETH_AF_XDP_NUM_BUFFERS , +250 ,0 , +ETH_AF_XDP_FRAME_SIZE - +ETH_AF_XDP_MBUF_OVERHEAD , +MEMPOOL_F_NO_SPREAD |MEMPOOL_F_PAGE_ALIGN , +SOCKET_ID_ANY ); + +if (!umem->mb_pool || umem->mb_pool->nb_mem_chunks !=1 ) {RTE_LOG (ERR ,PMD , -"Failed to create rte_ring\n" ); +"Failed to create rte_mempool\n" ); goto err; } + base_addr = (void *)get_base_addr (umem->mb_pool); -for (i =0 ; i <ETH_AF_XDP_NUM_BUFFERS ; i++) -rte_ring_enqueue (umem->buf_ring, - (void *)(i *ETH_AF_XDP_FRAME_SIZE + -ETH_AF_XDP_DATA_HEADROOM )); - -if (posix_memalign (&bufs,getpagesize (), -ETH_AF_XDP_NUM_BUFFERS *ETH_AF_XDP_FRAME_SIZE )) { -RTE_LOG (ERR ,PMD ,"Failed to allocate memory pool.\n" ); - goto err; - } - ret =xsk_umem__create (&umem->umem, bufs, + ret =xsk_umem__create (&umem->umem, base_addr,ETH_AF_XDP_NUM_BUFFERS *ETH_AF_XDP_FRAME_SIZE , &umem->fq, &umem->cq, &usr_config);
@@ -484,7 +514,7 @@ static struct xsk_umem_info *xdp_umem_configure(void)RTE_LOG (ERR ,PMD ,"Failed to create umem" ); goto err; } - umem->buffer = bufs; + umem->buffer = base_addr;return umem;
@@ -880,8 +910,7 @@ rte_pmd_af_xdp_remove(struct rte_vdev_device *dev) internals = eth_dev->data->dev_private; -rte_ring_free (internals->umem->buf_ring); -rte_free (internals->umem->buffer); +rte_mempool_free (internals->umem->mb_pool);rte_free (internals->umem);rte_free (internals);
Signature · 1 line
2.17.1
PATCH 5/6 net/af_xdp: enable zero copy
In reply to Xiaolong YeTry to check if external mempool (from rx_queue_setup) is fit for af_xdp, if it is, it will be registered to af_xdp socket directly and there will be no packet data copy on Rx and Tx.
--- drivers/net/af_xdp/rte_eth_af_xdp.c | 126 ++++++++++++++++++++-------- 1 file changed, 91 insertions(+), 35 deletions(-)
drivers/net/af_xdp/rte_eth_af_xdp.c +91 -35
@@ -60,6 +60,7 @@ struct xsk_umem_info {struct xsk_umem *umem;struct rte_mempool *mb_pool;void *buffer; +uint8_t zc; };struct pkt_rx_queue {
@@ -74,6 +75,7 @@ struct pkt_rx_queue {struct pkt_tx_queue *pair;uint16_t queue_idx; +uint8_t zc; };struct pkt_tx_queue {
@@ -187,17 +189,24 @@ eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)uint32_t len =xsk_ring_cons__rx_desc (rx, idx_rx++)->len;char *pkt =xsk_umem__get_data (rxq->umem->buffer, addr); - mbuf =rte_pktmbuf_alloc (rxq->mb_pool); -if (mbuf) { - memcpy(rte_pktmbuf_mtod(mbuf, void*), pkt, len); +if (rxq->zc) { + mbuf =addr_to_mbuf (rxq->umem, addr);rte_pktmbuf_pkt_len (mbuf) =rte_pktmbuf_data_len (mbuf) = len; - rx_bytes += len; bufs[count++] = mbuf; }else { - dropped++; + mbuf =rte_pktmbuf_alloc (rxq->mb_pool); +if (mbuf) { + memcpy(rte_pktmbuf_mtod(mbuf, void*), pkt, len); +rte_pktmbuf_pkt_len (mbuf) = +rte_pktmbuf_data_len (mbuf) = len; + rx_bytes += len; + bufs[count++] = mbuf; + }else { + dropped++; + } +rte_pktmbuf_free (addr_to_mbuf (umem, addr)); } -rte_pktmbuf_free (addr_to_mbuf (umem, addr)); }xsk_ring_cons__release (rx, rcvd);
@@ -278,22 +287,29 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) -ETH_AF_XDP_DATA_HEADROOM ; desc =xsk_ring_prod__tx_desc (&txq->tx, idx_tx + i); mbuf = bufs[i]; -if (mbuf->pkt_len <= buf_len) { - mbuf_to_tx =rte_pktmbuf_alloc (umem->mb_pool); -if (!mbuf_to_tx) { -rte_pktmbuf_free (mbuf); -continue ; - } - desc->addr =mbuf_to_addr (umem, mbuf_to_tx); +if (txq->pair->zc && mbuf->pool == umem->mb_pool) { + desc->addr =mbuf_to_addr (umem, mbuf); desc->len = mbuf->pkt_len; - pkt =xsk_umem__get_data (umem->buffer, - desc->addr); - memcpy(pkt, rte_pktmbuf_mtod(mbuf, void *), - desc->len); valid++; tx_bytes += mbuf->pkt_len; + }else { +if (mbuf->pkt_len <= buf_len) { + mbuf_to_tx =rte_pktmbuf_alloc (umem->mb_pool); +if (!mbuf_to_tx) { +rte_pktmbuf_free (mbuf); +continue ; + } + desc->addr =mbuf_to_addr (umem, mbuf_to_tx); + desc->len = mbuf->pkt_len; + pkt =xsk_umem__get_data (umem->buffer, + desc->addr); + memcpy(pkt, rte_pktmbuf_mtod(mbuf, void *), + desc->len); + valid++; + tx_bytes += mbuf->pkt_len; + } +rte_pktmbuf_free (mbuf); } -rte_pktmbuf_free (mbuf); }xsk_ring_prod__submit (&txq->tx, nb_pkts);
@@ -471,7 +487,7 @@ static inline uint64_t get_len(struct rte_mempool *mp)return (uint64_t )(memhdr->len); } -static struct xsk_umem_info *xdp_umem_configure (void ) +static struct xsk_umem_info *xdp_umem_configure (struct rte_mempool *mb_pool) {struct xsk_umem_info *umem;struct xsk_umem_config usr_config = {
@@ -489,20 +505,26 @@ static struct xsk_umem_info *xdp_umem_configure(void)return NULL ; } -snprintf (pool_name,0x100 ,"af_xdp_ring" ); - umem->mb_pool =rte_pktmbuf_pool_create_with_flags (pool_name, -ETH_AF_XDP_NUM_BUFFERS , -250 ,0 , -ETH_AF_XDP_FRAME_SIZE - -ETH_AF_XDP_MBUF_OVERHEAD , -MEMPOOL_F_NO_SPREAD |MEMPOOL_F_PAGE_ALIGN , -SOCKET_ID_ANY ); - -if (!umem->mb_pool || umem->mb_pool->nb_mem_chunks !=1 ) { -RTE_LOG (ERR ,PMD , -"Failed to create rte_mempool\n" ); - goto err; +if (!mb_pool) { +snprintf (pool_name,0x100 ,"af_xdp_ring" ); + umem->mb_pool =rte_pktmbuf_pool_create_with_flags (pool_name, +ETH_AF_XDP_NUM_BUFFERS , +250 ,0 , +ETH_AF_XDP_FRAME_SIZE - +ETH_AF_XDP_MBUF_OVERHEAD , +MEMPOOL_F_NO_SPREAD |MEMPOOL_F_PAGE_ALIGN , +SOCKET_ID_ANY ); + +if (!umem->mb_pool || umem->mb_pool->nb_mem_chunks !=1 ) { +RTE_LOG (ERR ,PMD , +"Failed to create rte_mempool\n" ); + goto err; + } + }else { + umem->mb_pool = mb_pool; + umem->zc =1 ; } + base_addr = (void *)get_base_addr (umem->mb_pool); ret = xsk_umem__create(&umem->umem, base_addr,
@@ -523,16 +545,43 @@ static struct xsk_umem_info *xdp_umem_configure(void)return NULL ; } +static uint8_t +check_mempool_zc (struct rte_mempool *mp) +{ +RTE_ASSERT (mp); + +/* must continues */ +if (mp->nb_mem_chunks >1 ) +return 0 ; + +/* check header size */ +if (mp->header_size !=RTE_CACHE_LINE_SIZE ) +return 0 ; + +/* check base address */ +if ((uint64_t )get_base_addr (mp) %getpagesize () !=0 ) +return 0 ; + +/* check chunk size */ +if ((mp->elt_size + mp->header_size + mp->trailer_size) % +ETH_AF_XDP_FRAME_SIZE !=0 ) +return 0 ; + +return 1 ; +} +static int xsk_configure (struct pmd_internals *internals,struct pkt_rx_queue *rxq, -int ring_size) +int ring_size,struct rte_mempool *mb_pool) {struct xsk_socket_config cfg;struct pkt_tx_queue *txq = rxq->pair; +struct rte_mempool *mp;int ret =0 ;int reserve_size; - rxq->umem =xdp_umem_configure (); + mp =check_mempool_zc (mb_pool) ? mb_pool :NULL ; + rxq->umem =xdp_umem_configure (mp);if (!rxq->umem) { ret = -ENOMEM ; goto err;
@@ -633,7 +682,7 @@ eth_rx_queue_setup(struct rte_eth_dev *dev, rxq->mb_pool = mb_pool; -if (xsk_configure (internals, rxq, nb_rx_desc)) { +if (xsk_configure (internals, rxq, nb_rx_desc, mb_pool)) {RTE_LOG (ERR ,PMD ,"Failed to configure xdp socket\n" ); ret = -EINVAL ;
@@ -642,6 +691,13 @@ eth_rx_queue_setup(struct rte_eth_dev *dev, internals->umem = rxq->umem; +if (mb_pool == internals->umem->mb_pool) + rxq->zc = internals->umem->zc; + +if (rxq->zc) +RTE_LOG (INFO ,PMD , +"zero copy enabled on rx queue %d\n" , rx_queue_id); + dev->data->rx_queues[rx_queue_id] = rxq;return 0 ;
Signature · 1 line
2.17.1
PATCH 6/6 app/testpmd: add mempool flags parameter
In reply to Xiaolong YeWhen create rte_mempool, flags can be parsed from command line. Now, it is possible for testpmd to create a af_xdp friendly mempool (which enable zero copy).
--- app/test-pmd/parameters.c | 12 ++++++++++++ app/test-pmd/testpmd.c | 17 ++++++++++------- app/test-pmd/testpmd.h | 1 + 3 files changed, 23 insertions(+), 7 deletions(-)
app/test-pmd/parameters.c +12 -0
@@ -61,6 +61,7 @@ usage(char* progname)"--tx-first | --stats-period=PERIOD | " "--coremask=COREMASK --portmask=PORTMASK --numa " "--mbuf-size= | --total-num-mbufs= | " +"--mp-flags= | " "--nb-cores= | --nb-ports= | " #ifdef RTE_LIBRTE_CMDLINE"--eth-peers-configfile= | "
@@ -105,6 +106,7 @@ usage(char* progname)printf (" --socket-num=N: set socket from which all memory is allocated " "in NUMA mode.\n" );printf (" --mbuf-size=N: set the data size of mbuf to N bytes.\n" ); +printf (" --mp-flags=N: set the flags when create mbuf memory pool.\n" );printf (" --total-num-mbufs=N: set the number of mbufs to be allocated " "in mbuf pools.\n" );printf (" --max-pkt-len=N: set the maximum size of packet to N bytes.\n" );
@@ -585,6 +587,7 @@ launch_args_parse(int argc, char** argv) { "ring-numa-config", 1, 0, 0 }, { "socket-num", 1, 0, 0 }, { "mbuf-size", 1, 0, 0 }, + { "mp-flags", 1, 0, 0 }, { "total-num-mbufs", 1, 0, 0 }, { "max-pkt-len", 1, 0, 0 }, { "pkt-filter-mode", 1, 0, 0 },
@@ -811,6 +814,15 @@ launch_args_parse(int argc, char** argv)rte_exit (EXIT_FAILURE ,"mbuf-size should be > 0 and < 65536\n" ); } +if (!strcmp (lgopts[opt_idx].name,"mp-flags" )) { + n =atoi (optarg); +if (n >0 && n <=0xFFFF ) + mp_flags = (uint16_t )n; +else +rte_exit (EXIT_FAILURE , +"mp-flags should be > 0 and < 65536\n" ); + } +if (!strcmp (lgopts[opt_idx].name,"total-num-mbufs" )) { n =atoi (optarg);if (n >1024 )
app/test-pmd/testpmd.c +10 -7
@@ -195,6 +195,7 @@ uint32_t burst_tx_delay_time = BURST_TX_WAIT_US;uint32_t burst_tx_retry_num =BURST_TX_RETRIES ;uint16_t mbuf_data_size =DEFAULT_MBUF_DATA_SIZE ;/**< Mbuf data space size. */ +uint16_t mp_flags =0 ;/**< flags parsed when create mempool */ uint32_t param_total_num_mbufs =0 ;/**< number of mbufs in all pools - if * specified on command-line. */ uint16_t stats_period;/**< Period to show statistics (disabled by default) */
@@ -834,6 +835,7 @@ setup_extmem(uint32_t nb_mbufs, uint32_t mbuf_sz, bool huge)*/ static void mbuf_pool_create (uint16_t mbuf_seg_size,unsigned nb_mbuf, +unsigned int flags,unsigned int socket_id) {char pool_name[RTE_MEMPOOL_NAMESIZE ];
@@ -853,8 +855,8 @@ mbuf_pool_create(uint16_t mbuf_seg_size, unsigned nb_mbuf,/* wrapper to rte_mempool_create() */ TESTPMD_LOG (INFO ,"preferred mempool ops selected: %s\n" ,rte_mbuf_best_mempool_ops ()); - rte_mp =rte_pktmbuf_pool_create (pool_name, nb_mbuf, - mb_mempool_cache,0 , mbuf_seg_size, socket_id); + rte_mp =rte_pktmbuf_pool_create_with_flags (pool_name, nb_mbuf, + mb_mempool_cache,0 , mbuf_seg_size, flags, socket_id);break ; }case MP_ALLOC_ANON :
@@ -891,8 +893,8 @@ mbuf_pool_create(uint16_t mbuf_seg_size, unsigned nb_mbuf,TESTPMD_LOG (INFO ,"preferred mempool ops selected: %s\n" ,rte_mbuf_best_mempool_ops ()); - rte_mp =rte_pktmbuf_pool_create (pool_name, nb_mbuf, - mb_mempool_cache,0 , mbuf_seg_size, + rte_mp =rte_pktmbuf_pool_create_with_flags (pool_name, nb_mbuf, + mb_mempool_cache,0 , mbuf_seg_size, flags, heap_socket);break ; }
@@ -1128,13 +1130,14 @@ init_config(void)for (i =0 ; i < num_sockets; i++)mbuf_pool_create (mbuf_data_size, nb_mbuf_per_pool, - socket_ids[i]); + mp_flags, socket_ids[i]); }else {if (socket_num ==UMA_NO_CONFIG ) -mbuf_pool_create (mbuf_data_size, nb_mbuf_per_pool,0 ); +mbuf_pool_create (mbuf_data_size, nb_mbuf_per_pool, + mp_flags,0 );else mbuf_pool_create (mbuf_data_size, nb_mbuf_per_pool, - socket_num); + mp_flags, socket_num); }init_port_config ();
app/test-pmd/testpmd.h +1 -0
@@ -408,6 +408,7 @@ extern uint8_t dcb_config;extern uint8_t dcb_test;extern uint16_t mbuf_data_size;/**< Mbuf data space size. */ +extern uint16_t mp_flags;/**< flags for mempool creation. */ extern uint32_t param_total_num_mbufs;extern uint16_t stats_period;
Signature · 1 line
2.17.1
Re: [PATCH v1 6/6] app/testpmd: add mempool flags parameter
Quoted 6 lines · On Fri, 1 Mar 2019 16:09:47 +0800 Xiaolong Ye <[email protected]> wrote:
When create
rte_mempool, flags can be parsed from command line. Now, it is possible for testpmd to create aaf_xdpfriendly mempool (which enable zero copy).Signed-off-by Qi Zhang <[email protected]>Signed-off-by Xiaolong Ye <[email protected]>
Please don't expose numeric values to user api
Re: [PATCH v1 6/6] app/testpmd: add mempool flags parameter
Hi,
Quoted 11 lines · Stephen Hemminger
Quoted 6 lines · On Fri, 1 Mar 2019 16:09:47 +0800 Xiaolong Ye <[email protected]> wrote:
When create
rte_mempool, flags can be parsed from command line. Now, it is possible for testpmd to create aaf_xdpfriendly mempool (which enable zero copy).Signed-off-by Qi Zhang <[email protected]>Signed-off-by Xiaolong Ye <[email protected]>Please don't expose numeric values to user api
Do you mean instead of "--mp-flags=<number>", we should use something like "--mp_flags=<strings_of_flags>"?
Thanks,
Xiaolong
Re: [PATCH v1 6/6] app/testpmd: add mempool flags parameter
In reply to Xiaolong YeQuoted 10 lines · Xiaolong Ye
When create
rte_mempool, flags can be parsed from command line. Now, it is possible for testpmd to create aaf_xdpfriendly mempool (which enable zero copy).Signed-off-by Qi Zhang <[email protected]>Signed-off-by Xiaolong Ye <[email protected]>--- app/test-pmd/parameters.c | 12 ++++++++++++ app/test-pmd/testpmd.c | 17 ++++++++++------- app/test-pmd/testpmd.h | 1 +
Can you please document new added command line option in doc/guides/testpmd_app_ug/run_app.rst ?
Re: [PATCH v1 6/6] app/testpmd: add mempool flags parameter
Quoted 12 lines · Ferruh Yigit
Quoted 10 lines · Xiaolong Ye
When create
rte_mempool, flags can be parsed from command line. Now, it is possible for testpmd to create aaf_xdpfriendly mempool (which enable zero copy).Signed-off-by Qi Zhang <[email protected]>Signed-off-by Xiaolong Ye <[email protected]>--- app/test-pmd/parameters.c | 12 ++++++++++++ app/test-pmd/testpmd.c | 17 ++++++++++------- app/test-pmd/testpmd.h | 1 +
Quoted 2 lines
Can you please document new added command line option in
doc/guides/testpmd_app_ug/run_app.rst?
Got it, will add in next version.
Thanks,
Xiaolong
Quoted 1 line
Quoted 11 lines · Xiaolong Ye
Overview
========This patchset adds a new PMD driver for
AF_XDPwhich is a proposed faster version ofAF_PACKETinterface in Linux, see below links [1] [2] for details ofAF_XDPintroduction:
AF_XDProadmap
==============
-AF_XDPis included in upstream kernel since 4.18, andAF_XDPsupport in libbpf has been merged in bpf-next/master. [3]
And it seems it has been merged into main repo [1], I assume it will be part of 5.1, which I guess will be released mid May.
And we have release on 10 May. Taking into account that libbpf APIs used extensively, does it mean we can't release af_xdp on 19.05?
<...>
11/03/2019 17:43, Ferruh Yigit:
Quoted 13 lines · Xiaolong Ye
Quoted 11 lines
Overview
========This patchset adds a new PMD driver for
AF_XDPwhich is a proposed faster version ofAF_PACKETinterface in Linux, see below links [1] [2] for details ofAF_XDPintroduction:
AF_XDProadmap
==============
-AF_XDPis included in upstream kernel since 4.18, andAF_XDPsupport in libbpf has been merged in bpf-next/master. [3]
Quoted 8 lines
And it seems it has been merged into main repo [1], I assume it will be part of 5.1, which I guess will be released mid May.
And we have release on 10 May. Taking into account that libbpf APIs used extensively, does it mean we can't release
af_xdpon 19.05?
I think the requirement is to have all dependencies upstream, so we avoid releasing a feature not working when dependency is ready.
If all is ready in Linux mainline branch, I guess we are fine, are we?
+ Magnus & Bjorn who can give more accurate comment about kernel upstream status.
Quoted 37 lines
-----Original Message-----
From: Thomas Monjalon [mailto:[email protected]]
Sent: Tuesday, March 12, 2019 1:20 AM
To: Yigit, Ferruh <[email protected]>; Ye, Xiaolong <[email protected]>
Cc: [email protected]; Zhang, Qi Z <[email protected]>; Richardson, Bruce <[email protected]>
Subject: Re: [dpdk-dev] [PATCH v1 0/6] IntroduceAF_XDPPMD11/03/2019 17:43, Ferruh Yigit:
Quoted 16 lines · Xiaolong Ye
Quoted 11 lines
Overview
========This patchset adds a new PMD driver for
AF_XDPwhich is a proposed faster version ofAF_PACKETinterface in Linux, see below links [1]
[2] for details ofAF_XDPintroduction:
AF_XDProadmap
==============
-AF_XDPis included in upstream kernel since 4.18, andAF_XDPsupport in libbpf has been merged in bpf-next/master. [3]And it seems it has been merged into main repo [1], I assume it will be part of 5.1, which I guess will be released mid May.
Quoted 6 lines
And we have release on 10 May. Taking into account that libbpf APIs used extensively, does it mean we can't release
af_xdpon 19.05?[1]
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/com
mit?id=1cad078842396f0047a796694b6130fc096d97e2I think the requirement is to have all dependencies upstream, so we avoid releasing a feature not working when dependency is ready.
If all is ready in Linux mainline branch, I guess we are fine, are we?
Quoted 12 lines
-----Original Message-----
From: Zhang, Qi Z
Sent: Tuesday, March 12, 2019 2:52 AM
To: Thomas Monjalon <[email protected]>; Yigit, Ferruh <[email protected]>; Ye, Xiaolong <[email protected]>
Cc: [email protected]; Richardson, Bruce <[email protected]>; Karlsson, Magnus <[email protected]>; Topel, Bjorn <[email protected]>
Subject: RE: [dpdk-dev] [PATCH v1 0/6] IntroduceAF_XDPPMD
Quoted 42 lines
+ Magnus & Bjorn who can give more accurate comment about kernel upstream status.
Quoted 38 lines
-----Original Message-----
From: Thomas Monjalon [mailto:[email protected]]
Sent: Tuesday, March 12, 2019 1:20 AM
To: Yigit, Ferruh <[email protected]>; Ye, Xiaolong <[email protected]>
Cc: [email protected]; Zhang, Qi Z <[email protected]>; Richardson, Bruce <[email protected]>
Subject: Re: [dpdk-dev] [PATCH v1 0/6] IntroduceAF_XDPPMD11/03/2019 17:43, Ferruh Yigit:
Quoted 16 lines · Xiaolong Ye
Quoted 11 lines
Overview
========This patchset adds a new PMD driver for
AF_XDPwhich is a proposed faster version ofAF_PACKETinterface in Linux, see below links
[1] [2] for details ofAF_XDPintroduction:
AF_XDProadmap
==============
-AF_XDPis included in upstream kernel since 4.18, andAF_XDPsupport in libbpf has been merged in bpf-next/master. [3]And it seems it has been merged into main repo [1], I assume it will be part of 5.1, which I guess will be released mid May.
Quoted 7 lines
And we have release on 10 May. Taking into account that libbpf APIs used extensively, does it mean we can't release
af_xdpon 19.05?[1]
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/c
om
mit?id=1cad078842396f0047a796694b6130fc096d97e2I think the requirement is to have all dependencies upstream, so we avoid releasing a feature not working when dependency is ready.
If all is ready in Linux mainline branch, I guess we are fine, are we?
Libbpf is in linux-next which means it will be in the 5.1 release. This release will likely be in the end of April if we count the normal 8 week cadence of releases. 8 weeks means April 28, 9 weeks = May 5. 10 weeks = May 12. So unless we go all the way to 10 weeks, this feature should be in a stable release before DPDK 19.05.
/Magnus
v2 cover Introduce AF_XDP PMD
In reply to Xiaolong YeOverview
========
This patchset adds a new PMD driver for AF_XDP which is a proposed faster version of AF_PACKET interface in Linux, see below links [1] [2] for details of AF_XDP introduction:
AF_XDP roadmap
==============
- AF_XDP is included in upstream kernel since 4.18, and AF_XDP support in libbpf has been merged in v5.1-rc1.
- Now i40e and ixgbe drivers have supported zero copy mode.
Change logs
===========
V2:
- Fix a NULL pointer reference crash issue
- Fix txonly stop sending traffic in zc mode
- Refactor rte_mbuf.c to avoid ABI breakage.
- Fix multiple style issues pointed by Ferruh, David, Stephen, Luca.
changes vs RFC sent by Qi last Aug:
- Re-work base on AF_XDP's interface changes since the new libbpf has provided higher-level APIs that hide many of the details of the AP_XDP uapi. After the rework, it helps to reduce 300+ lines of code.
- multi-queues is not supported due to current libbpf doesn't support multi-sockets on a single umem.
- No extra steps to load xdp program manually, since the current behavior of libbpf would load a default xdp program when user calls xsk_socket__create. userspace application only needs to handle the cleanup.
How to try
==========
1. take the kernel >= v5.1-rc1, build kernel and replace your host kernel with it.
make sure you turn on XDP sockets when compiling
Networking support -->
Networking options -->
[ * ] XDP sockets
2. build libbpf in tools/lib/bpf, and copy the libbpf.a and libbpf.so to /usr/lib64
cd tools/lib/bpf
make
3. ethtool -L enp59s0f0 combined 1
4. start testpmd
./build/app/testpmd -c 0xc -n 4 --vdev eth_af_xdp,iface=enp59s0f0,queue=0 -- -i --rxq=1 --txq=1
in this case, default xdp program will be loaded and linked to queue 0 of enp59s0f0, network traffics travel to queue 0 will be redirected to af_xdp socket.
[1] https://lwn.net/Articles/750845/
[2] https://fosdem.org/2018/schedule/event/af_xdp/
Xiaolong Ye (6):
net/af_xdp: introduce AF XDP PMD driver lib/mbuf: introduce helper to create mempool with flags lib/mempool: allow page size aligned mempool net/af_xdp: use mbuf mempool for buffer management net/af_xdp: enable zero copy
app/testpmd: add mempool flags parameter
MAINTAINERS | 6 + app/test-pmd/parameters.c | 12 + app/test-pmd/testpmd.c | 17 +- app/test-pmd/testpmd.h | 1 + config/common_base | 5 + config/common_linux | 1 + doc/guides/nics/af_xdp.rst | 45 + doc/guides/nics/features/af_xdp.ini | 11 + doc/guides/nics/index.rst | 1 + doc/guides/rel_notes/release_19_05.rst | 7 + doc/guides/testpmd_app_ug/run_app.rst | 4 + drivers/net/Makefile | 1 + drivers/net/af_xdp/Makefile | 33 + drivers/net/af_xdp/meson.build | 21 + drivers/net/af_xdp/rte_eth_af_xdp.c | 1013 +++++++++++++++++ drivers/net/af_xdp/rte_pmd_af_xdp_version.map | 3 + drivers/net/meson.build | 1 + lib/librte_mbuf/rte_mbuf.c | 29 +- lib/librte_mbuf/rte_mbuf.h | 45 + lib/librte_mempool/rte_mempool.c | 3 + lib/librte_mempool/rte_mempool.h | 1 + mk/rte.app.mk | 1 + 22 files changed, 1249 insertions(+), 12 deletions(-) create mode 100644 doc/guides/nics/af_xdp.rst create mode 100644 doc/guides/nics/features/af_xdp.ini create mode 100644 drivers/net/af_xdp/Makefile create mode 100644 drivers/net/af_xdp/meson.build create mode 100644 drivers/net/af_xdp/rte_eth_af_xdp.c create mode 100644 drivers/net/af_xdp/rte_pmd_af_xdp_version.map
Signature · 1 line
2.17.1
PATCH v2 1/6 net/af_xdp: introduce AF XDP PMD driver
Add a new PMD driver for AF_XDP which is a proposed faster version of AF_PACKET interface in Linux. More info about AF_XDP, please refer to [1] [2].
This is the vanilla version PMD which just uses a raw buffer registered as the umem.
[1] https://fosdem.org/2018/schedule/event/af_xdp/
[2] https://lwn.net/Articles/745934/
--- MAINTAINERS | 6 + config/common_base | 5 + config/common_linux | 1 + doc/guides/nics/af_xdp.rst | 45 + doc/guides/nics/features/af_xdp.ini | 11 + doc/guides/nics/index.rst | 1 + doc/guides/rel_notes/release_19_05.rst | 7 + drivers/net/Makefile | 1 + drivers/net/af_xdp/Makefile | 33 + drivers/net/af_xdp/meson.build | 21 + drivers/net/af_xdp/rte_eth_af_xdp.c | 930 ++++++++++++++++++ drivers/net/af_xdp/rte_pmd_af_xdp_version.map | 3 + drivers/net/meson.build | 1 + mk/rte.app.mk | 1 + 14 files changed, 1066 insertions(+) create mode 100644 doc/guides/nics/af_xdp.rst create mode 100644 doc/guides/nics/features/af_xdp.ini create mode 100644 drivers/net/af_xdp/Makefile create mode 100644 drivers/net/af_xdp/meson.build create mode 100644 drivers/net/af_xdp/rte_eth_af_xdp.c create mode 100644 drivers/net/af_xdp/rte_pmd_af_xdp_version.map
MAINTAINERS +6 -0
@@ -468,6 +468,12 @@ M: John W. Linville <[email protected]> F: drivers/net/af_packet/ F: doc/guides/nics/features/afpacket.ini +Linux AF_XDP +M: Xiaolong Ye <[email protected]> +M: Qi Zhang <[email protected]> +F: drivers/net/af_xdp/ +F: doc/guides/nics/features/af_xdp.rst + Amazon ENA M: Marcin Wojtas <[email protected]> M: Michal Krawczyk <[email protected]>
config/common_base +5 -0
@@ -416,6 +416,11 @@ CONFIG_RTE_LIBRTE_VMXNET3_DEBUG_TX_FREE=n # CONFIG_RTE_LIBRTE_PMD_AF_PACKET=n +# +# Compile software PMD backed by AF_XDP sockets (Linux only) +# +CONFIG_RTE_LIBRTE_PMD_AF_XDP=n + # # Compile link bonding PMD library #
config/common_linux +1 -0
@@ -19,6 +19,7 @@ CONFIG_RTE_LIBRTE_VHOST_POSTCOPY=n CONFIG_RTE_LIBRTE_PMD_VHOST=y CONFIG_RTE_LIBRTE_IFC_PMD=y CONFIG_RTE_LIBRTE_PMD_AF_PACKET=y +CONFIG_RTE_LIBRTE_PMD_AF_XDP=y CONFIG_RTE_LIBRTE_PMD_SOFTNIC=y CONFIG_RTE_LIBRTE_PMD_TAP=y CONFIG_RTE_LIBRTE_AVP_PMD=y
doc/guides/nics/af_xdp.rst new file +45 -0
45 lines not on this page · raw message
doc/guides/nics/features/af_xdp.ini new file +11 -0
@@ -0,0 +1,11 @@ +; +; Supported features of the 'af_xdp' network poll mode driver. +; +; Refer to default.ini for the full list of available PMD features. +; +[Features] +Link status = Y +MTU update = Y +Promiscuous mode = Y +Stats per queue = Y +x86-64 = Y
doc/guides/nics/index.rst +1 -0
@@ -12,6 +12,7 @@ Network Interface Controller Drivers features build_and_test af_packet + af_xdp ark atlantic avp
doc/guides/rel_notes/release_19_05.rst +7 -0
@@ -65,6 +65,13 @@ New Features process. * Added support for Rx packet types list in a secondary process. +* **Added the AF_XDP PMD.** + + Added a Linux-specific PMD driver for AF_XDP, it can create the AF_XDP socket + and bind it to a specific netdev queue, it allows a DPDK application to send + and receive raw packets through the socket which would bypass the kernel + network stack to achieve high performance packet processing. + * **Updated Mellanox drivers.** New features and improvements were done in mlx4 and mlx5 PMDs:
drivers/net/Makefile +1 -0
@@ -9,6 +9,7 @@ ifeq ($(CONFIG_RTE_LIBRTE_THUNDERX_NICVF_PMD),d) endif DIRS-$(CONFIG_RTE_LIBRTE_PMD_AF_PACKET) += af_packet +DIRS-$(CONFIG_RTE_LIBRTE_PMD_AF_XDP) += af_xdp DIRS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += ark DIRS-$(CONFIG_RTE_LIBRTE_ATLANTIC_PMD) += atlantic DIRS-$(CONFIG_RTE_LIBRTE_AVP_PMD) += avp
drivers/net/af_xdp/Makefile new file +33 -0
@@ -0,0 +1,33 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Intel Corporation + +include $(RTE_SDK)/mk/rte.vars.mk + +# +# library name +# +LIB = librte_pmd_af_xdp.a + +EXPORT_MAP := rte_pmd_af_xdp_version.map + +LIBABIVER := 1 + +CFLAGS += -O3 + +# require kernel version >= v5.1-rc1 +LINUX_VERSION := $(shell uname -r) +CFLAGS += -I/lib/modules/$(LINUX_VERSION)/build/tools/include +CFLAGS += -I/lib/modules/$(LINUX_VERSION)/build/tools/lib/bpf + +CFLAGS += $(WERROR_FLAGS) +LDLIBS += -lrte_eal -lrte_mbuf -lrte_mempool -lrte_ring +LDLIBS += -lrte_ethdev -lrte_net -lrte_kvargs +LDLIBS += -lrte_bus_vdev +LDLIBS += -lbpf + +# +# all source are stored in SRCS-y +# +SRCS-$(CONFIG_RTE_LIBRTE_PMD_AF_XDP) += rte_eth_af_xdp.c + +include $(RTE_SDK)/mk/rte.lib.mk
drivers/net/af_xdp/meson.build new file +21 -0
@@ -0,0 +1,21 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Intel Corporation + +if host_machine.system() != 'linux' + build = false +endif + +bpf_dep = dependency('libbpf', required: false) +if bpf_dep.found() + build = true +else + bpf_dep = cc.find_library('libbpf', required: false) + if bpf_dep.found() and cc.has_header('xsk.h', dependencies: bpf_dep) + build = true + pkgconfig_extra_libs += '-lbpf' + else + build = false + endif +endif +sources = files('rte_eth_af_xdp.c') +ext_deps += bpf_dep
drivers/net/af_xdp/rte_eth_af_xdp.c new file +930 -0
930 lines not on this page · raw message
drivers/net/af_xdp/rte_pmd_af_xdp_version.map new file +3 -0
@@ -0,0 +1,3 @@ +DPDK_19.05 { + local: *; +};
drivers/net/meson.build +1 -0
@@ -2,6 +2,7 @@ # Copyright(c) 2017 Intel Corporation drivers = ['af_packet', + 'af_xdp', 'ark', 'atlantic', 'avp',
mk/rte.app.mk +1 -0
@@ -143,6 +143,7 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_DPAA2_MEMPOOL) += -lrte_mempool_dpaa2 endif _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AF_PACKET) += -lrte_pmd_af_packet +_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AF_XDP) += -lrte_pmd_af_xdp -lelf -lbpf _LDLIBS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += -lrte_pmd_ark _LDLIBS-$(CONFIG_RTE_LIBRTE_ATLANTIC_PMD) += -lrte_pmd_atlantic _LDLIBS-$(CONFIG_RTE_LIBRTE_AVP_PMD) += -lrte_pmd_avp
Signature · 1 line
2.17.1
Re: [PATCH v2 1/6] net/af_xdp: introduce AF XDP PMD driver
Quoted 268 lines · Xiaolong Ye
Add a new PMD driver for
AF_XDPwhich is a proposed faster version ofAF_PACKETinterface in Linux. More info aboutAF_XDP, please refer to [1] [2].This is the vanilla version PMD which just uses a raw buffer registered as the umem.
[1] https://fosdem.org/2018/schedule/event/af_xdp/
[2] https://lwn.net/Articles/745934/Signed-off-by Xiaolong Ye <[email protected]>--- MAINTAINERS | 6 + config/common_base | 5 + config/common_linux | 1 + doc/guides/nics/af_xdp.rst | 45 + doc/guides/nics/features/af_xdp.ini | 11 + doc/guides/nics/index.rst | 1 + doc/guides/rel_notes/release_19_05.rst | 7 + drivers/net/Makefile | 1 + drivers/net/af_xdp/Makefile | 33 + drivers/net/af_xdp/meson.build | 21 + drivers/net/af_xdp/rte_eth_af_xdp.c | 930 ++++++++++++++++++ drivers/net/af_xdp/rte_pmd_af_xdp_version.map | 3 + drivers/net/meson.build | 1 + mk/rte.app.mk | 1 + 14 files changed, 1066 insertions(+)create mode 100644
doc/guides/nics/af_xdp.rst
create mode 100644 doc/guides/nics/features/af_xdp.ini create mode 100644drivers/net/af_xdp/Makefile
create mode 100644drivers/net/af_xdp/meson.buildcreate mode 100644drivers/net/af_xdp/rte_eth_af_xdp.ccreate mode 100644drivers/net/af_xdp/rte_pmd_af_xdp_version.mapMAINTAINERS +6
-0@@ -468,6 +468,12 @@ M: John W. Linville <[email protected]> F: drivers/net/af_packet/ F: doc/guides/nics/features/afpacket.ini +Linux AF_XDP +M: Xiaolong Ye <[email protected]> +M: Qi Zhang <[email protected]> +F: drivers/net/af_xdp/ +F: doc/guides/nics/features/af_xdp.rst + Amazon ENA M: Marcin Wojtas <[email protected]> M: Michal Krawczyk <[email protected]>config/common_base +5
-0@@ -416,6 +416,11 @@ CONFIG_RTE_LIBRTE_VMXNET3_DEBUG_TX_FREE=n # CONFIG_RTE_LIBRTE_PMD_AF_PACKET=n +# +# Compile software PMD backed by AF_XDP sockets (Linux only) +# +CONFIG_RTE_LIBRTE_PMD_AF_XDP=n + # # Compile link bonding PMD library #config/common_linux +1
-0@@ -19,6 +19,7 @@ CONFIG_RTE_LIBRTE_VHOST_POSTCOPY=n CONFIG_RTE_LIBRTE_PMD_VHOST=y CONFIG_RTE_LIBRTE_IFC_PMD=y CONFIG_RTE_LIBRTE_PMD_AF_PACKET=y +CONFIG_RTE_LIBRTE_PMD_AF_XDP=y CONFIG_RTE_LIBRTE_PMD_SOFTNIC=y CONFIG_RTE_LIBRTE_PMD_TAP=y CONFIG_RTE_LIBRTE_AVP_PMD=ydoc/guides/nics/af_xdp.rst new file +45
-0@@ -0,0 +1,45 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2018 Intel Corporation. + +AF_XDP Poll Mode Driver +========================== + +AF_XDP is an address family that is optimized for high performance +packet processing. AF_XDP sockets enable the possibility for XDP program to +redirect packets to a memory buffer in userspace. + +For the full details behind AF_XDP socket, you can refer to +`AF_XDP documentation in the Kernel +<https://www.kernel.org/doc/Documentation/networking/af_xdp.rst>`_. + +This Linux-specific PMD driver creates the AF_XDP socket and binds it to a +specific netdev queue, it allows a DPDK application to send and receive raw +packets through the socket which would bypass the kernel network stack. +Current implementation only supports single queue, multi-queues feature will +be added later. + +Options +------- + +The following options can be provided to set up an af_xdp port in DPDK. + +* ``iface`` - name of the Kernel interface to attach to (required); +* ``queue`` - netdev queue id (optional, default 0); + +Prerequisites +------------- + +This is a Linux-specific PMD, thus the following prerequisites apply: + +* A Linux Kernel (version > 4.18) with XDP sockets configuration enabled; +* libbpf (within kernel version > 5.1) with latest af_xdp support installed +* A Kernel bound interface to attach to. + +Set up an af_xdp interface +----------------------------- + +The following example will set up an af_xdp interface in DPDK: + +.. code-block:: console + + --vdev eth_af_xdp,iface=ens786f1,queue=0doc/guides/nics/features/af_xdp.ini new file +11
-0@@ -0,0 +1,11 @@ +; +; Supported features of the 'af_xdp' network poll mode driver. +; +; Refer to default.ini for the full list of available PMD features. +; +[Features] +Link status = Y +MTU update = Y +Promiscuous mode = Y +Stats per queue = Y +x86-64 = Ydoc/guides/nics/index.rst +1
-0@@ -12,6 +12,7 @@ Network Interface Controller Drivers features build_and_test af_packet + af_xdp ark atlantic avpdoc/guides/rel_notes/release_19_05.rst +7
-0@@ -65,6 +65,13 @@ New Features process. * Added support for Rx packet types list in a secondary process. +* **Added the AF_XDP PMD.** + + Added a Linux-specific PMD driver for AF_XDP, it can create the AF_XDP socket + and bind it to a specific netdev queue, it allows a DPDK application to send + and receive raw packets through the socket which would bypass the kernel + network stack to achieve high performance packet processing. + * **Updated Mellanox drivers.** New features and improvements were done in mlx4 and mlx5 PMDs:drivers/net/Makefile +1
-0@@ -9,6 +9,7 @@ ifeq ($(CONFIG_RTE_LIBRTE_THUNDERX_NICVF_PMD),d) endif DIRS-$(CONFIG_RTE_LIBRTE_PMD_AF_PACKET) += af_packet +DIRS-$(CONFIG_RTE_LIBRTE_PMD_AF_XDP) += af_xdp DIRS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += ark DIRS-$(CONFIG_RTE_LIBRTE_ATLANTIC_PMD) += atlantic DIRS-$(CONFIG_RTE_LIBRTE_AVP_PMD) += avpdrivers/net/af_xdp/Makefile new file +33
-0@@ -0,0 +1,33 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Intel Corporation + +include $(RTE_SDK)/mk/rte.vars.mk + +# +# library name +# +LIB = librte_pmd_af_xdp.a + +EXPORT_MAP := rte_pmd_af_xdp_version.map + +LIBABIVER := 1 + +CFLAGS += -O3 + +# require kernel version >= v5.1-rc1 +LINUX_VERSION := $(shell uname -r) +CFLAGS += -I/lib/modules/$(LINUX_VERSION)/build/tools/include +CFLAGS += -I/lib/modules/$(LINUX_VERSION)/build/tools/lib/bpf + +CFLAGS += $(WERROR_FLAGS) +LDLIBS += -lrte_eal -lrte_mbuf -lrte_mempool -lrte_ring +LDLIBS += -lrte_ethdev -lrte_net -lrte_kvargs +LDLIBS += -lrte_bus_vdev +LDLIBS += -lbpf + +# +# all source are stored in SRCS-y +# +SRCS-$(CONFIG_RTE_LIBRTE_PMD_AF_XDP) += rte_eth_af_xdp.c + +include $(RTE_SDK)/mk/rte.lib.mkdrivers/net/af_xdp/meson.build new file +21
-0@@ -0,0 +1,21 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Intel Corporation + +if host_machine.system() != 'linux' + build = false +endif + +bpf_dep = dependency('libbpf', required: false) +if bpf_dep.found() + build = true +else + bpf_dep = cc.find_library('libbpf', required: false) + if bpf_dep.found() and cc.has_header('xsk.h', dependencies: bpf_dep) + build = true + pkgconfig_extra_libs += '-lbpf' + else + build = false + endif +endif +sources = files('rte_eth_af_xdp.c') +ext_deps += bpf_depdrivers/net/af_xdp/rte_eth_af_xdp.c new file +6
-0@@ -0,0 +1,930 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2019 Intel Corporation. +*/ + +#include <rte_mbuf.h> +#include <rte_ethdev_driver.h>
drivers/net/af_xdp/rte_eth_af_xdp.c line 16
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+#include <rte_ethdev_vdev.h> +#include <rte_malloc.h> +#include <rte_kvargs.h> +#include <rte_bus_vdev.h> +#include <rte_string_fns.h> + +#include <linux/if_ether.h> +#include <linux/if_xdp.h> +#include <linux/if_link.h> +#include <asm/barrier.h>
Is this include used?
Quoted 93 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +93
-0+#include <arpa/inet.h> +#include <net/if.h> +#include <sys/types.h> +#include <sys/socket.h> +#include <sys/ioctl.h> +#include <sys/mman.h> +#include <unistd.h> +#include <poll.h> +#include <bpf/bpf.h> +#include <xsk.h> + +#define RTE_LOGTYPE_AF_XDP RTE_LOGTYPE_USER1 +#ifndef SOL_XDP +#define SOL_XDP 283 +#endif + +#ifndef AF_XDP +#define AF_XDP 44 +#endif + +#ifndef PF_XDP +#define PF_XDP AF_XDP +#endif + +#define ETH_AF_XDP_IFACE_ARG "iface" +#define ETH_AF_XDP_QUEUE_IDX_ARG "queue" + +#define ETH_AF_XDP_FRAME_SIZE XSK_UMEM__DEFAULT_FRAME_SIZE +#define ETH_AF_XDP_NUM_BUFFERS 4096 +#define ETH_AF_XDP_DATA_HEADROOM 0 +#define ETH_AF_XDP_DFLT_NUM_DESCS XSK_RING_CONS__DEFAULT_NUM_DESCS +#define ETH_AF_XDP_DFLT_QUEUE_IDX 0 + +#define ETH_AF_XDP_RX_BATCH_SIZE 32 +#define ETH_AF_XDP_TX_BATCH_SIZE 32 + +#define ETH_AF_XDP_MAX_QUEUE_PAIRS 16 + +struct xsk_umem_info { +struct xsk_ring_prod fq; +struct xsk_ring_cons cq; +struct xsk_umem *umem; +struct rte_ring *buf_ring; +void *buffer; +}; + +struct pkt_rx_queue { +struct xsk_ring_cons rx; +struct xsk_umem_info *umem; +struct xsk_socket *xsk; +struct rte_mempool *mb_pool; + +uint64_t rx_pkts; +uint64_t rx_bytes; +uint64_t rx_dropped; + +struct pkt_tx_queue *pair; +uint16_t queue_idx; +}; + +struct pkt_tx_queue { +struct xsk_ring_prod tx; + +uint64_t tx_pkts; +uint64_t err_pkts; +uint64_t tx_bytes; + +struct pkt_rx_queue *pair; +uint16_t queue_idx; +}; + +struct pmd_internals { +int if_index; +char if_name[IFNAMSIZ ]; +uint16_t queue_idx; +struct ether_addr eth_addr; +struct xsk_umem_info *umem; +struct rte_mempool *mb_pool_share; + +struct pkt_rx_queue rx_queues[ETH_AF_XDP_MAX_QUEUE_PAIRS ]; +struct pkt_tx_queue tx_queues[ETH_AF_XDP_MAX_QUEUE_PAIRS ]; +}; + +static const char *const valid_arguments[] = { +ETH_AF_XDP_IFACE_ARG , +ETH_AF_XDP_QUEUE_IDX_ARG , +NULL +}; + +static struct rte_eth_link pmd_link = { + .link_speed =ETH_SPEED_NUM_10G , + .link_duplex =ETH_LINK_FULL_DUPLEX , + .link_status =ETH_LINK_DOWN ,
drivers/net/af_xdp/rte_eth_af_xdp.c line 119
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+ .link_autoneg = ETH_LINK_AUTONEG +}; + +static inline int +reserve_fill_queue (struct xsk_umem_info *umem,int reserve_size) +{ +struct xsk_ring_prod *fq = &umem->fq; +uint32_t idx; +void *addr =NULL ; +int i, ret =0 ;
No need to initialize 'ret'. Is there a point to set 'addr'?
drivers/net/af_xdp/rte_eth_af_xdp.c line 129
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+ + ret =xsk_ring_prod__reserve (fq, reserve_size, &idx); +if (!ret) { +RTE_LOG (ERR ,AF_XDP ,"Failed to reserve enough fq descs.\n" ); +return ret; + } + +for (i =0 ; i < reserve_size; i++) { +rte_ring_dequeue (umem->buf_ring, &addr); + *xsk_ring_prod__fill_addr (fq, idx++) = (uint64_t )addr;
Consider introducing a tmp variable to make this more readable.
Quoted 16 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +16
-0+ } + +xsk_ring_prod__submit (fq, reserve_size); + +return 0 ; +} + +static uint16_t +eth_af_xdp_rx (void *queue,struct rte_mbuf **bufs,uint16_t nb_pkts) +{ +struct pkt_rx_queue *rxq = queue; +struct xsk_ring_cons *rx = &rxq->rx; +struct xsk_umem_info *umem = rxq->umem; +struct xsk_ring_prod *fq = &umem->fq; +uint32_t idx_rx; +uint32_t free_thresh = fq->size >>1 ;
drivers/net/af_xdp/rte_eth_af_xdp.c
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+struct rte_mbuf *mbuf; +unsigned long dropped =0 ; +unsigned long rx_bytes =0 ; +uint16_t count =0 ; +int rcvd, i; + + nb_pkts = nb_pkts <ETH_AF_XDP_RX_BATCH_SIZE ? + nb_pkts :ETH_AF_XDP_RX_BATCH_SIZE ; + + rcvd = xsk_ring_cons__peek(rx, nb_pkts, &idx_rx); > + if (!rcvd)
Since peek returns the number of entries, not a boolean, do:
rcvd == 0
drivers/net/af_xdp/rte_eth_af_xdp.c line 164
Quoted 8 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +8
-0+return 0 ; + +if (xsk_prod_nb_free (fq, free_thresh) >= free_thresh) + (void )reserve_fill_queue (umem,ETH_AF_XDP_RX_BATCH_SIZE ); + +for (i =0 ; i < rcvd; i++) { +uint64_t addr =xsk_ring_cons__rx_desc (rx, idx_rx)->addr; +uint32_t len =xsk_ring_cons__rx_desc (rx, idx_rx++)->len;
Use a tmp variable, instead of two calls.
drivers/net/af_xdp/rte_eth_af_xdp.c line 166
Quoted 2 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +2
-0+char *pkt =xsk_umem__get_data (rxq->umem->buffer, addr); +
Don't mix declaration and code. Why is this a char pointer? As opppose to void.
drivers/net/af_xdp/rte_eth_af_xdp.c line 168
Quoted 2 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +2
-0+ mbuf =rte_pktmbuf_alloc (rxq->mb_pool); +if (mbuf) {
1.8.1
drivers/net/af_xdp/rte_eth_af_xdp.c line 169
Quoted 1 line
drivers/net/af_xdp/rte_eth_af_xdp.c new file +1
-0+ memcpy(rte_pktmbuf_mtod(mbuf, void*), pkt, len);
rte_memcpy()
drivers/net/af_xdp/rte_eth_af_xdp.c line 171
Quoted 2 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +2
-0+rte_pktmbuf_pkt_len (mbuf) = +rte_pktmbuf_data_len (mbuf) = len;
Consider splitting this into two statements.
Quoted 16 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +16
-0+ rx_bytes += len; + bufs[count++] = mbuf; + }else { + dropped++; + } +rte_ring_enqueue (umem->buf_ring, (void *)addr); + } + +xsk_ring_cons__release (rx, rcvd); + +/* statistics */ + rxq->rx_pkts += (rcvd - dropped); + rxq->rx_bytes += rx_bytes; + rxq->rx_dropped += dropped; + +return count;
drivers/net/af_xdp/rte_eth_af_xdp.c line 197
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+} + +static void pull_umem_cq (struct xsk_umem_info *umem,int size) +{ +struct xsk_ring_cons *cq = &umem->cq; +int i, n; +uint32_t idx_cq; +uint64_t addr; + + n =xsk_ring_cons__peek (cq, size, &idx_cq);
Use size_t for n.
drivers/net/af_xdp/rte_eth_af_xdp.c line 199
Quoted 2 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +2
-0+if (n >0 ) { +for (i =0 ; i < n; i++) {
Consider declaring 'addr' in this scope.
Quoted 4 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +4
-0+ addr = *xsk_ring_cons__comp_addr (cq, + idx_cq++); > + rte_ring_enqueue(umem->buf_ring, (void *)addr); + } +
drivers/net/af_xdp/rte_eth_af_xdp.c line 214
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+xsk_ring_cons__release (cq, n); + } +} + +static void kick_tx (struct pkt_tx_queue *txq) +{ +struct xsk_umem_info *umem = txq->pair->umem; +int ret; + +while (1 ) {
for (;;)
drivers/net/af_xdp/rte_eth_af_xdp.c line 219
Quoted 5 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +5
-0+ ret =sendto (xsk_socket__fd (txq->pair->xsk),NULL ,0 , +MSG_DONTWAIT ,NULL ,0 ); + +/* everything is ok */ +if (ret >=0 )
Use likely()?
Quoted 16 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +16
-0+break ; + +/* some thing unexpected */ +if (errno !=EBUSY && errno !=EAGAIN ) +break ; + +/* pull from complete qeueu to leave more space */ +if (errno ==EAGAIN ) +pull_umem_cq (umem,ETH_AF_XDP_TX_BATCH_SIZE ); + } +pull_umem_cq (umem,ETH_AF_XDP_TX_BATCH_SIZE ); +} + +static uint16_t +eth_af_xdp_tx (void *queue,struct rte_mbuf **bufs,uint16_t nb_pkts) +{
drivers/net/af_xdp/rte_eth_af_xdp.c line 245
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+struct pkt_tx_queue *txq = queue; +struct xsk_umem_info *umem = txq->pair->umem; +struct rte_mbuf *mbuf; +void *addrs[ETH_AF_XDP_TX_BATCH_SIZE ]; +unsigned long tx_bytes =0 ; +int i, valid =0 ; +uint32_t idx_tx; + + nb_pkts = nb_pkts <ETH_AF_XDP_TX_BATCH_SIZE ? + nb_pkts :ETH_AF_XDP_TX_BATCH_SIZE ;
Use RTE_MIN().
drivers/net/af_xdp/rte_eth_af_xdp.c line 251
Quoted 6 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +6
-0+ +pull_umem_cq (umem, nb_pkts); + + nb_pkts =rte_ring_dequeue_bulk (umem->buf_ring, addrs, + nb_pkts,NULL ); +if (!nb_pkts)
nb_pkts == 0
drivers/net/af_xdp/rte_eth_af_xdp.c line 261
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+return 0 ; + +if (xsk_ring_prod__reserve (&txq->tx, nb_pkts, &idx_tx) != nb_pkts) { +kick_tx (txq); +return 0 ; + } + +for (i =0 ; i < nb_pkts; i++) { +struct xdp_desc *desc; +char *pkt;
Use void pointer?
drivers/net/af_xdp/rte_eth_af_xdp.c line 263
Quoted 2 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +2
-0+unsigned int buf_len =ETH_AF_XDP_FRAME_SIZE + -ETH_AF_XDP_DATA_HEADROOM ;
Use uint32_t, as you seem to do elsewhere.
drivers/net/af_xdp/rte_eth_af_xdp.c line 272
Quoted 9 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +9
-0+ desc =xsk_ring_prod__tx_desc (&txq->tx, idx_tx + i); + mbuf = bufs[i]; +if (mbuf->pkt_len <= buf_len) { + desc->addr = (uint64_t )addrs[valid]; + desc->len = mbuf->pkt_len; + pkt =xsk_umem__get_data (umem->buffer, + desc->addr); + memcpy(pkt, rte_pktmbuf_mtod(mbuf, void *), + desc->len);
rte_memcpy()
Quoted 178 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +178
-0+ valid++; + tx_bytes += mbuf->pkt_len; + } +rte_pktmbuf_free (mbuf); + } + +xsk_ring_prod__submit (&txq->tx, nb_pkts); + +kick_tx (txq); + +if (valid < nb_pkts) +rte_ring_enqueue_bulk (umem->buf_ring, &addrs[valid], + nb_pkts - valid,NULL ); + + txq->err_pkts += nb_pkts - valid; + txq->tx_pkts += valid; + txq->tx_bytes += tx_bytes; + +return nb_pkts; +} + +static int +eth_dev_start (struct rte_eth_dev *dev) +{ + dev->data->dev_link.link_status =ETH_LINK_UP ; + +return 0 ; +} + +/* This function gets called when the current port gets stopped. */ +static void +eth_dev_stop (struct rte_eth_dev *dev) +{ + dev->data->dev_link.link_status =ETH_LINK_DOWN ; +} + +static int +eth_dev_configure(struct rte_eth_dev *dev __rte_unused) +{ +/* rx/tx must be paired */ +if (dev->data->nb_rx_queues != dev->data->nb_tx_queues) +return -EINVAL ; + +return 0 ; +} + +static void +eth_dev_info (struct rte_eth_dev *dev,struct rte_eth_dev_info *dev_info) +{ +struct pmd_internals *internals = dev->data->dev_private; + + dev_info->if_index = internals->if_index; + dev_info->max_mac_addrs =1 ; + dev_info->max_rx_pktlen = (uint32_t )ETH_FRAME_LEN ; + dev_info->max_rx_queues =1 ; + dev_info->max_tx_queues =1 ; + dev_info->min_rx_bufsize =0 ; + + dev_info->default_rxportconf.nb_queues =1 ; + dev_info->default_txportconf.nb_queues =1 ; + dev_info->default_rxportconf.ring_size =ETH_AF_XDP_DFLT_NUM_DESCS ; + dev_info->default_txportconf.ring_size =ETH_AF_XDP_DFLT_NUM_DESCS ; +} + +static int +eth_stats_get (struct rte_eth_dev *dev,struct rte_eth_stats *stats) +{ +struct pmd_internals *internals = dev->data->dev_private; +struct xdp_statistics xdp_stats; +struct pkt_rx_queue *rxq; +socklen_t optlen; +int i; + + optlen =sizeof (struct xdp_statistics ); +for (i =0 ; i < dev->data->nb_rx_queues; i++) { + rxq = &internals->rx_queues[i]; + stats->q_ipackets[i] = internals->rx_queues[i].rx_pkts; + stats->q_ibytes[i] = internals->rx_queues[i].rx_bytes; + + stats->q_opackets[i] = internals->tx_queues[i].tx_pkts; + stats->q_obytes[i] = internals->tx_queues[i].tx_bytes; + + stats->ipackets += stats->q_ipackets[i]; + stats->ibytes += stats->q_ibytes[i]; + stats->imissed += internals->rx_queues[i].rx_dropped; +getsockopt (xsk_socket__fd (rxq->xsk),SOL_XDP ,XDP_STATISTICS , + &xdp_stats, &optlen); + stats->imissed += xdp_stats.rx_dropped; + + stats->opackets += stats->q_opackets[i]; + stats->oerrors += stats->q_errors[i]; + stats->oerrors += internals->tx_queues[i].err_pkts; + stats->obytes += stats->q_obytes[i]; + } + +return 0 ; +} + +static void +eth_stats_reset (struct rte_eth_dev *dev) +{ +struct pmd_internals *internals = dev->data->dev_private; +int i; + +for (i =0 ; i <ETH_AF_XDP_MAX_QUEUE_PAIRS ; i++) { + internals->rx_queues[i].rx_pkts =0 ; + internals->rx_queues[i].rx_bytes =0 ; + internals->rx_queues[i].rx_dropped =0 ; + + internals->tx_queues[i].tx_pkts =0 ; + internals->tx_queues[i].err_pkts =0 ; + internals->tx_queues[i].tx_bytes =0 ; + } +} + +static void remove_xdp_program (struct pmd_internals *internals) +{ +uint32_t curr_prog_id =0 ; + +if (bpf_get_link_xdp_id (internals->if_index, &curr_prog_id, +XDP_FLAGS_UPDATE_IF_NOEXIST )) { +RTE_LOG (ERR ,AF_XDP ,"bpf_get_link_xdp_id failed\n" ); +return ; + } +bpf_set_link_xdp_fd (internals->if_index,-1 , +XDP_FLAGS_UPDATE_IF_NOEXIST ); +} + +static void +eth_dev_close (struct rte_eth_dev *dev) +{ +struct pmd_internals *internals = dev->data->dev_private; +struct pkt_rx_queue *rxq; +int i; + +RTE_LOG (INFO ,AF_XDP ,"Closing AF_XDP ethdev on numa socket %u\n" , +rte_socket_id ()); + +for (i =0 ; i <ETH_AF_XDP_MAX_QUEUE_PAIRS ; i++) { + rxq = &internals->rx_queues[i]; +if (!rxq->umem) +break ; +xsk_socket__delete (rxq->xsk); + } + + (void )xsk_umem__delete (internals->umem->umem); +remove_xdp_program (internals); +} + +static void +eth_queue_release(void *q __rte_unused) +{ +} + +static int +eth_link_update(struct rte_eth_dev *dev __rte_unused, + int wait_to_complete __rte_unused) +{ +return 0 ; +} + +static void xdp_umem_destroy (struct xsk_umem_info *umem) +{ +free (umem->buffer); + umem->buffer =NULL ; + +rte_ring_free (umem->buf_ring); + umem->buf_ring =NULL ; + +free (umem); + umem =NULL ; +} + +static struct xsk_umem_info *xdp_umem_configure (void ) +{ +struct xsk_umem_info *umem; +struct xsk_umem_config usr_config = { + .fill_size =ETH_AF_XDP_DFLT_NUM_DESCS ,
drivers/net/af_xdp/rte_eth_af_xdp.c line 460
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+ .comp_size = ETH_AF_XDP_DFLT_NUM_DESCS, + .frame_size = ETH_AF_XDP_FRAME_SIZE, + .frame_headroom = ETH_AF_XDP_DATA_HEADROOM }; +void *bufs =NULL ; +char ring_name[0x100 ]; +int ret; +uint64_t i; + + umem =calloc (1 ,sizeof (*umem)); +if (!umem) {
1.8.1
drivers/net/af_xdp/rte_eth_af_xdp.c line 465
Quoted 5 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +5
-0+RTE_LOG (ERR ,AF_XDP ,"Failed to allocate umem info" ); +return NULL ; + } + +snprintf (ring_name,0x100 ,"af_xdp_ring" );
Again the magical 0x100.
drivers/net/af_xdp/rte_eth_af_xdp.c line 470
Quoted 5 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +5
-0+ umem->buf_ring =rte_ring_create (ring_name, +ETH_AF_XDP_NUM_BUFFERS , +SOCKET_ID_ANY , +0x0 ); +if (!umem->buf_ring) {
1.8.1
Quoted 34 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +34
-0+RTE_LOG (ERR ,AF_XDP , +"Failed to create rte_ring\n" ); + goto err; + } + +for (i =0 ; i <ETH_AF_XDP_NUM_BUFFERS ; i++) +rte_ring_enqueue (umem->buf_ring, + (void *)(i *ETH_AF_XDP_FRAME_SIZE + +ETH_AF_XDP_DATA_HEADROOM )); + +if (posix_memalign (&bufs,getpagesize (), +ETH_AF_XDP_NUM_BUFFERS *ETH_AF_XDP_FRAME_SIZE )) { +RTE_LOG (ERR ,AF_XDP ,"Failed to allocate memory pool.\n" ); + goto err; + } + ret =xsk_umem__create (&umem->umem, bufs, +ETH_AF_XDP_NUM_BUFFERS *ETH_AF_XDP_FRAME_SIZE , + &umem->fq, &umem->cq, + &usr_config); + +if (ret) { +RTE_LOG (ERR ,AF_XDP ,"Failed to create umem" ); + goto err; + } + umem->buffer = bufs; + +return umem; + +err: +xdp_umem_destroy (umem); +return NULL ; +} + +static int
drivers/net/af_xdp/rte_eth_af_xdp.c line 514
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+xsk_configure(struct pmd_internals *internals, struct pkt_rx_queue *rxq, + int ring_size) +{ + struct xsk_socket_config cfg; +struct pkt_tx_queue *txq = rxq->pair; +int ret =0 ; +int reserve_size; + + rxq->umem =xdp_umem_configure (); +if (!rxq->umem) {
1.8.1
Quoted 32 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +32
-0+ ret = -ENOMEM ; + goto err; + } + + cfg.rx_size = ring_size; + cfg.tx_size = ring_size; + cfg.libbpf_flags =0 ; + cfg.xdp_flags =XDP_FLAGS_UPDATE_IF_NOEXIST ; + cfg.bind_flags =0 ; + ret =xsk_socket__create (&rxq->xsk, internals->if_name, + internals->queue_idx, rxq->umem->umem, &rxq->rx, + &txq->tx, &cfg); +if (ret) { +RTE_LOG (ERR ,AF_XDP ,"Failed to create xsk socket.\n" ); + goto err; + } + + reserve_size =ETH_AF_XDP_DFLT_NUM_DESCS /2 ; + ret =reserve_fill_queue (rxq->umem, reserve_size); +if (ret) { +RTE_LOG (ERR ,AF_XDP ,"Failed to reserve fill queue.\n" ); + goto err; + } + +return 0 ; + +err: +xdp_umem_destroy (rxq->umem); + +return ret; +} +
drivers/net/af_xdp/rte_eth_af_xdp.c line 556
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+static void +queue_reset (struct pmd_internals *internals,uint16_t queue_idx) +{ +struct pkt_rx_queue *rxq = &internals->rx_queues[queue_idx]; +struct pkt_tx_queue *txq = rxq->pair; +int xsk_fd =xsk_socket__fd (rxq->xsk); + +if (xsk_fd) { +close (xsk_fd); +if (internals->umem) {
1.8.1
Quoted 12 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +12
-0+xdp_umem_destroy (internals->umem); + internals->umem =NULL ; + } + } +memset (rxq,0 ,sizeof (*rxq)); +memset (txq,0 ,sizeof (*txq)); + rxq->pair = txq; + txq->pair = rxq; + rxq->queue_idx = queue_idx; + txq->queue_idx = queue_idx; +} +
drivers/net/af_xdp/rte_eth_af_xdp.c line 578
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+static int +eth_rx_queue_setup (struct rte_eth_dev *dev, +uint16_t rx_queue_id, +uint16_t nb_rx_desc, + unsigned int socket_id __rte_unused, + const struct rte_eth_rxconf *rx_conf __rte_unused, +struct rte_mempool *mb_pool) +{ +struct pmd_internals *internals = dev->data->dev_private; +unsigned int buf_size, data_size;
uint32_t
drivers/net/af_xdp/rte_eth_af_xdp.c line 580
Quoted 2 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +2
-0+struct pkt_rx_queue *rxq; +int ret =0 ;
No need to set 'ret'. Alternatively, restructure so you always return 'ret'.
Quoted 127 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +127
-0+ + rxq = &internals->rx_queues[rx_queue_id]; +queue_reset (internals, rx_queue_id); + +/* Now get the space available for data in the mbuf */ + buf_size =rte_pktmbuf_data_room_size (mb_pool) - +RTE_PKTMBUF_HEADROOM ; + data_size =ETH_AF_XDP_FRAME_SIZE -ETH_AF_XDP_DATA_HEADROOM ; + +if (data_size > buf_size) { +RTE_LOG (ERR ,AF_XDP , +"%s: %d bytes will not fit in mbuf (%d bytes)\n" , + dev->device->name, data_size, buf_size); + ret = -ENOMEM ; + goto err; + } + + rxq->mb_pool = mb_pool; + +if (xsk_configure (internals, rxq, nb_rx_desc)) { +RTE_LOG (ERR ,AF_XDP , +"Failed to configure xdp socket\n" ); + ret = -EINVAL ; + goto err; + } + + internals->umem = rxq->umem; + + dev->data->rx_queues[rx_queue_id] = rxq; +return 0 ; + +err: +queue_reset (internals, rx_queue_id); +return ret; +} + +static int +eth_tx_queue_setup (struct rte_eth_dev *dev, +uint16_t tx_queue_id, + uint16_t nb_tx_desc __rte_unused, + unsigned int socket_id __rte_unused, + const struct rte_eth_txconf *tx_conf __rte_unused) +{ +struct pmd_internals *internals = dev->data->dev_private; +struct pkt_tx_queue *txq; + + txq = &internals->tx_queues[tx_queue_id]; + + dev->data->tx_queues[tx_queue_id] = txq; +return 0 ; +} + +static int +eth_dev_mtu_set (struct rte_eth_dev *dev,uint16_t mtu) +{ +struct pmd_internals *internals = dev->data->dev_private; +struct ifreq ifr = { .ifr_mtu = mtu }; +int ret; +int s; + + s =socket (PF_INET ,SOCK_DGRAM ,0 ); +if (s <0 ) +return -EINVAL ; + +strlcpy (ifr.ifr_name, internals->if_name,IFNAMSIZ ); + ret =ioctl (s,SIOCSIFMTU , &ifr); +close (s); + +if (ret <0 ) +return -EINVAL ; + +return 0 ; +} + +static void +eth_dev_change_flags (char *if_name,uint32_t flags,uint32_t mask) +{ +struct ifreq ifr; +int s; + + s =socket (PF_INET ,SOCK_DGRAM ,0 ); +if (s <0 ) +return ; + +strlcpy (ifr.ifr_name, if_name,IFNAMSIZ ); +if (ioctl (s,SIOCGIFFLAGS , &ifr) <0 ) + goto out; + ifr.ifr_flags &= mask; + ifr.ifr_flags |= flags; +if (ioctl (s,SIOCSIFFLAGS , &ifr) <0 ) + goto out; +out: +close (s); +} + +static void +eth_dev_promiscuous_enable (struct rte_eth_dev *dev) +{ +struct pmd_internals *internals = dev->data->dev_private; + +eth_dev_change_flags (internals->if_name,IFF_PROMISC , ~0 ); +} + +static void +eth_dev_promiscuous_disable (struct rte_eth_dev *dev) +{ +struct pmd_internals *internals = dev->data->dev_private; + +eth_dev_change_flags (internals->if_name,0 , ~IFF_PROMISC ); +} + +static const struct eth_dev_ops ops = { + .dev_start = eth_dev_start, + .dev_stop = eth_dev_stop, + .dev_close = eth_dev_close, + .dev_configure = eth_dev_configure, + .dev_infos_get = eth_dev_info, + .mtu_set = eth_dev_mtu_set, + .promiscuous_enable = eth_dev_promiscuous_enable, + .promiscuous_disable = eth_dev_promiscuous_disable, + .rx_queue_setup = eth_rx_queue_setup, + .tx_queue_setup = eth_tx_queue_setup, + .rx_queue_release = eth_queue_release, + .tx_queue_release = eth_queue_release, + .link_update = eth_link_update, + .stats_get = eth_stats_get, + .stats_reset = eth_stats_reset,
drivers/net/af_xdp/rte_eth_af_xdp.c line 717
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+}; + +/** parse integer from integer argument */ +static int +parse_integer_arg(const char *key __rte_unused, +const char *value,void *extra_args) +{ +int *i = (int *)extra_args; + + *i =atoi (value);
Use strtol().
Quoted 6 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +6
-0+if (*i <0 ) { +RTE_LOG (ERR ,AF_XDP ,"Argument has to be positive.\n" ); +return -EINVAL ; + } + +return 0 ;
drivers/net/af_xdp/rte_eth_af_xdp.c line 733
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+} + +/** parse name argument */ +static int +parse_name_arg(const char *key __rte_unused, +const char *value,void *extra_args) +{ +char *name = extra_args; + +if (strlen (value) >IFNAMSIZ ) {
The buffer is IFNAMSIZ bytes (which it should be), so it can't hold a string with strlen() == IFNAMSIZ.
Quoted 6 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +6
-0+RTE_LOG (ERR ,AF_XDP ,"Invalid name %s, should be less than " +"%u bytes.\n" , value,IFNAMSIZ ); +return -EINVAL ; + } + +strlcpy (name, value,IFNAMSIZ );
drivers/net/af_xdp/rte_eth_af_xdp.c line 749
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+ +return 0 ; +} + +static int +parse_parameters (struct rte_kvargs *kvlist, +char *if_name, +int *queue_idx) +{ +int ret =0 ;
Should not be initialized.
drivers/net/af_xdp/rte_eth_af_xdp.c line 759
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+ + ret =rte_kvargs_process (kvlist,ETH_AF_XDP_IFACE_ARG , + &parse_name_arg, if_name); +if (ret <0 ) + goto free_kvlist; + + ret =rte_kvargs_process (kvlist,ETH_AF_XDP_QUEUE_IDX_ARG , + &parse_integer_arg, queue_idx); +if (ret <0 ) + goto free_kvlist;
I fail to see the point of this goto, but maybe there's more code to follow in future patches.
Quoted 39 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +39
-0+ +free_kvlist: +rte_kvargs_free (kvlist); +return ret; +} + +static int +get_iface_info (const char *if_name, +struct ether_addr *eth_addr, +int *if_index) +{ +struct ifreq ifr; +int sock =socket (AF_INET ,SOCK_DGRAM ,IPPROTO_IP ); + +if (sock <0 ) +return -1 ; + +strlcpy (ifr.ifr_name, if_name,IFNAMSIZ ); +if (ioctl (sock,SIOCGIFINDEX , &ifr)) + goto error; + +if (ioctl (sock,SIOCGIFHWADDR , &ifr)) + goto error; + +rte_memcpy (eth_addr, ifr.ifr_hwaddr.sa_data,ETHER_ADDR_LEN ); + +close (sock); + *if_index =if_nametoindex (if_name); +return 0 ; + +error: +close (sock); +return -1 ; +} + +static int +init_internals(struct rte_vdev_device *dev, + const char *if_name, + int queue_idx,
drivers/net/af_xdp/rte_eth_af_xdp.c line 808
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+ struct rte_eth_dev **eth_dev) +{ +const char *name =rte_vdev_device_name (dev); +const unsigned int numa_node = dev->device.numa_node; +struct pmd_internals *internals =NULL ; +int ret; +int i; + + internals =rte_zmalloc_socket (name,sizeof (*internals),0 , numa_node); +if (!internals)
1.8.1
Quoted 7 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +7
-0+return -ENOMEM ; + + internals->queue_idx = queue_idx; +strlcpy (internals->if_name, if_name,IFNAMSIZ ); + +for (i =0 ; i <ETH_AF_XDP_MAX_QUEUE_PAIRS ; i++) { + internals->tx_queues[i].pair = &internals->rx_queues[i];
drivers/net/af_xdp/rte_eth_af_xdp.c line 825
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+ internals->rx_queues[i].pair = &internals->tx_queues[i]; + } + + ret =get_iface_info (if_name, &internals->eth_addr, + &internals->if_index); +if (ret) + goto err; + + *eth_dev =rte_eth_vdev_allocate (dev,0 ); +if (!*eth_dev)
1.8.1
Quoted 139 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +105
-0+ goto err; + + (*eth_dev)->data->dev_private = internals; + (*eth_dev)->data->dev_link = pmd_link; + (*eth_dev)->data->mac_addrs = &internals->eth_addr; + (*eth_dev)->dev_ops = &ops; + (*eth_dev)->rx_pkt_burst = eth_af_xdp_rx; + (*eth_dev)->tx_pkt_burst = eth_af_xdp_tx; + +return 0 ; + +err: +rte_free (internals); +return -1 ; +} + +static int +rte_pmd_af_xdp_probe (struct rte_vdev_device *dev) +{ +struct rte_kvargs *kvlist; +char if_name[IFNAMSIZ ]; +int xsk_queue_idx =ETH_AF_XDP_DFLT_QUEUE_IDX ; +struct rte_eth_dev *eth_dev =NULL ; +const char *name; +int ret; + +RTE_LOG (INFO ,AF_XDP ,"Initializing pmd_af_xdp for %s\n" , +rte_vdev_device_name (dev)); + + name =rte_vdev_device_name (dev); +if (rte_eal_process_type () ==RTE_PROC_SECONDARY && +strlen (rte_vdev_device_args (dev)) ==0 ) { + eth_dev =rte_eth_dev_attach_secondary (name); +if (!eth_dev) { +RTE_LOG (ERR ,AF_XDP ,"Failed to probe %s\n" , name); +return -EINVAL ; + } + eth_dev->dev_ops = &ops; +rte_eth_dev_probing_finish (eth_dev); +return 0 ; + } + + kvlist =rte_kvargs_parse (rte_vdev_device_args (dev), valid_arguments); +if (!kvlist) { +RTE_LOG (ERR ,AF_XDP ,"Invalid kvargs key\n" ); +return -EINVAL ; + } + +if (dev->device.numa_node ==SOCKET_ID_ANY ) + dev->device.numa_node =rte_socket_id (); + +if (parse_parameters (kvlist, if_name, &xsk_queue_idx) <0 ) { +RTE_LOG (ERR ,AF_XDP ,"Invalid kvargs value\n" ); +return -EINVAL ; + } + + ret =init_internals (dev, if_name, xsk_queue_idx, ð_dev); +if (ret) { +RTE_LOG (ERR ,AF_XDP ,"Failed to init internals\n" ); +return ret; + } + +rte_eth_dev_probing_finish (eth_dev); + +return 0 ; +} + +static int +rte_pmd_af_xdp_remove (struct rte_vdev_device *dev) +{ +struct rte_eth_dev *eth_dev =NULL ; +struct pmd_internals *internals; + +RTE_LOG (INFO ,AF_XDP ,"Removing AF_XDP ethdev on numa socket %u\n" , +rte_socket_id ()); + +if (!dev) +return -1 ; + +/* find the ethdev entry */ + eth_dev =rte_eth_dev_allocated (rte_vdev_device_name (dev)); +if (!eth_dev) +return -1 ; + + internals = eth_dev->data->dev_private; + +rte_ring_free (internals->umem->buf_ring); +rte_free (internals->umem->buffer); +rte_free (internals->umem); + +rte_eth_dev_release_port (eth_dev); + + +return 0 ; +} + +static struct rte_vdev_driver pmd_af_xdp_drv = { + .probe = rte_pmd_af_xdp_probe, + .remove = rte_pmd_af_xdp_remove, +}; + +RTE_PMD_REGISTER_VDEV (eth_af_xdp, pmd_af_xdp_drv); +RTE_PMD_REGISTER_PARAM_STRING (eth_af_xdp, +"iface=<string> " +"queue=<int> " );drivers/net/af_xdp/rte_pmd_af_xdp_version.map new file +3
-0@@ -0,0 +1,3 @@ +DPDK_19.05 { + local: *; +};drivers/net/meson.build +1
-0@@ -2,6 +2,7 @@ # Copyright(c) 2017 Intel Corporation drivers = ['af_packet', + 'af_xdp', 'ark', 'atlantic', 'avp',mk/rte.app.mk +1
-0@@ -143,6 +143,7 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_DPAA2_MEMPOOL) += -lrte_mempool_dpaa2 endif _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AF_PACKET) += -lrte_pmd_af_packet +_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AF_XDP) += -lrte_pmd_af_xdp -lelf -lbpf _LDLIBS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += -lrte_pmd_ark _LDLIBS-$(CONFIG_RTE_LIBRTE_ATLANTIC_PMD) += -lrte_pmd_atlantic _LDLIBS-$(CONFIG_RTE_LIBRTE_AVP_PMD) += -lrte_pmd_avp
Re: [PATCH v2 1/6] net/af_xdp: introduce AF XDP PMD driver
Hi, Mattias
Thanks for your comments.
Quoted 268 lines · Mattias Rönnblom
Quoted 267 lines · Xiaolong Ye
Add a new PMD driver for
AF_XDPwhich is a proposed faster version ofAF_PACKETinterface in Linux. More info aboutAF_XDP, please refer to [1] [2].This is the vanilla version PMD which just uses a raw buffer registered as the umem.
[1] https://fosdem.org/2018/schedule/event/af_xdp/
[2] https://lwn.net/Articles/745934/Signed-off-by Xiaolong Ye <[email protected]>--- MAINTAINERS | 6 + config/common_base | 5 + config/common_linux | 1 + doc/guides/nics/af_xdp.rst | 45 + doc/guides/nics/features/af_xdp.ini | 11 + doc/guides/nics/index.rst | 1 + doc/guides/rel_notes/release_19_05.rst | 7 + drivers/net/Makefile | 1 + drivers/net/af_xdp/Makefile | 33 + drivers/net/af_xdp/meson.build | 21 + drivers/net/af_xdp/rte_eth_af_xdp.c | 930 ++++++++++++++++++ drivers/net/af_xdp/rte_pmd_af_xdp_version.map | 3 + drivers/net/meson.build | 1 + mk/rte.app.mk | 1 + 14 files changed, 1066 insertions(+)create mode 100644
doc/guides/nics/af_xdp.rst
create mode 100644 doc/guides/nics/features/af_xdp.ini create mode 100644drivers/net/af_xdp/Makefile
create mode 100644drivers/net/af_xdp/meson.buildcreate mode 100644drivers/net/af_xdp/rte_eth_af_xdp.ccreate mode 100644drivers/net/af_xdp/rte_pmd_af_xdp_version.mapMAINTAINERS +6
-0@@ -468,6 +468,12 @@ M: John W. Linville <[email protected]> F: drivers/net/af_packet/ F: doc/guides/nics/features/afpacket.ini +Linux AF_XDP +M: Xiaolong Ye <[email protected]> +M: Qi Zhang <[email protected]> +F: drivers/net/af_xdp/ +F: doc/guides/nics/features/af_xdp.rst + Amazon ENA M: Marcin Wojtas <[email protected]> M: Michal Krawczyk <[email protected]>config/common_base +5
-0@@ -416,6 +416,11 @@ CONFIG_RTE_LIBRTE_VMXNET3_DEBUG_TX_FREE=n # CONFIG_RTE_LIBRTE_PMD_AF_PACKET=n +# +# Compile software PMD backed by AF_XDP sockets (Linux only) +# +CONFIG_RTE_LIBRTE_PMD_AF_XDP=n + # # Compile link bonding PMD library #config/common_linux +1
-0@@ -19,6 +19,7 @@ CONFIG_RTE_LIBRTE_VHOST_POSTCOPY=n CONFIG_RTE_LIBRTE_PMD_VHOST=y CONFIG_RTE_LIBRTE_IFC_PMD=y CONFIG_RTE_LIBRTE_PMD_AF_PACKET=y +CONFIG_RTE_LIBRTE_PMD_AF_XDP=y CONFIG_RTE_LIBRTE_PMD_SOFTNIC=y CONFIG_RTE_LIBRTE_PMD_TAP=y CONFIG_RTE_LIBRTE_AVP_PMD=ydoc/guides/nics/af_xdp.rst new file +45
-0@@ -0,0 +1,45 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2018 Intel Corporation. + +AF_XDP Poll Mode Driver +========================== + +AF_XDP is an address family that is optimized for high performance +packet processing. AF_XDP sockets enable the possibility for XDP program to +redirect packets to a memory buffer in userspace. + +For the full details behind AF_XDP socket, you can refer to +`AF_XDP documentation in the Kernel +<https://www.kernel.org/doc/Documentation/networking/af_xdp.rst>`_. + +This Linux-specific PMD driver creates the AF_XDP socket and binds it to a +specific netdev queue, it allows a DPDK application to send and receive raw +packets through the socket which would bypass the kernel network stack. +Current implementation only supports single queue, multi-queues feature will +be added later. + +Options +------- + +The following options can be provided to set up an af_xdp port in DPDK. + +* ``iface`` - name of the Kernel interface to attach to (required); +* ``queue`` - netdev queue id (optional, default 0); + +Prerequisites +------------- + +This is a Linux-specific PMD, thus the following prerequisites apply: + +* A Linux Kernel (version > 4.18) with XDP sockets configuration enabled; +* libbpf (within kernel version > 5.1) with latest af_xdp support installed +* A Kernel bound interface to attach to. + +Set up an af_xdp interface +----------------------------- + +The following example will set up an af_xdp interface in DPDK: + +.. code-block:: console + + --vdev eth_af_xdp,iface=ens786f1,queue=0doc/guides/nics/features/af_xdp.ini new file +11
-0@@ -0,0 +1,11 @@ +; +; Supported features of the 'af_xdp' network poll mode driver. +; +; Refer to default.ini for the full list of available PMD features. +; +[Features] +Link status = Y +MTU update = Y +Promiscuous mode = Y +Stats per queue = Y +x86-64 = Ydoc/guides/nics/index.rst +1
-0@@ -12,6 +12,7 @@ Network Interface Controller Drivers features build_and_test af_packet + af_xdp ark atlantic avpdoc/guides/rel_notes/release_19_05.rst +7
-0@@ -65,6 +65,13 @@ New Features process. * Added support for Rx packet types list in a secondary process. +* **Added the AF_XDP PMD.** + + Added a Linux-specific PMD driver for AF_XDP, it can create the AF_XDP socket + and bind it to a specific netdev queue, it allows a DPDK application to send + and receive raw packets through the socket which would bypass the kernel + network stack to achieve high performance packet processing. + * **Updated Mellanox drivers.** New features and improvements were done in mlx4 and mlx5 PMDs:drivers/net/Makefile +1
-0@@ -9,6 +9,7 @@ ifeq ($(CONFIG_RTE_LIBRTE_THUNDERX_NICVF_PMD),d) endif DIRS-$(CONFIG_RTE_LIBRTE_PMD_AF_PACKET) += af_packet +DIRS-$(CONFIG_RTE_LIBRTE_PMD_AF_XDP) += af_xdp DIRS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += ark DIRS-$(CONFIG_RTE_LIBRTE_ATLANTIC_PMD) += atlantic DIRS-$(CONFIG_RTE_LIBRTE_AVP_PMD) += avpdrivers/net/af_xdp/Makefile new file +33
-0@@ -0,0 +1,33 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Intel Corporation + +include $(RTE_SDK)/mk/rte.vars.mk + +# +# library name +# +LIB = librte_pmd_af_xdp.a + +EXPORT_MAP := rte_pmd_af_xdp_version.map + +LIBABIVER := 1 + +CFLAGS += -O3 + +# require kernel version >= v5.1-rc1 +LINUX_VERSION := $(shell uname -r) +CFLAGS += -I/lib/modules/$(LINUX_VERSION)/build/tools/include +CFLAGS += -I/lib/modules/$(LINUX_VERSION)/build/tools/lib/bpf + +CFLAGS += $(WERROR_FLAGS) +LDLIBS += -lrte_eal -lrte_mbuf -lrte_mempool -lrte_ring +LDLIBS += -lrte_ethdev -lrte_net -lrte_kvargs +LDLIBS += -lrte_bus_vdev +LDLIBS += -lbpf + +# +# all source are stored in SRCS-y +# +SRCS-$(CONFIG_RTE_LIBRTE_PMD_AF_XDP) += rte_eth_af_xdp.c + +include $(RTE_SDK)/mk/rte.lib.mkdrivers/net/af_xdp/meson.build new file +21
-0@@ -0,0 +1,21 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Intel Corporation + +if host_machine.system() != 'linux' + build = false +endif + +bpf_dep = dependency('libbpf', required: false) +if bpf_dep.found() + build = true +else + bpf_dep = cc.find_library('libbpf', required: false) + if bpf_dep.found() and cc.has_header('xsk.h', dependencies: bpf_dep) + build = true + pkgconfig_extra_libs += '-lbpf' + else + build = false + endif +endif +sources = files('rte_eth_af_xdp.c') +ext_deps += bpf_depdrivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0@@ -0,0 +1,930 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2019 Intel Corporation. +*/ + +#include <rte_mbuf.h> +#include <rte_ethdev_driver.h> +#include <rte_ethdev_vdev.h> +#include <rte_malloc.h> +#include <rte_kvargs.h> +#include <rte_bus_vdev.h>
Quoted 8 lines
drivers/net/af_xdp/rte_eth_af_xdp.c line 16
Quoted 6 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +6
-0+#include <rte_string_fns.h> + +#include <linux/if_ether.h> +#include <linux/if_xdp.h> +#include <linux/if_link.h> +#include <asm/barrier.h> Is this include used?
Yes, this is needed by xsk.h below which calls smp_wmb() & smp_rmb().
Quoted 98 lines
Quoted 98 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +97
-0+#include <arpa/inet.h> +#include <net/if.h> +#include <sys/types.h> +#include <sys/socket.h> +#include <sys/ioctl.h> +#include <sys/mman.h> +#include <unistd.h> +#include <poll.h> +#include <bpf/bpf.h> +#include <xsk.h> + +#define RTE_LOGTYPE_AF_XDP RTE_LOGTYPE_USER1 +#ifndef SOL_XDP +#define SOL_XDP 283 +#endif + +#ifndef AF_XDP +#define AF_XDP 44 +#endif + +#ifndef PF_XDP +#define PF_XDP AF_XDP +#endif + +#define ETH_AF_XDP_IFACE_ARG "iface" +#define ETH_AF_XDP_QUEUE_IDX_ARG "queue" + +#define ETH_AF_XDP_FRAME_SIZE XSK_UMEM__DEFAULT_FRAME_SIZE +#define ETH_AF_XDP_NUM_BUFFERS 4096 +#define ETH_AF_XDP_DATA_HEADROOM 0 +#define ETH_AF_XDP_DFLT_NUM_DESCS XSK_RING_CONS__DEFAULT_NUM_DESCS +#define ETH_AF_XDP_DFLT_QUEUE_IDX 0 + +#define ETH_AF_XDP_RX_BATCH_SIZE 32 +#define ETH_AF_XDP_TX_BATCH_SIZE 32 + +#define ETH_AF_XDP_MAX_QUEUE_PAIRS 16 + +struct xsk_umem_info { +struct xsk_ring_prod fq; +struct xsk_ring_cons cq; +struct xsk_umem *umem; +struct rte_ring *buf_ring; +void *buffer; +}; + +struct pkt_rx_queue { +struct xsk_ring_cons rx; +struct xsk_umem_info *umem; +struct xsk_socket *xsk; +struct rte_mempool *mb_pool; + +uint64_t rx_pkts; +uint64_t rx_bytes; +uint64_t rx_dropped; + +struct pkt_tx_queue *pair; +uint16_t queue_idx; +}; + +struct pkt_tx_queue { +struct xsk_ring_prod tx; + +uint64_t tx_pkts; +uint64_t err_pkts; +uint64_t tx_bytes; + +struct pkt_rx_queue *pair; +uint16_t queue_idx; +}; + +struct pmd_internals { +int if_index; +char if_name[IFNAMSIZ ]; +uint16_t queue_idx; +struct ether_addr eth_addr; +struct xsk_umem_info *umem; +struct rte_mempool *mb_pool_share; + +struct pkt_rx_queue rx_queues[ETH_AF_XDP_MAX_QUEUE_PAIRS ]; +struct pkt_tx_queue tx_queues[ETH_AF_XDP_MAX_QUEUE_PAIRS ]; +}; + +static const char *const valid_arguments[] = { +ETH_AF_XDP_IFACE_ARG , +ETH_AF_XDP_QUEUE_IDX_ARG , +NULL +}; + +static struct rte_eth_link pmd_link = { + .link_speed =ETH_SPEED_NUM_10G , + .link_duplex =ETH_LINK_FULL_DUPLEX , + .link_status =ETH_LINK_DOWN , + .link_autoneg =ETH_LINK_AUTONEG +}; + +static inline int
Quoted 8 lines
drivers/net/af_xdp/rte_eth_af_xdp.c line 119
Quoted 6 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +6
-0+reserve_fill_queue(struct xsk_umem_info *umem, int reserve_size) +{ +struct xsk_ring_prod *fq = &umem->fq; +uint32_t idx; +void *addr =NULL ; +int i, ret =0 ;No need to initialize 'ret'. Is there a point to set 'addr'?
Ok, I'll keep it uninitialized.
addr needs be set otherwise there would be a build error:
drivers/net/af_xdp/rte_eth_af_xdp.c:129:40: error: ‘addr’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
*xsk_ring_prod__fill_addr(fq, idx++) = (uint64_t)addr;
^Quoted 13 lines
drivers/net/af_xdp/rte_eth_af_xdp.c line 129
Quoted 11 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+ + ret =xsk_ring_prod__reserve (fq, reserve_size, &idx); +if (!ret) { +RTE_LOG (ERR ,AF_XDP ,"Failed to reserve enough fq descs.\n" ); +return ret; + } + +for (i =0 ; i < reserve_size; i++) { +rte_ring_dequeue (umem->buf_ring, &addr); + *xsk_ring_prod__fill_addr (fq, idx++) = (uint64_t )addr;Consider introducing a tmp variable to make this more readable.
Got it.
Quoted 22 lines
Quoted 22 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +21
-0+ } + +xsk_ring_prod__submit (fq, reserve_size); + +return 0 ; +} + +static uint16_t +eth_af_xdp_rx (void *queue,struct rte_mbuf **bufs,uint16_t nb_pkts) +{ +struct pkt_rx_queue *rxq = queue; +struct xsk_ring_cons *rx = &rxq->rx; +struct xsk_umem_info *umem = rxq->umem; +struct xsk_ring_prod *fq = &umem->fq; +uint32_t idx_rx; +uint32_t free_thresh = fq->size >>1 ; +struct rte_mbuf *mbuf; +unsigned long dropped =0 ; +unsigned long rx_bytes =0 ; +uint16_t count =0 ; +int rcvd, i;
Quoted 8 lines
drivers/net/af_xdp/rte_eth_af_xdp.c
Quoted 5 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +5
-0+ + nb_pkts = nb_pkts <ETH_AF_XDP_RX_BATCH_SIZE ? + nb_pkts :ETH_AF_XDP_RX_BATCH_SIZE ; + + rcvd = xsk_ring_cons__peek(rx, nb_pkts, &idx_rx); > + if (!rcvd)Since peek returns the number of entries, not a boolean, do:
rcvd == 0
Got it.
Quoted 11 lines
drivers/net/af_xdp/rte_eth_af_xdp.c line 164
Quoted 9 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +8
-0+return 0 ; + +if (xsk_prod_nb_free (fq, free_thresh) >= free_thresh) + (void )reserve_fill_queue (umem,ETH_AF_XDP_RX_BATCH_SIZE ); + +for (i =0 ; i < rcvd; i++) { +uint64_t addr =xsk_ring_cons__rx_desc (rx, idx_rx)->addr; +uint32_t len =xsk_ring_cons__rx_desc (rx, idx_rx++)->len;Use a tmp variable, instead of two calls.
Got it.
Quoted 6 lines
drivers/net/af_xdp/rte_eth_af_xdp.c line 166
Quoted 3 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +2
-0+char *pkt =xsk_umem__get_data (rxq->umem->buffer, addr); +Don't mix declaration and code. Why is this a char pointer? As opppose to void.
I'll split the declaration and code.
pkt should be a void *, will correct it in next version.
Quoted 5 lines
drivers/net/af_xdp/rte_eth_af_xdp.c line 168
Quoted 3 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +2
-0+ mbuf =rte_pktmbuf_alloc (rxq->mb_pool); +if (mbuf) {1.8.1
Got it.
Quoted 4 lines
drivers/net/af_xdp/rte_eth_af_xdp.c line 169
Quoted 2 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +1
-0+ memcpy(rte_pktmbuf_mtod(mbuf, void*), pkt, len);
rte_memcpy()
Got it.
Quoted 5 lines
drivers/net/af_xdp/rte_eth_af_xdp.c line 171
Quoted 3 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +2
-0+rte_pktmbuf_pkt_len (mbuf) = +rte_pktmbuf_data_len (mbuf) = len;Consider splitting this into two statements.
Got it.
Quoted 21 lines
Quoted 21 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +20
-0+ rx_bytes += len; + bufs[count++] = mbuf; + }else { + dropped++; + } +rte_ring_enqueue (umem->buf_ring, (void *)addr); + } + +xsk_ring_cons__release (rx, rcvd); + +/* statistics */ + rxq->rx_pkts += (rcvd - dropped); + rxq->rx_bytes += rx_bytes; + rxq->rx_dropped += dropped; + +return count; +} + +static void pull_umem_cq (struct xsk_umem_info *umem,int size) +{
Quoted 8 lines
drivers/net/af_xdp/rte_eth_af_xdp.c line 197
Quoted 6 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +6
-0+struct xsk_ring_cons *cq = &umem->cq; +int i, n; +uint32_t idx_cq; +uint64_t addr; + + n =xsk_ring_cons__peek (cq, size, &idx_cq);Use
size_tfor n.
Got it.
Quoted 5 lines
drivers/net/af_xdp/rte_eth_af_xdp.c line 199
Quoted 3 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +2
-0+if (n >0 ) { +for (i =0 ; i < n; i++) {Consider declaring 'addr' in this scope.
Got it.
Quoted 17 lines
Quoted 5 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +4
-0+ addr = *xsk_ring_cons__comp_addr (cq, + idx_cq++); > + rte_ring_enqueue(umem->buf_ring, (void *)addr); + } +drivers/net/af_xdp/rte_eth_af_xdp.c line 214
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+xsk_ring_cons__release (cq, n); + } +} + +static void kick_tx (struct pkt_tx_queue *txq) +{ +struct xsk_umem_info *umem = txq->pair->umem; +int ret; + +while (1 ) {for (;;)
Got it, is it a preference in DPDK convention for loop?
Quoted 8 lines
drivers/net/af_xdp/rte_eth_af_xdp.c line 219
Quoted 6 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +5
-0+ ret =sendto (xsk_socket__fd (txq->pair->xsk),NULL ,0 , +MSG_DONTWAIT ,NULL ,0 ); + +/* everything is ok */ +if (ret >=0 )Use
likely()?
Got it.
Quoted 21 lines
Quoted 21 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +20
-0+break ; + +/* some thing unexpected */ +if (errno !=EBUSY && errno !=EAGAIN ) +break ; + +/* pull from complete qeueu to leave more space */ +if (errno ==EAGAIN ) +pull_umem_cq (umem,ETH_AF_XDP_TX_BATCH_SIZE ); + } +pull_umem_cq (umem,ETH_AF_XDP_TX_BATCH_SIZE ); +} + +static uint16_t +eth_af_xdp_tx (void *queue,struct rte_mbuf **bufs,uint16_t nb_pkts) +{ +struct pkt_tx_queue *txq = queue; +struct xsk_umem_info *umem = txq->pair->umem; +struct rte_mbuf *mbuf; +void *addrs[ETH_AF_XDP_TX_BATCH_SIZE ];
Quoted 8 lines
drivers/net/af_xdp/rte_eth_af_xdp.c line 245
Quoted 6 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +6
-0+unsigned long tx_bytes =0 ; +int i, valid =0 ; +uint32_t idx_tx; + + nb_pkts = nb_pkts <ETH_AF_XDP_TX_BATCH_SIZE ? + nb_pkts :ETH_AF_XDP_TX_BATCH_SIZE ;Use
RTE_MIN().
Got it.
Quoted 9 lines
drivers/net/af_xdp/rte_eth_af_xdp.c line 251
Quoted 7 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +6
-0+ +pull_umem_cq (umem, nb_pkts); + + nb_pkts =rte_ring_dequeue_bulk (umem->buf_ring, addrs, + nb_pkts,NULL ); +if (!nb_pkts)
nb_pkts== 0
Got it.
Quoted 13 lines
drivers/net/af_xdp/rte_eth_af_xdp.c line 261
Quoted 11 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+return 0 ; + +if (xsk_ring_prod__reserve (&txq->tx, nb_pkts, &idx_tx) != nb_pkts) { +kick_tx (txq); +return 0 ; + } + +for (i =0 ; i < nb_pkts; i++) { +struct xdp_desc *desc; +char *pkt;Use void pointer?
Got it.
Quoted 5 lines
drivers/net/af_xdp/rte_eth_af_xdp.c line 263
Quoted 3 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +2
-0+unsigned int buf_len =ETH_AF_XDP_FRAME_SIZE + -ETH_AF_XDP_DATA_HEADROOM ;Use
uint32_t, as you seem to do elsewhere.
Got it.
Quoted 12 lines
drivers/net/af_xdp/rte_eth_af_xdp.c line 272
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +9
-0+ desc =xsk_ring_prod__tx_desc (&txq->tx, idx_tx + i); + mbuf = bufs[i]; +if (mbuf->pkt_len <= buf_len) { + desc->addr = (uint64_t )addrs[valid]; + desc->len = mbuf->pkt_len; + pkt =xsk_umem__get_data (umem->buffer, + desc->addr); + memcpy(pkt, rte_pktmbuf_mtod(mbuf, void *), + desc->len);
rte_memcpy()
Got it.
Quoted 183 lines
Quoted 183 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +182
-0+ valid++; + tx_bytes += mbuf->pkt_len; + } +rte_pktmbuf_free (mbuf); + } + +xsk_ring_prod__submit (&txq->tx, nb_pkts); + +kick_tx (txq); + +if (valid < nb_pkts) +rte_ring_enqueue_bulk (umem->buf_ring, &addrs[valid], + nb_pkts - valid,NULL ); + + txq->err_pkts += nb_pkts - valid; + txq->tx_pkts += valid; + txq->tx_bytes += tx_bytes; + +return nb_pkts; +} + +static int +eth_dev_start (struct rte_eth_dev *dev) +{ + dev->data->dev_link.link_status =ETH_LINK_UP ; + +return 0 ; +} + +/* This function gets called when the current port gets stopped. */ +static void +eth_dev_stop (struct rte_eth_dev *dev) +{ + dev->data->dev_link.link_status =ETH_LINK_DOWN ; +} + +static int +eth_dev_configure(struct rte_eth_dev *dev __rte_unused) +{ +/* rx/tx must be paired */ +if (dev->data->nb_rx_queues != dev->data->nb_tx_queues) +return -EINVAL ; + +return 0 ; +} + +static void +eth_dev_info (struct rte_eth_dev *dev,struct rte_eth_dev_info *dev_info) +{ +struct pmd_internals *internals = dev->data->dev_private; + + dev_info->if_index = internals->if_index; + dev_info->max_mac_addrs =1 ; + dev_info->max_rx_pktlen = (uint32_t )ETH_FRAME_LEN ; + dev_info->max_rx_queues =1 ; + dev_info->max_tx_queues =1 ; + dev_info->min_rx_bufsize =0 ; + + dev_info->default_rxportconf.nb_queues =1 ; + dev_info->default_txportconf.nb_queues =1 ; + dev_info->default_rxportconf.ring_size =ETH_AF_XDP_DFLT_NUM_DESCS ; + dev_info->default_txportconf.ring_size =ETH_AF_XDP_DFLT_NUM_DESCS ; +} + +static int +eth_stats_get (struct rte_eth_dev *dev,struct rte_eth_stats *stats) +{ +struct pmd_internals *internals = dev->data->dev_private; +struct xdp_statistics xdp_stats; +struct pkt_rx_queue *rxq; +socklen_t optlen; +int i; + + optlen =sizeof (struct xdp_statistics ); +for (i =0 ; i < dev->data->nb_rx_queues; i++) { + rxq = &internals->rx_queues[i]; + stats->q_ipackets[i] = internals->rx_queues[i].rx_pkts; + stats->q_ibytes[i] = internals->rx_queues[i].rx_bytes; + + stats->q_opackets[i] = internals->tx_queues[i].tx_pkts; + stats->q_obytes[i] = internals->tx_queues[i].tx_bytes; + + stats->ipackets += stats->q_ipackets[i]; + stats->ibytes += stats->q_ibytes[i]; + stats->imissed += internals->rx_queues[i].rx_dropped; +getsockopt (xsk_socket__fd (rxq->xsk),SOL_XDP ,XDP_STATISTICS , + &xdp_stats, &optlen); + stats->imissed += xdp_stats.rx_dropped; + + stats->opackets += stats->q_opackets[i]; + stats->oerrors += stats->q_errors[i]; + stats->oerrors += internals->tx_queues[i].err_pkts; + stats->obytes += stats->q_obytes[i]; + } + +return 0 ; +} + +static void +eth_stats_reset (struct rte_eth_dev *dev) +{ +struct pmd_internals *internals = dev->data->dev_private; +int i; + +for (i =0 ; i <ETH_AF_XDP_MAX_QUEUE_PAIRS ; i++) { + internals->rx_queues[i].rx_pkts =0 ; + internals->rx_queues[i].rx_bytes =0 ; + internals->rx_queues[i].rx_dropped =0 ; + + internals->tx_queues[i].tx_pkts =0 ; + internals->tx_queues[i].err_pkts =0 ; + internals->tx_queues[i].tx_bytes =0 ; + } +} + +static void remove_xdp_program (struct pmd_internals *internals) +{ +uint32_t curr_prog_id =0 ; + +if (bpf_get_link_xdp_id (internals->if_index, &curr_prog_id, +XDP_FLAGS_UPDATE_IF_NOEXIST )) { +RTE_LOG (ERR ,AF_XDP ,"bpf_get_link_xdp_id failed\n" ); +return ; + } +bpf_set_link_xdp_fd (internals->if_index,-1 , +XDP_FLAGS_UPDATE_IF_NOEXIST ); +} + +static void +eth_dev_close (struct rte_eth_dev *dev) +{ +struct pmd_internals *internals = dev->data->dev_private; +struct pkt_rx_queue *rxq; +int i; + +RTE_LOG (INFO ,AF_XDP ,"Closing AF_XDP ethdev on numa socket %u\n" , +rte_socket_id ()); + +for (i =0 ; i <ETH_AF_XDP_MAX_QUEUE_PAIRS ; i++) { + rxq = &internals->rx_queues[i]; +if (!rxq->umem) +break ; +xsk_socket__delete (rxq->xsk); + } + + (void )xsk_umem__delete (internals->umem->umem); +remove_xdp_program (internals); +} + +static void +eth_queue_release(void *q __rte_unused) +{ +} + +static int +eth_link_update(struct rte_eth_dev *dev __rte_unused, + int wait_to_complete __rte_unused) +{ +return 0 ; +} + +static void xdp_umem_destroy (struct xsk_umem_info *umem) +{ +free (umem->buffer); + umem->buffer =NULL ; + +rte_ring_free (umem->buf_ring); + umem->buf_ring =NULL ; + +free (umem); + umem =NULL ; +} + +static struct xsk_umem_info *xdp_umem_configure (void ) +{ +struct xsk_umem_info *umem; +struct xsk_umem_config usr_config = { + .fill_size =ETH_AF_XDP_DFLT_NUM_DESCS , + .comp_size =ETH_AF_XDP_DFLT_NUM_DESCS , + .frame_size =ETH_AF_XDP_FRAME_SIZE , + .frame_headroom =ETH_AF_XDP_DATA_HEADROOM }; +void *bufs =NULL ;
Quoted 8 lines
drivers/net/af_xdp/rte_eth_af_xdp.c line 460
Quoted 6 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +6
-0+char ring_name[0x100 ]; +int ret; +uint64_t i; + + umem =calloc (1 ,sizeof (*umem)); +if (!umem) {1.8.1
Got it.
Quoted 8 lines
drivers/net/af_xdp/rte_eth_af_xdp.c line 465
Quoted 6 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +5
-0+RTE_LOG (ERR ,AF_XDP ,"Failed to allocate umem info" ); +return NULL ; + } + +snprintf (ring_name,0x100 ,"af_xdp_ring" );Again the magical
0x100.
Will correct it.
Quoted 8 lines
drivers/net/af_xdp/rte_eth_af_xdp.c line 470
Quoted 6 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +5
-0+ umem->buf_ring =rte_ring_create (ring_name, +ETH_AF_XDP_NUM_BUFFERS , +SOCKET_ID_ANY , +0x0 ); +if (!umem->buf_ring) {1.8.1
Got it.
Quoted 39 lines
Quoted 39 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +38
-0+RTE_LOG (ERR ,AF_XDP , +"Failed to create rte_ring\n" ); + goto err; + } + +for (i =0 ; i <ETH_AF_XDP_NUM_BUFFERS ; i++) +rte_ring_enqueue (umem->buf_ring, + (void *)(i *ETH_AF_XDP_FRAME_SIZE + +ETH_AF_XDP_DATA_HEADROOM )); + +if (posix_memalign (&bufs,getpagesize (), +ETH_AF_XDP_NUM_BUFFERS *ETH_AF_XDP_FRAME_SIZE )) { +RTE_LOG (ERR ,AF_XDP ,"Failed to allocate memory pool.\n" ); + goto err; + } + ret =xsk_umem__create (&umem->umem, bufs, +ETH_AF_XDP_NUM_BUFFERS *ETH_AF_XDP_FRAME_SIZE , + &umem->fq, &umem->cq, + &usr_config); + +if (ret) { +RTE_LOG (ERR ,AF_XDP ,"Failed to create umem" ); + goto err; + } + umem->buffer = bufs; + +return umem; + +err: +xdp_umem_destroy (umem); +return NULL ; +} + +static int +xsk_configure (struct pmd_internals *internals,struct pkt_rx_queue *rxq, +int ring_size) +{ +struct xsk_socket_config cfg;
Quoted 8 lines
drivers/net/af_xdp/rte_eth_af_xdp.c line 514
Quoted 6 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +6
-0+struct pkt_tx_queue *txq = rxq->pair; +int ret =0 ; +int reserve_size; + + rxq->umem =xdp_umem_configure (); +if (!rxq->umem) {1.8.1
Got it.
Quoted 37 lines
Quoted 37 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +36
-0+ ret = -ENOMEM ; + goto err; + } + + cfg.rx_size = ring_size; + cfg.tx_size = ring_size; + cfg.libbpf_flags =0 ; + cfg.xdp_flags =XDP_FLAGS_UPDATE_IF_NOEXIST ; + cfg.bind_flags =0 ; + ret =xsk_socket__create (&rxq->xsk, internals->if_name, + internals->queue_idx, rxq->umem->umem, &rxq->rx, + &txq->tx, &cfg); +if (ret) { +RTE_LOG (ERR ,AF_XDP ,"Failed to create xsk socket.\n" ); + goto err; + } + + reserve_size =ETH_AF_XDP_DFLT_NUM_DESCS /2 ; + ret =reserve_fill_queue (rxq->umem, reserve_size); +if (ret) { +RTE_LOG (ERR ,AF_XDP ,"Failed to reserve fill queue.\n" ); + goto err; + } + +return 0 ; + +err: +xdp_umem_destroy (rxq->umem); + +return ret; +} + +static void +queue_reset (struct pmd_internals *internals,uint16_t queue_idx) +{ +struct pkt_rx_queue *rxq = &internals->rx_queues[queue_idx];
Quoted 8 lines
drivers/net/af_xdp/rte_eth_af_xdp.c line 556
Quoted 6 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +6
-0+struct pkt_tx_queue *txq = rxq->pair; +int xsk_fd =xsk_socket__fd (rxq->xsk); + +if (xsk_fd) { +close (xsk_fd); +if (internals->umem) {1.8.1
Got it.
Quoted 17 lines
Quoted 17 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +16
-0+xdp_umem_destroy (internals->umem); + internals->umem =NULL ; + } + } +memset (rxq,0 ,sizeof (*rxq)); +memset (txq,0 ,sizeof (*txq)); + rxq->pair = txq; + txq->pair = rxq; + rxq->queue_idx = queue_idx; + txq->queue_idx = queue_idx; +} + +static int +eth_rx_queue_setup(struct rte_eth_dev *dev, + uint16_t rx_queue_id, + uint16_t nb_rx_desc,
Quoted 8 lines
drivers/net/af_xdp/rte_eth_af_xdp.c line 578
Quoted 6 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +6
-0+ unsigned int socket_id __rte_unused, + const struct rte_eth_rxconf *rx_conf __rte_unused, + struct rte_mempool *mb_pool) +{ + struct pmd_internals *internals = dev->data->dev_private; +unsigned int buf_size, data_size;
uint32_t
Got it.
Quoted 5 lines
drivers/net/af_xdp/rte_eth_af_xdp.c line 580
Quoted 3 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +2
-0+struct pkt_rx_queue *rxq; +int ret =0 ;No need to set 'ret'. Alternatively, restructure so you always return 'ret'.
Got it.
Quoted 132 lines
Quoted 132 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +131
-0+ + rxq = &internals->rx_queues[rx_queue_id]; +queue_reset (internals, rx_queue_id); + +/* Now get the space available for data in the mbuf */ + buf_size =rte_pktmbuf_data_room_size (mb_pool) - +RTE_PKTMBUF_HEADROOM ; + data_size =ETH_AF_XDP_FRAME_SIZE -ETH_AF_XDP_DATA_HEADROOM ; + +if (data_size > buf_size) { +RTE_LOG (ERR ,AF_XDP , +"%s: %d bytes will not fit in mbuf (%d bytes)\n" , + dev->device->name, data_size, buf_size); + ret = -ENOMEM ; + goto err; + } + + rxq->mb_pool = mb_pool; + +if (xsk_configure (internals, rxq, nb_rx_desc)) { +RTE_LOG (ERR ,AF_XDP , +"Failed to configure xdp socket\n" ); + ret = -EINVAL ; + goto err; + } + + internals->umem = rxq->umem; + + dev->data->rx_queues[rx_queue_id] = rxq; +return 0 ; + +err: +queue_reset (internals, rx_queue_id); +return ret; +} + +static int +eth_tx_queue_setup (struct rte_eth_dev *dev, +uint16_t tx_queue_id, + uint16_t nb_tx_desc __rte_unused, + unsigned int socket_id __rte_unused, + const struct rte_eth_txconf *tx_conf __rte_unused) +{ +struct pmd_internals *internals = dev->data->dev_private; +struct pkt_tx_queue *txq; + + txq = &internals->tx_queues[tx_queue_id]; + + dev->data->tx_queues[tx_queue_id] = txq; +return 0 ; +} + +static int +eth_dev_mtu_set (struct rte_eth_dev *dev,uint16_t mtu) +{ +struct pmd_internals *internals = dev->data->dev_private; +struct ifreq ifr = { .ifr_mtu = mtu }; +int ret; +int s; + + s =socket (PF_INET ,SOCK_DGRAM ,0 ); +if (s <0 ) +return -EINVAL ; + +strlcpy (ifr.ifr_name, internals->if_name,IFNAMSIZ ); + ret =ioctl (s,SIOCSIFMTU , &ifr); +close (s); + +if (ret <0 ) +return -EINVAL ; + +return 0 ; +} + +static void +eth_dev_change_flags (char *if_name,uint32_t flags,uint32_t mask) +{ +struct ifreq ifr; +int s; + + s =socket (PF_INET ,SOCK_DGRAM ,0 ); +if (s <0 ) +return ; + +strlcpy (ifr.ifr_name, if_name,IFNAMSIZ ); +if (ioctl (s,SIOCGIFFLAGS , &ifr) <0 ) + goto out; + ifr.ifr_flags &= mask; + ifr.ifr_flags |= flags; +if (ioctl (s,SIOCSIFFLAGS , &ifr) <0 ) + goto out; +out: +close (s); +} + +static void +eth_dev_promiscuous_enable (struct rte_eth_dev *dev) +{ +struct pmd_internals *internals = dev->data->dev_private; + +eth_dev_change_flags (internals->if_name,IFF_PROMISC , ~0 ); +} + +static void +eth_dev_promiscuous_disable (struct rte_eth_dev *dev) +{ +struct pmd_internals *internals = dev->data->dev_private; + +eth_dev_change_flags (internals->if_name,0 , ~IFF_PROMISC ); +} + +static const struct eth_dev_ops ops = { + .dev_start = eth_dev_start, + .dev_stop = eth_dev_stop, + .dev_close = eth_dev_close, + .dev_configure = eth_dev_configure, + .dev_infos_get = eth_dev_info, + .mtu_set = eth_dev_mtu_set, + .promiscuous_enable = eth_dev_promiscuous_enable, + .promiscuous_disable = eth_dev_promiscuous_disable, + .rx_queue_setup = eth_rx_queue_setup, + .tx_queue_setup = eth_tx_queue_setup, + .rx_queue_release = eth_queue_release, + .tx_queue_release = eth_queue_release, + .link_update = eth_link_update, + .stats_get = eth_stats_get, + .stats_reset = eth_stats_reset, +}; + +/** parse integer from integer argument */ +static int
Quoted 8 lines
drivers/net/af_xdp/rte_eth_af_xdp.c line 717
Quoted 6 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +6
-0+parse_integer_arg(const char *key __rte_unused, + const char *value, void *extra_args) +{ + int *i = (int *)extra_args; + + *i =atoi (value);Use
strtol().
Got it.
Quoted 12 lines
Quoted 12 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +11
-0+if (*i <0 ) { +RTE_LOG (ERR ,AF_XDP ,"Argument has to be positive.\n" ); +return -EINVAL ; + } + +return 0 ; +} + +/** parse name argument */ +static int +parse_name_arg(const char *key __rte_unused,
Quoted 8 lines
drivers/net/af_xdp/rte_eth_af_xdp.c line 733
Quoted 5 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +5
-0+ const char *value, void *extra_args) +{ + char *name = extra_args; + +if (strlen (value) >IFNAMSIZ ) {The buffer is IFNAMSIZ bytes (which it should be), so it can't hold a string with
strlen()== IFNAMSIZ.
Good catch. should be strlen(value) > IFNAMSIZ - 1
Quoted 11 lines
Quoted 11 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+RTE_LOG (ERR ,AF_XDP ,"Invalid name %s, should be less than " +"%u bytes.\n" , value,IFNAMSIZ ); +return -EINVAL ; + } + +strlcpy (name, value,IFNAMSIZ ); + +return 0 ; +} +
Quoted 8 lines
drivers/net/af_xdp/rte_eth_af_xdp.c line 749
Quoted 6 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +6
-0+static int +parse_parameters (struct rte_kvargs *kvlist, +char *if_name, +int *queue_idx) +{ +int ret =0 ;Should not be initialized.
Got it.
Quoted 14 lines
drivers/net/af_xdp/rte_eth_af_xdp.c line 759
Quoted 11 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+ + ret =rte_kvargs_process (kvlist,ETH_AF_XDP_IFACE_ARG , + &parse_name_arg, if_name); +if (ret <0 ) + goto free_kvlist; + + ret =rte_kvargs_process (kvlist,ETH_AF_XDP_QUEUE_IDX_ARG , + &parse_integer_arg, queue_idx); +if (ret <0 ) + goto free_kvlist;I fail to see the point of this goto, but maybe there's more code to follow in future patches.
Yes, We'll add more parameters in the future.
Quoted 44 lines
Quoted 44 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +43
-0+ +free_kvlist: +rte_kvargs_free (kvlist); +return ret; +} + +static int +get_iface_info (const char *if_name, +struct ether_addr *eth_addr, +int *if_index) +{ +struct ifreq ifr; +int sock =socket (AF_INET ,SOCK_DGRAM ,IPPROTO_IP ); + +if (sock <0 ) +return -1 ; + +strlcpy (ifr.ifr_name, if_name,IFNAMSIZ ); +if (ioctl (sock,SIOCGIFINDEX , &ifr)) + goto error; + +if (ioctl (sock,SIOCGIFHWADDR , &ifr)) + goto error; + +rte_memcpy (eth_addr, ifr.ifr_hwaddr.sa_data,ETHER_ADDR_LEN ); + +close (sock); + *if_index =if_nametoindex (if_name); +return 0 ; + +error: +close (sock); +return -1 ; +} + +static int +init_internals (struct rte_vdev_device *dev, +const char *if_name, +int queue_idx, +struct rte_eth_dev **eth_dev) +{ +const char *name =rte_vdev_device_name (dev); +const unsigned int numa_node = dev->device.numa_node;
Quoted 8 lines
drivers/net/af_xdp/rte_eth_af_xdp.c line 808
Quoted 6 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +6
-0+struct pmd_internals *internals =NULL ; +int ret; +int i; + + internals =rte_zmalloc_socket (name,sizeof (*internals),0 , numa_node); +if (!internals)1.8.1
Got it.
Quoted 12 lines
Quoted 12 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +11
-0+return -ENOMEM ; + + internals->queue_idx = queue_idx; +strlcpy (internals->if_name, if_name,IFNAMSIZ ); + +for (i =0 ; i <ETH_AF_XDP_MAX_QUEUE_PAIRS ; i++) { + internals->tx_queues[i].pair = &internals->rx_queues[i]; + internals->rx_queues[i].pair = &internals->tx_queues[i]; + } + + ret = get_iface_info(if_name, &internals->eth_addr,
Quoted 8 lines
drivers/net/af_xdp/rte_eth_af_xdp.c line 825
Quoted 6 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +6
-0+ &internals->if_index); +if (ret) + goto err; + + *eth_dev =rte_eth_vdev_allocate (dev,0 ); +if (!*eth_dev)1.8.1
Got it.
Quoted 138 lines
Quoted 138 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +105
-0+ goto err; + + (*eth_dev)->data->dev_private = internals; + (*eth_dev)->data->dev_link = pmd_link; + (*eth_dev)->data->mac_addrs = &internals->eth_addr; + (*eth_dev)->dev_ops = &ops; + (*eth_dev)->rx_pkt_burst = eth_af_xdp_rx; + (*eth_dev)->tx_pkt_burst = eth_af_xdp_tx; + +return 0 ; + +err: +rte_free (internals); +return -1 ; +} + +static int +rte_pmd_af_xdp_probe (struct rte_vdev_device *dev) +{ +struct rte_kvargs *kvlist; +char if_name[IFNAMSIZ ]; +int xsk_queue_idx =ETH_AF_XDP_DFLT_QUEUE_IDX ; +struct rte_eth_dev *eth_dev =NULL ; +const char *name; +int ret; + +RTE_LOG (INFO ,AF_XDP ,"Initializing pmd_af_xdp for %s\n" , +rte_vdev_device_name (dev)); + + name =rte_vdev_device_name (dev); +if (rte_eal_process_type () ==RTE_PROC_SECONDARY && +strlen (rte_vdev_device_args (dev)) ==0 ) { + eth_dev =rte_eth_dev_attach_secondary (name); +if (!eth_dev) { +RTE_LOG (ERR ,AF_XDP ,"Failed to probe %s\n" , name); +return -EINVAL ; + } + eth_dev->dev_ops = &ops; +rte_eth_dev_probing_finish (eth_dev); +return 0 ; + } + + kvlist =rte_kvargs_parse (rte_vdev_device_args (dev), valid_arguments); +if (!kvlist) { +RTE_LOG (ERR ,AF_XDP ,"Invalid kvargs key\n" ); +return -EINVAL ; + } + +if (dev->device.numa_node ==SOCKET_ID_ANY ) + dev->device.numa_node =rte_socket_id (); + +if (parse_parameters (kvlist, if_name, &xsk_queue_idx) <0 ) { +RTE_LOG (ERR ,AF_XDP ,"Invalid kvargs value\n" ); +return -EINVAL ; + } + + ret =init_internals (dev, if_name, xsk_queue_idx, ð_dev); +if (ret) { +RTE_LOG (ERR ,AF_XDP ,"Failed to init internals\n" ); +return ret; + } + +rte_eth_dev_probing_finish (eth_dev); + +return 0 ; +} + +static int +rte_pmd_af_xdp_remove (struct rte_vdev_device *dev) +{ +struct rte_eth_dev *eth_dev =NULL ; +struct pmd_internals *internals; + +RTE_LOG (INFO ,AF_XDP ,"Removing AF_XDP ethdev on numa socket %u\n" , +rte_socket_id ()); + +if (!dev) +return -1 ; + +/* find the ethdev entry */ + eth_dev =rte_eth_dev_allocated (rte_vdev_device_name (dev)); +if (!eth_dev) +return -1 ; + + internals = eth_dev->data->dev_private; + +rte_ring_free (internals->umem->buf_ring); +rte_free (internals->umem->buffer); +rte_free (internals->umem); + +rte_eth_dev_release_port (eth_dev); + + +return 0 ; +} + +static struct rte_vdev_driver pmd_af_xdp_drv = { + .probe = rte_pmd_af_xdp_probe, + .remove = rte_pmd_af_xdp_remove, +}; + +RTE_PMD_REGISTER_VDEV (eth_af_xdp, pmd_af_xdp_drv); +RTE_PMD_REGISTER_PARAM_STRING (eth_af_xdp, +"iface=<string> " +"queue=<int> " );drivers/net/af_xdp/rte_pmd_af_xdp_version.map new file +3
-0@@ -0,0 +1,3 @@ +DPDK_19.05 { + local: *; +};drivers/net/meson.build +1
-0@@ -2,6 +2,7 @@ # Copyright(c) 2017 Intel Corporation drivers = ['af_packet', + 'af_xdp', 'ark', 'atlantic', 'avp',mk/rte.app.mk +1
-0@@ -143,6 +143,7 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_DPAA2_MEMPOOL) += -lrte_mempool_dpaa2 endif _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AF_PACKET) += -lrte_pmd_af_packet +_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AF_XDP) += -lrte_pmd_af_xdp -lelf -lbpf _LDLIBS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += -lrte_pmd_ark _LDLIBS-$(CONFIG_RTE_LIBRTE_ATLANTIC_PMD) += -lrte_pmd_atlantic _LDLIBS-$(CONFIG_RTE_LIBRTE_AVP_PMD) += -lrte_pmd_avp
Re: [PATCH v2 1/6] net/af_xdp: introduce AF XDP PMD driver
In reply to Xiaolong YeLots of little review comments. This is what I saw in 30 minutes. Expect more.
drivers/net/af_xdp/rte_eth_af_xdp.c line 153
Quoted 2 lines · On Tue, 19 Mar 2019 15:12:51 +0800 Xiaolong Ye <[email protected]> wrote:
drivers/net/af_xdp/rte_eth_af_xdp.c new file +2
-0+ nb_pkts = nb_pkts <ETH_AF_XDP_RX_BATCH_SIZE ? + nb_pkts :ETH_AF_XDP_RX_BATCH_SIZE ;
Maybe use RTE_MIN() ?
drivers/net/af_xdp/rte_eth_af_xdp.c line 169
Quoted 3 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +3
-0+ mbuf =rte_pktmbuf_alloc (rxq->mb_pool); +if (mbuf) { + memcpy(rte_pktmbuf_mtod(mbuf, void*), pkt, len);
Space necessary in "void*"
Use rte_memcpy.
drivers/net/af_xdp/rte_eth_af_xdp.c line 199
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+static void pull_umem_cq (struct xsk_umem_info *umem,int size) +{ +struct xsk_ring_cons *cq = &umem->cq; +int i, n; +uint32_t idx_cq; +uint64_t addr; + + n =xsk_ring_cons__peek (cq, size, &idx_cq); +if (n >0 ) { +for (i =0 ; i < n; i++) {
You don't need the if (n > 0) since if n <= 0 the for loop would happen 0 times.
drivers/net/af_xdp/rte_eth_af_xdp.c line 220
Quoted 13 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +13
-0+ +static void kick_tx (struct pkt_tx_queue *txq) +{ +struct xsk_umem_info *umem = txq->pair->umem; +int ret; + +while (1 ) { + ret =sendto (xsk_socket__fd (txq->pair->xsk),NULL ,0 , +MSG_DONTWAIT ,NULL ,0 ); + +/* everything is ok */ +if (ret >=0 ) +break ;
I would prefer:
while ((send(xsk_socket__fd(fd, NULL, 0, MSG_DONTWAIT) < 0) {Because:
- use while() to make looping clearer rather than while(1)
- use send() rather than sendto() because you aren't sending with addr
- you don't care about return value (ie.ret)
drivers/net/af_xdp/rte_eth_af_xdp.c line 224
Quoted 4 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +4
-0+ +/* some thing unexpected */ +if (errno !=EBUSY && errno !=EAGAIN ) +break ;
What about EINTR
drivers/net/af_xdp/rte_eth_af_xdp.c line 326
Quoted 8 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +8
-0+static void +eth_dev_info (struct rte_eth_dev *dev,struct rte_eth_dev_info *dev_info) +{ +struct pmd_internals *internals = dev->data->dev_private; + + dev_info->if_index = internals->if_index; + dev_info->max_mac_addrs =1 ; + dev_info->max_rx_pktlen = (uint32_t )ETH_FRAME_LEN ;
Cast here is unnecessary.
drivers/net/af_xdp/rte_eth_af_xdp.c line 329
Quoted 3 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +3
-0+ dev_info->max_rx_queues =1 ; + dev_info->max_tx_queues =1 ; + dev_info->min_rx_bufsize =0 ;
dev_info is already zero, don't need to fill other values.
Quoted 19 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +19
-0+ +static void +eth_dev_info (struct rte_eth_dev *dev,struct rte_eth_dev_info *dev_info) +{ +struct pmd_internals *internals = dev->data->dev_private; + + dev_info->if_index = internals->if_index; + dev_info->max_mac_addrs =1 ; + dev_info->max_rx_pktlen = (uint32_t )ETH_FRAME_LEN ; + dev_info->max_rx_queues =1 ; + dev_info->max_tx_queues =1 ; + dev_info->min_rx_bufsize =0 ; + + dev_info->default_rxportconf.nb_queues =1 ; + dev_info->default_txportconf.nb_queues =1 ; + dev_info->default_rxportconf.ring_size =ETH_AF_XDP_DFLT_NUM_DESCS ; + dev_info->default_txportconf.ring_size =ETH_AF_XDP_DFLT_NUM_DESCS ; +} +
drivers/net/af_xdp/rte_eth_af_xdp.c line 346
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+static int +eth_stats_get (struct rte_eth_dev *dev,struct rte_eth_stats *stats) +{ +struct pmd_internals *internals = dev->data->dev_private; +struct xdp_statistics xdp_stats; +struct pkt_rx_queue *rxq; +socklen_t optlen; +int i; + + optlen =sizeof (struct xdp_statistics );
In theory each call to getsockopt() could change or reduce the value of optlen. Best to initialize in loop before each call.
drivers/net/af_xdp/rte_eth_af_xdp.c line 359
Quoted 13 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +13
-0+for (i =0 ; i < dev->data->nb_rx_queues; i++) { + rxq = &internals->rx_queues[i]; + stats->q_ipackets[i] = internals->rx_queues[i].rx_pkts; + stats->q_ibytes[i] = internals->rx_queues[i].rx_bytes; + + stats->q_opackets[i] = internals->tx_queues[i].tx_pkts; + stats->q_obytes[i] = internals->tx_queues[i].tx_bytes; + + stats->ipackets += stats->q_ipackets[i]; + stats->ibytes += stats->q_ibytes[i]; + stats->imissed += internals->rx_queues[i].rx_dropped; +getsockopt (xsk_socket__fd (rxq->xsk),SOL_XDP ,XDP_STATISTICS , + &xdp_stats, &optlen);
You need to check return value of getsockopt() otherwise coverity will complain.
Quoted 4 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +4
-0+static void +eth_stats_reset (struct rte_eth_dev *dev) +{ +struct pmd_internals *internals = dev->data->dev_private;
drivers/net/af_xdp/rte_eth_af_xdp.c line 384
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+int i; + +for (i =0 ; i <ETH_AF_XDP_MAX_QUEUE_PAIRS ; i++) { + internals->rx_queues[i].rx_pkts =0 ; + internals->rx_queues[i].rx_bytes =0 ; + internals->rx_queues[i].rx_dropped =0 ; + + internals->tx_queues[i].tx_pkts =0 ; + internals->tx_queues[i].err_pkts =0 ; + internals->tx_queues[i].tx_bytes =0 ;
Put all the statistics together and use memset?
drivers/net/af_xdp/rte_eth_af_xdp.c line 455
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+static struct xsk_umem_info *xdp_umem_configure (void ) +{ +struct xsk_umem_info *umem; +struct xsk_umem_config usr_config = { + .fill_size =ETH_AF_XDP_DFLT_NUM_DESCS , + .comp_size =ETH_AF_XDP_DFLT_NUM_DESCS , + .frame_size =ETH_AF_XDP_FRAME_SIZE , + .frame_headroom =ETH_AF_XDP_DATA_HEADROOM }; +void *bufs =NULL ; +char ring_name[0x100 ];
0x100 is unconvential here. Instead use RTE_RING_NAMESIZE but variable is unnecessary, see below
drivers/net/af_xdp/rte_eth_af_xdp.c line 459
Quoted 4 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +4
-0+int ret; +uint64_t i; + + umem =calloc (1 ,sizeof (*umem));
Why not use rte_zmalloc_node to:
1. work with primary/secondary
2. guarantee memory on correct numa node?
drivers/net/af_xdp/rte_eth_af_xdp.c line 465
Quoted 6 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +6
-0+if (!umem) { +RTE_LOG (ERR ,AF_XDP ,"Failed to allocate umem info" ); +return NULL ; + } + +snprintf (ring_name,0x100 ,"af_xdp_ring" );
If ring is always the same, why copy it. Just use string literal
drivers/net/af_xdp/rte_eth_af_xdp.c line 733
Quoted 8 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +8
-0+/** parse name argument */ +static int +parse_name_arg(const char *key __rte_unused, +const char *value,void *extra_args) +{ +char *name = extra_args; + +if (strlen (value) >IFNAMSIZ ) {
Why not:
if (strnlen(value, IFNAMSIZ) >= IFNAMSIZ) {drivers/net/af_xdp/rte_eth_af_xdp.c line 799
Quoted 6 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +6
-0+ +static int +init_internals (struct rte_vdev_device *dev, +const char *if_name, +int queue_idx, +struct rte_eth_dev **eth_dev)
If you changed the function to return the new eth_dev (and return NULL on error) then you wouldn't have to pass eth_dev by reference.
static struct rte_eth_dev *allocate_ethdev(struct rte_vdev_device *dev, const char *if_name, uint16_t queue_idx) {
drivers/net/af_xdp/rte_eth_af_xdp.c line 803
Quoted 4 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +4
-0+{ +const char *name =rte_vdev_device_name (dev); +const unsigned int numa_node = dev->device.numa_node; +struct pmd_internals *internals =NULL ;
Useless initialization, first thing you do is allocate this.
Quoted 4 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +4
-0+int ret; +int i; + + internals =rte_zmalloc_socket (name,sizeof (*internals),0 , numa_node);
Re: [PATCH v2 1/6] net/af_xdp: introduce AF XDP PMD driver
Hi, Stephen
Quoted 2 lines · Stephen Hemminger
Lots of little review comments. This is what I saw in 30 minutes. Expect more.
Thanks for taking time to review my patch. They are all valuable inputs.
Quoted 9 lines
drivers/net/af_xdp/rte_eth_af_xdp.c line 153
Quoted 2 lines · On Tue, 19 Mar 2019 15:12:51 +0800 Xiaolong Ye <[email protected]> wrote:
drivers/net/af_xdp/rte_eth_af_xdp.c new file +2
-0+ nb_pkts = nb_pkts <ETH_AF_XDP_RX_BATCH_SIZE ? + nb_pkts :ETH_AF_XDP_RX_BATCH_SIZE ;Maybe use
RTE_MIN()?
Will do.
Quoted 7 lines
drivers/net/af_xdp/rte_eth_af_xdp.c line 169
Quoted 4 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +3
-0+ mbuf =rte_pktmbuf_alloc (rxq->mb_pool); +if (mbuf) { + memcpy(rte_pktmbuf_mtod(mbuf, void*), pkt, len);Space necessary in "void*"
Userte_memcpy.
Will do.
Quoted 14 lines
drivers/net/af_xdp/rte_eth_af_xdp.c line 199
Quoted 11 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+static void pull_umem_cq (struct xsk_umem_info *umem,int size) +{ +struct xsk_ring_cons *cq = &umem->cq; +int i, n; +uint32_t idx_cq; +uint64_t addr; + + n =xsk_ring_cons__peek (cq, size, &idx_cq); +if (n >0 ) { +for (i =0 ; i < n; i++) {You don't need the if (n > 0) since if n <= 0 the for loop would happen 0 times.
Will remove the redundant "if (n > 0)"
Quoted 32 lines
Quoted 4 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +3
-0+ +static void kick_tx (struct pkt_tx_queue *txq) +{drivers/net/af_xdp/rte_eth_af_xdp.c line 220
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+struct xsk_umem_info *umem = txq->pair->umem; +int ret; + +while (1 ) { + ret =sendto (xsk_socket__fd (txq->pair->xsk),NULL ,0 , +MSG_DONTWAIT ,NULL ,0 ); + +/* everything is ok */ +if (ret >=0 ) +break ;I would prefer:
while ((send(xsk_socket__fd(fd, NULL, 0, MSG_DONTWAIT) < 0) {Because:
- usewhile()to make looping clearer rather than while(1)
- usesend()rather thansendto()because you aren't sending with addr
- you don't care about return value (ie.ret)drivers/net/af_xdp/rte_eth_af_xdp.c line 224
Quoted 4 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +4
-0+ +/* some thing unexpected */ +if (errno !=EBUSY && errno !=EAGAIN ) +break ;What about EINTR
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c line 326
Quoted 8 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +8
-0+static void +eth_dev_info (struct rte_eth_dev *dev,struct rte_eth_dev_info *dev_info) +{ +struct pmd_internals *internals = dev->data->dev_private; + + dev_info->if_index = internals->if_index; + dev_info->max_mac_addrs =1 ; + dev_info->max_rx_pktlen = (uint32_t )ETH_FRAME_LEN ;Cast here is unnecessary.
Got it.
Quoted 6 lines
drivers/net/af_xdp/rte_eth_af_xdp.c line 329
Quoted 4 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +3
-0+ dev_info->max_rx_queues =1 ; + dev_info->max_tx_queues =1 ; + dev_info->min_rx_bufsize =0 ;
dev_infois already zero, don't need to fill other values.
Got it.
Quoted 25 lines
Quoted 25 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +24
-0+ +static void +eth_dev_info (struct rte_eth_dev *dev,struct rte_eth_dev_info *dev_info) +{ +struct pmd_internals *internals = dev->data->dev_private; + + dev_info->if_index = internals->if_index; + dev_info->max_mac_addrs =1 ; + dev_info->max_rx_pktlen = (uint32_t )ETH_FRAME_LEN ; + dev_info->max_rx_queues =1 ; + dev_info->max_tx_queues =1 ; + dev_info->min_rx_bufsize =0 ; + + dev_info->default_rxportconf.nb_queues =1 ; + dev_info->default_txportconf.nb_queues =1 ; + dev_info->default_rxportconf.ring_size =ETH_AF_XDP_DFLT_NUM_DESCS ; + dev_info->default_txportconf.ring_size =ETH_AF_XDP_DFLT_NUM_DESCS ; +} + +static int +eth_stats_get (struct rte_eth_dev *dev,struct rte_eth_stats *stats) +{ +struct pmd_internals *internals = dev->data->dev_private; +struct xdp_statistics xdp_stats;
Quoted 8 lines
drivers/net/af_xdp/rte_eth_af_xdp.c line 346
Quoted 5 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +5
-0+struct pkt_rx_queue *rxq; +socklen_t optlen; +int i; + + optlen =sizeof (struct xdp_statistics );In theory each call to
getsockopt()could change or reduce the value of optlen. Best to initialize in loop before each call.
Got it.
Quoted 16 lines
Quoted 4 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +3
-0+for (i =0 ; i < dev->data->nb_rx_queues; i++) { + rxq = &internals->rx_queues[i]; + stats->q_ipackets[i] = internals->rx_queues[i].rx_pkts;drivers/net/af_xdp/rte_eth_af_xdp.c line 359
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+ stats->q_ibytes[i] = internals->rx_queues[i].rx_bytes; + + stats->q_opackets[i] = internals->tx_queues[i].tx_pkts; + stats->q_obytes[i] = internals->tx_queues[i].tx_bytes; + + stats->ipackets += stats->q_ipackets[i]; + stats->ibytes += stats->q_ibytes[i]; + stats->imissed += internals->rx_queues[i].rx_dropped; +getsockopt (xsk_socket__fd (rxq->xsk),SOL_XDP ,XDP_STATISTICS , + &xdp_stats, &optlen);You need to check return value of
getsockopt()otherwise coverity will complain.
Got it.
Quoted 17 lines
Quoted 5 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +4
-0+static void +eth_stats_reset (struct rte_eth_dev *dev) +{ +struct pmd_internals *internals = dev->data->dev_private;drivers/net/af_xdp/rte_eth_af_xdp.c line 384
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+int i; + +for (i =0 ; i <ETH_AF_XDP_MAX_QUEUE_PAIRS ; i++) { + internals->rx_queues[i].rx_pkts =0 ; + internals->rx_queues[i].rx_bytes =0 ; + internals->rx_queues[i].rx_dropped =0 ; + + internals->tx_queues[i].tx_pkts =0 ; + internals->tx_queues[i].err_pkts =0 ; + internals->tx_queues[i].tx_bytes =0 ;Put all the statistics together and use memset?
Sounds good, will do.
Quoted 14 lines
drivers/net/af_xdp/rte_eth_af_xdp.c line 455
Quoted 11 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+static struct xsk_umem_info *xdp_umem_configure (void ) +{ +struct xsk_umem_info *umem; +struct xsk_umem_config usr_config = { + .fill_size =ETH_AF_XDP_DFLT_NUM_DESCS , + .comp_size =ETH_AF_XDP_DFLT_NUM_DESCS , + .frame_size =ETH_AF_XDP_FRAME_SIZE , + .frame_headroom =ETH_AF_XDP_DATA_HEADROOM }; +void *bufs =NULL ; +char ring_name[0x100 ];
0x100is unconvential here. Instead useRTE_RING_NAMESIZEbut variable is unnecessary, see below
Got it.
Quoted 9 lines
drivers/net/af_xdp/rte_eth_af_xdp.c line 459
Quoted 5 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +4
-0+int ret; +uint64_t i; + + umem =calloc (1 ,sizeof (*umem));Why not use
rte_zmalloc_nodeto:
1. work with primary/secondary
2. guarantee memory on correct numa node?
Will do.
Quoted 9 lines
drivers/net/af_xdp/rte_eth_af_xdp.c line 465
Quoted 7 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +6
-0+if (!umem) { +RTE_LOG (ERR ,AF_XDP ,"Failed to allocate umem info" ); +return NULL ; + } + +snprintf (ring_name,0x100 ,"af_xdp_ring" );If ring is always the same, why copy it. Just use string literal
Got it.
Quoted 11 lines
drivers/net/af_xdp/rte_eth_af_xdp.c line 733
Quoted 9 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +8
-0+/** parse name argument */ +static int +parse_name_arg(const char *key __rte_unused, +const char *value,void *extra_args) +{ +char *name = extra_args; + +if (strlen (value) >IFNAMSIZ ) {Why not:
Quoted 1 line
if (strnlen(value, IFNAMSIZ) >= IFNAMSIZ) {Will do.
Quoted 11 lines
drivers/net/af_xdp/rte_eth_af_xdp.c line 799
Quoted 7 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +6
-0+ +static int +init_internals (struct rte_vdev_device *dev, +const char *if_name, +int queue_idx, +struct rte_eth_dev **eth_dev)If you changed the function to return the new
eth_dev(and return NULL on error) then you wouldn't have to passeth_devby reference.
Quoted 2 lines
static struct
rte_eth_dev*allocate_ethdev(struct rte_vdev_device *dev, const char *if_name, uint16_t queue_idx)
Sounds better, will do.
Quoted 8 lines
{
drivers/net/af_xdp/rte_eth_af_xdp.c line 803
Quoted 4 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +4
-0+{ +const char *name =rte_vdev_device_name (dev); +const unsigned int numa_node = dev->device.numa_node; +struct pmd_internals *internals =NULL ;Useless initialization, first thing you do is allocate this.
Got it.
Thanks,
Xiaolong
Quoted 7 lines
Quoted 6 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +4
-0+int ret; +int i; + + internals =rte_zmalloc_socket (name,sizeof (*internals),0 , numa_node);
Re: [PATCH v2 1/6] net/af_xdp: introduce AF XDP PMD driver
In reply to Xiaolong YeQuoted 138 lines · On Tue, 19 Mar 2019 15:12:51 +0800 Xiaolong Ye <[email protected]> wrote:
Add a new PMD driver for
AF_XDPwhich is a proposed faster version ofAF_PACKETinterface in Linux. More info aboutAF_XDP, please refer to [1] [2].This is the vanilla version PMD which just uses a raw buffer registered as the umem.
[1] https://fosdem.org/2018/schedule/event/af_xdp/
[2] https://lwn.net/Articles/745934/Signed-off-by Xiaolong Ye <[email protected]>--- MAINTAINERS | 6 + config/common_base | 5 + config/common_linux | 1 + doc/guides/nics/af_xdp.rst | 45 + doc/guides/nics/features/af_xdp.ini | 11 + doc/guides/nics/index.rst | 1 + doc/guides/rel_notes/release_19_05.rst | 7 + drivers/net/Makefile | 1 + drivers/net/af_xdp/Makefile | 33 + drivers/net/af_xdp/meson.build | 21 + drivers/net/af_xdp/rte_eth_af_xdp.c | 930 ++++++++++++++++++ drivers/net/af_xdp/rte_pmd_af_xdp_version.map | 3 + drivers/net/meson.build | 1 + mk/rte.app.mk | 1 + 14 files changed, 1066 insertions(+) create mode 100644 doc/guides/nics/af_xdp.rst create mode 100644 doc/guides/nics/features/af_xdp.ini create mode 100644 drivers/net/af_xdp/Makefile create mode 100644 drivers/net/af_xdp/meson.build create mode 100644 drivers/net/af_xdp/rte_eth_af_xdp.c create mode 100644 drivers/net/af_xdp/rte_pmd_af_xdp_version.mapMAINTAINERS +6
-0@@ -468,6 +468,12 @@ M: John W. Linville <[email protected]> F: drivers/net/af_packet/ F: doc/guides/nics/features/afpacket.ini +Linux AF_XDP +M: Xiaolong Ye <[email protected]> +M: Qi Zhang <[email protected]> +F: drivers/net/af_xdp/ +F: doc/guides/nics/features/af_xdp.rst + Amazon ENA M: Marcin Wojtas <[email protected]> M: Michal Krawczyk <[email protected]>config/common_base +5
-0@@ -416,6 +416,11 @@ CONFIG_RTE_LIBRTE_VMXNET3_DEBUG_TX_FREE=n # CONFIG_RTE_LIBRTE_PMD_AF_PACKET=n +# +# Compile software PMD backed by AF_XDP sockets (Linux only) +# +CONFIG_RTE_LIBRTE_PMD_AF_XDP=n + # # Compile link bonding PMD library #config/common_linux +1
-0@@ -19,6 +19,7 @@ CONFIG_RTE_LIBRTE_VHOST_POSTCOPY=n CONFIG_RTE_LIBRTE_PMD_VHOST=y CONFIG_RTE_LIBRTE_IFC_PMD=y CONFIG_RTE_LIBRTE_PMD_AF_PACKET=y +CONFIG_RTE_LIBRTE_PMD_AF_XDP=y CONFIG_RTE_LIBRTE_PMD_SOFTNIC=y CONFIG_RTE_LIBRTE_PMD_TAP=y CONFIG_RTE_LIBRTE_AVP_PMD=ydoc/guides/nics/af_xdp.rst new file +45
-0@@ -0,0 +1,45 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2018 Intel Corporation. + +AF_XDP Poll Mode Driver +========================== + +AF_XDP is an address family that is optimized for high performance +packet processing. AF_XDP sockets enable the possibility for XDP program to +redirect packets to a memory buffer in userspace. + +For the full details behind AF_XDP socket, you can refer to +`AF_XDP documentation in the Kernel +<https://www.kernel.org/doc/Documentation/networking/af_xdp.rst>`_. + +This Linux-specific PMD driver creates the AF_XDP socket and binds it to a +specific netdev queue, it allows a DPDK application to send and receive raw +packets through the socket which would bypass the kernel network stack. +Current implementation only supports single queue, multi-queues feature will +be added later. + +Options +------- + +The following options can be provided to set up an af_xdp port in DPDK. + +* ``iface`` - name of the Kernel interface to attach to (required); +* ``queue`` - netdev queue id (optional, default 0); + +Prerequisites +------------- + +This is a Linux-specific PMD, thus the following prerequisites apply: + +* A Linux Kernel (version > 4.18) with XDP sockets configuration enabled; +* libbpf (within kernel version > 5.1) with latest af_xdp support installed +* A Kernel bound interface to attach to. + +Set up an af_xdp interface +----------------------------- + +The following example will set up an af_xdp interface in DPDK: + +.. code-block:: console + + --vdev eth_af_xdp,iface=ens786f1,queue=0doc/guides/nics/features/af_xdp.ini new file +2
-0@@ -0,0 +1,11 @@ +; +; Supported features of the 'af_xdp' network poll mode driver.
doc/guides/nics/features/af_xdp.ini line 11
Quoted 10 lines
doc/guides/nics/features/af_xdp.ini new file +9
-0+; +; Refer to default.ini for the full list of available PMD features. +; +[Features] +Link status = Y +MTU update = Y +Promiscuous mode = Y +Stats per queue = Y +x86-64 = Y
This is bad. Configure your editor to always put newline at end of file. In .emacs
(setq require-final-newline t)
Re: [PATCH v2 1/6] net/af_xdp: introduce AF XDP PMD driver
Quoted 156 lines · Stephen Hemminger
Quoted 138 lines · On Tue, 19 Mar 2019 15:12:51 +0800 Xiaolong Ye <[email protected]> wrote:
Add a new PMD driver for
AF_XDPwhich is a proposed faster version ofAF_PACKETinterface in Linux. More info aboutAF_XDP, please refer to [1] [2].This is the vanilla version PMD which just uses a raw buffer registered as the umem.
[1] https://fosdem.org/2018/schedule/event/af_xdp/
[2] https://lwn.net/Articles/745934/Signed-off-by Xiaolong Ye <[email protected]>--- MAINTAINERS | 6 + config/common_base | 5 + config/common_linux | 1 + doc/guides/nics/af_xdp.rst | 45 + doc/guides/nics/features/af_xdp.ini | 11 + doc/guides/nics/index.rst | 1 + doc/guides/rel_notes/release_19_05.rst | 7 + drivers/net/Makefile | 1 + drivers/net/af_xdp/Makefile | 33 + drivers/net/af_xdp/meson.build | 21 + drivers/net/af_xdp/rte_eth_af_xdp.c | 930 ++++++++++++++++++ drivers/net/af_xdp/rte_pmd_af_xdp_version.map | 3 + drivers/net/meson.build | 1 + mk/rte.app.mk | 1 + 14 files changed, 1066 insertions(+) create mode 100644 doc/guides/nics/af_xdp.rst create mode 100644 doc/guides/nics/features/af_xdp.ini create mode 100644 drivers/net/af_xdp/Makefile create mode 100644 drivers/net/af_xdp/meson.build create mode 100644 drivers/net/af_xdp/rte_eth_af_xdp.c create mode 100644 drivers/net/af_xdp/rte_pmd_af_xdp_version.mapMAINTAINERS +6
-0@@ -468,6 +468,12 @@ M: John W. Linville <[email protected]> F: drivers/net/af_packet/ F: doc/guides/nics/features/afpacket.ini +Linux AF_XDP +M: Xiaolong Ye <[email protected]> +M: Qi Zhang <[email protected]> +F: drivers/net/af_xdp/ +F: doc/guides/nics/features/af_xdp.rst + Amazon ENA M: Marcin Wojtas <[email protected]> M: Michal Krawczyk <[email protected]>config/common_base +5
-0@@ -416,6 +416,11 @@ CONFIG_RTE_LIBRTE_VMXNET3_DEBUG_TX_FREE=n # CONFIG_RTE_LIBRTE_PMD_AF_PACKET=n +# +# Compile software PMD backed by AF_XDP sockets (Linux only) +# +CONFIG_RTE_LIBRTE_PMD_AF_XDP=n + # # Compile link bonding PMD library #config/common_linux +1
-0@@ -19,6 +19,7 @@ CONFIG_RTE_LIBRTE_VHOST_POSTCOPY=n CONFIG_RTE_LIBRTE_PMD_VHOST=y CONFIG_RTE_LIBRTE_IFC_PMD=y CONFIG_RTE_LIBRTE_PMD_AF_PACKET=y +CONFIG_RTE_LIBRTE_PMD_AF_XDP=y CONFIG_RTE_LIBRTE_PMD_SOFTNIC=y CONFIG_RTE_LIBRTE_PMD_TAP=y CONFIG_RTE_LIBRTE_AVP_PMD=ydoc/guides/nics/af_xdp.rst new file +45
-0@@ -0,0 +1,45 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2018 Intel Corporation. + +AF_XDP Poll Mode Driver +========================== + +AF_XDP is an address family that is optimized for high performance +packet processing. AF_XDP sockets enable the possibility for XDP program to +redirect packets to a memory buffer in userspace. + +For the full details behind AF_XDP socket, you can refer to +`AF_XDP documentation in the Kernel +<https://www.kernel.org/doc/Documentation/networking/af_xdp.rst>`_. + +This Linux-specific PMD driver creates the AF_XDP socket and binds it to a +specific netdev queue, it allows a DPDK application to send and receive raw +packets through the socket which would bypass the kernel network stack. +Current implementation only supports single queue, multi-queues feature will +be added later. + +Options +------- + +The following options can be provided to set up an af_xdp port in DPDK. + +* ``iface`` - name of the Kernel interface to attach to (required); +* ``queue`` - netdev queue id (optional, default 0); + +Prerequisites +------------- + +This is a Linux-specific PMD, thus the following prerequisites apply: + +* A Linux Kernel (version > 4.18) with XDP sockets configuration enabled; +* libbpf (within kernel version > 5.1) with latest af_xdp support installed +* A Kernel bound interface to attach to. + +Set up an af_xdp interface +----------------------------- + +The following example will set up an af_xdp interface in DPDK: + +.. code-block:: console + + --vdev eth_af_xdp,iface=ens786f1,queue=0doc/guides/nics/features/af_xdp.ini new file +2
-0@@ -0,0 +1,11 @@ +; +; Supported features of the 'af_xdp' network poll mode driver.doc/guides/nics/features/af_xdp.ini line 11
Quoted 10 lines
doc/guides/nics/features/af_xdp.ini new file +9
-0+; +; Refer to default.ini for the full list of available PMD features. +; +[Features] +Link status = Y +MTU update = Y +Promiscuous mode = Y +Stats per queue = Y +x86-64 = YThis is bad. Configure your editor to always put newline at end of file. In .emacs
Quoted 2 lines
(setq require-final-newline t)
Or use Vim which needs no extra configuration to do this. :-)
Re: [PATCH v2 1/6] net/af_xdp: introduce AF XDP PMD driver
Quoted 6 lines · Bruce Richardson
Quoted 5 lines
This is bad. Configure your editor to always put newline at end of file. In .emacs
(setq require-final-newline t)
Or use Vim which needs no extra configuration to do this. :-)
I used to be a Vimer, just switch to emacs recently. :-)
Quoted 1 line
Re: [PATCH v2 1/6] net/af_xdp: introduce AF XDP PMD driver
In reply to Stephen HemmingerQuoted 8 lines · Stephen Hemminger
Quoted 1 line
\ No newline at end of file
This is bad. Configure your editor to always put newline at end of file. In .emacs
(setq require-final-newline t)
I wasn't aware of it before, thanks for the info, will configure it for my emacs.
Thanks,
Xiaolong
Re: [PATCH v2 1/6] net/af_xdp: introduce AF XDP PMD driver
In reply to Xiaolong YeQuoted 9 lines · Xiaolong Ye
doc/guides/nics/features/af_xdp.ini new file +1
-0@@ -0,0 +1,11 @@ +;
doc/guides/nics/features/af_xdp.ini line 11
Quoted 10 lines
doc/guides/nics/features/af_xdp.ini new file +10
-0+; Supported features of the 'af_xdp' network poll mode driver. +; +; Refer to default.ini for the full list of available PMD features. +; +[Features] +Link status = Y +MTU update = Y +Promiscuous mode = Y +Stats per queue = Y +x86-64 = Y
Is there really a limitation on x86?
drivers/net/Makefile +0 -0
Quoted 18 lines
index
502869a87..5d401b8c5100644drivers/net/Makefile +1
-0@@ -9,6 +9,7 @@ ifeq ($(CONFIG_RTE_LIBRTE_THUNDERX_NICVF_PMD),d) endif DIRS-$(CONFIG_RTE_LIBRTE_PMD_AF_PACKET) += af_packet +DIRS-$(CONFIG_RTE_LIBRTE_PMD_AF_XDP) += af_xdp DIRS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += ark DIRS-$(CONFIG_RTE_LIBRTE_ATLANTIC_PMD) += atlantic DIRS-$(CONFIG_RTE_LIBRTE_AVP_PMD) += avpdrivers/net/af_xdp/Makefile new file +0
-0
drivers/net/af_xdp/Makefile line 2
Quoted 2 lines
drivers/net/af_xdp/Makefile new file +2
-0@@ -0,0 +1,33 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Intel Corporation
2018? 2019?
+
Quoted 8 lines
drivers/net/af_xdp/Makefile new file +7
-0+include $(RTE_SDK)/mk/rte.vars.mk + +# +# library name +# +LIB = librte_pmd_af_xdp.a +
drivers/net/af_xdp/Makefile line 19
Quoted 10 lines
drivers/net/af_xdp/Makefile new file +10
-0+EXPORT_MAP := rte_pmd_af_xdp_version.map + +LIBABIVER := 1 + +CFLAGS += -O3 + +# require kernel version >= v5.1-rc1 +LINUX_VERSION := $(shell uname -r) +CFLAGS += -I/lib/modules/$(LINUX_VERSION)/build/tools/include +CFLAGS += -I/lib/modules/$(LINUX_VERSION)/build/tools/lib/bpf
We can reuse RTE_KERNELDIR here (even if the docs state that this was to build kmods so far).
Quoted 7 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +0
-0
[snip]
Quoted 24 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +22
-0+static int +eth_stats_get (struct rte_eth_dev *dev,struct rte_eth_stats *stats) +{ +struct pmd_internals *internals = dev->data->dev_private; +struct xdp_statistics xdp_stats; +struct pkt_rx_queue *rxq; +socklen_t optlen; +int i; + + optlen =sizeof (struct xdp_statistics ); +for (i =0 ; i < dev->data->nb_rx_queues; i++) { + rxq = &internals->rx_queues[i]; + stats->q_ipackets[i] = internals->rx_queues[i].rx_pkts; + stats->q_ibytes[i] = internals->rx_queues[i].rx_bytes; + + stats->q_opackets[i] = internals->tx_queues[i].tx_pkts; + stats->q_obytes[i] = internals->tx_queues[i].tx_bytes; + + stats->ipackets += stats->q_ipackets[i]; + stats->ibytes += stats->q_ibytes[i]; + stats->imissed += internals->rx_queues[i].rx_dropped; + getsockopt(xsk_socket__fd(rxq->xsk), SOL_XDP,
XDP_STATISTICS,
drivers/net/af_xdp/rte_eth_af_xdp.c
Quoted 5 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +5
-0+ &xdp_stats, &optlen); + stats->imissed += xdp_stats.rx_dropped; + + stats->opackets += stats->q_opackets[i]; + stats->oerrors += stats->q_errors[i];
You forgot to remove stats->q_errors[i];
+ stats->oerrors += internals->tx_queues[i].err_pkts;
Quoted 5 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +5
-0+ stats->obytes += stats->q_obytes[i]; + } + +return 0 ; +}
Signature · 1 line
David Marchand
Re: [PATCH v2 1/6] net/af_xdp: introduce AF XDP PMD driver
Thanks for your comments.
Quoted 15 lines · David Marchand
Quoted 13 lines · Xiaolong Ye
doc/guides/nics/features/af_xdp.ini new file +6
-0@@ -0,0 +1,11 @@ +; +; Supported features of the 'af_xdp' network poll mode driver. +; +; Refer to default.ini for the full list of available PMD features. +; +[Features]
Quoted 8 lines
doc/guides/nics/features/af_xdp.ini line 11
Quoted 6 lines
doc/guides/nics/features/af_xdp.ini new file +5
-0+Link status = Y +MTU update = Y +Promiscuous mode = Y +Stats per queue = Y +x86-64 = YIs there really a limitation on x86?
I think no, just don't have a chance to try it on a x86-32 machine.
Quoted 9 lines
drivers/net/Makefile +0
-0Quoted 6 lines
index
502869a87..5d401b8c5100644drivers/net/Makefile +0
-0@@ -9,6 +9,7 @@ ifeq ($(CONFIG_RTE_LIBRTE_THUNDERX_NICVF_PMD),d) endif
Quoted 16 lines
Quoted 12 lines
drivers/net/Makefile +1
-0DIRS-$(CONFIG_RTE_LIBRTE_PMD_AF_PACKET) += af_packet +DIRS-$(CONFIG_RTE_LIBRTE_PMD_AF_XDP) += af_xdp DIRS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += ark DIRS-$(CONFIG_RTE_LIBRTE_ATLANTIC_PMD) += atlantic DIRS-$(CONFIG_RTE_LIBRTE_AVP_PMD) += avpdrivers/net/af_xdp/Makefile new file +0
-0drivers/net/af_xdp/Makefile line 2
Quoted 2 lines
drivers/net/af_xdp/Makefile new file +2
-0@@ -0,0 +1,33 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Intel Corporation2018? 2019?
Will correct it to 2019.
Quoted 15 lines
+
Quoted 13 lines
drivers/net/af_xdp/Makefile new file +13
-0+include $(RTE_SDK)/mk/rte.vars.mk + +# +# library name +# +LIB = librte_pmd_af_xdp.a + +EXPORT_MAP := rte_pmd_af_xdp_version.map + +LIBABIVER := 1 + +CFLAGS += -O3 +
Quoted 8 lines
drivers/net/af_xdp/Makefile line 19
Quoted 5 lines
drivers/net/af_xdp/Makefile new file +4
-0+# require kernel version >= v5.1-rc1 +LINUX_VERSION := $(shell uname -r) +CFLAGS += -I/lib/modules/$(LINUX_VERSION)/build/tools/include +CFLAGS += -I/lib/modules/$(LINUX_VERSION)/build/tools/lib/bpfWe can reuse
RTE_KERNELDIRhere (even if the docs state that this was to build kmods so far).
Will do.
Quoted 45 lines
Quoted 9 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +0
-0[snip]
Quoted 24 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +22
-0+static int +eth_stats_get (struct rte_eth_dev *dev,struct rte_eth_stats *stats) +{ +struct pmd_internals *internals = dev->data->dev_private; +struct xdp_statistics xdp_stats; +struct pkt_rx_queue *rxq; +socklen_t optlen; +int i; + + optlen =sizeof (struct xdp_statistics ); +for (i =0 ; i < dev->data->nb_rx_queues; i++) { + rxq = &internals->rx_queues[i]; + stats->q_ipackets[i] = internals->rx_queues[i].rx_pkts; + stats->q_ibytes[i] = internals->rx_queues[i].rx_bytes; + + stats->q_opackets[i] = internals->tx_queues[i].tx_pkts; + stats->q_obytes[i] = internals->tx_queues[i].tx_bytes; + + stats->ipackets += stats->q_ipackets[i]; + stats->ibytes += stats->q_ibytes[i]; + stats->imissed += internals->rx_queues[i].rx_dropped; + getsockopt(xsk_socket__fd(rxq->xsk), SOL_XDP,
XDP_STATISTICS,drivers/net/af_xdp/rte_eth_af_xdp.c
Quoted 5 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +5
-0+ &xdp_stats, &optlen); + stats->imissed += xdp_stats.rx_dropped; + + stats->opackets += stats->q_opackets[i]; + stats->oerrors += stats->q_errors[i];You forgot to remove stats->q_errors[i];
Quoted 1 line
+
stats->oerrors+= internals->tx_queues[i].err_pkts;
My bad, will remove it in next version.
Thanks,
Xiaolong
Quoted 10 lines
Quoted 5 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +5
-0+ stats->obytes += stats->q_obytes[i]; + } + +return 0 ; +}Signature · 1 line
David Marchand
PATCH v2 2/6 lib/mbuf: introduce helper to create mempool with flags
In reply to Xiaolong YeThis allows applications to create mbuf mempool with specific flags such as MEMPOOL_F_NO_SPREAD if they want fixed size memory objects.
--- lib/librte_mbuf/rte_mbuf.c | 29 +++++++++++++++++++----- lib/librte_mbuf/rte_mbuf.h | 45 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 69 insertions(+), 5 deletions(-)
lib/librte_mbuf/rte_mbuf.c +24 -5
@@ -106,11 +106,10 @@ rte_pktmbuf_init(struct rte_mempool *mp, m->next =NULL ; } -/* Helper to create a mbuf pool with given mempool ops name*/ -struct rte_mempool * -rte_pktmbuf_pool_create_by_ops (const char *name,unsigned int n, +static struct rte_mempool * +rte_pktmbuf_pool_create_by_ops_with_flags (const char *name,unsigned int n,unsigned int cache_size,uint16_t priv_size,uint16_t data_room_size, -int socket_id,const char *ops_name) +unsigned int flags,int socket_id,const char *ops_name) {struct rte_mempool *mp;struct rte_pktmbuf_pool_private mbp_priv;
@@ -130,7 +129,7 @@ rte_pktmbuf_pool_create_by_ops(const char *name, unsigned int n, mbp_priv.mbuf_priv_size = priv_size; mp =rte_mempool_create_empty (name, n, elt_size, cache_size, -sizeof (struct rte_pktmbuf_pool_private ), socket_id,0 ); +sizeof (struct rte_pktmbuf_pool_private ), socket_id, flags);if (mp ==NULL )return NULL ;
@@ -157,6 +156,16 @@ rte_pktmbuf_pool_create_by_ops(const char *name, unsigned int n,return mp; } +/* Helper to create a mbuf pool with given mempool ops name*/ +struct rte_mempool * +rte_pktmbuf_pool_create_by_ops (const char *name,unsigned int n, +unsigned int cache_size,uint16_t priv_size,uint16_t data_room_size, +int socket_id,const char *ops_name) +{ +return rte_pktmbuf_pool_create_by_ops_with_flags (name, n, cache_size, + priv_size, data_room_size,0 , socket_id, ops_name); +} +/* helper to create a mbuf pool */ struct rte_mempool * rte_pktmbuf_pool_create(const char *name, unsigned int n,
@@ -167,6 +176,16 @@ rte_pktmbuf_pool_create(const char *name, unsigned int n, data_room_size, socket_id, NULL); } +/* helper to create a mbuf pool with flags (e.g. NO_SPREAD) */ +struct rte_mempool * __rte_experimental +rte_pktmbuf_pool_create_with_flags (const char *name,unsigned int n, +unsigned int cache_size,uint16_t priv_size,uint16_t data_room_size, +unsigned int flags,int socket_id) +{ +return rte_pktmbuf_pool_create_by_ops_with_flags (name, n, cache_size, + priv_size, data_room_size, flags, socket_id,NULL ); +} +/* do some sanity checks on a mbuf: panic if it fails */ void rte_mbuf_sanity_check (const struct rte_mbuf *m,int is_header)
lib/librte_mbuf/rte_mbuf.h +45 -0
51 lines not on this page · raw message
Signature · 1 line
2.17.1
PATCH v2 3/6 lib/mempool: allow page size aligned mempool
In reply to Xiaolong YeAllow create a mempool with page size aligned base address.
--- lib/librte_mempool/rte_mempool.c | 3 +++ lib/librte_mempool/rte_mempool.h | 1 + 2 files changed, 4 insertions(+)
lib/librte_mempool/rte_mempool.c +3 -0
@@ -543,6 +543,9 @@ rte_mempool_populate_default(struct rte_mempool *mp)if (try_contig) flags |=RTE_MEMZONE_IOVA_CONTIG ; +if (mp->flags &MEMPOOL_F_PAGE_ALIGN ) + align =getpagesize (); + mz =rte_memzone_reserve_aligned (mz_name, mem_size, mp->socket_id, flags, align);
lib/librte_mempool/rte_mempool.h +1 -0
@@ -264,6 +264,7 @@ struct rte_mempool {#define MEMPOOL_F_POOL_CREATED 0x0010/**< Internal: pool is created. */ #define MEMPOOL_F_NO_IOVA_CONTIG 0x0020/**< Don't need IOVA contiguous objs. */ #define MEMPOOL_F_NO_PHYS_CONTIG MEMPOOL_F_NO_IOVA_CONTIG/* deprecated */ +#define MEMPOOL_F_PAGE_ALIGN 0x0040/**< Chunk's base address is page aligned */ /** * @internal When debug is enabled, store some statistics.
Signature · 1 line
2.17.1
PATCH v2 4/6 net/af_xdp: use mbuf mempool for buffer management
In reply to Xiaolong YeNow, af_xdp registered memory buffer is managed by rte_mempool. mbuf be allocated from rte_mempool can be convert to xdp_desc's address and vice versa.
--- drivers/net/af_xdp/rte_eth_af_xdp.c | 127 +++++++++++++++++----------- 1 file changed, 78 insertions(+), 49 deletions(-)
drivers/net/af_xdp/rte_eth_af_xdp.c +78 -49
@@ -43,7 +43,11 @@#define ETH_AF_XDP_FRAME_SIZE XSK_UMEM__DEFAULT_FRAME_SIZE#define ETH_AF_XDP_NUM_BUFFERS 4096 -#define ETH_AF_XDP_DATA_HEADROOM 0 +/* mempool hdrobj size (64 bytes) + sizeof(struct rte_mbuf) (128 bytes) */ +#define ETH_AF_XDP_MBUF_OVERHEAD 192 +/* data start from offset 320 (192 + 128) bytes */ +#define ETH_AF_XDP_DATA_HEADROOM \ + (ETH_AF_XDP_MBUF_OVERHEAD + RTE_PKTMBUF_HEADROOM)#define ETH_AF_XDP_DFLT_NUM_DESCS XSK_RING_CONS__DEFAULT_NUM_DESCS#define ETH_AF_XDP_DFLT_QUEUE_IDX 0
@@ -56,7 +60,7 @@ struct xsk_umem_info {struct xsk_ring_prod fq;struct xsk_ring_cons cq;struct xsk_umem *umem; -struct rte_ring *buf_ring; +struct rte_mempool *mb_pool;void *buffer; };
@@ -110,12 +114,32 @@ static struct rte_eth_link pmd_link = { .link_autoneg = ETH_LINK_AUTONEG }; +static inline struct rte_mbuf * +addr_to_mbuf (struct xsk_umem_info *umem,uint64_t addr) +{ +uint64_t offset = (addr /ETH_AF_XDP_FRAME_SIZE * +ETH_AF_XDP_FRAME_SIZE ); +struct rte_mbuf *mbuf = (struct rte_mbuf *)((uint64_t )umem->buffer + + offset +ETH_AF_XDP_MBUF_OVERHEAD - +sizeof (struct rte_mbuf )); + mbuf->data_off = addr - offset -ETH_AF_XDP_MBUF_OVERHEAD ; +return mbuf; +} + +static inline uint64_t +mbuf_to_addr (struct xsk_umem_info *umem,struct rte_mbuf *mbuf) +{ +return (uint64_t )mbuf->buf_addr + mbuf->data_off - + (uint64_t )umem->buffer; +} +static inline int reserve_fill_queue (struct xsk_umem_info *umem,int reserve_size) {struct xsk_ring_prod *fq = &umem->fq; +struct rte_mbuf *mbuf;uint32_t idx; -void *addr =NULL ; +uint64_t addr;int i, ret =0 ; ret =xsk_ring_prod__reserve (fq, reserve_size, &idx);
@@ -125,11 +149,14 @@ reserve_fill_queue(struct xsk_umem_info *umem, int reserve_size) }for (i =0 ; i < reserve_size; i++) { -rte_ring_dequeue (umem->buf_ring, &addr); - *xsk_ring_prod__fill_addr (fq, idx++) = (uint64_t )addr; + mbuf =rte_pktmbuf_alloc (umem->mb_pool); +if (!mbuf) +break ; + addr =mbuf_to_addr (umem, mbuf); + *xsk_ring_prod__fill_addr (fq, idx++) = addr; } -xsk_ring_prod__submit (fq, reserve_size); +xsk_ring_prod__submit (fq, i);return 0 ; }
@@ -174,7 +201,7 @@ eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) }else { dropped++; } -rte_ring_enqueue (umem->buf_ring, (void *)addr); +rte_pktmbuf_free (addr_to_mbuf (umem, addr)); }xsk_ring_cons__release (rx, rcvd);
@@ -197,9 +224,8 @@ static void pull_umem_cq(struct xsk_umem_info *umem, int size) n =xsk_ring_cons__peek (cq, size, &idx_cq);if (n >0 ) {for (i =0 ; i < n; i++) { - addr = *xsk_ring_cons__comp_addr (cq, - idx_cq++); -rte_ring_enqueue (umem->buf_ring, (void *)addr); + addr = *xsk_ring_cons__comp_addr (cq, idx_cq++); +rte_pktmbuf_free (addr_to_mbuf (umem, addr)); }xsk_ring_cons__release (cq, n);
@@ -236,7 +262,7 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)struct pkt_tx_queue *txq = queue;struct xsk_umem_info *umem = txq->pair->umem;struct rte_mbuf *mbuf; -void *addrs[ETH_AF_XDP_TX_BATCH_SIZE ]; +struct rte_mbuf *mbuf_to_tx;unsigned long tx_bytes =0 ;int i, valid =0 ;uint32_t idx_tx;
@@ -246,10 +272,6 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)pull_umem_cq (umem, nb_pkts); - nb_pkts =rte_ring_dequeue_bulk (umem->buf_ring, addrs, - nb_pkts,NULL ); -if (!nb_pkts) -return 0 ;if (xsk_ring_prod__reserve (&txq->tx, nb_pkts, &idx_tx) != nb_pkts) {kick_tx (txq);
@@ -264,7 +286,12 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) desc =xsk_ring_prod__tx_desc (&txq->tx, idx_tx + i); mbuf = bufs[i];if (mbuf->pkt_len <= buf_len) { - desc->addr = (uint64_t )addrs[valid]; + mbuf_to_tx =rte_pktmbuf_alloc (umem->mb_pool); +if (!mbuf_to_tx) { +rte_pktmbuf_free (mbuf); +continue ; + } + desc->addr =mbuf_to_addr (umem, mbuf_to_tx); desc->len = mbuf->pkt_len; pkt =xsk_umem__get_data (umem->buffer, desc->addr);
@@ -280,10 +307,6 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)kick_tx (txq); -if (valid < nb_pkts) -rte_ring_enqueue_bulk (umem->buf_ring, &addrs[valid], - nb_pkts - valid,NULL ); - txq->err_pkts += nb_pkts - valid; txq->tx_pkts += valid; txq->tx_bytes += tx_bytes;
@@ -433,16 +456,29 @@ eth_link_update(struct rte_eth_dev *dev __rte_unused,static void xdp_umem_destroy (struct xsk_umem_info *umem) { -free (umem->buffer); - umem->buffer =NULL ; - -rte_ring_free (umem->buf_ring); - umem->buf_ring =NULL ; +rte_mempool_free (umem->mb_pool); + umem->mb_pool =NULL ;free (umem); umem =NULL ; } +static inline uint64_t get_base_addr (struct rte_mempool *mp) +{ +struct rte_mempool_memhdr *memhdr; + + memhdr =STAILQ_FIRST (&mp->mem_list); +return (uint64_t )(memhdr->addr); +} + +static inline uint64_t get_len (struct rte_mempool *mp) +{ +struct rte_mempool_memhdr *memhdr; + + memhdr =STAILQ_FIRST (&mp->mem_list); +return (uint64_t )(memhdr->len); +} +static struct xsk_umem_info *xdp_umem_configure (void ) {struct xsk_umem_info *umem;
@@ -451,10 +487,9 @@ static struct xsk_umem_info *xdp_umem_configure(void) .comp_size = ETH_AF_XDP_DFLT_NUM_DESCS, .frame_size = ETH_AF_XDP_FRAME_SIZE, .frame_headroom = ETH_AF_XDP_DATA_HEADROOM }; -void *bufs =NULL ; -char ring_name[0x100 ]; +void *base_addr =NULL ; +char pool_name[0x100 ];int ret; -uint64_t i; umem =calloc (1 ,sizeof (*umem));if (!umem) {
@@ -462,28 +497,23 @@ static struct xsk_umem_info *xdp_umem_configure(void)return NULL ; } -snprintf (ring_name,0x100 ,"af_xdp_ring" ); - umem->buf_ring =rte_ring_create (ring_name, -ETH_AF_XDP_NUM_BUFFERS , -SOCKET_ID_ANY , -0x0 ); -if (!umem->buf_ring) { +snprintf (pool_name,0x100 ,"af_xdp_ring" ); + umem->mb_pool =rte_pktmbuf_pool_create_with_flags (pool_name, +ETH_AF_XDP_NUM_BUFFERS , +250 ,0 , +ETH_AF_XDP_FRAME_SIZE - +ETH_AF_XDP_MBUF_OVERHEAD , +MEMPOOL_F_NO_SPREAD |MEMPOOL_F_PAGE_ALIGN , +SOCKET_ID_ANY ); + +if (!umem->mb_pool || umem->mb_pool->nb_mem_chunks !=1 ) {RTE_LOG (ERR ,AF_XDP , -"Failed to create rte_ring\n" ); +"Failed to create rte_mempool\n" ); goto err; } + base_addr = (void *)get_base_addr (umem->mb_pool); -for (i =0 ; i <ETH_AF_XDP_NUM_BUFFERS ; i++) -rte_ring_enqueue (umem->buf_ring, - (void *)(i *ETH_AF_XDP_FRAME_SIZE + -ETH_AF_XDP_DATA_HEADROOM )); - -if (posix_memalign (&bufs,getpagesize (), -ETH_AF_XDP_NUM_BUFFERS *ETH_AF_XDP_FRAME_SIZE )) { -RTE_LOG (ERR ,AF_XDP ,"Failed to allocate memory pool.\n" ); - goto err; - } - ret =xsk_umem__create (&umem->umem, bufs, + ret =xsk_umem__create (&umem->umem, base_addr,ETH_AF_XDP_NUM_BUFFERS *ETH_AF_XDP_FRAME_SIZE , &umem->fq, &umem->cq, &usr_config);
@@ -492,7 +522,7 @@ static struct xsk_umem_info *xdp_umem_configure(void)RTE_LOG (ERR ,AF_XDP ,"Failed to create umem" ); goto err; } - umem->buffer = bufs; + umem->buffer = base_addr;return umem;
@@ -909,8 +939,7 @@ rte_pmd_af_xdp_remove(struct rte_vdev_device *dev) internals = eth_dev->data->dev_private; -rte_ring_free (internals->umem->buf_ring); -rte_free (internals->umem->buffer); +rte_mempool_free (internals->umem->mb_pool);rte_free (internals->umem);rte_eth_dev_release_port (eth_dev);
Signature · 1 line
2.17.1
PATCH v2 5/6 net/af_xdp: enable zero copy
In reply to Xiaolong YeTry to check if external mempool (from rx_queue_setup) is fit for af_xdp, if it is, it will be registered to af_xdp socket directly and there will be no packet data copy on Rx and Tx.
--- drivers/net/af_xdp/rte_eth_af_xdp.c | 128 ++++++++++++++++++++-------- 1 file changed, 91 insertions(+), 37 deletions(-)
drivers/net/af_xdp/rte_eth_af_xdp.c +91 -37
@@ -62,6 +62,7 @@ struct xsk_umem_info {struct xsk_umem *umem;struct rte_mempool *mb_pool;void *buffer; +uint8_t zc; };struct pkt_rx_queue {
@@ -76,6 +77,7 @@ struct pkt_rx_queue {struct pkt_tx_queue *pair;uint16_t queue_idx; +uint8_t zc; };struct pkt_tx_queue {
@@ -191,17 +193,24 @@ eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)uint32_t len =xsk_ring_cons__rx_desc (rx, idx_rx++)->len;char *pkt =xsk_umem__get_data (rxq->umem->buffer, addr); - mbuf =rte_pktmbuf_alloc (rxq->mb_pool); -if (mbuf) { - memcpy(rte_pktmbuf_mtod(mbuf, void*), pkt, len); +if (rxq->zc) { + mbuf =addr_to_mbuf (rxq->umem, addr);rte_pktmbuf_pkt_len (mbuf) =rte_pktmbuf_data_len (mbuf) = len; - rx_bytes += len; bufs[count++] = mbuf; }else { - dropped++; + mbuf =rte_pktmbuf_alloc (rxq->mb_pool); +if (mbuf) { + memcpy(rte_pktmbuf_mtod(mbuf, void*), pkt, len); +rte_pktmbuf_pkt_len (mbuf) = +rte_pktmbuf_data_len (mbuf) = len; + rx_bytes += len; + bufs[count++] = mbuf; + }else { + dropped++; + } +rte_pktmbuf_free (addr_to_mbuf (umem, addr)); } -rte_pktmbuf_free (addr_to_mbuf (umem, addr)); }xsk_ring_cons__release (rx, rcvd);
@@ -285,22 +294,29 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) -ETH_AF_XDP_DATA_HEADROOM ; desc =xsk_ring_prod__tx_desc (&txq->tx, idx_tx + i); mbuf = bufs[i]; -if (mbuf->pkt_len <= buf_len) { - mbuf_to_tx =rte_pktmbuf_alloc (umem->mb_pool); -if (!mbuf_to_tx) { -rte_pktmbuf_free (mbuf); -continue ; - } - desc->addr =mbuf_to_addr (umem, mbuf_to_tx); +if (txq->pair->zc && mbuf->pool == umem->mb_pool) { + desc->addr =mbuf_to_addr (umem, mbuf); desc->len = mbuf->pkt_len; - pkt =xsk_umem__get_data (umem->buffer, - desc->addr); - memcpy(pkt, rte_pktmbuf_mtod(mbuf, void *), - desc->len); valid++; tx_bytes += mbuf->pkt_len; + }else { +if (mbuf->pkt_len <= buf_len) { + mbuf_to_tx =rte_pktmbuf_alloc (umem->mb_pool); +if (!mbuf_to_tx) { +rte_pktmbuf_free (mbuf); +continue ; + } + desc->addr =mbuf_to_addr (umem, mbuf_to_tx); + desc->len = mbuf->pkt_len; + pkt =xsk_umem__get_data (umem->buffer, + desc->addr); + memcpy(pkt, rte_pktmbuf_mtod(mbuf, void *), + desc->len); + valid++; + tx_bytes += mbuf->pkt_len; + } +rte_pktmbuf_free (mbuf); } -rte_pktmbuf_free (mbuf); }xsk_ring_prod__submit (&txq->tx, nb_pkts);
@@ -479,7 +495,7 @@ static inline uint64_t get_len(struct rte_mempool *mp)return (uint64_t )(memhdr->len); } -static struct xsk_umem_info *xdp_umem_configure (void ) +static struct xsk_umem_info *xdp_umem_configure (struct rte_mempool *mb_pool) {struct xsk_umem_info *umem;struct xsk_umem_config usr_config = {
@@ -497,20 +513,25 @@ static struct xsk_umem_info *xdp_umem_configure(void)return NULL ; } -snprintf (pool_name,0x100 ,"af_xdp_ring" ); - umem->mb_pool =rte_pktmbuf_pool_create_with_flags (pool_name, -ETH_AF_XDP_NUM_BUFFERS , -250 ,0 , -ETH_AF_XDP_FRAME_SIZE - -ETH_AF_XDP_MBUF_OVERHEAD , -MEMPOOL_F_NO_SPREAD |MEMPOOL_F_PAGE_ALIGN , -SOCKET_ID_ANY ); - -if (!umem->mb_pool || umem->mb_pool->nb_mem_chunks !=1 ) { -RTE_LOG (ERR ,AF_XDP , -"Failed to create rte_mempool\n" ); - goto err; +if (!mb_pool) { +snprintf (pool_name,0x100 ,"af_xdp_ring" ); + umem->mb_pool =rte_pktmbuf_pool_create_with_flags (pool_name, +ETH_AF_XDP_NUM_BUFFERS , +250 ,0 , +ETH_AF_XDP_FRAME_SIZE - +ETH_AF_XDP_MBUF_OVERHEAD , +MEMPOOL_F_NO_SPREAD |MEMPOOL_F_PAGE_ALIGN , +SOCKET_ID_ANY ); + +if (!umem->mb_pool || umem->mb_pool->nb_mem_chunks !=1 ) { +RTE_LOG (ERR ,AF_XDP ,"Failed to create rte_mempool\n" ); + goto err; + } + }else { + umem->mb_pool = mb_pool; + umem->zc =1 ; } + base_addr = (void *)get_base_addr (umem->mb_pool); ret = xsk_umem__create(&umem->umem, base_addr,
@@ -531,16 +552,43 @@ static struct xsk_umem_info *xdp_umem_configure(void)return NULL ; } +static uint8_t +check_mempool_zc (struct rte_mempool *mp) +{ +RTE_ASSERT (mp); + +/* must continues */ +if (mp->nb_mem_chunks >1 ) +return 0 ; + +/* check header size */ +if (mp->header_size !=RTE_CACHE_LINE_SIZE ) +return 0 ; + +/* check base address */ +if ((uint64_t )get_base_addr (mp) %getpagesize () !=0 ) +return 0 ; + +/* check chunk size */ +if ((mp->elt_size + mp->header_size + mp->trailer_size) % +ETH_AF_XDP_FRAME_SIZE !=0 ) +return 0 ; + +return 1 ; +} +static int xsk_configure (struct pmd_internals *internals,struct pkt_rx_queue *rxq, -int ring_size) +int ring_size,struct rte_mempool *mb_pool) {struct xsk_socket_config cfg;struct pkt_tx_queue *txq = rxq->pair; +struct rte_mempool *mp;int ret =0 ;int reserve_size; - rxq->umem =xdp_umem_configure (); + mp =check_mempool_zc (mb_pool) ? mb_pool :NULL ; + rxq->umem =xdp_umem_configure (mp);if (!rxq->umem) { ret = -ENOMEM ; goto err;
@@ -627,15 +675,21 @@ eth_rx_queue_setup(struct rte_eth_dev *dev, rxq->mb_pool = mb_pool; -if (xsk_configure (internals, rxq, nb_rx_desc)) { -RTE_LOG (ERR ,AF_XDP , -"Failed to configure xdp socket\n" ); +if (xsk_configure (internals, rxq, nb_rx_desc, mb_pool)) { +RTE_LOG (ERR ,AF_XDP ,"Failed to configure xdp socket\n" ); ret = -EINVAL ; goto err; } internals->umem = rxq->umem; +if (mb_pool == internals->umem->mb_pool) + rxq->zc = internals->umem->zc; + +if (rxq->zc) +RTE_LOG (INFO ,AF_XDP , +"zero copy enabled on rx queue %d\n" , rx_queue_id); + dev->data->rx_queues[rx_queue_id] = rxq;return 0 ;
Signature · 1 line
2.17.1
Re: [PATCH v2 5/6] net/af_xdp: enable zero copy
Quoted 36 lines · Xiaolong Ye
Try to check if external mempool (from
rx_queue_setup) is fit foraf_xdp, if it is, it will be registered toaf_xdpsocket directly and there will be no packet data copy on Rx and Tx.Signed-off-by Xiaolong Ye <[email protected]>--- drivers/net/af_xdp/rte_eth_af_xdp.c | 128 ++++++++++++++++++++-------- 1 file changed, 91 insertions(+), 37 deletions(-)drivers/net/af_xdp/rte_eth_af_xdp.c +2
-3@@ -62,6 +62,7 @@ struct xsk_umem_info {struct xsk_umem *umem;struct rte_mempool *mb_pool;void *buffer; +uint8_t zc; };struct pkt_rx_queue {@@ -76,6 +77,7 @@ struct pkt_rx_queue {struct pkt_tx_queue *pair;uint16_t queue_idx; +uint8_t zc; };struct pkt_tx_queue {@@ -191,17 +193,24 @@ eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)uint32_t len =xsk_ring_cons__rx_desc (rx, idx_rx++)->len;char *pkt =xsk_umem__get_data (rxq->umem->buffer, addr); - mbuf =rte_pktmbuf_alloc (rxq->mb_pool); -if (mbuf) { - memcpy(rte_pktmbuf_mtod(mbuf, void*), pkt, len);
drivers/net/af_xdp/rte_eth_af_xdp.c eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) · line 203
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c +4
-2+if (rxq->zc) { + mbuf =addr_to_mbuf (rxq->umem, addr);rte_pktmbuf_pkt_len (mbuf) =rte_pktmbuf_data_len (mbuf) = len; - rx_bytes += len; bufs[count++] = mbuf; }else { - dropped++; + mbuf =rte_pktmbuf_alloc (rxq->mb_pool); +if (mbuf) {
if (likely(mbuf != NULL))
drivers/net/af_xdp/rte_eth_af_xdp.c eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) · line 204
Quoted 1 line
drivers/net/af_xdp/rte_eth_af_xdp.c +1
-0+ memcpy(rte_pktmbuf_mtod(mbuf, void*), pkt, len);
Use rte_memcpy.
Quoted 27 lines
drivers/net/af_xdp/rte_eth_af_xdp.c +10
-8+rte_pktmbuf_pkt_len (mbuf) = +rte_pktmbuf_data_len (mbuf) = len; + rx_bytes += len; + bufs[count++] = mbuf; + }else { + dropped++; + } +rte_pktmbuf_free (addr_to_mbuf (umem, addr)); } -rte_pktmbuf_free (addr_to_mbuf (umem, addr)); }xsk_ring_cons__release (rx, rcvd);@@ -285,22 +294,29 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) -ETH_AF_XDP_DATA_HEADROOM ; desc =xsk_ring_prod__tx_desc (&txq->tx, idx_tx + i); mbuf = bufs[i]; -if (mbuf->pkt_len <= buf_len) { - mbuf_to_tx =rte_pktmbuf_alloc (umem->mb_pool); -if (!mbuf_to_tx) { -rte_pktmbuf_free (mbuf); -continue ; - } - desc->addr =mbuf_to_addr (umem, mbuf_to_tx); +if (txq->pair->zc && mbuf->pool == umem->mb_pool) { + desc->addr =mbuf_to_addr (umem, mbuf); desc->len = mbuf->pkt_len;
drivers/net/af_xdp/rte_eth_af_xdp.c eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) · line 305
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c +4
-4- pkt =xsk_umem__get_data (umem->buffer, - desc->addr); - memcpy(pkt, rte_pktmbuf_mtod(mbuf, void *), - desc->len); valid++; tx_bytes += mbuf->pkt_len; + }else { +if (mbuf->pkt_len <= buf_len) { + mbuf_to_tx =rte_pktmbuf_alloc (umem->mb_pool); +if (!mbuf_to_tx) {
if (unlikely(mbuf_to_tx == NULL))
See DPDK coding conventions 1.8.1 for how to do pointer comparisons.
drivers/net/af_xdp/rte_eth_af_xdp.c eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) · line 314
Quoted 9 lines
drivers/net/af_xdp/rte_eth_af_xdp.c +9
-0+rte_pktmbuf_free (mbuf); +continue ; + } + desc->addr =mbuf_to_addr (umem, mbuf_to_tx); + desc->len = mbuf->pkt_len; + pkt =xsk_umem__get_data (umem->buffer, + desc->addr); + memcpy(pkt, rte_pktmbuf_mtod(mbuf, void *), + desc->len);
rte_memcpy()
Quoted 26 lines
drivers/net/af_xdp/rte_eth_af_xdp.c +5
-6+ valid++; + tx_bytes += mbuf->pkt_len; + } +rte_pktmbuf_free (mbuf); } -rte_pktmbuf_free (mbuf); }xsk_ring_prod__submit (&txq->tx, nb_pkts);@@ -479,7 +495,7 @@ static inline uint64_t get_len(struct rte_mempool *mp)return (uint64_t )(memhdr->len); } -static struct xsk_umem_info *xdp_umem_configure (void ) +static struct xsk_umem_info *xdp_umem_configure (struct rte_mempool *mb_pool) {struct xsk_umem_info *umem;struct xsk_umem_config usr_config = {@@ -497,20 +513,25 @@ static struct xsk_umem_info *xdp_umem_configure(void)return NULL ; } -snprintf (pool_name,0x100 ,"af_xdp_ring" ); - umem->mb_pool = rte_pktmbuf_pool_create_with_flags(pool_name, - ETH_AF_XDP_NUM_BUFFERS, - 250, 0,
drivers/net/af_xdp/rte_eth_af_xdp.c static struct xsk_umem_info *xdp_umem_configure(void) · line 516
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c +1
-9-ETH_AF_XDP_FRAME_SIZE - -ETH_AF_XDP_MBUF_OVERHEAD , -MEMPOOL_F_NO_SPREAD |MEMPOOL_F_PAGE_ALIGN , - SOCKET_ID_ANY); - -if (!umem->mb_pool || umem->mb_pool->nb_mem_chunks !=1 ) { -RTE_LOG (ERR ,AF_XDP , -"Failed to create rte_mempool\n" ); - goto err; +if (!mb_pool) {
1.8.1
drivers/net/af_xdp/rte_eth_af_xdp.c static struct xsk_umem_info *xdp_umem_configure(void) · line 517
Quoted 1 line
drivers/net/af_xdp/rte_eth_af_xdp.c +1
-0+snprintf (pool_name,0x100 ,"af_xdp_ring" );
0x100??
drivers/net/af_xdp/rte_eth_af_xdp.c static struct xsk_umem_info *xdp_umem_configure(void) · line 526
Quoted 9 lines
drivers/net/af_xdp/rte_eth_af_xdp.c +9
-0+ umem->mb_pool =rte_pktmbuf_pool_create_with_flags (pool_name, +ETH_AF_XDP_NUM_BUFFERS , +250 ,0 , +ETH_AF_XDP_FRAME_SIZE - +ETH_AF_XDP_MBUF_OVERHEAD , +MEMPOOL_F_NO_SPREAD |MEMPOOL_F_PAGE_ALIGN , +SOCKET_ID_ANY ); + +if (!umem->mb_pool || umem->mb_pool->nb_mem_chunks !=1 ) {
1.8.1
Quoted 20 lines
drivers/net/af_xdp/rte_eth_af_xdp.c +12
-0+RTE_LOG (ERR ,AF_XDP ,"Failed to create rte_mempool\n" ); + goto err; + } + }else { + umem->mb_pool = mb_pool; + umem->zc =1 ; } + base_addr = (void *)get_base_addr (umem->mb_pool); ret = xsk_umem__create(&umem->umem, base_addr,@@ -531,16 +552,43 @@ static struct xsk_umem_info *xdp_umem_configure(void)return NULL ; } +static uint8_t +check_mempool_zc (struct rte_mempool *mp) +{ +RTE_ASSERT (mp); +
drivers/net/af_xdp/rte_eth_af_xdp.c static struct xsk_umem_info *xdp_umem_configure(void) · line 569
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c +10
-0+/* must continues */ +if (mp->nb_mem_chunks >1 ) +return 0 ; + +/* check header size */ +if (mp->header_size !=RTE_CACHE_LINE_SIZE ) +return 0 ; + +/* check base address */ +if ((uint64_t )get_base_addr (mp) %getpagesize () !=0 )
This should be an uintptr_t cast.
Quoted 53 lines
drivers/net/af_xdp/rte_eth_af_xdp.c +23
-5+return 0 ; + +/* check chunk size */ +if ((mp->elt_size + mp->header_size + mp->trailer_size) % +ETH_AF_XDP_FRAME_SIZE !=0 ) +return 0 ; + +return 1 ; +} +static int xsk_configure (struct pmd_internals *internals,struct pkt_rx_queue *rxq, -int ring_size) +int ring_size,struct rte_mempool *mb_pool) {struct xsk_socket_config cfg;struct pkt_tx_queue *txq = rxq->pair; +struct rte_mempool *mp;int ret =0 ;int reserve_size; - rxq->umem =xdp_umem_configure (); + mp =check_mempool_zc (mb_pool) ? mb_pool :NULL ; + rxq->umem =xdp_umem_configure (mp);if (!rxq->umem) { ret = -ENOMEM ; goto err;@@ -627,15 +675,21 @@ eth_rx_queue_setup(struct rte_eth_dev *dev, rxq->mb_pool = mb_pool; -if (xsk_configure (internals, rxq, nb_rx_desc)) { -RTE_LOG (ERR ,AF_XDP , -"Failed to configure xdp socket\n" ); +if (xsk_configure (internals, rxq, nb_rx_desc, mb_pool)) { +RTE_LOG (ERR ,AF_XDP ,"Failed to configure xdp socket\n" ); ret = -EINVAL ; goto err; } internals->umem = rxq->umem; +if (mb_pool == internals->umem->mb_pool) + rxq->zc = internals->umem->zc; + +if (rxq->zc) +RTE_LOG (INFO ,AF_XDP , +"zero copy enabled on rx queue %d\n" , rx_queue_id); + dev->data->rx_queues[rx_queue_id] = rxq;return 0 ;
Re: [PATCH v2 5/6] net/af_xdp: enable zero copy
Hi, Mattias
Thanks for the review.
Quoted 37 lines · Mattias Rönnblom
Quoted 36 lines · Xiaolong Ye
Try to check if external mempool (from
rx_queue_setup) is fit foraf_xdp, if it is, it will be registered toaf_xdpsocket directly and there will be no packet data copy on Rx and Tx.Signed-off-by Xiaolong Ye <[email protected]>--- drivers/net/af_xdp/rte_eth_af_xdp.c | 128 ++++++++++++++++++++-------- 1 file changed, 91 insertions(+), 37 deletions(-)drivers/net/af_xdp/rte_eth_af_xdp.c +4
-3@@ -62,6 +62,7 @@ struct xsk_umem_info {struct xsk_umem *umem;struct rte_mempool *mb_pool;void *buffer; +uint8_t zc; };struct pkt_rx_queue {@@ -76,6 +77,7 @@ struct pkt_rx_queue {struct pkt_tx_queue *pair;uint16_t queue_idx; +uint8_t zc; };struct pkt_tx_queue {@@ -191,17 +193,24 @@ eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)uint32_t len =xsk_ring_cons__rx_desc (rx, idx_rx++)->len;char *pkt =xsk_umem__get_data (rxq->umem->buffer, addr); - mbuf =rte_pktmbuf_alloc (rxq->mb_pool); -if (mbuf) { - memcpy(rte_pktmbuf_mtod(mbuf, void*), pkt, len); +if (rxq->zc) { + mbuf =addr_to_mbuf (rxq->umem, addr);rte_pktmbuf_pkt_len (mbuf) =rte_pktmbuf_data_len (mbuf) = len;
Quoted 8 lines
drivers/net/af_xdp/rte_eth_af_xdp.c eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) · line 203
Quoted 6 lines
drivers/net/af_xdp/rte_eth_af_xdp.c +2
-2- rx_bytes += len; bufs[count++] = mbuf; }else { - dropped++; + mbuf =rte_pktmbuf_alloc (rxq->mb_pool); +if (mbuf) {if (likely(mbuf != NULL))
Got it.
Quoted 4 lines
drivers/net/af_xdp/rte_eth_af_xdp.c eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) · line 204
Quoted 2 lines
drivers/net/af_xdp/rte_eth_af_xdp.c +1
-0+ memcpy(rte_pktmbuf_mtod(mbuf, void*), pkt, len);Use
rte_memcpy.
Got it.
Quoted 40 lines
Quoted 27 lines
drivers/net/af_xdp/rte_eth_af_xdp.c +10
-8+rte_pktmbuf_pkt_len (mbuf) = +rte_pktmbuf_data_len (mbuf) = len; + rx_bytes += len; + bufs[count++] = mbuf; + }else { + dropped++; + } +rte_pktmbuf_free (addr_to_mbuf (umem, addr)); } -rte_pktmbuf_free (addr_to_mbuf (umem, addr)); }xsk_ring_cons__release (rx, rcvd);@@ -285,22 +294,29 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) -ETH_AF_XDP_DATA_HEADROOM ; desc =xsk_ring_prod__tx_desc (&txq->tx, idx_tx + i); mbuf = bufs[i]; -if (mbuf->pkt_len <= buf_len) { - mbuf_to_tx =rte_pktmbuf_alloc (umem->mb_pool); -if (!mbuf_to_tx) { -rte_pktmbuf_free (mbuf); -continue ; - } - desc->addr =mbuf_to_addr (umem, mbuf_to_tx); +if (txq->pair->zc && mbuf->pool == umem->mb_pool) { + desc->addr =mbuf_to_addr (umem, mbuf); desc->len = mbuf->pkt_len;drivers/net/af_xdp/rte_eth_af_xdp.c eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) · line 305
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c +4
-4- pkt =xsk_umem__get_data (umem->buffer, - desc->addr); - memcpy(pkt, rte_pktmbuf_mtod(mbuf, void *), - desc->len); valid++; tx_bytes += mbuf->pkt_len; + }else { +if (mbuf->pkt_len <= buf_len) { + mbuf_to_tx =rte_pktmbuf_alloc (umem->mb_pool); +if (!mbuf_to_tx) {if (
unlikely(mbuf_to_tx == NULL))
Quoted 1 line
See DPDK coding conventions 1.8.1 for how to do pointer comparisons.
I'll check it and do it correctly in next version.
Quoted 12 lines
drivers/net/af_xdp/rte_eth_af_xdp.c eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) · line 314
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c +9
-0+rte_pktmbuf_free (mbuf); +continue ; + } + desc->addr =mbuf_to_addr (umem, mbuf_to_tx); + desc->len = mbuf->pkt_len; + pkt =xsk_umem__get_data (umem->buffer, + desc->addr); + memcpy(pkt, rte_pktmbuf_mtod(mbuf, void *), + desc->len);
rte_memcpy()
Got it.
Quoted 28 lines
Quoted 28 lines
drivers/net/af_xdp/rte_eth_af_xdp.c +5
-10+ valid++; + tx_bytes += mbuf->pkt_len; + } +rte_pktmbuf_free (mbuf); } -rte_pktmbuf_free (mbuf); }xsk_ring_prod__submit (&txq->tx, nb_pkts);@@ -479,7 +495,7 @@ static inline uint64_t get_len(struct rte_mempool *mp)return (uint64_t )(memhdr->len); } -static struct xsk_umem_info *xdp_umem_configure (void ) +static struct xsk_umem_info *xdp_umem_configure (struct rte_mempool *mb_pool) {struct xsk_umem_info *umem;struct xsk_umem_config usr_config = {@@ -497,20 +513,25 @@ static struct xsk_umem_info *xdp_umem_configure(void)return NULL ; } -snprintf (pool_name,0x100 ,"af_xdp_ring" ); - umem->mb_pool =rte_pktmbuf_pool_create_with_flags (pool_name, -ETH_AF_XDP_NUM_BUFFERS , -250 ,0 , -ETH_AF_XDP_FRAME_SIZE - -ETH_AF_XDP_MBUF_OVERHEAD , -MEMPOOL_F_NO_SPREAD |MEMPOOL_F_PAGE_ALIGN , -SOCKET_ID_ANY );
Quoted 8 lines
drivers/net/af_xdp/rte_eth_af_xdp.c static struct xsk_umem_info *xdp_umem_configure(void) · line 516
Quoted 6 lines
drivers/net/af_xdp/rte_eth_af_xdp.c +1
-5- -if (!umem->mb_pool || umem->mb_pool->nb_mem_chunks !=1 ) { -RTE_LOG (ERR ,AF_XDP , -"Failed to create rte_mempool\n" ); - goto err; +if (!mb_pool) {1.8.1
Got it.
Quoted 4 lines
drivers/net/af_xdp/rte_eth_af_xdp.c static struct xsk_umem_info *xdp_umem_configure(void) · line 517
Quoted 2 lines
drivers/net/af_xdp/rte_eth_af_xdp.c +1
-0+snprintf (pool_name,0x100 ,"af_xdp_ring" );
0x100??
Will use RTE_MEMPOOL_NAMESIZE instead.
Quoted 12 lines
drivers/net/af_xdp/rte_eth_af_xdp.c static struct xsk_umem_info *xdp_umem_configure(void) · line 526
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c +9
-0+ umem->mb_pool =rte_pktmbuf_pool_create_with_flags (pool_name, +ETH_AF_XDP_NUM_BUFFERS , +250 ,0 , +ETH_AF_XDP_FRAME_SIZE - +ETH_AF_XDP_MBUF_OVERHEAD , +MEMPOOL_F_NO_SPREAD |MEMPOOL_F_PAGE_ALIGN , +SOCKET_ID_ANY ); + +if (!umem->mb_pool || umem->mb_pool->nb_mem_chunks !=1 ) {1.8.1
Got it.
Quoted 23 lines
Quoted 23 lines
drivers/net/af_xdp/rte_eth_af_xdp.c +16
-0+RTE_LOG (ERR ,AF_XDP ,"Failed to create rte_mempool\n" ); + goto err; + } + }else { + umem->mb_pool = mb_pool; + umem->zc =1 ; } + base_addr = (void *)get_base_addr (umem->mb_pool); ret = xsk_umem__create(&umem->umem, base_addr,@@ -531,16 +552,43 @@ static struct xsk_umem_info *xdp_umem_configure(void)return NULL ; } +static uint8_t +check_mempool_zc (struct rte_mempool *mp) +{ +RTE_ASSERT (mp); + +/* must continues */ +if (mp->nb_mem_chunks >1 ) +return 0 ; +
Quoted 8 lines
drivers/net/af_xdp/rte_eth_af_xdp.c static struct xsk_umem_info *xdp_umem_configure(void) · line 569
Quoted 6 lines
drivers/net/af_xdp/rte_eth_af_xdp.c +6
-0+/* check header size */ +if (mp->header_size !=RTE_CACHE_LINE_SIZE ) +return 0 ; + +/* check base address */ +if ((uint64_t )get_base_addr (mp) %getpagesize () !=0 )This should be an
uintptr_tcast.
Got it.
Thanks,
Xiaolong
Quoted 48 lines
Quoted 48 lines
drivers/net/af_xdp/rte_eth_af_xdp.c +23
-5+return 0 ; + +/* check chunk size */ +if ((mp->elt_size + mp->header_size + mp->trailer_size) % +ETH_AF_XDP_FRAME_SIZE !=0 ) +return 0 ; + +return 1 ; +} +static int xsk_configure (struct pmd_internals *internals,struct pkt_rx_queue *rxq, -int ring_size) +int ring_size,struct rte_mempool *mb_pool) {struct xsk_socket_config cfg;struct pkt_tx_queue *txq = rxq->pair; +struct rte_mempool *mp;int ret =0 ;int reserve_size; - rxq->umem =xdp_umem_configure (); + mp =check_mempool_zc (mb_pool) ? mb_pool :NULL ; + rxq->umem =xdp_umem_configure (mp);if (!rxq->umem) { ret = -ENOMEM ; goto err;@@ -627,15 +675,21 @@ eth_rx_queue_setup(struct rte_eth_dev *dev, rxq->mb_pool = mb_pool; -if (xsk_configure (internals, rxq, nb_rx_desc)) { -RTE_LOG (ERR ,AF_XDP , -"Failed to configure xdp socket\n" ); +if (xsk_configure (internals, rxq, nb_rx_desc, mb_pool)) { +RTE_LOG (ERR ,AF_XDP ,"Failed to configure xdp socket\n" ); ret = -EINVAL ; goto err; } internals->umem = rxq->umem; +if (mb_pool == internals->umem->mb_pool) + rxq->zc = internals->umem->zc; + +if (rxq->zc) +RTE_LOG (INFO ,AF_XDP , +"zero copy enabled on rx queue %d\n" , rx_queue_id); + dev->data->rx_queues[rx_queue_id] = rxq;return 0 ;
Re: [PATCH v2 5/6] net/af_xdp: enable zero copy
In reply to Xiaolong YeQuoted 52 lines · Xiaolong Ye
Try to check if external mempool (from
rx_queue_setup) is fit foraf_xdp, if it is, it will be registered toaf_xdpsocket directly and there will be no packet data copy on Rx and Tx.Signed-off-by Xiaolong Ye <[email protected]>--- drivers/net/af_xdp/rte_eth_af_xdp.c | 128 ++++++++++++++++++++-------- 1 file changed, 91 insertions(+), 37 deletions(-)drivers/net/af_xdp/rte_eth_af_xdp.c +8
-5@@ -62,6 +62,7 @@ struct xsk_umem_info {struct xsk_umem *umem;struct rte_mempool *mb_pool;void *buffer; +uint8_t zc; };struct pkt_rx_queue {@@ -76,6 +77,7 @@ struct pkt_rx_queue {struct pkt_tx_queue *pair;uint16_t queue_idx; +uint8_t zc; };struct pkt_tx_queue {@@ -191,17 +193,24 @@ eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)uint32_t len =xsk_ring_cons__rx_desc (rx, idx_rx++)->len;char *pkt =xsk_umem__get_data (rxq->umem->buffer, addr); - mbuf =rte_pktmbuf_alloc (rxq->mb_pool); -if (mbuf) { - memcpy(rte_pktmbuf_mtod(mbuf, void*), pkt, len); +if (rxq->zc) { + mbuf =addr_to_mbuf (rxq->umem, addr);rte_pktmbuf_pkt_len (mbuf) =rte_pktmbuf_data_len (mbuf) = len; - rx_bytes += len; bufs[count++] = mbuf; }else { - dropped++; + mbuf =rte_pktmbuf_alloc (rxq->mb_pool); +if (mbuf) { + memcpy(rte_pktmbuf_mtod(mbuf, void*), pkt, len); + rte_pktmbuf_pkt_len(mbuf) =
drivers/net/af_xdp/rte_eth_af_xdp.c eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, · line 216
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c +7
-1+rte_pktmbuf_data_len (mbuf) = len; + rx_bytes += len; + bufs[count++] = mbuf; + }else { + dropped++; + } +rte_pktmbuf_free (addr_to_mbuf (umem, addr)); } -rte_pktmbuf_free (addr_to_mbuf (umem, addr)); }
Did not understand how the zc parts are working, but at least looking at the rx_burst function, when multi q will be supported, is there any reason we would have zc enabled on one rxq and not others?
If the answer is that we would have either all or none rxq with zc, we could have dedicated rx_burst functions and avoid this per mbuf test on rxq->zc.
For the tx part, I don't understand the relation between rx and tx. Should not the zc capability be global to the ethdev port ?
You might also want to look at "simple" tx burst functions like in i40e so that you only need to look at the first mbuf to check its originating pool.
Signature · 1 line
David Marchand
Re: [PATCH v2 5/6] net/af_xdp: enable zero copy
From: David Marchand [mailto:[email protected]]
Sent: Wednesday, March 20, 2019 5:22 PM
To: Ye, Xiaolong <[email protected]>
Cc: dev <[email protected]>; Zhang, Qi Z <[email protected]>; Karlsson, Magnus <[email protected]>; Topel, Bjorn <[email protected]>
Subject: Re: [dpdk-dev] [PATCH v2 5/6] net/af_xdp: enable zero copy
On Tue, Mar 19, 2019 at 8:17 AM Xiaolong Ye <[email protected]> wrote:
Try to check if external mempool (from rx_queue_setup) is fit for af_xdp, if it is, it will be registered to af_xdp socket directly and there will be no packet data copy on Rx and Tx.
---
drivers/net/af_xdp/rte_eth_af_xdp.c | 128 ++++++++++++++++++++--------
1 file changed, 91 insertions(+), 37 deletions(-)
drivers/net/af_xdp/rte_eth_af_xdp.c +0 -0
@@ -62,6 +62,7 @@ struct xsk_umem_info {
struct xsk_umem *umem;
struct rte_mempool *mb_pool;
void *buffer;
+ uint8_t zc;
};
struct pkt_rx_queue {
@@ -76,6 +77,7 @@ struct pkt_rx_queue {
struct pkt_tx_queue *pair;
uint16_t queue_idx;
+ uint8_t zc;
};
struct pkt_tx_queue {
@@ -191,17 +193,24 @@ eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) uint32_t len = xsk_ring_cons__rx_desc(rx, idx_rx++)->len; char *pkt = xsk_umem__get_data(rxq->umem->buffer, addr);
- mbuf = rte_pktmbuf_alloc(rxq->mb_pool);
- if (mbuf) {
- memcpy(rte_pktmbuf_mtod(mbuf, void*), pkt, len); + if (rxq->zc) {
+ mbuf = addr_to_mbuf(rxq->umem, addr); rte_pktmbuf_pkt_len(mbuf) = rte_pktmbuf_data_len(mbuf) = len;
- rx_bytes += len;
bufs[count++] = mbuf;
} else {
- dropped++;
+ mbuf = rte_pktmbuf_alloc(rxq->mb_pool);
+ if (mbuf) {
+ memcpy(rte_pktmbuf_mtod(mbuf, void*), pkt, len); + rte_pktmbuf_pkt_len(mbuf) =
+ rte_pktmbuf_data_len(mbuf) = len; + rx_bytes += len;
+ bufs[count++] = mbuf;
+ } else {
+ dropped++;
+ }
+ rte_pktmbuf_free(addr_to_mbuf(umem, addr)); }
- rte_pktmbuf_free(addr_to_mbuf(umem, addr));
}
Did not understand how the zc parts are working, but at least looking at the rx_burst function, when multi q will be supported, is there any reason we would have zc enabled on one rxq and not others?
[Qi:] the answer is no, we can't anticipate which memory pool application use during rx queue setup, also at the case multi queue share the same memory pool, umem still can't be shared due to race condition, so only one queue could be zc. To make all the queue have zc, we have to assign each queue different memory pool.
If the answer is that we would have either all or none rxq with zc, we could have dedicated rx_burst functions and avoid this per mbuf test on rxq->zc.
For the tx part, I don't understand the relation between rx and tx. Should not the zc capability be global to the ethdev port ?
You might also want to look at "simple" tx burst functions like in i40e so that you only need to look at the first mbuf to check its originating pool.
[Qi:] if you mean DEV_TX_OFFLOAD_MBUF_FAST_FREE, yes I think that's good point.
Signature · 1 line
David Marchand
PATCH v2 6/6 app/testpmd: add mempool flags parameter
In reply to Xiaolong YeWhen create rte_mempool, flags can be parsed from command line. Now, it is possible for testpmd to create a af_xdp friendly mempool (which enable zero copy).
--- app/test-pmd/parameters.c | 12 ++++++++++++ app/test-pmd/testpmd.c | 17 ++++++++++------- app/test-pmd/testpmd.h | 1 + doc/guides/testpmd_app_ug/run_app.rst | 4 ++++ 4 files changed, 27 insertions(+), 7 deletions(-)
app/test-pmd/parameters.c +12 -0
@@ -61,6 +61,7 @@ usage(char* progname)"--tx-first | --stats-period=PERIOD | " "--coremask=COREMASK --portmask=PORTMASK --numa " "--mbuf-size= | --total-num-mbufs= | " +"--mp-flags= | " "--nb-cores= | --nb-ports= | " #ifdef RTE_LIBRTE_CMDLINE"--eth-peers-configfile= | "
@@ -105,6 +106,7 @@ usage(char* progname)printf (" --socket-num=N: set socket from which all memory is allocated " "in NUMA mode.\n" );printf (" --mbuf-size=N: set the data size of mbuf to N bytes.\n" ); +printf (" --mp-flags=N: set the flags when create mbuf memory pool.\n" );printf (" --total-num-mbufs=N: set the number of mbufs to be allocated " "in mbuf pools.\n" );printf (" --max-pkt-len=N: set the maximum size of packet to N bytes.\n" );
@@ -585,6 +587,7 @@ launch_args_parse(int argc, char** argv) { "ring-numa-config", 1, 0, 0 }, { "socket-num", 1, 0, 0 }, { "mbuf-size", 1, 0, 0 }, + { "mp-flags", 1, 0, 0 }, { "total-num-mbufs", 1, 0, 0 }, { "max-pkt-len", 1, 0, 0 }, { "pkt-filter-mode", 1, 0, 0 },
@@ -811,6 +814,15 @@ launch_args_parse(int argc, char** argv)rte_exit (EXIT_FAILURE ,"mbuf-size should be > 0 and < 65536\n" ); } +if (!strcmp (lgopts[opt_idx].name,"mp-flags" )) { + n =atoi (optarg); +if (n >0 && n <=0xFFFF ) + mp_flags = (uint16_t )n; +else +rte_exit (EXIT_FAILURE , +"mp-flags should be > 0 and < 65536\n" ); + } +if (!strcmp (lgopts[opt_idx].name,"total-num-mbufs" )) { n =atoi (optarg);if (n >1024 )
app/test-pmd/testpmd.c +10 -7
@@ -195,6 +195,7 @@ uint32_t burst_tx_delay_time = BURST_TX_WAIT_US;uint32_t burst_tx_retry_num =BURST_TX_RETRIES ;uint16_t mbuf_data_size =DEFAULT_MBUF_DATA_SIZE ;/**< Mbuf data space size. */ +uint16_t mp_flags =0 ;/**< flags parsed when create mempool */ uint32_t param_total_num_mbufs =0 ;/**< number of mbufs in all pools - if * specified on command-line. */ uint16_t stats_period;/**< Period to show statistics (disabled by default) */
@@ -834,6 +835,7 @@ setup_extmem(uint32_t nb_mbufs, uint32_t mbuf_sz, bool huge)*/ static void mbuf_pool_create (uint16_t mbuf_seg_size,unsigned nb_mbuf, +unsigned int flags,unsigned int socket_id) {char pool_name[RTE_MEMPOOL_NAMESIZE ];
@@ -853,8 +855,8 @@ mbuf_pool_create(uint16_t mbuf_seg_size, unsigned nb_mbuf,/* wrapper to rte_mempool_create() */ TESTPMD_LOG (INFO ,"preferred mempool ops selected: %s\n" ,rte_mbuf_best_mempool_ops ()); - rte_mp =rte_pktmbuf_pool_create (pool_name, nb_mbuf, - mb_mempool_cache,0 , mbuf_seg_size, socket_id); + rte_mp =rte_pktmbuf_pool_create_with_flags (pool_name, nb_mbuf, + mb_mempool_cache,0 , mbuf_seg_size, flags, socket_id);break ; }case MP_ALLOC_ANON :
@@ -891,8 +893,8 @@ mbuf_pool_create(uint16_t mbuf_seg_size, unsigned nb_mbuf,TESTPMD_LOG (INFO ,"preferred mempool ops selected: %s\n" ,rte_mbuf_best_mempool_ops ()); - rte_mp =rte_pktmbuf_pool_create (pool_name, nb_mbuf, - mb_mempool_cache,0 , mbuf_seg_size, + rte_mp =rte_pktmbuf_pool_create_with_flags (pool_name, nb_mbuf, + mb_mempool_cache,0 , mbuf_seg_size, flags, heap_socket);break ; }
@@ -1128,13 +1130,14 @@ init_config(void)for (i =0 ; i < num_sockets; i++)mbuf_pool_create (mbuf_data_size, nb_mbuf_per_pool, - socket_ids[i]); + mp_flags, socket_ids[i]); }else {if (socket_num ==UMA_NO_CONFIG ) -mbuf_pool_create (mbuf_data_size, nb_mbuf_per_pool,0 ); +mbuf_pool_create (mbuf_data_size, nb_mbuf_per_pool, + mp_flags,0 );else mbuf_pool_create (mbuf_data_size, nb_mbuf_per_pool, - socket_num); + mp_flags, socket_num); }init_port_config ();
app/test-pmd/testpmd.h +1 -0
@@ -408,6 +408,7 @@ extern uint8_t dcb_config;extern uint8_t dcb_test;extern uint16_t mbuf_data_size;/**< Mbuf data space size. */ +extern uint16_t mp_flags;/**< flags for mempool creation. */ extern uint32_t param_total_num_mbufs;extern uint16_t stats_period;
doc/guides/testpmd_app_ug/run_app.rst +4 -0
@@ -392,6 +392,10 @@ The commandline options are: * xmemhuge: create and populate mempool using externally and anonymously allocated hugepage area +* ``--mp-flag=<N>`` + + Select mempool allocation flag. + * ``--noisy-tx-sw-buffer-size`` Set the number of maximum elements of the FIFO queue to be created
Signature · 1 line
2.17.1
Re: [PATCH v2 6/6] app/testpmd: add mempool flags parameter
Quoted 11 lines · Xiaolong Ye
When create
rte_mempool, flags can be parsed from command line. Now, it is possible for testpmd to create aaf_xdpfriendly mempool (which enable zero copy).Signed-off-by Qi Zhang <[email protected]>Signed-off-by Xiaolong Ye <[email protected]>--- app/test-pmd/parameters.c | 12 ++++++++++++ app/test-pmd/testpmd.c | 17 ++++++++++------- app/test-pmd/testpmd.h | 1 + doc/guides/testpmd_app_ug/run_app.rst | 4 ++++
If I understand it correctly, The user needs to change all the application in order to avail zero copy feature of XDP.
If so,
How about creating wrapper mempool driver for xdp at drivers/mempool/? and mempool's best mempool feature to select the required mempool driver for XDP at runtime without changing the apps.
see rte_mbuf_best_mempool_ops()
see struct eth_dev_ops::pool_ops_supported
/Jerin
Re: [PATCH v2 6/6] app/testpmd: add mempool flags parameter
Hi,
Quoted 16 lines · Jerin Jacob Kollanukkaran
Quoted 11 lines · Xiaolong Ye
When create
rte_mempool, flags can be parsed from command line. Now, it is possible for testpmd to create aaf_xdpfriendly mempool (which enable zero copy).Signed-off-by Qi Zhang <[email protected]>Signed-off-by Xiaolong Ye <[email protected]>--- app/test-pmd/parameters.c | 12 ++++++++++++ app/test-pmd/testpmd.c | 17 ++++++++++------- app/test-pmd/testpmd.h | 1 + doc/guides/testpmd_app_ug/run_app.rst | 4 ++++If I understand it correctly, The user needs to change all the application in order to avail zero copy feature of XDP.
Quoted 8 lines
If so,
How about creating wrapper mempool driver for xdp at
drivers/mempool/? and mempool's best mempool feature to select the required mempool driver for XDP at runtime without changing the apps.see
rte_mbuf_best_mempool_ops()
see structeth_dev_ops::pool_ops_supported
Sounds a good suggestion, I'll investigate and see how to implement it.
Thanks,
Xiaolong
Quoted 6 lines
/Jerin
Re: [PATCH v2 6/6] app/testpmd: add mempool flags parameter
In reply to Jerin Jacob KollanukkaranQuoted 16 lines · Jerin Jacob Kollanukkaran
Quoted 11 lines · Xiaolong Ye
When create
rte_mempool, flags can be parsed from command line. Now, it is possible for testpmd to create aaf_xdpfriendly mempool (which enable zero copy).Signed-off-by Qi Zhang <[email protected]>Signed-off-by Xiaolong Ye <[email protected]>--- app/test-pmd/parameters.c | 12 ++++++++++++ app/test-pmd/testpmd.c | 17 ++++++++++------- app/test-pmd/testpmd.h | 1 + doc/guides/testpmd_app_ug/run_app.rst | 4 ++++If I understand it correctly, The user needs to change all the application in order to avail zero copy feature of XDP.
Quoted 9 lines
If so,
How about creating wrapper mempool driver for xdp at
drivers/mempool/? and mempool's best mempool feature to select the required mempool driver for XDP at runtime without changing the apps.see
rte_mbuf_best_mempool_ops()
see structeth_dev_ops::pool_ops_supported
Glab to read this, I was under the same impression :-)
Signature · 1 line
David Marchand
Re: [PATCH v2 6/6] app/testpmd: add mempool flags parameter
Quoted 25 lines · David Marchand
Quoted 22 lines · Jerin Jacob Kollanukkaran
Quoted 11 lines · Xiaolong Ye
When create
rte_mempool, flags can be parsed from command line. Now, it is possible for testpmd to create aaf_xdpfriendly mempool (which enable zero copy).Signed-off-by Qi Zhang <[email protected]>Signed-off-by Xiaolong Ye <[email protected]>--- app/test-pmd/parameters.c | 12 ++++++++++++ app/test-pmd/testpmd.c | 17 ++++++++++------- app/test-pmd/testpmd.h | 1 + doc/guides/testpmd_app_ug/run_app.rst | 4 ++++If I understand it correctly, The user needs to change all the application in order to avail zero copy feature of XDP.
If so,
How about creating wrapper mempool driver for xdp at
drivers/mempool/? and mempool's best mempool feature to select the required mempool driver for XDP at runtime without changing the apps.
Quoted 5 lines
Quoted 3 lines
see
rte_mbuf_best_mempool_ops()
see structeth_dev_ops::pool_ops_supportedGlab to read this, I was under the same impression :-)
We plan to separate this in another patchset and may target for next release.
Thanks,
Xiaolong
Quoted 4 lines
Signature · 2 lines
-- David Marchand
v3 cover Introduce AF_XDP PMD
In reply to Xiaolong YeOverview
========
This patchset adds a new PMD driver for AF_XDP which is a proposed faster version of AF_PACKET interface in Linux, see below links [1] [2] for details of AF_XDP introduction:
AF_XDP roadmap
==============
- AF_XDP is included in upstream kernel since 4.18, and AF_XDP support in libbpf has been merged in v5.1-rc1.
- Now i40e and ixgbe drivers have supported zero copy mode.
Change logs
===========
V3:
- Fix all style issues pointed by Stephen, Mattias, David.
- Drop the testpmd change as we'll adopt Jerin's suggestion to add a new mempool driver to handle the application use of AF_XDP's zero copy feature.
V2:
- Fix a NULL pointer reference crash issue
- Fix txonly stop sending traffic in zc mode
- Refactor rte_mbuf.c to avoid ABI breakage.
- Fix multiple style issues pointed by Ferruh, David, Stephen, Luca.
changes vs RFC sent by Qi last Aug:
- Re-work base on AF_XDP's interface changes since the new libbpf has provided higher-level APIs that hide many of the details of the AP_XDP uapi. After the rework, it helps to reduce 300+ lines of code.
- multi-queues is not supported due to current libbpf doesn't support multi-sockets on a single umem.
- No extra steps to load xdp program manually, since the current behavior of libbpf would load a default xdp program when user calls xsk_socket__create. userspace application only needs to handle the cleanup.
How to try
==========
1. take the kernel >= v5.1-rc1, build kernel and replace your host kernel with it.
make sure you turn on XDP sockets when compiling
Networking support -->
Networking options -->
[ * ] XDP sockets
2. build libbpf in tools/lib/bpf, and copy the libbpf.a and libbpf.so to /usr/lib64
cd tools/lib/bpf
make
3. ethtool -L enp59s0f0 combined 1
4. start testpmd
./build/app/testpmd -c 0xc -n 4 --vdev eth_af_xdp,iface=enp59s0f0,queue=0 -- -i --rxq=1 --txq=1
in this case, default xdp program will be loaded and linked to queue 0 of enp59s0f0, network traffics travel to queue 0 will be redirected to af_xdp socket.
Xiaolong Ye (5):
net/af_xdp: introduce AF XDP PMD driver
lib/mbuf: introduce helper to create mempool with flags
lib/mempool: allow page size aligned mempool
net/af_xdp: use mbuf mempool for buffer management
net/af_xdp: enable zero copy
MAINTAINERS | 6 + config/common_base | 5 + config/common_linux | 1 + doc/guides/nics/af_xdp.rst | 45 + doc/guides/nics/features/af_xdp.ini | 11 + doc/guides/nics/index.rst | 1 + doc/guides/rel_notes/release_19_05.rst | 7 + drivers/net/Makefile | 1 + drivers/net/af_xdp/Makefile | 32 + drivers/net/af_xdp/meson.build | 21 + drivers/net/af_xdp/rte_eth_af_xdp.c | 1018 +++++++++++++++++ drivers/net/af_xdp/rte_pmd_af_xdp_version.map | 3 + drivers/net/meson.build | 1 + lib/librte_mbuf/rte_mbuf.c | 29 +- lib/librte_mbuf/rte_mbuf.h | 45 + lib/librte_mempool/rte_mempool.c | 3 + lib/librte_mempool/rte_mempool.h | 1 + mk/rte.app.mk | 1 + 18 files changed, 1226 insertions(+), 5 deletions(-) create mode 100644 doc/guides/nics/af_xdp.rst create mode 100644 doc/guides/nics/features/af_xdp.ini create mode 100644 drivers/net/af_xdp/Makefile create mode 100644 drivers/net/af_xdp/meson.build create mode 100644 drivers/net/af_xdp/rte_eth_af_xdp.c create mode 100644 drivers/net/af_xdp/rte_pmd_af_xdp_version.map
Signature · 1 line
2.17.1
PATCH v3 1/5 net/af_xdp: introduce AF XDP PMD driver
Add a new PMD driver for AF_XDP which is a proposed faster version of AF_PACKET interface in Linux. More info about AF_XDP, please refer to [1] [2].
This is the vanilla version PMD which just uses a raw buffer registered as the umem.
[1] https://fosdem.org/2018/schedule/event/af_xdp/
[2] https://lwn.net/Articles/745934/
--- MAINTAINERS | 6 + config/common_base | 5 + config/common_linux | 1 + doc/guides/nics/af_xdp.rst | 45 + doc/guides/nics/features/af_xdp.ini | 11 + doc/guides/nics/index.rst | 1 + doc/guides/rel_notes/release_19_05.rst | 7 + drivers/net/Makefile | 1 + drivers/net/af_xdp/Makefile | 32 + drivers/net/af_xdp/meson.build | 21 + drivers/net/af_xdp/rte_eth_af_xdp.c | 932 ++++++++++++++++++ drivers/net/af_xdp/rte_pmd_af_xdp_version.map | 3 + drivers/net/meson.build | 1 + mk/rte.app.mk | 1 + 14 files changed, 1067 insertions(+) create mode 100644 doc/guides/nics/af_xdp.rst create mode 100644 doc/guides/nics/features/af_xdp.ini create mode 100644 drivers/net/af_xdp/Makefile create mode 100644 drivers/net/af_xdp/meson.build create mode 100644 drivers/net/af_xdp/rte_eth_af_xdp.c create mode 100644 drivers/net/af_xdp/rte_pmd_af_xdp_version.map
MAINTAINERS +6 -0
@@ -468,6 +468,12 @@ M: John W. Linville <[email protected]> F: drivers/net/af_packet/ F: doc/guides/nics/features/afpacket.ini +Linux AF_XDP +M: Xiaolong Ye <[email protected]> +M: Qi Zhang <[email protected]> +F: drivers/net/af_xdp/ +F: doc/guides/nics/features/af_xdp.rst + Amazon ENA M: Marcin Wojtas <[email protected]> M: Michal Krawczyk <[email protected]>
config/common_base +5 -0
@@ -416,6 +416,11 @@ CONFIG_RTE_LIBRTE_VMXNET3_DEBUG_TX_FREE=n # CONFIG_RTE_LIBRTE_PMD_AF_PACKET=n +# +# Compile software PMD backed by AF_XDP sockets (Linux only) +# +CONFIG_RTE_LIBRTE_PMD_AF_XDP=n + # # Compile link bonding PMD library #
config/common_linux +1 -0
@@ -19,6 +19,7 @@ CONFIG_RTE_LIBRTE_VHOST_POSTCOPY=n CONFIG_RTE_LIBRTE_PMD_VHOST=y CONFIG_RTE_LIBRTE_IFC_PMD=y CONFIG_RTE_LIBRTE_PMD_AF_PACKET=y +CONFIG_RTE_LIBRTE_PMD_AF_XDP=y CONFIG_RTE_LIBRTE_PMD_SOFTNIC=y CONFIG_RTE_LIBRTE_PMD_TAP=y CONFIG_RTE_LIBRTE_AVP_PMD=y
doc/guides/nics/af_xdp.rst new file +45 -0
45 lines not on this page · raw message
doc/guides/nics/features/af_xdp.ini new file +11 -0
@@ -0,0 +1,11 @@ +; +; Supported features of the 'af_xdp' network poll mode driver. +; +; Refer to default.ini for the full list of available PMD features. +; +[Features] +Link status = Y +MTU update = Y +Promiscuous mode = Y +Stats per queue = Y +x86-64 = Y
doc/guides/nics/index.rst +1 -0
@@ -12,6 +12,7 @@ Network Interface Controller Drivers features build_and_test af_packet + af_xdp ark atlantic avp
doc/guides/rel_notes/release_19_05.rst +7 -0
@@ -65,6 +65,13 @@ New Features process. * Added support for Rx packet types list in a secondary process. +* **Added the AF_XDP PMD.** + + Added a Linux-specific PMD driver for AF_XDP, it can create the AF_XDP socket + and bind it to a specific netdev queue, it allows a DPDK application to send + and receive raw packets through the socket which would bypass the kernel + network stack to achieve high performance packet processing. + * **Updated Mellanox drivers.** New features and improvements were done in mlx4 and mlx5 PMDs:
drivers/net/Makefile +1 -0
@@ -9,6 +9,7 @@ ifeq ($(CONFIG_RTE_LIBRTE_THUNDERX_NICVF_PMD),d) endif DIRS-$(CONFIG_RTE_LIBRTE_PMD_AF_PACKET) += af_packet +DIRS-$(CONFIG_RTE_LIBRTE_PMD_AF_XDP) += af_xdp DIRS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += ark DIRS-$(CONFIG_RTE_LIBRTE_ATLANTIC_PMD) += atlantic DIRS-$(CONFIG_RTE_LIBRTE_AVP_PMD) += avp
drivers/net/af_xdp/Makefile new file +32 -0
@@ -0,0 +1,32 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Intel Corporation + +include $(RTE_SDK)/mk/rte.vars.mk + +# +# library name +# +LIB = librte_pmd_af_xdp.a + +EXPORT_MAP := rte_pmd_af_xdp_version.map + +LIBABIVER := 1 + +CFLAGS += -O3 + +# require kernel version >= v5.1-rc1 +CFLAGS += -I$(RTE_KERNELDIR)/tools/include +CFLAGS += -I$(RTE_KERNELDIR)/tools/lib/bpf + +CFLAGS += $(WERROR_FLAGS) +LDLIBS += -lrte_eal -lrte_mbuf -lrte_mempool -lrte_ring +LDLIBS += -lrte_ethdev -lrte_net -lrte_kvargs +LDLIBS += -lrte_bus_vdev +LDLIBS += -lbpf + +# +# all source are stored in SRCS-y +# +SRCS-$(CONFIG_RTE_LIBRTE_PMD_AF_XDP) += rte_eth_af_xdp.c + +include $(RTE_SDK)/mk/rte.lib.mk
drivers/net/af_xdp/meson.build new file +21 -0
@@ -0,0 +1,21 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Intel Corporation + +if host_machine.system() != 'linux' + build = false +endif + +bpf_dep = dependency('libbpf', required: false) +if bpf_dep.found() + build = true +else + bpf_dep = cc.find_library('libbpf', required: false) + if bpf_dep.found() and cc.has_header('xsk.h', dependencies: bpf_dep) + build = true + pkgconfig_extra_libs += '-lbpf' + else + build = false + endif +endif +sources = files('rte_eth_af_xdp.c') +ext_deps += bpf_dep
drivers/net/af_xdp/rte_eth_af_xdp.c new file +932 -0
932 lines not on this page · raw message
drivers/net/af_xdp/rte_pmd_af_xdp_version.map new file +3 -0
@@ -0,0 +1,3 @@ +DPDK_19.05 { + local: *; +};
drivers/net/meson.build +1 -0
@@ -2,6 +2,7 @@ # Copyright(c) 2017 Intel Corporation drivers = ['af_packet', + 'af_xdp', 'ark', 'atlantic', 'avp',
mk/rte.app.mk +1 -0
@@ -143,6 +143,7 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_DPAA2_MEMPOOL) += -lrte_mempool_dpaa2 endif _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AF_PACKET) += -lrte_pmd_af_packet +_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AF_XDP) += -lrte_pmd_af_xdp -lelf -lbpf _LDLIBS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += -lrte_pmd_ark _LDLIBS-$(CONFIG_RTE_LIBRTE_ATLANTIC_PMD) += -lrte_pmd_atlantic _LDLIBS-$(CONFIG_RTE_LIBRTE_AVP_PMD) += -lrte_pmd_avp
Signature · 1 line
2.17.1
Re: [PATCH v3 1/5] net/af_xdp: introduce AF XDP PMD driver
Quoted 8 lines · On Thu, 21 Mar 2019 17:18:41 +0800 Xiaolong Ye <[email protected]> wrote:
drivers/net/af_xdp/rte_eth_af_xdp.c new file +8
-0+ +static inline int +reserve_fill_queue (struct xsk_umem_info *umem,int reserve_size) +{ +struct xsk_ring_prod *fq = &umem->fq; +uint32_t idx; +void *addr =NULL ; +int i, ret;
drivers/net/af_xdp/rte_eth_af_xdp.c line 137
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+ + ret =xsk_ring_prod__reserve (fq, reserve_size, &idx); +if (!ret) { +RTE_LOG (ERR ,AF_XDP ,"Failed to reserve enough fq descs.\n" ); +return ret; + } + +for (i =0 ; i < reserve_size; i++) { +__u64 *fq_addr; +rte_ring_dequeue (umem->buf_ring, &addr);
You should check return value of dequeue, otherwise static checkers will (rightly) complain that "everyone else checks return value of of rte_ring_dequeue() why not here?"
Re: [PATCH v3 1/5] net/af_xdp: introduce AF XDP PMD driver
Quoted 17 lines · Stephen Hemminger
Quoted 14 lines · On Thu, 21 Mar 2019 17:18:41 +0800 Xiaolong Ye <[email protected]> wrote:
drivers/net/af_xdp/rte_eth_af_xdp.c new file +14
-0+ +static inline int +reserve_fill_queue (struct xsk_umem_info *umem,int reserve_size) +{ +struct xsk_ring_prod *fq = &umem->fq; +uint32_t idx; +void *addr =NULL ; +int i, ret; + + ret =xsk_ring_prod__reserve (fq, reserve_size, &idx); +if (!ret) { +RTE_LOG (ERR ,AF_XDP ,"Failed to reserve enough fq descs.\n" ); +return ret; + }
Quoted 8 lines
drivers/net/af_xdp/rte_eth_af_xdp.c line 137
Quoted 4 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +4
-0+ +for (i =0 ; i < reserve_size; i++) { +__u64 *fq_addr; +rte_ring_dequeue (umem->buf_ring, &addr);You should check return value of dequeue, otherwise static checkers will (rightly) complain that "everyone else checks return value of of
rte_ring_dequeue()why not here?"
Got it, will do.
Re: [PATCH v3 1/5] net/af_xdp: introduce AF XDP PMD driver
In reply to Xiaolong Yedrivers/net/af_xdp/rte_eth_af_xdp.c line 182
Quoted 12 lines · On Thu, 21 Mar 2019 17:18:41 +0800 Xiaolong Ye <[email protected]> wrote:
drivers/net/af_xdp/rte_eth_af_xdp.c new file +12
-0+for (i =0 ; i < rcvd; i++) { +const struct xdp_desc *desc; +uint64_t addr; +uint32_t len; +void *pkt; + + desc =xsk_ring_cons__rx_desc (rx, idx_rx++); + addr = desc->addr; + len = desc->len; + pkt =xsk_umem__get_data (rxq->umem->buffer, addr); + + mbuf =rte_pktmbuf_alloc (rxq->mb_pool);
You could use rte_pktmbuf_alloc_bulk to get the mbufs in one call before doing this. It saves rcvd-1 atomic operations.
Re: [PATCH v3 1/5] net/af_xdp: introduce AF XDP PMD driver
Quoted 18 lines · Stephen Hemminger
drivers/net/af_xdp/rte_eth_af_xdp.c line 182
Quoted 12 lines · On Thu, 21 Mar 2019 17:18:41 +0800 Xiaolong Ye <[email protected]> wrote:
drivers/net/af_xdp/rte_eth_af_xdp.c new file +12
-0+for (i =0 ; i < rcvd; i++) { +const struct xdp_desc *desc; +uint64_t addr; +uint32_t len; +void *pkt; + + desc =xsk_ring_cons__rx_desc (rx, idx_rx++); + addr = desc->addr; + len = desc->len; + pkt =xsk_umem__get_data (rxq->umem->buffer, addr); + + mbuf =rte_pktmbuf_alloc (rxq->mb_pool);You could use
rte_pktmbuf_alloc_bulkto get the mbufs in one call before doing this. It saves rcvd-1 atomic operations.
Got it, will do.
Re: [PATCH v3 1/5] net/af_xdp: introduce AF XDP PMD driver
In reply to Xiaolong YeQuoted 4 lines · On Thu, 21 Mar 2019 17:18:41 +0800 Xiaolong Ye <[email protected]> wrote:
static void
kick_tx(struct pkt_tx_queue *txq)drivers/net/af_xdp/rte_eth_af_xdp.c new file +3
-0+{ +struct xsk_umem_info *umem = txq->pair->umem; +
drivers/net/af_xdp/rte_eth_af_xdp.c line 235
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+while (send (xsk_socket__fd (txq->pair->xsk),NULL , +0 ,MSG_DONTWAIT ) <0 ) { +/* some thing unexpected */ +if (errno !=EBUSY && errno !=EAGAIN ) +break ; + +/* pull from complete qeueu to leave more space */ +if (errno ==EAGAIN ) +pull_umem_cq (umem,ETH_AF_XDP_TX_BATCH_SIZE ); + }
What about EINTR??
You should retry the send then.
Re: [PATCH v3 1/5] net/af_xdp: introduce AF XDP PMD driver
Quoted 12 lines · Stephen Hemminger
Quoted 9 lines · On Thu, 21 Mar 2019 17:18:41 +0800 Xiaolong Ye <[email protected]> wrote:
static void
kick_tx(struct pkt_tx_queue *txq)drivers/net/af_xdp/rte_eth_af_xdp.c new file +8
-0+{ +struct xsk_umem_info *umem = txq->pair->umem; + +while (send (xsk_socket__fd (txq->pair->xsk),NULL , +0 ,MSG_DONTWAIT ) <0 ) { +/* some thing unexpected */ +if (errno !=EBUSY && errno !=EAGAIN ) +break ;
Quoted 8 lines
drivers/net/af_xdp/rte_eth_af_xdp.c line 235
Quoted 5 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +5
-0+ +/* pull from complete qeueu to leave more space */ +if (errno ==EAGAIN ) +pull_umem_cq (umem,ETH_AF_XDP_TX_BATCH_SIZE ); + }What about EINTR??
You should retry the send then.
Will do.
Re: [PATCH v3 1/5] net/af_xdp: introduce AF XDP PMD driver
In reply to Xiaolong Yedrivers/net/af_xdp/rte_eth_af_xdp.c line 366
Quoted 3 lines · On Thu, 21 Mar 2019 17:18:41 +0800 Xiaolong Ye <[email protected]> wrote:
drivers/net/af_xdp/rte_eth_af_xdp.c new file +3
-0+if (ret !=0 ) { +RTE_LOG (ERR ,AF_XDP ,"getsockopt() failed for XDP_STATISTICS.\n" ); +return -1 ;
You need to use the new dynamic log types and not have a global logtype.
Re: [PATCH v3 1/5] net/af_xdp: introduce AF XDP PMD driver
Quoted 8 lines · Stephen Hemminger
drivers/net/af_xdp/rte_eth_af_xdp.c line 366
Quoted 3 lines · On Thu, 21 Mar 2019 17:18:41 +0800 Xiaolong Ye <[email protected]> wrote:
drivers/net/af_xdp/rte_eth_af_xdp.c new file +3
-0+if (ret !=0 ) { +RTE_LOG (ERR ,AF_XDP ,"getsockopt() failed for XDP_STATISTICS.\n" ); +return -1 ;You need to use the new dynamic log types and not have a global logtype.
You mean for all the logs in this driver, right? Is it due to the global logtype will be deprecated?
Will investigate and implement the dynamic log type.
Thanks,
Xiaolong
Re: [PATCH v3 1/5] net/af_xdp: introduce AF XDP PMD driver
Quoted 4 lines · On Fri, 22 Mar 2019 10:15:23 +0800 Ye Xiaolong <[email protected]> wrote:
Quoted 3 lines · Stephen Hemminger
On Thu, 21 Mar 2019 17:18:41 +0800
Xiaolong Ye <[email protected]> wrote:
Quoted 8 lines
Quoted 5 lines
drivers/net/af_xdp/rte_eth_af_xdp.c line 366
Quoted 3 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +3
-0+if (ret !=0 ) { +RTE_LOG (ERR ,AF_XDP ,"getsockopt() failed for XDP_STATISTICS.\n" ); +return -1 ;You need to use the new dynamic log types and not have a global logtype.
You mean for all the logs in this driver, right? Is it due to the global logtype will be deprecated?
Global log types should not be used or added by any new code.
Re: [PATCH v3 1/5] net/af_xdp: introduce AF XDP PMD driver
Quoted 13 lines · Stephen Hemminger
Quoted 10 lines · On Fri, 22 Mar 2019 10:15:23 +0800 Ye Xiaolong <[email protected]> wrote:
Quoted 8 lines · Stephen Hemminger
drivers/net/af_xdp/rte_eth_af_xdp.c line 366
Quoted 3 lines · On Thu, 21 Mar 2019 17:18:41 +0800 Xiaolong Ye <[email protected]> wrote:
drivers/net/af_xdp/rte_eth_af_xdp.c new file +3
-0+if (ret !=0 ) { +RTE_LOG (ERR ,AF_XDP ,"getsockopt() failed for XDP_STATISTICS.\n" ); +return -1 ;You need to use the new dynamic log types and not have a global logtype.
Quoted 4 lines
Quoted 2 lines
You mean for all the logs in this driver, right? Is it due to the global logtype will be deprecated?
Global log types should not be used or added by any new code.
Got it.
Re: [PATCH v3 1/5] net/af_xdp: introduce AF XDP PMD driver
In reply to Xiaolong Yedrivers/net/af_xdp/rte_eth_af_xdp.c line 654
Quoted 5 lines · On Thu, 21 Mar 2019 17:18:41 +0800 Xiaolong Ye <[email protected]> wrote:
drivers/net/af_xdp/rte_eth_af_xdp.c new file +5
-0+ +if (ret <0 ) +return -EINVAL ; + +return 0 ;
You could propogate kernel errno into DPDK?
return (ret < 0) ? -errno : 0;
Re: [PATCH v3 1/5] net/af_xdp: introduce AF XDP PMD driver
Quoted 12 lines · Stephen Hemminger
drivers/net/af_xdp/rte_eth_af_xdp.c line 654
Quoted 5 lines · On Thu, 21 Mar 2019 17:18:41 +0800 Xiaolong Ye <[email protected]> wrote:
drivers/net/af_xdp/rte_eth_af_xdp.c new file +5
-0+ +if (ret <0 ) +return -EINVAL ; + +return 0 ;You could propogate kernel errno into DPDK?
return (ret < 0) ? -errno : 0;
Sorry, could you share the advantage of doing this?
Thanks,
Xiaolong
Re: [PATCH v3 1/5] net/af_xdp: introduce AF XDP PMD driver
Quoted 14 lines · On Fri, 22 Mar 2019 10:01:57 +0800 Ye Xiaolong <[email protected]> wrote:
Quoted 12 lines · Stephen Hemminger
drivers/net/af_xdp/rte_eth_af_xdp.c line 654
Quoted 5 lines · On Thu, 21 Mar 2019 17:18:41 +0800 Xiaolong Ye <[email protected]> wrote:
drivers/net/af_xdp/rte_eth_af_xdp.c new file +5
-0+ +if (ret <0 ) +return -EINVAL ; + +return 0 ;You could propogate kernel errno into DPDK?
return (ret < 0) ? -errno : 0;
Quoted 4 lines
Sorry, could you share the advantage of doing this?
Thanks,
Xiaolong
Suppose kernel returned -ENOTSUPP or other error, it could go back to the caller rather than juse invalid.
Re: [PATCH v3 1/5] net/af_xdp: introduce AF XDP PMD driver
Quoted 17 lines · Stephen Hemminger
Quoted 14 lines · On Fri, 22 Mar 2019 10:01:57 +0800 Ye Xiaolong <[email protected]> wrote:
Quoted 12 lines · Stephen Hemminger
drivers/net/af_xdp/rte_eth_af_xdp.c line 654
Quoted 5 lines · On Thu, 21 Mar 2019 17:18:41 +0800 Xiaolong Ye <[email protected]> wrote:
drivers/net/af_xdp/rte_eth_af_xdp.c new file +5
-0+ +if (ret <0 ) +return -EINVAL ; + +return 0 ;You could propogate kernel errno into DPDK?
return (ret < 0) ? -errno : 0;
Quoted 7 lines
Quoted 4 lines
Sorry, could you share the advantage of doing this?
Thanks,
XiaolongSuppose kernel returned
-ENOTSUPPor other error, it could go back to the caller rather than juse invalid.
Got it.
Re: [PATCH v3 1/5] net/af_xdp: introduce AF XDP PMD driver
In reply to Xiaolong Yedrivers/net/af_xdp/rte_eth_af_xdp.c
Quoted 3 lines · On Thu, 21 Mar 2019 17:18:41 +0800 Xiaolong Ye <[email protected]> wrote:
drivers/net/af_xdp/rte_eth_af_xdp.c new file +3
-0+if (strnlen (value,IFNAMSIZ ) >IFNAMSIZ -1 ) { +RTE_LOG (ERR ,AF_XDP ,"Invalid name %s, should be less than " +"%u bytes.\n" , value,IFNAMSIZ )
Please don't break error message strings across multiple source lines. It makes it harder to use tools like grep to find errors in source.
Re: [PATCH v3 1/5] net/af_xdp: introduce AF XDP PMD driver
Quoted 9 lines · Stephen Hemminger
drivers/net/af_xdp/rte_eth_af_xdp.c
Quoted 3 lines · On Thu, 21 Mar 2019 17:18:41 +0800 Xiaolong Ye <[email protected]> wrote:
drivers/net/af_xdp/rte_eth_af_xdp.c new file +3
-0+if (strnlen (value,IFNAMSIZ ) >IFNAMSIZ -1 ) { +RTE_LOG (ERR ,AF_XDP ,"Invalid name %s, should be less than " +"%u bytes.\n" , value,IFNAMSIZ )Please don't break error message strings across multiple source lines. It makes it harder to use tools like grep to find errors in source.
Good point, will keep this in mind.
Thanks,
Xiaolong
Re: [PATCH v3 1/5] net/af_xdp: introduce AF XDP PMD driver
In reply to Xiaolong Yedrivers/net/af_xdp/rte_eth_af_xdp.c line 790
Quoted 12 lines · On Thu, 21 Mar 2019 17:18:41 +0800 Xiaolong Ye <[email protected]> wrote:
drivers/net/af_xdp/rte_eth_af_xdp.c new file +12
-0+ +strlcpy (ifr.ifr_name, if_name,IFNAMSIZ ); +if (ioctl (sock,SIOCGIFINDEX , &ifr)) + goto error; + +if (ioctl (sock,SIOCGIFHWADDR , &ifr)) + goto error; + +rte_memcpy (eth_addr, ifr.ifr_hwaddr.sa_data,ETHER_ADDR_LEN ); + +close (sock); + *if_index =if_nametoindex (if_name);
This seems confused:
- first you get ifindex with SIOCGIFINDEX, then you ignore the result
- then get MAC address.
- then use if_nametoindex() which does SIOCGIFINDEX internally
Re: [PATCH v3 1/5] net/af_xdp: introduce AF XDP PMD driver
Quoted 12 lines · Stephen Hemminger
Quoted 9 lines · On Thu, 21 Mar 2019 17:18:41 +0800 Xiaolong Ye <[email protected]> wrote:
drivers/net/af_xdp/rte_eth_af_xdp.c new file +9
-0+ +strlcpy (ifr.ifr_name, if_name,IFNAMSIZ ); +if (ioctl (sock,SIOCGIFINDEX , &ifr)) + goto error; + +if (ioctl (sock,SIOCGIFHWADDR , &ifr)) + goto error; + +rte_memcpy (eth_addr, ifr.ifr_hwaddr.sa_data,ETHER_ADDR_LEN );
Quoted 8 lines
drivers/net/af_xdp/rte_eth_af_xdp.c line 790
Quoted 3 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +3
-0+ +close (sock); + *if_index =if_nametoindex (if_name);This seems confused:
- first you get ifindex with SIOCGIFINDEX, then you ignore the result
- then get MAC address.
- then useif_nametoindex()which does SIOCGIFINDEX internally
You're right, the code is chaotic here, will improve it in next version.
Thanks,
Xiaolong
Re: [PATCH v3 1/5] net/af_xdp: introduce AF XDP PMD driver
In reply to Xiaolong Yedrivers/net/af_xdp/rte_eth_af_xdp.c line 929
Quoted 2 lines · On Thu, 21 Mar 2019 17:18:41 +0800 Xiaolong Ye <[email protected]> wrote:
drivers/net/af_xdp/rte_eth_af_xdp.c new file +2
-0+ +RTE_PMD_REGISTER_VDEV (eth_af_xdp, pmd_af_xdp_drv);
The convention in other network drivers is to use net_XXX in the vdev name. In AF_XDP that would be:
RTE_PMD_REGISTER_VDEV(net_af_xdp, pmd_af_xdp_drv);
About naming, I would just drop AF_ from the name everywhere, the driver is about running over XDP, and the "AF_" is just a prefix for address family.
Why not:
net/xdp
Re: [PATCH v3 1/5] net/af_xdp: introduce AF XDP PMD driver
Quoted 10 lines · Stephen Hemminger
drivers/net/af_xdp/rte_eth_af_xdp.c line 929
Quoted 2 lines · On Thu, 21 Mar 2019 17:18:41 +0800 Xiaolong Ye <[email protected]> wrote:
drivers/net/af_xdp/rte_eth_af_xdp.c new file +2
-0+ +RTE_PMD_REGISTER_VDEV (eth_af_xdp, pmd_af_xdp_drv);The convention in other network drivers is to use
net_XXXin the vdev name. InAF_XDPthat would be:
RTE_PMD_REGISTER_VDEV(net_af_xdp, pmd_af_xdp_drv);
Got it.
Quoted 6 lines
About naming, I would just drop
AF_from the name everywhere, the driver is about running over XDP, and the "AF_" is just a prefix for address family.Why not:
net/xdp
Thanks for the advice, Actually this driver is more about AF_XDP rathan than XDP, the foundational objects it uses such as umem, umem fill ring, umem completion ring, tx ring, rx ring which are all AF_XDP specfic, so I would rather keep the naming.
Thanks,
Xiaolong
Quoted 1 line
Re: [PATCH v3 1/5] net/af_xdp: introduce AF XDP PMD driver
Quoted 18 lines · Ye Xiaolong
Quoted 10 lines · Stephen Hemminger
drivers/net/af_xdp/rte_eth_af_xdp.c line 929
Quoted 2 lines · On Thu, 21 Mar 2019 17:18:41 +0800 Xiaolong Ye <[email protected]> wrote:
drivers/net/af_xdp/rte_eth_af_xdp.c new file +2
-0+ +RTE_PMD_REGISTER_VDEV (eth_af_xdp, pmd_af_xdp_drv);The convention in other network drivers is to use
net_XXXin the vdev name. InAF_XDPthat would be:
RTE_PMD_REGISTER_VDEV(net_af_xdp, pmd_af_xdp_drv);Got it.
Quoted 4 lines
About naming, I would just drop
AF_from the name everywhere, the driver is about running over XDP, and the "AF_" is just a prefix for address family.
Quoted 8 lines
Quoted 2 lines
Why not:
net/xdpThanks for the advice, Actually this driver is more about
AF_XDPrathan than XDP, the foundational objects it uses such as umem, umem fill ring, umem completion ring, tx ring, rx ring which are allAF_XDPspecfic, so I would rather keep the naming.
+1 for the naming. AF_XDP is something different from XDP itself, though the former does use the latter.
/Bruce
PATCH v3 2/5 lib/mbuf: introduce helper to create mempool with flags
In reply to Xiaolong YeThis allows applications to create mbuf mempool with specific flags such as MEMPOOL_F_NO_SPREAD if they want fixed size memory objects.
--- lib/librte_mbuf/rte_mbuf.c | 29 +++++++++++++++++++----- lib/librte_mbuf/rte_mbuf.h | 45 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 69 insertions(+), 5 deletions(-)
lib/librte_mbuf/rte_mbuf.c +24 -5
@@ -106,11 +106,10 @@ rte_pktmbuf_init(struct rte_mempool *mp, m->next =NULL ; } -/* Helper to create a mbuf pool with given mempool ops name*/ -struct rte_mempool * -rte_pktmbuf_pool_create_by_ops (const char *name,unsigned int n, +static struct rte_mempool * +rte_pktmbuf_pool_create_by_ops_with_flags (const char *name,unsigned int n,unsigned int cache_size,uint16_t priv_size,uint16_t data_room_size, -int socket_id,const char *ops_name) +unsigned int flags,int socket_id,const char *ops_name) {struct rte_mempool *mp;struct rte_pktmbuf_pool_private mbp_priv;
@@ -130,7 +129,7 @@ rte_pktmbuf_pool_create_by_ops(const char *name, unsigned int n, mbp_priv.mbuf_priv_size = priv_size; mp =rte_mempool_create_empty (name, n, elt_size, cache_size, -sizeof (struct rte_pktmbuf_pool_private ), socket_id,0 ); +sizeof (struct rte_pktmbuf_pool_private ), socket_id, flags);if (mp ==NULL )return NULL ;
@@ -157,6 +156,16 @@ rte_pktmbuf_pool_create_by_ops(const char *name, unsigned int n,return mp; } +/* Helper to create a mbuf pool with given mempool ops name*/ +struct rte_mempool * +rte_pktmbuf_pool_create_by_ops (const char *name,unsigned int n, +unsigned int cache_size,uint16_t priv_size,uint16_t data_room_size, +int socket_id,const char *ops_name) +{ +return rte_pktmbuf_pool_create_by_ops_with_flags (name, n, cache_size, + priv_size, data_room_size,0 , socket_id, ops_name); +} +/* helper to create a mbuf pool */ struct rte_mempool * rte_pktmbuf_pool_create(const char *name, unsigned int n,
@@ -167,6 +176,16 @@ rte_pktmbuf_pool_create(const char *name, unsigned int n, data_room_size, socket_id, NULL); } +/* helper to create a mbuf pool with flags (e.g. NO_SPREAD) */ +struct rte_mempool * __rte_experimental +rte_pktmbuf_pool_create_with_flags (const char *name,unsigned int n, +unsigned int cache_size,uint16_t priv_size,uint16_t data_room_size, +unsigned int flags,int socket_id) +{ +return rte_pktmbuf_pool_create_by_ops_with_flags (name, n, cache_size, + priv_size, data_room_size, flags, socket_id,NULL ); +} +/* do some sanity checks on a mbuf: panic if it fails */ void rte_mbuf_sanity_check (const struct rte_mbuf *m,int is_header)
lib/librte_mbuf/rte_mbuf.h +45 -0
51 lines not on this page · raw message
Signature · 1 line
2.17.1
PATCH v3 3/5 lib/mempool: allow page size aligned mempool
In reply to Xiaolong YeAllow create a mempool with page size aligned base address.
--- lib/librte_mempool/rte_mempool.c | 3 +++ lib/librte_mempool/rte_mempool.h | 1 + 2 files changed, 4 insertions(+)
lib/librte_mempool/rte_mempool.c +3 -0
@@ -543,6 +543,9 @@ rte_mempool_populate_default(struct rte_mempool *mp)if (try_contig) flags |=RTE_MEMZONE_IOVA_CONTIG ; +if (mp->flags &MEMPOOL_F_PAGE_ALIGN ) + align =getpagesize (); + mz =rte_memzone_reserve_aligned (mz_name, mem_size, mp->socket_id, flags, align);
lib/librte_mempool/rte_mempool.h +1 -0
@@ -264,6 +264,7 @@ struct rte_mempool {#define MEMPOOL_F_POOL_CREATED 0x0010/**< Internal: pool is created. */ #define MEMPOOL_F_NO_IOVA_CONTIG 0x0020/**< Don't need IOVA contiguous objs. */ #define MEMPOOL_F_NO_PHYS_CONTIG MEMPOOL_F_NO_IOVA_CONTIG/* deprecated */ +#define MEMPOOL_F_PAGE_ALIGN 0x0040/**< Chunk's base address is page aligned */ /** * @internal When debug is enabled, store some statistics.
Signature · 1 line
2.17.1
Re: [PATCH v3 3/5] lib/mempool: allow page size aligned mempool
Quoted 24 lines
-----Original Message-----
From: dev [mailto:[email protected]] On Behalf Of Xiaolong Ye
Sent: Thursday, March 21, 2019 9:19 AM
To: [email protected]
Cc: Zhang, Qi Z <[email protected]>; Karlsson, Magnus <[email protected]>; Topel, Bjorn <[email protected]>; Ye, Xiaolong <[email protected]>
Subject: [dpdk-dev] [PATCH v3 3/5] lib/mempool: allow page size aligned mempoolAllow create a mempool with page size aligned base address.
Signed-off-by Qi Zhang <[email protected]>Signed-off-by Xiaolong Ye <[email protected]>--- lib/librte_mempool/rte_mempool.c | 3 +++ lib/librte_mempool/rte_mempool.h | 1 + 2 files changed, 4 insertions(+)lib/librte_mempool/rte_mempool.c +0
-0
lib/librte_mempool/rte_mempool.c rte_mempool_populate_default(struct rte_mempool *mp) · line 548
Quoted 6 lines
lib/librte_mempool/rte_mempool.c +3
-0@@ -543,6 +543,9 @@ rte_mempool_populate_default(struct rte_mempool *mp)if (try_contig) flags |=RTE_MEMZONE_IOVA_CONTIG ; +if (mp->flags &MEMPOOL_F_PAGE_ALIGN ) + align =getpagesize (); +
Might be a bit safer:pg_sz = getpagesize();
align = RTE_MAX(align, pg_sz);
BTW, why do you need it always default page size aligned? Is it for 'external' memory allocation or even for eal hugepages too? Konstantin
Quoted 3 lines
lib/librte_mempool/rte_mempool.c +0
-0mz =rte_memzone_reserve_aligned (mz_name, mem_size, mp->socket_id, flags, align);
Re: [PATCH v3 3/5] lib/mempool: allow page size aligned mempool
Quoted 51 lines
-----Original Message-----
From: Ananyev, Konstantin
Sent: Thursday, March 21, 2019 10:01 PM
To: Ye, Xiaolong <[email protected]>; [email protected]
Cc: Zhang, Qi Z <[email protected]>; Karlsson, Magnus <[email protected]>; Topel, Bjorn <[email protected]>; Ye, Xiaolong <[email protected]>
Subject: RE: [dpdk-dev] [PATCH v3 3/5] lib/mempool: allow page size aligned mempoollib/librte_mempool/rte_mempool.c rte_mempool_populate_default(struct rte_mempool
Quoted 25 lines
-----Original Message-----
From: dev [mailto:[email protected]] On Behalf Of Xiaolong Ye
Sent: Thursday, March 21, 2019 9:19 AM
To: [email protected]
Cc: Zhang, Qi Z <[email protected]>; Karlsson, Magnus <[email protected]>; Topel, Bjorn <[email protected]>; Ye, Xiaolong <[email protected]>
Subject: [dpdk-dev] [PATCH v3 3/5] lib/mempool: allow page size aligned mempoolAllow create a mempool with page size aligned base address.
Signed-off-by Qi Zhang <[email protected]>Signed-off-by Xiaolong Ye <[email protected]>--- lib/librte_mempool/rte_mempool.c | 3 +++ lib/librte_mempool/rte_mempool.h | 1 + 2 files changed, 4 insertions(+)lib/librte_mempool/rte_mempool.c +0
-0@@ -543,6 +543,9 @@ rte_mempool_populate_default(struct rte_mempool*mp)
lib/librte_mempool/rte_mempool.c rte_mempool_populate_default(struct rte_mempool *mp) · line 548
Quoted 6 lines
lib/librte_mempool/rte_mempool.c +3
-0if (try_contig) flags |=RTE_MEMZONE_IOVA_CONTIG ; +if (mp->flags &MEMPOOL_F_PAGE_ALIGN ) + align =getpagesize (); +Might be a bit safer:
pg_sz=getpagesize();
align =RTE_MAX(align, pg_sz);
Quoted 2 lines
BTW, why do you need it always default page size aligned? Is it for 'external' memory allocation or even for eal hugepages too?
this help us to enable zero copy between xdp umem to mbuf. af_xdp umem require 2K chunk size and also aligned on 2K address,
Qi
Quoted 5 lines
Konstantin
Quoted 3 lines
lib/librte_mempool/rte_mempool.c +0
-0mz =rte_memzone_reserve_aligned (mz_name, mem_size, mp->socket_id, flags, align);
PATCH v3 4/5 net/af_xdp: use mbuf mempool for buffer management
In reply to Xiaolong YeNow, af_xdp registered memory buffer is managed by rte_mempool. mbuf be allocated from rte_mempool can be convert to xdp_desc's address and vice versa.
--- drivers/net/af_xdp/rte_eth_af_xdp.c | 123 +++++++++++++++++----------- 1 file changed, 77 insertions(+), 46 deletions(-)
drivers/net/af_xdp/rte_eth_af_xdp.c +77 -46
@@ -43,7 +43,11 @@#define ETH_AF_XDP_FRAME_SIZE XSK_UMEM__DEFAULT_FRAME_SIZE#define ETH_AF_XDP_NUM_BUFFERS 4096 -#define ETH_AF_XDP_DATA_HEADROOM 0 +/* mempool hdrobj size (64 bytes) + sizeof(struct rte_mbuf) (128 bytes) */ +#define ETH_AF_XDP_MBUF_OVERHEAD 192 +/* data start from offset 320 (192 + 128) bytes */ +#define ETH_AF_XDP_DATA_HEADROOM \ + (ETH_AF_XDP_MBUF_OVERHEAD + RTE_PKTMBUF_HEADROOM)#define ETH_AF_XDP_DFLT_NUM_DESCS XSK_RING_CONS__DEFAULT_NUM_DESCS#define ETH_AF_XDP_DFLT_QUEUE_IDX 0
@@ -56,7 +60,7 @@ struct xsk_umem_info {struct xsk_ring_prod fq;struct xsk_ring_cons cq;struct xsk_umem *umem; -struct rte_ring *buf_ring; +struct rte_mempool *mb_pool;void *buffer; };
@@ -118,12 +122,32 @@ static struct rte_eth_link pmd_link = { .link_autoneg = ETH_LINK_AUTONEG }; +static inline struct rte_mbuf * +addr_to_mbuf (struct xsk_umem_info *umem,uint64_t addr) +{ +uint64_t offset = (addr /ETH_AF_XDP_FRAME_SIZE * +ETH_AF_XDP_FRAME_SIZE ); +struct rte_mbuf *mbuf = (struct rte_mbuf *)((uint64_t )umem->buffer + + offset +ETH_AF_XDP_MBUF_OVERHEAD - +sizeof (struct rte_mbuf )); + mbuf->data_off = addr - offset -ETH_AF_XDP_MBUF_OVERHEAD ; +return mbuf; +} + +static inline uint64_t +mbuf_to_addr (struct xsk_umem_info *umem,struct rte_mbuf *mbuf) +{ +return (uint64_t )mbuf->buf_addr + mbuf->data_off - + (uint64_t )umem->buffer; +} +static inline int reserve_fill_queue (struct xsk_umem_info *umem,int reserve_size) {struct xsk_ring_prod *fq = &umem->fq; +struct rte_mbuf *mbuf;uint32_t idx; -void *addr =NULL ; +uint64_t addr;int i, ret; ret =xsk_ring_prod__reserve (fq, reserve_size, &idx);
@@ -134,12 +158,17 @@ reserve_fill_queue(struct xsk_umem_info *umem, int reserve_size)for (i =0 ; i < reserve_size; i++) {__u64 *fq_addr; -rte_ring_dequeue (umem->buf_ring, &addr); + mbuf =rte_pktmbuf_alloc (umem->mb_pool); +if (mbuf ==NULL ) { + i--; +break ; + } + addr =mbuf_to_addr (umem, mbuf); fq_addr =xsk_ring_prod__fill_addr (fq, idx++); - *fq_addr = (uint64_t )addr; + *fq_addr = addr; } -xsk_ring_prod__submit (fq, reserve_size); +xsk_ring_prod__submit (fq, i);return 0 ; }
@@ -189,7 +218,7 @@ eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) }else { dropped++; } -rte_ring_enqueue (umem->buf_ring, (void *)addr); +rte_pktmbuf_free (addr_to_mbuf (umem, addr)); }xsk_ring_cons__release (rx, rcvd);
@@ -213,7 +242,7 @@ static void pull_umem_cq(struct xsk_umem_info *umem, int size)for (i =0 ; i < n; i++) {uint64_t addr; addr = *xsk_ring_cons__comp_addr (cq, idx_cq++); -rte_ring_enqueue (umem->buf_ring, (void *)addr); +rte_pktmbuf_free (addr_to_mbuf (umem, addr)); }xsk_ring_cons__release (cq, n);
@@ -242,7 +271,7 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)struct pkt_tx_queue *txq = queue;struct xsk_umem_info *umem = txq->pair->umem;struct rte_mbuf *mbuf; -void *addrs[ETH_AF_XDP_TX_BATCH_SIZE ]; +struct rte_mbuf *mbuf_to_tx;unsigned long tx_bytes =0 ;int i, valid =0 ;uint32_t idx_tx;
@@ -251,11 +280,6 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)pull_umem_cq (umem, nb_pkts); - nb_pkts =rte_ring_dequeue_bulk (umem->buf_ring, addrs, - nb_pkts,NULL ); -if (nb_pkts ==0 ) -return 0 ; -if (xsk_ring_prod__reserve (&txq->tx, nb_pkts, &idx_tx) != nb_pkts) {kick_tx (txq);return 0 ;
@@ -269,7 +293,12 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) desc =xsk_ring_prod__tx_desc (&txq->tx, idx_tx + i); mbuf = bufs[i];if (mbuf->pkt_len <= buf_len) { - desc->addr = (uint64_t )addrs[valid]; + mbuf_to_tx =rte_pktmbuf_alloc (umem->mb_pool); +if (mbuf_to_tx ==NULL ) { +rte_pktmbuf_free (mbuf); +continue ; + } + desc->addr =mbuf_to_addr (umem, mbuf_to_tx); desc->len = mbuf->pkt_len; pkt =xsk_umem__get_data (umem->buffer, desc->addr);
@@ -285,10 +314,6 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)kick_tx (txq); -if (valid < nb_pkts) -rte_ring_enqueue_bulk (umem->buf_ring, &addrs[valid], - nb_pkts - valid,NULL ); - txq->stats.err_pkts += nb_pkts - valid; txq->stats.tx_pkts += valid; txq->stats.tx_bytes += tx_bytes;
@@ -437,16 +462,29 @@ eth_link_update(struct rte_eth_dev *dev __rte_unused,static void xdp_umem_destroy (struct xsk_umem_info *umem) { -free (umem->buffer); - umem->buffer =NULL ; - -rte_ring_free (umem->buf_ring); - umem->buf_ring =NULL ; +rte_mempool_free (umem->mb_pool); + umem->mb_pool =NULL ;free (umem); umem =NULL ; } +static inline uint64_t get_base_addr (struct rte_mempool *mp) +{ +struct rte_mempool_memhdr *memhdr; + + memhdr =STAILQ_FIRST (&mp->mem_list); +return (uint64_t )(memhdr->addr); +} + +static inline uint64_t get_len (struct rte_mempool *mp) +{ +struct rte_mempool_memhdr *memhdr; + + memhdr =STAILQ_FIRST (&mp->mem_list); +return (uint64_t )(memhdr->len); +} +static struct xsk_umem_info *xdp_umem_configure (void ) {struct xsk_umem_info *umem;
@@ -455,9 +493,8 @@ static struct xsk_umem_info *xdp_umem_configure(void) .comp_size = ETH_AF_XDP_DFLT_NUM_DESCS, .frame_size = ETH_AF_XDP_FRAME_SIZE, .frame_headroom = ETH_AF_XDP_DATA_HEADROOM }; -void *bufs =NULL ; +void *base_addr =NULL ;int ret; -uint64_t i; umem =rte_zmalloc_socket ("umem" ,sizeof (*umem),0 ,rte_socket_id ());if (umem ==NULL ) {
@@ -465,27 +502,22 @@ static struct xsk_umem_info *xdp_umem_configure(void)return NULL ; } - umem->buf_ring =rte_ring_create ("af_xdp_ring" , -ETH_AF_XDP_NUM_BUFFERS , -SOCKET_ID_ANY , -0x0 ); -if (umem->buf_ring ==NULL ) { + umem->mb_pool =rte_pktmbuf_pool_create_with_flags ("af_xdp_mempool" , +ETH_AF_XDP_NUM_BUFFERS , +250 ,0 , +ETH_AF_XDP_FRAME_SIZE - +ETH_AF_XDP_MBUF_OVERHEAD , +MEMPOOL_F_NO_SPREAD |MEMPOOL_F_PAGE_ALIGN , +SOCKET_ID_ANY ); + +if (umem->mb_pool ==NULL || umem->mb_pool->nb_mem_chunks !=1 ) {RTE_LOG (ERR ,AF_XDP , -"Failed to create rte_ring\n" ); +"Failed to create rte_mempool\n" ); goto err; } + base_addr = (void *)get_base_addr (umem->mb_pool); -for (i =0 ; i <ETH_AF_XDP_NUM_BUFFERS ; i++) -rte_ring_enqueue (umem->buf_ring, - (void *)(i *ETH_AF_XDP_FRAME_SIZE + -ETH_AF_XDP_DATA_HEADROOM )); - -if (posix_memalign (&bufs,getpagesize (), -ETH_AF_XDP_NUM_BUFFERS *ETH_AF_XDP_FRAME_SIZE )) { -RTE_LOG (ERR ,AF_XDP ,"Failed to allocate memory pool.\n" ); - goto err; - } - ret =xsk_umem__create (&umem->umem, bufs, + ret =xsk_umem__create (&umem->umem, base_addr,ETH_AF_XDP_NUM_BUFFERS *ETH_AF_XDP_FRAME_SIZE , &umem->fq, &umem->cq, &usr_config);
@@ -494,7 +526,7 @@ static struct xsk_umem_info *xdp_umem_configure(void)RTE_LOG (ERR ,AF_XDP ,"Failed to create umem" ); goto err; } - umem->buffer = bufs; + umem->buffer = base_addr;return umem;
@@ -911,8 +943,7 @@ rte_pmd_af_xdp_remove(struct rte_vdev_device *dev) internals = eth_dev->data->dev_private; -rte_ring_free (internals->umem->buf_ring); -rte_free (internals->umem->buffer); +rte_mempool_free (internals->umem->mb_pool);rte_free (internals->umem);rte_eth_dev_release_port (eth_dev);
Signature · 1 line
2.17.1
PATCH v3 5/5 net/af_xdp: enable zero copy
In reply to Xiaolong YeTry to check if external mempool (from rx_queue_setup) is fit for af_xdp, if it is, it will be registered to af_xdp socket directly and there will be no packet data copy on Rx and Tx.
--- drivers/net/af_xdp/rte_eth_af_xdp.c | 125 ++++++++++++++++++++-------- 1 file changed, 90 insertions(+), 35 deletions(-)
drivers/net/af_xdp/rte_eth_af_xdp.c +90 -35
@@ -62,6 +62,7 @@ struct xsk_umem_info {struct xsk_umem *umem;struct rte_mempool *mb_pool;void *buffer; +uint8_t zc; };struct rx_stats {
@@ -80,6 +81,7 @@ struct pkt_rx_queue {struct pkt_tx_queue *pair;uint16_t queue_idx; +uint8_t zc; };struct tx_stats {
@@ -208,17 +210,25 @@ eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) len = desc->len; pkt =xsk_umem__get_data (rxq->umem->buffer, addr); - mbuf =rte_pktmbuf_alloc (rxq->mb_pool); -if (mbuf !=NULL ) { - rte_memcpy(rte_pktmbuf_mtod(mbuf, void *), pkt, len); +if (rxq->zc) { + mbuf =addr_to_mbuf (rxq->umem, addr);rte_pktmbuf_pkt_len (mbuf) = len;rte_pktmbuf_data_len (mbuf) = len; rx_bytes += len; bufs[count++] = mbuf; }else { - dropped++; + mbuf =rte_pktmbuf_alloc (rxq->mb_pool); +if (mbuf !=NULL ) { + memcpy(rte_pktmbuf_mtod(mbuf, void*), pkt, len); +rte_pktmbuf_pkt_len (mbuf) = len; +rte_pktmbuf_data_len (mbuf) = len; + rx_bytes += len; + bufs[count++] = mbuf; + }else { + dropped++; + } +rte_pktmbuf_free (addr_to_mbuf (umem, addr)); } -rte_pktmbuf_free (addr_to_mbuf (umem, addr)); }xsk_ring_cons__release (rx, rcvd);
@@ -292,22 +302,29 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) -ETH_AF_XDP_DATA_HEADROOM ; desc =xsk_ring_prod__tx_desc (&txq->tx, idx_tx + i); mbuf = bufs[i]; -if (mbuf->pkt_len <= buf_len) { - mbuf_to_tx =rte_pktmbuf_alloc (umem->mb_pool); -if (mbuf_to_tx ==NULL ) { -rte_pktmbuf_free (mbuf); -continue ; - } - desc->addr =mbuf_to_addr (umem, mbuf_to_tx); +if (txq->pair->zc && mbuf->pool == umem->mb_pool) { + desc->addr =mbuf_to_addr (umem, mbuf); desc->len = mbuf->pkt_len; - pkt =xsk_umem__get_data (umem->buffer, - desc->addr); - rte_memcpy(pkt, rte_pktmbuf_mtod(mbuf, void *), - desc->len); valid++; tx_bytes += mbuf->pkt_len; + }else { +if (mbuf->pkt_len <= buf_len) { + mbuf_to_tx =rte_pktmbuf_alloc (umem->mb_pool); +if (!mbuf_to_tx) { +rte_pktmbuf_free (mbuf); +continue ; + } + desc->addr =mbuf_to_addr (umem, mbuf_to_tx); + desc->len = mbuf->pkt_len; + pkt =xsk_umem__get_data (umem->buffer, + desc->addr); + memcpy(pkt, rte_pktmbuf_mtod(mbuf, void *), + desc->len); + valid++; + tx_bytes += mbuf->pkt_len; + } +rte_pktmbuf_free (mbuf); } -rte_pktmbuf_free (mbuf); }xsk_ring_prod__submit (&txq->tx, nb_pkts);
@@ -485,7 +502,7 @@ static inline uint64_t get_len(struct rte_mempool *mp)return (uint64_t )(memhdr->len); } -static struct xsk_umem_info *xdp_umem_configure (void ) +static struct xsk_umem_info *xdp_umem_configure (struct rte_mempool *mb_pool) {struct xsk_umem_info *umem;struct xsk_umem_config usr_config = {
@@ -502,18 +519,23 @@ static struct xsk_umem_info *xdp_umem_configure(void)return NULL ; } - umem->mb_pool =rte_pktmbuf_pool_create_with_flags ("af_xdp_mempool" , -ETH_AF_XDP_NUM_BUFFERS , -250 ,0 , -ETH_AF_XDP_FRAME_SIZE - -ETH_AF_XDP_MBUF_OVERHEAD , -MEMPOOL_F_NO_SPREAD |MEMPOOL_F_PAGE_ALIGN , -SOCKET_ID_ANY ); - -if (umem->mb_pool ==NULL || umem->mb_pool->nb_mem_chunks !=1 ) { -RTE_LOG (ERR ,AF_XDP , -"Failed to create rte_mempool\n" ); - goto err; +if (!mb_pool) { + umem->mb_pool =rte_pktmbuf_pool_create_with_flags ("af_xdp_mempool" , +ETH_AF_XDP_NUM_BUFFERS , +250 ,0 , +ETH_AF_XDP_FRAME_SIZE - +ETH_AF_XDP_MBUF_OVERHEAD , +MEMPOOL_F_NO_SPREAD |MEMPOOL_F_PAGE_ALIGN , +SOCKET_ID_ANY ); + +if (umem->mb_pool !=NULL || + umem->mb_pool->nb_mem_chunks !=1 ) { +RTE_LOG (ERR ,AF_XDP ,"Failed to create rte_mempool\n" ); + goto err; + } + }else { + umem->mb_pool = mb_pool; + umem->zc =1 ; } base_addr = (void *)get_base_addr (umem->mb_pool);
@@ -535,16 +557,43 @@ static struct xsk_umem_info *xdp_umem_configure(void)return NULL ; } +static uint8_t +check_mempool_zc (struct rte_mempool *mp) +{ +RTE_ASSERT (mp); + +/* must continues */ +if (mp->nb_mem_chunks >1 ) +return 0 ; + +/* check header size */ +if (mp->header_size !=RTE_CACHE_LINE_SIZE ) +return 0 ; + +/* check base address */ +if ((uint64_t )get_base_addr (mp) %getpagesize () !=0 ) +return 0 ; + +/* check chunk size */ +if ((mp->elt_size + mp->header_size + mp->trailer_size) % +ETH_AF_XDP_FRAME_SIZE !=0 ) +return 0 ; + +return 1 ; +} +static int xsk_configure (struct pmd_internals *internals,struct pkt_rx_queue *rxq, -int ring_size) +int ring_size,struct rte_mempool *mb_pool) {struct xsk_socket_config cfg;struct pkt_tx_queue *txq = rxq->pair; +struct rte_mempool *mp;int ret =0 ;int reserve_size; - rxq->umem =xdp_umem_configure (); + mp =check_mempool_zc (mb_pool) ? mb_pool :NULL ; + rxq->umem =xdp_umem_configure (mp);if (rxq->umem ==NULL ) { ret = -ENOMEM ; goto err;
@@ -631,15 +680,21 @@ eth_rx_queue_setup(struct rte_eth_dev *dev, rxq->mb_pool = mb_pool; -if (xsk_configure (internals, rxq, nb_rx_desc)) { -RTE_LOG (ERR ,AF_XDP , -"Failed to configure xdp socket\n" ); +if (xsk_configure (internals, rxq, nb_rx_desc, mb_pool)) { +RTE_LOG (ERR ,AF_XDP ,"Failed to configure xdp socket\n" ); ret = -EINVAL ; goto err; } internals->umem = rxq->umem; +if (mb_pool == internals->umem->mb_pool) + rxq->zc = internals->umem->zc; + +if (rxq->zc) +RTE_LOG (INFO ,AF_XDP , +"zero copy enabled on rx queue %d\n" , rx_queue_id); + dev->data->rx_queues[rx_queue_id] = rxq;return 0 ;
Signature · 1 line
2.17.1
v4 cover Introduce AF_XDP PMD
In reply to Xiaolong YeOverview
========
This patchset adds a new PMD driver for AF_XDP which is a proposed faster version of AF_PACKET interface in Linux, see below links [1] [2] for details of AF_XDP introduction:
AF_XDP roadmap
==============
- AF_XDP is included in upstream kernel since 4.18, and AF_XDP support in libbpf has been merged in v5.1-rc1.
- Now i40e and ixgbe drivers have supported zero copy mode.
Change logs
===========
V4:
- change vdev name to net_af_xdp
- adopt dynamic log type for all logging
- Fix other style issues raised by Stephen
V3:
- Fix all style issues pointed by Stephen, Mattias, David.
- Drop the testpmd change as we'll adopt Jerin's suggestion to add a new mempool driver to handle the application use of AF_XDP's zero copy feature.
V2:
- Fix a NULL pointer reference crash issue
- Fix txonly stop sending traffic in zc mode
- Refactor rte_mbuf.c to avoid ABI breakage.
- Fix multiple style issues pointed by Ferruh, David, Stephen, Luca.
changes vs RFC sent by Qi last Aug:
- Re-work base on AF_XDP's interface changes since the new libbpf has provided higher-level APIs that hide many of the details of the AP_XDP uapi. After the rework, it helps to reduce 300+ lines of code.
- multi-queues is not supported due to current libbpf doesn't support multi-sockets on a single umem.
- No extra steps to load xdp program manually, since the current behavior of libbpf would load a default xdp program when user calls xsk_socket__create. userspace application only needs to handle the cleanup.
How to try
==========
1. take the kernel >= v5.1-rc1, build kernel and replace your host kernel with it.
make sure you turn on XDP sockets when compiling
Networking support -->
Networking options -->
[ * ] XDP sockets
2. build libbpf in tools/lib/bpf, and copy the libbpf.a and libbpf.so to /usr/lib64
cd tools/lib/bpf
make
3. ethtool -L enp59s0f0 combined 1
4. start testpmd
./build/app/testpmd -c 0xc -n 4 --vdev net_af_xdp,iface=enp59s0f0,queue=0 -- -i --rxq=1 --txq=1
in this case, default xdp program will be loaded and linked to queue 0 of enp59s0f0, network traffics travel to queue 0 will be redirected to af_xdp socket.
Xiaolong Ye (5):
net/af_xdp: introduce AF XDP PMD driver lib/mbuf: introduce helper to create mempool with flags lib/mempool: allow page size aligned mempool net/af_xdp: use mbuf mempool for buffer management net/af_xdp: enable zero copy
MAINTAINERS | 6 + config/common_base | 5 + config/common_linux | 1 + doc/guides/nics/af_xdp.rst | 45 + doc/guides/nics/features/af_xdp.ini | 11 + doc/guides/nics/index.rst | 1 + doc/guides/rel_notes/release_19_05.rst | 7 + drivers/net/Makefile | 1 + drivers/net/af_xdp/Makefile | 32 + drivers/net/af_xdp/meson.build | 21 + drivers/net/af_xdp/rte_eth_af_xdp.c | 1028 +++++++++++++++++ drivers/net/af_xdp/rte_pmd_af_xdp_version.map | 3 + drivers/net/meson.build | 1 + lib/librte_mbuf/rte_mbuf.c | 29 +- lib/librte_mbuf/rte_mbuf.h | 45 + lib/librte_mempool/rte_mempool.c | 3 + lib/librte_mempool/rte_mempool.h | 1 + mk/rte.app.mk | 1 + 18 files changed, 1236 insertions(+), 5 deletions(-) create mode 100644 doc/guides/nics/af_xdp.rst create mode 100644 doc/guides/nics/features/af_xdp.ini create mode 100644 drivers/net/af_xdp/Makefile create mode 100644 drivers/net/af_xdp/meson.build create mode 100644 drivers/net/af_xdp/rte_eth_af_xdp.c create mode 100644 drivers/net/af_xdp/rte_pmd_af_xdp_version.map
Signature · 1 line
2.17.1
PATCH v4 1/5 net/af_xdp: introduce AF XDP PMD driver
Add a new PMD driver for AF_XDP which is a proposed faster version of AF_PACKET interface in Linux. More info about AF_XDP, please refer to [1] [2].
This is the vanilla version PMD which just uses a raw buffer registered as the umem.
[1] https://fosdem.org/2018/schedule/event/af_xdp/
[2] https://lwn.net/Articles/745934/
--- MAINTAINERS | 6 + config/common_base | 5 + config/common_linux | 1 + doc/guides/nics/af_xdp.rst | 45 + doc/guides/nics/features/af_xdp.ini | 11 + doc/guides/nics/index.rst | 1 + doc/guides/rel_notes/release_19_05.rst | 7 + drivers/net/Makefile | 1 + drivers/net/af_xdp/Makefile | 32 + drivers/net/af_xdp/meson.build | 21 + drivers/net/af_xdp/rte_eth_af_xdp.c | 940 ++++++++++++++++++ drivers/net/af_xdp/rte_pmd_af_xdp_version.map | 3 + drivers/net/meson.build | 1 + mk/rte.app.mk | 1 + 14 files changed, 1075 insertions(+) create mode 100644 doc/guides/nics/af_xdp.rst create mode 100644 doc/guides/nics/features/af_xdp.ini create mode 100644 drivers/net/af_xdp/Makefile create mode 100644 drivers/net/af_xdp/meson.build create mode 100644 drivers/net/af_xdp/rte_eth_af_xdp.c create mode 100644 drivers/net/af_xdp/rte_pmd_af_xdp_version.map
MAINTAINERS +6 -0
@@ -468,6 +468,12 @@ M: John W. Linville <[email protected]> F: drivers/net/af_packet/ F: doc/guides/nics/features/afpacket.ini +Linux AF_XDP +M: Xiaolong Ye <[email protected]> +M: Qi Zhang <[email protected]> +F: drivers/net/af_xdp/ +F: doc/guides/nics/features/af_xdp.rst + Amazon ENA M: Marcin Wojtas <[email protected]> M: Michal Krawczyk <[email protected]>
config/common_base +5 -0
@@ -416,6 +416,11 @@ CONFIG_RTE_LIBRTE_VMXNET3_DEBUG_TX_FREE=n # CONFIG_RTE_LIBRTE_PMD_AF_PACKET=n +# +# Compile software PMD backed by AF_XDP sockets (Linux only) +# +CONFIG_RTE_LIBRTE_PMD_AF_XDP=n + # # Compile link bonding PMD library #
config/common_linux +1 -0
@@ -19,6 +19,7 @@ CONFIG_RTE_LIBRTE_VHOST_POSTCOPY=n CONFIG_RTE_LIBRTE_PMD_VHOST=y CONFIG_RTE_LIBRTE_IFC_PMD=y CONFIG_RTE_LIBRTE_PMD_AF_PACKET=y +CONFIG_RTE_LIBRTE_PMD_AF_XDP=y CONFIG_RTE_LIBRTE_PMD_SOFTNIC=y CONFIG_RTE_LIBRTE_PMD_TAP=y CONFIG_RTE_LIBRTE_AVP_PMD=y
doc/guides/nics/af_xdp.rst new file +45 -0
45 lines not on this page · raw message
doc/guides/nics/features/af_xdp.ini new file +11 -0
@@ -0,0 +1,11 @@ +; +; Supported features of the 'af_xdp' network poll mode driver. +; +; Refer to default.ini for the full list of available PMD features. +; +[Features] +Link status = Y +MTU update = Y +Promiscuous mode = Y +Stats per queue = Y +x86-64 = Y
doc/guides/nics/index.rst +1 -0
@@ -12,6 +12,7 @@ Network Interface Controller Drivers features build_and_test af_packet + af_xdp ark atlantic avp
doc/guides/rel_notes/release_19_05.rst +7 -0
@@ -65,6 +65,13 @@ New Features process. * Added support for Rx packet types list in a secondary process. +* **Added the AF_XDP PMD.** + + Added a Linux-specific PMD driver for AF_XDP, it can create the AF_XDP socket + and bind it to a specific netdev queue, it allows a DPDK application to send + and receive raw packets through the socket which would bypass the kernel + network stack to achieve high performance packet processing. + * **Updated Mellanox drivers.** New features and improvements were done in mlx4 and mlx5 PMDs:
drivers/net/Makefile +1 -0
@@ -9,6 +9,7 @@ ifeq ($(CONFIG_RTE_LIBRTE_THUNDERX_NICVF_PMD),d) endif DIRS-$(CONFIG_RTE_LIBRTE_PMD_AF_PACKET) += af_packet +DIRS-$(CONFIG_RTE_LIBRTE_PMD_AF_XDP) += af_xdp DIRS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += ark DIRS-$(CONFIG_RTE_LIBRTE_ATLANTIC_PMD) += atlantic DIRS-$(CONFIG_RTE_LIBRTE_AVP_PMD) += avp
drivers/net/af_xdp/Makefile new file +32 -0
@@ -0,0 +1,32 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Intel Corporation + +include $(RTE_SDK)/mk/rte.vars.mk + +# +# library name +# +LIB = librte_pmd_af_xdp.a + +EXPORT_MAP := rte_pmd_af_xdp_version.map + +LIBABIVER := 1 + +CFLAGS += -O3 + +# require kernel version >= v5.1-rc1 +CFLAGS += -I$(RTE_KERNELDIR)/tools/include +CFLAGS += -I$(RTE_KERNELDIR)/tools/lib/bpf + +CFLAGS += $(WERROR_FLAGS) +LDLIBS += -lrte_eal -lrte_mbuf -lrte_mempool -lrte_ring +LDLIBS += -lrte_ethdev -lrte_net -lrte_kvargs +LDLIBS += -lrte_bus_vdev +LDLIBS += -lbpf + +# +# all source are stored in SRCS-y +# +SRCS-$(CONFIG_RTE_LIBRTE_PMD_AF_XDP) += rte_eth_af_xdp.c + +include $(RTE_SDK)/mk/rte.lib.mk
drivers/net/af_xdp/meson.build new file +21 -0
@@ -0,0 +1,21 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Intel Corporation + +if host_machine.system() != 'linux' + build = false +endif + +bpf_dep = dependency('libbpf', required: false) +if bpf_dep.found() + build = true +else + bpf_dep = cc.find_library('libbpf', required: false) + if bpf_dep.found() and cc.has_header('xsk.h', dependencies: bpf_dep) + build = true + pkgconfig_extra_libs += '-lbpf' + else + build = false + endif +endif +sources = files('rte_eth_af_xdp.c') +ext_deps += bpf_dep
drivers/net/af_xdp/rte_eth_af_xdp.c new file +940 -0
940 lines not on this page · raw message
drivers/net/af_xdp/rte_pmd_af_xdp_version.map new file +3 -0
@@ -0,0 +1,3 @@ +DPDK_19.05 { + local: *; +};
drivers/net/meson.build +1 -0
@@ -2,6 +2,7 @@ # Copyright(c) 2017 Intel Corporation drivers = ['af_packet', + 'af_xdp', 'ark', 'atlantic', 'avp',
mk/rte.app.mk +1 -0
@@ -143,6 +143,7 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_DPAA2_MEMPOOL) += -lrte_mempool_dpaa2 endif _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AF_PACKET) += -lrte_pmd_af_packet +_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AF_XDP) += -lrte_pmd_af_xdp -lelf -lbpf _LDLIBS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += -lrte_pmd_ark _LDLIBS-$(CONFIG_RTE_LIBRTE_ATLANTIC_PMD) += -lrte_pmd_atlantic _LDLIBS-$(CONFIG_RTE_LIBRTE_AVP_PMD) += -lrte_pmd_avp
Signature · 1 line
2.17.1
Re: [PATCH v4 1/5] net/af_xdp: introduce AF XDP PMD driver
Quoted 34 lines · Xiaolong Ye
Add a new PMD driver for
AF_XDPwhich is a proposed faster version ofAF_PACKETinterface in Linux. More info aboutAF_XDP, please refer to [1] [2].This is the vanilla version PMD which just uses a raw buffer registered as the umem.
[1] https://fosdem.org/2018/schedule/event/af_xdp/
[2] https://lwn.net/Articles/745934/Signed-off-by Xiaolong Ye <[email protected]>--- MAINTAINERS | 6 + config/common_base | 5 + config/common_linux | 1 + doc/guides/nics/af_xdp.rst | 45 + doc/guides/nics/features/af_xdp.ini | 11 + doc/guides/nics/index.rst | 1 + doc/guides/rel_notes/release_19_05.rst | 7 + drivers/net/Makefile | 1 + drivers/net/af_xdp/Makefile | 32 + drivers/net/af_xdp/meson.build | 21 + drivers/net/af_xdp/rte_eth_af_xdp.c | 940 ++++++++++++++++++ drivers/net/af_xdp/rte_pmd_af_xdp_version.map | 3 + drivers/net/meson.build | 1 + mk/rte.app.mk | 1 + 14 files changed, 1075 insertions(+)create mode 100644
doc/guides/nics/af_xdp.rst
create mode 100644 doc/guides/nics/features/af_xdp.ini create mode 100644drivers/net/af_xdp/Makefile
create mode 100644drivers/net/af_xdp/meson.buildcreate mode 100644drivers/net/af_xdp/rte_eth_af_xdp.ccreate mode 100644drivers/net/af_xdp/rte_pmd_af_xdp_version.map
...
Quoted 21 lines
config/common_base +5
-0@@ -416,6 +416,11 @@ CONFIG_RTE_LIBRTE_VMXNET3_DEBUG_TX_FREE=n # CONFIG_RTE_LIBRTE_PMD_AF_PACKET=n +# +# Compile software PMD backed by AF_XDP sockets (Linux only) +# +CONFIG_RTE_LIBRTE_PMD_AF_XDP=n + # # Compile link bonding PMD library #config/common_linux +0
-0
config/common_linux CONFIG_RTE_LIBRTE_VHOST_POSTCOPY=n · line 22
Quoted 4 lines
config/common_linux +1
-0@@ -19,6 +19,7 @@ CONFIG_RTE_LIBRTE_VHOST_POSTCOPY=n CONFIG_RTE_LIBRTE_PMD_VHOST=y CONFIG_RTE_LIBRTE_IFC_PMD=y CONFIG_RTE_LIBRTE_PMD_AF_PACKET=y +CONFIG_RTE_LIBRTE_PMD_AF_XDP=y
It has to be disabled by default as it requires headers from kernels more recent than minimum kernel version supported by DPDK.
Quoted 3 lines
config/common_linux +0
-0CONFIG_RTE_LIBRTE_PMD_SOFTNIC=y CONFIG_RTE_LIBRTE_PMD_TAP=y CONFIG_RTE_LIBRTE_AVP_PMD=y
...
Quoted 15 lines
drivers/net/af_xdp/meson.build new file +9
-0@@ -0,0 +1,21 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Intel Corporation + +if host_machine.system() != 'linux' + build = false +endif + +bpf_dep = dependency('libbpf', required: false) +if bpf_dep.found()
drivers/net/af_xdp/meson.build line 19
Quoted 10 lines
drivers/net/af_xdp/meson.build new file +10
-0+ build = true +else + bpf_dep = cc.find_library('libbpf', required: false) + if bpf_dep.found() and cc.has_header('xsk.h', dependencies: bpf_dep) + build = true + pkgconfig_extra_libs += '-lbpf' + else + build = false + endif +endif
I think you need to add more checks, as above does not cover linux/if_xdp.h header IIUC.
Quoted 133 lines
drivers/net/af_xdp/meson.build new file +2
-0+sources = files('rte_eth_af_xdp.c') +ext_deps += bpf_depdrivers/net/af_xdp/rte_eth_af_xdp.c new file +125
-0@@ -0,0 +1,940 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2019 Intel Corporation. +*/ + +#include <rte_mbuf.h> +#include <rte_ethdev_driver.h> +#include <rte_ethdev_vdev.h> +#include <rte_malloc.h> +#include <rte_kvargs.h> +#include <rte_bus_vdev.h> +#include <rte_string_fns.h> + +#include <linux/if_ether.h> +#include <linux/if_xdp.h> +#include <linux/if_link.h> +#include <asm/barrier.h> +#include <arpa/inet.h> +#include <net/if.h> +#include <sys/types.h> +#include <sys/socket.h> +#include <sys/ioctl.h> +#include <sys/mman.h> +#include <unistd.h> +#include <poll.h> +#include <bpf/bpf.h> +#include <xsk.h> + +#ifndef SOL_XDP +#define SOL_XDP 283 +#endif + +#ifndef AF_XDP +#define AF_XDP 44 +#endif + +#ifndef PF_XDP +#define PF_XDP AF_XDP +#endif + +static int af_xdp_logtype; + +#define AF_XDP_LOG(level, fmt, args...) \ + rte_log(RTE_LOG_ ## level, af_xdp_logtype, \ + "%s(): " fmt "\n", __func__, ##args) + +#define ETH_AF_XDP_IFACE_ARG "iface" +#define ETH_AF_XDP_QUEUE_IDX_ARG "queue" + +#define ETH_AF_XDP_FRAME_SIZE XSK_UMEM__DEFAULT_FRAME_SIZE +#define ETH_AF_XDP_NUM_BUFFERS 4096 +#define ETH_AF_XDP_DATA_HEADROOM 0 +#define ETH_AF_XDP_DFLT_NUM_DESCS XSK_RING_CONS__DEFAULT_NUM_DESCS +#define ETH_AF_XDP_DFLT_QUEUE_IDX 0 + +#define ETH_AF_XDP_RX_BATCH_SIZE 32 +#define ETH_AF_XDP_TX_BATCH_SIZE 32 + +#define ETH_AF_XDP_MAX_QUEUE_PAIRS 16 + +struct xsk_umem_info { +struct xsk_ring_prod fq; +struct xsk_ring_cons cq; +struct xsk_umem *umem; +struct rte_ring *buf_ring; +void *buffer; +}; + +struct rx_stats { +uint64_t rx_pkts; +uint64_t rx_bytes; +uint64_t rx_dropped; +}; + +struct pkt_rx_queue { +struct xsk_ring_cons rx; +struct xsk_umem_info *umem; +struct xsk_socket *xsk; +struct rte_mempool *mb_pool; + +struct rx_stats stats; + +struct pkt_tx_queue *pair; +uint16_t queue_idx; +}; + +struct tx_stats { +uint64_t tx_pkts; +uint64_t err_pkts; +uint64_t tx_bytes; +}; + +struct pkt_tx_queue { +struct xsk_ring_prod tx; + +struct tx_stats stats; + +struct pkt_rx_queue *pair; +uint16_t queue_idx; +}; + +struct pmd_internals { +int if_index; +char if_name[IFNAMSIZ ]; +uint16_t queue_idx; +struct ether_addr eth_addr; +struct xsk_umem_info *umem; +struct rte_mempool *mb_pool_share; + +struct pkt_rx_queue rx_queues[ETH_AF_XDP_MAX_QUEUE_PAIRS ]; +struct pkt_tx_queue tx_queues[ETH_AF_XDP_MAX_QUEUE_PAIRS ]; +}; + +static const char *const valid_arguments[] = { +ETH_AF_XDP_IFACE_ARG , +ETH_AF_XDP_QUEUE_IDX_ARG , +NULL +}; + +static struct rte_eth_link pmd_link = { + .link_speed =ETH_SPEED_NUM_10G , + .link_duplex =ETH_LINK_FULL_DUPLEX , + .link_status =ETH_LINK_DOWN , + .link_autoneg =ETH_LINK_AUTONEG +}; +
drivers/net/af_xdp/rte_eth_af_xdp.c line 135
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+static inline int +reserve_fill_queue (struct xsk_umem_info *umem,int reserve_size) +{ +struct xsk_ring_prod *fq = &umem->fq; +uint32_t idx; +void *addr =NULL ; +int i, ret; + + ret =xsk_ring_prod__reserve (fq, reserve_size, &idx); +if (!ret) {
You could use unlikely() here as it is called in the hot path.
drivers/net/af_xdp/rte_eth_af_xdp.c line 141
Quoted 6 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +6
-0+AF_XDP_LOG (ERR ,"Failed to reserve enough fq descs.\n" ); +return ret; + } + +for (i =0 ; i < reserve_size; i++) { +__u64 *fq_addr;
For consistency, you could either declare addr here, or fq_addr at the top of the function.
Quoted 28 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +28
-0+if (rte_ring_dequeue (umem->buf_ring, &addr)) { + i--; +break ; + } + fq_addr =xsk_ring_prod__fill_addr (fq, idx++); + *fq_addr = (uint64_t )addr; + } + +xsk_ring_prod__submit (fq, i); + +return 0 ; +} + +static uint16_t +eth_af_xdp_rx (void *queue,struct rte_mbuf **bufs,uint16_t nb_pkts) +{ +struct pkt_rx_queue *rxq = queue; +struct xsk_ring_cons *rx = &rxq->rx; +struct xsk_umem_info *umem = rxq->umem; +struct xsk_ring_prod *fq = &umem->fq; +uint32_t idx_rx; +uint32_t free_thresh = fq->size >>1 ; +struct rte_mbuf *mbufs[ETH_AF_XDP_TX_BATCH_SIZE ]; +unsigned long dropped =0 ; +unsigned long rx_bytes =0 ; +uint16_t count =0 ; +int rcvd, i; +
drivers/net/af_xdp/rte_eth_af_xdp.c line 179
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+ nb_pkts =RTE_MIN (nb_pkts,ETH_AF_XDP_TX_BATCH_SIZE ); + + rcvd =xsk_ring_cons__peek (rx, nb_pkts, &idx_rx); +if (rcvd ==0 ) +return 0 ; + +if (xsk_prod_nb_free (fq, free_thresh) >= free_thresh) + (void )reserve_fill_queue (umem,ETH_AF_XDP_RX_BATCH_SIZE ); + +if (rte_pktmbuf_alloc_bulk (rxq->mb_pool, mbufs, rcvd) !=0 )
unlikely()?
Quoted 31 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +31
-0+return 0 ; + +for (i =0 ; i < rcvd; i++) { +const struct xdp_desc *desc; +uint64_t addr; +uint32_t len; +void *pkt; + + desc =xsk_ring_cons__rx_desc (rx, idx_rx++); + addr = desc->addr; + len = desc->len; + pkt =xsk_umem__get_data (rxq->umem->buffer, addr); + + rte_memcpy(rte_pktmbuf_mtod(mbufs[i], void *), pkt, len); +rte_pktmbuf_pkt_len (mbufs[i]) = len; +rte_pktmbuf_data_len (mbufs[i]) = len; + rx_bytes += len; + bufs[count++] = mbufs[i]; + +rte_ring_enqueue (umem->buf_ring, (void *)addr); + } + +xsk_ring_cons__release (rx, rcvd); + +/* statistics */ + rxq->stats.rx_pkts += (rcvd - dropped); + rxq->stats.rx_bytes += rx_bytes; + +return count; +} +
...
drivers/net/af_xdp/rte_eth_af_xdp.c line 321
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+ +/* This function gets called when the current port gets stopped. */ +static void +eth_dev_stop (struct rte_eth_dev *dev) +{ + dev->data->dev_link.link_status =ETH_LINK_DOWN ; +} + +static int +eth_dev_configure(struct rte_eth_dev *dev __rte_unused)
Remove __rte_unused.
Quoted 8 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +8
-0+{ +/* rx/tx must be paired */ +if (dev->data->nb_rx_queues != dev->data->nb_tx_queues) +return -EINVAL ; + +return 0 ; +} +
...
Quoted 4 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +4
-0+ +static int +xsk_configure (struct pmd_internals *internals,struct pkt_rx_queue *rxq, +int ring_size)
drivers/net/af_xdp/rte_eth_af_xdp.c line 523
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+{ +struct xsk_socket_config cfg; +struct pkt_tx_queue *txq = rxq->pair; +int ret =0 ; +int reserve_size; + + rxq->umem =xdp_umem_configure (); +if (rxq->umem ==NULL ) { + ret = -ENOMEM ; + goto err;
You can return directly here as umem == NULL.
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+ } + + cfg.rx_size = ring_size; + cfg.tx_size = ring_size; + cfg.libbpf_flags =0 ; + cfg.xdp_flags =XDP_FLAGS_UPDATE_IF_NOEXIST ; + cfg.bind_flags =0 ; + ret =xsk_socket__create (&rxq->xsk, internals->if_name, + internals->queue_idx, rxq->umem->umem, &rxq->rx, + &txq->tx, &cfg);
drivers/net/af_xdp/rte_eth_af_xdp.c line 543
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+if (ret) { +AF_XDP_LOG (ERR ,"Failed to create xsk socket.\n" ); + goto err; + } + + reserve_size =ETH_AF_XDP_DFLT_NUM_DESCS /2 ; + ret =reserve_fill_queue (rxq->umem, reserve_size); +if (ret) { +AF_XDP_LOG (ERR ,"Failed to reserve fill queue.\n" ); + goto err;
Shouldn't you call xsk_socket__delete(rxq->xsk) here?
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+ } + +return 0 ; + +err: +xdp_umem_destroy (rxq->umem); + +return ret; +} +
drivers/net/af_xdp/rte_eth_af_xdp.c line 563
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+static void +queue_reset (struct pmd_internals *internals,uint16_t queue_idx) +{ +struct pkt_rx_queue *rxq = &internals->rx_queues[queue_idx]; +struct pkt_tx_queue *txq = rxq->pair; +int xsk_fd =xsk_socket__fd (rxq->xsk); + +if (xsk_fd) { +close (xsk_fd); +if (internals->umem !=NULL ) {
Moving this condition out would work and be cleaner.
Anyway, it seems to never enter this condition as internal->umem is not set yet when queue_reset() is called.
Quoted 39 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +39
-0+xdp_umem_destroy (internals->umem); + internals->umem =NULL ; + } + } +memset (rxq,0 ,sizeof (*rxq)); +memset (txq,0 ,sizeof (*txq)); + rxq->pair = txq; + txq->pair = rxq; + rxq->queue_idx = queue_idx; + txq->queue_idx = queue_idx; +} + +static int +eth_rx_queue_setup (struct rte_eth_dev *dev, +uint16_t rx_queue_id, +uint16_t nb_rx_desc, + unsigned int socket_id __rte_unused, + const struct rte_eth_rxconf *rx_conf __rte_unused, +struct rte_mempool *mb_pool) +{ +struct pmd_internals *internals = dev->data->dev_private; +uint32_t buf_size, data_size; +struct pkt_rx_queue *rxq; +int ret; + + rxq = &internals->rx_queues[rx_queue_id]; +queue_reset (internals, rx_queue_id); + +/* Now get the space available for data in the mbuf */ + buf_size =rte_pktmbuf_data_room_size (mb_pool) - +RTE_PKTMBUF_HEADROOM ; + data_size =ETH_AF_XDP_FRAME_SIZE -ETH_AF_XDP_DATA_HEADROOM ; + +if (data_size > buf_size) { +AF_XDP_LOG (ERR ,"%s: %d bytes will not fit in mbuf (%d bytes)\n" , + dev->device->name, data_size, buf_size); + ret = -ENOMEM ; + goto err; + }
drivers/net/af_xdp/rte_eth_af_xdp.c line 612
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+ + rxq->mb_pool = mb_pool; + +if (xsk_configure (internals, rxq, nb_rx_desc)) { +AF_XDP_LOG (ERR ,"Failed to configure xdp socket\n" ); + ret = -EINVAL ; + goto err; + } + + internals->umem = rxq->umem;
If my previous comment is wrong, i.e. internals->umem may be already set when queue_reset() is called, then it means you might have a leak here.
Quoted 9 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +9
-0+ + dev->data->rx_queues[rx_queue_id] = rxq; +return 0 ; + +err: +queue_reset (internals, rx_queue_id); +return ret; +} +
Re: [PATCH v4 1/5] net/af_xdp: introduce AF XDP PMD driver
Quoted 59 lines · Maxime Coquelin
Quoted 34 lines · Xiaolong Ye
Add a new PMD driver for
AF_XDPwhich is a proposed faster version ofAF_PACKETinterface in Linux. More info aboutAF_XDP, please refer to [1] [2].This is the vanilla version PMD which just uses a raw buffer registered as the umem.
[1] https://fosdem.org/2018/schedule/event/af_xdp/
[2] https://lwn.net/Articles/745934/Signed-off-by Xiaolong Ye <[email protected]>--- MAINTAINERS | 6 + config/common_base | 5 + config/common_linux | 1 + doc/guides/nics/af_xdp.rst | 45 + doc/guides/nics/features/af_xdp.ini | 11 + doc/guides/nics/index.rst | 1 + doc/guides/rel_notes/release_19_05.rst | 7 + drivers/net/Makefile | 1 + drivers/net/af_xdp/Makefile | 32 + drivers/net/af_xdp/meson.build | 21 + drivers/net/af_xdp/rte_eth_af_xdp.c | 940 ++++++++++++++++++ drivers/net/af_xdp/rte_pmd_af_xdp_version.map | 3 + drivers/net/meson.build | 1 + mk/rte.app.mk | 1 + 14 files changed, 1075 insertions(+)create mode 100644
doc/guides/nics/af_xdp.rst
create mode 100644 doc/guides/nics/features/af_xdp.ini create mode 100644drivers/net/af_xdp/Makefile
create mode 100644drivers/net/af_xdp/meson.buildcreate mode 100644drivers/net/af_xdp/rte_eth_af_xdp.ccreate mode 100644drivers/net/af_xdp/rte_pmd_af_xdp_version.map...
Quoted 19 lines
config/common_base +5
-0@@ -416,6 +416,11 @@ CONFIG_RTE_LIBRTE_VMXNET3_DEBUG_TX_FREE=n # CONFIG_RTE_LIBRTE_PMD_AF_PACKET=n +# +# Compile software PMD backed by AF_XDP sockets (Linux only) +# +CONFIG_RTE_LIBRTE_PMD_AF_XDP=n + # # Compile link bonding PMD library #config/common_linux +0
-0
Quoted 8 lines
config/common_linux CONFIG_RTE_LIBRTE_VHOST_POSTCOPY=n · line 22
Quoted 5 lines
config/common_linux +1
-0@@ -19,6 +19,7 @@ CONFIG_RTE_LIBRTE_VHOST_POSTCOPY=n CONFIG_RTE_LIBRTE_PMD_VHOST=y CONFIG_RTE_LIBRTE_IFC_PMD=y CONFIG_RTE_LIBRTE_PMD_AF_PACKET=y +CONFIG_RTE_LIBRTE_PMD_AF_XDP=yIt has to be disabled by default as it requires headers from kernels more recent than minimum kernel version supported by DPDK.
Ok, got it.
Quoted 27 lines
Quoted 4 lines
config/common_linux +0
-0CONFIG_RTE_LIBRTE_PMD_SOFTNIC=y CONFIG_RTE_LIBRTE_PMD_TAP=y CONFIG_RTE_LIBRTE_AVP_PMD=y...
Quoted 20 lines
drivers/net/af_xdp/meson.build new file +14
-0@@ -0,0 +1,21 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Intel Corporation + +if host_machine.system() != 'linux' + build = false +endif + +bpf_dep = dependency('libbpf', required: false) +if bpf_dep.found() + build = true +else + bpf_dep = cc.find_library('libbpf', required: false) + if bpf_dep.found() and cc.has_header('xsk.h', dependencies: bpf_dep) + build = true
Quoted 8 lines
drivers/net/af_xdp/meson.build line 19
Quoted 5 lines
drivers/net/af_xdp/meson.build new file +5
-0+ pkgconfig_extra_libs += '-lbpf' + else + build = false + endif +endifI think you need to add more checks, as above does not cover
linux/if_xdp.hheader IIUC.
will add more.
Quoted 138 lines
Quoted 138 lines
drivers/net/af_xdp/meson.build new file +2
-0+sources = files('rte_eth_af_xdp.c') +ext_deps += bpf_depdrivers/net/af_xdp/rte_eth_af_xdp.c new file +129
-0@@ -0,0 +1,940 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2019 Intel Corporation. +*/ + +#include <rte_mbuf.h> +#include <rte_ethdev_driver.h> +#include <rte_ethdev_vdev.h> +#include <rte_malloc.h> +#include <rte_kvargs.h> +#include <rte_bus_vdev.h> +#include <rte_string_fns.h> + +#include <linux/if_ether.h> +#include <linux/if_xdp.h> +#include <linux/if_link.h> +#include <asm/barrier.h> +#include <arpa/inet.h> +#include <net/if.h> +#include <sys/types.h> +#include <sys/socket.h> +#include <sys/ioctl.h> +#include <sys/mman.h> +#include <unistd.h> +#include <poll.h> +#include <bpf/bpf.h> +#include <xsk.h> + +#ifndef SOL_XDP +#define SOL_XDP 283 +#endif + +#ifndef AF_XDP +#define AF_XDP 44 +#endif + +#ifndef PF_XDP +#define PF_XDP AF_XDP +#endif + +static int af_xdp_logtype; + +#define AF_XDP_LOG(level, fmt, args...) \ + rte_log(RTE_LOG_ ## level, af_xdp_logtype, \ + "%s(): " fmt "\n", __func__, ##args) + +#define ETH_AF_XDP_IFACE_ARG "iface" +#define ETH_AF_XDP_QUEUE_IDX_ARG "queue" + +#define ETH_AF_XDP_FRAME_SIZE XSK_UMEM__DEFAULT_FRAME_SIZE +#define ETH_AF_XDP_NUM_BUFFERS 4096 +#define ETH_AF_XDP_DATA_HEADROOM 0 +#define ETH_AF_XDP_DFLT_NUM_DESCS XSK_RING_CONS__DEFAULT_NUM_DESCS +#define ETH_AF_XDP_DFLT_QUEUE_IDX 0 + +#define ETH_AF_XDP_RX_BATCH_SIZE 32 +#define ETH_AF_XDP_TX_BATCH_SIZE 32 + +#define ETH_AF_XDP_MAX_QUEUE_PAIRS 16 + +struct xsk_umem_info { +struct xsk_ring_prod fq; +struct xsk_ring_cons cq; +struct xsk_umem *umem; +struct rte_ring *buf_ring; +void *buffer; +}; + +struct rx_stats { +uint64_t rx_pkts; +uint64_t rx_bytes; +uint64_t rx_dropped; +}; + +struct pkt_rx_queue { +struct xsk_ring_cons rx; +struct xsk_umem_info *umem; +struct xsk_socket *xsk; +struct rte_mempool *mb_pool; + +struct rx_stats stats; + +struct pkt_tx_queue *pair; +uint16_t queue_idx; +}; + +struct tx_stats { +uint64_t tx_pkts; +uint64_t err_pkts; +uint64_t tx_bytes; +}; + +struct pkt_tx_queue { +struct xsk_ring_prod tx; + +struct tx_stats stats; + +struct pkt_rx_queue *pair; +uint16_t queue_idx; +}; + +struct pmd_internals { +int if_index; +char if_name[IFNAMSIZ ]; +uint16_t queue_idx; +struct ether_addr eth_addr; +struct xsk_umem_info *umem; +struct rte_mempool *mb_pool_share; + +struct pkt_rx_queue rx_queues[ETH_AF_XDP_MAX_QUEUE_PAIRS ]; +struct pkt_tx_queue tx_queues[ETH_AF_XDP_MAX_QUEUE_PAIRS ]; +}; + +static const char *const valid_arguments[] = { +ETH_AF_XDP_IFACE_ARG , +ETH_AF_XDP_QUEUE_IDX_ARG , +NULL +}; + +static struct rte_eth_link pmd_link = { + .link_speed =ETH_SPEED_NUM_10G , + .link_duplex =ETH_LINK_FULL_DUPLEX , + .link_status =ETH_LINK_DOWN , + .link_autoneg =ETH_LINK_AUTONEG +}; + +static inline int +reserve_fill_queue (struct xsk_umem_info *umem,int reserve_size) +{ +struct xsk_ring_prod *fq = &umem->fq;
Quoted 8 lines
drivers/net/af_xdp/rte_eth_af_xdp.c line 135
Quoted 6 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +6
-0+uint32_t idx; +void *addr =NULL ; +int i, ret; + + ret =xsk_ring_prod__reserve (fq, reserve_size, &idx); +if (!ret) {You could use
unlikely()here as it is called in the hot path.
Got it.
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c line 141
Quoted 7 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +6
-0+AF_XDP_LOG (ERR ,"Failed to reserve enough fq descs.\n" ); +return ret; + } + +for (i =0 ; i < reserve_size; i++) { +__u64 *fq_addr;For consistency, you could either declare addr here, or
fq_addrat the top of the function.
ok, will make the code consistent in next version.
Quoted 32 lines
Quoted 32 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +31
-0+if (rte_ring_dequeue (umem->buf_ring, &addr)) { + i--; +break ; + } + fq_addr =xsk_ring_prod__fill_addr (fq, idx++); + *fq_addr = (uint64_t )addr; + } + +xsk_ring_prod__submit (fq, i); + +return 0 ; +} + +static uint16_t +eth_af_xdp_rx (void *queue,struct rte_mbuf **bufs,uint16_t nb_pkts) +{ +struct pkt_rx_queue *rxq = queue; +struct xsk_ring_cons *rx = &rxq->rx; +struct xsk_umem_info *umem = rxq->umem; +struct xsk_ring_prod *fq = &umem->fq; +uint32_t idx_rx; +uint32_t free_thresh = fq->size >>1 ; +struct rte_mbuf *mbufs[ETH_AF_XDP_TX_BATCH_SIZE ]; +unsigned long dropped =0 ; +unsigned long rx_bytes =0 ; +uint16_t count =0 ; +int rcvd, i; + + nb_pkts =RTE_MIN (nb_pkts,ETH_AF_XDP_TX_BATCH_SIZE ); + + rcvd =xsk_ring_cons__peek (rx, nb_pkts, &idx_rx);
Quoted 8 lines
drivers/net/af_xdp/rte_eth_af_xdp.c line 179
Quoted 7 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +7
-0+if (rcvd ==0 ) +return 0 ; + +if (xsk_prod_nb_free (fq, free_thresh) >= free_thresh) + (void )reserve_fill_queue (umem,ETH_AF_XDP_RX_BATCH_SIZE ); + +if (rte_pktmbuf_alloc_bulk (rxq->mb_pool, mbufs, rcvd) !=0 )
unlikely()?
Got it.
Quoted 34 lines
Quoted 31 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +31
-0+return 0 ; + +for (i =0 ; i < rcvd; i++) { +const struct xdp_desc *desc; +uint64_t addr; +uint32_t len; +void *pkt; + + desc =xsk_ring_cons__rx_desc (rx, idx_rx++); + addr = desc->addr; + len = desc->len; + pkt =xsk_umem__get_data (rxq->umem->buffer, addr); + + rte_memcpy(rte_pktmbuf_mtod(mbufs[i], void *), pkt, len); +rte_pktmbuf_pkt_len (mbufs[i]) = len; +rte_pktmbuf_data_len (mbufs[i]) = len; + rx_bytes += len; + bufs[count++] = mbufs[i]; + +rte_ring_enqueue (umem->buf_ring, (void *)addr); + } + +xsk_ring_cons__release (rx, rcvd); + +/* statistics */ + rxq->stats.rx_pkts += (rcvd - dropped); + rxq->stats.rx_bytes += rx_bytes; + +return count; +} +...
Quoted 12 lines
drivers/net/af_xdp/rte_eth_af_xdp.c line 321
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+ +/* This function gets called when the current port gets stopped. */ +static void +eth_dev_stop (struct rte_eth_dev *dev) +{ + dev->data->dev_link.link_status =ETH_LINK_DOWN ; +} + +static int +eth_dev_configure(struct rte_eth_dev *dev __rte_unused)Remove
__rte_unused.
oops, my bad, will remove it.
Quoted 12 lines
Quoted 9 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +8
-0+{ +/* rx/tx must be paired */ +if (dev->data->nb_rx_queues != dev->data->nb_tx_queues) +return -EINVAL ; + +return 0 ; +} +...
Quoted 15 lines
Quoted 4 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +4
-0+ +static int +xsk_configure (struct pmd_internals *internals,struct pkt_rx_queue *rxq, +int ring_size)drivers/net/af_xdp/rte_eth_af_xdp.c line 523
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+{ +struct xsk_socket_config cfg; +struct pkt_tx_queue *txq = rxq->pair; +int ret =0 ; +int reserve_size; + + rxq->umem =xdp_umem_configure (); +if (rxq->umem ==NULL ) { + ret = -ENOMEM ; + goto err;You can return directly here as umem == NULL.
Got it.
Quoted 15 lines
Quoted 15 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +14
-0+ } + + cfg.rx_size = ring_size; + cfg.tx_size = ring_size; + cfg.libbpf_flags =0 ; + cfg.xdp_flags =XDP_FLAGS_UPDATE_IF_NOEXIST ; + cfg.bind_flags =0 ; + ret =xsk_socket__create (&rxq->xsk, internals->if_name, + internals->queue_idx, rxq->umem->umem, &rxq->rx, + &txq->tx, &cfg); +if (ret) { +AF_XDP_LOG (ERR ,"Failed to create xsk socket.\n" ); + goto err; + }
Quoted 8 lines
drivers/net/af_xdp/rte_eth_af_xdp.c line 543
Quoted 6 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +6
-0+ + reserve_size =ETH_AF_XDP_DFLT_NUM_DESCS /2 ; + ret =reserve_fill_queue (rxq->umem, reserve_size); +if (ret) { +AF_XDP_LOG (ERR ,"Failed to reserve fill queue.\n" ); + goto err;Shouldn't you call
xsk_socket__delete(rxq->xsk)here?
Good catch, xsk socket does need to be deleted here.
Quoted 24 lines
Quoted 11 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+ } + +return 0 ; + +err: +xdp_umem_destroy (rxq->umem); + +return ret; +} +drivers/net/af_xdp/rte_eth_af_xdp.c line 563
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+static void +queue_reset (struct pmd_internals *internals,uint16_t queue_idx) +{ +struct pkt_rx_queue *rxq = &internals->rx_queues[queue_idx]; +struct pkt_tx_queue *txq = rxq->pair; +int xsk_fd =xsk_socket__fd (rxq->xsk); + +if (xsk_fd) { +close (xsk_fd); +if (internals->umem !=NULL ) {Moving this condition out would work and be cleaner.
Quoted 2 lines
Anyway, it seems to never enter this condition as
internal->umemis not set yet whenqueue_reset()is called.
You are right, internal->umem and xsk_fd shouldn't be handled here, will refine the code.
Quoted 63 lines
Quoted 40 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +39
-0+xdp_umem_destroy (internals->umem); + internals->umem =NULL ; + } + } +memset (rxq,0 ,sizeof (*rxq)); +memset (txq,0 ,sizeof (*txq)); + rxq->pair = txq; + txq->pair = rxq; + rxq->queue_idx = queue_idx; + txq->queue_idx = queue_idx; +} + +static int +eth_rx_queue_setup (struct rte_eth_dev *dev, +uint16_t rx_queue_id, +uint16_t nb_rx_desc, + unsigned int socket_id __rte_unused, + const struct rte_eth_rxconf *rx_conf __rte_unused, +struct rte_mempool *mb_pool) +{ +struct pmd_internals *internals = dev->data->dev_private; +uint32_t buf_size, data_size; +struct pkt_rx_queue *rxq; +int ret; + + rxq = &internals->rx_queues[rx_queue_id]; +queue_reset (internals, rx_queue_id); + +/* Now get the space available for data in the mbuf */ + buf_size =rte_pktmbuf_data_room_size (mb_pool) - +RTE_PKTMBUF_HEADROOM ; + data_size =ETH_AF_XDP_FRAME_SIZE -ETH_AF_XDP_DATA_HEADROOM ; + +if (data_size > buf_size) { +AF_XDP_LOG (ERR ,"%s: %d bytes will not fit in mbuf (%d bytes)\n" , + dev->device->name, data_size, buf_size); + ret = -ENOMEM ; + goto err; + }drivers/net/af_xdp/rte_eth_af_xdp.c line 612
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +10
-0+ + rxq->mb_pool = mb_pool; + +if (xsk_configure (internals, rxq, nb_rx_desc)) { +AF_XDP_LOG (ERR ,"Failed to configure xdp socket\n" ); + ret = -EINVAL ; + goto err; + } + + internals->umem = rxq->umem;If my previous comment is wrong, i.e.
internals->umemmay be already set whenqueue_reset()is called, then it means you might have a leak here.Quoted 9 lines
drivers/net/af_xdp/rte_eth_af_xdp.c new file +9
-0+ + dev->data->rx_queues[rx_queue_id] = rxq; +return 0 ; + +err: +queue_reset (internals, rx_queue_id); +return ret; +} +
Re: [PATCH v4 1/5] net/af_xdp: introduce AF XDP PMD driver
In reply to Xiaolong Yedrivers/net/af_xdp/rte_pmd_af_xdp_version.map line 3
Quoted 10 lines · Xiaolong Ye
drivers/net/af_xdp/rte_pmd_af_xdp_version.map new file +3
-0@@ -0,0 +1,3 @@ +DPDK_19.05 { + local: *; +};
This is a new PMD, shouldn't it be EXPERIMENTAL rather than DPDK_19.05?
Signature · 2 lines
Kind regards, Luca Boccassi
Re: [PATCH v4 1/5] net/af_xdp: introduce AF XDP PMD driver
24/03/2019 13:10, Luca Boccassi:
Quoted 5 lines · Xiaolong Ye
Quoted 4 lines
drivers/net/af_xdp/rte_pmd_af_xdp_version.map new file +0
-0
Quoted 8 lines
drivers/net/af_xdp/rte_pmd_af_xdp_version.map line 3
Quoted 6 lines
drivers/net/af_xdp/rte_pmd_af_xdp_version.map new file +3
-0@@ -0,0 +1,3 @@ +DPDK_19.05 { + local: *; +};This is a new PMD, shouldn't it be EXPERIMENTAL rather than DPDK_19.05?
I don't think so, because there is no API here.
PATCH v4 2/5 lib/mbuf: introduce helper to create mempool with flags
In reply to Xiaolong YeThis allows applications to create mbuf mempool with specific flags such as MEMPOOL_F_NO_SPREAD if they want fixed size memory objects.
--- lib/librte_mbuf/rte_mbuf.c | 29 +++++++++++++++++++----- lib/librte_mbuf/rte_mbuf.h | 45 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 69 insertions(+), 5 deletions(-)
lib/librte_mbuf/rte_mbuf.c +24 -5
@@ -106,11 +106,10 @@ rte_pktmbuf_init(struct rte_mempool *mp, m->next =NULL ; } -/* Helper to create a mbuf pool with given mempool ops name*/ -struct rte_mempool * -rte_pktmbuf_pool_create_by_ops (const char *name,unsigned int n, +static struct rte_mempool * +rte_pktmbuf_pool_create_by_ops_with_flags (const char *name,unsigned int n,unsigned int cache_size,uint16_t priv_size,uint16_t data_room_size, -int socket_id,const char *ops_name) +unsigned int flags,int socket_id,const char *ops_name) {struct rte_mempool *mp;struct rte_pktmbuf_pool_private mbp_priv;
@@ -130,7 +129,7 @@ rte_pktmbuf_pool_create_by_ops(const char *name, unsigned int n, mbp_priv.mbuf_priv_size = priv_size; mp =rte_mempool_create_empty (name, n, elt_size, cache_size, -sizeof (struct rte_pktmbuf_pool_private ), socket_id,0 ); +sizeof (struct rte_pktmbuf_pool_private ), socket_id, flags);if (mp ==NULL )return NULL ;
@@ -157,6 +156,16 @@ rte_pktmbuf_pool_create_by_ops(const char *name, unsigned int n,return mp; } +/* Helper to create a mbuf pool with given mempool ops name*/ +struct rte_mempool * +rte_pktmbuf_pool_create_by_ops (const char *name,unsigned int n, +unsigned int cache_size,uint16_t priv_size,uint16_t data_room_size, +int socket_id,const char *ops_name) +{ +return rte_pktmbuf_pool_create_by_ops_with_flags (name, n, cache_size, + priv_size, data_room_size,0 , socket_id, ops_name); +} +/* helper to create a mbuf pool */ struct rte_mempool * rte_pktmbuf_pool_create(const char *name, unsigned int n,
@@ -167,6 +176,16 @@ rte_pktmbuf_pool_create(const char *name, unsigned int n, data_room_size, socket_id, NULL); } +/* helper to create a mbuf pool with flags (e.g. NO_SPREAD) */ +struct rte_mempool * __rte_experimental +rte_pktmbuf_pool_create_with_flags (const char *name,unsigned int n, +unsigned int cache_size,uint16_t priv_size,uint16_t data_room_size, +unsigned int flags,int socket_id) +{ +return rte_pktmbuf_pool_create_by_ops_with_flags (name, n, cache_size, + priv_size, data_room_size, flags, socket_id,NULL ); +} +/* do some sanity checks on a mbuf: panic if it fails */ void rte_mbuf_sanity_check (const struct rte_mbuf *m,int is_header)
lib/librte_mbuf/rte_mbuf.h +45 -0
51 lines not on this page · raw message
Signature · 1 line
2.17.1
Re: [PATCH v4 2/5] lib/mbuf: introduce helper to create mempool with flags
Quoted 50 lines · Xiaolong Ye
This allows applications to create mbuf mempool with specific flags such as
MEMPOOL_F_NO_SPREADif they want fixed size memory objects.Signed-off-by Qi Zhang <[email protected]>Signed-off-by Xiaolong Ye <[email protected]>--- lib/librte_mbuf/rte_mbuf.c | 29 +++++++++++++++++++----- lib/librte_mbuf/rte_mbuf.h | 45 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 69 insertions(+), 5 deletions(-)lib/librte_mbuf/rte_mbuf.h +38
-0+/** +* Create a mbuf pool with flags. +* +* This function creates and initializes a packet mbuf pool. It is +* a wrapper to rte_mempool functions. +* +* @warning +* @b EXPERIMENTAL: This API may change without prior notice. +* +* @param name +* The name of the mbuf pool. +* @param n +* The number of elements in the mbuf pool. The optimum size (in terms +* of memory usage) for a mempool is when n is a power of two minus one: +* n = (2^q - 1). +* @param cache_size +* Size of the per-core object cache. See rte_mempool_create() for +* details. +* @param priv_size +* Size of application private are between the rte_mbuf structure +* and the data buffer. This value must be aligned to RTE_MBUF_PRIV_ALIGN. +* @param data_room_size +* Size of data buffer in each mbuf, including RTE_PKTMBUF_HEADROOM. +* @param flags +* Flags controlling the behavior of the mempool. See +* rte_mempool_create() for details. +* @param socket_id +* The socket identifier where the memory should be allocated. The +* value can be *SOCKET_ID_ANY* if there is no NUMA constraint for the +* reserved zone. +* @return +* The pointer to the new allocated mempool, on success. NULL on error +* with rte_errno set appropriately. Possible rte_errno values include: +* - E_RTE_NO_CONFIG - function could not get pointer to rte_config structure +* - E_RTE_SECONDARY - function was called from a secondary process instance +* - EINVAL - cache size provided is too large, or priv_size is not aligned. +* - ENOSPC - the maximum number of memzones has already been allocated +* - EEXIST - a memzone with the same name already exists
lib/librte_mbuf/rte_mbuf.h rte_pktmbuf_pool_create(const char *name, unsigned n, · line 1316
Quoted 10 lines
lib/librte_mbuf/rte_mbuf.h +7
-0+* - ENOMEM - no appropriate memory area found in which to create memzone +*/ +struct rte_mempool * __rte_experimental +rte_pktmbuf_pool_create_with_flags (const char *name,unsigned int n, +unsigned int cache_size,uint16_t priv_size,uint16_t data_room_size, +unsigned int flags,int socket_id); +/** * Create a mbuf pool with a given mempool ops name *
You need to add it to rte_mbuf_version.map too.
Re: [PATCH v4 2/5] lib/mbuf: introduce helper to create mempool with flags
Quoted 57 lines · Maxime Coquelin
Quoted 54 lines · Xiaolong Ye
This allows applications to create mbuf mempool with specific flags such as
MEMPOOL_F_NO_SPREADif they want fixed size memory objects.Signed-off-by Qi Zhang <[email protected]>Signed-off-by Xiaolong Ye <[email protected]>--- lib/librte_mbuf/rte_mbuf.c | 29 +++++++++++++++++++----- lib/librte_mbuf/rte_mbuf.h | 45 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 69 insertions(+), 5 deletions(-)lib/librte_mbuf/rte_mbuf.h +43
-0+/** +* Create a mbuf pool with flags. +* +* This function creates and initializes a packet mbuf pool. It is +* a wrapper to rte_mempool functions. +* +* @warning +* @b EXPERIMENTAL: This API may change without prior notice. +* +* @param name +* The name of the mbuf pool. +* @param n +* The number of elements in the mbuf pool. The optimum size (in terms +* of memory usage) for a mempool is when n is a power of two minus one: +* n = (2^q - 1). +* @param cache_size +* Size of the per-core object cache. See rte_mempool_create() for +* details. +* @param priv_size +* Size of application private are between the rte_mbuf structure +* and the data buffer. This value must be aligned to RTE_MBUF_PRIV_ALIGN. +* @param data_room_size +* Size of data buffer in each mbuf, including RTE_PKTMBUF_HEADROOM. +* @param flags +* Flags controlling the behavior of the mempool. See +* rte_mempool_create() for details. +* @param socket_id +* The socket identifier where the memory should be allocated. The +* value can be *SOCKET_ID_ANY* if there is no NUMA constraint for the +* reserved zone. +* @return +* The pointer to the new allocated mempool, on success. NULL on error +* with rte_errno set appropriately. Possible rte_errno values include: +* - E_RTE_NO_CONFIG - function could not get pointer to rte_config structure +* - E_RTE_SECONDARY - function was called from a secondary process instance +* - EINVAL - cache size provided is too large, or priv_size is not aligned. +* - ENOSPC - the maximum number of memzones has already been allocated +* - EEXIST - a memzone with the same name already exists +* - ENOMEM - no appropriate memory area found in which to create memzone +*/ +struct rte_mempool * __rte_experimental +rte_pktmbuf_pool_create_with_flags(const char *name, unsigned int n, + unsigned int cache_size, uint16_t priv_size, uint16_t data_room_size,
Quoted 8 lines
lib/librte_mbuf/rte_mbuf.h rte_pktmbuf_pool_create(const char *name, unsigned n, · line 1316
Quoted 6 lines
lib/librte_mbuf/rte_mbuf.h +2
-0+ unsigned int flags, int socket_id); +/** * Create a mbuf pool with a given mempool ops name * You need to add it to rte_mbuf_version.map too.
Got it, will add in next version.
Thanks,
Xiaolong
PATCH v4 3/5 lib/mempool: allow page size aligned mempool
In reply to Xiaolong YeAllow create a mempool with page size aligned base address.
--- lib/librte_mempool/rte_mempool.c | 3 +++ lib/librte_mempool/rte_mempool.h | 1 + 2 files changed, 4 insertions(+)
lib/librte_mempool/rte_mempool.c +3 -0
@@ -543,6 +543,9 @@ rte_mempool_populate_default(struct rte_mempool *mp)if (try_contig) flags |=RTE_MEMZONE_IOVA_CONTIG ; +if (mp->flags &MEMPOOL_F_PAGE_ALIGN ) + align =RTE_MAX (align, (size_t )getpagesize ()); + mz =rte_memzone_reserve_aligned (mz_name, mem_size, mp->socket_id, flags, align);
lib/librte_mempool/rte_mempool.h +1 -0
@@ -264,6 +264,7 @@ struct rte_mempool {#define MEMPOOL_F_POOL_CREATED 0x0010/**< Internal: pool is created. */ #define MEMPOOL_F_NO_IOVA_CONTIG 0x0020/**< Don't need IOVA contiguous objs. */ #define MEMPOOL_F_NO_PHYS_CONTIG MEMPOOL_F_NO_IOVA_CONTIG/* deprecated */ +#define MEMPOOL_F_PAGE_ALIGN 0x0040/**< Chunk's base address is page aligned */ /** * @internal When debug is enabled, store some statistics.
Signature · 1 line
2.17.1
PATCH v4 4/5 net/af_xdp: use mbuf mempool for buffer management
In reply to Xiaolong YeNow, af_xdp registered memory buffer is managed by rte_mempool. mbuf be allocated from rte_mempool can be convert to xdp_desc's address and vice versa.
--- drivers/net/af_xdp/rte_eth_af_xdp.c | 117 +++++++++++++++++----------- 1 file changed, 72 insertions(+), 45 deletions(-)
drivers/net/af_xdp/rte_eth_af_xdp.c +72 -45
@@ -48,7 +48,11 @@ static int af_xdp_logtype;#define ETH_AF_XDP_FRAME_SIZE XSK_UMEM__DEFAULT_FRAME_SIZE#define ETH_AF_XDP_NUM_BUFFERS 4096 -#define ETH_AF_XDP_DATA_HEADROOM 0 +/* mempool hdrobj size (64 bytes) + sizeof(struct rte_mbuf) (128 bytes) */ +#define ETH_AF_XDP_MBUF_OVERHEAD 192 +/* data start from offset 320 (192 + 128) bytes */ +#define ETH_AF_XDP_DATA_HEADROOM \ + (ETH_AF_XDP_MBUF_OVERHEAD + RTE_PKTMBUF_HEADROOM)#define ETH_AF_XDP_DFLT_NUM_DESCS XSK_RING_CONS__DEFAULT_NUM_DESCS#define ETH_AF_XDP_DFLT_QUEUE_IDX 0
@@ -61,7 +65,7 @@ struct xsk_umem_info {struct xsk_ring_prod fq;struct xsk_ring_cons cq;struct xsk_umem *umem; -struct rte_ring *buf_ring; +struct rte_mempool *mb_pool;void *buffer; };
@@ -123,12 +127,32 @@ static struct rte_eth_link pmd_link = { .link_autoneg = ETH_LINK_AUTONEG }; +static inline struct rte_mbuf * +addr_to_mbuf (struct xsk_umem_info *umem,uint64_t addr) +{ +uint64_t offset = (addr /ETH_AF_XDP_FRAME_SIZE * +ETH_AF_XDP_FRAME_SIZE ); +struct rte_mbuf *mbuf = (struct rte_mbuf *)((uint64_t )umem->buffer + + offset +ETH_AF_XDP_MBUF_OVERHEAD - +sizeof (struct rte_mbuf )); + mbuf->data_off = addr - offset -ETH_AF_XDP_MBUF_OVERHEAD ; +return mbuf; +} + +static inline uint64_t +mbuf_to_addr (struct xsk_umem_info *umem,struct rte_mbuf *mbuf) +{ +return (uint64_t )mbuf->buf_addr + mbuf->data_off - + (uint64_t )umem->buffer; +} +static inline int reserve_fill_queue (struct xsk_umem_info *umem,int reserve_size) {struct xsk_ring_prod *fq = &umem->fq; +struct rte_mbuf *mbuf;uint32_t idx; -void *addr =NULL ; +uint64_t addr;int i, ret; ret =xsk_ring_prod__reserve (fq, reserve_size, &idx);
@@ -139,12 +163,14 @@ reserve_fill_queue(struct xsk_umem_info *umem, int reserve_size)for (i =0 ; i < reserve_size; i++) {__u64 *fq_addr; -if (rte_ring_dequeue (umem->buf_ring, &addr)) { + mbuf =rte_pktmbuf_alloc (umem->mb_pool); +if (mbuf ==NULL ) { i--;break ; } + addr =mbuf_to_addr (umem, mbuf); fq_addr =xsk_ring_prod__fill_addr (fq, idx++); - *fq_addr = (uint64_t )addr; + *fq_addr = addr; }xsk_ring_prod__submit (fq, i);
@@ -196,7 +222,7 @@ eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) rx_bytes += len; bufs[count++] = mbufs[i]; -rte_ring_enqueue (umem->buf_ring, (void *)addr); +rte_pktmbuf_free (addr_to_mbuf (umem, addr)); }xsk_ring_cons__release (rx, rcvd);
@@ -219,7 +245,7 @@ static void pull_umem_cq(struct xsk_umem_info *umem, int size)for (i =0 ; i < n; i++) {uint64_t addr; addr = *xsk_ring_cons__comp_addr (cq, idx_cq++); -rte_ring_enqueue (umem->buf_ring, (void *)addr); +rte_pktmbuf_free (addr_to_mbuf (umem, addr)); }xsk_ring_cons__release (cq, n);
@@ -248,7 +274,7 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)struct pkt_tx_queue *txq = queue;struct xsk_umem_info *umem = txq->pair->umem;struct rte_mbuf *mbuf; -void *addrs[ETH_AF_XDP_TX_BATCH_SIZE ]; +struct rte_mbuf *mbuf_to_tx;unsigned long tx_bytes =0 ;int i, valid =0 ;uint32_t idx_tx;
@@ -257,11 +283,6 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)pull_umem_cq (umem, nb_pkts); - nb_pkts =rte_ring_dequeue_bulk (umem->buf_ring, addrs, - nb_pkts,NULL ); -if (nb_pkts ==0 ) -return 0 ; -if (xsk_ring_prod__reserve (&txq->tx, nb_pkts, &idx_tx) != nb_pkts) {kick_tx (txq);return 0 ;
@@ -275,7 +296,12 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) desc =xsk_ring_prod__tx_desc (&txq->tx, idx_tx + i); mbuf = bufs[i];if (mbuf->pkt_len <= buf_len) { - desc->addr = (uint64_t )addrs[valid]; + mbuf_to_tx =rte_pktmbuf_alloc (umem->mb_pool); +if (mbuf_to_tx ==NULL ) { +rte_pktmbuf_free (mbuf); +continue ; + } + desc->addr =mbuf_to_addr (umem, mbuf_to_tx); desc->len = mbuf->pkt_len; pkt =xsk_umem__get_data (umem->buffer, desc->addr);
@@ -291,10 +317,6 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)kick_tx (txq); -if (valid < nb_pkts) -rte_ring_enqueue_bulk (umem->buf_ring, &addrs[valid], - nb_pkts - valid,NULL ); - txq->stats.err_pkts += nb_pkts - valid; txq->stats.tx_pkts += valid; txq->stats.tx_bytes += tx_bytes;
@@ -443,16 +465,29 @@ eth_link_update(struct rte_eth_dev *dev __rte_unused,static void xdp_umem_destroy (struct xsk_umem_info *umem) { -free (umem->buffer); - umem->buffer =NULL ; - -rte_ring_free (umem->buf_ring); - umem->buf_ring =NULL ; +rte_mempool_free (umem->mb_pool); + umem->mb_pool =NULL ;rte_free (umem); umem =NULL ; } +static inline uint64_t get_base_addr (struct rte_mempool *mp) +{ +struct rte_mempool_memhdr *memhdr; + + memhdr =STAILQ_FIRST (&mp->mem_list); +return (uint64_t )(memhdr->addr); +} + +static inline uint64_t get_len (struct rte_mempool *mp) +{ +struct rte_mempool_memhdr *memhdr; + + memhdr =STAILQ_FIRST (&mp->mem_list); +return (uint64_t )(memhdr->len); +} +static struct xsk_umem_info *xdp_umem_configure (void ) {struct xsk_umem_info *umem;
@@ -461,9 +496,8 @@ static struct xsk_umem_info *xdp_umem_configure(void) .comp_size = ETH_AF_XDP_DFLT_NUM_DESCS, .frame_size = ETH_AF_XDP_FRAME_SIZE, .frame_headroom = ETH_AF_XDP_DATA_HEADROOM }; -void *bufs =NULL ; +void *base_addr =NULL ;int ret; -uint64_t i; umem =rte_zmalloc_socket ("umem" ,sizeof (*umem),0 ,rte_socket_id ());if (umem ==NULL ) {
@@ -471,26 +505,20 @@ static struct xsk_umem_info *xdp_umem_configure(void)return NULL ; } - umem->buf_ring =rte_ring_create ("af_xdp_ring" , -ETH_AF_XDP_NUM_BUFFERS , -SOCKET_ID_ANY , -0x0 ); -if (umem->buf_ring ==NULL ) { -AF_XDP_LOG (ERR ,"Failed to create rte_ring\n" ); + umem->mb_pool =rte_pktmbuf_pool_create_with_flags ("af_xdp_mempool" , +ETH_AF_XDP_NUM_BUFFERS , +250 ,0 , +ETH_AF_XDP_FRAME_SIZE - +ETH_AF_XDP_MBUF_OVERHEAD , +MEMPOOL_F_NO_SPREAD |MEMPOOL_F_PAGE_ALIGN , +SOCKET_ID_ANY ); +if (umem->mb_pool ==NULL || umem->mb_pool->nb_mem_chunks !=1 ) { +AF_XDP_LOG (ERR ,"Failed to create mempool\n" ); goto err; } + base_addr = (void *)get_base_addr (umem->mb_pool); -for (i =0 ; i <ETH_AF_XDP_NUM_BUFFERS ; i++) -rte_ring_enqueue (umem->buf_ring, - (void *)(i *ETH_AF_XDP_FRAME_SIZE + -ETH_AF_XDP_DATA_HEADROOM )); - -if (posix_memalign (&bufs,getpagesize (), -ETH_AF_XDP_NUM_BUFFERS *ETH_AF_XDP_FRAME_SIZE )) { -AF_XDP_LOG (ERR ,"Failed to allocate memory pool.\n" ); - goto err; - } - ret =xsk_umem__create (&umem->umem, bufs, + ret =xsk_umem__create (&umem->umem, base_addr,ETH_AF_XDP_NUM_BUFFERS *ETH_AF_XDP_FRAME_SIZE , &umem->fq, &umem->cq, &usr_config);
@@ -499,7 +527,7 @@ static struct xsk_umem_info *xdp_umem_configure(void)AF_XDP_LOG (ERR ,"Failed to create umem" ); goto err; } - umem->buffer = bufs; + umem->buffer = base_addr;return umem;
@@ -912,10 +940,9 @@ rte_pmd_af_xdp_remove(struct rte_vdev_device *dev) internals = eth_dev->data->dev_private; -rte_ring_free (internals->umem->buf_ring); -rte_free (internals->umem->buffer);rte_free (internals->umem); +rte_mempool_free (internals->umem->mb_pool);rte_eth_dev_release_port (eth_dev);
Signature · 1 line
2.17.1
Re: [PATCH v4 4/5] net/af_xdp: use mbuf mempool for buffer management
Quoted 1 line · Xiaolong Ye
Now,
af_xdpregistered memory buffer is managed byrte_mempool. mbuf be
s/mbuf be allocated/mbuf allocated/
Quoted 1 line
allocated from
rte_mempoolcan be convert to xdp_desc's address and vice
s/convert/converted/
Quoted 68 lines
versa.
Signed-off-by Xiaolong Ye <[email protected]>--- drivers/net/af_xdp/rte_eth_af_xdp.c | 117 +++++++++++++++++----------- 1 file changed, 72 insertions(+), 45 deletions(-)drivers/net/af_xdp/rte_eth_af_xdp.c +27
-3@@ -48,7 +48,11 @@ static int af_xdp_logtype;#define ETH_AF_XDP_FRAME_SIZE XSK_UMEM__DEFAULT_FRAME_SIZE#define ETH_AF_XDP_NUM_BUFFERS 4096 -#define ETH_AF_XDP_DATA_HEADROOM 0 +/* mempool hdrobj size (64 bytes) + sizeof(struct rte_mbuf) (128 bytes) */ +#define ETH_AF_XDP_MBUF_OVERHEAD 192 +/* data start from offset 320 (192 + 128) bytes */ +#define ETH_AF_XDP_DATA_HEADROOM \ + (ETH_AF_XDP_MBUF_OVERHEAD + RTE_PKTMBUF_HEADROOM)#define ETH_AF_XDP_DFLT_NUM_DESCS XSK_RING_CONS__DEFAULT_NUM_DESCS#define ETH_AF_XDP_DFLT_QUEUE_IDX 0@@ -61,7 +65,7 @@ struct xsk_umem_info {struct xsk_ring_prod fq;struct xsk_ring_cons cq;struct xsk_umem *umem; -struct rte_ring *buf_ring; +struct rte_mempool *mb_pool;void *buffer; };@@ -123,12 +127,32 @@ static struct rte_eth_link pmd_link = { .link_autoneg = ETH_LINK_AUTONEG }; +static inline struct rte_mbuf * +addr_to_mbuf (struct xsk_umem_info *umem,uint64_t addr) +{ +uint64_t offset = (addr /ETH_AF_XDP_FRAME_SIZE * +ETH_AF_XDP_FRAME_SIZE ); +struct rte_mbuf *mbuf = (struct rte_mbuf *)((uint64_t )umem->buffer + + offset +ETH_AF_XDP_MBUF_OVERHEAD - +sizeof (struct rte_mbuf )); + mbuf->data_off = addr - offset -ETH_AF_XDP_MBUF_OVERHEAD ; +return mbuf; +} + +static inline uint64_t +mbuf_to_addr (struct xsk_umem_info *umem,struct rte_mbuf *mbuf) +{ +return (uint64_t )mbuf->buf_addr + mbuf->data_off - + (uint64_t )umem->buffer; +} +static inline int reserve_fill_queue (struct xsk_umem_info *umem,int reserve_size) {struct xsk_ring_prod *fq = &umem->fq; +struct rte_mbuf *mbuf;uint32_t idx; -void *addr =NULL ; +uint64_t addr;int i, ret; ret =xsk_ring_prod__reserve (fq, reserve_size, &idx);
drivers/net/af_xdp/rte_eth_af_xdp.c reserve_fill_queue(struct xsk_umem_info *umem, int reserve_size) · line 167
Quoted 6 lines
drivers/net/af_xdp/rte_eth_af_xdp.c +2
-1@@ -139,12 +163,14 @@ reserve_fill_queue(struct xsk_umem_info *umem, int reserve_size)for (i =0 ; i < reserve_size; i++) {__u64 *fq_addr; -if (rte_ring_dequeue (umem->buf_ring, &addr)) { + mbuf =rte_pktmbuf_alloc (umem->mb_pool); +if (mbuf ==NULL ) {
unlikely()
Quoted 50 lines
drivers/net/af_xdp/rte_eth_af_xdp.c +5
-9i--;break ; } + addr =mbuf_to_addr (umem, mbuf); fq_addr =xsk_ring_prod__fill_addr (fq, idx++); - *fq_addr = (uint64_t )addr; + *fq_addr = addr; }xsk_ring_prod__submit (fq, i);@@ -196,7 +222,7 @@ eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) rx_bytes += len; bufs[count++] = mbufs[i]; -rte_ring_enqueue (umem->buf_ring, (void *)addr); +rte_pktmbuf_free (addr_to_mbuf (umem, addr)); }xsk_ring_cons__release (rx, rcvd);@@ -219,7 +245,7 @@ static void pull_umem_cq(struct xsk_umem_info *umem, int size)for (i =0 ; i < n; i++) {uint64_t addr; addr = *xsk_ring_cons__comp_addr (cq, idx_cq++); -rte_ring_enqueue (umem->buf_ring, (void *)addr); +rte_pktmbuf_free (addr_to_mbuf (umem, addr)); }xsk_ring_cons__release (cq, n);@@ -248,7 +274,7 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)struct pkt_tx_queue *txq = queue;struct xsk_umem_info *umem = txq->pair->umem;struct rte_mbuf *mbuf; -void *addrs[ETH_AF_XDP_TX_BATCH_SIZE ]; +struct rte_mbuf *mbuf_to_tx;unsigned long tx_bytes =0 ;int i, valid =0 ;uint32_t idx_tx;@@ -257,11 +283,6 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)pull_umem_cq (umem, nb_pkts); - nb_pkts =rte_ring_dequeue_bulk (umem->buf_ring, addrs, - nb_pkts,NULL ); -if (nb_pkts ==0 ) -return 0 ; -if (xsk_ring_prod__reserve (&txq->tx, nb_pkts, &idx_tx) != nb_pkts) {kick_tx (txq);return 0 ;
drivers/net/af_xdp/rte_eth_af_xdp.c eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) · line 300
Quoted 6 lines
drivers/net/af_xdp/rte_eth_af_xdp.c +2
-1@@ -275,7 +296,12 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) desc =xsk_ring_prod__tx_desc (&txq->tx, idx_tx + i); mbuf = bufs[i];if (mbuf->pkt_len <= buf_len) { - desc->addr = (uint64_t )addrs[valid]; + mbuf_to_tx =rte_pktmbuf_alloc (umem->mb_pool); +if (mbuf_to_tx ==NULL ) {
unlikely()
Quoted 103 lines
drivers/net/af_xdp/rte_eth_af_xdp.c +34
-28+rte_pktmbuf_free (mbuf); +continue ; + } + desc->addr =mbuf_to_addr (umem, mbuf_to_tx); desc->len = mbuf->pkt_len; pkt =xsk_umem__get_data (umem->buffer, desc->addr);@@ -291,10 +317,6 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)kick_tx (txq); -if (valid < nb_pkts) -rte_ring_enqueue_bulk (umem->buf_ring, &addrs[valid], - nb_pkts - valid,NULL ); - txq->stats.err_pkts += nb_pkts - valid; txq->stats.tx_pkts += valid; txq->stats.tx_bytes += tx_bytes;@@ -443,16 +465,29 @@ eth_link_update(struct rte_eth_dev *dev __rte_unused,static void xdp_umem_destroy (struct xsk_umem_info *umem) { -free (umem->buffer); - umem->buffer =NULL ; - -rte_ring_free (umem->buf_ring); - umem->buf_ring =NULL ; +rte_mempool_free (umem->mb_pool); + umem->mb_pool =NULL ;rte_free (umem); umem =NULL ; } +static inline uint64_t get_base_addr (struct rte_mempool *mp) +{ +struct rte_mempool_memhdr *memhdr; + + memhdr =STAILQ_FIRST (&mp->mem_list); +return (uint64_t )(memhdr->addr); +} + +static inline uint64_t get_len (struct rte_mempool *mp) +{ +struct rte_mempool_memhdr *memhdr; + + memhdr =STAILQ_FIRST (&mp->mem_list); +return (uint64_t )(memhdr->len); +} +static struct xsk_umem_info *xdp_umem_configure (void ) {struct xsk_umem_info *umem;@@ -461,9 +496,8 @@ static struct xsk_umem_info *xdp_umem_configure(void) .comp_size = ETH_AF_XDP_DFLT_NUM_DESCS, .frame_size = ETH_AF_XDP_FRAME_SIZE, .frame_headroom = ETH_AF_XDP_DATA_HEADROOM }; -void *bufs =NULL ; +void *base_addr =NULL ;int ret; -uint64_t i; umem =rte_zmalloc_socket ("umem" ,sizeof (*umem),0 ,rte_socket_id ());if (umem ==NULL ) {@@ -471,26 +505,20 @@ static struct xsk_umem_info *xdp_umem_configure(void)return NULL ; } - umem->buf_ring =rte_ring_create ("af_xdp_ring" , -ETH_AF_XDP_NUM_BUFFERS , -SOCKET_ID_ANY , -0x0 ); -if (umem->buf_ring ==NULL ) { -AF_XDP_LOG (ERR ,"Failed to create rte_ring\n" ); + umem->mb_pool =rte_pktmbuf_pool_create_with_flags ("af_xdp_mempool" , +ETH_AF_XDP_NUM_BUFFERS , +250 ,0 , +ETH_AF_XDP_FRAME_SIZE - +ETH_AF_XDP_MBUF_OVERHEAD , +MEMPOOL_F_NO_SPREAD |MEMPOOL_F_PAGE_ALIGN , +SOCKET_ID_ANY ); +if (umem->mb_pool ==NULL || umem->mb_pool->nb_mem_chunks !=1 ) { +AF_XDP_LOG (ERR ,"Failed to create mempool\n" ); goto err; } + base_addr = (void *)get_base_addr (umem->mb_pool); -for (i =0 ; i <ETH_AF_XDP_NUM_BUFFERS ; i++) -rte_ring_enqueue (umem->buf_ring, - (void *)(i *ETH_AF_XDP_FRAME_SIZE + -ETH_AF_XDP_DATA_HEADROOM )); - -if (posix_memalign (&bufs,getpagesize (), -ETH_AF_XDP_NUM_BUFFERS *ETH_AF_XDP_FRAME_SIZE )) { -AF_XDP_LOG (ERR ,"Failed to allocate memory pool.\n" ); - goto err; - } - ret =xsk_umem__create (&umem->umem, bufs, + ret =xsk_umem__create (&umem->umem, base_addr,ETH_AF_XDP_NUM_BUFFERS *ETH_AF_XDP_FRAME_SIZE , &umem->fq, &umem->cq, &usr_config);
drivers/net/af_xdp/rte_eth_af_xdp.c static struct xsk_umem_info *xdp_umem_configure(void) · line 528
Quoted 2 lines
drivers/net/af_xdp/rte_eth_af_xdp.c +0
-0@@ -499,7 +527,7 @@ static struct xsk_umem_info *xdp_umem_configure(void)AF_XDP_LOG (ERR ,"Failed to create umem" ); goto err;
You need to destroy mb_pool if xsk_umem__create() fails.
Quoted 19 lines
drivers/net/af_xdp/rte_eth_af_xdp.c +2
-3} - umem->buffer = bufs; + umem->buffer = base_addr;return umem;@@ -912,10 +940,9 @@ rte_pmd_af_xdp_remove(struct rte_vdev_device *dev) internals = eth_dev->data->dev_private; -rte_ring_free (internals->umem->buf_ring); -rte_free (internals->umem->buffer);rte_free (internals->umem); +rte_mempool_free (internals->umem->mb_pool);rte_eth_dev_release_port (eth_dev);
Re: [PATCH v4 4/5] net/af_xdp: use mbuf mempool for buffer management
Quoted 8 lines · Maxime Coquelin
Quoted 1 line · Xiaolong Ye
Now,
af_xdpregistered memory buffer is managed byrte_mempool. mbuf bes/mbuf be allocated/mbuf allocated/
Quoted 1 line
allocated from
rte_mempoolcan be convert to xdp_desc's address and vices/convert/converted/
Quoted 1 line
versa.
Will fix these spell errors.
Quoted 61 lines
Quoted 61 lines
Signed-off-by Xiaolong Ye <[email protected]>--- drivers/net/af_xdp/rte_eth_af_xdp.c | 117 +++++++++++++++++----------- 1 file changed, 72 insertions(+), 45 deletions(-)drivers/net/af_xdp/rte_eth_af_xdp.c +27
-3@@ -48,7 +48,11 @@ static int af_xdp_logtype;#define ETH_AF_XDP_FRAME_SIZE XSK_UMEM__DEFAULT_FRAME_SIZE#define ETH_AF_XDP_NUM_BUFFERS 4096 -#define ETH_AF_XDP_DATA_HEADROOM 0 +/* mempool hdrobj size (64 bytes) + sizeof(struct rte_mbuf) (128 bytes) */ +#define ETH_AF_XDP_MBUF_OVERHEAD 192 +/* data start from offset 320 (192 + 128) bytes */ +#define ETH_AF_XDP_DATA_HEADROOM \ + (ETH_AF_XDP_MBUF_OVERHEAD + RTE_PKTMBUF_HEADROOM)#define ETH_AF_XDP_DFLT_NUM_DESCS XSK_RING_CONS__DEFAULT_NUM_DESCS#define ETH_AF_XDP_DFLT_QUEUE_IDX 0@@ -61,7 +65,7 @@ struct xsk_umem_info {struct xsk_ring_prod fq;struct xsk_ring_cons cq;struct xsk_umem *umem; -struct rte_ring *buf_ring; +struct rte_mempool *mb_pool;void *buffer; };@@ -123,12 +127,32 @@ static struct rte_eth_link pmd_link = { .link_autoneg = ETH_LINK_AUTONEG }; +static inline struct rte_mbuf * +addr_to_mbuf (struct xsk_umem_info *umem,uint64_t addr) +{ +uint64_t offset = (addr /ETH_AF_XDP_FRAME_SIZE * +ETH_AF_XDP_FRAME_SIZE ); +struct rte_mbuf *mbuf = (struct rte_mbuf *)((uint64_t )umem->buffer + + offset +ETH_AF_XDP_MBUF_OVERHEAD - +sizeof (struct rte_mbuf )); + mbuf->data_off = addr - offset -ETH_AF_XDP_MBUF_OVERHEAD ; +return mbuf; +} + +static inline uint64_t +mbuf_to_addr (struct xsk_umem_info *umem,struct rte_mbuf *mbuf) +{ +return (uint64_t )mbuf->buf_addr + mbuf->data_off - + (uint64_t )umem->buffer; +} +static inline int reserve_fill_queue (struct xsk_umem_info *umem,int reserve_size) {struct xsk_ring_prod *fq = &umem->fq; +struct rte_mbuf *mbuf;uint32_t idx; -void *addr =NULL ; +uint64_t addr;int i, ret; ret =xsk_ring_prod__reserve (fq, reserve_size, &idx);
Quoted 8 lines
drivers/net/af_xdp/rte_eth_af_xdp.c reserve_fill_queue(struct xsk_umem_info *umem, int reserve_size) · line 166
Quoted 6 lines
drivers/net/af_xdp/rte_eth_af_xdp.c +2
-1@@ -139,12 +163,14 @@ reserve_fill_queue(struct xsk_umem_info *umem, int reserve_size)for (i =0 ; i < reserve_size; i++) {__u64 *fq_addr; -if (rte_ring_dequeue (umem->buf_ring, &addr)) { + mbuf =rte_pktmbuf_alloc (umem->mb_pool); +if (mbuf ==NULL ) {
unlikely()
Got it.
Quoted 45 lines
Quoted 45 lines
drivers/net/af_xdp/rte_eth_af_xdp.c +5
-9i--;break ; } + addr =mbuf_to_addr (umem, mbuf); fq_addr =xsk_ring_prod__fill_addr (fq, idx++); - *fq_addr = (uint64_t )addr; + *fq_addr = addr; }xsk_ring_prod__submit (fq, i);@@ -196,7 +222,7 @@ eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) rx_bytes += len; bufs[count++] = mbufs[i]; -rte_ring_enqueue (umem->buf_ring, (void *)addr); +rte_pktmbuf_free (addr_to_mbuf (umem, addr)); }xsk_ring_cons__release (rx, rcvd);@@ -219,7 +245,7 @@ static void pull_umem_cq(struct xsk_umem_info *umem, int size)for (i =0 ; i < n; i++) {uint64_t addr; addr = *xsk_ring_cons__comp_addr (cq, idx_cq++); -rte_ring_enqueue (umem->buf_ring, (void *)addr); +rte_pktmbuf_free (addr_to_mbuf (umem, addr)); }xsk_ring_cons__release (cq, n);@@ -248,7 +274,7 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)struct pkt_tx_queue *txq = queue;struct xsk_umem_info *umem = txq->pair->umem;struct rte_mbuf *mbuf; -void *addrs[ETH_AF_XDP_TX_BATCH_SIZE ]; +struct rte_mbuf *mbuf_to_tx;unsigned long tx_bytes =0 ;int i, valid =0 ;uint32_t idx_tx;@@ -257,11 +283,6 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)pull_umem_cq (umem, nb_pkts); - nb_pkts =rte_ring_dequeue_bulk (umem->buf_ring, addrs, - nb_pkts,NULL ); -if (nb_pkts ==0 ) -return 0 ; -if (xsk_ring_prod__reserve (&txq->tx, nb_pkts, &idx_tx) != nb_pkts) {kick_tx (txq);return 0 ;@@ -275,7 +296,12 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)
Quoted 8 lines
drivers/net/af_xdp/rte_eth_af_xdp.c eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) · line 300
Quoted 6 lines
drivers/net/af_xdp/rte_eth_af_xdp.c +2
-1desc =xsk_ring_prod__tx_desc (&txq->tx, idx_tx + i); mbuf = bufs[i];if (mbuf->pkt_len <= buf_len) { - desc->addr = (uint64_t )addrs[valid]; + mbuf_to_tx =rte_pktmbuf_alloc (umem->mb_pool); +if (mbuf_to_tx ==NULL ) {
unlikely()
Got it.
Quoted 92 lines
Quoted 92 lines
drivers/net/af_xdp/rte_eth_af_xdp.c +34
-28+rte_pktmbuf_free (mbuf); +continue ; + } + desc->addr =mbuf_to_addr (umem, mbuf_to_tx); desc->len = mbuf->pkt_len; pkt =xsk_umem__get_data (umem->buffer, desc->addr);@@ -291,10 +317,6 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)kick_tx (txq); -if (valid < nb_pkts) -rte_ring_enqueue_bulk (umem->buf_ring, &addrs[valid], - nb_pkts - valid,NULL ); - txq->stats.err_pkts += nb_pkts - valid; txq->stats.tx_pkts += valid; txq->stats.tx_bytes += tx_bytes;@@ -443,16 +465,29 @@ eth_link_update(struct rte_eth_dev *dev __rte_unused,static void xdp_umem_destroy (struct xsk_umem_info *umem) { -free (umem->buffer); - umem->buffer =NULL ; - -rte_ring_free (umem->buf_ring); - umem->buf_ring =NULL ; +rte_mempool_free (umem->mb_pool); + umem->mb_pool =NULL ;rte_free (umem); umem =NULL ; } +static inline uint64_t get_base_addr (struct rte_mempool *mp) +{ +struct rte_mempool_memhdr *memhdr; + + memhdr =STAILQ_FIRST (&mp->mem_list); +return (uint64_t )(memhdr->addr); +} + +static inline uint64_t get_len (struct rte_mempool *mp) +{ +struct rte_mempool_memhdr *memhdr; + + memhdr =STAILQ_FIRST (&mp->mem_list); +return (uint64_t )(memhdr->len); +} +static struct xsk_umem_info *xdp_umem_configure (void ) {struct xsk_umem_info *umem;@@ -461,9 +496,8 @@ static struct xsk_umem_info *xdp_umem_configure(void) .comp_size = ETH_AF_XDP_DFLT_NUM_DESCS, .frame_size = ETH_AF_XDP_FRAME_SIZE, .frame_headroom = ETH_AF_XDP_DATA_HEADROOM }; -void *bufs =NULL ; +void *base_addr =NULL ;int ret; -uint64_t i; umem =rte_zmalloc_socket ("umem" ,sizeof (*umem),0 ,rte_socket_id ());if (umem ==NULL ) {@@ -471,26 +505,20 @@ static struct xsk_umem_info *xdp_umem_configure(void)return NULL ; } - umem->buf_ring =rte_ring_create ("af_xdp_ring" , -ETH_AF_XDP_NUM_BUFFERS , -SOCKET_ID_ANY , -0x0 ); -if (umem->buf_ring ==NULL ) { -AF_XDP_LOG (ERR ,"Failed to create rte_ring\n" ); + umem->mb_pool =rte_pktmbuf_pool_create_with_flags ("af_xdp_mempool" , +ETH_AF_XDP_NUM_BUFFERS , +250 ,0 , +ETH_AF_XDP_FRAME_SIZE - +ETH_AF_XDP_MBUF_OVERHEAD , +MEMPOOL_F_NO_SPREAD |MEMPOOL_F_PAGE_ALIGN , +SOCKET_ID_ANY ); +if (umem->mb_pool ==NULL || umem->mb_pool->nb_mem_chunks !=1 ) { +AF_XDP_LOG (ERR ,"Failed to create mempool\n" ); goto err; } + base_addr = (void *)get_base_addr (umem->mb_pool); -for (i =0 ; i <ETH_AF_XDP_NUM_BUFFERS ; i++) -rte_ring_enqueue (umem->buf_ring, - (void *)(i *ETH_AF_XDP_FRAME_SIZE + -ETH_AF_XDP_DATA_HEADROOM )); - -if (posix_memalign (&bufs,getpagesize (), -ETH_AF_XDP_NUM_BUFFERS *ETH_AF_XDP_FRAME_SIZE )) { -AF_XDP_LOG (ERR ,"Failed to allocate memory pool.\n" ); - goto err; - } - ret = xsk_umem__create(&umem->umem, bufs, + ret = xsk_umem__create(&umem->umem, base_addr,
Quoted 8 lines
drivers/net/af_xdp/rte_eth_af_xdp.c static struct xsk_umem_info *xdp_umem_configure(void) · line 528
Quoted 6 lines
drivers/net/af_xdp/rte_eth_af_xdp.c +0
-0ETH_AF_XDP_NUM_BUFFERS *ETH_AF_XDP_FRAME_SIZE , &umem->fq, &umem->cq, &usr_config);@@ -499,7 +527,7 @@ static struct xsk_umem_info *xdp_umem_configure(void)AF_XDP_LOG (ERR ,"Failed to create umem" ); goto err;You need to destroy
mb_poolifxsk_umem__create()fails.
Will do.
Thanks,
Xiaolong
Quoted 13 lines
Quoted 13 lines
drivers/net/af_xdp/rte_eth_af_xdp.c +2
-3} - umem->buffer = bufs; + umem->buffer = base_addr;return umem;@@ -912,10 +940,9 @@ rte_pmd_af_xdp_remove(struct rte_vdev_device *dev) internals = eth_dev->data->dev_private; -rte_ring_free (internals->umem->buf_ring); -rte_free (internals->umem->buffer);rte_free (internals->umem); +rte_mempool_free (internals->umem->mb_pool);rte_eth_dev_release_port (eth_dev);
Re: [PATCH v4 4/5] net/af_xdp: use mbuf mempool for buffer management
Quoted 11 lines · Ye Xiaolong
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c static struct xsk_umem_info *xdp_umem_configure(void) · line 528
Quoted 8 lines
drivers/net/af_xdp/rte_eth_af_xdp.c +1
-1- ret =xsk_umem__create (&umem->umem, bufs, + ret =xsk_umem__create (&umem->umem, base_addr,ETH_AF_XDP_NUM_BUFFERS *ETH_AF_XDP_FRAME_SIZE , &umem->fq, &umem->cq, &usr_config);@@ -499,7 +527,7 @@ static struct xsk_umem_info *xdp_umem_configure(void)AF_XDP_LOG (ERR ,"Failed to create umem" ); goto err;You need to destroy
mb_poolifxsk_umem__create()fails.
Quoted 1 line
Will do.
Correction, mp_pool destrcttion has alrealy been hanled in xdp_umem_destory.
Thanks,
Xiaolong
Quoted 16 lines
Thanks,
XiaolongQuoted 13 lines
Quoted 13 lines
drivers/net/af_xdp/rte_eth_af_xdp.c +2
-3} - umem->buffer = bufs; + umem->buffer = base_addr;return umem;@@ -912,10 +940,9 @@ rte_pmd_af_xdp_remove(struct rte_vdev_device *dev) internals = eth_dev->data->dev_private; -rte_ring_free (internals->umem->buf_ring); -rte_free (internals->umem->buffer);rte_free (internals->umem); +rte_mempool_free (internals->umem->mb_pool);rte_eth_dev_release_port (eth_dev);
PATCH v4 5/5 net/af_xdp: enable zero copy
In reply to Xiaolong YeTry to check if external mempool (from rx_queue_setup) is fit for af_xdp, if it is, it will be registered to af_xdp socket directly and there will be no packet data copy on Rx and Tx.
--- drivers/net/af_xdp/rte_eth_af_xdp.c | 129 ++++++++++++++++++++-------- 1 file changed, 95 insertions(+), 34 deletions(-)
drivers/net/af_xdp/rte_eth_af_xdp.c +95 -34
@@ -67,6 +67,7 @@ struct xsk_umem_info {struct xsk_umem *umem;struct rte_mempool *mb_pool;void *buffer; +uint8_t zc; };struct rx_stats {
@@ -85,6 +86,7 @@ struct pkt_rx_queue {struct pkt_tx_queue *pair;uint16_t queue_idx; +uint8_t zc; };struct tx_stats {
@@ -202,7 +204,8 @@ eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)if (xsk_prod_nb_free (fq, free_thresh) >= free_thresh) (void )reserve_fill_queue (umem,ETH_AF_XDP_RX_BATCH_SIZE ); -if (rte_pktmbuf_alloc_bulk (rxq->mb_pool, mbufs, rcvd) !=0 ) +if (!rxq->zc && +rte_pktmbuf_alloc_bulk (rxq->mb_pool, mbufs, rcvd) !=0 )return 0 ;for (i =0 ; i < rcvd; i++) {
@@ -216,13 +219,23 @@ eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) len = desc->len; pkt =xsk_umem__get_data (rxq->umem->buffer, addr); - rte_memcpy(rte_pktmbuf_mtod(mbufs[i], void *), pkt, len); -rte_pktmbuf_pkt_len (mbufs[i]) = len; -rte_pktmbuf_data_len (mbufs[i]) = len; - rx_bytes += len; - bufs[count++] = mbufs[i]; - -rte_pktmbuf_free (addr_to_mbuf (umem, addr)); +if (rxq->zc) { +struct rte_mbuf *mbuf; + mbuf =addr_to_mbuf (rxq->umem, addr); +rte_pktmbuf_pkt_len (mbuf) = len; +rte_pktmbuf_data_len (mbuf) = len; + rx_bytes += len; + bufs[count++] = mbuf; + }else { + rte_memcpy(rte_pktmbuf_mtod(mbufs[i], void *), + pkt, len); +rte_pktmbuf_pkt_len (mbufs[i]) = len; +rte_pktmbuf_data_len (mbufs[i]) = len; + rx_bytes += len; + bufs[count++] = mbufs[i]; + +rte_pktmbuf_free (addr_to_mbuf (umem, addr)); + } }xsk_ring_cons__release (rx, rcvd);
@@ -295,22 +308,29 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) -ETH_AF_XDP_DATA_HEADROOM ; desc =xsk_ring_prod__tx_desc (&txq->tx, idx_tx + i); mbuf = bufs[i]; -if (mbuf->pkt_len <= buf_len) { - mbuf_to_tx =rte_pktmbuf_alloc (umem->mb_pool); -if (mbuf_to_tx ==NULL ) { -rte_pktmbuf_free (mbuf); -continue ; - } - desc->addr =mbuf_to_addr (umem, mbuf_to_tx); +if (txq->pair->zc && mbuf->pool == umem->mb_pool) { + desc->addr =mbuf_to_addr (umem, mbuf); desc->len = mbuf->pkt_len; - pkt =xsk_umem__get_data (umem->buffer, - desc->addr); - rte_memcpy(pkt, rte_pktmbuf_mtod(mbuf, void *), - desc->len); valid++; tx_bytes += mbuf->pkt_len; + }else { +if (mbuf->pkt_len <= buf_len) { + mbuf_to_tx =rte_pktmbuf_alloc (umem->mb_pool); +if (!mbuf_to_tx) { +rte_pktmbuf_free (mbuf); +continue ; + } + desc->addr =mbuf_to_addr (umem, mbuf_to_tx); + desc->len = mbuf->pkt_len; + pkt =xsk_umem__get_data (umem->buffer, + desc->addr); + memcpy(pkt, rte_pktmbuf_mtod(mbuf, void *), + desc->len); + valid++; + tx_bytes += mbuf->pkt_len; + } +rte_pktmbuf_free (mbuf); } -rte_pktmbuf_free (mbuf); }xsk_ring_prod__submit (&txq->tx, nb_pkts);
@@ -488,7 +508,7 @@ static inline uint64_t get_len(struct rte_mempool *mp)return (uint64_t )(memhdr->len); } -static struct xsk_umem_info *xdp_umem_configure (void ) +static struct xsk_umem_info *xdp_umem_configure (struct rte_mempool *mb_pool) {struct xsk_umem_info *umem;struct xsk_umem_config usr_config = {
@@ -505,16 +525,23 @@ static struct xsk_umem_info *xdp_umem_configure(void)return NULL ; } - umem->mb_pool =rte_pktmbuf_pool_create_with_flags ("af_xdp_mempool" , -ETH_AF_XDP_NUM_BUFFERS , -250 ,0 , -ETH_AF_XDP_FRAME_SIZE - -ETH_AF_XDP_MBUF_OVERHEAD , -MEMPOOL_F_NO_SPREAD |MEMPOOL_F_PAGE_ALIGN , -SOCKET_ID_ANY ); -if (umem->mb_pool ==NULL || umem->mb_pool->nb_mem_chunks !=1 ) { -AF_XDP_LOG (ERR ,"Failed to create mempool\n" ); - goto err; +if (!mb_pool) { + umem->mb_pool =rte_pktmbuf_pool_create_with_flags ("af_xdp_mempool" , +ETH_AF_XDP_NUM_BUFFERS , +250 ,0 , +ETH_AF_XDP_FRAME_SIZE - +ETH_AF_XDP_MBUF_OVERHEAD , +MEMPOOL_F_NO_SPREAD |MEMPOOL_F_PAGE_ALIGN , +SOCKET_ID_ANY ); + +if (umem->mb_pool ==NULL || + umem->mb_pool->nb_mem_chunks !=1 ) { +AF_XDP_LOG (ERR ,"Failed to create mempool\n" ); + goto err; + } + }else { + umem->mb_pool = mb_pool; + umem->zc =1 ; } base_addr = (void *)get_base_addr (umem->mb_pool);
@@ -536,16 +563,43 @@ static struct xsk_umem_info *xdp_umem_configure(void)return NULL ; } +static uint8_t +check_mempool_zc (struct rte_mempool *mp) +{ +RTE_ASSERT (mp); + +/* must continues */ +if (mp->nb_mem_chunks >1 ) +return 0 ; + +/* check header size */ +if (mp->header_size !=RTE_CACHE_LINE_SIZE ) +return 0 ; + +/* check base address */ +if ((uint64_t )get_base_addr (mp) %getpagesize () !=0 ) +return 0 ; + +/* check chunk size */ +if ((mp->elt_size + mp->header_size + mp->trailer_size) % +ETH_AF_XDP_FRAME_SIZE !=0 ) +return 0 ; + +return 1 ; +} +static int xsk_configure (struct pmd_internals *internals,struct pkt_rx_queue *rxq, -int ring_size) +int ring_size,struct rte_mempool *mb_pool) {struct xsk_socket_config cfg;struct pkt_tx_queue *txq = rxq->pair; +struct rte_mempool *mp;int ret =0 ;int reserve_size; - rxq->umem =xdp_umem_configure (); + mp =check_mempool_zc (mb_pool) ? mb_pool :NULL ; + rxq->umem =xdp_umem_configure (mp);if (rxq->umem ==NULL ) { ret = -ENOMEM ; goto err;
@@ -631,7 +685,7 @@ eth_rx_queue_setup(struct rte_eth_dev *dev, rxq->mb_pool = mb_pool; -if (xsk_configure (internals, rxq, nb_rx_desc)) { +if (xsk_configure (internals, rxq, nb_rx_desc, mb_pool)) {AF_XDP_LOG (ERR ,"Failed to configure xdp socket\n" ); ret = -EINVAL ; goto err;
@@ -639,6 +693,13 @@ eth_rx_queue_setup(struct rte_eth_dev *dev, internals->umem = rxq->umem; +if (mb_pool == internals->umem->mb_pool) + rxq->zc = internals->umem->zc; + +if (rxq->zc) +AF_XDP_LOG (INFO , +"zero copy enabled on rx queue %d\n" , rx_queue_id); + dev->data->rx_queues[rx_queue_id] = rxq;return 0 ;
Signature · 1 line
2.17.1
v5 cover Introduce AF_XDP PMD
In reply to Xiaolong YeOverview
========
This patchset adds a new PMD driver for AF_XDP which is a proposed faster version of AF_PACKET interface in Linux, see below links [1] [2] for details of AF_XDP introduction:
AF_XDP roadmap
==============
- AF_XDP is included in upstream kernel since 4.18, and AF_XDP support in libbpf has been merged in v5.1-rc1.
- Now i40e and ixgbe drivers have supported zero copy mode.
Change logs
===========
V5:
- disable AF_XDP pmd by default due to it requires kernel more recent than minimum kernel version supported by DPDK
- address other review comments of Maxime
V4:
- change vdev name to net_af_xdp
- adopt dynamic log type for all logging
- Fix other style issues raised by Stephen
V3:
- Fix all style issues pointed by Stephen, Mattias, David.
- Drop the testpmd change as we'll adopt Jerin's suggestion to add a new mempool driver to handle the application use of AF_XDP's zero copy feature.
V2:
- Fix a NULL pointer reference crash issue
- Fix txonly stop sending traffic in zc mode
- Refactor rte_mbuf.c to avoid ABI breakage.
- Fix multiple style issues pointed by Ferruh, David, Stephen, Luca.
changes vs RFC sent by Qi last Aug:
- Re-work base on AF_XDP's interface changes since the new libbpf has provided higher-level APIs that hide many of the details of the AP_XDP uapi. After the rework, it helps to reduce 300+ lines of code.
- multi-queues is not supported due to current libbpf doesn't support multi-sockets on a single umem.
- No extra steps to load xdp program manually, since the current behavior of libbpf would load a default xdp program when user calls xsk_socket__create. userspace application only needs to handle the cleanup.
How to try
==========
1. take the kernel >= v5.1-rc1, build kernel and replace your host kernel with it.
make sure you turn on XDP sockets when compiling
Networking support -->
Networking options -->
[ * ] XDP sockets
2. build libbpf in tools/lib/bpf, and copy the libbpf.a and libbpf.so to /usr/lib64
cd tools/lib/bpf
make
3. ethtool -L enp59s0f0 combined 1
4. extra step to build dpdk
explicitly enable AF_XDP pmd by adding below line to config/common_linux
CONFIG_RTE_LIBRTE_PMD_AF_XDP=y
5. start testpmd
./build/app/testpmd -c 0xc -n 4 --vdev net_af_xdp,iface=enp59s0f0,queue=0 -- -i --rxq=1 --txq=1
in this case, default xdp program will be loaded and linked to queue 0 of enp59s0f0, network traffics travel to queue 0 will be redirected to af_xdp socket.
Xiaolong Ye (5):
net/af_xdp: introduce AF XDP PMD driver lib/mbuf: introduce helper to create mempool with flags lib/mempool: allow page size aligned mempool net/af_xdp: use mbuf mempool for buffer management net/af_xdp: enable zero copy
MAINTAINERS | 6 + config/common_base | 5 + doc/guides/nics/af_xdp.rst | 45 + doc/guides/nics/features/af_xdp.ini | 11 + doc/guides/nics/index.rst | 1 + doc/guides/rel_notes/release_19_05.rst | 7 + drivers/net/Makefile | 1 + drivers/net/af_xdp/Makefile | 32 + drivers/net/af_xdp/meson.build | 21 + drivers/net/af_xdp/rte_eth_af_xdp.c | 1020 +++++++++++++++++ drivers/net/af_xdp/rte_pmd_af_xdp_version.map | 3 + drivers/net/meson.build | 1 + lib/librte_mbuf/rte_mbuf.c | 29 +- lib/librte_mbuf/rte_mbuf.h | 45 + lib/librte_mbuf/rte_mbuf_version.map | 1 + lib/librte_mempool/rte_mempool.c | 3 + lib/librte_mempool/rte_mempool.h | 1 + mk/rte.app.mk | 1 + 18 files changed, 1228 insertions(+), 5 deletions(-) create mode 100644 doc/guides/nics/af_xdp.rst create mode 100644 doc/guides/nics/features/af_xdp.ini create mode 100644 drivers/net/af_xdp/Makefile create mode 100644 drivers/net/af_xdp/meson.build create mode 100644 drivers/net/af_xdp/rte_eth_af_xdp.c create mode 100644 drivers/net/af_xdp/rte_pmd_af_xdp_version.map
Signature · 1 line
2.17.1