Skip to contentSkip to conversation

net/ice: add Neon-optimised Rx/Tx vector paths

PATCH In review Jay Wang 2 participants 21 messages Mar 23 – Apr 16

Subsystems (2)
lore
Share & tools
lore

Jay Wang

author
#0 lore raw

cover net/ice: add Neon-optimised Rx/Tx vector paths

This patch introduces Neon-optimised Rx/Tx vector paths for the Intel ICE driver on AArch64 platforms.

The implementation mirrors the existing x86 vector paths while leveraging Arm Neon intrinsics to achieve comparable performance.

Tested on:
- Arm Neoverse (Ampere One)
- DPDK testpmd/l3fwd functional validation

Jay Wang (1):
net/ice: add Neon-optimised Rx/Tx vector paths

 .mailmap                                  |   1 +
 drivers/net/intel/ice/ice_ethdev.h        |   3 +
 drivers/net/intel/ice/ice_rxtx.c          |  53 +-
 drivers/net/intel/ice/ice_rxtx.h          |   6 +
 drivers/net/intel/ice/ice_rxtx_vec_neon.c | 747 ++++++++++++++++++++++
 drivers/net/intel/ice/meson.build         |   2 +
 6 files changed, 810 insertions(+), 2 deletions(-)
 create mode 100644 drivers/net/intel/ice/ice_rxtx_vec_neon.c
Signature · 1 line
2.43.0
author
#1 lore raw

PATCH 1/1 net/ice: add Neon-optimised Rx/Tx vector paths

This patch adds the Neon-optimised Rx and Tx paths to the ice driver.

Tested on Ampere One platform with Intel E810-C NIC and 100G connection. Tested with a single core and testpmd io forwarding mode. Observed ~30% performance boost in the above test compared to the default scalar path.

---
 .mailmap                                  |   1 +
 drivers/net/intel/ice/ice_ethdev.h        |   3 +
 drivers/net/intel/ice/ice_rxtx.c          |  53 +-
 drivers/net/intel/ice/ice_rxtx.h          |   6 +
 drivers/net/intel/ice/ice_rxtx_vec_neon.c | 747 ++++++++++++++++++++++
 drivers/net/intel/ice/meson.build         |   2 +
 6 files changed, 810 insertions(+), 2 deletions(-)
 create mode 100644 drivers/net/intel/ice/ice_rxtx_vec_neon.c
.mailmap +1 -0
index beccc84425..dfe92b0399 100644
@@ -695,6 +695,7 @@ Javen Xu <[email protected]>
 Jay Ding <[email protected]>
 Jay Jayatheerthan <[email protected]>
 Jay Rolette <[email protected]>
+Jay Wang <[email protected]>
 Jay Zhou <[email protected]>
 Jayaprakash Shanmugam <[email protected]>
 Jean Dao <[email protected]>
drivers/net/intel/ice/ice_ethdev.h +3 -0
index 4b3718f715..f6fd3bf106 100644
@@ -204,6 +204,8 @@ enum ice_rx_func_type {
 	ICE_RX_AVX512_SCATTERED,
 	ICE_RX_AVX512_OFFLOAD,
 	ICE_RX_AVX512_SCATTERED_OFFLOAD,
+	ICE_RX_NEON,
+	ICE_RX_NEON_SCATTERED,
 };
 
 enum ice_tx_func_type {
@@ -213,6 +215,7 @@ enum ice_tx_func_type {
 	ICE_TX_AVX2_OFFLOAD,
 	ICE_TX_AVX512,
 	ICE_TX_AVX512_OFFLOAD,
+	ICE_TX_NEON,
 };
 
 struct ice_adapter;
drivers/net/intel/ice/ice_rxtx.c +51 -2

89 lines not on this page · raw message

drivers/net/intel/ice/ice_rxtx.h +6 -0
index 77ed41f9fd..999b6b30d6 100644
@@ -261,6 +261,12 @@ const uint32_t *ice_dev_supported_ptypes_get(struct rte_eth_dev *dev,
 void ice_select_rxd_to_pkt_fields_handler(struct ci_rx_queue *rxq,
 					  uint32_t rxdid);
 
+uint16_t ice_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
+				uint16_t nb_pkts);
+uint16_t ice_recv_scattered_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
+				uint16_t nb_pkts);
+uint16_t ice_xmit_pkts_vec(void *tx_queue, struct rte_mbuf **tx_pkts,
+				uint16_t nb_pkts);
 int ice_rx_vec_dev_check(struct rte_eth_dev *dev);
 int ice_tx_vec_dev_check(struct rte_eth_dev *dev);
 int ice_rxq_vec_setup(struct ci_rx_queue *rxq);
drivers/net/intel/ice/ice_rxtx_vec_neon.c new file +747 -0

747 lines not on this page · raw message

drivers/net/intel/ice/meson.build +2 -0
index 293577676f..1dc7c0109a 100644
@@ -33,6 +33,8 @@ endif
 if arch_subdir == 'x86'
     sources_avx2 += files('ice_rxtx_vec_avx2.c')
     sources_avx512 += files('ice_rxtx_vec_avx512.c')
+elif arch_subdir == 'arm'
+    sources += files('ice_rxtx_vec_neon.c')
 endif
 
 sources += files(
Signature · 1 line
2.43.0
author
#2 lore raw

cover net/ice: add NEON-optimised Rx/Tx vector paths

This patch introduces NEON-optimised Rx/Tx vector paths for the Intel ICE driver on AArch64 platforms.

The implementation mirrors the existing x86 vector paths while leveraging Arm NEON intrinsics to achieve comparable performance.

Tested on:
- Arm Neoverse (Ampere One)
- DPDK testpmd/l3fwd functional validation

Jay Wang (1):
net/ice: add NEON-optimised Rx/Tx vector paths

---
v2:
* Fixed the coding style issues in the comments

 .mailmap                                  |   1 +
 drivers/net/intel/ice/ice_ethdev.h        |   3 +
 drivers/net/intel/ice/ice_rxtx.c          |  53 +-
 drivers/net/intel/ice/ice_rxtx.h          |   6 +
 drivers/net/intel/ice/ice_rxtx_vec_neon.c | 747 ++++++++++++++++++++++
 drivers/net/intel/ice/meson.build         |   2 +
 6 files changed, 810 insertions(+), 2 deletions(-)
 create mode 100644 drivers/net/intel/ice/ice_rxtx_vec_neon.c
Signature · 1 line
2.43.0
author
#3 lore raw

PATCH 1/1 net/ice: add Neon-optimised Rx/Tx vector paths

This patch adds the Neon-optimised Rx and Tx paths to the ice driver.

Tested on Ampere One platform with Intel E810-C NIC and 100G connection. Tested with a single core and testpmd io forwarding mode. Observed ~30% performance boost in the above test compared to the default scalar path.

---
 .mailmap                                  |   1 +
 drivers/net/intel/ice/ice_ethdev.h        |   3 +
 drivers/net/intel/ice/ice_rxtx.c          |  53 +-
 drivers/net/intel/ice/ice_rxtx.h          |   6 +
 drivers/net/intel/ice/ice_rxtx_vec_neon.c | 747 ++++++++++++++++++++++
 drivers/net/intel/ice/meson.build         |   2 +
 6 files changed, 810 insertions(+), 2 deletions(-)
 create mode 100644 drivers/net/intel/ice/ice_rxtx_vec_neon.c
.mailmap +1 -0
index beccc84425..dfe92b0399 100644
@@ -695,6 +695,7 @@ Javen Xu <[email protected]>
 Jay Ding <[email protected]>
 Jay Jayatheerthan <[email protected]>
 Jay Rolette <[email protected]>
+Jay Wang <[email protected]>
 Jay Zhou <[email protected]>
 Jayaprakash Shanmugam <[email protected]>
 Jean Dao <[email protected]>
drivers/net/intel/ice/ice_ethdev.h +3 -0
index 4b3718f715..f6fd3bf106 100644
@@ -204,6 +204,8 @@ enum ice_rx_func_type {
 	ICE_RX_AVX512_SCATTERED,
 	ICE_RX_AVX512_OFFLOAD,
 	ICE_RX_AVX512_SCATTERED_OFFLOAD,
+	ICE_RX_NEON,
+	ICE_RX_NEON_SCATTERED,
 };
 
 enum ice_tx_func_type {
@@ -213,6 +215,7 @@ enum ice_tx_func_type {
 	ICE_TX_AVX2_OFFLOAD,
 	ICE_TX_AVX512,
 	ICE_TX_AVX512_OFFLOAD,
+	ICE_TX_NEON,
 };
 
 struct ice_adapter;
drivers/net/intel/ice/ice_rxtx.c +51 -2

89 lines not on this page · raw message

drivers/net/intel/ice/ice_rxtx.h +6 -0
index 77ed41f9fd..999b6b30d6 100644
@@ -261,6 +261,12 @@ const uint32_t *ice_dev_supported_ptypes_get(struct rte_eth_dev *dev,
 void ice_select_rxd_to_pkt_fields_handler(struct ci_rx_queue *rxq,
 					  uint32_t rxdid);
 
+uint16_t ice_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
+				uint16_t nb_pkts);
+uint16_t ice_recv_scattered_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
+				uint16_t nb_pkts);
+uint16_t ice_xmit_pkts_vec(void *tx_queue, struct rte_mbuf **tx_pkts,
+				uint16_t nb_pkts);
 int ice_rx_vec_dev_check(struct rte_eth_dev *dev);
 int ice_tx_vec_dev_check(struct rte_eth_dev *dev);
 int ice_rxq_vec_setup(struct ci_rx_queue *rxq);
drivers/net/intel/ice/ice_rxtx_vec_neon.c new file +747 -0

747 lines not on this page · raw message

drivers/net/intel/ice/meson.build +2 -0
index 293577676f..1dc7c0109a 100644
@@ -33,6 +33,8 @@ endif
 if arch_subdir == 'x86'
     sources_avx2 += files('ice_rxtx_vec_avx2.c')
     sources_avx512 += files('ice_rxtx_vec_avx512.c')
+elif arch_subdir == 'arm'
+    sources += files('ice_rxtx_vec_neon.c')
 endif
 
 sources += files(
Signature · 1 line
2.43.0
author
#4 lore raw

PATCH v2 1/1 net/ice: add NEON-optimised Rx/Tx vector paths

In reply to Jay Wang

This patch adds the NEON-optimised Rx and Tx paths to the ice driver.

Tested on Ampere One platform with Intel E810-C NIC and 100G connection. Tested with a single core and testpmd io forwarding mode. Observed ~30% performance boost in the above test compared to the default scalar path.

---
 .mailmap                                  |   1 +
 drivers/net/intel/ice/ice_ethdev.h        |   3 +
 drivers/net/intel/ice/ice_rxtx.c          |  53 +-
 drivers/net/intel/ice/ice_rxtx.h          |   6 +
 drivers/net/intel/ice/ice_rxtx_vec_neon.c | 761 ++++++++++++++++++++++
 drivers/net/intel/ice/meson.build         |   2 +
 6 files changed, 824 insertions(+), 2 deletions(-)
 create mode 100644 drivers/net/intel/ice/ice_rxtx_vec_neon.c
.mailmap +1 -0
index beccc84425..dfe92b0399 100644
@@ -695,6 +695,7 @@ Javen Xu <[email protected]>
 Jay Ding <[email protected]>
 Jay Jayatheerthan <[email protected]>
 Jay Rolette <[email protected]>
+Jay Wang <[email protected]>
 Jay Zhou <[email protected]>
 Jayaprakash Shanmugam <[email protected]>
 Jean Dao <[email protected]>
drivers/net/intel/ice/ice_ethdev.h +3 -0
index 4b3718f715..f6fd3bf106 100644
@@ -204,6 +204,8 @@ enum ice_rx_func_type {
 	ICE_RX_AVX512_SCATTERED,
 	ICE_RX_AVX512_OFFLOAD,
 	ICE_RX_AVX512_SCATTERED_OFFLOAD,
+	ICE_RX_NEON,
+	ICE_RX_NEON_SCATTERED,
 };
 
 enum ice_tx_func_type {
@@ -213,6 +215,7 @@ enum ice_tx_func_type {
 	ICE_TX_AVX2_OFFLOAD,
 	ICE_TX_AVX512,
 	ICE_TX_AVX512_OFFLOAD,
+	ICE_TX_NEON,
 };
 
 struct ice_adapter;
drivers/net/intel/ice/ice_rxtx.c +51 -2

89 lines not on this page · raw message

drivers/net/intel/ice/ice_rxtx.h +6 -0
index 77ed41f9fd..999b6b30d6 100644
@@ -261,6 +261,12 @@ const uint32_t *ice_dev_supported_ptypes_get(struct rte_eth_dev *dev,
 void ice_select_rxd_to_pkt_fields_handler(struct ci_rx_queue *rxq,
 					  uint32_t rxdid);
 
+uint16_t ice_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
+				uint16_t nb_pkts);
+uint16_t ice_recv_scattered_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
+				uint16_t nb_pkts);
+uint16_t ice_xmit_pkts_vec(void *tx_queue, struct rte_mbuf **tx_pkts,
+				uint16_t nb_pkts);
 int ice_rx_vec_dev_check(struct rte_eth_dev *dev);
 int ice_tx_vec_dev_check(struct rte_eth_dev *dev);
 int ice_rxq_vec_setup(struct ci_rx_queue *rxq);
drivers/net/intel/ice/ice_rxtx_vec_neon.c new file +761 -0

761 lines not on this page · raw message

drivers/net/intel/ice/meson.build +2 -0
index 293577676f..1dc7c0109a 100644
@@ -33,6 +33,8 @@ endif
 if arch_subdir == 'x86'
     sources_avx2 += files('ice_rxtx_vec_avx2.c')
     sources_avx512 += files('ice_rxtx_vec_avx512.c')
+elif arch_subdir == 'arm'
+    sources += files('ice_rxtx_vec_neon.c')
 endif
 
 sources += files(
Signature · 1 line
2.43.0
Acked-by on v2 1/1 Applied
#5 lore raw

Re: [PATCH v2] net/ice: add NEON-optimised Rx/Tx vector paths

Quoted 8 lines · Jay Wang

This patch adds the NEON-optimised Rx and Tx paths to the ice driver.

Tested on Ampere One platform with Intel E810-C NIC and 100G connection. Tested with a single core and testpmd io forwarding mode. Observed ~30% performance boost in the above test compared to the default scalar path.

Quoted 10 lines
---
 .mailmap                                  |   1 +
 drivers/net/intel/ice/ice_ethdev.h        |   3 +
 drivers/net/intel/ice/ice_rxtx.c          |  53 +-
 drivers/net/intel/ice/ice_rxtx.h          |   6 +
 drivers/net/intel/ice/ice_rxtx_vec_neon.c | 761 ++++++++++++++++++++++
 drivers/net/intel/ice/meson.build         |   2 +
 6 files changed, 824 insertions(+), 2 deletions(-)
 create mode 100644 drivers/net/intel/ice/ice_rxtx_vec_neon.c

Applied to dpdk-next-net-intel

Thanks,
/Bruce

#6 lore raw

Re: [PATCH v2] net/ice: add NEON-optimised Rx/Tx vector paths

Quoted 19 lines · Bruce Richardson
Quoted 8 lines · Jay Wang

This patch adds the NEON-optimised Rx and Tx paths to the ice driver.

Tested on Ampere One platform with Intel E810-C NIC and 100G connection. Tested with a single core and testpmd io forwarding mode. Observed ~30% performance boost in the above test compared to the default scalar path.

Quoted 10 lines
---
 .mailmap                                  |   1 +
 drivers/net/intel/ice/ice_ethdev.h        |   3 +
 drivers/net/intel/ice/ice_rxtx.c          |  53 +-
 drivers/net/intel/ice/ice_rxtx.h          |   6 +
 drivers/net/intel/ice/ice_rxtx_vec_neon.c | 761 ++++++++++++++++++++++
 drivers/net/intel/ice/meson.build         |   2 +
 6 files changed, 824 insertions(+), 2 deletions(-)
 create mode 100644 drivers/net/intel/ice/ice_rxtx_vec_neon.c
Quoted 4 lines

Applied to dpdk-next-net-intel

Unfortunately, I've had to back this patch out again as it has caused errors in the CI over the weekend e.g. [1]. No errors were reported with the patch before I merged it, but I see now that not all tests were successfully run on it. Can you please take a look at the CI errors reported and submit a new version of this patch.

Thanks,
/Bruce

[1] https://lab.dpdk.org/results/dashboard/testruns/logs/1985291/

author
#7 lore raw

RE: [PATCH v2] net/ice: add NEON-optimised Rx/Tx vector paths

Hi Bruce,

Quoted 1 line

Unfortunately, I've had to back this patch out again as it has caused errors in the CI over the weekend e.g. [1]. No errors were reported with the patch before I merged it, but I see now that not all tests were successfully run on it. Can you please take a look at the CI errors reported and submit a new version of this patch.

Thanks for reporting the issue!
I didn't realise that some of the intrinsics, such as vzip1q/vzip2q, are only supported on AArch64. This explains the compilation errors we're seeing on AArch32 when cross-building DPDK for the AArch32 unit tests. I'll address this in the next revision by using vzip intrinsics to ensure compatibility across both AArch64 and AArch32.

Best Regards,
Jay Wang

-----Original Message-----
From: Bruce Richardson <[email protected]>
Sent: Tuesday, April 7, 2026 11:47 AM
To: Jay Wang <[email protected]>
Cc: [email protected]; Anatoly Burakov <[email protected]>; [email protected]; nd <[email protected]>
Subject: Re: [PATCH v2] net/ice: add NEON-optimised Rx/Tx vector paths

Quoted 19 lines · Bruce Richardson
Quoted 16 lines · Jay Wang

This patch adds the NEON-optimised Rx and Tx paths to the ice driver.

Tested on Ampere One platform with Intel E810-C NIC and 100G connection. Tested with a single core and testpmd io forwarding mode. Observed ~30% performance boost in the above test compared to the default scalar path.

---
 .mailmap                                  |   1 +
 drivers/net/intel/ice/ice_ethdev.h        |   3 +
 drivers/net/intel/ice/ice_rxtx.c          |  53 +-
 drivers/net/intel/ice/ice_rxtx.h          |   6 +
 drivers/net/intel/ice/ice_rxtx_vec_neon.c | 761 ++++++++++++++++++++++
 drivers/net/intel/ice/meson.build         |   2 +
 6 files changed, 824 insertions(+), 2 deletions(-)  create mode 
Quoted 2 lines

100644 drivers/net/intel/ice/ice_rxtx_vec_neon.c

Quoted 4 lines

Applied to dpdk-next-net-intel

Unfortunately, I've had to back this patch out again as it has caused errors in the CI over the weekend e.g. [1]. No errors were reported with the patch before I merged it, but I see now that not all tests were successfully run on it. Can you please take a look at the CI errors reported and submit a new version of this patch.

Thanks,
/Bruce

[1] https://lab.dpdk.org/results/dashboard/testruns/logs/1985291/

author
#8 lore raw

cover net/ice: add NEON-optimised Rx/Tx vector paths

In reply to Jay Wang

This patch introduces NEON-optimised Rx/Tx vector paths for the Intel ICE driver on AArch64 platforms.

The implementation mirrors the existing x86 vector paths while leveraging Arm NEON intrinsics to achieve comparable performance.

Tested on:
- Arm Neoverse (Ampere One)
- DPDK testpmd/l3fwd functional validation

Jay Wang (1):
net/ice: add NEON-optimised Rx/Tx vector paths

---
v3:
* Restricted the compilation of ice_rxtx_vec_neon.c to AArch64
* Fixed the compiling errors when cross build AArch32 on AArch64

v2:
* Fixed the coding style issues in the comments

 .mailmap                                  |   1 +
 drivers/net/intel/ice/ice_ethdev.h        |   3 +
 drivers/net/intel/ice/ice_rxtx.c          |  53 +-
 drivers/net/intel/ice/ice_rxtx.h          |   6 +
 drivers/net/intel/ice/ice_rxtx_vec_neon.c | 747 ++++++++++++++++++++++
 drivers/net/intel/ice/meson.build         |   2 +
 6 files changed, 810 insertions(+), 2 deletions(-)
 create mode 100644 drivers/net/intel/ice/ice_rxtx_vec_neon.c
Signature · 1 line
2.43.0
author
#9 lore raw

PATCH 1/1 net/ice: add Neon-optimised Rx/Tx vector paths

This patch adds the Neon-optimised Rx and Tx paths to the ice driver.

Tested on Ampere One platform with Intel E810-C NIC and 100G connection. Tested with a single core and testpmd io forwarding mode. Observed ~30% performance boost in the above test compared to the default scalar path.

---
 .mailmap                                  |   1 +
 drivers/net/intel/ice/ice_ethdev.h        |   3 +
 drivers/net/intel/ice/ice_rxtx.c          |  53 +-
 drivers/net/intel/ice/ice_rxtx.h          |   6 +
 drivers/net/intel/ice/ice_rxtx_vec_neon.c | 747 ++++++++++++++++++++++
 drivers/net/intel/ice/meson.build         |   2 +
 6 files changed, 810 insertions(+), 2 deletions(-)
 create mode 100644 drivers/net/intel/ice/ice_rxtx_vec_neon.c
.mailmap +1 -0
index beccc84425..dfe92b0399 100644
@@ -695,6 +695,7 @@ Javen Xu <[email protected]>
 Jay Ding <[email protected]>
 Jay Jayatheerthan <[email protected]>
 Jay Rolette <[email protected]>
+Jay Wang <[email protected]>
 Jay Zhou <[email protected]>
 Jayaprakash Shanmugam <[email protected]>
 Jean Dao <[email protected]>
drivers/net/intel/ice/ice_ethdev.h +3 -0
index 4b3718f715..f6fd3bf106 100644
@@ -204,6 +204,8 @@ enum ice_rx_func_type {
 	ICE_RX_AVX512_SCATTERED,
 	ICE_RX_AVX512_OFFLOAD,
 	ICE_RX_AVX512_SCATTERED_OFFLOAD,
+	ICE_RX_NEON,
+	ICE_RX_NEON_SCATTERED,
 };
 
 enum ice_tx_func_type {
@@ -213,6 +215,7 @@ enum ice_tx_func_type {
 	ICE_TX_AVX2_OFFLOAD,
 	ICE_TX_AVX512,
 	ICE_TX_AVX512_OFFLOAD,
+	ICE_TX_NEON,
 };
 
 struct ice_adapter;
drivers/net/intel/ice/ice_rxtx.c +51 -2

89 lines not on this page · raw message

drivers/net/intel/ice/ice_rxtx.h +6 -0
index 77ed41f9fd..999b6b30d6 100644
@@ -261,6 +261,12 @@ const uint32_t *ice_dev_supported_ptypes_get(struct rte_eth_dev *dev,
 void ice_select_rxd_to_pkt_fields_handler(struct ci_rx_queue *rxq,
 					  uint32_t rxdid);
 
+uint16_t ice_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
+				uint16_t nb_pkts);
+uint16_t ice_recv_scattered_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
+				uint16_t nb_pkts);
+uint16_t ice_xmit_pkts_vec(void *tx_queue, struct rte_mbuf **tx_pkts,
+				uint16_t nb_pkts);
 int ice_rx_vec_dev_check(struct rte_eth_dev *dev);
 int ice_tx_vec_dev_check(struct rte_eth_dev *dev);
 int ice_rxq_vec_setup(struct ci_rx_queue *rxq);
drivers/net/intel/ice/ice_rxtx_vec_neon.c new file +747 -0

747 lines not on this page · raw message

drivers/net/intel/ice/meson.build +2 -0
index 293577676f..1dc7c0109a 100644
@@ -33,6 +33,8 @@ endif
 if arch_subdir == 'x86'
     sources_avx2 += files('ice_rxtx_vec_avx2.c')
     sources_avx512 += files('ice_rxtx_vec_avx512.c')
+elif arch_subdir == 'arm'
+    sources += files('ice_rxtx_vec_neon.c')
 endif
 
 sources += files(
Signature · 1 line
2.43.0
author
#10 lore raw

PATCH v2 1/1 net/ice: add NEON-optimised Rx/Tx vector paths

In reply to Jay Wang

This patch adds the NEON-optimised Rx and Tx paths to the ice driver.

Tested on Ampere One platform with Intel E810-C NIC and 100G connection. Tested with a single core and testpmd io forwarding mode. Observed ~30% performance boost in the above test compared to the default scalar path.

---
 .mailmap                                  |   1 +
 drivers/net/intel/ice/ice_ethdev.h        |   3 +
 drivers/net/intel/ice/ice_rxtx.c          |  53 +-
 drivers/net/intel/ice/ice_rxtx.h          |   6 +
 drivers/net/intel/ice/ice_rxtx_vec_neon.c | 761 ++++++++++++++++++++++
 drivers/net/intel/ice/meson.build         |   2 +
 6 files changed, 824 insertions(+), 2 deletions(-)
 create mode 100644 drivers/net/intel/ice/ice_rxtx_vec_neon.c
.mailmap +1 -0
index beccc84425..dfe92b0399 100644
@@ -695,6 +695,7 @@ Javen Xu <[email protected]>
 Jay Ding <[email protected]>
 Jay Jayatheerthan <[email protected]>
 Jay Rolette <[email protected]>
+Jay Wang <[email protected]>
 Jay Zhou <[email protected]>
 Jayaprakash Shanmugam <[email protected]>
 Jean Dao <[email protected]>
drivers/net/intel/ice/ice_ethdev.h +3 -0
index 4b3718f715..f6fd3bf106 100644
@@ -204,6 +204,8 @@ enum ice_rx_func_type {
 	ICE_RX_AVX512_SCATTERED,
 	ICE_RX_AVX512_OFFLOAD,
 	ICE_RX_AVX512_SCATTERED_OFFLOAD,
+	ICE_RX_NEON,
+	ICE_RX_NEON_SCATTERED,
 };
 
 enum ice_tx_func_type {
@@ -213,6 +215,7 @@ enum ice_tx_func_type {
 	ICE_TX_AVX2_OFFLOAD,
 	ICE_TX_AVX512,
 	ICE_TX_AVX512_OFFLOAD,
+	ICE_TX_NEON,
 };
 
 struct ice_adapter;
drivers/net/intel/ice/ice_rxtx.c +51 -2

89 lines not on this page · raw message

drivers/net/intel/ice/ice_rxtx.h +6 -0
index 77ed41f9fd..999b6b30d6 100644
@@ -261,6 +261,12 @@ const uint32_t *ice_dev_supported_ptypes_get(struct rte_eth_dev *dev,
 void ice_select_rxd_to_pkt_fields_handler(struct ci_rx_queue *rxq,
 					  uint32_t rxdid);
 
+uint16_t ice_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
+				uint16_t nb_pkts);
+uint16_t ice_recv_scattered_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
+				uint16_t nb_pkts);
+uint16_t ice_xmit_pkts_vec(void *tx_queue, struct rte_mbuf **tx_pkts,
+				uint16_t nb_pkts);
 int ice_rx_vec_dev_check(struct rte_eth_dev *dev);
 int ice_tx_vec_dev_check(struct rte_eth_dev *dev);
 int ice_rxq_vec_setup(struct ci_rx_queue *rxq);
drivers/net/intel/ice/ice_rxtx_vec_neon.c new file +761 -0

761 lines not on this page · raw message

drivers/net/intel/ice/meson.build +2 -0
index 293577676f..1dc7c0109a 100644
@@ -33,6 +33,8 @@ endif
 if arch_subdir == 'x86'
     sources_avx2 += files('ice_rxtx_vec_avx2.c')
     sources_avx512 += files('ice_rxtx_vec_avx512.c')
+elif arch_subdir == 'arm'
+    sources += files('ice_rxtx_vec_neon.c')
 endif
 
 sources += files(
Signature · 1 line
2.43.0
author
#11 lore raw

PATCH v3 1/1 net/ice: add NEON-optimised Rx/Tx vector paths

In reply to Jay Wang

This patch adds the NEON-optimised Rx and Tx paths to the ice driver.

Tested on Ampere One platform with Intel E810-C NIC and 100G connection. Tested with a single core and testpmd io forwarding mode. Observed ~30% performance boost in the above test compared to the default scalar path.

---
 .mailmap                                  |   1 +
 drivers/net/intel/ice/ice_ethdev.h        |   3 +
 drivers/net/intel/ice/ice_rxtx.c          |  53 +-
 drivers/net/intel/ice/ice_rxtx.h          |   6 +
 drivers/net/intel/ice/ice_rxtx_vec_neon.c | 761 ++++++++++++++++++++++
 drivers/net/intel/ice/meson.build         |   2 +
 6 files changed, 824 insertions(+), 2 deletions(-)
 create mode 100644 drivers/net/intel/ice/ice_rxtx_vec_neon.c
.mailmap +1 -0
index beccc84425..dfe92b0399 100644
@@ -695,6 +695,7 @@ Javen Xu <[email protected]>
 Jay Ding <[email protected]>
 Jay Jayatheerthan <[email protected]>
 Jay Rolette <[email protected]>
+Jay Wang <[email protected]>
 Jay Zhou <[email protected]>
 Jayaprakash Shanmugam <[email protected]>
 Jean Dao <[email protected]>
drivers/net/intel/ice/ice_ethdev.h +3 -0
index 4b3718f715..f6fd3bf106 100644
@@ -204,6 +204,8 @@ enum ice_rx_func_type {
 	ICE_RX_AVX512_SCATTERED,
 	ICE_RX_AVX512_OFFLOAD,
 	ICE_RX_AVX512_SCATTERED_OFFLOAD,
+	ICE_RX_NEON,
+	ICE_RX_NEON_SCATTERED,
 };
 
 enum ice_tx_func_type {
@@ -213,6 +215,7 @@ enum ice_tx_func_type {
 	ICE_TX_AVX2_OFFLOAD,
 	ICE_TX_AVX512,
 	ICE_TX_AVX512_OFFLOAD,
+	ICE_TX_NEON,
 };
 
 struct ice_adapter;
drivers/net/intel/ice/ice_rxtx.c +51 -2

89 lines not on this page · raw message

drivers/net/intel/ice/ice_rxtx.h +6 -0
index 77ed41f9fd..999b6b30d6 100644
@@ -261,6 +261,12 @@ const uint32_t *ice_dev_supported_ptypes_get(struct rte_eth_dev *dev,
 void ice_select_rxd_to_pkt_fields_handler(struct ci_rx_queue *rxq,
 					  uint32_t rxdid);
 
+uint16_t ice_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
+				uint16_t nb_pkts);
+uint16_t ice_recv_scattered_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
+				uint16_t nb_pkts);
+uint16_t ice_xmit_pkts_vec(void *tx_queue, struct rte_mbuf **tx_pkts,
+				uint16_t nb_pkts);
 int ice_rx_vec_dev_check(struct rte_eth_dev *dev);
 int ice_tx_vec_dev_check(struct rte_eth_dev *dev);
 int ice_rxq_vec_setup(struct ci_rx_queue *rxq);
drivers/net/intel/ice/ice_rxtx_vec_neon.c new file +761 -0

761 lines not on this page · raw message

drivers/net/intel/ice/meson.build +2 -0
index 293577676f..a205304c89 100644
@@ -33,6 +33,8 @@ endif
 if arch_subdir == 'x86'
     sources_avx2 += files('ice_rxtx_vec_avx2.c')
     sources_avx512 += files('ice_rxtx_vec_avx512.c')
+elif arch_subdir == 'arm' and dpdk_conf.get('RTE_ARCH_64')
+    sources += files('ice_rxtx_vec_neon.c')
 endif
 
 sources += files(
Signature · 1 line
2.43.0
author
#12 lore raw

RE: [PATCH v3] net/ice: add NEON-optimised Rx/Tx vector paths

Hi Bruce,

I think it would be better to restrict compilation of the vectorised paths to AArch64 only.
So, rather than modifying the intrinsics as I proposed earlier, I added an extra check in the meson build configuration file, as shown in the following code snippet. Could you please help verify the patch again? I also did a sanity check by cross building DPDK for AArch32 on my local AArch64 machine, and the file was not compiled, so no build errors were reported.

Sorry for duplicating the previous patches on the website. I've marked them as superseded.

Thanks,
Jay Wang

Quoted 11 lines
drivers/net/intel/ice/meson.build +2 -0
index 293577676f..a205304c89 100644
@@ -33,6 +33,8 @@  endif
 if arch_subdir == 'x86'
  sources_avx2 += files('ice_rxtx_vec_avx2.c')
  sources_avx512 += files('ice_rxtx_vec_avx512.c')
+elif arch_subdir == 'arm' and dpdk_conf.get('RTE_ARCH_64')
+    sources += files('ice_rxtx_vec_neon.c')
 endif
#13 lore raw

Re: [PATCH v3] net/ice: add NEON-optimised Rx/Tx vector paths

Quoted 6 lines · Jay Wang

Hi Bruce,

I think it would be better to restrict compilation of the vectorised paths to AArch64 only.
So, rather than modifying the intrinsics as I proposed earlier, I added an extra check in the meson build configuration file, as shown in the following code snippet. Could you please help verify the patch again? I also did a sanity check by cross building DPDK for AArch32 on my local AArch64 machine, and the file was not compiled, so no build errors were reported.

The CI should run this patch through the various builds and report the output in patchwork. Once that goes clean I can consider merging the patch, and if not, you can submit a v4.

Quoted 2 lines

Sorry for duplicating the previous patches on the website. I've marked them as superseded.

No problem.

Quoted 14 lines

Thanks,
Jay Wang

Quoted 11 lines
drivers/net/intel/ice/meson.build +2 -0
index 293577676f..a205304c89 100644
@@ -33,6 +33,8 @@  endif
 if arch_subdir == 'x86'
  sources_avx2 += files('ice_rxtx_vec_avx2.c')
  sources_avx512 += files('ice_rxtx_vec_avx512.c')
+elif arch_subdir == 'arm' and dpdk_conf.get('RTE_ARCH_64')
+    sources += files('ice_rxtx_vec_neon.c')
 endif
author
#14 lore raw

RE: [PATCH v3] net/ice: add NEON-optimised Rx/Tx vector paths

Hi Bruce,

Quoted 1 line

The CI should run this patch through the various builds and report the output in patchwork. Once that goes clean I can consider merging the patch, and if not, you can submit a v4.

I just wanted to let you know that I've been able to reproduce the failing test cases(pmd_buffer_scatter, port_control, port_stats, dynamic_config) on the Arm Grace + Intel E810 testbed in my local environment with my patch applied.

I'm currently root-causing the issue locally and will submit a v4 once I have identified and fixed the problem.

Thanks very much for your patience and support!

Cheers,
Jay

#15 lore raw

Re: [PATCH v3] net/ice: add NEON-optimised Rx/Tx vector paths

In reply to Jay Wang
Quoted 9 lines · Jay Wang

This patch adds the NEON-optimised Rx and Tx paths to the ice driver.

Tested on Ampere One platform with Intel E810-C NIC and 100G connection. Tested with a single core and testpmd io forwarding mode. Observed ~30% performance boost in the above test compared to the default scalar path.

---

Recheck-request: iol-intel-Functional, rebase=next-net-intel

author
#16 lore raw

v4 cover net/ice: add NEON-optimised Rx/Tx vector paths

In reply to Jay Wang

This patch introduces NEON-optimised Rx/Tx vector paths for the Intel ICE driver on AArch64.

The implementation mirrors the existing x86 vector paths while leveraging Arm NEON intrinsics to achieve comparable performance

Tested on:
- Arm Neoverse (Ampere One)
- DPDK testpmd/l3fwd function validataion

---
v4:
* Fixed incorrect Rx vector DD bit extraction
* Added the check for AArch64 128-bit SIMD in selecting Tx funcs

v3:
* Restricted the compilation of ice_rxtx_vec_neon.c to AArch64
* Fixed the compiling errors when cross build AArch32 on AArch64

v2:
* Fixed the coding style issues in the comments

Jay Wang (1):
net/ice: add NEON-optimised Rx/Tx vector paths

 .mailmap                                  |   1 +
 drivers/net/intel/ice/ice_ethdev.h        |   3 +
 drivers/net/intel/ice/ice_rxtx.c          |  58 +-
 drivers/net/intel/ice/ice_rxtx.h          |   6 +
 drivers/net/intel/ice/ice_rxtx_vec_neon.c | 745 ++++++++++++++++++++++
 drivers/net/intel/ice/meson.build         |   2 +
 6 files changed, 813 insertions(+), 2 deletions(-)
 create mode 100644 drivers/net/intel/ice/ice_rxtx_vec_neon.c
Signature · 1 line
2.43.0
author
#17 lore raw

PATCH v4 1/1 net/ice: add NEON-optimised Rx/Tx vector paths

This patch adds the NEON-optimised Rx and Tx paths to the ice driver.

Tested on Ampere One platform with Intel E810-C NIC and 100G connection. Tested with a single core and testpmd io forwarding mode. Observed ~30% performance boost in the above test compared to the default scalar path.

---
 .mailmap                                  |   1 +
 drivers/net/intel/ice/ice_ethdev.h        |   3 +
 drivers/net/intel/ice/ice_rxtx.c          |  58 +-
 drivers/net/intel/ice/ice_rxtx.h          |   6 +
 drivers/net/intel/ice/ice_rxtx_vec_neon.c | 745 ++++++++++++++++++++++
 drivers/net/intel/ice/meson.build         |   2 +
 6 files changed, 813 insertions(+), 2 deletions(-)
 create mode 100644 drivers/net/intel/ice/ice_rxtx_vec_neon.c
.mailmap +1 -0
index beccc84425..dfe92b0399 100644
@@ -695,6 +695,7 @@ Javen Xu <[email protected]>
 Jay Ding <[email protected]>
 Jay Jayatheerthan <[email protected]>
 Jay Rolette <[email protected]>
+Jay Wang <[email protected]>
 Jay Zhou <[email protected]>
 Jayaprakash Shanmugam <[email protected]>
 Jean Dao <[email protected]>
drivers/net/intel/ice/ice_ethdev.h +3 -0
index 4b3718f715..f6fd3bf106 100644
@@ -204,6 +204,8 @@ enum ice_rx_func_type {
 	ICE_RX_AVX512_SCATTERED,
 	ICE_RX_AVX512_OFFLOAD,
 	ICE_RX_AVX512_SCATTERED_OFFLOAD,
+	ICE_RX_NEON,
+	ICE_RX_NEON_SCATTERED,
 };
 
 enum ice_tx_func_type {
@@ -213,6 +215,7 @@ enum ice_tx_func_type {
 	ICE_TX_AVX2_OFFLOAD,
 	ICE_TX_AVX512,
 	ICE_TX_AVX512_OFFLOAD,
+	ICE_TX_NEON,
 };
 
 struct ice_adapter;
drivers/net/intel/ice/ice_rxtx.c +56 -2

102 lines not on this page · raw message

drivers/net/intel/ice/ice_rxtx.h +6 -0
index 77ed41f9fd..999b6b30d6 100644
@@ -261,6 +261,12 @@ const uint32_t *ice_dev_supported_ptypes_get(struct rte_eth_dev *dev,
 void ice_select_rxd_to_pkt_fields_handler(struct ci_rx_queue *rxq,
 					  uint32_t rxdid);
 
+uint16_t ice_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
+				uint16_t nb_pkts);
+uint16_t ice_recv_scattered_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
+				uint16_t nb_pkts);
+uint16_t ice_xmit_pkts_vec(void *tx_queue, struct rte_mbuf **tx_pkts,
+				uint16_t nb_pkts);
 int ice_rx_vec_dev_check(struct rte_eth_dev *dev);
 int ice_tx_vec_dev_check(struct rte_eth_dev *dev);
 int ice_rxq_vec_setup(struct ci_rx_queue *rxq);
drivers/net/intel/ice/ice_rxtx_vec_neon.c new file +745 -0

745 lines not on this page · raw message

drivers/net/intel/ice/meson.build +2 -0
index 293577676f..a205304c89 100644
@@ -33,6 +33,8 @@ endif
 if arch_subdir == 'x86'
     sources_avx2 += files('ice_rxtx_vec_avx2.c')
     sources_avx512 += files('ice_rxtx_vec_avx512.c')
+elif arch_subdir == 'arm' and dpdk_conf.get('RTE_ARCH_64')
+    sources += files('ice_rxtx_vec_neon.c')
 endif
 
 sources += files(
Signature · 1 line
2.43.0
author
#18 lore raw

PATCH v5 1/1 net/ice: add NEON-optimised Rx/Tx vector paths

This patch adds the NEON-optimised Rx and Tx paths to the ice driver on AArch64.

The single-core testpmd benchmarking was conducted by Andrew Bailey on an Arm Grace system with an Intel E810 100G NIC. The test was run with txq/rxq = 1 and 2048 descriptors. The throughput performance uplift is shown in the following table.

| buffer sz (B) | prev (MPPS) | optimised (MPPS) | uplift |
| ------------- | ----------- | ---------------- | ------ |
| 64            | 77.613      | 110.874          | 42.9%  |
| 128           | 77.187      | 103.547          | 34.2%  |
| 256           | 55.183      | 55.240           | 0.1%   |
| 512           | 28.800      | 29.031           | 0.8%   |
| 1024          | 14.609      | 14.556           | -0.4%  |
| 1518          | 9.726       | 9.756            | 0.3%   |

---
v5:
- Updated benchmark results on Grace with Intel E810
- Fixed a few coding style issues

v4:
- Resolved the icorrect Rx vector DD bit extraction issue leading to the wrong czt-based counting of the received packets

v3:
- Fixed the compiling errors when cross build AArch32 on AArch64 by restricting the compilation of ice_rxtx_vec_neon.c to AArch64

v2:
- Fixed the coding style issues reported by the checkpatch script

 .mailmap                                  |   1 +
 drivers/net/intel/ice/ice_ethdev.h        |   3 +
 drivers/net/intel/ice/ice_rxtx.c          |  58 +-
 drivers/net/intel/ice/ice_rxtx.h          |   6 +
 drivers/net/intel/ice/ice_rxtx_vec_neon.c | 745 ++++++++++++++++++++++
 drivers/net/intel/ice/meson.build         |   2 +
 6 files changed, 813 insertions(+), 2 deletions(-)
 create mode 100644 drivers/net/intel/ice/ice_rxtx_vec_neon.c
.mailmap +1 -0
index 0e0d83e1c6..873bc62d0d 100644
@@ -695,6 +695,7 @@ Javen Xu <[email protected]>
 Jay Ding <[email protected]>
 Jay Jayatheerthan <[email protected]>
 Jay Rolette <[email protected]>
+Jay Wang <[email protected]>
 Jay Zhou <[email protected]>
 Jayaprakash Shanmugam <[email protected]>
 Jean Dao <[email protected]>
drivers/net/intel/ice/ice_ethdev.h +3 -0
index 4b3718f715..f6fd3bf106 100644
@@ -204,6 +204,8 @@ enum ice_rx_func_type {
 	ICE_RX_AVX512_SCATTERED,
 	ICE_RX_AVX512_OFFLOAD,
 	ICE_RX_AVX512_SCATTERED_OFFLOAD,
+	ICE_RX_NEON,
+	ICE_RX_NEON_SCATTERED,
 };
 
 enum ice_tx_func_type {
@@ -213,6 +215,7 @@ enum ice_tx_func_type {
 	ICE_TX_AVX2_OFFLOAD,
 	ICE_TX_AVX512,
 	ICE_TX_AVX512_OFFLOAD,
+	ICE_TX_NEON,
 };
 
 struct ice_adapter;
drivers/net/intel/ice/ice_rxtx.c +56 -2

102 lines not on this page · raw message

drivers/net/intel/ice/ice_rxtx.h +6 -0
index 77ed41f9fd..999b6b30d6 100644
@@ -261,6 +261,12 @@ const uint32_t *ice_dev_supported_ptypes_get(struct rte_eth_dev *dev,
 void ice_select_rxd_to_pkt_fields_handler(struct ci_rx_queue *rxq,
 					  uint32_t rxdid);
 
+uint16_t ice_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
+				uint16_t nb_pkts);
+uint16_t ice_recv_scattered_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
+				uint16_t nb_pkts);
+uint16_t ice_xmit_pkts_vec(void *tx_queue, struct rte_mbuf **tx_pkts,
+				uint16_t nb_pkts);
 int ice_rx_vec_dev_check(struct rte_eth_dev *dev);
 int ice_tx_vec_dev_check(struct rte_eth_dev *dev);
 int ice_rxq_vec_setup(struct ci_rx_queue *rxq);
drivers/net/intel/ice/ice_rxtx_vec_neon.c new file +745 -0

745 lines not on this page · raw message

drivers/net/intel/ice/meson.build +2 -0
index 293577676f..a205304c89 100644
@@ -33,6 +33,8 @@ endif
 if arch_subdir == 'x86'
     sources_avx2 += files('ice_rxtx_vec_avx2.c')
     sources_avx512 += files('ice_rxtx_vec_avx512.c')
+elif arch_subdir == 'arm' and dpdk_conf.get('RTE_ARCH_64')
+    sources += files('ice_rxtx_vec_neon.c')
 endif
 
 sources += files(
Signature · 1 line
2.43.0
#19 lore raw

Re: [PATCH v5] net/ice: add NEON-optimised Rx/Tx vector paths

Quoted 17 lines · Jay Wang

This patch adds the NEON-optimised Rx and Tx paths to the ice driver on AArch64.

The single-core testpmd benchmarking was conducted by Andrew Bailey on an Arm Grace system with an Intel E810 100G NIC. The test was run with txq/rxq = 1 and 2048 descriptors. The throughput performance uplift is shown in the following table.

| buffer sz (B) | prev (MPPS) | optimised (MPPS) | uplift |
| ------------- | ----------- | ---------------- | ------ |
| 64            | 77.613      | 110.874          | 42.9%  |
| 128           | 77.187      | 103.547          | 34.2%  |
| 256           | 55.183      | 55.240           | 0.1%   |
| 512           | 28.800      | 29.031           | 0.8%   |
| 1024          | 14.609      | 14.556           | -0.4%  |
| 1518          | 9.726       | 9.756            | 0.3%   |
Quoted 3 lines

---

Applied to dpdk-next-net-intel.

Thanks,
/Bruce

#20 lore raw
In reply to Jay Wang
Quoted 11 lines · Jay Wang

This patch introduces Neon-optimised Rx/Tx vector paths for the Intel ICE driver on AArch64 platforms.

The implementation mirrors the existing x86 vector paths while leveraging Arm Neon intrinsics to achieve comparable performance.

Tested on: - Arm Neoverse (Ampere One) - DPDK testpmd/l3fwd functional validation

Jay Wang (1): net/ice: add Neon-optimised Rx/Tx vector paths

Thanks for this patch. I see one error reported in the CIs in patchwork, which should be investigated. Otherwise, since it's late in the release cycle for 26.03, I'm going to defer this patch for 26.07, and will look at it as part of that release cycle.

Thanks,
/Bruce