l3fwd improvements
cover l3fwd improvements
This series of patches include changes to l3fwd example application. Some improvements are made for better usage of CPU cycles and memory.
Ruifeng Wang (4):
examples/l3fwd: tune prefetch for better performance examples/l3fwd: eliminate unnecessary calculations examples/l3fwd: eliminate unnecessary reloads in loop examples/l3fwd: make data struct to be memory efficient
examples/l3fwd/l3fwd.h | 12 ++++++------ examples/l3fwd/l3fwd_common.h | 4 ++-- examples/l3fwd/l3fwd_em.c | 6 +++--- examples/l3fwd/l3fwd_lpm.c | 16 +++++++++------- examples/l3fwd/l3fwd_lpm_neon.h | 20 ++++++++++---------- 5 files changed, 30 insertions(+), 28 deletions(-)
Signature · 1 line
2.25.1
PATCH 1/4 examples/l3fwd: tune prefetch for better performance
Packet header is prefetched before packet processing for better memory access performance. As L2 header will be updated by l3fwd, using of prefetch for store hint will set cache line to proper status and reduce cache maintenance overhead.
With this change, 12.9% performance uplift was measured on N1SDP platform with MLX5 NIC.
--- examples/l3fwd/l3fwd_lpm_neon.h | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
examples/l3fwd/l3fwd_lpm_neon.h +5 -5
@@ -97,13 +97,13 @@ l3fwd_lpm_send_packets(int nb_rx, struct rte_mbuf **pkts_burst,if (k) {for (i =0 ; i <FWDSTEP ; i++) { -rte_prefetch0 (rte_pktmbuf_mtod (pkts_burst [i], +rte_prefetch0_write (rte_pktmbuf_mtod (pkts_burst [i], struct rte_ether_hdr *) + 1); }for (j =0 ; j != k -FWDSTEP ; j +=FWDSTEP ) {for (i =0 ; i <FWDSTEP ; i++) { -rte_prefetch0 (rte_pktmbuf_mtod ( +rte_prefetch0_write (rte_pktmbuf_mtod (pkts_burst [j + i +FWDSTEP ], struct rte_ether_hdr *) + 1); }
@@ -124,17 +124,17 @@ l3fwd_lpm_send_packets(int nb_rx, struct rte_mbuf **pkts_burst,/* Prefetch last up to 3 packets one by one */ switch (m) {case 3 : -rte_prefetch0 (rte_pktmbuf_mtod (pkts_burst [j], +rte_prefetch0_write (rte_pktmbuf_mtod (pkts_burst [j], struct rte_ether_hdr *) + 1); j++;/* fallthrough */ case 2 : -rte_prefetch0 (rte_pktmbuf_mtod (pkts_burst [j], +rte_prefetch0_write (rte_pktmbuf_mtod (pkts_burst [j], struct rte_ether_hdr *) + 1); j++;/* fallthrough */ case 1 : -rte_prefetch0 (rte_pktmbuf_mtod (pkts_burst [j], +rte_prefetch0_write (rte_pktmbuf_mtod (pkts_burst [j], struct rte_ether_hdr *) + 1); j++; }
Signature · 1 line
2.25.1
Re: [dpdk-dev] [PATCH 1/4] examples/l3fwd: tune prefetch for better performance
Quoted 5 lines · Ruifeng Wang
Packet header is prefetched before packet processing for better memory access performance. As L2 header will be updated by l3fwd, using of prefetch for store hint will set cache line to proper status and reduce cache maintenance overhead.
The code does read the cache line too. Right?
Quoted 8 lines
With this change, 12.9% performance uplift was measured on N1SDP platform with MLX5 NIC.
Suggested-by Honnappa Nagarahalli <[email protected]>Signed-off-by Ruifeng Wang <[email protected]>Reviewed-by Honnappa Nagarahalli <[email protected]>
On the octeontx2 platform, It is 2% regression.
Looks like micro architecture-specific item of handing write hint on the memory the area that does read and write.
I am testing the LPM lookup miss case.
My test command:
./build/examples/dpdk-l3fwd -c 0x0100 -- -p 0x1 --config="(0,0,8)" -P
Quoted 48 lines
--- examples/l3fwd/l3fwd_lpm_neon.h | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)examples/l3fwd/l3fwd_lpm_neon.h +5
-6@@ -97,13 +97,13 @@ l3fwd_lpm_send_packets(int nb_rx, struct rte_mbuf **pkts_burst,if (k) {for (i =0 ; i <FWDSTEP ; i++) { -rte_prefetch0 (rte_pktmbuf_mtod (pkts_burst [i], +rte_prefetch0_write (rte_pktmbuf_mtod (pkts_burst [i], struct rte_ether_hdr *) + 1); }for (j =0 ; j != k -FWDSTEP ; j +=FWDSTEP ) {for (i =0 ; i <FWDSTEP ; i++) { -rte_prefetch0 (rte_pktmbuf_mtod ( +rte_prefetch0_write (rte_pktmbuf_mtod (pkts_burst [j + i +FWDSTEP ], struct rte_ether_hdr *) + 1); }@@ -124,17 +124,17 @@ l3fwd_lpm_send_packets(int nb_rx, struct rte_mbuf **pkts_burst,/* Prefetch last up to 3 packets one by one */ switch (m) {case 3 : -rte_prefetch0 (rte_pktmbuf_mtod (pkts_burst [j], +rte_prefetch0_write (rte_pktmbuf_mtod (pkts_burst [j], struct rte_ether_hdr *) + 1); j++;/* fallthrough */ case 2 : -rte_prefetch0 (rte_pktmbuf_mtod (pkts_burst [j], +rte_prefetch0_write (rte_pktmbuf_mtod (pkts_burst [j], struct rte_ether_hdr *) + 1); j++;/* fallthrough */ case 1 : -rte_prefetch0 (rte_pktmbuf_mtod (pkts_burst [j], +rte_prefetch0_write (rte_pktmbuf_mtod (pkts_burst [j], struct rte_ether_hdr *) + 1); j++; } --2.25.1
Re: [dpdk-dev] [PATCH 1/4] examples/l3fwd: tune prefetch for better performance
<snip>
Quoted 14 lines
Quoted 5 lines · Ruifeng Wang
Packet header is prefetched before packet processing for better memory access performance. As L2 header will be updated by l3fwd, using of prefetch for store hint will set cache line to proper status and reduce cache maintenance overhead.
The code does read the cache line too. Right?
Quoted 4 lines
With this change, 12.9% performance uplift was measured on N1SDP platform with MLX5 NIC.
Quoted 7 lines
Quoted 4 lines
Suggested-by Honnappa Nagarahalli <[email protected]>Signed-off-by Ruifeng Wang <[email protected]>Reviewed-by Honnappa Nagarahalli <[email protected]>On the octeontx2 platform, It is 2% regression.
Thanks Jerin for testing.
It would be good to know the results from others with A72 platforms.
Quoted 66 lines
Looks like micro architecture-specific item of handing write hint on the memory the area that does read and write.
I am testing the LPM lookup miss case.
My test command:
./build/examples/dpdk-l3fwd -c0x0100-- -p0x1--config="(0,0,8)" -Pexamples/l3fwd/l3fwd_lpm_neon.h l3fwd_lpm_send_packets(int nb_rx, struct rte_mbuf
Quoted 27 lines
--- examples/l3fwd/l3fwd_lpm_neon.h | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)(quoted diff) +0
-0
b/examples/l3fwd/l3fwd_lpm_neon.hindexd6c0ba64a..ae8840694100644examples/l3fwd/l3fwd_lpm_neon.h +3
-2@@ -97,13 +97,13 @@ l3fwd_lpm_send_packets(int nb_rx, struct rte_mbuf **pkts_burst, if (k) {for (i =0 ; i <FWDSTEP ; i++) { -rte_prefetch0 (rte_pktmbuf_mtod (pkts_burst [i], + +rte_prefetch0_write (rte_pktmbuf_mtod (pkts_burst [i], struct rte_ether_hdr *) + 1); }for (j =0 ; j != k -FWDSTEP ; j +=FWDSTEP ) {for (i =0 ; i <FWDSTEP ; i++) { -rte_prefetch0 (rte_pktmbuf_mtod ( +rte_prefetch0_write (rte_pktmbuf_mtod (pkts_burst [j + i +FWDSTEP ], struct rte_ether_hdr *) + 1); }@@ -124,17 +124,17 @@ l3fwd_lpm_send_packets(int nb_rx, struct rte_mbuf**pkts_burst,
Quoted 26 lines
examples/l3fwd/l3fwd_lpm_neon.h +6
-4/* Prefetch last up to 3 packets one by one */ switch (m) {case 3 : -rte_prefetch0 (rte_pktmbuf_mtod (pkts_burst [j], + +rte_prefetch0_write (rte_pktmbuf_mtod (pkts_burst [j], struct rte_ether_hdr *) + 1); j++;/* fallthrough */ case 2 : -rte_prefetch0 (rte_pktmbuf_mtod (pkts_burst [j], + +rte_prefetch0_write (rte_pktmbuf_mtod (pkts_burst [j], struct rte_ether_hdr *) + 1); j++;/* fallthrough */ case 1 : -rte_prefetch0 (rte_pktmbuf_mtod (pkts_burst [j], + +rte_prefetch0_write (rte_pktmbuf_mtod (pkts_burst [j], struct rte_ether_hdr *) + 1); j++; } --2.25.1
Re: [dpdk-dev] [PATCH 1/4] examples/l3fwd: tune prefetch for better performance
In reply to Jerin JacobQuoted 14 lines
-----Original Message-----
From: Jerin Jacob <[email protected]>
Sent: Wednesday, April 14, 2021 2:51 AM
To: Ruifeng Wang <[email protected]>
Cc: [email protected]; [email protected]; Ferruh Yigit <[email protected]>; [email protected]; David Marchand <[email protected]>; dpdk-dev <[email protected]>; nd <[email protected]>; Honnappa Nagarahalli <[email protected]>
Subject: Re: [dpdk-dev] [PATCH 1/4] examples/l3fwd: tune prefetch for better performanceQuoted 1 line · Ruifeng Wang
Quoted 7 lines
Quoted 4 lines
Packet header is prefetched before packet processing for better memory access performance. As L2 header will be updated by l3fwd, using of prefetch for store hint will set cache line to proper status and reduce cache maintenance overhead.
The code does read the cache line too. Right?
Yes, the code also read the cache line.
And prefetch to write helps writes. It saves snooping cost.
Quoted 6 lines
Quoted 6 lines
With this change, 12.9% performance uplift was measured on N1SDP platform with MLX5 NIC.
Suggested-by Honnappa Nagarahalli <[email protected]>
Quoted 9 lines
Quoted 2 lines
Signed-off-by Ruifeng Wang <[email protected]>Reviewed-by Honnappa Nagarahalli <[email protected]>On the octeontx2 platform, It is 2% regression.
Looks like micro architecture-specific item of handing write hint on the memory the area that does read and write.
OK. Performance impact of the write hint may be different on various micro architecture implementations.
The 12% measured on N1 is not a small enough number to ignore. How about using a flag to distinguish prefetches invoked on different SoCs? A compile time flag like RTE_USE_PREFETCH_WRITE is introduced. And SoCs enable it based on need.
Quoted 63 lines
I am testing the LPM lookup miss case.
My test command:
./build/examples/dpdk-l3fwd -c0x0100-- -p0x1--config="(0,0,8)" -Pexamples/l3fwd/l3fwd_lpm_neon.h l3fwd_lpm_send_packets(int nb_rx, struct
Quoted 9 lines
--- examples/l3fwd/l3fwd_lpm_neon.h | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)(quoted diff) +0
-0
b/examples/l3fwd/l3fwd_lpm_neon.hindexd6c0ba64a..ae8840694100644examples/l3fwd/l3fwd_lpm_neon.h +0
-0@@ -97,13 +97,13 @@ l3fwd_lpm_send_packets(int nb_rx, struct
rte_mbufexamples/l3fwd/l3fwd_lpm_neon.h l3fwd_lpm_send_packets(int nb_rx, struct
Quoted 18 lines
examples/l3fwd/l3fwd_lpm_neon.h +3
-2**pkts_burst, if (k) {for (i =0 ; i <FWDSTEP ; i++) { -rte_prefetch0 (rte_pktmbuf_mtod (pkts_burst [i], + +rte_prefetch0_write (rte_pktmbuf_mtod (pkts_burst [i], struct rte_ether_hdr *) + 1); }for (j =0 ; j != k -FWDSTEP ; j +=FWDSTEP ) {for (i =0 ; i <FWDSTEP ; i++) { -rte_prefetch0 (rte_pktmbuf_mtod ( +rte_prefetch0_write (rte_pktmbuf_mtod (pkts_burst [j + i +FWDSTEP ], struct rte_ether_hdr *) + 1); }@@ -124,17 +124,17 @@ l3fwd_lpm_send_packets(int nb_rx, struct
rte_mbuf**pkts_burst,Quoted 26 lines
examples/l3fwd/l3fwd_lpm_neon.h +6
-4/* Prefetch last up to 3 packets one by one */ switch (m) {case 3 : -rte_prefetch0 (rte_pktmbuf_mtod (pkts_burst [j], + +rte_prefetch0_write (rte_pktmbuf_mtod (pkts_burst [j], struct rte_ether_hdr *) + 1); j++;/* fallthrough */ case 2 : -rte_prefetch0 (rte_pktmbuf_mtod (pkts_burst [j], + +rte_prefetch0_write (rte_pktmbuf_mtod (pkts_burst [j], struct rte_ether_hdr *) + 1); j++;/* fallthrough */ case 1 : -rte_prefetch0 (rte_pktmbuf_mtod (pkts_burst [j], + +rte_prefetch0_write (rte_pktmbuf_mtod (pkts_burst [j], struct rte_ether_hdr *) + 1); j++; } --2.25.1
PATCH 2/4 examples/l3fwd: eliminate unnecessary calculations
In reply to Ruifeng WangBoth L2 and L3 headers will be used in forward processing. And these two headers are in the same cache line. It has the same effect for prefetching with L2 header address and prefetching with L3 header address.
Changed to use L2 header address for prefetching. The change showed no measurable performance improvement, but it definitely removed unnecessary instructions for address calculation.
--- examples/l3fwd/l3fwd_lpm_neon.h | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
examples/l3fwd/l3fwd_lpm_neon.h +5 -5
@@ -98,14 +98,14 @@ l3fwd_lpm_send_packets(int nb_rx, struct rte_mbuf **pkts_burst,if (k) {for (i =0 ; i <FWDSTEP ; i++) {rte_prefetch0_write (rte_pktmbuf_mtod(pkts_burst [i], - struct rte_ether_hdr *) + 1); +void *)); }for (j =0 ; j != k -FWDSTEP ; j +=FWDSTEP ) {for (i =0 ; i <FWDSTEP ; i++) {rte_prefetch0_write (rte_pktmbuf_mtod(pkts_burst [j + i +FWDSTEP ], - struct rte_ether_hdr *) + 1); +void *)); }processx4_step1 (&pkts_burst[j], &dip, &ipv4_flag);
@@ -125,17 +125,17 @@ l3fwd_lpm_send_packets(int nb_rx, struct rte_mbuf **pkts_burst,switch (m) {case 3 :rte_prefetch0_write (rte_pktmbuf_mtod (pkts_burst [j], - struct rte_ether_hdr *) + 1); +void *)); j++;/* fallthrough */ case 2 :rte_prefetch0_write (rte_pktmbuf_mtod (pkts_burst [j], - struct rte_ether_hdr *) + 1); +void *)); j++;/* fallthrough */ case 1 :rte_prefetch0_write (rte_pktmbuf_mtod (pkts_burst [j], - struct rte_ether_hdr *) + 1); +void *)); j++; }
Signature · 1 line
2.25.1
Re: [dpdk-dev] [PATCH 2/4] examples/l3fwd: eliminate unnecessary calculations
Quoted 8 lines · Ruifeng Wang
Both L2 and L3 headers will be used in forward processing. And these two headers are in the same cache line. It has the same effect for prefetching with L2 header address and prefetching with L3 header address.
Changed to use L2 header address for prefetching. The change showed no measurable performance improvement, but it definitely removed
Same here.
Quoted 5 lines
unnecessary instructions for address calculation.
Signed-off-by Ruifeng Wang <[email protected]>
Quoted 49 lines
--- examples/l3fwd/l3fwd_lpm_neon.h | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)examples/l3fwd/l3fwd_lpm_neon.h +5
-6@@ -98,14 +98,14 @@ l3fwd_lpm_send_packets(int nb_rx, struct rte_mbuf **pkts_burst,if (k) {for (i =0 ; i <FWDSTEP ; i++) {rte_prefetch0_write (rte_pktmbuf_mtod(pkts_burst [i], - struct rte_ether_hdr *) + 1); +void *)); }for (j =0 ; j != k -FWDSTEP ; j +=FWDSTEP ) {for (i =0 ; i <FWDSTEP ; i++) {rte_prefetch0_write (rte_pktmbuf_mtod(pkts_burst [j + i +FWDSTEP ], - struct rte_ether_hdr *) + 1); +void *)); }processx4_step1 (&pkts_burst[j], &dip, &ipv4_flag);@@ -125,17 +125,17 @@ l3fwd_lpm_send_packets(int nb_rx, struct rte_mbuf **pkts_burst,switch (m) {case 3 :rte_prefetch0_write (rte_pktmbuf_mtod (pkts_burst [j], - struct rte_ether_hdr *) + 1); +void *)); j++;/* fallthrough */ case 2 :rte_prefetch0_write (rte_pktmbuf_mtod (pkts_burst [j], - struct rte_ether_hdr *) + 1); +void *)); j++;/* fallthrough */ case 1 :rte_prefetch0_write (rte_pktmbuf_mtod (pkts_burst [j], - struct rte_ether_hdr *) + 1); +void *)); j++; } --2.25.1
PATCH 3/4 examples/l3fwd: eliminate unnecessary reloads in loop
In reply to Ruifeng WangNumber of rx queue and number of rx port in lcore config are constants during the period of l3 forward application running. But compiler has no this information.
Copied values from lcore config to local variables and used the local variables for iteration. Compiler can see that the local variables are not changed, so qconf reloads at each iteration can be eliminated.
The change showed 1.8% performance uplift in single core, single port, single queue test on N1SDP platform with MLX5 NIC.
--- examples/l3fwd/l3fwd_lpm.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-)
examples/l3fwd/l3fwd_lpm.c +6 -4
@@ -190,14 +190,16 @@ lpm_main_loop(__rte_unused void *dummy) lcore_id =rte_lcore_id (); qconf = &lcore_conf[lcore_id]; -if (qconf->n_rx_queue ==0 ) { +uint16_t n_rx_q = qconf->n_rx_queue; +uint16_t n_tx_p = qconf->n_tx_port; +if (n_rx_q ==0 ) {RTE_LOG (INFO ,L3FWD ,"lcore %u has nothing to do\n" , lcore_id);return 0 ; }RTE_LOG (INFO ,L3FWD ,"entering main loop on lcore %u\n" , lcore_id); -for (i =0 ; i < qconf->n_rx_queue; i++) { +for (i =0 ; i < n_rx_q; i++) { portid = qconf->rx_queue_list[i].port_id; queueid = qconf->rx_queue_list[i].queue_id;
@@ -216,7 +218,7 @@ lpm_main_loop(__rte_unused void *dummy) diff_tsc = cur_tsc - prev_tsc;if (unlikely (diff_tsc > drain_tsc)) { -for (i =0 ; i < qconf->n_tx_port; ++i) { +for (i =0 ; i < n_tx_p; ++i) { portid = qconf->tx_port_id[i];if (qconf->tx_mbufs[portid].len ==0 )continue ;
@@ -232,7 +234,7 @@ lpm_main_loop(__rte_unused void *dummy)/* * Read packet from RX queues */ -for (i =0 ; i < qconf->n_rx_queue; ++i) { +for (i =0 ; i < n_rx_q; ++i) { portid = qconf->rx_queue_list[i].port_id; queueid = qconf->rx_queue_list[i].queue_id; nb_rx = rte_eth_rx_burst(portid, queueid, pkts_burst,
Signature · 1 line
2.25.1
Re: [dpdk-dev] [PATCH 3/4] examples/l3fwd: eliminate unnecessary reloads in loop
Quoted 11 lines · Ruifeng Wang
Number of rx queue and number of rx port in lcore config are constants during the period of l3 forward application running. But compiler has no this information.
Copied values from lcore config to local variables and used the local variables for iteration. Compiler can see that the local variables are not changed, so qconf reloads at each iteration can be eliminated.
The change showed 1.8% performance uplift in single core, single port, single queue test on N1SDP platform with MLX5 NIC.
At least, in octeontx2, I dont see any performance improvement. But change looks good. Please find below a comment.
Quoted 11 lines
Signed-off-by Ruifeng Wang <[email protected]>--- examples/l3fwd/l3fwd_lpm.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-)examples/l3fwd/l3fwd_lpm.c +0
-0
examples/l3fwd/l3fwd_lpm.c lpm_main_loop(__rte_unused void *dummy) · line 194
Quoted 6 lines
examples/l3fwd/l3fwd_lpm.c +2
-1@@ -190,14 +190,16 @@ lpm_main_loop(__rte_unused void *dummy) lcore_id =rte_lcore_id (); qconf = &lcore_conf[lcore_id]; -if (qconf->n_rx_queue ==0 ) { +uint16_t n_rx_q = qconf->n_rx_queue; +uint16_t n_tx_p = qconf->n_tx_port;
How about adding const?
Re: [dpdk-dev] [PATCH 3/4] examples/l3fwd: eliminate unnecessary reloads in loop
Quoted 42 lines
-----Original Message-----
From: Jerin Jacob <[email protected]>
Sent: Wednesday, April 14, 2021 1:43 AM
To: Ruifeng Wang <[email protected]>
Cc: [email protected]; [email protected]; Ferruh Yigit <[email protected]>; [email protected]; David Marchand <[email protected]>; dpdk-dev <[email protected]>; nd <[email protected]>; Honnappa Nagarahalli <[email protected]>
Subject: Re: [dpdk-dev] [PATCH 3/4] examples/l3fwd: eliminate unnecessary reloads in loopQuoted 11 lines · Ruifeng Wang
Number of rx queue and number of rx port in lcore config are constants during the period of l3 forward application running. But compiler has no this information.
Copied values from lcore config to local variables and used the local variables for iteration. Compiler can see that the local variables are not changed, so qconf reloads at each iteration can be eliminated.
The change showed 1.8% performance uplift in single core, single port, single queue test on N1SDP platform with MLX5 NIC.
At least, in octeontx2, I dont see any performance improvement. But change looks good. Please find below a comment.
Quoted 14 lines
Signed-off-by Ruifeng Wang <[email protected]>--- examples/l3fwd/l3fwd_lpm.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-)examples/l3fwd/l3fwd_lpm.c +0
-0@@ -190,14 +190,16 @@ lpm_main_loop(__rte_unused void *dummy) lcore_id =rte_lcore_id (); qconf = &lcore_conf[lcore_id];
Quoted 5 lines
examples/l3fwd/l3fwd_lpm.c lpm_main_loop(__rte_unused void *dummy) · line 194
Quoted 3 lines
examples/l3fwd/l3fwd_lpm.c +2
-1-if (qconf->n_rx_queue ==0 ) { +uint16_t n_rx_q = qconf->n_rx_queue; +uint16_t n_tx_p = qconf->n_tx_port;How about adding const?
Ack. The values are not expected to be changed.
Will update in next version.
PATCH 4/4 examples/l3fwd: make data struct to be memory efficient
In reply to Ruifeng WangThere are some holes in data struct lcore_conf. The holes are due to alignment requirement.
For struct lcore_rx_queue, there is no need to make every element of this type to be cache line aligned, because the data is not shared between cores.
Member len of struct mbuf_table can be moved out. So data can be packed and there will be no need to load an extra cache line when mbuf table is empty.
The change showed slight performance improvement on N1SDP platform.
--- examples/l3fwd/l3fwd.h | 12 ++++++------ examples/l3fwd/l3fwd_common.h | 4 ++-- examples/l3fwd/l3fwd_em.c | 6 +++--- examples/l3fwd/l3fwd_lpm.c | 6 +++--- 4 files changed, 14 insertions(+), 14 deletions(-)
examples/l3fwd/l3fwd.h +6 -6
@@ -57,22 +57,22 @@#define HASH_ENTRY_NUMBER_DEFAULT 4struct mbuf_table { -uint16_t len;struct rte_mbuf *m_table[MAX_PKT_BURST ]; };struct lcore_rx_queue {uint16_t port_id;uint8_t queue_id; -} __rte_cache_aligned; +};struct lcore_conf { -uint16_t n_rx_queue;struct lcore_rx_queue rx_queue_list[MAX_RX_QUEUE_PER_LCORE ]; -uint16_t n_tx_port;uint16_t tx_port_id[RTE_MAX_ETHPORTS ];uint16_t tx_queue_id[RTE_MAX_ETHPORTS ]; +uint16_t tx_mbuf_len[RTE_MAX_ETHPORTS ];struct mbuf_table tx_mbufs[RTE_MAX_ETHPORTS ]; +uint16_t n_rx_queue; +uint16_t n_tx_port;void *ipv4_lookup_struct;void *ipv6_lookup_struct; } __rte_cache_aligned;
@@ -122,7 +122,7 @@ send_single_packet(struct lcore_conf *qconf, {uint16_t len; - len = qconf->tx_mbufs[port].len; + len = qconf->tx_mbuf_len[port]; qconf->tx_mbufs[port].m_table[len] = m; len++;
@@ -132,7 +132,7 @@ send_single_packet(struct lcore_conf *qconf, len =0 ; } - qconf->tx_mbufs[port].len = len; + qconf->tx_mbuf_len[port] = len;return 0 ; }
examples/l3fwd/l3fwd_common.h +2 -2
@@ -183,7 +183,7 @@ send_packetsx4(struct lcore_conf *qconf, uint16_t port, struct rte_mbuf *m[], {uint32_t len, j, n; - len = qconf->tx_mbufs[port].len; + len = qconf->tx_mbuf_len[port];/* * If TX buffer for that queue is empty, and we have enough packets,
@@ -258,7 +258,7 @@ send_packetsx4(struct lcore_conf *qconf, uint16_t port, struct rte_mbuf *m[], } } - qconf->tx_mbufs[port].len = len; + qconf->tx_mbuf_len[port] = len; }#endif /* _L3FWD_COMMON_H_ */
examples/l3fwd/l3fwd_em.c +3 -3
@@ -662,12 +662,12 @@ em_main_loop(__rte_unused void *dummy)for (i =0 ; i < qconf->n_tx_port; ++i) { portid = qconf->tx_port_id[i]; -if (qconf->tx_mbufs[portid].len ==0 ) +if (qconf->tx_mbuf_len[portid] ==0 )continue ;send_burst (qconf, - qconf->tx_mbufs[portid].len, + qconf->tx_mbuf_len[portid], portid); - qconf->tx_mbufs[portid].len =0 ; + qconf->tx_mbuf_len[portid] =0 ; } prev_tsc = cur_tsc;
examples/l3fwd/l3fwd_lpm.c +3 -3
@@ -220,12 +220,12 @@ lpm_main_loop(__rte_unused void *dummy)for (i =0 ; i < n_tx_p; ++i) { portid = qconf->tx_port_id[i]; -if (qconf->tx_mbufs[portid].len ==0 ) +if (qconf->tx_mbuf_len[portid] ==0 )continue ;send_burst (qconf, - qconf->tx_mbufs[portid].len, + qconf->tx_mbuf_len[portid], portid); - qconf->tx_mbufs[portid].len =0 ; + qconf->tx_mbuf_len[portid] =0 ; } prev_tsc = cur_tsc;
Signature · 1 line
2.25.1
Re: [dpdk-dev] [PATCH 4/4] examples/l3fwd: make data struct to be memory efficient
Quoted 12 lines · Ruifeng Wang
There are some holes in data struct
lcore_conf. The holes are due to alignment requirement.For struct
lcore_rx_queue, there is no need to make every element of this type to be cache line aligned, because the data is not shared between cores.Member len of struct
mbuf_tablecan be moved out. So data can be packed and there will be no need to load an extra cache line when mbuf table is empty.
Quoted 5 lines
The change showed slight performance improvement on N1SDP platform.
Suggested-by Honnappa Nagarahalli <[email protected]>Signed-off-by Ruifeng Wang <[email protected]>
This change alone is OK in the octeontx2 platform.(No difference in performance) combining with 3/4 shows some regression. Probably is due to prefetch or 128B cache line tuning specifics.
Quoted 121 lines
--- examples/l3fwd/l3fwd.h | 12 ++++++------ examples/l3fwd/l3fwd_common.h | 4 ++-- examples/l3fwd/l3fwd_em.c | 6 +++--- examples/l3fwd/l3fwd_lpm.c | 6 +++--- 4 files changed, 14 insertions(+), 14 deletions(-)examples/l3fwd/l3fwd.h +6
-6@@ -57,22 +57,22 @@#define HASH_ENTRY_NUMBER_DEFAULT 4struct mbuf_table { -uint16_t len;struct rte_mbuf *m_table[MAX_PKT_BURST ]; };struct lcore_rx_queue {uint16_t port_id;uint8_t queue_id; -} __rte_cache_aligned; +};struct lcore_conf { -uint16_t n_rx_queue;struct lcore_rx_queue rx_queue_list[MAX_RX_QUEUE_PER_LCORE ]; -uint16_t n_tx_port;uint16_t tx_port_id[RTE_MAX_ETHPORTS ];uint16_t tx_queue_id[RTE_MAX_ETHPORTS ]; +uint16_t tx_mbuf_len[RTE_MAX_ETHPORTS ];struct mbuf_table tx_mbufs[RTE_MAX_ETHPORTS ]; +uint16_t n_rx_queue; +uint16_t n_tx_port;void *ipv4_lookup_struct;void *ipv6_lookup_struct; } __rte_cache_aligned;@@ -122,7 +122,7 @@ send_single_packet(struct lcore_conf *qconf, {uint16_t len; - len = qconf->tx_mbufs[port].len; + len = qconf->tx_mbuf_len[port]; qconf->tx_mbufs[port].m_table[len] = m; len++;@@ -132,7 +132,7 @@ send_single_packet(struct lcore_conf *qconf, len =0 ; } - qconf->tx_mbufs[port].len = len; + qconf->tx_mbuf_len[port] = len;return 0 ; }examples/l3fwd/l3fwd_common.h +2
-2@@ -183,7 +183,7 @@ send_packetsx4(struct lcore_conf *qconf, uint16_t port, struct rte_mbuf *m[], {uint32_t len, j, n; - len = qconf->tx_mbufs[port].len; + len = qconf->tx_mbuf_len[port];/* * If TX buffer for that queue is empty, and we have enough packets, @@ -258,7 +258,7 @@ send_packetsx4(struct lcore_conf *qconf, uint16_t port, struct rte_mbuf *m[], } } - qconf->tx_mbufs[port].len = len; + qconf->tx_mbuf_len[port] = len; }#endif /* _L3FWD_COMMON_H_ */ examples/l3fwd/l3fwd_em.c +3
-3@@ -662,12 +662,12 @@ em_main_loop(__rte_unused void *dummy)for (i =0 ; i < qconf->n_tx_port; ++i) { portid = qconf->tx_port_id[i]; -if (qconf->tx_mbufs[portid].len ==0 ) +if (qconf->tx_mbuf_len[portid] ==0 )continue ;send_burst (qconf, - qconf->tx_mbufs[portid].len, + qconf->tx_mbuf_len[portid], portid); - qconf->tx_mbufs[portid].len =0 ; + qconf->tx_mbuf_len[portid] =0 ; } prev_tsc = cur_tsc;examples/l3fwd/l3fwd_lpm.c +3
-4@@ -220,12 +220,12 @@ lpm_main_loop(__rte_unused void *dummy)for (i =0 ; i < n_tx_p; ++i) { portid = qconf->tx_port_id[i]; -if (qconf->tx_mbufs[portid].len ==0 ) +if (qconf->tx_mbuf_len[portid] ==0 )continue ;send_burst (qconf, - qconf->tx_mbufs[portid].len, + qconf->tx_mbuf_len[portid], portid); - qconf->tx_mbufs[portid].len =0 ; + qconf->tx_mbuf_len[portid] =0 ; } prev_tsc = cur_tsc; --2.25.1
Re: [dpdk-dev] [PATCH 4/4] examples/l3fwd: make data struct to be memory efficient
Quoted 12 lines · Jerin Jacob
Quoted 11 lines · Ruifeng Wang
There are some holes in data struct
lcore_conf. The holes are due to alignment requirement.For struct
lcore_rx_queue, there is no need to make every element of this type to be cache line aligned, because the data is not shared between cores.Member len of struct
mbuf_tablecan be moved out. So data can be packed and there will be no need to load an extra cache line when mbuf table is empty.
Quoted 8 lines
Quoted 5 lines
The change showed slight performance improvement on N1SDP platform.
Suggested-by Honnappa Nagarahalli <[email protected]>Signed-off-by Ruifeng Wang <[email protected]>This change alone is OK in the octeontx2 platform.(No difference in performance) combining with 3/4 shows some regression. Probably is due to prefetch or 128B cache line tuning specifics.
We checked it on Layerscape LS2088A platform. No difference for 1-2 core case. However observing ~2% regression for 4-8 cores.
Regards,
Hemant
Re: [dpdk-dev] [PATCH 4/4] examples/l3fwd: make data struct to be memory efficient
<snip>
Quoted 21 lines
Quoted 11 lines
Quoted 11 lines
There are some holes in data struct
lcore_conf. The holes are due to alignment requirement.For struct
lcore_rx_queue, there is no need to make every element of this type to be cache line aligned, because the data is not shared between cores.Member len of struct
mbuf_tablecan be moved out. So data can be packed and there will be no need to load an extra cache line when mbuf table is empty.Quoted 6 lines
Quoted 5 lines
The change showed slight performance improvement on N1SDP platform.
Suggested-by Honnappa Nagarahalli <[email protected]>Signed-off-by Ruifeng Wang <[email protected]>This change alone is OK in the octeontx2 platform.(No difference in
performance)
Quoted 2 lines
combining with 3/4 shows some regression. Probably is due to prefetch or 128B cache line tuning specifics.
Quoted 7 lines
We checked it on Layerscape LS2088A platform. No difference for 1-2 core case. However observing ~2% regression for 4-8 cores.
Regards,
Hemant
Hi Ruifeng,
l3fwd will no longer build with this patch as you have changed a struct used by the FIB lookup method. This patch will need to be updated to also update the FIB lookup method as you have done with EM and LPM.
Thanks,
Conor.
Re: [dpdk-dev] [PATCH 4/4] examples/l3fwd: make data struct to be memory efficient
Quoted 38 lines
-----Original Message-----
From: Walsh, Conor <[email protected]>
Sent: Monday, April 26, 2021 6:55 PM
To: [email protected]; Jerin Jacob <[email protected]>; Ruifeng Wang <[email protected]>
Cc: [email protected]; Yigit, Ferruh <[email protected]>; [email protected]; David Marchand <[email protected]>; dpdk-dev <[email protected]>; nd <[email protected]>; Honnappa Nagarahalli <[email protected]>
Subject: RE: [dpdk-dev] [PATCH 4/4] examples/l3fwd: make data struct to be memory efficient<snip>
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
There are some holes in data struct
lcore_conf. The holes are due to alignment requirement.For struct
lcore_rx_queue, there is no need to make every element of this type to be cache line aligned, because the data is not shared between cores.Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Member len of struct
mbuf_tablecan be moved out. So data can be packed and there will be no need to load an extra cache line when mbuf table is empty.The change showed slight performance improvement on N1SDP
platform.
Quoted 12 lines
Quoted 5 lines
Quoted 4 lines
Suggested-by Honnappa Nagarahalli <[email protected]>Signed-off-by Ruifeng Wang <[email protected]>This change alone is OK in the octeontx2 platform.(No difference in
performance)
Quoted 2 lines
combining with 3/4 shows some regression. Probably is due to prefetch or 128B cache line tuning specifics.
We checked it on Layerscape LS2088A platform. No difference for 1-2 core case. However observing ~2% regression for 4-8 cores.
Quoted 6 lines
Quoted 4 lines
Regards,
Hemant
Hi Ruifeng,
Hi Conor,
Quoted 5 lines
l3fwd will no longer build with this patch as you have changed a struct used by the FIB lookup method.
This patch will need to be updated to also update the FIB lookup method as you have done with EM and LPM.
Thanks for the comments.
I will provide v2 with updates. I'm considering to drop this one in v2 series.
Quoted 3 lines
Thanks,
Conor.
Hello,
This patch series targeted to improve L3fwd example. Performance gain was observed on N1SDP platform.
It would be good if you can run this series on your platforms and see if there is any performance impact.
Thanks,
Ruifeng
Quoted 28 lines
-----Original Message-----
From: Ruifeng Wang <[email protected]>
Sent: Thursday, March 18, 2021 6:26 PM
To: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Cc: [email protected]; nd <[email protected]>; Honnappa Nagarahalli <[email protected]>; Ruifeng Wang <[email protected]>
Subject: [PATCH 0/4] l3fwd improvementsThis series of patches include changes to l3fwd example application. Some improvements are made for better usage of CPU cycles and memory.
Ruifeng Wang (4):
examples/l3fwd: tune prefetch for better performance examples/l3fwd: eliminate unnecessary calculations examples/l3fwd: eliminate unnecessary reloads in loop examples/l3fwd: make data struct to be memory efficientexamples/l3fwd/l3fwd.h | 12 ++++++------ examples/l3fwd/l3fwd_common.h | 4 ++-- examples/l3fwd/l3fwd_em.c | 6 +++--- examples/l3fwd/l3fwd_lpm.c | 16 +++++++++------- examples/l3fwd/l3fwd_lpm_neon.h | 20 ++++++++++---------- 5 files changed, 30 insertions(+), 28 deletions(-)--
2.25.1
Quoted 6 lines · Ruifeng Wang
Hello,
This patch series targeted to improve L3fwd example. Performance gain was observed on N1SDP platform.
It would be good if you can run this series on your platforms and see if there is any performance impact.
I will test and update and update in the respective patches.
Quoted 33 lines
Thanks,
RuifengQuoted 28 lines
-----Original Message-----
From: Ruifeng Wang <[email protected]>
Sent: Thursday, March 18, 2021 6:26 PM
To: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Cc: [email protected]; nd <[email protected]>; Honnappa Nagarahalli <[email protected]>; Ruifeng Wang <[email protected]>
Subject: [PATCH 0/4] l3fwd improvementsThis series of patches include changes to l3fwd example application. Some improvements are made for better usage of CPU cycles and memory.
Ruifeng Wang (4):
examples/l3fwd: tune prefetch for better performance examples/l3fwd: eliminate unnecessary calculations examples/l3fwd: eliminate unnecessary reloads in loop examples/l3fwd: make data struct to be memory efficientexamples/l3fwd/l3fwd.h | 12 ++++++------ examples/l3fwd/l3fwd_common.h | 4 ++-- examples/l3fwd/l3fwd_em.c | 6 +++--- examples/l3fwd/l3fwd_lpm.c | 16 +++++++++------- examples/l3fwd/l3fwd_lpm_neon.h | 20 ++++++++++---------- 5 files changed, 30 insertions(+), 28 deletions(-)--
2.25.1
Quoted 10 lines
-----Original Message-----
From: dev <[email protected]> On Behalf Of Ruifeng Wang
Sent: Thursday, March 18, 2021 06:26 PM
To: [email protected]; [email protected]; Yigit, Ferruh <[email protected]>; [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Ruifeng Wang <[email protected]>
Subject: [dpdk-dev] [PATCH 0/4] l3fwd improvements
v2 cover l3fwd improvements
In reply to Ruifeng WangThis series of patches include changes to l3fwd example application. Some improvements are made for better usage of CPU cycles and memory.
v2:
Dropped 1/4 prefetch to write change from v1.
Dropped 4/4 data struct change from v1.
Added 1/3 code reorganize.
Updated 3/3 to add 'const'. (Jerin)
Ruifeng Wang (3):
examples/l3fwd: reorganize code for better performance examples/l3fwd: eliminate unnecessary calculations examples/l3fwd: eliminate unnecessary reloads in loop
examples/l3fwd/l3fwd_lpm.c | 10 ++++++---- examples/l3fwd/l3fwd_lpm_neon.h | 10 +++++----- examples/l3fwd/l3fwd_neon.h | 10 +++++----- 3 files changed, 16 insertions(+), 14 deletions(-)
Signature · 1 line
2.25.1
PATCH v2 1/3 examples/l3fwd: reorganize code for better performance
Moved rfc1812 process prior to NEON registers store.
On N1SDP, this reorganization mitigates CPU frontend stall and backend stall when forwarding.
On N1SDP with MLX5 40G NIC, this change showed 10.2% performance gain in single port single core MRR test.
On ThunderX2, this changed showed no performance degradation.
--- examples/l3fwd/l3fwd_neon.h | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
examples/l3fwd/l3fwd_neon.h +5 -5
@@ -43,11 +43,6 @@ processx4_step3(struct rte_mbuf *pkt[FWDSTEP], uint16_t dst_port[FWDSTEP]) ve[2 ] =vsetq_lane_u32 (vgetq_lane_u32 (te[2 ],3 ), ve[2 ],3 ); ve[3 ] =vsetq_lane_u32 (vgetq_lane_u32 (te[3 ],3 ), ve[3 ],3 ); -vst1q_u32 (p[0 ], ve[0 ]); -vst1q_u32 (p[1 ], ve[1 ]); -vst1q_u32 (p[2 ], ve[2 ]); -vst1q_u32 (p[3 ], ve[3 ]); -rfc1812_process ((struct rte_ipv4_hdr *) ((struct rte_ether_hdr *)p[0 ] +1 ), &dst_port[0 ], pkt[0 ]->packet_type);
@@ -60,6 +55,11 @@ processx4_step3(struct rte_mbuf *pkt[FWDSTEP], uint16_t dst_port[FWDSTEP])rfc1812_process ((struct rte_ipv4_hdr *) ((struct rte_ether_hdr *)p[3 ] +1 ), &dst_port[3 ], pkt[3 ]->packet_type); + +vst1q_u32 (p[0 ], ve[0 ]); +vst1q_u32 (p[1 ], ve[1 ]); +vst1q_u32 (p[2 ], ve[2 ]); +vst1q_u32 (p[3 ], ve[3 ]); }/*
Signature · 1 line
2.25.1
Re: [dpdk-dev] [PATCH v2 1/3] examples/l3fwd: reorganize code for better performance
Quoted 7 lines · Ruifeng Wang
Moved rfc1812 process prior to NEON registers store.
On N1SDP, this reorganization mitigates CPU frontend stall and backend stall when forwarding.On N1SDP with MLX5 40G NIC, this change showed 10.2% performance gain in single port single core MRR test.
I think, it may not have anything to do with N1SDP, It could be just the prefetch window timing
with MLX5 driver on Tx mbuf on touching with tx_burst() and L1 cache pressure timing.
I think, tuning the driver parameters can switch the window to some driver code.
On Octeontx2, this change has regression of -3.1% flow lookup miss case. so NACK.
Quoted 38 lines
On ThunderX2, this changed showed no performance degradation.
Signed-off-by Ruifeng Wang <[email protected]>--- examples/l3fwd/l3fwd_neon.h | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)examples/l3fwd/l3fwd_neon.h +5
-6@@ -43,11 +43,6 @@ processx4_step3(struct rte_mbuf *pkt[FWDSTEP], uint16_t dst_port[FWDSTEP]) ve[2 ] =vsetq_lane_u32 (vgetq_lane_u32 (te[2 ],3 ), ve[2 ],3 ); ve[3 ] =vsetq_lane_u32 (vgetq_lane_u32 (te[3 ],3 ), ve[3 ],3 ); -vst1q_u32 (p[0 ], ve[0 ]); -vst1q_u32 (p[1 ], ve[1 ]); -vst1q_u32 (p[2 ], ve[2 ]); -vst1q_u32 (p[3 ], ve[3 ]); -rfc1812_process ((struct rte_ipv4_hdr *) ((struct rte_ether_hdr *)p[0 ] +1 ), &dst_port[0 ], pkt[0 ]->packet_type);@@ -60,6 +55,11 @@ processx4_step3(struct rte_mbuf *pkt[FWDSTEP], uint16_t dst_port[FWDSTEP])rfc1812_process ((struct rte_ipv4_hdr *) ((struct rte_ether_hdr *)p[3 ] +1 ), &dst_port[3 ], pkt[3 ]->packet_type); + +vst1q_u32 (p[0 ], ve[0 ]); +vst1q_u32 (p[1 ], ve[1 ]); +vst1q_u32 (p[2 ], ve[2 ]); +vst1q_u32 (p[3 ], ve[3 ]); }/* -- 2.25.1
PATCH v2 2/3 examples/l3fwd: eliminate unnecessary calculations
In reply to Ruifeng WangBoth L2 and L3 headers will be used in forward processing. And these two headers are in the same cache line. It has the same effect for prefetching with L2 header address and prefetching with L3 header address.
Changed to use L2 header address for prefetching. The change showed no measurable performance improvement, but it definitely removed unnecessary instructions for address calculation.
--- examples/l3fwd/l3fwd_lpm_neon.h | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
examples/l3fwd/l3fwd_lpm_neon.h +5 -5
@@ -98,14 +98,14 @@ l3fwd_lpm_send_packets(int nb_rx, struct rte_mbuf **pkts_burst,if (k) {for (i =0 ; i <FWDSTEP ; i++) {rte_prefetch0 (rte_pktmbuf_mtod(pkts_burst [i], - struct rte_ether_hdr *) + 1); +void *)); }for (j =0 ; j != k -FWDSTEP ; j +=FWDSTEP ) {for (i =0 ; i <FWDSTEP ; i++) {rte_prefetch0 (rte_pktmbuf_mtod(pkts_burst [j + i +FWDSTEP ], - struct rte_ether_hdr *) + 1); +void *)); }processx4_step1 (&pkts_burst[j], &dip, &ipv4_flag);
@@ -125,17 +125,17 @@ l3fwd_lpm_send_packets(int nb_rx, struct rte_mbuf **pkts_burst,switch (m) {case 3 :rte_prefetch0 (rte_pktmbuf_mtod (pkts_burst [j], - struct rte_ether_hdr *) + 1); +void *)); j++;/* fallthrough */ case 2 :rte_prefetch0 (rte_pktmbuf_mtod (pkts_burst [j], - struct rte_ether_hdr *) + 1); +void *)); j++;/* fallthrough */ case 1 :rte_prefetch0 (rte_pktmbuf_mtod (pkts_burst [j], - struct rte_ether_hdr *) + 1); +void *)); j++; }
Signature · 1 line
2.25.1
PATCH v2 3/3 examples/l3fwd: eliminate unnecessary reloads in loop
In reply to Ruifeng WangNumber of rx queue and number of rx port in lcore config are constants during the period of l3 forward application running. But compiler has no this information.
Copied values from lcore config to local variables and used the local variables for iteration. Compiler can see that the local variables are not changed, so qconf reloads at each iteration can be eliminated.
The change showed 1.8% performance uplift in single core, single port, single queue test on N1SDP platform with MLX5 NIC.
--- examples/l3fwd/l3fwd_lpm.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-)
examples/l3fwd/l3fwd_lpm.c +6 -4
@@ -154,14 +154,16 @@ lpm_main_loop(__rte_unused void *dummy) lcore_id =rte_lcore_id (); qconf = &lcore_conf[lcore_id]; -if (qconf->n_rx_queue ==0 ) { +const uint16_t n_rx_q = qconf->n_rx_queue; +const uint16_t n_tx_p = qconf->n_tx_port; +if (n_rx_q ==0 ) {RTE_LOG (INFO ,L3FWD ,"lcore %u has nothing to do\n" , lcore_id);return 0 ; }RTE_LOG (INFO ,L3FWD ,"entering main loop on lcore %u\n" , lcore_id); -for (i =0 ; i < qconf->n_rx_queue; i++) { +for (i =0 ; i < n_rx_q; i++) { portid = qconf->rx_queue_list[i].port_id; queueid = qconf->rx_queue_list[i].queue_id;
@@ -181,7 +183,7 @@ lpm_main_loop(__rte_unused void *dummy) diff_tsc = cur_tsc - prev_tsc;if (unlikely (diff_tsc > drain_tsc)) { -for (i =0 ; i < qconf->n_tx_port; ++i) { +for (i =0 ; i < n_tx_p; ++i) { portid = qconf->tx_port_id[i];if (qconf->tx_mbufs[portid].len ==0 )continue ;
@@ -197,7 +199,7 @@ lpm_main_loop(__rte_unused void *dummy)/* * Read packet from RX queues */ -for (i =0 ; i < qconf->n_rx_queue; ++i) { +for (i =0 ; i < n_rx_q; ++i) { portid = qconf->rx_queue_list[i].port_id; queueid = qconf->rx_queue_list[i].queue_id; nb_rx = rte_eth_rx_burst(portid, queueid, pkts_burst,
Signature · 1 line
2.25.1
Re: [dpdk-dev] [PATCH v2 3/3] examples/l3fwd: eliminate unnecessary reloads in loop
Quoted 9 lines · Ruifeng Wang
Number of rx queue and number of rx port in lcore config are constants during the period of l3 forward application running. But compiler has no this information.
Copied values from lcore config to local variables and used the local variables for iteration. Compiler can see that the local variables are not changed, so qconf reloads at each iteration can be eliminated.
Quoted 6 lines
The change showed 1.8% performance uplift in single core, single port, single queue test on N1SDP platform with MLX5 NIC.
Signed-off-by Ruifeng Wang <[email protected]>Reviewed-by Ola Liljedahl <[email protected]>Reviewed-by Honnappa Nagarahalli <[email protected]>
No performance regression with octeontx2.
Quoted 48 lines
--- examples/l3fwd/l3fwd_lpm.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-)examples/l3fwd/l3fwd_lpm.c +6
-5@@ -154,14 +154,16 @@ lpm_main_loop(__rte_unused void *dummy) lcore_id =rte_lcore_id (); qconf = &lcore_conf[lcore_id]; -if (qconf->n_rx_queue ==0 ) { +const uint16_t n_rx_q = qconf->n_rx_queue; +const uint16_t n_tx_p = qconf->n_tx_port; +if (n_rx_q ==0 ) {RTE_LOG (INFO ,L3FWD ,"lcore %u has nothing to do\n" , lcore_id);return 0 ; }RTE_LOG (INFO ,L3FWD ,"entering main loop on lcore %u\n" , lcore_id); -for (i =0 ; i < qconf->n_rx_queue; i++) { +for (i =0 ; i < n_rx_q; i++) { portid = qconf->rx_queue_list[i].port_id; queueid = qconf->rx_queue_list[i].queue_id;@@ -181,7 +183,7 @@ lpm_main_loop(__rte_unused void *dummy) diff_tsc = cur_tsc - prev_tsc;if (unlikely (diff_tsc > drain_tsc)) { -for (i =0 ; i < qconf->n_tx_port; ++i) { +for (i =0 ; i < n_tx_p; ++i) { portid = qconf->tx_port_id[i];if (qconf->tx_mbufs[portid].len ==0 )continue ;@@ -197,7 +199,7 @@ lpm_main_loop(__rte_unused void *dummy)/* * Read packet from RX queues */ -for (i =0 ; i < qconf->n_rx_queue; ++i) { +for (i =0 ; i < n_rx_q; ++i) { portid = qconf->rx_queue_list[i].port_id; queueid = qconf->rx_queue_list[i].queue_id; nb_rx = rte_eth_rx_burst(portid, queueid, pkts_burst, --2.25.1
v3 cover l3fwd improvements
In reply to Ruifeng WangThis series of patches include changes to l3fwd example application. Some improvements are made for better usage of CPU cycles and memory.
v3:
Dropped 1/3 code reorg from v2.
v2:
Dropped 1/4 prefetch to write change from v1.
Dropped 4/4 data struct change from v1.
Added 1/3 code reorganize.
Updated 3/3 to add 'const'. (Jerin)
Ruifeng Wang (2):
examples/l3fwd: eliminate unnecessary calculations examples/l3fwd: eliminate unnecessary reloads in loop
examples/l3fwd/l3fwd_lpm.c | 10 ++++++---- examples/l3fwd/l3fwd_lpm_neon.h | 10 +++++----- 2 files changed, 11 insertions(+), 9 deletions(-)
Signature · 1 line
2.25.1
PATCH v3 1/2 examples/l3fwd: eliminate unnecessary calculations
Both L2 and L3 headers will be used in forward processing. And these two headers are in the same cache line. It has the same effect for prefetching with L2 header address and prefetching with L3 header address.
Changed to use L2 header address for prefetching. The change showed no measurable performance improvement, but it definitely removed unnecessary instructions for address calculation.
--- examples/l3fwd/l3fwd_lpm_neon.h | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
examples/l3fwd/l3fwd_lpm_neon.h +5 -5
@@ -98,14 +98,14 @@ l3fwd_lpm_send_packets(int nb_rx, struct rte_mbuf **pkts_burst,if (k) {for (i =0 ; i <FWDSTEP ; i++) {rte_prefetch0 (rte_pktmbuf_mtod(pkts_burst [i], - struct rte_ether_hdr *) + 1); +void *)); }for (j =0 ; j != k -FWDSTEP ; j +=FWDSTEP ) {for (i =0 ; i <FWDSTEP ; i++) {rte_prefetch0 (rte_pktmbuf_mtod(pkts_burst [j + i +FWDSTEP ], - struct rte_ether_hdr *) + 1); +void *)); }processx4_step1 (&pkts_burst[j], &dip, &ipv4_flag);
@@ -125,17 +125,17 @@ l3fwd_lpm_send_packets(int nb_rx, struct rte_mbuf **pkts_burst,switch (m) {case 3 :rte_prefetch0 (rte_pktmbuf_mtod (pkts_burst [j], - struct rte_ether_hdr *) + 1); +void *)); j++;/* fallthrough */ case 2 :rte_prefetch0 (rte_pktmbuf_mtod (pkts_burst [j], - struct rte_ether_hdr *) + 1); +void *)); j++;/* fallthrough */ case 1 :rte_prefetch0 (rte_pktmbuf_mtod (pkts_burst [j], - struct rte_ether_hdr *) + 1); +void *)); j++; }
Signature · 1 line
2.25.1
PATCH v3 2/2 examples/l3fwd: eliminate unnecessary reloads in loop
In reply to Ruifeng WangNumber of rx queue and number of rx port in lcore config are constants during the period of l3 forward application running. But compiler has no this information.
Copied values from lcore config to local variables and used the local variables for iteration. Compiler can see that the local variables are not changed, so qconf reloads at each iteration can be eliminated.
The change showed 1.8% performance uplift in single core, single port, single queue test on N1SDP platform with MLX5 NIC.
--- examples/l3fwd/l3fwd_lpm.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-)
examples/l3fwd/l3fwd_lpm.c +6 -4
@@ -154,14 +154,16 @@ lpm_main_loop(__rte_unused void *dummy) lcore_id =rte_lcore_id (); qconf = &lcore_conf[lcore_id]; -if (qconf->n_rx_queue ==0 ) { +const uint16_t n_rx_q = qconf->n_rx_queue; +const uint16_t n_tx_p = qconf->n_tx_port; +if (n_rx_q ==0 ) {RTE_LOG (INFO ,L3FWD ,"lcore %u has nothing to do\n" , lcore_id);return 0 ; }RTE_LOG (INFO ,L3FWD ,"entering main loop on lcore %u\n" , lcore_id); -for (i =0 ; i < qconf->n_rx_queue; i++) { +for (i =0 ; i < n_rx_q; i++) { portid = qconf->rx_queue_list[i].port_id; queueid = qconf->rx_queue_list[i].queue_id;
@@ -181,7 +183,7 @@ lpm_main_loop(__rte_unused void *dummy) diff_tsc = cur_tsc - prev_tsc;if (unlikely (diff_tsc > drain_tsc)) { -for (i =0 ; i < qconf->n_tx_port; ++i) { +for (i =0 ; i < n_tx_p; ++i) { portid = qconf->tx_port_id[i];if (qconf->tx_mbufs[portid].len ==0 )continue ;
@@ -197,7 +199,7 @@ lpm_main_loop(__rte_unused void *dummy)/* * Read packet from RX queues */ -for (i =0 ; i < qconf->n_rx_queue; ++i) { +for (i =0 ; i < n_rx_q; ++i) { portid = qconf->rx_queue_list[i].port_id; queueid = qconf->rx_queue_list[i].queue_id; nb_rx = rte_eth_rx_burst(portid, queueid, pkts_burst,
Signature · 1 line
2.25.1
Quoted 4 lines · Ruifeng Wang
This series of patches include changes to l3fwd example application. Some improvements are made for better usage of CPU cycles and memory.
Quoted 9 lines
v3:
Dropped 1/3 code reorg from v2.v2:
Dropped 1/4 prefetch to write change from v1.
Dropped 4/4 data struct change from v1.
Added 1/3 code reorganize.
Updated 3/3 to add 'const'. (Jerin)
Series applied, thanks.
Signature · 1 line
David Marchand