examples/eventdev: move eth stop to the end
PATCH 1/1 examples/eventdev: move eth stop to the end
Move eth stop code from "signal_handler" function to the end of "main" function. There are two reasons for this:
First, this improves code maintenance and makes code look simple and clear. Based on this change, after receiving the interrupt signal, "fdata->done" is set as 1. Then the main thread will wait all worker lcores to jump out of the loop. Finally, the main thread will stop and then close eth dev port.
Second, for older version, the main thread first stops eth dev port and then waits the end of worker lcore. This may cause errors because it may stop the eth dev port which worker lcores are using. This moving change can fix this by waiting all worker threads to exit and then stop the eth dev port.
In the meanwhile, remove wmb in signal_handler.
This is because when the main lcore receive the stop signal, it stores 1 into fdata->done. And then the worker lcores load "fdata->done" and jump out of the loop to stop running. Nothing should be stored after updating fdata->done, so the wmb is unnecessary.
--- examples/eventdev_pipeline/main.c | 15 ++++----------- 1 file changed, 4 insertions(+), 11 deletions(-)
examples/eventdev_pipeline/main.c +4 -11
@@ -291,17 +291,6 @@ signal_handler(int signum)rte_event_dev_dump (0 , stdout); once =1 ; fdata->done =1 ; -rte_smp_wmb (); - -RTE_ETH_FOREACH_DEV (portid) { -rte_event_eth_rx_adapter_stop (portid); -rte_event_eth_tx_adapter_stop (portid); -if (rte_eth_dev_stop (portid) <0 ) -printf ("Failed to stop port %u" , portid); - } - -rte_eal_mp_wait_lcore (); - }if (signum ==SIGTSTP )rte_event_dev_dump (0 , stdout);
@@ -465,6 +454,10 @@ main(int argc, char **argv) }RTE_ETH_FOREACH_DEV (portid) { +rte_event_eth_rx_adapter_stop (portid); +rte_event_eth_tx_adapter_stop (portid); +if (rte_eth_dev_stop (portid) <0 ) +printf ("Failed to stop port %u" , portid);rte_eth_dev_close (portid); }
Signature · 1 line
2.17.1
PATCH v2 1/1 examples/eventdev: move eth stop to the end
Move eth stop code from "signal_handler" function to the end of "main" function. There are two reasons for this:
First, this improves code maintenance and makes code look simple and clear. Based on this change, after receiving the interrupt signal, "fdata->done" is set as 1. Then the main thread will wait all worker lcores to jump out of the loop. Finally, the main thread will stop and then close eth dev port.
Second, for older version, the main thread first stops eth dev port and then waits the end of worker lcore. This may cause errors because it may stop the eth dev port which worker lcores are using. This moving change can fix this by waiting all worker threads to exit and then stop the eth dev port.
In the meanwhile, remove wmb in signal_handler.
This is because when the main lcore receive the stop signal, it stores 1 into fdata->done. And then the worker lcores load "fdata->done" and jump out of the loop to stop running. Nothing should be stored after updating fdata->done, so the wmb is unnecessary.
---
v2:
1. Delete unused variable to fix build error
2. Reduce commit message length to fix coding style issues
examples/eventdev_pipeline/main.c | 16 ++++------------ 1 file changed, 4 insertions(+), 12 deletions(-)
examples/eventdev_pipeline/main.c +4 -12
@@ -280,7 +280,6 @@ static void signal_handler(int signum) {static uint8_t once; -uint16_t portid;if (fdata->done)rte_exit (1 ,"Exiting on signal %d\n" , signum);
@@ -291,17 +290,6 @@ signal_handler(int signum)rte_event_dev_dump (0 , stdout); once =1 ; fdata->done =1 ; -rte_smp_wmb (); - -RTE_ETH_FOREACH_DEV (portid) { -rte_event_eth_rx_adapter_stop (portid); -rte_event_eth_tx_adapter_stop (portid); -if (rte_eth_dev_stop (portid) <0 ) -printf ("Failed to stop port %u" , portid); - } - -rte_eal_mp_wait_lcore (); - }if (signum ==SIGTSTP )rte_event_dev_dump (0 , stdout);
@@ -465,6 +453,10 @@ main(int argc, char **argv) }RTE_ETH_FOREACH_DEV (portid) { +rte_event_eth_rx_adapter_stop (portid); +rte_event_eth_tx_adapter_stop (portid); +if (rte_eth_dev_stop (portid) <0 ) +printf ("Failed to stop port %u" , portid);rte_eth_dev_close (portid); }
Signature · 1 line
2.17.1
Quoted 8 lines
-----Original Message-----
From: Feifei Wang <[email protected]>
Sent: Monday, December 21, 2020 5:35 AM
To: Van Haaren, Harry <[email protected]>; Rao, Nikhil <[email protected]>; Pavan Nikhilesh <[email protected]>
Cc: [email protected]; [email protected]; Feifei Wang <[email protected]>; [email protected]; [email protected]
Subject: [PATCH] examples/eventdev: move eth stop to the end
Suggested title improvement? examples/eventdev_pipeline: refactor ethdev port stop
Quoted 8 lines
Move eth stop code from "
signal_handler" function to the end of "main" function. There are two reasons for this:First, this improves code maintenance and makes code look simple and clear. Based on this change, after receiving the interrupt signal, "
fdata->done" is set as 1. Then the main thread will wait all worker lcores to jump out of the loop. Finally, the main thread will stop and then close eth dev port.
Quoted 4 lines
Second, for older version, the main thread first stops eth dev port and then waits the end of worker lcore. This may cause errors because it may stop the eth dev port which worker lcores are using. This moving change can fix this by waiting all worker threads to exit and then stop the eth dev port.
I'm OK with the above changes, and agree that moving eth dev port close to after lcores return is a worthy change.
Quoted 7 lines
In the meanwhile, remove wmb in
signal_handler.This is because when the main lcore receive the stop signal, it stores 1 into
fdata->done. And then the worker lcores load "fdata->done" and jump out of the loop to stop running. Nothing should be stored after updatingfdata->done, so the wmb is unnecessary.
Quoted 8 lines
Fixes 085edac2ca38 ("examples/eventdev_pipeline: support Tx adapter")Suggested-by Ruifeng Wang <[email protected]>Signed-off-by Feifei Wang <[email protected]>Reviewed-by Ruifeng Wang <[email protected]>Reviewed-by Honnappa Nagarahalli <[email protected]>
+Cc Jerin for Eventdev tree;
-Cc Pavan's old email address
Ack-ed by: Harry van Haaren <[email protected]>
Hi, Van Haaren
Quoted 19 lines
-----邮件原件-----
发件人: Van Haaren, Harry <[email protected]> 发送时间: 2020年12月21日 17:57
收件人: Feifei Wang <[email protected]>; Rao, Nikhil <[email protected]>
抄送: [email protected]; nd <[email protected]>; [email protected]; [email protected]; [email protected]
主题: RE: [PATCH] examples/eventdev: move eth stop to the endQuoted 9 lines
-----Original Message-----
From: Feifei Wang <[email protected]>
Sent: Monday, December 21, 2020 5:35 AM
To: Van Haaren, Harry <[email protected]>; Rao, Nikhil <[email protected]>; Pavan Nikhilesh <[email protected]>
Cc: [email protected]; [email protected]; Feifei Wang <[email protected]>; [email protected]; [email protected]
Subject: [PATCH] examples/eventdev: move eth stop to the end
Quoted 2 lines
Suggested title improvement? examples/eventdev_pipeline: refactor ethdev port stop
That's OK. I will apply this in the next version.
Quoted 37 lines
Quoted 5 lines
Move eth stop code from "
signal_handler" function to the end of "main" function. There are two reasons for this:First, this improves code maintenance and makes code look simple and
clear.
Quoted 3 lines
Based on this change, after receiving the interrupt signal, "
fdata->done" is set as 1. Then the main thread will wait all worker lcores to jump out of the loop. Finally, the main thread will stop and then close eth devport.
Quoted 5 lines
Second, for older version, the main thread first stops eth dev port and then waits the end of worker lcore. This may cause errors because it may stop the eth dev port which worker lcores are using. This moving change can fix this by waiting all worker threads to exit and then
stop the eth dev port.
I'm OK with the above changes, and agree that moving eth dev port close to after lcores return is a worthy change.
Quoted 16 lines
In the meanwhile, remove wmb in
signal_handler.This is because when the main lcore receive the stop signal, it stores 1 into
fdata->done. And then the worker lcores load "fdata->done" and jump out of the loop to stop running. Nothing should be stored after updatingfdata->done, so the wmb is unnecessary.Fixes 085edac2ca38 ("examples/eventdev_pipeline: support Tx adapter")Suggested-by Ruifeng Wang <[email protected]>Signed-off-by Feifei Wang <[email protected]>Reviewed-by Ruifeng Wang <[email protected]>Reviewed-by Honnappa Nagarahalli <[email protected]>
Review
Quoted 2 lines
(quoted diff) +1
-1+Cc Jerin for Eventdev tree; -Cc Pavan's old email address
Thanks very much for this change.
Quoted 2 lines
Ack-ed by: Harry van Haaren <[email protected]>
Best Regards
Feifei
PATCH v3 1/1 examples/eventdev: refactor ethdev port stop
In reply to Feifei WangMove eth stop code from "signal_handler" function to the end of "main" function. There are two reasons for this:
First, this improves code maintenance and makes code look simple and clear. Based on this change, after receiving the interrupt signal, "fdata->done" is set as 1. Then the main thread will wait all worker lcores to jump out of the loop. Finally, the main thread will stop and then close eth dev port.
Second, for older version, the main thread first stops eth dev port and then waits the end of worker lcore. This may cause errors because it may stop the eth dev port which worker lcores are using. This moving change can fix this by waiting all worker threads to exit and then stop the eth dev port.
In the meanwhile, remove wmb in signal_handler.
This is because when the main lcore receive the stop signal, it stores 1 into fdata->done. And then the worker lcores load "fdata->done" and jump out of the loop to stop running. Nothing should be stored after updating fdata->done, so the wmb is unnecessary.
---
v2:
1. Delete unused variable to fix build error
2. Reduce commit message length to fix coding style issues
v3:
1. Title improvement (Van Haaren)
examples/eventdev_pipeline/main.c | 16 ++++------------ 1 file changed, 4 insertions(+), 12 deletions(-)
examples/eventdev_pipeline/main.c +4 -12
@@ -280,7 +280,6 @@ static void signal_handler(int signum) {static uint8_t once; -uint16_t portid;if (fdata->done)rte_exit (1 ,"Exiting on signal %d\n" , signum);
@@ -291,17 +290,6 @@ signal_handler(int signum)rte_event_dev_dump (0 , stdout); once =1 ; fdata->done =1 ; -rte_smp_wmb (); - -RTE_ETH_FOREACH_DEV (portid) { -rte_event_eth_rx_adapter_stop (portid); -rte_event_eth_tx_adapter_stop (portid); -if (rte_eth_dev_stop (portid) <0 ) -printf ("Failed to stop port %u" , portid); - } - -rte_eal_mp_wait_lcore (); - }if (signum ==SIGTSTP )rte_event_dev_dump (0 , stdout);
@@ -465,6 +453,10 @@ main(int argc, char **argv) }RTE_ETH_FOREACH_DEV (portid) { +rte_event_eth_rx_adapter_stop (portid); +rte_event_eth_tx_adapter_stop (portid); +if (rte_eth_dev_stop (portid) <0 ) +printf ("Failed to stop port %u" , portid);rte_eth_dev_close (portid); }
Signature · 1 line
2.25.1
Re: [dpdk-dev] [EXT] [PATCH v3] examples/eventdev: refactor ethdev port stop
Hi Feifei,
Quoted 10 lines
Move eth stop code from "
signal_handler" function to the end of "main"
function. There are two reasons for this:First, this improves code maintenance and makes code look simple and clear. Based on this change, after receiving the interrupt signal, "
fdata->done" is set as 1. Then the main thread will wait all worker lcores to jump out of the loop. Finally, the main thread will stop and then close eth dev port.
Quoted 7 lines
Second, for older version, the main thread first stops eth dev port and then waits the end of worker lcore. This may cause errors because it may
stop the eth dev port which worker lcores are using. This moving change
can fix this by waiting all worker threads to exit and then stop the eth dev port.
Apologies for the delayed reply,
In case of event dev the workers don't interact with eth device directly, Instead eth device "injects" packets into event device and event device is responsible for scheduling them to the workers.
If the producer is not stopped i.e. in this case eth device then the worker threads might never exit and the main core would wait indefinitely for workers to exit. This will be predominantly seen in cases where there are only a few flows and large number of workers causing a lot of intra thread dependency.
Regards,
Pavan.
Quoted 77 lines
In the meanwhile, remove wmb in
signal_handler.This is because when the main lcore receive the stop signal, it stores 1 into
fdata->done. And then the worker lcores load "fdata->done" and jump
out of the loop to stop running. Nothing should be stored after updatingfdata->done, so the wmb is unnecessary.Fixes 085edac2ca38 ("examples/eventdev_pipeline: support Txadapter")
Cc: [email protected]
Cc: [email protected]Suggested-by Ruifeng Wang <[email protected]>Signed-off-by Feifei Wang <[email protected]>Reviewed-by Ruifeng Wang <[email protected]>Reviewed-by Honnappa NagarahalliAcked-by Harry van Haaren <[email protected]>---
v2:
1. Delete unused variable to fix build error
2. Reduce commit message length to fix coding style issuesv3:
1. Title improvement (Van Haaren)
examples/eventdev_pipeline/main.c| 16 ++++------------1 file changed, 4 insertions(+), 12 deletions(-)examples/eventdev_pipeline/main.c +4
-13@@ -280,7 +280,6 @@ static void signal_handler(int signum) {static uint8_t once; -uint16_t portid;if (fdata->done)rte_exit (1 ,"Exiting on signal %d\n" , signum);@@ -291,17 +290,6 @@ signal_handler(int signum)rte_event_dev_dump (0 , stdout); once =1 ; fdata->done =1 ; -rte_smp_wmb (); - -RTE_ETH_FOREACH_DEV (portid) { -rte_event_eth_rx_adapter_stop (portid); -rte_event_eth_tx_adapter_stop (portid); -if (rte_eth_dev_stop (portid) <0 ) -printf ("Failed to stop port %u" , portid); - } - -rte_eal_mp_wait_lcore (); - }if (signum ==SIGTSTP )rte_event_dev_dump (0 , stdout);@@ -465,6 +453,10 @@ main(int argc, char **argv) }RTE_ETH_FOREACH_DEV (portid) { +rte_event_eth_rx_adapter_stop (portid); +rte_event_eth_tx_adapter_stop (portid); +if (rte_eth_dev_stop (portid) <0 ) +printf ("Failed to stop port %u" , portid);rte_eth_dev_close (portid); } --2.25.1
[dpdk-dev] 回复: [EXT] [PATCH v3] examples/eventdev: refactor ethdev port stop
Quoted 11 lines
-----邮件原件-----
发件人: Pavan Nikhilesh Bhagavatula <[email protected]> 发送时间: 2021年1月5日 18:09
收件人: Feifei Wang <[email protected]>; Harry van Haaren <[email protected]>; Nikhil Rao <[email protected]>; Pavan Nikhilesh <[email protected]>
抄送: [email protected]; [email protected]; nd <[email protected]>; [email protected]; Ruifeng Wang <[email protected]>; Honnappa Nagarahalli <[email protected]>
主题: RE: [EXT] [PATCH v3] examples/eventdev: refactor ethdev port stop
Hi, Pavan
Quoted 23 lines
Hi Feifei,
Quoted 9 lines
Move eth stop code from "
signal_handler" function to the end of "main" function. There are two reasons for this:First, this improves code maintenance and makes code look simple and clear. Based on this change, after receiving the interrupt signal, "
fdata->done" is set as 1. Then the main thread will wait all worker lcores to jump out of the loop. Finally, the main thread will stop and then close eth dev port.Quoted 5 lines
Second, for older version, the main thread first stops eth dev port and then waits the end of worker lcore. This may cause errors because it may stop the eth dev port which worker lcores are using. This moving change can fix this by waiting all worker threads to exit and then stop the eth dev port.
Apologies for the delayed reply,
In case of event dev the workers don't interact with eth device directly, Instead eth device "injects" packets into event device and event device is responsible for scheduling them to the workers.
Quoted 5 lines
If the producer is not stopped i.e. in this case eth device then the worker threads might never exit and the main core would wait indefinitely for workers to exit. This will be predominantly seen in cases where there are only a few flows and large number of workers causing a lot of intra thread dependency.
For the case that the event device scheduling packets to the workers. Though the producer is not stopped (eth device), when the main core receive the interrupt signal, the "fdata->done" will be set as 1. Then all the workers load the value 1 of "fdata->done" and jump out of the loop to finish their thread.
Best Regards
Feifei
Quoted 81 lines
Regards,
Pavan.Quoted 38 lines
In the meanwhile, remove wmb in
signal_handler.This is because when the main lcore receive the stop signal, it stores 1 into
fdata->done. And then the worker lcores load "fdata->done" and jump out of the loop to stop running. Nothing should be stored after updatingfdata->done, so the wmb is unnecessary.Fixes 085edac2ca38 ("examples/eventdev_pipeline: support Txadapter")
Cc: [email protected]
Cc: [email protected]Suggested-by Ruifeng Wang <[email protected]>Signed-off-by Feifei Wang <[email protected]>Reviewed-by Ruifeng Wang <[email protected]>Reviewed-by Honnappa NagarahalliAcked-by Harry van Haaren <[email protected]>---
v2:
1. Delete unused variable to fix build error 2. Reduce commit message length to fix coding style issuesv3:
1. Title improvement (Van Haaren)
examples/eventdev_pipeline/main.c| 16 ++++------------1 file changed, 4 insertions(+), 12 deletions(-)examples/eventdev_pipeline/main.c +0
-0examples/eventdev_pipeline/main.c static void · line 285
Quoted 7 lines
examples/eventdev_pipeline/main.c +0
-1@@ -280,7 +280,6 @@ static void signal_handler(int signum) {static uint8_t once; -uint16_t portid;if (fdata->done) rte_exit(1, "Exiting on signal %d\n", signum); @@ -291,17+290,6 @@
Quoted 31 lines
signal_handler(int signum)rte_event_dev_dump(0, stdout); once = 1; fdata->done = 1;examples/eventdev_pipeline/main.c +0
-11-rte_smp_wmb (); - -RTE_ETH_FOREACH_DEV (portid) { -rte_event_eth_rx_adapter_stop (portid); -rte_event_eth_tx_adapter_stop (portid); -if (rte_eth_dev_stop (portid) <0 ) -printf ("Failed to stop port %u" , portid); - } - -rte_eal_mp_wait_lcore (); -} if (signum == SIGTSTP) rte_event_dev_dump(0, stdout);examples/eventdev_pipeline/main.c +4
-1@@ -465,6 +453,10 @@ main(int argc, char **argv) }RTE_ETH_FOREACH_DEV (portid) { +rte_event_eth_rx_adapter_stop (portid); +rte_event_eth_tx_adapter_stop (portid); +if (rte_eth_dev_stop (portid) <0 ) +printf ("Failed to stop port %u" , portid);rte_eth_dev_close (portid); } --2.25.1
[dpdk-dev] 回复: [EXT] [PATCH v3] examples/eventdev: refactor ethdev port stop
Quoted 56 lines
-----邮件原件-----
发件人: Feifei Wang
发送时间: 2021年1月14日 14:24
收件人: Pavan Nikhilesh Bhagavatula <[email protected]>; Harry van Haaren <[email protected]>; Nikhil Rao <[email protected]>; Pavan Nikhilesh <[email protected]>
抄送: [email protected]; [email protected]; nd <[email protected]>; [email protected]; Ruifeng Wang <[email protected]>; Honnappa Nagarahalli <[email protected]>; nd <[email protected]> 主题: 回复: [EXT] [PATCH v3] examples/eventdev: refactor ethdev port stopQuoted 11 lines
-----邮件原件-----
发件人: Pavan Nikhilesh Bhagavatula <[email protected]> 发送时间: 2021年1月5日 18:09
收件人: Feifei Wang <[email protected]>; Harry van Haaren <[email protected]>; Nikhil Rao <[email protected]>; Pavan Nikhilesh <[email protected]>
抄送: [email protected]; [email protected]; nd <[email protected]>; [email protected]; Ruifeng Wang <[email protected]>; Honnappa Nagarahalli <[email protected]>
主题: RE: [EXT] [PATCH v3] examples/eventdev: refactor ethdev port stopHi, Pavan
Quoted 23 lines
Hi Feifei,
Quoted 9 lines
Move eth stop code from "
signal_handler" function to the end of "main" function. There are two reasons for this:First, this improves code maintenance and makes code look simple and clear. Based on this change, after receiving the interrupt signal, "
fdata->done" is set as 1. Then the main thread will wait all worker lcores to jump out of the loop. Finally, the main thread will stop and then close eth dev port.Quoted 5 lines
Second, for older version, the main thread first stops eth dev port and then waits the end of worker lcore. This may cause errors because it may stop the eth dev port which worker lcores are using. This moving change can fix this by waiting all worker threads to exit and then stop the eth dev port.
Apologies for the delayed reply,
In case of event dev the workers don't interact with eth device directly, Instead eth device "injects" packets into event device and event device is responsible for scheduling them to the workers.
Quoted 5 lines
If the producer is not stopped i.e. in this case eth device then the worker threads might never exit and the main core would wait indefinitely for workers to exit. This will be predominantly seen in cases where there are only a few flows and large number of workers causing a lot of intra thread dependency.
Quoted 6 lines
For the case that the event device scheduling packets to the workers. Though the producer is not stopped (eth device), when the main core receive the interrupt signal, the "
fdata->done"
will be set as 1. Then all the workers load the value 1 of "fdata->done" and jump out of the loop to finish their thread.
And I also test the case you said above with this patch, when I send interrupt signal(ctrl + c), IXIA is running and eth device is also working, then the program can exit normally.
HW:
Traffic generator: IXIA
Nics: ixgbe 82599ES 10-Gigabit
Architecture: aarch64
CPU: Cortex-A72
With this patch:
______________________________________________________________________________________________
Command: sudo ./dpdk-eventdev_pipeline -l 0-2,8-11 --vdev=event_sw0 -- -r2 -t2 -e4 -w F00 -s4 -n0 -c32 -W1000 -D
.........
Config:
ports: 2
workers: 4
packets: 9223372036854775807
Queue-prio: 0
qid0 type: atomic
Cores available: 7
Cores used: 6
Eventdev 0: event_sw
Stages:
Stage 0, Type Atomic Priority = 128
Stage 1, Type Atomic Priority = 128
Stage 2, Type Atomic Priority = 128
Stage 3, Type Atomic Priority = 128Port 0 modified RSS hash function based on hardware support,requested:0x3afbc configured:0x38d34 Port 0 MAC: 90 e2 ba 56 ed 6c
Port 1 modified RSS hash function based on hardware support,requested:0x3afbc configured:0x38d34 Port 1 MAC: 90 e2 ba 56 ed 6d
[dump_core_info()] lcore 1 executing NIC Rx
[dump_core_info()] lcore 1 executing NIC Tx
[dump_core_info()] lcore 2 executing scheduler
[dump_core_info()] lcore 8 executing worker, using eventdev port 0 [dump_core_info()] lcore 9 executing worker, using eventdev port 1 [dump_core_info()] lcore 10 executing worker, using eventdev port 2 [dump_core_info()] lcore 11 executing worker, using eventdev port 3
Command: ctrl + c
.........
worker 1 thread done. RX=0 TX=0
worker 2 thread done. RX=0 TX=0
worker 11 thread done. RX=1844023 TX=1844023
worker 9 thread done. RX=1844888 TX=1844888
worker 8 thread done. RX=1845490 TX=1845490
worker 10 thread done. RX=1844903 TX=1844903
Port Workload distribution:
worker 0 : 25.0 % (1845474 pkts) worker 1 : 25.0 % (1844872 pkts) worker 2 : 25.0 % (1844887 pkts) worker 3 : 25.0 % (1844007 pkts)
Invalid port_id=0
Invalid port_id=1
______________________________________________________________________________________________
Quoted 84 lines
Best Regards
FeifeiQuoted 82 lines
Regards,
Pavan.Quoted 39 lines
In the meanwhile, remove wmb in
signal_handler.This is because when the main lcore receive the stop signal, it stores
1 intofdata->done. And then the worker lcores load "fdata->done" and jump out of the loop to stop running. Nothing should be stored after updatingfdata->done, so the wmb is unnecessary.Fixes 085edac2ca38 ("examples/eventdev_pipeline: support Txadapter")
Cc: [email protected]
Cc: [email protected]Suggested-by Ruifeng Wang <[email protected]>Signed-off-by Feifei Wang <[email protected]>Reviewed-by Ruifeng Wang <[email protected]>Reviewed-by Honnappa NagarahalliAcked-by Harry van Haaren <[email protected]>---
v2:
1. Delete unused variable to fix build error 2. Reduce commit message length to fix coding style issuesv3:
1. Title improvement (Van Haaren)
examples/eventdev_pipeline/main.c| 16 ++++------------1 file changed, 4 insertions(+), 12 deletions(-)examples/eventdev_pipeline/main.c +0
-0examples/eventdev_pipeline/main.c static void · line 285
Quoted 7 lines
examples/eventdev_pipeline/main.c +0
-1@@ -280,7 +280,6 @@ static void signal_handler(int signum) {static uint8_t once; -uint16_t portid;if (fdata->done) rte_exit(1, "Exiting on signal %d\n", signum); @@ -291,17+290,6 @@
Quoted 31 lines
signal_handler(int signum)rte_event_dev_dump(0, stdout); once = 1; fdata->done = 1;examples/eventdev_pipeline/main.c +0
-11-rte_smp_wmb (); - -RTE_ETH_FOREACH_DEV (portid) { -rte_event_eth_rx_adapter_stop (portid); -rte_event_eth_tx_adapter_stop (portid); -if (rte_eth_dev_stop (portid) <0 ) -printf ("Failed to stop port %u" , portid); - } - -rte_eal_mp_wait_lcore (); -} if (signum == SIGTSTP) rte_event_dev_dump(0, stdout);examples/eventdev_pipeline/main.c +4
-1@@ -465,6 +453,10 @@ main(int argc, char **argv) }RTE_ETH_FOREACH_DEV (portid) { +rte_event_eth_rx_adapter_stop (portid); +rte_event_eth_tx_adapter_stop (portid); +if (rte_eth_dev_stop (portid) <0 ) +printf ("Failed to stop port %u" , portid);rte_eth_dev_close (portid); } --2.25.1
v4 cover fix bugs for examples/evnentdev
In reply to Feifei WangFix some bugs for eventdev pipeline example.
v2:
1. Delete unused variable to fix build error
2. Reduce commit message length to fix coding style issues
v3:
1. Title improvement (Van Haaren)
v4:
1. fix core enabled bug
2. add info typo for main core
Feifei Wang (3):
doc: fix core enabled bug for eventdev pipeline example examples/eventdev: add info typo for main core examples/eventdev: move eth stop to the end
.../sample_app_ug/eventdev_pipeline.rst | 5 +- examples/eventdev_pipeline/main.c | 78 +++++++++++-------- 2 files changed, 47 insertions(+), 36 deletions(-)
Signature · 1 line
2.25.1
PATCH v4 1/3 doc: fix core enabled bug for eventdev pipeline example
In the case that the cores are isolated, if "-l" or "-c" parameter is not added, the cores will not be enabled and can not launch worker function correctly. In the meanwhile, no error information is reported.
For example:
totally CPUs:16
isolated CPUs:1-8
command: sudo gdb -args ./dpdk-eventdev_pipeline --vdev event_sw0 \
-- -r1 -t1 -e4 -w F00 -s4 -n0 -c32 -W1000 -Dcores information:rte_config->lcore_role = {ROLE_RTE, ROLE_OFF, ROLE_OFF, ROLE_OFF,
ROLE_OFF, ROLE_OFF, ROLE_OFF, ROLE_OFF, ROLE_OFF, ROLE_RTE, ROLE_RTE, ROLE_RTE, ROLE_RTE, ROLE_RTE, ROLE_RTE, ROLE_RTE}
output information:
...
[main()] lcore 9 executing worker, using eventdev port 0 [main()] lcore 10 executing worker, using eventdev port 1 [main()] lcore 11 executing worker, using eventdev port 2
This is because "RTE_LCORE_FOREACH_WORKER" chooses the enabled core. In the case that the cores are isolated, "the lcore_role" flag of isolated cores are set as "ROLE_OFF" by default(not enabled). So if we choose these isolated cores as workers, "RTE_LCORE_FOREACH_WORKER" will ignore these cores and not launch worker functions on them.
To fix this, add "-l" parameters to doc and add lcore enabled check.
--- doc/guides/sample_app_ug/eventdev_pipeline.rst | 5 +++-- examples/eventdev_pipeline/main.c | 7 ++++++- 2 files changed, 9 insertions(+), 3 deletions(-)
doc/guides/sample_app_ug/eventdev_pipeline.rst +3 -2
@@ -34,6 +34,7 @@ options. An example eventdev pipeline running with the software eventdev PMD using these settings is shown below: + * ``-l 0,2,8-15``: lcore to use * ``-r1``: core mask 0x1 for RX * ``-t1``: core mask 0x1 for TX * ``-e4``: core mask 0x4 for the software scheduler
@@ -46,8 +47,8 @@ these settings is shown below: .. code-block:: console - ./<build_dir>/examples/dpdk-eventdev_pipeline --vdev event_sw0 -- -r1 -t1 \ - -e4 -w FF00 -s4 -n0 -c32 -W1000 -D + ./<build_dir>/examples/dpdk-eventdev_pipeline -l 0,2,8-15 --vdev event_sw0 \ + -- -r1 -t1 -e4 -w FF00 -s4 -n0 -c32 -W1000 -D The application has some sanity checking built-in, so if there is a function (e.g.; the RX core) which doesn't have a cpu core mask assigned, the application
examples/eventdev_pipeline/main.c +6 -1
@@ -239,8 +239,13 @@ parse_app_args(int argc, char **argv)if (fdata->worker_core[i]) cdata.num_workers++; -if (core_in_use (i)) +if (core_in_use (i)) { +if (!rte_lcore_is_enabled (i)) { +printf ("error: lcore %d is not enabled in lcore list\n" , i); +rte_exit (EXIT_FAILURE ,"check lcore params failed\n" ); + } cdata.active_cores++; + } } }
Signature · 1 line
2.25.1
PATCH v4 2/3 examples/eventdev: add info output for main core
In reply to Feifei WangWhen the main core is set as tx/rx/sched/worker core, it also needs to print some information to show this. Thus, add info output for the main core, and add a "dump" function to print core information for the sake of code simplicity and easy maintenance.
In the meanwhile, fix the count error. For the variable "worker_idx", it should be incremented when the core is set as worker core. However, when the main core is set as rx/tx/sched core, the worker_idx is also incremented. Though this error may not have a substantial impact due to that the main core is the last launched core, but it should be corrected from the perspective of code correctness.
--- examples/eventdev_pipeline/main.c | 55 +++++++++++++++++++------------ 1 file changed, 34 insertions(+), 21 deletions(-)
examples/eventdev_pipeline/main.c +34 -21
@@ -22,6 +22,32 @@ struct config_data cdata = { .worker_cq_depth = 16 }; +static void +dump_core_info (unsigned int lcore_id,struct worker_data *data, +unsigned int worker_idx) +{ +if (fdata->rx_core[lcore_id]) +printf ( +"[%s()] lcore %d executing NIC Rx\n" , + __func__, lcore_id); + +if (fdata->tx_core[lcore_id]) +printf ( +"[%s()] lcore %d executing NIC Tx\n" , + __func__, lcore_id); + +if (fdata->sched_core[lcore_id]) +printf ( +"[%s()] lcore %d executing scheduler\n" , + __func__, lcore_id); + +if (fdata->worker_core[lcore_id]) +printf ( +"[%s()] lcore %d executing worker, using eventdev port %u\n" , + __func__, lcore_id, + data[worker_idx].port_id); +} +static bool core_in_use (unsigned int lcore_id) { return (fdata->rx_core[lcore_id] || fdata->sched_core[lcore_id] ||
@@ -411,25 +437,7 @@ main(int argc, char **argv) !fdata->sched_core[lcore_id])continue ; -if (fdata->rx_core[lcore_id]) -printf ( -"[%s()] lcore %d executing NIC Rx\n" , - __func__, lcore_id); - -if (fdata->tx_core[lcore_id]) -printf ( -"[%s()] lcore %d executing NIC Tx\n" , - __func__, lcore_id); - -if (fdata->sched_core[lcore_id]) -printf ("[%s()] lcore %d executing scheduler\n" , - __func__, lcore_id); - -if (fdata->worker_core[lcore_id]) -printf ( -"[%s()] lcore %d executing worker, using eventdev port %u\n" , - __func__, lcore_id, - worker_data[worker_idx].port_id); +dump_core_info (lcore_id, worker_data, worker_idx); err =rte_eal_remote_launch (fdata->cap.worker, &worker_data[worker_idx], lcore_id);
@@ -444,8 +452,13 @@ main(int argc, char **argv) lcore_id =rte_lcore_id (); -if (core_in_use (lcore_id)) - fdata->cap.worker (&worker_data[worker_idx++]); +if (core_in_use (lcore_id)) { +dump_core_info (lcore_id, worker_data, worker_idx); + fdata->cap.worker (&worker_data[worker_idx]); + +if (fdata->worker_core[lcore_id]) + worker_idx++; + }rte_eal_mp_wait_lcore ();
Signature · 1 line
2.25.1
PATCH v4 3/3 examples/eventdev: move eth stop to the end
In reply to Feifei WangMove eth stop code from "signal_handler" function to the end of "main" function. There are two reasons for this:
First, this improves code maintenance and makes code look simple and clear. Based on this change, after receiving the interrupt signal, "fdata->done" is set as 1. Then the main thread will wait all worker lcores to jump out of the loop. Finally, the main thread will stop and then close eth dev port.
Second, for older version, the main thread first stops eth dev port and then waits the end of worker lcore. This may cause errors because it may stop the eth dev port which worker lcores are using. This moving change can fix this by waiting all worker threads to exit and then stop the eth dev port.
In the meanwhile, remove wmb in signal_handler.
This is because when the main lcore receive the stop signal, it stores 1 into fdata->done. And then the worker lcores load "fdata->done" and jump out of the loop to stop running. Nothing should be stored after updating fdata->done, so the wmb is unnecessary.
--- examples/eventdev_pipeline/main.c | 16 ++++------------ 1 file changed, 4 insertions(+), 12 deletions(-)
examples/eventdev_pipeline/main.c +4 -12
@@ -311,7 +311,6 @@ static void signal_handler(int signum) {static uint8_t once; -uint16_t portid;if (fdata->done)rte_exit (1 ,"Exiting on signal %d\n" , signum);
@@ -322,17 +321,6 @@ signal_handler(int signum)rte_event_dev_dump (0 , stdout); once =1 ; fdata->done =1 ; -rte_smp_wmb (); - -RTE_ETH_FOREACH_DEV (portid) { -rte_event_eth_rx_adapter_stop (portid); -rte_event_eth_tx_adapter_stop (portid); -if (rte_eth_dev_stop (portid) <0 ) -printf ("Failed to stop port %u" , portid); - } - -rte_eal_mp_wait_lcore (); - }if (signum ==SIGTSTP )rte_event_dev_dump (0 , stdout);
@@ -483,6 +471,10 @@ main(int argc, char **argv) }RTE_ETH_FOREACH_DEV (portid) { +rte_event_eth_rx_adapter_stop (portid); +rte_event_eth_tx_adapter_stop (portid); +if (rte_eth_dev_stop (portid) <0 ) +printf ("Failed to stop port %u" , portid);rte_eth_dev_close (portid); }
Signature · 1 line
2.25.1
Quoted 32 lines
Move eth stop code from "
signal_handler" function to the end of "main"
function. There are two reasons for this:First, this improves code maintenance and makes code look simple and clear. Based on this change, after receiving the interrupt signal, "
fdata->done" is set as 1. Then the main thread will wait all worker lcores to jump out of the loop. Finally, the main thread will stop and then close eth dev port.Second, for older version, the main thread first stops eth dev port and then waits the end of worker lcore. This may cause errors because it may
stop the eth dev port which worker lcores are using. This moving change
can fix this by waiting all worker threads to exit and then stop the eth dev port.In the meanwhile, remove wmb in
signal_handler.This is because when the main lcore receive the stop signal, it stores 1 into
fdata->done. And then the worker lcores load "fdata->done" and jump
out of the loop to stop running. Nothing should be stored after updatingfdata->done, so the wmb is unnecessary.Fixes 085edac2ca38 ("examples/eventdev_pipeline: support Txadapter")
Cc: [email protected]
Cc: [email protected]
Quoted 6 lines
Suggested-by Ruifeng Wang <[email protected]>Signed-off-by Feifei Wang <[email protected]>Reviewed-by Ruifeng Wang <[email protected]>Reviewed-by Honnappa NagarahalliAcked-by Harry van Haaren <[email protected]>
Quoted 48 lines
---
examples/eventdev_pipeline/main.c| 16 ++++------------1 file changed, 4 insertions(+), 12 deletions(-)examples/eventdev_pipeline/main.c +4
-13@@ -311,7 +311,6 @@ static void signal_handler(int signum) {static uint8_t once; -uint16_t portid;if (fdata->done)rte_exit (1 ,"Exiting on signal %d\n" , signum);@@ -322,17 +321,6 @@ signal_handler(int signum)rte_event_dev_dump (0 , stdout); once =1 ; fdata->done =1 ; -rte_smp_wmb (); - -RTE_ETH_FOREACH_DEV (portid) { -rte_event_eth_rx_adapter_stop (portid); -rte_event_eth_tx_adapter_stop (portid); -if (rte_eth_dev_stop (portid) <0 ) -printf ("Failed to stop port %u" , portid); - } - -rte_eal_mp_wait_lcore (); - }if (signum ==SIGTSTP )rte_event_dev_dump (0 , stdout);@@ -483,6 +471,10 @@ main(int argc, char **argv) }RTE_ETH_FOREACH_DEV (portid) { +rte_event_eth_rx_adapter_stop (portid); +rte_event_eth_tx_adapter_stop (portid); +if (rte_eth_dev_stop (portid) <0 ) +printf ("Failed to stop port %u" , portid);rte_eth_dev_close (portid); } --2.25.1
Quoted 33 lines · Pavan Nikhilesh Bhagavatula
Quoted 33 lines
Move eth stop code from "
signal_handler" function to the end of "main"
function. There are two reasons for this:First, this improves code maintenance and makes code look simple and clear. Based on this change, after receiving the interrupt signal, "
fdata->done" is set as 1. Then the main thread will wait all worker lcores to jump out of the loop. Finally, the main thread will stop and then close eth dev port.Second, for older version, the main thread first stops eth dev port and then waits the end of worker lcore. This may cause errors because it may
stop the eth dev port which worker lcores are using. This moving change
can fix this by waiting all worker threads to exit and then stop the eth dev port.In the meanwhile, remove wmb in
signal_handler.This is because when the main lcore receive the stop signal, it stores 1 into
fdata->done. And then the worker lcores load "fdata->done" and jump
out of the loop to stop running. Nothing should be stored after updatingfdata->done, so the wmb is unnecessary.Fixes 085edac2ca38 ("examples/eventdev_pipeline: support Txadapter")
Cc: [email protected]
Cc: [email protected]
Quoted 8 lines
Quoted 6 lines
Suggested-by Ruifeng Wang <[email protected]>Signed-off-by Feifei Wang <[email protected]>Reviewed-by Ruifeng Wang <[email protected]>Reviewed-by Honnappa NagarahalliAcked-by Harry van Haaren <[email protected]>Acked-by Pavan Nikhilesh <[email protected]>
Series applied to dpdk-next-eventdev/for-main. Thanks.
Quoted 50 lines
Quoted 49 lines
---
examples/eventdev_pipeline/main.c| 16 ++++------------1 file changed, 4 insertions(+), 12 deletions(-)examples/eventdev_pipeline/main.c +4
-13@@ -311,7 +311,6 @@ static void signal_handler(int signum) {static uint8_t once; -uint16_t portid;if (fdata->done)rte_exit (1 ,"Exiting on signal %d\n" , signum);@@ -322,17 +321,6 @@ signal_handler(int signum)rte_event_dev_dump (0 , stdout); once =1 ; fdata->done =1 ; -rte_smp_wmb (); - -RTE_ETH_FOREACH_DEV (portid) { -rte_event_eth_rx_adapter_stop (portid); -rte_event_eth_tx_adapter_stop (portid); -if (rte_eth_dev_stop (portid) <0 ) -printf ("Failed to stop port %u" , portid); - } - -rte_eal_mp_wait_lcore (); - }if (signum ==SIGTSTP )rte_event_dev_dump (0 , stdout);@@ -483,6 +471,10 @@ main(int argc, char **argv) }RTE_ETH_FOREACH_DEV (portid) { +rte_event_eth_rx_adapter_stop (portid); +rte_event_eth_tx_adapter_stop (portid); +if (rte_eth_dev_stop (portid) <0 ) +printf ("Failed to stop port %u" , portid);rte_eth_dev_close (portid); } --2.25.1