Skip to contentSkip to conversation

examples/eventdev: move eth stop to the end

PATCH Superseded Feifei Wang 4 participants 14 messages Dec 21, 2020 – Jan 26, 2021

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 examples/eventdev: move eth stop to the end

Move eth stop code from "signal_handler" function to the end of "main" function. There are two reasons for this:

First, this improves code maintenance and makes code look simple and clear. Based on this change, after receiving the interrupt signal, "fdata->done" is set as 1. Then the main thread will wait all worker lcores to jump out of the loop. Finally, the main thread will stop and then close eth dev port.

Second, for older version, the main thread first stops eth dev port and then waits the end of worker lcore. This may cause errors because it may stop the eth dev port which worker lcores are using. This moving change can fix this by waiting all worker threads to exit and then stop the eth dev port.

In the meanwhile, remove wmb in signal_handler.

This is because when the main lcore receive the stop signal, it stores 1 into fdata->done. And then the worker lcores load "fdata->done" and jump out of the loop to stop running. Nothing should be stored after updating fdata->done, so the wmb is unnecessary.

Fixes 085edac2ca38 ("examples/eventdev_pipeline: support Tx adapter")
---
 examples/eventdev_pipeline/main.c | 15 ++++-----------
 1 file changed, 4 insertions(+), 11 deletions(-)
examples/eventdev_pipeline/main.c +4 -11
index 823f8b51c..ac2ace5c4 100644
@@ -291,17 +291,6 @@ signal_handler(int signum)
 			rte_event_dev_dump(0, stdout);
 		once = 1;
 		fdata->done = 1;
-		rte_smp_wmb();
-
-		RTE_ETH_FOREACH_DEV(portid) {
-			rte_event_eth_rx_adapter_stop(portid);
-			rte_event_eth_tx_adapter_stop(portid);
-			if (rte_eth_dev_stop(portid) < 0)
-				printf("Failed to stop port %u", portid);
-		}
-
-		rte_eal_mp_wait_lcore();
-
 	}
 	if (signum == SIGTSTP)
 		rte_event_dev_dump(0, stdout);
@@ -465,6 +454,10 @@ main(int argc, char **argv)
 	}
 
 	RTE_ETH_FOREACH_DEV(portid) {
+		rte_event_eth_rx_adapter_stop(portid);
+		rte_event_eth_tx_adapter_stop(portid);
+		if (rte_eth_dev_stop(portid) < 0)
+			printf("Failed to stop port %u", portid);
 		rte_eth_dev_close(portid);
 	}
 
Signature · 1 line
2.17.1
#1 lore raw

PATCH v2 1/1 examples/eventdev: move eth stop to the end

Move eth stop code from "signal_handler" function to the end of "main" function. There are two reasons for this:

First, this improves code maintenance and makes code look simple and clear. Based on this change, after receiving the interrupt signal, "fdata->done" is set as 1. Then the main thread will wait all worker lcores to jump out of the loop. Finally, the main thread will stop and then close eth dev port.

Second, for older version, the main thread first stops eth dev port and then waits the end of worker lcore. This may cause errors because it may stop the eth dev port which worker lcores are using. This moving change can fix this by waiting all worker threads to exit and then stop the eth dev port.

In the meanwhile, remove wmb in signal_handler.

This is because when the main lcore receive the stop signal, it stores 1 into fdata->done. And then the worker lcores load "fdata->done" and jump out of the loop to stop running. Nothing should be stored after updating fdata->done, so the wmb is unnecessary.

Fixes 085edac2ca38 ("examples/eventdev_pipeline: support Tx adapter")

---

v2:
1. Delete unused variable to fix build error
2. Reduce commit message length to fix coding style issues

 examples/eventdev_pipeline/main.c | 16 ++++------------
 1 file changed, 4 insertions(+), 12 deletions(-)
examples/eventdev_pipeline/main.c +4 -12
index 823f8b51c..fdbaf667b 100644
@@ -280,7 +280,6 @@ static void
 signal_handler(int signum)
 {
 	static uint8_t once;
-	uint16_t portid;
 
 	if (fdata->done)
 		rte_exit(1, "Exiting on signal %d\n", signum);
@@ -291,17 +290,6 @@ signal_handler(int signum)
 			rte_event_dev_dump(0, stdout);
 		once = 1;
 		fdata->done = 1;
-		rte_smp_wmb();
-
-		RTE_ETH_FOREACH_DEV(portid) {
-			rte_event_eth_rx_adapter_stop(portid);
-			rte_event_eth_tx_adapter_stop(portid);
-			if (rte_eth_dev_stop(portid) < 0)
-				printf("Failed to stop port %u", portid);
-		}
-
-		rte_eal_mp_wait_lcore();
-
 	}
 	if (signum == SIGTSTP)
 		rte_event_dev_dump(0, stdout);
@@ -465,6 +453,10 @@ main(int argc, char **argv)
 	}
 
 	RTE_ETH_FOREACH_DEV(portid) {
+		rte_event_eth_rx_adapter_stop(portid);
+		rte_event_eth_tx_adapter_stop(portid);
+		if (rte_eth_dev_stop(portid) < 0)
+			printf("Failed to stop port %u", portid);
 		rte_eth_dev_close(portid);
 	}
 
Signature · 1 line
2.17.1
#2 lore raw
In reply to Feifei Wang
Quoted 8 lines

-----Original Message-----
From: Feifei Wang <[email protected]>
Sent: Monday, December 21, 2020 5:35 AM
To: Van Haaren, Harry <[email protected]>; Rao, Nikhil <[email protected]>; Pavan Nikhilesh <[email protected]>
Cc: [email protected]; [email protected]; Feifei Wang <[email protected]>; [email protected]; [email protected]
Subject: [PATCH] examples/eventdev: move eth stop to the end

Suggested title improvement? examples/eventdev_pipeline: refactor ethdev port stop

Quoted 8 lines

Move eth stop code from "signal_handler" function to the end of "main" function. There are two reasons for this:

First, this improves code maintenance and makes code look simple and clear. Based on this change, after receiving the interrupt signal, "fdata->done" is set as 1. Then the main thread will wait all worker lcores to jump out of the loop. Finally, the main thread will stop and then close eth dev port.

Quoted 4 lines

Second, for older version, the main thread first stops eth dev port and then waits the end of worker lcore. This may cause errors because it may stop the eth dev port which worker lcores are using. This moving change can fix this by waiting all worker threads to exit and then stop the eth dev port.

I'm OK with the above changes, and agree that moving eth dev port close to after lcores return is a worthy change.

Quoted 7 lines

In the meanwhile, remove wmb in signal_handler.

This is because when the main lcore receive the stop signal, it stores 1 into fdata->done. And then the worker lcores load "fdata->done" and jump out of the loop to stop running. Nothing should be stored after updating fdata->done, so the wmb is unnecessary.

Quoted 8 lines
Fixes 085edac2ca38 ("examples/eventdev_pipeline: support Tx adapter")

+Cc Jerin for Eventdev tree;
-Cc Pavan's old email address

Ack-ed by: Harry van Haaren <[email protected]>

#3 lore raw

Hi, Van Haaren

Quoted 19 lines

-----邮件原件-----
发件人: Van Haaren, Harry <[email protected]> 发送时间: 2020年12月21日 17:57
收件人: Feifei Wang <[email protected]>; Rao, Nikhil <[email protected]>
抄送: [email protected]; nd <[email protected]>; [email protected]; [email protected]; [email protected]
主题: RE: [PATCH] examples/eventdev: move eth stop to the end

Quoted 9 lines

-----Original Message-----
From: Feifei Wang <[email protected]>
Sent: Monday, December 21, 2020 5:35 AM
To: Van Haaren, Harry <[email protected]>; Rao, Nikhil <[email protected]>; Pavan Nikhilesh <[email protected]>
Cc: [email protected]; [email protected]; Feifei Wang <[email protected]>; [email protected]; [email protected]
Subject: [PATCH] examples/eventdev: move eth stop to the end

Quoted 2 lines

Suggested title improvement? examples/eventdev_pipeline: refactor ethdev port stop

That's OK. I will apply this in the next version.

Quoted 37 lines
Quoted 5 lines

Move eth stop code from "signal_handler" function to the end of "main" function. There are two reasons for this:

First, this improves code maintenance and makes code look simple and

clear.

Quoted 3 lines

Based on this change, after receiving the interrupt signal, "fdata->done" is set as 1. Then the main thread will wait all worker lcores to jump out of the loop. Finally, the main thread will stop and then close eth dev

port.

Quoted 5 lines

Second, for older version, the main thread first stops eth dev port and then waits the end of worker lcore. This may cause errors because it may stop the eth dev port which worker lcores are using. This moving change can fix this by waiting all worker threads to exit and then

stop the eth dev port.

I'm OK with the above changes, and agree that moving eth dev port close to after lcores return is a worthy change.

Quoted 16 lines

In the meanwhile, remove wmb in signal_handler.

This is because when the main lcore receive the stop signal, it stores 1 into fdata->done. And then the worker lcores load "fdata->done" and jump out of the loop to stop running. Nothing should be stored after updating
fdata->done, so the wmb is unnecessary.

Fixes 085edac2ca38 ("examples/eventdev_pipeline: support Tx adapter")

Review

Quoted 2 lines
(quoted diff) +1 -1
+Cc Jerin for Eventdev tree;
-Cc Pavan's old email address

Thanks very much for this change.

Quoted 2 lines

Ack-ed by: Harry van Haaren <[email protected]>

Best Regards
Feifei

#4 lore raw

PATCH v3 1/1 examples/eventdev: refactor ethdev port stop

In reply to Feifei Wang

Move eth stop code from "signal_handler" function to the end of "main" function. There are two reasons for this:

First, this improves code maintenance and makes code look simple and clear. Based on this change, after receiving the interrupt signal, "fdata->done" is set as 1. Then the main thread will wait all worker lcores to jump out of the loop. Finally, the main thread will stop and then close eth dev port.

Second, for older version, the main thread first stops eth dev port and then waits the end of worker lcore. This may cause errors because it may stop the eth dev port which worker lcores are using. This moving change can fix this by waiting all worker threads to exit and then stop the eth dev port.

In the meanwhile, remove wmb in signal_handler.

This is because when the main lcore receive the stop signal, it stores 1 into fdata->done. And then the worker lcores load "fdata->done" and jump out of the loop to stop running. Nothing should be stored after updating fdata->done, so the wmb is unnecessary.

Fixes 085edac2ca38 ("examples/eventdev_pipeline: support Tx adapter")

---

v2:
1. Delete unused variable to fix build error
2. Reduce commit message length to fix coding style issues

v3:
1. Title improvement (Van Haaren)

 examples/eventdev_pipeline/main.c | 16 ++++------------
 1 file changed, 4 insertions(+), 12 deletions(-)
examples/eventdev_pipeline/main.c +4 -12
index 823f8b51c..fdbaf667b 100644
@@ -280,7 +280,6 @@ static void
 signal_handler(int signum)
 {
 	static uint8_t once;
-	uint16_t portid;
 
 	if (fdata->done)
 		rte_exit(1, "Exiting on signal %d\n", signum);
@@ -291,17 +290,6 @@ signal_handler(int signum)
 			rte_event_dev_dump(0, stdout);
 		once = 1;
 		fdata->done = 1;
-		rte_smp_wmb();
-
-		RTE_ETH_FOREACH_DEV(portid) {
-			rte_event_eth_rx_adapter_stop(portid);
-			rte_event_eth_tx_adapter_stop(portid);
-			if (rte_eth_dev_stop(portid) < 0)
-				printf("Failed to stop port %u", portid);
-		}
-
-		rte_eal_mp_wait_lcore();
-
 	}
 	if (signum == SIGTSTP)
 		rte_event_dev_dump(0, stdout);
@@ -465,6 +453,10 @@ main(int argc, char **argv)
 	}
 
 	RTE_ETH_FOREACH_DEV(portid) {
+		rte_event_eth_rx_adapter_stop(portid);
+		rte_event_eth_tx_adapter_stop(portid);
+		if (rte_eth_dev_stop(portid) < 0)
+			printf("Failed to stop port %u", portid);
 		rte_eth_dev_close(portid);
 	}
 
Signature · 1 line
2.25.1
#5 lore raw

Re: [dpdk-dev] [EXT] [PATCH v3] examples/eventdev: refactor ethdev port stop

Hi Feifei,

Quoted 10 lines

Move eth stop code from "signal_handler" function to the end of "main"
function. There are two reasons for this:

First, this improves code maintenance and makes code look simple and clear. Based on this change, after receiving the interrupt signal, "fdata->done" is set as 1. Then the main thread will wait all worker lcores to jump out of the loop. Finally, the main thread will stop and then close eth dev port.

Quoted 7 lines

Second, for older version, the main thread first stops eth dev port and then waits the end of worker lcore. This may cause errors because it may
stop the eth dev port which worker lcores are using. This moving change
can fix this by waiting all worker threads to exit and then stop the eth dev port.

Apologies for the delayed reply,

In case of event dev the workers don't interact with eth device directly, Instead eth device "injects" packets into event device and event device is responsible for scheduling them to the workers.

If the producer is not stopped i.e. in this case eth device then the worker threads might never exit and the main core would wait indefinitely for workers to exit. This will be predominantly seen in cases where there are only a few flows and large number of workers causing a lot of intra thread dependency.

Regards,
Pavan.

Quoted 77 lines

In the meanwhile, remove wmb in signal_handler.

This is because when the main lcore receive the stop signal, it stores 1 into fdata->done. And then the worker lcores load "fdata->done" and jump
out of the loop to stop running. Nothing should be stored after updating
fdata->done, so the wmb is unnecessary.

Fixes 085edac2ca38 ("examples/eventdev_pipeline: support Tx

adapter")
Cc: [email protected]
Cc: [email protected]

Reviewed-by Honnappa Nagarahalli

<[email protected]>

---

v2:
1. Delete unused variable to fix build error
2. Reduce commit message length to fix coding style issues

v3:
1. Title improvement (Van Haaren)

examples/eventdev_pipeline/main.c | 16 ++++------------

1 file changed, 4 insertions(+), 12 deletions(-)
examples/eventdev_pipeline/main.c +4 -13
index 823f8b51c..fdbaf667b 100644
@@ -280,7 +280,6 @@ static void
 signal_handler(int signum)
 {
 	static uint8_t once;
-	uint16_t portid;
 
 	if (fdata->done)
 		rte_exit(1, "Exiting on signal %d\n", signum);
@@ -291,17 +290,6 @@ signal_handler(int signum)
 			rte_event_dev_dump(0, stdout);
 		once = 1;
 		fdata->done = 1;
-		rte_smp_wmb();
-
-		RTE_ETH_FOREACH_DEV(portid) {
-			rte_event_eth_rx_adapter_stop(portid);
-			rte_event_eth_tx_adapter_stop(portid);
-			if (rte_eth_dev_stop(portid) < 0)
-				printf("Failed to stop port %u", portid);
-		}
-
-		rte_eal_mp_wait_lcore();
-
 	}
 	if (signum == SIGTSTP)
 		rte_event_dev_dump(0, stdout);
@@ -465,6 +453,10 @@ main(int argc, char **argv)
 	}
 
 	RTE_ETH_FOREACH_DEV(portid) {
+		rte_event_eth_rx_adapter_stop(portid);
+		rte_event_eth_tx_adapter_stop(portid);
+		if (rte_eth_dev_stop(portid) < 0)
+			printf("Failed to stop port %u", portid);
 		rte_eth_dev_close(portid);
 	}
 
--

2.25.1

#6 lore raw

[dpdk-dev] 回复: [EXT] [PATCH v3] examples/eventdev: refactor ethdev port stop

Quoted 11 lines

-----邮件原件-----
发件人: Pavan Nikhilesh Bhagavatula <[email protected]> 发送时间: 2021年1月5日 18:09
收件人: Feifei Wang <[email protected]>; Harry van Haaren <[email protected]>; Nikhil Rao <[email protected]>; Pavan Nikhilesh <[email protected]>
抄送: [email protected]; [email protected]; nd <[email protected]>; [email protected]; Ruifeng Wang <[email protected]>; Honnappa Nagarahalli <[email protected]>
主题: RE: [EXT] [PATCH v3] examples/eventdev: refactor ethdev port stop

Hi, Pavan

Quoted 23 lines

Hi Feifei,

Quoted 9 lines

Move eth stop code from "signal_handler" function to the end of "main" function. There are two reasons for this:

First, this improves code maintenance and makes code look simple and clear. Based on this change, after receiving the interrupt signal, "fdata->done" is set as 1. Then the main thread will wait all worker lcores to jump out of the loop. Finally, the main thread will stop and then close eth dev port.

Quoted 5 lines

Second, for older version, the main thread first stops eth dev port and then waits the end of worker lcore. This may cause errors because it may stop the eth dev port which worker lcores are using. This moving change can fix this by waiting all worker threads to exit and then stop the eth dev port.

Apologies for the delayed reply,

In case of event dev the workers don't interact with eth device directly, Instead eth device "injects" packets into event device and event device is responsible for scheduling them to the workers.

Quoted 5 lines

If the producer is not stopped i.e. in this case eth device then the worker threads might never exit and the main core would wait indefinitely for workers to exit. This will be predominantly seen in cases where there are only a few flows and large number of workers causing a lot of intra thread dependency.

For the case that the event device scheduling packets to the workers. Though the producer is not stopped (eth device), when the main core receive the interrupt signal, the "fdata->done" will be set as 1. Then all the workers load the value 1 of "fdata->done" and jump out of the loop to finish their thread.

Best Regards
Feifei

Quoted 81 lines

Regards,
Pavan.

Quoted 38 lines

In the meanwhile, remove wmb in signal_handler.

This is because when the main lcore receive the stop signal, it stores 1 into fdata->done. And then the worker lcores load "fdata->done" and jump out of the loop to stop running. Nothing should be stored after updating
fdata->done, so the wmb is unnecessary.

Fixes 085edac2ca38 ("examples/eventdev_pipeline: support Tx

adapter")
Cc: [email protected]
Cc: [email protected]

Reviewed-by Honnappa Nagarahalli

<[email protected]>

---

v2:
1. Delete unused variable to fix build error 2. Reduce commit message length to fix coding style issues

v3:
1. Title improvement (Van Haaren)

examples/eventdev_pipeline/main.c | 16 ++++------------

1 file changed, 4 insertions(+), 12 deletions(-)
examples/eventdev_pipeline/main.c +0 -0
index 823f8b51c..fdbaf667b 100644

examples/eventdev_pipeline/main.c static void · line 285

Quoted 7 lines
examples/eventdev_pipeline/main.c +0 -1
@@ -280,7 +280,6 @@ static void
 signal_handler(int signum)
 {
 	static uint8_t once;
-	uint16_t portid;
 
 	if (fdata->done)
 		rte_exit(1, "Exiting on signal %d\n", signum); @@ -291,17

+290,6 @@

Quoted 31 lines

signal_handler(int signum)

			rte_event_dev_dump(0, stdout);
		once = 1;
		fdata->done = 1;
examples/eventdev_pipeline/main.c +0 -11
-		rte_smp_wmb();
-
-		RTE_ETH_FOREACH_DEV(portid) {
-			rte_event_eth_rx_adapter_stop(portid);
-			rte_event_eth_tx_adapter_stop(portid);
-			if (rte_eth_dev_stop(portid) < 0)
-				printf("Failed to stop port %u", portid);
-		}
-
-		rte_eal_mp_wait_lcore();
-
	}
	if (signum == SIGTSTP)
		rte_event_dev_dump(0, stdout);
examples/eventdev_pipeline/main.c +4 -1
@@ -465,6 +453,10 @@ main(int argc, char **argv)
 	}
 
 	RTE_ETH_FOREACH_DEV(portid) {
+		rte_event_eth_rx_adapter_stop(portid);
+		rte_event_eth_tx_adapter_stop(portid);
+		if (rte_eth_dev_stop(portid) < 0)
+			printf("Failed to stop port %u", portid);
 		rte_eth_dev_close(portid);
 	}
 
--

2.25.1

#7 lore raw

[dpdk-dev] 回复: [EXT] [PATCH v3] examples/eventdev: refactor ethdev port stop

Quoted 56 lines

-----邮件原件-----
发件人: Feifei Wang
发送时间: 2021年1月14日 14:24
收件人: Pavan Nikhilesh Bhagavatula <[email protected]>; Harry van Haaren <[email protected]>; Nikhil Rao <[email protected]>; Pavan Nikhilesh <[email protected]>
抄送: [email protected]; [email protected]; nd <[email protected]>; [email protected]; Ruifeng Wang <[email protected]>; Honnappa Nagarahalli <[email protected]>; nd <[email protected]> 主题: 回复: [EXT] [PATCH v3] examples/eventdev: refactor ethdev port stop

Quoted 11 lines

-----邮件原件-----
发件人: Pavan Nikhilesh Bhagavatula <[email protected]> 发送时间: 2021年1月5日 18:09
收件人: Feifei Wang <[email protected]>; Harry van Haaren <[email protected]>; Nikhil Rao <[email protected]>; Pavan Nikhilesh <[email protected]>
抄送: [email protected]; [email protected]; nd <[email protected]>; [email protected]; Ruifeng Wang <[email protected]>; Honnappa Nagarahalli <[email protected]>
主题: RE: [EXT] [PATCH v3] examples/eventdev: refactor ethdev port stop

Hi, Pavan

Quoted 23 lines

Hi Feifei,

Quoted 9 lines

Move eth stop code from "signal_handler" function to the end of "main" function. There are two reasons for this:

First, this improves code maintenance and makes code look simple and clear. Based on this change, after receiving the interrupt signal, "fdata->done" is set as 1. Then the main thread will wait all worker lcores to jump out of the loop. Finally, the main thread will stop and then close eth dev port.

Quoted 5 lines

Second, for older version, the main thread first stops eth dev port and then waits the end of worker lcore. This may cause errors because it may stop the eth dev port which worker lcores are using. This moving change can fix this by waiting all worker threads to exit and then stop the eth dev port.

Apologies for the delayed reply,

In case of event dev the workers don't interact with eth device directly, Instead eth device "injects" packets into event device and event device is responsible for scheduling them to the workers.

Quoted 5 lines

If the producer is not stopped i.e. in this case eth device then the worker threads might never exit and the main core would wait indefinitely for workers to exit. This will be predominantly seen in cases where there are only a few flows and large number of workers causing a lot of intra thread dependency.

Quoted 6 lines

For the case that the event device scheduling packets to the workers. Though the producer is not stopped (eth device), when the main core receive the interrupt signal, the "fdata->done"
will be set as 1. Then all the workers load the value 1 of "fdata->done" and jump out of the loop to finish their thread.

And I also test the case you said above with this patch, when I send interrupt signal(ctrl + c), IXIA is running and eth device is also working, then the program can exit normally.

HW:
Traffic generator: IXIA
Nics: ixgbe 82599ES 10-Gigabit
Architecture: aarch64
CPU: Cortex-A72

With this patch:
______________________________________________________________________________________________
Command: sudo ./dpdk-eventdev_pipeline -l 0-2,8-11 --vdev=event_sw0 -- -r2 -t2 -e4 -w F00 -s4 -n0 -c32 -W1000 -D
.........
Config:
ports: 2
workers: 4
packets: 9223372036854775807
Queue-prio: 0
qid0 type: atomic
Cores available: 7
Cores used: 6
Eventdev 0: event_sw
Stages:

        Stage 0, Type Atomic    Priority = 128
        Stage 1, Type Atomic    Priority = 128
        Stage 2, Type Atomic    Priority = 128
        Stage 3, Type Atomic    Priority = 128

Port 0 modified RSS hash function based on hardware support,requested:0x3afbc configured:0x38d34 Port 0 MAC: 90 e2 ba 56 ed 6c
Port 1 modified RSS hash function based on hardware support,requested:0x3afbc configured:0x38d34 Port 1 MAC: 90 e2 ba 56 ed 6d
[dump_core_info()] lcore 1 executing NIC Rx
[dump_core_info()] lcore 1 executing NIC Tx
[dump_core_info()] lcore 2 executing scheduler
[dump_core_info()] lcore 8 executing worker, using eventdev port 0 [dump_core_info()] lcore 9 executing worker, using eventdev port 1 [dump_core_info()] lcore 10 executing worker, using eventdev port 2 [dump_core_info()] lcore 11 executing worker, using eventdev port 3

Command: ctrl + c
.........
worker 1 thread done. RX=0 TX=0
worker 2 thread done. RX=0 TX=0
worker 11 thread done. RX=1844023 TX=1844023
worker 9 thread done. RX=1844888 TX=1844888
worker 8 thread done. RX=1845490 TX=1845490
worker 10 thread done. RX=1844903 TX=1844903

Port Workload distribution:

worker 0 :      25.0 % (1845474 pkts)
worker 1 :      25.0 % (1844872 pkts)
worker 2 :      25.0 % (1844887 pkts)
worker 3 :      25.0 % (1844007 pkts)

Invalid port_id=0
Invalid port_id=1
______________________________________________________________________________________________

Quoted 84 lines

Best Regards
Feifei

Quoted 82 lines

Regards,
Pavan.

Quoted 39 lines

In the meanwhile, remove wmb in signal_handler.

This is because when the main lcore receive the stop signal, it stores
1 into fdata->done. And then the worker lcores load "fdata->done" and jump out of the loop to stop running. Nothing should be stored after updating
fdata->done, so the wmb is unnecessary.

Fixes 085edac2ca38 ("examples/eventdev_pipeline: support Tx

adapter")
Cc: [email protected]
Cc: [email protected]

Reviewed-by Honnappa Nagarahalli

<[email protected]>

---

v2:
1. Delete unused variable to fix build error 2. Reduce commit message length to fix coding style issues

v3:
1. Title improvement (Van Haaren)

examples/eventdev_pipeline/main.c | 16 ++++------------

1 file changed, 4 insertions(+), 12 deletions(-)
examples/eventdev_pipeline/main.c +0 -0
index 823f8b51c..fdbaf667b 100644

examples/eventdev_pipeline/main.c static void · line 285

Quoted 7 lines
examples/eventdev_pipeline/main.c +0 -1
@@ -280,7 +280,6 @@ static void
 signal_handler(int signum)
 {
 	static uint8_t once;
-	uint16_t portid;
 
 	if (fdata->done)
 		rte_exit(1, "Exiting on signal %d\n", signum); @@ -291,17

+290,6 @@

Quoted 31 lines

signal_handler(int signum)

			rte_event_dev_dump(0, stdout);
		once = 1;
		fdata->done = 1;
examples/eventdev_pipeline/main.c +0 -11
-		rte_smp_wmb();
-
-		RTE_ETH_FOREACH_DEV(portid) {
-			rte_event_eth_rx_adapter_stop(portid);
-			rte_event_eth_tx_adapter_stop(portid);
-			if (rte_eth_dev_stop(portid) < 0)
-				printf("Failed to stop port %u", portid);
-		}
-
-		rte_eal_mp_wait_lcore();
-
	}
	if (signum == SIGTSTP)
		rte_event_dev_dump(0, stdout);
examples/eventdev_pipeline/main.c +4 -1
@@ -465,6 +453,10 @@ main(int argc, char **argv)
 	}
 
 	RTE_ETH_FOREACH_DEV(portid) {
+		rte_event_eth_rx_adapter_stop(portid);
+		rte_event_eth_tx_adapter_stop(portid);
+		if (rte_eth_dev_stop(portid) < 0)
+			printf("Failed to stop port %u", portid);
 		rte_eth_dev_close(portid);
 	}
 
--

2.25.1

#8 lore raw

v4 cover fix bugs for examples/evnentdev

In reply to Feifei Wang

Fix some bugs for eventdev pipeline example.

v2:
1. Delete unused variable to fix build error
2. Reduce commit message length to fix coding style issues

v3:
1. Title improvement (Van Haaren)

v4:
1. fix core enabled bug
2. add info typo for main core

Feifei Wang (3):
doc: fix core enabled bug for eventdev pipeline example examples/eventdev: add info typo for main core examples/eventdev: move eth stop to the end

 .../sample_app_ug/eventdev_pipeline.rst       |  5 +-
 examples/eventdev_pipeline/main.c             | 78 +++++++++++--------
 2 files changed, 47 insertions(+), 36 deletions(-)
Signature · 1 line
2.25.1
#9 lore raw

PATCH v4 1/3 doc: fix core enabled bug for eventdev pipeline example

In the case that the cores are isolated, if "-l" or "-c" parameter is not added, the cores will not be enabled and can not launch worker function correctly. In the meanwhile, no error information is reported.

For example:
totally CPUs:16
isolated CPUs:1-8

command: sudo gdb -args ./dpdk-eventdev_pipeline --vdev event_sw0 \
        -- -r1 -t1 -e4 -w F00 -s4 -n0 -c32 -W1000 -D

cores information:
rte_config->lcore_role = {ROLE_RTE, ROLE_OFF, ROLE_OFF, ROLE_OFF,
ROLE_OFF, ROLE_OFF, ROLE_OFF, ROLE_OFF, ROLE_OFF, ROLE_RTE, ROLE_RTE, ROLE_RTE, ROLE_RTE, ROLE_RTE, ROLE_RTE, ROLE_RTE}

output information:
...
[main()] lcore 9 executing worker, using eventdev port 0 [main()] lcore 10 executing worker, using eventdev port 1 [main()] lcore 11 executing worker, using eventdev port 2

This is because "RTE_LCORE_FOREACH_WORKER" chooses the enabled core. In the case that the cores are isolated, "the lcore_role" flag of isolated cores are set as "ROLE_OFF" by default(not enabled). So if we choose these isolated cores as workers, "RTE_LCORE_FOREACH_WORKER" will ignore these cores and not launch worker functions on them.

To fix this, add "-l" parameters to doc and add lcore enabled check.

Fixes 1094ca96689c ("doc: add SW eventdev pipeline to sample app guide")
---
 doc/guides/sample_app_ug/eventdev_pipeline.rst | 5 +++--
 examples/eventdev_pipeline/main.c              | 7 ++++++-
 2 files changed, 9 insertions(+), 3 deletions(-)
doc/guides/sample_app_ug/eventdev_pipeline.rst +3 -2
index 4508c3dcc..19ff53803 100644
@@ -34,6 +34,7 @@ options.
 An example eventdev pipeline running with the software eventdev PMD using
 these settings is shown below:
 
+ * ``-l 0,2,8-15``: lcore to use
  * ``-r1``: core mask 0x1 for RX
  * ``-t1``: core mask 0x1 for TX
  * ``-e4``: core mask 0x4 for the software scheduler
@@ -46,8 +47,8 @@ these settings is shown below:
 
 .. code-block:: console
 
-    ./<build_dir>/examples/dpdk-eventdev_pipeline --vdev event_sw0 -- -r1 -t1 \
-    -e4 -w FF00 -s4 -n0 -c32 -W1000 -D
+    ./<build_dir>/examples/dpdk-eventdev_pipeline -l 0,2,8-15 --vdev event_sw0 \
+    -- -r1 -t1 -e4 -w FF00 -s4 -n0 -c32 -W1000 -D
 
 The application has some sanity checking built-in, so if there is a function
 (e.g.; the RX core) which doesn't have a cpu core mask assigned, the application
examples/eventdev_pipeline/main.c +6 -1
index 823f8b51c..e9a591134 100644
@@ -239,8 +239,13 @@ parse_app_args(int argc, char **argv)
 
 		if (fdata->worker_core[i])
 			cdata.num_workers++;
-		if (core_in_use(i))
+		if (core_in_use(i)) {
+			if (!rte_lcore_is_enabled(i)) {
+				printf("error: lcore %d is not enabled in lcore list\n", i);
+				rte_exit(EXIT_FAILURE, "check lcore params failed\n");
+			}
 			cdata.active_cores++;
+		}
 	}
 }
 
Signature · 1 line
2.25.1
#10 lore raw

PATCH v4 2/3 examples/eventdev: add info output for main core

In reply to Feifei Wang

When the main core is set as tx/rx/sched/worker core, it also needs to print some information to show this. Thus, add info output for the main core, and add a "dump" function to print core information for the sake of code simplicity and easy maintenance.

In the meanwhile, fix the count error. For the variable "worker_idx", it should be incremented when the core is set as worker core. However, when the main core is set as rx/tx/sched core, the worker_idx is also incremented. Though this error may not have a substantial impact due to that the main core is the last launched core, but it should be corrected from the perspective of code correctness.

Fixes 1094ca96689c ("doc: add SW eventdev pipeline to sample app guide")
---
 examples/eventdev_pipeline/main.c | 55 +++++++++++++++++++------------
 1 file changed, 34 insertions(+), 21 deletions(-)
examples/eventdev_pipeline/main.c +34 -21
index e9a591134..3526d4d3d 100644
@@ -22,6 +22,32 @@ struct config_data cdata = {
 	.worker_cq_depth = 16
 };
 
+static void
+dump_core_info(unsigned int lcore_id, struct worker_data *data,
+		unsigned int worker_idx)
+{
+	if (fdata->rx_core[lcore_id])
+		printf(
+			"[%s()] lcore %d executing NIC Rx\n",
+			__func__, lcore_id);
+
+	if (fdata->tx_core[lcore_id])
+		printf(
+			"[%s()] lcore %d executing NIC Tx\n",
+			__func__, lcore_id);
+
+	if (fdata->sched_core[lcore_id])
+		printf(
+			"[%s()] lcore %d executing scheduler\n",
+			__func__, lcore_id);
+
+	if (fdata->worker_core[lcore_id])
+		printf(
+			"[%s()] lcore %d executing worker, using eventdev port %u\n",
+			__func__, lcore_id,
+			data[worker_idx].port_id);
+}
+
 static bool
 core_in_use(unsigned int lcore_id) {
 	return (fdata->rx_core[lcore_id] || fdata->sched_core[lcore_id] ||
@@ -411,25 +437,7 @@ main(int argc, char **argv)
 			!fdata->sched_core[lcore_id])
 			continue;
 
-		if (fdata->rx_core[lcore_id])
-			printf(
-				"[%s()] lcore %d executing NIC Rx\n",
-				__func__, lcore_id);
-
-		if (fdata->tx_core[lcore_id])
-			printf(
-				"[%s()] lcore %d executing NIC Tx\n",
-				__func__, lcore_id);
-
-		if (fdata->sched_core[lcore_id])
-			printf("[%s()] lcore %d executing scheduler\n",
-					__func__, lcore_id);
-
-		if (fdata->worker_core[lcore_id])
-			printf(
-				"[%s()] lcore %d executing worker, using eventdev port %u\n",
-				__func__, lcore_id,
-				worker_data[worker_idx].port_id);
+		dump_core_info(lcore_id, worker_data, worker_idx);
 
 		err = rte_eal_remote_launch(fdata->cap.worker,
 				&worker_data[worker_idx], lcore_id);
@@ -444,8 +452,13 @@ main(int argc, char **argv)
 
 	lcore_id = rte_lcore_id();
 
-	if (core_in_use(lcore_id))
-		fdata->cap.worker(&worker_data[worker_idx++]);
+	if (core_in_use(lcore_id)) {
+		dump_core_info(lcore_id, worker_data, worker_idx);
+		fdata->cap.worker(&worker_data[worker_idx]);
+
+		if (fdata->worker_core[lcore_id])
+			worker_idx++;
+	}
 
 	rte_eal_mp_wait_lcore();
 
Signature · 1 line
2.25.1
#11 lore raw

PATCH v4 3/3 examples/eventdev: move eth stop to the end

In reply to Feifei Wang

Move eth stop code from "signal_handler" function to the end of "main" function. There are two reasons for this:

First, this improves code maintenance and makes code look simple and clear. Based on this change, after receiving the interrupt signal, "fdata->done" is set as 1. Then the main thread will wait all worker lcores to jump out of the loop. Finally, the main thread will stop and then close eth dev port.

Second, for older version, the main thread first stops eth dev port and then waits the end of worker lcore. This may cause errors because it may stop the eth dev port which worker lcores are using. This moving change can fix this by waiting all worker threads to exit and then stop the eth dev port.

In the meanwhile, remove wmb in signal_handler.

This is because when the main lcore receive the stop signal, it stores 1 into fdata->done. And then the worker lcores load "fdata->done" and jump out of the loop to stop running. Nothing should be stored after updating fdata->done, so the wmb is unnecessary.

Fixes 085edac2ca38 ("examples/eventdev_pipeline: support Tx adapter")
---
 examples/eventdev_pipeline/main.c | 16 ++++------------
 1 file changed, 4 insertions(+), 12 deletions(-)
examples/eventdev_pipeline/main.c +4 -12
index 3526d4d3d..4621e8a89 100644
@@ -311,7 +311,6 @@ static void
 signal_handler(int signum)
 {
 	static uint8_t once;
-	uint16_t portid;
 
 	if (fdata->done)
 		rte_exit(1, "Exiting on signal %d\n", signum);
@@ -322,17 +321,6 @@ signal_handler(int signum)
 			rte_event_dev_dump(0, stdout);
 		once = 1;
 		fdata->done = 1;
-		rte_smp_wmb();
-
-		RTE_ETH_FOREACH_DEV(portid) {
-			rte_event_eth_rx_adapter_stop(portid);
-			rte_event_eth_tx_adapter_stop(portid);
-			if (rte_eth_dev_stop(portid) < 0)
-				printf("Failed to stop port %u", portid);
-		}
-
-		rte_eal_mp_wait_lcore();
-
 	}
 	if (signum == SIGTSTP)
 		rte_event_dev_dump(0, stdout);
@@ -483,6 +471,10 @@ main(int argc, char **argv)
 	}
 
 	RTE_ETH_FOREACH_DEV(portid) {
+		rte_event_eth_rx_adapter_stop(portid);
+		rte_event_eth_tx_adapter_stop(portid);
+		if (rte_eth_dev_stop(portid) < 0)
+			printf("Failed to stop port %u", portid);
 		rte_eth_dev_close(portid);
 	}
 
Signature · 1 line
2.25.1
Acked-by on v4 3/3
#12 lore raw
Quoted 32 lines

Move eth stop code from "signal_handler" function to the end of "main"
function. There are two reasons for this:

First, this improves code maintenance and makes code look simple and clear. Based on this change, after receiving the interrupt signal, "fdata->done" is set as 1. Then the main thread will wait all worker lcores to jump out of the loop. Finally, the main thread will stop and then close eth dev port.

Second, for older version, the main thread first stops eth dev port and then waits the end of worker lcore. This may cause errors because it may
stop the eth dev port which worker lcores are using. This moving change
can fix this by waiting all worker threads to exit and then stop the eth dev port.

In the meanwhile, remove wmb in signal_handler.

This is because when the main lcore receive the stop signal, it stores 1 into fdata->done. And then the worker lcores load "fdata->done" and jump
out of the loop to stop running. Nothing should be stored after updating
fdata->done, so the wmb is unnecessary.

Fixes 085edac2ca38 ("examples/eventdev_pipeline: support Tx

adapter")
Cc: [email protected]
Cc: [email protected]

Quoted 6 lines
Reviewed-by Honnappa Nagarahalli

<[email protected]>

Quoted 48 lines

---
examples/eventdev_pipeline/main.c | 16 ++++------------

1 file changed, 4 insertions(+), 12 deletions(-)
examples/eventdev_pipeline/main.c +4 -13
index 3526d4d3d..4621e8a89 100644
@@ -311,7 +311,6 @@ static void
 signal_handler(int signum)
 {
 	static uint8_t once;
-	uint16_t portid;
 
 	if (fdata->done)
 		rte_exit(1, "Exiting on signal %d\n", signum);
@@ -322,17 +321,6 @@ signal_handler(int signum)
 			rte_event_dev_dump(0, stdout);
 		once = 1;
 		fdata->done = 1;
-		rte_smp_wmb();
-
-		RTE_ETH_FOREACH_DEV(portid) {
-			rte_event_eth_rx_adapter_stop(portid);
-			rte_event_eth_tx_adapter_stop(portid);
-			if (rte_eth_dev_stop(portid) < 0)
-				printf("Failed to stop port %u", portid);
-		}
-
-		rte_eal_mp_wait_lcore();
-
 	}
 	if (signum == SIGTSTP)
 		rte_event_dev_dump(0, stdout);
@@ -483,6 +471,10 @@ main(int argc, char **argv)
 	}
 
 	RTE_ETH_FOREACH_DEV(portid) {
+		rte_event_eth_rx_adapter_stop(portid);
+		rte_event_eth_tx_adapter_stop(portid);
+		if (rte_eth_dev_stop(portid) < 0)
+			printf("Failed to stop port %u", portid);
 		rte_eth_dev_close(portid);
 	}
 
--

2.25.1

#13 lore raw
Quoted 33 lines · Pavan Nikhilesh Bhagavatula
Quoted 33 lines

Move eth stop code from "signal_handler" function to the end of "main"
function. There are two reasons for this:

First, this improves code maintenance and makes code look simple and clear. Based on this change, after receiving the interrupt signal, "fdata->done" is set as 1. Then the main thread will wait all worker lcores to jump out of the loop. Finally, the main thread will stop and then close eth dev port.

Second, for older version, the main thread first stops eth dev port and then waits the end of worker lcore. This may cause errors because it may
stop the eth dev port which worker lcores are using. This moving change
can fix this by waiting all worker threads to exit and then stop the eth dev port.

In the meanwhile, remove wmb in signal_handler.

This is because when the main lcore receive the stop signal, it stores 1 into fdata->done. And then the worker lcores load "fdata->done" and jump
out of the loop to stop running. Nothing should be stored after updating
fdata->done, so the wmb is unnecessary.

Fixes 085edac2ca38 ("examples/eventdev_pipeline: support Tx

adapter")
Cc: [email protected]
Cc: [email protected]

Quoted 8 lines
Quoted 6 lines
Reviewed-by Honnappa Nagarahalli

<[email protected]>

Series applied to dpdk-next-eventdev/for-main. Thanks.

Quoted 50 lines
Quoted 49 lines

---
examples/eventdev_pipeline/main.c | 16 ++++------------

1 file changed, 4 insertions(+), 12 deletions(-)
examples/eventdev_pipeline/main.c +4 -13
index 3526d4d3d..4621e8a89 100644
@@ -311,7 +311,6 @@ static void
 signal_handler(int signum)
 {
      static uint8_t once;
-      uint16_t portid;
 
      if (fdata->done)
              rte_exit(1, "Exiting on signal %d\n", signum);
@@ -322,17 +321,6 @@ signal_handler(int signum)
                      rte_event_dev_dump(0, stdout);
              once = 1;
              fdata->done = 1;
-              rte_smp_wmb();
-
-              RTE_ETH_FOREACH_DEV(portid) {
-                      rte_event_eth_rx_adapter_stop(portid);
-                      rte_event_eth_tx_adapter_stop(portid);
-                      if (rte_eth_dev_stop(portid) < 0)
-                              printf("Failed to stop port %u", portid);
-              }
-
-              rte_eal_mp_wait_lcore();
-
      }
      if (signum == SIGTSTP)
              rte_event_dev_dump(0, stdout);
@@ -483,6 +471,10 @@ main(int argc, char **argv)
      }
 
      RTE_ETH_FOREACH_DEV(portid) {
+              rte_event_eth_rx_adapter_stop(portid);
+              rte_event_eth_tx_adapter_stop(portid);
+              if (rte_eth_dev_stop(portid) < 0)
+                      printf("Failed to stop port %u", portid);
              rte_eth_dev_close(portid);
      }
 
--

2.25.1