Skip to contentSkip to conversation

app/testpmd: fix secondary process cannot dump packet

PATCH In review [email protected] 8 participants 23 messages Jun 23, 2022 – Oct 17, 2022

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 app/testpmd: fix secondary process cannot dump packet

From: Peng Zhang <[email protected]>

The origin design is whether testpmd is primary or not, if state of receive queue is stop, then packets will not be dumped for show. While to secondary process, receive queue will not be set up, and state will still be stop even if testpmd is started. So packets of stated secondary process cannot be dumped for show.

The current design is to secondary process state of queue will be set to start after testpmd is started. Then packets of started secondary process can be dumped for show.

Fixes a550baf24af9 ("app/testpmd: support multi-process")
---
 app/test-pmd/testpmd.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)
app/test-pmd/testpmd.c +12 -0
index 205d98ee3d..93ba7e7c9b 100644
@@ -3007,6 +3007,18 @@ start_port(portid_t pid)
 			if (setup_hairpin_queues(pi, p_pi, cnt_pi) != 0)
 				return -1;
 		}
+
+		if (port->need_reconfig_queues > 0 && !is_proc_primary()) {
+			struct rte_eth_rxconf *rx_conf;
+			for (qi = 0; qi < nb_rxq; qi++) {
+				rx_conf = &(port->rxq[qi].conf);
+				ports[pi].rxq[qi].state =
+					rx_conf->rx_deferred_start ?
+					RTE_ETH_QUEUE_STATE_STOPPED :
+					RTE_ETH_QUEUE_STATE_STARTED;
+			}
+		}
+
 		configure_rxtx_dump_callbacks(verbose_level);
 		if (clear_ptypes) {
 			diag = rte_eth_dev_set_ptypes(pi, RTE_PTYPE_UNKNOWN,
Signature · 1 line
2.25.1
#1 lore raw
Quoted 27 lines · On 6/23/22 21:15, [email protected] wrote:

From: Peng Zhang <[email protected]>

The origin design is whether testpmd is primary or not, if state of receive queue is stop, then packets will not be dumped for show. While to secondary process, receive queue will not be set up, and state will still be stop even if testpmd is started. So packets of stated secondary process cannot be dumped for show.

The current design is to secondary process state of queue will be set to start after testpmd is started. Then packets of started secondary process can be dumped for show.

Fixes a550baf24af9 ("app/testpmd: support multi-process")
---
  app/test-pmd/testpmd.c | 12 ++++++++++++
  1 file changed, 12 insertions(+)
app/test-pmd/testpmd.c +0 -0
index 205d98ee3d..93ba7e7c9b 100644
@@ -3007,6 +3007,18 @@ start_port(portid_t pid)
  			if (setup_hairpin_queues(pi, p_pi, cnt_pi) != 0)
  				return -1;

app/test-pmd/testpmd.c start_port(portid_t pid) · line 3018

Quoted 10 lines
app/test-pmd/testpmd.c +9 -0
  		}
+
+		if (port->need_reconfig_queues > 0 && !is_proc_primary()) {
+			struct rte_eth_rxconf *rx_conf;
+			for (qi = 0; qi < nb_rxq; qi++) {
+				rx_conf = &(port->rxq[qi].conf);
+				ports[pi].rxq[qi].state =
+					rx_conf->rx_deferred_start ?
+					RTE_ETH_QUEUE_STATE_STOPPED :
+					RTE_ETH_QUEUE_STATE_STARTED;

I'm not sure why it is correct to assume that deferred queue is not yet started.

Quoted 6 lines
app/test-pmd/testpmd.c +3 -0
+			}
+		}
+
  		configure_rxtx_dump_callbacks(verbose_level);
  		if (clear_ptypes) {
  			diag = rte_eth_dev_set_ptypes(pi, RTE_PTYPE_UNKNOWN,
#2 lore raw
Quoted 33 lines · Andrew Rybchenko
Quoted 32 lines · On 6/23/22 21:15, [email protected] wrote:

From: Peng Zhang <[email protected]>

The origin design is whether testpmd is primary or not, if state of receive queue is stop, then packets will not be dumped for show. While to secondary process, receive queue will not be set up, and state will still be stop even if testpmd is started. So packets of stated secondary process cannot be dumped for show.

The current design is to secondary process state of queue will be set to start after testpmd is started. Then packets of started secondary process can be dumped for show.

Fixes a550baf24af9 ("app/testpmd: support multi-process")

---
  app/test-pmd/testpmd.c | 12 ++++++++++++

  1 file changed, 12 insertions(+)
app/test-pmd/testpmd.c +4 -0
index 205d98ee3d..93ba7e7c9b 100644
@@ -3007,6 +3007,18 @@ start_port(portid_t pid)
               if (setup_hairpin_queues(pi, p_pi, cnt_pi) != 0)
                   return -1;
           }
+
+        if (port->need_reconfig_queues > 0 && !is_proc_primary()) {
+            struct rte_eth_rxconf *rx_conf;
+            for (qi = 0; qi < nb_rxq; qi++) {
Quoted 8 lines

app/test-pmd/testpmd.c start_port(portid_t pid) · line 3018

Quoted 5 lines
app/test-pmd/testpmd.c +5 -0
+                rx_conf = &(port->rxq[qi].conf);
+                ports[pi].rxq[qi].state =
+                    rx_conf->rx_deferred_start ?
+                    RTE_ETH_QUEUE_STATE_STOPPED :
+                    RTE_ETH_QUEUE_STATE_STARTED;

I'm not sure why it is correct to assume that deferred queue is not yet started.

+1.

We should also consider whether the queue state can be changed in secondary. The 'rx_conf->rx_deferred_start' is the data in secondary.
Why not use 'dev->data->rx_queue_state[]'.

In fact, the issue you memtioned was introduced the following patch:

Fixes 3c4426db54fc ("app/testpmd: do not poll stopped queues")

The root cause of this issue is that the default value of Rx/Tx queue state maintained by testpmd is 'RTE_ETH_QUEUE_STATE_STOPPED'. As a result, secondary doesn't start polling thread to receive packets when start packet forwarding. And now, secondary cannot receive and send any packets.

Could you fix it together?

Quoted 9 lines

app/test-pmd/testpmd.c start_port(portid_t pid) · line 3024

Quoted 7 lines
app/test-pmd/testpmd.c +3 -0
+            }
+        }
+
           configure_rxtx_dump_callbacks(verbose_level);
           if (clear_ptypes) {
               diag = rte_eth_dev_set_ptypes(pi, RTE_PTYPE_UNKNOWN,

.

#3 lore raw

Hi,
In fact, the patch is aim to fix this issue that secondary process cannot dump packet after start testpmd. This issue is induced by commit id is 3c4426db54fc ("app/testpmd: do not poll stopped queues"). After secondary process start, the default value of Rx/Tx queue state maintained by testpmd is 'RTE_ETH_QUEUE_STATE_STOPPED', the 'fsm[sm_id]->disabled' flag will set true according to queues state, then packet cannot forward and dump.

The reason why not use 'dev->data->rx_queue_state' is whether queue state is start or stop in primary process depend on rx_conf->rx_deferred_start after start testpmd. And after having started testpmd, queue state can be controlled by command for example 'port x rxq x start'.
Should we align with the same behavior of queues state for primary and secondary process after start testpmd?

Quoted 76 lines

-----Original Message-----
From: lihuisong (C) <[email protected]>
Sent: Wednesday, June 29, 2022 10:55 AM
To: Andrew Rybchenko <[email protected]>; Zhang, Peng1X <[email protected]>; [email protected]
Cc: Singh, Aman Deep <[email protected]>; Zhang, Yuying <[email protected]>; [email protected]
Subject: Re: [PATCH] app/testpmd: fix secondary process cannot dump packet

Quoted 33 lines · Andrew Rybchenko
Quoted 32 lines · On 6/23/22 21:15, [email protected] wrote:

From: Peng Zhang <[email protected]>

The origin design is whether testpmd is primary or not, if state of receive queue is stop, then packets will not be dumped for show. While to secondary process, receive queue will not be set up, and state will still be stop even if testpmd is started. So packets of stated secondary process cannot be dumped for show.

The current design is to secondary process state of queue will be set to start after testpmd is started. Then packets of started secondary process can be dumped for show.

Fixes a550baf24af9 ("app/testpmd: support multi-process")

---
  app/test-pmd/testpmd.c | 12 ++++++++++++

  1 file changed, 12 insertions(+)
app/test-pmd/testpmd.c index renamed from app/test-pmd/testpmd.c +0 -0

205d98ee3d..93ba7e7c9b 100644

app/test-pmd/testpmd.c +4 -0
@@ -3007,6 +3007,18 @@ start_port(portid_t pid)
               if (setup_hairpin_queues(pi, p_pi, cnt_pi) != 0)
                   return -1;
           }
+
+        if (port->need_reconfig_queues > 0 && !is_proc_primary()) {
+            struct rte_eth_rxconf *rx_conf;
+            for (qi = 0; qi < nb_rxq; qi++) {
Quoted 8 lines

app/test-pmd/testpmd.c start_port(portid_t pid) · line 3018

Quoted 5 lines
app/test-pmd/testpmd.c +5 -0
+                rx_conf = &(port->rxq[qi].conf);
+                ports[pi].rxq[qi].state =
+                    rx_conf->rx_deferred_start ?
+                    RTE_ETH_QUEUE_STATE_STOPPED :
+                    RTE_ETH_QUEUE_STATE_STARTED;

I'm not sure why it is correct to assume that deferred queue is not yet started.

+1.

We should also consider whether the queue state can be changed in secondary. The 'rx_conf->rx_deferred_start' is the data in secondary.
Why not use 'dev->data->rx_queue_state[]'.

In fact, the issue you memtioned was introduced the following patch:

Fixes 3c4426db54fc ("app/testpmd: do not poll stopped queues")

The root cause of this issue is that the default value of Rx/Tx queue state maintained by testpmd is 'RTE_ETH_QUEUE_STATE_STOPPED'. As a result, secondary doesn't start polling thread to receive packets when start packet forwarding. And now, secondary cannot receive and send any packets.

Could you fix it together?

Quoted 9 lines

app/test-pmd/testpmd.c start_port(portid_t pid) · line 3024

Quoted 7 lines
app/test-pmd/testpmd.c +3 -0
+            }
+        }
+
           configure_rxtx_dump_callbacks(verbose_level);
           if (clear_ptypes) {
               diag = rte_eth_dev_set_ptypes(pi, RTE_PTYPE_UNKNOWN,

.

#4 lore raw

Hi Peng1X,

Quoted 6 lines · 在 2022/7/1 19:36, Zhang, Peng1X 写道:

Hi,
In fact, the patch is aim to fix this issue that secondary process cannot dump packet after start testpmd. This issue is induced by commit id is 3c4426db54fc ("app/testpmd: do not poll stopped queues"). After secondary process start, the default value of Rx/Tx queue state maintained by testpmd is 'RTE_ETH_QUEUE_STATE_STOPPED', the 'fsm[sm_id]->disabled' flag will set true according to queues state, then packet cannot forward and dump.

I get your meaning.
However, failing to dump packet isn't the first exception, and the first one is that testpmd doesn't call 'struct fwd_engine::packet_fwd()' to receive or send packet. So, I think you should describe and resolve this problem from this point. This patch cannot completely resolve this problem. The Tx queue state should also be added here.

Quoted 5 lines

The reason why not use 'dev->data->rx_queue_state' is whether queue state is start or stop in primary process depend on rx_conf->rx_deferred_start after start testpmd. And after having started testpmd, queue state can be controlled by command for example 'port x rxq x start'.
Should we align with the same behavior of queues state for primary and secondary process after start testpmd?

If primary process stops a queue, but secondary doesn't know. we have to simplify this queue state problem like you momentioned if we don't have a good idea.

@Andrew, what do you think?

Thanks,

Huisong

Quoted 74 lines
Quoted 74 lines

-----Original Message-----
From: lihuisong (C) <[email protected]>
Sent: Wednesday, June 29, 2022 10:55 AM
To: Andrew Rybchenko <[email protected]>; Zhang, Peng1X <[email protected]>; [email protected]
Cc: Singh, Aman Deep <[email protected]>; Zhang, Yuying <[email protected]>; [email protected]
Subject: Re: [PATCH] app/testpmd: fix secondary process cannot dump packet

Quoted 32 lines · Andrew Rybchenko
Quoted 31 lines · On 6/23/22 21:15, [email protected] wrote:

From: Peng Zhang <[email protected]>

The origin design is whether testpmd is primary or not, if state of receive queue is stop, then packets will not be dumped for show. While to secondary process, receive queue will not be set up, and state will still be stop even if testpmd is started. So packets of stated secondary process cannot be dumped for show.

The current design is to secondary process state of queue will be set to start after testpmd is started. Then packets of started secondary process can be dumped for show.

Fixes a550baf24af9 ("app/testpmd: support multi-process")
---
   app/test-pmd/testpmd.c | 12 ++++++++++++
   1 file changed, 12 insertions(+)
app/test-pmd/testpmd.c index renamed from app/test-pmd/testpmd.c +0 -0

205d98ee3d..93ba7e7c9b 100644

app/test-pmd/testpmd.c +3 -0
@@ -3007,6 +3007,18 @@ start_port(portid_t pid)
               if (setup_hairpin_queues(pi, p_pi, cnt_pi) != 0)
                   return -1;
           }
+
+        if (port->need_reconfig_queues > 0 && !is_proc_primary()) {
+            struct rte_eth_rxconf *rx_conf;
Quoted 8 lines

app/test-pmd/testpmd.c start_port(portid_t pid) · line 3018

Quoted 6 lines
app/test-pmd/testpmd.c +6 -0
+            for (qi = 0; qi < nb_rxq; qi++) {
+                rx_conf = &(port->rxq[qi].conf);
+                ports[pi].rxq[qi].state =
+                    rx_conf->rx_deferred_start ?
+                    RTE_ETH_QUEUE_STATE_STOPPED :
+                    RTE_ETH_QUEUE_STATE_STARTED;

I'm not sure why it is correct to assume that deferred queue is not yet started.

+1.

We should also consider whether the queue state can be changed in secondary. The 'rx_conf->rx_deferred_start' is the data in secondary.
Why not use 'dev->data->rx_queue_state[]'.

In fact, the issue you memtioned was introduced the following patch:

Fixes 3c4426db54fc ("app/testpmd: do not poll stopped queues")

The root cause of this issue is that the default value of Rx/Tx queue state maintained by testpmd is 'RTE_ETH_QUEUE_STATE_STOPPED'. As a result, secondary doesn't start polling thread to receive packets when start packet forwarding. And now, secondary cannot receive and send any packets.

Could you fix it together?

Quoted 7 lines

app/test-pmd/testpmd.c start_port(portid_t pid) · line 3024

Quoted 6 lines
app/test-pmd/testpmd.c +3 -0
+            }
+        }
+
           configure_rxtx_dump_callbacks(verbose_level);
           if (clear_ptypes) {
               diag = rte_eth_dev_set_ptypes(pi, RTE_PTYPE_UNKNOWN,

.

#5 lore raw
Quoted 6 lines · lihuisong (C)

[...]
在 2022/7/1 19:36, Zhang, Peng1X 写道:
[...]

Quoted 1 line

The reason why not use 'dev->data->rx_queue_state' is whether queue

state is start or stop in primary

Quoted 1 line

process depend on rx_conf->rx_deferred_start after start testpmd. And

Quoted 9 lines

after having started testpmd,

Quoted 1 line

queue state can be controlled by command for example 'port x rxq x

start'.

Quoted 1 line

Should we align with the same behavior of queues state for primary and

secondary process after start testpmd?
If primary process stops a queue, but secondary doesn't know. we have to simplify this queue state problem like you momentioned if we don't have a good idea.

AFAIU, dev->data->rx_queue_state should be aligned with rx_conf->rx_deferred_start.
The reason why testpmd manages the queue state itself in the offending patch is that not all PMDs implement rte_eth_rx/tx_queue_info_get() and ethdev won't return dev->data->rx_queue_state in this case.

#6 lore raw
In reply to lihuisong (C)

Hi Song,
Currently this patch is just fix the issue detected for rx queue on secondary process. Later patch for tx queue will be submit.

@Andrew, what's your opinion about the solution of this patch?

Thanks,
Peng

Quoted 123 lines

-----Original Message-----
From: lihuisong (C) <[email protected]>
Sent: Monday, July 4, 2022 10:37 AM
To: Zhang, Peng1X <[email protected]>; Andrew Rybchenko <[email protected]>; [email protected]
Cc: Singh, Aman Deep <[email protected]>; Zhang, Yuying <[email protected]>; [email protected]
Subject: Re: [PATCH] app/testpmd: fix secondary process cannot dump packet

Hi Peng1X,

Quoted 2 lines · 在 2022/7/1 19:36, Zhang, Peng1X 写道:

Hi,
In fact, the patch is aim to fix this issue that secondary process cannot dump

packet after start testpmd.

Quoted 4 lines

This issue is induced by commit id is 3c4426db54fc ("app/testpmd: do not poll stopped queues"). After secondary process start, the default value of Rx/Tx queue state maintained by testpmd is 'RTE_ETH_QUEUE_STATE_STOPPED', the 'fsm[sm_id]->disabled' flag will set

true according to queues state, then packet cannot forward and dump. I get your meaning.
However, failing to dump packet isn't the first exception, and the first one is that testpmd doesn't call 'struct fwd_engine::packet_fwd()' to receive or send packet.
So, I think you should describe and resolve this problem from this point. This patch cannot completely resolve this problem. The Tx queue state should also be added here.

Quoted 4 lines

The reason why not use 'dev->data->rx_queue_state' is whether queue state is start or stop in primary process depend on rx_conf->rx_deferred_start after start testpmd. And after having started

testpmd, queue state can be controlled by command for example 'port x rxq x start'.

Quoted 1 line

Should we align with the same behavior of queues state for primary and

secondary process after start testpmd?
If primary process stops a queue, but secondary doesn't know.
we have to simplify this queue state problem like you momentioned if we don't have a good idea.

@Andrew, what do you think?

Thanks,

Huisong

Quoted 54 lines
Quoted 54 lines

-----Original Message-----
From: lihuisong (C) <[email protected]>
Sent: Wednesday, June 29, 2022 10:55 AM
To: Andrew Rybchenko <[email protected]>; Zhang, Peng1X <[email protected]>; [email protected]
Cc: Singh, Aman Deep <[email protected]>; Zhang, Yuying <[email protected]>; [email protected]
Subject: Re: [PATCH] app/testpmd: fix secondary process cannot dump packet

Quoted 41 lines · Andrew Rybchenko
Quoted 28 lines · On 6/23/22 21:15, [email protected] wrote:

From: Peng Zhang <[email protected]>

The origin design is whether testpmd is primary or not, if state of receive queue is stop, then packets will not be dumped for show. While to secondary process, receive queue will not be set up, and state will still be stop even if testpmd is started. So packets of stated secondary process cannot be dumped for show.

The current design is to secondary process state of queue will be set to start after testpmd is started. Then packets of started secondary process can be dumped for show.

Fixes a550baf24af9 ("app/testpmd: support multi-process")
---
   app/test-pmd/testpmd.c | 12 ++++++++++++
   1 file changed, 12 insertions(+)
app/test-pmd/testpmd.c index renamed from app/test-pmd/testpmd.c +0 -0

205d98ee3d..93ba7e7c9b 100644

app/test-pmd/testpmd.c +0 -0
@@ -3007,6 +3007,18 @@ start_port(portid_t pid)
               if (setup_hairpin_queues(pi, p_pi, cnt_pi) != 0)
                   return -1;
           }

app/test-pmd/testpmd.c start_port(portid_t pid) · line 3019

Quoted 10 lines
app/test-pmd/testpmd.c +10 -0
+
+        if (port->need_reconfig_queues > 0 && !is_proc_primary())
+{
+            struct rte_eth_rxconf *rx_conf;
+            for (qi = 0; qi < nb_rxq; qi++) {
+                rx_conf = &(port->rxq[qi].conf);
+                ports[pi].rxq[qi].state =
+                    rx_conf->rx_deferred_start ?
+                    RTE_ETH_QUEUE_STATE_STOPPED :
+                    RTE_ETH_QUEUE_STATE_STARTED;

I'm not sure why it is correct to assume that deferred queue is not yet started.

Quoted 3 lines
Quoted 3 lines

+1.

We should also consider whether the queue state can be changed in

secondary.

Quoted 10 lines
Quoted 10 lines

The 'rx_conf->rx_deferred_start' is the data in secondary. Why not use 'dev->data->rx_queue_state[]'.

In fact, the issue you memtioned was introduced the following patch:

Fixes 3c4426db54fc ("app/testpmd: do not poll stopped queues")

The root cause of this issue is that the default value of Rx/Tx queue state maintained by testpmd is 'RTE_ETH_QUEUE_STATE_STOPPED'. As a result, secondary doesn't start polling thread to receive packets when start packet forwarding. And now, secondary cannot receive and send

any packets.

Quoted 9 lines
Quoted 9 lines

Could you fix it together?

Quoted 7 lines

app/test-pmd/testpmd.c start_port(portid_t pid) · line 3024

Quoted 6 lines
app/test-pmd/testpmd.c +3 -0
+            }
+        }
+
           configure_rxtx_dump_callbacks(verbose_level);
           if (clear_ptypes) {
               diag = rte_eth_dev_set_ptypes(pi, RTE_PTYPE_UNKNOWN,

.

#7 lore raw
Quoted 3 lines · 在 2022/7/5 18:12, Zhang, Peng1X 写道:

Hi Song,
Currently this patch is just fix the issue detected for rx queue on secondary process. Later patch for tx queue will be submit.

Firstly, the Rx packet dump failure is not displayed in your commit log.

In addition, even if this patch is aimed to fix the Rx issue on secondary process, it doesn't work well when if testpmd run in 'mac' or 'swap' forwarding mode, right? Because it is also affected by the Tx queue state. Therefore, it is better to place the modification of Rx and Tx queue state in the same patch to fix this issue.

@Peng1X, @Andrew and @Thomas
Currently, the secondary process cannot send and receive packets. It is suggested that this problem should be solved quickly. After all, 22.07 will be released soon.

Quoted 125 lines

@Andrew, what's your opinion about the solution of this patch?

Thanks,
Peng

Quoted 120 lines

-----Original Message-----
From: lihuisong (C) <[email protected]>
Sent: Monday, July 4, 2022 10:37 AM
To: Zhang, Peng1X <[email protected]>; Andrew Rybchenko <[email protected]>; [email protected]
Cc: Singh, Aman Deep <[email protected]>; Zhang, Yuying <[email protected]>; [email protected]
Subject: Re: [PATCH] app/testpmd: fix secondary process cannot dump packet

Hi Peng1X,

Quoted 2 lines · 在 2022/7/1 19:36, Zhang, Peng1X 写道:

Hi,
In fact, the patch is aim to fix this issue that secondary process cannot dump

packet after start testpmd.

Quoted 4 lines

This issue is induced by commit id is 3c4426db54fc ("app/testpmd: do not poll stopped queues"). After secondary process start, the default value of Rx/Tx queue state maintained by testpmd is 'RTE_ETH_QUEUE_STATE_STOPPED', the 'fsm[sm_id]->disabled' flag will set

true according to queues state, then packet cannot forward and dump. I get your meaning.
However, failing to dump packet isn't the first exception, and the first one is that testpmd doesn't call 'struct fwd_engine::packet_fwd()' to receive or send packet.
So, I think you should describe and resolve this problem from this point. This patch cannot completely resolve this problem. The Tx queue state should also be added here.

Quoted 3 lines

The reason why not use 'dev->data->rx_queue_state' is whether queue state is start or stop in primary process depend on rx_conf->rx_deferred_start after start testpmd. And after having started

testpmd, queue state can be controlled by command for example 'port x rxq x start'.

Quoted 1 line

Should we align with the same behavior of queues state for primary and

secondary process after start testpmd?
If primary process stops a queue, but secondary doesn't know.
we have to simplify this queue state problem like you momentioned if we don't have a good idea.

@Andrew, what do you think?

Thanks,

Huisong

Quoted 53 lines
Quoted 53 lines

-----Original Message-----
From: lihuisong (C) <[email protected]>
Sent: Wednesday, June 29, 2022 10:55 AM
To: Andrew Rybchenko <[email protected]>; Zhang, Peng1X <[email protected]>; [email protected]
Cc: Singh, Aman Deep <[email protected]>; Zhang, Yuying <[email protected]>; [email protected]
Subject: Re: [PATCH] app/testpmd: fix secondary process cannot dump packet

Quoted 41 lines · Andrew Rybchenko
Quoted 28 lines · On 6/23/22 21:15, [email protected] wrote:

From: Peng Zhang <[email protected]>

The origin design is whether testpmd is primary or not, if state of receive queue is stop, then packets will not be dumped for show. While to secondary process, receive queue will not be set up, and state will still be stop even if testpmd is started. So packets of stated secondary process cannot be dumped for show.

The current design is to secondary process state of queue will be set to start after testpmd is started. Then packets of started secondary process can be dumped for show.

Fixes a550baf24af9 ("app/testpmd: support multi-process")
---
    app/test-pmd/testpmd.c | 12 ++++++++++++
    1 file changed, 12 insertions(+)
app/test-pmd/testpmd.c index renamed from app/test-pmd/testpmd.c +0 -0

205d98ee3d..93ba7e7c9b 100644

app/test-pmd/testpmd.c +0 -0
@@ -3007,6 +3007,18 @@ start_port(portid_t pid)
                if (setup_hairpin_queues(pi, p_pi, cnt_pi) != 0)
                    return -1;
            }

app/test-pmd/testpmd.c start_port(portid_t pid) · line 3019

Quoted 10 lines
app/test-pmd/testpmd.c +10 -0
+
+        if (port->need_reconfig_queues > 0 && !is_proc_primary())
+{
+            struct rte_eth_rxconf *rx_conf;
+            for (qi = 0; qi < nb_rxq; qi++) {
+                rx_conf = &(port->rxq[qi].conf);
+                ports[pi].rxq[qi].state =
+                    rx_conf->rx_deferred_start ?
+                    RTE_ETH_QUEUE_STATE_STOPPED :
+                    RTE_ETH_QUEUE_STATE_STARTED;

I'm not sure why it is correct to assume that deferred queue is not yet started.

Quoted 3 lines
Quoted 3 lines

+1.

We should also consider whether the queue state can be changed in

secondary.

Quoted 10 lines
Quoted 10 lines

The 'rx_conf->rx_deferred_start' is the data in secondary. Why not use 'dev->data->rx_queue_state[]'.

In fact, the issue you memtioned was introduced the following patch:

Fixes 3c4426db54fc ("app/testpmd: do not poll stopped queues")

The root cause of this issue is that the default value of Rx/Tx queue state maintained by testpmd is 'RTE_ETH_QUEUE_STATE_STOPPED'. As a result, secondary doesn't start polling thread to receive packets when start packet forwarding. And now, secondary cannot receive and send

any packets.

Quoted 8 lines
Quoted 8 lines

Could you fix it together?

Quoted 7 lines

app/test-pmd/testpmd.c start_port(portid_t pid) · line 3024

Quoted 6 lines
app/test-pmd/testpmd.c +3 -0
+            }
+        }
+
            configure_rxtx_dump_callbacks(verbose_level);
            if (clear_ptypes) {
                diag = rte_eth_dev_set_ptypes(pi, RTE_PTYPE_UNKNOWN,

.

#8 lore raw
Quoted 11 lines · lihuisong (C)
Quoted 3 lines · 在 2022/7/5 18:12, Zhang, Peng1X 写道:

Hi Song,
Currently this patch is just fix the issue detected for rx queue on secondary process. Later patch for tx queue will be submit.

Firstly, the Rx packet dump failure is not displayed in your commit log.

In addition, even if this patch is aimed to fix the Rx issue on secondary process, it doesn't work well when if testpmd run in 'mac' or 'swap' forwarding mode, right? Because it is also affected by the Tx queue state. Therefore, it is better to place the modification of Rx and Tx queue state in the same patch to fix this issue.

If the problem is generic, it is better to find generic solution rather than concentrate on one aspect of the problem.

Quoted 6 lines

@Peng1X, @Andrew and @Thomas
Currently, the secondary process cannot send and receive packets. It is suggested that this problem should be solved quickly. After all, 22.07 will be released soon.

Quoted 1 line

@Andrew, what's your opinion about the solution of this patch?

IMHO the patch is simply incorrect since it uses rx_deferred_start incorrectly. The attribute does not say if an Rx queue is started or stopped right now.

Quoted 36 lines
Quoted 36 lines

Thanks,
Peng

Quoted 32 lines

-----Original Message-----
From: lihuisong (C) <[email protected]>
Sent: Monday, July 4, 2022 10:37 AM
To: Zhang, Peng1X <[email protected]>; Andrew Rybchenko <[email protected]>; [email protected]
Cc: Singh, Aman Deep <[email protected]>; Zhang, Yuying <[email protected]>; [email protected]
Subject: Re: [PATCH] app/testpmd: fix secondary process cannot dump packet

Hi Peng1X,

Quoted 2 lines · 在 2022/7/1 19:36, Zhang, Peng1X 写道:

Hi,
In fact, the patch is aim to fix this issue that secondary process cannot dump

packet after start testpmd.

Quoted 4 lines

This issue is induced by commit id is 3c4426db54fc ("app/testpmd: do not poll stopped queues"). After secondary process start, the default value of Rx/Tx queue state maintained by testpmd is 'RTE_ETH_QUEUE_STATE_STOPPED', the 'fsm[sm_id]->disabled' flag will set

true according to queues state, then packet cannot forward and dump. I get your meaning.
However, failing to dump packet isn't the first exception, and the first one is that testpmd doesn't call 'struct fwd_engine::packet_fwd()' to receive or send packet.
So, I think you should describe and resolve this problem from this point. This patch cannot completely resolve this problem. The Tx queue state should also be added here.

Quoted 3 lines

The reason why not use 'dev->data->rx_queue_state' is whether queue state is start or stop in primary process depend on rx_conf->rx_deferred_start after start testpmd. And after having started

testpmd, queue state can be controlled by command for example 'port x rxq x start'.

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 1 line

Should we align with the same behavior of queues state for primary and

secondary process after start testpmd?
If primary process stops a queue, but secondary doesn't know.
we have to simplify this queue state problem like you momentioned if we don't have a good idea.

@Andrew, what do you think?

I'm sorry to say, but multi-process support in testpmd is not well- thought. So, it is hard to suggest a good solution. I think that trying to mirror state in secondary testpmd process is a bad idea. It could become out-of-sync by definition. So, I'd either share testpmd state or use state from ethdev (which is located in shared memory). However, it sounds like a long story and non-trivial changes which are too late at the current release cycle state. See below.

Quoted 64 lines
Quoted 64 lines
Quoted 64 lines

Thanks,

Huisong

Quoted 59 lines
Quoted 59 lines

-----Original Message-----
From: lihuisong (C) <[email protected]>
Sent: Wednesday, June 29, 2022 10:55 AM
To: Andrew Rybchenko <[email protected]>; Zhang, Peng1X <[email protected]>; [email protected]
Cc: Singh, Aman Deep <[email protected]>; Zhang, Yuying <[email protected]>; [email protected]
Subject: Re: [PATCH] app/testpmd: fix secondary process cannot dump packet

Quoted 33 lines · Andrew Rybchenko
Quoted 32 lines · On 6/23/22 21:15, [email protected] wrote:

From: Peng Zhang <[email protected]>

The origin design is whether testpmd is primary or not, if state of receive queue is stop, then packets will not be dumped for show. While to secondary process, receive queue will not be set up, and state will still be stop even if testpmd is started. So packets of stated secondary process cannot be dumped for show.

The current design is to secondary process state of queue will be set to start after testpmd is started. Then packets of started secondary process can be dumped for show.

Fixes a550baf24af9 ("app/testpmd: support multi-process")

---
    app/test-pmd/testpmd.c | 12 ++++++++++++

    1 file changed, 12 insertions(+)
app/test-pmd/testpmd.c index renamed from app/test-pmd/testpmd.c +0 -0

205d98ee3d..93ba7e7c9b 100644

app/test-pmd/testpmd.c +4 -0
@@ -3007,6 +3007,18 @@ start_port(portid_t pid)
                 if (setup_hairpin_queues(pi, p_pi, cnt_pi) != 0)
                     return -1;
             }
+
+        if (port->need_reconfig_queues > 0 && !is_proc_primary())
+{
+            struct rte_eth_rxconf *rx_conf;
Quoted 8 lines

app/test-pmd/testpmd.c start_port(portid_t pid) · line 3019

Quoted 6 lines
app/test-pmd/testpmd.c +6 -0
+            for (qi = 0; qi < nb_rxq; qi++) {
+                rx_conf = &(port->rxq[qi].conf);
+                ports[pi].rxq[qi].state =
+                    rx_conf->rx_deferred_start ?
+                    RTE_ETH_QUEUE_STATE_STOPPED :
+                    RTE_ETH_QUEUE_STATE_STARTED;

I'm not sure why it is correct to assume that deferred queue is not yet started.

+1.

We should also consider whether the queue state can be changed in secondary. The 'rx_conf->rx_deferred_start' is the data in secondary.
Why not use 'dev->data->rx_queue_state[]'.

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

In fact, the issue you memtioned was introduced the following patch:

Fixes 3c4426db54fc ("app/testpmd: do not poll stopped queues")

The root cause of this issue is that the default value of Rx/Tx queue state maintained by testpmd is 'RTE_ETH_QUEUE_STATE_STOPPED'. As a result, secondary doesn't start polling thread to receive packets
when start packet forwarding. And now, secondary cannot receive and send any packets. Could you fix it together?

As a simple fix which should work in assumption that everything is started before secondary process start-up, I'd just set state to RTE_ETH_QUEUE_STATE_STARTED.

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 6 lines
app/test-pmd/testpmd.c +3 -0
+            }
+        }
+
             configure_rxtx_dump_callbacks(verbose_level);
             if (clear_ptypes) {

                diag = rte_eth_dev_set_ptypes(pi, RTE_PTYPE_UNKNOWN,

.

#9 lore raw
In reply to [email protected]

Hi Peng,

Quoted 9 lines

-----Original Message-----
From: Zhang, Peng1X <[email protected]>
Sent: Friday, June 24, 2022 2:15 AM
To: [email protected]
Cc: Singh, Aman Deep <[email protected]>; Zhang, Yuying <[email protected]>; Zhang, Peng1X <[email protected]>; [email protected]
Subject: [PATCH] app/testpmd: fix secondary process cannot dump packet

Quoted 7 lines

From: Peng Zhang <[email protected]>

The origin design is whether testpmd is primary or not, if state of receive queue is stop, then packets will not be dumped for show.
While to secondary process, receive queue will not be set up, and state will still be stop even if testpmd is started. So packets of stated secondary process cannot be dumped for show.

Current description is confusing. Please refine the commit log to define the issue clearly.

Quoted 39 lines

The current design is to secondary process state of queue will be set to start after testpmd is started. Then packets of started secondary process can be dumped for show.

Fixes a550baf24af9 ("app/testpmd: support multi-process")
---
 app/test-pmd/testpmd.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)
app/test-pmd/testpmd.c index renamed from app/test-pmd/testpmd.c +0 -0

205d98ee3d..93ba7e7c9b 100644

app/test-pmd/testpmd.c +12 -0
@@ -3007,6 +3007,18 @@ start_port(portid_t pid)
 			if (setup_hairpin_queues(pi, p_pi, cnt_pi) != 0)
 				return -1;
 		}
+
+		if (port->need_reconfig_queues > 0 && !is_proc_primary()) {
+			struct rte_eth_rxconf *rx_conf;
+			for (qi = 0; qi < nb_rxq; qi++) {
+				rx_conf = &(port->rxq[qi].conf);
+				ports[pi].rxq[qi].state =
+					rx_conf->rx_deferred_start ?
+					RTE_ETH_QUEUE_STATE_STOPPED :
+					RTE_ETH_QUEUE_STATE_STARTED;
+			}
+		}
+
 		configure_rxtx_dump_callbacks(verbose_level);
 		if (clear_ptypes) {
 			diag = rte_eth_dev_set_ptypes(pi,

RTE_PTYPE_UNKNOWN,
--
2.25.1

Acked-by on v1 1/1
#10 lore raw
In reply to [email protected]

Hi,

Quoted 17 lines

-----Original Message-----
From: Zhang, Peng1X <[email protected]>
Sent: Friday, June 24, 2022 2:15 AM
To: [email protected]
Cc: Singh, Aman Deep <[email protected]>; Zhang, Yuying <[email protected]>; Zhang, Peng1X <[email protected]>; [email protected]
Subject: [PATCH] app/testpmd: fix secondary process cannot dump packet

From: Peng Zhang <[email protected]>

The origin design is whether testpmd is primary or not, if state of receive queue is stop, then packets will not be dumped for show.
While to secondary process, receive queue will not be set up, and state will still be stop even if testpmd is started. So packets of stated secondary process cannot be dumped for show.

Quoted 8 lines

The current design is to secondary process state of queue will be set to start after testpmd is started. Then packets of started secondary process can be dumped for show.

Fixes a550baf24af9 ("app/testpmd: support multi-process")
Quoted 30 lines
---
 app/test-pmd/testpmd.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)
app/test-pmd/testpmd.c index renamed from app/test-pmd/testpmd.c +0 -0

205d98ee3d..93ba7e7c9b 100644

app/test-pmd/testpmd.c +12 -0
@@ -3007,6 +3007,18 @@ start_port(portid_t pid)
 			if (setup_hairpin_queues(pi, p_pi, cnt_pi) != 0)
 				return -1;
 		}
+
+		if (port->need_reconfig_queues > 0 && !is_proc_primary()) {
+			struct rte_eth_rxconf *rx_conf;
+			for (qi = 0; qi < nb_rxq; qi++) {
+				rx_conf = &(port->rxq[qi].conf);
+				ports[pi].rxq[qi].state =
+					rx_conf->rx_deferred_start ?
+					RTE_ETH_QUEUE_STATE_STOPPED :
+					RTE_ETH_QUEUE_STATE_STARTED;
+			}
+		}
+
 		configure_rxtx_dump_callbacks(verbose_level);
 		if (clear_ptypes) {
 			diag = rte_eth_dev_set_ptypes(pi,

RTE_PTYPE_UNKNOWN,
--
2.25.1

#11 lore raw

PATCH v2 1/1 app/testpmd: fix incorrect queues state of secondary process

In reply to [email protected]

From: Peng Zhang <[email protected]>

Primary process could set up queues state correctly when starting port, but under multi-process scenario, "stream_init" function would get wrong queues state for secondary process.

This commit is to get queues state from ethdev which is located in shared memory.

Fixes 3c4426db54fc ("app/testpmd: do not poll stopped queues")
---
 app/test-pmd/testpmd.c | 20 ++++++++++++++++++--
 1 file changed, 18 insertions(+), 2 deletions(-)
app/test-pmd/testpmd.c +18 -2
index addcbcac85..70f907d96b 100644
@@ -75,6 +75,8 @@
 
 #include "testpmd.h"
 
+#include <ethdev_driver.h>
+
 #ifndef MAP_HUGETLB
 /* FreeBSD may not have MAP_HUGETLB (in fact, it probably doesn't) */
 #define HUGE_FLAG (0x40000)
@@ -2402,9 +2404,23 @@ start_packet_forwarding(int with_tx_first)
 	if (!pkt_fwd_shared_rxq_check())
 		return;
 
-	if (stream_init != NULL)
-		for (i = 0; i < cur_fwd_config.nb_fwd_streams; i++)
+	if (stream_init != NULL) {
+		for (i = 0; i < cur_fwd_config.nb_fwd_streams; i++) {
+			if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
+				struct fwd_stream *fs = fwd_streams[i];
+				struct rte_eth_dev_data *dev_rx_data, *dev_tx_data;
+
+				dev_rx_data = (&rte_eth_devices[fs->rx_port])->data;
+				dev_tx_data = (&rte_eth_devices[fs->tx_port])->data;
+
+				uint8_t rx_state = dev_rx_data->rx_queue_state[fs->rx_port];
+				ports[fs->rx_port].rxq[fs->rx_queue].state = rx_state;
+				uint8_t tx_state = dev_tx_data->tx_queue_state[fs->tx_port];
+				ports[fs->tx_port].txq[fs->tx_queue].state = tx_state;
+			}
 			stream_init(fwd_streams[i]);
+		}
+	}
 
 	port_fwd_begin = cur_fwd_config.fwd_eng->port_fwd_begin;
 	if (port_fwd_begin != NULL) {
Signature · 1 line
2.25.1
#12 lore raw

Re: [PATCH v2] app/testpmd: fix incorrect queues state of secondary process

Hi Peng,

Quoted 36 lines · On 8/19/2022 3:39 PM, [email protected] wrote:

From: Peng Zhang <[email protected]>

Primary process could set up queues state correctly when starting port, but under multi-process scenario, "stream_init" function would get wrong queues state for secondary process.

This commit is to get queues state from ethdev which is located in shared memory.

Fixes 3c4426db54fc ("app/testpmd: do not poll stopped queues")
---
  app/test-pmd/testpmd.c | 20 ++++++++++++++++++--
  1 file changed, 18 insertions(+), 2 deletions(-)
app/test-pmd/testpmd.c +2 -2
index addcbcac85..70f907d96b 100644
@@ -75,6 +75,8 @@
 
  #include "testpmd.h"
 
+#include <ethdev_driver.h>
+
  #ifndef MAP_HUGETLB
  /* FreeBSD may not have MAP_HUGETLB (in fact, it probably doesn't) */
  #define HUGE_FLAG (0x40000)
@@ -2402,9 +2404,23 @@ start_packet_forwarding(int with_tx_first)
  	if (!pkt_fwd_shared_rxq_check())
  		return;
 
-	if (stream_init != NULL)
-		for (i = 0; i < cur_fwd_config.nb_fwd_streams; i++)

app/test-pmd/testpmd.c start_packet_forwarding(int with_tx_first) · line 2416

Quoted 10 lines
app/test-pmd/testpmd.c +10 -0
+	if (stream_init != NULL) {
+		for (i = 0; i < cur_fwd_config.nb_fwd_streams; i++) {
+			if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
+				struct fwd_stream *fs = fwd_streams[i];
+				struct rte_eth_dev_data *dev_rx_data, *dev_tx_data;
+
+				dev_rx_data = (&rte_eth_devices[fs->rx_port])->data;
+				dev_tx_data = (&rte_eth_devices[fs->tx_port])->data;
+
+				uint8_t rx_state = dev_rx_data->rx_queue_state[fs->rx_port];

To get the queue state, the array rx_queue_state[] should be indexed by the queue number. Using fs->rx_port may not give desired queue's state.

app/test-pmd/testpmd.c start_packet_forwarding(int with_tx_first) · line 2418

Quoted 2 lines
app/test-pmd/testpmd.c +2 -0
+				ports[fs->rx_port].rxq[fs->rx_queue].state = rx_state;
+				uint8_t tx_state = dev_tx_data->tx_queue_state[fs->tx_port];

Same as rx queue above. We might need to root cause the issue further.

Quoted 8 lines
app/test-pmd/testpmd.c +4 -0
+				ports[fs->tx_port].txq[fs->tx_queue].state = tx_state;
+			}
  			stream_init(fwd_streams[i]);
+		}
+	}
 
  	port_fwd_begin = cur_fwd_config.fwd_eng->port_fwd_begin;
  	if (port_fwd_begin != NULL) {
#13 lore raw

RE: [PATCH v2] app/testpmd: fix incorrect queues state of secondary process

Hi Aman Deep,

Thanks for your comment, I will follow it. And root cause is to secondary process, it cannot get queue state directly, but from share memory. So when starting port, secondary process cannot get correct queue state even if queue has been started.

Quoted 82 lines

-----Original Message-----
From: Singh, Aman Deep <[email protected]>
Sent: Thursday, August 25, 2022 2:22 AM
To: Zhang, Peng1X <[email protected]>; [email protected]
Cc: Zhang, Yuying <[email protected]>; [email protected]
Subject: Re: [PATCH v2] app/testpmd: fix incorrect queues state of secondary process

Hi Peng,

Quoted 31 lines · On 8/19/2022 3:39 PM, [email protected] wrote:

From: Peng Zhang <[email protected]>

Primary process could set up queues state correctly when starting port, but under multi-process scenario, "stream_init" function would get wrong queues state for secondary process.

This commit is to get queues state from ethdev which is located in shared memory.

Fixes 3c4426db54fc ("app/testpmd: do not poll stopped queues")
---
  app/test-pmd/testpmd.c | 20 ++++++++++++++++++--
  1 file changed, 18 insertions(+), 2 deletions(-)
app/test-pmd/testpmd.c index renamed from app/test-pmd/testpmd.c +0 -0

addcbcac85..70f907d96b 100644

app/test-pmd/testpmd.c +2 -0
@@ -75,6 +75,8 @@
 
  #include "testpmd.h"
 
+#include <ethdev_driver.h>
+
  #ifndef MAP_HUGETLB
  /* FreeBSD may not have MAP_HUGETLB (in fact, it probably doesn't) */
  #define HUGE_FLAG (0x40000)

app/test-pmd/testpmd.c start_packet_forwarding(int with_tx_first) · line 2411

Quoted 10 lines
app/test-pmd/testpmd.c +5 -2
@@ -2402,9 +2404,23 @@ start_packet_forwarding(int with_tx_first)
  	if (!pkt_fwd_shared_rxq_check())
  		return;
 
-	if (stream_init != NULL)
-		for (i = 0; i < cur_fwd_config.nb_fwd_streams; i++)
+	if (stream_init != NULL) {
+		for (i = 0; i < cur_fwd_config.nb_fwd_streams; i++) {
+			if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
+				struct fwd_stream *fs = fwd_streams[i];
+				struct rte_eth_dev_data *dev_rx_data,

*dev_tx_data;

app/test-pmd/testpmd.c start_packet_forwarding(int with_tx_first) · line 2419

Quoted 8 lines
app/test-pmd/testpmd.c +5 -0
+
+				dev_rx_data = (&rte_eth_devices[fs-
 rx_port])->data;
+				dev_tx_data = (&rte_eth_devices[fs->tx_port])-
 data;
+
+				uint8_t rx_state = dev_rx_data-
 rx_queue_state[fs->rx_port];

To get the queue state, the array rx_queue_state[] should be indexed by the queue number.
Using fs->rx_port may not give desired queue's state.

app/test-pmd/testpmd.c start_packet_forwarding(int with_tx_first) · line 2420

Quoted 1 line
app/test-pmd/testpmd.c +1 -0
+				ports[fs->rx_port].rxq[fs->rx_queue].state =

rx_state;

app/test-pmd/testpmd.c start_packet_forwarding(int with_tx_first) · line 2422

Quoted 2 lines
app/test-pmd/testpmd.c +1 -0
+				uint8_t tx_state = dev_tx_data-
 tx_queue_state[fs->tx_port];

Same as rx queue above. We might need to root cause the issue further.

app/test-pmd/testpmd.c start_packet_forwarding(int with_tx_first) · line 2423

Quoted 1 line
app/test-pmd/testpmd.c +1 -0
+				ports[fs->tx_port].txq[fs->tx_queue].state =

tx_state;

Quoted 7 lines
app/test-pmd/testpmd.c +3 -0
+			}
  			stream_init(fwd_streams[i]);
+		}
+	}
 
  	port_fwd_begin = cur_fwd_config.fwd_eng->port_fwd_begin;
  	if (port_fwd_begin != NULL) {
author
#14 lore raw

PATCH v3 1/1 app/testpmd: fix incorrect queues state of secondary process

In reply to [email protected]

Primary process could set up queues state correctly when starting port, while secondary process not. Under multi-process scenario, "stream_init" function would get wrong queues state for secondary process.

This commit is to get queues state from ethdev which is located in shared memory.

Fixes 3c4426db54fc ("app/testpmd: do not poll stopped queues")

---
v3:
- Modify the parameter of rx or tx queue state array v2:
- Change the way of getting secondary process queues states

---
 app/test-pmd/testpmd.c | 22 +++++++++++++++++++---
 1 file changed, 19 insertions(+), 3 deletions(-)
app/test-pmd/testpmd.c +19 -3
index addcbcac85..977ec4fa28 100644
@@ -75,6 +75,8 @@
 
 #include "testpmd.h"
 
+#include <ethdev_driver.h>
+
 #ifndef MAP_HUGETLB
 /* FreeBSD may not have MAP_HUGETLB (in fact, it probably doesn't) */
 #define HUGE_FLAG (0x40000)
@@ -2402,10 +2404,24 @@ start_packet_forwarding(int with_tx_first)
 	if (!pkt_fwd_shared_rxq_check())
 		return;
 
-	if (stream_init != NULL)
-		for (i = 0; i < cur_fwd_config.nb_fwd_streams; i++)
-			stream_init(fwd_streams[i]);
+	if (stream_init != NULL) {
+		for (i = 0; i < cur_fwd_config.nb_fwd_streams; i++) {
+			if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
+				struct fwd_stream *fs = fwd_streams[i];
+				struct rte_eth_dev_data *dev_rx_data, *dev_tx_data;
+
+				dev_rx_data = (&rte_eth_devices[fs->rx_port])->data;
+				dev_tx_data = (&rte_eth_devices[fs->tx_port])->data;
+
+				uint8_t rx_state = dev_rx_data->rx_queue_state[fs->rx_queue];
+				ports[fs->rx_port].rxq[fs->rx_queue].state = rx_state;
+				uint8_t tx_state = dev_tx_data->tx_queue_state[fs->tx_queue];
+				ports[fs->tx_port].txq[fs->tx_queue].state = tx_state;
+			}
 
+			stream_init(fwd_streams[i]);
+		}
+	}
 	port_fwd_begin = cur_fwd_config.fwd_eng->port_fwd_begin;
 	if (port_fwd_begin != NULL) {
 		for (i = 0; i < cur_fwd_config.nb_fwd_ports; i++) {
Signature · 1 line
2.25.1
#15 lore raw

Re: [PATCH v3] app/testpmd: fix incorrect queues state of secondary process

Quoted 26 lines · Peng Zhang

Primary process could set up queues state correctly when starting port, while secondary process not. Under multi-process scenario, "stream_init" function would get wrong queues state for secondary process.

This commit is to get queues state from ethdev which is located in shared memory.

Fixes 3c4426db54fc ("app/testpmd: do not poll stopped queues")

---
v3:
- Modify the parameter of rx or tx queue state array v2:
- Change the way of getting secondary process queues states

---
  app/test-pmd/testpmd.c | 22 +++++++++++++++++++---
  1 file changed, 19 insertions(+), 3 deletions(-)
app/test-pmd/testpmd.c +0 -0
index addcbcac85..977ec4fa28 100644

app/test-pmd/testpmd.c line 78

Quoted 4 lines
app/test-pmd/testpmd.c +1 -0
@@ -75,6 +75,8 @@
 
  #include "testpmd.h"
 
+#include <ethdev_driver.h>

This header file is internal, app shouldn't use it.

Quoted 20 lines
app/test-pmd/testpmd.c +10 -3
+
  #ifndef MAP_HUGETLB
  /* FreeBSD may not have MAP_HUGETLB (in fact, it probably doesn't) */
  #define HUGE_FLAG (0x40000)
@@ -2402,10 +2404,24 @@ start_packet_forwarding(int with_tx_first)
  	if (!pkt_fwd_shared_rxq_check())
  		return;
 
-	if (stream_init != NULL)
-		for (i = 0; i < cur_fwd_config.nb_fwd_streams; i++)
-			stream_init(fwd_streams[i]);
+	if (stream_init != NULL) {
+		for (i = 0; i < cur_fwd_config.nb_fwd_streams; i++) {
+			if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
+				struct fwd_stream *fs = fwd_streams[i];
+				struct rte_eth_dev_data *dev_rx_data, *dev_tx_data;
+
+				dev_rx_data = (&rte_eth_devices[fs->rx_port])->data;
+				dev_tx_data = (&rte_eth_devices[fs->tx_port])->data;
+

app/test-pmd/testpmd.c start_packet_forwarding(int with_tx_first) · line 2425

Quoted 10 lines
app/test-pmd/testpmd.c +8 -0
+				uint8_t rx_state = dev_rx_data->rx_queue_state[fs->rx_queue];
+				ports[fs->rx_port].rxq[fs->rx_queue].state = rx_state;
+				uint8_t tx_state = dev_tx_data->tx_queue_state[fs->tx_queue];
+				ports[fs->tx_port].txq[fs->tx_queue].state = tx_state;
+			}
 
+			stream_init(fwd_streams[i]);
+		}
+	}
 

Suggest that an API in ethdev layer can be encapsulated to obtain the device Rx/Tx queue state. Both primary and secondary process get or set queue state by this API.

Quoted 3 lines
app/test-pmd/testpmd.c +0 -0
  	port_fwd_begin = cur_fwd_config.fwd_eng->port_fwd_begin;
  	if (port_fwd_begin != NULL) {
  		for (i = 0; i < cur_fwd_config.nb_fwd_ports; i++) {
#16 lore raw

RE: [PATCH v3] app/testpmd: fix incorrect queues state of secondary process

Quoted 42 lines

-----Original Message-----
From: lihuisong (C) <[email protected]>
Sent: Wednesday, September 7, 2022 9:53 AM
To: Zhang, Peng1X <[email protected]>; [email protected]
Cc: [email protected]; Singh, Aman Deep <[email protected]>; Zhang, Yuying <[email protected]>; [email protected]
Subject: Re: [PATCH v3] app/testpmd: fix incorrect queues state of secondary process

Quoted 2 lines · Peng Zhang

Primary process could set up queues state correctly when starting port, while secondary process not. Under multi-process scenario,

"stream_init"

Quoted 25 lines

function would get wrong queues state for secondary process.

This commit is to get queues state from ethdev which is located in shared memory.

Fixes 3c4426db54fc ("app/testpmd: do not poll stopped queues")

---
v3:
- Modify the parameter of rx or tx queue state array v2:
- Change the way of getting secondary process queues states

---
  app/test-pmd/testpmd.c | 22 +++++++++++++++++++---
  1 file changed, 19 insertions(+), 3 deletions(-)
app/test-pmd/testpmd.c index renamed from app/test-pmd/testpmd.c +0 -0

addcbcac85..977ec4fa28 100644

app/test-pmd/testpmd.c +0 -0
@@ -75,6 +75,8 @@
 
Quoted 4 lines

app/test-pmd/testpmd.c line 78

Quoted 3 lines
app/test-pmd/testpmd.c +1 -0
  #include "testpmd.h"
 
+#include <ethdev_driver.h>

This header file is internal, app shouldn't use it.

In this case not all PMDs implement 'rte_eth_rx/tx_queue_info_get()' and ethdev won't return 'dev->data->rx/tx_queue_state'.

Quoted 33 lines
Quoted 6 lines
app/test-pmd/testpmd.c +1 -0
+
  #ifndef MAP_HUGETLB
  /* FreeBSD may not have MAP_HUGETLB (in fact, it probably doesn't) */
  #define HUGE_FLAG (0x40000)
@@ -2402,10 +2404,24 @@ start_packet_forwarding(int with_tx_first)
  	if (!pkt_fwd_shared_rxq_check())

app/test-pmd/testpmd.c start_packet_forwarding(int with_tx_first) · line 2411

Quoted 10 lines
app/test-pmd/testpmd.c +5 -3
  		return;
 
-	if (stream_init != NULL)
-		for (i = 0; i < cur_fwd_config.nb_fwd_streams; i++)
-			stream_init(fwd_streams[i]);
+	if (stream_init != NULL) {
+		for (i = 0; i < cur_fwd_config.nb_fwd_streams; i++) {
+			if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
+				struct fwd_stream *fs = fwd_streams[i];
+				struct rte_eth_dev_data *dev_rx_data,

*dev_tx_data;

app/test-pmd/testpmd.c start_packet_forwarding(int with_tx_first) · line 2420

Quoted 9 lines
app/test-pmd/testpmd.c +6 -0
+
+				dev_rx_data = (&rte_eth_devices[fs->rx_port])-
 data;
+				dev_tx_data = (&rte_eth_devices[fs->tx_port])-
 data;
+
+				uint8_t rx_state = dev_rx_data-
 rx_queue_state[fs->rx_queue];
+				ports[fs->rx_port].rxq[fs->rx_queue].state =

rx_state;

app/test-pmd/testpmd.c start_packet_forwarding(int with_tx_first) · line 2423

Quoted 3 lines
app/test-pmd/testpmd.c +2 -0
+				uint8_t tx_state = dev_tx_data-
 tx_queue_state[fs->tx_queue];
+				ports[fs->tx_port].txq[fs->tx_queue].state =

tx_state;

Quoted 2 lines
app/test-pmd/testpmd.c +1 -0
+			}
Quoted 7 lines

app/test-pmd/testpmd.c start_packet_forwarding(int with_tx_first) · line 2424

Quoted 4 lines
app/test-pmd/testpmd.c +3 -0
+			stream_init(fwd_streams[i]);
+		}
+	}

Suggest that an API in ethdev layer can be encapsulated to obtain the device Rx/Tx queue state.
Both primary and secondary process get or set queue state by this API.

Suggestion sounds good, maybe better to add a new API in ethdev layer.

@andrew, what's your opinion about this solution and huisong's suggestion?

Quoted 3 lines
Quoted 3 lines
  	port_fwd_begin = cur_fwd_config.fwd_eng->port_fwd_begin;
  	if (port_fwd_begin != NULL) {
  		for (i = 0; i < cur_fwd_config.nb_fwd_ports; i++) {
#17 lore raw

Re: [PATCH v3] app/testpmd: fix incorrect queues state of secondary process

Quoted 41 lines · 在 2022/9/10 17:21, Zhang, Peng1X 写道:
Quoted 41 lines

-----Original Message-----
From: lihuisong (C) <[email protected]>
Sent: Wednesday, September 7, 2022 9:53 AM
To: Zhang, Peng1X <[email protected]>; [email protected]
Cc: [email protected]; Singh, Aman Deep <[email protected]>; Zhang, Yuying <[email protected]>; [email protected]
Subject: Re: [PATCH v3] app/testpmd: fix incorrect queues state of secondary process

Quoted 2 lines · Peng Zhang

Primary process could set up queues state correctly when starting port, while secondary process not. Under multi-process scenario,

"stream_init"

Quoted 25 lines

function would get wrong queues state for secondary process.

This commit is to get queues state from ethdev which is located in shared memory.

Fixes 3c4426db54fc ("app/testpmd: do not poll stopped queues")

---
v3:
- Modify the parameter of rx or tx queue state array v2:
- Change the way of getting secondary process queues states

---
   app/test-pmd/testpmd.c | 22 +++++++++++++++++++---
   1 file changed, 19 insertions(+), 3 deletions(-)
app/test-pmd/testpmd.c index renamed from app/test-pmd/testpmd.c +0 -0

addcbcac85..977ec4fa28 100644

app/test-pmd/testpmd.c +0 -0
@@ -75,6 +75,8 @@
 
Quoted 5 lines
Quoted 4 lines

app/test-pmd/testpmd.c line 78

Quoted 3 lines
app/test-pmd/testpmd.c +1 -0
   #include "testpmd.h"
 
+#include <ethdev_driver.h>

This header file is internal, app shouldn't use it.

In this case not all PMDs implement 'rte_eth_rx/tx_queue_info_get()' and ethdev won't return 'dev->data->rx/tx_queue_state'.

I don't think Rx/Tx queue state needs to be put in the API above.
The queue state is modified by calling 'rte_eth_dev_rx/tx_queue_stop/start'. Can we create a new API to report queue state? like, 'rte_eth_dev_rx/tx_queue_state_get'. If some PMDs do not support 'rte_eth_dev_rx/tx_queue_stop/start', the new API always return 'START' state and preserves its original behavior.

What do you think?

Quoted 48 lines
Quoted 34 lines
Quoted 7 lines
app/test-pmd/testpmd.c +1 -0
+
   #ifndef MAP_HUGETLB
   /* FreeBSD may not have MAP_HUGETLB (in fact, it probably doesn't) */
   #define HUGE_FLAG (0x40000)
@@ -2402,10 +2404,24 @@ start_packet_forwarding(int with_tx_first)
   	if (!pkt_fwd_shared_rxq_check())

app/test-pmd/testpmd.c start_packet_forwarding(int with_tx_first) · line 2411

Quoted 10 lines
app/test-pmd/testpmd.c +5 -3
   		return;
 
-	if (stream_init != NULL)
-		for (i = 0; i < cur_fwd_config.nb_fwd_streams; i++)
-			stream_init(fwd_streams[i]);
+	if (stream_init != NULL) {
+		for (i = 0; i < cur_fwd_config.nb_fwd_streams; i++) {
+			if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
+				struct fwd_stream *fs = fwd_streams[i];
+				struct rte_eth_dev_data *dev_rx_data,

*dev_tx_data;

app/test-pmd/testpmd.c start_packet_forwarding(int with_tx_first) · line 2420

Quoted 9 lines
app/test-pmd/testpmd.c +6 -0
+
+				dev_rx_data = (&rte_eth_devices[fs->rx_port])-
 data;
+				dev_tx_data = (&rte_eth_devices[fs->tx_port])-
 data;
+
+				uint8_t rx_state = dev_rx_data-
 rx_queue_state[fs->rx_queue];
+				ports[fs->rx_port].rxq[fs->rx_queue].state =

rx_state;

app/test-pmd/testpmd.c start_packet_forwarding(int with_tx_first) · line 2423

Quoted 3 lines
app/test-pmd/testpmd.c +2 -0
+				uint8_t tx_state = dev_tx_data-
 tx_queue_state[fs->tx_queue];
+				ports[fs->tx_port].txq[fs->tx_queue].state =

tx_state;

Quoted 2 lines
app/test-pmd/testpmd.c +1 -0
+			}
Quoted 7 lines

app/test-pmd/testpmd.c start_packet_forwarding(int with_tx_first) · line 2424

Quoted 4 lines
app/test-pmd/testpmd.c +3 -0
+			stream_init(fwd_streams[i]);
+		}
+	}

Suggest that an API in ethdev layer can be encapsulated to obtain the device Rx/Tx queue state.
Both primary and secondary process get or set queue state by this API.

Suggestion sounds good, maybe better to add a new API in ethdev layer.

@andrew, what's your opinion about this solution and huisong's suggestion?

Quoted 3 lines
Quoted 3 lines
   	port_fwd_begin = cur_fwd_config.fwd_eng->port_fwd_begin;
   	if (port_fwd_begin != NULL) {
   		for (i = 0; i < cur_fwd_config.nb_fwd_ports; i++) {
#18 lore raw

Re: [PATCH v3] app/testpmd: fix incorrect queues state of secondary process

Quoted 43 lines · lihuisong (C)
Quoted 41 lines · 在 2022/9/10 17:21, Zhang, Peng1X 写道:
Quoted 41 lines

-----Original Message-----
From: lihuisong (C) <[email protected]>
Sent: Wednesday, September 7, 2022 9:53 AM
To: Zhang, Peng1X <[email protected]>; [email protected]
Cc: [email protected]; Singh, Aman Deep <[email protected]>; Zhang, Yuying <[email protected]>; [email protected]
Subject: Re: [PATCH v3] app/testpmd: fix incorrect queues state of secondary process

Quoted 2 lines · Peng Zhang

Primary process could set up queues state correctly when starting port, while secondary process not. Under multi-process scenario,

"stream_init"

Quoted 25 lines

function would get wrong queues state for secondary process.

This commit is to get queues state from ethdev which is located in shared memory.

Fixes 3c4426db54fc ("app/testpmd: do not poll stopped queues")

---
   v3:
   - Modify the parameter of rx or tx queue state array v2:
   - Change the way of getting secondary process queues states
---
   app/test-pmd/testpmd.c | 22 +++++++++++++++++++---

   1 file changed, 19 insertions(+), 3 deletions(-)
app/test-pmd/testpmd.c index renamed from app/test-pmd/testpmd.c +0 -0

addcbcac85..977ec4fa28 100644

app/test-pmd/testpmd.c +0 -0
@@ -75,6 +75,8 @@
 
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

app/test-pmd/testpmd.c line 78

Quoted 3 lines
app/test-pmd/testpmd.c +1 -0
    #include "testpmd.h"
 
+#include <ethdev_driver.h>

This header file is internal, app shouldn't use it.

+1

Quoted 4 lines
Quoted 1 line

In this case not all PMDs implement 'rte_eth_rx/tx_queue_info_get()' and ethdev won't return 'dev->data->rx/tx_queue_state'.

I don't think Rx/Tx queue state needs to be put in the API above.
The queue state is modified by calling 'rte_eth_dev_rx/tx_queue_stop/start'. Can we create a new API to report queue state? like, 'rte_eth_dev_rx/tx_queue_state_get'.

We can and it does not require driver callback since ethdev layer knows the queue state, but do we really need it? Application controls queues state and should know the state internally.

Quoted 2 lines

If some PMDs do not support 'rte_eth_dev_rx/tx_queue_stop/start', the new API always return 'START' state and preserves its original behavior.

It is not required since ethdev layer knows queue state.

Quoted 47 lines

What do you think?

Quoted 45 lines
Quoted 34 lines
Quoted 7 lines
app/test-pmd/testpmd.c +1 -0
+
    #ifndef MAP_HUGETLB
    /* FreeBSD may not have MAP_HUGETLB (in fact, it probably doesn't) */
    #define HUGE_FLAG (0x40000)
@@ -2402,10 +2404,24 @@ start_packet_forwarding(int with_tx_first)
        if (!pkt_fwd_shared_rxq_check())

app/test-pmd/testpmd.c start_packet_forwarding(int with_tx_first) · line 2411

Quoted 10 lines
app/test-pmd/testpmd.c +5 -3
            return;
 
-    if (stream_init != NULL)
-        for (i = 0; i < cur_fwd_config.nb_fwd_streams; i++)
-            stream_init(fwd_streams[i]);
+    if (stream_init != NULL) {
+        for (i = 0; i < cur_fwd_config.nb_fwd_streams; i++) {
+            if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
+                struct fwd_stream *fs = fwd_streams[i];
+                struct rte_eth_dev_data *dev_rx_data,

*dev_tx_data;

app/test-pmd/testpmd.c start_packet_forwarding(int with_tx_first) · line 2420

Quoted 9 lines
app/test-pmd/testpmd.c +6 -0
+
+                dev_rx_data = (&rte_eth_devices[fs->rx_port])-
 data;
+                dev_tx_data = (&rte_eth_devices[fs->tx_port])-
 data;
+
+                uint8_t rx_state = dev_rx_data-
 rx_queue_state[fs->rx_queue];
+                ports[fs->rx_port].rxq[fs->rx_queue].state =

rx_state;

app/test-pmd/testpmd.c start_packet_forwarding(int with_tx_first) · line 2423

Quoted 3 lines
app/test-pmd/testpmd.c +2 -0
+                uint8_t tx_state = dev_tx_data-
 tx_queue_state[fs->tx_queue];
+                ports[fs->tx_port].txq[fs->tx_queue].state =

tx_state;

Quoted 2 lines
app/test-pmd/testpmd.c +1 -0
+            }
Quoted 7 lines

app/test-pmd/testpmd.c start_packet_forwarding(int with_tx_first) · line 2424

Quoted 4 lines
app/test-pmd/testpmd.c +3 -0
+            stream_init(fwd_streams[i]);
+        }
+    }

Suggest that an API in ethdev layer can be encapsulated to obtain the device Rx/Tx queue state.
Both primary and secondary process get or set queue state by this API.

Suggestion sounds good, maybe better to add a new API in ethdev layer.

@andrew, what's your opinion about this solution and huisong's suggestion?

May be it is really more convenient to have ethdev API to get queue state, but may be it is too late for the API in the release cycle phase. I'm sorry for late review.

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

       port_fwd_begin = cur_fwd_config.fwd_eng->port_fwd_begin; if (port_fwd_begin != NULL) {
           for (i = 0; i < cur_fwd_config.nb_fwd_ports; i++) {

#19 lore raw

RE: [PATCH v3] app/testpmd: fix incorrect queues state of secondary process

In reply to Zhang, Peng1X
Quoted 78 lines
Quoted 28 lines
Quoted 2 lines

Primary process could set up queues state correctly when starting port, while secondary process not. Under multi-process scenario,

"stream_init"

Quoted 25 lines

function would get wrong queues state for secondary process.

This commit is to get queues state from ethdev which is located in shared memory.

Fixes 3c4426db54fc ("app/testpmd: do not poll stopped queues")

---
v3:
- Modify the parameter of rx or tx queue state array v2:
- Change the way of getting secondary process queues states

---
  app/test-pmd/testpmd.c | 22 +++++++++++++++++++---
  1 file changed, 19 insertions(+), 3 deletions(-)
app/test-pmd/testpmd.c index renamed from app/test-pmd/testpmd.c +0 -0

addcbcac85..977ec4fa28 100644

app/test-pmd/testpmd.c +0 -0
@@ -75,6 +75,8 @@
 
Quoted 4 lines

app/test-pmd/testpmd.c line 78

Quoted 3 lines
app/test-pmd/testpmd.c +1 -0
  #include "testpmd.h"
 
+#include <ethdev_driver.h>

This header file is internal, app shouldn't use it.

In this case not all PMDs implement 'rte_eth_rx/tx_queue_info_get()' and ethdev won't return 'dev->data->rx/tx_queue_state'.

Quoted 34 lines
Quoted 6 lines
app/test-pmd/testpmd.c +1 -0
+
  #ifndef MAP_HUGETLB
  /* FreeBSD may not have MAP_HUGETLB (in fact, it probably doesn't) */
  #define HUGE_FLAG (0x40000)
@@ -2402,10 +2404,24 @@ start_packet_forwarding(int with_tx_first)
  	if (!pkt_fwd_shared_rxq_check())

app/test-pmd/testpmd.c start_packet_forwarding(int with_tx_first) · line 2411

Quoted 10 lines
app/test-pmd/testpmd.c +5 -3
  		return;
 
-	if (stream_init != NULL)
-		for (i = 0; i < cur_fwd_config.nb_fwd_streams; i++)
-			stream_init(fwd_streams[i]);
+	if (stream_init != NULL) {
+		for (i = 0; i < cur_fwd_config.nb_fwd_streams; i++) {
+			if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
+				struct fwd_stream *fs = fwd_streams[i];
+				struct rte_eth_dev_data *dev_rx_data,

*dev_tx_data;

app/test-pmd/testpmd.c start_packet_forwarding(int with_tx_first) · line 2413

Quoted 2 lines
app/test-pmd/testpmd.c +2 -0
+
+				dev_rx_data = (&rte_eth_devices[fs-

rx_port])-

app/test-pmd/testpmd.c start_packet_forwarding(int with_tx_first) · line 2420

Quoted 7 lines
app/test-pmd/testpmd.c +4 -0
 data;
+				dev_tx_data = (&rte_eth_devices[fs->tx_port])-
 data;
+
+				uint8_t rx_state = dev_rx_data-
 rx_queue_state[fs->rx_queue];
+				ports[fs->rx_port].rxq[fs->rx_queue].state =

rx_state;

app/test-pmd/testpmd.c start_packet_forwarding(int with_tx_first) · line 2423

Quoted 3 lines
app/test-pmd/testpmd.c +2 -0
+				uint8_t tx_state = dev_tx_data-
 tx_queue_state[fs->tx_queue];
+				ports[fs->tx_port].txq[fs->tx_queue].state =

tx_state;

Quoted 2 lines
app/test-pmd/testpmd.c +1 -0
+			}
Quoted 7 lines

app/test-pmd/testpmd.c start_packet_forwarding(int with_tx_first) · line 2424

Quoted 4 lines
app/test-pmd/testpmd.c +3 -0
+			stream_init(fwd_streams[i]);
+		}
+	}

Suggest that an API in ethdev layer can be encapsulated to obtain the device Rx/Tx queue state.
Both primary and secondary process get or set queue state by this API.

Quoted 7 lines

Suggestion sounds good, maybe better to add a new API in ethdev layer.

@andrew, what's your opinion about this solution and huisong's suggestion?

Quoted 3 lines
Quoted 3 lines
  	port_fwd_begin = cur_fwd_config.fwd_eng->port_fwd_begin;
  	if (port_fwd_begin != NULL) {
  		for (i = 0; i < cur_fwd_config.nb_fwd_ports; i++) {

Hi, @[email protected] can you please help review this patch? Thanks.

#20 lore raw

RE: [PATCH v3] app/testpmd: fix incorrect queues state of secondary process

Hi

Quoted 22 lines

-----Original Message-----
From: Zhou, YidingX <[email protected]>
Sent: Thursday, September 29, 2022 9:58 AM
To: Zhang, Peng1X <[email protected]>; lihuisong (C) <[email protected]>; [email protected]; [email protected]
Cc: Singh, Aman Deep <[email protected]>; Zhang, Yuying <[email protected]>
Subject: RE: [PATCH v3] app/testpmd: fix incorrect queues state of secondary process

Quoted 12 lines
Quoted 12 lines
Quoted 2 lines

Primary process could set up queues state correctly when starting port, while secondary process not. Under multi-process scenario,

"stream_init"

Quoted 9 lines

function would get wrong queues state for secondary process.

This commit is to get queues state from ethdev which is located in shared memory.

Fixes 3c4426db54fc ("app/testpmd: do not poll stopped queues")
#21 lore raw

Re: [PATCH v3] app/testpmd: fix incorrect queues state of secondary process

In reply to Peng Zhang

app/test-pmd/testpmd.c start_packet_forwarding(int with_tx_first) · line 2420

Quoted 12 lines · On Tue, 6 Sep 2022 22:53:10 +0800 Peng Zhang <[email protected]> wrote:
app/test-pmd/testpmd.c +12 -0
+			if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
+				struct fwd_stream *fs = fwd_streams[i];
+				struct rte_eth_dev_data *dev_rx_data, *dev_tx_data;
+
+				dev_rx_data = (&rte_eth_devices[fs->rx_port])->data;
+				dev_tx_data = (&rte_eth_devices[fs->tx_port])->data;
+
+				uint8_t rx_state = dev_rx_data->rx_queue_state[fs->rx_queue];
+				ports[fs->rx_port].rxq[fs->rx_queue].state = rx_state;
+				uint8_t tx_state = dev_tx_data->tx_queue_state[fs->tx_queue];
+				ports[fs->tx_port].txq[fs->tx_queue].state = tx_state;
+			}

Could the logic be put in stream_init() like this?

It keeps with the function pointer model object style model in that code. Also, it makes testpmd more dependent on data structures that should be hidden and internal only (rte_eth_devices).

#22 lore raw

RE: [PATCH v3] app/testpmd: fix incorrect queues state of secondary process

Quoted 19 lines

app/test-pmd/testpmd.c start_packet_forwarding(int with_tx_first)

Quoted 4 lines
app/test-pmd/testpmd.c +3 -0
+			if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
+				struct fwd_stream *fs = fwd_streams[i];
+				struct rte_eth_dev_data *dev_rx_data,

*dev_tx_data;

Review

Quoted 9 lines
(quoted diff) +2 -0
+
+				dev_rx_data = (&rte_eth_devices[fs-

rx_port])->data;

(quoted diff) +1 -0
+				dev_tx_data = (&rte_eth_devices[fs->tx_port])-

data;

(quoted diff) +2 -0
+
+				uint8_t rx_state = dev_rx_data-

rx_queue_state[fs->rx_queue];

(quoted diff) +1 -0
+				ports[fs->rx_port].rxq[fs->rx_queue].state =

rx_state;

Review

Quoted 3 lines
(quoted diff) +1 -0
+				uint8_t tx_state = dev_tx_data-

tx_queue_state[fs->tx_queue];

(quoted diff) +1 -0
+				ports[fs->tx_port].txq[fs->tx_queue].state =

tx_state;

Quoted 8 lines

Review

Quoted 1 line
(quoted diff) +1 -0
+			}

Could the logic be put in stream_init() like this?

It keeps with the function pointer model object style model in that code. Also, it makes testpmd more dependent on data structures that should be hidden and internal only (rte_eth_devices).

'strean_init' is a function pointer, putting this logic in will produce a lot of similar redundant code in all forwarding modules, like 'iofwd' 'macfwd' and etc. It's better to hide 'rte_eth_devices' from testpmd, but this requires an API at the EAL layer to access the queue state, 'rte_eth_rx/tx_queue_info_get()' can not get queue's state now.