Skip to contentSkip to conversation

net/gve: Change ERR to DEBUG to prevent flooding of logs for Tx-Dqo.

PATCH In review Rushil Gupta 3 participants 6 messages Feb 19, 2024 – Feb 20, 2024 Network drivers

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 net/gve: Change ERR to DEBUG to prevent flooding of logs for Tx-Dqo.

This was causing failure for testpmd runs (for queues >=15) presumably due to flooding of logs due to descriptor ring being overwritten.

Fixes a01854 ("net/gve: fix dqo bug for chained descriptors")
---
 drivers/net/gve/gve_tx_dqo.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/gve/gve_tx_dqo.c +1 -1
index 1a8eb96ea9..30a1455b20 100644
@@ -116,7 +116,7 @@ gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
 		first_sw_id = sw_id;
 		do {
 			if (sw_ring[sw_id] != NULL)
-				PMD_DRV_LOG(ERR, "Overwriting an entry in sw_ring");
+				PMD_DRV_LOG(DEBUG, "Overwriting an entry in sw_ring");
 
 			txd = &txr[tx_id];
 			sw_ring[sw_id] = tx_pkt;
Signature · 1 line
2.44.0.rc0.258.g7320e95886-goog
#1 lore raw
Quoted 18 lines · Rushil Gupta

This was causing failure for testpmd runs (for queues >=15) presumably due to flooding of logs due to descriptor ring being overwritten.

Fixes a01854 ("net/gve: fix dqo bug for chained descriptors")
---
 drivers/net/gve/gve_tx_dqo.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/gve/gve_tx_dqo.c +0 -0
index 1a8eb96ea9..30a1455b20 100644

drivers/net/gve/gve_tx_dqo.c gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) · line 122

Quoted 8 lines
drivers/net/gve/gve_tx_dqo.c +1 -1
@@ -116,7 +116,7 @@ gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
 		first_sw_id = sw_id;
 		do {
 			if (sw_ring[sw_id] != NULL)
-				PMD_DRV_LOG(ERR, "Overwriting an entry in sw_ring");
+				PMD_DRV_LOG(DEBUG, "Overwriting an entry in sw_ring");
 
 			txd = &txr[tx_id];
 			sw_ring[sw_id] = tx_pkt;

Hi Rushil,

I will continue with this patch, BUT
logging in the datapath has potential problems like this, also it may have performance impact even log is not printed, because of additional check in the log function.

For datapath, you can prefer:
- Add log macros controlled by RTE_ETHDEV_DEBUG_RX & RTE_ETHDEV_DEBUG_TX flags
- Or use RTE_LOG_DP() macro which is compiled out if default log level is less than the log uses

Also another perspective for the logs, when end-user observes this bloat of logs, what she can do?
Can she change some driver arguments or environment variables to fix the issue, if not what is the point of the log?
If this is a condition that can occur dynamically based on received traffic and user doesn't have control on it, maybe driver should handle the error without log?
And if this is a log for driver developer, perhaps it should be assert or similar that is disabled in the release build?

#2 lore raw

These are very useful insights Ferruh. I think RTE_LOG_DP() is something that would have been more suitable here.
Also, the DEBUG statement combined with a statistic will be more useful than ERR from developer perspective if they see a potential memory leak in their program.

Quoted 55 lines · Ferruh Yigit

drivers/net/gve/gve_tx_dqo.c gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf

Quoted 18 lines · Rushil Gupta

This was causing failure for testpmd runs (for queues >=15) presumably due to flooding of logs due to descriptor ring being overwritten.

Fixes a01854 ("net/gve: fix dqo bug for chained descriptors")
---
 drivers/net/gve/gve_tx_dqo.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/gve/gve_tx_dqo.c +0 -0
index 1a8eb96ea9..30a1455b20 100644
@@ -116,7 +116,7 @@ gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf

**tx_pkts, uint16_t nb_pkts)

drivers/net/gve/gve_tx_dqo.c gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)

Quoted 4 lines
drivers/net/gve/gve_tx_dqo.c +0 -1
              first_sw_id = sw_id;
              do {
                      if (sw_ring[sw_id] != NULL)
-                             PMD_DRV_LOG(ERR, "Overwriting an entry in

sw_ring");

drivers/net/gve/gve_tx_dqo.c gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf · line 119

Quoted 1 line
drivers/net/gve/gve_tx_dqo.c +1 -0
+                             PMD_DRV_LOG(DEBUG, "Overwriting an entry

in sw_ring");

drivers/net/gve/gve_tx_dqo.c gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) · line 122

Quoted 3 lines
drivers/net/gve/gve_tx_dqo.c +0 -0
                      txd = &txr[tx_id];
                      sw_ring[sw_id] = tx_pkt;

Hi Rushil,

I will continue with this patch, BUT
logging in the datapath has potential problems like this, also it may have performance impact even log is not printed, because of additional check in the log function.

For datapath, you can prefer:
- Add log macros controlled by RTE_ETHDEV_DEBUG_RX & RTE_ETHDEV_DEBUG_TX flags
- Or use RTE_LOG_DP() macro which is compiled out if default log level is less than the log uses

Also another perspective for the logs, when end-user observes this bloat of logs, what she can do?
Can she change some driver arguments or environment variables to fix the issue, if not what is the point of the log?
If this is a condition that can occur dynamically based on received traffic and user doesn't have control on it, maybe driver should handle the error without log?
And if this is a log for driver developer, perhaps it should be assert or similar that is disabled in the release build?

#4 lore raw
In reply to Rushil Gupta
Quoted 9 lines · On Mon, 19 Feb 2024 02:44:35 +0000 Rushil Gupta <[email protected]> wrote:

This was causing failure for testpmd runs (for queues >=15) presumably due to flooding of logs due to descriptor ring being overwritten.

Fixes a01854 ("net/gve: fix dqo bug for chained descriptors")

Isn't this still an error. What about the descriptor overwritten is there an mbuf leak? Maybe a statistic would be better than a message?

#5 lore raw

I agree.
This bug has manifested for a while before I fixed it partially in "[PATCH] net/gve: fix dqo bug for chained descriptors"
However, for higher queue counts (> 13); we still see this behavior. I'll add a statistic.

Quoted 17 lines · Stephen Hemminger
Quoted 9 lines · On Mon, 19 Feb 2024 02:44:35 +0000 Rushil Gupta <[email protected]> wrote:

This was causing failure for testpmd runs (for queues >=15) presumably due to flooding of logs due to descriptor ring being overwritten.

Fixes a01854 ("net/gve: fix dqo bug for chained descriptors")

Isn't this still an error. What about the descriptor overwritten is there an mbuf leak?
Maybe a statistic would be better than a message?