Skip to contentSkip to conversation

app/testpmd: fix issue with memory leaks when quit testpmd

PATCH Superseded Ke Zhang 5 participants 9 messages Mar 1, 2022 – Jun 8, 2022

lore
Share & tools
lore

Ke Zhang

author
#0 lore raw

PATCH 1/1 app/testpmd: fix issue with memory leaks when quit testpmd

when dpdk is compiled in ASan, there is a memory leaks after quit testpmd if set mcast_addr, this patch fix this issue.

Error info as following:
ERROR: LeakSanitizer: detected memory leaksDirect leak of

       192 byte(s)
0 0x7f6a2e0aeffe in __interceptor_realloc
	(/lib/x86_64-linux-gnu/libasan.so.5+0x10dffe)
1 0x565361eb340f in mcast_addr_pool_extend
	../app/test-pmd/config.c:5162
2 0x565361eb3556 in mcast_addr_pool_append
	../app/test-pmd/config.c:5180
3 0x565361eb3aae in mcast_addr_add
	../app/test-pmd/config.c:5243
---
 app/test-pmd/testpmd.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)
app/test-pmd/testpmd.c +12 -0
index fe2ce19f99..fa7f80fdf7 100644
@@ -2742,6 +2742,8 @@ start_port(portid_t pid)
 			continue;
 		}
 
+		port->mc_addr_pool = NULL;
+
 		if (port->need_reconfig > 0) {
 			struct rte_eth_conf dev_conf;
 			int k;
@@ -3065,6 +3067,16 @@ stop_port(portid_t pid)
 		if (eth_dev_stop_mp(pi) != 0)
 			RTE_LOG(ERR, EAL, "rte_eth_dev_stop failed for port %u\n",
 				pi);
+		/*
+		 * free the pool of multicast addresses. If it is NULL,
+		 * it means there is no mc addr.Make sure the mc_addr_pool
+		 * is NULL at port init.
+		 */
+		if (port->mc_addr_pool != NULL) {
+			free(port->mc_addr_pool);
+			port->mc_addr_pool = NULL;
+		}
+		port->mc_addr_nb = 0;
 
 		if (port->port_status == RTE_PORT_HANDLING)
 			port->port_status = RTE_PORT_STOPPED;
Signature · 1 line
2.25.1
#1 lore raw
Quoted 25 lines · Ke Zhang

when dpdk is compiled in ASan, there is a memory leaks after quit testpmd if set mcast_addr, this patch fix this issue.

Error info as following:
ERROR: LeakSanitizer: detected memory leaksDirect leak of

        192 byte(s)
0 0x7f6a2e0aeffe in __interceptor_realloc
	(/lib/x86_64-linux-gnu/libasan.so.5+0x10dffe)
1 0x565361eb340f in mcast_addr_pool_extend
	../app/test-pmd/config.c:5162
2 0x565361eb3556 in mcast_addr_pool_append
	../app/test-pmd/config.c:5180
3 0x565361eb3aae in mcast_addr_add
	../app/test-pmd/config.c:5243
---
  app/test-pmd/testpmd.c | 12 ++++++++++++
  1 file changed, 12 insertions(+)
app/test-pmd/testpmd.c +0 -0
index fe2ce19f99..fa7f80fdf7 100644

app/test-pmd/testpmd.c start_port(portid_t pid) · line 2746

Quoted 5 lines
app/test-pmd/testpmd.c +2 -0
@@ -2742,6 +2742,8 @@ start_port(portid_t pid)
  			continue;
  		}
 
+		port->mc_addr_pool = NULL;
+

User can do consecutive start / stop / start / ..

Won't user expect the previous allocated and set MAC address to be valid after this?

What about to free the 'port->mc_addr_pool' in close() instead of stop()?

Quoted 7 lines
app/test-pmd/testpmd.c +0 -0
  		if (port->need_reconfig > 0) {
  			struct rte_eth_conf dev_conf;
  			int k;
@@ -3065,6 +3067,16 @@ stop_port(portid_t pid)
  		if (eth_dev_stop_mp(pi) != 0)
  			RTE_LOG(ERR, EAL, "rte_eth_dev_stop failed for port %u\n",
  				pi);

app/test-pmd/testpmd.c stop_port(portid_t pid) · line 3079

Quoted 10 lines
app/test-pmd/testpmd.c +10 -0
+		/*
+		 * free the pool of multicast addresses. If it is NULL,
+		 * it means there is no mc addr.Make sure the mc_addr_pool
+		 * is NULL at port init.
+		 */
+		if (port->mc_addr_pool != NULL) {
+			free(port->mc_addr_pool);
+			port->mc_addr_pool = NULL;
+		}
+		port->mc_addr_nb = 0;

will it work to use 'port->mc_addr_nb' to detect if the 'port->mc_addr_pool' pointer is valid

if (port->mc_addr_nb) {
free(port->mc_addr_pool);
port->mc_addr_pool = NULL;
port->mc_addr_nb = 0;
}

Quoted 3 lines
app/test-pmd/testpmd.c +0 -0
  		if (port->port_status == RTE_PORT_HANDLING)
  			port->port_status = RTE_PORT_STOPPED;
#2 lore raw
Quoted 45 lines

-----Original Message-----
From: Yigit, Ferruh <[email protected]>
Sent: Saturday, March 5, 2022 12:44 AM
To: Zhang, Ke1X <[email protected]>; Li, Xiaoyun <[email protected]>; Singh, Aman Deep <[email protected]>; Zhang, Yuying <[email protected]>; [email protected]
Subject: Re: [PATCH] app/testpmd: fix issue with memory leaks when quit testpmd

Quoted 25 lines · Ke Zhang

when dpdk is compiled in ASan, there is a memory leaks after quit testpmd if set mcast_addr, this patch fix this issue.

Error info as following:
ERROR: LeakSanitizer: detected memory leaksDirect leak of

        192 byte(s)
0 0x7f6a2e0aeffe in __interceptor_realloc
	(/lib/x86_64-linux-gnu/libasan.so.5+0x10dffe)
1 0x565361eb340f in mcast_addr_pool_extend
	../app/test-pmd/config.c:5162
2 0x565361eb3556 in mcast_addr_pool_append
	../app/test-pmd/config.c:5180
3 0x565361eb3aae in mcast_addr_add
	../app/test-pmd/config.c:5243
---
  app/test-pmd/testpmd.c | 12 ++++++++++++
  1 file changed, 12 insertions(+)
app/test-pmd/testpmd.c index renamed from app/test-pmd/testpmd.c +0 -0

fe2ce19f99..fa7f80fdf7 100644

app/test-pmd/testpmd.c start_port(portid_t pid) · line 2746

Quoted 5 lines
app/test-pmd/testpmd.c +2 -0
@@ -2742,6 +2742,8 @@ start_port(portid_t pid)
  			continue;
  		}
 
+		port->mc_addr_pool = NULL;
+

User can do consecutive start / stop / start / ..

Quoted 5 lines

Won't user expect the previous allocated and set MAC address to be valid after this?

What about to free the 'port->mc_addr_pool' in close() instead of stop()?

Thanks for your comments, I checked the code in dpdk, when start, the resource associated with port Will be allocated, when stop, the resource will be free from hardware and software struct.For example, In the function ice_dev_stop(), the rx and tx resource will be free, and also in start, resource like tx,
it will be allocated and request hardware to allocate the resource using ice_aqc_opc_add_txqs admQ
command, and the mc_addr_pool is a port resource. So what's your opinion about this ?

Quoted 22 lines

app/test-pmd/testpmd.c stop_port(portid_t pid) · line 3068

Quoted 6 lines
app/test-pmd/testpmd.c +0 -0
  		if (port->need_reconfig > 0) {
  			struct rte_eth_conf dev_conf;
  			int k;
@@ -3065,6 +3067,16 @@ stop_port(portid_t pid)
  		if (eth_dev_stop_mp(pi) != 0)
  			RTE_LOG(ERR, EAL, "rte_eth_dev_stop failed for

port %u\n",

app/test-pmd/testpmd.c stop_port(portid_t pid) · line 3079

Quoted 11 lines
app/test-pmd/testpmd.c +10 -0
  				pi);
+		/*
+		 * free the pool of multicast addresses. If it is NULL,
+		 * it means there is no mc addr.Make sure the mc_addr_pool
+		 * is NULL at port init.
+		 */
+		if (port->mc_addr_pool != NULL) {
+			free(port->mc_addr_pool);
+			port->mc_addr_pool = NULL;
+		}
+		port->mc_addr_nb = 0;

will it work to use 'port->mc_addr_nb' to detect if the 'port->mc_addr_pool' pointer is valid

Quoted 6 lines
if (port->mc_addr_nb) {
   free(port->mc_addr_pool);
   port->mc_addr_pool = NULL;
   port->mc_addr_nb = 0;
}

Yes , it is a better way using " if (port->mc_addr_nb) ", I will modify my code.

Quoted 3 lines
Quoted 3 lines
app/test-pmd/testpmd.c +0 -0
  		if (port->port_status == RTE_PORT_HANDLING)
  			port->port_status = RTE_PORT_STOPPED;
author
#3 lore raw

PATCH v2 1/1 app/testpmd: fix issue with memory leaks when quit testpmd

In reply to Ke Zhang

when dpdk is compiled in ASan, there is a memory leaks after quit testpmd if set mcast_addr, this patch fix this issue.

Error info as following:
ERROR: LeakSanitizer: detected memory leaksDirect leak of

       192 byte(s)
0 0x7f6a2e0aeffe in __interceptor_realloc
	(/lib/x86_64-linux-gnu/libasan.so.5+0x10dffe)
1 0x565361eb340f in mcast_addr_pool_extend
	../app/test-pmd/config.c:5162
2 0x565361eb3556 in mcast_addr_pool_append
	../app/test-pmd/config.c:5180
3 0x565361eb3aae in mcast_addr_add
	../app/test-pmd/config.c:5243
---
 app/test-pmd/testpmd.c | 6 ++++++
 1 file changed, 6 insertions(+)
app/test-pmd/testpmd.c +6 -0
index fe2ce19f99..f7e18aee25 100644
@@ -3136,6 +3136,12 @@ close_port(portid_t pid)
 			continue;
 		}
 
+		if (port->mc_addr_nb != 0) {
+			/* free the pool of multicast addresses. */
+			free(port->mc_addr_pool);
+			port->mc_addr_pool = NULL;
+		}
+
 		if (is_proc_primary()) {
 			port_flow_flush(pi);
 			port_flex_item_flush(pi);
Signature · 1 line
2.25.1
#4 lore raw
Quoted 3 lines · Ke Zhang

when dpdk is compiled in ASan, there is a memory leaks after quit testpmd if set mcast_addr, this patch fix this issue.

The memory leak is present regardless of ASan being compiled in. Plus, afaiu, the issue happens too when closing a port.

I'd rather rephrase like:
"""
A multicast address pool is allocated for a port when using mcast_addr testpmd commands.

When closing a port or stopping testpmd, this pool was not freed, resulting in a leak.
This issue has been caught using ASan.

Free this pool when closing the port.
"""

Quoted 13 lines

Error info as following:
ERROR: LeakSanitizer: detected memory leaksDirect leak of

       192 byte(s)
0 0x7f6a2e0aeffe in __interceptor_realloc
        (/lib/x86_64-linux-gnu/libasan.so.5+0x10dffe)
1 0x565361eb340f in mcast_addr_pool_extend
        ../app/test-pmd/config.c:5162
2 0x565361eb3556 in mcast_addr_pool_append
        ../app/test-pmd/config.c:5180
3 0x565361eb3aae in mcast_addr_add
        ../app/test-pmd/config.c:5243

This patch needs a Fixes: tag.

Quoted 10 lines
---
 app/test-pmd/testpmd.c | 6 ++++++
 1 file changed, 6 insertions(+)
app/test-pmd/testpmd.c +0 -0
index fe2ce19f99..f7e18aee25 100644

app/test-pmd/testpmd.c close_port(portid_t pid) · line 3144

Quoted 9 lines
app/test-pmd/testpmd.c +6 -0
@@ -3136,6 +3136,12 @@ close_port(portid_t pid)
                        continue;
                }
 
+               if (port->mc_addr_nb != 0) {
+                       /* free the pool of multicast addresses. */
+                       free(port->mc_addr_pool);
+                       port->mc_addr_pool = NULL;
+               }
+

Nit: I would introduce a helper in app/test-pmd/config.c, for example, mcast_addr_pool_destroy.

Quoted 6 lines
app/test-pmd/testpmd.c +0 -1
                if (is_proc_primary()) {
                        port_flow_flush(pi);
                        port_flex_item_flush(pi);
--

2.25.1

Signature · 1 line
David Marchand
#5 lore raw
In reply to Ke Zhang

Hi Ke,

Quoted 35 lines

-----Original Message-----
From: Zhang, Ke1X <[email protected]>
Sent: Monday, March 14, 2022 1:53 PM
To: Li, Xiaoyun <[email protected]>; Singh, Aman Deep <[email protected]>; Zhang, Yuying <[email protected]>; [email protected]
Cc: Zhang, Ke1X <[email protected]>
Subject: [PATCH v2] app/testpmd: fix issue with memory leaks when quit testpmd

when dpdk is compiled in ASan, there is a memory leaks after quit testpmd if set mcast_addr, this patch fix this issue.

Error info as following:
ERROR: LeakSanitizer: detected memory leaksDirect leak of

       192 byte(s)
0 0x7f6a2e0aeffe in __interceptor_realloc
	(/lib/x86_64-linux-gnu/libasan.so.5+0x10dffe)
1 0x565361eb340f in mcast_addr_pool_extend
	../app/test-pmd/config.c:5162
2 0x565361eb3556 in mcast_addr_pool_append
	../app/test-pmd/config.c:5180
3 0x565361eb3aae in mcast_addr_add
	../app/test-pmd/config.c:5243
---
 app/test-pmd/testpmd.c | 6 ++++++
 1 file changed, 6 insertions(+)
app/test-pmd/testpmd.c index renamed from app/test-pmd/testpmd.c +0 -0

fe2ce19f99..f7e18aee25 100644

app/test-pmd/testpmd.c close_port(portid_t pid) · line 3144

Quoted 9 lines
app/test-pmd/testpmd.c +6 -0
@@ -3136,6 +3136,12 @@ close_port(portid_t pid)
 			continue;
 		}
 
+		if (port->mc_addr_nb != 0) {
+			/* free the pool of multicast addresses. */
+			free(port->mc_addr_pool);
+			port->mc_addr_pool = NULL;
+		}
+

Is port->mc_addr_pool located in shared memory and may it be freed in primary process? BTW, you can write a function in config.c such as mcast_addr_pool_extend() and reference in close_port().

Quoted 5 lines
app/test-pmd/testpmd.c +0 -1
 		if (is_proc_primary()) {
 			port_flow_flush(pi);
 			port_flex_item_flush(pi);
--

2.25.1

author
#6 lore raw

PATCH v3 1/1 app/testpmd: fix issue with memory leaks when quit testpmd

In reply to Ke Zhang

A multicast address pool is allocated for a port when using mcast_addr testpmd commands.

When closing a port or stopping testpmd, this pool was not freed, resulting in a leak.This issue has been caught using ASan.

Free this pool when closing the port.

Error info as following:
ERROR: LeakSanitizer: detected memory leaksDirect leak of

       192 byte(s)
0 0x7f6a2e0aeffe in __interceptor_realloc
	(/lib/x86_64-linux-gnu/libasan.so.5+0x10dffe)
1 0x565361eb340f in mcast_addr_pool_extend
	../app/test-pmd/config.c:5162
2 0x565361eb3556 in mcast_addr_pool_append
	../app/test-pmd/config.c:5180
3 0x565361eb3aae in mcast_addr_add
	../app/test-pmd/config.c:5243
---
 app/test-pmd/config.c  | 19 +++++++++++++++++++
 app/test-pmd/testpmd.c |  1 +
 app/test-pmd/testpmd.h |  1 +
 3 files changed, 21 insertions(+)
app/test-pmd/config.c +19 -0
index cc8e7aa138..a6fa9be3ef 100644
@@ -5978,6 +5978,25 @@ mcast_addr_pool_remove(struct rte_port *port, uint32_t addr_idx)
 		sizeof(struct rte_ether_addr) * (port->mc_addr_nb - addr_idx));
 }
 
+int
+mcast_addr_pool_destroy(portid_t port_id)
+{
+	struct rte_port *port;
+
+	if (port_id_is_invalid(port_id, ENABLED_WARN) ||
+	    port_id == (portid_t)RTE_PORT_ALL)
+		return -EINVAL;
+	port = &ports[port_id];
+
+	if (port->mc_addr_nb != 0) {
+		/* free the pool of multicast addresses. */
+		free(port->mc_addr_pool);
+		port->mc_addr_pool = NULL;
+		port->mc_addr_nb = 0;
+	}
+	return 0;
+}
+
 static int
 eth_port_multicast_addr_list_set(portid_t port_id)
 {
app/test-pmd/testpmd.c +1 -0
index fe2ce19f99..1861a02c2f 100644
@@ -3137,6 +3137,7 @@ close_port(portid_t pid)
 		}
 
 		if (is_proc_primary()) {
+			mcast_addr_pool_destroy(pi);
 			port_flow_flush(pi);
 			port_flex_item_flush(pi);
 			rte_eth_dev_close(pi);
app/test-pmd/testpmd.h +1 -0
index 31f766c965..daa3c08317 100644
@@ -965,6 +965,7 @@ int port_flow_create(portid_t port_id,
 int port_action_handle_query(portid_t port_id, uint32_t id);
 void update_age_action_context(const struct rte_flow_action *actions,
 		     struct port_flow *pf);
+int mcast_addr_pool_destroy(portid_t port_id);
 int port_flow_destroy(portid_t port_id, uint32_t n, const uint32_t *rule);
 int port_flow_flush(portid_t port_id);
 int port_flow_dump(portid_t port_id, bool dump_all,
Signature · 1 line
2.25.1
Acked-by on v3 1/1
#7 lore raw

Hi Ke,

Quoted 30 lines

-----Original Message-----
From: Zhang, Ke1X <[email protected]>
Sent: Friday, March 25, 2022 4:36 PM
To: Li, Xiaoyun <[email protected]>; Singh, Aman Deep <[email protected]>; Zhang, Yuying <[email protected]>; [email protected]
Cc: Zhang, Ke1X <[email protected]>
Subject: [PATCH v3] app/testpmd: fix issue with memory leaks when quit testpmd

A multicast address pool is allocated for a port when using mcast_addr testpmd commands.

When closing a port or stopping testpmd, this pool was not freed, resulting in a leak.This issue has been caught using ASan.

Free this pool when closing the port.

Error info as following:
ERROR: LeakSanitizer: detected memory leaksDirect leak of

       192 byte(s)
0 0x7f6a2e0aeffe in __interceptor_realloc
	(/lib/x86_64-linux-gnu/libasan.so.5+0x10dffe)
1 0x565361eb340f in mcast_addr_pool_extend
	../app/test-pmd/config.c:5162
2 0x565361eb3556 in mcast_addr_pool_append
	../app/test-pmd/config.c:5180
3 0x565361eb3aae in mcast_addr_add
	../app/test-pmd/config.c:5243
Quoted 1 line
Quoted 60 lines

---
app/test-pmd/config.c | 19 +++++++++++++++++++ app/test-
pmd/testpmd.c | 1 + app/test-pmd/testpmd.h | 1 +

 3 files changed, 21 insertions(+)
app/test-pmd/config.c index renamed from app/test-pmd/config.c +0 -0

cc8e7aa138..a6fa9be3ef 100644

app/test-pmd/config.c +18 -0
@@ -5978,6 +5978,25 @@ mcast_addr_pool_remove(struct rte_port *port,
 uint32_t addr_idx)
 		sizeof(struct rte_ether_addr) * (port->mc_addr_nb -
 addr_idx));  }
 
+int
+mcast_addr_pool_destroy(portid_t port_id) {
+	struct rte_port *port;
+
+	if (port_id_is_invalid(port_id, ENABLED_WARN) ||
+	    port_id == (portid_t)RTE_PORT_ALL)
+		return -EINVAL;
+	port = &ports[port_id];
+
+	if (port->mc_addr_nb != 0) {
+		/* free the pool of multicast addresses. */
+		free(port->mc_addr_pool);
+		port->mc_addr_pool = NULL;
+		port->mc_addr_nb = 0;
+	}
+	return 0;
+}
+
 static int
 eth_port_multicast_addr_list_set(portid_t port_id)  { diff --git a/app/test-

pmd/testpmd.c b/app/test-pmd/testpmd.c index fe2ce19f99..1861a02c2f 100644

app/test-pmd/testpmd.c +1 -0
@@ -3137,6 +3137,7 @@ close_port(portid_t pid)
 		}
 
 		if (is_proc_primary()) {
+			mcast_addr_pool_destroy(pi);
 			port_flow_flush(pi);
 			port_flex_item_flush(pi);
 			rte_eth_dev_close(pi);
app/test-pmd/testpmd.h index renamed from app/test-pmd/testpmd.h +0 -0

31f766c965..daa3c08317 100644

app/test-pmd/testpmd.h +0 -0
@@ -965,6 +965,7 @@ int port_flow_create(portid_t port_id,  int

port_action_handle_query(portid_t port_id, uint32_t id); void update_age_action_context(const struct rte_flow_action *actions,

app/test-pmd/testpmd.h +1 -0
 		     struct port_flow *pf);
+int mcast_addr_pool_destroy(portid_t port_id);
 int port_flow_destroy(portid_t port_id, uint32_t n, const uint32_t *rule);  int

port_flow_flush(portid_t port_id); int port_flow_dump(portid_t port_id, bool dump_all,
--
2.25.1

Acked-by Applied
#8 lore raw
Quoted 32 lines · Yuying

Hi Ke,

Quoted 30 lines

-----Original Message-----
From: Zhang, Ke1X <[email protected]>
Sent: Friday, March 25, 2022 4:36 PM
To: Li, Xiaoyun <[email protected]>; Singh, Aman Deep <[email protected]>; Zhang, Yuying <[email protected]>; [email protected]
Cc: Zhang, Ke1X <[email protected]>
Subject: [PATCH v3] app/testpmd: fix issue with memory leaks when quit testpmd

A multicast address pool is allocated for a port when using mcast_addr testpmd commands.

When closing a port or stopping testpmd, this pool was not freed, resulting in a leak.This issue has been caught using ASan.

Free this pool when closing the port.

Error info as following:
ERROR: LeakSanitizer: detected memory leaksDirect leak of

        192 byte(s)
0 0x7f6a2e0aeffe in __interceptor_realloc
	(/lib/x86_64-linux-gnu/libasan.so.5+0x10dffe)
1 0x565361eb340f in mcast_addr_pool_extend
	../app/test-pmd/config.c:5162
2 0x565361eb3556 in mcast_addr_pool_append
	../app/test-pmd/config.c:5180
3 0x565361eb3aae in mcast_addr_add
	../app/test-pmd/config.c:5243
Quoted 4 lines
Quoted 1 line
Fixes 8fff667578a7 ("app/testpmd: new command to add/remove

multicast MAC addresses")
Cc: [email protected]

Applied to dpdk-next-net/main, thanks.

<...>

Quoted 7 lines
Quoted 7 lines

pmd/testpmd.c b/app/test-pmd/testpmd.c index fe2ce19f99..1861a02c2f 100644

app/test-pmd/testpmd.c +0 -0
@@ -3137,6 +3137,7 @@ close_port(portid_t pid)
  		}
 
Quoted 5 lines

app/test-pmd/testpmd.c close_port(portid_t pid) · line 3143

Quoted 5 lines
app/test-pmd/testpmd.c +1 -0
  		if (is_proc_primary()) {
+			mcast_addr_pool_destroy(pi);
  			port_flow_flush(pi);
  			port_flex_item_flush(pi);
  			rte_eth_dev_close(pi);

'rte_eth_dev_close()' may fail, like if the port is not stopped, in that case mcast pool will be freed although port is not closed.
This can be prevented by saving existing mcast poll and restore it if close failed, but I think it doesn't worth the complexity it brings, so OK to continue as it is.