Skip to contentSkip to conversation

net/vhost: clear data of packet mbuf after sending pkts

PATCH In review Yuying Zhang 5 participants 8 messages Mar 1, 2022 – Mar 3, 2022 Network drivers

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 net/vhost: clear data of packet mbuf after sending pkts

The PMD frees a packet mbuf back into its original mempool after sending a packet. However, old data is not cleaned up which causes error in payload of new packets. This patch clear data of packet mbuf before freeing mbuf.

Fixes ee584e9710b9 ("vhost: add driver on top of the library")
---
 drivers/net/vhost/rte_eth_vhost.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)
drivers/net/vhost/rte_eth_vhost.c +8 -2
index 070f0e6dfd..92ed07a334 100644
@@ -417,10 +417,11 @@ static uint16_t
 eth_vhost_tx(void *q, struct rte_mbuf **bufs, uint16_t nb_bufs)
 {
 	struct vhost_queue *r = q;
-	uint16_t i, nb_tx = 0;
+	uint16_t i, j, nb_tx = 0;
 	uint16_t nb_send = 0;
 	uint64_t nb_bytes = 0;
 	uint64_t nb_missed = 0;
+	void *data = NULL;
 
 	if (unlikely(rte_atomic32_read(&r->allow_queuing) == 0))
 		return 0;
@@ -483,8 +484,13 @@ eth_vhost_tx(void *q, struct rte_mbuf **bufs, uint16_t nb_bufs)
 	for (i = nb_tx; i < nb_bufs; i++)
 		vhost_count_xcast_packets(r, bufs[i]);
 
-	for (i = 0; likely(i < nb_tx); i++)
+	for (i = 0; likely(i < nb_tx); i++) {
+		for (j = 0; j < bufs[i]->nb_segs; j++) {
+			data = rte_pktmbuf_mtod(bufs[i], void *);
+			memset(data, 0, bufs[i]->data_len);
+		}
 		rte_pktmbuf_free(bufs[i]);
+	}
 out:
 	rte_atomic32_set(&r->while_queuing, 0);
 
Signature · 1 line
2.25.1
Tested-by on v1 1/1
#1 lore raw
Quoted 12 lines

-----Original Message-----
From: Yuying Zhang <[email protected]>
Sent: Tuesday, March 1, 2022 3:28 PM
To: [email protected]; [email protected]; Xia, Chenbo <[email protected]>
Cc: Zhang, Yuying <[email protected]>; [email protected]
Subject: [PATCH v1] net/vhost: clear data of packet mbuf after sending pkts

The PMD frees a packet mbuf back into its original mempool after sending a packet. However, old data is not cleaned up which causes error in payload of new packets. This patch clear data of packet mbuf before freeing mbuf.

Quoted 5 lines
Fixes ee584e9710b9 ("vhost: add driver on top of the library")

---

#2 lore raw
In reply to Yuying Zhang
Quoted 5 lines · Yuying Zhang

The PMD frees a packet mbuf back into its original mempool after sending a packet. However, old data is not cleaned up which causes error in payload of new packets. This patch clear data of packet mbuf before freeing mbuf.

This patch looks wrong to me.
What is the actual issue you want to fix?

Signature · 1 line
David Marchand
#3 lore raw

Hi Marchand,

Quoted 10 lines

-----Original Message-----
From: David Marchand <[email protected]>
Sent: Tuesday, March 1, 2022 4:44 PM
To: Zhang, Yuying <[email protected]>
Cc: dev <[email protected]>; Maxime Coquelin <[email protected]>; Xia, Chenbo <[email protected]>; dpdk stable <[email protected]>
Subject: Re: [PATCH v1] net/vhost: clear data of packet mbuf after sending pkts

Quoted 1 line · Yuying Zhang
Quoted 7 lines
Quoted 4 lines

The PMD frees a packet mbuf back into its original mempool after sending a packet. However, old data is not cleaned up which causes error in payload of new packets. This patch clear data of packet mbuf before freeing mbuf.

This patch looks wrong to me.
What is the actual issue you want to fix?

eth_vhost_tx() frees the packet mbuf back into its original mempool every time after a packet sent without clearing the data field. Then packet transmit function will get bulk directly without reset. New generated packet contains old data of previous packet. This is wrong.

Quoted 4 lines

--
David Marchand

#4 lore raw
Quoted 15 lines · Yuying
Quoted 15 lines

-----Original Message-----
From: David Marchand <[email protected]>
Sent: Tuesday, March 1, 2022 4:44 PM
To: Zhang, Yuying <[email protected]>
Cc: dev <[email protected]>; Maxime Coquelin <[email protected]>; Xia, Chenbo <[email protected]>; dpdk stable <[email protected]>
Subject: Re: [PATCH v1] net/vhost: clear data of packet mbuf after sending pkts

Quoted 5 lines · Yuying Zhang

The PMD frees a packet mbuf back into its original mempool after sending a packet. However, old data is not cleaned up which causes error in payload of new packets. This patch clear data of packet mbuf before freeing mbuf.

Quoted 5 lines
Quoted 2 lines

This patch looks wrong to me.
What is the actual issue you want to fix?

eth_vhost_tx() frees the packet mbuf back into its original mempool every time after a packet sent without clearing the data field. Then packet transmit function will get bulk directly without reset. New generated packet contains old data of previous packet. This is wrong.

With the proposed patch, if the mbuf refcnt is != 0, you are shooting the data while some other part of the application might be needing it.

Plus, there should be no expectation about a mbuf data content when retrieving one from a mempool.
The only bytes that are guaranteed to be initialised by the mbuf API are its metadata.

If there is an issue somewhere in dpdk where the mbuf data content is expected to be 0 on allocation, please point at it.
Or share the full test that failed.

Signature · 1 line
David Marchand
#5 lore raw
Quoted 31 lines · On Tue, 1 Mar 2022 10:47:32 +0100 David Marchand <[email protected]>…
Quoted 15 lines · Yuying
Quoted 15 lines

-----Original Message-----
From: David Marchand <[email protected]>
Sent: Tuesday, March 1, 2022 4:44 PM
To: Zhang, Yuying <[email protected]>
Cc: dev <[email protected]>; Maxime Coquelin <[email protected]>; Xia, Chenbo <[email protected]>; dpdk stable <[email protected]>
Subject: Re: [PATCH v1] net/vhost: clear data of packet mbuf after sending pkts

Quoted 5 lines · Yuying Zhang

The PMD frees a packet mbuf back into its original mempool after sending a packet. However, old data is not cleaned up which causes error in payload of new packets. This patch clear data of packet mbuf before freeing mbuf.

Quoted 5 lines
Quoted 2 lines

This patch looks wrong to me.
What is the actual issue you want to fix?

eth_vhost_tx() frees the packet mbuf back into its original mempool every time after a packet sent without clearing the data field. Then packet transmit function will get bulk directly without reset. New generated packet contains old data of previous packet. This is wrong.

With the proposed patch, if the mbuf refcnt is != 0, you are shooting the data while some other part of the application might be needing it.

Plus, there should be no expectation about a mbuf data content when retrieving one from a mempool.
The only bytes that are guaranteed to be initialised by the mbuf API are its metadata.

Quoted 5 lines

If there is an issue somewhere in dpdk where the mbuf data content is expected to be 0 on allocation, please point at it.
Or share the full test that failed.

Agree. There is no guarantee that mbuf you get was not just used by some other driver or library. Only the fields in rte_pktmbuf_reset are guaranteed to be set.

#6 lore raw
In reply to David Marchand

Hi Marchand,

Quoted 10 lines

-----Original Message-----
From: David Marchand <[email protected]>
Sent: Tuesday, March 1, 2022 5:48 PM
To: Zhang, Yuying <[email protected]>
Cc: dev <[email protected]>; Maxime Coquelin <[email protected]>; Xia, Chenbo <[email protected]>; dpdk stable <[email protected]>
Subject: Re: [PATCH v1] net/vhost: clear data of packet mbuf after sending pkts

On Tue, Mar 1, 2022 at 10:02 AM Zhang, Yuying <[email protected]> wrote:

...

Quoted 15 lines
Quoted 2 lines

eth_vhost_tx() frees the packet mbuf back into its original mempool every

time after a packet sent without clearing the data field.

Quoted 1 line

Then packet transmit function will get bulk directly without reset. New

generated packet contains old data of previous packet. This is wrong.

With the proposed patch, if the mbuf refcnt is != 0, you are shooting the data while some other part of the application might be needing it.

Plus, there should be no expectation about a mbuf data content when retrieving one from a mempool.
The only bytes that are guaranteed to be initialised by the mbuf API are its metadata.

Quoted 3 lines

If there is an issue somewhere in dpdk where the mbuf data content is expected to be 0 on allocation, please point at it.
Or share the full test that failed.

According to the test_plan guide of dpdk (https://doc.dpdk.org/dts/test_plans/loopback_virtio_user_server_mode_test_plan.html),
Test Case 13 (loopback packed ring all path payload check test using server mode and multi-queues), the payload of each packet must be the same. The packet of first stream is initialized value 0. Then this packet is put back into mempool(actually, the local cache of the core). The packet of rest stream is got from local_cache directly and contains the first packet's header data in the payload. Therefore, the payload of the packets are different.

Quoted 4 lines

--
David Marchand

#7 lore raw
Quoted 48 lines

-----Original Message-----
From: Zhang, Yuying <[email protected]>
Sent: Wednesday, March 2, 2022 4:59 PM
To: David Marchand <[email protected]>
Cc: dev <[email protected]>; Maxime Coquelin <[email protected]>; Xia, Chenbo <[email protected]>; dpdk stable <[email protected]>; [email protected]
Subject: RE: [PATCH v1] net/vhost: clear data of packet mbuf after sending pkts

Hi Marchand,

Quoted 7 lines

-----Original Message-----
From: David Marchand <[email protected]>
Sent: Tuesday, March 1, 2022 5:48 PM
To: Zhang, Yuying <[email protected]>
Cc: dev <[email protected]>; Maxime Coquelin <[email protected]>; Xia, Chenbo <[email protected]>; dpdk stable <[email protected]>
Subject: Re: [PATCH v1] net/vhost: clear data of packet mbuf after sending

pkts

Quoted 3 lines

On Tue, Mar 1, 2022 at 10:02 AM Zhang, Yuying <[email protected]> wrote:

...

Quoted 7 lines
Quoted 2 lines

eth_vhost_tx() frees the packet mbuf back into its original mempool every

time after a packet sent without clearing the data field.

Quoted 1 line

Then packet transmit function will get bulk directly without reset. New

generated packet contains old data of previous packet. This is wrong.

With the proposed patch, if the mbuf refcnt is != 0, you are shooting the

data

Quoted 3 lines

while some other part of the application might be needing it.

Plus, there should be no expectation about a mbuf data content when

retrieving

Quoted 6 lines

one from a mempool.
The only bytes that are guaranteed to be initialised by the mbuf API are its metadata.

If there is an issue somewhere in dpdk where the mbuf data content is

expected

Quoted 2 lines

to be 0 on allocation, please point at it.
Or share the full test that failed.

Quoted 11 lines

According to the test_plan guide of dpdk
(https://doc.dpdk.org/dts/test_plans/loopback_virtio_user_server_mode_test_pla
n.html),
Test Case 13 (loopback packed ring all path payload check test using server mode and multi-queues), the payload of each packet must be the same. The packet of first stream is initialized value 0. Then this packet is put back into mempool(actually, the local cache of the core).
The packet of rest stream is got from local_cache directly and contains the first packet's header data in the payload. Therefore, the payload of the packets
are different.

Could you explain more about the problem?

But anyway I think this fix is wrong. After we're clear about the problem, there should be another solution.

Thanks,
Chenbo

Quoted 5 lines
Quoted 5 lines

--
David Marchand