Skip to contentSkip to conversation

vhost: add support to large linear mbufs

PATCH In review Flavio Leitner 6 participants 32 messages Oct 1, 2019 – Oct 29, 2019 Network drivers

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 vhost: add support to large linear mbufs

The rte_vhost_dequeue_burst supports two ways of dequeuing data. If the data fits into a buffer, then all data is copied and a single linear buffer is returned. Otherwise it allocates additional mbufs and chains them together to return a multiple segments mbuf.

While that covers most use cases, it forces applications that need to work with larger data sizes to support multiple segments mbufs. The non-linear characteristic brings complexity and performance implications to the application.

To resolve the issue, change the API so that the application can optionally provide a second mempool containing larger mbufs. If that is not provided (NULL), the behavior remains as before the change. Otherwise, the data size is checked and the corresponding mempool is used to return linear mbufs.

---
 drivers/net/vhost/rte_eth_vhost.c |  4 +--
 examples/tep_termination/main.c   |  2 +-
 examples/vhost/main.c             |  2 +-
 lib/librte_vhost/rte_vhost.h      |  5 ++-
 lib/librte_vhost/virtio_net.c     | 57 +++++++++++++++++++++++--------
 5 files changed, 50 insertions(+), 20 deletions(-)
drivers/net/vhost/rte_eth_vhost.c +2 -2
index 46f01a7f4..ce7f68a5b 100644
@@ -393,8 +393,8 @@ eth_vhost_rx(void *q, struct rte_mbuf **bufs, uint16_t nb_bufs)
 						 VHOST_MAX_PKT_BURST);
 
 		nb_pkts = rte_vhost_dequeue_burst(r->vid, r->virtqueue_id,
-						  r->mb_pool, &bufs[nb_rx],
-						  num);
+						  r->mb_pool, NULL,
+						  &bufs[nb_rx], num);
 
 		nb_rx += nb_pkts;
 		nb_receive -= nb_pkts;
examples/tep_termination/main.c +1 -1
index ab956ad7c..3ebf0fa6e 100644
@@ -697,7 +697,7 @@ switch_worker(__rte_unused void *arg)
 			if (likely(!vdev->remove)) {
 				/* Handle guest TX*/
 				tx_count = rte_vhost_dequeue_burst(vdev->vid,
-						VIRTIO_TXQ, mbuf_pool,
+						VIRTIO_TXQ, mbuf_pool, NULL,
 						pkts_burst, MAX_PKT_BURST);
 				/* If this is the first received packet we need to learn the MAC */
 				if (unlikely(vdev->ready == DEVICE_MAC_LEARNING) && tx_count) {
examples/vhost/main.c +1 -1
index ab649bf14..e9b306af3 100644
@@ -1092,7 +1092,7 @@ drain_virtio_tx(struct vhost_dev *vdev)
 					pkts, MAX_PKT_BURST);
 	} else {
 		count = rte_vhost_dequeue_burst(vdev->vid, VIRTIO_TXQ,
-					mbuf_pool, pkts, MAX_PKT_BURST);
+					mbuf_pool, NULL, pkts, MAX_PKT_BURST);
 	}
 
 	/* setup VMDq for the first packet */
lib/librte_vhost/rte_vhost.h +4 -1
index 19474bca0..b05fd8e2a 100644
@@ -593,6 +593,8 @@ uint16_t rte_vhost_enqueue_burst(int vid, uint16_t queue_id,
  *  virtio queue index in mq case
  * @param mbuf_pool
  *  mbuf_pool where host mbuf is allocated.
+ * @param mbuf_pool_large
+ *  mbuf_pool where larger host mbuf is allocated.
  * @param pkts
  *  array to contain packets to be dequeued
  * @param count
@@ -601,7 +603,8 @@ uint16_t rte_vhost_enqueue_burst(int vid, uint16_t queue_id,
  *  num of packets dequeued
  */
 uint16_t rte_vhost_dequeue_burst(int vid, uint16_t queue_id,
-	struct rte_mempool *mbuf_pool, struct rte_mbuf **pkts, uint16_t count);
+	struct rte_mempool *mbuf_pool, struct rte_mempool *mbuf_pool_large,
+	struct rte_mbuf **pkts, uint16_t count);
 
 /**
  * Get guest mem table: a list of memory regions.
lib/librte_vhost/virtio_net.c +42 -15

141 lines not on this page · raw message

Signature · 1 line
2.20.1
#1 lore raw
Quoted 10 lines · On Tue, 1 Oct 2019 19:19:35 -0300 Flavio Leitner <[email protected]> wrote:

The rte_vhost_dequeue_burst supports two ways of dequeuing data. If the data fits into a buffer, then all data is copied and a single linear buffer is returned. Otherwise it allocates additional mbufs and chains them together to return a multiple segments mbuf.

While that covers most use cases, it forces applications that need to work with larger data sizes to support multiple segments mbufs. The non-linear characteristic brings complexity and performance implications to the application.

Quoted 7 lines

To resolve the issue, change the API so that the application can optionally provide a second mempool containing larger mbufs. If that is not provided (NULL), the behavior remains as before the change. Otherwise, the data size is checked and the corresponding mempool is used to return linear mbufs.

The use case behind the patch is TSO support in DPDK accelerated Open vSwitch (customers are using ovs-dpdk to send big packets too!). The effort has been going on for many months now. There is a recent proposal in OvS dev mailing using the API as is today here:
https://mail.openvswitch.org/pipermail/ovs-dev/2019-September/362573.html

There is also a reply from me in that thread with the patches implementing a Proof-Of-Concept with the proposed API change:
https://mail.openvswitch.org/pipermail/ovs-dev/2019-September/362881.html

I opted to improve the current API instead of adding a new one, but I have no strong opinion either way.

I know it's a bit late for v19.11, but since the TSO support effort is going on for months now and OvS uses only LTS, I wonder if this can be part of v19.11.

Thanks,
fbl

#2 lore raw
In reply to Flavio Leitner

Wednesday, October 2, 2019 1:20 AM, Flavio Leitner:

Quoted 12 lines

Subject: [dpdk-dev] [PATCH] vhost: add support to large linear mbufs

The rte_vhost_dequeue_burst supports two ways of dequeuing data. If the data fits into a buffer, then all data is copied and a single linear buffer is returned. Otherwise it allocates additional mbufs and chains them together to return a multiple segments mbuf.

While that covers most use cases, it forces applications that need to work with larger data sizes to support multiple segments mbufs.
The non-linear characteristic brings complexity and performance implications to the application.

Quoted 5 lines

To resolve the issue, change the API so that the application can optionally provide a second mempool containing larger mbufs. If that is not provided (NULL), the behavior remains as before the change.
Otherwise, the data size is checked and the corresponding mempool is used to return linear mbufs.

I understand the motivation.
However, providing a static pool w/ large buffers is not so efficient in terms of memory footprint. You will need to prepare to worst case (all packet are large) w/ max size of 64KB. Also, the two mempools are quite restrictive as the memory fill of the mbufs might be very sparse. E.g. mempool1 mbuf.size = 1.5K , mempool2 mbuf.size = 64K, packet size 4KB.

Instead, how about using the mbuf external buffer feature?
The flow will be:
1. vhost PMD always receive a single mempool (like today)
2. on dequeue, PMD looks on the virtio packet size. If smaller than the mbuf size use the mbuf as is (like today)
3. otherwise, allocate a new buffer (inside the PMD) and link it to the mbuf as external buffer (rte_pktmbuf_attach_extbuf)

The pros of this approach is that you have full flexibility on the memory allocation, and therefore a lower footprint. The cons is the OVS will need to know how to handle mbuf w/ external buffers (not too complex IMO).

Quoted 262 lines
---
 drivers/net/vhost/rte_eth_vhost.c |  4 +--
 examples/tep_termination/main.c   |  2 +-
 examples/vhost/main.c             |  2 +-
 lib/librte_vhost/rte_vhost.h      |  5 ++-
 lib/librte_vhost/virtio_net.c     | 57 +++++++++++++++++++++++--------
 5 files changed, 50 insertions(+), 20 deletions(-)
drivers/net/vhost/rte_eth_vhost.c +2 -2
index 46f01a7f4..ce7f68a5b 100644
@@ -393,8 +393,8 @@ eth_vhost_rx(void *q, struct rte_mbuf **bufs,
 uint16_t nb_bufs)
 						 VHOST_MAX_PKT_BURST);
 
 		nb_pkts = rte_vhost_dequeue_burst(r->vid, r-
 >virtqueue_id,
-						  r->mb_pool, &bufs[nb_rx],
-						  num);
+						  r->mb_pool, NULL,
+						  &bufs[nb_rx], num);
 
drivers/net/vhost/rte_eth_vhost.c +0 -0
 		nb_rx += nb_pkts;
 		nb_receive -= nb_pkts;
(quoted diff) +0 -0

b/examples/tep_termination/main.c index ab956ad7c..3ebf0fa6e 100644

examples/tep_termination/main.c +1 -1
@@ -697,7 +697,7 @@ switch_worker(__rte_unused void *arg)
 			if (likely(!vdev->remove)) {
 				/* Handle guest TX*/
 				tx_count = rte_vhost_dequeue_burst(vdev-
 >vid,
-						VIRTIO_TXQ, mbuf_pool,
+						VIRTIO_TXQ, mbuf_pool,
 NULL,
 						pkts_burst,

MAX_PKT_BURST);
/* If this is the first received packet we need
to learn the MAC */
if (unlikely(vdev->ready ==
DEVICE_MAC_LEARNING) && tx_count) { diff --git a/examples/vhost/main.c b/examples/vhost/main.c index ab649bf14..e9b306af3 100644

examples/vhost/main.c +1 -1
@@ -1092,7 +1092,7 @@ drain_virtio_tx(struct vhost_dev *vdev)
 					pkts, MAX_PKT_BURST);
 	} else {
 		count = rte_vhost_dequeue_burst(vdev->vid, VIRTIO_TXQ,
-					mbuf_pool, pkts, MAX_PKT_BURST);
+					mbuf_pool, NULL, pkts,
 MAX_PKT_BURST);
 	}
 
examples/vhost/main.c +0 -0
 	/* setup VMDq for the first packet */
lib/librte_vhost/rte_vhost.h index renamed from lib/librte_vhost/rte_vhost.h +0 -0

19474bca0..b05fd8e2a 100644

lib/librte_vhost/rte_vhost.h +2 -0
@@ -593,6 +593,8 @@ uint16_t rte_vhost_enqueue_burst(int vid, uint16_t
 queue_id,
  *  virtio queue index in mq case
  * @param mbuf_pool
  *  mbuf_pool where host mbuf is allocated.
+ * @param mbuf_pool_large
+ *  mbuf_pool where larger host mbuf is allocated.
  * @param pkts
  *  array to contain packets to be dequeued

* @param count

lib/librte_vhost/rte_vhost.h +2 -1
@@ -601,7 +603,8 @@ uint16_t rte_vhost_enqueue_burst(int vid, uint16_t
 queue_id,
  *  num of packets dequeued
  */
 uint16_t rte_vhost_dequeue_burst(int vid, uint16_t queue_id,
-	struct rte_mempool *mbuf_pool, struct rte_mbuf **pkts, uint16_t
 count);
+	struct rte_mempool *mbuf_pool, struct rte_mempool
 *mbuf_pool_large,
+	struct rte_mbuf **pkts, uint16_t count);

/**
* Get guest mem table: a list of memory regions.

lib/librte_vhost/virtio_net.c index renamed from lib/librte_vhost/virtio_net.c +0 -0

5b85b832d..da9d77732 100644

lib/librte_vhost/virtio_net.c +3 -1
@@ -1291,10 +1291,12 @@ get_zmbuf(struct vhost_virtqueue *vq)
 
 static __rte_noinline uint16_t
 virtio_dev_tx_split(struct virtio_net *dev, struct vhost_virtqueue *vq,
-	struct rte_mempool *mbuf_pool, struct rte_mbuf **pkts, uint16_t
 count)
+	struct rte_mempool *mbuf_pool, struct rte_mempool
 *mbuf_pool_large,
+	struct rte_mbuf **pkts, uint16_t count)
 {
 	uint16_t i;
 	uint16_t free_entries;
+	uint16_t mbuf_avail;
 
lib/librte_vhost/virtio_net.c +16 -5
 	if (unlikely(dev->dequeue_zero_copy)) {
 		struct zcopy_mbuf *zmbuf, *next;
@@ -1340,32 +1342,42 @@ virtio_dev_tx_split(struct virtio_net *dev, struct
 vhost_virtqueue *vq,
 	VHOST_LOG_DEBUG(VHOST_DATA, "(%d) about to dequeue %u
 buffers\n",
 			dev->vid, count);
 
+	/* If the large mpool is provided, find the threshold */
+	mbuf_avail = 0;
+	if (mbuf_pool_large)
+		mbuf_avail = rte_pktmbuf_data_room_size(mbuf_pool) -
+RTE_PKTMBUF_HEADROOM;
+
 	for (i = 0; i < count; i++) {
 		struct buf_vector buf_vec[BUF_VECTOR_MAX];
 		uint16_t head_idx;
-		uint32_t dummy_len;
+		uint32_t buf_len;
 		uint16_t nr_vec = 0;
+		struct rte_mempool *mpool;
 		int err;
 
 		if (unlikely(fill_vec_buf_split(dev, vq,
 						vq->last_avail_idx + i,
 						&nr_vec, buf_vec,
-						&head_idx, &dummy_len,
+						&head_idx, &buf_len,
 						VHOST_ACCESS_RO) < 0))
 			break;
 
 		if (likely(dev->dequeue_zero_copy == 0))
 			update_shadow_used_ring_split(vq, head_idx, 0);
 
-		pkts[i] = rte_pktmbuf_alloc(mbuf_pool);
+		if (mbuf_pool_large && buf_len > mbuf_avail)
+			mpool = mbuf_pool_large;
+		else
+			mpool = mbuf_pool;
+
+		pkts[i] = rte_pktmbuf_alloc(mpool);
 		if (unlikely(pkts[i] == NULL)) {
 			RTE_LOG(ERR, VHOST_DATA,
 				"Failed to allocate memory for mbuf.\n");
 			break;
 		}
 
-		err = copy_desc_to_mbuf(dev, vq, buf_vec, nr_vec, pkts[i],
-				mbuf_pool);
+		err = copy_desc_to_mbuf(dev, vq, buf_vec, nr_vec, pkts[i],

mpool);

lib/librte_vhost/virtio_net.c +3 -1
 		if (unlikely(err)) {
 			rte_pktmbuf_free(pkts[i]);
 			break;
@@ -1411,9 +1423,11 @@ virtio_dev_tx_split(struct virtio_net *dev, struct
 vhost_virtqueue *vq,
 
 static __rte_noinline uint16_t
 virtio_dev_tx_packed(struct virtio_net *dev, struct vhost_virtqueue *vq,
-	struct rte_mempool *mbuf_pool, struct rte_mbuf **pkts, uint16_t
 count)
+	struct rte_mempool *mbuf_pool, struct rte_mempool
 *mbuf_pool_large,
+	struct rte_mbuf **pkts, uint16_t count)
 {
 	uint16_t i;
+	uint16_t mbuf_avail;
lib/librte_vhost/virtio_net.c +16 -5
 	if (unlikely(dev->dequeue_zero_copy)) {
 		struct zcopy_mbuf *zmbuf, *next;
@@ -1448,17 +1462,23 @@ virtio_dev_tx_packed(struct virtio_net *dev,
 struct vhost_virtqueue *vq,
 	VHOST_LOG_DEBUG(VHOST_DATA, "(%d) about to dequeue %u
 buffers\n",
 			dev->vid, count);
 
+	/* If the large mpool is provided, find the threshold */
+	mbuf_avail = 0;
+	if (mbuf_pool_large)
+		mbuf_avail = rte_pktmbuf_data_room_size(mbuf_pool) -
+RTE_PKTMBUF_HEADROOM;
+
 	for (i = 0; i < count; i++) {
 		struct buf_vector buf_vec[BUF_VECTOR_MAX];
 		uint16_t buf_id;
-		uint32_t dummy_len;
+		uint32_t buf_len;
 		uint16_t desc_count, nr_vec = 0;
+		struct rte_mempool *mpool;
 		int err;
 
 		if (unlikely(fill_vec_buf_packed(dev, vq,
 						vq->last_avail_idx,
 &desc_count,
 						buf_vec, &nr_vec,
-						&buf_id, &dummy_len,
+						&buf_id, &buf_len,
 						VHOST_ACCESS_RO) < 0))
 			break;
@@ -1466,15 +1486,19 @@ virtio_dev_tx_packed(struct virtio_net *dev,
 struct vhost_virtqueue *vq,
 			update_shadow_used_ring_packed(vq, buf_id, 0,
 					desc_count);
 
-		pkts[i] = rte_pktmbuf_alloc(mbuf_pool);
+		if (mbuf_pool_large && buf_len > mbuf_avail)
+			mpool = mbuf_pool_large;
+		else
+			mpool = mbuf_pool;
+
+		pkts[i] = rte_pktmbuf_alloc(mpool);
 		if (unlikely(pkts[i] == NULL)) {
 			RTE_LOG(ERR, VHOST_DATA,
 				"Failed to allocate memory for mbuf.\n");
 			break;
 		}
 
-		err = copy_desc_to_mbuf(dev, vq, buf_vec, nr_vec, pkts[i],
-				mbuf_pool);
+		err = copy_desc_to_mbuf(dev, vq, buf_vec, nr_vec, pkts[i],
 mpool);
 		if (unlikely(err)) {
lib/librte_vhost/virtio_net.c +2 -1
 			rte_pktmbuf_free(pkts[i]);
 			break;
@@ -1526,7 +1550,8 @@ virtio_dev_tx_packed(struct virtio_net *dev, struct
 vhost_virtqueue *vq,
 
 uint16_t
 rte_vhost_dequeue_burst(int vid, uint16_t queue_id,
-	struct rte_mempool *mbuf_pool, struct rte_mbuf **pkts, uint16_t
 count)
+	struct rte_mempool *mbuf_pool, struct rte_mempool
 *mbuf_pool_large,
+	struct rte_mbuf **pkts, uint16_t count)

{

lib/librte_vhost/virtio_net.c +3 -2
 	struct virtio_net *dev;
 	struct rte_mbuf *rarp_mbuf = NULL;
@@ -1598,9 +1623,11 @@ rte_vhost_dequeue_burst(int vid, uint16_t
 queue_id,
 	}
 
 	if (vq_is_packed(dev))
-		count = virtio_dev_tx_packed(dev, vq, mbuf_pool, pkts,
 count);
+		count = virtio_dev_tx_packed(dev, vq, mbuf_pool,
 mbuf_pool_large, pkts,
+                                     count);
 	else
-		count = virtio_dev_tx_split(dev, vq, mbuf_pool, pkts, count);
+		count = virtio_dev_tx_split(dev, vq, mbuf_pool,

mbuf_pool_large, pkts,

lib/librte_vhost/virtio_net.c +1 -1
+                                    count);
 
 out:
 	if (dev->features & (1ULL << VIRTIO_F_IOMMU_PLATFORM))
--

2.20.1

#3 lore raw

Hello Shahaf,

Quoted 24 lines · Shahaf Shuler

Wednesday, October 2, 2019 1:20 AM, Flavio Leitner:

Quoted 12 lines

Subject: [dpdk-dev] [PATCH] vhost: add support to large linear mbufs

The rte_vhost_dequeue_burst supports two ways of dequeuing data. If the data fits into a buffer, then all data is copied and a single linear buffer is returned. Otherwise it allocates additional mbufs and chains them together to return a multiple segments mbuf.

While that covers most use cases, it forces applications that need to work with larger data sizes to support multiple segments mbufs.
The non-linear characteristic brings complexity and performance implications to the application.

Quoted 5 lines

To resolve the issue, change the API so that the application can optionally provide a second mempool containing larger mbufs. If that is not provided (NULL), the behavior remains as before the change.
Otherwise, the data size is checked and the corresponding mempool is used to return linear mbufs.

I understand the motivation.
However, providing a static pool w/ large buffers is not so efficient in terms of memory footprint. You will need to prepare to worst case (all packet are large) w/ max size of 64KB. Also, the two mempools are quite restrictive as the memory fill of the mbufs might be very sparse. E.g. mempool1 mbuf.size = 1.5K , mempool2 mbuf.size = 64K, packet size 4KB.

Quoted 5 lines

Instead, how about using the mbuf external buffer feature?
The flow will be:
1. vhost PMD always receive a single mempool (like today)
2. on dequeue, PMD looks on the virtio packet size. If smaller than the mbuf size use the mbuf as is (like today)
3. otherwise, allocate a new buffer (inside the PMD) and link it to the mbuf as external buffer (rte_pktmbuf_attach_extbuf)

I am missing some piece here.
Which pool would the PMD take those external buffers from?

If it is from an additional mempool passed to the vhost pmd, I can't see the difference with Flavio proposal.

Quoted 2 lines

The pros of this approach is that you have full flexibility on the memory allocation, and therefore a lower footprint. The cons is the OVS will need to know how to handle mbuf w/ external buffers (not too complex IMO).

Signature · 1 line
David Marchand
#4 lore raw

Wednesday, October 2, 2019 11:05 AM, David Marchand:

Quoted 42 lines

Subject: Re: [dpdk-dev] [PATCH] vhost: add support to large linear mbufs

Hello Shahaf,

Quoted 14 lines · Shahaf Shuler

Wednesday, October 2, 2019 1:20 AM, Flavio Leitner:

Quoted 12 lines

Subject: [dpdk-dev] [PATCH] vhost: add support to large linear mbufs

The rte_vhost_dequeue_burst supports two ways of dequeuing data. If the data fits into a buffer, then all data is copied and a single linear buffer is returned. Otherwise it allocates additional mbufs and chains them together to return a multiple segments mbuf.

While that covers most use cases, it forces applications that need to work with larger data sizes to support multiple segments mbufs. The non-linear characteristic brings complexity and performance implications to the application.

Quoted 8 lines
Quoted 5 lines

To resolve the issue, change the API so that the application can optionally provide a second mempool containing larger mbufs. If that is not provided (NULL), the behavior remains as before the change. Otherwise, the data size is checked and the corresponding mempool is used to return linear mbufs.

I understand the motivation.
However, providing a static pool w/ large buffers is not so efficient in terms

of memory footprint. You will need to prepare to worst case (all packet are large) w/ max size of 64KB.

Quoted 1 line

Also, the two mempools are quite restrictive as the memory fill of the

mbufs might be very sparse. E.g. mempool1 mbuf.size = 1.5K , mempool2 mbuf.size = 64K, packet size 4KB.

Quoted 8 lines

Instead, how about using the mbuf external buffer feature? The flow will be:
1. vhost PMD always receive a single mempool (like today) 2. on dequeue, PMD looks on the virtio packet size. If smaller than the mbuf size use the mbuf as is (like today) 3. otherwise, allocate a new buffer (inside the PMD) and link it to the mbuf as external buffer (rte_pktmbuf_attach_extbuf)

Quoted 2 lines

I am missing some piece here.
Which pool would the PMD take those external buffers from?

The mbuf is always taken from the single mempool associated w/ the rxq.
The buffer for the mbuf may be allocated (in case virtio payload is bigger than current mbuf size) from DPDK hugepages or any other system memory and be attached to the mbuf.

You can see example implementation of it in mlx5 PMD (checkout rte_pktmbuf_attach_extbuf call)

Quoted 13 lines

If it is from an additional mempool passed to the vhost pmd, I can't see the difference with Flavio proposal.

Quoted 1 line

The pros of this approach is that you have full flexibility on the memory

allocation, and therefore a lower footprint.

Quoted 1 line

The cons is the OVS will need to know how to handle mbuf w/ external

buffers (not too complex IMO).

--
David Marchand

#5 lore raw

Hi Shahaf,

Thanks for looking into this, see my inline comments.

Quoted 51 lines · On Wed, 2 Oct 2019 09:00:11 +0000 Shahaf Shuler <[email protected]> wrote:

Wednesday, October 2, 2019 11:05 AM, David Marchand:

Quoted 47 lines

Subject: Re: [dpdk-dev] [PATCH] vhost: add support to large linear mbufs

Hello Shahaf,

Quoted 22 lines · Shahaf Shuler

Wednesday, October 2, 2019 1:20 AM, Flavio Leitner:

Quoted 14 lines

Subject: [dpdk-dev] [PATCH] vhost: add support to large linear mbufs

The rte_vhost_dequeue_burst supports two ways of dequeuing data. If the data fits into a buffer, then all data is copied and a single linear buffer is returned. Otherwise it allocates additional mbufs and chains them together to return a multiple segments mbuf.

While that covers most use cases, it forces applications that need to work with larger data sizes to support multiple segments mbufs. The non-linear characteristic brings complexity and performance implications to the application.

Quoted 5 lines

To resolve the issue, change the API so that the application can optionally provide a second mempool containing larger mbufs. If that is not provided (NULL), the behavior remains as before the change. Otherwise, the data size is checked and the corresponding mempool is used to return linear mbufs.

Quoted 3 lines

I understand the motivation.
However, providing a static pool w/ large buffers is not so efficient in terms

of memory footprint. You will need to prepare to worst case (all packet are large) w/ max size of 64KB.

Quoted 2 lines

Also, the two mempools are quite restrictive as the memory fill of the

mbufs might be very sparse. E.g. mempool1 mbuf.size = 1.5K , mempool2 mbuf.size = 64K, packet size 4KB.

Quoted 8 lines

Instead, how about using the mbuf external buffer feature? The flow will be:
1. vhost PMD always receive a single mempool (like today) 2. on dequeue, PMD looks on the virtio packet size. If smaller than the mbuf size use the mbuf as is (like today) 3. otherwise, allocate a new buffer (inside the PMD) and link it to the mbuf as external buffer (rte_pktmbuf_attach_extbuf)

Quoted 2 lines

I am missing some piece here.
Which pool would the PMD take those external buffers from?

Quoted 7 lines

The mbuf is always taken from the single mempool associated w/ the rxq. The buffer for the mbuf may be allocated (in case virtio payload is bigger than current mbuf size) from DPDK hugepages or any other system memory and be attached to the mbuf.

You can see example implementation of it in mlx5 PMD (checkout rte_pktmbuf_attach_extbuf call)

Thanks, I wasn't aware of external buffers.

I see that attaching external buffers of the correct size would be more efficient in terms of saving memory/avoiding sparsing.

However, we still need to be prepared to the worse case scenario (all packets 64K), so that doesn't help with the total memory required.

The current patch pushes the decision to the application which knows better the workload. If more memory is available, it can optionally use large buffers, otherwise just don't pass that. Or even decide whether to share the same 64K mempool between multiple vhost ports or use one mempool per port.

Perhaps I missed something, but managing memory with mempool still require us to have buffers of 64K regardless if the data consumes less space. Otherwise the application or the PMD will have to manage memory itself.

If we let the PMD manages the memory, what happens if a port/queue is closed and one or more buffers are still in use (switching)? I don't see how to solve this cleanly.

fbl

Quoted 16 lines
Quoted 16 lines

If it is from an additional mempool passed to the vhost pmd, I can't see the difference with Flavio proposal.

Quoted 2 lines

The pros of this approach is that you have full flexibility on the memory

allocation, and therefore a lower footprint.

Quoted 2 lines

The cons is the OVS will need to know how to handle mbuf w/ external

buffers (not too complex IMO).

--
David Marchand

#6 lore raw

Wednesday, October 2, 2019 3:59 PM, Flavio Leitner:

Quoted 21 lines

Obrembski MichalX <[email protected]>; Stokes Ian <[email protected]>
Subject: Re: [dpdk-dev] [PATCH] vhost: add support to large linear mbufs

Hi Shahaf,

Thanks for looking into this, see my inline comments.

Quoted 9 lines · On Wed, 2 Oct 2019 09:00:11 +0000 Shahaf Shuler <[email protected]> wrote:

Wednesday, October 2, 2019 11:05 AM, David Marchand:

Quoted 8 lines

Subject: Re: [dpdk-dev] [PATCH] vhost: add support to large linear mbufs

Hello Shahaf,

Quoted 1 line · Shahaf Shuler

[...]

Quoted 12 lines
Quoted 11 lines
Quoted 3 lines

I am missing some piece here.
Which pool would the PMD take those external buffers from?

The mbuf is always taken from the single mempool associated w/ the rxq. The buffer for the mbuf may be allocated (in case virtio payload is bigger than current mbuf size) from DPDK hugepages or any other system memory and be attached to the mbuf.

You can see example implementation of it in mlx5 PMD (checkout rte_pktmbuf_attach_extbuf call)

Quoted 7 lines

Thanks, I wasn't aware of external buffers.

I see that attaching external buffers of the correct size would be more efficient in terms of saving memory/avoiding sparsing.

However, we still need to be prepared to the worse case scenario (all packets 64K), so that doesn't help with the total memory required.

Am not sure why.
The allocation can be per demand. That is - only when you encounter a large buffer.

Having buffer allocated in advance will benefit only from removing the cost of the rte_*malloc. However on such big buffers, and further more w/ device offloads like TSO, am not sure that is an issue.

Quoted 6 lines

The current patch pushes the decision to the application which knows better the workload. If more memory is available, it can optionally use large buffers, otherwise just don't pass that. Or even decide whether to share the same 64K mempool between multiple vhost ports or use one mempool per port.

Quoted 7 lines

Perhaps I missed something, but managing memory with mempool still require us to have buffers of 64K regardless if the data consumes less space. Otherwise the application or the PMD will have to manage memory itself.

If we let the PMD manages the memory, what happens if a port/queue is closed and one or more buffers are still in use (switching)? I don't see how to solve this cleanly.

Closing of the dev should return EBUSY till all buffers are free.
What is the use case of closing a port while still having packet pending on other port of the switch? And why we cannot wait for them to complete transmission?

Quoted 19 lines

fbl

Quoted 16 lines
Quoted 16 lines

If it is from an additional mempool passed to the vhost pmd, I can't see the difference with Flavio proposal.

Quoted 2 lines

The pros of this approach is that you have full flexibility on the memory

allocation, and therefore a lower footprint.

Quoted 2 lines

The cons is the OVS will need to know how to handle mbuf w/ external

buffers (not too complex IMO).

--
David Marchand

#7 lore raw
Quoted 47 lines · On Wed, 2 Oct 2019 17:50:41 +0000 Shahaf Shuler <[email protected]> wrote:

Wednesday, October 2, 2019 3:59 PM, Flavio Leitner:

Quoted 22 lines

Obrembski MichalX <[email protected]>; Stokes Ian <[email protected]>
Subject: Re: [dpdk-dev] [PATCH] vhost: add support to large linear mbufs

Hi Shahaf,

Thanks for looking into this, see my inline comments.

Quoted 9 lines · On Wed, 2 Oct 2019 09:00:11 +0000 Shahaf Shuler <[email protected]> wrote:

Wednesday, October 2, 2019 11:05 AM, David Marchand:

Quoted 8 lines

Subject: Re: [dpdk-dev] [PATCH] vhost: add support to large linear mbufs

Hello Shahaf,

Quoted 1 line · Shahaf Shuler

[...]

Quoted 12 lines
Quoted 11 lines
Quoted 3 lines

I am missing some piece here.
Which pool would the PMD take those external buffers from?

The mbuf is always taken from the single mempool associated w/ the rxq. The buffer for the mbuf may be allocated (in case virtio payload is bigger than current mbuf size) from DPDK hugepages or any other system memory and be attached to the mbuf.

You can see example implementation of it in mlx5 PMD (checkout rte_pktmbuf_attach_extbuf call)

Quoted 8 lines

Thanks, I wasn't aware of external buffers.

I see that attaching external buffers of the correct size would be more efficient in terms of saving memory/avoiding sparsing.

However, we still need to be prepared to the worse case scenario (all packets 64K), so that doesn't help with the total memory required.

Quoted 7 lines

Am not sure why.
The allocation can be per demand. That is - only when you encounter a large buffer.

Having buffer allocated in advance will benefit only from removing the cost of the rte_*malloc. However on such big buffers, and further more w/ device offloads like TSO, am not sure that is an issue.

Now I see what you're saying. I was thinking we had to reserve the memory before, like mempool does, then get the buffers as needed.

OK, I can give a try with rte_*malloc and see how it goes.

Quoted 11 lines
Quoted 11 lines

The current patch pushes the decision to the application which knows better the workload. If more memory is available, it can optionally use large buffers, otherwise just don't pass that. Or even decide whether to share the same 64K mempool between multiple vhost ports or use one mempool per port.

Perhaps I missed something, but managing memory with mempool still require us to have buffers of 64K regardless if the data consumes less space. Otherwise the application or the PMD will have to manage memory itself.

Quoted 8 lines
Quoted 3 lines

If we let the PMD manages the memory, what happens if a port/queue is closed and one or more buffers are still in use (switching)? I don't see how to solve this cleanly.

Closing of the dev should return EBUSY till all buffers are free. What is the use case of closing a port while still having packet pending on other port of the switch? And why we cannot wait for them to complete transmission?

The vswitch gets the request from outside and the assumption is that the command will succeed. AFAIK, there is no retry mechanism.

Thanks Shahaf!
fbl

#8 lore raw
Quoted 54 lines · Flavio Leitner
Quoted 51 lines · On Wed, 2 Oct 2019 17:50:41 +0000 Shahaf Shuler <[email protected]> wrote:

Wednesday, October 2, 2019 3:59 PM, Flavio Leitner:

Quoted 22 lines

Obrembski MichalX <[email protected]>; Stokes Ian <[email protected]>
Subject: Re: [dpdk-dev] [PATCH] vhost: add support to large linear mbufs

Hi Shahaf,

Thanks for looking into this, see my inline comments.

Quoted 9 lines · On Wed, 2 Oct 2019 09:00:11 +0000 Shahaf Shuler <[email protected]> wrote:

Wednesday, October 2, 2019 11:05 AM, David Marchand:

Quoted 8 lines

Subject: Re: [dpdk-dev] [PATCH] vhost: add support to large linear mbufs

Hello Shahaf,

Quoted 1 line · Shahaf Shuler

[...]

Quoted 12 lines
Quoted 11 lines
Quoted 3 lines

I am missing some piece here.
Which pool would the PMD take those external buffers from?

The mbuf is always taken from the single mempool associated w/ the rxq. The buffer for the mbuf may be allocated (in case virtio payload is bigger than current mbuf size) from DPDK hugepages or any other system memory and be attached to the mbuf.

You can see example implementation of it in mlx5 PMD (checkout rte_pktmbuf_attach_extbuf call)

Quoted 8 lines

Thanks, I wasn't aware of external buffers.

I see that attaching external buffers of the correct size would be more efficient in terms of saving memory/avoiding sparsing.

However, we still need to be prepared to the worse case scenario (all packets 64K), so that doesn't help with the total memory required.

Am not sure why.
The allocation can be per demand. That is - only when you encounter a large buffer.

Quoted 8 lines
Quoted 3 lines

Having buffer allocated in advance will benefit only from removing the cost of the rte_*malloc. However on such big buffers, and further more w/ device offloads like TSO, am not sure that is an issue.

Now I see what you're saying. I was thinking we had to reserve the memory before, like mempool does, then get the buffers as needed.

OK, I can give a try with rte_*malloc and see how it goes.

This way we actually could have a nice API. For example, by introducing some new flag RTE_VHOST_USER_NO_CHAINED_MBUFS (there might be better name) which could be passed to driver_register(). On receive, depending on this flag, function will create chained mbufs or allocate new contiguous memory chunk and attach it as an external buffer if the data could not be stored in a single mbuf from the registered memory pool.

Supporting external memory in mbufs will require some additional work from the OVS side (e.g. better work with ol_flags), but we'll have to do it anyway for upgrade to DPDK 19.11.

Best regards, Ilya Maximets.

Quoted 26 lines
Quoted 12 lines
Quoted 12 lines

The current patch pushes the decision to the application which knows better the workload. If more memory is available, it can optionally use large buffers, otherwise just don't pass that. Or even decide whether to share the same 64K mempool between multiple vhost ports or use one mempool per port.

Perhaps I missed something, but managing memory with mempool still require us to have buffers of 64K regardless if the data consumes less space. Otherwise the application or the PMD will have to manage memory itself.

Quoted 8 lines
Quoted 3 lines

If we let the PMD manages the memory, what happens if a port/queue is closed and one or more buffers are still in use (switching)? I don't see how to solve this cleanly.

Closing of the dev should return EBUSY till all buffers are free. What is the use case of closing a port while still having packet pending on other port of the switch? And why we cannot wait for them to complete transmission?

The vswitch gets the request from outside and the assumption is that the command will succeed. AFAIK, there is no retry mechanism.

Thanks Shahaf!
fbl

#9 lore raw
Quoted 73 lines · On Thu, 3 Oct 2019 18:57:32 +0200 Ilya Maximets <[email protected]> wrote:
Quoted 59 lines · Flavio Leitner
Quoted 47 lines · On Wed, 2 Oct 2019 17:50:41 +0000 Shahaf Shuler <[email protected]> wrote:

Wednesday, October 2, 2019 3:59 PM, Flavio Leitner:

Quoted 22 lines

Obrembski MichalX <[email protected]>; Stokes Ian <[email protected]>
Subject: Re: [dpdk-dev] [PATCH] vhost: add support to large linear mbufs

Hi Shahaf,

Thanks for looking into this, see my inline comments.

Quoted 9 lines · On Wed, 2 Oct 2019 09:00:11 +0000 Shahaf Shuler <[email protected]> wrote:

Wednesday, October 2, 2019 11:05 AM, David Marchand:

Quoted 8 lines

Subject: Re: [dpdk-dev] [PATCH] vhost: add support to large linear mbufs

Hello Shahaf,

Quoted 1 line · Shahaf Shuler

[...]

Quoted 12 lines
Quoted 11 lines
Quoted 3 lines

I am missing some piece here.
Which pool would the PMD take those external buffers from?

The mbuf is always taken from the single mempool associated w/ the rxq. The buffer for the mbuf may be allocated (in case virtio payload is bigger than current mbuf size) from DPDK hugepages or any other system memory and be attached to the mbuf.

You can see example implementation of it in mlx5 PMD (checkout rte_pktmbuf_attach_extbuf call)

Quoted 8 lines

Thanks, I wasn't aware of external buffers.

I see that attaching external buffers of the correct size would be more efficient in terms of saving memory/avoiding sparsing.

However, we still need to be prepared to the worse case scenario (all packets 64K), so that doesn't help with the total memory required.

Quoted 8 lines

Am not sure why.
The allocation can be per demand. That is - only when you encounter a large buffer.

Having buffer allocated in advance will benefit only from removing the cost of the rte_*malloc. However on such big buffers, and further more w/ device offloads like TSO, am not sure that is an issue.

Quoted 4 lines

Now I see what you're saying. I was thinking we had to reserve the memory before, like mempool does, then get the buffers as needed.

OK, I can give a try with rte_*malloc and see how it goes.

This way we actually could have a nice API. For example, by introducing some new flag RTE_VHOST_USER_NO_CHAINED_MBUFS (there might be better name) which could be passed to driver_register(). On receive, depending on this flag, function will create chained mbufs or allocate new contiguous memory chunk and attach it as an external buffer if the data could not be stored in a single mbuf from the registered memory pool.

Quoted 3 lines

Supporting external memory in mbufs will require some additional work from the OVS side (e.g. better work with ol_flags), but we'll have to do it anyway for upgrade to DPDK 19.11.

Agreed. Looks like rte_malloc is fast enough after all. I have a PoC running iperf3 from VM to another baremetal using vhost-user client with TSO enabled:
[...]

[  5] 140.00-141.00 sec  4.60 GBytes  39.5 Gbits/sec    0   1.26 MBytes
[  5] 141.00-142.00 sec  4.65 GBytes  39.9 Gbits/sec    0   1.26 MBytes
[  5] 142.00-143.00 sec  4.65 GBytes  40.0 Gbits/sec    0   1.26 MBytes
[  5] 143.00-144.00 sec  4.65 GBytes  39.9 Gbits/sec    9   1.04 MBytes
[  5] 144.00-145.00 sec  4.59 GBytes  39.4 Gbits/sec    0   1.16 MBytes
[  5] 145.00-146.00 sec  4.58 GBytes  39.3 Gbits/sec    0   1.26 MBytes
[  5] 146.00-147.00 sec  4.48 GBytes  38.5 Gbits/sec  700    973 KBytes

[...]
(The physical link is 40Gbps)

I will clean that, test more and post the patches soon. Thanks!
fbl

#10 lore raw
In reply to Flavio Leitner

Hi Flavio,

Quoted 10 lines · Flavio Leitner

The rte_vhost_dequeue_burst supports two ways of dequeuing data. If the data fits into a buffer, then all data is copied and a single linear buffer is returned. Otherwise it allocates additional mbufs and chains them together to return a multiple segments mbuf.

While that covers most use cases, it forces applications that need to work with larger data sizes to support multiple segments mbufs. The non-linear characteristic brings complexity and performance implications to the application.

Quoted 5 lines

To resolve the issue, change the API so that the application can optionally provide a second mempool containing larger mbufs. If that is not provided (NULL), the behavior remains as before the change. Otherwise, the data size is checked and the corresponding mempool is used to return linear mbufs.

It is not necessary to break the API (and it would require a deprecation notice), but instead you could create a new API:

static inline uint16_t
vhost_dequeue_burst(int vid, uint16_t queue_id,
struct rte_mempool *mbuf_pool, struct rte_mempool *mbuf_pool_large, struct rte_mbuf **pkts, uint16_t count);

uint16_t rte_vhost_dequeue_burst(int vid, uint16_t queue_id,
struct rte_mempool *mbuf_pool, struct rte_mbuf **pkts, uint16_t count)

{
	return vhost_dequeue_burst(vid, queue_id, mbuf_pool, NULL,
				pkts, count);
}

uint16_t rte_vhost_dequeue_burst2(int vid, uint16_t queue_id,
struct rte_mempool *mbuf_pool, struct rte_mempool *mbuf_pool_large, struct rte_mbuf **pkts, uint16_t count)

{
	return vhost_dequeue_burst(vid, queue_id, mbuf_pool,
				mbuf_pool_large,
				pkts, count);
}

Yeah, the name isn't very creative, I'm sure you'll have better idea!

Quoted 9 lines
---
 drivers/net/vhost/rte_eth_vhost.c |  4 +--
 examples/tep_termination/main.c   |  2 +-
 examples/vhost/main.c             |  2 +-
 lib/librte_vhost/rte_vhost.h      |  5 ++-
 lib/librte_vhost/virtio_net.c     | 57 +++++++++++++++++++++++--------
 5 files changed, 50 insertions(+), 20 deletions(-)

Maxime

#11 lore raw

PATCH v2 1/1 vhost: add support for large buffers.

In reply to Flavio Leitner

The rte_vhost_dequeue_burst supports two ways of dequeuing data. If the data fits into a buffer, then all data is copied and a single linear buffer is returned. Otherwise it allocates additional mbufs and chains them together to return a multiple segments mbuf.

While that covers most use cases, it forces applications that need to work with larger data sizes to support multiple segments mbufs. The non-linear characteristic brings complexity and performance implications to the application.

To resolve the issue, let the host provide during registration if attaching an external buffer to pktmbuf is supported and if only linear buffer are supported.

---
 doc/guides/prog_guide/vhost_lib.rst |  35 ++++++++++
 lib/librte_vhost/rte_vhost.h        |   4 ++
 lib/librte_vhost/socket.c           |  10 +++
 lib/librte_vhost/vhost.c            |  22 ++++++
 lib/librte_vhost/vhost.h            |   4 ++
 lib/librte_vhost/virtio_net.c       | 103 ++++++++++++++++++++++++++--
 6 files changed, 172 insertions(+), 6 deletions(-)

- Changelog:
V2:
- Used rte_malloc() instead of another mempool as suggested by Shahaf.
- Added the documentation section.
- Using driver registration to negotiate the features.
- OvS PoC code:
https://github.com/fleitner/ovs/commit/8fc197c40b1d4fda331686a7b919e9e2b670dda7

doc/guides/prog_guide/vhost_lib.rst +35 -0
index fc3ee4353..07e40e3c5 100644
@@ -117,6 +117,41 @@ The following is an overview of some key Vhost API functions:
     Enabling this flag should only be done when the calling application does
     not pre-fault the guest shared memory, otherwise migration would fail.
 
+  - ``RTE_VHOST_USER_LINEARBUF_SUPPORT``
+
+    Enabling this flag forces vhost dequeue function to only provide linear
+    pktmbuf (no multi-segmented pktmbuf).
+
+    The vhost library by default provides a single pktmbuf for given a
+    packet, but if for some reason the data doesn't fit into a single
+    pktmbuf (e.g., TSO is enabled), the library will allocate additional
+    pktmbufs from the same mempool and chain them together to create a
+    multi-segmented pktmbuf.
+
+    However, the vhost application needs to support multi-segmented format.
+    If the vhost application does not support that format and requires large
+    buffers to be dequeue, this flag should be enabled to force only linear
+    buffers (see RTE_VHOST_USER_EXTBUF_SUPPORT) or drop the packet.
+
+    It is disabled by default.
+
+  - ``RTE_VHOST_USER_EXTBUF_SUPPORT``
+
+    Enabling this flag allows vhost dequeue function to allocate and attach
+    an external buffer to a pktmbuf if the pkmbuf doesn't provide enough
+    space to store all data.
+
+    This is useful when the vhost application wants to support large packets
+    but doesn't want to increase the default mempool object size nor to
+    support multi-segmented mbufs (non-linear). In this case, a fresh buffer
+    is allocated using rte_malloc() which gets attached to a pktmbuf using
+    rte_pktmbuf_attach_extbuf().
+
+    See RTE_VHOST_USER_LINEARBUF_SUPPORT as well to disable multi-segmented
+    mbufs for application that doesn't support chained mbufs.
+
+    It is disabled by default.
+
 * ``rte_vhost_driver_set_features(path, features)``
 
   This function sets the feature bits the vhost-user driver supports. The
lib/librte_vhost/rte_vhost.h +4 -0
index 19474bca0..b821b5df4 100644
@@ -30,6 +30,10 @@ extern "C" {
 #define RTE_VHOST_USER_DEQUEUE_ZERO_COPY	(1ULL << 2)
 #define RTE_VHOST_USER_IOMMU_SUPPORT	(1ULL << 3)
 #define RTE_VHOST_USER_POSTCOPY_SUPPORT		(1ULL << 4)
+/* support mbuf with external buffer attached */
+#define RTE_VHOST_USER_EXTBUF_SUPPORT	(1ULL << 5)
+/* support only linear buffers (no chained mbufs) */
+#define RTE_VHOST_USER_LINEARBUF_SUPPORT	(1ULL << 6)
 
 /** Protocol features. */
 #ifndef VHOST_USER_PROTOCOL_F_MQ
lib/librte_vhost/socket.c +10 -0
index 274988c4d..0ba610fda 100644
@@ -40,6 +40,8 @@ struct vhost_user_socket {
 	bool dequeue_zero_copy;
 	bool iommu_support;
 	bool use_builtin_virtio_net;
+	bool extbuf;
+	bool linearbuf;
 
 	/*
 	 * The "supported_features" indicates the feature bits the
@@ -232,6 +234,12 @@ vhost_user_add_connection(int fd, struct vhost_user_socket *vsocket)
 	if (vsocket->dequeue_zero_copy)
 		vhost_enable_dequeue_zero_copy(vid);
 
+	if (vsocket->extbuf)
+		vhost_enable_extbuf(vid);
+
+	if (vsocket->linearbuf)
+		vhost_enable_linearbuf(vid);
+
 	RTE_LOG(INFO, VHOST_CONFIG, "new device, handle is %d\n", vid);
 
 	if (vsocket->notify_ops->new_connection) {
@@ -870,6 +878,8 @@ rte_vhost_driver_register(const char *path, uint64_t flags)
 		goto out_free;
 	}
 	vsocket->dequeue_zero_copy = flags & RTE_VHOST_USER_DEQUEUE_ZERO_COPY;
+	vsocket->extbuf = flags & RTE_VHOST_USER_EXTBUF_SUPPORT;
+	vsocket->linearbuf = flags & RTE_VHOST_USER_LINEARBUF_SUPPORT;
 
 	/*
 	 * Set the supported features correctly for the builtin vhost-user
lib/librte_vhost/vhost.c +22 -0
index cea44df8c..77457f538 100644
@@ -605,6 +605,28 @@ vhost_set_builtin_virtio_net(int vid, bool enable)
 		dev->flags &= ~VIRTIO_DEV_BUILTIN_VIRTIO_NET;
 }
 
+void
+vhost_enable_extbuf(int vid)
+{
+	struct virtio_net *dev = get_device(vid);
+
+	if (dev == NULL)
+		return;
+
+	dev->extbuf = 1;
+}
+
+void
+vhost_enable_linearbuf(int vid)
+{
+	struct virtio_net *dev = get_device(vid);
+
+	if (dev == NULL)
+		return;
+
+	dev->linearbuf = 1;
+}
+
 int
 rte_vhost_get_mtu(int vid, uint16_t *mtu)
 {
lib/librte_vhost/vhost.h +4 -0
index 5131a97a3..0346bd118 100644
@@ -302,6 +302,8 @@ struct virtio_net {
 	rte_atomic16_t		broadcast_rarp;
 	uint32_t		nr_vring;
 	int			dequeue_zero_copy;
+	int			extbuf;
+	int			linearbuf;
 	struct vhost_virtqueue	*virtqueue[VHOST_MAX_QUEUE_PAIRS * 2];
 #define IF_NAME_SZ (PATH_MAX > IFNAMSIZ ? PATH_MAX : IFNAMSIZ)
 	char			ifname[IF_NAME_SZ];
@@ -476,6 +478,8 @@ void vhost_attach_vdpa_device(int vid, int did);
 void vhost_set_ifname(int, const char *if_name, unsigned int if_len);
 void vhost_enable_dequeue_zero_copy(int vid);
 void vhost_set_builtin_virtio_net(int vid, bool enable);
+void vhost_enable_extbuf(int vid);
+void vhost_enable_linearbuf(int vid);
 
 struct vhost_device_ops const *vhost_driver_callback_get(const char *path);
 
lib/librte_vhost/virtio_net.c +97 -6

151 lines not on this page · raw message

Signature · 1 line
2.20.1
Acked-by on v2 1/1
#12 lore raw

Re: [dpdk-dev] [PATCH v2] vhost: add support for large buffers.

Friday, October 4, 2019 11:10 PM, Flavio Leitner:

Quoted 12 lines

Subject: [dpdk-dev] [PATCH v2] vhost: add support for large buffers.

The rte_vhost_dequeue_burst supports two ways of dequeuing data.
If the data fits into a buffer, then all data is copied and a single linear buffer is returned. Otherwise it allocates additional mbufs and chains them together to return a multiple segments mbuf.

While that covers most use cases, it forces applications that need to work with larger data sizes to support multiple segments mbufs. The non-linear characteristic brings complexity and performance implications to the application.

Quoted 5 lines

To resolve the issue, let the host provide during registration if attaching an external buffer to pktmbuf is supported and if only linear buffer are supported.

Havn't reviewed the code in details. But for the high level direction

Quoted 350 lines
---
 doc/guides/prog_guide/vhost_lib.rst |  35 ++++++++++
 lib/librte_vhost/rte_vhost.h        |   4 ++
 lib/librte_vhost/socket.c           |  10 +++
 lib/librte_vhost/vhost.c            |  22 ++++++
 lib/librte_vhost/vhost.h            |   4 ++
 lib/librte_vhost/virtio_net.c       | 103 ++++++++++++++++++++++++++--
 6 files changed, 172 insertions(+), 6 deletions(-)

- Changelog:
V2:
- Used rte_malloc() instead of another mempool as suggested by Shahaf.
- Added the documentation section.
- Using driver registration to negotiate the features.
- OvS PoC code:

https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithu
b.com%2Ffleitner%2Fovs%2Fcommit%2F8fc197c40b1d4fda331686a7b919e9e 2b670dda7&amp;data=02%7C01%7Cshahafs%40mellanox.com%7C505d8fd73 8da4cfe988a08d74906e87d%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7 C0%7C637058166345885355&amp;sdata=SlYdj%2BoTG%2BufcAgkwp%2FfHo nXR6x4JuGvMmr7gZGsJRE%3D&amp;reserved=0

doc/guides/prog_guide/vhost_lib.rst +0 -0
index fc3ee4353..07e40e3c5 100644
@@ -117,6 +117,41 @@ The following is an overview of some key Vhost API

functions:
Enabling this flag should only be done when the calling application does not pre-fault the guest shared memory, otherwise migration would fail.

doc/guides/prog_guide/vhost_lib.rst +12 -0
+  - ``RTE_VHOST_USER_LINEARBUF_SUPPORT``
+
+    Enabling this flag forces vhost dequeue function to only provide linear
+    pktmbuf (no multi-segmented pktmbuf).
+
+    The vhost library by default provides a single pktmbuf for given a
+    packet, but if for some reason the data doesn't fit into a single
+    pktmbuf (e.g., TSO is enabled), the library will allocate additional
+    pktmbufs from the same mempool and chain them together to create a
+    multi-segmented pktmbuf.
+
+    However, the vhost application needs to support multi-segmented

format.

doc/guides/prog_guide/vhost_lib.rst +19 -0
+    If the vhost application does not support that format and requires large
+    buffers to be dequeue, this flag should be enabled to force only linear
+    buffers (see RTE_VHOST_USER_EXTBUF_SUPPORT) or drop the packet.
+
+    It is disabled by default.
+
+  - ``RTE_VHOST_USER_EXTBUF_SUPPORT``
+
+    Enabling this flag allows vhost dequeue function to allocate and attach
+    an external buffer to a pktmbuf if the pkmbuf doesn't provide enough
+    space to store all data.
+
+    This is useful when the vhost application wants to support large packets
+    but doesn't want to increase the default mempool object size nor to
+    support multi-segmented mbufs (non-linear). In this case, a fresh buffer
+    is allocated using rte_malloc() which gets attached to a pktmbuf using
+    rte_pktmbuf_attach_extbuf().
+
+    See RTE_VHOST_USER_LINEARBUF_SUPPORT as well to disable multi-

segmented

doc/guides/prog_guide/vhost_lib.rst +4 -0
+    mbufs for application that doesn't support chained mbufs.
+
+    It is disabled by default.
+
 * ``rte_vhost_driver_set_features(path, features)``
 
   This function sets the feature bits the vhost-user driver supports. The diff --

git a/lib/librte_vhost/rte_vhost.h b/lib/librte_vhost/rte_vhost.h index 19474bca0..b821b5df4 100644

lib/librte_vhost/rte_vhost.h +4 -0
@@ -30,6 +30,10 @@ extern "C" {
 #define RTE_VHOST_USER_DEQUEUE_ZERO_COPY	(1ULL << 2)
 #define RTE_VHOST_USER_IOMMU_SUPPORT	(1ULL << 3)
 #define RTE_VHOST_USER_POSTCOPY_SUPPORT		(1ULL << 4)
+/* support mbuf with external buffer attached */
+#define RTE_VHOST_USER_EXTBUF_SUPPORT	(1ULL << 5)
+/* support only linear buffers (no chained mbufs) */
+#define RTE_VHOST_USER_LINEARBUF_SUPPORT	(1ULL << 6)
 
 /** Protocol features. */
 #ifndef VHOST_USER_PROTOCOL_F_MQ
lib/librte_vhost/socket.c index renamed from lib/librte_vhost/socket.c +0 -0

274988c4d..0ba610fda 100644

lib/librte_vhost/socket.c +2 -0
@@ -40,6 +40,8 @@ struct vhost_user_socket {
 	bool dequeue_zero_copy;
 	bool iommu_support;
 	bool use_builtin_virtio_net;
+	bool extbuf;
+	bool linearbuf;
 
 	/*
 	 * The "supported_features" indicates the feature bits the @@ -

232,6 +234,12 @@ vhost_user_add_connection(int fd, struct vhost_user_socket *vsocket)

lib/librte_vhost/socket.c +6 -0
 	if (vsocket->dequeue_zero_copy)
 		vhost_enable_dequeue_zero_copy(vid);
 
+	if (vsocket->extbuf)
+		vhost_enable_extbuf(vid);
+
+	if (vsocket->linearbuf)
+		vhost_enable_linearbuf(vid);
+
 	RTE_LOG(INFO, VHOST_CONFIG, "new device, handle is %d\n", vid);
 
 	if (vsocket->notify_ops->new_connection) { @@ -870,6 +878,8 @@

rte_vhost_driver_register(const char *path, uint64_t flags)

lib/librte_vhost/socket.c +0 -0
 		goto out_free;
 	}
 	vsocket->dequeue_zero_copy = flags &

RTE_VHOST_USER_DEQUEUE_ZERO_COPY;

lib/librte_vhost/socket.c +2 -0
+	vsocket->extbuf = flags & RTE_VHOST_USER_EXTBUF_SUPPORT;
+	vsocket->linearbuf = flags &

RTE_VHOST_USER_LINEARBUF_SUPPORT;

lib/librte_vhost/socket.c +0 -1
 	/*
 	 * Set the supported features correctly for the builtin vhost-user diff
--git a/lib/librte_vhost/vhost.c b/lib/librte_vhost/vhost.c index

cea44df8c..77457f538 100644

lib/librte_vhost/vhost.c +22 -0
@@ -605,6 +605,28 @@ vhost_set_builtin_virtio_net(int vid, bool enable)
 		dev->flags &= ~VIRTIO_DEV_BUILTIN_VIRTIO_NET;  }
 
+void
+vhost_enable_extbuf(int vid)
+{
+	struct virtio_net *dev = get_device(vid);
+
+	if (dev == NULL)
+		return;
+
+	dev->extbuf = 1;
+}
+
+void
+vhost_enable_linearbuf(int vid)
+{
+	struct virtio_net *dev = get_device(vid);
+
+	if (dev == NULL)
+		return;
+
+	dev->linearbuf = 1;
+}
+
 int
 rte_vhost_get_mtu(int vid, uint16_t *mtu)  { diff --git

a/lib/librte_vhost/vhost.h b/lib/librte_vhost/vhost.h index 5131a97a3..0346bd118 100644

lib/librte_vhost/vhost.h +2 -0
@@ -302,6 +302,8 @@ struct virtio_net {
 	rte_atomic16_t		broadcast_rarp;
 	uint32_t		nr_vring;
 	int			dequeue_zero_copy;
+	int			extbuf;
+	int			linearbuf;
 	struct vhost_virtqueue	*virtqueue[VHOST_MAX_QUEUE_PAIRS *
 2];
 #define IF_NAME_SZ (PATH_MAX > IFNAMSIZ ? PATH_MAX : IFNAMSIZ)
lib/librte_vhost/vhost.h +0 -0
 	char			ifname[IF_NAME_SZ];
@@ -476,6 +478,8 @@ void vhost_attach_vdpa_device(int vid, int did);  void

vhost_set_ifname(int, const char *if_name, unsigned int if_len); void vhost_enable_dequeue_zero_copy(int vid); void vhost_set_builtin_virtio_net(int vid, bool enable);

lib/librte_vhost/vhost.h +2 -0
+void vhost_enable_extbuf(int vid);
+void vhost_enable_linearbuf(int vid);
 
 struct vhost_device_ops const *vhost_driver_callback_get(const char

*path);

lib/librte_vhost/virtio_net.c index renamed from lib/librte_vhost/virtio_net.c +0 -0

5b85b832d..fca75161d 100644

lib/librte_vhost/virtio_net.c +17 -0
@@ -2,6 +2,7 @@
  * Copyright(c) 2010-2016 Intel Corporation
  */
 
+#include <assert.h>
 #include <stdint.h>
 #include <stdbool.h>
 #include <linux/virtio_net.h>
@@ -1289,6 +1290,96 @@ get_zmbuf(struct vhost_virtqueue *vq)
 	return NULL;
 }
 
+static void
+virtio_dev_extbuf_free(void *addr __rte_unused, void *opaque) {
+	rte_free(opaque);
+}
+
+static int
+virtio_dev_extbuf_alloc(struct rte_mbuf *pkt, uint16_t size) {
+	struct rte_mbuf_ext_shared_info *shinfo;
+	uint16_t buf_len;
+	rte_iova_t iova;
+	void *buf;
+
+	shinfo = NULL;
+	buf_len = size + RTE_PKTMBUF_HEADROOM;
+
+	/* Try to use pkt buffer to store shinfo to reduce the amount of

memory

lib/librte_vhost/virtio_net.c +5 -0
+	 * required, otherwise store shinfo in the new buffer.
+	 */
+	if (rte_pktmbuf_tailroom(pkt) > sizeof(*shinfo))
+		shinfo = rte_pktmbuf_mtod(pkt,
+					  struct rte_mbuf_ext_shared_info

*);

lib/librte_vhost/virtio_net.c +24 -0
+	else {
+		if (unlikely(buf_len + sizeof(shinfo) > UINT16_MAX)) {
+			RTE_LOG(ERR, VHOST_DATA,
+				"buffer size exceeded maximum.\n");
+			return -ENOSPC;
+		}
+
+		buf_len += sizeof(shinfo);
+	}
+
+	buf = rte_malloc(NULL, buf_len, RTE_CACHE_LINE_SIZE);
+	if (unlikely(buf == NULL)) {
+		RTE_LOG(ERR, VHOST_DATA,
+				"Failed to allocate memory for mbuf.\n");
+		return -ENOMEM;
+	}
+
+	/* initialize shinfo */
+	if (shinfo) {
+		shinfo->free_cb = virtio_dev_extbuf_free;
+		shinfo->fcb_opaque = buf;
+		rte_mbuf_ext_refcnt_set(shinfo, 1);
+	} else {
+		shinfo = rte_pktmbuf_ext_shinfo_init_helper(buf,

&buf_len,

lib/librte_vhost/virtio_net.c +43 -0
+					      virtio_dev_extbuf_free, buf);
+		assert(shinfo);
+	}
+
+	iova = rte_mempool_virt2iova(buf);
+	rte_pktmbuf_attach_extbuf(pkt, buf, iova, buf_len, shinfo);
+	rte_pktmbuf_reset_headroom(pkt);
+	assert(pkt->ol_flags == EXT_ATTACHED_MBUF);
+
+	return 0;
+}
+
+/*
+ * Allocate a host supported pktmbuf.
+ */
+static __rte_always_inline struct rte_mbuf *
+virtio_dev_pktmbuf_alloc(struct virtio_net *dev, struct rte_mempool *mp,
+			 uint16_t data_len)
+{
+	struct rte_mbuf *pkt = rte_pktmbuf_alloc(mp);
+
+	if (unlikely(pkt == NULL))
+		return NULL;
+
+	if (rte_pktmbuf_tailroom(pkt) >= data_len)
+		return pkt;
+
+	/* attach an external buffer if supported */
+	if (dev->extbuf && !virtio_dev_extbuf_alloc(pkt, data_len))
+		return pkt;
+
+	/* check if chained buffers are allowed */
+	if (!dev->linearbuf)
+		return pkt;
+
+	/* Data doesn't fit into the buffer and the host supports
+	 * only linear buffers
+	 */
+	rte_pktmbuf_free(pkt);
+
+	return NULL;
+}
+
 static __rte_noinline uint16_t
 virtio_dev_tx_split(struct virtio_net *dev, struct vhost_virtqueue *vq,
 	struct rte_mempool *mbuf_pool, struct rte_mbuf **pkts, uint16_t

count) @@ -1343,21 +1434,21 @@ virtio_dev_tx_split(struct virtio_net *dev, struct vhost_virtqueue *vq,

lib/librte_vhost/virtio_net.c +3 -3
 	for (i = 0; i < count; i++) {
 		struct buf_vector buf_vec[BUF_VECTOR_MAX];
 		uint16_t head_idx;
-		uint32_t dummy_len;
+		uint32_t buf_len;
 		uint16_t nr_vec = 0;
 		int err;
 
 		if (unlikely(fill_vec_buf_split(dev, vq,
 						vq->last_avail_idx + i,
 						&nr_vec, buf_vec,
-						&head_idx, &dummy_len,
+						&head_idx, &buf_len,
 						VHOST_ACCESS_RO) < 0))
 			break;
 
 		if (likely(dev->dequeue_zero_copy == 0))
 			update_shadow_used_ring_split(vq, head_idx, 0);
 
-		pkts[i] = rte_pktmbuf_alloc(mbuf_pool);
+		pkts[i] = virtio_dev_pktmbuf_alloc(dev, mbuf_pool,

buf_len);

lib/librte_vhost/virtio_net.c +2 -2
 		if (unlikely(pkts[i] == NULL)) {
 			RTE_LOG(ERR, VHOST_DATA,
 				"Failed to allocate memory for mbuf.\n");
@@ -1451,14 +1542,14 @@ virtio_dev_tx_packed(struct virtio_net *dev,
 struct vhost_virtqueue *vq,
 	for (i = 0; i < count; i++) {
 		struct buf_vector buf_vec[BUF_VECTOR_MAX];
 		uint16_t buf_id;
-		uint32_t dummy_len;
+		uint32_t buf_len;
 		uint16_t desc_count, nr_vec = 0;
 		int err;
 
 		if (unlikely(fill_vec_buf_packed(dev, vq,
 						vq->last_avail_idx,
 &desc_count,
 						buf_vec, &nr_vec,
-						&buf_id, &dummy_len,
+						&buf_id, &buf_len,
 						VHOST_ACCESS_RO) < 0))
lib/librte_vhost/virtio_net.c +1 -1
 			break;
@@ -1466,7 +1557,7 @@ virtio_dev_tx_packed(struct virtio_net *dev, struct
 vhost_virtqueue *vq,
 			update_shadow_used_ring_packed(vq, buf_id, 0,
 					desc_count);
 
-		pkts[i] = rte_pktmbuf_alloc(mbuf_pool);
+		pkts[i] = virtio_dev_pktmbuf_alloc(dev, mbuf_pool,
 buf_len);
 		if (unlikely(pkts[i] == NULL)) {
lib/librte_vhost/virtio_net.c +0 -1
 			RTE_LOG(ERR, VHOST_DATA,
 				"Failed to allocate memory for mbuf.\n");
--

2.20.1

#13 lore raw

Re: [dpdk-dev] [PATCH v2] vhost: add support for large buffers.

In reply to Flavio Leitner

[PATCH v2] vhost: add support for large buffers.

There is a warning reported by devtools/check-git-log.sh The '.' at the end of the title should be dropped.

Quoted 121 lines · Flavio Leitner

The rte_vhost_dequeue_burst supports two ways of dequeuing data. If the data fits into a buffer, then all data is copied and a single linear buffer is returned. Otherwise it allocates additional mbufs and chains them together to return a multiple segments mbuf.

While that covers most use cases, it forces applications that need to work with larger data sizes to support multiple segments mbufs. The non-linear characteristic brings complexity and performance implications to the application.

To resolve the issue, let the host provide during registration if attaching an external buffer to pktmbuf is supported and if only linear buffer are supported.

---
 doc/guides/prog_guide/vhost_lib.rst |  35 ++++++++++
 lib/librte_vhost/rte_vhost.h        |   4 ++
 lib/librte_vhost/socket.c           |  10 +++
 lib/librte_vhost/vhost.c            |  22 ++++++
 lib/librte_vhost/vhost.h            |   4 ++
 lib/librte_vhost/virtio_net.c       | 103 ++++++++++++++++++++++++++--
 6 files changed, 172 insertions(+), 6 deletions(-)

- Changelog:
V2:
- Used rte_malloc() instead of another mempool as suggested by Shahaf.
- Added the documentation section.
- Using driver registration to negotiate the features.
- OvS PoC code:
https://github.com/fleitner/ovs/commit/8fc197c40b1d4fda331686a7b919e9e2b670dda7

doc/guides/prog_guide/vhost_lib.rst +35 -0
index fc3ee4353..07e40e3c5 100644
@@ -117,6 +117,41 @@ The following is an overview of some key Vhost API functions:
     Enabling this flag should only be done when the calling application does
     not pre-fault the guest shared memory, otherwise migration would fail.
 
+  - ``RTE_VHOST_USER_LINEARBUF_SUPPORT``
+
+    Enabling this flag forces vhost dequeue function to only provide linear
+    pktmbuf (no multi-segmented pktmbuf).
+
+    The vhost library by default provides a single pktmbuf for given a
+    packet, but if for some reason the data doesn't fit into a single
+    pktmbuf (e.g., TSO is enabled), the library will allocate additional
+    pktmbufs from the same mempool and chain them together to create a
+    multi-segmented pktmbuf.
+
+    However, the vhost application needs to support multi-segmented format.
+    If the vhost application does not support that format and requires large
+    buffers to be dequeue, this flag should be enabled to force only linear
+    buffers (see RTE_VHOST_USER_EXTBUF_SUPPORT) or drop the packet.
+
+    It is disabled by default.
+
+  - ``RTE_VHOST_USER_EXTBUF_SUPPORT``
+
+    Enabling this flag allows vhost dequeue function to allocate and attach
+    an external buffer to a pktmbuf if the pkmbuf doesn't provide enough
+    space to store all data.
+
+    This is useful when the vhost application wants to support large packets
+    but doesn't want to increase the default mempool object size nor to
+    support multi-segmented mbufs (non-linear). In this case, a fresh buffer
+    is allocated using rte_malloc() which gets attached to a pktmbuf using
+    rte_pktmbuf_attach_extbuf().
+
+    See RTE_VHOST_USER_LINEARBUF_SUPPORT as well to disable multi-segmented
+    mbufs for application that doesn't support chained mbufs.
+
+    It is disabled by default.
+
 * ``rte_vhost_driver_set_features(path, features)``
 
   This function sets the feature bits the vhost-user driver supports. The
lib/librte_vhost/rte_vhost.h +4 -0
index 19474bca0..b821b5df4 100644
@@ -30,6 +30,10 @@ extern "C" {
 #define RTE_VHOST_USER_DEQUEUE_ZERO_COPY	(1ULL << 2)
 #define RTE_VHOST_USER_IOMMU_SUPPORT	(1ULL << 3)
 #define RTE_VHOST_USER_POSTCOPY_SUPPORT		(1ULL << 4)
+/* support mbuf with external buffer attached */
+#define RTE_VHOST_USER_EXTBUF_SUPPORT	(1ULL << 5)
+/* support only linear buffers (no chained mbufs) */
+#define RTE_VHOST_USER_LINEARBUF_SUPPORT	(1ULL << 6)
 
 /** Protocol features. */
 #ifndef VHOST_USER_PROTOCOL_F_MQ
lib/librte_vhost/socket.c +8 -0
index 274988c4d..0ba610fda 100644
@@ -40,6 +40,8 @@ struct vhost_user_socket {
 	bool dequeue_zero_copy;
 	bool iommu_support;
 	bool use_builtin_virtio_net;
+	bool extbuf;
+	bool linearbuf;
 
 	/*
 	 * The "supported_features" indicates the feature bits the
@@ -232,6 +234,12 @@ vhost_user_add_connection(int fd, struct vhost_user_socket *vsocket)
 	if (vsocket->dequeue_zero_copy)
 		vhost_enable_dequeue_zero_copy(vid);
 
+	if (vsocket->extbuf)
+		vhost_enable_extbuf(vid);
+
+	if (vsocket->linearbuf)
+		vhost_enable_linearbuf(vid);
+
 	RTE_LOG(INFO, VHOST_CONFIG, "new device, handle is %d\n", vid);
 
 	if (vsocket->notify_ops->new_connection) {

lib/librte_vhost/socket.c rte_vhost_driver_register(const char *path, uint64_t flags) · line 882

Quoted 5 lines
lib/librte_vhost/socket.c +2 -0
@@ -870,6 +878,8 @@ rte_vhost_driver_register(const char *path, uint64_t flags)
 		goto out_free;
 	}
 	vsocket->dequeue_zero_copy = flags & RTE_VHOST_USER_DEQUEUE_ZERO_COPY;
+	vsocket->extbuf = flags & RTE_VHOST_USER_EXTBUF_SUPPORT;
+	vsocket->linearbuf = flags & RTE_VHOST_USER_LINEARBUF_SUPPORT;

It could be problematic to use dequeue_zero_copy with above two features at the same time.
We may want to reject such combination.

Quoted 75 lines
lib/librte_vhost/socket.c +0 -0
 	/*
 	 * Set the supported features correctly for the builtin vhost-user
lib/librte_vhost/vhost.c +22 -0
index cea44df8c..77457f538 100644
@@ -605,6 +605,28 @@ vhost_set_builtin_virtio_net(int vid, bool enable)
 		dev->flags &= ~VIRTIO_DEV_BUILTIN_VIRTIO_NET;
 }
 
+void
+vhost_enable_extbuf(int vid)
+{
+	struct virtio_net *dev = get_device(vid);
+
+	if (dev == NULL)
+		return;
+
+	dev->extbuf = 1;
+}
+
+void
+vhost_enable_linearbuf(int vid)
+{
+	struct virtio_net *dev = get_device(vid);
+
+	if (dev == NULL)
+		return;
+
+	dev->linearbuf = 1;
+}
+
 int
 rte_vhost_get_mtu(int vid, uint16_t *mtu)
 {
lib/librte_vhost/vhost.h +4 -0
index 5131a97a3..0346bd118 100644
@@ -302,6 +302,8 @@ struct virtio_net {
 	rte_atomic16_t		broadcast_rarp;
 	uint32_t		nr_vring;
 	int			dequeue_zero_copy;
+	int			extbuf;
+	int			linearbuf;
 	struct vhost_virtqueue	*virtqueue[VHOST_MAX_QUEUE_PAIRS * 2];
 #define IF_NAME_SZ (PATH_MAX > IFNAMSIZ ? PATH_MAX : IFNAMSIZ)
 	char			ifname[IF_NAME_SZ];
@@ -476,6 +478,8 @@ void vhost_attach_vdpa_device(int vid, int did);
 void vhost_set_ifname(int, const char *if_name, unsigned int if_len);
 void vhost_enable_dequeue_zero_copy(int vid);
 void vhost_set_builtin_virtio_net(int vid, bool enable);
+void vhost_enable_extbuf(int vid);
+void vhost_enable_linearbuf(int vid);
 
 struct vhost_device_ops const *vhost_driver_callback_get(const char *path);
 
lib/librte_vhost/virtio_net.c +2 -0
index 5b85b832d..fca75161d 100644
@@ -2,6 +2,7 @@
  * Copyright(c) 2010-2016 Intel Corporation
  */
 
+#include <assert.h>
 #include <stdint.h>
 #include <stdbool.h>
 #include <linux/virtio_net.h>
@@ -1289,6 +1290,96 @@ get_zmbuf(struct vhost_virtqueue *vq)
 	return NULL;
 }
 
+static void

lib/librte_vhost/virtio_net.c get_zmbuf(struct vhost_virtqueue *vq) · line 1303

Quoted 10 lines
lib/librte_vhost/virtio_net.c +10 -0
+virtio_dev_extbuf_free(void *addr __rte_unused, void *opaque)
+{
+	rte_free(opaque);
+}
+
+static int
+virtio_dev_extbuf_alloc(struct rte_mbuf *pkt, uint16_t size)
+{
+	struct rte_mbuf_ext_shared_info *shinfo;
+	uint16_t buf_len;

It also includes RTE_PKTMBUF_HEADROOM and sizeof(*shinfo). Is uint16_t big enough?

Quoted 4 lines
lib/librte_vhost/virtio_net.c +4 -0
+	rte_iova_t iova;
+	void *buf;
+
+	shinfo = NULL;

lib/librte_vhost/virtio_net.c get_zmbuf(struct vhost_virtqueue *vq) · line 1317

Quoted 10 lines
lib/librte_vhost/virtio_net.c +10 -0
+	buf_len = size + RTE_PKTMBUF_HEADROOM;
+
+	/* Try to use pkt buffer to store shinfo to reduce the amount of memory
+	 * required, otherwise store shinfo in the new buffer.
+	 */
+	if (rte_pktmbuf_tailroom(pkt) > sizeof(*shinfo))
+		shinfo = rte_pktmbuf_mtod(pkt,
+					  struct rte_mbuf_ext_shared_info *);
+	else {
+		if (unlikely(buf_len + sizeof(shinfo) > UINT16_MAX)) {

Should be sizeof(*shinfo)

lib/librte_vhost/virtio_net.c get_zmbuf(struct vhost_virtqueue *vq) · line 1323

Quoted 6 lines
lib/librte_vhost/virtio_net.c +6 -0
+			RTE_LOG(ERR, VHOST_DATA,
+				"buffer size exceeded maximum.\n");
+			return -ENOSPC;
+		}
+
+		buf_len += sizeof(shinfo);

Ditto.

lib/librte_vhost/virtio_net.c get_zmbuf(struct vhost_virtqueue *vq) · line 1329

Quoted 6 lines
lib/librte_vhost/virtio_net.c +6 -0
+	}
+
+	buf = rte_malloc(NULL, buf_len, RTE_CACHE_LINE_SIZE);
+	if (unlikely(buf == NULL)) {
+		RTE_LOG(ERR, VHOST_DATA,
+				"Failed to allocate memory for mbuf.\n");

Looks better to just have 3 tabs before the string.

lib/librte_vhost/virtio_net.c get_zmbuf(struct vhost_virtqueue *vq) · line 1341

Quoted 12 lines
lib/librte_vhost/virtio_net.c +12 -0
+		return -ENOMEM;
+	}
+
+	/* initialize shinfo */
+	if (shinfo) {
+		shinfo->free_cb = virtio_dev_extbuf_free;
+		shinfo->fcb_opaque = buf;
+		rte_mbuf_ext_refcnt_set(shinfo, 1);
+	} else {
+		shinfo = rte_pktmbuf_ext_shinfo_init_helper(buf, &buf_len,
+					      virtio_dev_extbuf_free, buf);
+		assert(shinfo);

As a library, we shouldn't abort the process.
We can just return the error.

lib/librte_vhost/virtio_net.c get_zmbuf(struct vhost_virtqueue *vq) · line 1344

Quoted 3 lines
lib/librte_vhost/virtio_net.c +3 -0
+	}
+
+	iova = rte_mempool_virt2iova(buf);

Should be rte_malloc_virt2iova().

lib/librte_vhost/virtio_net.c get_zmbuf(struct vhost_virtqueue *vq) · line 1347

Quoted 3 lines
lib/librte_vhost/virtio_net.c +3 -0
+	rte_pktmbuf_attach_extbuf(pkt, buf, iova, buf_len, shinfo);
+	rte_pktmbuf_reset_headroom(pkt);
+	assert(pkt->ol_flags == EXT_ATTACHED_MBUF);

Ditto.

Quoted 92 lines
lib/librte_vhost/virtio_net.c +41 -6
+
+	return 0;
+}
+
+/*
+ * Allocate a host supported pktmbuf.
+ */
+static __rte_always_inline struct rte_mbuf *
+virtio_dev_pktmbuf_alloc(struct virtio_net *dev, struct rte_mempool *mp,
+			 uint16_t data_len)
+{
+	struct rte_mbuf *pkt = rte_pktmbuf_alloc(mp);
+
+	if (unlikely(pkt == NULL))
+		return NULL;
+
+	if (rte_pktmbuf_tailroom(pkt) >= data_len)
+		return pkt;
+
+	/* attach an external buffer if supported */
+	if (dev->extbuf && !virtio_dev_extbuf_alloc(pkt, data_len))
+		return pkt;
+
+	/* check if chained buffers are allowed */
+	if (!dev->linearbuf)
+		return pkt;
+
+	/* Data doesn't fit into the buffer and the host supports
+	 * only linear buffers
+	 */
+	rte_pktmbuf_free(pkt);
+
+	return NULL;
+}
+
 static __rte_noinline uint16_t
 virtio_dev_tx_split(struct virtio_net *dev, struct vhost_virtqueue *vq,
 	struct rte_mempool *mbuf_pool, struct rte_mbuf **pkts, uint16_t count)
@@ -1343,21 +1434,21 @@ virtio_dev_tx_split(struct virtio_net *dev, struct vhost_virtqueue *vq,
 	for (i = 0; i < count; i++) {
 		struct buf_vector buf_vec[BUF_VECTOR_MAX];
 		uint16_t head_idx;
-		uint32_t dummy_len;
+		uint32_t buf_len;
 		uint16_t nr_vec = 0;
 		int err;
 
 		if (unlikely(fill_vec_buf_split(dev, vq,
 						vq->last_avail_idx + i,
 						&nr_vec, buf_vec,
-						&head_idx, &dummy_len,
+						&head_idx, &buf_len,
 						VHOST_ACCESS_RO) < 0))
 			break;
 
 		if (likely(dev->dequeue_zero_copy == 0))
 			update_shadow_used_ring_split(vq, head_idx, 0);
 
-		pkts[i] = rte_pktmbuf_alloc(mbuf_pool);
+		pkts[i] = virtio_dev_pktmbuf_alloc(dev, mbuf_pool, buf_len);
 		if (unlikely(pkts[i] == NULL)) {
 			RTE_LOG(ERR, VHOST_DATA,
 				"Failed to allocate memory for mbuf.\n");
@@ -1451,14 +1542,14 @@ virtio_dev_tx_packed(struct virtio_net *dev, struct vhost_virtqueue *vq,
 	for (i = 0; i < count; i++) {
 		struct buf_vector buf_vec[BUF_VECTOR_MAX];
 		uint16_t buf_id;
-		uint32_t dummy_len;
+		uint32_t buf_len;
 		uint16_t desc_count, nr_vec = 0;
 		int err;
 
 		if (unlikely(fill_vec_buf_packed(dev, vq,
 						vq->last_avail_idx, &desc_count,
 						buf_vec, &nr_vec,
-						&buf_id, &dummy_len,
+						&buf_id, &buf_len,
 						VHOST_ACCESS_RO) < 0))
 			break;
 
@@ -1466,7 +1557,7 @@ virtio_dev_tx_packed(struct virtio_net *dev, struct vhost_virtqueue *vq,
 			update_shadow_used_ring_packed(vq, buf_id, 0,
 					desc_count);
 
-		pkts[i] = rte_pktmbuf_alloc(mbuf_pool);
+		pkts[i] = virtio_dev_pktmbuf_alloc(dev, mbuf_pool, buf_len);
 		if (unlikely(pkts[i] == NULL)) {
 			RTE_LOG(ERR, VHOST_DATA,
 				"Failed to allocate memory for mbuf.\n");
Signature · 1 line
2.20.1
#14 lore raw

Re: [dpdk-dev] [PATCH v2] vhost: add support for large buffers.

Hi Tiwei,

Thanks for the review. I will follow up with an improved patch. fbl

Quoted 410 lines · On Thu, 10 Oct 2019 13:12:57 +0800 Tiwei Bie <[email protected]> wrote:

[PATCH v2] vhost: add support for large buffers.

There is a warning reported by devtools/check-git-log.sh The '.' at the end of the title should be dropped.

Quoted 138 lines · Flavio Leitner

The rte_vhost_dequeue_burst supports two ways of dequeuing data. If the data fits into a buffer, then all data is copied and a single linear buffer is returned. Otherwise it allocates additional mbufs and chains them together to return a multiple segments mbuf.

While that covers most use cases, it forces applications that need to work with larger data sizes to support multiple segments mbufs. The non-linear characteristic brings complexity and performance implications to the application.

To resolve the issue, let the host provide during registration if attaching an external buffer to pktmbuf is supported and if only linear buffer are supported.

---
 doc/guides/prog_guide/vhost_lib.rst |  35 ++++++++++
 lib/librte_vhost/rte_vhost.h        |   4 ++
 lib/librte_vhost/socket.c           |  10 +++
 lib/librte_vhost/vhost.c            |  22 ++++++
 lib/librte_vhost/vhost.h            |   4 ++
 lib/librte_vhost/virtio_net.c       | 103

++++++++++++++++++++++++++-- 6 files changed, 172 insertions(+), 6 deletions(-)

- Changelog:
V2:
- Used rte_malloc() instead of another mempool as suggested by
Shahaf.
- Added the documentation section.
- Using driver registration to negotiate the features.
- OvS PoC code:
https://github.com/fleitner/ovs/commit/8fc197c40b1d4fda331686a7b919e9e2b670dda7

(quoted diff) +0 -0

b/doc/guides/prog_guide/vhost_lib.rst index fc3ee4353..07e40e3c5 100644 --- a/doc/guides/prog_guide/vhost_lib.rst
+++ b/doc/guides/prog_guide/vhost_lib.rst

(quoted diff) +0 -0
@@ -117,6 +117,41 @@ The following is an overview of some key Vhost

API functions: Enabling this flag should only be done when the calling application does not pre-fault the guest shared memory, otherwise migration would fail.

doc/guides/prog_guide/vhost_lib.rst +3 -0
+  - ``RTE_VHOST_USER_LINEARBUF_SUPPORT``
+
+    Enabling this flag forces vhost dequeue function to only

provide linear

doc/guides/prog_guide/vhost_lib.rst +3 -0
+    pktmbuf (no multi-segmented pktmbuf).
+
+    The vhost library by default provides a single pktmbuf for

given a
+ packet, but if for some reason the data doesn't fit into a
single
+ pktmbuf (e.g., TSO is enabled), the library will allocate
additional
+ pktmbufs from the same mempool and chain them together to
create a

doc/guides/prog_guide/vhost_lib.rst +3 -0
+    multi-segmented pktmbuf.
+
+    However, the vhost application needs to support

multi-segmented format.
+ If the vhost application does not support that format and
requires large
+ buffers to be dequeue, this flag should be enabled to force
only linear

doc/guides/prog_guide/vhost_lib.rst +7 -0
+    buffers (see RTE_VHOST_USER_EXTBUF_SUPPORT) or drop the packet.
+
+    It is disabled by default.
+
+  - ``RTE_VHOST_USER_EXTBUF_SUPPORT``
+
+    Enabling this flag allows vhost dequeue function to allocate

and attach
+ an external buffer to a pktmbuf if the pkmbuf doesn't provide
enough

doc/guides/prog_guide/vhost_lib.rst +3 -0
+    space to store all data.
+
+    This is useful when the vhost application wants to support

large packets
+ but doesn't want to increase the default mempool object size
nor to
+ support multi-segmented mbufs (non-linear). In this case, a
fresh buffer
+ is allocated using rte_malloc() which gets attached to a
pktmbuf using

doc/guides/prog_guide/vhost_lib.rst +3 -0
+    rte_pktmbuf_attach_extbuf().
+
+    See RTE_VHOST_USER_LINEARBUF_SUPPORT as well to disable

multi-segmented

doc/guides/prog_guide/vhost_lib.rst +4 -0
+    mbufs for application that doesn't support chained mbufs.
+
+    It is disabled by default.
+
 * ``rte_vhost_driver_set_features(path, features)``
 
   This function sets the feature bits the vhost-user driver

supports. The diff --git a/lib/librte_vhost/rte_vhost.h b/lib/librte_vhost/rte_vhost.h index 19474bca0..b821b5df4 100644

lib/librte_vhost/rte_vhost.h +4 -0
@@ -30,6 +30,10 @@ extern "C" {
 #define RTE_VHOST_USER_DEQUEUE_ZERO_COPY	(1ULL << 2)
 #define RTE_VHOST_USER_IOMMU_SUPPORT	(1ULL << 3)
 #define RTE_VHOST_USER_POSTCOPY_SUPPORT		(1ULL << 4)
+/* support mbuf with external buffer attached */
+#define RTE_VHOST_USER_EXTBUF_SUPPORT	(1ULL << 5)
+/* support only linear buffers (no chained mbufs) */
+#define RTE_VHOST_USER_LINEARBUF_SUPPORT	(1ULL << 6)
 
 /** Protocol features. */
 #ifndef VHOST_USER_PROTOCOL_F_MQ
lib/librte_vhost/socket.c +8 -0
index 274988c4d..0ba610fda 100644
@@ -40,6 +40,8 @@ struct vhost_user_socket {
 	bool dequeue_zero_copy;
 	bool iommu_support;
 	bool use_builtin_virtio_net;
+	bool extbuf;
+	bool linearbuf;
 
 	/*
 	 * The "supported_features" indicates the feature bits the
@@ -232,6 +234,12 @@ vhost_user_add_connection(int fd, struct
 vhost_user_socket *vsocket) if (vsocket->dequeue_zero_copy)
 		vhost_enable_dequeue_zero_copy(vid);
 
+	if (vsocket->extbuf)
+		vhost_enable_extbuf(vid);
+
+	if (vsocket->linearbuf)
+		vhost_enable_linearbuf(vid);
+
 	RTE_LOG(INFO, VHOST_CONFIG, "new device, handle is %d\n",
 vid); 
 	if (vsocket->notify_ops->new_connection) {

lib/librte_vhost/socket.c rte_vhost_driver_register(const char *path, · line 884

Quoted 7 lines
lib/librte_vhost/socket.c +2 -0
@@ -870,6 +878,8 @@ rte_vhost_driver_register(const char *path,
 uint64_t flags) goto out_free;
 	}
 	vsocket->dequeue_zero_copy = flags &
 RTE_VHOST_USER_DEQUEUE_ZERO_COPY;
+	vsocket->extbuf = flags & RTE_VHOST_USER_EXTBUF_SUPPORT;
+	vsocket->linearbuf = flags &
 RTE_VHOST_USER_LINEARBUF_SUPPORT;  

It could be problematic to use dequeue_zero_copy with above two features at the same time.
We may want to reject such combination.

Quoted 76 lines
lib/librte_vhost/socket.c +0 -0
 	/*

* Set the supported features correctly for the builtin
vhost-user diff --git a/lib/librte_vhost/vhost.c b/lib/librte_vhost/vhost.c index cea44df8c..77457f538 100644

lib/librte_vhost/vhost.c +22 -0
@@ -605,6 +605,28 @@ vhost_set_builtin_virtio_net(int vid, bool
 enable) dev->flags &= ~VIRTIO_DEV_BUILTIN_VIRTIO_NET;
 }
 
+void
+vhost_enable_extbuf(int vid)
+{
+	struct virtio_net *dev = get_device(vid);
+
+	if (dev == NULL)
+		return;
+
+	dev->extbuf = 1;
+}
+
+void
+vhost_enable_linearbuf(int vid)
+{
+	struct virtio_net *dev = get_device(vid);
+
+	if (dev == NULL)
+		return;
+
+	dev->linearbuf = 1;
+}
+
 int
 rte_vhost_get_mtu(int vid, uint16_t *mtu)
 {
lib/librte_vhost/vhost.h +2 -0
index 5131a97a3..0346bd118 100644
@@ -302,6 +302,8 @@ struct virtio_net {
 	rte_atomic16_t		broadcast_rarp;
 	uint32_t		nr_vring;
 	int			dequeue_zero_copy;
+	int			extbuf;
+	int			linearbuf;
 	struct vhost_virtqueue
 *virtqueue[VHOST_MAX_QUEUE_PAIRS * 2]; #define IF_NAME_SZ (PATH_MAX
Quoted 2 lines

IFNAMSIZ ? PATH_MAX : IFNAMSIZ) char
ifname[IF_NAME_SZ];

lib/librte_vhost/vhost.h +2 -0
@@ -476,6 +478,8 @@ void vhost_attach_vdpa_device(int vid, int did);
 void vhost_set_ifname(int, const char *if_name, unsigned int
 if_len); void vhost_enable_dequeue_zero_copy(int vid);
 void vhost_set_builtin_virtio_net(int vid, bool enable);
+void vhost_enable_extbuf(int vid);
+void vhost_enable_linearbuf(int vid);
 
 struct vhost_device_ops const *vhost_driver_callback_get(const
 char *path); 
(quoted diff) +0 -0

b/lib/librte_vhost/virtio_net.c index 5b85b832d..fca75161d 100644

lib/librte_vhost/virtio_net.c +2 -0
@@ -2,6 +2,7 @@
  * Copyright(c) 2010-2016 Intel Corporation
  */
 
+#include <assert.h>
 #include <stdint.h>
 #include <stdbool.h>
 #include <linux/virtio_net.h>
@@ -1289,6 +1290,96 @@ get_zmbuf(struct vhost_virtqueue *vq)
 	return NULL;
 }
 
+static void

lib/librte_vhost/virtio_net.c get_zmbuf(struct vhost_virtqueue *vq) · line 1303

Quoted 10 lines
lib/librte_vhost/virtio_net.c +10 -0
+virtio_dev_extbuf_free(void *addr __rte_unused, void *opaque)
+{
+	rte_free(opaque);
+}
+
+static int
+virtio_dev_extbuf_alloc(struct rte_mbuf *pkt, uint16_t size)
+{
+	struct rte_mbuf_ext_shared_info *shinfo;
+	uint16_t buf_len;  

It also includes RTE_PKTMBUF_HEADROOM and sizeof(*shinfo). Is uint16_t big enough?

Quoted 16 lines
lib/librte_vhost/virtio_net.c +7 -0
+	rte_iova_t iova;
+	void *buf;
+
+	shinfo = NULL;
+	buf_len = size + RTE_PKTMBUF_HEADROOM;
+
+	/* Try to use pkt buffer to store shinfo to reduce the

amount of memory

lib/librte_vhost/virtio_net.c +5 -0
+	 * required, otherwise store shinfo in the new buffer.
+	 */
+	if (rte_pktmbuf_tailroom(pkt) > sizeof(*shinfo))
+		shinfo = rte_pktmbuf_mtod(pkt,
+					  struct

rte_mbuf_ext_shared_info *);

lib/librte_vhost/virtio_net.c +2 -0
+	else {
+		if (unlikely(buf_len + sizeof(shinfo) >
Quoted 1 line

UINT16_MAX)) {

Should be sizeof(*shinfo)

lib/librte_vhost/virtio_net.c get_zmbuf(struct vhost_virtqueue *vq) · line 1323

Quoted 6 lines
lib/librte_vhost/virtio_net.c +6 -0
+			RTE_LOG(ERR, VHOST_DATA,
+				"buffer size exceeded maximum.\n");
+			return -ENOSPC;
+		}
+
+		buf_len += sizeof(shinfo);  

Ditto.

Quoted 7 lines
lib/librte_vhost/virtio_net.c +6 -0
+	}
+
+	buf = rte_malloc(NULL, buf_len, RTE_CACHE_LINE_SIZE);
+	if (unlikely(buf == NULL)) {
+		RTE_LOG(ERR, VHOST_DATA,
+				"Failed to allocate memory for

mbuf.\n");

Looks better to just have 3 tabs before the string.

Quoted 13 lines
lib/librte_vhost/virtio_net.c +10 -0
+		return -ENOMEM;
+	}
+
+	/* initialize shinfo */
+	if (shinfo) {
+		shinfo->free_cb = virtio_dev_extbuf_free;
+		shinfo->fcb_opaque = buf;
+		rte_mbuf_ext_refcnt_set(shinfo, 1);
+	} else {
+		shinfo = rte_pktmbuf_ext_shinfo_init_helper(buf,

&buf_len,
+
virtio_dev_extbuf_free, buf);

lib/librte_vhost/virtio_net.c get_zmbuf(struct vhost_virtqueue *vq) · line 1341

Quoted 1 line
lib/librte_vhost/virtio_net.c +1 -0
+		assert(shinfo);  

As a library, we shouldn't abort the process.
We can just return the error.

lib/librte_vhost/virtio_net.c get_zmbuf(struct vhost_virtqueue *vq) · line 1344

Quoted 3 lines
lib/librte_vhost/virtio_net.c +3 -0
+	}
+
+	iova = rte_mempool_virt2iova(buf);  

Should be rte_malloc_virt2iova().

lib/librte_vhost/virtio_net.c get_zmbuf(struct vhost_virtqueue *vq) · line 1347

Quoted 3 lines
lib/librte_vhost/virtio_net.c +3 -0
+	rte_pktmbuf_attach_extbuf(pkt, buf, iova, buf_len, shinfo);
+	rte_pktmbuf_reset_headroom(pkt);
+	assert(pkt->ol_flags == EXT_ATTACHED_MBUF);  

Ditto.

Quoted 95 lines
lib/librte_vhost/virtio_net.c +9 -0
+
+	return 0;
+}
+
+/*
+ * Allocate a host supported pktmbuf.
+ */
+static __rte_always_inline struct rte_mbuf *
+virtio_dev_pktmbuf_alloc(struct virtio_net *dev, struct

rte_mempool *mp,

lib/librte_vhost/virtio_net.c +26 -0
+			 uint16_t data_len)
+{
+	struct rte_mbuf *pkt = rte_pktmbuf_alloc(mp);
+
+	if (unlikely(pkt == NULL))
+		return NULL;
+
+	if (rte_pktmbuf_tailroom(pkt) >= data_len)
+		return pkt;
+
+	/* attach an external buffer if supported */
+	if (dev->extbuf && !virtio_dev_extbuf_alloc(pkt, data_len))
+		return pkt;
+
+	/* check if chained buffers are allowed */
+	if (!dev->linearbuf)
+		return pkt;
+
+	/* Data doesn't fit into the buffer and the host supports
+	 * only linear buffers
+	 */
+	rte_pktmbuf_free(pkt);
+
+	return NULL;
+}
+
 static __rte_noinline uint16_t
 virtio_dev_tx_split(struct virtio_net *dev, struct vhost_virtqueue

*vq, struct rte_mempool *mbuf_pool, struct rte_mbuf **pkts, uint16_t count) @@ -1343,21 +1434,21 @@ virtio_dev_tx_split(struct virtio_net *dev, struct vhost_virtqueue *vq, for (i = 0; i < count; i++) { struct buf_vector buf_vec[BUF_VECTOR_MAX];

lib/librte_vhost/virtio_net.c +2 -2
 		uint16_t head_idx;
-		uint32_t dummy_len;
+		uint32_t buf_len;
 		uint16_t nr_vec = 0;
 		int err;
 
 		if (unlikely(fill_vec_buf_split(dev, vq,
 						vq->last_avail_idx
+ i, &nr_vec, buf_vec,
-						&head_idx,

&dummy_len,

lib/librte_vhost/virtio_net.c +1 -0
+						&head_idx,

&buf_len, VHOST_ACCESS_RO) < 0))

lib/librte_vhost/virtio_net.c +0 -0
 			break;
 
 		if (likely(dev->dequeue_zero_copy == 0))
 			update_shadow_used_ring_split(vq,

head_idx, 0);

lib/librte_vhost/virtio_net.c +1 -1
-		pkts[i] = rte_pktmbuf_alloc(mbuf_pool);
+		pkts[i] = virtio_dev_pktmbuf_alloc(dev, mbuf_pool,

buf_len); if (unlikely(pkts[i] == NULL)) {

lib/librte_vhost/virtio_net.c +0 -0
 			RTE_LOG(ERR, VHOST_DATA,
 				"Failed to allocate memory for

mbuf.\n"); @@ -1451,14 +1542,14 @@ virtio_dev_tx_packed(struct virtio_net *dev, struct vhost_virtqueue *vq, for (i = 0; i < count; i++) { struct buf_vector buf_vec[BUF_VECTOR_MAX];

lib/librte_vhost/virtio_net.c +1 -1
 		uint16_t buf_id;
-		uint32_t dummy_len;
+		uint32_t buf_len;
 		uint16_t desc_count, nr_vec = 0;
 		int err;
 
 		if (unlikely(fill_vec_buf_packed(dev, vq,
 						vq->last_avail_idx,

&desc_count, buf_vec, &nr_vec,

lib/librte_vhost/virtio_net.c +0 -1
-						&buf_id,

&dummy_len,

lib/librte_vhost/virtio_net.c +1 -0
+						&buf_id, &buf_len,
 						VHOST_ACCESS_RO) <

0)) break;

lib/librte_vhost/virtio_net.c +1 -1
@@ -1466,7 +1557,7 @@ virtio_dev_tx_packed(struct virtio_net *dev,
 struct vhost_virtqueue *vq, update_shadow_used_ring_packed(vq,
 buf_id, 0, desc_count);
 
-		pkts[i] = rte_pktmbuf_alloc(mbuf_pool);
+		pkts[i] = virtio_dev_pktmbuf_alloc(dev, mbuf_pool,
 buf_len); if (unlikely(pkts[i] == NULL)) {
 			RTE_LOG(ERR, VHOST_DATA,
 				"Failed to allocate memory for

mbuf.\n"); --
2.20.1

#15 lore raw

PATCH v3 1/1 vhost: add support for large buffers

In reply to Flavio Leitner

The rte_vhost_dequeue_burst supports two ways of dequeuing data. If the data fits into a buffer, then all data is copied and a single linear buffer is returned. Otherwise it allocates additional mbufs and chains them together to return a multiple segments mbuf.

While that covers most use cases, it forces applications that need to work with larger data sizes to support multiple segments mbufs. The non-linear characteristic brings complexity and performance implications to the application.

To resolve the issue, add support to attach external buffer to a pktmbuf and let the host provide during registration if attaching an external buffer to pktmbuf is supported and if only linear buffer are supported.

---
 doc/guides/prog_guide/vhost_lib.rst |  35 +++++++++
 lib/librte_vhost/rte_vhost.h        |   4 ++
 lib/librte_vhost/socket.c           |  22 ++++++
 lib/librte_vhost/vhost.c            |  22 ++++++
 lib/librte_vhost/vhost.h            |   4 ++
 lib/librte_vhost/virtio_net.c       | 108 ++++++++++++++++++++++++----
 6 files changed, 181 insertions(+), 14 deletions(-)

- Changelog:
V3:
- prevent the new features to be used with zero copy
- fixed sizeof() usage
- fixed log msg indentation
- removed/replaced asserts
- used the correct virt2iova function
- fixed the patch's title
- OvS PoC code:
https://github.com/fleitner/ovs/tree/rte_malloc-v3
V2:
- Used rte_malloc() instead of another mempool as suggested by Shahaf.
- Added the documentation section.
- Using driver registration to negotiate the features.
- OvS PoC code:
https://github.com/fleitner/ovs/commit/8fc197c40b1d4fda331686a7b919e9e2b670dda7

doc/guides/prog_guide/vhost_lib.rst +35 -0
index fc3ee4353..07e40e3c5 100644
@@ -117,6 +117,41 @@ The following is an overview of some key Vhost API functions:
     Enabling this flag should only be done when the calling application does
     not pre-fault the guest shared memory, otherwise migration would fail.
 
+  - ``RTE_VHOST_USER_LINEARBUF_SUPPORT``
+
+    Enabling this flag forces vhost dequeue function to only provide linear
+    pktmbuf (no multi-segmented pktmbuf).
+
+    The vhost library by default provides a single pktmbuf for given a
+    packet, but if for some reason the data doesn't fit into a single
+    pktmbuf (e.g., TSO is enabled), the library will allocate additional
+    pktmbufs from the same mempool and chain them together to create a
+    multi-segmented pktmbuf.
+
+    However, the vhost application needs to support multi-segmented format.
+    If the vhost application does not support that format and requires large
+    buffers to be dequeue, this flag should be enabled to force only linear
+    buffers (see RTE_VHOST_USER_EXTBUF_SUPPORT) or drop the packet.
+
+    It is disabled by default.
+
+  - ``RTE_VHOST_USER_EXTBUF_SUPPORT``
+
+    Enabling this flag allows vhost dequeue function to allocate and attach
+    an external buffer to a pktmbuf if the pkmbuf doesn't provide enough
+    space to store all data.
+
+    This is useful when the vhost application wants to support large packets
+    but doesn't want to increase the default mempool object size nor to
+    support multi-segmented mbufs (non-linear). In this case, a fresh buffer
+    is allocated using rte_malloc() which gets attached to a pktmbuf using
+    rte_pktmbuf_attach_extbuf().
+
+    See RTE_VHOST_USER_LINEARBUF_SUPPORT as well to disable multi-segmented
+    mbufs for application that doesn't support chained mbufs.
+
+    It is disabled by default.
+
 * ``rte_vhost_driver_set_features(path, features)``
 
   This function sets the feature bits the vhost-user driver supports. The
lib/librte_vhost/rte_vhost.h +4 -0
index 19474bca0..b821b5df4 100644
@@ -30,6 +30,10 @@ extern "C" {
 #define RTE_VHOST_USER_DEQUEUE_ZERO_COPY	(1ULL << 2)
 #define RTE_VHOST_USER_IOMMU_SUPPORT	(1ULL << 3)
 #define RTE_VHOST_USER_POSTCOPY_SUPPORT		(1ULL << 4)
+/* support mbuf with external buffer attached */
+#define RTE_VHOST_USER_EXTBUF_SUPPORT	(1ULL << 5)
+/* support only linear buffers (no chained mbufs) */
+#define RTE_VHOST_USER_LINEARBUF_SUPPORT	(1ULL << 6)
 
 /** Protocol features. */
 #ifndef VHOST_USER_PROTOCOL_F_MQ
lib/librte_vhost/socket.c +22 -0
index 274988c4d..e546be2a8 100644
@@ -40,6 +40,8 @@ struct vhost_user_socket {
 	bool dequeue_zero_copy;
 	bool iommu_support;
 	bool use_builtin_virtio_net;
+	bool extbuf;
+	bool linearbuf;
 
 	/*
 	 * The "supported_features" indicates the feature bits the
@@ -232,6 +234,12 @@ vhost_user_add_connection(int fd, struct vhost_user_socket *vsocket)
 	if (vsocket->dequeue_zero_copy)
 		vhost_enable_dequeue_zero_copy(vid);
 
+	if (vsocket->extbuf)
+		vhost_enable_extbuf(vid);
+
+	if (vsocket->linearbuf)
+		vhost_enable_linearbuf(vid);
+
 	RTE_LOG(INFO, VHOST_CONFIG, "new device, handle is %d\n", vid);
 
 	if (vsocket->notify_ops->new_connection) {
@@ -870,6 +878,8 @@ rte_vhost_driver_register(const char *path, uint64_t flags)
 		goto out_free;
 	}
 	vsocket->dequeue_zero_copy = flags & RTE_VHOST_USER_DEQUEUE_ZERO_COPY;
+	vsocket->extbuf = flags & RTE_VHOST_USER_EXTBUF_SUPPORT;
+	vsocket->linearbuf = flags & RTE_VHOST_USER_LINEARBUF_SUPPORT;
 
 	/*
 	 * Set the supported features correctly for the builtin vhost-user
@@ -894,6 +904,18 @@ rte_vhost_driver_register(const char *path, uint64_t flags)
 	 * not compatible with postcopy.
 	 */
 	if (vsocket->dequeue_zero_copy) {
+		if (vsocket->extbuf) {
+			RTE_LOG(ERR, VHOST_CONFIG,
+			"error: zero copy is incompatible with external buffers\n");
+			ret = -1;
+			goto out_free;
+		}
+		if (vsocket->linearbuf) {
+			RTE_LOG(ERR, VHOST_CONFIG,
+			"error: zero copy is incompatible with linear buffers\n");
+			ret = -1;
+			goto out_free;
+		}
 		vsocket->supported_features &= ~(1ULL << VIRTIO_F_IN_ORDER);
 		vsocket->features &= ~(1ULL << VIRTIO_F_IN_ORDER);
 
lib/librte_vhost/vhost.c +22 -0
index cea44df8c..77457f538 100644
@@ -605,6 +605,28 @@ vhost_set_builtin_virtio_net(int vid, bool enable)
 		dev->flags &= ~VIRTIO_DEV_BUILTIN_VIRTIO_NET;
 }
 
+void
+vhost_enable_extbuf(int vid)
+{
+	struct virtio_net *dev = get_device(vid);
+
+	if (dev == NULL)
+		return;
+
+	dev->extbuf = 1;
+}
+
+void
+vhost_enable_linearbuf(int vid)
+{
+	struct virtio_net *dev = get_device(vid);
+
+	if (dev == NULL)
+		return;
+
+	dev->linearbuf = 1;
+}
+
 int
 rte_vhost_get_mtu(int vid, uint16_t *mtu)
 {
lib/librte_vhost/vhost.h +4 -0
index 5131a97a3..0346bd118 100644
@@ -302,6 +302,8 @@ struct virtio_net {
 	rte_atomic16_t		broadcast_rarp;
 	uint32_t		nr_vring;
 	int			dequeue_zero_copy;
+	int			extbuf;
+	int			linearbuf;
 	struct vhost_virtqueue	*virtqueue[VHOST_MAX_QUEUE_PAIRS * 2];
 #define IF_NAME_SZ (PATH_MAX > IFNAMSIZ ? PATH_MAX : IFNAMSIZ)
 	char			ifname[IF_NAME_SZ];
@@ -476,6 +478,8 @@ void vhost_attach_vdpa_device(int vid, int did);
 void vhost_set_ifname(int, const char *if_name, unsigned int if_len);
 void vhost_enable_dequeue_zero_copy(int vid);
 void vhost_set_builtin_virtio_net(int vid, bool enable);
+void vhost_enable_extbuf(int vid);
+void vhost_enable_linearbuf(int vid);
 
 struct vhost_device_ops const *vhost_driver_callback_get(const char *path);
 
lib/librte_vhost/virtio_net.c +94 -14

152 lines not on this page · raw message

Signature · 1 line
2.20.1
#16 lore raw

Re: [dpdk-dev] [PATCH v3] vhost: add support for large buffers

Quoted 17 lines · Flavio Leitner

The rte_vhost_dequeue_burst supports two ways of dequeuing data. If the data fits into a buffer, then all data is copied and a single linear buffer is returned. Otherwise it allocates additional mbufs and chains them together to return a multiple segments mbuf.

While that covers most use cases, it forces applications that need to work with larger data sizes to support multiple segments mbufs. The non-linear characteristic brings complexity and performance implications to the application.

To resolve the issue, add support to attach external buffer to a pktmbuf and let the host provide during registration if attaching an external buffer to pktmbuf is supported and if only linear buffer are supported.

Quoted 8 lines
---
 doc/guides/prog_guide/vhost_lib.rst |  35 +++++++++
 lib/librte_vhost/rte_vhost.h        |   4 ++
 lib/librte_vhost/socket.c           |  22 ++++++
 lib/librte_vhost/vhost.c            |  22 ++++++
 lib/librte_vhost/vhost.h            |   4 ++
 lib/librte_vhost/virtio_net.c       | 108 ++++++++++++++++++++++++----
 6 files changed, 181 insertions(+), 14 deletions(-)

Thanks for the new version!
Overall looks good to me, few minor comments inlined.

lib/librte_vhost/virtio_net.c get_zmbuf(struct vhost_virtqueue *vq) · line 1309

Quoted 12 lines
lib/librte_vhost/virtio_net.c +12 -0
+static int
+virtio_dev_extbuf_alloc(struct rte_mbuf *pkt, uint16_t size)
+{
+	struct rte_mbuf_ext_shared_info *shinfo = NULL;
+	uint16_t buf_len = 0;
+	rte_iova_t iova;
+	void *buf;
+
+	/* Try to use pkt buffer to store shinfo to reduce the amount of memory
+	 * required, otherwise store shinfo in the new buffer.
+	 */
+	if (rte_pktmbuf_tailroom(pkt) > sizeof(*shinfo))

How about "rte_pktmbuf_tailroom(pkt) >= sizeof(*shinfo)"?

lib/librte_vhost/virtio_net.c get_zmbuf(struct vhost_virtqueue *vq) · line 1313

Quoted 4 lines
lib/librte_vhost/virtio_net.c +4 -0
+		shinfo = rte_pktmbuf_mtod(pkt,
+					  struct rte_mbuf_ext_shared_info *);
+	else
+		buf_len += sizeof(*shinfo);

The space taken by rte_pktmbuf_ext_shinfo_init_helper() may be bigger than this depending on buf_end.

https://github.com/DPDK/dpdk/blob/cc8627bc6d7a7bd3ccc2653b746aac4fbaa0bc50/lib/librte_mbuf/rte_mbuf.h#L1585-L1589

lib/librte_vhost/virtio_net.c get_zmbuf(struct vhost_virtqueue *vq) · line 1316

Quoted 3 lines
lib/librte_vhost/virtio_net.c +3 -0
+
+	if (unlikely(buf_len + size + RTE_PKTMBUF_HEADROOM > UINT16_MAX)) {
+		RTE_LOG(ERR, VHOST_DATA,

We may have fallback available when extbuf alloc fails, logging this as error may be too noisy.

Quoted 10 lines
lib/librte_vhost/virtio_net.c +10 -0
+			"buffer size %d exceeded maximum.\n", buf_len);
+		return -ENOSPC;
+	}
+
+	buf_len += size + RTE_PKTMBUF_HEADROOM;
+	buf = rte_malloc(NULL, buf_len, RTE_CACHE_LINE_SIZE);
+	if (unlikely(buf == NULL))
+		return -ENOMEM;
+
+	/* initialize shinfo */

lib/librte_vhost/virtio_net.c get_zmbuf(struct vhost_virtqueue *vq) · line 1336

Quoted 10 lines
lib/librte_vhost/virtio_net.c +10 -0
+	if (shinfo) {
+		shinfo->free_cb = virtio_dev_extbuf_free;
+		shinfo->fcb_opaque = buf;
+		rte_mbuf_ext_refcnt_set(shinfo, 1);
+	} else {
+		shinfo = rte_pktmbuf_ext_shinfo_init_helper(buf, &buf_len,
+					      virtio_dev_extbuf_free, buf);
+		if (unlikely(shinfo == NULL)) {
+			RTE_LOG(ERR, VHOST_DATA, "Failed to init shinfo\n");
+			return -1;

buf should be freed.

Quoted 15 lines
lib/librte_vhost/virtio_net.c +2 -2
@@ -1343,26 +1429,23 @@ virtio_dev_tx_split(struct virtio_net *dev, struct vhost_virtqueue *vq,
 	for (i = 0; i < count; i++) {
 		struct buf_vector buf_vec[BUF_VECTOR_MAX];
 		uint16_t head_idx;
-		uint32_t dummy_len;
+		uint32_t buf_len;
 		uint16_t nr_vec = 0;
 		int err;
 
 		if (unlikely(fill_vec_buf_split(dev, vq,
 						vq->last_avail_idx + i,
 						&nr_vec, buf_vec,
-						&head_idx, &dummy_len,
+						&head_idx, &buf_len,
 						VHOST_ACCESS_RO) < 0))

lib/librte_vhost/virtio_net.c virtio_dev_tx_split(struct virtio_net *dev, struct vhost_virtqueue *vq, · line 1446

Quoted 10 lines
lib/librte_vhost/virtio_net.c +1 -4
 			break;
 
 		if (likely(dev->dequeue_zero_copy == 0))
 			update_shadow_used_ring_split(vq, head_idx, 0);
 
-		pkts[i] = rte_pktmbuf_alloc(mbuf_pool);
-		if (unlikely(pkts[i] == NULL)) {
-			RTE_LOG(ERR, VHOST_DATA,
-				"Failed to allocate memory for mbuf.\n");
+		pkts[i] = virtio_dev_pktmbuf_alloc(dev, mbuf_pool, buf_len);

The len in descriptor set by guests is 32bit, we may not want to trim it to 16bit unconditionally like this.

Quoted 24 lines
lib/librte_vhost/virtio_net.c +3 -3
+		if (unlikely(pkts[i] == NULL))
 			break;
-		}
 
 		err = copy_desc_to_mbuf(dev, vq, buf_vec, nr_vec, pkts[i],
 				mbuf_pool);
@@ -1451,14 +1534,14 @@ virtio_dev_tx_packed(struct virtio_net *dev, struct vhost_virtqueue *vq,
 	for (i = 0; i < count; i++) {
 		struct buf_vector buf_vec[BUF_VECTOR_MAX];
 		uint16_t buf_id;
-		uint32_t dummy_len;
+		uint32_t buf_len;
 		uint16_t desc_count, nr_vec = 0;
 		int err;
 
 		if (unlikely(fill_vec_buf_packed(dev, vq,
 						vq->last_avail_idx, &desc_count,
 						buf_vec, &nr_vec,
-						&buf_id, &dummy_len,
+						&buf_id, &buf_len,
 						VHOST_ACCESS_RO) < 0))
 			break;
 

lib/librte_vhost/virtio_net.c virtio_dev_tx_packed(struct virtio_net *dev, struct vhost_virtqueue *vq, · line 1552

Quoted 8 lines
lib/librte_vhost/virtio_net.c +1 -4
@@ -1466,12 +1549,9 @@ virtio_dev_tx_packed(struct virtio_net *dev, struct vhost_virtqueue *vq,
 			update_shadow_used_ring_packed(vq, buf_id, 0,
 					desc_count);
 
-		pkts[i] = rte_pktmbuf_alloc(mbuf_pool);
-		if (unlikely(pkts[i] == NULL)) {
-			RTE_LOG(ERR, VHOST_DATA,
-				"Failed to allocate memory for mbuf.\n");
+		pkts[i] = virtio_dev_pktmbuf_alloc(dev, mbuf_pool, buf_len);

Ditto.

Quoted 9 lines
lib/librte_vhost/virtio_net.c +1 -1
+		if (unlikely(pkts[i] == NULL))
 			break;
-		}
 
 		err = copy_desc_to_mbuf(dev, vq, buf_vec, nr_vec, pkts[i],
 				mbuf_pool);
Signature · 1 line
2.20.1
#17 lore raw

PATCH v4 1/1 vhost: add support for large buffers

In reply to Flavio Leitner

The rte_vhost_dequeue_burst supports two ways of dequeuing data. If the data fits into a buffer, then all data is copied and a single linear buffer is returned. Otherwise it allocates additional mbufs and chains them together to return a multiple segments mbuf.

While that covers most use cases, it forces applications that need to work with larger data sizes to support multiple segments mbufs. The non-linear characteristic brings complexity and performance implications to the application.

To resolve the issue, add support to attach external buffer to a pktmbuf and let the host provide during registration if attaching an external buffer to pktmbuf is supported and if only linear buffer are supported.

---
 doc/guides/prog_guide/vhost_lib.rst |  35 +++++++++
 lib/librte_vhost/rte_vhost.h        |   4 +
 lib/librte_vhost/socket.c           |  22 ++++++
 lib/librte_vhost/vhost.c            |  22 ++++++
 lib/librte_vhost/vhost.h            |   4 +
 lib/librte_vhost/virtio_net.c       | 109 ++++++++++++++++++++++++----
 6 files changed, 182 insertions(+), 14 deletions(-)

- Changelog:
v4:
- allow to use pktmbuf if there is exact space
- removed log message if the buffer is too big
- fixed the length to include align padding
- free allocated buf if shinfo fails
v3:
- prevent the new features to be used with zero copy
- fixed sizeof() usage
- fixed log msg indentation
- removed/replaced asserts
- used the correct virt2iova function
- fixed the patch's title
- OvS PoC code:
https://github.com/fleitner/ovs/tree/rte_malloc-v3
v2:
- Used rte_malloc() instead of another mempool as suggested by Shahaf.
- Added the documentation section.
- Using driver registration to negotiate the features.
- OvS PoC code:
https://github.com/fleitner/ovs/commit/8fc197c40b1d4fda331686a7b919e9e2b670dda7

doc/guides/prog_guide/vhost_lib.rst +35 -0
index fc3ee4353..07e40e3c5 100644
@@ -117,6 +117,41 @@ The following is an overview of some key Vhost API functions:
     Enabling this flag should only be done when the calling application does
     not pre-fault the guest shared memory, otherwise migration would fail.
 
+  - ``RTE_VHOST_USER_LINEARBUF_SUPPORT``
+
+    Enabling this flag forces vhost dequeue function to only provide linear
+    pktmbuf (no multi-segmented pktmbuf).
+
+    The vhost library by default provides a single pktmbuf for given a
+    packet, but if for some reason the data doesn't fit into a single
+    pktmbuf (e.g., TSO is enabled), the library will allocate additional
+    pktmbufs from the same mempool and chain them together to create a
+    multi-segmented pktmbuf.
+
+    However, the vhost application needs to support multi-segmented format.
+    If the vhost application does not support that format and requires large
+    buffers to be dequeue, this flag should be enabled to force only linear
+    buffers (see RTE_VHOST_USER_EXTBUF_SUPPORT) or drop the packet.
+
+    It is disabled by default.
+
+  - ``RTE_VHOST_USER_EXTBUF_SUPPORT``
+
+    Enabling this flag allows vhost dequeue function to allocate and attach
+    an external buffer to a pktmbuf if the pkmbuf doesn't provide enough
+    space to store all data.
+
+    This is useful when the vhost application wants to support large packets
+    but doesn't want to increase the default mempool object size nor to
+    support multi-segmented mbufs (non-linear). In this case, a fresh buffer
+    is allocated using rte_malloc() which gets attached to a pktmbuf using
+    rte_pktmbuf_attach_extbuf().
+
+    See RTE_VHOST_USER_LINEARBUF_SUPPORT as well to disable multi-segmented
+    mbufs for application that doesn't support chained mbufs.
+
+    It is disabled by default.
+
 * ``rte_vhost_driver_set_features(path, features)``
 
   This function sets the feature bits the vhost-user driver supports. The
lib/librte_vhost/rte_vhost.h +4 -0
index 19474bca0..b821b5df4 100644
@@ -30,6 +30,10 @@ extern "C" {
 #define RTE_VHOST_USER_DEQUEUE_ZERO_COPY	(1ULL << 2)
 #define RTE_VHOST_USER_IOMMU_SUPPORT	(1ULL << 3)
 #define RTE_VHOST_USER_POSTCOPY_SUPPORT		(1ULL << 4)
+/* support mbuf with external buffer attached */
+#define RTE_VHOST_USER_EXTBUF_SUPPORT	(1ULL << 5)
+/* support only linear buffers (no chained mbufs) */
+#define RTE_VHOST_USER_LINEARBUF_SUPPORT	(1ULL << 6)
 
 /** Protocol features. */
 #ifndef VHOST_USER_PROTOCOL_F_MQ
lib/librte_vhost/socket.c +22 -0
index 274988c4d..e546be2a8 100644
@@ -40,6 +40,8 @@ struct vhost_user_socket {
 	bool dequeue_zero_copy;
 	bool iommu_support;
 	bool use_builtin_virtio_net;
+	bool extbuf;
+	bool linearbuf;
 
 	/*
 	 * The "supported_features" indicates the feature bits the
@@ -232,6 +234,12 @@ vhost_user_add_connection(int fd, struct vhost_user_socket *vsocket)
 	if (vsocket->dequeue_zero_copy)
 		vhost_enable_dequeue_zero_copy(vid);
 
+	if (vsocket->extbuf)
+		vhost_enable_extbuf(vid);
+
+	if (vsocket->linearbuf)
+		vhost_enable_linearbuf(vid);
+
 	RTE_LOG(INFO, VHOST_CONFIG, "new device, handle is %d\n", vid);
 
 	if (vsocket->notify_ops->new_connection) {
@@ -870,6 +878,8 @@ rte_vhost_driver_register(const char *path, uint64_t flags)
 		goto out_free;
 	}
 	vsocket->dequeue_zero_copy = flags & RTE_VHOST_USER_DEQUEUE_ZERO_COPY;
+	vsocket->extbuf = flags & RTE_VHOST_USER_EXTBUF_SUPPORT;
+	vsocket->linearbuf = flags & RTE_VHOST_USER_LINEARBUF_SUPPORT;
 
 	/*
 	 * Set the supported features correctly for the builtin vhost-user
@@ -894,6 +904,18 @@ rte_vhost_driver_register(const char *path, uint64_t flags)
 	 * not compatible with postcopy.
 	 */
 	if (vsocket->dequeue_zero_copy) {
+		if (vsocket->extbuf) {
+			RTE_LOG(ERR, VHOST_CONFIG,
+			"error: zero copy is incompatible with external buffers\n");
+			ret = -1;
+			goto out_free;
+		}
+		if (vsocket->linearbuf) {
+			RTE_LOG(ERR, VHOST_CONFIG,
+			"error: zero copy is incompatible with linear buffers\n");
+			ret = -1;
+			goto out_free;
+		}
 		vsocket->supported_features &= ~(1ULL << VIRTIO_F_IN_ORDER);
 		vsocket->features &= ~(1ULL << VIRTIO_F_IN_ORDER);
 
lib/librte_vhost/vhost.c +22 -0
index cea44df8c..77457f538 100644
@@ -605,6 +605,28 @@ vhost_set_builtin_virtio_net(int vid, bool enable)
 		dev->flags &= ~VIRTIO_DEV_BUILTIN_VIRTIO_NET;
 }
 
+void
+vhost_enable_extbuf(int vid)
+{
+	struct virtio_net *dev = get_device(vid);
+
+	if (dev == NULL)
+		return;
+
+	dev->extbuf = 1;
+}
+
+void
+vhost_enable_linearbuf(int vid)
+{
+	struct virtio_net *dev = get_device(vid);
+
+	if (dev == NULL)
+		return;
+
+	dev->linearbuf = 1;
+}
+
 int
 rte_vhost_get_mtu(int vid, uint16_t *mtu)
 {
lib/librte_vhost/vhost.h +4 -0
index 5131a97a3..0346bd118 100644
@@ -302,6 +302,8 @@ struct virtio_net {
 	rte_atomic16_t		broadcast_rarp;
 	uint32_t		nr_vring;
 	int			dequeue_zero_copy;
+	int			extbuf;
+	int			linearbuf;
 	struct vhost_virtqueue	*virtqueue[VHOST_MAX_QUEUE_PAIRS * 2];
 #define IF_NAME_SZ (PATH_MAX > IFNAMSIZ ? PATH_MAX : IFNAMSIZ)
 	char			ifname[IF_NAME_SZ];
@@ -476,6 +478,8 @@ void vhost_attach_vdpa_device(int vid, int did);
 void vhost_set_ifname(int, const char *if_name, unsigned int if_len);
 void vhost_enable_dequeue_zero_copy(int vid);
 void vhost_set_builtin_virtio_net(int vid, bool enable);
+void vhost_enable_extbuf(int vid);
+void vhost_enable_linearbuf(int vid);
 
 struct vhost_device_ops const *vhost_driver_callback_get(const char *path);
 
lib/librte_vhost/virtio_net.c +95 -14

153 lines not on this page · raw message

Signature · 1 line
2.20.1
#18 lore raw

Re: [dpdk-dev] [PATCH v4] vhost: add support for large buffers

Hi.
Not a full review. Few comments inline.

Best regards, Ilya Maximets.

Quoted 148 lines · Flavio Leitner

The rte_vhost_dequeue_burst supports two ways of dequeuing data. If the data fits into a buffer, then all data is copied and a single linear buffer is returned. Otherwise it allocates additional mbufs and chains them together to return a multiple segments mbuf.

While that covers most use cases, it forces applications that need to work with larger data sizes to support multiple segments mbufs. The non-linear characteristic brings complexity and performance implications to the application.

To resolve the issue, add support to attach external buffer to a pktmbuf and let the host provide during registration if attaching an external buffer to pktmbuf is supported and if only linear buffer are supported.

---
  doc/guides/prog_guide/vhost_lib.rst |  35 +++++++++
  lib/librte_vhost/rte_vhost.h        |   4 +
  lib/librte_vhost/socket.c           |  22 ++++++
  lib/librte_vhost/vhost.c            |  22 ++++++
  lib/librte_vhost/vhost.h            |   4 +
  lib/librte_vhost/virtio_net.c       | 109 ++++++++++++++++++++++++----
  6 files changed, 182 insertions(+), 14 deletions(-)

- Changelog:
v4:
- allow to use pktmbuf if there is exact space
- removed log message if the buffer is too big
- fixed the length to include align padding
- free allocated buf if shinfo fails
v3:
- prevent the new features to be used with zero copy
- fixed sizeof() usage
- fixed log msg indentation
- removed/replaced asserts
- used the correct virt2iova function
- fixed the patch's title
- OvS PoC code:
https://github.com/fleitner/ovs/tree/rte_malloc-v3
v2:
- Used rte_malloc() instead of another mempool as suggested by Shahaf.
- Added the documentation section.
- Using driver registration to negotiate the features.
- OvS PoC code:
https://github.com/fleitner/ovs/commit/8fc197c40b1d4fda331686a7b919e9e2b670dda7

doc/guides/prog_guide/vhost_lib.rst +35 -0
index fc3ee4353..07e40e3c5 100644
@@ -117,6 +117,41 @@ The following is an overview of some key Vhost API functions:
      Enabling this flag should only be done when the calling application does
      not pre-fault the guest shared memory, otherwise migration would fail.
 
+  - ``RTE_VHOST_USER_LINEARBUF_SUPPORT``
+
+    Enabling this flag forces vhost dequeue function to only provide linear
+    pktmbuf (no multi-segmented pktmbuf).
+
+    The vhost library by default provides a single pktmbuf for given a
+    packet, but if for some reason the data doesn't fit into a single
+    pktmbuf (e.g., TSO is enabled), the library will allocate additional
+    pktmbufs from the same mempool and chain them together to create a
+    multi-segmented pktmbuf.
+
+    However, the vhost application needs to support multi-segmented format.
+    If the vhost application does not support that format and requires large
+    buffers to be dequeue, this flag should be enabled to force only linear
+    buffers (see RTE_VHOST_USER_EXTBUF_SUPPORT) or drop the packet.
+
+    It is disabled by default.
+
+  - ``RTE_VHOST_USER_EXTBUF_SUPPORT``
+
+    Enabling this flag allows vhost dequeue function to allocate and attach
+    an external buffer to a pktmbuf if the pkmbuf doesn't provide enough
+    space to store all data.
+
+    This is useful when the vhost application wants to support large packets
+    but doesn't want to increase the default mempool object size nor to
+    support multi-segmented mbufs (non-linear). In this case, a fresh buffer
+    is allocated using rte_malloc() which gets attached to a pktmbuf using
+    rte_pktmbuf_attach_extbuf().
+
+    See RTE_VHOST_USER_LINEARBUF_SUPPORT as well to disable multi-segmented
+    mbufs for application that doesn't support chained mbufs.
+
+    It is disabled by default.
+
  * ``rte_vhost_driver_set_features(path, features)``
 
    This function sets the feature bits the vhost-user driver supports. The
lib/librte_vhost/rte_vhost.h +4 -0
index 19474bca0..b821b5df4 100644
@@ -30,6 +30,10 @@ extern "C" {
  #define RTE_VHOST_USER_DEQUEUE_ZERO_COPY	(1ULL << 2)
  #define RTE_VHOST_USER_IOMMU_SUPPORT	(1ULL << 3)
  #define RTE_VHOST_USER_POSTCOPY_SUPPORT		(1ULL << 4)
+/* support mbuf with external buffer attached */
+#define RTE_VHOST_USER_EXTBUF_SUPPORT	(1ULL << 5)
+/* support only linear buffers (no chained mbufs) */
+#define RTE_VHOST_USER_LINEARBUF_SUPPORT	(1ULL << 6)
 
  /** Protocol features. */
  #ifndef VHOST_USER_PROTOCOL_F_MQ
lib/librte_vhost/socket.c +10 -0
index 274988c4d..e546be2a8 100644
@@ -40,6 +40,8 @@ struct vhost_user_socket {
  	bool dequeue_zero_copy;
  	bool iommu_support;
  	bool use_builtin_virtio_net;
+	bool extbuf;
+	bool linearbuf;
 
  	/*
  	 * The "supported_features" indicates the feature bits the
@@ -232,6 +234,12 @@ vhost_user_add_connection(int fd, struct vhost_user_socket *vsocket)
  	if (vsocket->dequeue_zero_copy)
  		vhost_enable_dequeue_zero_copy(vid);
 
+	if (vsocket->extbuf)
+		vhost_enable_extbuf(vid);
+
+	if (vsocket->linearbuf)
+		vhost_enable_linearbuf(vid);
+
  	RTE_LOG(INFO, VHOST_CONFIG, "new device, handle is %d\n", vid);
 
  	if (vsocket->notify_ops->new_connection) {
@@ -870,6 +878,8 @@ rte_vhost_driver_register(const char *path, uint64_t flags)
  		goto out_free;
  	}
  	vsocket->dequeue_zero_copy = flags & RTE_VHOST_USER_DEQUEUE_ZERO_COPY;
+	vsocket->extbuf = flags & RTE_VHOST_USER_EXTBUF_SUPPORT;
+	vsocket->linearbuf = flags & RTE_VHOST_USER_LINEARBUF_SUPPORT;
 
  	/*
  	 * Set the supported features correctly for the builtin vhost-user

lib/librte_vhost/socket.c rte_vhost_driver_register(const char *path, uint64_t flags) · line 911

Quoted 8 lines
lib/librte_vhost/socket.c +5 -0
@@ -894,6 +904,18 @@ rte_vhost_driver_register(const char *path, uint64_t flags)
  	 * not compatible with postcopy.
  	 */
  	if (vsocket->dequeue_zero_copy) {
+		if (vsocket->extbuf) {
+			RTE_LOG(ERR, VHOST_CONFIG,
+			"error: zero copy is incompatible with external buffers\n");
+			ret = -1;
+			goto out_free;

There should be 'out_mutex'.

lib/librte_vhost/socket.c rte_vhost_driver_register(const char *path, uint64_t flags) · line 917

Quoted 6 lines
lib/librte_vhost/socket.c +6 -0
+		}
+		if (vsocket->linearbuf) {
+			RTE_LOG(ERR, VHOST_CONFIG,
+			"error: zero copy is incompatible with linear buffers\n");
+			ret = -1;
+			goto out_free;

Ditto.

Quoted 135 lines
lib/librte_vhost/socket.c +1 -0
+		}
  		vsocket->supported_features &= ~(1ULL << VIRTIO_F_IN_ORDER);
  		vsocket->features &= ~(1ULL << VIRTIO_F_IN_ORDER);
 
lib/librte_vhost/vhost.c +22 -0
index cea44df8c..77457f538 100644
@@ -605,6 +605,28 @@ vhost_set_builtin_virtio_net(int vid, bool enable)
  		dev->flags &= ~VIRTIO_DEV_BUILTIN_VIRTIO_NET;
  }
 
+void
+vhost_enable_extbuf(int vid)
+{
+	struct virtio_net *dev = get_device(vid);
+
+	if (dev == NULL)
+		return;
+
+	dev->extbuf = 1;
+}
+
+void
+vhost_enable_linearbuf(int vid)
+{
+	struct virtio_net *dev = get_device(vid);
+
+	if (dev == NULL)
+		return;
+
+	dev->linearbuf = 1;
+}
+
  int
  rte_vhost_get_mtu(int vid, uint16_t *mtu)
  {
lib/librte_vhost/vhost.h +4 -0
index 5131a97a3..0346bd118 100644
@@ -302,6 +302,8 @@ struct virtio_net {
  	rte_atomic16_t		broadcast_rarp;
  	uint32_t		nr_vring;
  	int			dequeue_zero_copy;
+	int			extbuf;
+	int			linearbuf;
  	struct vhost_virtqueue	*virtqueue[VHOST_MAX_QUEUE_PAIRS * 2];
  #define IF_NAME_SZ (PATH_MAX > IFNAMSIZ ? PATH_MAX : IFNAMSIZ)
  	char			ifname[IF_NAME_SZ];
@@ -476,6 +478,8 @@ void vhost_attach_vdpa_device(int vid, int did);
  void vhost_set_ifname(int, const char *if_name, unsigned int if_len);
  void vhost_enable_dequeue_zero_copy(int vid);
  void vhost_set_builtin_virtio_net(int vid, bool enable);
+void vhost_enable_extbuf(int vid);
+void vhost_enable_linearbuf(int vid);
 
  struct vhost_device_ops const *vhost_driver_callback_get(const char *path);
 
lib/librte_vhost/virtio_net.c +68 -0
index 5b85b832d..da69ab1db 100644
@@ -1289,6 +1289,93 @@ get_zmbuf(struct vhost_virtqueue *vq)
  	return NULL;
  }
 
+static void
+virtio_dev_extbuf_free(void *addr __rte_unused, void *opaque)
+{
+	rte_free(opaque);
+}
+
+static int
+virtio_dev_extbuf_alloc(struct rte_mbuf *pkt, uint32_t size)
+{
+	struct rte_mbuf_ext_shared_info *shinfo = NULL;
+	uint32_t total_len = RTE_PKTMBUF_HEADROOM + size;
+	uint16_t buf_len;
+	rte_iova_t iova;
+	void *buf;
+
+	/* Try to use pkt buffer to store shinfo to reduce the amount of memory
+	 * required, otherwise store shinfo in the new buffer.
+	 */
+	if (rte_pktmbuf_tailroom(pkt) >= sizeof(*shinfo))
+		shinfo = rte_pktmbuf_mtod(pkt,
+					  struct rte_mbuf_ext_shared_info *);
+	else {
+		total_len += sizeof(*shinfo) + sizeof(uintptr_t);
+		total_len = RTE_ALIGN_CEIL(total_len, sizeof(uintptr_t));
+	}
+
+	if (unlikely(total_len > UINT16_MAX))
+		return -ENOSPC;
+
+	buf_len = total_len;
+	buf = rte_malloc(NULL, buf_len, RTE_CACHE_LINE_SIZE);
+	if (unlikely(buf == NULL))
+		return -ENOMEM;
+
+	/* Initialize shinfo */
+	if (shinfo) {
+		shinfo->free_cb = virtio_dev_extbuf_free;
+		shinfo->fcb_opaque = buf;
+		rte_mbuf_ext_refcnt_set(shinfo, 1);
+	} else {
+		shinfo = rte_pktmbuf_ext_shinfo_init_helper(buf, &buf_len,
+					      virtio_dev_extbuf_free, buf);
+		if (unlikely(shinfo == NULL)) {
+			rte_free(buf);
+			RTE_LOG(ERR, VHOST_DATA, "Failed to init shinfo\n");
+			return -1;
+		}
+	}
+
+	iova = rte_malloc_virt2iova(buf);
+	rte_pktmbuf_attach_extbuf(pkt, buf, iova, buf_len, shinfo);
+	rte_pktmbuf_reset_headroom(pkt);
+
+	return 0;
+}
+
+/*
+ * Allocate a host supported pktmbuf.
+ */
+static __rte_always_inline struct rte_mbuf *
+virtio_dev_pktmbuf_alloc(struct virtio_net *dev, struct rte_mempool *mp,
+			 uint32_t data_len)
+{
+	struct rte_mbuf *pkt = rte_pktmbuf_alloc(mp);
+
+	if (unlikely(pkt == NULL))
+		return NULL;
+

lib/librte_vhost/virtio_net.c get_zmbuf(struct vhost_virtqueue *vq) · line 1369

Quoted 10 lines
lib/librte_vhost/virtio_net.c +10 -0
+	if (rte_pktmbuf_tailroom(pkt) >= data_len)
+		return pkt;
+
+	/* attach an external buffer if supported */
+	if (dev->extbuf && !virtio_dev_extbuf_alloc(pkt, data_len))
+		return pkt;
+
+	/* check if chained buffers are allowed */
+	if (!dev->linearbuf)
+		return pkt;

I guess, checking of the 'linearbuf' should go before checking the 'extbuf'. The usecase is that allocation of several buffers from the memory pool is, probably, faster than rte_malloc() + memory attaching. So, if the 'linearbuf' is not requested, it might be faster to use chained mbufs.

BTW, I'm not sure if we really need 2 separate options for this. i.e. 1. +linear +extbuf --> extbuf allocated
2. +linear -extbuf --> buffer dropped (is this case is useful at all?)
3. -linear +extbuf --> chained mbufs might be preferred (see above)
4. -linear -extbuf --> chained mbufs

Case 4 is a default case. Case 1 is our target case for supporting large buffers. Case 3 might makes no much sense as the result is equal to case 4. Case 2 might be not interesting for as at all, because it will lead to random packet drops depending on their size.

But, if only cases 1 and 4 are valid and interesting to us, we could easily merge both flags.

Thoughts?

#19 lore raw

Re: [dpdk-dev] [PATCH v4] vhost: add support for large buffers

Quoted 4 lines · On Tue, 15 Oct 2019 19:41:52 +0200 Ilya Maximets <[email protected]> wrote:

Hi.
Not a full review. Few comments inline.

Best regards, Ilya Maximets.

Thanks for reviewing Ilya, see below.

[...]

Quoted 15 lines
Quoted 15 lines
(quoted diff) +2 -0
@@ -870,6 +878,8 @@ rte_vhost_driver_register(const char *path,
 uint64_t flags) goto out_free;
  	}
  	vsocket->dequeue_zero_copy = flags &
 RTE_VHOST_USER_DEQUEUE_ZERO_COPY;
+	vsocket->extbuf = flags & RTE_VHOST_USER_EXTBUF_SUPPORT;
+	vsocket->linearbuf = flags &
 RTE_VHOST_USER_LINEARBUF_SUPPORT; 
  	/*

* Set the supported features correctly for the builtin
vhost-user @@ -894,6 +904,18 @@ rte_vhost_driver_register(const char *path, uint64_t flags)
* not compatible with postcopy.
*/
if (vsocket->dequeue_zero_copy) {

Quoted 8 lines

Review

Quoted 6 lines
lib/librte_vhost/socket.c +3 -0
+		if (vsocket->extbuf) {
+			RTE_LOG(ERR, VHOST_CONFIG,
+			"error: zero copy is incompatible with

external buffers\n");

(quoted diff) +2 -0
+			ret = -1;
+			goto out_free;  

There should be 'out_mutex'.

Good catch, going to fix that.

Quoted 10 lines

Review

Quoted 8 lines
lib/librte_vhost/socket.c +4 -0
+		}
+		if (vsocket->linearbuf) {
+			RTE_LOG(ERR, VHOST_CONFIG,
+			"error: zero copy is incompatible with

linear buffers\n");

(quoted diff) +2 -0
+			ret = -1;
+			goto out_free;  

Ditto.

Yeah

[...]

Quoted 21 lines
Quoted 21 lines
lib/librte_vhost/virtio_net.c +5 -0
+/*
+ * Allocate a host supported pktmbuf.
+ */
+static __rte_always_inline struct rte_mbuf *
+virtio_dev_pktmbuf_alloc(struct virtio_net *dev, struct

rte_mempool *mp,

lib/librte_vhost/virtio_net.c +15 -0
+			 uint32_t data_len)
+{
+	struct rte_mbuf *pkt = rte_pktmbuf_alloc(mp);
+
+	if (unlikely(pkt == NULL))
+		return NULL;
+
+	if (rte_pktmbuf_tailroom(pkt) >= data_len)
+		return pkt;
+
+	/* attach an external buffer if supported */
+	if (dev->extbuf && !virtio_dev_extbuf_alloc(pkt, data_len))
+		return pkt;
+
+	/* check if chained buffers are allowed */
Quoted 8 lines

lib/librte_vhost/virtio_net.c get_zmbuf(struct vhost_virtqueue *vq) · line 1369

Quoted 2 lines
lib/librte_vhost/virtio_net.c +2 -0
+	if (!dev->linearbuf)
+		return pkt;  

I guess, checking of the 'linearbuf' should go before checking the 'extbuf'. The usecase is that allocation of several buffers from the memory pool is, probably, faster than rte_malloc() + memory attaching. So, if the 'linearbuf' is not requested, it might be faster to use chained mbufs.

The speed seems to depend on the length of the packet and it is not just the allocation, but the overall overhead to deal with multiple segments. If the preferred way is chained buffers, then just don't pass any of the new flags because that's the default behavior today.

Quoted 7 lines

BTW, I'm not sure if we really need 2 separate options for this. i.e. 1. +linear +extbuf --> extbuf allocated
2. +linear -extbuf --> buffer dropped (is this case is useful
at all?) 3. -linear +extbuf --> chained mbufs might be preferred (see above) 4. -linear -extbuf --> chained mbufs

Case 4 is a default case.

Yes, in this case if the packet length fits into the pktmbuf, a single linear mbuf is returned otherwise it uses chained mbufs.

Quoted 2 lines

Case 1 is our target case for supporting
large buffers.

Right, here it enables external buffers, but we don't want chained mbufs.

Quoted 2 lines

Case 3 might makes no much sense as the result is
equal to case 4.

Well case #3 either uses a single mbuf from mpool or use an external buffer for big packets while in case #4 there is no support for big packets at all.

Quoted 1 line

Case 2 might be not interesting for as at all,

Case #2 says the app only supports what fits into a single mbuf and no external buffer nor chained buffers is supported.

Quoted 1 line

because it will lead to random packet drops depending on their size.

Either that or wrong processing.

Quoted 2 lines

But, if only cases 1 and 4 are valid and interesting to us, we could easily merge both flags.

For OvS use-case today yes, but I created different flags because they seemed different useful features to me and changing flags later can get the API very confusing.

Thanks!
fbl

#20 lore raw

PATCH v5 1/1 vhost: add support for large buffers

In reply to Flavio Leitner

The rte_vhost_dequeue_burst supports two ways of dequeuing data. If the data fits into a buffer, then all data is copied and a single linear buffer is returned. Otherwise it allocates additional mbufs and chains them together to return a multiple segments mbuf.

While that covers most use cases, it forces applications that need to work with larger data sizes to support multiple segments mbufs. The non-linear characteristic brings complexity and performance implications to the application.

To resolve the issue, add support to attach external buffer to a pktmbuf and let the host provide during registration if attaching an external buffer to pktmbuf is supported and if only linear buffer are supported.

---
 doc/guides/prog_guide/vhost_lib.rst |  35 +++++++++
 lib/librte_vhost/rte_vhost.h        |   4 +
 lib/librte_vhost/socket.c           |  22 ++++++
 lib/librte_vhost/vhost.c            |  22 ++++++
 lib/librte_vhost/vhost.h            |   4 +
 lib/librte_vhost/virtio_net.c       | 109 ++++++++++++++++++++++++----
 6 files changed, 182 insertions(+), 14 deletions(-)

- Changelog:
v5:
- fixed to destroy mutex if incompatible flags
v4:
- allow to use pktmbuf if there is exact space
- removed log message if the buffer is too big
- fixed the length to include align padding
- free allocated buf if shinfo fails
v3:
- prevent the new features to be used with zero copy
- fixed sizeof() usage
- fixed log msg indentation
- removed/replaced asserts
- used the correct virt2iova function
- fixed the patch's title
- OvS PoC code:
https://github.com/fleitner/ovs/tree/rte_malloc-v3
v2:
- Used rte_malloc() instead of another mempool as suggested by Shahaf.
- Added the documentation section.
- Using driver registration to negotiate the features.
- OvS PoC code:
https://github.com/fleitner/ovs/commit/8fc197c40b1d4fda331686a7b919e9e2b670dda7

doc/guides/prog_guide/vhost_lib.rst +35 -0
index fc3ee4353..07e40e3c5 100644
@@ -117,6 +117,41 @@ The following is an overview of some key Vhost API functions:
     Enabling this flag should only be done when the calling application does
     not pre-fault the guest shared memory, otherwise migration would fail.
 
+  - ``RTE_VHOST_USER_LINEARBUF_SUPPORT``
+
+    Enabling this flag forces vhost dequeue function to only provide linear
+    pktmbuf (no multi-segmented pktmbuf).
+
+    The vhost library by default provides a single pktmbuf for given a
+    packet, but if for some reason the data doesn't fit into a single
+    pktmbuf (e.g., TSO is enabled), the library will allocate additional
+    pktmbufs from the same mempool and chain them together to create a
+    multi-segmented pktmbuf.
+
+    However, the vhost application needs to support multi-segmented format.
+    If the vhost application does not support that format and requires large
+    buffers to be dequeue, this flag should be enabled to force only linear
+    buffers (see RTE_VHOST_USER_EXTBUF_SUPPORT) or drop the packet.
+
+    It is disabled by default.
+
+  - ``RTE_VHOST_USER_EXTBUF_SUPPORT``
+
+    Enabling this flag allows vhost dequeue function to allocate and attach
+    an external buffer to a pktmbuf if the pkmbuf doesn't provide enough
+    space to store all data.
+
+    This is useful when the vhost application wants to support large packets
+    but doesn't want to increase the default mempool object size nor to
+    support multi-segmented mbufs (non-linear). In this case, a fresh buffer
+    is allocated using rte_malloc() which gets attached to a pktmbuf using
+    rte_pktmbuf_attach_extbuf().
+
+    See RTE_VHOST_USER_LINEARBUF_SUPPORT as well to disable multi-segmented
+    mbufs for application that doesn't support chained mbufs.
+
+    It is disabled by default.
+
 * ``rte_vhost_driver_set_features(path, features)``
 
   This function sets the feature bits the vhost-user driver supports. The
lib/librte_vhost/rte_vhost.h +4 -0
index 19474bca0..b821b5df4 100644
@@ -30,6 +30,10 @@ extern "C" {
 #define RTE_VHOST_USER_DEQUEUE_ZERO_COPY	(1ULL << 2)
 #define RTE_VHOST_USER_IOMMU_SUPPORT	(1ULL << 3)
 #define RTE_VHOST_USER_POSTCOPY_SUPPORT		(1ULL << 4)
+/* support mbuf with external buffer attached */
+#define RTE_VHOST_USER_EXTBUF_SUPPORT	(1ULL << 5)
+/* support only linear buffers (no chained mbufs) */
+#define RTE_VHOST_USER_LINEARBUF_SUPPORT	(1ULL << 6)
 
 /** Protocol features. */
 #ifndef VHOST_USER_PROTOCOL_F_MQ
lib/librte_vhost/socket.c +22 -0
index 274988c4d..a8d433c49 100644
@@ -40,6 +40,8 @@ struct vhost_user_socket {
 	bool dequeue_zero_copy;
 	bool iommu_support;
 	bool use_builtin_virtio_net;
+	bool extbuf;
+	bool linearbuf;
 
 	/*
 	 * The "supported_features" indicates the feature bits the
@@ -232,6 +234,12 @@ vhost_user_add_connection(int fd, struct vhost_user_socket *vsocket)
 	if (vsocket->dequeue_zero_copy)
 		vhost_enable_dequeue_zero_copy(vid);
 
+	if (vsocket->extbuf)
+		vhost_enable_extbuf(vid);
+
+	if (vsocket->linearbuf)
+		vhost_enable_linearbuf(vid);
+
 	RTE_LOG(INFO, VHOST_CONFIG, "new device, handle is %d\n", vid);
 
 	if (vsocket->notify_ops->new_connection) {
@@ -870,6 +878,8 @@ rte_vhost_driver_register(const char *path, uint64_t flags)
 		goto out_free;
 	}
 	vsocket->dequeue_zero_copy = flags & RTE_VHOST_USER_DEQUEUE_ZERO_COPY;
+	vsocket->extbuf = flags & RTE_VHOST_USER_EXTBUF_SUPPORT;
+	vsocket->linearbuf = flags & RTE_VHOST_USER_LINEARBUF_SUPPORT;
 
 	/*
 	 * Set the supported features correctly for the builtin vhost-user
@@ -894,6 +904,18 @@ rte_vhost_driver_register(const char *path, uint64_t flags)
 	 * not compatible with postcopy.
 	 */
 	if (vsocket->dequeue_zero_copy) {
+		if (vsocket->extbuf) {
+			RTE_LOG(ERR, VHOST_CONFIG,
+			"error: zero copy is incompatible with external buffers\n");
+			ret = -1;
+			goto out_mutex;
+		}
+		if (vsocket->linearbuf) {
+			RTE_LOG(ERR, VHOST_CONFIG,
+			"error: zero copy is incompatible with linear buffers\n");
+			ret = -1;
+			goto out_mutex;
+		}
 		vsocket->supported_features &= ~(1ULL << VIRTIO_F_IN_ORDER);
 		vsocket->features &= ~(1ULL << VIRTIO_F_IN_ORDER);
 
lib/librte_vhost/vhost.c +22 -0
index cea44df8c..77457f538 100644
@@ -605,6 +605,28 @@ vhost_set_builtin_virtio_net(int vid, bool enable)
 		dev->flags &= ~VIRTIO_DEV_BUILTIN_VIRTIO_NET;
 }
 
+void
+vhost_enable_extbuf(int vid)
+{
+	struct virtio_net *dev = get_device(vid);
+
+	if (dev == NULL)
+		return;
+
+	dev->extbuf = 1;
+}
+
+void
+vhost_enable_linearbuf(int vid)
+{
+	struct virtio_net *dev = get_device(vid);
+
+	if (dev == NULL)
+		return;
+
+	dev->linearbuf = 1;
+}
+
 int
 rte_vhost_get_mtu(int vid, uint16_t *mtu)
 {
lib/librte_vhost/vhost.h +4 -0
index 5131a97a3..0346bd118 100644
@@ -302,6 +302,8 @@ struct virtio_net {
 	rte_atomic16_t		broadcast_rarp;
 	uint32_t		nr_vring;
 	int			dequeue_zero_copy;
+	int			extbuf;
+	int			linearbuf;
 	struct vhost_virtqueue	*virtqueue[VHOST_MAX_QUEUE_PAIRS * 2];
 #define IF_NAME_SZ (PATH_MAX > IFNAMSIZ ? PATH_MAX : IFNAMSIZ)
 	char			ifname[IF_NAME_SZ];
@@ -476,6 +478,8 @@ void vhost_attach_vdpa_device(int vid, int did);
 void vhost_set_ifname(int, const char *if_name, unsigned int if_len);
 void vhost_enable_dequeue_zero_copy(int vid);
 void vhost_set_builtin_virtio_net(int vid, bool enable);
+void vhost_enable_extbuf(int vid);
+void vhost_enable_linearbuf(int vid);
 
 struct vhost_device_ops const *vhost_driver_callback_get(const char *path);
 
lib/librte_vhost/virtio_net.c +95 -14

153 lines not on this page · raw message

Signature · 1 line
2.20.1
Reviewed-by on v5 1/1
#21 lore raw

Re: [dpdk-dev] [PATCH v5] vhost: add support for large buffers

Hi Flavio,

Quoted 41 lines · Flavio Leitner

The rte_vhost_dequeue_burst supports two ways of dequeuing data. If the data fits into a buffer, then all data is copied and a single linear buffer is returned. Otherwise it allocates additional mbufs and chains them together to return a multiple segments mbuf.

While that covers most use cases, it forces applications that need to work with larger data sizes to support multiple segments mbufs. The non-linear characteristic brings complexity and performance implications to the application.

To resolve the issue, add support to attach external buffer to a pktmbuf and let the host provide during registration if attaching an external buffer to pktmbuf is supported and if only linear buffer are supported.

---
 doc/guides/prog_guide/vhost_lib.rst |  35 +++++++++
 lib/librte_vhost/rte_vhost.h        |   4 +
 lib/librte_vhost/socket.c           |  22 ++++++
 lib/librte_vhost/vhost.c            |  22 ++++++
 lib/librte_vhost/vhost.h            |   4 +
 lib/librte_vhost/virtio_net.c       | 109 ++++++++++++++++++++++++----
 6 files changed, 182 insertions(+), 14 deletions(-)

- Changelog:
v5:
- fixed to destroy mutex if incompatible flags
v4:
- allow to use pktmbuf if there is exact space
- removed log message if the buffer is too big
- fixed the length to include align padding
- free allocated buf if shinfo fails
v3:
- prevent the new features to be used with zero copy
- fixed sizeof() usage
- fixed log msg indentation
- removed/replaced asserts
- used the correct virt2iova function
- fixed the patch's title

Quoted 9 lines

- OvS PoC code:
https://github.com/fleitner/ovs/tree/rte_malloc-v3
v2:
- Used rte_malloc() instead of another mempool as suggested by Shahaf.
- Added the documentation section.
- Using driver registration to negotiate the features.
- OvS PoC code:
https://github.com/fleitner/ovs/commit/8fc197c40b1d4fda331686a7b919e9e2b670dda7

I went through the patch, and it looks good to me.
I appreciate the fact that there is no rte_vhost_dequeue_burst API change.

Thanks,
Maxime

#22 lore raw

Re: [dpdk-dev] [PATCH v5] vhost: add support for large buffers

In reply to Flavio Leitner
Quoted 41 lines · Flavio Leitner

The rte_vhost_dequeue_burst supports two ways of dequeuing data. If the data fits into a buffer, then all data is copied and a single linear buffer is returned. Otherwise it allocates additional mbufs and chains them together to return a multiple segments mbuf.

While that covers most use cases, it forces applications that need to work with larger data sizes to support multiple segments mbufs. The non-linear characteristic brings complexity and performance implications to the application.

To resolve the issue, add support to attach external buffer to a pktmbuf and let the host provide during registration if attaching an external buffer to pktmbuf is supported and if only linear buffer are supported.

---
 doc/guides/prog_guide/vhost_lib.rst |  35 +++++++++
 lib/librte_vhost/rte_vhost.h        |   4 +
 lib/librte_vhost/socket.c           |  22 ++++++
 lib/librte_vhost/vhost.c            |  22 ++++++
 lib/librte_vhost/vhost.h            |   4 +
 lib/librte_vhost/virtio_net.c       | 109 ++++++++++++++++++++++++----
 6 files changed, 182 insertions(+), 14 deletions(-)

- Changelog:
v5:
- fixed to destroy mutex if incompatible flags
v4:
- allow to use pktmbuf if there is exact space
- removed log message if the buffer is too big
- fixed the length to include align padding
- free allocated buf if shinfo fails
v3:
- prevent the new features to be used with zero copy
- fixed sizeof() usage
- fixed log msg indentation
- removed/replaced asserts
- used the correct virt2iova function
- fixed the patch's title

Quoted 8 lines

- OvS PoC code:
https://github.com/fleitner/ovs/tree/rte_malloc-v3
v2:
- Used rte_malloc() instead of another mempool as suggested by Shahaf.
- Added the documentation section.
- Using driver registration to negotiate the features.
- OvS PoC code:
https://github.com/fleitner/ovs/commit/8fc197c40b1d4fda331686a7b919e9e2b670dda7

Applied to dpdk-next-virtio/master.

Thanks,
Maxime

#23 lore raw

Re: [dpdk-dev] [PATCH v5] vhost: add support for large buffers

Quoted 53 lines · Maxime Coquelin
Quoted 41 lines · Flavio Leitner

The rte_vhost_dequeue_burst supports two ways of dequeuing data. If the data fits into a buffer, then all data is copied and a single linear buffer is returned. Otherwise it allocates additional mbufs and chains them together to return a multiple segments mbuf.

While that covers most use cases, it forces applications that need to work with larger data sizes to support multiple segments mbufs. The non-linear characteristic brings complexity and performance implications to the application.

To resolve the issue, add support to attach external buffer to a pktmbuf and let the host provide during registration if attaching an external buffer to pktmbuf is supported and if only linear buffer are supported.

---
  doc/guides/prog_guide/vhost_lib.rst |  35 +++++++++
  lib/librte_vhost/rte_vhost.h        |   4 +
  lib/librte_vhost/socket.c           |  22 ++++++
  lib/librte_vhost/vhost.c            |  22 ++++++
  lib/librte_vhost/vhost.h            |   4 +
  lib/librte_vhost/virtio_net.c       | 109 ++++++++++++++++++++++++----
  6 files changed, 182 insertions(+), 14 deletions(-)

- Changelog:
v5:
- fixed to destroy mutex if incompatible flags
v4:
- allow to use pktmbuf if there is exact space
- removed log message if the buffer is too big
- fixed the length to include align padding
- free allocated buf if shinfo fails
v3:
- prevent the new features to be used with zero copy
- fixed sizeof() usage
- fixed log msg indentation
- removed/replaced asserts
- used the correct virt2iova function
- fixed the patch's title

Quoted 8 lines

- OvS PoC code:
https://github.com/fleitner/ovs/tree/rte_malloc-v3
v2:
- Used rte_malloc() instead of another mempool as suggested by Shahaf.
- Added the documentation section.
- Using driver registration to negotiate the features.
- OvS PoC code:
https://github.com/fleitner/ovs/commit/8fc197c40b1d4fda331686a7b919e9e2b670dda7

Quoted 1 line

Applied to dpdk-next-virtio/master.

Thanks Maxime,

But can we return back the mbuf allocation failure message?

I mean:

lib/librte_vhost/virtio_net.c +4 -1
index 66f0c7206..f8af4e0b3 100644
@@ -1354,8 +1354,11 @@ virtio_dev_pktmbuf_alloc(struct virtio_net *dev, struct rte_mempool *mp,
 {
 	struct rte_mbuf *pkt = rte_pktmbuf_alloc(mp);
 
-	if (unlikely(pkt == NULL))
+	if (unlikely(pkt == NULL)) {
+		RTE_LOG(ERR, VHOST_DATA,
+			"Failed to allocate memory for mbuf.\n");
 		return NULL;
+	}
 
 	if (rte_pktmbuf_tailroom(pkt) >= data_len)
 		return pkt;

---

It's a hard failure that highlights some significant issues with memory pool size or a mbuf leak.

We still have the message for subsequent chained mbufs, but not for the first one. Without this error message we could never notice that something is wrong with our memory pool. Only the network traffic will stop flowing.
The message was very useful previously while catching the root cause of the mbuf leak in OVS.

I could send a separate patch for this if needed.

Best regards, Ilya Maximets.

#24 lore raw

Re: [dpdk-dev] [PATCH v5] vhost: add support for large buffers

Quoted 57 lines · Ilya Maximets
Quoted 56 lines · Maxime Coquelin
Quoted 52 lines · Flavio Leitner

The rte_vhost_dequeue_burst supports two ways of dequeuing data. If the data fits into a buffer, then all data is copied and a single linear buffer is returned. Otherwise it allocates additional mbufs and chains them together to return a multiple segments mbuf.

While that covers most use cases, it forces applications that need to work with larger data sizes to support multiple segments mbufs. The non-linear characteristic brings complexity and performance implications to the application.

To resolve the issue, add support to attach external buffer to a pktmbuf and let the host provide during registration if attaching an external buffer to pktmbuf is supported and if only linear buffer are supported.

---
  doc/guides/prog_guide/vhost_lib.rst |  35 +++++++++ lib/librte_vhost/rte_vhost.h        |   4 + lib/librte_vhost/socket.c           |  22 ++++++ lib/librte_vhost/vhost.c            |  22 ++++++ lib/librte_vhost/vhost.h            |   4 + lib/librte_vhost/virtio_net.c       | 109 ++++++++++++++++++++++++----

  6 files changed, 182 insertions(+), 14 deletions(-)

- Changelog:
   v5:
     - fixed to destroy mutex if incompatible flags
   v4:
     - allow to use pktmbuf if there is exact space
     - removed log message if the buffer is too big
     - fixed the length to include align padding
     - free allocated buf if shinfo fails
   v3:
     - prevent the new features to be used with zero copy
     - fixed sizeof() usage
     - fixed log msg indentation
     - removed/replaced asserts
     - used the correct virt2iova function
     - fixed the patch's title
     - OvS PoC code:
       https://github.com/fleitner/ovs/tree/rte_malloc-v3
   v2:
     - Used rte_malloc() instead of another mempool as suggested by Shahaf.
     - Added the documentation section.
     - Using driver registration to negotiate the features.
     - OvS PoC code:

https://github.com/fleitner/ovs/commit/8fc197c40b1d4fda331686a7b919e9e2b670dda7

Quoted 5 lines
Quoted 1 line

Applied to dpdk-next-virtio/master.

Thanks Maxime,

But can we return back the mbuf allocation failure message?

Good point, I missed it.

Quoted 22 lines

I mean:

lib/librte_vhost/virtio_net.c +0 -0
index 66f0c7206..f8af4e0b3 100644
@@ -1354,8 +1354,11 @@ virtio_dev_pktmbuf_alloc(struct virtio_net *dev,
 struct rte_mempool *mp,
  {
      struct rte_mbuf *pkt = rte_pktmbuf_alloc(mp);


-    if (unlikely(pkt == NULL))
+    if (unlikely(pkt == NULL)) {
+        RTE_LOG(ERR, VHOST_DATA,
+            "Failed to allocate memory for mbuf.\n"); return NULL;
+    }

     if (rte_pktmbuf_tailroom(pkt) >= data_len) return pkt;
---

Quoted 2 lines

It's a hard failure that highlights some significant issues with memory pool size or a mbuf leak.

I agree, we need this error message.

Quoted 8 lines

We still have the message for subsequent chained mbufs, but not for the first one.  Without this error message we could never notice that something is wrong with our memory pool.  Only the network traffic will stop flowing.
The message was very useful previously while catching the root cause of the mbuf leak in OVS.

I could send a separate patch for this if needed.

We need a separate patch.
If you want to do it, please do! Otherwise I'll do it.

Thanks!
Maxime

Quoted 1 line

Best regards, Ilya Maximets.

#25 lore raw

Re: [dpdk-dev] [PATCH v5] vhost: add support for large buffers

Quoted 89 lines · On Wed, 16 Oct 2019 15:46:15 +0200 Maxime Coquelin…
Quoted 58 lines · Ilya Maximets
Quoted 57 lines · Maxime Coquelin
Quoted 53 lines · Flavio Leitner

The rte_vhost_dequeue_burst supports two ways of dequeuing data. If the data fits into a buffer, then all data is copied and a single linear buffer is returned. Otherwise it allocates additional mbufs and chains them together to return a multiple segments mbuf.

While that covers most use cases, it forces applications that need to work with larger data sizes to support multiple segments mbufs. The non-linear characteristic brings complexity and performance implications to the application.

To resolve the issue, add support to attach external buffer to a pktmbuf and let the host provide during registration if attaching an external buffer to pktmbuf is supported and if only linear buffer are supported.

---
  doc/guides/prog_guide/vhost_lib.rst |  35 +++++++++ lib/librte_vhost/rte_vhost.h        |   4 + lib/librte_vhost/socket.c           |  22 ++++++ lib/librte_vhost/vhost.c            |  22 ++++++ lib/librte_vhost/vhost.h            |   4 + lib/librte_vhost/virtio_net.c       | 109 ++++++++++++++++++++++++---- 6 files changed, 182 insertions(+), 14 deletions(-)

- Changelog:
   v5:
     - fixed to destroy mutex if incompatible flags v4:
     - allow to use pktmbuf if there is exact space
     - removed log message if the buffer is too big
     - fixed the length to include align padding
     - free allocated buf if shinfo fails
   v3:
     - prevent the new features to be used with zero copy
     - fixed sizeof() usage
     - fixed log msg indentation
     - removed/replaced asserts
     - used the correct virt2iova function
     - fixed the patch's title
     - OvS PoC code:
       https://github.com/fleitner/ovs/tree/rte_malloc-v3
   v2:
     - Used rte_malloc() instead of another mempool as suggested by Shahaf.
     - Added the documentation section.
     - Using driver registration to negotiate the features.
     - OvS PoC code:

https://github.com/fleitner/ovs/commit/8fc197c40b1d4fda331686a7b919e9e2b670dda7

Quoted 5 lines
Quoted 1 line

Applied to dpdk-next-virtio/master.

Thanks Maxime,

But can we return back the mbuf allocation failure message?

Good point, I missed it.

Quoted 22 lines

I mean:

(quoted diff) +0 -0

b/lib/librte_vhost/virtio_net.c index 66f0c7206..f8af4e0b3 100644

lib/librte_vhost/virtio_net.c +0 -0
@@ -1354,8 +1354,11 @@ virtio_dev_pktmbuf_alloc(struct virtio_net
 *dev, struct rte_mempool *mp,
  {
      struct rte_mbuf *pkt = rte_pktmbuf_alloc(mp);


-    if (unlikely(pkt == NULL))
+    if (unlikely(pkt == NULL)) {
+        RTE_LOG(ERR, VHOST_DATA,
+            "Failed to allocate memory for mbuf.\n"); return NULL;
+    }

     if (rte_pktmbuf_tailroom(pkt) >= data_len) return pkt;
---

Quoted 4 lines
Quoted 2 lines

It's a hard failure that highlights some significant issues with memory pool size or a mbuf leak.

I agree, we need this error message.

If it runs out of memory and there are many packets coming, then it will flood the logs. Maybe we could use something to report in a more moderated way, for example:

lib/librte_vhost/virtio_net.c +7 -1
index da69ab1db..b1572b3a4 100644
@@ -1354,8 +1354,14 @@ virtio_dev_pktmbuf_alloc(struct virtio_net *dev, struct rte_mempool *mp,
 {
 	struct rte_mbuf *pkt = rte_pktmbuf_alloc(mp);
 
-	if (unlikely(pkt == NULL))
+	if (unlikely(pkt == NULL)) {
+		static int rate_lim = 0;
+
+		if (rate_lim++ % 1000 == 0)
+			RTE_LOG...
+
 		return NULL;
+	}
 
 	if (rte_pktmbuf_tailroom(pkt) >= data_len)
 		return pkt;

Or track this at mempool level and keep stats of failed requests and report that in OvS. That would cover other points too.

fbl

#26 lore raw

Re: [dpdk-dev] [PATCH v5] vhost: add support for large buffers

Quoted 123 lines · Flavio Leitner
Quoted 85 lines · On Wed, 16 Oct 2019 15:46:15 +0200 Maxime Coquelin…
Quoted 55 lines · Ilya Maximets
Quoted 54 lines · Maxime Coquelin
Quoted 51 lines · Flavio Leitner

The rte_vhost_dequeue_burst supports two ways of dequeuing data. If the data fits into a buffer, then all data is copied and a single linear buffer is returned. Otherwise it allocates additional mbufs and chains them together to return a multiple segments mbuf.

While that covers most use cases, it forces applications that need to work with larger data sizes to support multiple segments mbufs. The non-linear characteristic brings complexity and performance implications to the application.

To resolve the issue, add support to attach external buffer to a pktmbuf and let the host provide during registration if attaching an external buffer to pktmbuf is supported and if only linear buffer are supported.

---
   doc/guides/prog_guide/vhost_lib.rst |  35 +++++++++
   lib/librte_vhost/rte_vhost.h        |   4 +
   lib/librte_vhost/socket.c           |  22 ++++++
   lib/librte_vhost/vhost.c            |  22 ++++++
   lib/librte_vhost/vhost.h            |   4 +
   lib/librte_vhost/virtio_net.c       | 109

++++++++++++++++++++++++---- 6 files changed, 182 insertions(+), 14 deletions(-)

- Changelog:
   v5:
     - fixed to destroy mutex if incompatible flags v4:
     - allow to use pktmbuf if there is exact space
     - removed log message if the buffer is too big
     - fixed the length to include align padding
     - free allocated buf if shinfo fails
   v3:
     - prevent the new features to be used with zero copy
     - fixed sizeof() usage
     - fixed log msg indentation
     - removed/replaced asserts
     - used the correct virt2iova function
     - fixed the patch's title
     - OvS PoC code:
       https://github.com/fleitner/ovs/tree/rte_malloc-v3
   v2:
     - Used rte_malloc() instead of another mempool as suggested
by Shahaf.
     - Added the documentation section.
     - Using driver registration to negotiate the features.
     - OvS PoC code:

Quoted 8 lines
Quoted 4 lines
Quoted 2 lines

https://github.com/fleitner/ovs/commit/8fc197c40b1d4fda331686a7b919e9e2b670dda7

Applied to dpdk-next-virtio/master.

Thanks Maxime,

But can we return back the mbuf allocation failure message?

Good point, I missed it.

Quoted 18 lines

I mean:

(quoted diff) +0 -0

b/lib/librte_vhost/virtio_net.c index 66f0c7206..f8af4e0b3 100644

lib/librte_vhost/virtio_net.c +4 -1
@@ -1354,8 +1354,11 @@ virtio_dev_pktmbuf_alloc(struct virtio_net
 *dev, struct rte_mempool *mp,
  {
      struct rte_mbuf *pkt = rte_pktmbuf_alloc(mp);
 
-    if (unlikely(pkt == NULL))
+    if (unlikely(pkt == NULL)) {
+        RTE_LOG(ERR, VHOST_DATA,
+            "Failed to allocate memory for mbuf.\n");
          return NULL;
+    }
 
Quoted 8 lines
Quoted 6 lines
lib/librte_vhost/virtio_net.c +0 -0
      if (rte_pktmbuf_tailroom(pkt) >= data_len)

         return pkt;
---

It's a hard failure that highlights some significant issues with memory pool size or a mbuf leak.

I agree, we need this error message.

If it runs out of memory and there are many packets coming, then it will flood the logs. Maybe we could use something to report in a more moderated way, for example:

lib/librte_vhost/virtio_net.c +7 -1
index da69ab1db..b1572b3a4 100644
@@ -1354,8 +1354,14 @@ virtio_dev_pktmbuf_alloc(struct virtio_net *dev, struct rte_mempool *mp,
  {
  	struct rte_mbuf *pkt = rte_pktmbuf_alloc(mp);
 
-	if (unlikely(pkt == NULL))
+	if (unlikely(pkt == NULL)) {
+		static int rate_lim = 0;
+
+		if (rate_lim++ % 1000 == 0)
+			RTE_LOG...
+
  		return NULL;
+	}
 
  	if (rte_pktmbuf_tailroom(pkt) >= data_len)
  		return pkt;
Quoted 2 lines

Or track this at mempool level and keep stats of failed requests and report that in OvS. That would cover other points too.

OVS is already rete-limiting DPDK logs. Basically, I introduced this functionality to limit exactly this message.
See: 9fd38f6867df ("netdev-dpdk: Limit rate of DPDK logs.")

Best regards, Ilya Maximets.

#27 lore raw

Re: [dpdk-dev] [PATCH v5] vhost: add support for large buffers

Quoted 124 lines · On Wed, 16 Oct 2019 16:08:54 +0200 Ilya Maximets <[email protected]> wrote:
Quoted 123 lines · Flavio Leitner
Quoted 85 lines · On Wed, 16 Oct 2019 15:46:15 +0200 Maxime Coquelin…
Quoted 55 lines · Ilya Maximets
Quoted 54 lines · Maxime Coquelin
Quoted 51 lines · Flavio Leitner

The rte_vhost_dequeue_burst supports two ways of dequeuing data. If the data fits into a buffer, then all data is copied and a single linear buffer is returned. Otherwise it allocates additional mbufs and chains them together to return a multiple segments mbuf.

While that covers most use cases, it forces applications that need to work with larger data sizes to support multiple segments mbufs. The non-linear characteristic brings complexity and performance implications to the application.

To resolve the issue, add support to attach external buffer to a pktmbuf and let the host provide during registration if attaching an external buffer to pktmbuf is supported and if only linear buffer are supported.

---
   doc/guides/prog_guide/vhost_lib.rst |  35 +++++++++
   lib/librte_vhost/rte_vhost.h        |   4 +
   lib/librte_vhost/socket.c           |  22 ++++++
   lib/librte_vhost/vhost.c            |  22 ++++++
   lib/librte_vhost/vhost.h            |   4 +
   lib/librte_vhost/virtio_net.c       | 109

++++++++++++++++++++++++---- 6 files changed, 182 insertions(+), 14 deletions(-)

- Changelog:
   v5:
     - fixed to destroy mutex if incompatible flags v4:
     - allow to use pktmbuf if there is exact space
     - removed log message if the buffer is too big
     - fixed the length to include align padding
     - free allocated buf if shinfo fails
   v3:
     - prevent the new features to be used with zero copy
     - fixed sizeof() usage
     - fixed log msg indentation
     - removed/replaced asserts
     - used the correct virt2iova function
     - fixed the patch's title
     - OvS PoC code:
       https://github.com/fleitner/ovs/tree/rte_malloc-v3
   v2:
     - Used rte_malloc() instead of another mempool as
suggested by Shahaf.
     - Added the documentation section.
     - Using driver registration to negotiate the features.
     - OvS PoC code:

Quoted 8 lines
Quoted 4 lines
Quoted 2 lines

https://github.com/fleitner/ovs/commit/8fc197c40b1d4fda331686a7b919e9e2b670dda7

Applied to dpdk-next-virtio/master.

Thanks Maxime,

But can we return back the mbuf allocation failure message?

Good point, I missed it.

Quoted 18 lines

I mean:

(quoted diff) +0 -0

b/lib/librte_vhost/virtio_net.c index 66f0c7206..f8af4e0b3 100644

lib/librte_vhost/virtio_net.c +4 -1
@@ -1354,8 +1354,11 @@ virtio_dev_pktmbuf_alloc(struct virtio_net
 *dev, struct rte_mempool *mp,
  {
      struct rte_mbuf *pkt = rte_pktmbuf_alloc(mp);
 
-    if (unlikely(pkt == NULL))
+    if (unlikely(pkt == NULL)) {
+        RTE_LOG(ERR, VHOST_DATA,
+            "Failed to allocate memory for mbuf.\n");
          return NULL;
+    }
 
Quoted 8 lines
Quoted 6 lines
lib/librte_vhost/virtio_net.c +0 -0
      if (rte_pktmbuf_tailroom(pkt) >= data_len)

         return pkt;
---

It's a hard failure that highlights some significant issues with memory pool size or a mbuf leak.

I agree, we need this error message.

If it runs out of memory and there are many packets coming, then it will flood the logs. Maybe we could use something to report in a more moderated way, for example:

(quoted diff) +0 -0

b/lib/librte_vhost/virtio_net.c index da69ab1db..b1572b3a4 100644

lib/librte_vhost/virtio_net.c +7 -1
@@ -1354,8 +1354,14 @@ virtio_dev_pktmbuf_alloc(struct virtio_net
 *dev, struct rte_mempool *mp, {
  	struct rte_mbuf *pkt = rte_pktmbuf_alloc(mp);
 
-	if (unlikely(pkt == NULL))
+	if (unlikely(pkt == NULL)) {
+		static int rate_lim = 0;
+
+		if (rate_lim++ % 1000 == 0)
+			RTE_LOG...
+
  		return NULL;
+	}
 
  	if (rte_pktmbuf_tailroom(pkt) >= data_len)
  		return pkt;
Quoted 6 lines
Quoted 2 lines

Or track this at mempool level and keep stats of failed requests and report that in OvS. That would cover other points too.

OVS is already rete-limiting DPDK logs. Basically, I introduced this functionality to limit exactly this message.
See: 9fd38f6867df ("netdev-dpdk: Limit rate of DPDK logs.")

Thanks, I missed that commit.
fbl

#28 lore raw

Re: [dpdk-dev] [PATCH v5] vhost: add support for large buffers

In reply to Maxime Coquelin
Quoted 102 lines · Maxime Coquelin
Quoted 54 lines · Ilya Maximets
Quoted 53 lines · Maxime Coquelin
Quoted 50 lines · Flavio Leitner

The rte_vhost_dequeue_burst supports two ways of dequeuing data. If the data fits into a buffer, then all data is copied and a single linear buffer is returned. Otherwise it allocates additional mbufs and chains them together to return a multiple segments mbuf.

While that covers most use cases, it forces applications that need to work with larger data sizes to support multiple segments mbufs. The non-linear characteristic brings complexity and performance implications to the application.

To resolve the issue, add support to attach external buffer to a pktmbuf and let the host provide during registration if attaching an external buffer to pktmbuf is supported and if only linear buffer are supported.

---
   doc/guides/prog_guide/vhost_lib.rst |  35 +++++++++
   lib/librte_vhost/rte_vhost.h        |   4 +
   lib/librte_vhost/socket.c           |  22 ++++++
   lib/librte_vhost/vhost.c            |  22 ++++++
   lib/librte_vhost/vhost.h            |   4 +
   lib/librte_vhost/virtio_net.c       | 109 ++++++++++++++++++++++++----
   6 files changed, 182 insertions(+), 14 deletions(-)

- Changelog:
   v5:
     - fixed to destroy mutex if incompatible flags
   v4:
     - allow to use pktmbuf if there is exact space
     - removed log message if the buffer is too big
     - fixed the length to include align padding
     - free allocated buf if shinfo fails
   v3:
     - prevent the new features to be used with zero copy
     - fixed sizeof() usage
     - fixed log msg indentation
     - removed/replaced asserts
     - used the correct virt2iova function
     - fixed the patch's title
     - OvS PoC code:
       https://github.com/fleitner/ovs/tree/rte_malloc-v3
   v2:
     - Used rte_malloc() instead of another mempool as suggested by
Shahaf.
     - Added the documentation section.
     - Using driver registration to negotiate the features.
     - OvS PoC code:

Quoted 8 lines
Quoted 4 lines
Quoted 2 lines

https://github.com/fleitner/ovs/commit/8fc197c40b1d4fda331686a7b919e9e2b670dda7

Applied to dpdk-next-virtio/master.

Thanks Maxime,

But can we return back the mbuf allocation failure message?

Good point, I missed it.

Quoted 22 lines

I mean:

lib/librte_vhost/virtio_net.c +4 -1
index 66f0c7206..f8af4e0b3 100644
@@ -1354,8 +1354,11 @@ virtio_dev_pktmbuf_alloc(struct virtio_net *dev,
 struct rte_mempool *mp,
  {
      struct rte_mbuf *pkt = rte_pktmbuf_alloc(mp);
 
-    if (unlikely(pkt == NULL))
+    if (unlikely(pkt == NULL)) {
+        RTE_LOG(ERR, VHOST_DATA,
+            "Failed to allocate memory for mbuf.\n");
          return NULL;
+    }
 
      if (rte_pktmbuf_tailroom(pkt) >= data_len)

         return pkt;
---

Quoted 2 lines

It's a hard failure that highlights some significant issues with memory pool size or a mbuf leak.

I agree, we need this error message.

Quoted 7 lines

We still have the message for subsequent chained mbufs, but not for the first one.  Without this error message we could never notice that something is wrong with our memory pool.  Only the network traffic will stop flowing.
The message was very useful previously while catching the root cause of the mbuf leak in OVS.

Quoted 4 lines
Quoted 1 line

I could send a separate patch for this if needed.

We need a separate patch.
If you want to do it, please do! Otherwise I'll do it.

OK. I'll send a patch soon.

Best regards, Ilya Maximets.

#29 lore raw

Re: [dpdk-dev] [PATCH v5] vhost: add support for large buffers

In reply to Flavio Leitner
Quoted 29 lines · Flavio Leitner
lib/librte_vhost/virtio_net.c +21 -0
index 5b85b832d..da69ab1db 100644
@@ -1289,6 +1289,93 @@ get_zmbuf(struct vhost_virtqueue *vq)
        return NULL;
 }
 
+static void
+virtio_dev_extbuf_free(void *addr __rte_unused, void *opaque)
+{
+       rte_free(opaque);
+}
+
+static int
+virtio_dev_extbuf_alloc(struct rte_mbuf *pkt, uint32_t size)
+{
+       struct rte_mbuf_ext_shared_info *shinfo = NULL;
+       uint32_t total_len = RTE_PKTMBUF_HEADROOM + size;
+       uint16_t buf_len;
+       rte_iova_t iova;
+       void *buf;
+
+       /* Try to use pkt buffer to store shinfo to reduce the amount of memory
+        * required, otherwise store shinfo in the new buffer.
+        */
+       if (rte_pktmbuf_tailroom(pkt) >= sizeof(*shinfo))
+               shinfo = rte_pktmbuf_mtod(pkt,
+                                         struct rte_mbuf_ext_shared_info *);

lib/librte_vhost/virtio_net.c get_zmbuf(struct vhost_virtqueue *vq) · line 1322

Quoted 10 lines
lib/librte_vhost/virtio_net.c +10 -0
+       else {
+               total_len += sizeof(*shinfo) + sizeof(uintptr_t);
+               total_len = RTE_ALIGN_CEIL(total_len, sizeof(uintptr_t));
+       }
+
+       if (unlikely(total_len > UINT16_MAX))
+               return -ENOSPC;
+
+       buf_len = total_len;
+       buf = rte_malloc(NULL, buf_len, RTE_CACHE_LINE_SIZE);

Using rte_malloc() means that the allocation can end up on any numa node. This external buffer might end up on a different node than the mbuf (which resides on mp->socket_id node).

Signature · 1 line
David Marchand
#30 lore raw

Re: [dpdk-dev] [PATCH v5] vhost: add support for large buffers

Quoted 40 lines · On Tue, 29 Oct 2019 10:02:57 +0100 David Marchand <[email protected]>…
Quoted 38 lines · Flavio Leitner
(quoted diff) +0 -0

b/lib/librte_vhost/virtio_net.c index 5b85b832d..da69ab1db 100644

lib/librte_vhost/virtio_net.c +16 -0
@@ -1289,6 +1289,93 @@ get_zmbuf(struct vhost_virtqueue *vq)
        return NULL;
 }
 
+static void
+virtio_dev_extbuf_free(void *addr __rte_unused, void *opaque)
+{
+       rte_free(opaque);
+}
+
+static int
+virtio_dev_extbuf_alloc(struct rte_mbuf *pkt, uint32_t size)
+{
+       struct rte_mbuf_ext_shared_info *shinfo = NULL;
+       uint32_t total_len = RTE_PKTMBUF_HEADROOM + size;
+       uint16_t buf_len;
+       rte_iova_t iova;
+       void *buf;
+
+       /* Try to use pkt buffer to store shinfo to reduce the

amount of memory

lib/librte_vhost/virtio_net.c +5 -0
+        * required, otherwise store shinfo in the new buffer.
+        */
+       if (rte_pktmbuf_tailroom(pkt) >= sizeof(*shinfo))
+               shinfo = rte_pktmbuf_mtod(pkt,
+                                         struct

rte_mbuf_ext_shared_info *);

lib/librte_vhost/virtio_net.c +3 -0
+       else {
+               total_len += sizeof(*shinfo) + sizeof(uintptr_t);
+               total_len = RTE_ALIGN_CEIL(total_len,

sizeof(uintptr_t));

lib/librte_vhost/virtio_net.c +3 -0
+       }
+
+       if (unlikely(total_len > UINT16_MAX))
Quoted 8 lines

lib/librte_vhost/virtio_net.c get_zmbuf(struct vhost_virtqueue *vq) · line 1322

Quoted 4 lines
lib/librte_vhost/virtio_net.c +4 -0
+               return -ENOSPC;
+
+       buf_len = total_len;
+       buf = rte_malloc(NULL, buf_len, RTE_CACHE_LINE_SIZE);  

Using rte_malloc() means that the allocation can end up on any numa node. This external buffer might end up on a different node than the mbuf (which resides on mp->socket_id node).

Only if there is no memory in the local socket. It seems better than a failure to me.

fbl

#31 lore raw

Re: [dpdk-dev] [PATCH v5] vhost: add support for large buffers

Quoted 8 lines · Flavio Leitner
Quoted 3 lines · On Tue, 29 Oct 2019 10:02:57 +0100 David Marchand <[email protected]>…

Using rte_malloc() means that the allocation can end up on any numa node. This external buffer might end up on a different node than the mbuf (which resides on mp->socket_id node).

Only if there is no memory in the local socket. It seems better than a failure to me.

This won't fail, but you don't know how it will impact the application.

We are moving from a model where the vhost library used a mempool with a fixed number of buffers on a known socket to a model where we can't guarantee how much memory vhost will eat, and where the allocations happen.
I am just scared we will get bitten by random allocation failures on a system that was "used to run fine".

--
David Marchand