Skip to contentSkip to conversation

Some fixes for vhost zero copy

PATCH Applied Tiwei Bie 2 participants 13 messages Feb 22, 2019 – Feb 27, 2019

lore
Share & tools
lore

Tiwei Bie

author
#0 lore raw

cover Some fixes for vhost zero copy

Tiwei Bie (4):
vhost: restore mbuf first when freeing zmbuf
vhost: fix potential use-after-free for zero copy mbuf
vhost: fix potential use-after-free for memory region
doc: improve vhost zero copy guide

 doc/guides/prog_guide/vhost_lib.rst |  3 +++
 lib/librte_vhost/vhost.h            | 34 +++++++++++++++++++++++
 lib/librte_vhost/vhost_user.c       | 42 ++++++++++++++++++++++-------
 lib/librte_vhost/virtio_net.c       | 34 -----------------------
 4 files changed, 70 insertions(+), 43 deletions(-)
Signature · 1 line
2.17.1
author
#1 lore raw

PATCH 1/4 vhost: restore mbuf first when freeing zmbuf

The mbufs should also be restored in free_zmbufs().

Fixes b0a985d1f340 ("vhost: add dequeue zero copy")
Fixes 3ebd930588b7 ("vhost: fix mbuf free")
---
 lib/librte_vhost/vhost.h      | 16 ++++++++++++++++
 lib/librte_vhost/vhost_user.c |  1 +
 lib/librte_vhost/virtio_net.c | 16 ----------------
 3 files changed, 17 insertions(+), 16 deletions(-)
lib/librte_vhost/vhost.h +16 -0
index fc31796bf..bcfce274b 100644
@@ -741,4 +741,20 @@ free_ind_table(void *idesc)
 	rte_free(idesc);
 }
 
+static __rte_always_inline void
+restore_mbuf(struct rte_mbuf *m)
+{
+	uint32_t mbuf_size, priv_size;
+
+	while (m) {
+		priv_size = rte_pktmbuf_priv_size(m->pool);
+		mbuf_size = sizeof(struct rte_mbuf) + priv_size;
+		/* start of buffer is after mbuf structure and priv data */
+
+		m->buf_addr = (char *)m + mbuf_size;
+		m->buf_iova = rte_mempool_virt2iova(m) + mbuf_size;
+		m = m->next;
+	}
+}
+
 #endif /* _VHOST_NET_CDEV_H_ */
lib/librte_vhost/vhost_user.c +1 -0
index b086ad95f..e3ddf2589 100644
@@ -1218,6 +1218,7 @@ free_zmbufs(struct vhost_virtqueue *vq)
 	     zmbuf != NULL; zmbuf = next) {
 		next = TAILQ_NEXT(zmbuf, next);
 
+		restore_mbuf(zmbuf->mbuf);
 		rte_pktmbuf_free(zmbuf->mbuf);
 		TAILQ_REMOVE(&vq->zmbuf_list, zmbuf, next);
 	}
lib/librte_vhost/virtio_net.c +0 -16
index 37a4c00d2..862ca5e1a 100644
@@ -1318,22 +1318,6 @@ mbuf_is_consumed(struct rte_mbuf *m)
 	return true;
 }
 
-static __rte_always_inline void
-restore_mbuf(struct rte_mbuf *m)
-{
-	uint32_t mbuf_size, priv_size;
-
-	while (m) {
-		priv_size = rte_pktmbuf_priv_size(m->pool);
-		mbuf_size = sizeof(struct rte_mbuf) + priv_size;
-		/* start of buffer is after mbuf structure and priv data */
-
-		m->buf_addr = (char *)m + mbuf_size;
-		m->buf_iova = rte_mempool_virt2iova(m) + mbuf_size;
-		m = m->next;
-	}
-}
-
 static __rte_always_inline uint16_t
 virtio_dev_tx_split(struct virtio_net *dev, struct vhost_virtqueue *vq,
 	struct rte_mempool *mbuf_pool, struct rte_mbuf **pkts, uint16_t count)
Signature · 1 line
2.17.1
Reviewed-by on v1 1/4
#2 lore raw

Re: [PATCH 1/4] vhost: restore mbuf first when freeing zmbuf

Quoted 6 lines · Tiwei Bie

The mbufs should also be restored in free_zmbufs().

Fixes b0a985d1f340 ("vhost: add dequeue zero copy")
Fixes 3ebd930588b7 ("vhost: fix mbuf free")
Quoted 7 lines
---
  lib/librte_vhost/vhost.h      | 16 ++++++++++++++++
  lib/librte_vhost/vhost_user.c |  1 +
  lib/librte_vhost/virtio_net.c | 16 ----------------
  3 files changed, 17 insertions(+), 16 deletions(-)

Thanks,
Maxime

author
#3 lore raw

PATCH 2/4 vhost: fix potential use-after-free for zero copy mbuf

In reply to Tiwei Bie

Don't free the zero copy mbufs before they have been consumed, otherwise there could be use-after-free.

Fixes b0a985d1f340 ("vhost: add dequeue zero copy")
---
 lib/librte_vhost/vhost.h      | 12 ++++++++++++
 lib/librte_vhost/vhost_user.c |  3 +++
 lib/librte_vhost/virtio_net.c | 12 ------------
 3 files changed, 15 insertions(+), 12 deletions(-)
lib/librte_vhost/vhost.h +12 -0
index bcfce274b..044651b19 100644
@@ -757,4 +757,16 @@ restore_mbuf(struct rte_mbuf *m)
 	}
 }
 
+static __rte_always_inline bool
+mbuf_is_consumed(struct rte_mbuf *m)
+{
+	while (m) {
+		if (rte_mbuf_refcnt_read(m) > 1)
+			return false;
+		m = m->next;
+	}
+
+	return true;
+}
+
 #endif /* _VHOST_NET_CDEV_H_ */
lib/librte_vhost/vhost_user.c +3 -0
index e3ddf2589..6d8253514 100644
@@ -1218,6 +1218,9 @@ free_zmbufs(struct vhost_virtqueue *vq)
 	     zmbuf != NULL; zmbuf = next) {
 		next = TAILQ_NEXT(zmbuf, next);
 
+		while (!mbuf_is_consumed(zmbuf->mbuf))
+			usleep(1000);
+
 		restore_mbuf(zmbuf->mbuf);
 		rte_pktmbuf_free(zmbuf->mbuf);
 		TAILQ_REMOVE(&vq->zmbuf_list, zmbuf, next);
lib/librte_vhost/virtio_net.c +0 -12
index 862ca5e1a..40a292364 100644
@@ -1306,18 +1306,6 @@ get_zmbuf(struct vhost_virtqueue *vq)
 	return NULL;
 }
 
-static __rte_always_inline bool
-mbuf_is_consumed(struct rte_mbuf *m)
-{
-	while (m) {
-		if (rte_mbuf_refcnt_read(m) > 1)
-			return false;
-		m = m->next;
-	}
-
-	return true;
-}
-
 static __rte_always_inline uint16_t
 virtio_dev_tx_split(struct virtio_net *dev, struct vhost_virtqueue *vq,
 	struct rte_mempool *mbuf_pool, struct rte_mbuf **pkts, uint16_t count)
Signature · 1 line
2.17.1
Reviewed-by on v1 2/4
#4 lore raw

Re: [PATCH 2/4] vhost: fix potential use-after-free for zero copy mbuf

Quoted 6 lines · Tiwei Bie

Don't free the zero copy mbufs before they have been consumed, otherwise there could be use-after-free.

Fixes b0a985d1f340 ("vhost: add dequeue zero copy")
Quoted 6 lines
---
  lib/librte_vhost/vhost.h      | 12 ++++++++++++
  lib/librte_vhost/vhost_user.c |  3 +++
  lib/librte_vhost/virtio_net.c | 12 ------------
  3 files changed, 15 insertions(+), 12 deletions(-)

Thanks,
Maxime

author
#5 lore raw

PATCH 3/4 vhost: fix potential use-after-free for memory region

In reply to Tiwei Bie

Reclaim outstanding zmbufs first before freeing memory regions, otherwise there could be use-after-free.

Fixes b0a985d1f340 ("vhost: add dequeue zero copy")
---
 lib/librte_vhost/vhost.h      |  6 +++++
 lib/librte_vhost/vhost_user.c | 46 +++++++++++++++++++++++++----------
 lib/librte_vhost/virtio_net.c |  6 -----
 3 files changed, 39 insertions(+), 19 deletions(-)
lib/librte_vhost/vhost.h +6 -0
index 044651b19..f008ec43b 100644
@@ -769,4 +769,10 @@ mbuf_is_consumed(struct rte_mbuf *m)
 	return true;
 }
 
+static __rte_always_inline void
+put_zmbuf(struct zcopy_mbuf *zmbuf)
+{
+	zmbuf->in_use = 0;
+}
+
 #endif /* _VHOST_NET_CDEV_H_ */
lib/librte_vhost/vhost_user.c +33 -13

67 lines not on this page · raw message

lib/librte_vhost/virtio_net.c +0 -6
index 40a292364..a6a33a101 100644
@@ -1063,12 +1063,6 @@ vhost_dequeue_offload(struct virtio_net_hdr *hdr, struct rte_mbuf *m)
 	}
 }
 
-static __rte_always_inline void
-put_zmbuf(struct zcopy_mbuf *zmbuf)
-{
-	zmbuf->in_use = 0;
-}
-
 static __rte_always_inline int
 copy_desc_to_mbuf(struct virtio_net *dev, struct vhost_virtqueue *vq,
 		  struct buf_vector *buf_vec, uint16_t nr_vec,
Signature · 1 line
2.17.1
Reviewed-by on v1 3/4
#6 lore raw

Re: [PATCH 3/4] vhost: fix potential use-after-free for memory region

Quoted 6 lines · Tiwei Bie

Reclaim outstanding zmbufs first before freeing memory regions, otherwise there could be use-after-free.

Fixes b0a985d1f340 ("vhost: add dequeue zero copy")
Quoted 6 lines
---
  lib/librte_vhost/vhost.h      |  6 +++++
  lib/librte_vhost/vhost_user.c | 46 +++++++++++++++++++++++++----------
  lib/librte_vhost/virtio_net.c |  6 -----
  3 files changed, 39 insertions(+), 19 deletions(-)

Thanks,
Maxime

author
#7 lore raw

PATCH 4/4 doc: improve vhost zero copy guide

In reply to Tiwei Bie

Highlight that vhost zero copy mbufs should be consumed as soon as possible.

---
 doc/guides/prog_guide/vhost_lib.rst | 3 +++
 1 file changed, 3 insertions(+)
doc/guides/prog_guide/vhost_lib.rst +3 -0
index c77df338f..a86c07a62 100644
@@ -92,6 +92,9 @@ The following is an overview of some key Vhost API functions:
       to use vfio-pci driver, please insert vfio-pci kernel module in noiommu
       mode.
 
+    * The consumer of zero copy mbufs should consume these mbufs as soon as
+      possible, otherwise it may block the operations in vhost.
+
   - ``RTE_VHOST_USER_IOMMU_SUPPORT``
 
     IOMMU support will be enabled when this flag is set. It is disabled by
Signature · 1 line
2.17.1
Reviewed-by on v1 4/4
#8 lore raw

Re: [PATCH 4/4] doc: improve vhost zero copy guide

Quoted 14 lines · Tiwei Bie

Highlight that vhost zero copy mbufs should be consumed as soon as possible.

---
  doc/guides/prog_guide/vhost_lib.rst | 3 +++
  1 file changed, 3 insertions(+)
doc/guides/prog_guide/vhost_lib.rst +0 -0
index c77df338f..a86c07a62 100644

doc/guides/prog_guide/vhost_lib.rst The following is an overview of some key Vhost API functions: · line 100

Quoted 9 lines
doc/guides/prog_guide/vhost_lib.rst +3 -0
@@ -92,6 +92,9 @@ The following is an overview of some key Vhost API functions:
        to use vfio-pci driver, please insert vfio-pci kernel module in noiommu
        mode.
 
+    * The consumer of zero copy mbufs should consume these mbufs as soon as
+      possible, otherwise it may block the operations in vhost.
+
    - ``RTE_VHOST_USER_IOMMU_SUPPORT``
 
      IOMMU support will be enabled when this flag is set. It is disabled by

Indeed, and it may also block QEMU. That's what I find really fragile with dequeue zero-copy.

Thanks,
Maxime

#9 lore raw
In reply to Tiwei Bie
Quoted 6 lines · Tiwei Bie

Tiwei Bie (4):
vhost: restore mbuf first when freeing zmbuf
vhost: fix potential use-after-free for zero copy mbuf
vhost: fix potential use-after-free for memory region
doc: improve vhost zero copy guide

Quoted 6 lines
  doc/guides/prog_guide/vhost_lib.rst |  3 +++
  lib/librte_vhost/vhost.h            | 34 +++++++++++++++++++++++
  lib/librte_vhost/vhost_user.c       | 42 ++++++++++++++++++++++-------
  lib/librte_vhost/virtio_net.c       | 34 -----------------------
  4 files changed, 70 insertions(+), 43 deletions(-)

Looking at the spec, I think we may need also to drain zmbufs in the VHOST_USER_SET_VRING_ENABLE for the disable case:

""
If VHOST_USER_F_PROTOCOL_FEATURES has been negotiated, the ring is initialized in a disabled state. Client must not pass data to/from the backend until ring is enabled by VHOST_USER_SET_VRING_ENABLE with parameter 1, or after it has been disabled by VHOST_USER_SET_VRING_ENABLE with parameter 0.

Each ring is initialized in a stopped state, client must not process it until ring is started, or *after it has been stopped*.
""

Do you take care of this or I send a patch on top?

Thanks,
Maxime

author
#10 lore raw
Quoted 26 lines · Maxime Coquelin
Quoted 6 lines · Tiwei Bie

Tiwei Bie (4):
vhost: restore mbuf first when freeing zmbuf
vhost: fix potential use-after-free for zero copy mbuf
vhost: fix potential use-after-free for memory region
doc: improve vhost zero copy guide

Quoted 6 lines
  doc/guides/prog_guide/vhost_lib.rst |  3 +++
  lib/librte_vhost/vhost.h            | 34 +++++++++++++++++++++++
  lib/librte_vhost/vhost_user.c       | 42 ++++++++++++++++++++++-------
  lib/librte_vhost/virtio_net.c       | 34 -----------------------
  4 files changed, 70 insertions(+), 43 deletions(-)

Looking at the spec, I think we may need also to drain zmbufs in the VHOST_USER_SET_VRING_ENABLE for the disable case:

""
If VHOST_USER_F_PROTOCOL_FEATURES has been negotiated, the ring is initialized
in a disabled state. Client must not pass data to/from the backend until ring is enabled by
VHOST_USER_SET_VRING_ENABLE with parameter 1, or after it has been disabled by
VHOST_USER_SET_VRING_ENABLE with parameter 0.

Quoted 6 lines

Each ring is initialized in a stopped state, client must not process it until
ring is started, or *after it has been stopped*.
""

Do you take care of this or I send a patch on top?

Agree. Please feel free to send any patch on top.

Thanks!
Tiwei

Quoted 3 lines

Thanks,
Maxime

#11 lore raw
Quoted 27 lines · Tiwei Bie
Quoted 26 lines · Maxime Coquelin
Quoted 6 lines · Tiwei Bie

Tiwei Bie (4):
vhost: restore mbuf first when freeing zmbuf
vhost: fix potential use-after-free for zero copy mbuf
vhost: fix potential use-after-free for memory region
doc: improve vhost zero copy guide

Quoted 6 lines
   doc/guides/prog_guide/vhost_lib.rst |  3 +++
   lib/librte_vhost/vhost.h            | 34 +++++++++++++++++++++++
   lib/librte_vhost/vhost_user.c       | 42 ++++++++++++++++++++++-------
   lib/librte_vhost/virtio_net.c       | 34 -----------------------
   4 files changed, 70 insertions(+), 43 deletions(-)

Looking at the spec, I think we may need also to drain zmbufs in the VHOST_USER_SET_VRING_ENABLE for the disable case:

""
If VHOST_USER_F_PROTOCOL_FEATURES has been negotiated, the ring is initialized
in a disabled state. Client must not pass data to/from the backend until ring is enabled by
VHOST_USER_SET_VRING_ENABLE with parameter 1, or after it has been disabled by
VHOST_USER_SET_VRING_ENABLE with parameter 0.

Quoted 8 lines
Quoted 6 lines

Each ring is initialized in a stopped state, client must not process it until
ring is started, or *after it has been stopped*.
""

Do you take care of this or I send a patch on top?

Agree. Please feel free to send any patch on top.

Good, I'll do the patch now.

Maxime

Quoted 6 lines

Thanks!
Tiwei

Quoted 3 lines

Thanks,
Maxime

#12 lore raw
In reply to Tiwei Bie
Quoted 6 lines · Tiwei Bie

Tiwei Bie (4):
vhost: restore mbuf first when freeing zmbuf
vhost: fix potential use-after-free for zero copy mbuf
vhost: fix potential use-after-free for memory region
doc: improve vhost zero copy guide

Quoted 6 lines
  doc/guides/prog_guide/vhost_lib.rst |  3 +++
  lib/librte_vhost/vhost.h            | 34 +++++++++++++++++++++++
  lib/librte_vhost/vhost_user.c       | 42 ++++++++++++++++++++++-------
  lib/librte_vhost/virtio_net.c       | 34 -----------------------
  4 files changed, 70 insertions(+), 43 deletions(-)

Applied to dpdk-next-virtio/master.

Thanks,
Maxime