Skip to contentSkip to conversation

vhost: add port mirroring function in the vhost lib

RFC In review Cheng Jiang 3 participants 7 messages Aug 14, 2022 – Aug 25, 2022

lore
Share & tools
lore
#0 lore raw

cover vhost: add port mirroring function in the vhost lib

Similar to the port mirroring function on the switch or router, this patch set also implements such function on the Vhost lib. When data is sent to a front-end, it will also send the data to its mirror front-end. When data is received from a front-end, it will also send the data to its mirrorfront-end.

Cheng Jiang (1):
vhost: add egress API for port mirroring datapath

Wenwu Ma (1):
vhost: add ingress API for port mirroring datapath

 lib/vhost/rte_vhost_async.h |   26 +-
 lib/vhost/version.map       |    5 +
 lib/vhost/vhost.h           |    3 +-
 lib/vhost/virtio_net.c      | 2755 ++++++++++++++++++++++++++---------
 4 files changed, 2101 insertions(+), 688 deletions(-)

--
2.35.1

#1 lore raw

RFC 1/2 vhost: add ingress API for port mirroring datapath

From: Wenwu Ma <[email protected]>

Similar to the port mirroring function on the switch or router, this patch also implements an ingress function on the Vhost lib. When data is sent to a front-end, it will also send the data to its mirror front-end.

---
 lib/vhost/rte_vhost_async.h |  14 +-
 lib/vhost/version.map       |   2 +
 lib/vhost/vhost.h           |   3 +-
 lib/vhost/virtio_net.c      | 666 +++++++++++++++++++++++++++++++++++-
 4 files changed, 679 insertions(+), 6 deletions(-)
lib/vhost/rte_vhost_async.h +13 -1
index ad71555a7f..b199af078c 100644
@@ -29,6 +29,7 @@ struct rte_vhost_async_desc {
 	struct rte_vhost_iov_iter *src;
 	/** destination memory iov_iter */
 	struct rte_vhost_iov_iter *dst;
+	struct rte_vhost_iov_iter *mirror_dst;
 };
 
 /**
@@ -64,7 +65,7 @@ struct rte_vhost_async_channel_ops {
 	int32_t (*transfer_data)(int vid, uint16_t queue_id,
 		struct rte_vhost_async_desc *descs,
 		struct rte_vhost_async_status *opaque_data,
-		uint16_t count);
+		uint16_t count, bool mirr_flag);
 	/**
 	 * check copy-completed packets from the async engine
 	 * @param vid
@@ -200,6 +201,12 @@ __rte_experimental
 uint16_t rte_vhost_submit_enqueue_burst(int vid, uint16_t queue_id,
 		struct rte_mbuf **pkts, uint16_t count);
 
+__rte_experimental
+uint16_t
+rte_vhost_submit_ingress_mirroring_burst(int vid, uint16_t queue_id,
+		int mirror_vid, uint16_t mirror_queue_id, struct rte_mbuf **pkts, uint16_t count);
+
+
 /**
  * This function checks async completion status for a specific vhost
  * device queue. Packets which finish copying (enqueue) operation
@@ -220,6 +227,11 @@ __rte_experimental
 uint16_t rte_vhost_poll_enqueue_completed(int vid, uint16_t queue_id,
 		struct rte_mbuf **pkts, uint16_t count);
 
+__rte_experimental
+uint16_t rte_vhost_poll_ingress_completed(int vid, uint16_t queue_id,
+		int mirror_vid, uint16_t mirror_queue_id,
+		struct rte_mbuf **pkts, uint16_t count);
+
 /**
  * This function returns the amount of in-flight packets for the vhost
  * queue which uses async channel acceleration.
lib/vhost/version.map +2 -0
index c92a9d4962..4c35fa4555 100644
@@ -76,6 +76,8 @@ EXPERIMENTAL {
 	rte_vhost_async_channel_unregister;
 	rte_vhost_submit_enqueue_burst;
 	rte_vhost_poll_enqueue_completed;
+	rte_vhost_submit_ingress_mirroring_burst;
+	rte_vhost_poll_ingress_completed;
 
 	# added in 21.05
 	rte_vhost_get_negotiated_protocol_features;
lib/vhost/vhost.h +2 -1
index 1e56311725..89a31e4ca8 100644
@@ -49,7 +49,8 @@
 #define MAX_PKT_BURST 32
 
 #define VHOST_MAX_ASYNC_IT (MAX_PKT_BURST * 2)
-#define VHOST_MAX_ASYNC_VEC (BUF_VECTOR_MAX * 4)
+#define MAX_ASYNC_COPY_VECTOR 1024
+#define VHOST_MAX_ASYNC_VEC (MAX_ASYNC_COPY_VECTOR * 2)
 
 #define PACKED_DESC_ENQUEUE_USED_FLAG(w)	\
 	((w) ? (VRING_DESC_F_AVAIL | VRING_DESC_F_USED | VRING_DESC_F_WRITE) : \
lib/vhost/virtio_net.c +662 -4

696 lines not on this page · raw message

Signature · 1 line
2.35.1
#2 lore raw

Re: [RFC 1/2] vhost: add ingress API for port mirroring datapath

Quoted 9 lines · On Sun, 14 Aug 2022 12:49:19 +0000 Cheng Jiang <[email protected]> wrote:

From: Wenwu Ma <[email protected]>

Similar to the port mirroring function on the switch or router, this patch also implements an ingress function on the Vhost lib. When data is sent to a front-end, it will also send the data to its mirror front-end.

We already have rte_flow, packet capture, and rx/tx callbacks. This seems like re-invention.

#3 lore raw

RE: [RFC 1/2] vhost: add ingress API for port mirroring datapath

Hi,

Quoted 20 lines

-----Original Message-----
From: Stephen Hemminger <[email protected]>
Sent: Sunday, August 14, 2022 10:58 PM
To: Jiang, Cheng1 <[email protected]>
Cc: [email protected]; Xia, Chenbo <[email protected]>; [email protected]; Hu, Jiayu <[email protected]>; Ding, Xuan <[email protected]>; Ma, WenwuX <[email protected]>; Wang, YuanX <[email protected]>; Yang, YvonneX <[email protected]>
Subject: Re: [RFC 1/2] vhost: add ingress API for port mirroring datapath

Quoted 7 lines · On Sun, 14 Aug 2022 12:49:19 +0000 Cheng Jiang <[email protected]> wrote:

From: Wenwu Ma <[email protected]>

Similar to the port mirroring function on the switch or router, this patch also implements an ingress function on the Vhost lib. When data is sent to a front-end, it will also send the data to its mirror front-end.

Quoted 5 lines
Quoted 2 lines

We already have rte_flow, packet capture, and rx/tx callbacks. This seems like re-invention.

Sorry that I didn't make it clear in the v1 commit message. This port mirror function is based on async vhost which is accelerated by DMA device. Compared with other mirror implements: 1. It's targeted for vhost. 2. The performance is really good. Its use scenario is to let one front-end(mirror-VM) monitor the traffic of another front-end(VM). It's different from the things you mentioned above. So, IMO I don't think it's re-invention.

Thanks,
Cheng

#4 lore raw

RE: [RFC 1/2] vhost: add ingress API for port mirroring datapath

Quoted 35 lines

From: Jiang, Cheng1 [mailto:[email protected]]
Sent: Thursday, 18 August 2022 08.58

Hi,

Quoted 10 lines

-----Original Message-----
From: Stephen Hemminger <[email protected]>
Sent: Sunday, August 14, 2022 10:58 PM

Quoted 3 lines · On Sun, 14 Aug 2022 12:49:19 +0000 Cheng Jiang <[email protected]> wrote:

From: Wenwu Ma <[email protected]>

Similar to the port mirroring function on the switch or router,

this

Quoted 1 line
Quoted 1 line

patch also implements an ingress function on the Vhost lib. When

data

Quoted 8 lines
Quoted 5 lines

is sent to a front-end, it will also send the data to its mirror front-end.

We already have rte_flow, packet capture, and rx/tx callbacks. This seems like re-invention.

Sorry that I didn't make it clear in the v1 commit message. This port mirror function is based on async vhost which is accelerated by DMA device. Compared with other mirror implements: 1. It's targeted for vhost. 2. The performance is really good. Its use scenario is to let one front-end(mirror-VM) monitor the traffic of another front-end(VM). It's different from the things you mentioned above. So, IMO I don't think it's re-invention.

Quoted 2 lines

Thanks,
Cheng

Thank you for elaborating the use case.

In other words: This is a performance optimization for a specific use case.

This raises two questions:

1. Please convince us that this is a common use case?
2. What is the performance compared to implementing it at the application level?

Overall, I totally agree with Stephen:

Port Mirroring was removed from ethdev with the 21.11 release [1] for the reasons mentioned by Stephen. Please don't try to reintroduce it.

If the application needs to mirror all (or some) packets, why not just let the application do it?

Furthermore, the application might need to mirror the packets to a physical port, and perhaps encapsulate it for remote packet capture. Sampling could also be required. Here's a prediction for you: More features will creep into to this API over time, and it will end up like the ethdev mirroring API being removed because the alternatives are better and more versatile.

[1]: https://doc.dpdk.org/guides/rel_notes/release_21_11.html#removed-items

#5 lore raw

RE: [RFC 1/2] vhost: add ingress API for port mirroring datapath

Hi,

Quoted 49 lines

-----Original Message-----
From: Morten Brørup <[email protected]>
Sent: Thursday, August 18, 2022 4:17 PM
To: Jiang, Cheng1 <[email protected]>; Stephen Hemminger <[email protected]>
Cc: [email protected]; Xia, Chenbo <[email protected]>; [email protected]; Hu, Jiayu <[email protected]>; Ding, Xuan <[email protected]>; Ma, WenwuX <[email protected]>; Wang, YuanX <[email protected]>; Yang, YvonneX <[email protected]>
Subject: RE: [RFC 1/2] vhost: add ingress API for port mirroring datapath

Quoted 35 lines

From: Jiang, Cheng1 [mailto:[email protected]]
Sent: Thursday, 18 August 2022 08.58

Hi,

Quoted 10 lines

-----Original Message-----
From: Stephen Hemminger <[email protected]>
Sent: Sunday, August 14, 2022 10:58 PM

Quoted 3 lines · On Sun, 14 Aug 2022 12:49:19 +0000 Cheng Jiang <[email protected]> wrote:

From: Wenwu Ma <[email protected]>

Similar to the port mirroring function on the switch or router,

this

Quoted 1 line
Quoted 1 line

patch also implements an ingress function on the Vhost lib. When

data

Quoted 8 lines
Quoted 5 lines

is sent to a front-end, it will also send the data to its mirror front-end.

We already have rte_flow, packet capture, and rx/tx callbacks. This seems like re-invention.

Sorry that I didn't make it clear in the v1 commit message. This port mirror function is based on async vhost which is accelerated by DMA device. Compared with other mirror implements: 1. It's targeted for vhost. 2. The performance is really good. Its use scenario is to let one front-end(mirror-VM) monitor the traffic of another front-end(VM). It's different from the things you mentioned above. So, IMO I don't think it's re-invention.

Quoted 2 lines

Thanks,
Cheng

Quoted 7 lines

Thank you for elaborating the use case.

In other words: This is a performance optimization for a specific use case.

This raises two questions:

1. Please convince us that this is a common use case?

Using VM to monitor the other VM's traffic is a common use case. This is commonly used in an intrusion detection system, passive probe or real user monitoring technology.

Quoted 3 lines

2. What is the performance compared to implementing it at the application level?

This API is introduced mainly to enable DMA acceleration in vhost port mirror scenario. And leverage DMA's strong performance.

Quoted 6 lines

Overall, I totally agree with Stephen:

Port Mirroring was removed from ethdev with the 21.11 release [1] for the reasons mentioned by Stephen. Please don't try to reintroduce it.

Quoted 8 lines

If the application needs to mirror all (or some) packets, why not just let the application do it?

Furthermore, the application might need to mirror the packets to a physical port, and perhaps encapsulate it for remote packet capture. Sampling could also be required. Here's a prediction for you: More features will creep into to this API over time, and it will end up like the ethdev mirroring API being removed because the alternatives are better and more versatile.

First of all, These are Vhost lib APIs based on async vhost which is accelerated by DMA device.
And they are provided for Vhost PMD or other Vhost applications. So, I am not sure whether these APIs are similar to ethdev mirroring APIs which are removed.

As far as I understand, RTE_FLOW is used for ethdev. I'm not sure there are conflicts between mirror API in Vhost and RTE_FLOW, since vhost/virtio is a virtual protocol.

We will consider your suggestion carefully again.
Thank you very much.
Cheng

Quoted 3 lines

[1]: https://doc.dpdk.org/guides/rel_notes/release_21_11.html#removed-
items

#6 lore raw

RFC 2/2 vhost: add egress API for port mirroring datapath

In reply to Cheng Jiang

This patch implements such an egress function on the Vhost lib. When data is received from a front-end, it will also send the data to its mirrorfront-end.

---
 lib/vhost/rte_vhost_async.h |  12 +-
 lib/vhost/version.map       |   3 +
 lib/vhost/virtio_net.c      | 729 ++++++++++++++++++++++++++++++++++++
 3 files changed, 742 insertions(+), 2 deletions(-)
lib/vhost/rte_vhost_async.h +10 -2
index b199af078c..b0772c2db7 100644
@@ -85,11 +85,12 @@ struct rte_vhost_async_channel_ops {
 };
 
 /**
- * inflight async packet information
+ * in-flight async packet information
  */
 struct async_inflight_info {
 	struct rte_mbuf *mbuf;
-	uint16_t descs; /* num of descs inflight */
+	struct virtio_net_hdr nethdr;
+	uint16_t descs; /* num of descs in-flight */
 	uint16_t nr_buffers; /* num of buffers inflight for packed ring */
 };
 
@@ -268,4 +269,11 @@ __rte_experimental
 uint16_t rte_vhost_clear_queue_thread_unsafe(int vid, uint16_t queue_id,
 		struct rte_mbuf **pkts, uint16_t count);
 
+__rte_experimental
+uint16_t
+rte_vhost_async_try_egress_burst(int vid, uint16_t queue_id,
+	int mirr_vid, uint16_t mirr_queue_id,
+	struct rte_mempool *mbuf_pool, struct rte_mbuf **pkts, uint16_t count,
+	int *nr_inflight);
+
 #endif /* _RTE_VHOST_ASYNC_H_ */
lib/vhost/version.map +3 -0
index 4c35fa4555..2529943a91 100644
@@ -87,4 +87,7 @@ EXPERIMENTAL {
 	rte_vhost_async_channel_register_thread_unsafe;
 	rte_vhost_async_channel_unregister_thread_unsafe;
 	rte_vhost_clear_queue_thread_unsafe;
+
+	# added in 21.11
+	rte_vhost_async_try_egress_burst;
 };
lib/vhost/virtio_net.c +729 -0

732 lines not on this page · raw message

Signature · 1 line
2.35.1