Skip to contentSkip to conversation

vhost-user: fix enabling of queue pair

PATCH In review Victor Kaplansky 3 participants 6 messages Nov 20, 2015 – Nov 24, 2015

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 vhost-user: fix enabling of queue pair

The VHOST_USER_SET_VRING_ENABLE request is sent for each queue in queue-pair separately. So, a queue-pair should be considered enabled only when both RX and TX queues are enabled.

The old code caused segfault when last TX queue was enabled.

---
 lib/librte_vhost/vhost_user/virtio-net-user.c | 14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)
lib/librte_vhost/vhost_user/virtio-net-user.c +9 -5
index d07452a..c06e53e 100644
@@ -317,21 +317,25 @@ user_set_vring_enable(struct vhost_device_ctx ctx,
 		      struct vhost_vring_state *state)
 {
 	struct virtio_net *dev = get_device(ctx);
-	uint16_t base_idx = state->index;
+	uint16_t idx = state->index;
+	uint16_t base_idx = (idx / VIRTIO_QNUM) * VIRTIO_QNUM;
 	int enable = (int)state->num;
+	int qpair_enabled;
 
 	RTE_LOG(INFO, VHOST_CONFIG,
 		"set queue enable: %d to qp idx: %d\n",
 		enable, state->index);
 
+	dev->virtqueue[idx]->enabled = enable;
+	qpair_enabled =
+		dev->virtqueue[base_idx + VIRTIO_RXQ]->enabled &&
+		dev->virtqueue[base_idx + VIRTIO_TXQ]->enabled;
+
 	if (notify_ops->vring_state_changed) {
 		notify_ops->vring_state_changed(dev, base_idx / VIRTIO_QNUM,
-						enable);
+						qpair_enabled);
 	}
 
-	dev->virtqueue[base_idx + VIRTIO_RXQ]->enabled = enable;
-	dev->virtqueue[base_idx + VIRTIO_TXQ]->enabled = enable;
-
 	return 0;
 }
 
Signature · 1 line
--Victor
#1 lore raw
Quoted 5 lines · Victor Kaplansky

The VHOST_USER_SET_VRING_ENABLE request is sent for each queue in queue-pair separately. So, a queue-pair should be considered enabled only when both RX and TX queues are enabled.

The old code caused segfault when last TX queue was enabled.

Since the problem causes segfault, it's important to review this soon before QEMU 2.5 is out.

Thanks,
-- Victor

Quoted 43 lines
---
 lib/librte_vhost/vhost_user/virtio-net-user.c | 14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)
lib/librte_vhost/vhost_user/virtio-net-user.c +9 -5
index d07452a..c06e53e 100644
@@ -317,21 +317,25 @@ user_set_vring_enable(struct vhost_device_ctx ctx,
 		      struct vhost_vring_state *state)
 {
 	struct virtio_net *dev = get_device(ctx);
-	uint16_t base_idx = state->index;
+	uint16_t idx = state->index;
+	uint16_t base_idx = (idx / VIRTIO_QNUM) * VIRTIO_QNUM;
 	int enable = (int)state->num;
+	int qpair_enabled;
 
 	RTE_LOG(INFO, VHOST_CONFIG,
 		"set queue enable: %d to qp idx: %d\n",
 		enable, state->index);
 
+	dev->virtqueue[idx]->enabled = enable;
+	qpair_enabled =
+		dev->virtqueue[base_idx + VIRTIO_RXQ]->enabled &&
+		dev->virtqueue[base_idx + VIRTIO_TXQ]->enabled;
+
 	if (notify_ops->vring_state_changed) {
 		notify_ops->vring_state_changed(dev, base_idx / VIRTIO_QNUM,
-						enable);
+						qpair_enabled);
 	}
 
-	dev->virtqueue[base_idx + VIRTIO_RXQ]->enabled = enable;
-	dev->virtqueue[base_idx + VIRTIO_TXQ]->enabled = enable;
-
 	return 0;
 }
 
Signature · 1 line
--Victor
#2 lore raw
In reply to Victor Kaplansky
Quoted 3 lines · Victor Kaplansky

The VHOST_USER_SET_VRING_ENABLE request is sent for each queue in queue-pair separately. So, a queue-pair should be considered enabled only when both RX and TX queues are enabled.

Hi Victor,

It was sent per queue-pair, and I found that Michael changed it to per vring. The reason I made it per queue-pair is that the backend can handle both queues in one queue-pair. However, I agree that making it per queue (vring) is better here, to keep the consistency of sending all requests (say vring_call) per vring.

Quoted 25 lines

The old code caused segfault when last TX queue was enabled.

---
 lib/librte_vhost/vhost_user/virtio-net-user.c | 14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)
lib/librte_vhost/vhost_user/virtio-net-user.c +3 -1
index d07452a..c06e53e 100644
@@ -317,21 +317,25 @@ user_set_vring_enable(struct vhost_device_ctx ctx,
 		      struct vhost_vring_state *state)
 {
 	struct virtio_net *dev = get_device(ctx);
-	uint16_t base_idx = state->index;
+	uint16_t idx = state->index;
+	uint16_t base_idx = (idx / VIRTIO_QNUM) * VIRTIO_QNUM;
 	int enable = (int)state->num;
+	int qpair_enabled;
 
 	RTE_LOG(INFO, VHOST_CONFIG,
 		"set queue enable: %d to qp idx: %d\n",
 		enable, state->index);

lib/librte_vhost/vhost_user/virtio-net-user.c user_set_vring_enable(struct vhost_device_ctx ctx, · line 336

Quoted 10 lines
lib/librte_vhost/vhost_user/virtio-net-user.c +6 -1
 
+	dev->virtqueue[idx]->enabled = enable;
+	qpair_enabled =
+		dev->virtqueue[base_idx + VIRTIO_RXQ]->enabled &&
+		dev->virtqueue[base_idx + VIRTIO_TXQ]->enabled;
+
 	if (notify_ops->vring_state_changed) {
 		notify_ops->vring_state_changed(dev, base_idx / VIRTIO_QNUM,
-						enable);
+						qpair_enabled);

Here we could make the vring_state_changed callback be per vring then, instead of per queue-pair. That make the code simpler, as well as more consistent.

Thanks.

	--yliu
Quoted 10 lines
lib/librte_vhost/vhost_user/virtio-net-user.c +0 -3
 	}
 
-	dev->virtqueue[base_idx + VIRTIO_RXQ]->enabled = enable;
-	dev->virtqueue[base_idx + VIRTIO_TXQ]->enabled = enable;
-
 	return 0;
 }
 
Signature · 1 line
--Victor
#3 lore raw

PATCH v2 1/1 vhost-user: fix enabling of queue pair

In reply to Victor Kaplansky

From: Victor Kaplansky <[email protected]>

The VHOST_USER_SET_VRING_ENABLE request was sent for each queue-pair. However, it's changed to be sent per queue in the queue-pair by QEMU commit dc3db6ad ("vhost-user: start/stop all rings"). The change is reasonable, as we send all other request per queue, instead of queue-pair.

Hence we should do proper changes to adapt to the QEMU change here. Otherwise, a segfault will be triggered when last TX queue was enabled.

[ commit log reword -- Yuanhan Liu ]

---

v2: invoke vring_state_changed() callback per-vring as well

This fixes a fatal issue, without that MQ will not work properly. Seems that Victor is busy at something else (or out of office so far), hence I sent v2 on behalf of him.

---
 lib/librte_vhost/vhost_user/virtio-net-user.c | 7 ++-----
 1 file changed, 2 insertions(+), 5 deletions(-)
lib/librte_vhost/vhost_user/virtio-net-user.c +2 -5
index d07452a..b2ada9f 100644
@@ -317,7 +317,6 @@ user_set_vring_enable(struct vhost_device_ctx ctx,
 		      struct vhost_vring_state *state)
 {
 	struct virtio_net *dev = get_device(ctx);
-	uint16_t base_idx = state->index;
 	int enable = (int)state->num;
 
 	RTE_LOG(INFO, VHOST_CONFIG,
@@ -325,12 +324,10 @@ user_set_vring_enable(struct vhost_device_ctx ctx,
 		enable, state->index);
 
 	if (notify_ops->vring_state_changed) {
-		notify_ops->vring_state_changed(dev, base_idx / VIRTIO_QNUM,
-						enable);
+		notify_ops->vring_state_changed(dev, state->index, enable);
 	}
 
-	dev->virtqueue[base_idx + VIRTIO_RXQ]->enabled = enable;
-	dev->virtqueue[base_idx + VIRTIO_TXQ]->enabled = enable;
+	dev->virtqueue[state->index]->enabled = enable;
 
 	return 0;
 }
Signature · 1 line
1.9.3
#4 lore raw

Yuanhan, looks fine. Thanks for handling this.
-- Victor

Quoted 56 lines · Yuanhan Liu

From: Victor Kaplansky <[email protected]>

The VHOST_USER_SET_VRING_ENABLE request was sent for each queue-pair. However, it's changed to be sent per queue in the queue-pair by QEMU commit dc3db6ad ("vhost-user: start/stop all rings"). The change is reasonable, as we send all other request per queue, instead of queue-pair.

Hence we should do proper changes to adapt to the QEMU change here. Otherwise, a segfault will be triggered when last TX queue was enabled.

[ commit log reword -- Yuanhan Liu ]

---

v2: invoke vring_state_changed() callback per-vring as well

This fixes a fatal issue, without that MQ will not work properly. Seems that Victor is busy at something else (or out of office so far), hence I sent v2 on behalf of him.

---
 lib/librte_vhost/vhost_user/virtio-net-user.c | 7 ++-----
 1 file changed, 2 insertions(+), 5 deletions(-)
lib/librte_vhost/vhost_user/virtio-net-user.c +2 -5
index d07452a..b2ada9f 100644
@@ -317,7 +317,6 @@ user_set_vring_enable(struct vhost_device_ctx ctx,
 		      struct vhost_vring_state *state)
 {
 	struct virtio_net *dev = get_device(ctx);
-	uint16_t base_idx = state->index;
 	int enable = (int)state->num;
 
 	RTE_LOG(INFO, VHOST_CONFIG,
@@ -325,12 +324,10 @@ user_set_vring_enable(struct vhost_device_ctx ctx,
 		enable, state->index);
 
 	if (notify_ops->vring_state_changed) {
-		notify_ops->vring_state_changed(dev, base_idx / VIRTIO_QNUM,
-						enable);
+		notify_ops->vring_state_changed(dev, state->index, enable);
 	}
 
-	dev->virtqueue[base_idx + VIRTIO_RXQ]->enabled = enable;
-	dev->virtqueue[base_idx + VIRTIO_TXQ]->enabled = enable;
+	dev->virtqueue[state->index]->enabled = enable;
 
 	return 0;
 }
Signature · 1 line
1.9.3
#5 lore raw

2015-11-24 09:43, Victor Kaplansky:

Quoted 26 lines

Yuanhan, looks fine. Thanks for handling this.
-- Victor

Quoted 22 lines · Yuanhan Liu

From: Victor Kaplansky <[email protected]>

The VHOST_USER_SET_VRING_ENABLE request was sent for each queue-pair. However, it's changed to be sent per queue in the queue-pair by QEMU commit dc3db6ad ("vhost-user: start/stop all rings"). The change is reasonable, as we send all other request per queue, instead of queue-pair.

Hence we should do proper changes to adapt to the QEMU change here. Otherwise, a segfault will be triggered when last TX queue was enabled.

[ commit log reword -- Yuanhan Liu ]

---

v2: invoke vring_state_changed() callback per-vring as well

This fixes a fatal issue, without that MQ will not work properly. Seems that Victor is busy at something else (or out of office so far), hence I sent v2 on behalf of him.

Applied, thanks