Skip to contentSkip to conversation

net/vhost: fix access to freed memory

PATCH Applied Yuan Wang 3 participants 4 messages Mar 11, 2022 – May 5, 2022 Network drivers

lore
Share & tools
lore

Yuan Wang

author
#0 lore raw

PATCH 1/1 net/vhost: fix access to freed memory

This patch fixes heap-use-after-free reported by ASan.

It is possible for the rte_vhost_dequeue_burst() to access the vq is freed when numa_realloc() gets called in the device running state. The control plane will set the vq->access_lock to protected the vq from the data plane. Unfortunately the lock will fail at the moment the vq is freed, allowing the rte_vhost_dequeue_burst() to access the fields of the vq, which will trigger a heap-use-after-free error.

In the case of multiple queues, the vhost pmd can access other queues that are not ready when the first queue is ready, which makes no sense and also allows numa_realloc() and rte_vhost_dequeue_burst() access to vq to happen at the same time. By controlling vq->allow_queuing we can make the pmd access only the queues that are ready.

Fixes 1ce3c7fe149 ("net/vhost: emulate device start/stop behavior")
---
 drivers/net/vhost/rte_eth_vhost.c | 15 +++++++++++++--
 1 file changed, 13 insertions(+), 2 deletions(-)
drivers/net/vhost/rte_eth_vhost.c +13 -2
index 070f0e6dfd..8a6595504a 100644
@@ -720,6 +720,7 @@ update_queuing_status(struct rte_eth_dev *dev)
 {
 	struct pmd_internal *internal = dev->data->dev_private;
 	struct vhost_queue *vq;
+	struct rte_vhost_vring_state *state;
 	unsigned int i;
 	int allow_queuing = 1;
 
@@ -730,12 +731,17 @@ update_queuing_status(struct rte_eth_dev *dev)
 	    rte_atomic32_read(&internal->dev_attached) == 0)
 		allow_queuing = 0;
 
+	state = vring_states[dev->data->port_id];
+
 	/* Wait until rx/tx_pkt_burst stops accessing vhost device */
 	for (i = 0; i < dev->data->nb_rx_queues; i++) {
 		vq = dev->data->rx_queues[i];
 		if (vq == NULL)
 			continue;
-		rte_atomic32_set(&vq->allow_queuing, allow_queuing);
+		if (allow_queuing && state->cur[vq->virtqueue_id])
+			rte_atomic32_set(&vq->allow_queuing, 1);
+		else
+			rte_atomic32_set(&vq->allow_queuing, 0);
 		while (rte_atomic32_read(&vq->while_queuing))
 			rte_pause();
 	}
@@ -744,7 +750,10 @@ update_queuing_status(struct rte_eth_dev *dev)
 		vq = dev->data->tx_queues[i];
 		if (vq == NULL)
 			continue;
-		rte_atomic32_set(&vq->allow_queuing, allow_queuing);
+		if (allow_queuing && state->cur[vq->virtqueue_id])
+			rte_atomic32_set(&vq->allow_queuing, 1);
+		else
+			rte_atomic32_set(&vq->allow_queuing, 0);
 		while (rte_atomic32_read(&vq->while_queuing))
 			rte_pause();
 	}
@@ -967,6 +976,8 @@ vring_state_changed(int vid, uint16_t vring, int enable)
 	state->max_vring = RTE_MAX(vring, state->max_vring);
 	rte_spinlock_unlock(&state->lock);
 
+	update_queuing_status(eth_dev);
+
 	VHOST_LOG(INFO, "vring%u is %s\n",
 			vring, enable ? "enabled" : "disabled");
 
Signature · 1 line
2.25.1
Tested-by on v1 1/1
#1 lore raw
Quoted 23 lines

-----Original Message-----
From: Wang, YuanX <[email protected]>
Sent: Saturday, March 12, 2022 12:35 AM
To: [email protected]; Xia, Chenbo <[email protected]>
Cc: [email protected]; Hu, Jiayu <[email protected]>; Ling, WeiX <[email protected]>; Wang, YuanX <[email protected]>
Subject: [PATCH] net/vhost: fix access to freed memory

This patch fixes heap-use-after-free reported by ASan.

It is possible for the rte_vhost_dequeue_burst() to access the vq is freed when numa_realloc() gets called in the device running state.
The control plane will set the vq->access_lock to protected the vq from the data plane. Unfortunately the lock will fail at the moment the vq is freed, allowing the rte_vhost_dequeue_burst() to access the fields of the vq, which will trigger a heap-use-after-free error.

In the case of multiple queues, the vhost pmd can access other queues that are not ready when the first queue is ready, which makes no sense and also allows numa_realloc() and rte_vhost_dequeue_burst() access to vq to happen at the same time. By controlling vq->allow_queuing we can make the pmd access only the queues that are ready.

Quoted 4 lines
Fixes 1ce3c7fe149 ("net/vhost: emulate device start/stop behavior")

---

Reviewed-by on v1 1/1
#2 lore raw
In reply to Yuan Wang

Hi Yuan,

Quoted 15 lines · Yuan Wang

This patch fixes heap-use-after-free reported by ASan.

It is possible for the rte_vhost_dequeue_burst() to access the vq is freed when numa_realloc() gets called in the device running state. The control plane will set the vq->access_lock to protected the vq from the data plane. Unfortunately the lock will fail at the moment the vq is freed, allowing the rte_vhost_dequeue_burst() to access the fields of the vq, which will trigger a heap-use-after-free error.

In the case of multiple queues, the vhost pmd can access other queues that are not ready when the first queue is ready, which makes no sense and also allows numa_realloc() and rte_vhost_dequeue_burst() access to vq to happen at the same time. By controlling vq->allow_queuing we can make the pmd access only the queues that are ready.

Quoted 7 lines
Fixes 1ce3c7fe149 ("net/vhost: emulate device start/stop behavior")
---
  drivers/net/vhost/rte_eth_vhost.c | 15 +++++++++++++--
  1 file changed, 13 insertions(+), 2 deletions(-)

It is indeed better for the Vhost PMD to not access virtqueues that aren't ready.

Thanks,
Maxime

#3 lore raw
In reply to Yuan Wang
Quoted 15 lines · Yuan Wang

This patch fixes heap-use-after-free reported by ASan.

It is possible for the rte_vhost_dequeue_burst() to access the vq is freed when numa_realloc() gets called in the device running state. The control plane will set the vq->access_lock to protected the vq from the data plane. Unfortunately the lock will fail at the moment the vq is freed, allowing the rte_vhost_dequeue_burst() to access the fields of the vq, which will trigger a heap-use-after-free error.

In the case of multiple queues, the vhost pmd can access other queues that are not ready when the first queue is ready, which makes no sense and also allows numa_realloc() and rte_vhost_dequeue_burst() access to vq to happen at the same time. By controlling vq->allow_queuing we can make the pmd access only the queues that are ready.

Quoted 7 lines
Fixes 1ce3c7fe149 ("net/vhost: emulate device start/stop behavior")
---
  drivers/net/vhost/rte_eth_vhost.c | 15 +++++++++++++--
  1 file changed, 13 insertions(+), 2 deletions(-)

Applied to dpdk-next-virtio/main.

Thanks,
Maxime