Skip to contentSkip to conversation

vsock/virtio: fix worker access after virtqueue teardown

PATCH Superseded Weiming Shi 5 participants 9 messages Jul 29 – Aug 4

Subsystems (6)
lore
Share & tools
lore
#0 lore raw

v2 cover vsock/virtio: fix worker access after virtqueue teardown

Virtio-vsock workers can remain queued while freeze deletes the virtqueues. This series prevents workers delayed across freeze and restore from retaining pointers to deleted queues, and prevents the RX worker from refilling its queue after teardown.

Changes in v2:
- Split the worker pointer changes from the RX refill fix because they fix different commits.
- Use bd50c5dc182b as the Fixes tag for the worker pointer changes.
- Keep b917507e5ad9 as the Fixes tag for the RX refill fix.
- Use the suggested out_nofill label when rx_run is clear.

Weiming Shi (2):
vsock/virtio: read virtqueues under worker locks vsock/virtio: avoid refilling the RX queue after teardown

 net/vmw_vsock/virtio_transport.c | 14 ++++++++------
 1 file changed, 8 insertions(+), 6 deletions(-)

base-commit: 51b093a7ba27476e1f639455f005e8d2e75390e4

Signature · 1 line
2.55.0
#1 lore raw

v3 cover vsock/virtio: fix worker access after virtqueue teardown

Virtio-vsock workers can remain queued while freeze deletes the virtqueues. This series prevents workers delayed across freeze and restore from retaining pointers to deleted queues, and prevents the RX worker from refilling its queue after teardown.

Changes in v3:
- Resend the series with proper email threading; no code changes.

Changes in v2:
- Split the worker pointer changes from the RX refill fix because they fix different commits.
- Use bd50c5dc182b as the Fixes tag for the worker pointer changes.
- Keep b917507e5ad9 as the Fixes tag for the RX refill fix.
- Use the suggested out_nofill label when rx_run is clear.

Weiming Shi (2):
vsock/virtio: read virtqueues under worker locks vsock/virtio: avoid refilling the RX queue after teardown

 net/vmw_vsock/virtio_transport.c | 14 ++++++++------
 1 file changed, 8 insertions(+), 6 deletions(-)

base-commit: 51b093a7ba27476e1f639455f005e8d2e75390e4

Signature · 1 line
2.55.0
#2 lore raw

PATCH v3 1/2 vsock/virtio: read virtqueues under worker locks

Commit bd50c5dc182b ("vsock/virtio: add support for device suspend/resume") made the *_run flags transition from false to true when restore installs replacement virtqueues. The RX, TX and event workers read their virtqueue before locking and checking the corresponding flag, so a worker delayed across freeze and restore can observe the replacement queue's running state while retaining a pointer to the deleted queue.

Read each virtqueue under its mutex after checking the run flag, keeping the pointer and state in the same queue generation.

Fixes bd50c5dc182b ("vsock/virtio: add support for device suspend/resume")
Assisted-by OpenAI-Codex:gpt-5
---
 net/vmw_vsock/virtio_transport.c | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)
net/vmw_vsock/virtio_transport.c +6 -5
index 57f2d6ec3ffc..a8e1dd95ba8c 100644
@@ -346,12 +346,13 @@ static void virtio_transport_tx_work(struct work_struct *work)
 	struct virtqueue *vq;
 	bool added = false;
 
-	vq = vsock->vqs[VSOCK_VQ_TX];
 	mutex_lock(&vsock->tx_lock);
 
 	if (!vsock->tx_run)
 		goto out;
 
+	vq = vsock->vqs[VSOCK_VQ_TX];
+
 	do {
 		struct sk_buff *skb;
 		unsigned int len;
@@ -451,13 +452,13 @@ static void virtio_transport_event_work(struct work_struct *work)
 		container_of(work, struct virtio_vsock, event_work);
 	struct virtqueue *vq;
 
-	vq = vsock->vqs[VSOCK_VQ_EVENT];
-
 	mutex_lock(&vsock->event_lock);
 
 	if (!vsock->event_run)
 		goto out;
 
+	vq = vsock->vqs[VSOCK_VQ_EVENT];
+
 	do {
 		struct virtio_vsock_event *event;
 		unsigned int len;
@@ -634,13 +635,13 @@ static void virtio_transport_rx_work(struct work_struct *work)
 		container_of(work, struct virtio_vsock, rx_work);
 	struct virtqueue *vq;
 
-	vq = vsock->vqs[VSOCK_VQ_RX];
-
 	mutex_lock(&vsock->rx_lock);
 
 	if (!vsock->rx_run)
 		goto out;
 
+	vq = vsock->vqs[VSOCK_VQ_RX];
+
 	do {
 		virtqueue_disable_cb(vq);
 		for (;;) {
Signature · 1 line
2.55.0
Reviewed-by on v3 1/2
#3 lore raw

Re: [PATCH v3 1/2] vsock/virtio: read virtqueues under worker locks

Quoted 74 lines · Weiming Shi

Commit bd50c5dc182b ("vsock/virtio: add support for device suspend/resume") made the *_run flags transition from false to true when restore installs replacement virtqueues. The RX, TX and event workers read their virtqueue before locking and checking the corresponding flag, so a worker delayed across freeze and restore can observe the replacement queue's running state while retaining a pointer to the deleted queue.

Read each virtqueue under its mutex after checking the run flag, keeping the pointer and state in the same queue generation.

Fixes bd50c5dc182b ("vsock/virtio: add support for device suspend/resume")
Assisted-by OpenAI-Codex:gpt-5
---
 net/vmw_vsock/virtio_transport.c | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)
net/vmw_vsock/virtio_transport.c +6 -5
index 57f2d6ec3ffc..a8e1dd95ba8c 100644
@@ -346,12 +346,13 @@ static void virtio_transport_tx_work(struct work_struct *work)
 	struct virtqueue *vq;
 	bool added = false;
 
-	vq = vsock->vqs[VSOCK_VQ_TX];
 	mutex_lock(&vsock->tx_lock);
 
 	if (!vsock->tx_run)
 		goto out;
 
+	vq = vsock->vqs[VSOCK_VQ_TX];
+
 	do {
 		struct sk_buff *skb;
 		unsigned int len;
@@ -451,13 +452,13 @@ static void virtio_transport_event_work(struct work_struct *work)
 		container_of(work, struct virtio_vsock, event_work);
 	struct virtqueue *vq;
 
-	vq = vsock->vqs[VSOCK_VQ_EVENT];
-
 	mutex_lock(&vsock->event_lock);
 
 	if (!vsock->event_run)
 		goto out;
 
+	vq = vsock->vqs[VSOCK_VQ_EVENT];
+
 	do {
 		struct virtio_vsock_event *event;
 		unsigned int len;
@@ -634,13 +635,13 @@ static void virtio_transport_rx_work(struct work_struct *work)
 		container_of(work, struct virtio_vsock, rx_work);
 	struct virtqueue *vq;
 
-	vq = vsock->vqs[VSOCK_VQ_RX];
-
 	mutex_lock(&vsock->rx_lock);
 
 	if (!vsock->rx_run)
 		goto out;
 
+	vq = vsock->vqs[VSOCK_VQ_RX];
+
 	do {
 		virtqueue_disable_cb(vq);
 		for (;;) {
Signature · 1 line
2.55.0
#4 lore raw

PATCH v3 2/2 vsock/virtio: avoid refilling the RX queue after teardown

In reply to Weiming Shi

Commit b917507e5ad9 ("vsock/virtio: stop workers during the .remove()") made the RX worker jump to its common exit when rx_run is clear. That exit still refills the RX queue when the buffer count is low, so work queued across virtio_vsock_vqs_del() can add buffers after the virtqueues have been deleted.

BUG: KASAN: slab-use-after-free in virtqueue_add_sgs
Read of size 4 by task kworker/0:1
Workqueue: virtio_vsock virtio_transport_rx_work
Call Trace:
 virtqueue_add_sgs (drivers/virtio/virtio_ring.c:2796)
 virtio_vsock_rx_fill (net/vmw_vsock/virtio_transport.c:332)
 virtio_transport_rx_work (net/vmw_vsock/virtio_transport.c:701)
 process_one_work (kernel/workqueue.c:3314)
 worker_thread (kernel/workqueue.c:3478)
 kthread (kernel/kthread.c:436)
 ret_from_fork (arch/x86/kernel/process.c:158)
 ret_from_fork_asm (arch/x86/entry/entry_64.S:245)
...
Freed by task 141:
 kfree (mm/slub.c:6566)
 vp_del_vq (drivers/virtio/virtio_pci_common.c:259)
 vp_del_vqs (drivers/virtio/virtio_pci_common.c:285)
 virtio_vsock_freeze (net/vmw_vsock/virtio_transport.c:912)
 virtio_device_freeze (drivers/virtio/virtio.c:658)
 virtio_pci_freeze (drivers/virtio/virtio_pci_common.c:601)
 pci_pm_freeze (drivers/pci/pci-driver.c:1098)
 device_suspend (drivers/base/power/main.c:1968)
Kernel panic - not syncing: KASAN: panic_on_warn set ...

Jump to a no-refill exit when rx_run is clear, leaving the normal exit to replenish a running queue.

Fixes b917507e5ad9 ("vsock/virtio: stop workers during the .remove()")
Assisted-by OpenAI-Codex:gpt-5
---
 net/vmw_vsock/virtio_transport.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
net/vmw_vsock/virtio_transport.c +2 -1
index a8e1dd95ba8c..96c9fe8d357c 100644
@@ -638,7 +638,7 @@ static void virtio_transport_rx_work(struct work_struct *work)
 	mutex_lock(&vsock->rx_lock);
 
 	if (!vsock->rx_run)
-		goto out;
+		goto out_nofill;
 
 	vq = vsock->vqs[VSOCK_VQ_RX];
 
@@ -692,6 +692,7 @@ static void virtio_transport_rx_work(struct work_struct *work)
 out:
 	if (vsock->rx_buf_nr < vsock->rx_buf_max_nr / 2)
 		virtio_vsock_rx_fill(vsock);
+out_nofill:
 	mutex_unlock(&vsock->rx_lock);
 }
 
Signature · 1 line
2.55.0
#5 lore raw

Re: [PATCH v3 2/2] vsock/virtio: avoid refilling the RX queue after teardown

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Pre-existing issues:
- [High] Incorrect flush_work ordering in virtio_vsock_remove() allows virtio_transport_rx_work to execute after the vsock object has been freed. --

--- Patch [2]: [PATCH v3 2/2] vsock/virtio: avoid refilling the RX queue after teardown --- commit e28975cab51c80e977d0774ef653bc9c4660fff0
Author: Weiming Shi <[email protected]>

vsock/virtio: avoid refilling the RX queue after teardown

Commit b917507e5ad9 ("vsock/virtio: stop workers during the .remove()") made the RX worker jump to its common exit when rx_run is clear. That exit still refills the RX queue when the buffer count is low, so work queued across virtio_vsock_vqs_del() can add buffers after the virtqueues have been deleted.

Quoted 4 lines
net/vmw_vsock/virtio_transport.c +0 -0
index 57f2d6ec3ffc6..96c9fe8d357c6 100644

[ ... ]

net/vmw_vsock/virtio_transport.c static void virtio_transport_rx_work(struct work_struct *work) · line 641

Quoted 11 lines
net/vmw_vsock/virtio_transport.c +1 -3
@@ -634,12 +635,12 @@ static void virtio_transport_rx_work(struct work_struct *work)
 		container_of(work, struct virtio_vsock, rx_work);
 	struct virtqueue *vq;
 
-	vq = vsock->vqs[VSOCK_VQ_RX];
-
 	mutex_lock(&vsock->rx_lock);
 
 	if (!vsock->rx_run)
-		goto out;
+		goto out_nofill;

[Severity: High]
This is a pre-existing issue, but I have a question about the workqueue teardown logic that can still trigger a use-after-free for vsock.

In virtio_vsock_remove(), the workqueues are flushed in this order:

net/vmw_vsock/virtio_transport.c:virtio_vsock_remove() {
	...
	flush_work(&vsock->rx_work);
	flush_work(&vsock->tx_work);
	flush_work(&vsock->event_work);
	flush_work(&vsock->send_pkt_work);
	...
}

However, virtio_transport_send_pkt_work() can queue rx_work without holding locks if restart_rx is true:

net/vmw_vsock/virtio_transport.c:virtio_transport_send_pkt_work() {
	...
out:
	mutex_unlock(&vsock->tx_lock);

	if (restart_rx)
		queue_work(virtio_vsock_workqueue, &vsock->rx_work);
}

If a worker executing virtio_transport_send_pkt_work() is preempted just before calling queue_work(), and a concurrent virtio_vsock_remove() flushes rx_work and then blocks flushing send_pkt_work, couldn't the preempted worker then resume and queue a new rx_work?

This newly queued rx_work would be left pending after virtio_vsock_remove() completes, causing a use-after-free when the vsock structure is freed.

Should virtio_vsock_remove() flush send_pkt_work before rx_work to prevent this race?

Quoted 5 lines
net/vmw_vsock/virtio_transport.c +2 -0
+
+	vq = vsock->vqs[VSOCK_VQ_RX];
 
 	do {
 		virtqueue_disable_cb(vq);
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
Reviewed-by on v3 2/2
#6 lore raw

Re: [PATCH v3 2/2] vsock/virtio: avoid refilling the RX queue after teardown

In reply to Weiming Shi
Quoted 68 lines · Weiming Shi

Commit b917507e5ad9 ("vsock/virtio: stop workers during the .remove()") made the RX worker jump to its common exit when rx_run is clear. That exit still refills the RX queue when the buffer count is low, so work queued across virtio_vsock_vqs_del() can add buffers after the virtqueues have been deleted.

BUG: KASAN: slab-use-after-free in virtqueue_add_sgs
Read of size 4 by task kworker/0:1
Workqueue: virtio_vsock virtio_transport_rx_work
Call Trace:
 virtqueue_add_sgs (drivers/virtio/virtio_ring.c:2796)
 virtio_vsock_rx_fill (net/vmw_vsock/virtio_transport.c:332)
 virtio_transport_rx_work (net/vmw_vsock/virtio_transport.c:701)
 process_one_work (kernel/workqueue.c:3314)
 worker_thread (kernel/workqueue.c:3478)
 kthread (kernel/kthread.c:436)
 ret_from_fork (arch/x86/kernel/process.c:158)
 ret_from_fork_asm (arch/x86/entry/entry_64.S:245)
...
Freed by task 141:
 kfree (mm/slub.c:6566)
 vp_del_vq (drivers/virtio/virtio_pci_common.c:259)
 vp_del_vqs (drivers/virtio/virtio_pci_common.c:285)
 virtio_vsock_freeze (net/vmw_vsock/virtio_transport.c:912)
 virtio_device_freeze (drivers/virtio/virtio.c:658)
 virtio_pci_freeze (drivers/virtio/virtio_pci_common.c:601)
 pci_pm_freeze (drivers/pci/pci-driver.c:1098)
 device_suspend (drivers/base/power/main.c:1968)
Kernel panic - not syncing: KASAN: panic_on_warn set ...

Jump to a no-refill exit when rx_run is clear, leaving the normal exit to replenish a running queue.

Fixes b917507e5ad9 ("vsock/virtio: stop workers during the .remove()")
Assisted-by OpenAI-Codex:gpt-5
---
 net/vmw_vsock/virtio_transport.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
net/vmw_vsock/virtio_transport.c +2 -1
index a8e1dd95ba8c..96c9fe8d357c 100644
@@ -638,7 +638,7 @@ static void virtio_transport_rx_work(struct work_struct *work)
 	mutex_lock(&vsock->rx_lock);
 
 	if (!vsock->rx_run)
-		goto out;
+		goto out_nofill;
 
 	vq = vsock->vqs[VSOCK_VQ_RX];
 
@@ -692,6 +692,7 @@ static void virtio_transport_rx_work(struct work_struct *work)
 out:
 	if (vsock->rx_buf_nr < vsock->rx_buf_max_nr / 2)
 		virtio_vsock_rx_fill(vsock);
+out_nofill:
 	mutex_unlock(&vsock->rx_lock);
 }
 
Signature · 1 line
2.55.0
#7 lore raw
In reply to Weiming Shi
This series was applied to netdev/net.git (main) by Jakub Kicinski <[email protected]>:

Hello:

This series was applied to netdev/net.git (main)
by Jakub Kicinski <[email protected]>:

Quoted 9 lines · On Wed, 29 Jul 2026 12:16:53 -0700 you wrote:

Virtio-vsock workers can remain queued while freeze deletes the virtqueues. This series prevents workers delayed across freeze and restore from retaining pointers to deleted queues, and prevents the RX worker from refilling its queue after teardown.

Changes in v3:
- Resend the series with proper email threading; no code changes.

[...]

Here is the summary with links:
- [v3,1/2] vsock/virtio: read virtqueues under worker locks

    https://git.kernel.org/netdev/net/c/ebac8f6b1ef0
  - [v3,2/2] vsock/virtio: avoid refilling the RX queue after teardown
    https://git.kernel.org/netdev/net/c/a31e0ad44469

You are awesome, thank you!

Signature · 2 lines
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html
maintainer Reviewed-by on the v3 series
#8 lore raw
In reply to Weiming Shi
Quoted 8 lines · Weiming Shi

Virtio-vsock workers can remain queued while freeze deletes the virtqueues. This series prevents workers delayed across freeze and restore from retaining pointers to deleted queues, and prevents the RX worker from refilling its queue after teardown.

Changes in v3:
- Resend the series with proper email threading; no code changes.

Quoted 6 lines

Changes in v2:
- Split the worker pointer changes from the RX refill fix because they fix different commits.
- Use bd50c5dc182b as the Fixes tag for the worker pointer changes.
- Keep b917507e5ad9 as the Fixes tag for the RX refill fix.
- Use the suggested out_nofill label when rx_run is clear.

Thanks, it's already queued but sice you fixed all my comments, just for record: