Skip to contentSkip to conversation

scsi: virtio_scsi: move INIT_WORK calls to virtscsi_probe

PATCH Applied Joshua Daley 3 participants 7 messages Mar 25 – Apr 3

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

v4 cover scsi: virtio_scsi: move INIT_WORK calls to virtscsi_probe

TITLE CHANGED! Original series title:
"scsi: virtio_scsi: move INIT_WORK calls to virtscsi_init" Previous version:
https://lore.kernel.org/linux-scsi/[email protected]/T/#t

Changelog v3 -> v4:

[PATCH v3 1/3] scsi: virtio_scsi: kick event_list unconditionally
-> [PATCH v4 2/2] scsi: virtio_scsi: kick event_list unconditionally
- this patch now comes after moving the INIT_WORK calls, to address bisection concerns:
https://lore.kernel.org/linux-scsi/[email protected]/
- the code changes are a bit different given that the INIT_WORK calls are now moved into virtscsi_probe(), but the concept is the same.

[PATCH v3 2/3] scsi: virtio_scsi: move INIT_WORK calls to virtscsi_init -> [PATCH v4 1/2] scsi: virtio_scsi: move INIT_WORK calls to virtscsi_probe
- the INIT_WORK calls are now moved to virtscsi_probe() to address concerns with suspend/resume. The title of the series is changed to reflect this.

[PATCH v3 3/3] scsi: virtio_scsi: remove unnecessary fn declaration
- squashed into [PATCH v4 1/2]

-----

v3 cover letter:

Changelog v2 -> v3:

- switched the order of patches 2 & 3 to fix compilation error.
- added reviewed-by tags.

-----

v2 cover letter:

Changelog v1 -> v2:

- Added 2 additional patches:
- [PATCH v2 1/3] scsi: virtio_scsi: kick event_list unconditionally
- Removes the conditions surrounding event_list operations (suggested by Stefan Hajnoczi [<[email protected]>](mailto:[email protected]))
- [PATCH v2 2/3] scsi: virtio_scsi: remove unnecessary fn declaration
- Removes virtscsi_handle_event() prototype (suggested by Eric Farman [<[email protected]>](mailto:[email protected]))

- [PATCH 1/1] -> [PATCH v2 3/3] scsi: virtio_scsi: move INIT_WORK calls to virtscsi_init
- Removed the condition surrounding INIT_WORK calls

-----

v1 cover letter:

This patch avoids a kernel warning that may occur if a virtio_scsi controller is detached immediately following a disk detach. See the commit message for details. The following are instructions to produce the warning (without the proposed patch).

Timing matters--if all event work items call INIT_WORK before they are flushed by cancel_work_sync, then the warning will not occur.

The warning will occur consistently if a sleep is added in virtscsi_kick_event before the INIT_WORK call, like so:

#include <linux/delay.h>

static int virtscsi_kick_event(struct virtio_scsi *vscsi,
struct virtio_scsi_event_node *event_node)

{
    int err;
    struct scatterlist sg;
    unsigned long flags;
 -> msleep(1000);
    INIT_WORK(&event_node->work, virtscsi_handle_event);
	
    ...
}

Then, just detach a disk and its controller in quick succession:

virsh detach-device --domain <domain> disk.xml; \
virsh detach-device --domain <domain> controller.xml

where disk.xml and controller.xml are text files containing the XML of the disk and controller.

Or, with the libvirt python module:

domain.detachDevice(str(disk_xml))
domain.detachDevice(str(controller_xml))

Joshua Daley (2):
scsi: virtio_scsi: move INIT_WORK calls to virtscsi_probe
scsi: virtio_scsi: kick event_list unconditionally

 drivers/scsi/virtio_scsi.c | 14 ++++++--------
 1 file changed, 6 insertions(+), 8 deletions(-)
Signature · 1 line
2.34.1
#1 lore raw

PATCH v4 1/2 scsi: virtio_scsi: move INIT_WORK calls to virtscsi_probe

The last step of virtscsi_handle_event is to call virtscsi_kick_event, which calls INIT_WORK on it's own work item. INIT_WORK resets the work item's data bits to 0.

If this occurs while the work item is being flushed by cancel_work_sync, then kernel/workqueue.c/work_offqd_enable triggers a kernel warning, as it expects the "disable" bit to be 1:

[   21.450115] workqueue: work disable count underflowed
[   21.450117] WARNING: CPU: 1 PID: 56 at kernel/workqueue.c:4328 enable_work+0x10a/0x120
...
[   21.450171] Call Trace:
[   21.450173]  [<000003db2e5bdc3e>] enable_work+0x10e/0x120
[   21.450176] ([<000003db2e5bdc3a>] enable_work+0x10a/0x120)
[   21.450178]  [<000003db2e5bdd86>] cancel_work_sync+0x86/0xa0
[   21.450181]  [<000003daae97d9e4>] virtscsi_remove+0xb4/0xd0 [virtio_scsi]
[   21.450184]  [<000003db2ef3b5ca>] virtio_dev_remove+0x6a/0xd0
[   21.450186]  [<000003db2ef9106c>] device_release_driver_internal+0x1ac/0x260
[   21.450190]  [<000003db2ef8edc8>] bus_remove_device+0xf8/0x190
[   21.450192]  [<000003db2ef88d72>] device_del+0x142/0x340
[   21.450194]  [<000003db2ef88fa0>] device_unregister+0x30/0xa0
[   21.450196]  [<000003db2ef3b2fa>] unregister_virtio_device+0x2a/0x40

This warning may occur if a controller is detached immediately following a disk detach.

Move the INIT_WORK call to prevent this. Don't re-init event list work items in virtscsi_kick_event, init them only once in virtscsi_probe instead.

---
 drivers/scsi/virtio_scsi.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)
drivers/scsi/virtio_scsi.c +4 -3
index 0ed8558dad72..64b6c942f572 100644
@@ -233,7 +233,6 @@ static void virtscsi_ctrl_done(struct virtqueue *vq)
 	virtscsi_vq_done(vscsi, &vscsi->ctrl_vq, virtscsi_complete_free);
 };
 
-static void virtscsi_handle_event(struct work_struct *work);
 
 static int virtscsi_kick_event(struct virtio_scsi *vscsi,
 			       struct virtio_scsi_event_node *event_node)
@@ -242,7 +241,6 @@ static int virtscsi_kick_event(struct virtio_scsi *vscsi,
 	struct scatterlist sg;
 	unsigned long flags;
 
-	INIT_WORK(&event_node->work, virtscsi_handle_event);
 	sg_init_one(&sg, event_node->event, sizeof(struct virtio_scsi_event));
 
 	spin_lock_irqsave(&vscsi->event_vq.vq_lock, flags);
@@ -984,8 +982,11 @@ static int virtscsi_probe(struct virtio_device *vdev)
 
 	virtio_device_ready(vdev);
 
-	if (virtio_has_feature(vdev, VIRTIO_SCSI_F_HOTPLUG))
+	if (virtio_has_feature(vdev, VIRTIO_SCSI_F_HOTPLUG)) {
+		for (int i = 0; i < VIRTIO_SCSI_EVENT_LEN; i++)
+			INIT_WORK(&vscsi->event_list[i].work, virtscsi_handle_event);
 		virtscsi_kick_event_all(vscsi);
+	}
 
 	scsi_scan_host(shost);
 	return 0;
Signature · 1 line
2.34.1
reviewer Reviewed-by on v4 1/2
#2 lore raw
Quoted 26 lines · Joshua Daley

The last step of virtscsi_handle_event is to call virtscsi_kick_event, which calls INIT_WORK on it's own work item. INIT_WORK resets the work item's data bits to 0.

If this occurs while the work item is being flushed by cancel_work_sync, then kernel/workqueue.c/work_offqd_enable triggers a kernel warning, as it expects the "disable" bit to be 1:

[   21.450115] workqueue: work disable count underflowed
[   21.450117] WARNING: CPU: 1 PID: 56 at kernel/workqueue.c:4328 enable_work+0x10a/0x120
...
[   21.450171] Call Trace:
[   21.450173]  [<000003db2e5bdc3e>] enable_work+0x10e/0x120
[   21.450176] ([<000003db2e5bdc3a>] enable_work+0x10a/0x120)
[   21.450178]  [<000003db2e5bdd86>] cancel_work_sync+0x86/0xa0
[   21.450181]  [<000003daae97d9e4>] virtscsi_remove+0xb4/0xd0 [virtio_scsi]
[   21.450184]  [<000003db2ef3b5ca>] virtio_dev_remove+0x6a/0xd0
[   21.450186]  [<000003db2ef9106c>] device_release_driver_internal+0x1ac/0x260
[   21.450190]  [<000003db2ef8edc8>] bus_remove_device+0xf8/0x190
[   21.450192]  [<000003db2ef88d72>] device_del+0x142/0x340
[   21.450194]  [<000003db2ef88fa0>] device_unregister+0x30/0xa0
[   21.450196]  [<000003db2ef3b2fa>] unregister_virtio_device+0x2a/0x40

This warning may occur if a controller is detached immediately following a disk detach.

Quoted 8 lines

Move the INIT_WORK call to prevent this. Don't re-init event list work items in virtscsi_kick_event, init them only once in virtscsi_probe instead.

---
 drivers/scsi/virtio_scsi.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)
#3 lore raw

PATCH v4 2/2 scsi: virtio_scsi: kick event_list unconditionally

In reply to Joshua Daley

The event_list processes non-hotplug events (such as LUN capacity changes), so remove the conditions that guard the initial kicks in _probe() and _restore(), as well as the work cancellation in _remove().

---
 drivers/scsi/virtio_scsi.c | 15 ++++++---------
 1 file changed, 6 insertions(+), 9 deletions(-)
drivers/scsi/virtio_scsi.c +6 -9
index 64b6c942f572..5fdaa71f0652 100644
@@ -982,11 +982,10 @@ static int virtscsi_probe(struct virtio_device *vdev)
 
 	virtio_device_ready(vdev);
 
-	if (virtio_has_feature(vdev, VIRTIO_SCSI_F_HOTPLUG)) {
-		for (int i = 0; i < VIRTIO_SCSI_EVENT_LEN; i++)
-			INIT_WORK(&vscsi->event_list[i].work, virtscsi_handle_event);
-		virtscsi_kick_event_all(vscsi);
-	}
+	for (int i = 0; i < VIRTIO_SCSI_EVENT_LEN; i++)
+		INIT_WORK(&vscsi->event_list[i].work, virtscsi_handle_event);
+
+	virtscsi_kick_event_all(vscsi);
 
 	scsi_scan_host(shost);
 	return 0;
@@ -1003,8 +1002,7 @@ static void virtscsi_remove(struct virtio_device *vdev)
 	struct Scsi_Host *shost = virtio_scsi_host(vdev);
 	struct virtio_scsi *vscsi = shost_priv(shost);
 
-	if (virtio_has_feature(vdev, VIRTIO_SCSI_F_HOTPLUG))
-		virtscsi_cancel_event_work(vscsi);
+	virtscsi_cancel_event_work(vscsi);
 
 	scsi_remove_host(shost);
 	virtscsi_remove_vqs(vdev);
@@ -1030,8 +1028,7 @@ static int virtscsi_restore(struct virtio_device *vdev)
 
 	virtio_device_ready(vdev);
 
-	if (virtio_has_feature(vdev, VIRTIO_SCSI_F_HOTPLUG))
-		virtscsi_kick_event_all(vscsi);
+	virtscsi_kick_event_all(vscsi);
 
 	return err;
 }
Signature · 1 line
2.34.1
reviewer Reviewed-by on v4 2/2
#4 lore raw

Re: [PATCH v4 2/2] scsi: virtio_scsi: kick event_list unconditionally

Quoted 9 lines · Joshua Daley

The event_list processes non-hotplug events (such as LUN capacity changes), so remove the conditions that guard the initial kicks in _probe() and _restore(), as well as the work cancellation in _remove().

---
 drivers/scsi/virtio_scsi.c | 15 ++++++---------
 1 file changed, 6 insertions(+), 9 deletions(-)
#6 lore raw
In reply to Joshua Daley
Quoted 8 lines · Joshua Daley

TITLE CHANGED! Original series title:
"scsi: virtio_scsi: move INIT_WORK calls to virtscsi_init" Previous version:
https://lore.kernel.org/linux-scsi/[email protected]/T/#t

Changelog v3 -> v4:

[...]

Applied to 7.1/scsi-queue, thanks!

[1/2] scsi: virtio_scsi: move INIT_WORK calls to virtscsi_probe

      https://git.kernel.org/mkp/scsi/c/da3159a3b3fd
[2/2] scsi: virtio_scsi: kick event_list unconditionally
      https://git.kernel.org/mkp/scsi/c/0019a3a5756b
Signature · 1 line
Martin K. Petersen