Revert "Call blk_mq_free_tag_set() earlier"
cover Revert "Call blk_mq_free_tag_set() earlier"
Hi Martin,
Since a device, target or host reference may be held when scsi_remove_host() or scsi_remove_target() is called and since te patch series "Call blk_mq_free_tag_set() earlier" makes these functions wait until all references are gone, that patch series may trigger a deadlock. Hence this request to revert the patch series "Call blk_mq_free_tag_set() earlier".
Thanks,
Bart.
Bart Van Assche (4):
scsi: core: Revert "Call blk_mq_free_tag_set() earlier"
scsi: core: Revert "Simplify LLD module reference counting"
scsi: core: Revert "Make sure that hosts outlive targets"
scsi: core: Revert "Make sure that targets outlive devices"
drivers/scsi/hosts.c | 18 +++++------------- drivers/scsi/scsi.c | 9 +++------ drivers/scsi/scsi_scan.c | 9 --------- drivers/scsi/scsi_sysfs.c | 29 ++++++++++++----------------- include/scsi/scsi_device.h | 2 -- include/scsi/scsi_host.h | 3 --- 6 files changed, 20 insertions(+), 50 deletions(-)
PATCH 1/4 scsi: core: Revert "Call blk_mq_free_tag_set() earlier"
Revert the patch series "Call blk_mq_free_tag_set() earlier" because it introduces a deadlock if the scsi_remove_host() caller holds a reference on a device, target or host.
--- drivers/scsi/hosts.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
drivers/scsi/hosts.c +5 -5
@@ -197,8 +197,6 @@ void scsi_remove_host(struct Scsi_Host *shost)* the dependent SCSI targets and devices are gone before returning. */ wait_event (shost->targets_wq,atomic_read (&shost->target_count) ==0 ); - -scsi_mq_destroy_tags (shost); }EXPORT_SYMBOL (scsi_remove_host);
@@ -309,8 +307,8 @@ int scsi_add_host_with_dma(struct Scsi_Host *shost, struct device *dev,return error;/* -* Any resources associated with the SCSI host in this function except -* the tag set will be freed by scsi_host_dev_release(). +* Any host allocation in this function will be freed in +* scsi_host_dev_release(). */ out_del_dev:device_del (&shost->shost_dev);
@@ -326,7 +324,6 @@ int scsi_add_host_with_dma(struct Scsi_Host *shost, struct device *dev,pm_runtime_disable (&shost->shost_gendev);pm_runtime_set_suspended (&shost->shost_gendev);pm_runtime_put_noidle (&shost->shost_gendev); -scsi_mq_destroy_tags (shost); fail:return error; }
@@ -360,6 +357,9 @@ static void scsi_host_dev_release(struct device *dev)kfree (dev_name (&shost->shost_dev)); } +if (shost->tag_set.tags) +scsi_mq_destroy_tags (shost); +kfree (shost->shost_data);ida_free (&host_index_ida, shost->host_no);
PATCH 2/4 scsi: core: Revert "Simplify LLD module reference counting"
In reply to Bart Van AsscheRevert the patch series "Call blk_mq_free_tag_set() earlier" because it introduces a deadlock if the scsi_remove_host() caller holds a reference on a device, target or host.
--- drivers/scsi/scsi.c | 9 +++------ drivers/scsi/scsi_sysfs.c | 9 +++++++++ 2 files changed, 12 insertions(+), 6 deletions(-)
drivers/scsi/scsi.c +3 -6
@@ -586,13 +586,10 @@ EXPORT_SYMBOL(scsi_device_get);*/ void scsi_device_put (struct scsi_device *sdev) { -/* -* Decreasing the module reference count before the device reference -* count is safe since scsi_remove_host() only returns after all -* devices have been removed. -*/ -module_put (sdev->host->hostt->module); +struct module *mod = sdev->host->hostt->module; +put_device (&sdev->sdev_gendev); +module_put (mod); }EXPORT_SYMBOL (scsi_device_put);
drivers/scsi/scsi_sysfs.c +9 -0
@@ -452,6 +452,9 @@ static void scsi_device_dev_release_usercontext(struct work_struct *work)struct scsi_vpd *vpd_pg0 =NULL , *vpd_pg89 =NULL ;struct scsi_vpd *vpd_pgb0 =NULL , *vpd_pgb1 =NULL , *vpd_pgb2 =NULL ;unsigned long flags; +struct module *mod; + + mod = sdev->host->hostt->module;scsi_dh_release_device (sdev);
@@ -518,11 +521,17 @@ static void scsi_device_dev_release_usercontext(struct work_struct *work)if (parent)put_device (parent); +module_put (mod); }static void scsi_device_dev_release (struct device *dev) {struct scsi_device *sdp =to_scsi_device (dev); + +/* Set module pointer as NULL in case of module unloading */ +if (!try_module_get (sdp->host->hostt->module)) + sdp->host->hostt->module =NULL ; +execute_in_process_context (scsi_device_dev_release_usercontext, &sdp->ew); }
PATCH 3/4 scsi: core: Revert "Make sure that hosts outlive targets"
In reply to Bart Van AsscheRevert the patch series "Call blk_mq_free_tag_set() earlier" because it introduces a deadlock if the scsi_remove_host() caller holds a reference on a device, target or host.
--- drivers/scsi/hosts.c | 8 -------- drivers/scsi/scsi_scan.c | 7 ------- include/scsi/scsi_host.h | 3 --- 3 files changed, 18 deletions(-)
drivers/scsi/hosts.c +0 -8
@@ -190,13 +190,6 @@ void scsi_remove_host(struct Scsi_Host *shost)transport_unregister_device (&shost->shost_gendev);device_unregister (&shost->shost_dev);device_del (&shost->shost_gendev); - -/* -* After scsi_remove_host() has returned the scsi LLD module can be -* unloaded and/or the host resources can be released. Hence wait until -* the dependent SCSI targets and devices are gone before returning. -*/ -wait_event (shost->targets_wq,atomic_read (&shost->target_count) ==0 ); }EXPORT_SYMBOL (scsi_remove_host);
@@ -406,7 +399,6 @@ struct Scsi_Host *scsi_host_alloc(struct scsi_host_template *sht, int privsize)INIT_LIST_HEAD (&shost->starved_list);init_waitqueue_head (&shost->host_wait);mutex_init (&shost->scan_mutex); -init_waitqueue_head (&shost->targets_wq); index =ida_alloc (&host_index_ida,GFP_KERNEL );if (index <0 ) {
drivers/scsi/scsi_scan.c +0 -7
@@ -406,14 +406,9 @@ static void scsi_target_destroy(struct scsi_target *starget)static void scsi_target_dev_release (struct device *dev) {struct device *parent = dev->parent; -struct Scsi_Host *shost =dev_to_shost (parent);struct scsi_target *starget =to_scsi_target (dev);kfree (starget); - -if (atomic_dec_return (&shost->target_count) ==0 ) -wake_up (&shost->targets_wq); -put_device (parent); }
@@ -528,8 +523,6 @@ static struct scsi_target *scsi_alloc_target(struct device *parent, starget->max_target_blocked =SCSI_DEFAULT_TARGET_BLOCKED ;init_waitqueue_head (&starget->sdev_wq); -atomic_inc (&shost->target_count); - retry:spin_lock_irqsave (shost->host_lock, flags);
include/scsi/scsi_host.h +0 -3
@@ -690,9 +690,6 @@ struct Scsi_Host {/* ldm bits */ struct device shost_gendev, shost_dev; -atomic_t target_count; -wait_queue_head_t targets_wq; -/* * Points to the transport data (if any) which is allocated * separately
PATCH 4/4 scsi: core: Revert "Make sure that targets outlive devices"
In reply to Bart Van AsscheRevert the patch series "Call blk_mq_free_tag_set() earlier" because it introduces a deadlock if the scsi_remove_host() caller holds a reference on a device, target or host.
--- drivers/scsi/scsi_scan.c | 2 -- drivers/scsi/scsi_sysfs.c | 20 +++----------------- include/scsi/scsi_device.h | 2 -- 3 files changed, 3 insertions(+), 21 deletions(-)
drivers/scsi/scsi_scan.c +0 -2
@@ -521,8 +521,6 @@ static struct scsi_target *scsi_alloc_target(struct device *parent, starget->state =STARGET_CREATED ; starget->scsi_level =SCSI_2 ; starget->max_target_blocked =SCSI_DEFAULT_TARGET_BLOCKED ; -init_waitqueue_head (&starget->sdev_wq); - retry:spin_lock_irqsave (shost->host_lock, flags);
drivers/scsi/scsi_sysfs.c +3 -17
@@ -443,9 +443,7 @@ static void scsi_device_cls_release(struct device *class_dev)static void scsi_device_dev_release_usercontext (struct work_struct *work) { - struct scsi_device *sdev = container_of(work, struct scsi_device, - ew.work); -struct scsi_target *starget = sdev->sdev_target; +struct scsi_device *sdev;struct device *parent;struct list_head *this, *tmp;struct scsi_vpd *vpd_pg80 =NULL , *vpd_pg83 =NULL ;
@@ -454,6 +452,8 @@ static void scsi_device_dev_release_usercontext(struct work_struct *work)unsigned long flags;struct module *mod; + sdev = container_of(work, struct scsi_device, ew.work); + mod = sdev->host->hostt->module;scsi_dh_release_device (sdev);
@@ -516,9 +516,6 @@ static void scsi_device_dev_release_usercontext(struct work_struct *work)kfree (sdev->inquiry);kfree (sdev); -if (starget &&atomic_dec_return (&starget->sdev_count) ==0 ) -wake_up (&starget->sdev_wq); -if (parent)put_device (parent);module_put (mod);
@@ -1538,14 +1535,6 @@ static void __scsi_remove_target(struct scsi_target *starget) goto restart; }spin_unlock_irqrestore (shost->host_lock, flags); - -/* -* After scsi_remove_target() returns its caller can remove resources -* associated with @starget, e.g. an rport or session. Wait until all -* devices associated with @starget have been removed to prevent that -* a SCSI error handling callback function triggers a use-after-free. -*/ -wait_event (starget->sdev_wq,atomic_read (&starget->sdev_count) ==0 ); }/**
@@ -1656,9 +1645,6 @@ void scsi_sysfs_device_initialize(struct scsi_device *sdev)list_add_tail (&sdev->same_target_siblings, &starget->devices);list_add_tail (&sdev->siblings, &shost->__devices);spin_unlock_irqrestore (shost->host_lock, flags); - -atomic_inc (&starget->sdev_count); -/* * device can now only be removed via __scsi_remove_device() so hold * the target. Target will be held in CREATED state until something
include/scsi/scsi_device.h +0 -2
@@ -309,8 +309,6 @@ struct scsi_target {struct list_head devices;struct device dev;struct kref reap_ref;/* last put renders target invisible */ -atomic_t sdev_count; -wait_queue_head_t sdev_wq;unsigned int channel;unsigned int id;/* target id ... replace * scsi_device.id eventually */
Hi Bart,
Quoted 8 lines · Bart Van Assche
Hi Martin,
Since a device, target or host reference may be held when
scsi_remove_host()orscsi_remove_target()is called and since te patch series "Callblk_mq_free_tag_set()earlier" makes these functions wait until all references are gone, that patch series may trigger a deadlock. Hence this request to revert the patch series "Callblk_mq_free_tag_set()earlier".
Care to share the deadlock details? Such as dmesg log or theory behind.
Thanks,
Ming
Quoted 8 lines · Ming Lei
Quoted 5 lines · Bart Van Assche
Since a device, target or host reference may be held when
scsi_remove_host()orscsi_remove_target()is called and since te patch series "Callblk_mq_free_tag_set()earlier" makes these functions wait until all references are gone, that patch series may trigger a deadlock. Hence this request to revert the patch series "Callblk_mq_free_tag_set()earlier".Care to share the deadlock details? Such as dmesg log or theory behind.
Hi Ming,
Details of two different deadlock scenarios are available here:
* [syzbot] INFO: task hung in scsi_remove_host (https://lore.kernel.org/all/[email protected]/).
* https://lore.kernel.org/all/Yv%[email protected]/. The link [2] in this email includes a call trace and instructions for reproducing this issue. My root cause analysis of this deadlock is available here: https://lore.kernel.org/all/[email protected]/.
Thanks,
Bart.
Martin K. Petersen
AppliedQuoted 9 lines · Bart Van Assche
Since a device, target or host reference may be held when
scsi_remove_host()orscsi_remove_target()is called and since te patch series "Callblk_mq_free_tag_set()earlier" makes these functions wait until all references are gone, that patch series may trigger a deadlock. Hence this request to revert the patch series "Callblk_mq_free_tag_set()earlier".Thanks,
[...]
Applied to 6.0/scsi-fixes, thanks!
[1/4] scsi: core: Revert "Call blk_mq_free_tag_set() earlier"
https://git.kernel.org/mkp/scsi/c/2b36209ca818 [2/4] scsi: core: Revert "Simplify LLD module reference counting" https://git.kernel.org/mkp/scsi/c/70e8d057bef5 [3/4] scsi: core: Revert "Make sure that hosts outlive targets" https://git.kernel.org/mkp/scsi/c/d94b2d00f7bf [4/4] scsi: core: Revert "Make sure that targets outlive devices" https://git.kernel.org/mkp/scsi/c/f782201ebc2b
Signature · 1 line
Martin K. Petersen Oracle Linux Engineering