scsi: core: put LLD module refcnt after SCSI device is released
PATCH v2 1/1 scsi: core: put LLD module refcnt after SCSI device is released
SCSI host release is triggered when SCSI device is freed, and we have to make sure that LLD module won't be unloaded before SCSI host instance is released because shost->hostt is required in host release handler.
So put LLD module refcnt after SCSI device is released.
The real release handler can be run from wq context in case of in_interrupt(), so add one atomic counter for serializing putting module via current and wq context. This way is fine since we don't call scsi_device_put() in fast IO path.
--- drivers/scsi/scsi.c | 8 +++++++- drivers/scsi/scsi_sysfs.c | 10 ++++++++++ include/scsi/scsi_device.h | 2 ++ 3 files changed, 19 insertions(+), 1 deletion(-)
drivers/scsi/scsi.c +7 -1
@@ -553,8 +553,14 @@ EXPORT_SYMBOL(scsi_device_get);*/ void scsi_device_put (struct scsi_device *sdev) { -module_put (sdev->host->hostt->module); +struct module *mod = sdev->host->hostt->module; + +atomic_inc (&sdev->put_dev_cnt); +put_device (&sdev->sdev_gendev); + +if (atomic_dec_if_positive (&sdev->put_dev_cnt) >=0 ) +module_put (mod); }EXPORT_SYMBOL (scsi_device_put);
drivers/scsi/scsi_sysfs.c +10 -0
@@ -449,9 +449,16 @@ static void scsi_device_dev_release_usercontext(struct work_struct *work)struct scsi_vpd *vpd_pg80 =NULL , *vpd_pg83 =NULL ;struct scsi_vpd *vpd_pg0 =NULL , *vpd_pg89 =NULL ;unsigned long flags; +struct module *mod; +bool put_mod = false; sdev = container_of(work, struct scsi_device, ew.work); +if (atomic_dec_if_positive (&sdev->put_dev_cnt) >=0 ) { + put_mod = true; + mod = sdev->host->hostt->module; + } +scsi_dh_release_device (sdev); parent = sdev->sdev_gendev.parent;
@@ -502,6 +509,9 @@ static void scsi_device_dev_release_usercontext(struct work_struct *work)if (parent)put_device (parent); + +if (put_mod) +module_put (mod); }static void scsi_device_dev_release (struct device *dev)
include/scsi/scsi_device.h +2 -0
@@ -111,6 +111,8 @@ struct scsi_device {struct sbitmap budget_map;atomic_t device_blocked;/* Device returned QUEUE_FULL. */ +atomic_t put_dev_cnt;/* increased by 1 when we are putting device */ +atomic_t restarts;spinlock_t list_lock;struct list_head starved_entry;
Signature · 1 line
2.31.1
Quoted 28 lines · Ming Lei
SCSI host release is triggered when SCSI device is freed, and we have to make sure that LLD module won't be unloaded before SCSI host instance is released because
shost->hosttis required in host release handler.So put LLD module refcnt after SCSI device is released.
The real release handler can be run from wq context in case of
in_interrupt(), so add one atomic counter for serializing putting module via current and wq context. This way is fine since we don't callscsi_device_put()in fast IO path.Reported-by Changhui Zhong <[email protected]>Reported-by Yi Zhang <[email protected]>Cc Greg Kroah-Hartman <[email protected]>Signed-off-by Ming Lei <[email protected]>--- drivers/scsi/scsi.c | 8 +++++++- drivers/scsi/scsi_sysfs.c | 10 ++++++++++ include/scsi/scsi_device.h | 2 ++ 3 files changed, 19 insertions(+), 1 deletion(-)drivers/scsi/scsi.c +0
-0@@ -553,8 +553,14 @@ EXPORT_SYMBOL(scsi_device_get);*/ void scsi_device_put (struct scsi_device *sdev)
drivers/scsi/scsi.c EXPORT_SYMBOL(scsi_device_get); · line 563
Quoted 10 lines
drivers/scsi/scsi.c +7
-1{ -module_put (sdev->host->hostt->module); +struct module *mod = sdev->host->hostt->module; + +atomic_inc (&sdev->put_dev_cnt); +put_device (&sdev->sdev_gendev); + +if (atomic_dec_if_positive (&sdev->put_dev_cnt) >=0 ) +module_put (mod);
oops, sdev can be freed now, so this approach isn't good too, :-(
Will think further about the solution.
thanks,
Ming
Quoted 26 lines · Ming Lei
SCSI host release is triggered when SCSI device is freed, and we have to make sure that LLD module won't be unloaded before SCSI host instance is released because
shost->hosttis required in host release handler.So put LLD module refcnt after SCSI device is released.
The real release handler can be run from wq context in case of
in_interrupt(), so add one atomic counter for serializing putting module via current and wq context. This way is fine since we don't callscsi_device_put()in fast IO path.Reported-by Changhui Zhong <[email protected]>Reported-by Yi Zhang <[email protected]>Cc Greg Kroah-Hartman <[email protected]>Signed-off-by Ming Lei <[email protected]>--- drivers/scsi/scsi.c | 8 +++++++- drivers/scsi/scsi_sysfs.c | 10 ++++++++++ include/scsi/scsi_device.h | 2 ++ 3 files changed, 19 insertions(+), 1 deletion(-)drivers/scsi/scsi.c +0
-0
drivers/scsi/scsi.c EXPORT_SYMBOL(scsi_device_get); · line 558
Quoted 7 lines
drivers/scsi/scsi.c +3
-1@@ -553,8 +553,14 @@ EXPORT_SYMBOL(scsi_device_get);*/ void scsi_device_put (struct scsi_device *sdev) { -module_put (sdev->host->hostt->module); +struct module *mod = sdev->host->hostt->module; + +atomic_inc (&sdev->put_dev_cnt);
Ick, no! Why are you making a new lock and reference count for no reason?
drivers/scsi/scsi.c EXPORT_SYMBOL(scsi_device_get); · line 563
Quoted 5 lines
drivers/scsi/scsi.c +4
-0+put_device (&sdev->sdev_gendev); + +if (atomic_dec_if_positive (&sdev->put_dev_cnt) >=0 ) +module_put (mod);
How do you know if your module pointer is still valid here?
Why do you care?
What problem are you trying to solve and why is it unique to scsi devices?
thanks,
greg k-h
Quoted 22 lines · Greg Kroah-Hartman
Quoted 21 lines · Ming Lei
SCSI host release is triggered when SCSI device is freed, and we have to make sure that LLD module won't be unloaded before SCSI host instance is released because
shost->hosttis required in host release handler.So put LLD module refcnt after SCSI device is released.
The real release handler can be run from wq context in case of
in_interrupt(), so add one atomic counter for serializing putting module via current and wq context. This way is fine since we don't callscsi_device_put()in fast IO path.Reported-by Changhui Zhong <[email protected]>Reported-by Yi Zhang <[email protected]>Cc Greg Kroah-Hartman <[email protected]>Signed-off-by Ming Lei <[email protected]>--- drivers/scsi/scsi.c | 8 +++++++- drivers/scsi/scsi_sysfs.c | 10 ++++++++++ include/scsi/scsi_device.h | 2 ++ 3 files changed, 19 insertions(+), 1 deletion(-)
Quoted 15 lines
drivers/scsi/scsi.c EXPORT_SYMBOL(scsi_device_get); · line 558
Quoted 12 lines
drivers/scsi/scsi.c +3
-1@@ -553,8 +553,14 @@ EXPORT_SYMBOL(scsi_device_get);*/ void scsi_device_put (struct scsi_device *sdev) { -module_put (sdev->host->hostt->module); +struct module *mod = sdev->host->hostt->module; + +atomic_inc (&sdev->put_dev_cnt);Ick, no! Why are you making a new lock and reference count for no reason?
The reason is to make sure that the LLD module is only put from either scsi_device_put() and scsi_device_dev_release_usercontext().
Quoted 8 lines
drivers/scsi/scsi.c EXPORT_SYMBOL(scsi_device_get); · line 563
Quoted 6 lines
drivers/scsi/scsi.c +4
-0+put_device (&sdev->sdev_gendev); + +if (atomic_dec_if_positive (&sdev->put_dev_cnt) >=0 ) +module_put (mod);How do you know if your module pointer is still valid here?
module refcnt is grabbed in scsi_device_get(), so it is valid.
Quoted 5 lines
Why do you care?
What problem are you trying to solve and why is it unique to scsi devices?
See it from the commit log:
SCSI host release is triggered when SCSI device is freed, and we have to make sure that LLD module won't be unloaded before SCSI host instance is released because shost->hostt is required in host release handler.
So put LLD module refcnt after SCSI device is released.
and the upstream report on the issue:
Thanks,
Ming
Quoted 51 lines · Ming Lei
Quoted 22 lines · Greg Kroah-Hartman
Quoted 21 lines · Ming Lei
SCSI host release is triggered when SCSI device is freed, and we have to make sure that LLD module won't be unloaded before SCSI host instance is released because
shost->hosttis required in host release handler.So put LLD module refcnt after SCSI device is released.
The real release handler can be run from wq context in case of
in_interrupt(), so add one atomic counter for serializing putting module via current and wq context. This way is fine since we don't callscsi_device_put()in fast IO path.Reported-by Changhui Zhong <[email protected]>Reported-by Yi Zhang <[email protected]>Cc Greg Kroah-Hartman <[email protected]>Signed-off-by Ming Lei <[email protected]>--- drivers/scsi/scsi.c | 8 +++++++- drivers/scsi/scsi_sysfs.c | 10 ++++++++++ include/scsi/scsi_device.h | 2 ++ 3 files changed, 19 insertions(+), 1 deletion(-)Quoted 15 lines
drivers/scsi/scsi.c EXPORT_SYMBOL(scsi_device_get); · line 558
Quoted 12 lines
drivers/scsi/scsi.c +3
-1@@ -553,8 +553,14 @@ EXPORT_SYMBOL(scsi_device_get);*/ void scsi_device_put (struct scsi_device *sdev) { -module_put (sdev->host->hostt->module); +struct module *mod = sdev->host->hostt->module; + +atomic_inc (&sdev->put_dev_cnt);Ick, no! Why are you making a new lock and reference count for no reason?
The reason is to make sure that the LLD module is only put from either
scsi_device_put()andscsi_device_dev_release_usercontext().Quoted 8 lines
drivers/scsi/scsi.c EXPORT_SYMBOL(scsi_device_get); · line 563
Quoted 6 lines
drivers/scsi/scsi.c +4
-0+put_device (&sdev->sdev_gendev); + +if (atomic_dec_if_positive (&sdev->put_dev_cnt) >=0 ) +module_put (mod);How do you know if your module pointer is still valid here?
Quoted 1 line
module refcnt is grabbed in
scsi_device_get(), so it is valid.
Then you don't need the extra atomic variable.
Quoted 11 lines
Quoted 5 lines
Why do you care?
What problem are you trying to solve and why is it unique to scsi devices?
See it from the commit log:
SCSI host release is triggered when SCSI device is freed, and we have to make sure that LLD module won't be unloaded before SCSI host instance is released because
shost->hosttis required in host release handler.
What is "hostt"?
Quoted 2 lines
So put LLD module refcnt after SCSI device is released.
Why not just drop it explicitly when you drop the reference count of the device object? Like you tried to do here, but no need for the extra atomic variable.
thanks,
greg k-h
Quoted 62 lines · Greg Kroah-Hartman
Quoted 51 lines · Ming Lei
Quoted 22 lines · Greg Kroah-Hartman
Quoted 21 lines · Ming Lei
SCSI host release is triggered when SCSI device is freed, and we have to make sure that LLD module won't be unloaded before SCSI host instance is released because
shost->hosttis required in host release handler.So put LLD module refcnt after SCSI device is released.
The real release handler can be run from wq context in case of
in_interrupt(), so add one atomic counter for serializing putting module via current and wq context. This way is fine since we don't callscsi_device_put()in fast IO path.Reported-by Changhui Zhong <[email protected]>Reported-by Yi Zhang <[email protected]>Cc Greg Kroah-Hartman <[email protected]>Signed-off-by Ming Lei <[email protected]>--- drivers/scsi/scsi.c | 8 +++++++- drivers/scsi/scsi_sysfs.c | 10 ++++++++++ include/scsi/scsi_device.h | 2 ++ 3 files changed, 19 insertions(+), 1 deletion(-)Quoted 15 lines
drivers/scsi/scsi.c EXPORT_SYMBOL(scsi_device_get); · line 558
Quoted 12 lines
drivers/scsi/scsi.c +3
-1@@ -553,8 +553,14 @@ EXPORT_SYMBOL(scsi_device_get);*/ void scsi_device_put (struct scsi_device *sdev) { -module_put (sdev->host->hostt->module); +struct module *mod = sdev->host->hostt->module; + +atomic_inc (&sdev->put_dev_cnt);Ick, no! Why are you making a new lock and reference count for no reason?
The reason is to make sure that the LLD module is only put from either
scsi_device_put()andscsi_device_dev_release_usercontext().Quoted 8 lines
drivers/scsi/scsi.c EXPORT_SYMBOL(scsi_device_get); · line 563
Quoted 6 lines
drivers/scsi/scsi.c +4
-0+put_device (&sdev->sdev_gendev); + +if (atomic_dec_if_positive (&sdev->put_dev_cnt) >=0 ) +module_put (mod);How do you know if your module pointer is still valid here?
Quoted 1 line
module refcnt is grabbed in
scsi_device_get(), so it is valid.Then you don't need the extra atomic variable.
Quoted 6 lines
Quoted 5 lines
Why do you care?
What problem are you trying to solve and why is it unique to scsi devices?
Quoted 7 lines
Quoted 5 lines
See it from the commit log:
SCSI host release is triggered when SCSI device is freed, and we have to make sure that LLD module won't be unloaded before SCSI host instance is released because
shost->hosttis required in host release handler.What is "hostt"?
hostt is 'struct scsi_host_template' which is defined in LLD module, and often allocated as static global variable, that is what try_get_module() tries to protect.
Quoted 7 lines
Quoted 3 lines
So put LLD module refcnt after SCSI device is released.
Why not just drop it explicitly when you drop the reference count of the device object? Like you tried to do here, but no need for the extra atomic variable.
scsi_device_dev_release_usercontext() may be scheduled via schedule_work from the device object's release handler for releasing the scsi_device, which may trigger scsi host's release handler in which hostt is required.
If we simply call module_put() after put_device() simply, the module refcnt may be dropped earlier than running scsi_device_dev_release_usercontext(), then the kernel panic still can't be addressed.
Thanks,
Ming
Quoted 79 lines · Ming Lei
Quoted 62 lines · Greg Kroah-Hartman
Quoted 51 lines · Ming Lei
Quoted 22 lines · Greg Kroah-Hartman
Quoted 21 lines · Ming Lei
SCSI host release is triggered when SCSI device is freed, and we have to make sure that LLD module won't be unloaded before SCSI host instance is released because
shost->hosttis required in host release handler.So put LLD module refcnt after SCSI device is released.
The real release handler can be run from wq context in case of
in_interrupt(), so add one atomic counter for serializing putting module via current and wq context. This way is fine since we don't callscsi_device_put()in fast IO path.Reported-by Changhui Zhong <[email protected]>Reported-by Yi Zhang <[email protected]>Cc Greg Kroah-Hartman <[email protected]>Signed-off-by Ming Lei <[email protected]>--- drivers/scsi/scsi.c | 8 +++++++- drivers/scsi/scsi_sysfs.c | 10 ++++++++++ include/scsi/scsi_device.h | 2 ++ 3 files changed, 19 insertions(+), 1 deletion(-)Quoted 15 lines
drivers/scsi/scsi.c EXPORT_SYMBOL(scsi_device_get); · line 558
Quoted 12 lines
drivers/scsi/scsi.c +3
-1@@ -553,8 +553,14 @@ EXPORT_SYMBOL(scsi_device_get);*/ void scsi_device_put (struct scsi_device *sdev) { -module_put (sdev->host->hostt->module); +struct module *mod = sdev->host->hostt->module; + +atomic_inc (&sdev->put_dev_cnt);Ick, no! Why are you making a new lock and reference count for no reason?
The reason is to make sure that the LLD module is only put from either
scsi_device_put()andscsi_device_dev_release_usercontext().Quoted 8 lines
drivers/scsi/scsi.c EXPORT_SYMBOL(scsi_device_get); · line 563
Quoted 6 lines
drivers/scsi/scsi.c +4
-0+put_device (&sdev->sdev_gendev); + +if (atomic_dec_if_positive (&sdev->put_dev_cnt) >=0 ) +module_put (mod);How do you know if your module pointer is still valid here?
Quoted 1 line
module refcnt is grabbed in
scsi_device_get(), so it is valid.Then you don't need the extra atomic variable.
Quoted 6 lines
Quoted 5 lines
Why do you care?
What problem are you trying to solve and why is it unique to scsi devices?
Quoted 7 lines
Quoted 5 lines
See it from the commit log:
SCSI host release is triggered when SCSI device is freed, and we have to make sure that LLD module won't be unloaded before SCSI host instance is released because
shost->hosttis required in host release handler.What is "hostt"?
hostt is 'struct
scsi_host_template' which is defined in LLD module, and often allocated as static global variable, that is whattry_get_module()tries to protect.Quoted 4 lines
Quoted 3 lines
So put LLD module refcnt after SCSI device is released.
Quoted 7 lines
Quoted 3 lines
Why not just drop it explicitly when you drop the reference count of the device object? Like you tried to do here, but no need for the extra atomic variable.
scsi_device_dev_release_usercontext()may be scheduled viaschedule_workfrom the device object's release handler for releasing thescsi_device, which may trigger scsi host's release handler in which hostt is required.
If a release handler can be called from the device release function, then that is when you need to drop the reference, after that function is finished being called, right?
thanks,
greg k-h
Quoted 84 lines · Greg Kroah-Hartman
Quoted 83 lines · Ming Lei
Quoted 62 lines · Greg Kroah-Hartman
Quoted 51 lines · Ming Lei
Quoted 22 lines · Greg Kroah-Hartman
Quoted 21 lines · Ming Lei
SCSI host release is triggered when SCSI device is freed, and we have to make sure that LLD module won't be unloaded before SCSI host instance is released because
shost->hosttis required in host release handler.So put LLD module refcnt after SCSI device is released.
The real release handler can be run from wq context in case of
in_interrupt(), so add one atomic counter for serializing putting module via current and wq context. This way is fine since we don't callscsi_device_put()in fast IO path.Reported-by Changhui Zhong <[email protected]>Reported-by Yi Zhang <[email protected]>Cc Greg Kroah-Hartman <[email protected]>Signed-off-by Ming Lei <[email protected]>--- drivers/scsi/scsi.c | 8 +++++++- drivers/scsi/scsi_sysfs.c | 10 ++++++++++ include/scsi/scsi_device.h | 2 ++ 3 files changed, 19 insertions(+), 1 deletion(-)Quoted 15 lines
drivers/scsi/scsi.c EXPORT_SYMBOL(scsi_device_get); · line 558
Quoted 12 lines
drivers/scsi/scsi.c +3
-1@@ -553,8 +553,14 @@ EXPORT_SYMBOL(scsi_device_get);*/ void scsi_device_put (struct scsi_device *sdev) { -module_put (sdev->host->hostt->module); +struct module *mod = sdev->host->hostt->module; + +atomic_inc (&sdev->put_dev_cnt);Ick, no! Why are you making a new lock and reference count for no reason?
The reason is to make sure that the LLD module is only put from either
scsi_device_put()andscsi_device_dev_release_usercontext().Quoted 8 lines
drivers/scsi/scsi.c EXPORT_SYMBOL(scsi_device_get); · line 563
Quoted 6 lines
drivers/scsi/scsi.c +4
-0+put_device (&sdev->sdev_gendev); + +if (atomic_dec_if_positive (&sdev->put_dev_cnt) >=0 ) +module_put (mod);How do you know if your module pointer is still valid here?
Quoted 1 line
module refcnt is grabbed in
scsi_device_get(), so it is valid.Then you don't need the extra atomic variable.
Quoted 6 lines
Quoted 5 lines
Why do you care?
What problem are you trying to solve and why is it unique to scsi devices?
Quoted 7 lines
Quoted 5 lines
See it from the commit log:
SCSI host release is triggered when SCSI device is freed, and we have to make sure that LLD module won't be unloaded before SCSI host instance is released because
shost->hosttis required in host release handler.What is "hostt"?
hostt is 'struct
scsi_host_template' which is defined in LLD module, and often allocated as static global variable, that is whattry_get_module()tries to protect.Quoted 7 lines
Quoted 3 lines
So put LLD module refcnt after SCSI device is released.
Why not just drop it explicitly when you drop the reference count of the device object? Like you tried to do here, but no need for the extra atomic variable.
Quoted 7 lines
Quoted 3 lines
scsi_device_dev_release_usercontext()may be scheduled viaschedule_workfrom the device object's release handler for releasing thescsi_device, which may trigger scsi host's release handler in which hostt is required.If a release handler can be called from the device release function, then that is when you need to drop the reference, after that function is finished being called, right?
No.
Not like device object refcnt, module refcnt is special, it is grabbed when someone is using the device, then module won't be unloaded when there is active user. When no one uses the device, we should allow rmmod to unload the module.
SCSI stack models the device uses via scsi_device_get() and scsi_device_put(), such as, when one disk is opened, scsi_device_get() is called, and scsi_device_put() is called when the disk is closed.
Wrt. this issue, we can think that the LLD module holds one device refcnt of the scsi_host, and the refcnt is dropped when running module_exit(). But one disk attached to the host may still be opened by userspace, so the scsi_host won't be released until the disk is closed, and the LLD module won't be unloaded until the disk is closed too. When the disk is closed, scsi_device_put() is called and module_put() can unload the LLD module immediately if rmmod is started.
So what the patch is doing is that we need to make sure that module refcnt is put after the device is released since the host is released from its release handler directly if rmmod has been started.
Thanks,
Ming