mmc: block: Prevent new req entering queue while freeing up the queue
Subsystems (2)
PATCH 1/1 mmc: block: Prevent new req entering queue while freeing up the queue
The commit bbdc74dc19e0 ("mmc: block: Prevent new req entering queue after its cleanup") has introduced this change but it got moved after del_gendisk() with commit 57678e5a3d51 ("mmc: block: Delete gendisk before cleaning up the request queue").
It is blocking reboot with below Call stack().
INFO: task reboot:3086 blocked for more than 122 seconds.
__schedule
schedule
schedule_timeout
io_schedule_timeout
do_wait_for_common
wait_for_completion_io
submit_bio_wait
blkdev_issue_flush
ext4_sync_fs
__sync_filesystem
sync_filesystem
fsync_bdev
invalidate_partition
del_gendisk
mmc_blk_remove_req
mmc_blk_remove
mmc_bus_remove
device_release_driver_internal
device_release_driver
bus_remove_device
device_del
mmc_remove_card
mmc_remove
mmc_stop_host
mmc_remove_host
sdhci_remove_host
sdhci_msm_remove
sdhci_msm_shutdown
platform_drv_shutdown
device_shutdown
kernel_restart_prepare
kernel_restartSo bringing this change back.
---
I'm observing this issue 100% of the time with shutdown callback added to sdhci-msm driver. I'm trying on 5.4 kernel with ChromeOS.
Please let me know if this can be fixed in a better way.
---
drivers/mmc/core/block.c | 1 + 1 file changed, 1 insertion(+)
drivers/mmc/core/block.c +1 -0
@@ -2627,6 +2627,7 @@ static void mmc_blk_remove_req(struct mmc_blk_data *md)* from being accepted. */ card = md->queue.card; +blk_set_queue_dying (md->queue.queue);if (md->disk->flags &GENHD_FL_UP ) {device_remove_file (disk_to_dev (md->disk), &md->force_ro); if ((md->area_type & MMC_BLK_DATA_AREA_BOOT) &&
Signature · 1 line
Qualcomm India Private Limited, on behalf of Qualcomm Innovation Center, Inc., is a member of Code Aurora Forum, a Linux Foundation Collaborative Project
Hi Ulf, Adrian,
Gentle reminder. Can you share your comments on this issue and change?
Thanks
Quoted 67 lines · Veerabhadrarao Badiganti
The commit
bbdc74dc19e0("mmc: block: Prevent new req entering queue after its cleanup") has introduced this change but it got moved afterdel_gendisk()with commit57678e5a3d51("mmc: block: Delete gendisk before cleaning up the request queue").It is blocking reboot with below Call
stack().INFO: task reboot:3086 blocked for more than 122 seconds. __schedule schedule schedule_timeout io_schedule_timeout do_wait_for_common wait_for_completion_io submit_bio_wait blkdev_issue_flush ext4_sync_fs __sync_filesystem sync_filesystem fsync_bdev invalidate_partition del_gendisk mmc_blk_remove_req mmc_blk_remove mmc_bus_remove device_release_driver_internal device_release_driver bus_remove_device device_del mmc_remove_card mmc_remove mmc_stop_host mmc_remove_host sdhci_remove_host sdhci_msm_remove sdhci_msm_shutdown platform_drv_shutdown device_shutdown kernel_restart_prepare kernel_restartSo bringing this change back.
Signed-off-by Veerabhadrarao Badiganti <[email protected]>---
I'm observing this issue 100% of the time with shutdown callback added to sdhci-msm driver. I'm trying on 5.4 kernel with ChromeOS.
Please let me know if this can be fixed in a better way.
---drivers/mmc/core/block.c | 1 + 1 file changed, 1 insertion(+)drivers/mmc/core/block.c +1
-0@@ -2627,6 +2627,7 @@ static void mmc_blk_remove_req(struct mmc_blk_data *md)* from being accepted. */ card = md->queue.card; +blk_set_queue_dying (md->queue.queue);if (md->disk->flags &GENHD_FL_UP ) {device_remove_file (disk_to_dev (md->disk), &md->force_ro); if ((md->area_type & MMC_BLK_DATA_AREA_BOOT) &&
Quoted 5 lines · Veerabhadrarao Badiganti
The commit
bbdc74dc19e0("mmc: block: Prevent new req entering queue after its cleanup") has introduced this change but it got moved afterdel_gendisk()with commit57678e5a3d51("mmc: block: Delete gendisk before cleaning up the request queue").
This isn't the first time we have spotted errors in this path. Seems like a difficult path to get correct. :-)
Quoted 23 lines
It is blocking reboot with below Call
stack().INFO: task reboot:3086 blocked for more than 122 seconds. __schedule schedule schedule_timeout io_schedule_timeout do_wait_for_common wait_for_completion_io submit_bio_wait blkdev_issue_flush ext4_sync_fs __sync_filesystem sync_filesystem fsync_bdev invalidate_partition del_gendisk mmc_blk_remove_req mmc_blk_remove mmc_bus_remove device_release_driver_internal device_release_driver
Quoted 8 lines
bus_remove_device
device_del
mmc_remove_card
mmc_remove
mmc_stop_host
mmc_remove_host
sdhci_remove_host
sdhci_msm_removeWhy do you call sdhci_msm_remove() from the shutdown callback? What specific operations do you need to run in the shutdown path for sdhci msm?
The important part should be to do a graceful shutdown of the card (and the block device) - is there anything else?
Or you are just using the shutdown callback as a simple way to trigger this problem? Could unbinding the driver trigger the same issue?
Quoted 8 lines
sdhci_msm_shutdown
platform_drv_shutdown
device_shutdown
kernel_restart_prepare
kernel_restartSo bringing this change back.
Quoted 7 lines
Signed-off-by Veerabhadrarao Badiganti <[email protected]>---
I'm observing this issue 100% of the time with shutdown callback added to sdhci-msm driver. I'm trying on 5.4 kernel with ChromeOS.
Please let me know if this can be fixed in a better way.
I don't know yet, but I will have a closer look. Let's also see if Adrian has some thoughts.
Kind regards
Uffe
Quoted 20 lines
---
drivers/mmc/core/block.c | 1 + 1 file changed, 1 insertion(+)drivers/mmc/core/block.c +1
-1@@ -2627,6 +2627,7 @@ static void mmc_blk_remove_req(struct mmc_blk_data *md)* from being accepted. */ card = md->queue.card; +blk_set_queue_dying (md->queue.queue);if (md->disk->flags &GENHD_FL_UP ) {device_remove_file (disk_to_dev (md->disk), &md->force_ro); if ((md->area_type & MMC_BLK_DATA_AREA_BOOT) && --Qualcomm India Private Limited, on behalf of Qualcomm Innovation Center, Inc., is a member of Code Aurora Forum, a Linux Foundation Collaborative Project
Quoted 43 lines · Ulf Hansson
Quoted 5 lines · Veerabhadrarao Badiganti
The commit
bbdc74dc19e0("mmc: block: Prevent new req entering queue after its cleanup") has introduced this change but it got moved afterdel_gendisk()with commit57678e5a3d51("mmc: block: Delete gendisk before cleaning up the request queue").This isn't the first time we have spotted errors in this path. Seems like a difficult path to get correct. :-)
Quoted 23 lines
It is blocking reboot with below Call
stack().INFO: task reboot:3086 blocked for more than 122 seconds. __schedule schedule schedule_timeout io_schedule_timeout do_wait_for_common wait_for_completion_io submit_bio_wait blkdev_issue_flush ext4_sync_fs __sync_filesystem sync_filesystem fsync_bdev invalidate_partition del_gendisk mmc_blk_remove_req mmc_blk_remove mmc_bus_remove device_release_driver_internal device_release_driverQuoted 8 lines
bus_remove_device device_del mmc_remove_card mmc_remove mmc_stop_host mmc_remove_host sdhci_remove_host sdhci_msm_remove
Quoted 3 lines
Why do you call
sdhci_msm_remove()from the shutdown callback? What specific operations do you need to run in the shutdown path for sdhci msm?
Yes, the problem is that upper layers, like the mmc block driver, have already shut down, so doing operations like remove will get into deadlocks.
Quoted 49 lines
The important part should be to do a graceful shutdown of the card (and the block device) - is there anything else?
Or you are just using the shutdown callback as a simple way to trigger this problem? Could unbinding the driver trigger the same issue?
Quoted 8 lines
sdhci_msm_shutdown
platform_drv_shutdown
device_shutdown
kernel_restart_prepare
kernel_restartSo bringing this change back.
Quoted 7 lines
Signed-off-by Veerabhadrarao Badiganti <[email protected]>---
I'm observing this issue 100% of the time with shutdown callback added to sdhci-msm driver. I'm trying on 5.4 kernel with ChromeOS.
Please let me know if this can be fixed in a better way.
I don't know yet, but I will have a closer look. Let's also see if Adrian has some thoughts.
Kind regards
UffeQuoted 20 lines
---
drivers/mmc/core/block.c | 1 + 1 file changed, 1 insertion(+)drivers/mmc/core/block.c +1
-1@@ -2627,6 +2627,7 @@ static void mmc_blk_remove_req(struct mmc_blk_data *md)* from being accepted. */ card = md->queue.card; +blk_set_queue_dying (md->queue.queue);if (md->disk->flags &GENHD_FL_UP ) {device_remove_file (disk_to_dev (md->disk), &md->force_ro); if ((md->area_type & MMC_BLK_DATA_AREA_BOOT) && --Qualcomm India Private Limited, on behalf of Qualcomm Innovation Center, Inc., is a member of Code Aurora Forum, a Linux Foundation Collaborative Project
Quoted 39 lines · Ulf Hansson
Quoted 4 lines · Veerabhadrarao Badiganti
The commit
bbdc74dc19e0("mmc: block: Prevent new req entering queue after its cleanup") has introduced this change but it got moved afterdel_gendisk()with commit57678e5a3d51("mmc: block: Delete gendisk before cleaning up the request queue").This isn't the first time we have spotted errors in this path. Seems like a difficult path to get correct. :-)
Quoted 22 lines
It is blocking reboot with below Call
stack().INFO: task reboot:3086 blocked for more than 122 seconds. __schedule schedule schedule_timeout io_schedule_timeout do_wait_for_common wait_for_completion_io submit_bio_wait blkdev_issue_flush ext4_sync_fs __sync_filesystem sync_filesystem fsync_bdev invalidate_partition del_gendisk mmc_blk_remove_req mmc_blk_remove mmc_bus_remove device_release_driver_internal device_release_driverQuoted 8 lines
bus_remove_device device_del mmc_remove_card mmc_remove mmc_stop_host mmc_remove_host sdhci_remove_host sdhci_msm_remove
Quoted 3 lines
Why do you call
sdhci_msm_remove()from the shutdown callback? What specific operations do you need to run in the shutdown path for sdhci msm?
I was suggested to add shutdown callback by memory team to gracefully de-register with smmu driver during reboot/shutdown. So tried adding it. Since SMMU team.
I just need to ensure that controller is not active anymore and not accessing memory related to any requests and de-register with smmu driver during shutdown.
Quoted 48 lines
The important part should be to do a graceful shutdown of the card (and the block device) - is there anything else?
Or you are just using the shutdown callback as a simple way to trigger this problem? Could unbinding the driver trigger the same issue?
Quoted 8 lines
sdhci_msm_shutdown
platform_drv_shutdown
device_shutdown
kernel_restart_prepare
kernel_restartSo bringing this change back.
Quoted 7 lines
Signed-off-by Veerabhadrarao Badiganti <[email protected]>---
I'm observing this issue 100% of the time with shutdown callback added to sdhci-msm driver. I'm trying on 5.4 kernel with ChromeOS.
Please let me know if this can be fixed in a better way.
I don't know yet, but I will have a closer look. Let's also see if Adrian has some thoughts.
Kind regards
UffeQuoted 20 lines
---
drivers/mmc/core/block.c | 1 + 1 file changed, 1 insertion(+)drivers/mmc/core/block.c +1
-1@@ -2627,6 +2627,7 @@ static void mmc_blk_remove_req(struct mmc_blk_data *md)* from being accepted. */ card = md->queue.card; +blk_set_queue_dying (md->queue.queue);if (md->disk->flags &GENHD_FL_UP ) {device_remove_file (disk_to_dev (md->disk), &md->force_ro); if ((md->area_type & MMC_BLK_DATA_AREA_BOOT) && --Qualcomm India Private Limited, on behalf of Qualcomm Innovation Center, Inc., is a member of Code Aurora Forum, a Linux Foundation Collaborative Project