Skip to contentSkip to conversation

nvme: Add abrupt shutdown support

PATCH In review [email protected] 4 participants 9 messages Jul 5, 2021 – Jul 6, 2021

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 nvme: Add abrupt shutdown support

From: Shivamurthy Shastri <[email protected]>

Enabling the abrupt shutdown support. In this shutdown type host does not need to send Delete I/O Submission Queue and Delete I/O Completion queue commands to the device.

---
 drivers/nvme/host/core.c | 11 ++++++++++-
 drivers/nvme/host/nvme.h |  2 ++
 drivers/nvme/host/pci.c  |  5 +++--
 3 files changed, 15 insertions(+), 3 deletions(-)
drivers/nvme/host/core.c +10 -1
index 80c656dcbbac..c1d1bd6a03af 100644
@@ -40,6 +40,11 @@ module_param_named(io_timeout, nvme_io_timeout, uint, 0644);
 MODULE_PARM_DESC(io_timeout, "timeout in seconds for I/O");
 EXPORT_SYMBOL_GPL(nvme_io_timeout);
 
+bool shutdown_type;
+EXPORT_SYMBOL_GPL(shutdown_type);
+module_param(shutdown_type, bool, 0644);
+MODULE_PARM_DESC(shutdown_type, "Type of controller shutdown");
+
 static unsigned char shutdown_timeout = 5;
 module_param(shutdown_timeout, byte, 0644);
 MODULE_PARM_DESC(shutdown_timeout, "timeout in seconds for controller shutdown");
@@ -2166,7 +2171,11 @@ int nvme_shutdown_ctrl(struct nvme_ctrl *ctrl)
 	int ret;
 
 	ctrl->ctrl_config &= ~NVME_CC_SHN_MASK;
-	ctrl->ctrl_config |= NVME_CC_SHN_NORMAL;
+
+	if (shutdown_type)
+		ctrl->ctrl_config |= NVME_CC_SHN_ABRUPT;
+	else
+		ctrl->ctrl_config |= NVME_CC_SHN_NORMAL;
 
 	ret = ctrl->ops->reg_write32(ctrl, NVME_REG_CC, ctrl->ctrl_config);
 	if (ret)
drivers/nvme/host/nvme.h +2 -0
index 75420ceacc10..af14ee1cd7d8 100644
@@ -23,6 +23,8 @@
 extern unsigned int nvme_io_timeout;
 #define NVME_IO_TIMEOUT	(nvme_io_timeout * HZ)
 
+extern bool shutdown_type;
+
 extern unsigned int admin_timeout;
 #define NVME_ADMIN_TIMEOUT	(admin_timeout * HZ)
 
drivers/nvme/host/pci.c +3 -2
index d3c5086673bc..6a78233d9645 100644
@@ -2474,13 +2474,14 @@ static void nvme_dev_disable(struct nvme_dev *dev, bool shutdown)
 	 * Give the controller a chance to complete all entered requests if
 	 * doing a safe shutdown.
 	 */
-	if (!dead && shutdown && freeze)
+	if (!dead && shutdown && !shutdown_type && freeze)
 		nvme_wait_freeze_timeout(&dev->ctrl, NVME_IO_TIMEOUT);
 
 	nvme_stop_queues(&dev->ctrl);
 
 	if (!dead && dev->ctrl.queue_count > 0) {
-		nvme_disable_io_queues(dev);
+		if (!shutdown_type)
+			nvme_disable_io_queues(dev);
 		nvme_disable_admin_queue(dev, shutdown);
 	}
 	nvme_suspend_io_queues(dev);
Signature · 1 line
2.25.1
#1 lore raw
Quoted 5 lines · On Mon, Jul 05, 2021 at 10:11:57AM +0000, [email protected] wrote:

From: Shivamurthy Shastri <[email protected]>

Enabling the abrupt shutdown support. In this shutdown type host does not need to send Delete I/O Submission Queue and Delete I/O Completion queue commands to the device.

Why?

#2 lore raw

Micron Confidential

Hi Christoph,

Quoted 10 lines · On Mon, Jul 05, 2021 at 10:11:57AM +0000, [email protected] wrote:
Quoted 5 lines

From: Shivamurthy Shastri <[email protected]>

Enabling the abrupt shutdown support. In this shutdown type host does not need to send Delete I/O Submission Queue and Delete I/O Completion queue commands to the device.

Why?

This is to achieve faster shutdown at the cost of data in-flight.

As per the spec Deleting Submission queues and Completion queues are not explicitly mentioned for performing the abrupt shutdown (CC.SHN=10b).

Thanks,
Shiva

Micron Confidential

#3 lore raw
Quoted 9 lines · Shivamurthy Shastri (sshivamurthy)

Micron Confidential

Hi Christoph,

Quoted 5 lines · On Mon, Jul 05, 2021 at 10:11:57AM +0000, [email protected] wrote:
Quoted 2 lines

From: Shivamurthy Shastri <[email protected]>

Quoted 7 lines
Quoted 5 lines
Quoted 3 lines

Enabling the abrupt shutdown support. In this shutdown type host does not need to send Delete I/O Submission Queue and Delete I/O Completion queue commands to the device.

Why?

This is to achieve faster shutdown at the cost of data in-flight.

What is the actual use case, and why would a module paramter be a good interface?

#4 lore raw

Micron Confidential

Quoted 13 lines
Quoted 6 lines

Hi Christoph,

Quoted 3 lines

On Mon, Jul 05, 2021 at 10:11:57AM +0000, [email protected]

Quoted 6 lines · wrote:
Quoted 6 lines
Quoted 5 lines

From: Shivamurthy Shastri <[email protected]>

Enabling the abrupt shutdown support. In this shutdown type host does not need to send Delete I/O Submission Queue and Delete I/O Completion queue commands to the device.

Quoted 6 lines
Quoted 3 lines
Quoted 1 line

Why?

This is to achieve faster shutdown at the cost of data in-flight.

What is the actual use case, and why would a module paramter be a good interface?

For systems with passive electronic components to handle power-loss/battery-removal, CC.SHN=10b will help to achieve safer shutdown with relatively lower power back-up requirements (at lower cost).

The module parameter gives flexibility to the host to decide shutdown type even during boot failure.

Thanks,
Shiva

Micron Confidential

#5 lore raw
Quoted 6 lines · Shivamurthy Shastri (sshivamurthy)

For systems with passive electronic components to handle power-loss/battery-removal, CC.SHN=10b will help to achieve safer shutdown with relatively lower power back-up requirements (at lower cost).

The module parameter gives flexibility to the host to decide shutdown type even during boot failure.

If you have a device that always prefers an abrupt please introduce a TP in the NVMe technical working group so that the hosts automatically do the right thing. Requiring sysadmin intervention using a global kernel parameter is not the way to do this.

maintainer
#6 lore raw
Quoted 5 lines · Christoph Hellwig
Quoted 4 lines · Shivamurthy Shastri (sshivamurthy)

For systems with passive electronic components to handle power-loss/battery-removal, CC.SHN=10b will help to achieve safer shutdown with relatively lower power back-up requirements (at lower cost).

Quoted 7 lines
Quoted 2 lines

The module parameter gives flexibility to the host to decide shutdown type even during boot failure.

If you have a device that always prefers an abrupt please introduce a TP in the NVMe technical working group so that the hosts automatically do the right thing. Requiring sysadmin intervention using a global kernel parameter is not the way to do this.

Is it the device that wants an abrupt shutdown or the platform? If the platform's power is running on limited back-up supply, and could inform the kernel's power management subsystem of this, then a driver could use that to determine if the quick shutdown is appropriate.

#7 lore raw

Micron Confidential

Hi,

Quoted 19 lines · Christoph Hellwig
Quoted 1 line

On Mon, Jul 05, 2021 at 11:15:51AM +0000, Shivamurthy Shastri

Quoted 1 line · (sshivamurthy)
Quoted 1 line

For systems with passive electronic components to handle power-

loss/battery-removal,

Quoted 1 line
Quoted 1 line

CC.SHN=10b will help to achieve safer shutdown with relatively lower power

back-up

Quoted 3 lines
Quoted 3 lines

requirements (at lower cost).

The module parameter gives flexibility to the host to decide shutdown type

even during

Quoted 6 lines
Quoted 1 line

boot failure.

If you have a device that always prefers an abrupt please introduce a TP in the NVMe technical working group so that the hosts automatically do the right thing. Requiring sysadmin intervention using a global kernel parameter is not the way to do this.

The abrupt shutdown notification is already part of NVMe Specification.

Quoted 5 lines

Is it the device that wants an abrupt shutdown or the platform? If the platform's power is running on limited back-up supply, and could inform the kernel's power management subsystem of this, then a driver could use that to determine if the quick shutdown is appropriate.

Yes, the platform is running on limited back-up supply. In this case, abrupt shutdown option will help to achieve shutdown with relatively low back-up requirements.

Please let me know if you want to include the abrupt shutdown as part of

static const struct dev_pm_ops nvme_dev_pm_ops = {
	.suspend	= nvme_suspend,
               .....

Thanks,
Shiva

Micron Confidential

maintainer
#8 lore raw
Quoted 5 lines · Shivamurthy Shastri (sshivamurthy)
Quoted 4 lines

Is it the device that wants an abrupt shutdown or the platform? If the platform's power is running on limited back-up supply, and could inform the kernel's power management subsystem of this, then a driver could use that to determine if the quick shutdown is appropriate.

Quoted 8 lines

Yes, the platform is running on limited back-up supply. In this case, abrupt shutdown option will help to achieve shutdown with relatively low back-up requirements.

Please let me know if you want to include the abrupt shutdown as part of

static const struct dev_pm_ops nvme_dev_pm_ops = {
	.suspend	= nvme_suspend,
               .....

Are you suggesting creating a new pm callback just for this? I don't think that's necessary.

I was considering just adding a bit to the pm_suspend_global_flags, something like the patch below. You just have to fill in the platform specific parts to set the flag.

---

include/linux/suspend.h +24 -3
index 8af13ba60c7e..f43c0b2313d0 100644
@@ -210,9 +210,10 @@ extern int suspend_valid_only_mem(suspend_state_t state);
 
 extern unsigned int pm_suspend_global_flags;
 
-#define PM_SUSPEND_FLAG_FW_SUSPEND	BIT(0)
-#define PM_SUSPEND_FLAG_FW_RESUME	BIT(1)
-#define PM_SUSPEND_FLAG_NO_PLATFORM	BIT(2)
+#define PM_SUSPEND_FLAG_FW_SUSPEND		BIT(0)
+#define PM_SUSPEND_FLAG_FW_RESUME		BIT(1)
+#define PM_SUSPEND_FLAG_NO_PLATFORM		BIT(2)
+#define PM_SUSPEND_FLAG_POWER_LOSS_IMMINENT	BIT(3)
 
 static inline void pm_suspend_clear_flags(void)
 {
@@ -234,6 +235,11 @@ static inline void pm_set_suspend_no_platform(void)
 	pm_suspend_global_flags |= PM_SUSPEND_FLAG_NO_PLATFORM;
 }
 
+static inline void pm_set_power_loss_imminent(void)
+{
+	pm_suspend_global_flags |= PM_SUSPEND_FLAG_POWER_LOSS_IMMINENT;
+}
+
 /**
  * pm_suspend_via_firmware - Check if platform firmware will suspend the system.
  *
@@ -291,6 +297,21 @@ static inline bool pm_suspend_no_platform(void)
 	return !!(pm_suspend_global_flags & PM_SUSPEND_FLAG_NO_PLATFORM);
 }
 
+/**
+ * pm_power_loss_imminent - Check if platform is running on limited backup
+ * 			    power source
+ *
+ * To be called during system-wide power management transitions to sleep states.
+ *
+ * Return 'true' if power loss may be imminent due to platform running on
+ * limited backup supply. If set during a shutdown, drivers should use any
+ * available shortcuts to prepare their device for abrupt power loss.
+ */
+static inline bool pm_power_loss_imminent(void)
+{
+	return !!(pm_suspend_global_flags & PM_SUSPEND_FLAG_POWER_LOSS_IMMINENT);
+}
+
 /* Suspend-to-idle state machnine. */
 enum s2idle_states {
 	S2IDLE_STATE_NONE,      /* Not suspended/suspending. */

--