Skip to contentSkip to conversation

nvme: put some AMD PCIE downstream NVME device to simple suspend/resume path

PATCH Superseded Prike Liang 4 participants 8 messages Mar 31, 2021 – Apr 11, 2021 NVMe

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 nvme: put some AMD PCIE downstream NVME device to simple suspend/resume path

The NVME device pluged in some AMD PCIE root port will resume timeout from s2idle which caused by NVME power CFG lost in the SMU FW restore. This issue can be workaround by using PCIe power set with simple suspend/resume process path instead of APST. In the onwards ASIC will try do the NVME shutdown save and restore in the BIOS and still need PCIe power seting to resume from RTD3 for s2idle.

---
Changes in v2:
Fix the patch format and check chip root complex DID instead of PCIe RP
to avoid the storage device plugged in internal PCIe RP by USB adaptor.

---
 drivers/nvme/host/pci.c | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)
drivers/nvme/host/pci.c +19 -0
index 6bad4d4..1952da1 100644
@@ -42,6 +42,9 @@
 #define NVME_MAX_KB_SZ	4096
 #define NVME_MAX_SEGS	127
 
+#define AMD_PCIE_ID_CZN 0x1630
+#define AMD_PCIE_ID_RN AMD_PCIE_ID_CZN
+
 static int use_threaded_interrupts;
 module_param(use_threaded_interrupts, int, 0);
 
@@ -70,6 +73,12 @@ static unsigned int io_queue_depth = 1024;
 module_param_cb(io_queue_depth, &io_queue_depth_ops, &io_queue_depth, 0644);
 MODULE_PARM_DESC(io_queue_depth, "set io queue depth, should >= 2");
 
+static const struct pci_device_id pcie_ids[] = {
+	{ PCI_DEVICE(PCI_VENDOR_ID_AMD, AMD_PCIE_ID_RN) },
+	{ PCI_DEVICE(PCI_VENDOR_ID_AMD, AMD_PCIE_ID_CZN) },
+	{ }
+};
+
 static int io_queue_count_set(const char *val, const struct kernel_param *kp)
 {
 	unsigned int n;
@@ -2832,6 +2841,7 @@ static bool nvme_acpi_storage_d3(struct pci_dev *dev)
 {
 	struct acpi_device *adev;
 	struct pci_dev *root;
+	struct pci_dev *rdev;
 	acpi_handle handle;
 	acpi_status status;
 	u8 val;
@@ -2845,6 +2855,15 @@ static bool nvme_acpi_storage_d3(struct pci_dev *dev)
 	if (!root)
 		return false;
 
+	rdev = pci_get_domain_bus_and_slot(0, 0, PCI_DEVFN(0, 0));
+	if (rdev && pci_match_id(pcie_ids, rdev)) {
+	/* Detect NVME device which pluged in the Renoir PCIE root port
+	 * then force use quick suspend/resume path for handling the NVME
+	 * timeout during s2idle resume.
+	 */
+		return NVME_QUIRK_SIMPLE_SUSPEND;
+	}
+
 	adev = ACPI_COMPANION(&root->dev);
 	if (!adev)
 		return false;
Signature · 7 lines
2.7.4


_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme
#1 lore raw
Quoted 2 lines

try do the NVME shutdown save and restore in the BIOS and still need PCIe power seting to resume from RTD3 for s2idle.

s/seting/setting/ ? idk.

Quoted 24 lines

---
Changes in v2:
Fix the patch format and check chip root complex DID instead of PCIe RP
to avoid the storage device plugged in internal PCIe RP by USB adaptor.

---
 drivers/nvme/host/pci.c | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)
drivers/nvme/host/pci.c +3 -0
index 6bad4d4..1952da1 100644
@@ -42,6 +42,9 @@
 #define NVME_MAX_KB_SZ	4096
 #define NVME_MAX_SEGS	127
 
+#define AMD_PCIE_ID_CZN 0x1630
+#define AMD_PCIE_ID_RN AMD_PCIE_ID_CZN
+
 static int use_threaded_interrupts;
 module_param(use_threaded_interrupts, int, 0);
 

drivers/nvme/host/pci.c static unsigned int io_queue_depth = 1024; · line 81

Quoted 9 lines
drivers/nvme/host/pci.c +6 -0
@@ -70,6 +73,12 @@ static unsigned int io_queue_depth = 1024;
 module_param_cb(io_queue_depth, &io_queue_depth_ops, &io_queue_depth, 0644);
 MODULE_PARM_DESC(io_queue_depth, "set io queue depth, should >= 2");
 
+static const struct pci_device_id pcie_ids[] = {
+	{ PCI_DEVICE(PCI_VENDOR_ID_AMD, AMD_PCIE_ID_RN) },
+	{ PCI_DEVICE(PCI_VENDOR_ID_AMD, AMD_PCIE_ID_CZN) },
+	{ }
+};
+

why new global pci_device_id array is needed ?

Quoted 12 lines
drivers/nvme/host/pci.c +1 -0
 static int io_queue_count_set(const char *val, const struct kernel_param *kp)
 {
 	unsigned int n;
@@ -2832,6 +2841,7 @@ static bool nvme_acpi_storage_d3(struct pci_dev *dev)
 {
 	struct acpi_device *adev;
 	struct pci_dev *root;
+	struct pci_dev *rdev;
 	acpi_handle handle;
 	acpi_status status;
 	u8 val;

drivers/nvme/host/pci.c static bool nvme_acpi_storage_d3(struct pci_dev *dev) · line 2863

Quoted 9 lines
drivers/nvme/host/pci.c +6 -0
@@ -2845,6 +2855,15 @@ static bool nvme_acpi_storage_d3(struct pci_dev *dev)
 	if (!root)
 		return false;
 
+	rdev = pci_get_domain_bus_and_slot(0, 0, PCI_DEVFN(0, 0));
+	if (rdev && pci_match_id(pcie_ids, rdev)) {
+	/* Detect NVME device which pluged in the Renoir PCIE root port
+	 * then force use quick suspend/resume path for handling the NVME
+	 * timeout during s2idle resume.
+	 */

The comment should something like this :-

              if (rdev && pci_match_id(pcie_ids, rdev)) {

		/* 
		 * Detect NVMe device which pluged in the Renoir PCIe root port
		 * then force use quick suspend/resume path for handling the NVMe
		 * timeout during s2idle resume.
		 */return NVME_QUIRK_SIMPLE_SUSPEND;

          }

drivers/nvme/host/pci.c static bool nvme_acpi_storage_d3(struct pci_dev *dev) · line 2869

Quoted 6 lines
drivers/nvme/host/pci.c +3 -0
+		return NVME_QUIRK_SIMPLE_SUSPEND;
+	}
+
 	adev = ACPI_COMPANION(&root->dev);
 	if (!adev)
 		return false;

_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme

#2 lore raw

[AMD Official Use Only - Internal Distribution Only]

Quoted 8 lines

-----Original Message-----
From: Chaitanya Kulkarni <[email protected]>

Quoted 2 lines

try do the NVME shutdown save and restore in the BIOS and still need PCIe power seting to resume from RTD3 for s2idle.

s/seting/setting/ ? idk.

[Prike] It's should be a typo and will correct it.

Quoted 30 lines
Quoted 24 lines

---
Changes in v2:
Fix the patch format and check chip root complex DID instead of PCIe RP
to avoid the storage device plugged in internal PCIe RP by USB adaptor.

---
 drivers/nvme/host/pci.c | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)
drivers/nvme/host/pci.c index renamed from drivers/nvme/host/pci.c +0 -0

6bad4d4..1952da1 100644

drivers/nvme/host/pci.c +3 -0
@@ -42,6 +42,9 @@
 #define NVME_MAX_KB_SZ4096
 #define NVME_MAX_SEGS127
 
+#define AMD_PCIE_ID_CZN 0x1630
+#define AMD_PCIE_ID_RN AMD_PCIE_ID_CZN
+
 static int use_threaded_interrupts;
 module_param(use_threaded_interrupts, int, 0);
 

drivers/nvme/host/pci.c static unsigned int io_queue_depth = 1024; · line 73

Quoted 1 line
drivers/nvme/host/pci.c +0 -0
@@ -70,6 +73,12 @@ static unsigned int io_queue_depth = 1024;
 module_param_cb(io_queue_depth, &io_queue_depth_ops,

&io_queue_depth,

drivers/nvme/host/pci.c static unsigned int io_queue_depth = 1024; · line 74

Quoted 1 line
drivers/nvme/host/pci.c +0 -0
 0644);  MODULE_PARM_DESC(io_queue_depth, "set io queue depth,

should

Quoted 2 lines
drivers/nvme/host/pci.c +0 -0
 >= 2");
 
Quoted 9 lines

drivers/nvme/host/pci.c static unsigned int io_queue_depth = 1024; · line 81

Quoted 6 lines
drivers/nvme/host/pci.c +6 -0
+static const struct pci_device_id pcie_ids[] = {
+{ PCI_DEVICE(PCI_VENDOR_ID_AMD, AMD_PCIE_ID_RN) },
+{ PCI_DEVICE(PCI_VENDOR_ID_AMD, AMD_PCIE_ID_CZN) },
+{ }
+};
+

why new global pci_device_id array is needed ?

[Prike] This pci_device_id array contains the platform root complex DID which need shutdown the NVME in this armed platform for s2idle.

Quoted 16 lines

drivers/nvme/host/pci.c static bool nvme_acpi_storage_d3(struct

Quoted 12 lines
drivers/nvme/host/pci.c +0 -0
 static int io_queue_count_set(const char *val, const struct
 kernel_param *kp)  {

unsigned int n;

drivers/nvme/host/pci.c +1 -0
@@ -2832,6 +2841,7 @@ static bool nvme_acpi_storage_d3(struct pci_dev
 *dev)  {
 struct acpi_device *adev;
 struct pci_dev *root;
+struct pci_dev *rdev;
 acpi_handle handle;
 acpi_status status;
 u8 val;
@@ -2845,6 +2855,15 @@ static bool nvme_acpi_storage_d3(struct

pci_dev *dev)

Quoted 3 lines
drivers/nvme/host/pci.c +0 -0
 if (!root)
 return false;
 
Quoted 8 lines

drivers/nvme/host/pci.c static bool nvme_acpi_storage_d3(struct pci_dev *dev) · line 2863

Quoted 6 lines
drivers/nvme/host/pci.c +6 -0
+rdev = pci_get_domain_bus_and_slot(0, 0, PCI_DEVFN(0, 0));
+if (rdev && pci_match_id(pcie_ids, rdev)) {
+/* Detect NVME device which pluged in the Renoir PCIE root port
+ * then force use quick suspend/resume path for handling the NVME
+ * timeout during s2idle resume.
+ */

The comment should something like this :-

[Prike] Thanks point out and miss check this during this quick send.

Quoted 11 lines
              if (rdev && pci_match_id(pcie_ids, rdev)) {

/*
* Detect NVMe device which pluged in the Renoir PCIe root
port
* then force use quick suspend/resume path for handling
the NVMe
* timeout during s2idle resume.
*/return NVME_QUIRK_SIMPLE_SUSPEND;

Quoted 7 lines
          }

drivers/nvme/host/pci.c static bool nvme_acpi_storage_d3(struct pci_dev *dev) · line 2869

Quoted 6 lines
drivers/nvme/host/pci.c +3 -0
+return NVME_QUIRK_SIMPLE_SUSPEND;
+}
+
 adev = ACPI_COMPANION(&root->dev);
 if (!adev)
 return false;

_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme

maintainer
#3 lore raw
Quoted 1 line · Prike

[AMD Official Use Only - Internal Distribution Only]

This header is not appropriate for public list communication.

Quoted 10 lines
Quoted 9 lines

drivers/nvme/host/pci.c static unsigned int io_queue_depth = 1024; · line 81

Quoted 6 lines
drivers/nvme/host/pci.c +6 -0
+static const struct pci_device_id pcie_ids[] = {
+{ PCI_DEVICE(PCI_VENDOR_ID_AMD, AMD_PCIE_ID_RN) },
+{ PCI_DEVICE(PCI_VENDOR_ID_AMD, AMD_PCIE_ID_CZN) },
+{ }
+};
+

why new global pci_device_id array is needed ?

[Prike] This pci_device_id array contains the platform root complex DID which need shutdown the NVME in this armed platform for s2idle.

I think Chaitanya is just suggesting you move this declaration to the scope that uses it.

_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme

#4 lore raw
Quoted 3 lines · Keith Busch
Quoted 1 line · Prike

[AMD Official Use Only - Internal Distribution Only]

This header is not appropriate for public list communication.

Yes please remove this from next patches.

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

Review

Quoted 6 lines
(quoted diff) +6 -0
+static const struct pci_device_id pcie_ids[] = {
+{ PCI_DEVICE(PCI_VENDOR_ID_AMD, AMD_PCIE_ID_RN) },
+{ PCI_DEVICE(PCI_VENDOR_ID_AMD, AMD_PCIE_ID_CZN) },
+{ }
+};
+

why new global pci_device_id array is needed ?

Quoted 4 lines
Quoted 1 line

[Prike] This pci_device_id array contains the platform root complex DID which need shutdown the NVME in this armed platform for s2idle.

I think Chaitanya is just suggesting you move this declaration to the scope that uses it.

Yes, something like this on the top of your patch totally untested :-

drivers/nvme/host/pci.c +0 -3
index fa81cea41f48..8b2f619eabab 100644
@@ -42,9 +42,6 @@
 #define NVME_MAX_KB_SZ 4096
 #define NVME_MAX_SEGS  127
 
-#define AMD_PCIE_ID_CZN 0x1630
-#define AMD_PCIE_ID_RN AMD_PCIE_ID_CZN
-
 static int use_threaded_interrupts;
 module_param(use_threaded_interrupts, int, 0);
 
@@ -73,12 +70,6 @@ static unsigned int io_queue_depth = 1024;
 module_param_cb(io_queue_depth, &io_queue_depth_ops, &io_queue_depth,

0644);
MODULE_PARM_DESC(io_queue_depth, "set io queue depth, should >= 2");

-static const struct pci_device_id pcie_ids[] = {

-       { PCI_DEVICE(PCI_VENDOR_ID_AMD, AMD_PCIE_ID_RN) },
-       { PCI_DEVICE(PCI_VENDOR_ID_AMD, AMD_PCIE_ID_CZN) },
-       { }

-};
-
static int io_queue_count_set(const char *val, const struct
kernel_param *kp)

 {
        unsigned int n;

@@ -2835,6 +2826,10 @@ static unsigned long check_vendor_combination_bug(struct pci_dev *pdev)
}

#ifdef CONFIG_ACPI
+
+#define AMD_PCIE_ID_CZN 0x1630
+#define AMD_PCIE_ID_RN AMD_PCIE_ID_CZN
+
static bool nvme_acpi_storage_d3(struct pci_dev *dev)

 {
        struct acpi_device *adev;
@@ -2843,6 +2838,12 @@ static bool nvme_acpi_storage_d3(struct pci_dev *dev)
        acpi_handle handle;
        acpi_status status;
        u8 val;
+       const struct pci_device_id pcie_ids[] = {
+               { PCI_DEVICE(PCI_VENDOR_ID_AMD, AMD_PCIE_ID_RN) },
+               { PCI_DEVICE(PCI_VENDOR_ID_AMD, AMD_PCIE_ID_CZN) },
+               { }

+ };
+

/*
* Look for _DSD property specifying that the storage device on
the port

_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme

#5 lore raw

[AMD Public Use]

Quoted 3 lines
Quoted 1 line

This header is not appropriate for public list communication.

Yes please remove this from next patches.

[Prike] Yeah, the header should be replaced to public info.

Quoted 13 lines
Quoted 8 lines
Quoted 8 lines
Quoted 7 lines

Review

Quoted 5 lines
(quoted diff) +4 -0
+static const struct pci_device_id pcie_ids[] = { {
+PCI_DEVICE(PCI_VENDOR_ID_AMD, AMD_PCIE_ID_RN) }, {
+PCI_DEVICE(PCI_VENDOR_ID_AMD, AMD_PCIE_ID_CZN) }, { } };
+

why new global pci_device_id array is needed ?

[Prike] This pci_device_id array contains the platform root complex DID

which need shutdown the NVME in this armed platform for s2idle.

Quoted 3 lines

I think Chaitanya is just suggesting you move this declaration to the scope that uses it.

[Prike] Thanks proposal and that's sound good to move the reference PCI ID table into the caller scope directly.

Quoted 53 lines

Yes, something like this on the top of your patch totally untested :-

drivers/nvme/host/pci.c index renamed from drivers/nvme/host/pci.c +0 -0

fa81cea41f48..8b2f619eabab 100644

drivers/nvme/host/pci.c +0 -3
@@ -42,9 +42,6 @@
 #define NVME_MAX_KB_SZ 4096
 #define NVME_MAX_SEGS  127
 
-#define AMD_PCIE_ID_CZN 0x1630
-#define AMD_PCIE_ID_RN AMD_PCIE_ID_CZN
-
 static int use_threaded_interrupts;
 module_param(use_threaded_interrupts, int, 0);
 
@@ -73,12 +70,6 @@ static unsigned int io_queue_depth = 1024;
 module_param_cb(io_queue_depth, &io_queue_depth_ops,

&io_queue_depth, 0644); MODULE_PARM_DESC(io_queue_depth, "set io queue depth, should >= 2");

drivers/nvme/host/pci.c +0 -6
-static const struct pci_device_id pcie_ids[] = {
-       { PCI_DEVICE(PCI_VENDOR_ID_AMD, AMD_PCIE_ID_RN) },
-       { PCI_DEVICE(PCI_VENDOR_ID_AMD, AMD_PCIE_ID_CZN) },
-       { }
-};
-
 static int io_queue_count_set(const char *val, const struct kernel_param *kp)
{
        unsigned int n;
drivers/nvme/host/pci.c +10 -0
@@ -2835,6 +2826,10 @@ static unsigned long
 check_vendor_combination_bug(struct pci_dev *pdev)  }
 
 #ifdef CONFIG_ACPI
+
+#define AMD_PCIE_ID_CZN 0x1630
+#define AMD_PCIE_ID_RN AMD_PCIE_ID_CZN
+
 static bool nvme_acpi_storage_d3(struct pci_dev *dev)  {
        struct acpi_device *adev;
@@ -2843,6 +2838,12 @@ static bool nvme_acpi_storage_d3(struct pci_dev
 *dev)
        acpi_handle handle;
        acpi_status status;
        u8 val;
+       const struct pci_device_id pcie_ids[] = {
+               { PCI_DEVICE(PCI_VENDOR_ID_AMD, AMD_PCIE_ID_RN) },
+               { PCI_DEVICE(PCI_VENDOR_ID_AMD, AMD_PCIE_ID_CZN) },
+               { }
+       };
+
 
        /*
Quoted 1 line

* Look for _DSD property specifying that the storage device on the port

_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme

#6 lore raw
In reply to Prike Liang
Quoted 6 lines · Prike Liang

The NVME device pluged in some AMD PCIE root port will resume timeout from s2idle which caused by NVME power CFG lost in the SMU FW restore. This issue can be workaround by using PCIe power set with simple suspend/resume process path instead of APST. In the onwards ASIC will try do the NVME shutdown save and restore in the BIOS and still need PCIe power seting to resume from RTD3 for s2idle.

Looks like this is a generic PCIe-level quirk and should go into the core drivers/pci/ quirk table, with NVMe just looking at the quirk flag.

_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme

#7 lore raw

[AMD Public Use]

Quoted 11 lines · Prike Liang
Quoted 6 lines

The NVME device pluged in some AMD PCIE root port will resume timeout from s2idle which caused by NVME power CFG lost in the SMU FW restore. This issue can be workaround by using PCIe power set with simple suspend/resume process path instead of APST. In the onwards ASIC will try do the NVME shutdown save and restore in the BIOS and still need PCIe power seting to resume from RTD3 for s2idle.

Looks like this is a generic PCIe-level quirk and should go into the core drivers/pci/ quirk table, with NVMe just looking at the quirk flag.

[Prike] Thanks proposal, put the PCI identify to PCIe quirk out of NVMe seems more modularization but seems may not simplify the logic. Will update the v3 patch according.

_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme