Skip to contentSkip to conversation

drm/amdgpu: handle mp1 state properly on suspend or device removal

PATCH In review Evan Quan 2 participants 5 messages Dec 26, 2019 – Jan 2, 2020

Subsystems (2)
lore
Share & tools
lore

Evan Quan

author
#0 lore raw

PATCH 1/2 drm/amdgpu: handle mp1 state properly on suspend or device removal

Set mp1 state properly for non gpu reset cases.

Change-Id I2a007910da6b5e2d1f8915d17827621ebd2e8c59
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_device.c |  3 +++
 drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c    | 16 ++++++++++++++--
 drivers/gpu/drm/amd/powerplay/amdgpu_smu.c | 13 +++++++++++++
 3 files changed, 30 insertions(+), 2 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +3 -0
index d36245321cb0..739ff4e4e845 100644
@@ -1089,6 +1089,7 @@ static int amdgpu_device_check_arguments(struct amdgpu_device *adev)
 static void amdgpu_switcheroo_set_state(struct pci_dev *pdev, enum vga_switcheroo_state state)
 {
 	struct drm_device *dev = pci_get_drvdata(pdev);
+	struct amdgpu_device *adev = dev->dev_private;
 	int r;
 
 	if (amdgpu_device_supports_boco(dev) && state == VGA_SWITCHEROO_OFF)
@@ -1112,7 +1113,9 @@ static void amdgpu_switcheroo_set_state(struct pci_dev *pdev, enum vga_switchero
 		pr_info("amdgpu: switched off\n");
 		drm_kms_helper_poll_disable(dev);
 		dev->switch_power_state = DRM_SWITCH_POWER_CHANGING;
+		adev->mp1_state = PP_MP1_STATE_SHUTDOWN;
 		amdgpu_device_suspend(dev, true);
+		adev->mp1_state = PP_MP1_STATE_NONE;
 		pci_save_state(dev->pdev);
 		/* Shut down the device */
 		pci_disable_device(dev->pdev);
drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c +14 -2
index 61dc26515c7e..13278f1c1b14 100644
@@ -1157,8 +1157,14 @@ amdgpu_pci_shutdown(struct pci_dev *pdev)
 static int amdgpu_pmops_suspend(struct device *dev)
 {
 	struct drm_device *drm_dev = dev_get_drvdata(dev);
+	struct amdgpu_device *adev = drm_dev->dev_private;
+	int r;
+
+	adev->mp1_state = PP_MP1_STATE_SHUTDOWN;
+	r = amdgpu_device_suspend(drm_dev, true);
+	adev->mp1_state = PP_MP1_STATE_NONE;
 
-	return amdgpu_device_suspend(drm_dev, true);
+	return r;
 }
 
 static int amdgpu_pmops_resume(struct device *dev)
@@ -1198,8 +1204,14 @@ static int amdgpu_pmops_thaw(struct device *dev)
 static int amdgpu_pmops_poweroff(struct device *dev)
 {
 	struct drm_device *drm_dev = dev_get_drvdata(dev);
+	struct amdgpu_device *adev = drm_dev->dev_private;
+	int r;
+
+	adev->mp1_state = PP_MP1_STATE_SHUTDOWN;
+	r = amdgpu_device_suspend(drm_dev, true);
+	adev->mp1_state = PP_MP1_STATE_NONE;
 
-	return amdgpu_device_suspend(drm_dev, true);
+	return r;
 }
 
 static int amdgpu_pmops_restore(struct device *dev)
drivers/gpu/drm/amd/powerplay/amdgpu_smu.c +13 -0
index d07c4f2ccee7..4a7fb6b15635 100644
@@ -1370,6 +1370,18 @@ int smu_reset(struct smu_context *smu)
 	return ret;
 }
 
+void smu_late_fini(void *handle)
+{
+	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+	struct smu_context *smu = &adev->smu;
+
+	/*
+	 * Put the mp1 in the right state.
+	 * This gets called on driver unloading.
+	 */
+	smu_send_smc_msg(smu, SMU_MSG_PrepareMp1ForShutdown);
+}
+
 static int smu_suspend(void *handle)
 {
 	int ret;
@@ -1931,6 +1943,7 @@ const struct amd_ip_funcs smu_ip_funcs = {
 	.sw_fini = smu_sw_fini,
 	.hw_init = smu_hw_init,
 	.hw_fini = smu_hw_fini,
+	.late_fini = smu_late_fini,
 	.suspend = smu_suspend,
 	.resume = smu_resume,
 	.is_idle = NULL,
Signature · 6 lines
2.24.0

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
author
#1 lore raw

PATCH 2/2 drm/amdgpu: avoid unnecessary SMC firmware reloading

E.g. During non-RAS baco reset, the SMC is still alive. Thus there is no need to reload the SMC firmware.

Change-Id I73f6284541d0ca0e82761380a27e32484fb0061c
---
 drivers/gpu/drm/amd/amdgpu/psp_v11_0.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)
drivers/gpu/drm/amd/amdgpu/psp_v11_0.c +14 -0
index c66ca8cc2ebd..ba761e9366e3 100644
@@ -676,6 +676,19 @@ static bool psp_v11_0_compare_sram_data(struct psp_context *psp,
 	return true;
 }
 
+/*
+ * Check whether SMU is still alive. If that's true
+ * (e.g. for non-RAS baco reset), we need to skip SMC firmware reloading.
+ */
+static bool psp_v11_0_smu_reload_quirk(struct psp_context *psp)
+{
+	struct amdgpu_device *adev = psp->adev;
+	uint32_t reg;
+
+	reg = RREG32_PCIE(smnMP1_FIRMWARE_FLAGS | 0x03b00000);
+	return (reg & MP1_FIRMWARE_FLAGS__INTERRUPTS_ENABLED_MASK) ? true : false;
+}
+
 static int psp_v11_0_mode1_reset(struct psp_context *psp)
 {
 	int ret;
@@ -1070,6 +1083,7 @@ static const struct psp_funcs psp_v11_0_funcs = {
 	.ring_stop = psp_v11_0_ring_stop,
 	.ring_destroy = psp_v11_0_ring_destroy,
 	.compare_sram_data = psp_v11_0_compare_sram_data,
+	.smu_reload_quirk = psp_v11_0_smu_reload_quirk,
 	.mode1_reset = psp_v11_0_mode1_reset,
 	.xgmi_get_topology_info = psp_v11_0_xgmi_get_topology_info,
 	.xgmi_set_topology_info = psp_v11_0_xgmi_set_topology_info,
Signature · 6 lines
2.24.0

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
maintainer Acked-by on v1 2/2
#2 lore raw

RE: [PATCH 2/2] drm/amdgpu: avoid unnecessary SMC firmware reloading

[AMD Public Use]

Quoted 9 lines

-----Original Message-----
From: Quan, Evan <[email protected]>
Sent: Thursday, December 26, 2019 3:11 AM
To: [email protected]
Cc: Deucher, Alexander <[email protected]>; Quan, Evan <[email protected]>
Subject: [PATCH 2/2] drm/amdgpu: avoid unnecessary SMC firmware reloading

Quoted 5 lines

E.g. During non-RAS baco reset, the SMC is still alive. Thus there is no need to reload the SMC firmware.

Change-Id I73f6284541d0ca0e82761380a27e32484fb0061c
Quoted 38 lines
---
 drivers/gpu/drm/amd/amdgpu/psp_v11_0.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)
drivers/gpu/drm/amd/amdgpu/psp_v11_0.c +10 -0
index c66ca8cc2ebd..ba761e9366e3 100644
@@ -676,6 +676,19 @@ static bool psp_v11_0_compare_sram_data(struct
 psp_context *psp,
 	return true;
 }
 
+/*
+ * Check whether SMU is still alive. If that's true
+ * (e.g. for non-RAS baco reset), we need to skip SMC firmware reloading.
+ */
+static bool psp_v11_0_smu_reload_quirk(struct psp_context *psp) {
+	struct amdgpu_device *adev = psp->adev;
+	uint32_t reg;
+
+	reg = RREG32_PCIE(smnMP1_FIRMWARE_FLAGS | 0x03b00000);
+	return (reg &

MP1_FIRMWARE_FLAGS__INTERRUPTS_ENABLED_MASK) ? true :

drivers/gpu/drm/amd/amdgpu/psp_v11_0.c +3 -1
+false; }
+
 static int psp_v11_0_mode1_reset(struct psp_context *psp)  {
 	int ret;
@@ -1070,6 +1083,7 @@ static const struct psp_funcs psp_v11_0_funcs = {
 	.ring_stop = psp_v11_0_ring_stop,
 	.ring_destroy = psp_v11_0_ring_destroy,
 	.compare_sram_data = psp_v11_0_compare_sram_data,
+	.smu_reload_quirk = psp_v11_0_smu_reload_quirk,
 	.mode1_reset = psp_v11_0_mode1_reset,
 	.xgmi_get_topology_info = psp_v11_0_xgmi_get_topology_info,
 	.xgmi_set_topology_info = psp_v11_0_xgmi_set_topology_info,
--
Quoted 1 line

2.24.0

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

#3 lore raw
In reply to Evan Quan

[AMD Public Use]

Quoted 66 lines

-----Original Message-----
From: Quan, Evan <[email protected]>
Sent: Thursday, December 26, 2019 3:11 AM
To: [email protected]
Cc: Deucher, Alexander <[email protected]>; Quan, Evan <[email protected]>
Subject: [PATCH 1/2] drm/amdgpu: handle mp1 state properly on suspend or device removal

Set mp1 state properly for non gpu reset cases.

Change-Id I2a007910da6b5e2d1f8915d17827621ebd2e8c59
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_device.c |  3 +++
 drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c    | 16 ++++++++++++++--
 drivers/gpu/drm/amd/powerplay/amdgpu_smu.c | 13 +++++++++++++
 3 files changed, 30 insertions(+), 2 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +0 -0
index d36245321cb0..739ff4e4e845 100644
@@ -1089,6 +1089,7 @@ static int amdgpu_device_check_arguments(struct

amdgpu_device *adev) static void amdgpu_switcheroo_set_state(struct pci_dev *pdev, enum vga_switcheroo_state state) {

drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +1 -0
 	struct drm_device *dev = pci_get_drvdata(pdev);
+	struct amdgpu_device *adev = dev->dev_private;
 	int r;
 
 	if (amdgpu_device_supports_boco(dev) && state ==

VGA_SWITCHEROO_OFF) @@ -1112,7 +1113,9 @@ static void amdgpu_switcheroo_set_state(struct pci_dev *pdev, enum vga_switchero

drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +0 -0
 		pr_info("amdgpu: switched off\n");
 		drm_kms_helper_poll_disable(dev);
 		dev->switch_power_state =

DRM_SWITCH_POWER_CHANGING;

drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +2 -0
+		adev->mp1_state = PP_MP1_STATE_SHUTDOWN;
 		amdgpu_device_suspend(dev, true);
+		adev->mp1_state = PP_MP1_STATE_NONE;
 		pci_save_state(dev->pdev);
 		/* Shut down the device */
 		pci_disable_device(dev->pdev);
drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c +7 -1
index 61dc26515c7e..13278f1c1b14 100644
@@ -1157,8 +1157,14 @@ amdgpu_pci_shutdown(struct pci_dev *pdev)
 static int amdgpu_pmops_suspend(struct device *dev)  {
 	struct drm_device *drm_dev = dev_get_drvdata(dev);
+	struct amdgpu_device *adev = drm_dev->dev_private;
+	int r;
+
+	adev->mp1_state = PP_MP1_STATE_SHUTDOWN;
+	r = amdgpu_device_suspend(drm_dev, true);
+	adev->mp1_state = PP_MP1_STATE_NONE;
 
-	return amdgpu_device_suspend(drm_dev, true);
+	return r;
 }
 
 static int amdgpu_pmops_resume(struct device *dev) @@ -1198,8 +1204,14

@@ static int amdgpu_pmops_thaw(struct device *dev) static int amdgpu_pmops_poweroff(struct device *dev) {

drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c amdgpu_pci_shutdown(struct pci_dev *pdev) · line 1163

Quoted 5 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c +4 -0
 	struct drm_device *drm_dev = dev_get_drvdata(dev);
+	struct amdgpu_device *adev = drm_dev->dev_private;
+	int r;
+
+	adev->mp1_state = PP_MP1_STATE_SHUTDOWN;

Should this be STATE_UNLOAD for poweroff?

Quoted 18 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c +3 -1
+	r = amdgpu_device_suspend(drm_dev, true);
+	adev->mp1_state = PP_MP1_STATE_NONE;
 
-	return amdgpu_device_suspend(drm_dev, true);
+	return r;
 }
 
 static int amdgpu_pmops_restore(struct device *dev) diff --git

a/drivers/gpu/drm/amd/powerplay/amdgpu_smu.c
b/drivers/gpu/drm/amd/powerplay/amdgpu_smu.c
index d07c4f2ccee7..4a7fb6b15635 100644

drivers/gpu/drm/amd/powerplay/amdgpu_smu.c +1 -0
@@ -1370,6 +1370,18 @@ int smu_reset(struct smu_context *smu)
 	return ret;
 }
 
+void smu_late_fini(void *handle)

drivers/gpu/drm/amd/powerplay/amdgpu_smu.c int smu_reset(struct smu_context *smu) · line 1383

Quoted 10 lines
drivers/gpu/drm/amd/powerplay/amdgpu_smu.c +10 -0
+{
+	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+	struct smu_context *smu = &adev->smu;
+
+	/*
+	 * Put the mp1 in the right state.
+	 * This gets called on driver unloading.
+	 */
+	smu_send_smc_msg(smu, SMU_MSG_PrepareMp1ForShutdown); }
+

Do we need something similar for the powerplay code? E.g., pp_late_fini()? Also shouldn't this be adev->mp1_state or prepare for unload rather than hardcoding it to prepare for shutdown?

Alex

Quoted 12 lines
drivers/gpu/drm/amd/powerplay/amdgpu_smu.c +1 -1
 static int smu_suspend(void *handle)
 {
 	int ret;
@@ -1931,6 +1943,7 @@ const struct amd_ip_funcs smu_ip_funcs = {
 	.sw_fini = smu_sw_fini,
 	.hw_init = smu_hw_init,
 	.hw_fini = smu_hw_fini,
+	.late_fini = smu_late_fini,
 	.suspend = smu_suspend,
 	.resume = smu_resume,
 	.is_idle = NULL,
--
Quoted 1 line

2.24.0

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

author
#4 lore raw
Quoted 74 lines

-----Original Message-----
From: Deucher, Alexander <[email protected]>
Sent: Sunday, December 29, 2019 2:43 PM
To: Quan, Evan <[email protected]>; [email protected]
Subject: RE: [PATCH 1/2] drm/amdgpu: handle mp1 state properly on suspend or device removal

[AMD Public Use]

Quoted 63 lines

-----Original Message-----
From: Quan, Evan <[email protected]>
Sent: Thursday, December 26, 2019 3:11 AM
To: [email protected]
Cc: Deucher, Alexander <[email protected]>; Quan, Evan <[email protected]>
Subject: [PATCH 1/2] drm/amdgpu: handle mp1 state properly on suspend or device removal

Set mp1 state properly for non gpu reset cases.

Change-Id I2a007910da6b5e2d1f8915d17827621ebd2e8c59
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_device.c |  3 +++
 drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c    | 16 ++++++++++++++--
 drivers/gpu/drm/amd/powerplay/amdgpu_smu.c | 13 +++++++++++++
 3 files changed, 30 insertions(+), 2 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +0 -0
index d36245321cb0..739ff4e4e845 100644
@@ -1089,6 +1089,7 @@ static int amdgpu_device_check_arguments(struct

amdgpu_device *adev) static void amdgpu_switcheroo_set_state(struct pci_dev *pdev, enum vga_switcheroo_state state) {

drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +1 -0
 	struct drm_device *dev = pci_get_drvdata(pdev);
+	struct amdgpu_device *adev = dev->dev_private;
 	int r;
 
 	if (amdgpu_device_supports_boco(dev) && state ==

VGA_SWITCHEROO_OFF) @@ -1112,7 +1113,9 @@ static void amdgpu_switcheroo_set_state(struct pci_dev *pdev, enum vga_switchero

drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +0 -0
 		pr_info("amdgpu: switched off\n");
 		drm_kms_helper_poll_disable(dev);
 		dev->switch_power_state =

DRM_SWITCH_POWER_CHANGING;

drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +2 -0
+		adev->mp1_state = PP_MP1_STATE_SHUTDOWN;
 		amdgpu_device_suspend(dev, true);
+		adev->mp1_state = PP_MP1_STATE_NONE;
 		pci_save_state(dev->pdev);
 		/* Shut down the device */
 		pci_disable_device(dev->pdev);
drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c +7 -1
index 61dc26515c7e..13278f1c1b14 100644
@@ -1157,8 +1157,14 @@ amdgpu_pci_shutdown(struct pci_dev *pdev)
 static int amdgpu_pmops_suspend(struct device *dev)  {
 	struct drm_device *drm_dev = dev_get_drvdata(dev);
+	struct amdgpu_device *adev = drm_dev->dev_private;
+	int r;
+
+	adev->mp1_state = PP_MP1_STATE_SHUTDOWN;
+	r = amdgpu_device_suspend(drm_dev, true);
+	adev->mp1_state = PP_MP1_STATE_NONE;
 
-	return amdgpu_device_suspend(drm_dev, true);
+	return r;
 }
 
Quoted 10 lines

drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c amdgpu_pci_shutdown(struct pci_dev *pdev) · line 1163

Quoted 8 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c +0 -0
 static int amdgpu_pmops_resume(struct device *dev) @@ -1198,8 +1204,14

@@ static int amdgpu_pmops_thaw(struct device *dev) static int amdgpu_pmops_poweroff(struct device *dev) {

drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c +4 -0
 	struct drm_device *drm_dev = dev_get_drvdata(dev);
+	struct amdgpu_device *adev = drm_dev->dev_private;
+	int r;
+
+	adev->mp1_state = PP_MP1_STATE_SHUTDOWN;

Should this be STATE_UNLOAD for poweroff?

[Quan, Evan] "PrepareMp1ForUnload is for PnP cases, where ASIC is not reset. If the whole ASIC is reset or need to shutdown, then we can simply use PrepareMp1ForShutdown." - a quote from a SMC firmware developer. According to that, I think PrepareMp1ForShutdown is more proper. How do you think?

Quoted 18 lines
Quoted 18 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c +3 -1
+	r = amdgpu_device_suspend(drm_dev, true);
+	adev->mp1_state = PP_MP1_STATE_NONE;
 
-	return amdgpu_device_suspend(drm_dev, true);
+	return r;
 }
 
 static int amdgpu_pmops_restore(struct device *dev) diff --git

a/drivers/gpu/drm/amd/powerplay/amdgpu_smu.c
b/drivers/gpu/drm/amd/powerplay/amdgpu_smu.c
index d07c4f2ccee7..4a7fb6b15635 100644

drivers/gpu/drm/amd/powerplay/amdgpu_smu.c +0 -0
@@ -1370,6 +1370,18 @@ int smu_reset(struct smu_context *smu)
 	return ret;
 }
 
Quoted 15 lines

drivers/gpu/drm/amd/powerplay/amdgpu_smu.c int smu_reset(struct smu_context *smu) · line 1384

Quoted 11 lines
drivers/gpu/drm/amd/powerplay/amdgpu_smu.c +11 -0
+void smu_late_fini(void *handle)
+{
+	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+	struct smu_context *smu = &adev->smu;
+
+	/*
+	 * Put the mp1 in the right state.
+	 * This gets called on driver unloading.
+	 */
+	smu_send_smc_msg(smu, SMU_MSG_PrepareMp1ForShutdown); }
+

Do we need something similar for the powerplay code? E.g., pp_late_fini()? Also shouldn't this be adev->mp1_state or prepare for unload rather than hardcoding it to prepare for shutdown?

[Quan, Evan] This is specifically for the amdgpu_driver_unload_kms() use case.
For that, the call path is amdgpu_driver_unload_kms() -> amdgpu_device_fini() -> amdgpu_device_ip_fini() -> hw_fini(). And the suspend routines are not on the call path. So, the adev->mp1_state way is not workable.
Yes, I think powerplay code needs this also.
Maybe we can use prepareMp1forunload. Considering amdgpu_driver_unload_kms() may be called on runtime driver/module unloading and the ASIC needs no power down or reset for that.

Quoted 16 lines

Alex

Quoted 12 lines
drivers/gpu/drm/amd/powerplay/amdgpu_smu.c +1 -1
 static int smu_suspend(void *handle)
 {
 	int ret;
@@ -1931,6 +1943,7 @@ const struct amd_ip_funcs smu_ip_funcs = {
 	.sw_fini = smu_sw_fini,
 	.hw_init = smu_hw_init,
 	.hw_fini = smu_hw_fini,
+	.late_fini = smu_late_fini,
 	.suspend = smu_suspend,
 	.resume = smu_resume,
 	.is_idle = NULL,
--
Quoted 1 line

2.24.0

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx