Skip to contentSkip to conversation

pc10 entry fixes for skl.

PATCH Applied Animesh Manna 8 participants 51 messages Aug 25, 2015 – Oct 19, 2015

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover pc10 entry fixes for skl.

The following patches helps to solve PC10 entry issue for SKL. Detailed description about the changes done to solve the issue is mentioned in commit message of each patch.

v1: http://lists.freedesktop.org/archives/intel-gfx/2015-August/072870.html

v2: Based on review comments from Daniel, changes made in the current version.

DMC redesign patch series has dependencies with current patch series. Need to rework on few patches, planning to send after initial review feedback of the current patch series.
http://lists.freedesktop.org/archives/intel-gfx/2015-August/072921.html

Animesh Manna (5):
drm/i915/skl: Added a check for the hardware status of csr fw before
loading.
drm/i915/skl Remove the call for csr uninitialization from suspend
path
drm/i915/skl: Making DC6 entry is the last call in suspend flow. drm/i915/skl: Do not disable cdclk PLL if csr firmware is present drm/i915/skl: Block disable call for pw1 if dmc firmware is present.

 drivers/gpu/drm/i915/i915_drv.c         | 19 +++++++++++++------
 drivers/gpu/drm/i915/intel_csr.c        |  9 +++++++++
 drivers/gpu/drm/i915/intel_display.c    | 14 ++++++++++----
 drivers/gpu/drm/i915/intel_drv.h        |  2 ++
 drivers/gpu/drm/i915/intel_runtime_pm.c | 31 ++++++++++++++++---------------
 5 files changed, 50 insertions(+), 25 deletions(-)
Signature · 6 lines
2.0.2

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#1 lore raw

PATCH 1/5 drm/i915/skl: Added a check for the hardware status of csr fw before loading.

Dmc will restore the csr program except DC9, cold boot, warm reset, PCI function level reset, and hibernate/suspend.

intel_csr_load_program() function is used to load the firmware data from kernel memory to csr address space.

All values of csr address space will be zero if it got reset and the first byte of csr program is always a non-zero if firmware is loaded successfuly. Based on hardware status will load the firmware.

Without this condition check if we overwrite the firmware data the counters exposed for dc5/dc6 (help for debugging) will be nullified.

v1: Initial version.

v2: Based on review comments from Daniel,
- Added a check to know hardware status and load the firmware if not loaded.

Cc Daniel Vetter <[email protected]>
Cc Damien Lespiau <[email protected]>
Cc Imre Deak <[email protected]>
Cc Sunil Kamath <[email protected]>
---
 drivers/gpu/drm/i915/intel_csr.c | 9 +++++++++
 1 file changed, 9 insertions(+)
drivers/gpu/drm/i915/intel_csr.c +9 -0
index ba1ae03..682cc26 100644
@@ -252,6 +252,15 @@ void intel_csr_load_program(struct drm_device *dev)
 		return;
 	}
 
+	/*
+	 * Dmc will restore the csr the program except DC9, cold boot,
+	 * warm reset, PCI function level reset, and hibernate/suspend.
+	 * This condition will help to check if csr address space is reset/
+	 * not loaded.
+	 */
+	if (I915_READ(CSR_PROGRAM_BASE))
+		return;
+
 	mutex_lock(&dev_priv->csr_lock);
 	fw_size = dev_priv->csr.dmc_fw_size;
 	for (i = 0; i < fw_size; i++)
Signature · 6 lines
2.0.2

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#2 lore raw

Re: [DMC_BUGFIX_SKL_V2 1/5] drm/i915/skl: Added a check for the hardware status of csr fw before loading.

Quoted 34 lines · Animesh Manna

Dmc will restore the csr program except DC9, cold boot, warm reset, PCI function level reset, and hibernate/suspend.

intel_csr_load_program() function is used to load the firmware data from kernel memory to csr address space.

All values of csr address space will be zero if it got reset and the first byte of csr program is always a non-zero if firmware is loaded successfuly. Based on hardware status will load the firmware.

Without this condition check if we overwrite the firmware data the counters exposed for dc5/dc6 (help for debugging) will be nullified.

v1: Initial version.

v2: Based on review comments from Daniel,
- Added a check to know hardware status and load the firmware if not loaded.

Cc Daniel Vetter <[email protected]>
Cc Damien Lespiau <[email protected]>
Cc Imre Deak <[email protected]>
Cc Sunil Kamath <[email protected]>
---
 drivers/gpu/drm/i915/intel_csr.c | 9 +++++++++
 1 file changed, 9 insertions(+)
drivers/gpu/drm/i915/intel_csr.c +0 -0
index ba1ae03..682cc26 100644

drivers/gpu/drm/i915/intel_csr.c void intel_csr_load_program(struct drm_device *dev) · line 260

Quoted 9 lines
drivers/gpu/drm/i915/intel_csr.c +6 -0
@@ -252,6 +252,15 @@ void intel_csr_load_program(struct drm_device *dev)
 		return;
 	}
 
+	/*
+	 * Dmc will restore the csr the program except DC9, cold boot,
+	 * warm reset, PCI function level reset, and hibernate/suspend.
+	 * This condition will help to check if csr address space is reset/
+	 * not loaded.
+	 */

Atm we call this from driver load and resume, which doesn seem to cover all the cases you mention in the comment. Should this be a WARN_ON instead? Or do we have troubles in our init sequence where we load too many times?

Either way I can't reconcile your commit message with the comment here. -Daniel

Quoted 13 lines
drivers/gpu/drm/i915/intel_csr.c +3 -0
+	if (I915_READ(CSR_PROGRAM_BASE))
+		return;
+
 	mutex_lock(&dev_priv->csr_lock);
 	fw_size = dev_priv->csr.dmc_fw_size;
 	for (i = 0; i < fw_size; i++)
Signature · 6 lines
2.0.2

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#3 lore raw

Re: [DMC_BUGFIX_SKL_V2 1/5] drm/i915/skl: Added a check for the hardware status of csr fw before loading.

Quoted 7 lines · Daniel Vetter
Quoted 6 lines · Animesh Manna

Dmc will restore the csr program except DC9, cold boot, warm reset, PCI function level reset, and hibernate/suspend.

intel_csr_load_program() function is used to load the firmware data from kernel memory to csr address space.

Quoted 7 lines
Quoted 7 lines

All values of csr address space will be zero if it got reset and the first byte of csr program is always a non-zero if firmware is loaded successfuly. Based on hardware status will load the firmware.

Without this condition check if we overwrite the firmware data the counters exposed for dc5/dc6 (help for debugging) will be nullified.

Bacause of the above reason mentioned just above we need to block firmware loading again. So only WARN_ON will not help.

Quoted 24 lines
Quoted 24 lines

v1: Initial version.

v2: Based on review comments from Daniel,
- Added a check to know hardware status and load the firmware if not loaded.

Cc Daniel Vetter <[email protected]>
Cc Damien Lespiau <[email protected]>
Cc Imre Deak <[email protected]>
Cc Sunil Kamath <[email protected]>
---
  drivers/gpu/drm/i915/intel_csr.c | 9 +++++++++
  1 file changed, 9 insertions(+)
drivers/gpu/drm/i915/intel_csr.c +0 -0
index ba1ae03..682cc26 100644
@@ -252,6 +252,15 @@ void intel_csr_load_program(struct drm_device *dev)
  		return;
  	}
 
Quoted 10 lines

drivers/gpu/drm/i915/intel_csr.c void intel_csr_load_program(struct drm_device *dev) · line 260

Quoted 6 lines
drivers/gpu/drm/i915/intel_csr.c +6 -0
+	/*
+	 * Dmc will restore the csr the program except DC9, cold boot,
+	 * warm reset, PCI function level reset, and hibernate/suspend.
+	 * This condition will help to check if csr address space is reset/
+	 * not loaded.
+	 */

Atm we call this from driver load and resume, which doesn seem to cover all the cases you mention in the comment. Should this be a WARN_ON instead? Or do we have troubles in our init sequence where we load too many times?

Yes, the above statement taken from bspec to describe about the special cases dmc will not restore the firmware. Agree, In our cases cold boot and hibernate/suspend mainly we need to load the firmware again, so in my second sentence I wanted to comment mainly regarding this condition check added for suspend-hibernate(reset) and cold boot(not loaded).

Anyways the same api later can be used to load the firmware from anywhere, so my intention to check firmware loaded or not. If already loaded then not to overwrite the csr address space to maintain the dc5/dc6 counter value.

Can the below comment more clear to you.

/*
* Dmc will restore the csr the program except DC9, cold boot,
* warm reset, PCI function level reset, and hibernate/suspend.
* If firmware is restored by dmc then no need to load again which
* will keep the dc5/dc6 counter exposed by firmware. */

No issue in init sequence.

-Animesh

Quoted 13 lines

Either way I can't reconcile your commit message with the comment here. -Daniel

Quoted 9 lines
drivers/gpu/drm/i915/intel_csr.c +3 -0
+	if (I915_READ(CSR_PROGRAM_BASE))
+		return;
+
  	mutex_lock(&dev_priv->csr_lock);
  	fw_size = dev_priv->csr.dmc_fw_size;
  	for (i = 0; i < fw_size; i++)
Signature · 1 line
2.0.2
Quoted 4 lines
Quoted 4 lines

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

#4 lore raw

Re: [DMC_BUGFIX_SKL_V2 1/5] drm/i915/skl: Added a check for the hardware status of csr fw before loading.

Quoted 66 lines · Animesh Manna
Quoted 7 lines · Daniel Vetter
Quoted 6 lines · Animesh Manna

Dmc will restore the csr program except DC9, cold boot, warm reset, PCI function level reset, and hibernate/suspend.

intel_csr_load_program() function is used to load the firmware data from kernel memory to csr address space.

Quoted 7 lines
Quoted 7 lines

All values of csr address space will be zero if it got reset and the first byte of csr program is always a non-zero if firmware is loaded successfuly. Based on hardware status will load the firmware.

Without this condition check if we overwrite the firmware data the counters exposed for dc5/dc6 (help for debugging) will be nullified.

Bacause of the above reason mentioned just above we need to block firmware loading again. So only WARN_ON will not help.

Quoted 25 lines
Quoted 25 lines

v1: Initial version.

v2: Based on review comments from Daniel,
- Added a check to know hardware status and load the firmware if not loaded.

Cc Daniel Vetter <[email protected]>
Cc Damien Lespiau <[email protected]>
Cc Imre Deak <[email protected]>
Cc Sunil Kamath <[email protected]>
---
 drivers/gpu/drm/i915/intel_csr.c | 9 +++++++++
 1 file changed, 9 insertions(+)
drivers/gpu/drm/i915/intel_csr.c +2 -0
index ba1ae03..682cc26 100644
@@ -252,6 +252,15 @@ void intel_csr_load_program(struct drm_device *dev)
 		return;
 	}
+	/*
+	 * Dmc will restore the csr the program except DC9, cold boot,
Quoted 8 lines

drivers/gpu/drm/i915/intel_csr.c void intel_csr_load_program(struct drm_device *dev) · line 260

Quoted 4 lines
drivers/gpu/drm/i915/intel_csr.c +4 -0
+	 * warm reset, PCI function level reset, and hibernate/suspend.
+	 * This condition will help to check if csr address space is reset/
+	 * not loaded.
+	 */

Atm we call this from driver load and resume, which doesn seem to cover all the cases you mention in the comment. Should this be a WARN_ON instead? Or do we have troubles in our init sequence where we load too many times?

Yes, the above statement taken from bspec to describe about the special cases dmc will not restore the firmware. Agree, In our cases cold boot and hibernate/suspend mainly we need to load the firmware again, so in my second sentence I wanted to comment mainly regarding this condition check added for suspend-hibernate(reset) and cold boot(not loaded).

Anyways the same api later can be used to load the firmware from anywhere, so my intention to check firmware loaded or not. If already loaded then not to overwrite the csr address space to maintain the dc5/dc6 counter value.

Can the below comment more clear to you.

Quoted 8 lines

/*
* Dmc will restore the csr the program except DC9, cold boot,
* warm reset, PCI function level reset, and hibernate/suspend.
* If firmware is restored by dmc then no need to load again which
* will keep the dc5/dc6 counter exposed by firmware. */

No issue in init sequence.

That seems to still cover all the callers of the function afaics - we do pci resets over suspend resume unconditionally. So I still don't understand where exactly we try to load the dmc firmware in i915.ko when it's already loaded.
-Daniel

Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#5 lore raw

Re: [DMC_BUGFIX_SKL_V2 1/5] drm/i915/skl: Added a check for the hardware status of csr fw before loading.

Quoted 70 lines · Daniel Vetter
Quoted 69 lines · Animesh Manna
Quoted 7 lines · Daniel Vetter
Quoted 6 lines · Animesh Manna

Dmc will restore the csr program except DC9, cold boot, warm reset, PCI function level reset, and hibernate/suspend.

intel_csr_load_program() function is used to load the firmware data from kernel memory to csr address space.

Quoted 7 lines
Quoted 7 lines

All values of csr address space will be zero if it got reset and the first byte of csr program is always a non-zero if firmware is loaded successfuly. Based on hardware status will load the firmware.

Without this condition check if we overwrite the firmware data the counters exposed for dc5/dc6 (help for debugging) will be nullified.

Bacause of the above reason mentioned just above we need to block firmware loading again. So only WARN_ON will not help.

Quoted 24 lines
Quoted 24 lines

v1: Initial version.

v2: Based on review comments from Daniel,
- Added a check to know hardware status and load the firmware if not loaded.

Cc Daniel Vetter <[email protected]>
Cc Damien Lespiau <[email protected]>
Cc Imre Deak <[email protected]>
Cc Sunil Kamath <[email protected]>
---
  drivers/gpu/drm/i915/intel_csr.c | 9 +++++++++
  1 file changed, 9 insertions(+)
drivers/gpu/drm/i915/intel_csr.c +2 -0
index ba1ae03..682cc26 100644
@@ -252,6 +252,15 @@ void intel_csr_load_program(struct drm_device *dev)
  		return;
  	}
+	/*
+	 * Dmc will restore the csr the program except DC9, cold boot,
Quoted 8 lines

drivers/gpu/drm/i915/intel_csr.c void intel_csr_load_program(struct drm_device *dev) · line 260

Quoted 4 lines
drivers/gpu/drm/i915/intel_csr.c +4 -0
+	 * warm reset, PCI function level reset, and hibernate/suspend.
+	 * This condition will help to check if csr address space is reset/
+	 * not loaded.
+	 */

Atm we call this from driver load and resume, which doesn seem to cover all the cases you mention in the comment. Should this be a WARN_ON instead? Or do we have troubles in our init sequence where we load too many times?

Yes, the above statement taken from bspec to describe about the special cases dmc will not restore the firmware. Agree, In our cases cold boot and hibernate/suspend mainly we need to load the firmware again, so in my second sentence I wanted to comment mainly regarding this condition check added for suspend-hibernate(reset) and cold boot(not loaded).

Anyways the same api later can be used to load the firmware from anywhere, so my intention to check firmware loaded or not. If already loaded then not to overwrite the csr address space to maintain the dc5/dc6 counter value.

Can the below comment more clear to you.

/*
* Dmc will restore the csr the program except DC9, cold boot,
* warm reset, PCI function level reset, and hibernate/suspend.
* If firmware is restored by dmc then no need to load again which
* will keep the dc5/dc6 counter exposed by firmware. */

Quoted 5 lines
Quoted 1 line

No issue in init sequence.

That seems to still cover all the callers of the function afaics - we do pci resets over suspend resume unconditionally. So I still don't understand where exactly we try to load the dmc firmware in i915.ko when it's already loaded.

During resume intel_csr_load_program() will be called from intel_runtime_resume().

intel_runtime_resume()-> skl_resume_prepare()-> intel_csr_load_program()

During Pc10 entry testing I can see dmc is restoring back the firmware always, but as you mentioned pci-reset can happen unconditionally, but still then also during resume intel_runtime_resume() will be called and based on register read of csr-base-address firmware loading will happen.

-Animesh

Quoted 1 line

-Daniel

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

#6 lore raw

Re: [DMC_BUGFIX_SKL_V2 1/5] drm/i915/skl: Added a check for the hardware status of csr fw before loading.

Quoted 82 lines · Animesh Manna
Quoted 70 lines · Daniel Vetter
Quoted 69 lines · Animesh Manna
Quoted 7 lines · Daniel Vetter
Quoted 6 lines · Animesh Manna

Dmc will restore the csr program except DC9, cold boot, warm reset, PCI function level reset, and hibernate/suspend.

intel_csr_load_program() function is used to load the firmware data from kernel memory to csr address space.

Quoted 7 lines
Quoted 7 lines

All values of csr address space will be zero if it got reset and the first byte of csr program is always a non-zero if firmware is loaded successfuly. Based on hardware status will load the firmware.

Without this condition check if we overwrite the firmware data the counters exposed for dc5/dc6 (help for debugging) will be nullified.

Bacause of the above reason mentioned just above we need to block firmware loading again. So only WARN_ON will not help.

Quoted 24 lines
Quoted 24 lines

v1: Initial version.

v2: Based on review comments from Daniel,
- Added a check to know hardware status and load the firmware if not loaded.

Cc Daniel Vetter <[email protected]>
Cc Damien Lespiau <[email protected]>
Cc Imre Deak <[email protected]>
Cc Sunil Kamath <[email protected]>
---
 drivers/gpu/drm/i915/intel_csr.c | 9 +++++++++
 1 file changed, 9 insertions(+)
drivers/gpu/drm/i915/intel_csr.c +2 -0
index ba1ae03..682cc26 100644
@@ -252,6 +252,15 @@ void intel_csr_load_program(struct drm_device *dev)
 		return;
 	}
+	/*
+	 * Dmc will restore the csr the program except DC9, cold boot,
Quoted 8 lines

drivers/gpu/drm/i915/intel_csr.c void intel_csr_load_program(struct drm_device *dev) · line 260

Quoted 4 lines
drivers/gpu/drm/i915/intel_csr.c +4 -0
+	 * warm reset, PCI function level reset, and hibernate/suspend.
+	 * This condition will help to check if csr address space is reset/
+	 * not loaded.
+	 */

Atm we call this from driver load and resume, which doesn seem to cover all the cases you mention in the comment. Should this be a WARN_ON instead? Or do we have troubles in our init sequence where we load too many times?

Yes, the above statement taken from bspec to describe about the special cases dmc will not restore the firmware. Agree, In our cases cold boot and hibernate/suspend mainly we need to load the firmware again, so in my second sentence I wanted to comment mainly regarding this condition check added for suspend-hibernate(reset) and cold boot(not loaded).

Anyways the same api later can be used to load the firmware from anywhere, so my intention to check firmware loaded or not. If already loaded then not to overwrite the csr address space to maintain the dc5/dc6 counter value.

Can the below comment more clear to you.

/*
* Dmc will restore the csr the program except DC9, cold boot,
* warm reset, PCI function level reset, and hibernate/suspend.
* If firmware is restored by dmc then no need to load again which
* will keep the dc5/dc6 counter exposed by firmware. */

Quoted 5 lines
Quoted 1 line

No issue in init sequence.

That seems to still cover all the callers of the function afaics - we do pci resets over suspend resume unconditionally. So I still don't understand where exactly we try to load the dmc firmware in i915.ko when it's already loaded.

During resume intel_csr_load_program() will be called from intel_runtime_resume().

Quoted 6 lines

intel_runtime_resume()-> skl_resume_prepare()-> intel_csr_load_program()

During Pc10 entry testing I can see dmc is restoring back the firmware always, but as you mentioned pci-reset can happen unconditionally, but still then also during resume intel_runtime_resume() will be called and based on register read of csr-base-address firmware loading will happen.

But in your comment you're saying it won't get restored in case of dc9 and suspend. So that seems to mismatch what you're saying here (and what the commit message says) and what the code does. And this function here is called for resume after suspend/hibernate only.

So I still see a mismatch between what you're saying here and what the code does vs. what the comment above says. It would be clear if there's just no comment. And we don't call intel_csr_load_program from random places just for luck, if there's unecessary calls of that its kinda a bug in the code (and we should just remove it).
-Daniel

Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#7 lore raw

Re: [DMC_BUGFIX_SKL_V2 1/5] drm/i915/skl: Added a check for the hardware status of csr fw before loading.

Quoted 82 lines · Daniel Vetter
Quoted 81 lines · Animesh Manna
Quoted 69 lines · Daniel Vetter
Quoted 68 lines · Animesh Manna
Quoted 7 lines · Daniel Vetter
Quoted 6 lines · Animesh Manna

Dmc will restore the csr program except DC9, cold boot, warm reset, PCI function level reset, and hibernate/suspend.

intel_csr_load_program() function is used to load the firmware data from kernel memory to csr address space.

Quoted 7 lines
Quoted 7 lines

All values of csr address space will be zero if it got reset and the first byte of csr program is always a non-zero if firmware is loaded successfuly. Based on hardware status will load the firmware.

Without this condition check if we overwrite the firmware data the counters exposed for dc5/dc6 (help for debugging) will be nullified.

Bacause of the above reason mentioned just above we need to block firmware loading again. So only WARN_ON will not help.

Quoted 24 lines
Quoted 24 lines

v1: Initial version.

v2: Based on review comments from Daniel,
- Added a check to know hardware status and load the firmware if not loaded.

Cc Daniel Vetter <[email protected]>
Cc Damien Lespiau <[email protected]>
Cc Imre Deak <[email protected]>
Cc Sunil Kamath <[email protected]>
---
  drivers/gpu/drm/i915/intel_csr.c | 9 +++++++++
  1 file changed, 9 insertions(+)
drivers/gpu/drm/i915/intel_csr.c +2 -0
index ba1ae03..682cc26 100644
@@ -252,6 +252,15 @@ void intel_csr_load_program(struct drm_device *dev)
  		return;
  	}
+	/*
+	 * Dmc will restore the csr the program except DC9, cold boot,
Quoted 8 lines

drivers/gpu/drm/i915/intel_csr.c void intel_csr_load_program(struct drm_device *dev) · line 260

Quoted 4 lines
drivers/gpu/drm/i915/intel_csr.c +4 -0
+	 * warm reset, PCI function level reset, and hibernate/suspend.
+	 * This condition will help to check if csr address space is reset/
+	 * not loaded.
+	 */

Atm we call this from driver load and resume, which doesn seem to cover all the cases you mention in the comment. Should this be a WARN_ON instead? Or do we have troubles in our init sequence where we load too many times?

Yes, the above statement taken from bspec to describe about the special cases dmc will not restore the firmware. Agree, In our cases cold boot and hibernate/suspend mainly we need to load the firmware again, so in my second sentence I wanted to comment mainly regarding this condition check added for suspend-hibernate(reset) and cold boot(not loaded).

Anyways the same api later can be used to load the firmware from anywhere, so my intention to check firmware loaded or not. If already loaded then not to overwrite the csr address space to maintain the dc5/dc6 counter value.

Can the below comment more clear to you.

/*
* Dmc will restore the csr the program except DC9, cold boot,
* warm reset, PCI function level reset, and hibernate/suspend.
* If firmware is restored by dmc then no need to load again which
* will keep the dc5/dc6 counter exposed by firmware. */

Quoted 5 lines
Quoted 1 line

No issue in init sequence.

That seems to still cover all the callers of the function afaics - we do pci resets over suspend resume unconditionally. So I still don't understand where exactly we try to load the dmc firmware in i915.ko when it's already loaded.

During resume intel_csr_load_program() will be called from intel_runtime_resume().

intel_runtime_resume()-> skl_resume_prepare()-> intel_csr_load_program()

Quoted 8 lines
Quoted 4 lines

During Pc10 entry testing I can see dmc is restoring back the firmware always, but as you mentioned pci-reset can happen unconditionally, but still then also during resume intel_runtime_resume() will be called and based on register read of csr-base-address firmware loading will happen.

But in your comment you're saying it won't get restored in case of dc9 and suspend. So that seems to mismatch what you're saying here (and what the commit message says) and what the code does. And this function here is called for resume after suspend/hibernate only.

pc10 entry explanation I told is for skylake. dc9 in skylake is not possible. I think you are confusing between dc6 and dc9. Pc10 can be achieved by entering into dc6 (not dc9) for skylake. dc9 is the lowest possible state for broxton which is not present for skylake.

Here intel_csr_load_program() will be used for both skylake and broxton, and instruction execution flow will be different in case of suspend/resume which I think is confusing you.

I am ready explain you in detail. It will be good if we discuss specific use-case scenario and itz software design for specific platform. Another point - as dmc related code for broxton is not merged better first we close design for skylake. Now, I have added dc9 description in comment thinking of future. If you want I can remove for now and later can add in bxt patch series for enabling dmc. Will wait for your reply.

-Animesh

Quoted 7 lines

So I still see a mismatch between what you're saying here and what the code does vs. what the comment above says. It would be clear if there's just no comment. And we don't call intel_csr_load_program from random places just for luck, if there's unecessary calls of that its kinda a bug in the code (and we should just remove it).
-Daniel

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

#8 lore raw

Re: [DMC_BUGFIX_SKL_V2 1/5] drm/i915/skl: Added a check for the hardware status of csr fw before loading.

In reply to Daniel Vetter
Quoted 82 lines · Daniel Vetter
Quoted 81 lines · Animesh Manna
Quoted 69 lines · Daniel Vetter
Quoted 68 lines · Animesh Manna
Quoted 7 lines · Daniel Vetter
Quoted 6 lines · Animesh Manna

Dmc will restore the csr program except DC9, cold boot, warm reset, PCI function level reset, and hibernate/suspend.

intel_csr_load_program() function is used to load the firmware data from kernel memory to csr address space.

Quoted 7 lines
Quoted 7 lines

All values of csr address space will be zero if it got reset and the first byte of csr program is always a non-zero if firmware is loaded successfuly. Based on hardware status will load the firmware.

Without this condition check if we overwrite the firmware data the counters exposed for dc5/dc6 (help for debugging) will be nullified.

Bacause of the above reason mentioned just above we need to block firmware loading again. So only WARN_ON will not help.

Quoted 24 lines
Quoted 24 lines

v1: Initial version.

v2: Based on review comments from Daniel,
- Added a check to know hardware status and load the firmware if not loaded.

Cc Daniel Vetter <[email protected]>
Cc Damien Lespiau <[email protected]>
Cc Imre Deak <[email protected]>
Cc Sunil Kamath <[email protected]>
---
  drivers/gpu/drm/i915/intel_csr.c | 9 +++++++++
  1 file changed, 9 insertions(+)
drivers/gpu/drm/i915/intel_csr.c +2 -0
index ba1ae03..682cc26 100644
@@ -252,6 +252,15 @@ void intel_csr_load_program(struct drm_device *dev)
  		return;
  	}
+	/*
+	 * Dmc will restore the csr the program except DC9, cold boot,
Quoted 8 lines

drivers/gpu/drm/i915/intel_csr.c void intel_csr_load_program(struct drm_device *dev) · line 260

Quoted 4 lines
drivers/gpu/drm/i915/intel_csr.c +4 -0
+	 * warm reset, PCI function level reset, and hibernate/suspend.
+	 * This condition will help to check if csr address space is reset/
+	 * not loaded.
+	 */

Atm we call this from driver load and resume, which doesn seem to cover all the cases you mention in the comment. Should this be a WARN_ON instead? Or do we have troubles in our init sequence where we load too many times?

Yes, the above statement taken from bspec to describe about the special cases dmc will not restore the firmware. Agree, In our cases cold boot and hibernate/suspend mainly we need to load the firmware again, so in my second sentence I wanted to comment mainly regarding this condition check added for suspend-hibernate(reset) and cold boot(not loaded).

Anyways the same api later can be used to load the firmware from anywhere, so my intention to check firmware loaded or not. If already loaded then not to overwrite the csr address space to maintain the dc5/dc6 counter value.

Can the below comment more clear to you.

/*
* Dmc will restore the csr the program except DC9, cold boot,
* warm reset, PCI function level reset, and hibernate/suspend.
* If firmware is restored by dmc then no need to load again which
* will keep the dc5/dc6 counter exposed by firmware. */

Quoted 5 lines
Quoted 1 line

No issue in init sequence.

That seems to still cover all the callers of the function afaics - we do pci resets over suspend resume unconditionally. So I still don't understand where exactly we try to load the dmc firmware in i915.ko when it's already loaded.

During resume intel_csr_load_program() will be called from intel_runtime_resume().

intel_runtime_resume()-> skl_resume_prepare()-> intel_csr_load_program()

Quoted 8 lines
Quoted 4 lines

During Pc10 entry testing I can see dmc is restoring back the firmware always, but as you mentioned pci-reset can happen unconditionally, but still then also during resume intel_runtime_resume() will be called and based on register read of csr-base-address firmware loading will happen.

But in your comment you're saying it won't get restored in case of dc9 and suspend. So that seems to mismatch what you're saying here (and what the commit message says) and what the code does. And this function here is called for resume after suspend/hibernate only.

pc10 entry explanation I told is for skylake. dc9 in skylake is not possible. I think you are confusing between dc6 and dc9. Pc10 can be achieved by entering into dc6 (not dc9) for skylake. dc9 is the lowest possible state for broxton which is not present for skylake.

Here intel_csr_load_program() will be used for both skylake and broxton, and instruction execution flow will be different in case of suspend/resume which I think is confusing you.

I am ready explain you in detail. It will be good if we discuss specific use-case scenario and itz software design for specific platform. Another point - as dmc related code for broxton is not merged better first we close design for skylake. Now, I have added dc9 description in comment thinking of future. If you want I can remove for now and later can add in bxt patch series for enabling dmc. Will wait for your reply.

-Animesh

Quoted 7 lines

So I still see a mismatch between what you're saying here and what the code does vs. what the comment above says. It would be clear if there's just no comment. And we don't call intel_csr_load_program from random places just for luck, if there's unecessary calls of that its kinda a bug in the code (and we should just remove it).
-Daniel

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

#9 lore raw

Re: [DMC_BUGFIX_SKL_V2 1/5] drm/i915/skl: Added a check for the hardware status of csr fw before loading.

Quoted 94 lines · Animesh Manna
Quoted 82 lines · Daniel Vetter
Quoted 81 lines · Animesh Manna
Quoted 69 lines · Daniel Vetter
Quoted 68 lines · Animesh Manna
Quoted 7 lines · Daniel Vetter
Quoted 6 lines · Animesh Manna

Dmc will restore the csr program except DC9, cold boot, warm reset, PCI function level reset, and hibernate/suspend.

intel_csr_load_program() function is used to load the firmware data from kernel memory to csr address space.

Quoted 7 lines
Quoted 7 lines

All values of csr address space will be zero if it got reset and the first byte of csr program is always a non-zero if firmware is loaded successfuly. Based on hardware status will load the firmware.

Without this condition check if we overwrite the firmware data the counters exposed for dc5/dc6 (help for debugging) will be nullified.

Bacause of the above reason mentioned just above we need to block firmware loading again. So only WARN_ON will not help.

Quoted 24 lines
Quoted 24 lines

v1: Initial version.

v2: Based on review comments from Daniel,
- Added a check to know hardware status and load the firmware if not loaded.

Cc Daniel Vetter <[email protected]>
Cc Damien Lespiau <[email protected]>
Cc Imre Deak <[email protected]>
Cc Sunil Kamath <[email protected]>
---
 drivers/gpu/drm/i915/intel_csr.c | 9 +++++++++
 1 file changed, 9 insertions(+)
drivers/gpu/drm/i915/intel_csr.c +2 -0
index ba1ae03..682cc26 100644
@@ -252,6 +252,15 @@ void intel_csr_load_program(struct drm_device *dev)
 		return;
 	}
+	/*
+	 * Dmc will restore the csr the program except DC9, cold boot,
Quoted 8 lines

drivers/gpu/drm/i915/intel_csr.c void intel_csr_load_program(struct drm_device *dev) · line 259

Quoted 4 lines
drivers/gpu/drm/i915/intel_csr.c +4 -0
+	 * warm reset, PCI function level reset, and hibernate/suspend.
+	 * This condition will help to check if csr address space is reset/
+	 * not loaded.
+	 */

Atm we call this from driver load and resume, which doesn seem to cover all the cases you mention in the comment. Should this be a WARN_ON instead? Or do we have troubles in our init sequence where we load too many times?

Yes, the above statement taken from bspec to describe about the special cases dmc will not restore the firmware. Agree, In our cases cold boot and hibernate/suspend mainly we need to load the firmware again, so in my second sentence I wanted to comment mainly regarding this condition check added for suspend-hibernate(reset) and cold boot(not loaded).

Anyways the same api later can be used to load the firmware from anywhere, so my intention to check firmware loaded or not. If already loaded then not to overwrite the csr address space to maintain the dc5/dc6 counter value.

Can the below comment more clear to you.

/*
* Dmc will restore the csr the program except DC9, cold boot,
* warm reset, PCI function level reset, and hibernate/suspend.
* If firmware is restored by dmc then no need to load again which
* will keep the dc5/dc6 counter exposed by firmware. */

Quoted 5 lines
Quoted 1 line

No issue in init sequence.

That seems to still cover all the callers of the function afaics - we do pci resets over suspend resume unconditionally. So I still don't understand where exactly we try to load the dmc firmware in i915.ko when it's already loaded.

During resume intel_csr_load_program() will be called from intel_runtime_resume().

intel_runtime_resume()-> skl_resume_prepare()-> intel_csr_load_program()

Quoted 8 lines
Quoted 4 lines

During Pc10 entry testing I can see dmc is restoring back the firmware always, but as you mentioned pci-reset can happen unconditionally, but still then also during resume intel_runtime_resume() will be called and based on register read of csr-base-address firmware loading will happen.

But in your comment you're saying it won't get restored in case of dc9 and suspend. So that seems to mismatch what you're saying here (and what the commit message says) and what the code does. And this function here is called for resume after suspend/hibernate only.

Quoted 4 lines

pc10 entry explanation I told is for skylake. dc9 in skylake is not possible. I think you are confusing between dc6 and dc9. Pc10 can be achieved by entering into dc6 (not dc9) for skylake. dc9 is the lowest possible state for broxton which is not present for skylake.

I have no idea at all about different pc levels on skl. What I'm talking about is system suspend/resume and driver load, which are the places this function gets called. At least afaics.

Quoted 3 lines

Here intel_csr_load_program() will be used for both skylake and broxton, and instruction execution flow will be different in case of suspend/resume which I think is confusing you.

That seems like really important information. What's different on bxt? These are the kind of details you should explain in the commit message ...

Quoted 5 lines

I am ready explain you in detail. It will be good if we discuss specific use-case scenario and itz software design for specific platform. Another point - as dmc related code for broxton is not merged better first we close design for skylake. Now, I have added dc9 description in comment thinking of future. If you want I can remove for now and later can add in bxt patch series for enabling dmc. Will wait for your reply.

This question here isn't about the overall design and how to handle power wells in skl/bxt. That's a separate discussion and tracked somewhere else. I'm really just confused about when exactly we need to reload to firmware, and why we need a runtime check for that. Normally we should know when to reload the firmware and just either reload or not, without checking hw state. And I don't like checking for hw state since at least in the past that kind of code ended up being fragile - it's an illusion that it does the right thing no matter what, since often there's other tricky ordering constraints. And if you have automatic duct-tape like then no one will ever spot those other, harder to spot issues, until an expensive customer escalation happens.

So what I want to know here is:
- When exactly do we need to reload dmc firmware.
- What exactly is the reason why we can't make that decision statically in the code (by calling csr_load at the right spots).

Thanks, Daniel

Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#10 lore raw

Re: [DMC_BUGFIX_SKL_V2 1/5] drm/i915/skl: Added a check for the hardware status of csr fw before loading.

Quoted 123 lines · Daniel Vetter
Quoted 87 lines · Animesh Manna
Quoted 85 lines · Daniel Vetter
Quoted 84 lines · Animesh Manna
Quoted 69 lines · Daniel Vetter
Quoted 68 lines · Animesh Manna
Quoted 7 lines · Daniel Vetter
Quoted 6 lines · Animesh Manna

Dmc will restore the csr program except DC9, cold boot, warm reset, PCI function level reset, and hibernate/suspend.

intel_csr_load_program() function is used to load the firmware data from kernel memory to csr address space.

Quoted 7 lines
Quoted 7 lines

All values of csr address space will be zero if it got reset and the first byte of csr program is always a non-zero if firmware is loaded successfuly. Based on hardware status will load the firmware.

Without this condition check if we overwrite the firmware data the counters exposed for dc5/dc6 (help for debugging) will be nullified.

Bacause of the above reason mentioned just above we need to block firmware loading again. So only WARN_ON will not help.

Quoted 24 lines
Quoted 24 lines

v1: Initial version.

v2: Based on review comments from Daniel,
- Added a check to know hardware status and load the firmware if not loaded.

Cc Daniel Vetter <[email protected]>
Cc Damien Lespiau <[email protected]>
Cc Imre Deak <[email protected]>
Cc Sunil Kamath <[email protected]>
---
  drivers/gpu/drm/i915/intel_csr.c | 9 +++++++++
  1 file changed, 9 insertions(+)
drivers/gpu/drm/i915/intel_csr.c +2 -0
index ba1ae03..682cc26 100644
@@ -252,6 +252,15 @@ void intel_csr_load_program(struct drm_device *dev)
  		return;
  	}
+	/*
+	 * Dmc will restore the csr the program except DC9, cold boot,
Quoted 8 lines

drivers/gpu/drm/i915/intel_csr.c void intel_csr_load_program(struct drm_device *dev) · line 259

Quoted 4 lines
drivers/gpu/drm/i915/intel_csr.c +4 -0
+	 * warm reset, PCI function level reset, and hibernate/suspend.
+	 * This condition will help to check if csr address space is reset/
+	 * not loaded.
+	 */

Atm we call this from driver load and resume, which doesn seem to cover all the cases you mention in the comment. Should this be a WARN_ON instead? Or do we have troubles in our init sequence where we load too many times?

Yes, the above statement taken from bspec to describe about the special cases dmc will not restore the firmware. Agree, In our cases cold boot and hibernate/suspend mainly we need to load the firmware again, so in my second sentence I wanted to comment mainly regarding this condition check added for suspend-hibernate(reset) and cold boot(not loaded).

Anyways the same api later can be used to load the firmware from anywhere, so my intention to check firmware loaded or not. If already loaded then not to overwrite the csr address space to maintain the dc5/dc6 counter value.

Can the below comment more clear to you.

/*
* Dmc will restore the csr the program except DC9, cold boot,
* warm reset, PCI function level reset, and hibernate/suspend.
* If firmware is restored by dmc then no need to load again which
* will keep the dc5/dc6 counter exposed by firmware. */

Quoted 5 lines
Quoted 1 line

No issue in init sequence.

That seems to still cover all the callers of the function afaics - we do pci resets over suspend resume unconditionally. So I still don't understand where exactly we try to load the dmc firmware in i915.ko when it's already loaded.

During resume intel_csr_load_program() will be called from intel_runtime_resume().

intel_runtime_resume()-> skl_resume_prepare()-> intel_csr_load_program()

During Pc10 entry testing I can see dmc is restoring back the firmware always, but as you mentioned pci-reset can happen unconditionally, but still then also during resume intel_runtime_resume() will be called and based on register read of csr-base-address firmware loading will happen.

Quoted 8 lines
Quoted 4 lines

But in your comment you're saying it won't get restored in case of dc9 and suspend. So that seems to mismatch what you're saying here (and what the commit message says) and what the code does. And this function here is called for resume after suspend/hibernate only.

pc10 entry explanation I told is for skylake. dc9 in skylake is not possible. I think you are confusing between dc6 and dc9. Pc10 can be achieved by entering into dc6 (not dc9) for skylake. dc9 is the lowest possible state for broxton which is not present for skylake.

I have no idea at all about different pc levels on skl. What I'm talking about is system suspend/resume and driver load, which are the places this function gets called. At least afaics.

Quoted 3 lines

Here intel_csr_load_program() will be used for both skylake and broxton, and instruction execution flow will be different in case of suspend/resume which I think is confusing you.

That seems like really important information. What's different on bxt? These are the kind of details you should explain in the commit message ...

Quoted 5 lines

I am ready explain you in detail. It will be good if we discuss specific use-case scenario and itz software design for specific platform. Another point - as dmc related code for broxton is not merged better first we close design for skylake. Now, I have added dc9 description in comment thinking of future. If you want I can remove for now and later can add in bxt patch series for enabling dmc. Will wait for your reply.

This question here isn't about the overall design and how to handle power wells in skl/bxt. That's a separate discussion and tracked somewhere else. I'm really just confused about when exactly we need to reload to firmware, and why we need a runtime check for that. Normally we should know when to reload the firmware and just either reload or not, without checking hw state. And I don't like checking for hw state since at least in the past that kind of code ended up being fragile - it's an illusion that it does the right thing no matter what, since often there's other tricky ordering constraints. And if you have automatic duct-tape like then no one will ever spot those other, harder to spot issues, until an expensive customer escalation happens.

Quoted 2 lines

So what I want to know here is:
- When exactly do we need to reload dmc firmware.

In skl, during driver load first time we load the firmware, during normal suspend-resume (dc6 entry/exit) no need to reload the firmware again as dmc will take care of it. But during suspend/hibernation dmc will not restore the firmware. In that case driver need to reload it again. I do not know how to differentiate pm-suspend and suspend-hibernation and thought both the cases intel_runtime_resume() will be called where we can check the h/w state and reload the firmware if dmc is not restored.

In bxt, during driver load first time we load the firmware, during normal suspend-resume display engine will enter into dc9 and dmc will not restore the firmware. So every suspend-resume we need to reload the firmware.

Quoted 2 lines

- What exactly is the reason why we can't make that decision statically in the code (by calling csr_load at the right spots).

As I mentioned before in case of skylake can we differentiate between "resume from pm-suspend" with "resume from suspend-hibernation" inside driver?

In case of broxton, every time we need to reload, so we can decide statically.

-Animesh

Quoted 2 lines

Thanks, Daniel

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

#11 lore raw

Re: [DMC_BUGFIX_SKL_V2 1/5] drm/i915/skl: Added a check for the hardware status of csr fw before loading.

Quoted 145 lines · Animesh Manna
Quoted 123 lines · Daniel Vetter
Quoted 87 lines · Animesh Manna
Quoted 85 lines · Daniel Vetter
Quoted 84 lines · Animesh Manna
Quoted 69 lines · Daniel Vetter
Quoted 68 lines · Animesh Manna
Quoted 7 lines · Daniel Vetter
Quoted 6 lines · Animesh Manna

Dmc will restore the csr program except DC9, cold boot, warm reset, PCI function level reset, and hibernate/suspend.

intel_csr_load_program() function is used to load the firmware data from kernel memory to csr address space.

Quoted 7 lines
Quoted 7 lines

All values of csr address space will be zero if it got reset and the first byte of csr program is always a non-zero if firmware is loaded successfuly. Based on hardware status will load the firmware.

Without this condition check if we overwrite the firmware data the counters exposed for dc5/dc6 (help for debugging) will be nullified.

Bacause of the above reason mentioned just above we need to block firmware loading again. So only WARN_ON will not help.

Quoted 24 lines
Quoted 24 lines

v1: Initial version.

v2: Based on review comments from Daniel,
- Added a check to know hardware status and load the firmware if not loaded.

Cc Daniel Vetter <[email protected]>
Cc Damien Lespiau <[email protected]>
Cc Imre Deak <[email protected]>
Cc Sunil Kamath <[email protected]>
---
 drivers/gpu/drm/i915/intel_csr.c | 9 +++++++++
 1 file changed, 9 insertions(+)
drivers/gpu/drm/i915/intel_csr.c +2 -0
index ba1ae03..682cc26 100644
@@ -252,6 +252,15 @@ void intel_csr_load_program(struct drm_device *dev)
 		return;
 	}
+	/*
+	 * Dmc will restore the csr the program except DC9, cold boot,
Quoted 8 lines

drivers/gpu/drm/i915/intel_csr.c void intel_csr_load_program(struct drm_device *dev) · line 259

Quoted 4 lines
drivers/gpu/drm/i915/intel_csr.c +4 -0
+	 * warm reset, PCI function level reset, and hibernate/suspend.
+	 * This condition will help to check if csr address space is reset/
+	 * not loaded.
+	 */

Atm we call this from driver load and resume, which doesn seem to cover all the cases you mention in the comment. Should this be a WARN_ON instead? Or do we have troubles in our init sequence where we load too many times?

Yes, the above statement taken from bspec to describe about the special cases dmc will not restore the firmware. Agree, In our cases cold boot and hibernate/suspend mainly we need to load the firmware again, so in my second sentence I wanted to comment mainly regarding this condition check added for suspend-hibernate(reset) and cold boot(not loaded).

Anyways the same api later can be used to load the firmware from anywhere, so my intention to check firmware loaded or not. If already loaded then not to overwrite the csr address space to maintain the dc5/dc6 counter value.

Can the below comment more clear to you.

/*
* Dmc will restore the csr the program except DC9, cold boot,
* warm reset, PCI function level reset, and hibernate/suspend.
* If firmware is restored by dmc then no need to load again which
* will keep the dc5/dc6 counter exposed by firmware. */

Quoted 5 lines
Quoted 1 line

No issue in init sequence.

That seems to still cover all the callers of the function afaics - we do pci resets over suspend resume unconditionally. So I still don't understand where exactly we try to load the dmc firmware in i915.ko when it's already loaded.

During resume intel_csr_load_program() will be called from intel_runtime_resume().

intel_runtime_resume()-> skl_resume_prepare()-> intel_csr_load_program()

During Pc10 entry testing I can see dmc is restoring back the firmware always, but as you mentioned pci-reset can happen unconditionally, but still then also during resume intel_runtime_resume() will be called and based on register read of csr-base-address firmware loading will happen.

Quoted 8 lines
Quoted 4 lines

But in your comment you're saying it won't get restored in case of dc9 and suspend. So that seems to mismatch what you're saying here (and what the commit message says) and what the code does. And this function here is called for resume after suspend/hibernate only.

pc10 entry explanation I told is for skylake. dc9 in skylake is not possible. I think you are confusing between dc6 and dc9. Pc10 can be achieved by entering into dc6 (not dc9) for skylake. dc9 is the lowest possible state for broxton which is not present for skylake.

I have no idea at all about different pc levels on skl. What I'm talking about is system suspend/resume and driver load, which are the places this function gets called. At least afaics.

Quoted 3 lines

Here intel_csr_load_program() will be used for both skylake and broxton, and instruction execution flow will be different in case of suspend/resume which I think is confusing you.

That seems like really important information. What's different on bxt? These are the kind of details you should explain in the commit message ...

Quoted 5 lines

I am ready explain you in detail. It will be good if we discuss specific use-case scenario and itz software design for specific platform. Another point - as dmc related code for broxton is not merged better first we close design for skylake. Now, I have added dc9 description in comment thinking of future. If you want I can remove for now and later can add in bxt patch series for enabling dmc. Will wait for your reply.

This question here isn't about the overall design and how to handle power wells in skl/bxt. That's a separate discussion and tracked somewhere else. I'm really just confused about when exactly we need to reload to firmware, and why we need a runtime check for that. Normally we should know when to reload the firmware and just either reload or not, without checking hw state. And I don't like checking for hw state since at least in the past that kind of code ended up being fragile - it's an illusion that it does the right thing no matter what, since often there's other tricky ordering constraints. And if you have automatic duct-tape like then no one will ever spot those other, harder to spot issues, until an expensive customer escalation happens.

Quoted 2 lines

So what I want to know here is:
- When exactly do we need to reload dmc firmware.

In skl, during driver load first time we load the firmware, during normal suspend-resume (dc6 entry/exit)
no need to reload the firmware again as dmc will take care of it. But during suspend/hibernation
dmc will not restore the firmware. In that case driver need to reload it again. I do not know
how to differentiate pm-suspend and suspend-hibernation and thought both the cases
intel_runtime_resume() will be called where we can check the h/w state and reload the
firmware if dmc is not restored.

In bxt, during driver load first time we load the firmware, during normal suspend-resume
display engine will enter into dc9 and dmc will not restore the firmware. So every
suspend-resume we need to reload the firmware.

Quoted 8 lines
Quoted 2 lines

- What exactly is the reason why we can't make that decision statically in the code (by calling csr_load at the right spots).

As I mentioned before in case of skylake can we differentiate between "resume from pm-suspend" with "resume from suspend-hibernation" inside driver?

In case of broxton, every time we need to reload, so we can decide statically.

Of course we can differentiate between all the different resume paths, and we also have a per-platform split to take care of bxt vs. skl. And there are actually 3 different resume paths:

- runtime PM resume. This calls the runtime_resume hook. It sounds like on skl we should _not_ load the csr firmware, but on bxt we should load it. This can be fixed by removing the intel_csr_load_program call from skl_resume_prepare.
- resume from hibernate-to-disk (i.e. system completely off, state stored on the swap partition) is done by calling the thaw callbacks.
- resume from suspend-to-mem (i.e. system in low-power with only memory in self-refresh, all state stored in memory) is done by calling the resume callbacks.

For i915 we use unified handlers in our dev_pm_ops for both thaw and resume, but it sounds like that won't be a problem for skl/bxt since we need to reload the csr firmware in all cases. Although I'm not perfectly sure since you don't explain what kind of resume you mean exactly (since you don't use the linux names for them).

Anyway it sounds like we can replace this patch by one where we remove that errornous csr load call from skl runtime pm resume and that's all. But I suggest to make sure we get this right we keep the check you're adding here, but wrap it in a WARN_ON. Then we'll get a backtrace when this is going wrong again. Like this:

	if (WARN_ON(csr_loaded_already()))
		return;

Also when redoing the commits please explain in detail what exactly are the requirements like you've done above, but please use the standard linux names, i.e. "runtime PM" and "hibernate-to-disk" and "suspend-to-mem".

Thanks, Daniel

Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#12 lore raw

Re: [DMC_BUGFIX_SKL_V2 1/5] drm/i915/skl: Added a check for the hardware status of csr fw before loading.

Quoted 181 lines · Daniel Vetter
Quoted 145 lines · Animesh Manna
Quoted 123 lines · Daniel Vetter
Quoted 87 lines · Animesh Manna
Quoted 85 lines · Daniel Vetter
Quoted 84 lines · Animesh Manna
Quoted 69 lines · Daniel Vetter
Quoted 68 lines · Animesh Manna
Quoted 7 lines · Daniel Vetter
Quoted 6 lines · Animesh Manna

Dmc will restore the csr program except DC9, cold boot, warm reset, PCI function level reset, and hibernate/suspend.

intel_csr_load_program() function is used to load the firmware data from kernel memory to csr address space.

Quoted 7 lines
Quoted 7 lines

All values of csr address space will be zero if it got reset and the first byte of csr program is always a non-zero if firmware is loaded successfuly. Based on hardware status will load the firmware.

Without this condition check if we overwrite the firmware data the counters exposed for dc5/dc6 (help for debugging) will be nullified.

Bacause of the above reason mentioned just above we need to block firmware loading again. So only WARN_ON will not help.

Quoted 24 lines
Quoted 24 lines

v1: Initial version.

v2: Based on review comments from Daniel,
- Added a check to know hardware status and load the firmware if not loaded.

Cc Daniel Vetter <[email protected]>
Cc Damien Lespiau <[email protected]>
Cc Imre Deak <[email protected]>
Cc Sunil Kamath <[email protected]>
---
 drivers/gpu/drm/i915/intel_csr.c | 9 +++++++++
 1 file changed, 9 insertions(+)
drivers/gpu/drm/i915/intel_csr.c +2 -0
index ba1ae03..682cc26 100644
@@ -252,6 +252,15 @@ void intel_csr_load_program(struct drm_device *dev)
             return;
     }
+     /*
+      * Dmc will restore the csr the program except DC9, cold boot,
Quoted 8 lines

drivers/gpu/drm/i915/intel_csr.c void intel_csr_load_program(struct drm_device *dev) · line 259

Quoted 4 lines
drivers/gpu/drm/i915/intel_csr.c +4 -0
+      * warm reset, PCI function level reset, and hibernate/suspend.
+      * This condition will help to check if csr address space is reset/
+      * not loaded.
+      */

Atm we call this from driver load and resume, which doesn seem to cover all the cases you mention in the comment. Should this be a WARN_ON instead? Or do we have troubles in our init sequence where we load too many times?

Yes, the above statement taken from bspec to describe about the special cases dmc will not restore the firmware. Agree, In our cases cold boot and hibernate/suspend mainly we need to load the firmware again, so in my second sentence I wanted to comment mainly regarding this condition check added for suspend-hibernate(reset) and cold boot(not loaded).

Anyways the same api later can be used to load the firmware from anywhere, so my intention to check firmware loaded or not. If already loaded then not to overwrite the csr address space to maintain the dc5/dc6 counter value.

Can the below comment more clear to you.

/*
* Dmc will restore the csr the program except DC9, cold boot,
* warm reset, PCI function level reset, and hibernate/suspend.
* If firmware is restored by dmc then no need to load again which
* will keep the dc5/dc6 counter exposed by firmware. */

Quoted 5 lines
Quoted 1 line

No issue in init sequence.

That seems to still cover all the callers of the function afaics - we do pci resets over suspend resume unconditionally. So I still don't understand where exactly we try to load the dmc firmware in i915.ko when it's already loaded.

During resume intel_csr_load_program() will be called from intel_runtime_resume().

intel_runtime_resume()-> skl_resume_prepare()-> intel_csr_load_program()

During Pc10 entry testing I can see dmc is restoring back the firmware always, but as you mentioned pci-reset can happen unconditionally, but still then also during resume intel_runtime_resume() will be called and based on register read of csr-base-address firmware loading will happen.

Quoted 8 lines
Quoted 4 lines

But in your comment you're saying it won't get restored in case of dc9 and suspend. So that seems to mismatch what you're saying here (and what the commit message says) and what the code does. And this function here is called for resume after suspend/hibernate only.

pc10 entry explanation I told is for skylake. dc9 in skylake is not possible. I think you are confusing between dc6 and dc9. Pc10 can be achieved by entering into dc6 (not dc9) for skylake. dc9 is the lowest possible state for broxton which is not present for skylake.

I have no idea at all about different pc levels on skl. What I'm talking about is system suspend/resume and driver load, which are the places this function gets called. At least afaics.

Quoted 3 lines

Here intel_csr_load_program() will be used for both skylake and broxton, and instruction execution flow will be different in case of suspend/resume which I think is confusing you.

That seems like really important information. What's different on bxt? These are the kind of details you should explain in the commit message ...

Quoted 5 lines

I am ready explain you in detail. It will be good if we discuss specific use-case scenario and itz software design for specific platform. Another point - as dmc related code for broxton is not merged better first we close design for skylake. Now, I have added dc9 description in comment thinking of future. If you want I can remove for now and later can add in bxt patch series for enabling dmc. Will wait for your reply.

This question here isn't about the overall design and how to handle power wells in skl/bxt. That's a separate discussion and tracked somewhere else. I'm really just confused about when exactly we need to reload to firmware, and why we need a runtime check for that. Normally we should know when to reload the firmware and just either reload or not, without checking hw state. And I don't like checking for hw state since at least in the past that kind of code ended up being fragile - it's an illusion that it does the right thing no matter what, since often there's other tricky ordering constraints. And if you have automatic duct-tape like then no one will ever spot those other, harder to spot issues, until an expensive customer escalation happens.

Quoted 2 lines

So what I want to know here is:
- When exactly do we need to reload dmc firmware.

In skl, during driver load first time we load the firmware, during normal suspend-resume (dc6 entry/exit)
no need to reload the firmware again as dmc will take care of it. But during suspend/hibernation
dmc will not restore the firmware. In that case driver need to reload it again. I do not know
how to differentiate pm-suspend and suspend-hibernation and thought both the cases
intel_runtime_resume() will be called where we can check the h/w state and reload the
firmware if dmc is not restored.

In bxt, during driver load first time we load the firmware, during normal suspend-resume
display engine will enter into dc9 and dmc will not restore the firmware. So every
suspend-resume we need to reload the firmware.

Quoted 8 lines
Quoted 2 lines

- What exactly is the reason why we can't make that decision statically in the code (by calling csr_load at the right spots).

As I mentioned before in case of skylake can we differentiate between "resume from pm-suspend" with "resume from suspend-hibernation" inside driver?

In case of broxton, every time we need to reload, so we can decide statically.

Of course we can differentiate between all the different resume paths, and we also have a per-platform split to take care of bxt vs. skl. And there are actually 3 different resume paths:

- runtime PM resume. This calls the runtime_resume hook. It sounds like on skl we should _not_ load the csr firmware, but on bxt we should load it. This can be fixed by removing the intel_csr_load_program call from skl_resume_prepare.
- resume from hibernate-to-disk (i.e. system completely off, state stored on the swap partition) is done by calling the thaw callbacks.
- resume from suspend-to-mem (i.e. system in low-power with only memory in self-refresh, all state stored in memory) is done by calling the resume callbacks.

For i915 we use unified handlers in our dev_pm_ops for both thaw and resume, but it sounds like that won't be a problem for skl/bxt since we need to reload the csr firmware in all cases. Although I'm not perfectly sure since you don't explain what kind of resume you mean exactly (since you don't use the linux names for them).

Anyway it sounds like we can replace this patch by one where we remove that errornous csr load call from skl runtime pm resume and that's all. But I suggest to make sure we get this right we keep the check you're adding here, but wrap it in a WARN_ON. Then we'll get a backtrace when this is going wrong again. Like this:

Quoted 6 lines
        if (WARN_ON(csr_loaded_already()))
                return;

Also when redoing the commits please explain in detail what exactly are the requirements like you've done above, but please use the standard linux names, i.e. "runtime PM" and "hibernate-to-disk" and "suspend-to-mem".

Ok hooray there's more suspend-to-something things I've totally missed:
- suspend-to-idle (done by cat freeze > /sys/power/state) and
- suspend (done by cat suspend > /sys/power/state)

And apparently there's really no way to drivers to tell them apart. Rafael, is there really no way for drivers to take different paths for these 3 suspend cases? I tried grepping for PM_SUSPEND_ON/STANDY/MEM and didn't spot anything.

Also we're completely missing test coverage for that in igt. That is something that needs to be fixed asap (yet another case of combinatorial explosion in igt tests, yay). And at least one of those suspend-to-idle testcase better be in the BAT.
-Daniel

Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#13 lore raw

Re: [DMC_BUGFIX_SKL_V2 1/5] drm/i915/skl: Added a check for the hardware status of csr fw before loading.

Actually add Rafael this time around ...
-Daniel

Quoted 207 lines · Daniel Vetter
Quoted 181 lines · Daniel Vetter
Quoted 145 lines · Animesh Manna
Quoted 123 lines · Daniel Vetter
Quoted 87 lines · Animesh Manna
Quoted 85 lines · Daniel Vetter
Quoted 84 lines · Animesh Manna
Quoted 69 lines · Daniel Vetter
Quoted 68 lines · Animesh Manna
Quoted 7 lines · Daniel Vetter
Quoted 6 lines · Animesh Manna

Dmc will restore the csr program except DC9, cold boot, warm reset, PCI function level reset, and hibernate/suspend.

intel_csr_load_program() function is used to load the firmware data from kernel memory to csr address space.

Quoted 7 lines
Quoted 7 lines

All values of csr address space will be zero if it got reset and the first byte of csr program is always a non-zero if firmware is loaded successfuly. Based on hardware status will load the firmware.

Without this condition check if we overwrite the firmware data the counters exposed for dc5/dc6 (help for debugging) will be nullified.

Bacause of the above reason mentioned just above we need to block firmware loading again. So only WARN_ON will not help.

Quoted 24 lines
Quoted 24 lines

v1: Initial version.

v2: Based on review comments from Daniel,
- Added a check to know hardware status and load the firmware if not loaded.

Cc Daniel Vetter <[email protected]>
Cc Damien Lespiau <[email protected]>
Cc Imre Deak <[email protected]>
Cc Sunil Kamath <[email protected]>
---
 drivers/gpu/drm/i915/intel_csr.c | 9 +++++++++
 1 file changed, 9 insertions(+)
drivers/gpu/drm/i915/intel_csr.c +2 -0
index ba1ae03..682cc26 100644
@@ -252,6 +252,15 @@ void intel_csr_load_program(struct drm_device *dev)
             return;
     }
+     /*
+      * Dmc will restore the csr the program except DC9, cold boot,
Quoted 8 lines

drivers/gpu/drm/i915/intel_csr.c void intel_csr_load_program(struct drm_device *dev) · line 259

Quoted 4 lines
drivers/gpu/drm/i915/intel_csr.c +4 -0
+      * warm reset, PCI function level reset, and hibernate/suspend.
+      * This condition will help to check if csr address space is reset/
+      * not loaded.
+      */

Atm we call this from driver load and resume, which doesn seem to cover all the cases you mention in the comment. Should this be a WARN_ON instead? Or do we have troubles in our init sequence where we load too many times?

Yes, the above statement taken from bspec to describe about the special cases dmc will not restore the firmware. Agree, In our cases cold boot and hibernate/suspend mainly we need to load the firmware again, so in my second sentence I wanted to comment mainly regarding this condition check added for suspend-hibernate(reset) and cold boot(not loaded).

Anyways the same api later can be used to load the firmware from anywhere, so my intention to check firmware loaded or not. If already loaded then not to overwrite the csr address space to maintain the dc5/dc6 counter value.

Can the below comment more clear to you.

/*
* Dmc will restore the csr the program except DC9, cold boot,
* warm reset, PCI function level reset, and hibernate/suspend.
* If firmware is restored by dmc then no need to load again which
* will keep the dc5/dc6 counter exposed by firmware. */

Quoted 5 lines
Quoted 1 line

No issue in init sequence.

That seems to still cover all the callers of the function afaics - we do pci resets over suspend resume unconditionally. So I still don't understand where exactly we try to load the dmc firmware in i915.ko when it's already loaded.

During resume intel_csr_load_program() will be called from intel_runtime_resume().

intel_runtime_resume()-> skl_resume_prepare()-> intel_csr_load_program()

During Pc10 entry testing I can see dmc is restoring back the firmware always, but as you mentioned pci-reset can happen unconditionally, but still then also during resume intel_runtime_resume() will be called and based on register read of csr-base-address firmware loading will happen.

Quoted 8 lines
Quoted 4 lines

But in your comment you're saying it won't get restored in case of dc9 and suspend. So that seems to mismatch what you're saying here (and what the commit message says) and what the code does. And this function here is called for resume after suspend/hibernate only.

pc10 entry explanation I told is for skylake. dc9 in skylake is not possible. I think you are confusing between dc6 and dc9. Pc10 can be achieved by entering into dc6 (not dc9) for skylake. dc9 is the lowest possible state for broxton which is not present for skylake.

I have no idea at all about different pc levels on skl. What I'm talking about is system suspend/resume and driver load, which are the places this function gets called. At least afaics.

Quoted 3 lines

Here intel_csr_load_program() will be used for both skylake and broxton, and instruction execution flow will be different in case of suspend/resume which I think is confusing you.

That seems like really important information. What's different on bxt? These are the kind of details you should explain in the commit message ...

Quoted 5 lines

I am ready explain you in detail. It will be good if we discuss specific use-case scenario and itz software design for specific platform. Another point - as dmc related code for broxton is not merged better first we close design for skylake. Now, I have added dc9 description in comment thinking of future. If you want I can remove for now and later can add in bxt patch series for enabling dmc. Will wait for your reply.

This question here isn't about the overall design and how to handle power wells in skl/bxt. That's a separate discussion and tracked somewhere else. I'm really just confused about when exactly we need to reload to firmware, and why we need a runtime check for that. Normally we should know when to reload the firmware and just either reload or not, without checking hw state. And I don't like checking for hw state since at least in the past that kind of code ended up being fragile - it's an illusion that it does the right thing no matter what, since often there's other tricky ordering constraints. And if you have automatic duct-tape like then no one will ever spot those other, harder to spot issues, until an expensive customer escalation happens.

Quoted 2 lines

So what I want to know here is:
- When exactly do we need to reload dmc firmware.

In skl, during driver load first time we load the firmware, during normal suspend-resume (dc6 entry/exit)
no need to reload the firmware again as dmc will take care of it. But during suspend/hibernation
dmc will not restore the firmware. In that case driver need to reload it again. I do not know
how to differentiate pm-suspend and suspend-hibernation and thought both the cases
intel_runtime_resume() will be called where we can check the h/w state and reload the
firmware if dmc is not restored.

In bxt, during driver load first time we load the firmware, during normal suspend-resume
display engine will enter into dc9 and dmc will not restore the firmware. So every
suspend-resume we need to reload the firmware.

Quoted 8 lines
Quoted 2 lines

- What exactly is the reason why we can't make that decision statically in the code (by calling csr_load at the right spots).

As I mentioned before in case of skylake can we differentiate between "resume from pm-suspend" with "resume from suspend-hibernation" inside driver?

In case of broxton, every time we need to reload, so we can decide statically.

Of course we can differentiate between all the different resume paths, and we also have a per-platform split to take care of bxt vs. skl. And there are actually 3 different resume paths:

- runtime PM resume. This calls the runtime_resume hook. It sounds like on skl we should _not_ load the csr firmware, but on bxt we should load it. This can be fixed by removing the intel_csr_load_program call from skl_resume_prepare.
- resume from hibernate-to-disk (i.e. system completely off, state stored on the swap partition) is done by calling the thaw callbacks.
- resume from suspend-to-mem (i.e. system in low-power with only memory in self-refresh, all state stored in memory) is done by calling the resume callbacks.

For i915 we use unified handlers in our dev_pm_ops for both thaw and resume, but it sounds like that won't be a problem for skl/bxt since we need to reload the csr firmware in all cases. Although I'm not perfectly sure since you don't explain what kind of resume you mean exactly (since you don't use the linux names for them).

Anyway it sounds like we can replace this patch by one where we remove that errornous csr load call from skl runtime pm resume and that's all. But I suggest to make sure we get this right we keep the check you're adding here, but wrap it in a WARN_ON. Then we'll get a backtrace when this is going wrong again. Like this:

Quoted 6 lines
        if (WARN_ON(csr_loaded_already()))
                return;

Also when redoing the commits please explain in detail what exactly are the requirements like you've done above, but please use the standard linux names, i.e. "runtime PM" and "hibernate-to-disk" and "suspend-to-mem".

Ok hooray there's more suspend-to-something things I've totally missed:
- suspend-to-idle (done by cat freeze > /sys/power/state) and
- suspend (done by cat suspend > /sys/power/state)

And apparently there's really no way to drivers to tell them apart. Rafael, is there really no way for drivers to take different paths for these 3 suspend cases? I tried grepping for PM_SUSPEND_ON/STANDY/MEM and didn't spot anything.

Also we're completely missing test coverage for that in igt. That is something that needs to be fixed asap (yet another case of combinatorial explosion in igt tests, yay). And at least one of those suspend-to-idle testcase better be in the BAT.
-Daniel
--
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch

Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#14 lore raw

Re: [DMC_BUGFIX_SKL_V2 1/5] drm/i915/skl: Added a check for the hardware status of csr fw before loading.

acpi_target_system_state() seems to be almost the thing we're looking for, except that it's only valid in the suspend callbacks since it gets reset to ACPI_STATE_S0 when resuming. So probably we want something else ...
-Daniel

Quoted 218 lines · Daniel Vetter

Actually add Rafael this time around ...
-Daniel

Quoted 198 lines · Daniel Vetter
Quoted 181 lines · Daniel Vetter
Quoted 145 lines · Animesh Manna
Quoted 123 lines · Daniel Vetter
Quoted 87 lines · Animesh Manna
Quoted 85 lines · Daniel Vetter
Quoted 84 lines · Animesh Manna
Quoted 69 lines · Daniel Vetter
Quoted 68 lines · Animesh Manna
Quoted 7 lines · Daniel Vetter
Quoted 6 lines · Animesh Manna

Dmc will restore the csr program except DC9, cold boot, warm reset, PCI function level reset, and hibernate/suspend.

intel_csr_load_program() function is used to load the firmware data from kernel memory to csr address space.

Quoted 7 lines
Quoted 7 lines

All values of csr address space will be zero if it got reset and the first byte of csr program is always a non-zero if firmware is loaded successfuly. Based on hardware status will load the firmware.

Without this condition check if we overwrite the firmware data the counters exposed for dc5/dc6 (help for debugging) will be nullified.

Bacause of the above reason mentioned just above we need to block firmware loading again. So only WARN_ON will not help.

Quoted 24 lines
Quoted 24 lines

v1: Initial version.

v2: Based on review comments from Daniel,
- Added a check to know hardware status and load the firmware if not loaded.

Cc Daniel Vetter <[email protected]>
Cc Damien Lespiau <[email protected]>
Cc Imre Deak <[email protected]>
Cc Sunil Kamath <[email protected]>
---
 drivers/gpu/drm/i915/intel_csr.c | 9 +++++++++
 1 file changed, 9 insertions(+)
drivers/gpu/drm/i915/intel_csr.c +2 -0
index ba1ae03..682cc26 100644
@@ -252,6 +252,15 @@ void intel_csr_load_program(struct drm_device *dev)
             return;
     }
+     /*
+      * Dmc will restore the csr the program except DC9, cold boot,
Quoted 8 lines

drivers/gpu/drm/i915/intel_csr.c void intel_csr_load_program(struct drm_device *dev) · line 259

Quoted 4 lines
drivers/gpu/drm/i915/intel_csr.c +4 -0
+      * warm reset, PCI function level reset, and hibernate/suspend.
+      * This condition will help to check if csr address space is reset/
+      * not loaded.
+      */

Atm we call this from driver load and resume, which doesn seem to cover all the cases you mention in the comment. Should this be a WARN_ON instead? Or do we have troubles in our init sequence where we load too many times?

Yes, the above statement taken from bspec to describe about the special cases dmc will not restore the firmware. Agree, In our cases cold boot and hibernate/suspend mainly we need to load the firmware again, so in my second sentence I wanted to comment mainly regarding this condition check added for suspend-hibernate(reset) and cold boot(not loaded).

Anyways the same api later can be used to load the firmware from anywhere, so my intention to check firmware loaded or not. If already loaded then not to overwrite the csr address space to maintain the dc5/dc6 counter value.

Can the below comment more clear to you.

/*
* Dmc will restore the csr the program except DC9, cold boot,
* warm reset, PCI function level reset, and hibernate/suspend.
* If firmware is restored by dmc then no need to load again which
* will keep the dc5/dc6 counter exposed by firmware. */

Quoted 5 lines
Quoted 1 line

No issue in init sequence.

That seems to still cover all the callers of the function afaics - we do pci resets over suspend resume unconditionally. So I still don't understand where exactly we try to load the dmc firmware in i915.ko when it's already loaded.

During resume intel_csr_load_program() will be called from intel_runtime_resume().

intel_runtime_resume()-> skl_resume_prepare()-> intel_csr_load_program()

During Pc10 entry testing I can see dmc is restoring back the firmware always, but as you mentioned pci-reset can happen unconditionally, but still then also during resume intel_runtime_resume() will be called and based on register read of csr-base-address firmware loading will happen.

Quoted 8 lines
Quoted 4 lines

But in your comment you're saying it won't get restored in case of dc9 and suspend. So that seems to mismatch what you're saying here (and what the commit message says) and what the code does. And this function here is called for resume after suspend/hibernate only.

pc10 entry explanation I told is for skylake. dc9 in skylake is not possible. I think you are confusing between dc6 and dc9. Pc10 can be achieved by entering into dc6 (not dc9) for skylake. dc9 is the lowest possible state for broxton which is not present for skylake.

I have no idea at all about different pc levels on skl. What I'm talking about is system suspend/resume and driver load, which are the places this function gets called. At least afaics.

Quoted 3 lines

Here intel_csr_load_program() will be used for both skylake and broxton, and instruction execution flow will be different in case of suspend/resume which I think is confusing you.

That seems like really important information. What's different on bxt? These are the kind of details you should explain in the commit message ...

Quoted 5 lines

I am ready explain you in detail. It will be good if we discuss specific use-case scenario and itz software design for specific platform. Another point - as dmc related code for broxton is not merged better first we close design for skylake. Now, I have added dc9 description in comment thinking of future. If you want I can remove for now and later can add in bxt patch series for enabling dmc. Will wait for your reply.

This question here isn't about the overall design and how to handle power wells in skl/bxt. That's a separate discussion and tracked somewhere else. I'm really just confused about when exactly we need to reload to firmware, and why we need a runtime check for that. Normally we should know when to reload the firmware and just either reload or not, without checking hw state. And I don't like checking for hw state since at least in the past that kind of code ended up being fragile - it's an illusion that it does the right thing no matter what, since often there's other tricky ordering constraints. And if you have automatic duct-tape like then no one will ever spot those other, harder to spot issues, until an expensive customer escalation happens.

Quoted 2 lines

So what I want to know here is:
- When exactly do we need to reload dmc firmware.

In skl, during driver load first time we load the firmware, during normal suspend-resume (dc6 entry/exit)
no need to reload the firmware again as dmc will take care of it. But during suspend/hibernation
dmc will not restore the firmware. In that case driver need to reload it again. I do not know
how to differentiate pm-suspend and suspend-hibernation and thought both the cases
intel_runtime_resume() will be called where we can check the h/w state and reload the
firmware if dmc is not restored.

In bxt, during driver load first time we load the firmware, during normal suspend-resume
display engine will enter into dc9 and dmc will not restore the firmware. So every
suspend-resume we need to reload the firmware.

Quoted 8 lines
Quoted 2 lines

- What exactly is the reason why we can't make that decision statically in the code (by calling csr_load at the right spots).

As I mentioned before in case of skylake can we differentiate between "resume from pm-suspend" with "resume from suspend-hibernation" inside driver?

In case of broxton, every time we need to reload, so we can decide statically.

Of course we can differentiate between all the different resume paths, and we also have a per-platform split to take care of bxt vs. skl. And there are actually 3 different resume paths:

- runtime PM resume. This calls the runtime_resume hook. It sounds like on skl we should _not_ load the csr firmware, but on bxt we should load it. This can be fixed by removing the intel_csr_load_program call from skl_resume_prepare.
- resume from hibernate-to-disk (i.e. system completely off, state stored on the swap partition) is done by calling the thaw callbacks.
- resume from suspend-to-mem (i.e. system in low-power with only memory in self-refresh, all state stored in memory) is done by calling the resume callbacks.

For i915 we use unified handlers in our dev_pm_ops for both thaw and resume, but it sounds like that won't be a problem for skl/bxt since we need to reload the csr firmware in all cases. Although I'm not perfectly sure since you don't explain what kind of resume you mean exactly (since you don't use the linux names for them).

Anyway it sounds like we can replace this patch by one where we remove that errornous csr load call from skl runtime pm resume and that's all. But I suggest to make sure we get this right we keep the check you're adding here, but wrap it in a WARN_ON. Then we'll get a backtrace when this is going wrong again. Like this:

Quoted 6 lines
        if (WARN_ON(csr_loaded_already()))
                return;

Also when redoing the commits please explain in detail what exactly are the requirements like you've done above, but please use the standard linux names, i.e. "runtime PM" and "hibernate-to-disk" and "suspend-to-mem".

Ok hooray there's more suspend-to-something things I've totally missed:
- suspend-to-idle (done by cat freeze > /sys/power/state) and
- suspend (done by cat suspend > /sys/power/state)

And apparently there's really no way to drivers to tell them apart. Rafael, is there really no way for drivers to take different paths for these 3 suspend cases? I tried grepping for PM_SUSPEND_ON/STANDY/MEM and didn't spot anything.

Quoted 9 lines

Also we're completely missing test coverage for that in igt. That is something that needs to be fixed asap (yet another case of combinatorial explosion in igt tests, yay). And at least one of those suspend-to-idle testcase better be in the BAT.
-Daniel
--
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch

--
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch

Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#15 lore raw

Re: [DMC_BUGFIX_SKL_V2 1/5] drm/i915/skl: Added a check for the hardware status of csr fw before loading.

Quoted 4 lines · Daniel Vetter

acpi_target_system_state() seems to be almost the thing we're looking for, except that it's only valid in the suspend callbacks since it gets reset to ACPI_STATE_S0 when resuming. So probably we want something else ...

Right.

The idea is to add a way for drivers to check if
(a) suspend is going to enter the BIOS
(b) resume has been triggered by the BIOS and that's really what drivers need to know.

For suspend-to-idle those two will return false and for S3 they'll return true.

Would that help?

Thanks,
Rafael

Quoted 210 lines · Daniel Vetter
Quoted 209 lines

Actually add Rafael this time around ...
-Daniel

Quoted 203 lines · Daniel Vetter
Quoted 178 lines · Daniel Vetter
Quoted 143 lines · Animesh Manna
Quoted 122 lines · Daniel Vetter
Quoted 86 lines · Animesh Manna
Quoted 85 lines · Daniel Vetter
Quoted 84 lines · Animesh Manna
Quoted 69 lines · Daniel Vetter
Quoted 68 lines · Animesh Manna
Quoted 7 lines · Daniel Vetter
Quoted 6 lines · Animesh Manna

Dmc will restore the csr program except DC9, cold boot,
warm reset, PCI function level reset, and hibernate/suspend.

intel_csr_load_program() function is used to load the firmware
data from kernel memory to csr address space.

Quoted 7 lines
Quoted 7 lines

All values of csr address space will be zero if it got reset and
the first byte of csr program is always a non-zero if firmware
is loaded successfuly. Based on hardware status will load the
firmware.

Without this condition check if we overwrite the firmware data the
counters exposed for dc5/dc6 (help for debugging) will be nullified.

Bacause of the above reason mentioned just above we need to block firmware loading again. So only WARN_ON will not help.

Quoted 24 lines
Quoted 24 lines

v1: Initial version.

v2: Based on review comments from Daniel,
- Added a check to know hardware status and load the firmware if not loaded.

Cc: Daniel Vetter <[email protected]>
Cc: Damien Lespiau <[email protected]>
Cc: Imre Deak <[email protected]>
Cc: Sunil Kamath <[email protected]>
Signed-off-by: Animesh Manna <[email protected]>
Signed-off-by: Vathsala Nagaraju <[email protected]>
---
drivers/gpu/drm/i915/intel_csr.c | 9 +++++++++
1 file changed, 9 insertions(+)

diff --git a/drivers/gpu/drm/i915/intel_csr.c b/drivers/gpu/drm/i915/intel_csr.c
index ba1ae03..682cc26 100644
--- a/drivers/gpu/drm/i915/intel_csr.c
+++ b/drivers/gpu/drm/i915/intel_csr.c
@@ -252,6 +252,15 @@ void intel_csr_load_program(struct drm_device *dev)
return;
}
+ /*
+ * Dmc will restore the csr the program except DC9, cold boot,

Quoted 8 lines
Quoted 4 lines

+ * warm reset, PCI function level reset, and hibernate/suspend.
+ * This condition will help to check if csr address space is reset/
+ * not loaded.
+ */

Atm we call this from driver load and resume, which doesn seem to cover all the cases you mention in the comment. Should this be a WARN_ON instead? Or do we have troubles in our init sequence where we load too many times?

Yes, the above statement taken from bspec to describe about the special cases dmc will not restore the firmware. Agree, In our cases cold boot and hibernate/suspend mainly we need to load the firmware again, so in my second sentence I wanted to comment mainly regarding this condition check added for suspend-hibernate(reset) and cold boot(not loaded).

Anyways the same api later can be used to load the firmware from anywhere, so my intention to check firmware loaded or not. If already loaded then not to overwrite the csr address space to maintain the dc5/dc6 counter value.

Can the below comment more clear to you.

/*
* Dmc will restore the csr the program except DC9, cold boot,
* warm reset, PCI function level reset, and hibernate/suspend.
* If firmware is restored by dmc then no need to load again which
* will keep the dc5/dc6 counter exposed by firmware. */

Quoted 5 lines
Quoted 1 line

No issue in init sequence.

That seems to still cover all the callers of the function afaics - we do pci resets over suspend resume unconditionally. So I still don't understand where exactly we try to load the dmc firmware in i915.ko when it's already loaded.

During resume intel_csr_load_program() will be called from intel_runtime_resume().

intel_runtime_resume()-> skl_resume_prepare()-> intel_csr_load_program()

During Pc10 entry testing I can see dmc is restoring back the firmware always, but as you mentioned pci-reset can happen unconditionally, but still then also during resume intel_runtime_resume() will be called and based on register read of csr-base-address firmware loading will happen.

Quoted 8 lines
Quoted 4 lines

But in your comment you're saying it won't get restored in case of dc9 and suspend. So that seems to mismatch what you're saying here (and what the commit message says) and what the code does. And this function here is called for resume after suspend/hibernate only.

pc10 entry explanation I told is for skylake. dc9 in skylake is not possible. I think you are confusing between dc6 and dc9. Pc10 can be achieved by entering into dc6 (not dc9) for skylake. dc9 is the lowest possible state for broxton which is not present for skylake.

I have no idea at all about different pc levels on skl. What I'm talking about is system suspend/resume and driver load, which are the places this function gets called. At least afaics.

Quoted 3 lines

Here intel_csr_load_program() will be used for both skylake and broxton, and instruction execution flow will be different in case of suspend/resume which I think is confusing you.

That seems like really important information. What's different on bxt? These are the kind of details you should explain in the commit message ...

Quoted 5 lines

I am ready explain you in detail. It will be good if we discuss specific use-case scenario and itz software design for specific platform. Another point - as dmc related code for broxton is not merged better first we close design for skylake. Now, I have added dc9 description in comment thinking of future. If you want I can remove for now and later can add in bxt patch series for enabling dmc. Will wait for your reply.

This question here isn't about the overall design and how to handle power wells in skl/bxt. That's a separate discussion and tracked somewhere else. I'm really just confused about when exactly we need to reload to firmware, and why we need a runtime check for that. Normally we should know when to reload the firmware and just either reload or not, without checking hw state. And I don't like checking for hw state since at least in the past that kind of code ended up being fragile - it's an illusion that it does the right thing no matter what, since often there's other tricky ordering constraints. And if you have automatic duct-tape like then no one will ever spot those other, harder to spot issues, until an expensive customer escalation happens.

Quoted 2 lines

So what I want to know here is:
- When exactly do we need to reload dmc firmware.

In skl, during driver load first time we load the firmware, during normal suspend-resume (dc6 entry/exit)
no need to reload the firmware again as dmc will take care of it. But during suspend/hibernation
dmc will not restore the firmware. In that case driver need to reload it again. I do not know
how to differentiate pm-suspend and suspend-hibernation and thought both the cases
intel_runtime_resume() will be called where we can check the h/w state and reload the
firmware if dmc is not restored.

In bxt, during driver load first time we load the firmware, during normal suspend-resume
display engine will enter into dc9 and dmc will not restore the firmware. So every
suspend-resume we need to reload the firmware.

Quoted 8 lines
Quoted 2 lines

- What exactly is the reason why we can't make that decision statically in the code (by calling csr_load at the right spots).

As I mentioned before in case of skylake can we differentiate between "resume from pm-suspend" with "resume from suspend-hibernation" inside driver?

In case of broxton, every time we need to reload, so we can decide statically.

Of course we can differentiate between all the different resume paths, and we also have a per-platform split to take care of bxt vs. skl. And there are actually 3 different resume paths:

- runtime PM resume. This calls the runtime_resume hook. It sounds like on skl we should _not_ load the csr firmware, but on bxt we should load it. This can be fixed by removing the intel_csr_load_program call from skl_resume_prepare.
- resume from hibernate-to-disk (i.e. system completely off, state stored on the swap partition) is done by calling the thaw callbacks.
- resume from suspend-to-mem (i.e. system in low-power with only memory in self-refresh, all state stored in memory) is done by calling the resume callbacks.

For i915 we use unified handlers in our dev_pm_ops for both thaw and resume, but it sounds like that won't be a problem for skl/bxt since we need to reload the csr firmware in all cases. Although I'm not perfectly sure since you don't explain what kind of resume you mean exactly (since you don't use the linux names for them).

Anyway it sounds like we can replace this patch by one where we remove that errornous csr load call from skl runtime pm resume and that's all. But I suggest to make sure we get this right we keep the check you're adding here, but wrap it in a WARN_ON. Then we'll get a backtrace when this is going wrong again. Like this:

Quoted 6 lines
         if (WARN_ON(csr_loaded_already()))
                 return;

Also when redoing the commits please explain in detail what exactly are the requirements like you've done above, but please use the standard linux names, i.e. "runtime PM" and "hibernate-to-disk" and "suspend-to-mem".

Ok hooray there's more suspend-to-something things I've totally missed:
- suspend-to-idle (done by cat freeze > /sys/power/state) and
- suspend (done by cat suspend > /sys/power/state)

And apparently there's really no way to drivers to tell them apart. Rafael, is there really no way for drivers to take different paths for these 3 suspend cases? I tried grepping for PM_SUSPEND_ON/STANDY/MEM and didn't spot anything.

Also we're completely missing test coverage for that in igt. That is something that needs to be fixed asap (yet another case of combinatorial explosion in igt tests, yay). And at least one of those suspend-to-idle testcase better be in the BAT.
-Daniel
--
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch

Quoted 6 lines
Quoted 4 lines

--
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

#16 lore raw

Re: [DMC_BUGFIX_SKL_V2 1/5] drm/i915/skl: Added a check for the hardware status of csr fw before loading.

Quoted 13 lines · Rafael J. Wysocki
Quoted 4 lines · Daniel Vetter

acpi_target_system_state() seems to be almost the thing we're looking for, except that it's only valid in the suspend callbacks since it gets reset to ACPI_STATE_S0 when resuming. So probably we want something else ...

Right.

The idea is to add a way for drivers to check if
(a) suspend is going to enter the BIOS
(b) resume has been triggered by the BIOS and that's really what drivers need to know.

Quoted 4 lines

For suspend-to-idle those two will return false and for S3 they'll return true.

Would that help?

Not sure that matches exaxtly what we'd need here ... Essentially we need to know whether we've been in S3/S4 (firmware has been eaten) or in one of the higher suspend-to-idle/standby states (firmware still alive, don't disturb it). Additional fun that just crossed my mind is that if the suspend-to-mem is aborted (some other driver failed) then that function should _not_ indicate that we've been in S3. So maybe something like

acpi_source_system_state() which usually is S0 and only when acpi successfully went into the suspend state in platform_suspend_ops->enter it gets set to the value of acpi_target_system_state. And then reset once the resume has completed. I think that would be what we'd want here.

Anyway I'll pull in Animesh series meanwhile, amended with a FIXME comment.
-Daniel

Quoted 222 lines

Thanks,
Rafael

Quoted 216 lines · Daniel Vetter
Quoted 213 lines

Actually add Rafael this time around ...
-Daniel

Quoted 194 lines · Daniel Vetter
Quoted 178 lines · Daniel Vetter
Quoted 143 lines · Animesh Manna
Quoted 122 lines · Daniel Vetter
Quoted 86 lines · Animesh Manna
Quoted 85 lines · Daniel Vetter
Quoted 84 lines · Animesh Manna
Quoted 69 lines · Daniel Vetter
Quoted 68 lines · Animesh Manna
Quoted 14 lines · Daniel Vetter

On Wed, Aug 26, 2015 at 01:36:05AM +0530, Animesh Manna wrote:
Dmc will restore the csr program except DC9, cold boot,
warm reset, PCI function level reset, and hibernate/suspend.

intel_csr_load_program() function is used to load the firmware
data from kernel memory to csr address space.

All values of csr address space will be zero if it got reset and
the first byte of csr program is always a non-zero if firmware
is loaded successfuly. Based on hardware status will load the
firmware.

Without this condition check if we overwrite the firmware data the
counters exposed for dc5/dc6 (help for debugging) will be nullified.

Bacause of the above reason mentioned just above we need to block firmware loading again. So only WARN_ON will not help.

Quoted 32 lines

v1: Initial version.

v2: Based on review comments from Daniel,
- Added a check to know hardware status and load the firmware if not loaded.

Cc: Daniel Vetter <[email protected]>
Cc: Damien Lespiau <[email protected]>
Cc: Imre Deak <[email protected]>
Cc: Sunil Kamath <[email protected]>
Signed-off-by: Animesh Manna <[email protected]>
Signed-off-by: Vathsala Nagaraju <[email protected]>
---
drivers/gpu/drm/i915/intel_csr.c | 9 +++++++++
1 file changed, 9 insertions(+)

diff --git a/drivers/gpu/drm/i915/intel_csr.c b/drivers/gpu/drm/i915/intel_csr.c
index ba1ae03..682cc26 100644
--- a/drivers/gpu/drm/i915/intel_csr.c
+++ b/drivers/gpu/drm/i915/intel_csr.c
@@ -252,6 +252,15 @@ void intel_csr_load_program(struct drm_device *dev)
return;
}
+ /*
+ * Dmc will restore the csr the program except DC9, cold boot,
+ * warm reset, PCI function level reset, and hibernate/suspend.
+ * This condition will help to check if csr address space is reset/
+ * not loaded.
+ */
Atm we call this from driver load and resume, which doesn seem to cover
all the cases you mention in the comment. Should this be a WARN_ON
instead? Or do we have troubles in our init sequence where we load too
many times?

Yes, the above statement taken from bspec to describe about the special cases dmc will not restore the firmware. Agree, In our cases cold boot and hibernate/suspend mainly we need to load the firmware again, so in my second sentence I wanted to comment mainly regarding this condition check added for suspend-hibernate(reset) and cold boot(not loaded).

Anyways the same api later can be used to load the firmware from anywhere, so my intention to check firmware loaded or not. If already loaded then not to overwrite the csr address space to maintain the dc5/dc6 counter value.

Can the below comment more clear to you.

/*
* Dmc will restore the csr the program except DC9, cold boot,
* warm reset, PCI function level reset, and hibernate/suspend.
* If firmware is restored by dmc then no need to load again which
* will keep the dc5/dc6 counter exposed by firmware. */

Quoted 5 lines
Quoted 1 line

No issue in init sequence.

That seems to still cover all the callers of the function afaics - we do pci resets over suspend resume unconditionally. So I still don't understand where exactly we try to load the dmc firmware in i915.ko when it's already loaded.

During resume intel_csr_load_program() will be called from intel_runtime_resume().

intel_runtime_resume()-> skl_resume_prepare()-> intel_csr_load_program()

During Pc10 entry testing I can see dmc is restoring back the firmware always, but as you mentioned pci-reset can happen unconditionally, but still then also during resume intel_runtime_resume() will be called and based on register read of csr-base-address firmware loading will happen.

Quoted 8 lines
Quoted 4 lines

But in your comment you're saying it won't get restored in case of dc9 and suspend. So that seems to mismatch what you're saying here (and what the commit message says) and what the code does. And this function here is called for resume after suspend/hibernate only.

pc10 entry explanation I told is for skylake. dc9 in skylake is not possible. I think you are confusing between dc6 and dc9. Pc10 can be achieved by entering into dc6 (not dc9) for skylake. dc9 is the lowest possible state for broxton which is not present for skylake.

I have no idea at all about different pc levels on skl. What I'm talking about is system suspend/resume and driver load, which are the places this function gets called. At least afaics.

Quoted 3 lines

Here intel_csr_load_program() will be used for both skylake and broxton, and instruction execution flow will be different in case of suspend/resume which I think is confusing you.

That seems like really important information. What's different on bxt? These are the kind of details you should explain in the commit message ...

Quoted 5 lines

I am ready explain you in detail. It will be good if we discuss specific use-case scenario and itz software design for specific platform. Another point - as dmc related code for broxton is not merged better first we close design for skylake. Now, I have added dc9 description in comment thinking of future. If you want I can remove for now and later can add in bxt patch series for enabling dmc. Will wait for your reply.

This question here isn't about the overall design and how to handle power wells in skl/bxt. That's a separate discussion and tracked somewhere else. I'm really just confused about when exactly we need to reload to firmware, and why we need a runtime check for that. Normally we should know when to reload the firmware and just either reload or not, without checking hw state. And I don't like checking for hw state since at least in the past that kind of code ended up being fragile - it's an illusion that it does the right thing no matter what, since often there's other tricky ordering constraints. And if you have automatic duct-tape like then no one will ever spot those other, harder to spot issues, until an expensive customer escalation happens.

Quoted 2 lines

So what I want to know here is:
- When exactly do we need to reload dmc firmware.

In skl, during driver load first time we load the firmware, during normal suspend-resume (dc6 entry/exit)
no need to reload the firmware again as dmc will take care of it. But during suspend/hibernation
dmc will not restore the firmware. In that case driver need to reload it again. I do not know
how to differentiate pm-suspend and suspend-hibernation and thought both the cases
intel_runtime_resume() will be called where we can check the h/w state and reload the
firmware if dmc is not restored.

In bxt, during driver load first time we load the firmware, during normal suspend-resume
display engine will enter into dc9 and dmc will not restore the firmware. So every
suspend-resume we need to reload the firmware.

Quoted 8 lines
Quoted 2 lines

- What exactly is the reason why we can't make that decision statically in the code (by calling csr_load at the right spots).

As I mentioned before in case of skylake can we differentiate between "resume from pm-suspend" with "resume from suspend-hibernation" inside driver?

In case of broxton, every time we need to reload, so we can decide statically.

Of course we can differentiate between all the different resume paths, and we also have a per-platform split to take care of bxt vs. skl. And there are actually 3 different resume paths:

- runtime PM resume. This calls the runtime_resume hook. It sounds like on skl we should _not_ load the csr firmware, but on bxt we should load it. This can be fixed by removing the intel_csr_load_program call from skl_resume_prepare.
- resume from hibernate-to-disk (i.e. system completely off, state stored on the swap partition) is done by calling the thaw callbacks.
- resume from suspend-to-mem (i.e. system in low-power with only memory in self-refresh, all state stored in memory) is done by calling the resume callbacks.

For i915 we use unified handlers in our dev_pm_ops for both thaw and resume, but it sounds like that won't be a problem for skl/bxt since we need to reload the csr firmware in all cases. Although I'm not perfectly sure since you don't explain what kind of resume you mean exactly (since you don't use the linux names for them).

Anyway it sounds like we can replace this patch by one where we remove that errornous csr load call from skl runtime pm resume and that's all. But I suggest to make sure we get this right we keep the check you're adding here, but wrap it in a WARN_ON. Then we'll get a backtrace when this is going wrong again. Like this:

Quoted 6 lines
        if (WARN_ON(csr_loaded_already()))
                return;

Also when redoing the commits please explain in detail what exactly are the requirements like you've done above, but please use the standard linux names, i.e. "runtime PM" and "hibernate-to-disk" and "suspend-to-mem".

Ok hooray there's more suspend-to-something things I've totally missed:
- suspend-to-idle (done by cat freeze > /sys/power/state) and
- suspend (done by cat suspend > /sys/power/state)

And apparently there's really no way to drivers to tell them apart. Rafael, is there really no way for drivers to take different paths for these 3 suspend cases? I tried grepping for PM_SUSPEND_ON/STANDY/MEM and didn't spot anything.

Quoted 9 lines

Also we're completely missing test coverage for that in igt. That is something that needs to be fixed asap (yet another case of combinatorial explosion in igt tests, yay). And at least one of those suspend-to-idle testcase better be in the BAT.
-Daniel
--
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch

--
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch

Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#17 lore raw

Re: [DMC_BUGFIX_SKL_V2 1/5] drm/i915/skl: Added a check for the hardware status of csr fw before loading.

Quoted 16 lines · Daniel Vetter
Quoted 15 lines · Rafael J. Wysocki
Quoted 4 lines · Daniel Vetter

acpi_target_system_state() seems to be almost the thing we're looking for, except that it's only valid in the suspend callbacks since it gets reset to ACPI_STATE_S0 when resuming. So probably we want something else ...

Right.

The idea is to add a way for drivers to check if
(a) suspend is going to enter the BIOS
(b) resume has been triggered by the BIOS and that's really what drivers need to know.

For suspend-to-idle those two will return false and for S3 they'll return true.

Quoted 7 lines
Quoted 1 line

Would that help?

Not sure that matches exaxtly what we'd need here ... Essentially we need to know whether we've been in S3/S4 (firmware has been eaten) or in one of the higher suspend-to-idle/standby states (firmware still alive, don't disturb it). Additional fun that just crossed my mind is that if the suspend-to-mem is aborted (some other driver failed) then that function should _not_ indicate that we've been in S3. So maybe something like

So it really looks like the interface I was talking about would be suitable:

pm_suspend_via_firmware() == true -> firmware is going to be eaten (use that during suspend if needed) pm_resume_via_firmware() == true -> firmware was eaten

The latter will only return 'true' if we really have entered the BIOS (platform firmware).

Quoted 4 lines

acpi_source_system_state() which usually is S0 and only when acpi successfully went into the suspend state in platform_suspend_ops->enter it gets set to the value of acpi_target_system_state. And then reset once the resume has completed. I think that would be what we'd want here.

We need to new functions like the above, because some things already depend on acpi_target_system_state working the way it does currently.

I see no reason to make that ACPI-specific, though, in principle.

Quoted 1 line

Anyway I'll pull in Animesh series meanwhile, amended with a FIXME comment.

Fine by me.

Thanks,
Rafael

Quoted 208 lines
Quoted 208 lines
Quoted 208 lines · Daniel Vetter
Quoted 207 lines

Actually add Rafael this time around ...
-Daniel

Quoted 202 lines · Daniel Vetter
Quoted 177 lines · Daniel Vetter
Quoted 142 lines · Animesh Manna
Quoted 122 lines · Daniel Vetter
Quoted 86 lines · Animesh Manna
Quoted 85 lines · Daniel Vetter
Quoted 84 lines · Animesh Manna
Quoted 69 lines · Daniel Vetter
Quoted 68 lines · Animesh Manna

On 8/26/2015 6:40 PM, Daniel Vetter wrote:
On Wed, Aug 26, 2015 at 01:36:05AM +0530, Animesh Manna wrote:
Dmc will restore the csr program except DC9, cold boot,
warm reset, PCI function level reset, and hibernate/suspend.

intel_csr_load_program() function is used to load the firmware
data from kernel memory to csr address space.

All values of csr address space will be zero if it got reset and
the first byte of csr program is always a non-zero if firmware
is loaded successfuly. Based on hardware status will load the
firmware.

Without this condition check if we overwrite the firmware data the
counters exposed for dc5/dc6 (help for debugging) will be nullified.
Bacause of the above reason mentioned just above we need to block firmware loading again.
So only WARN_ON will not help.

v1: Initial version.

v2: Based on review comments from Daniel,
- Added a check to know hardware status and load the firmware if not loaded.

Cc: Daniel Vetter <[email protected]>
Cc: Damien Lespiau <[email protected]>
Cc: Imre Deak <[email protected]>
Cc: Sunil Kamath <[email protected]>
Signed-off-by: Animesh Manna <[email protected]>
Signed-off-by: Vathsala Nagaraju <[email protected]>
---
drivers/gpu/drm/i915/intel_csr.c | 9 +++++++++
1 file changed, 9 insertions(+)

diff --git a/drivers/gpu/drm/i915/intel_csr.c b/drivers/gpu/drm/i915/intel_csr.c
index ba1ae03..682cc26 100644
--- a/drivers/gpu/drm/i915/intel_csr.c
+++ b/drivers/gpu/drm/i915/intel_csr.c
@@ -252,6 +252,15 @@ void intel_csr_load_program(struct drm_device *dev)
return;
}
+ /*
+ * Dmc will restore the csr the program except DC9, cold boot,
+ * warm reset, PCI function level reset, and hibernate/suspend.
+ * This condition will help to check if csr address space is reset/
+ * not loaded.
+ */
Atm we call this from driver load and resume, which doesn seem to cover
all the cases you mention in the comment. Should this be a WARN_ON
instead? Or do we have troubles in our init sequence where we load too
many times?
Yes, the above statement taken from bspec to describe about the special cases dmc will not restore the firmware.
Agree, In our cases cold boot and hibernate/suspend mainly we need to load the firmware again, so in my
second sentence I wanted to comment mainly regarding this condition check added for suspend-hibernate(reset)
and cold boot(not loaded).

Anyways the same api later can be used to load the firmware from anywhere, so my intention to check firmware loaded or not.
If already loaded then not to overwrite the csr address space to maintain the dc5/dc6 counter value.

Can the below comment more clear to you.

/*
* Dmc will restore the csr the program except DC9, cold boot,
* warm reset, PCI function level reset, and hibernate/suspend.
* If firmware is restored by dmc then no need to load again which
* will keep the dc5/dc6 counter exposed by firmware.
*/

Quoted 5 lines
Quoted 1 line

No issue in init sequence.

That seems to still cover all the callers of the function afaics - we do pci resets over suspend resume unconditionally. So I still don't understand where exactly we try to load the dmc firmware in i915.ko when it's already loaded.

During resume intel_csr_load_program() will be called from intel_runtime_resume().

intel_runtime_resume()-> skl_resume_prepare()-> intel_csr_load_program()

During Pc10 entry testing I can see dmc is restoring back the firmware always, but as you mentioned pci-reset can happen unconditionally, but still then also during resume intel_runtime_resume() will be called and based on register read of csr-base-address firmware loading will happen.

Quoted 8 lines
Quoted 4 lines

But in your comment you're saying it won't get restored in case of dc9 and suspend. So that seems to mismatch what you're saying here (and what the commit message says) and what the code does. And this function here is called for resume after suspend/hibernate only.

pc10 entry explanation I told is for skylake. dc9 in skylake is not possible. I think you are confusing between dc6 and dc9. Pc10 can be achieved by entering into dc6 (not dc9) for skylake. dc9 is the lowest possible state for broxton which is not present for skylake.

I have no idea at all about different pc levels on skl. What I'm talking about is system suspend/resume and driver load, which are the places this function gets called. At least afaics.

Quoted 3 lines

Here intel_csr_load_program() will be used for both skylake and broxton, and instruction execution flow will be different in case of suspend/resume which I think is confusing you.

That seems like really important information. What's different on bxt? These are the kind of details you should explain in the commit message ...

Quoted 5 lines

I am ready explain you in detail. It will be good if we discuss specific use-case scenario and itz software design for specific platform. Another point - as dmc related code for broxton is not merged better first we close design for skylake. Now, I have added dc9 description in comment thinking of future. If you want I can remove for now and later can add in bxt patch series for enabling dmc. Will wait for your reply.

This question here isn't about the overall design and how to handle power wells in skl/bxt. That's a separate discussion and tracked somewhere else. I'm really just confused about when exactly we need to reload to firmware, and why we need a runtime check for that. Normally we should know when to reload the firmware and just either reload or not, without checking hw state. And I don't like checking for hw state since at least in the past that kind of code ended up being fragile - it's an illusion that it does the right thing no matter what, since often there's other tricky ordering constraints. And if you have automatic duct-tape like then no one will ever spot those other, harder to spot issues, until an expensive customer escalation happens.

Quoted 2 lines

So what I want to know here is:
- When exactly do we need to reload dmc firmware.

In skl, during driver load first time we load the firmware, during normal suspend-resume (dc6 entry/exit)
no need to reload the firmware again as dmc will take care of it. But during suspend/hibernation
dmc will not restore the firmware. In that case driver need to reload it again. I do not know
how to differentiate pm-suspend and suspend-hibernation and thought both the cases
intel_runtime_resume() will be called where we can check the h/w state and reload the
firmware if dmc is not restored.

In bxt, during driver load first time we load the firmware, during normal suspend-resume
display engine will enter into dc9 and dmc will not restore the firmware. So every
suspend-resume we need to reload the firmware.

Quoted 8 lines
Quoted 2 lines

- What exactly is the reason why we can't make that decision statically in the code (by calling csr_load at the right spots).

As I mentioned before in case of skylake can we differentiate between "resume from pm-suspend" with "resume from suspend-hibernation" inside driver?

In case of broxton, every time we need to reload, so we can decide statically.

Of course we can differentiate between all the different resume paths, and we also have a per-platform split to take care of bxt vs. skl. And there are actually 3 different resume paths:

- runtime PM resume. This calls the runtime_resume hook. It sounds like on skl we should _not_ load the csr firmware, but on bxt we should load it. This can be fixed by removing the intel_csr_load_program call from skl_resume_prepare.
- resume from hibernate-to-disk (i.e. system completely off, state stored on the swap partition) is done by calling the thaw callbacks.
- resume from suspend-to-mem (i.e. system in low-power with only memory in self-refresh, all state stored in memory) is done by calling the resume callbacks.

For i915 we use unified handlers in our dev_pm_ops for both thaw and resume, but it sounds like that won't be a problem for skl/bxt since we need to reload the csr firmware in all cases. Although I'm not perfectly sure since you don't explain what kind of resume you mean exactly (since you don't use the linux names for them).

Anyway it sounds like we can replace this patch by one where we remove that errornous csr load call from skl runtime pm resume and that's all. But I suggest to make sure we get this right we keep the check you're adding here, but wrap it in a WARN_ON. Then we'll get a backtrace when this is going wrong again. Like this:

Quoted 6 lines
         if (WARN_ON(csr_loaded_already()))
                 return;

Also when redoing the commits please explain in detail what exactly are the requirements like you've done above, but please use the standard linux names, i.e. "runtime PM" and "hibernate-to-disk" and "suspend-to-mem".

Ok hooray there's more suspend-to-something things I've totally missed:
- suspend-to-idle (done by cat freeze > /sys/power/state) and
- suspend (done by cat suspend > /sys/power/state)

And apparently there's really no way to drivers to tell them apart. Rafael, is there really no way for drivers to take different paths for these 3 suspend cases? I tried grepping for PM_SUSPEND_ON/STANDY/MEM and didn't spot anything.

Also we're completely missing test coverage for that in igt. That is something that needs to be fixed asap (yet another case of combinatorial explosion in igt tests, yay). And at least one of those suspend-to-idle testcase better be in the BAT.
-Daniel
--
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 4 lines

--
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

#18 lore raw

Re: [DMC_BUGFIX_SKL_V2 1/5] drm/i915/skl: Added a check for the hardware status of csr fw before loading.

Quoted 25 lines · Rafael J. Wysocki
Quoted 16 lines · Daniel Vetter
Quoted 15 lines · Rafael J. Wysocki
Quoted 4 lines · Daniel Vetter

acpi_target_system_state() seems to be almost the thing we're looking for, except that it's only valid in the suspend callbacks since it gets reset to ACPI_STATE_S0 when resuming. So probably we want something else ...

Right.

The idea is to add a way for drivers to check if
(a) suspend is going to enter the BIOS
(b) resume has been triggered by the BIOS and that's really what drivers need to know.

For suspend-to-idle those two will return false and for S3 they'll return true.

Quoted 7 lines
Quoted 1 line

Would that help?

Not sure that matches exaxtly what we'd need here ... Essentially we need to know whether we've been in S3/S4 (firmware has been eaten) or in one of the higher suspend-to-idle/standby states (firmware still alive, don't disturb it). Additional fun that just crossed my mind is that if the suspend-to-mem is aborted (some other driver failed) then that function should _not_ indicate that we've been in S3. So maybe something like

Quoted 8 lines

So it really looks like the interface I was talking about would be suitable:

pm_suspend_via_firmware() == true -> firmware is going to be eaten (use that during suspend if needed)
pm_resume_via_firmware() == true -> firmware was eaten

The latter will only return 'true' if we really have entered the BIOS (platform firmware).

Yeah that seems to fit the bill. We already have a check in our suspend paths to figure out whether we'll suspend to idle or go into full S3, so i915 could use them both. And making them generic would make sense I guess.
-Daniel

Quoted 232 lines
Quoted 4 lines

acpi_source_system_state() which usually is S0 and only when acpi successfully went into the suspend state in platform_suspend_ops->enter it gets set to the value of acpi_target_system_state. And then reset once the resume has completed. I think that would be what we'd want here.

We need to new functions like the above, because some things already depend on acpi_target_system_state working the way it does currently.

I see no reason to make that ACPI-specific, though, in principle.

Quoted 1 line

Anyway I'll pull in Animesh series meanwhile, amended with a FIXME comment.

Fine by me.

Thanks,
Rafael

Quoted 213 lines
Quoted 213 lines
Quoted 213 lines · Daniel Vetter
Quoted 211 lines

Actually add Rafael this time around ...
-Daniel

Quoted 193 lines · Daniel Vetter
Quoted 177 lines · Daniel Vetter
Quoted 142 lines · Animesh Manna
Quoted 122 lines · Daniel Vetter
Quoted 86 lines · Animesh Manna
Quoted 85 lines · Daniel Vetter
Quoted 84 lines · Animesh Manna
Quoted 74 lines · Daniel Vetter

On Wed, Aug 26, 2015 at 07:40:54PM +0530, Animesh Manna wrote:
On 8/26/2015 6:40 PM, Daniel Vetter wrote:
On Wed, Aug 26, 2015 at 01:36:05AM +0530, Animesh Manna wrote:
Dmc will restore the csr program except DC9, cold boot,
warm reset, PCI function level reset, and hibernate/suspend.

intel_csr_load_program() function is used to load the firmware
data from kernel memory to csr address space.

All values of csr address space will be zero if it got reset and
the first byte of csr program is always a non-zero if firmware
is loaded successfuly. Based on hardware status will load the
firmware.

Without this condition check if we overwrite the firmware data the
counters exposed for dc5/dc6 (help for debugging) will be nullified.
Bacause of the above reason mentioned just above we need to block firmware loading again.
So only WARN_ON will not help.

v1: Initial version.

v2: Based on review comments from Daniel,
- Added a check to know hardware status and load the firmware if not loaded.

Cc: Daniel Vetter <[email protected]>
Cc: Damien Lespiau <[email protected]>
Cc: Imre Deak <[email protected]>
Cc: Sunil Kamath <[email protected]>
Signed-off-by: Animesh Manna <[email protected]>
Signed-off-by: Vathsala Nagaraju <[email protected]>
---
drivers/gpu/drm/i915/intel_csr.c | 9 +++++++++
1 file changed, 9 insertions(+)

diff --git a/drivers/gpu/drm/i915/intel_csr.c b/drivers/gpu/drm/i915/intel_csr.c
index ba1ae03..682cc26 100644
--- a/drivers/gpu/drm/i915/intel_csr.c
+++ b/drivers/gpu/drm/i915/intel_csr.c
@@ -252,6 +252,15 @@ void intel_csr_load_program(struct drm_device *dev)
return;
}
+ /*
+ * Dmc will restore the csr the program except DC9, cold boot,
+ * warm reset, PCI function level reset, and hibernate/suspend.
+ * This condition will help to check if csr address space is reset/
+ * not loaded.
+ */
Atm we call this from driver load and resume, which doesn seem to cover
all the cases you mention in the comment. Should this be a WARN_ON
instead? Or do we have troubles in our init sequence where we load too
many times?
Yes, the above statement taken from bspec to describe about the special cases dmc will not restore the firmware.
Agree, In our cases cold boot and hibernate/suspend mainly we need to load the firmware again, so in my
second sentence I wanted to comment mainly regarding this condition check added for suspend-hibernate(reset)
and cold boot(not loaded).

Anyways the same api later can be used to load the firmware from anywhere, so my intention to check firmware loaded or not.
If already loaded then not to overwrite the csr address space to maintain the dc5/dc6 counter value.

Can the below comment more clear to you.

/*
* Dmc will restore the csr the program except DC9, cold boot,
* warm reset, PCI function level reset, and hibernate/suspend.
* If firmware is restored by dmc then no need to load again which
* will keep the dc5/dc6 counter exposed by firmware.
*/

No issue in init sequence.
That seems to still cover all the callers of the function afaics - we do
pci resets over suspend resume unconditionally. So I still don't
understand where exactly we try to load the dmc firmware in i915.ko when
it's already loaded.

During resume intel_csr_load_program() will be called from intel_runtime_resume().

intel_runtime_resume()-> skl_resume_prepare()-> intel_csr_load_program()

During Pc10 entry testing I can see dmc is restoring back the firmware always, but as you mentioned pci-reset can happen unconditionally, but still then also during resume intel_runtime_resume() will be called and based on register read of csr-base-address firmware loading will happen.

Quoted 8 lines
Quoted 4 lines

But in your comment you're saying it won't get restored in case of dc9 and suspend. So that seems to mismatch what you're saying here (and what the commit message says) and what the code does. And this function here is called for resume after suspend/hibernate only.

pc10 entry explanation I told is for skylake. dc9 in skylake is not possible. I think you are confusing between dc6 and dc9. Pc10 can be achieved by entering into dc6 (not dc9) for skylake. dc9 is the lowest possible state for broxton which is not present for skylake.

I have no idea at all about different pc levels on skl. What I'm talking about is system suspend/resume and driver load, which are the places this function gets called. At least afaics.

Quoted 3 lines

Here intel_csr_load_program() will be used for both skylake and broxton, and instruction execution flow will be different in case of suspend/resume which I think is confusing you.

That seems like really important information. What's different on bxt? These are the kind of details you should explain in the commit message ...

Quoted 5 lines

I am ready explain you in detail. It will be good if we discuss specific use-case scenario and itz software design for specific platform. Another point - as dmc related code for broxton is not merged better first we close design for skylake. Now, I have added dc9 description in comment thinking of future. If you want I can remove for now and later can add in bxt patch series for enabling dmc. Will wait for your reply.

This question here isn't about the overall design and how to handle power wells in skl/bxt. That's a separate discussion and tracked somewhere else. I'm really just confused about when exactly we need to reload to firmware, and why we need a runtime check for that. Normally we should know when to reload the firmware and just either reload or not, without checking hw state. And I don't like checking for hw state since at least in the past that kind of code ended up being fragile - it's an illusion that it does the right thing no matter what, since often there's other tricky ordering constraints. And if you have automatic duct-tape like then no one will ever spot those other, harder to spot issues, until an expensive customer escalation happens.

Quoted 2 lines

So what I want to know here is:
- When exactly do we need to reload dmc firmware.

In skl, during driver load first time we load the firmware, during normal suspend-resume (dc6 entry/exit)
no need to reload the firmware again as dmc will take care of it. But during suspend/hibernation
dmc will not restore the firmware. In that case driver need to reload it again. I do not know
how to differentiate pm-suspend and suspend-hibernation and thought both the cases
intel_runtime_resume() will be called where we can check the h/w state and reload the
firmware if dmc is not restored.

In bxt, during driver load first time we load the firmware, during normal suspend-resume
display engine will enter into dc9 and dmc will not restore the firmware. So every
suspend-resume we need to reload the firmware.

Quoted 8 lines
Quoted 2 lines

- What exactly is the reason why we can't make that decision statically in the code (by calling csr_load at the right spots).

As I mentioned before in case of skylake can we differentiate between "resume from pm-suspend" with "resume from suspend-hibernation" inside driver?

In case of broxton, every time we need to reload, so we can decide statically.

Of course we can differentiate between all the different resume paths, and we also have a per-platform split to take care of bxt vs. skl. And there are actually 3 different resume paths:

- runtime PM resume. This calls the runtime_resume hook. It sounds like on skl we should _not_ load the csr firmware, but on bxt we should load it. This can be fixed by removing the intel_csr_load_program call from skl_resume_prepare.
- resume from hibernate-to-disk (i.e. system completely off, state stored on the swap partition) is done by calling the thaw callbacks.
- resume from suspend-to-mem (i.e. system in low-power with only memory in self-refresh, all state stored in memory) is done by calling the resume callbacks.

For i915 we use unified handlers in our dev_pm_ops for both thaw and resume, but it sounds like that won't be a problem for skl/bxt since we need to reload the csr firmware in all cases. Although I'm not perfectly sure since you don't explain what kind of resume you mean exactly (since you don't use the linux names for them).

Anyway it sounds like we can replace this patch by one where we remove that errornous csr load call from skl runtime pm resume and that's all. But I suggest to make sure we get this right we keep the check you're adding here, but wrap it in a WARN_ON. Then we'll get a backtrace when this is going wrong again. Like this:

Quoted 6 lines
        if (WARN_ON(csr_loaded_already()))
                return;

Also when redoing the commits please explain in detail what exactly are the requirements like you've done above, but please use the standard linux names, i.e. "runtime PM" and "hibernate-to-disk" and "suspend-to-mem".

Ok hooray there's more suspend-to-something things I've totally missed:
- suspend-to-idle (done by cat freeze > /sys/power/state) and
- suspend (done by cat suspend > /sys/power/state)

And apparently there's really no way to drivers to tell them apart. Rafael, is there really no way for drivers to take different paths for these 3 suspend cases? I tried grepping for PM_SUSPEND_ON/STANDY/MEM and didn't spot anything.

Quoted 9 lines

Also we're completely missing test coverage for that in igt. That is something that needs to be fixed asap (yet another case of combinatorial explosion in igt tests, yay). And at least one of those suspend-to-idle testcase better be in the BAT.
-Daniel
--
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch

--
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch

Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#19 lore raw

Re: [DMC_BUGFIX_SKL_V2 1/5] drm/i915/skl: Added a check for the hardware status of csr fw before loading.

Quoted 31 lines · Daniel Vetter
Quoted 30 lines · Rafael J. Wysocki
Quoted 16 lines · Daniel Vetter
Quoted 15 lines · Rafael J. Wysocki
Quoted 4 lines · Daniel Vetter

acpi_target_system_state() seems to be almost the thing we're looking for, except that it's only valid in the suspend callbacks since it gets reset to ACPI_STATE_S0 when resuming. So probably we want something else ...

Right.

The idea is to add a way for drivers to check if
(a) suspend is going to enter the BIOS
(b) resume has been triggered by the BIOS and that's really what drivers need to know.

For suspend-to-idle those two will return false and for S3 they'll return true.

Quoted 7 lines
Quoted 1 line

Would that help?

Not sure that matches exaxtly what we'd need here ... Essentially we need to know whether we've been in S3/S4 (firmware has been eaten) or in one of the higher suspend-to-idle/standby states (firmware still alive, don't disturb it). Additional fun that just crossed my mind is that if the suspend-to-mem is aborted (some other driver failed) then that function should _not_ indicate that we've been in S3. So maybe something like

So it really looks like the interface I was talking about would be suitable:

pm_suspend_via_firmware() == true -> firmware is going to be eaten (use that during suspend if needed)
pm_resume_via_firmware() == true -> firmware was eaten

Quoted 6 lines
Quoted 2 lines

The latter will only return 'true' if we really have entered the BIOS (platform firmware).

Yeah that seems to fit the bill. We already have a check in our suspend paths to figure out whether we'll suspend to idle or go into full S3, so i915 could use them both. And making them generic would make sense I guess.

OK, sent patches (CCed you), please have a look.

Thanks,
Rafael

Quoted 223 lines
Quoted 223 lines
Quoted 4 lines

acpi_source_system_state() which usually is S0 and only when acpi successfully went into the suspend state in platform_suspend_ops->enter it gets set to the value of acpi_target_system_state. And then reset once the resume has completed. I think that would be what we'd want here.

We need to new functions like the above, because some things already depend on acpi_target_system_state working the way it does currently.

I see no reason to make that ACPI-specific, though, in principle.

Quoted 1 line

Anyway I'll pull in Animesh series meanwhile, amended with a FIXME comment.

Fine by me.

Thanks,
Rafael

Quoted 207 lines
Quoted 207 lines
Quoted 207 lines · Daniel Vetter
Quoted 206 lines

Actually add Rafael this time around ...
-Daniel

Quoted 202 lines · Daniel Vetter
Quoted 177 lines · Daniel Vetter
Quoted 142 lines · Animesh Manna
Quoted 122 lines · Daniel Vetter
Quoted 86 lines · Animesh Manna
Quoted 85 lines · Daniel Vetter
Quoted 84 lines · Animesh Manna

On 9/2/2015 2:24 PM, Daniel Vetter wrote:
On Wed, Aug 26, 2015 at 07:40:54PM +0530, Animesh Manna wrote:
On 8/26/2015 6:40 PM, Daniel Vetter wrote:
On Wed, Aug 26, 2015 at 01:36:05AM +0530, Animesh Manna wrote:
Dmc will restore the csr program except DC9, cold boot,
warm reset, PCI function level reset, and hibernate/suspend.

intel_csr_load_program() function is used to load the firmware
data from kernel memory to csr address space.

All values of csr address space will be zero if it got reset and
the first byte of csr program is always a non-zero if firmware
is loaded successfuly. Based on hardware status will load the
firmware.

Without this condition check if we overwrite the firmware data the
counters exposed for dc5/dc6 (help for debugging) will be nullified.
Bacause of the above reason mentioned just above we need to block firmware loading again.
So only WARN_ON will not help.

v1: Initial version.

v2: Based on review comments from Daniel,
- Added a check to know hardware status and load the firmware if not loaded.

Cc: Daniel Vetter <[email protected]>
Cc: Damien Lespiau <[email protected]>
Cc: Imre Deak <[email protected]>
Cc: Sunil Kamath <[email protected]>
Signed-off-by: Animesh Manna <[email protected]>
Signed-off-by: Vathsala Nagaraju <[email protected]>
---
drivers/gpu/drm/i915/intel_csr.c | 9 +++++++++
1 file changed, 9 insertions(+)

diff --git a/drivers/gpu/drm/i915/intel_csr.c b/drivers/gpu/drm/i915/intel_csr.c
index ba1ae03..682cc26 100644
--- a/drivers/gpu/drm/i915/intel_csr.c
+++ b/drivers/gpu/drm/i915/intel_csr.c
@@ -252,6 +252,15 @@ void intel_csr_load_program(struct drm_device *dev)
return;
}
+ /*
+ * Dmc will restore the csr the program except DC9, cold boot,
+ * warm reset, PCI function level reset, and hibernate/suspend.
+ * This condition will help to check if csr address space is reset/
+ * not loaded.
+ */
Atm we call this from driver load and resume, which doesn seem to cover
all the cases you mention in the comment. Should this be a WARN_ON
instead? Or do we have troubles in our init sequence where we load too
many times?
Yes, the above statement taken from bspec to describe about the special cases dmc will not restore the firmware.
Agree, In our cases cold boot and hibernate/suspend mainly we need to load the firmware again, so in my
second sentence I wanted to comment mainly regarding this condition check added for suspend-hibernate(reset)
and cold boot(not loaded).

Anyways the same api later can be used to load the firmware from anywhere, so my intention to check firmware loaded or not.
If already loaded then not to overwrite the csr address space to maintain the dc5/dc6 counter value.

Can the below comment more clear to you.

/*
* Dmc will restore the csr the program except DC9, cold boot,
* warm reset, PCI function level reset, and hibernate/suspend.
* If firmware is restored by dmc then no need to load again which
* will keep the dc5/dc6 counter exposed by firmware.
*/

No issue in init sequence.
That seems to still cover all the callers of the function afaics - we do
pci resets over suspend resume unconditionally. So I still don't
understand where exactly we try to load the dmc firmware in i915.ko when
it's already loaded.
During resume intel_csr_load_program() will be called from
intel_runtime_resume().

intel_runtime_resume()-> skl_resume_prepare()-> intel_csr_load_program()

During Pc10 entry testing I can see dmc is restoring back the firmware always,
but as you mentioned pci-reset can happen unconditionally, but still then
also during resume intel_runtime_resume() will be called and based on
register read of csr-base-address firmware loading will happen.

Quoted 8 lines
Quoted 4 lines

But in your comment you're saying it won't get restored in case of dc9 and suspend. So that seems to mismatch what you're saying here (and what the commit message says) and what the code does. And this function here is called for resume after suspend/hibernate only.

pc10 entry explanation I told is for skylake. dc9 in skylake is not possible. I think you are confusing between dc6 and dc9. Pc10 can be achieved by entering into dc6 (not dc9) for skylake. dc9 is the lowest possible state for broxton which is not present for skylake.

I have no idea at all about different pc levels on skl. What I'm talking about is system suspend/resume and driver load, which are the places this function gets called. At least afaics.

Quoted 3 lines

Here intel_csr_load_program() will be used for both skylake and broxton, and instruction execution flow will be different in case of suspend/resume which I think is confusing you.

That seems like really important information. What's different on bxt? These are the kind of details you should explain in the commit message ...

Quoted 5 lines

I am ready explain you in detail. It will be good if we discuss specific use-case scenario and itz software design for specific platform. Another point - as dmc related code for broxton is not merged better first we close design for skylake. Now, I have added dc9 description in comment thinking of future. If you want I can remove for now and later can add in bxt patch series for enabling dmc. Will wait for your reply.

This question here isn't about the overall design and how to handle power wells in skl/bxt. That's a separate discussion and tracked somewhere else. I'm really just confused about when exactly we need to reload to firmware, and why we need a runtime check for that. Normally we should know when to reload the firmware and just either reload or not, without checking hw state. And I don't like checking for hw state since at least in the past that kind of code ended up being fragile - it's an illusion that it does the right thing no matter what, since often there's other tricky ordering constraints. And if you have automatic duct-tape like then no one will ever spot those other, harder to spot issues, until an expensive customer escalation happens.

Quoted 2 lines

So what I want to know here is:
- When exactly do we need to reload dmc firmware.

In skl, during driver load first time we load the firmware, during normal suspend-resume (dc6 entry/exit)
no need to reload the firmware again as dmc will take care of it. But during suspend/hibernation
dmc will not restore the firmware. In that case driver need to reload it again. I do not know
how to differentiate pm-suspend and suspend-hibernation and thought both the cases
intel_runtime_resume() will be called where we can check the h/w state and reload the
firmware if dmc is not restored.

In bxt, during driver load first time we load the firmware, during normal suspend-resume
display engine will enter into dc9 and dmc will not restore the firmware. So every
suspend-resume we need to reload the firmware.

Quoted 8 lines
Quoted 2 lines

- What exactly is the reason why we can't make that decision statically in the code (by calling csr_load at the right spots).

As I mentioned before in case of skylake can we differentiate between "resume from pm-suspend" with "resume from suspend-hibernation" inside driver?

In case of broxton, every time we need to reload, so we can decide statically.

Of course we can differentiate between all the different resume paths, and we also have a per-platform split to take care of bxt vs. skl. And there are actually 3 different resume paths:

- runtime PM resume. This calls the runtime_resume hook. It sounds like on skl we should _not_ load the csr firmware, but on bxt we should load it. This can be fixed by removing the intel_csr_load_program call from skl_resume_prepare.
- resume from hibernate-to-disk (i.e. system completely off, state stored on the swap partition) is done by calling the thaw callbacks.
- resume from suspend-to-mem (i.e. system in low-power with only memory in self-refresh, all state stored in memory) is done by calling the resume callbacks.

For i915 we use unified handlers in our dev_pm_ops for both thaw and resume, but it sounds like that won't be a problem for skl/bxt since we need to reload the csr firmware in all cases. Although I'm not perfectly sure since you don't explain what kind of resume you mean exactly (since you don't use the linux names for them).

Anyway it sounds like we can replace this patch by one where we remove that errornous csr load call from skl runtime pm resume and that's all. But I suggest to make sure we get this right we keep the check you're adding here, but wrap it in a WARN_ON. Then we'll get a backtrace when this is going wrong again. Like this:

Quoted 6 lines
         if (WARN_ON(csr_loaded_already()))
                 return;

Also when redoing the commits please explain in detail what exactly are the requirements like you've done above, but please use the standard linux names, i.e. "runtime PM" and "hibernate-to-disk" and "suspend-to-mem".

Ok hooray there's more suspend-to-something things I've totally missed:
- suspend-to-idle (done by cat freeze > /sys/power/state) and
- suspend (done by cat suspend > /sys/power/state)

And apparently there's really no way to drivers to tell them apart. Rafael, is there really no way for drivers to take different paths for these 3 suspend cases? I tried grepping for PM_SUSPEND_ON/STANDY/MEM and didn't spot anything.

Also we're completely missing test coverage for that in igt. That is something that needs to be fixed asap (yet another case of combinatorial explosion in igt tests, yay). And at least one of those suspend-to-idle testcase better be in the BAT.
-Daniel
--
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

--
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

#20 lore raw

Re: [DMC_BUGFIX_SKL_V2 1/5] drm/i915/skl: Added a check for the hardware status of csr fw before loading.

Quoted 32 lines · Rafael J. Wysocki
Quoted 31 lines · Daniel Vetter
Quoted 30 lines · Rafael J. Wysocki
Quoted 16 lines · Daniel Vetter
Quoted 15 lines · Rafael J. Wysocki
Quoted 4 lines · Daniel Vetter

acpi_target_system_state() seems to be almost the thing we're looking for, except that it's only valid in the suspend callbacks since it gets reset to ACPI_STATE_S0 when resuming. So probably we want something else ...

Right.

The idea is to add a way for drivers to check if
(a) suspend is going to enter the BIOS
(b) resume has been triggered by the BIOS and that's really what drivers need to know.

For suspend-to-idle those two will return false and for S3 they'll return true.

Quoted 7 lines
Quoted 1 line

Would that help?

Not sure that matches exaxtly what we'd need here ... Essentially we need to know whether we've been in S3/S4 (firmware has been eaten) or in one of the higher suspend-to-idle/standby states (firmware still alive, don't disturb it). Additional fun that just crossed my mind is that if the suspend-to-mem is aborted (some other driver failed) then that function should _not_ indicate that we've been in S3. So maybe something like

So it really looks like the interface I was talking about would be suitable:

pm_suspend_via_firmware() == true -> firmware is going to be eaten (use that during suspend if needed)
pm_resume_via_firmware() == true -> firmware was eaten

Quoted 8 lines
Quoted 6 lines
Quoted 2 lines

The latter will only return 'true' if we really have entered the BIOS (platform firmware).

Yeah that seems to fit the bill. We already have a check in our suspend paths to figure out whether we'll suspend to idle or go into full S3, so i915 could use them both. And making them generic would make sense I guess.

OK, sent patches (CCed you), please have a look.

EXPORT_SYMBOL seems missing, but lgtm otherwise. I haven't tried to use them for the two cases in i915 though.
-Daniel

Quoted 233 lines

Thanks,
Rafael

Quoted 227 lines
Quoted 227 lines
Quoted 4 lines

acpi_source_system_state() which usually is S0 and only when acpi successfully went into the suspend state in platform_suspend_ops->enter it gets set to the value of acpi_target_system_state. And then reset once the resume has completed. I think that would be what we'd want here.

We need to new functions like the above, because some things already depend on acpi_target_system_state working the way it does currently.

I see no reason to make that ACPI-specific, though, in principle.

Quoted 1 line

Anyway I'll pull in Animesh series meanwhile, amended with a FIXME comment.

Fine by me.

Thanks,
Rafael

Quoted 211 lines
Quoted 211 lines
Quoted 211 lines · Daniel Vetter
Quoted 210 lines

Actually add Rafael this time around ...
-Daniel

Quoted 193 lines · Daniel Vetter
Quoted 177 lines · Daniel Vetter
Quoted 142 lines · Animesh Manna
Quoted 122 lines · Daniel Vetter
Quoted 86 lines · Animesh Manna
Quoted 85 lines · Daniel Vetter

On Thu, Sep 10, 2015 at 01:58:54AM +0530, Animesh Manna wrote:
On 9/2/2015 2:24 PM, Daniel Vetter wrote:
On Wed, Aug 26, 2015 at 07:40:54PM +0530, Animesh Manna wrote:
On 8/26/2015 6:40 PM, Daniel Vetter wrote:
On Wed, Aug 26, 2015 at 01:36:05AM +0530, Animesh Manna wrote:
Dmc will restore the csr program except DC9, cold boot,
warm reset, PCI function level reset, and hibernate/suspend.

intel_csr_load_program() function is used to load the firmware
data from kernel memory to csr address space.

All values of csr address space will be zero if it got reset and
the first byte of csr program is always a non-zero if firmware
is loaded successfuly. Based on hardware status will load the
firmware.

Without this condition check if we overwrite the firmware data the
counters exposed for dc5/dc6 (help for debugging) will be nullified.
Bacause of the above reason mentioned just above we need to block firmware loading again.
So only WARN_ON will not help.

v1: Initial version.

v2: Based on review comments from Daniel,
- Added a check to know hardware status and load the firmware if not loaded.

Cc: Daniel Vetter <[email protected]>
Cc: Damien Lespiau <[email protected]>
Cc: Imre Deak <[email protected]>
Cc: Sunil Kamath <[email protected]>
Signed-off-by: Animesh Manna <[email protected]>
Signed-off-by: Vathsala Nagaraju <[email protected]>
---
drivers/gpu/drm/i915/intel_csr.c | 9 +++++++++
1 file changed, 9 insertions(+)

diff --git a/drivers/gpu/drm/i915/intel_csr.c b/drivers/gpu/drm/i915/intel_csr.c
index ba1ae03..682cc26 100644
--- a/drivers/gpu/drm/i915/intel_csr.c
+++ b/drivers/gpu/drm/i915/intel_csr.c
@@ -252,6 +252,15 @@ void intel_csr_load_program(struct drm_device *dev)
return;
}
+ /*
+ * Dmc will restore the csr the program except DC9, cold boot,
+ * warm reset, PCI function level reset, and hibernate/suspend.
+ * This condition will help to check if csr address space is reset/
+ * not loaded.
+ */
Atm we call this from driver load and resume, which doesn seem to cover
all the cases you mention in the comment. Should this be a WARN_ON
instead? Or do we have troubles in our init sequence where we load too
many times?
Yes, the above statement taken from bspec to describe about the special cases dmc will not restore the firmware.
Agree, In our cases cold boot and hibernate/suspend mainly we need to load the firmware again, so in my
second sentence I wanted to comment mainly regarding this condition check added for suspend-hibernate(reset)
and cold boot(not loaded).

Anyways the same api later can be used to load the firmware from anywhere, so my intention to check firmware loaded or not.
If already loaded then not to overwrite the csr address space to maintain the dc5/dc6 counter value.

Can the below comment more clear to you.

/*
* Dmc will restore the csr the program except DC9, cold boot,
* warm reset, PCI function level reset, and hibernate/suspend.
* If firmware is restored by dmc then no need to load again which
* will keep the dc5/dc6 counter exposed by firmware.
*/

No issue in init sequence.
That seems to still cover all the callers of the function afaics - we do
pci resets over suspend resume unconditionally. So I still don't
understand where exactly we try to load the dmc firmware in i915.ko when
it's already loaded.
During resume intel_csr_load_program() will be called from
intel_runtime_resume().

intel_runtime_resume()-> skl_resume_prepare()-> intel_csr_load_program()

During Pc10 entry testing I can see dmc is restoring back the firmware always,
but as you mentioned pci-reset can happen unconditionally, but still then
also during resume intel_runtime_resume() will be called and based on
register read of csr-base-address firmware loading will happen.

Quoted 8 lines
Quoted 4 lines

But in your comment you're saying it won't get restored in case of dc9 and
suspend. So that seems to mismatch what you're saying here (and what the
commit message says) and what the code does. And this function here is
called for resume after suspend/hibernate only.

pc10 entry explanation I told is for skylake. dc9 in skylake is not possible. I think you are confusing between dc6 and dc9. Pc10 can be achieved by entering into dc6 (not dc9) for skylake. dc9 is the lowest possible state for broxton which is not present for skylake.

I have no idea at all about different pc levels on skl. What I'm talking about is system suspend/resume and driver load, which are the places this function gets called. At least afaics.

Quoted 3 lines

Here intel_csr_load_program() will be used for both skylake and broxton, and instruction execution flow will be different in case of suspend/resume which I think is confusing you.

That seems like really important information. What's different on bxt? These are the kind of details you should explain in the commit message ...

Quoted 5 lines

I am ready explain you in detail. It will be good if we discuss specific use-case scenario and itz software design for specific platform. Another point - as dmc related code for broxton is not merged better first we close design for skylake. Now, I have added dc9 description in comment thinking of future. If you want I can remove for now and later can add in bxt patch series for enabling dmc. Will wait for your reply.

This question here isn't about the overall design and how to handle power wells in skl/bxt. That's a separate discussion and tracked somewhere else. I'm really just confused about when exactly we need to reload to firmware, and why we need a runtime check for that. Normally we should know when to reload the firmware and just either reload or not, without checking hw state. And I don't like checking for hw state since at least in the past that kind of code ended up being fragile - it's an illusion that it does the right thing no matter what, since often there's other tricky ordering constraints. And if you have automatic duct-tape like then no one will ever spot those other, harder to spot issues, until an expensive customer escalation happens.

Quoted 2 lines

So what I want to know here is:
- When exactly do we need to reload dmc firmware.

In skl, during driver load first time we load the firmware, during normal suspend-resume (dc6 entry/exit)
no need to reload the firmware again as dmc will take care of it. But during suspend/hibernation
dmc will not restore the firmware. In that case driver need to reload it again. I do not know
how to differentiate pm-suspend and suspend-hibernation and thought both the cases
intel_runtime_resume() will be called where we can check the h/w state and reload the
firmware if dmc is not restored.

In bxt, during driver load first time we load the firmware, during normal suspend-resume
display engine will enter into dc9 and dmc will not restore the firmware. So every
suspend-resume we need to reload the firmware.

Quoted 8 lines
Quoted 2 lines

- What exactly is the reason why we can't make that decision statically in the code (by calling csr_load at the right spots).

As I mentioned before in case of skylake can we differentiate between "resume from pm-suspend" with "resume from suspend-hibernation" inside driver?

In case of broxton, every time we need to reload, so we can decide statically.

Of course we can differentiate between all the different resume paths, and we also have a per-platform split to take care of bxt vs. skl. And there are actually 3 different resume paths:

- runtime PM resume. This calls the runtime_resume hook. It sounds like on skl we should _not_ load the csr firmware, but on bxt we should load it. This can be fixed by removing the intel_csr_load_program call from skl_resume_prepare.
- resume from hibernate-to-disk (i.e. system completely off, state stored on the swap partition) is done by calling the thaw callbacks.
- resume from suspend-to-mem (i.e. system in low-power with only memory in self-refresh, all state stored in memory) is done by calling the resume callbacks.

For i915 we use unified handlers in our dev_pm_ops for both thaw and resume, but it sounds like that won't be a problem for skl/bxt since we need to reload the csr firmware in all cases. Although I'm not perfectly sure since you don't explain what kind of resume you mean exactly (since you don't use the linux names for them).

Anyway it sounds like we can replace this patch by one where we remove that errornous csr load call from skl runtime pm resume and that's all. But I suggest to make sure we get this right we keep the check you're adding here, but wrap it in a WARN_ON. Then we'll get a backtrace when this is going wrong again. Like this:

Quoted 6 lines
        if (WARN_ON(csr_loaded_already()))
                return;

Also when redoing the commits please explain in detail what exactly are the requirements like you've done above, but please use the standard linux names, i.e. "runtime PM" and "hibernate-to-disk" and "suspend-to-mem".

Ok hooray there's more suspend-to-something things I've totally missed:
- suspend-to-idle (done by cat freeze > /sys/power/state) and
- suspend (done by cat suspend > /sys/power/state)

And apparently there's really no way to drivers to tell them apart. Rafael, is there really no way for drivers to take different paths for these 3 suspend cases? I tried grepping for PM_SUSPEND_ON/STANDY/MEM and didn't spot anything.

Quoted 9 lines

Also we're completely missing test coverage for that in igt. That is something that needs to be fixed asap (yet another case of combinatorial explosion in igt tests, yay). And at least one of those suspend-to-idle testcase better be in the BAT.
-Daniel
--
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch

--
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch

Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#21 lore raw

Re: [DMC_BUGFIX_SKL_V2 1/5] drm/i915/skl: Added a check for the hardware status of csr fw before loading.

Quoted 33 lines · Daniel Vetter
Quoted 32 lines · Rafael J. Wysocki
Quoted 31 lines · Daniel Vetter
Quoted 30 lines · Rafael J. Wysocki
Quoted 16 lines · Daniel Vetter
Quoted 15 lines · Rafael J. Wysocki
Quoted 4 lines · Daniel Vetter

acpi_target_system_state() seems to be almost the thing we're looking for, except that it's only valid in the suspend callbacks since it gets reset to ACPI_STATE_S0 when resuming. So probably we want something else ...

Right.

The idea is to add a way for drivers to check if
(a) suspend is going to enter the BIOS
(b) resume has been triggered by the BIOS and that's really what drivers need to know.

For suspend-to-idle those two will return false and for S3 they'll return true.

Quoted 7 lines
Quoted 1 line

Would that help?

Not sure that matches exaxtly what we'd need here ... Essentially we need to know whether we've been in S3/S4 (firmware has been eaten) or in one of the higher suspend-to-idle/standby states (firmware still alive, don't disturb it). Additional fun that just crossed my mind is that if the suspend-to-mem is aborted (some other driver failed) then that function should _not_ indicate that we've been in S3. So maybe something like

So it really looks like the interface I was talking about would be suitable:

pm_suspend_via_firmware() == true -> firmware is going to be eaten (use that during suspend if needed)
pm_resume_via_firmware() == true -> firmware was eaten

Quoted 8 lines
Quoted 7 lines
Quoted 6 lines
Quoted 2 lines

The latter will only return 'true' if we really have entered the BIOS (platform firmware).

Yeah that seems to fit the bill. We already have a check in our suspend paths to figure out whether we'll suspend to idle or go into full S3, so i915 could use them both. And making them generic would make sense I guess.

OK, sent patches (CCed you), please have a look.

EXPORT_SYMBOL seems missing,

Good point, will fix.

Quoted 2 lines

but lgtm otherwise. I haven't tried to use
them for the two cases in i915 though.

OK, thanks!

Rafael

Quoted 224 lines
Quoted 224 lines
Quoted 224 lines
Quoted 224 lines
Quoted 5 lines

acpi_source_system_state() which usually is S0 and only when acpi successfully went into the suspend state in platform_suspend_ops->enter it gets set to the value of acpi_target_system_state. And then reset once the resume has completed. I think that would be what we'd want here.

We need to new functions like the above, because some things already depend on acpi_target_system_state working the way it does currently.

I see no reason to make that ACPI-specific, though, in principle.

Quoted 1 line

Anyway I'll pull in Animesh series meanwhile, amended with a FIXME comment.

Fine by me.

Thanks,
Rafael

Quoted 207 lines
Quoted 207 lines
Quoted 207 lines · Daniel Vetter
Quoted 206 lines

Actually add Rafael this time around ...
-Daniel

Quoted 202 lines · Daniel Vetter
Quoted 177 lines · Daniel Vetter
Quoted 142 lines · Animesh Manna
Quoted 122 lines · Daniel Vetter
Quoted 94 lines · Animesh Manna

On 9/10/2015 8:15 PM, Daniel Vetter wrote:
On Thu, Sep 10, 2015 at 01:58:54AM +0530, Animesh Manna wrote:
On 9/2/2015 2:24 PM, Daniel Vetter wrote:
On Wed, Aug 26, 2015 at 07:40:54PM +0530, Animesh Manna wrote:
On 8/26/2015 6:40 PM, Daniel Vetter wrote:
On Wed, Aug 26, 2015 at 01:36:05AM +0530, Animesh Manna wrote:
Dmc will restore the csr program except DC9, cold boot,
warm reset, PCI function level reset, and hibernate/suspend.

intel_csr_load_program() function is used to load the firmware
data from kernel memory to csr address space.

All values of csr address space will be zero if it got reset and
the first byte of csr program is always a non-zero if firmware
is loaded successfuly. Based on hardware status will load the
firmware.

Without this condition check if we overwrite the firmware data the
counters exposed for dc5/dc6 (help for debugging) will be nullified.
Bacause of the above reason mentioned just above we need to block firmware loading again.
So only WARN_ON will not help.

v1: Initial version.

v2: Based on review comments from Daniel,
- Added a check to know hardware status and load the firmware if not loaded.

Cc: Daniel Vetter <[email protected]>
Cc: Damien Lespiau <[email protected]>
Cc: Imre Deak <[email protected]>
Cc: Sunil Kamath <[email protected]>
Signed-off-by: Animesh Manna <[email protected]>
Signed-off-by: Vathsala Nagaraju <[email protected]>
---
drivers/gpu/drm/i915/intel_csr.c | 9 +++++++++
1 file changed, 9 insertions(+)

diff --git a/drivers/gpu/drm/i915/intel_csr.c b/drivers/gpu/drm/i915/intel_csr.c
index ba1ae03..682cc26 100644
--- a/drivers/gpu/drm/i915/intel_csr.c
+++ b/drivers/gpu/drm/i915/intel_csr.c
@@ -252,6 +252,15 @@ void intel_csr_load_program(struct drm_device *dev)
return;
}
+ /*
+ * Dmc will restore the csr the program except DC9, cold boot,
+ * warm reset, PCI function level reset, and hibernate/suspend.
+ * This condition will help to check if csr address space is reset/
+ * not loaded.
+ */
Atm we call this from driver load and resume, which doesn seem to cover
all the cases you mention in the comment. Should this be a WARN_ON
instead? Or do we have troubles in our init sequence where we load too
many times?
Yes, the above statement taken from bspec to describe about the special cases dmc will not restore the firmware.
Agree, In our cases cold boot and hibernate/suspend mainly we need to load the firmware again, so in my
second sentence I wanted to comment mainly regarding this condition check added for suspend-hibernate(reset)
and cold boot(not loaded).

Anyways the same api later can be used to load the firmware from anywhere, so my intention to check firmware loaded or not.
If already loaded then not to overwrite the csr address space to maintain the dc5/dc6 counter value.

Can the below comment more clear to you.

/*
* Dmc will restore the csr the program except DC9, cold boot,
* warm reset, PCI function level reset, and hibernate/suspend.
* If firmware is restored by dmc then no need to load again which
* will keep the dc5/dc6 counter exposed by firmware.
*/

No issue in init sequence.
That seems to still cover all the callers of the function afaics - we do
pci resets over suspend resume unconditionally. So I still don't
understand where exactly we try to load the dmc firmware in i915.ko when
it's already loaded.
During resume intel_csr_load_program() will be called from
intel_runtime_resume().

intel_runtime_resume()-> skl_resume_prepare()-> intel_csr_load_program()

During Pc10 entry testing I can see dmc is restoring back the firmware always,
but as you mentioned pci-reset can happen unconditionally, but still then
also during resume intel_runtime_resume() will be called and based on
register read of csr-base-address firmware loading will happen.
But in your comment you're saying it won't get restored in case of dc9 and
suspend. So that seems to mismatch what you're saying here (and what the
commit message says) and what the code does. And this function here is
called for resume after suspend/hibernate only.
pc10 entry explanation I told is for skylake. dc9 in skylake is not possible.
I think you are confusing between dc6 and dc9. Pc10 can be achieved by
entering into dc6 (not dc9) for skylake. dc9 is the lowest possible state
for broxton which is not present for skylake.

I have no idea at all about different pc levels on skl. What I'm talking about is system suspend/resume and driver load, which are the places this function gets called. At least afaics.

Quoted 3 lines

Here intel_csr_load_program() will be used for both skylake and broxton, and instruction
execution flow will be different in case of suspend/resume which I think is confusing
you.

That seems like really important information. What's different on bxt? These are the kind of details you should explain in the commit message ...

Quoted 5 lines

I am ready explain you in detail. It will be good if we discuss specific use-case scenario
and itz software design for specific platform. Another point - as dmc related code for
broxton is not merged better first we close design for skylake. Now, I have added dc9
description in comment thinking of future. If you want I can remove for now and later
can add in bxt patch series for enabling dmc. Will wait for your reply.

This question here isn't about the overall design and how to handle power wells in skl/bxt. That's a separate discussion and tracked somewhere else. I'm really just confused about when exactly we need to reload to firmware, and why we need a runtime check for that. Normally we should know when to reload the firmware and just either reload or not, without checking hw state. And I don't like checking for hw state since at least in the past that kind of code ended up being fragile - it's an illusion that it does the right thing no matter what, since often there's other tricky ordering constraints. And if you have automatic duct-tape like then no one will ever spot those other, harder to spot issues, until an expensive customer escalation happens.

Quoted 2 lines

So what I want to know here is:
- When exactly do we need to reload dmc firmware.

In skl, during driver load first time we load the firmware, during normal suspend-resume (dc6 entry/exit)
no need to reload the firmware again as dmc will take care of it. But during suspend/hibernation
dmc will not restore the firmware. In that case driver need to reload it again. I do not know
how to differentiate pm-suspend and suspend-hibernation and thought both the cases
intel_runtime_resume() will be called where we can check the h/w state and reload the
firmware if dmc is not restored.

In bxt, during driver load first time we load the firmware, during normal suspend-resume
display engine will enter into dc9 and dmc will not restore the firmware. So every
suspend-resume we need to reload the firmware.

Quoted 8 lines
Quoted 2 lines

- What exactly is the reason why we can't make that decision statically in the code (by calling csr_load at the right spots).

As I mentioned before in case of skylake can we differentiate between "resume from pm-suspend" with "resume from suspend-hibernation" inside driver?

In case of broxton, every time we need to reload, so we can decide statically.

Of course we can differentiate between all the different resume paths, and we also have a per-platform split to take care of bxt vs. skl. And there are actually 3 different resume paths:

- runtime PM resume. This calls the runtime_resume hook. It sounds like on skl we should _not_ load the csr firmware, but on bxt we should load it. This can be fixed by removing the intel_csr_load_program call from skl_resume_prepare.
- resume from hibernate-to-disk (i.e. system completely off, state stored on the swap partition) is done by calling the thaw callbacks.
- resume from suspend-to-mem (i.e. system in low-power with only memory in self-refresh, all state stored in memory) is done by calling the resume callbacks.

For i915 we use unified handlers in our dev_pm_ops for both thaw and resume, but it sounds like that won't be a problem for skl/bxt since we need to reload the csr firmware in all cases. Although I'm not perfectly sure since you don't explain what kind of resume you mean exactly (since you don't use the linux names for them).

Anyway it sounds like we can replace this patch by one where we remove that errornous csr load call from skl runtime pm resume and that's all. But I suggest to make sure we get this right we keep the check you're adding here, but wrap it in a WARN_ON. Then we'll get a backtrace when this is going wrong again. Like this:

Quoted 6 lines
         if (WARN_ON(csr_loaded_already()))
                 return;

Also when redoing the commits please explain in detail what exactly are the requirements like you've done above, but please use the standard linux names, i.e. "runtime PM" and "hibernate-to-disk" and "suspend-to-mem".

Ok hooray there's more suspend-to-something things I've totally missed:
- suspend-to-idle (done by cat freeze > /sys/power/state) and
- suspend (done by cat suspend > /sys/power/state)

And apparently there's really no way to drivers to tell them apart. Rafael, is there really no way for drivers to take different paths for these 3 suspend cases? I tried grepping for PM_SUSPEND_ON/STANDY/MEM and didn't spot anything.

Also we're completely missing test coverage for that in igt. That is something that needs to be fixed asap (yet another case of combinatorial explosion in igt tests, yay). And at least one of those suspend-to-idle testcase better be in the BAT.
-Daniel
--
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

--
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

Reviewed-by on v1 1/5
#22 lore raw

Re: [DMC_BUGFIX_SKL_V2 1/5] drm/i915/skl: Added a check for the hardware status of csr fw before loading.

In reply to Animesh Manna
Quoted 39 lines · Animesh Manna

Dmc will restore the csr program except DC9, cold boot, warm reset, PCI function level reset, and hibernate/suspend.

intel_csr_load_program() function is used to load the firmware data from kernel memory to csr address space.

All values of csr address space will be zero if it got reset and the first byte of csr program is always a non-zero if firmware is loaded successfuly. Based on hardware status will load the firmware.

Without this condition check if we overwrite the firmware data the counters exposed for dc5/dc6 (help for debugging) will be nullified.

v1: Initial version.

v2: Based on review comments from Daniel,
- Added a check to know hardware status and load the firmware if not loaded.

Cc Daniel Vetter <[email protected]>
Cc Damien Lespiau <[email protected]>
Cc Imre Deak <[email protected]>
Cc Sunil Kamath <[email protected]>
---
  drivers/gpu/drm/i915/intel_csr.c | 9 +++++++++
  1 file changed, 9 insertions(+)
drivers/gpu/drm/i915/intel_csr.c +2 -0
index ba1ae03..682cc26 100644
@@ -252,6 +252,15 @@ void intel_csr_load_program(struct drm_device *dev)
  		return;
  	}
 
+	/*
+	 * Dmc will restore the csr the program except DC9, cold boot,

drivers/gpu/drm/i915/intel_csr.c void intel_csr_load_program(struct drm_device *dev) · line 266

Quoted 10 lines
drivers/gpu/drm/i915/intel_csr.c +7 -0
+	 * warm reset, PCI function level reset, and hibernate/suspend.
+	 * This condition will help to check if csr address space is reset/
+	 * not loaded.
+	 */
+	if (I915_READ(CSR_PROGRAM_BASE))
+		return;
+
  	mutex_lock(&dev_priv->csr_lock);
  	fw_size = dev_priv->csr.dmc_fw_size;
  	for (i = 0; i < fw_size; i++)

Valid fix and patch is ready for merge now.

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

#23 lore raw

Re: [DMC_BUGFIX_SKL_V2 1/5] drm/i915/skl: Added a check for the hardware status of csr fw before loading.

Quoted 43 lines · Sunil Kamath
Quoted 42 lines · Animesh Manna

Dmc will restore the csr program except DC9, cold boot, warm reset, PCI function level reset, and hibernate/suspend.

intel_csr_load_program() function is used to load the firmware data from kernel memory to csr address space.

All values of csr address space will be zero if it got reset and the first byte of csr program is always a non-zero if firmware is loaded successfuly. Based on hardware status will load the firmware.

Without this condition check if we overwrite the firmware data the counters exposed for dc5/dc6 (help for debugging) will be nullified.

v1: Initial version.

v2: Based on review comments from Daniel,
- Added a check to know hardware status and load the firmware if not loaded.

Cc Daniel Vetter <[email protected]>
Cc Damien Lespiau <[email protected]>
Cc Imre Deak <[email protected]>
Cc Sunil Kamath <[email protected]>
---
 drivers/gpu/drm/i915/intel_csr.c | 9 +++++++++
 1 file changed, 9 insertions(+)
drivers/gpu/drm/i915/intel_csr.c +6 -0
index ba1ae03..682cc26 100644
@@ -252,6 +252,15 @@ void intel_csr_load_program(struct drm_device *dev)
 		return;
 	}
+	/*
+	 * Dmc will restore the csr the program except DC9, cold boot,
+	 * warm reset, PCI function level reset, and hibernate/suspend.
+	 * This condition will help to check if csr address space is reset/
+	 * not loaded.
+	 */
Quoted 8 lines

drivers/gpu/drm/i915/intel_csr.c void intel_csr_load_program(struct drm_device *dev) · line 266

Quoted 6 lines
drivers/gpu/drm/i915/intel_csr.c +3 -0
+	if (I915_READ(CSR_PROGRAM_BASE))
+		return;
+
 	mutex_lock(&dev_priv->csr_lock);
 	fw_size = dev_priv->csr.dmc_fw_size;
 	for (i = 0; i < fw_size; i++)

Valid fix and patch is ready for merge now.

I still have a question open so will wait until that's anserwered ...

Thanks, Daniel

Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#24 lore raw

Re: [DMC_BUGFIX_SKL_V2 1/5] drm/i915/skl: Added a check for the hardware status of csr fw before loading.

Quoted 51 lines · Daniel Vetter
Quoted 42 lines · Sunil Kamath
Quoted 41 lines · Animesh Manna

Dmc will restore the csr program except DC9, cold boot, warm reset, PCI function level reset, and hibernate/suspend.

intel_csr_load_program() function is used to load the firmware data from kernel memory to csr address space.

All values of csr address space will be zero if it got reset and the first byte of csr program is always a non-zero if firmware is loaded successfuly. Based on hardware status will load the firmware.

Without this condition check if we overwrite the firmware data the counters exposed for dc5/dc6 (help for debugging) will be nullified.

v1: Initial version.

v2: Based on review comments from Daniel,
- Added a check to know hardware status and load the firmware if not loaded.

Cc Daniel Vetter <[email protected]>
Cc Damien Lespiau <[email protected]>
Cc Imre Deak <[email protected]>
Cc Sunil Kamath <[email protected]>
---
  drivers/gpu/drm/i915/intel_csr.c | 9 +++++++++
  1 file changed, 9 insertions(+)
drivers/gpu/drm/i915/intel_csr.c +5 -0
index ba1ae03..682cc26 100644
@@ -252,6 +252,15 @@ void intel_csr_load_program(struct drm_device *dev)
  		return;
  	}
+	/*
+	 * Dmc will restore the csr the program except DC9, cold boot,
+	 * warm reset, PCI function level reset, and hibernate/suspend.
+	 * This condition will help to check if csr address space is reset/
+	 * not loaded.
Quoted 8 lines

drivers/gpu/drm/i915/intel_csr.c void intel_csr_load_program(struct drm_device *dev) · line 266

Quoted 7 lines
drivers/gpu/drm/i915/intel_csr.c +4 -0
+	 */
+	if (I915_READ(CSR_PROGRAM_BASE))
+		return;
+
  	mutex_lock(&dev_priv->csr_lock);
  	fw_size = dev_priv->csr.dmc_fw_size;
  	for (i = 0; i < fw_size; i++)

Valid fix and patch is ready for merge now.

Quoted 3 lines

I still have a question open so will wait until that's anserwered ...

Thanks, Daniel

Shared my view on other thread.

-Animesh

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

#25 lore raw

Re: [DMC_BUGFIX_SKL_V2 1/5] drm/i915/skl: Added a check for the hardware status of csr fw before loading.

In reply to Sunil Kamath
Quoted 52 lines · Sunil Kamath
Quoted 38 lines · Animesh Manna

Dmc will restore the csr program except DC9, cold boot, warm reset, PCI function level reset, and hibernate/suspend.

intel_csr_load_program() function is used to load the firmware data from kernel memory to csr address space.

All values of csr address space will be zero if it got reset and the first byte of csr program is always a non-zero if firmware is loaded successfuly. Based on hardware status will load the firmware.

Without this condition check if we overwrite the firmware data the counters exposed for dc5/dc6 (help for debugging) will be nullified.

v1: Initial version.

v2: Based on review comments from Daniel,
- Added a check to know hardware status and load the firmware if not loaded.

Cc Daniel Vetter <[email protected]>
Cc Damien Lespiau <[email protected]>
Cc Imre Deak <[email protected]>
Cc Sunil Kamath <[email protected]>
---
 drivers/gpu/drm/i915/intel_csr.c | 9 +++++++++
 1 file changed, 9 insertions(+)
drivers/gpu/drm/i915/intel_csr.c +2 -0
index ba1ae03..682cc26 100644
@@ -252,6 +252,15 @@ void intel_csr_load_program(struct drm_device *dev)
 		return;
 	}
+	/*
+	 * Dmc will restore the csr the program except DC9, cold boot,

drivers/gpu/drm/i915/intel_csr.c void intel_csr_load_program(struct drm_device *dev) · line 265

Quoted 10 lines
drivers/gpu/drm/i915/intel_csr.c +7 -0
+	 * warm reset, PCI function level reset, and hibernate/suspend.
+	 * This condition will help to check if csr address space is reset/
+	 * not loaded.
+	 */
+	if (I915_READ(CSR_PROGRAM_BASE))
+		return;
+
 	mutex_lock(&dev_priv->csr_lock);
 	fw_size = dev_priv->csr.dmc_fw_size;
 	for (i = 0; i < fw_size; i++)

Valid fix and patch is ready for merge now.

Quoted 1 line

Ok I changed the code comment to a FIXME to explain what's going on (DC9 is totally irrelevant if I understand this correctly now) and amended the commit message with a note about what I think is really going on. Please double-check.

Also I realized that we have 0 test coverage for suspend-to-idle (and oversight for merging this feature a few years back). Please add a new igt testcase (maybe subtest of drv_suspend) to exercise this feature.

Making sure that we have sufficient igt coverage is part of the review and especially something that _must_ be checked for bugfixes like this.

Thanks, Daniel

Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#26 lore raw

PATCH 2/5 drm/i915/skl Remove the call for csr uninitialization from suspend path

In reply to Animesh Manna

This patch remove the function call to set the firmware loading status as uninitialized during suspend.

Dmc firmware will restore the firmware in normal suspend. In previous patch added a check to directly read the hardware status and load the firmware if got reset during resume from suspend-hibernation.

---
 drivers/gpu/drm/i915/i915_drv.c | 6 ------
 1 file changed, 6 deletions(-)
drivers/gpu/drm/i915/i915_drv.c +0 -6
index 1d88745..478101c 100644
@@ -1015,12 +1015,6 @@ static int skl_suspend_complete(struct drm_i915_private *dev_priv)
 {
 	/* Enabling DC6 is not a hard requirement to enter runtime D3 */
 
-	/*
-	 * This is to ensure that CSR isn't identified as loaded before
-	 * CSR-loading program is called during runtime-resume.
-	 */
-	intel_csr_load_status_set(dev_priv, FW_UNINITIALIZED);
-
 	skl_uninit_cdclk(dev_priv);
 
 	return 0;
Signature · 6 lines
2.0.2

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
Reviewed-by on v1 2/5
#27 lore raw

Re: [DMC_BUGFIX_SKL_V2 2/5] drm/i915/skl Remove the call for csr uninitialization from suspend path

Quoted 21 lines · Animesh Manna

This patch remove the function call to set the firmware loading status as uninitialized during suspend.

Dmc firmware will restore the firmware in normal suspend. In previous patch added a check to directly read the hardware status and load the firmware if got reset during resume from suspend-hibernation.

---
  drivers/gpu/drm/i915/i915_drv.c | 6 ------
  1 file changed, 6 deletions(-)
drivers/gpu/drm/i915/i915_drv.c +0 -0
index 1d88745..478101c 100644
@@ -1015,12 +1015,6 @@ static int skl_suspend_complete(struct drm_i915_private *dev_priv)
  {
  	/* Enabling DC6 is not a hard requirement to enter runtime D3 */

drivers/gpu/drm/i915/i915_drv.c static int skl_suspend_complete(struct drm_i915_private *dev_priv) · line 1020

Quoted 10 lines
drivers/gpu/drm/i915/i915_drv.c +0 -6
 
-	/*
-	 * This is to ensure that CSR isn't identified as loaded before
-	 * CSR-loading program is called during runtime-resume.
-	 */
-	intel_csr_load_status_set(dev_priv, FW_UNINITIALIZED);
-
  	skl_uninit_cdclk(dev_priv);
 
  	return 0;

Valid fix and patch is ready for merge now.

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

#28 lore raw

PATCH 3/5 drm/i915/skl: Making DC6 entry is the last call in suspend flow.

In reply to Animesh Manna

Mmio register access after dc6/dc5 entry is not allowed when DC6 power states are enabled according to bspec (bspec-id 0527), so enabling dc6 as the last call in suspend flow.

v1: Initial version.

v2: Based on review comment from Daniel,
- created a seperate patch for csr uninitialization set call.

---
 drivers/gpu/drm/i915/i915_drv.c         | 13 +++++++++++++
 drivers/gpu/drm/i915/intel_drv.h        |  2 ++
 drivers/gpu/drm/i915/intel_runtime_pm.c | 19 +++++++------------
 3 files changed, 22 insertions(+), 12 deletions(-)
drivers/gpu/drm/i915/i915_drv.c +13 -0
index 478101c..fa66162 100644
@@ -1013,10 +1013,20 @@ static int i915_pm_resume(struct device *dev)
 
 static int skl_suspend_complete(struct drm_i915_private *dev_priv)
 {
+	enum csr_state state;
 	/* Enabling DC6 is not a hard requirement to enter runtime D3 */
 
 	skl_uninit_cdclk(dev_priv);
 
+	/* TODO: wait for a completion event or
+	 * similar here instead of busy
+	 * waiting using wait_for function.
+	 */
+	wait_for((state = intel_csr_load_status_get(dev_priv)) !=
+			FW_UNINITIALIZED, 1000);
+	if (state == FW_LOADED)
+		skl_enable_dc6(dev_priv);
+
 	return 0;
 }
 
@@ -1063,6 +1073,9 @@ static int skl_resume_prepare(struct drm_i915_private *dev_priv)
 {
 	struct drm_device *dev = dev_priv->dev;
 
+	if (intel_csr_load_status_get(dev_priv) == FW_LOADED)
+		skl_disable_dc6(dev_priv);
+
 	skl_init_cdclk(dev_priv);
 	intel_csr_load_program(dev);
 
drivers/gpu/drm/i915/intel_drv.h +2 -0
index 81b7d77..9cb7d4e 100644
@@ -1118,6 +1118,8 @@ void bxt_enable_dc9(struct drm_i915_private *dev_priv);
 void bxt_disable_dc9(struct drm_i915_private *dev_priv);
 void skl_init_cdclk(struct drm_i915_private *dev_priv);
 void skl_uninit_cdclk(struct drm_i915_private *dev_priv);
+void skl_enable_dc6(struct drm_i915_private *dev_priv);
+void skl_disable_dc6(struct drm_i915_private *dev_priv);
 void intel_dp_get_m_n(struct intel_crtc *crtc,
 		      struct intel_crtc_state *pipe_config);
 void intel_dp_set_m_n(struct intel_crtc *crtc, enum link_m_n_set m_n);
drivers/gpu/drm/i915/intel_runtime_pm.c +7 -12
index 821644d..23a3aa3 100644
@@ -548,7 +548,7 @@ static void assert_can_disable_dc6(struct drm_i915_private *dev_priv)
 		"DC6 already programmed to be disabled.\n");
 }
 
-static void skl_enable_dc6(struct drm_i915_private *dev_priv)
+void skl_enable_dc6(struct drm_i915_private *dev_priv)
 {
 	uint32_t val;
 
@@ -565,7 +565,7 @@ static void skl_enable_dc6(struct drm_i915_private *dev_priv)
 	POSTING_READ(DC_STATE_EN);
 }
 
-static void skl_disable_dc6(struct drm_i915_private *dev_priv)
+void skl_disable_dc6(struct drm_i915_private *dev_priv)
 {
 	uint32_t val;
 
@@ -626,10 +626,10 @@ static void skl_set_power_well(struct drm_i915_private *dev_priv,
 				!I915_READ(HSW_PWR_WELL_BIOS),
 				"Invalid for power well status to be enabled, unless done by the BIOS, \
 				when request is to disable!\n");
-			if ((GEN9_ENABLE_DC5(dev) || SKL_ENABLE_DC6(dev)) &&
-				power_well->data == SKL_DISP_PW_2) {
+			if (power_well->data == SKL_DISP_PW_2) {
+				if (GEN9_ENABLE_DC5(dev))
+					gen9_disable_dc5(dev_priv);
 				if (SKL_ENABLE_DC6(dev)) {
-					skl_disable_dc6(dev_priv);
 					/*
 					 * DDI buffer programming unnecessary during driver-load/resume
 					 * as it's already done during modeset initialization then.
@@ -637,8 +637,6 @@ static void skl_set_power_well(struct drm_i915_private *dev_priv,
 					 */
 					if (!dev_priv->power_domains.initializing)
 						intel_prepare_ddi(dev);
-				} else {
-					gen9_disable_dc5(dev_priv);
 				}
 			}
 			I915_WRITE(HSW_PWR_WELL_DRIVER, tmp | req_mask);
@@ -658,7 +656,7 @@ static void skl_set_power_well(struct drm_i915_private *dev_priv,
 			POSTING_READ(HSW_PWR_WELL_DRIVER);
 			DRM_DEBUG_KMS("Disabling %s\n", power_well->name);
 
-			if ((GEN9_ENABLE_DC5(dev) || SKL_ENABLE_DC6(dev)) &&
+			if (GEN9_ENABLE_DC5(dev) &&
 				power_well->data == SKL_DISP_PW_2) {
 				enum csr_state state;
 				/* TODO: wait for a completion event or
@@ -671,10 +669,7 @@ static void skl_set_power_well(struct drm_i915_private *dev_priv,
 					DRM_ERROR("CSR firmware not ready (%d)\n",
 							state);
 				else
-					if (SKL_ENABLE_DC6(dev))
-						skl_enable_dc6(dev_priv);
-					else
-						gen9_enable_dc5(dev_priv);
+					gen9_enable_dc5(dev_priv);
 			}
 		}
 	}
Signature · 6 lines
2.0.2

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
Reviewed-by on v1 3/5
#29 lore raw

Re: [DMC_BUGFIX_SKL_V2 3/5] drm/i915/skl: Making DC6 entry is the last call in suspend flow.

Quoted 126 lines · Animesh Manna

Mmio register access after dc6/dc5 entry is not allowed when DC6 power states are enabled according to bspec (bspec-id 0527), so enabling dc6 as the last call in suspend flow.

v1: Initial version.

v2: Based on review comment from Daniel,
- created a seperate patch for csr uninitialization set call.

---
  drivers/gpu/drm/i915/i915_drv.c         | 13 +++++++++++++
  drivers/gpu/drm/i915/intel_drv.h        |  2 ++
  drivers/gpu/drm/i915/intel_runtime_pm.c | 19 +++++++------------
  3 files changed, 22 insertions(+), 12 deletions(-)
drivers/gpu/drm/i915/i915_drv.c +13 -0
index 478101c..fa66162 100644
@@ -1013,10 +1013,20 @@ static int i915_pm_resume(struct device *dev)
 
  static int skl_suspend_complete(struct drm_i915_private *dev_priv)
  {
+	enum csr_state state;
  	/* Enabling DC6 is not a hard requirement to enter runtime D3 */
 
  	skl_uninit_cdclk(dev_priv);
 
+	/* TODO: wait for a completion event or
+	 * similar here instead of busy
+	 * waiting using wait_for function.
+	 */
+	wait_for((state = intel_csr_load_status_get(dev_priv)) !=
+			FW_UNINITIALIZED, 1000);
+	if (state == FW_LOADED)
+		skl_enable_dc6(dev_priv);
+
  	return 0;
  }
 
@@ -1063,6 +1073,9 @@ static int skl_resume_prepare(struct drm_i915_private *dev_priv)
  {
  	struct drm_device *dev = dev_priv->dev;
 
+	if (intel_csr_load_status_get(dev_priv) == FW_LOADED)
+		skl_disable_dc6(dev_priv);
+
  	skl_init_cdclk(dev_priv);
  	intel_csr_load_program(dev);
 
drivers/gpu/drm/i915/intel_drv.h +2 -0
index 81b7d77..9cb7d4e 100644
@@ -1118,6 +1118,8 @@ void bxt_enable_dc9(struct drm_i915_private *dev_priv);
  void bxt_disable_dc9(struct drm_i915_private *dev_priv);
  void skl_init_cdclk(struct drm_i915_private *dev_priv);
  void skl_uninit_cdclk(struct drm_i915_private *dev_priv);
+void skl_enable_dc6(struct drm_i915_private *dev_priv);
+void skl_disable_dc6(struct drm_i915_private *dev_priv);
  void intel_dp_get_m_n(struct intel_crtc *crtc,
  		      struct intel_crtc_state *pipe_config);
  void intel_dp_set_m_n(struct intel_crtc *crtc, enum link_m_n_set m_n);
drivers/gpu/drm/i915/intel_runtime_pm.c +6 -8
index 821644d..23a3aa3 100644
@@ -548,7 +548,7 @@ static void assert_can_disable_dc6(struct drm_i915_private *dev_priv)
  		"DC6 already programmed to be disabled.\n");
  }
 
-static void skl_enable_dc6(struct drm_i915_private *dev_priv)
+void skl_enable_dc6(struct drm_i915_private *dev_priv)
  {
  	uint32_t val;
 
@@ -565,7 +565,7 @@ static void skl_enable_dc6(struct drm_i915_private *dev_priv)
  	POSTING_READ(DC_STATE_EN);
  }
 
-static void skl_disable_dc6(struct drm_i915_private *dev_priv)
+void skl_disable_dc6(struct drm_i915_private *dev_priv)
  {
  	uint32_t val;
 
@@ -626,10 +626,10 @@ static void skl_set_power_well(struct drm_i915_private *dev_priv,
  				!I915_READ(HSW_PWR_WELL_BIOS),
  				"Invalid for power well status to be enabled, unless done by the BIOS, \
  				when request is to disable!\n");
-			if ((GEN9_ENABLE_DC5(dev) || SKL_ENABLE_DC6(dev)) &&
-				power_well->data == SKL_DISP_PW_2) {
+			if (power_well->data == SKL_DISP_PW_2) {
+				if (GEN9_ENABLE_DC5(dev))
+					gen9_disable_dc5(dev_priv);
  				if (SKL_ENABLE_DC6(dev)) {
-					skl_disable_dc6(dev_priv);
  					/*
  					 * DDI buffer programming unnecessary during driver-load/resume
  					 * as it's already done during modeset initialization then.
@@ -637,8 +637,6 @@ static void skl_set_power_well(struct drm_i915_private *dev_priv,
  					 */
  					if (!dev_priv->power_domains.initializing)
  						intel_prepare_ddi(dev);
-				} else {
-					gen9_disable_dc5(dev_priv);
  				}
  			}
  			I915_WRITE(HSW_PWR_WELL_DRIVER, tmp | req_mask);
@@ -658,7 +656,7 @@ static void skl_set_power_well(struct drm_i915_private *dev_priv,
  			POSTING_READ(HSW_PWR_WELL_DRIVER);
  			DRM_DEBUG_KMS("Disabling %s\n", power_well->name);
 
-			if ((GEN9_ENABLE_DC5(dev) || SKL_ENABLE_DC6(dev)) &&
+			if (GEN9_ENABLE_DC5(dev) &&
  				power_well->data == SKL_DISP_PW_2) {
  				enum csr_state state;
  				/* TODO: wait for a completion event or
@@ -671,10 +669,7 @@ static void skl_set_power_well(struct drm_i915_private *dev_priv,
  					DRM_ERROR("CSR firmware not ready (%d)\n",

drivers/gpu/drm/i915/intel_runtime_pm.c static void skl_set_power_well(struct drm_i915_private *dev_priv, · line 675

Quoted 10 lines
drivers/gpu/drm/i915/intel_runtime_pm.c +1 -4
  							state);
  				else
-					if (SKL_ENABLE_DC6(dev))
-						skl_enable_dc6(dev_priv);
-					else
-						gen9_enable_dc5(dev_priv);
+					gen9_enable_dc5(dev_priv);
  			}
  		}
  	}

Valid fix and patch is ready for merge now.

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

#30 lore raw

Re: [DMC_BUGFIX_SKL_V2 3/5] drm/i915/skl: Making DC6 entry is the last call in suspend flow.

In reply to Animesh Manna
Quoted 3 lines · Animesh Manna

Mmio register access after dc6/dc5 entry is not allowed when DC6 power states are enabled according to bspec (bspec-id 0527), so enabling dc6 as the last call in suspend flow.

We unconditionaly grab a power well reference for everything in our suspend/resume functions, which means dc6/5 should be prevented for long enough.

Also since dc6/5 are part of the power well framework you can't just enable/disable them directly, instead you need to get/put the corresponding power wells.

Finally this patch doesn't just do that, but also frobs around a lot in set power well code itself, and I have no idea what it does there and why. It does smell a bit like you're just breaking dc6 for runtime pm though, which wouldn't be good.

Anyway I decided to just merge this since this patch series has been floating around since forever, but then the patch didn't apply cleanly and so dropped it.
-Daniel

Quoted 140 lines

v1: Initial version.

v2: Based on review comment from Daniel,
- created a seperate patch for csr uninitialization set call.

---
 drivers/gpu/drm/i915/i915_drv.c         | 13 +++++++++++++
 drivers/gpu/drm/i915/intel_drv.h        |  2 ++
 drivers/gpu/drm/i915/intel_runtime_pm.c | 19 +++++++------------
 3 files changed, 22 insertions(+), 12 deletions(-)
drivers/gpu/drm/i915/i915_drv.c +13 -0
index 478101c..fa66162 100644
@@ -1013,10 +1013,20 @@ static int i915_pm_resume(struct device *dev)
 
 static int skl_suspend_complete(struct drm_i915_private *dev_priv)
 {
+	enum csr_state state;
 	/* Enabling DC6 is not a hard requirement to enter runtime D3 */
 
 	skl_uninit_cdclk(dev_priv);
 
+	/* TODO: wait for a completion event or
+	 * similar here instead of busy
+	 * waiting using wait_for function.
+	 */
+	wait_for((state = intel_csr_load_status_get(dev_priv)) !=
+			FW_UNINITIALIZED, 1000);
+	if (state == FW_LOADED)
+		skl_enable_dc6(dev_priv);
+
 	return 0;
 }
 
@@ -1063,6 +1073,9 @@ static int skl_resume_prepare(struct drm_i915_private *dev_priv)
 {
 	struct drm_device *dev = dev_priv->dev;
 
+	if (intel_csr_load_status_get(dev_priv) == FW_LOADED)
+		skl_disable_dc6(dev_priv);
+
 	skl_init_cdclk(dev_priv);
 	intel_csr_load_program(dev);
 
drivers/gpu/drm/i915/intel_drv.h +2 -0
index 81b7d77..9cb7d4e 100644
@@ -1118,6 +1118,8 @@ void bxt_enable_dc9(struct drm_i915_private *dev_priv);
 void bxt_disable_dc9(struct drm_i915_private *dev_priv);
 void skl_init_cdclk(struct drm_i915_private *dev_priv);
 void skl_uninit_cdclk(struct drm_i915_private *dev_priv);
+void skl_enable_dc6(struct drm_i915_private *dev_priv);
+void skl_disable_dc6(struct drm_i915_private *dev_priv);
 void intel_dp_get_m_n(struct intel_crtc *crtc,
 		      struct intel_crtc_state *pipe_config);
 void intel_dp_set_m_n(struct intel_crtc *crtc, enum link_m_n_set m_n);
drivers/gpu/drm/i915/intel_runtime_pm.c +7 -12
index 821644d..23a3aa3 100644
@@ -548,7 +548,7 @@ static void assert_can_disable_dc6(struct drm_i915_private *dev_priv)
 		"DC6 already programmed to be disabled.\n");
 }
 
-static void skl_enable_dc6(struct drm_i915_private *dev_priv)
+void skl_enable_dc6(struct drm_i915_private *dev_priv)
 {
 	uint32_t val;
 
@@ -565,7 +565,7 @@ static void skl_enable_dc6(struct drm_i915_private *dev_priv)
 	POSTING_READ(DC_STATE_EN);
 }
 
-static void skl_disable_dc6(struct drm_i915_private *dev_priv)
+void skl_disable_dc6(struct drm_i915_private *dev_priv)
 {
 	uint32_t val;
 
@@ -626,10 +626,10 @@ static void skl_set_power_well(struct drm_i915_private *dev_priv,
 				!I915_READ(HSW_PWR_WELL_BIOS),
 				"Invalid for power well status to be enabled, unless done by the BIOS, \
 				when request is to disable!\n");
-			if ((GEN9_ENABLE_DC5(dev) || SKL_ENABLE_DC6(dev)) &&
-				power_well->data == SKL_DISP_PW_2) {
+			if (power_well->data == SKL_DISP_PW_2) {
+				if (GEN9_ENABLE_DC5(dev))
+					gen9_disable_dc5(dev_priv);
 				if (SKL_ENABLE_DC6(dev)) {
-					skl_disable_dc6(dev_priv);
 					/*
 					 * DDI buffer programming unnecessary during driver-load/resume
 					 * as it's already done during modeset initialization then.
@@ -637,8 +637,6 @@ static void skl_set_power_well(struct drm_i915_private *dev_priv,
 					 */
 					if (!dev_priv->power_domains.initializing)
 						intel_prepare_ddi(dev);
-				} else {
-					gen9_disable_dc5(dev_priv);
 				}
 			}
 			I915_WRITE(HSW_PWR_WELL_DRIVER, tmp | req_mask);
@@ -658,7 +656,7 @@ static void skl_set_power_well(struct drm_i915_private *dev_priv,
 			POSTING_READ(HSW_PWR_WELL_DRIVER);
 			DRM_DEBUG_KMS("Disabling %s\n", power_well->name);
 
-			if ((GEN9_ENABLE_DC5(dev) || SKL_ENABLE_DC6(dev)) &&
+			if (GEN9_ENABLE_DC5(dev) &&
 				power_well->data == SKL_DISP_PW_2) {
 				enum csr_state state;
 				/* TODO: wait for a completion event or
@@ -671,10 +669,7 @@ static void skl_set_power_well(struct drm_i915_private *dev_priv,
 					DRM_ERROR("CSR firmware not ready (%d)\n",
 							state);
 				else
-					if (SKL_ENABLE_DC6(dev))
-						skl_enable_dc6(dev_priv);
-					else
-						gen9_enable_dc5(dev_priv);
+					gen9_enable_dc5(dev_priv);
 			}
 		}
 	}
Signature · 6 lines
2.0.2

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#31 lore raw

Re: [DMC_BUGFIX_SKL_V2 3/5] drm/i915/skl: Making DC6 entry is the last call in suspend flow.

You believe suspend w/ DC6 should work due to the unconditional power-well reference, or you don't know and just couldn't merge since the patch didn’t apply?

Gavin Hindman
Senior Program Manager
SSG/OTC – Open Source Technology Center

-----Original Message-----
From: Intel-gfx [mailto:[email protected]] On Behalf Of Daniel Vetter
Sent: Monday, September 28, 2015 12:22 AM
To: Manna, Animesh <[email protected]>
Cc: Bhardwaj, Rajneesh <[email protected]>; [email protected]; Vetter, Daniel <[email protected]>
Subject: Re: [Intel-gfx] [DMC_BUGFIX_SKL_V2 3/5] drm/i915/skl: Making DC6 entry is the last call in suspend flow.

Quoted 3 lines · Animesh Manna

Mmio register access after dc6/dc5 entry is not allowed when DC6 power states are enabled according to bspec (bspec-id 0527), so enabling dc6 as the last call in suspend flow.

We unconditionaly grab a power well reference for everything in our suspend/resume functions, which means dc6/5 should be prevented for long enough.

Also since dc6/5 are part of the power well framework you can't just enable/disable them directly, instead you need to get/put the corresponding power wells.

Finally this patch doesn't just do that, but also frobs around a lot in set power well code itself, and I have no idea what it does there and why. It does smell a bit like you're just breaking dc6 for runtime pm though, which wouldn't be good.

Anyway I decided to just merge this since this patch series has been floating around since forever, but then the patch didn't apply cleanly and so dropped it. -Daniel

Quoted 136 lines

v1: Initial version.

v2: Based on review comment from Daniel,
- created a seperate patch for csr uninitialization set call.

---
 drivers/gpu/drm/i915/i915_drv.c         | 13 +++++++++++++
 drivers/gpu/drm/i915/intel_drv.h        |  2 ++
 drivers/gpu/drm/i915/intel_runtime_pm.c | 19 +++++++------------
 3 files changed, 22 insertions(+), 12 deletions(-)
(quoted diff) +0 -0

b/drivers/gpu/drm/i915/i915_drv.c index 478101c..fa66162 100644

drivers/gpu/drm/i915/i915_drv.c +13 -0
@@ -1013,10 +1013,20 @@ static int i915_pm_resume(struct device *dev)
 
 static int skl_suspend_complete(struct drm_i915_private *dev_priv)  {
+	enum csr_state state;
 	/* Enabling DC6 is not a hard requirement to enter runtime D3 */
 
 	skl_uninit_cdclk(dev_priv);
 
+	/* TODO: wait for a completion event or
+	 * similar here instead of busy
+	 * waiting using wait_for function.
+	 */
+	wait_for((state = intel_csr_load_status_get(dev_priv)) !=
+			FW_UNINITIALIZED, 1000);
+	if (state == FW_LOADED)
+		skl_enable_dc6(dev_priv);
+
 	return 0;
 }
 
@@ -1063,6 +1073,9 @@ static int skl_resume_prepare(struct 
 drm_i915_private *dev_priv)  {
 	struct drm_device *dev = dev_priv->dev;
 
+	if (intel_csr_load_status_get(dev_priv) == FW_LOADED)
+		skl_disable_dc6(dev_priv);
+
 	skl_init_cdclk(dev_priv);
 	intel_csr_load_program(dev);
 
drivers/gpu/drm/i915/intel_drv.h +0 -0
index 81b7d77..9cb7d4e 100644
@@ -1118,6 +1118,8 @@ void bxt_enable_dc9(struct drm_i915_private 
 *dev_priv);  void bxt_disable_dc9(struct drm_i915_private *dev_priv);  

void skl_init_cdclk(struct drm_i915_private *dev_priv); void skl_uninit_cdclk(struct drm_i915_private *dev_priv);

drivers/gpu/drm/i915/intel_drv.h +2 -0
+void skl_enable_dc6(struct drm_i915_private *dev_priv); void 
+skl_disable_dc6(struct drm_i915_private *dev_priv);
 void intel_dp_get_m_n(struct intel_crtc *crtc,
 		      struct intel_crtc_state *pipe_config);  void 

intel_dp_set_m_n(struct intel_crtc *crtc, enum link_m_n_set m_n); diff --git a/drivers/gpu/drm/i915/intel_runtime_pm.c b/drivers/gpu/drm/i915/intel_runtime_pm.c
index 821644d..23a3aa3 100644

drivers/gpu/drm/i915/intel_runtime_pm.c +5 -7
@@ -548,7 +548,7 @@ static void assert_can_disable_dc6(struct drm_i915_private *dev_priv)
 		"DC6 already programmed to be disabled.\n");  }
 
-static void skl_enable_dc6(struct drm_i915_private *dev_priv)
+void skl_enable_dc6(struct drm_i915_private *dev_priv)
 {
 	uint32_t val;
 
@@ -565,7 +565,7 @@ static void skl_enable_dc6(struct drm_i915_private *dev_priv)
 	POSTING_READ(DC_STATE_EN);
 }
 
-static void skl_disable_dc6(struct drm_i915_private *dev_priv)
+void skl_disable_dc6(struct drm_i915_private *dev_priv)
 {
 	uint32_t val;
 
@@ -626,10 +626,10 @@ static void skl_set_power_well(struct drm_i915_private *dev_priv,
 				!I915_READ(HSW_PWR_WELL_BIOS),
 				"Invalid for power well status to be enabled, unless done by the BIOS, \
 				when request is to disable!\n");
-			if ((GEN9_ENABLE_DC5(dev) || SKL_ENABLE_DC6(dev)) &&
-				power_well->data == SKL_DISP_PW_2) {
+			if (power_well->data == SKL_DISP_PW_2) {
+				if (GEN9_ENABLE_DC5(dev))
+					gen9_disable_dc5(dev_priv);
 				if (SKL_ENABLE_DC6(dev)) {
-					skl_disable_dc6(dev_priv);
 					/*
 					 * DDI buffer programming unnecessary during driver-load/resume
 					 * as it's already done during modeset initialization then.
@@ -637,8 +637,6 @@ static void skl_set_power_well(struct drm_i915_private *dev_priv,
 					 */
 					if (!dev_priv->power_domains.initializing)
 						intel_prepare_ddi(dev);
-				} else {
-					gen9_disable_dc5(dev_priv);
 				}
 			}
 			I915_WRITE(HSW_PWR_WELL_DRIVER, tmp | req_mask); @@ -658,7 +656,7 

@@ static void skl_set_power_well(struct drm_i915_private *dev_priv,

drivers/gpu/drm/i915/intel_runtime_pm.c +1 -1
 			POSTING_READ(HSW_PWR_WELL_DRIVER);
 			DRM_DEBUG_KMS("Disabling %s\n", power_well->name);
 
-			if ((GEN9_ENABLE_DC5(dev) || SKL_ENABLE_DC6(dev)) &&
+			if (GEN9_ENABLE_DC5(dev) &&
 				power_well->data == SKL_DISP_PW_2) {
 				enum csr_state state;
 				/* TODO: wait for a completion event or @@ -671,10 +669,7 @@ 

static void skl_set_power_well(struct drm_i915_private *dev_priv,

drivers/gpu/drm/i915/intel_runtime_pm.c +1 -5
 					DRM_ERROR("CSR firmware not ready (%d)\n",
 							state);
 				else
-					if (SKL_ENABLE_DC6(dev))
-						skl_enable_dc6(dev_priv);
-					else
-						gen9_enable_dc5(dev_priv);
+					gen9_enable_dc5(dev_priv);
 			}
 		}
 	}
--

2.0.2

Quoted 4 lines

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

#32 lore raw

[DMC_BUGFIX_V3] drm/i915/skl: Making DC6 entry is the last call in suspend flow.

In reply to Daniel Vetter

Mmio register access after dc6/dc5 entry is not allowed when DC6 power states are enabled according to bspec (bspec-id 0527), so enabling dc6 as the last call in suspend flow.

v1: Initial version.

v2: Based on review comment from Daniel,
- created a seperate patch for csr uninitialization set call.

v3: Rebased on top of latest code.

---
 drivers/gpu/drm/i915/i915_drv.c         | 13 +++++++++++++
 drivers/gpu/drm/i915/intel_drv.h        |  2 ++
 drivers/gpu/drm/i915/intel_runtime_pm.c | 19 +++++++------------
 3 files changed, 22 insertions(+), 12 deletions(-)
drivers/gpu/drm/i915/i915_drv.c +13 -0
index 1cb6b82..51075d5 100644
@@ -1049,10 +1049,20 @@ static int i915_pm_resume(struct device *dev)
 
 static int skl_suspend_complete(struct drm_i915_private *dev_priv)
 {
+	enum csr_state state;
 	/* Enabling DC6 is not a hard requirement to enter runtime D3 */
 
 	skl_uninit_cdclk(dev_priv);
 
+	/* TODO: wait for a completion event or
+	 * similar here instead of busy
+	 * waiting using wait_for function.
+	 */
+	wait_for((state = intel_csr_load_status_get(dev_priv)) !=
+			FW_UNINITIALIZED, 1000);
+	if (state == FW_LOADED)
+		skl_enable_dc6(dev_priv);
+
 	return 0;
 }
 
@@ -1099,6 +1109,9 @@ static int skl_resume_prepare(struct drm_i915_private *dev_priv)
 {
 	struct drm_device *dev = dev_priv->dev;
 
+	if (intel_csr_load_status_get(dev_priv) == FW_LOADED)
+		skl_disable_dc6(dev_priv);
+
 	skl_init_cdclk(dev_priv);
 	intel_csr_load_program(dev);
 
drivers/gpu/drm/i915/intel_drv.h +2 -0
index c96289d..990161d 100644
@@ -1143,6 +1143,8 @@ void bxt_enable_dc9(struct drm_i915_private *dev_priv);
 void bxt_disable_dc9(struct drm_i915_private *dev_priv);
 void skl_init_cdclk(struct drm_i915_private *dev_priv);
 void skl_uninit_cdclk(struct drm_i915_private *dev_priv);
+void skl_enable_dc6(struct drm_i915_private *dev_priv);
+void skl_disable_dc6(struct drm_i915_private *dev_priv);
 void intel_dp_get_m_n(struct intel_crtc *crtc,
 		      struct intel_crtc_state *pipe_config);
 void intel_dp_set_m_n(struct intel_crtc *crtc, enum link_m_n_set m_n);
drivers/gpu/drm/i915/intel_runtime_pm.c +7 -12
index d8e9416..d6b4f61 100644
@@ -551,7 +551,7 @@ static void assert_can_disable_dc6(struct drm_i915_private *dev_priv)
 		  "DC6 already programmed to be disabled.\n");
 }
 
-static void skl_enable_dc6(struct drm_i915_private *dev_priv)
+void skl_enable_dc6(struct drm_i915_private *dev_priv)
 {
 	uint32_t val;
 
@@ -568,7 +568,7 @@ static void skl_enable_dc6(struct drm_i915_private *dev_priv)
 	POSTING_READ(DC_STATE_EN);
 }
 
-static void skl_disable_dc6(struct drm_i915_private *dev_priv)
+void skl_disable_dc6(struct drm_i915_private *dev_priv)
 {
 	uint32_t val;
 
@@ -629,10 +629,10 @@ static void skl_set_power_well(struct drm_i915_private *dev_priv,
 				!I915_READ(HSW_PWR_WELL_BIOS),
 				"Invalid for power well status to be enabled, unless done by the BIOS, \
 				when request is to disable!\n");
-			if ((GEN9_ENABLE_DC5(dev) || SKL_ENABLE_DC6(dev)) &&
-				power_well->data == SKL_DISP_PW_2) {
+			if (power_well->data == SKL_DISP_PW_2) {
+				if (GEN9_ENABLE_DC5(dev))
+					gen9_disable_dc5(dev_priv);
 				if (SKL_ENABLE_DC6(dev)) {
-					skl_disable_dc6(dev_priv);
 					/*
 					 * DDI buffer programming unnecessary during driver-load/resume
 					 * as it's already done during modeset initialization then.
@@ -640,8 +640,6 @@ static void skl_set_power_well(struct drm_i915_private *dev_priv,
 					 */
 					if (!dev_priv->power_domains.initializing)
 						intel_prepare_ddi(dev);
-				} else {
-					gen9_disable_dc5(dev_priv);
 				}
 			}
 			I915_WRITE(HSW_PWR_WELL_DRIVER, tmp | req_mask);
@@ -667,7 +665,7 @@ static void skl_set_power_well(struct drm_i915_private *dev_priv,
 				DRM_DEBUG_KMS("Disabling %s\n", power_well->name);
 			}
 
-			if ((GEN9_ENABLE_DC5(dev) || SKL_ENABLE_DC6(dev)) &&
+			if (GEN9_ENABLE_DC5(dev) &&
 				power_well->data == SKL_DISP_PW_2) {
 				enum csr_state state;
 				/* TODO: wait for a completion event or
@@ -680,10 +678,7 @@ static void skl_set_power_well(struct drm_i915_private *dev_priv,
 					DRM_DEBUG("CSR firmware not ready (%d)\n",
 							state);
 				else
-					if (SKL_ENABLE_DC6(dev))
-						skl_enable_dc6(dev_priv);
-					else
-						gen9_enable_dc5(dev_priv);
+					gen9_enable_dc5(dev_priv);
 			}
 		}
 	}
Signature · 6 lines
2.0.2

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
Acked-by
#33 lore raw

Re: [DMC_BUGFIX_V3] drm/i915/skl: Making DC6 entry is the last call in suspend flow.

Quoted 11 lines · Animesh Manna

Mmio register access after dc6/dc5 entry is not allowed when DC6 power states are enabled according to bspec (bspec-id 0527), so enabling dc6 as the last call in suspend flow.

v1: Initial version.

v2: Based on review comment from Daniel,
- created a seperate patch for csr uninitialization set call.

v3: Rebased on top of latest code.

Quoted 7 lines

Suggestion for the commit message:

Currently we keep DC6 enabled during modesets and DPAUX transfers, which is not allowed according to the specification. This can lead at least to PLL locking failures, DPAUX timeouts and prevent deeper package power states (PC9/10). Fix this for now by enabling DC6 only when we know the above events (modeset, DPAUX) can't happen.

This a temporary solution as some issues are still unsolved as described in [1] and [2], we'll address those as a follow-up.

[1]
http://lists.freedesktop.org/archives/intel-gfx/2015-October/077669.html
[2]
http://lists.freedesktop.org/archives/intel-gfx/2015-October/077787.html

Quoted 110 lines
---
 drivers/gpu/drm/i915/i915_drv.c         | 13 +++++++++++++
 drivers/gpu/drm/i915/intel_drv.h        |  2 ++
 drivers/gpu/drm/i915/intel_runtime_pm.c | 19 +++++++------------
 3 files changed, 22 insertions(+), 12 deletions(-)
drivers/gpu/drm/i915/i915_drv.c +13 -0
index 1cb6b82..51075d5 100644
@@ -1049,10 +1049,20 @@ static int i915_pm_resume(struct device *dev)
 
 static int skl_suspend_complete(struct drm_i915_private *dev_priv)
 {
+	enum csr_state state;
 	/* Enabling DC6 is not a hard requirement to enter runtime D3 */
 
 	skl_uninit_cdclk(dev_priv);
 
+	/* TODO: wait for a completion event or
+	 * similar here instead of busy
+	 * waiting using wait_for function.
+	 */
+	wait_for((state = intel_csr_load_status_get(dev_priv)) !=
+			FW_UNINITIALIZED, 1000);
+	if (state == FW_LOADED)
+		skl_enable_dc6(dev_priv);
+
 	return 0;
 }
 
@@ -1099,6 +1109,9 @@ static int skl_resume_prepare(struct drm_i915_private *dev_priv)
 {
 	struct drm_device *dev = dev_priv->dev;
 
+	if (intel_csr_load_status_get(dev_priv) == FW_LOADED)
+		skl_disable_dc6(dev_priv);
+
 	skl_init_cdclk(dev_priv);
 	intel_csr_load_program(dev);
 
drivers/gpu/drm/i915/intel_drv.h +2 -0
index c96289d..990161d 100644
@@ -1143,6 +1143,8 @@ void bxt_enable_dc9(struct drm_i915_private *dev_priv);
 void bxt_disable_dc9(struct drm_i915_private *dev_priv);
 void skl_init_cdclk(struct drm_i915_private *dev_priv);
 void skl_uninit_cdclk(struct drm_i915_private *dev_priv);
+void skl_enable_dc6(struct drm_i915_private *dev_priv);
+void skl_disable_dc6(struct drm_i915_private *dev_priv);
 void intel_dp_get_m_n(struct intel_crtc *crtc,
 		      struct intel_crtc_state *pipe_config);
 void intel_dp_set_m_n(struct intel_crtc *crtc, enum link_m_n_set m_n);
drivers/gpu/drm/i915/intel_runtime_pm.c +6 -8
index d8e9416..d6b4f61 100644
@@ -551,7 +551,7 @@ static void assert_can_disable_dc6(struct drm_i915_private *dev_priv)
 		  "DC6 already programmed to be disabled.\n");
 }
 
-static void skl_enable_dc6(struct drm_i915_private *dev_priv)
+void skl_enable_dc6(struct drm_i915_private *dev_priv)
 {
 	uint32_t val;
 
@@ -568,7 +568,7 @@ static void skl_enable_dc6(struct drm_i915_private *dev_priv)
 	POSTING_READ(DC_STATE_EN);
 }
 
-static void skl_disable_dc6(struct drm_i915_private *dev_priv)
+void skl_disable_dc6(struct drm_i915_private *dev_priv)
 {
 	uint32_t val;
 
@@ -629,10 +629,10 @@ static void skl_set_power_well(struct drm_i915_private *dev_priv,
 				!I915_READ(HSW_PWR_WELL_BIOS),
 				"Invalid for power well status to be enabled, unless done by the BIOS, \
 				when request is to disable!\n");
-			if ((GEN9_ENABLE_DC5(dev) || SKL_ENABLE_DC6(dev)) &&
-				power_well->data == SKL_DISP_PW_2) {
+			if (power_well->data == SKL_DISP_PW_2) {
+				if (GEN9_ENABLE_DC5(dev))
+					gen9_disable_dc5(dev_priv);
 				if (SKL_ENABLE_DC6(dev)) {
-					skl_disable_dc6(dev_priv);
 					/*
 					 * DDI buffer programming unnecessary during driver-load/resume
 					 * as it's already done during modeset initialization then.
@@ -640,8 +640,6 @@ static void skl_set_power_well(struct drm_i915_private *dev_priv,
 					 */
 					if (!dev_priv->power_domains.initializing)
 						intel_prepare_ddi(dev);
-				} else {
-					gen9_disable_dc5(dev_priv);
 				}
 			}
 			I915_WRITE(HSW_PWR_WELL_DRIVER, tmp | req_mask);
@@ -667,7 +665,7 @@ static void skl_set_power_well(struct drm_i915_private *dev_priv,
 				DRM_DEBUG_KMS("Disabling %s\n", power_well->name);
 			}
 
-			if ((GEN9_ENABLE_DC5(dev) || SKL_ENABLE_DC6(dev)) &&
+			if (GEN9_ENABLE_DC5(dev) &&
 				power_well->data == SKL_DISP_PW_2) {
 				enum csr_state state;
 				/* TODO: wait for a completion event or
@@ -680,10 +678,7 @@ static void skl_set_power_well(struct drm_i915_private *dev_priv,
 					DRM_DEBUG("CSR firmware not ready (%d)\n",

drivers/gpu/drm/i915/intel_runtime_pm.c static void skl_set_power_well(struct drm_i915_private *dev_priv, · line 684

Quoted 10 lines
drivers/gpu/drm/i915/intel_runtime_pm.c +1 -4
 							state);
 				else
-					if (SKL_ENABLE_DC6(dev))
-						skl_enable_dc6(dev_priv);
-					else
-						gen9_enable_dc5(dev_priv);
+					gen9_enable_dc5(dev_priv);
 			}
 		}
 	}

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

#34 lore raw

Re: [DMC_BUGFIX_V3] drm/i915/skl: Making DC6 entry is the last call in suspend flow.

Quoted 30 lines · Imre Deak
Quoted 11 lines · Animesh Manna

Mmio register access after dc6/dc5 entry is not allowed when DC6 power states are enabled according to bspec (bspec-id 0527), so enabling dc6 as the last call in suspend flow.

v1: Initial version.

v2: Based on review comment from Daniel,
- created a seperate patch for csr uninitialization set call.

v3: Rebased on top of latest code.

Quoted 7 lines

Suggestion for the commit message:

Currently we keep DC6 enabled during modesets and DPAUX transfers, which is not allowed according to the specification. This can lead at least to PLL locking failures, DPAUX timeouts and prevent deeper package power states (PC9/10). Fix this for now by enabling DC6 only when we know the above events (modeset, DPAUX) can't happen.

Quoted 7 lines

This a temporary solution as some issues are still unsolved as described in [1] and [2], we'll address those as a follow-up.

[1]
http://lists.freedesktop.org/archives/intel-gfx/2015-October/077669.html
[2]
http://lists.freedesktop.org/archives/intel-gfx/2015-October/077787.html

Queued for -next, thanks for the patch.
-Daniel

Quoted 128 lines
Quoted 112 lines

---
 drivers/gpu/drm/i915/i915_drv.c         | 13 +++++++++++++
 drivers/gpu/drm/i915/intel_drv.h        |  2 ++
 drivers/gpu/drm/i915/intel_runtime_pm.c | 19 +++++++------------
 3 files changed, 22 insertions(+), 12 deletions(-)
drivers/gpu/drm/i915/i915_drv.c +13 -0
index 1cb6b82..51075d5 100644
@@ -1049,10 +1049,20 @@ static int i915_pm_resume(struct device *dev)
 
 static int skl_suspend_complete(struct drm_i915_private *dev_priv)
 {
+	enum csr_state state;
 	/* Enabling DC6 is not a hard requirement to enter runtime D3 */
 
 	skl_uninit_cdclk(dev_priv);
 
+	/* TODO: wait for a completion event or
+	 * similar here instead of busy
+	 * waiting using wait_for function.
+	 */
+	wait_for((state = intel_csr_load_status_get(dev_priv)) !=
+			FW_UNINITIALIZED, 1000);
+	if (state == FW_LOADED)
+		skl_enable_dc6(dev_priv);
+
 	return 0;
 }
 
@@ -1099,6 +1109,9 @@ static int skl_resume_prepare(struct drm_i915_private *dev_priv)
 {
 	struct drm_device *dev = dev_priv->dev;
 
+	if (intel_csr_load_status_get(dev_priv) == FW_LOADED)
+		skl_disable_dc6(dev_priv);
+
 	skl_init_cdclk(dev_priv);
 	intel_csr_load_program(dev);
 
drivers/gpu/drm/i915/intel_drv.h +2 -0
index c96289d..990161d 100644
@@ -1143,6 +1143,8 @@ void bxt_enable_dc9(struct drm_i915_private *dev_priv);
 void bxt_disable_dc9(struct drm_i915_private *dev_priv);
 void skl_init_cdclk(struct drm_i915_private *dev_priv);
 void skl_uninit_cdclk(struct drm_i915_private *dev_priv);
+void skl_enable_dc6(struct drm_i915_private *dev_priv);
+void skl_disable_dc6(struct drm_i915_private *dev_priv);
 void intel_dp_get_m_n(struct intel_crtc *crtc,
 		      struct intel_crtc_state *pipe_config);
 void intel_dp_set_m_n(struct intel_crtc *crtc, enum link_m_n_set m_n);
drivers/gpu/drm/i915/intel_runtime_pm.c +6 -8
index d8e9416..d6b4f61 100644
@@ -551,7 +551,7 @@ static void assert_can_disable_dc6(struct drm_i915_private *dev_priv)
 		  "DC6 already programmed to be disabled.\n");
 }
 
-static void skl_enable_dc6(struct drm_i915_private *dev_priv)
+void skl_enable_dc6(struct drm_i915_private *dev_priv)
 {
 	uint32_t val;
 
@@ -568,7 +568,7 @@ static void skl_enable_dc6(struct drm_i915_private *dev_priv)
 	POSTING_READ(DC_STATE_EN);
 }
 
-static void skl_disable_dc6(struct drm_i915_private *dev_priv)
+void skl_disable_dc6(struct drm_i915_private *dev_priv)
 {
 	uint32_t val;
 
@@ -629,10 +629,10 @@ static void skl_set_power_well(struct drm_i915_private *dev_priv,
 				!I915_READ(HSW_PWR_WELL_BIOS),
 				"Invalid for power well status to be enabled, unless done by the BIOS, \
 				when request is to disable!\n");
-			if ((GEN9_ENABLE_DC5(dev) || SKL_ENABLE_DC6(dev)) &&
-				power_well->data == SKL_DISP_PW_2) {
+			if (power_well->data == SKL_DISP_PW_2) {
+				if (GEN9_ENABLE_DC5(dev))
+					gen9_disable_dc5(dev_priv);
 				if (SKL_ENABLE_DC6(dev)) {
-					skl_disable_dc6(dev_priv);
 					/*
 					 * DDI buffer programming unnecessary during driver-load/resume
 					 * as it's already done during modeset initialization then.
@@ -640,8 +640,6 @@ static void skl_set_power_well(struct drm_i915_private *dev_priv,
 					 */
 					if (!dev_priv->power_domains.initializing)
 						intel_prepare_ddi(dev);
-				} else {
-					gen9_disable_dc5(dev_priv);
 				}
 			}
 			I915_WRITE(HSW_PWR_WELL_DRIVER, tmp | req_mask);
@@ -667,7 +665,7 @@ static void skl_set_power_well(struct drm_i915_private *dev_priv,
 				DRM_DEBUG_KMS("Disabling %s\n", power_well->name);
 			}
 
-			if ((GEN9_ENABLE_DC5(dev) || SKL_ENABLE_DC6(dev)) &&
+			if (GEN9_ENABLE_DC5(dev) &&
 				power_well->data == SKL_DISP_PW_2) {
 				enum csr_state state;
 				/* TODO: wait for a completion event or
@@ -680,10 +678,7 @@ static void skl_set_power_well(struct drm_i915_private *dev_priv,
 					DRM_DEBUG("CSR firmware not ready (%d)\n",

drivers/gpu/drm/i915/intel_runtime_pm.c static void skl_set_power_well(struct drm_i915_private *dev_priv, · line 684

Quoted 10 lines
drivers/gpu/drm/i915/intel_runtime_pm.c +1 -4
 							state);
 				else
-					if (SKL_ENABLE_DC6(dev))
-						skl_enable_dc6(dev_priv);
-					else
-						gen9_enable_dc5(dev_priv);
+					gen9_enable_dc5(dev_priv);
 			}
 		}
 	}

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#35 lore raw

Re: [DMC_BUGFIX_SKL_V2 3/5] drm/i915/skl: Making DC6 entry is the last call in suspend flow.

In reply to Daniel Vetter
Quoted 12 lines · Daniel Vetter
Quoted 3 lines · Animesh Manna

Mmio register access after dc6/dc5 entry is not allowed when DC6 power states are enabled according to bspec (bspec-id 0527), so enabling dc6 as the last call in suspend flow.

We unconditionaly grab a power well reference for everything in our suspend/resume functions, which means dc6/5 should be prevented for long enough.

Also since dc6/5 are part of the power well framework you can't just enable/disable them directly, instead you need to get/put the corresponding power wells.

Quoted 8 lines

Finally this patch doesn't just do that, but also frobs around a lot in set power well code itself, and I have no idea what it does there and why. It does smell a bit like you're just breaking dc6 for runtime pm though, which wouldn't be good.

Anyway I decided to just merge this since this patch series has been floating around since forever, but then the patch didn't apply cleanly and so dropped it.

I mentioned in my commit message that we have a h/w workaround. "Mmio register access after dc6/dc5 entry is not allowed when DC6 power states are enabled"

This patch is mandatory to solve the pc10 entry issue for skylake. Planning to send again after rebase on top of tree.

-Animesh

Quoted 137 lines

-Daniel

Quoted 135 lines

v1: Initial version.

v2: Based on review comment from Daniel,
- created a seperate patch for csr uninitialization set call.

---
  drivers/gpu/drm/i915/i915_drv.c         | 13 +++++++++++++
  drivers/gpu/drm/i915/intel_drv.h        |  2 ++
  drivers/gpu/drm/i915/intel_runtime_pm.c | 19 +++++++------------
  3 files changed, 22 insertions(+), 12 deletions(-)
drivers/gpu/drm/i915/i915_drv.c +13 -0
index 478101c..fa66162 100644
@@ -1013,10 +1013,20 @@ static int i915_pm_resume(struct device *dev)
 
  static int skl_suspend_complete(struct drm_i915_private *dev_priv)
  {
+	enum csr_state state;
  	/* Enabling DC6 is not a hard requirement to enter runtime D3 */
 
  	skl_uninit_cdclk(dev_priv);
 
+	/* TODO: wait for a completion event or
+	 * similar here instead of busy
+	 * waiting using wait_for function.
+	 */
+	wait_for((state = intel_csr_load_status_get(dev_priv)) !=
+			FW_UNINITIALIZED, 1000);
+	if (state == FW_LOADED)
+		skl_enable_dc6(dev_priv);
+
  	return 0;
  }
 
@@ -1063,6 +1073,9 @@ static int skl_resume_prepare(struct drm_i915_private *dev_priv)
  {
  	struct drm_device *dev = dev_priv->dev;
 
+	if (intel_csr_load_status_get(dev_priv) == FW_LOADED)
+		skl_disable_dc6(dev_priv);
+
  	skl_init_cdclk(dev_priv);
  	intel_csr_load_program(dev);
 
drivers/gpu/drm/i915/intel_drv.h +2 -0
index 81b7d77..9cb7d4e 100644
@@ -1118,6 +1118,8 @@ void bxt_enable_dc9(struct drm_i915_private *dev_priv);
  void bxt_disable_dc9(struct drm_i915_private *dev_priv);
  void skl_init_cdclk(struct drm_i915_private *dev_priv);
  void skl_uninit_cdclk(struct drm_i915_private *dev_priv);
+void skl_enable_dc6(struct drm_i915_private *dev_priv);
+void skl_disable_dc6(struct drm_i915_private *dev_priv);
  void intel_dp_get_m_n(struct intel_crtc *crtc,
  		      struct intel_crtc_state *pipe_config);
  void intel_dp_set_m_n(struct intel_crtc *crtc, enum link_m_n_set m_n);
drivers/gpu/drm/i915/intel_runtime_pm.c +7 -12
index 821644d..23a3aa3 100644
@@ -548,7 +548,7 @@ static void assert_can_disable_dc6(struct drm_i915_private *dev_priv)
  		"DC6 already programmed to be disabled.\n");
  }
 
-static void skl_enable_dc6(struct drm_i915_private *dev_priv)
+void skl_enable_dc6(struct drm_i915_private *dev_priv)
  {
  	uint32_t val;
 
@@ -565,7 +565,7 @@ static void skl_enable_dc6(struct drm_i915_private *dev_priv)
  	POSTING_READ(DC_STATE_EN);
  }
 
-static void skl_disable_dc6(struct drm_i915_private *dev_priv)
+void skl_disable_dc6(struct drm_i915_private *dev_priv)
  {
  	uint32_t val;
 
@@ -626,10 +626,10 @@ static void skl_set_power_well(struct drm_i915_private *dev_priv,
  				!I915_READ(HSW_PWR_WELL_BIOS),
  				"Invalid for power well status to be enabled, unless done by the BIOS, \
  				when request is to disable!\n");
-			if ((GEN9_ENABLE_DC5(dev) || SKL_ENABLE_DC6(dev)) &&
-				power_well->data == SKL_DISP_PW_2) {
+			if (power_well->data == SKL_DISP_PW_2) {
+				if (GEN9_ENABLE_DC5(dev))
+					gen9_disable_dc5(dev_priv);
  				if (SKL_ENABLE_DC6(dev)) {
-					skl_disable_dc6(dev_priv);
  					/*
  					 * DDI buffer programming unnecessary during driver-load/resume
  					 * as it's already done during modeset initialization then.
@@ -637,8 +637,6 @@ static void skl_set_power_well(struct drm_i915_private *dev_priv,
  					 */
  					if (!dev_priv->power_domains.initializing)
  						intel_prepare_ddi(dev);
-				} else {
-					gen9_disable_dc5(dev_priv);
  				}
  			}
  			I915_WRITE(HSW_PWR_WELL_DRIVER, tmp | req_mask);
@@ -658,7 +656,7 @@ static void skl_set_power_well(struct drm_i915_private *dev_priv,
  			POSTING_READ(HSW_PWR_WELL_DRIVER);
  			DRM_DEBUG_KMS("Disabling %s\n", power_well->name);
 
-			if ((GEN9_ENABLE_DC5(dev) || SKL_ENABLE_DC6(dev)) &&
+			if (GEN9_ENABLE_DC5(dev) &&
  				power_well->data == SKL_DISP_PW_2) {
  				enum csr_state state;
  				/* TODO: wait for a completion event or
@@ -671,10 +669,7 @@ static void skl_set_power_well(struct drm_i915_private *dev_priv,
  					DRM_ERROR("CSR firmware not ready (%d)\n",
  							state);
  				else
-					if (SKL_ENABLE_DC6(dev))
-						skl_enable_dc6(dev_priv);
-					else
-						gen9_enable_dc5(dev_priv);
+					gen9_enable_dc5(dev_priv);
  			}
  		}
  	}
Signature · 1 line
2.0.2
Quoted 4 lines
Quoted 4 lines

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

#36 lore raw

Re: [DMC_BUGFIX_SKL_V2 3/5] drm/i915/skl: Making DC6 entry is the last call in suspend flow.

Quoted 24 lines · Animesh Manna
Quoted 12 lines · Daniel Vetter
Quoted 3 lines · Animesh Manna

Mmio register access after dc6/dc5 entry is not allowed when DC6 power states are enabled according to bspec (bspec-id 0527), so enabling dc6 as the last call in suspend flow.

We unconditionaly grab a power well reference for everything in our suspend/resume functions, which means dc6/5 should be prevented for long enough.

Also since dc6/5 are part of the power well framework you can't just enable/disable them directly, instead you need to get/put the corresponding power wells.

Quoted 8 lines

Finally this patch doesn't just do that, but also frobs around a lot in set power well code itself, and I have no idea what it does there and why. It does smell a bit like you're just breaking dc6 for runtime pm though, which wouldn't be good.

Anyway I decided to just merge this since this patch series has been floating around since forever, but then the patch didn't apply cleanly and so dropped it.

Quoted 6 lines

I mentioned in my commit message that we have a h/w workaround. "Mmio register access after dc6/dc5 entry is not allowed when DC6 power states are enabled"

This patch is mandatory to solve the pc10 entry issue for skylake. Planning to send again after rebase on top of tree.

The problem isn't that the patch didn't apply cleanly, but that it seems to have fundamental issues:
- We already grab power well references around suspend/resume code, see the call to intel_power_domains_init_hw and the calls to intel_display_set_init_power. That code is supposed to ensure that while suspend/resume is going on _all_ power wells are on (and hence dc6/5 are forbidden). It's a bit a big hammer, but we already have the code to do exactly what you're trying to do here. It could be though that for skl it's in the wrong order (but then the commit message must state very clearly what's the exact depency, the gpu is assembled from a pile of various different blocks which are all mostly independent).

- Your patch directly calls skl_enable/disable_dc6 instead of going through the power well abstraction. Breaking these abstraction needs a really good reason, adequate assurance that the refcounts are ok and nothing breaks (using WARN_ON) and an explanation why the refcount interface for display power management doesn't work.

- You also do changes to the power well code itself, which looks like it might break dc6 for runtime pm (by only going into dc5 at most). That needs to be a separate patch or have a much better explanation.

We probably need to set up a meeting to resolve this sooner than the few months it took to figure out the resume issue ...

Thanks, Daniel

Quoted 128 lines
Quoted 127 lines
Quoted 127 lines

v2: Based on review comment from Daniel,
- created a seperate patch for csr uninitialization set call.

---
 drivers/gpu/drm/i915/i915_drv.c         | 13 +++++++++++++
 drivers/gpu/drm/i915/intel_drv.h        |  2 ++
 drivers/gpu/drm/i915/intel_runtime_pm.c | 19 +++++++------------
 3 files changed, 22 insertions(+), 12 deletions(-)
drivers/gpu/drm/i915/i915_drv.c +13 -0
index 478101c..fa66162 100644
@@ -1013,10 +1013,20 @@ static int i915_pm_resume(struct device *dev)
 static int skl_suspend_complete(struct drm_i915_private *dev_priv)
 {
+	enum csr_state state;
 	/* Enabling DC6 is not a hard requirement to enter runtime D3 */
 	skl_uninit_cdclk(dev_priv);
+	/* TODO: wait for a completion event or
+	 * similar here instead of busy
+	 * waiting using wait_for function.
+	 */
+	wait_for((state = intel_csr_load_status_get(dev_priv)) !=
+			FW_UNINITIALIZED, 1000);
+	if (state == FW_LOADED)
+		skl_enable_dc6(dev_priv);
+
 	return 0;
 }
@@ -1063,6 +1073,9 @@ static int skl_resume_prepare(struct drm_i915_private *dev_priv)
 {
 	struct drm_device *dev = dev_priv->dev;
+	if (intel_csr_load_status_get(dev_priv) == FW_LOADED)
+		skl_disable_dc6(dev_priv);
+
 	skl_init_cdclk(dev_priv);
 	intel_csr_load_program(dev);
drivers/gpu/drm/i915/intel_drv.h +2 -0
index 81b7d77..9cb7d4e 100644
@@ -1118,6 +1118,8 @@ void bxt_enable_dc9(struct drm_i915_private *dev_priv);
 void bxt_disable_dc9(struct drm_i915_private *dev_priv);
 void skl_init_cdclk(struct drm_i915_private *dev_priv);
 void skl_uninit_cdclk(struct drm_i915_private *dev_priv);
+void skl_enable_dc6(struct drm_i915_private *dev_priv);
+void skl_disable_dc6(struct drm_i915_private *dev_priv);
 void intel_dp_get_m_n(struct intel_crtc *crtc,
 		      struct intel_crtc_state *pipe_config);
 void intel_dp_set_m_n(struct intel_crtc *crtc, enum link_m_n_set m_n);
drivers/gpu/drm/i915/intel_runtime_pm.c +7 -12
index 821644d..23a3aa3 100644
@@ -548,7 +548,7 @@ static void assert_can_disable_dc6(struct drm_i915_private *dev_priv)
 		"DC6 already programmed to be disabled.\n");
 }
-static void skl_enable_dc6(struct drm_i915_private *dev_priv)
+void skl_enable_dc6(struct drm_i915_private *dev_priv)
 {
 	uint32_t val;
@@ -565,7 +565,7 @@ static void skl_enable_dc6(struct drm_i915_private *dev_priv)
 	POSTING_READ(DC_STATE_EN);
 }
-static void skl_disable_dc6(struct drm_i915_private *dev_priv)
+void skl_disable_dc6(struct drm_i915_private *dev_priv)
 {
 	uint32_t val;
@@ -626,10 +626,10 @@ static void skl_set_power_well(struct drm_i915_private *dev_priv,
 				!I915_READ(HSW_PWR_WELL_BIOS),
 				"Invalid for power well status to be enabled, unless done by the BIOS, \
 				when request is to disable!\n");
-			if ((GEN9_ENABLE_DC5(dev) || SKL_ENABLE_DC6(dev)) &&
-				power_well->data == SKL_DISP_PW_2) {
+			if (power_well->data == SKL_DISP_PW_2) {
+				if (GEN9_ENABLE_DC5(dev))
+					gen9_disable_dc5(dev_priv);
 				if (SKL_ENABLE_DC6(dev)) {
-					skl_disable_dc6(dev_priv);
 					/*
 					 * DDI buffer programming unnecessary during driver-load/resume
 					 * as it's already done during modeset initialization then.
@@ -637,8 +637,6 @@ static void skl_set_power_well(struct drm_i915_private *dev_priv,
 					 */
 					if (!dev_priv->power_domains.initializing)
 						intel_prepare_ddi(dev);
-				} else {
-					gen9_disable_dc5(dev_priv);
 				}
 			}
 			I915_WRITE(HSW_PWR_WELL_DRIVER, tmp | req_mask);
@@ -658,7 +656,7 @@ static void skl_set_power_well(struct drm_i915_private *dev_priv,
 			POSTING_READ(HSW_PWR_WELL_DRIVER);
 			DRM_DEBUG_KMS("Disabling %s\n", power_well->name);
-			if ((GEN9_ENABLE_DC5(dev) || SKL_ENABLE_DC6(dev)) &&
+			if (GEN9_ENABLE_DC5(dev) &&
 				power_well->data == SKL_DISP_PW_2) {
 				enum csr_state state;
 				/* TODO: wait for a completion event or
@@ -671,10 +669,7 @@ static void skl_set_power_well(struct drm_i915_private *dev_priv,
 					DRM_ERROR("CSR firmware not ready (%d)\n",
 							state);
 				else
-					if (SKL_ENABLE_DC6(dev))
-						skl_enable_dc6(dev_priv);
-					else
-						gen9_enable_dc5(dev_priv);
+					gen9_enable_dc5(dev_priv);
 			}
 		}
 	}
Signature · 6 lines
2.0.2

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#37 lore raw

Re: [DMC_BUGFIX_SKL_V2 3/5] drm/i915/skl: Making DC6 entry is the last call in suspend flow.

Quoted 25 lines · Daniel Vetter
Quoted 24 lines · Animesh Manna
Quoted 20 lines · Daniel Vetter
Quoted 3 lines · Animesh Manna

Mmio register access after dc6/dc5 entry is not allowed when DC6 power states are enabled according to bspec (bspec-id 0527), so enabling dc6 as the last call in suspend flow.

We unconditionaly grab a power well reference for everything in our suspend/resume functions, which means dc6/5 should be prevented for long enough.

Also since dc6/5 are part of the power well framework you can't just enable/disable them directly, instead you need to get/put the corresponding power wells.

Finally this patch doesn't just do that, but also frobs around a lot in set power well code itself, and I have no idea what it does there and why. It does smell a bit like you're just breaking dc6 for runtime pm though, which wouldn't be good.

Anyway I decided to just merge this since this patch series has been floating around since forever, but then the patch didn't apply cleanly and so dropped it.

Quoted 6 lines
Quoted 6 lines

I mentioned in my commit message that we have a h/w workaround. "Mmio register access after dc6/dc5 entry is not allowed when DC6 power states are enabled"

This patch is mandatory to solve the pc10 entry issue for skylake. Planning to send again after rebase on top of tree.

Hi Animesh and Daniel

I'm trying to shed some light on this. It seems rather confusing atm. Animesh, please correct me if I'm wrong below.

Quoted 11 lines

The problem isn't that the patch didn't apply cleanly, but that it seems to have fundamental issues:
- We already grab power well references around suspend/resume code, see the call to intel_power_domains_init_hw and the calls to intel_display_set_init_power. That code is supposed to ensure that while suspend/resume is going on _all_ power wells are on (and hence dc6/5 are forbidden). It's a bit a big hammer, but we already have the code to do exactly what you're trying to do here. It could be though that for skl it's in the wrong order (but then the commit message must state very clearly what's the exact depency, the gpu is assembled from a pile of various different blocks which are all mostly independent).

DC6 can only be enabled when _all_ power wells are disabled. Hence this patch moves DC6 enabled/disabled to the runtime suspend/resume callbacks. Previously we only disabled DC6 when PW2 was enabled.

The DC state table says:
- Everything off: DC6 enabled
- PW0 enabled: DC5 enabled but DC6 is disabled
- PW1 enabled: DC5 and DC6 disabled
- PW2 enabled: DC5 and DC6 disabled

Quoted 5 lines

- Your patch directly calls skl_enable/disable_dc6 instead of going through the power well abstraction. Breaking these abstraction needs a really good reason, adequate assurance that the refcounts are ok and nothing breaks (using WARN_ON) and an explanation why the refcount interface for display power management doesn't work.

This is indeed a hack but since all we need is the guarantee that a power well is in fact enabled we can see the operation as: enable pw + disable dc state. This is currently hacked into skl_set_power_well(). It's not nice but it takes away the complexity of modeling DC states as power wells.

The alternative would be to turn DC5 and DC6 into power wells and add proper dependencies. What I've discovered is that it's quite a tight fit.

Quoted 3 lines

- You also do changes to the power well code itself, which looks like it might break dc6 for runtime pm (by only going into dc5 at most). That needs to be a separate patch or have a much better explanation.

If I understood correctly, that is the entire point of this patch. Don't allow DC6 unless all power wells are disabled. My question though is why we don't disable DC5 for PW1 as well?

Quoted 4 lines

We probably need to set up a meeting to resolve this sooner than the few months it took to figure out the resume issue ...

Thanks, Daniel

I'd like to join if possible.

Thanks
Patrik

Quoted 123 lines
Quoted 123 lines
Quoted 123 lines
Quoted 123 lines

v2: Based on review comment from Daniel,
- created a seperate patch for csr uninitialization set call.

---
 drivers/gpu/drm/i915/i915_drv.c         | 13 +++++++++++++
 drivers/gpu/drm/i915/intel_drv.h        |  2 ++
 drivers/gpu/drm/i915/intel_runtime_pm.c | 19 +++++++------------
 3 files changed, 22 insertions(+), 12 deletions(-)
drivers/gpu/drm/i915/i915_drv.c +13 -0
index 478101c..fa66162 100644
@@ -1013,10 +1013,20 @@ static int i915_pm_resume(struct device *dev)
 static int skl_suspend_complete(struct drm_i915_private *dev_priv)
 {
+	enum csr_state state;
 	/* Enabling DC6 is not a hard requirement to enter runtime D3 */
 	skl_uninit_cdclk(dev_priv);
+	/* TODO: wait for a completion event or
+	 * similar here instead of busy
+	 * waiting using wait_for function.
+	 */
+	wait_for((state = intel_csr_load_status_get(dev_priv)) !=
+			FW_UNINITIALIZED, 1000);
+	if (state == FW_LOADED)
+		skl_enable_dc6(dev_priv);
+
 	return 0;
 }
@@ -1063,6 +1073,9 @@ static int skl_resume_prepare(struct drm_i915_private *dev_priv)
 {
 	struct drm_device *dev = dev_priv->dev;
+	if (intel_csr_load_status_get(dev_priv) == FW_LOADED)
+		skl_disable_dc6(dev_priv);
+
 	skl_init_cdclk(dev_priv);
 	intel_csr_load_program(dev);
drivers/gpu/drm/i915/intel_drv.h +2 -0
index 81b7d77..9cb7d4e 100644
@@ -1118,6 +1118,8 @@ void bxt_enable_dc9(struct drm_i915_private *dev_priv);
 void bxt_disable_dc9(struct drm_i915_private *dev_priv);
 void skl_init_cdclk(struct drm_i915_private *dev_priv);
 void skl_uninit_cdclk(struct drm_i915_private *dev_priv);
+void skl_enable_dc6(struct drm_i915_private *dev_priv);
+void skl_disable_dc6(struct drm_i915_private *dev_priv);
 void intel_dp_get_m_n(struct intel_crtc *crtc,
 		      struct intel_crtc_state *pipe_config);
 void intel_dp_set_m_n(struct intel_crtc *crtc, enum link_m_n_set m_n);
drivers/gpu/drm/i915/intel_runtime_pm.c +7 -12
index 821644d..23a3aa3 100644
@@ -548,7 +548,7 @@ static void assert_can_disable_dc6(struct drm_i915_private *dev_priv)
 		"DC6 already programmed to be disabled.\n");
 }
-static void skl_enable_dc6(struct drm_i915_private *dev_priv)
+void skl_enable_dc6(struct drm_i915_private *dev_priv)
 {
 	uint32_t val;
@@ -565,7 +565,7 @@ static void skl_enable_dc6(struct drm_i915_private *dev_priv)
 	POSTING_READ(DC_STATE_EN);
 }
-static void skl_disable_dc6(struct drm_i915_private *dev_priv)
+void skl_disable_dc6(struct drm_i915_private *dev_priv)
 {
 	uint32_t val;
@@ -626,10 +626,10 @@ static void skl_set_power_well(struct drm_i915_private *dev_priv,
 				!I915_READ(HSW_PWR_WELL_BIOS),
 				"Invalid for power well status to be enabled, unless done by the BIOS, \
 				when request is to disable!\n");
-			if ((GEN9_ENABLE_DC5(dev) || SKL_ENABLE_DC6(dev)) &&
-				power_well->data == SKL_DISP_PW_2) {
+			if (power_well->data == SKL_DISP_PW_2) {
+				if (GEN9_ENABLE_DC5(dev))
+					gen9_disable_dc5(dev_priv);
 				if (SKL_ENABLE_DC6(dev)) {
-					skl_disable_dc6(dev_priv);
 					/*
 					 * DDI buffer programming unnecessary during driver-load/resume
 					 * as it's already done during modeset initialization then.
@@ -637,8 +637,6 @@ static void skl_set_power_well(struct drm_i915_private *dev_priv,
 					 */
 					if (!dev_priv->power_domains.initializing)
 						intel_prepare_ddi(dev);
-				} else {
-					gen9_disable_dc5(dev_priv);
 				}
 			}
 			I915_WRITE(HSW_PWR_WELL_DRIVER, tmp | req_mask);
@@ -658,7 +656,7 @@ static void skl_set_power_well(struct drm_i915_private *dev_priv,
 			POSTING_READ(HSW_PWR_WELL_DRIVER);
 			DRM_DEBUG_KMS("Disabling %s\n", power_well->name);
-			if ((GEN9_ENABLE_DC5(dev) || SKL_ENABLE_DC6(dev)) &&
+			if (GEN9_ENABLE_DC5(dev) &&
 				power_well->data == SKL_DISP_PW_2) {
 				enum csr_state state;
 				/* TODO: wait for a completion event or
@@ -671,10 +669,7 @@ static void skl_set_power_well(struct drm_i915_private *dev_priv,
 					DRM_ERROR("CSR firmware not ready (%d)\n",
 							state);
 				else
-					if (SKL_ENABLE_DC6(dev))
-						skl_enable_dc6(dev_priv);
-					else
-						gen9_enable_dc5(dev_priv);
+					gen9_enable_dc5(dev_priv);
 			}
 		}
 	}
Signature · 1 line
2.0.2
Quoted 14 lines
Quoted 5 lines
Quoted 4 lines
Quoted 4 lines

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

#38 lore raw

Re: [DMC_BUGFIX_SKL_V2 3/5] drm/i915/skl: Making DC6 entry is the last call in suspend flow.

Quoted 54 lines · Patrik Jakobsson
Quoted 25 lines · Daniel Vetter
Quoted 24 lines · Animesh Manna
Quoted 20 lines · Daniel Vetter
Quoted 3 lines · Animesh Manna

Mmio register access after dc6/dc5 entry is not allowed when DC6 power states are enabled according to bspec (bspec-id 0527), so enabling dc6 as the last call in suspend flow.

We unconditionaly grab a power well reference for everything in our suspend/resume functions, which means dc6/5 should be prevented for long enough.

Also since dc6/5 are part of the power well framework you can't just enable/disable them directly, instead you need to get/put the corresponding power wells.

Finally this patch doesn't just do that, but also frobs around a lot in set power well code itself, and I have no idea what it does there and why. It does smell a bit like you're just breaking dc6 for runtime pm though, which wouldn't be good.

Anyway I decided to just merge this since this patch series has been floating around since forever, but then the patch didn't apply cleanly and so dropped it.

Quoted 6 lines
Quoted 6 lines

I mentioned in my commit message that we have a h/w workaround. "Mmio register access after dc6/dc5 entry is not allowed when DC6 power states are enabled"

This patch is mandatory to solve the pc10 entry issue for skylake. Planning to send again after rebase on top of tree.

Hi Animesh and Daniel

I'm trying to shed some light on this. It seems rather confusing atm. Animesh, please correct me if I'm wrong below.

Quoted 11 lines

The problem isn't that the patch didn't apply cleanly, but that it seems to have fundamental issues:
- We already grab power well references around suspend/resume code, see the call to intel_power_domains_init_hw and the calls to intel_display_set_init_power. That code is supposed to ensure that while suspend/resume is going on _all_ power wells are on (and hence dc6/5 are forbidden). It's a bit a big hammer, but we already have the code to do exactly what you're trying to do here. It could be though that for skl it's in the wrong order (but then the commit message must state very clearly what's the exact depency, the gpu is assembled from a pile of various different blocks which are all mostly independent).

DC6 can only be enabled when _all_ power wells are disabled. Hence this patch moves DC6 enabled/disabled to the runtime suspend/resume callbacks. Previously we only disabled DC6 when PW2 was enabled.

Quoted 5 lines

The DC state table says:
- Everything off: DC6 enabled
- PW0 enabled: DC5 enabled but DC6 is disabled
- PW1 enabled: DC5 and DC6 disabled
- PW2 enabled: DC5 and DC6 disabled

Ok, so what the patch does (and that is at least self-consistent) is to move the DC6 enable/disable from PW2 to the top-level domain. That doesn't quite match what you're describing here since we still have PW1 managed by the driver, but that's short-circuited now.

Quoted 6 lines
Quoted 5 lines

- Your patch directly calls skl_enable/disable_dc6 instead of going through the power well abstraction. Breaking these abstraction needs a really good reason, adequate assurance that the refcounts are ok and nothing breaks (using WARN_ON) and an explanation why the refcount interface for display power management doesn't work.

Quoted 7 lines

This is indeed a hack but since all we need is the guarantee that a power well is in fact enabled we can see the operation as: enable pw + disable dc state. This is currently hacked into skl_set_power_well(). It's not nice but it takes away the complexity of modeling DC states as power wells.

The alternative would be to turn DC5 and DC6 into power wells and add proper dependencies. What I've discovered is that it's quite a tight fit.

Nah, if the goal is to move just DC6 the patch seems fine. I was just really confused that the commit message talked about both dc6 _and_ dc5 being impossible to do, and then only moving half of the story around. And I was also a bit confused with runtime pm vs. system suspend (again).

So if we can confirm that this is really only about DC6 and not also about dc5 like the commit message claims, then I can clean up the commit message and apply the patch.

Quoted 7 lines
Quoted 3 lines

- You also do changes to the power well code itself, which looks like it might break dc6 for runtime pm (by only going into dc5 at most). That needs to be a separate patch or have a much better explanation.

If I understood correctly, that is the entire point of this patch. Don't allow DC6 unless all power wells are disabled. My question though is why we don't disable DC5 for PW1 as well?

Afaiui dmc manages pw1 for us and we shouldn't ever touch it from the driver side. I'm not entirely sure about that though since the current fix is a hack. If we really shouldn't touch pw1 on skl then we should remove that power domain from our skl code entirely. -Daniel

Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#39 lore raw

Re: [DMC_BUGFIX_SKL_V2 3/5] drm/i915/skl: Making DC6 entry is the last call in suspend flow.

Quoted 62 lines · Daniel Vetter
Quoted 54 lines · Patrik Jakobsson
Quoted 25 lines · Daniel Vetter
Quoted 24 lines · Animesh Manna
Quoted 20 lines · Daniel Vetter
Quoted 3 lines · Animesh Manna

Mmio register access after dc6/dc5 entry is not allowed when DC6 power states are enabled according to bspec (bspec-id 0527), so enabling dc6 as the last call in suspend flow.

We unconditionaly grab a power well reference for everything in our suspend/resume functions, which means dc6/5 should be prevented for long enough.

Also since dc6/5 are part of the power well framework you can't just enable/disable them directly, instead you need to get/put the corresponding power wells.

Finally this patch doesn't just do that, but also frobs around a lot in set power well code itself, and I have no idea what it does there and why. It does smell a bit like you're just breaking dc6 for runtime pm though, which wouldn't be good.

Anyway I decided to just merge this since this patch series has been floating around since forever, but then the patch didn't apply cleanly and so dropped it.

Quoted 6 lines
Quoted 6 lines

I mentioned in my commit message that we have a h/w workaround. "Mmio register access after dc6/dc5 entry is not allowed when DC6 power states are enabled"

This patch is mandatory to solve the pc10 entry issue for skylake. Planning to send again after rebase on top of tree.

Hi Animesh and Daniel

I'm trying to shed some light on this. It seems rather confusing atm. Animesh, please correct me if I'm wrong below.

Quoted 11 lines

The problem isn't that the patch didn't apply cleanly, but that it seems to have fundamental issues:
- We already grab power well references around suspend/resume code, see the call to intel_power_domains_init_hw and the calls to intel_display_set_init_power. That code is supposed to ensure that while suspend/resume is going on _all_ power wells are on (and hence dc6/5 are forbidden). It's a bit a big hammer, but we already have the code to do exactly what you're trying to do here. It could be though that for skl it's in the wrong order (but then the commit message must state very clearly what's the exact depency, the gpu is assembled from a pile of various different blocks which are all mostly independent).

DC6 can only be enabled when _all_ power wells are disabled. Hence this patch moves DC6 enabled/disabled to the runtime suspend/resume callbacks. Previously we only disabled DC6 when PW2 was enabled.

Quoted 5 lines

The DC state table says:
- Everything off: DC6 enabled
- PW0 enabled: DC5 enabled but DC6 is disabled
- PW1 enabled: DC5 and DC6 disabled
- PW2 enabled: DC5 and DC6 disabled

Quoted 4 lines

Ok, so what the patch does (and that is at least self-consistent) is to move the DC6 enable/disable from PW2 to the top-level domain. That doesn't quite match what you're describing here since we still have PW1 managed by the driver, but that's short-circuited now.

Hmm. Why are we going back and forth on this all the time? Was there some problem with the plan [1] that Imre and I hatched?

[1] http://lists.freedesktop.org/archives/intel-gfx/2015-September/076612.html

Quoted 46 lines
Quoted 7 lines
Quoted 6 lines

- Your patch directly calls skl_enable/disable_dc6 instead of going through the power well abstraction. Breaking these abstraction needs a really good reason, adequate assurance that the refcounts are ok and nothing breaks (using WARN_ON) and an explanation why the refcount interface for display power management doesn't work.

Quoted 7 lines

This is indeed a hack but since all we need is the guarantee that a power well is in fact enabled we can see the operation as: enable pw + disable dc state. This is currently hacked into skl_set_power_well(). It's not nice but it takes away the complexity of modeling DC states as power wells.

The alternative would be to turn DC5 and DC6 into power wells and add proper dependencies. What I've discovered is that it's quite a tight fit.

Nah, if the goal is to move just DC6 the patch seems fine. I was just really confused that the commit message talked about both dc6 _and_ dc5 being impossible to do, and then only moving half of the story around. And I was also a bit confused with runtime pm vs. system suspend (again).

So if we can confirm that this is really only about DC6 and not also about dc5 like the commit message claims, then I can clean up the commit message and apply the patch.

Quoted 7 lines
Quoted 3 lines

- You also do changes to the power well code itself, which looks like it might break dc6 for runtime pm (by only going into dc5 at most). That needs to be a separate patch or have a much better explanation.

If I understood correctly, that is the entire point of this patch. Don't allow DC6 unless all power wells are disabled. My question though is why we don't disable DC5 for PW1 as well?

Afaiui dmc manages pw1 for us and we shouldn't ever touch it from the driver side. I'm not entirely sure about that though since the current fix is a hack. If we really shouldn't touch pw1 on skl then we should remove that power domain from our skl code entirely. -Daniel

Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
Signature · 6 lines
Ville Syrjälä
Intel OTC
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#40 lore raw

Re: [DMC_BUGFIX_SKL_V2 3/5] drm/i915/skl: Making DC6 entry is the last call in suspend flow.

Quoted 4 lines · Ville Syrjälä

Hmm. Why are we going back and forth on this all the time? Was there some problem with the plan [1] that Imre and I hatched?

[1] http://lists.freedesktop.org/archives/intel-gfx/2015-September/076612.html

Well this is just the interim bugfix with a confusing commit message. Compared to your overall plan we now have dc5&pw2 fused together, pw1 is still there but a no-op and dc6 is now (with this patch) fused with with pw0 and overall device d3 (or whatever we use to kill the entire thing and still appease the firmware).

So at least matches the overall plan.
-Daniel

Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#41 lore raw

PATCH 4/5 drm/i915/skl: Do not disable cdclk PLL if csr firmware is present

In reply to Animesh Manna

While display engine entering into low power state no need to disable cdclk pll as CSR firmware of dmc will take care. If pll is already enabled firmware execution sequence will be blocked. This is one of the criteria for dmc to work properly.

v1: Initial version.

v2: Based on review comment from Daniel added code commnent.

Cc Daniel Vetter <[email protected]>
Cc Damien Lespiau <[email protected]>
Cc Imre Deak <[email protected]>
Cc Sunil Kamath <[email protected]>

Signed-off-bt: Vathsala Nagaraju <[email protected]>

---
 drivers/gpu/drm/i915/intel_display.c | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)
drivers/gpu/drm/i915/intel_display.c +10 -4
index f604ce1..b6bef20 100644
@@ -5687,10 +5687,16 @@ void skl_uninit_cdclk(struct drm_i915_private *dev_priv)
 	if (I915_READ(DBUF_CTL) & DBUF_POWER_STATE)
 		DRM_ERROR("DBuf power disable timeout\n");
 
-	/* disable DPLL0 */
-	I915_WRITE(LCPLL1_CTL, I915_READ(LCPLL1_CTL) & ~LCPLL_PLL_ENABLE);
-	if (wait_for(!(I915_READ(LCPLL1_CTL) & LCPLL_PLL_LOCK), 1))
-		DRM_ERROR("Couldn't disable DPLL0\n");
+	/*
+	 * DMC assumes ownership of LCPLL and will get confused if we touch it.
+	 */
+	if (dev_priv->csr.dmc_payload) {
+		/* disable DPLL0 */
+		I915_WRITE(LCPLL1_CTL, I915_READ(LCPLL1_CTL) &
+					~LCPLL_PLL_ENABLE);
+		if (wait_for(!(I915_READ(LCPLL1_CTL) & LCPLL_PLL_LOCK), 1))
+			DRM_ERROR("Couldn't disable DPLL0\n");
+	}
 
 	intel_display_power_put(dev_priv, POWER_DOMAIN_PLLS);
 }
Signature · 6 lines
2.0.2

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#42 lore raw

Re: [DMC_BUGFIX_SKL_V2 4/5] drm/i915/skl: Do not disable cdclk PLL if csr firmware is present

Quoted 25 lines · Animesh Manna

While display engine entering into low power state no need to disable cdclk pll as CSR firmware of dmc will take care. If pll is already enabled firmware execution sequence will be blocked. This is one of the criteria for dmc to work properly.

v1: Initial version.

v2: Based on review comment from Daniel added code commnent.

Cc Daniel Vetter <[email protected]>
Cc Damien Lespiau <[email protected]>
Cc Imre Deak <[email protected]>
Cc Sunil Kamath <[email protected]>

Signed-off-bt: Vathsala Nagaraju <[email protected]>

---
 drivers/gpu/drm/i915/intel_display.c | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)
drivers/gpu/drm/i915/intel_display.c +0 -0
index f604ce1..b6bef20 100644

drivers/gpu/drm/i915/intel_display.c void skl_uninit_cdclk(struct drm_i915_private *dev_priv) · line 5691

Quoted 9 lines
drivers/gpu/drm/i915/intel_display.c +2 -4
@@ -5687,10 +5687,16 @@ void skl_uninit_cdclk(struct drm_i915_private *dev_priv)
 	if (I915_READ(DBUF_CTL) & DBUF_POWER_STATE)
 		DRM_ERROR("DBuf power disable timeout\n");
 
-	/* disable DPLL0 */
-	I915_WRITE(LCPLL1_CTL, I915_READ(LCPLL1_CTL) & ~LCPLL_PLL_ENABLE);
-	if (wait_for(!(I915_READ(LCPLL1_CTL) & LCPLL_PLL_LOCK), 1))
-		DRM_ERROR("Couldn't disable DPLL0\n");
+	/*
+	 * DMC assumes ownership of LCPLL and will get confused if we touch it.

This should get a FIXME - once we have dmc loading fixed up we require the firmware and there's no point in this check any more. Flexibilty just because is something we simply don't have the developer and validation resources for.
-Daniel

Quoted 18 lines
drivers/gpu/drm/i915/intel_display.c +8 -0
+	 */
+	if (dev_priv->csr.dmc_payload) {
+		/* disable DPLL0 */
+		I915_WRITE(LCPLL1_CTL, I915_READ(LCPLL1_CTL) &
+					~LCPLL_PLL_ENABLE);
+		if (wait_for(!(I915_READ(LCPLL1_CTL) & LCPLL_PLL_LOCK), 1))
+			DRM_ERROR("Couldn't disable DPLL0\n");
+	}
 
 	intel_display_power_put(dev_priv, POWER_DOMAIN_PLLS);
 }
Signature · 6 lines
2.0.2

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#43 lore raw

Re: [DMC_BUGFIX_SKL_V2 4/5] drm/i915/skl: Do not disable cdclk PLL if csr firmware is present

Quoted 29 lines · Daniel Vetter
Quoted 28 lines · Animesh Manna

While display engine entering into low power state no need to disable cdclk pll as CSR firmware of dmc will take care. If pll is already enabled firmware execution sequence will be blocked. This is one of the criteria for dmc to work properly.

v1: Initial version.

v2: Based on review comment from Daniel added code commnent.

Cc Daniel Vetter <[email protected]>
Cc Damien Lespiau <[email protected]>
Cc Imre Deak <[email protected]>
Cc Sunil Kamath <[email protected]>

Signed-off-bt: Vathsala Nagaraju <[email protected]>

---
  drivers/gpu/drm/i915/intel_display.c | 14 ++++++++++----
  1 file changed, 10 insertions(+), 4 deletions(-)
drivers/gpu/drm/i915/intel_display.c +0 -0
index f604ce1..b6bef20 100644
@@ -5687,10 +5687,16 @@ void skl_uninit_cdclk(struct drm_i915_private *dev_priv)
  	if (I915_READ(DBUF_CTL) & DBUF_POWER_STATE)
  		DRM_ERROR("DBuf power disable timeout\n");
 
Quoted 11 lines

drivers/gpu/drm/i915/intel_display.c void skl_uninit_cdclk(struct drm_i915_private *dev_priv) · line 5691

Quoted 6 lines
drivers/gpu/drm/i915/intel_display.c +2 -4
-	/* disable DPLL0 */
-	I915_WRITE(LCPLL1_CTL, I915_READ(LCPLL1_CTL) & ~LCPLL_PLL_ENABLE);
-	if (wait_for(!(I915_READ(LCPLL1_CTL) & LCPLL_PLL_LOCK), 1))
-		DRM_ERROR("Couldn't disable DPLL0\n");
+	/*
+	 * DMC assumes ownership of LCPLL and will get confused if we touch it.

This should get a FIXME - once we have dmc loading fixed up we require the firmware and there's no point in this check any more. Flexibilty just because is something we simply don't have the developer and validation resources for.
-Daniel

I am not sure if dmc firmware is mandatory for all user.
But the code is written based on design principle published in below series
http://www.spinics.net/lists/intel-gfx/msg72399.html

Job of Dmc firmware is to disable pw1 and cdclk pll during suspend.
If dmc firmware is not present driver can disable cdclk pll to save some more power.

Am I missing anything?

-Animesh

Quoted 15 lines
Quoted 15 lines
drivers/gpu/drm/i915/intel_display.c +8 -0
+	 */
+	if (dev_priv->csr.dmc_payload) {
+		/* disable DPLL0 */
+		I915_WRITE(LCPLL1_CTL, I915_READ(LCPLL1_CTL) &
+					~LCPLL_PLL_ENABLE);
+		if (wait_for(!(I915_READ(LCPLL1_CTL) & LCPLL_PLL_LOCK), 1))
+			DRM_ERROR("Couldn't disable DPLL0\n");
+	}
 
  	intel_display_power_put(dev_priv, POWER_DOMAIN_PLLS);
  }
Signature · 1 line
2.0.2
Quoted 4 lines
Quoted 4 lines

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

#44 lore raw

Re: [DMC_BUGFIX_SKL_V2 4/5] drm/i915/skl: Do not disable cdclk PLL if csr firmware is present

In reply to Daniel Vetter

Unless I'm misreading that would imply that we are moving away from our previous position that DMC FW is optional, correct? Would this not render power-sequencing broken if a distro chose not to include DMC FW?

Gavin Hindman
Senior Program Manager
SSG/OTC – Open Source Technology Center

-----Original Message-----
From: Intel-gfx [mailto:[email protected]] On Behalf Of Daniel Vetter
Sent: Wednesday, August 26, 2015 6:12 AM
To: Manna, Animesh
Cc: Bhardwaj, Rajneesh; [email protected]; Vetter, Daniel
Subject: Re: [Intel-gfx] [DMC_BUGFIX_SKL_V2 4/5] drm/i915/skl: Do not disable cdclk PLL if csr firmware is present

Quoted 26 lines · Animesh Manna

While display engine entering into low power state no need to disable cdclk pll as CSR firmware of dmc will take care. If pll is already enabled firmware execution sequence will be blocked. This is one of the criteria for dmc to work properly.

v1: Initial version.

v2: Based on review comment from Daniel added code commnent.

Cc Daniel Vetter <[email protected]>
Cc Damien Lespiau <[email protected]>
Cc Imre Deak <[email protected]>
Cc Sunil Kamath <[email protected]>

Signed-off-bt: Vathsala Nagaraju <[email protected]>

---
 drivers/gpu/drm/i915/intel_display.c | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)
drivers/gpu/drm/i915/intel_display.c +0 -0
index f604ce1..b6bef20 100644

drivers/gpu/drm/i915/intel_display.c void skl_uninit_cdclk(struct drm_i915_private *dev_priv) · line 5691

Quoted 9 lines
drivers/gpu/drm/i915/intel_display.c +2 -4
@@ -5687,10 +5687,16 @@ void skl_uninit_cdclk(struct drm_i915_private *dev_priv)
 	if (I915_READ(DBUF_CTL) & DBUF_POWER_STATE)
 		DRM_ERROR("DBuf power disable timeout\n");
 
-	/* disable DPLL0 */
-	I915_WRITE(LCPLL1_CTL, I915_READ(LCPLL1_CTL) & ~LCPLL_PLL_ENABLE);
-	if (wait_for(!(I915_READ(LCPLL1_CTL) & LCPLL_PLL_LOCK), 1))
-		DRM_ERROR("Couldn't disable DPLL0\n");
+	/*
+	 * DMC assumes ownership of LCPLL and will get confused if we touch it.

This should get a FIXME - once we have dmc loading fixed up we require the firmware and there's no point in this check any more. Flexibilty just because is something we simply don't have the developer and validation resources for. -Daniel

Quoted 13 lines
drivers/gpu/drm/i915/intel_display.c +8 -1
+	 */
+	if (dev_priv->csr.dmc_payload) {
+		/* disable DPLL0 */
+		I915_WRITE(LCPLL1_CTL, I915_READ(LCPLL1_CTL) &
+					~LCPLL_PLL_ENABLE);
+		if (wait_for(!(I915_READ(LCPLL1_CTL) & LCPLL_PLL_LOCK), 1))
+			DRM_ERROR("Couldn't disable DPLL0\n");
+	}
 
 	intel_display_power_put(dev_priv, POWER_DOMAIN_PLLS);  }
--

2.0.2

Quoted 4 lines

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

#45 lore raw

Re: [DMC_BUGFIX_SKL_V2 4/5] drm/i915/skl: Do not disable cdclk PLL if csr firmware is present

Quoted 3 lines · Gavin

Unless I'm misreading that would imply that we are moving away from our
previous position that DMC FW is optional, correct? Would this not
render power-sequencing broken if a distro chose not to include DMC FW?

For upstream we never had the stance (and I wouldn't accept the patches really without a really good reason) that we'll support runtime pm without DMC firmware.

We already have a really bad time just trying to keep things working in 1 configuration (there's a patch from me to enable runtime pm by default blocked on unfixed regressions which are open since months and no one's working on them), we absolutely don't have the resources to support any crazy configurations that's theoretically possible. If a distro won't include DMC then they won't get proper runtime pm and that's it.

Yes the original DMC patches (and the code still in tree) had code to handle that case, but follow-up patches rip that complexity out.

If we are in position where we can actually ship all the power features we develop then we could consider supporting more combinations of features, but right now I think that's in the very distant future.

Thanks, Daniel

Quoted 79 lines

Gavin Hindman
Senior Program Manager
SSG/OTC – Open Source Technology Center

-----Original Message-----
From: Intel-gfx [mailto:[email protected]] On Behalf Of Daniel Vetter
Sent: Wednesday, August 26, 2015 6:12 AM
To: Manna, Animesh
Cc: Bhardwaj, Rajneesh; [email protected]; Vetter, Daniel
Subject: Re: [Intel-gfx] [DMC_BUGFIX_SKL_V2 4/5] drm/i915/skl: Do not disable cdclk PLL if csr firmware is present

Quoted 26 lines · Animesh Manna

While display engine entering into low power state no need to disable cdclk pll as CSR firmware of dmc will take care. If pll is already enabled firmware execution sequence will be blocked. This is one of the criteria for dmc to work properly.

v1: Initial version.

v2: Based on review comment from Daniel added code commnent.

Cc Daniel Vetter <[email protected]>
Cc Damien Lespiau <[email protected]>
Cc Imre Deak <[email protected]>
Cc Sunil Kamath <[email protected]>

Signed-off-bt: Vathsala Nagaraju <[email protected]>

---
 drivers/gpu/drm/i915/intel_display.c | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)
drivers/gpu/drm/i915/intel_display.c +0 -0
index f604ce1..b6bef20 100644

drivers/gpu/drm/i915/intel_display.c void skl_uninit_cdclk(struct drm_i915_private *dev_priv) · line 5691

Quoted 9 lines
drivers/gpu/drm/i915/intel_display.c +2 -4
@@ -5687,10 +5687,16 @@ void skl_uninit_cdclk(struct drm_i915_private *dev_priv)
 	if (I915_READ(DBUF_CTL) & DBUF_POWER_STATE)
 		DRM_ERROR("DBuf power disable timeout\n");
 
-	/* disable DPLL0 */
-	I915_WRITE(LCPLL1_CTL, I915_READ(LCPLL1_CTL) & ~LCPLL_PLL_ENABLE);
-	if (wait_for(!(I915_READ(LCPLL1_CTL) & LCPLL_PLL_LOCK), 1))
-		DRM_ERROR("Couldn't disable DPLL0\n");
+	/*
+	 * DMC assumes ownership of LCPLL and will get confused if we touch it.

This should get a FIXME - once we have dmc loading fixed up we require the firmware and there's no point in this check any more. Flexibilty just because is something we simply don't have the developer and validation resources for. -Daniel

Quoted 13 lines
drivers/gpu/drm/i915/intel_display.c +8 -1
+	 */
+	if (dev_priv->csr.dmc_payload) {
+		/* disable DPLL0 */
+		I915_WRITE(LCPLL1_CTL, I915_READ(LCPLL1_CTL) &
+					~LCPLL_PLL_ENABLE);
+		if (wait_for(!(I915_READ(LCPLL1_CTL) & LCPLL_PLL_LOCK), 1))
+			DRM_ERROR("Couldn't disable DPLL0\n");
+	}
 
 	intel_display_power_put(dev_priv, POWER_DOMAIN_PLLS);  }
--

2.0.2

Quoted 4 lines

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
Reviewed-by on v1 4/5
#46 lore raw

Re: [DMC_BUGFIX_SKL_V2 4/5] drm/i915/skl: Do not disable cdclk PLL if csr firmware is present

In reply to Animesh Manna
Quoted 35 lines · Animesh Manna

While display engine entering into low power state no need to disable cdclk pll as CSR firmware of dmc will take care. If pll is already enabled firmware execution sequence will be blocked. This is one of the criteria for dmc to work properly.

v1: Initial version.

v2: Based on review comment from Daniel added code commnent.

Cc Daniel Vetter <[email protected]>
Cc Damien Lespiau <[email protected]>
Cc Imre Deak <[email protected]>
Cc Sunil Kamath <[email protected]>

Signed-off-bt: Vathsala Nagaraju <[email protected]>

---
  drivers/gpu/drm/i915/intel_display.c | 14 ++++++++++----
  1 file changed, 10 insertions(+), 4 deletions(-)
drivers/gpu/drm/i915/intel_display.c +3 -4
index f604ce1..b6bef20 100644
@@ -5687,10 +5687,16 @@ void skl_uninit_cdclk(struct drm_i915_private *dev_priv)
  	if (I915_READ(DBUF_CTL) & DBUF_POWER_STATE)
  		DRM_ERROR("DBuf power disable timeout\n");
 
-	/* disable DPLL0 */
-	I915_WRITE(LCPLL1_CTL, I915_READ(LCPLL1_CTL) & ~LCPLL_PLL_ENABLE);
-	if (wait_for(!(I915_READ(LCPLL1_CTL) & LCPLL_PLL_LOCK), 1))
-		DRM_ERROR("Couldn't disable DPLL0\n");
+	/*
+	 * DMC assumes ownership of LCPLL and will get confused if we touch it.
+	 */

drivers/gpu/drm/i915/intel_display.c void skl_uninit_cdclk(struct drm_i915_private *dev_priv) · line 5702

Quoted 10 lines
drivers/gpu/drm/i915/intel_display.c +7 -0
+	if (dev_priv->csr.dmc_payload) {
+		/* disable DPLL0 */
+		I915_WRITE(LCPLL1_CTL, I915_READ(LCPLL1_CTL) &
+					~LCPLL_PLL_ENABLE);
+		if (wait_for(!(I915_READ(LCPLL1_CTL) & LCPLL_PLL_LOCK), 1))
+			DRM_ERROR("Couldn't disable DPLL0\n");
+	}
 
  	intel_display_power_put(dev_priv, POWER_DOMAIN_PLLS);
  }

Valid fix and patch is ready for merge now.

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

#47 lore raw

PATCH 5/5 drm/i915/skl: Block disable call for pw1 if dmc firmware is present.

In reply to Animesh Manna

Another interesting criteria to work dmc as expected is pw1 to be enabled by driver and dmc will shut it off in its execution sequence. If already disabled by driver dmc will get confuse and behave differently than expected found during pc10 entry issue for skl.

So berfore we disable power-well 1, added check if dmc firmware is present and driver will not disable power well 1, but for any reason if firmware is not present of failed to load we can shut off the power well 1 which will save some power.

As skl is currently fully dependent on dmc to go in lowest possible power state (dc6) but the same is not applicable for bxt. Display engine can enter into dc9 without dmc, hence unblocking disable call.

v1: Initial version.

v2: Rebased as per current patch series.

Cc Daniel Vetter <[email protected]>
Cc Damien Lespiau <[email protected]>
Cc Imre Deak <[email protected]>
Cc Sunil Kamath <[email protected]>
---
 drivers/gpu/drm/i915/intel_runtime_pm.c | 12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)
drivers/gpu/drm/i915/intel_runtime_pm.c +9 -3
index 23a3aa3..340f386 100644
@@ -652,9 +652,15 @@ static void skl_set_power_well(struct drm_i915_private *dev_priv,
 		}
 	} else {
 		if (enable_requested) {
-			I915_WRITE(HSW_PWR_WELL_DRIVER,	tmp & ~req_mask);
-			POSTING_READ(HSW_PWR_WELL_DRIVER);
-			DRM_DEBUG_KMS("Disabling %s\n", power_well->name);
+			if (IS_SKYLAKE(dev) &&
+				(power_well->data == SKL_DISP_PW_1) &&
+				(intel_csr_load_status_get(dev_priv) == FW_LOADED))
+				DRM_DEBUG_KMS("Not Disabling PW1, dmc will handle\n");
+			else {
+				I915_WRITE(HSW_PWR_WELL_DRIVER,	tmp & ~req_mask);
+				POSTING_READ(HSW_PWR_WELL_DRIVER);
+				DRM_DEBUG_KMS("Disabling %s\n", power_well->name);
+			}
 
 			if (GEN9_ENABLE_DC5(dev) &&
 				power_well->data == SKL_DISP_PW_2) {
Signature · 6 lines
2.0.2

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
Reviewed-by on v1 5/5
#48 lore raw

Re: [DMC_BUGFIX_SKL_V2 5/5] drm/i915/skl: Block disable call for pw1 if dmc firmware is present.

Quoted 42 lines · Animesh Manna

Another interesting criteria to work dmc as expected is pw1 to be enabled by driver and dmc will shut it off in its execution sequence. If already disabled by driver dmc will get confuse and behave differently than expected found during pc10 entry issue for skl.

So berfore we disable power-well 1, added check if dmc firmware is present and driver will not disable power well 1, but for any reason if firmware is not present of failed to load we can shut off the power well 1 which will save some power.

As skl is currently fully dependent on dmc to go in lowest possible power state (dc6) but the same is not applicable for bxt. Display engine can enter into dc9 without dmc, hence unblocking disable call.

v1: Initial version.

v2: Rebased as per current patch series.

Cc Daniel Vetter <[email protected]>
Cc Damien Lespiau <[email protected]>
Cc Imre Deak <[email protected]>
Cc Sunil Kamath <[email protected]>
---
  drivers/gpu/drm/i915/intel_runtime_pm.c | 12 +++++++++---
  1 file changed, 9 insertions(+), 3 deletions(-)
drivers/gpu/drm/i915/intel_runtime_pm.c +2 -3
index 23a3aa3..340f386 100644
@@ -652,9 +652,15 @@ static void skl_set_power_well(struct drm_i915_private *dev_priv,
  		}
  	} else {
  		if (enable_requested) {
-			I915_WRITE(HSW_PWR_WELL_DRIVER,	tmp & ~req_mask);
-			POSTING_READ(HSW_PWR_WELL_DRIVER);
-			DRM_DEBUG_KMS("Disabling %s\n", power_well->name);
+			if (IS_SKYLAKE(dev) &&
+				(power_well->data == SKL_DISP_PW_1) &&

drivers/gpu/drm/i915/intel_runtime_pm.c static void skl_set_power_well(struct drm_i915_private *dev_priv, · line 666

Quoted 10 lines
drivers/gpu/drm/i915/intel_runtime_pm.c +7 -0
+				(intel_csr_load_status_get(dev_priv) == FW_LOADED))
+				DRM_DEBUG_KMS("Not Disabling PW1, dmc will handle\n");
+			else {
+				I915_WRITE(HSW_PWR_WELL_DRIVER,	tmp & ~req_mask);
+				POSTING_READ(HSW_PWR_WELL_DRIVER);
+				DRM_DEBUG_KMS("Disabling %s\n", power_well->name);
+			}
 
  			if (GEN9_ENABLE_DC5(dev) &&
  				power_well->data == SKL_DISP_PW_2) {

Valid fix and patch is ready for merge now.

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

#49 lore raw

Re: [DMC_BUGFIX_SKL_V2 5/5] drm/i915/skl: Block disable call for pw1 if dmc firmware is present.

Quoted 55 lines · Sunil Kamath
Quoted 41 lines · Animesh Manna

Another interesting criteria to work dmc as expected is pw1 to be enabled by driver and dmc will shut it off in its execution sequence. If already disabled by driver dmc will get confuse and behave differently than expected found during pc10 entry issue for skl.

So berfore we disable power-well 1, added check if dmc firmware is present and driver will not disable power well 1, but for any reason if firmware is not present of failed to load we can shut off the power well 1 which will save some power.

As skl is currently fully dependent on dmc to go in lowest possible power state (dc6) but the same is not applicable for bxt. Display engine can enter into dc9 without dmc, hence unblocking disable call.

v1: Initial version.

v2: Rebased as per current patch series.

Cc Daniel Vetter <[email protected]>
Cc Damien Lespiau <[email protected]>
Cc Imre Deak <[email protected]>
Cc Sunil Kamath <[email protected]>
---
 drivers/gpu/drm/i915/intel_runtime_pm.c | 12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)
drivers/gpu/drm/i915/intel_runtime_pm.c +1 -3
index 23a3aa3..340f386 100644
@@ -652,9 +652,15 @@ static void skl_set_power_well(struct drm_i915_private *dev_priv,
 		}
 	} else {
 		if (enable_requested) {
-			I915_WRITE(HSW_PWR_WELL_DRIVER,	tmp & ~req_mask);
-			POSTING_READ(HSW_PWR_WELL_DRIVER);
-			DRM_DEBUG_KMS("Disabling %s\n", power_well->name);
+			if (IS_SKYLAKE(dev) &&

drivers/gpu/drm/i915/intel_runtime_pm.c static void skl_set_power_well(struct drm_i915_private *dev_priv, · line 665

Quoted 10 lines
drivers/gpu/drm/i915/intel_runtime_pm.c +8 -0
+				(power_well->data == SKL_DISP_PW_1) &&
+				(intel_csr_load_status_get(dev_priv) == FW_LOADED))
+				DRM_DEBUG_KMS("Not Disabling PW1, dmc will handle\n");
+			else {
+				I915_WRITE(HSW_PWR_WELL_DRIVER,	tmp & ~req_mask);
+				POSTING_READ(HSW_PWR_WELL_DRIVER);
+				DRM_DEBUG_KMS("Disabling %s\n", power_well->name);
+			}
 			if (GEN9_ENABLE_DC5(dev) &&
 				power_well->data == SKL_DISP_PW_2) {

Valid fix and patch is ready for merge now.

Quoted 1 line

Ok I pulled them all in except patch 3.
-Daniel

Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#50 lore raw
In reply to Animesh Manna

After discussions with Art, Ville and others internally on how DC5/6 should normally work and existing open issues related to the HW automatic DC5/6 functionality, I have the following understanding.

In the future we will require the firmware for any display side runtime power management (and possibly some power savings during system power states like S0ix and S3), so in the following we can ignore the case when the firmware is not available.

Normally we need to run the BSpec display init sequence only when loading the driver and resuming from S4 if the BIOS hasn't done so already. PW1 and CDCLK is enabled as part of this sequence and will be toggled by the DMC firmware on demand, so we must not toggle these manually during runtime, since that would race with the FW. The same goes goes for other resources enabled/reset in the init sequence: PCH reset handshake, DBUF, Misc IO. The manual controls are DC5, DC6 and PW2 which the driver needs to toggle based on the active outputs. We also need to disable DC5/6 while doing modeset and according to our tests DPAUX transfers.

When HW enters DC6 and the GT side is idle too (in RC6) the system as a whole can enter a deeper power state (PC9/10).

Due to a HW issue atm we cannot rely on the HW transitioning automatically to DC6 and we need to use a manual way instead. We do this by keeping DC6 disabled all the time and running a special deinitialization sequence when both the display and GT side is idle, and the reverse when something becomes active. (that is in the driver's runtime suspend/resume hooks).

Currently the driver is not following the normal HW assisted flow, since it toggles DBUF, CDCLK and PW1 manually during runtime. It also lacks the manual sequence to enable PC9/10.

This patchset solves some of the above issues, by removing the manual CDCLK and PW1 control and disabling DC6. This will solve some of the stability issues, but leave PC9/10 disabled. As a follow-up we need to
- move the rest of the display init/uninit sequence out of the power_well/suspend/resume path and call it only during driver loading/S4 resume as necessary
- add separate DC5/DC6 power wells
- add the manual sequence to enable PC9/10
- add a module parameter to enable either HW assisted DC6, or the manual way, or disable DC6
- disable DC5/6 during modeset and other places like DPAUX as needed

I'll comment on the individual patches based on the above. I think with those addressed we could merge this patchset and do the rest as a follow-up.

--Imre

Quoted 23 lines · Animesh Manna

The following patches helps to solve PC10 entry issue for SKL. Detailed description about the changes done to solve the issue is mentioned in commit message of each patch.

v1: http://lists.freedesktop.org/archives/intel-gfx/2015-August/072870.html

v2: Based on review comments from Daniel, changes made in the current version.

DMC redesign patch series has dependencies with current patch series. Need to rework on few patches, planning to send after initial review feedback of the current patch series.
http://lists.freedesktop.org/archives/intel-gfx/2015-August/072921.html

Animesh Manna (5):
drm/i915/skl: Added a check for the hardware status of csr fw before
loading.
drm/i915/skl Remove the call for csr uninitialization from suspend
path
drm/i915/skl: Making DC6 entry is the last call in suspend flow. drm/i915/skl: Do not disable cdclk PLL if csr firmware is present drm/i915/skl: Block disable call for pw1 if dmc firmware is present.

Quoted 7 lines
 drivers/gpu/drm/i915/i915_drv.c         | 19 +++++++++++++------
 drivers/gpu/drm/i915/intel_csr.c        |  9 +++++++++
 drivers/gpu/drm/i915/intel_display.c    | 14 ++++++++++----
 drivers/gpu/drm/i915/intel_drv.h        |  2 ++
 drivers/gpu/drm/i915/intel_runtime_pm.c | 31 ++++++++++++++++---------------
 5 files changed, 50 insertions(+), 25 deletions(-)

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx