drm/i915/pcode: Wait 10 seconds for pcode to settle
Subsystems (2)
PATCH 1/1 drm/i915/pcode: Wait 10 seconds for pcode to settle
From: Aravind Iddamsetty <[email protected]>
During module load not all the punit transaction have completed and we might end up timing out, as shown by the following warning:
i915 0000:4d:00.0: drm_WARN_ON_ONCE(timeout_base_ms > 3)
Wait 10 seconds for the punit to settle and complete any outstanding transactions upon module load.
--- drivers/gpu/drm/i915/intel_pcode.c | 35 ++++++++++++++++++++++++++---- 1 file changed, 31 insertions(+), 4 deletions(-)
drivers/gpu/drm/i915/intel_pcode.c +31 -4
@@ -204,15 +204,42 @@ int skl_pcode_request(struct intel_uncore *uncore, u32 mbox, u32 request,#undef COND } +static int pcode_init_wait (struct intel_uncore *uncore,int timeout_ms) +{ +if (__intel_wait_for_register_fw (uncore, +GEN6_PCODE_MAILBOX , +GEN6_PCODE_READY ,0 , +500 , timeout_ms, +NULL )) +return -EPROBE_DEFER ; + +return skl_pcode_request (uncore, +DG1_PCODE_STATUS , +DG1_UNCORE_GET_INIT_STATUS , +DG1_UNCORE_INIT_STATUS_COMPLETE , +DG1_UNCORE_INIT_STATUS_COMPLETE , timeout_ms); +} +int intel_pcode_init (struct intel_uncore *uncore) { +int err; +if (!IS_DGFX (uncore->i915))return 0 ; -return skl_pcode_request (uncore,DG1_PCODE_STATUS , -DG1_UNCORE_GET_INIT_STATUS , -DG1_UNCORE_INIT_STATUS_COMPLETE , -DG1_UNCORE_INIT_STATUS_COMPLETE ,180000 ); +/* +* Wait 10 seconds so that the punit to settle and complete +* any outstanding transactions upon module load +*/ + err =pcode_init_wait (uncore,10000 ); + +if (err) { +drm_notice (&uncore->i915->drm, +"Waiting for HW initialisation...\n" ); + err =pcode_init_wait (uncore,180000 ); + } + +return err; }int snb_pcode_read_p (struct intel_uncore *uncore,u32 mbcmd,u32 p1,u32 p2,u32 *val)
Signature · 1 line
2.34.1
Patchwork
CI[Intel-gfx] ✗ Fi.CI.CHECKPATCH: warning for drm/i915/pcode: Wait 10 seconds for pcode to settle
== Series Details ==
== Series Details ==
Series: drm/i915/pcode: Wait 10 seconds for pcode to settle
URL : https://patchwork.freedesktop.org/series/112666/
State : warning
== Summary ==
Error: dim checkpatch failedf784820151fb drm/i915/pcode: Wait 10 seconds for pcode to settle
-:18: WARNING:BAD_SIGN_OFF: Co-developed-by: must be immediately followed by Signed-off-by:
#18:
total: 0 errors, 1 warnings, 0 checks, 46 lines checked
Patchwork
CI[Intel-gfx] ✗ Fi.CI.BAT: failure for drm/i915/pcode: Wait 10 seconds for pcode to settle
In reply to Andi Shyti== Series Details ==
== Series Details ==
Series: drm/i915/pcode: Wait 10 seconds for pcode to settle
URL : https://patchwork.freedesktop.org/series/112666/
State : failure
== Summary ==
CI Bug Log - changes from CI_DRM_12572 -> Patchwork_112666v1
====================================================
Summary
-------
**FAILURE**
Serious unknown changes coming with Patchwork_112666v1 absolutely need to be verified manually.
If you think the reported changes have nothing to do with the changes introduced in Patchwork_112666v1, please notify your bug team to allow them to document this new failure mode, which will reduce false positives in CI.
External URL: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_112666v1/index.html
Participating hosts (40 -> 35)
------------------------------
Additional (1): fi-bsw-kefka
Missing (6): bat-dg1-7 bat-dg1-6 fi-snb-2520m bat-adln-1 bat-rpls-1 bat-rpls-2
Possible new issues
-------------------
Here are the unknown changes that may have been introduced in Patchwork_112666v1:
### IGT changes ###
#### Possible regressions ####
* igt@i915_selftest@live@slpc:
- fi-kbl-soraka: [PASS][1] -> [INCOMPLETE][2]
[1]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12572/fi-kbl-soraka/igt@i915_selftest@[email protected]
[2]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_112666v1/fi-kbl-soraka/igt@i915_selftest@[email protected]
Known issues
------------
Here are the changes found in Patchwork_112666v1 that come from known issues:
### IGT changes ###
#### Issues hit ####
* igt@i915_selftest@live@gt_heartbeat:
- fi-cfl-guc: [PASS][3] -> [DMESG-FAIL][4] ([i915#5334])
[3]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12572/fi-cfl-guc/igt@i915_selftest@live@gt_heartbeat.html
[4]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_112666v1/fi-cfl-guc/igt@i915_selftest@live@gt_heartbeat.html
* igt@prime_vgem@basic-fence-flip:
- fi-bsw-kefka: NOTRUN -> [SKIP][5] ([fdo#109271]) +26 similar issues
[5]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_112666v1/fi-bsw-kefka/igt@[email protected]
#### Possible fixes ####
* igt@dmabuf@all-tests@dma_fence:
- fi-elk-e7500: [FAIL][6] -> [PASS][7]
[6]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12572/fi-elk-e7500/igt@dmabuf@all-tests@dma_fence.html
[7]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_112666v1/fi-elk-e7500/igt@dmabuf@all-tests@dma_fence.html
[fdo#109271]: https://bugs.freedesktop.org/show_bug.cgi?id=109271
[i915#5334]: https://gitlab.freedesktop.org/drm/intel/issues/5334
Build changes
-------------
* Linux: CI_DRM_12572 -> Patchwork_112666v1
CI-20190529: 20190529
CI_DRM_12572: 1d81095ce0a981d0c0b5979bf4b75fe6cf871973 @ git://anongit.freedesktop.org/gfx-ci/linux
IGT_7115: c162d70b00c6f4cf6a0ba1ca7a7e2ad8f7190646 @ https://gitlab.freedesktop.org/drm/igt-gpu-tools.git
Patchwork_112666v1: 1d81095ce0a981d0c0b5979bf4b75fe6cf871973 @ git://anongit.freedesktop.org/gfx-ci/linux
### Linux commits
a8fb3da8f21c drm/i915/pcode: Wait 10 seconds for pcode to settle
== Logs ==
For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_112666v1/index.html
Jani Nikula
maintainerQuoted 5 lines · Andi Shyti
From: Aravind Iddamsetty <[email protected]>
During module load not all the punit transaction have completed and we might end up timing out, as shown by the following warning:
Root cause?
Quoted 8 lines
i915 0000:4d:00.0: drm_WARN_ON_ONCE(timeout_base_ms > 3)Wait 10 seconds for the punit to settle and complete any outstanding transactions upon module load.
No blank lines between the tag lines please.
Quoted 14 lines
Signed-off-by Aravind Iddamsetty <[email protected]>Co-developed-by Chris Wilson <[email protected]>Cc Rodrigo Vivi <[email protected]>Signed-off-by Andi Shyti <[email protected]>--- drivers/gpu/drm/i915/intel_pcode.c | 35 ++++++++++++++++++++++++++---- 1 file changed, 31 insertions(+), 4 deletions(-)drivers/gpu/drm/i915/intel_pcode.c +0
-0@@ -204,15 +204,42 @@ int skl_pcode_request(struct intel_uncore *uncore, u32 mbox, u32 request,#undef COND
drivers/gpu/drm/i915/intel_pcode.c int skl_pcode_request(struct intel_uncore *uncore, u32 mbox, u32 request, · line 214
Quoted 10 lines
drivers/gpu/drm/i915/intel_pcode.c +8
-0} +static int pcode_init_wait (struct intel_uncore *uncore,int timeout_ms) +{ +if (__intel_wait_for_register_fw (uncore, +GEN6_PCODE_MAILBOX , +GEN6_PCODE_READY ,0 , +500 , timeout_ms, +NULL )) +return -EPROBE_DEFER ;
This is already done within skl_pcode_request -> skl_pcode_try_request -> __snb_pcode_rw path, with waits in skl_pcode_request.
Is there anyone who still understands what's being waited for, where, for how long, and why in the different code paths? I know I don't, and this isn't helping.
There's also no explanation on the -EPROBE_DEFER return in the commit message or comments or anywhere.
Again, root cause?
BR,
Jani.
Quoted 35 lines
drivers/gpu/drm/i915/intel_pcode.c +23
-4+ +return skl_pcode_request (uncore, +DG1_PCODE_STATUS , +DG1_UNCORE_GET_INIT_STATUS , +DG1_UNCORE_INIT_STATUS_COMPLETE , +DG1_UNCORE_INIT_STATUS_COMPLETE , timeout_ms); +} +int intel_pcode_init (struct intel_uncore *uncore) { +int err; +if (!IS_DGFX (uncore->i915))return 0 ; -return skl_pcode_request (uncore,DG1_PCODE_STATUS , -DG1_UNCORE_GET_INIT_STATUS , -DG1_UNCORE_INIT_STATUS_COMPLETE , -DG1_UNCORE_INIT_STATUS_COMPLETE ,180000 ); +/* +* Wait 10 seconds so that the punit to settle and complete +* any outstanding transactions upon module load +*/ + err =pcode_init_wait (uncore,10000 ); + +if (err) { +drm_notice (&uncore->i915->drm, +"Waiting for HW initialisation...\n" ); + err =pcode_init_wait (uncore,180000 ); + } + +return err; }int snb_pcode_read_p (struct intel_uncore *uncore,u32 mbcmd,u32 p1,u32 p2,u32 *val)
Signature · 1 line
Jani Nikula, Intel Open Source Graphics Center
Quoted 12 lines · Jani Nikula
Quoted 5 lines · Andi Shyti
From: Aravind Iddamsetty <[email protected]>
During module load not all the punit transaction have completed and we might end up timing out, as shown by the following warning:
Root cause?
Quoted 3 lines
i915 0000:4d:00.0: drm_WARN_ON_ONCE(timeout_base_ms > 3)
Quoted 7 lines
Quoted 5 lines
Wait 10 seconds for the punit to settle and complete any outstanding transactions upon module load.
No blank lines between the tag lines please.
I don't consider "Closes:" to be a tag even if someone is using it as such. AFAIK is not mentioned it in any of the kernel docs (e.g. Documentation/process/maintainer-tip.rst).
Not to confuse it with a normal tag, I wanted to put it in parenthesis or rephrase it.
Quoted 16 lines
Quoted 16 lines
Signed-off-by Aravind Iddamsetty <[email protected]>Co-developed-by Chris Wilson <[email protected]>Cc Rodrigo Vivi <[email protected]>Signed-off-by Andi Shyti <[email protected]>--- drivers/gpu/drm/i915/intel_pcode.c | 35 ++++++++++++++++++++++++++---- 1 file changed, 31 insertions(+), 4 deletions(-)drivers/gpu/drm/i915/intel_pcode.c +0
-0@@ -204,15 +204,42 @@ int skl_pcode_request(struct intel_uncore *uncore, u32 mbox, u32 request,#undef COND }
Quoted 11 lines
drivers/gpu/drm/i915/intel_pcode.c int skl_pcode_request(struct intel_uncore *uncore, u32 mbox, u32 request, · line 214
Quoted 8 lines
drivers/gpu/drm/i915/intel_pcode.c +8
-0+static int pcode_init_wait (struct intel_uncore *uncore,int timeout_ms) +{ +if (__intel_wait_for_register_fw (uncore, +GEN6_PCODE_MAILBOX , +GEN6_PCODE_READY ,0 , +500 , timeout_ms, +NULL )) +return -EPROBE_DEFER ;This is already done within
skl_pcode_request->skl_pcode_try_request->__snb_pcode_rwpath, with waits inskl_pcode_request.
the idea is to check for PCODE_READY even before checking if data are sent/received by pcode. And this is only during boot time. While skl_pcode_request is called in other contexts as well.
In other words here I want to start the communication when I already know that the punit is ready. Otherwise I would hit an -EAGAIN and fail.
Quoted 3 lines
Is there anyone who still understands what's being waited for, where, for how long, and why in the different code paths? I know I don't, and this isn't helping.
I think it depends on hardware. There are some documents roaming around with some boot time and reset time calculation.
Quoted 2 lines
There's also no explanation on the -EPROBE_DEFER return in the commit message or comments or anywhere.
we haven't really failed, right? We just need some time for the punit to be ready and try to probe again (remember we are here in module probe).
Thanks for the review,
Andi
Quoted 44 lines
Again, root cause?
BR,
Jani.Quoted 35 lines
drivers/gpu/drm/i915/intel_pcode.c +23
-4+ +return skl_pcode_request (uncore, +DG1_PCODE_STATUS , +DG1_UNCORE_GET_INIT_STATUS , +DG1_UNCORE_INIT_STATUS_COMPLETE , +DG1_UNCORE_INIT_STATUS_COMPLETE , timeout_ms); +} +int intel_pcode_init (struct intel_uncore *uncore) { +int err; +if (!IS_DGFX (uncore->i915))return 0 ; -return skl_pcode_request (uncore,DG1_PCODE_STATUS , -DG1_UNCORE_GET_INIT_STATUS , -DG1_UNCORE_INIT_STATUS_COMPLETE , -DG1_UNCORE_INIT_STATUS_COMPLETE ,180000 ); +/* +* Wait 10 seconds so that the punit to settle and complete +* any outstanding transactions upon module load +*/ + err =pcode_init_wait (uncore,10000 ); + +if (err) { +drm_notice (&uncore->i915->drm, +"Waiting for HW initialisation...\n" ); + err =pcode_init_wait (uncore,180000 ); + } + +return err; }int snb_pcode_read_p (struct intel_uncore *uncore,u32 mbcmd,u32 p1,u32 p2,u32 *val)Signature · 1 line
Jani Nikula, Intel Open Source Graphics Center
Jani Nikula
maintainerQuoted 16 lines · Andi Shyti
Quoted 15 lines · Jani Nikula
Quoted 5 lines · Andi Shyti
From: Aravind Iddamsetty <[email protected]>
During module load not all the punit transaction have completed and we might end up timing out, as shown by the following warning:
Root cause?
Quoted 6 lines
i915 0000:4d:00.0: drm_WARN_ON_ONCE(timeout_base_ms > 3)Wait 10 seconds for the punit to settle and complete any outstanding transactions upon module load.
Quoted 8 lines
Quoted 4 lines
Quoted 2 lines
No blank lines between the tag lines please.
I don't consider "Closes:" to be a tag even if someone is using it as such. AFAIK is not mentioned it in any of the kernel docs (e.g.
Documentation/process/maintainer-tip.rst).
Basically all of the Something: lines should be bundled together.
BR,
Jani.
Quoted 102 lines
Not to confuse it with a normal tag, I wanted to put it in parenthesis or rephrase it.
Quoted 16 lines
Quoted 16 lines
Signed-off-by Aravind Iddamsetty <[email protected]>Co-developed-by Chris Wilson <[email protected]>Cc Rodrigo Vivi <[email protected]>Signed-off-by Andi Shyti <[email protected]>--- drivers/gpu/drm/i915/intel_pcode.c | 35 ++++++++++++++++++++++++++---- 1 file changed, 31 insertions(+), 4 deletions(-)drivers/gpu/drm/i915/intel_pcode.c +0
-0@@ -204,15 +204,42 @@ int skl_pcode_request(struct intel_uncore *uncore, u32 mbox, u32 request,#undef COND }Quoted 11 lines
drivers/gpu/drm/i915/intel_pcode.c int skl_pcode_request(struct intel_uncore *uncore, u32 mbox, u32 request, · line 214
Quoted 8 lines
drivers/gpu/drm/i915/intel_pcode.c +8
-0+static int pcode_init_wait (struct intel_uncore *uncore,int timeout_ms) +{ +if (__intel_wait_for_register_fw (uncore, +GEN6_PCODE_MAILBOX , +GEN6_PCODE_READY ,0 , +500 , timeout_ms, +NULL )) +return -EPROBE_DEFER ;This is already done within
skl_pcode_request->skl_pcode_try_request->__snb_pcode_rwpath, with waits inskl_pcode_request.the idea is to check for
PCODE_READYeven before checking if data are sent/received by pcode. And this is only during boot time. Whileskl_pcode_requestis called in other contexts as well.In other words here I want to start the communication when I already know that the punit is ready. Otherwise I would hit an
-EAGAINand fail.Quoted 3 lines
Is there anyone who still understands what's being waited for, where, for how long, and why in the different code paths? I know I don't, and this isn't helping.
I think it depends on hardware. There are some documents roaming around with some boot time and reset time calculation.
Quoted 2 lines
There's also no explanation on the -EPROBE_DEFER return in the commit message or comments or anywhere.
we haven't really failed, right? We just need some time for the punit to be ready and try to probe again (remember we are here in module probe).
Thanks for the review,
AndiQuoted 44 lines
Again, root cause?
BR,
Jani.Quoted 35 lines
drivers/gpu/drm/i915/intel_pcode.c +23
-4+ +return skl_pcode_request (uncore, +DG1_PCODE_STATUS , +DG1_UNCORE_GET_INIT_STATUS , +DG1_UNCORE_INIT_STATUS_COMPLETE , +DG1_UNCORE_INIT_STATUS_COMPLETE , timeout_ms); +} +int intel_pcode_init (struct intel_uncore *uncore) { +int err; +if (!IS_DGFX (uncore->i915))return 0 ; -return skl_pcode_request (uncore,DG1_PCODE_STATUS , -DG1_UNCORE_GET_INIT_STATUS , -DG1_UNCORE_INIT_STATUS_COMPLETE , -DG1_UNCORE_INIT_STATUS_COMPLETE ,180000 ); +/* +* Wait 10 seconds so that the punit to settle and complete +* any outstanding transactions upon module load +*/ + err =pcode_init_wait (uncore,10000 ); + +if (err) { +drm_notice (&uncore->i915->drm, +"Waiting for HW initialisation...\n" ); + err =pcode_init_wait (uncore,180000 ); + } + +return err; }int snb_pcode_read_p (struct intel_uncore *uncore,u32 mbcmd,u32 p1,u32 p2,u32 *val)Signature · 1 line
Jani Nikula, Intel Open Source Graphics Center
Signature · 1 line
Jani Nikula, Intel Open Source Graphics Center
Quoted 9 lines · Andi Shyti
Quoted 8 lines · Jani Nikula
Quoted 5 lines · Andi Shyti
From: Aravind Iddamsetty <[email protected]>
During module load not all the punit transaction have completed and we might end up timing out, as shown by the following warning:
Root cause?
Hi Andi, looking at the log where this problem is reported,
https://gitlab.freedesktop.org/drm/intel/-/issues/7814
<4> [268.276908] i915 0000:4d:00.0: drm_WARN_ON_ONCE(timeout_base_ms > 3)
In the code above, the call stack is output, and the return value of intel_pcode_init() returns -11 (-EAGAIN).
<3> [268.329058] i915 0000:4d:00.0: [drm] *ERROR* gt0: intel_pcode_init failed -11
If you simplify the function call flow, you can see it as below. (The call of _wait_for(COND, timeout_base_ms * 1000, 10, 10) in skl_pcode_request() is omitted)
-------------------------------------------------------------------------intel_pcode_init()
|
+-> skl_pcode_request(uncore, DG1_PCODE_STATUS,
DG1_UNCORE_GET_INIT_STATUS,
DG1_UNCORE_INIT_STATUS_COMPLETE,
DG1_UNCORE_INIT_STATUS_COMPLETE, 180000);
|
+-> skl_pcode_try_request()
|
+-> *status = __snb_pcode_rw(uncore, mbox, &request, NULL,
500, 0, true);-------------------------------------------------------------------------
static int __snb_pcode_rw(struct intel_uncore *uncore, u32 mbox,
u32 *val, u32 *val1, int fast_timeout_us, int slow_timeout_ms, bool is_read)
{
...
if (intel_uncore_read_fw(uncore, GEN6_PCODE_MAILBOX) & GEN6_PCODE_READY) return -EAGAIN; intel_uncore_write_fw(uncore, GEN6_PCODE_DATA, *val); intel_uncore_write_fw(uncore, GEN6_PCODE_DATA1, val1 ? *val1 : 0); intel_uncore_write_fw(uncore, GEN6_PCODE_MAILBOX, GEN6_PCODE_READY | mbox); if (__intel_wait_for_register_fw(uncore, GEN6_PCODE_MAILBOX, GEN6_PCODE_READY, 0, fast_timeout_us, slow_timeout_ms, &mbox)) return -ETIMEDOUT;
...
}
-------------------------------------------------------------------------
The case where skl_pcode_request() returns -EAGAIN can be considered as the following scenario.
1. 1. When checking the GEN6_PCODE_MAILBOX register status before writing the value to the GEN6_PCODE_DATA register in __snb_pcode_rw(), it is always BUSY
2. _wait_for(COND, timeout_base_ms * 1000, 10, 10) of skl_pcode_request() returns -EAGAIN if the GEN6_PCODE_MAILBOX register indicates BUSY even after waiting 500us after writing a value to the GEN6_PCODE_DATA register in __snb_pcode_rw()
(Even if skl_pcode_request() gives a timeout of 180 seconds, the time used each time __snb_pcode_rw() is called is up to 500us. The rest of the time is used for sleep.)
In the situation where the problem is reported, it is not possible to confirm exactly which scenario code causes the problem with the current log information, and it seems that additional analysis is needed to confirm it. If the hardware takes more than 500us to succeed after PCODE_MAILBOX write under certain circumstances, it is thought that the root problem causing the problem should be fixed.
br,
G.G.
Quoted 117 lines
Quoted 11 lines
Quoted 9 lines
i915 0000:4d:00.0: drm_WARN_ON_ONCE(timeout_base_ms > 3)Wait 10 seconds for the punit to settle and complete any outstanding transactions upon module load.
No blank lines between the tag lines please.
I don't consider "Closes:" to be a tag even if someone is using it as such. AFAIK is not mentioned it in any of the kernel docs (e.g.
Documentation/process/maintainer-tip.rst).Not to confuse it with a normal tag, I wanted to put it in parenthesis or rephrase it.
Quoted 16 lines
Quoted 16 lines
Signed-off-by Aravind Iddamsetty <[email protected]>Co-developed-by Chris Wilson <[email protected]>Cc Rodrigo Vivi <[email protected]>Signed-off-by Andi Shyti <[email protected]>--- drivers/gpu/drm/i915/intel_pcode.c | 35 ++++++++++++++++++++++++++---- 1 file changed, 31 insertions(+), 4 deletions(-)drivers/gpu/drm/i915/intel_pcode.c +0
-0@@ -204,15 +204,42 @@ int skl_pcode_request(struct intel_uncore *uncore, u32 mbox, u32 request,#undef COND }Quoted 11 lines
drivers/gpu/drm/i915/intel_pcode.c int skl_pcode_request(struct intel_uncore *uncore, u32 mbox, u32 request, · line 214
Quoted 8 lines
drivers/gpu/drm/i915/intel_pcode.c +8
-0+static int pcode_init_wait (struct intel_uncore *uncore,int timeout_ms) +{ +if (__intel_wait_for_register_fw (uncore, +GEN6_PCODE_MAILBOX , +GEN6_PCODE_READY ,0 , +500 , timeout_ms, +NULL )) +return -EPROBE_DEFER ;This is already done within
skl_pcode_request->skl_pcode_try_request->__snb_pcode_rwpath, with waits inskl_pcode_request.the idea is to check for
PCODE_READYeven before checking if data are sent/received by pcode. And this is only during boot time. Whileskl_pcode_requestis called in other contexts as well.In other words here I want to start the communication when I already know that the punit is ready. Otherwise I would hit an
-EAGAINand fail.Quoted 3 lines
Is there anyone who still understands what's being waited for, where, for how long, and why in the different code paths? I know I don't, and this isn't helping.
I think it depends on hardware. There are some documents roaming around with some boot time and reset time calculation.
Quoted 2 lines
There's also no explanation on the -EPROBE_DEFER return in the commit message or comments or anywhere.
we haven't really failed, right? We just need some time for the punit to be ready and try to probe again (remember we are here in module probe).
Thanks for the review,
AndiQuoted 44 lines
Again, root cause?
BR,
Jani.Quoted 35 lines
drivers/gpu/drm/i915/intel_pcode.c +23
-4+ +return skl_pcode_request (uncore, +DG1_PCODE_STATUS , +DG1_UNCORE_GET_INIT_STATUS , +DG1_UNCORE_INIT_STATUS_COMPLETE , +DG1_UNCORE_INIT_STATUS_COMPLETE , timeout_ms); +} +int intel_pcode_init (struct intel_uncore *uncore) { +int err; +if (!IS_DGFX (uncore->i915))return 0 ; -return skl_pcode_request (uncore,DG1_PCODE_STATUS , -DG1_UNCORE_GET_INIT_STATUS , -DG1_UNCORE_INIT_STATUS_COMPLETE , -DG1_UNCORE_INIT_STATUS_COMPLETE ,180000 ); +/* +* Wait 10 seconds so that the punit to settle and complete +* any outstanding transactions upon module load +*/ + err =pcode_init_wait (uncore,10000 ); + +if (err) { +drm_notice (&uncore->i915->drm, +"Waiting for HW initialisation...\n" ); + err =pcode_init_wait (uncore,180000 ); + } + +return err; }int snb_pcode_read_p (struct intel_uncore *uncore,u32 mbcmd,u32 p1,u32 p2,u32 *val)Signature · 1 line
Jani Nikula, Intel Open Source Graphics Center
Hi Gwan-gyeong,
thanks for the review and the thorough explanation.
Quoted 73 lines · Gwan-gyeong Mun
Quoted 9 lines · Andi Shyti
Quoted 8 lines · Jani Nikula
Quoted 5 lines · Andi Shyti
From: Aravind Iddamsetty <[email protected]>
During module load not all the punit transaction have completed and we might end up timing out, as shown by the following warning:
Root cause?
Hi Andi, looking at the log where this problem is reported,
https://gitlab.freedesktop.org/drm/intel/-/issues/7814
<4> [268.276908] i915 0000:4d:00.0:
drm_WARN_ON_ONCE(timeout_base_ms > 3)In the code above, the call stack is output, and the return value of
intel_pcode_init()returns -11 (-EAGAIN).<3> [268.329058] i915 0000:4d:00.0: [drm] *ERROR* gt0:
intel_pcode_initfailed -11If you simplify the function call flow, you can see it as below. (The call of
_wait_for(COND, timeout_base_ms * 1000, 10, 10)inskl_pcode_request()is omitted)-------------------------------------------------------------------------
intel_pcode_init()
|
+-> skl_pcode_request(uncore,DG1_PCODE_STATUS,DG1_UNCORE_GET_INIT_STATUS, DG1_UNCORE_INIT_STATUS_COMPLETE, DG1_UNCORE_INIT_STATUS_COMPLETE, 180000); | +-> skl_pcode_try_request() | +-> *status = __snb_pcode_rw(uncore, mbox, &request, NULL, 500, 0, true);-------------------------------------------------------------------------
static int __snb_pcode_rw(structintel_uncore*uncore, u32 mbox,u32 *val, u32 *val1, int fast_timeout_us, int slow_timeout_ms, bool is_read){
...if (intel_uncore_read_fw(uncore, GEN6_PCODE_MAILBOX) & GEN6_PCODE_READY) return -EAGAIN; intel_uncore_write_fw(uncore, GEN6_PCODE_DATA, *val); intel_uncore_write_fw(uncore, GEN6_PCODE_DATA1, val1 ? *val1 : 0); intel_uncore_write_fw(uncore, GEN6_PCODE_MAILBOX, GEN6_PCODE_READY | mbox); if (__intel_wait_for_register_fw(uncore, GEN6_PCODE_MAILBOX, GEN6_PCODE_READY, 0, fast_timeout_us, slow_timeout_ms, &mbox)) return -ETIMEDOUT;...
}
-------------------------------------------------------------------------
Quoted 5 lines
The case where
skl_pcode_request()returns-EAGAINcan be considered as the following scenario.
1. 1. When checking theGEN6_PCODE_MAILBOXregister status before writing the value to theGEN6_PCODE_DATAregister in__snb_pcode_rw(), it is always BUSY
correct! We fail with EAGAIN because we are not able to communicate with the punit because the punit is busy.
Talking about this case we are in boot time and the gpu is performing its boot routine, the punit as well. While the punit is working we try communicate with it, but unfortunately, being busy, we fail with -EAGAIN exactly where you pointed.
Adding an extra wait_for_register_fw, i.e. waiting until the PCODE_READY register tells that the punit is ready, makes sure that the read/write will succeed.
Thus Chris has added a 10 seconds wait before the very first read and write. If the punit is not ready we don't fail with -EAGAIN and give up the driver loading as it happens now. But we give it another chance trying to probe it again later.
Quoted 4 lines
2.
_wait_for(COND, timeout_base_ms * 1000, 10, 10)ofskl_pcode_request()returns-EAGAINif theGEN6_PCODE_MAILBOXregister indicates BUSY even after waiting 500us after writing a value to theGEN6_PCODE_DATAregister in__snb_pcode_rw()
Isn't it the same as '1'?
Quoted 3 lines
(Even if
skl_pcode_request()gives a timeout of 180 seconds, the time used each time__snb_pcode_rw()is called is up to 500us. The rest of the time is used for sleep.)
There is one big, massive, huge difference... the timeout in skl_pcode_request() after the read/write, not before. So that at the first communication we fail, return -EAGAIN and give up probing without starting any timer.
Be aware of the fact that the timeout is not for the current communication, but for the next one. De facto we start the timeout after communicating, this makes sure that the next communication will work.
But no one makes sure that the very first communication works during probe. Thus the extra wait.
Quoted 6 lines
In the situation where the problem is reported, it is not possible to confirm exactly which scenario code causes the problem with the current log information, and it seems that additional analysis is needed to confirm it. If the hardware takes more than 500us to succeed after
PCODE_MAILBOXwrite under certain circumstances, it is thought that the root problem causing the problem should be fixed.
Actually I have tested it and indeed the extra check on the PCODE_READY is needed and makes the driver probe correctly.
Thanks,
Andi
Quoted 117 lines · Andi Shyti
Hi Gwan-gyeong,
thanks for the review and the thorough explanation.
Quoted 73 lines · Gwan-gyeong Mun
Quoted 9 lines · Andi Shyti
Quoted 8 lines · Jani Nikula
Quoted 5 lines · Andi Shyti
From: Aravind Iddamsetty <[email protected]>
During module load not all the punit transaction have completed and we might end up timing out, as shown by the following warning:
Root cause?
Hi Andi, looking at the log where this problem is reported,
https://gitlab.freedesktop.org/drm/intel/-/issues/7814
<4> [268.276908] i915 0000:4d:00.0:
drm_WARN_ON_ONCE(timeout_base_ms > 3)In the code above, the call stack is output, and the return value of
intel_pcode_init()returns -11 (-EAGAIN).<3> [268.329058] i915 0000:4d:00.0: [drm] *ERROR* gt0:
intel_pcode_initfailed -11If you simplify the function call flow, you can see it as below. (The call of
_wait_for(COND, timeout_base_ms * 1000, 10, 10)inskl_pcode_request()is omitted)-------------------------------------------------------------------------
intel_pcode_init()
|
+-> skl_pcode_request(uncore,DG1_PCODE_STATUS,DG1_UNCORE_GET_INIT_STATUS, DG1_UNCORE_INIT_STATUS_COMPLETE, DG1_UNCORE_INIT_STATUS_COMPLETE, 180000); | +-> skl_pcode_try_request() | +-> *status = __snb_pcode_rw(uncore, mbox, &request, NULL, 500, 0, true);-------------------------------------------------------------------------
static int __snb_pcode_rw(structintel_uncore*uncore, u32 mbox,u32 *val, u32 *val1, int fast_timeout_us, int slow_timeout_ms, bool is_read){
...if (intel_uncore_read_fw(uncore, GEN6_PCODE_MAILBOX) & GEN6_PCODE_READY) return -EAGAIN; intel_uncore_write_fw(uncore, GEN6_PCODE_DATA, *val); intel_uncore_write_fw(uncore, GEN6_PCODE_DATA1, val1 ? *val1 : 0); intel_uncore_write_fw(uncore, GEN6_PCODE_MAILBOX, GEN6_PCODE_READY | mbox); if (__intel_wait_for_register_fw(uncore, GEN6_PCODE_MAILBOX, GEN6_PCODE_READY, 0, fast_timeout_us, slow_timeout_ms, &mbox)) return -ETIMEDOUT;...
}
-------------------------------------------------------------------------Quoted 5 lines
The case where
skl_pcode_request()returns-EAGAINcan be considered as the following scenario.
1. 1. When checking theGEN6_PCODE_MAILBOXregister status before writing the value to theGEN6_PCODE_DATAregister in__snb_pcode_rw(), it is always BUSYcorrect! We fail with EAGAIN because we are not able to communicate with the punit because the punit is busy.
Talking about this case we are in boot time and the gpu is performing its boot routine, the punit as well. While the punit is working we try communicate with it, but unfortunately, being busy, we fail with
-EAGAINexactly where you pointed.Adding an extra
wait_for_register_fw, i.e. waiting until thePCODE_READYregister tells that the punit is ready, makes sure that the read/write will succeed.Thus Chris has added a 10 seconds wait before the very first read and write. If the punit is not ready we don't fail with
-EAGAINand give up the driver loading as it happens now. But we give it another chance trying to probe it again later.Quoted 4 lines
2.
_wait_for(COND, timeout_base_ms * 1000, 10, 10)ofskl_pcode_request()returns-EAGAINif theGEN6_PCODE_MAILBOXregister indicates BUSY even after waiting 500us after writing a value to theGEN6_PCODE_DATAregister in__snb_pcode_rw()Isn't it the same as '1'?
Quoted 3 lines
(Even if
skl_pcode_request()gives a timeout of 180 seconds, the time used each time__snb_pcode_rw()is called is up to 500us. The rest of the time is used for sleep.)There is one big, massive, huge difference... the timeout in
skl_pcode_request()after the read/write, not before. So that at the first communication we fail, return-EAGAINand give up probing without starting any timer.
Quoted 8 lines
Be aware of the fact that the timeout is not for the current communication, but for the next one. De facto we start the timeout after communicating, this makes sure that the next communication will work.
But no one makes sure that the very first communication works during probe. Thus the extra wait.
Hi Andi,
In the call flow invoked by intel_pcode_init(), I've added brief comments where further clarification is needed in this scenario, and a description of the suspicious scenario at the bottom.
-------------------------------------------------------------------------intel_pcode_init()
|
+-> skl_pcode_request(uncore, DG1_PCODE_STATUS,
DG1_UNCORE_GET_INIT_STATUS,
DG1_UNCORE_INIT_STATUS_COMPLETE,
DG1_UNCORE_INIT_STATUS_COMPLETE, 180000);
|
+-> skl_pcode_try_request()
|
+-> *status = __snb_pcode_rw(uncore, mbox, &request, NULL,
500, 0, true);-------------------------------------------------------------------------
static int __snb_pcode_rw(struct intel_uncore *uncore, u32 mbox,
u32 *val, u32 *val1,
int fast_timeout_us, int slow_timeout_ms,
bool is_read)
{
...
/* Before writing a value to the GEN6_PCODE_DATA register,
check if the bit in the GEN6_PCODE_MAILBOX register indicates
BUSY. */
if (intel_uncore_read_fw(uncore, GEN6_PCODE_MAILBOX) & GEN6_PCODE_READY)
return -EAGAIN;
/* write value to GEN6_PCODE_DATA register */
intel_uncore_write_fw(uncore, GEN6_PCODE_DATA, *val);
intel_uncore_write_fw(uncore, GEN6_PCODE_DATA1, val1 ? *val1 : 0);
/* In this scenario, the value
"DG1_PCODE_STATUS | GEN6_PCODE_READY"
is written to the GEN6_PCODE_MAILBOX register,
so that the Busy status of the GEN6_PCODE_MAILBOX register
can be checked later.
(When the value of the GEN6_PCODE_READY bit of the
GEN6_PCODE_MAILBOX register changes to 0, the operation can
be considered completed.) */
intel_uncore_write_fw(uncore,
GEN6_PCODE_MAILBOX, GEN6_PCODE_READY | mbox);
/* In this scenario, verify that the BUSY status bit in the
GEN6_PCODE_MAILBOX register turns off for up to 500us. */
if (__intel_wait_for_register_fw(uncore,
GEN6_PCODE_MAILBOX,
GEN6_PCODE_READY, 0,
fast_timeout_us,
slow_timeout_ms,
&mbox))
return -ETIMEDOUT;
/* If there is a failure here, it may be considered that the
"DG1_PCODE_STATUS | GEN6_PCODE_READY" operation was not
completed within 500us */...
}
int skl_pcode_request(struct intel_uncore *uncore, u32 mbox, u32 request,
u32 reply_mask, u32 reply, int timeout_base_ms)
{
u32 status;
int ret;
mutex_lock(&uncore->i915->sb_lock);#define COND \
skl_pcode_try_request(uncore, mbox, request, reply_mask, reply, &status)
/* the first trial for skl_pcode_try_request() can return
-EAGAIN or -ETIMEDOUT. And the code did not check the error
code here, so we don't know how far the __snb_pcode_rw()
function went. It is not known whether the pcode_mailbox
status was busy before writing the value to the
GEN6_PCODE_DATA register or after.*/
if (COND) {
ret = 0;
goto out;
}
/* In this scenario, skl_pcode_try_request() is invoked every
10us for 180 seconds. When skl_pcode_try_request() returns
-EAGAIN and -ETIMEDOUT by _wait_for(),
-ETIMEDOUT is returned to a variable ret. */
ret = _wait_for(COND, timeout_base_ms * 1000, 10, 10);
if (!ret)
goto out;
/*
* The above can time out if the number of requests was low (2 in the
* worst case) _and_ PCODE was busy for some reason even after a
* (queued) request and @timeout_base_ms delay. As a workaround retry
* the poll with preemption disabled to maximize the number of
* requests. Increase the timeout from @timeout_base_ms to 50ms to
* account for interrupts that could reduce the number of these
* requests, and for any quirks of the PCODE firmware that delays
* the request completion.
*/
drm_dbg_kms(&uncore->i915->drm,
"PCODE timeout, retrying with preemption disabled\n");
drm_WARN_ON_ONCE(&uncore->i915->drm, timeout_base_ms > 3);
preempt_disable();
ret = wait_for_atomic(COND, 50);
preempt_enable();out:
mutex_unlock(&uncore->i915->sb_lock); return status ? status : ret;
#undef COND
}
---------------------------------------------------------
If you try skl_pcode_try_request() for 180 seconds in skl_pcode_request(), and the first "intel_uncore_read_fw(uncore, GEN6_PCODE_MAILBOX) & GEN6_PCODE_READY)" call in __snb_pcode_rw() that skl_pcode_try_request() invokes always fails. if then it does not make sense to me why this patch fixes it by just waiting 10 seconds.This is because if it was called with the flow described above, 180 seconds is longer than 10 seconds, so the scenario you mentioned is also covered in the existing code.
To describe in more detail the second scenario I previously commented on:skl_pcode_request() tries skl_pcode_try_request() for 180 seconds
1) In skl_pcode_try_request(), the GEN6_PCODE_MAILBOX register bit is
not BUSY, so the value is written to the GEN6_PCODE_DATA1 register.
2) skl_pcode_try_request() returns -ETIMEDOUT value because the
operation of 1) does not complete within 500us.
3) Scenario in which the GEN6_PCODE_MAILBOX register bit is checked as
BUSY and returns -EAGAIN in the last call of skl_pcode_try_request() invoked by skl_pcode_request()
If the reason why this problem occurred is because of this scenario, shouldn't there be an attempt to increase fast_timeout_us used as an argument of __snb_pcode_rw() to 500us or more when skl_pcode_try_request() returns -ETIMEDOUT?
Br,
G.G.
Quoted 12 lines
Quoted 6 lines
In the situation where the problem is reported, it is not possible to confirm exactly which scenario code causes the problem with the current log information, and it seems that additional analysis is needed to confirm it. If the hardware takes more than 500us to succeed after
PCODE_MAILBOXwrite under certain circumstances, it is thought that the root problem causing the problem should be fixed.Actually I have tested it and indeed the extra check on the
PCODE_READYis needed and makes the driver probe correctly.Thanks,
Andi
Hi GG,
thanks for the deep analysis!
Quoted 20 lines
Hi Andi,
In the call flow invoked byintel_pcode_init(), I've added brief comments where further clarification is needed in this scenario, and a description of the suspicious scenario at the bottom.-------------------------------------------------------------------------
intel_pcode_init()
|
+-> skl_pcode_request(uncore,DG1_PCODE_STATUS,DG1_UNCORE_GET_INIT_STATUS, DG1_UNCORE_INIT_STATUS_COMPLETE, DG1_UNCORE_INIT_STATUS_COMPLETE, 180000); | +-> skl_pcode_try_request() | +-> *status = __snb_pcode_rw(uncore, mbox, &request, NULL, 500, 0, true);-------------------------------------------------------------------------
static int __snb_pcode_rw(structintel_uncore*uncore, u32 mbox,
Quoted 10 lines
u32 *val, u32 *val1,
int fast_timeout_us, int slow_timeout_ms,
bool is_read)
{
...
/* Before writing a value to the GEN6_PCODE_DATA register,
check if the bit in the GEN6_PCODE_MAILBOX register indicates
BUSY. */
if (intel_uncore_read_fw(uncore, GEN6_PCODE_MAILBOX) & GEN6_PCODE_READY)
return -EAGAIN;what if we fail here because the punit is still initializing and will be ready, say, in 10 seconds?
GG, without going any further, we fail here! The -EAGAIN we receive from the test comes from this point. We don't fail with -ETIMEDOUT, but with -EAGAIN and the reason is because the punit is not ready to perform the very fist communication and we fail the probing.
It doesn't mean, though, that there is anything wrong, we just need to wait a bit before "taking drastic decisions"!
This patch is suggesting to wait up to 10s for the punit to be ready and eventually try to probe again... and, indeed, it works!
Andi
Quoted 115 lines
/* write value to
GEN6_PCODE_DATAregister */intel_uncore_write_fw(uncore, GEN6_PCODE_DATA, *val);
intel_uncore_write_fw(uncore, GEN6_PCODE_DATA1, val1 ? *val1 : 0);/* In this scenario, the value
"DG1_PCODE_STATUS|GEN6_PCODE_READY"
is written to theGEN6_PCODE_MAILBOXregister,
so that the Busy status of theGEN6_PCODE_MAILBOXregister can be checked later.
(When the value of theGEN6_PCODE_READYbit of the
GEN6_PCODE_MAILBOXregister changes to 0, the operation can be considered completed.) */
intel_uncore_write_fw(uncore,
GEN6_PCODE_MAILBOX,GEN6_PCODE_READY| mbox);/* In this scenario, verify that the BUSY status bit in the
GEN6_PCODE_MAILBOXregister turns off for up to 500us. */
if (__intel_wait_for_register_fw(uncore,
GEN6_PCODE_MAILBOX,GEN6_PCODE_READY, 0,fast_timeout_us,slow_timeout_ms, &mbox))
return-ETIMEDOUT;
/* If there is a failure here, it may be considered that the
"DG1_PCODE_STATUS|GEN6_PCODE_READY" operation was not completed within 500us */
...
}int skl_pcode_request(struct
intel_uncore*uncore, u32 mbox, u32 request,
u32reply_mask, u32 reply, inttimeout_base_ms){ u32 status; int ret; mutex_lock(&uncore->i915->sb_lock);#define COND \ skl_pcode_try_request(uncore, mbox, request, reply_mask, reply, &status) /* the first trial for skl_pcode_try_request() can return -EAGAIN or -ETIMEDOUT. And the code did not check the error code here, so we don't know how far the __snb_pcode_rw() function went. It is not known whether the pcode_mailbox status was busy before writing the value to the GEN6_PCODE_DATA register or after.*/ if (COND) { ret = 0; goto out; } /* In this scenario, skl_pcode_try_request() is invoked every 10us for 180 seconds. When skl_pcode_try_request() returns -EAGAIN and -ETIMEDOUT by _wait_for(), -ETIMEDOUT is returned to a variable ret. */ ret = _wait_for(COND, timeout_base_ms * 1000, 10, 10); if (!ret) goto out; /* * The above can time out if the number of requests was low (2 in the * worst case) _and_ PCODE was busy for some reason even after a * (queued) request and @timeout_base_ms delay. As a workaround retry * the poll with preemption disabled to maximize the number of * requests. Increase the timeout from @timeout_base_ms to 50ms to * account for interrupts that could reduce the number of these * requests, and for any quirks of the PCODE firmware that delays * the request completion. */ drm_dbg_kms(&uncore->i915->drm, "PCODE timeout, retrying with preemption disabled\n"); drm_WARN_ON_ONCE(&uncore->i915->drm, timeout_base_ms > 3); preempt_disable(); ret = wait_for_atomic(COND, 50); preempt_enable();out:
mutex_unlock(&uncore->i915->sb_lock); return status ? status : ret;#undef COND
}---------------------------------------------------------
If you try
skl_pcode_try_request()for 180 seconds inskl_pcode_request(), and the first "intel_uncore_read_fw(uncore, GEN6_PCODE_MAILBOX)&GEN6_PCODE_READY)" call in__snb_pcode_rw()thatskl_pcode_try_request()invokes always fails. if then it does not make sense to me why this patch fixes it by just waiting 10 seconds.This is because if it was called with the flow described above, 180 seconds is longer than 10 seconds, so the scenario you mentioned is also covered in the existing code.To describe in more detail the second scenario I previously commented on:
skl_pcode_request()triesskl_pcode_try_request()for 180 seconds
1) Inskl_pcode_try_request(), theGEN6_PCODE_MAILBOXregister bit is not
BUSY, so the value is written to theGEN6_PCODE_DATA1register.
2)skl_pcode_try_request()returns-ETIMEDOUTvalue because the operation
of 1) does not complete within 500us.
3) Scenario in which theGEN6_PCODE_MAILBOXregister bit is checked as BUSY
and returns-EAGAINin the last call ofskl_pcode_try_request()invoked byskl_pcode_request()If the reason why this problem occurred is because of this scenario, shouldn't there be an attempt to increase
fast_timeout_usused as an argument of__snb_pcode_rw()to 500us or more whenskl_pcode_try_request()returns-ETIMEDOUT?Br,
G.G.
Rodrigo Vivi
maintainerQuoted 44 lines · Andi Shyti
Hi GG,
thanks for the deep analysis!
Quoted 20 lines
Hi Andi,
In the call flow invoked byintel_pcode_init(), I've added brief comments where further clarification is needed in this scenario, and a description of the suspicious scenario at the bottom.-------------------------------------------------------------------------
intel_pcode_init()
|
+-> skl_pcode_request(uncore,DG1_PCODE_STATUS,DG1_UNCORE_GET_INIT_STATUS, DG1_UNCORE_INIT_STATUS_COMPLETE, DG1_UNCORE_INIT_STATUS_COMPLETE, 180000); | +-> skl_pcode_try_request() | +-> *status = __snb_pcode_rw(uncore, mbox, &request, NULL, 500, 0, true);-------------------------------------------------------------------------
static int __snb_pcode_rw(structintel_uncore*uncore, u32 mbox,Quoted 10 lines
u32 *val, u32 *val1, int fast_timeout_us, int slow_timeout_ms, bool is_read) { ... /* Before writing a value to the GEN6_PCODE_DATA register, check if the bit in the GEN6_PCODE_MAILBOX register indicates BUSY. */ if (intel_uncore_read_fw(uncore, GEN6_PCODE_MAILBOX) & GEN6_PCODE_READY) return -EAGAIN;what if we fail here because the punit is still initializing and will be ready, say, in 10 seconds?
GG, without going any further, we fail here! The
-EAGAINwe receive from the test comes from this point. We don't fail with-ETIMEDOUT, but with-EAGAINand the reason is because the punit is not ready to perform the very fist communication and we fail the probing.
Quoted 5 lines
It doesn't mean, though, that there is anything wrong, we just need to wait a bit before "taking drastic decisions"!
This patch is suggesting to wait up to 10s for the punit to be ready and eventually try to probe again... and, indeed, it works!
As GG, what I still don't understand is how this extra 10 seconds wait helps... have you tried to simple add the 10 to the 180 and make the code 190 sec instead?
Quoted 118 lines
Andi
Quoted 115 lines
/* write value to
GEN6_PCODE_DATAregister */intel_uncore_write_fw(uncore, GEN6_PCODE_DATA, *val);
intel_uncore_write_fw(uncore, GEN6_PCODE_DATA1, val1 ? *val1 : 0);/* In this scenario, the value
"DG1_PCODE_STATUS|GEN6_PCODE_READY"
is written to theGEN6_PCODE_MAILBOXregister,
so that the Busy status of theGEN6_PCODE_MAILBOXregister can be checked later.
(When the value of theGEN6_PCODE_READYbit of the
GEN6_PCODE_MAILBOXregister changes to 0, the operation can be considered completed.) */
intel_uncore_write_fw(uncore,
GEN6_PCODE_MAILBOX,GEN6_PCODE_READY| mbox);/* In this scenario, verify that the BUSY status bit in the
GEN6_PCODE_MAILBOXregister turns off for up to 500us. */
if (__intel_wait_for_register_fw(uncore,
GEN6_PCODE_MAILBOX,GEN6_PCODE_READY, 0,fast_timeout_us,slow_timeout_ms, &mbox))
return-ETIMEDOUT;
/* If there is a failure here, it may be considered that the
"DG1_PCODE_STATUS|GEN6_PCODE_READY" operation was not completed within 500us */
...
}int skl_pcode_request(struct
intel_uncore*uncore, u32 mbox, u32 request,
u32reply_mask, u32 reply, inttimeout_base_ms){ u32 status; int ret; mutex_lock(&uncore->i915->sb_lock);#define COND \ skl_pcode_try_request(uncore, mbox, request, reply_mask, reply, &status) /* the first trial for skl_pcode_try_request() can return -EAGAIN or -ETIMEDOUT. And the code did not check the error code here, so we don't know how far the __snb_pcode_rw() function went. It is not known whether the pcode_mailbox status was busy before writing the value to the GEN6_PCODE_DATA register or after.*/ if (COND) { ret = 0; goto out; } /* In this scenario, skl_pcode_try_request() is invoked every 10us for 180 seconds. When skl_pcode_try_request() returns -EAGAIN and -ETIMEDOUT by _wait_for(), -ETIMEDOUT is returned to a variable ret. */ ret = _wait_for(COND, timeout_base_ms * 1000, 10, 10); if (!ret) goto out; /* * The above can time out if the number of requests was low (2 in the * worst case) _and_ PCODE was busy for some reason even after a * (queued) request and @timeout_base_ms delay. As a workaround retry * the poll with preemption disabled to maximize the number of * requests. Increase the timeout from @timeout_base_ms to 50ms to * account for interrupts that could reduce the number of these * requests, and for any quirks of the PCODE firmware that delays * the request completion. */ drm_dbg_kms(&uncore->i915->drm, "PCODE timeout, retrying with preemption disabled\n"); drm_WARN_ON_ONCE(&uncore->i915->drm, timeout_base_ms > 3); preempt_disable(); ret = wait_for_atomic(COND, 50); preempt_enable();out:
mutex_unlock(&uncore->i915->sb_lock); return status ? status : ret;#undef COND
}---------------------------------------------------------
If you try
skl_pcode_try_request()for 180 seconds inskl_pcode_request(), and the first "intel_uncore_read_fw(uncore, GEN6_PCODE_MAILBOX)&GEN6_PCODE_READY)" call in__snb_pcode_rw()thatskl_pcode_try_request()invokes always fails. if then it does not make sense to me why this patch fixes it by just waiting 10 seconds.This is because if it was called with the flow described above, 180 seconds is longer than 10 seconds, so the scenario you mentioned is also covered in the existing code.To describe in more detail the second scenario I previously commented on:
skl_pcode_request()triesskl_pcode_try_request()for 180 seconds
1) Inskl_pcode_try_request(), theGEN6_PCODE_MAILBOXregister bit is not
BUSY, so the value is written to theGEN6_PCODE_DATA1register.
2)skl_pcode_try_request()returns-ETIMEDOUTvalue because the operation
of 1) does not complete within 500us.
3) Scenario in which theGEN6_PCODE_MAILBOXregister bit is checked as BUSY
and returns-EAGAINin the last call ofskl_pcode_try_request()invoked byskl_pcode_request()If the reason why this problem occurred is because of this scenario, shouldn't there be an attempt to increase
fast_timeout_usused as an argument of__snb_pcode_rw()to 500us or more whenskl_pcode_try_request()returns-ETIMEDOUT?Br,
G.G.
Hi Rodrigo,
Quoted 42 lines
Quoted 42 lines
Quoted 19 lines
In the call flow invoked by
intel_pcode_init(), I've added brief comments where further clarification is needed in this scenario, and a description of the suspicious scenario at the bottom.-------------------------------------------------------------------------
intel_pcode_init()
|
+-> skl_pcode_request(uncore,DG1_PCODE_STATUS,DG1_UNCORE_GET_INIT_STATUS, DG1_UNCORE_INIT_STATUS_COMPLETE, DG1_UNCORE_INIT_STATUS_COMPLETE, 180000); | +-> skl_pcode_try_request() | +-> *status = __snb_pcode_rw(uncore, mbox, &request, NULL, 500, 0, true);-------------------------------------------------------------------------
static int __snb_pcode_rw(structintel_uncore*uncore, u32 mbox,Quoted 10 lines
u32 *val, u32 *val1, int fast_timeout_us, int slow_timeout_ms, bool is_read) { ... /* Before writing a value to the GEN6_PCODE_DATA register, check if the bit in the GEN6_PCODE_MAILBOX register indicates BUSY. */ if (intel_uncore_read_fw(uncore, GEN6_PCODE_MAILBOX) & GEN6_PCODE_READY) return -EAGAIN;what if we fail here because the punit is still initializing and will be ready, say, in 10 seconds?
GG, without going any further, we fail here! The
-EAGAINwe receive from the test comes from this point. We don't fail with-ETIMEDOUT, but with-EAGAINand the reason is because the punit is not ready to perform the very fist communication and we fail the probing.It doesn't mean, though, that there is anything wrong, we just need to wait a bit before "taking drastic decisions"!
Quoted 6 lines
Quoted 2 lines
This patch is suggesting to wait up to 10s for the punit to be ready and eventually try to probe again... and, indeed, it works!
As GG, what I still don't understand is how this extra 10 seconds wait helps... have you tried to simple add the 10 to the 180 and make the code 190 sec instead?
maybe I haven't been able to explain the issue properly.
I can even set that timer to 2hrs and a half and nothing changes because we fail before.
Here it's not a matter of how much do I wait but when do I check the pcode readiness (i.e. signalled by the GEN6_PCODE_READY bit in the GEN6_PCODE_MAILBOX register).
During a normal run we are always sure that communicating with the punit works, because we made it sure during the previous transaction.
During probe there is no previous transaction and we start communicating with the punit without making sure that it is ready. And indeed some times it is not, so that we suppress the probing on purpose instead of giving it another chance.
I admit that the commit message is not written properly and rather misleading, but here it's not at all a matter of how much do I wait.
Thanks, Rodrigo!
Andi
Quoted 118 lines
Quoted 118 lines
Andi
Quoted 115 lines
/* write value to
GEN6_PCODE_DATAregister */intel_uncore_write_fw(uncore, GEN6_PCODE_DATA, *val);
intel_uncore_write_fw(uncore, GEN6_PCODE_DATA1, val1 ? *val1 : 0);/* In this scenario, the value
"DG1_PCODE_STATUS|GEN6_PCODE_READY"
is written to theGEN6_PCODE_MAILBOXregister,
so that the Busy status of theGEN6_PCODE_MAILBOXregister can be checked later.
(When the value of theGEN6_PCODE_READYbit of the
GEN6_PCODE_MAILBOXregister changes to 0, the operation can be considered completed.) */
intel_uncore_write_fw(uncore,
GEN6_PCODE_MAILBOX,GEN6_PCODE_READY| mbox);/* In this scenario, verify that the BUSY status bit in the
GEN6_PCODE_MAILBOXregister turns off for up to 500us. */
if (__intel_wait_for_register_fw(uncore,
GEN6_PCODE_MAILBOX,GEN6_PCODE_READY, 0,fast_timeout_us,slow_timeout_ms, &mbox))
return-ETIMEDOUT;
/* If there is a failure here, it may be considered that the
"DG1_PCODE_STATUS|GEN6_PCODE_READY" operation was not completed within 500us */
...
}int skl_pcode_request(struct
intel_uncore*uncore, u32 mbox, u32 request,
u32reply_mask, u32 reply, inttimeout_base_ms){ u32 status; int ret; mutex_lock(&uncore->i915->sb_lock);#define COND \ skl_pcode_try_request(uncore, mbox, request, reply_mask, reply, &status) /* the first trial for skl_pcode_try_request() can return -EAGAIN or -ETIMEDOUT. And the code did not check the error code here, so we don't know how far the __snb_pcode_rw() function went. It is not known whether the pcode_mailbox status was busy before writing the value to the GEN6_PCODE_DATA register or after.*/ if (COND) { ret = 0; goto out; } /* In this scenario, skl_pcode_try_request() is invoked every 10us for 180 seconds. When skl_pcode_try_request() returns -EAGAIN and -ETIMEDOUT by _wait_for(), -ETIMEDOUT is returned to a variable ret. */ ret = _wait_for(COND, timeout_base_ms * 1000, 10, 10); if (!ret) goto out; /* * The above can time out if the number of requests was low (2 in the * worst case) _and_ PCODE was busy for some reason even after a * (queued) request and @timeout_base_ms delay. As a workaround retry * the poll with preemption disabled to maximize the number of * requests. Increase the timeout from @timeout_base_ms to 50ms to * account for interrupts that could reduce the number of these * requests, and for any quirks of the PCODE firmware that delays * the request completion. */ drm_dbg_kms(&uncore->i915->drm, "PCODE timeout, retrying with preemption disabled\n"); drm_WARN_ON_ONCE(&uncore->i915->drm, timeout_base_ms > 3); preempt_disable(); ret = wait_for_atomic(COND, 50); preempt_enable();out:
mutex_unlock(&uncore->i915->sb_lock); return status ? status : ret;#undef COND
}---------------------------------------------------------
If you try
skl_pcode_try_request()for 180 seconds inskl_pcode_request(), and the first "intel_uncore_read_fw(uncore, GEN6_PCODE_MAILBOX)&GEN6_PCODE_READY)" call in__snb_pcode_rw()thatskl_pcode_try_request()invokes always fails. if then it does not make sense to me why this patch fixes it by just waiting 10 seconds.This is because if it was called with the flow described above, 180 seconds is longer than 10 seconds, so the scenario you mentioned is also covered in the existing code.To describe in more detail the second scenario I previously commented on:
skl_pcode_request()triesskl_pcode_try_request()for 180 seconds
1) Inskl_pcode_try_request(), theGEN6_PCODE_MAILBOXregister bit is not
BUSY, so the value is written to theGEN6_PCODE_DATA1register.
2)skl_pcode_try_request()returns-ETIMEDOUTvalue because the operation
of 1) does not complete within 500us.
3) Scenario in which theGEN6_PCODE_MAILBOXregister bit is checked as BUSY
and returns-EAGAINin the last call ofskl_pcode_try_request()invoked byskl_pcode_request()If the reason why this problem occurred is because of this scenario, shouldn't there be an attempt to increase
fast_timeout_usused as an argument of__snb_pcode_rw()to 500us or more whenskl_pcode_try_request()returns-ETIMEDOUT?Br,
G.G.
Rodrigo Vivi
maintainerQuoted 64 lines · Andi Shyti
Hi Rodrigo,
Quoted 42 lines
Quoted 42 lines
Quoted 19 lines
In the call flow invoked by
intel_pcode_init(), I've added brief comments where further clarification is needed in this scenario, and a description of the suspicious scenario at the bottom.-------------------------------------------------------------------------
intel_pcode_init()
|
+-> skl_pcode_request(uncore,DG1_PCODE_STATUS,DG1_UNCORE_GET_INIT_STATUS, DG1_UNCORE_INIT_STATUS_COMPLETE, DG1_UNCORE_INIT_STATUS_COMPLETE, 180000); | +-> skl_pcode_try_request() | +-> *status = __snb_pcode_rw(uncore, mbox, &request, NULL, 500, 0, true);-------------------------------------------------------------------------
static int __snb_pcode_rw(structintel_uncore*uncore, u32 mbox,Quoted 10 lines
u32 *val, u32 *val1, int fast_timeout_us, int slow_timeout_ms, bool is_read) { ... /* Before writing a value to the GEN6_PCODE_DATA register, check if the bit in the GEN6_PCODE_MAILBOX register indicates BUSY. */ if (intel_uncore_read_fw(uncore, GEN6_PCODE_MAILBOX) & GEN6_PCODE_READY) return -EAGAIN;what if we fail here because the punit is still initializing and will be ready, say, in 10 seconds?
GG, without going any further, we fail here! The
-EAGAINwe receive from the test comes from this point. We don't fail with-ETIMEDOUT, but with-EAGAINand the reason is because the punit is not ready to perform the very fist communication and we fail the probing.It doesn't mean, though, that there is anything wrong, we just need to wait a bit before "taking drastic decisions"!
Quoted 6 lines
Quoted 2 lines
This patch is suggesting to wait up to 10s for the punit to be ready and eventually try to probe again... and, indeed, it works!
As GG, what I still don't understand is how this extra 10 seconds wait helps... have you tried to simple add the 10 to the 180 and make the code 190 sec instead?
maybe I haven't been able to explain the issue properly.
I can even set that timer to 2hrs and a half and nothing changes because we fail before.
Here it's not a matter of how much do I wait but when do I check the pcode readiness (i.e. signalled by the
GEN6_PCODE_READYbit in theGEN6_PCODE_MAILBOXregister).During a normal run we are always sure that communicating with the punit works, because we made it sure during the previous transaction.
Quoted 8 lines
During probe there is no previous transaction and we start communicating with the punit without making sure that it is ready. And indeed some times it is not, so that we suppress the probing on purpose instead of giving it another chance.
I admit that the commit message is not written properly and rather misleading, but here it's not at all a matter of how much do I wait.
The commit message was initially confused because it looks like we are just adding a wait, without doing anything....
But looking to the code we can see that it will wait until pcode is ready with a timeout of 10 seconds.
But if pcode is ready in 10 seconds, why pcode is not ready in 190 seconds. We are doing absolutely nothing more that could make pcode ready in 10 seconds that won't be in 190.
This is what we are missing here... The code as is doesn't make a lot of sense to us and it looks like it is solving the issue by the 10 extra seconds and not by some special status checking.
Quoted 122 lines
Thanks, Rodrigo!
AndiQuoted 118 lines
Quoted 118 lines
Andi
Quoted 115 lines
/* write value to
GEN6_PCODE_DATAregister */intel_uncore_write_fw(uncore, GEN6_PCODE_DATA, *val);
intel_uncore_write_fw(uncore, GEN6_PCODE_DATA1, val1 ? *val1 : 0);/* In this scenario, the value
"DG1_PCODE_STATUS|GEN6_PCODE_READY"
is written to theGEN6_PCODE_MAILBOXregister,
so that the Busy status of theGEN6_PCODE_MAILBOXregister can be checked later.
(When the value of theGEN6_PCODE_READYbit of the
GEN6_PCODE_MAILBOXregister changes to 0, the operation can be considered completed.) */
intel_uncore_write_fw(uncore,
GEN6_PCODE_MAILBOX,GEN6_PCODE_READY| mbox);/* In this scenario, verify that the BUSY status bit in the
GEN6_PCODE_MAILBOXregister turns off for up to 500us. */
if (__intel_wait_for_register_fw(uncore,
GEN6_PCODE_MAILBOX,GEN6_PCODE_READY, 0,fast_timeout_us,slow_timeout_ms, &mbox))
return-ETIMEDOUT;
/* If there is a failure here, it may be considered that the
"DG1_PCODE_STATUS|GEN6_PCODE_READY" operation was not completed within 500us */
...
}int skl_pcode_request(struct
intel_uncore*uncore, u32 mbox, u32 request,
u32reply_mask, u32 reply, inttimeout_base_ms){ u32 status; int ret; mutex_lock(&uncore->i915->sb_lock);#define COND \ skl_pcode_try_request(uncore, mbox, request, reply_mask, reply, &status) /* the first trial for skl_pcode_try_request() can return -EAGAIN or -ETIMEDOUT. And the code did not check the error code here, so we don't know how far the __snb_pcode_rw() function went. It is not known whether the pcode_mailbox status was busy before writing the value to the GEN6_PCODE_DATA register or after.*/ if (COND) { ret = 0; goto out; } /* In this scenario, skl_pcode_try_request() is invoked every 10us for 180 seconds. When skl_pcode_try_request() returns -EAGAIN and -ETIMEDOUT by _wait_for(), -ETIMEDOUT is returned to a variable ret. */ ret = _wait_for(COND, timeout_base_ms * 1000, 10, 10); if (!ret) goto out; /* * The above can time out if the number of requests was low (2 in the * worst case) _and_ PCODE was busy for some reason even after a * (queued) request and @timeout_base_ms delay. As a workaround retry * the poll with preemption disabled to maximize the number of * requests. Increase the timeout from @timeout_base_ms to 50ms to * account for interrupts that could reduce the number of these * requests, and for any quirks of the PCODE firmware that delays * the request completion. */ drm_dbg_kms(&uncore->i915->drm, "PCODE timeout, retrying with preemption disabled\n"); drm_WARN_ON_ONCE(&uncore->i915->drm, timeout_base_ms > 3); preempt_disable(); ret = wait_for_atomic(COND, 50); preempt_enable();out:
mutex_unlock(&uncore->i915->sb_lock); return status ? status : ret;#undef COND
}---------------------------------------------------------
If you try
skl_pcode_try_request()for 180 seconds inskl_pcode_request(), and the first "intel_uncore_read_fw(uncore, GEN6_PCODE_MAILBOX)&GEN6_PCODE_READY)" call in__snb_pcode_rw()thatskl_pcode_try_request()invokes always fails. if then it does not make sense to me why this patch fixes it by just waiting 10 seconds.This is because if it was called with the flow described above, 180 seconds is longer than 10 seconds, so the scenario you mentioned is also covered in the existing code.To describe in more detail the second scenario I previously commented on:
skl_pcode_request()triesskl_pcode_try_request()for 180 seconds
1) Inskl_pcode_try_request(), theGEN6_PCODE_MAILBOXregister bit is not
BUSY, so the value is written to theGEN6_PCODE_DATA1register.
2)skl_pcode_try_request()returns-ETIMEDOUTvalue because the operation
of 1) does not complete within 500us.
3) Scenario in which theGEN6_PCODE_MAILBOXregister bit is checked as BUSY
and returns-EAGAINin the last call ofskl_pcode_try_request()invoked byskl_pcode_request()If the reason why this problem occurred is because of this scenario, shouldn't there be an attempt to increase
fast_timeout_usused as an argument of__snb_pcode_rw()to 500us or more whenskl_pcode_try_request()returns-ETIMEDOUT?Br,
G.G.
Rodrigo Vivi
maintainerQuoted 80 lines · Rodrigo Vivi
Quoted 64 lines · Andi Shyti
Hi Rodrigo,
Quoted 42 lines
Quoted 42 lines
Quoted 19 lines
In the call flow invoked by
intel_pcode_init(), I've added brief comments where further clarification is needed in this scenario, and a description of the suspicious scenario at the bottom.-------------------------------------------------------------------------
intel_pcode_init()
|
+-> skl_pcode_request(uncore,DG1_PCODE_STATUS,DG1_UNCORE_GET_INIT_STATUS, DG1_UNCORE_INIT_STATUS_COMPLETE, DG1_UNCORE_INIT_STATUS_COMPLETE, 180000); | +-> skl_pcode_try_request() | +-> *status = __snb_pcode_rw(uncore, mbox, &request, NULL, 500, 0, true);-------------------------------------------------------------------------
static int __snb_pcode_rw(structintel_uncore*uncore, u32 mbox,Quoted 10 lines
u32 *val, u32 *val1, int fast_timeout_us, int slow_timeout_ms, bool is_read) { ... /* Before writing a value to the GEN6_PCODE_DATA register, check if the bit in the GEN6_PCODE_MAILBOX register indicates BUSY. */ if (intel_uncore_read_fw(uncore, GEN6_PCODE_MAILBOX) & GEN6_PCODE_READY) return -EAGAIN;what if we fail here because the punit is still initializing and will be ready, say, in 10 seconds?
GG, without going any further, we fail here! The
-EAGAINwe receive from the test comes from this point. We don't fail with-ETIMEDOUT, but with-EAGAINand the reason is because the punit is not ready to perform the very fist communication and we fail the probing.It doesn't mean, though, that there is anything wrong, we just need to wait a bit before "taking drastic decisions"!
Quoted 6 lines
Quoted 2 lines
This patch is suggesting to wait up to 10s for the punit to be ready and eventually try to probe again... and, indeed, it works!
As GG, what I still don't understand is how this extra 10 seconds wait helps... have you tried to simple add the 10 to the 180 and make the code 190 sec instead?
maybe I haven't been able to explain the issue properly.
I can even set that timer to 2hrs and a half and nothing changes because we fail before.
Here it's not a matter of how much do I wait but when do I check the pcode readiness (i.e. signalled by the
GEN6_PCODE_READYbit in theGEN6_PCODE_MAILBOXregister).During a normal run we are always sure that communicating with the punit works, because we made it sure during the previous transaction.
Quoted 8 lines
During probe there is no previous transaction and we start communicating with the punit without making sure that it is ready. And indeed some times it is not, so that we suppress the probing on purpose instead of giving it another chance.
I admit that the commit message is not written properly and rather misleading, but here it's not at all a matter of how much do I wait.
The commit message was initially confused because it looks like we are just adding a wait, without doing anything....
But looking to the code we can see that it will wait until pcode is ready with a timeout of 10 seconds.
Quoted 7 lines
But if pcode is ready in 10 seconds, why pcode is not ready in 190 seconds. We are doing absolutely nothing more that could make pcode ready in 10 seconds that won't be in 190.
This is what we are missing here... The code as is doesn't make a lot of sense to us and it looks like it is solving the issue by the 10 extra seconds and not by some special status checking.
Okay, after an offline talk I am convinced now that we need some check like this in some place.
But the commit message needs be be fully re-written.
It needs to be clear that underneath, the pcode communication functions will do a check for ready without any wait, what will make desired timeout to never really wait for the pcode done and prematurely return.
at __snb_pcode_rw():
if (intel_uncore_read_fw(uncore, GEN6_PCODE_MAILBOX) & GEN6_PCODE_READY)
return -EAGAIN;
So, for this reason we need to ensure that pcode is really ready before we wait.
Other options are to handle the EAGAIN return and then wait. Or even change the __snb_pcode_rw to ensure that it is ready...
Something like:
- if (intel_uncore_read_fw(uncore, GEN6_PCODE_MAILBOX) & GEN6_PCODE_READY)
- return -EAGAIN;
+ if (__intel_wait_for_register_fw(uncore, GEN6_PCODE_MAILBOX,
+ GEN6_PCODE_READY, GEN6_PCODE_READY, + fast_timeout_us, + slow_timeout_ms, + NULL))
+ return -ETIMEDOUT;
In the best regular case it will not change anything since the wait for will return 0 immediatelly... in the worst case we would double the timeout value, but this would be only in this insane case of 180 seconds anyway and if it goes really really nasty...
But anyway, now the motivation and the source of the issue is pretty clear, I can add my rv-b there, but I really want a better commit msg at least...
Quoted 123 lines
Quoted 123 lines
Thanks, Rodrigo!
AndiQuoted 118 lines
Quoted 118 lines
Andi
Quoted 115 lines
/* write value to
GEN6_PCODE_DATAregister */intel_uncore_write_fw(uncore, GEN6_PCODE_DATA, *val);
intel_uncore_write_fw(uncore, GEN6_PCODE_DATA1, val1 ? *val1 : 0);/* In this scenario, the value
"DG1_PCODE_STATUS|GEN6_PCODE_READY"
is written to theGEN6_PCODE_MAILBOXregister,
so that the Busy status of theGEN6_PCODE_MAILBOXregister can be checked later.
(When the value of theGEN6_PCODE_READYbit of the
GEN6_PCODE_MAILBOXregister changes to 0, the operation can be considered completed.) */
intel_uncore_write_fw(uncore,
GEN6_PCODE_MAILBOX,GEN6_PCODE_READY| mbox);/* In this scenario, verify that the BUSY status bit in the
GEN6_PCODE_MAILBOXregister turns off for up to 500us. */
if (__intel_wait_for_register_fw(uncore,
GEN6_PCODE_MAILBOX,GEN6_PCODE_READY, 0,fast_timeout_us,slow_timeout_ms, &mbox))
return-ETIMEDOUT;
/* If there is a failure here, it may be considered that the
"DG1_PCODE_STATUS|GEN6_PCODE_READY" operation was not completed within 500us */
...
}int skl_pcode_request(struct
intel_uncore*uncore, u32 mbox, u32 request,
u32reply_mask, u32 reply, inttimeout_base_ms){ u32 status; int ret; mutex_lock(&uncore->i915->sb_lock);#define COND \ skl_pcode_try_request(uncore, mbox, request, reply_mask, reply, &status) /* the first trial for skl_pcode_try_request() can return -EAGAIN or -ETIMEDOUT. And the code did not check the error code here, so we don't know how far the __snb_pcode_rw() function went. It is not known whether the pcode_mailbox status was busy before writing the value to the GEN6_PCODE_DATA register or after.*/ if (COND) { ret = 0; goto out; } /* In this scenario, skl_pcode_try_request() is invoked every 10us for 180 seconds. When skl_pcode_try_request() returns -EAGAIN and -ETIMEDOUT by _wait_for(), -ETIMEDOUT is returned to a variable ret. */ ret = _wait_for(COND, timeout_base_ms * 1000, 10, 10); if (!ret) goto out; /* * The above can time out if the number of requests was low (2 in the * worst case) _and_ PCODE was busy for some reason even after a * (queued) request and @timeout_base_ms delay. As a workaround retry * the poll with preemption disabled to maximize the number of * requests. Increase the timeout from @timeout_base_ms to 50ms to * account for interrupts that could reduce the number of these * requests, and for any quirks of the PCODE firmware that delays * the request completion. */ drm_dbg_kms(&uncore->i915->drm, "PCODE timeout, retrying with preemption disabled\n"); drm_WARN_ON_ONCE(&uncore->i915->drm, timeout_base_ms > 3); preempt_disable(); ret = wait_for_atomic(COND, 50); preempt_enable();out:
mutex_unlock(&uncore->i915->sb_lock); return status ? status : ret;#undef COND
}---------------------------------------------------------
If you try
skl_pcode_try_request()for 180 seconds inskl_pcode_request(), and the first "intel_uncore_read_fw(uncore, GEN6_PCODE_MAILBOX)&GEN6_PCODE_READY)" call in__snb_pcode_rw()thatskl_pcode_try_request()invokes always fails. if then it does not make sense to me why this patch fixes it by just waiting 10 seconds.This is because if it was called with the flow described above, 180 seconds is longer than 10 seconds, so the scenario you mentioned is also covered in the existing code.To describe in more detail the second scenario I previously commented on:
skl_pcode_request()triesskl_pcode_try_request()for 180 seconds
1) Inskl_pcode_try_request(), theGEN6_PCODE_MAILBOXregister bit is not
BUSY, so the value is written to theGEN6_PCODE_DATA1register.
2)skl_pcode_try_request()returns-ETIMEDOUTvalue because the operation
of 1) does not complete within 500us.
3) Scenario in which theGEN6_PCODE_MAILBOXregister bit is checked as BUSY
and returns-EAGAINin the last call ofskl_pcode_try_request()invoked byskl_pcode_request()If the reason why this problem occurred is because of this scenario, shouldn't there be an attempt to increase
fast_timeout_usused as an argument of__snb_pcode_rw()to 500us or more whenskl_pcode_try_request()returns-ETIMEDOUT?Br,
G.G.
Hi Rodrigo,
first of all, thanks for looking into this!
Quoted 108 lines
Quoted 77 lines
Quoted 62 lines
Quoted 42 lines
Quoted 42 lines
Quoted 19 lines
In the call flow invoked by
intel_pcode_init(), I've added brief comments where further clarification is needed in this scenario, and a description of the suspicious scenario at the bottom.-------------------------------------------------------------------------
intel_pcode_init()
|
+-> skl_pcode_request(uncore,DG1_PCODE_STATUS,DG1_UNCORE_GET_INIT_STATUS, DG1_UNCORE_INIT_STATUS_COMPLETE, DG1_UNCORE_INIT_STATUS_COMPLETE, 180000); | +-> skl_pcode_try_request() | +-> *status = __snb_pcode_rw(uncore, mbox, &request, NULL, 500, 0, true);-------------------------------------------------------------------------
static int __snb_pcode_rw(structintel_uncore*uncore, u32 mbox,Quoted 10 lines
u32 *val, u32 *val1, int fast_timeout_us, int slow_timeout_ms, bool is_read) { ... /* Before writing a value to the GEN6_PCODE_DATA register, check if the bit in the GEN6_PCODE_MAILBOX register indicates BUSY. */ if (intel_uncore_read_fw(uncore, GEN6_PCODE_MAILBOX) & GEN6_PCODE_READY) return -EAGAIN;what if we fail here because the punit is still initializing and will be ready, say, in 10 seconds?
GG, without going any further, we fail here! The
-EAGAINwe receive from the test comes from this point. We don't fail with-ETIMEDOUT, but with-EAGAINand the reason is because the punit is not ready to perform the very fist communication and we fail the probing.It doesn't mean, though, that there is anything wrong, we just need to wait a bit before "taking drastic decisions"!
Quoted 6 lines
Quoted 2 lines
This patch is suggesting to wait up to 10s for the punit to be ready and eventually try to probe again... and, indeed, it works!
As GG, what I still don't understand is how this extra 10 seconds wait helps... have you tried to simple add the 10 to the 180 and make the code 190 sec instead?
maybe I haven't been able to explain the issue properly.
I can even set that timer to 2hrs and a half and nothing changes because we fail before.
Here it's not a matter of how much do I wait but when do I check the pcode readiness (i.e. signalled by the
GEN6_PCODE_READYbit in theGEN6_PCODE_MAILBOXregister).During a normal run we are always sure that communicating with the punit works, because we made it sure during the previous transaction.
Quoted 8 lines
During probe there is no previous transaction and we start communicating with the punit without making sure that it is ready. And indeed some times it is not, so that we suppress the probing on purpose instead of giving it another chance.
I admit that the commit message is not written properly and rather misleading, but here it's not at all a matter of how much do I wait.
The commit message was initially confused because it looks like we are just adding a wait, without doing anything....
But looking to the code we can see that it will wait until pcode is ready with a timeout of 10 seconds.
Quoted 7 lines
But if pcode is ready in 10 seconds, why pcode is not ready in 190 seconds. We are doing absolutely nothing more that could make pcode ready in 10 seconds that won't be in 190.
This is what we are missing here... The code as is doesn't make a lot of sense to us and it looks like it is solving the issue by the 10 extra seconds and not by some special status checking.
Okay, after an offline talk I am convinced now that we need some check like this in some place.
But the commit message needs be be fully re-written.
It needs to be clear that underneath, the pcode communication functions will do a check for ready without any wait, what will make desired timeout to never really wait for the pcode done and prematurely return.
at
__snb_pcode_rw():if (
intel_uncore_read_fw(uncore, GEN6_PCODE_MAILBOX)&GEN6_PCODE_READY)return -EAGAIN;So, for this reason we need to ensure that pcode is really ready before we wait.
Other options are to handle the EAGAIN return and then wait. Or even change the
__snb_pcode_rwto ensure that it is ready...Something like:
Review
Quoted 8 lines
(quoted diff) +6
-2- if (intel_uncore_read_fw(uncore, GEN6_PCODE_MAILBOX) & GEN6_PCODE_READY) - return -EAGAIN; + if (__intel_wait_for_register_fw(uncore, GEN6_PCODE_MAILBOX, + GEN6_PCODE_READY, GEN6_PCODE_READY, + fast_timeout_us, + slow_timeout_ms, + NULL)) + return -ETIMEDOUT;
This works, but the difference is that you are putting it on any call to the __snb_pcode_rw(), even when not necessary.
Putting it in the schema we used in our offline chat, the original implementation was:
+-----------------------+ | r/w -> check if ready |-> r/w -> cir -> r/w -> cir -> r/w -> ... +-----------------------+
probe time
(where cir stands for "check if ready"). Because in the first r/w we didn't check the status of the punit we added:
+-------------------+ | cir -> r/w -> cir |-> r/w -> cir -> r/w -> cir -> r/w -> ... +-------------------+
probe time
So that we are sure that the first r/w works. What you are suggesting is:
+-------------------+ | cir -> r/w -> cir |-> cir -> r/w -> cir -> cir -> r/w -> cir ->
--------------------+
probe time
As you can se we have two "check if ready" in a raw, which might be a bit of an overkill, it doesn't hurt much but it would look like:
if (__intel_wait_for_register_fw())
return -EAGAIN;
intel_uncore_write_fw......
if (__intel_wait_for_register_fw())
return -TIMEDOUT;
and this for every single snb_pcode_r/w.
Besides some functions might need the first wait, some might not. Check, for example icl_tc_cold_exit() where the -EAGAIN is handled. With your version the retries would be quite ponderous.
I'm sorry, but I can't take your suggestion as it can have major consequences along i915, it requires more changes and and it needs to be carefully tested.
On top of that I like Ashutosh refactoring that is quite an elegant way to gain more flexibility at boot time without any further impact.
Quoted 8 lines
In the best regular case it will not change anything since the wait for will return 0 immediatelly... in the worst case we would double the timeout value, but this would be only in this insane case of 180 seconds anyway and if it goes really really nasty...
But anyway, now the motivation and the source of the issue is pretty clear, I can add my rv-b there, but I really want a better commit msg at least...
I definitely need to make a better commit message :)
Thanks a lot,
Andi
Quoted 123 lines
Quoted 123 lines
Quoted 123 lines
Thanks, Rodrigo!
AndiQuoted 118 lines
Quoted 118 lines
Andi
Quoted 115 lines
/* write value to
GEN6_PCODE_DATAregister */intel_uncore_write_fw(uncore, GEN6_PCODE_DATA, *val);
intel_uncore_write_fw(uncore, GEN6_PCODE_DATA1, val1 ? *val1 : 0);/* In this scenario, the value
"DG1_PCODE_STATUS|GEN6_PCODE_READY"
is written to theGEN6_PCODE_MAILBOXregister,
so that the Busy status of theGEN6_PCODE_MAILBOXregister can be checked later.
(When the value of theGEN6_PCODE_READYbit of the
GEN6_PCODE_MAILBOXregister changes to 0, the operation can be considered completed.) */
intel_uncore_write_fw(uncore,
GEN6_PCODE_MAILBOX,GEN6_PCODE_READY| mbox);/* In this scenario, verify that the BUSY status bit in the
GEN6_PCODE_MAILBOXregister turns off for up to 500us. */
if (__intel_wait_for_register_fw(uncore,
GEN6_PCODE_MAILBOX,GEN6_PCODE_READY, 0,fast_timeout_us,slow_timeout_ms, &mbox))
return-ETIMEDOUT;
/* If there is a failure here, it may be considered that the
"DG1_PCODE_STATUS|GEN6_PCODE_READY" operation was not completed within 500us */
...
}int skl_pcode_request(struct
intel_uncore*uncore, u32 mbox, u32 request,
u32reply_mask, u32 reply, inttimeout_base_ms){ u32 status; int ret; mutex_lock(&uncore->i915->sb_lock);#define COND \ skl_pcode_try_request(uncore, mbox, request, reply_mask, reply, &status) /* the first trial for skl_pcode_try_request() can return -EAGAIN or -ETIMEDOUT. And the code did not check the error code here, so we don't know how far the __snb_pcode_rw() function went. It is not known whether the pcode_mailbox status was busy before writing the value to the GEN6_PCODE_DATA register or after.*/ if (COND) { ret = 0; goto out; } /* In this scenario, skl_pcode_try_request() is invoked every 10us for 180 seconds. When skl_pcode_try_request() returns -EAGAIN and -ETIMEDOUT by _wait_for(), -ETIMEDOUT is returned to a variable ret. */ ret = _wait_for(COND, timeout_base_ms * 1000, 10, 10); if (!ret) goto out; /* * The above can time out if the number of requests was low (2 in the * worst case) _and_ PCODE was busy for some reason even after a * (queued) request and @timeout_base_ms delay. As a workaround retry * the poll with preemption disabled to maximize the number of * requests. Increase the timeout from @timeout_base_ms to 50ms to * account for interrupts that could reduce the number of these * requests, and for any quirks of the PCODE firmware that delays * the request completion. */ drm_dbg_kms(&uncore->i915->drm, "PCODE timeout, retrying with preemption disabled\n"); drm_WARN_ON_ONCE(&uncore->i915->drm, timeout_base_ms > 3); preempt_disable(); ret = wait_for_atomic(COND, 50); preempt_enable();out:
mutex_unlock(&uncore->i915->sb_lock); return status ? status : ret;#undef COND
}---------------------------------------------------------
If you try
skl_pcode_try_request()for 180 seconds inskl_pcode_request(), and the first "intel_uncore_read_fw(uncore, GEN6_PCODE_MAILBOX)&GEN6_PCODE_READY)" call in__snb_pcode_rw()thatskl_pcode_try_request()invokes always fails. if then it does not make sense to me why this patch fixes it by just waiting 10 seconds.This is because if it was called with the flow described above, 180 seconds is longer than 10 seconds, so the scenario you mentioned is also covered in the existing code.To describe in more detail the second scenario I previously commented on:
skl_pcode_request()triesskl_pcode_try_request()for 180 seconds
1) Inskl_pcode_try_request(), theGEN6_PCODE_MAILBOXregister bit is not
BUSY, so the value is written to theGEN6_PCODE_DATA1register.
2)skl_pcode_try_request()returns-ETIMEDOUTvalue because the operation
of 1) does not complete within 500us.
3) Scenario in which theGEN6_PCODE_MAILBOXregister bit is checked as BUSY
and returns-EAGAINin the last call ofskl_pcode_try_request()invoked byskl_pcode_request()If the reason why this problem occurred is because of this scenario, shouldn't there be an attempt to increase
fast_timeout_usused as an argument of__snb_pcode_rw()to 500us or more whenskl_pcode_try_request()returns-ETIMEDOUT?Br,
G.G.
Hi Andi,
You gave a lot of explanations, and confirmed that this patch solves the problem, but the root cause of this problem still seems to be unclear.
In the logs where this problem was reported, the logs were output in the following order.
[log]
<7> [268.276586] i915 0000:4d:00.0: [drm:skl_pcode_request [i915]] PCODE timeout, retrying with preemption disabled
...
<4> [268.276942] WARNING: CPU: 1 PID: 1678 at drivers/gpu/drm/i915/intel_pcode.c:196 skl_pcode_request+0x242/0x260 [i915]
...
<3> [268.329058] i915 0000:4d:00.0: [drm] *ERROR* gt0: intel_pcode_init failed -11
In other words, -EAGAIN was not returned when the first skl_pcode_try_request() was executed in the skl_pcode_request() function, but the logs confirmed that the _wait_for(COND ...) of the skl_pcode_request() function was executed.
And, I reviewed bpsec in more detail to see if I missed anything related to the status and restrictions of the initial GTDRIVER_MAILBOX_INTERFACE (PCODE).
I've reviewed bpsec in detail for the two below where GTDRIVER_MAILBOX_INTERFACE is mentioned, but I haven't identified any additional constraints other than those described below.
Pcode Driver Mailbox Spec: 34151
Driver Initialization Programming Sequence: 33149
RUN_BUSY: SW may write to the two mailbox registers only when RUN_BUSY is cleared (0b)
My suggestion is that the HW should explain and analyze the root cause of the situation where it takes more than 180s for the Busy bit of GTDRIVER_MAILBOX_INTERFACE to change to 0.
And the method of the proposed patch, the newly added line "__intel_wait_for_register_fw(uncore, GEN6_PCODE_MAILBOX, GEN6_PCODE_READY, 0, 500, 10000, NULL))" succeeded, but the existing method of trying 180s with checking intel_uncore_read_fw(uncore, GEN6_PCODE_MAILBOX)&GEN6_PCODE_READY) failed. Therefore please review it again. (Checking the code, it appears that both call usleep_range(wait__, wait__ * 2); in __wait_for().)
Br,
G.G.
int skl_pcode_request(struct intel_uncore *uncore, u32 mbox, u32 request,
u32 reply_mask, u32 reply, int timeout_base_ms)
{
u32 status;
int ret;
mutex_lock(&uncore->i915->sb_lock);#define COND \
skl_pcode_try_request(uncore, mbox, request, reply_mask, reply, &status)
/*
* Prime the PCODE by doing a request first. Normally it guarantees
* that a subsequent request, at most @timeout_base_ms later, succeeds.
* _wait_for() doesn't guarantee when its passed condition is evaluated
* first, so send the first request explicitly.
*/
if (COND) {
ret = 0;
goto out;
}
ret = _wait_for(COND, timeout_base_ms * 1000, 10, 10);
if (!ret)
goto out;
/*
* The above can time out if the number of requests was low (2 in the
* worst case) _and_ PCODE was busy for some reason even after a
* (queued) request and @timeout_base_ms delay. As a workaround retry
* the poll with preemption disabled to maximize the number of
* requests. Increase the timeout from @timeout_base_ms to 50ms to
* account for interrupts that could reduce the number of these
* requests, and for any quirks of the PCODE firmware that delays
* the request completion.
*/
drm_dbg_kms(&uncore->i915->drm,
"PCODE timeout, retrying with preemption disabled\n");
drm_WARN_ON_ONCE(&uncore->i915->drm, timeout_base_ms > 3);
preempt_disable();
ret = wait_for_atomic(COND, 50);
preempt_enable();out:
mutex_unlock(&uncore->i915->sb_lock); return status ? status : ret;
#undef COND
}
Quoted 311 lines · Andi Shyti
Hi Rodrigo,
first of all, thanks for looking into this!
Quoted 108 lines
Quoted 77 lines
Quoted 62 lines
Quoted 42 lines
Quoted 42 lines
Quoted 19 lines
In the call flow invoked by
intel_pcode_init(), I've added brief comments where further clarification is needed in this scenario, and a description of the suspicious scenario at the bottom.-------------------------------------------------------------------------
intel_pcode_init()
|
+-> skl_pcode_request(uncore,DG1_PCODE_STATUS,DG1_UNCORE_GET_INIT_STATUS, DG1_UNCORE_INIT_STATUS_COMPLETE, DG1_UNCORE_INIT_STATUS_COMPLETE, 180000); | +-> skl_pcode_try_request() | +-> *status = __snb_pcode_rw(uncore, mbox, &request, NULL, 500, 0, true);-------------------------------------------------------------------------
static int __snb_pcode_rw(structintel_uncore*uncore, u32 mbox,Quoted 10 lines
u32 *val, u32 *val1, int fast_timeout_us, int slow_timeout_ms, bool is_read) { ... /* Before writing a value to the GEN6_PCODE_DATA register, check if the bit in the GEN6_PCODE_MAILBOX register indicates BUSY. */ if (intel_uncore_read_fw(uncore, GEN6_PCODE_MAILBOX) & GEN6_PCODE_READY) return -EAGAIN;what if we fail here because the punit is still initializing and will be ready, say, in 10 seconds?
GG, without going any further, we fail here! The
-EAGAINwe receive from the test comes from this point. We don't fail with-ETIMEDOUT, but with-EAGAINand the reason is because the punit is not ready to perform the very fist communication and we fail the probing.It doesn't mean, though, that there is anything wrong, we just need to wait a bit before "taking drastic decisions"!
Quoted 6 lines
Quoted 2 lines
This patch is suggesting to wait up to 10s for the punit to be ready and eventually try to probe again... and, indeed, it works!
As GG, what I still don't understand is how this extra 10 seconds wait helps... have you tried to simple add the 10 to the 180 and make the code 190 sec instead?
maybe I haven't been able to explain the issue properly.
I can even set that timer to 2hrs and a half and nothing changes because we fail before.
Here it's not a matter of how much do I wait but when do I check the pcode readiness (i.e. signalled by the
GEN6_PCODE_READYbit in theGEN6_PCODE_MAILBOXregister).During a normal run we are always sure that communicating with the punit works, because we made it sure during the previous transaction.
Quoted 8 lines
During probe there is no previous transaction and we start communicating with the punit without making sure that it is ready. And indeed some times it is not, so that we suppress the probing on purpose instead of giving it another chance.
I admit that the commit message is not written properly and rather misleading, but here it's not at all a matter of how much do I wait.
The commit message was initially confused because it looks like we are just adding a wait, without doing anything....
But looking to the code we can see that it will wait until pcode is ready with a timeout of 10 seconds.
Quoted 7 lines
But if pcode is ready in 10 seconds, why pcode is not ready in 190 seconds. We are doing absolutely nothing more that could make pcode ready in 10 seconds that won't be in 190.
This is what we are missing here... The code as is doesn't make a lot of sense to us and it looks like it is solving the issue by the 10 extra seconds and not by some special status checking.
Okay, after an offline talk I am convinced now that we need some check like this in some place.
But the commit message needs be be fully re-written.
It needs to be clear that underneath, the pcode communication functions will do a check for ready without any wait, what will make desired timeout to never really wait for the pcode done and prematurely return.
at
__snb_pcode_rw():if (
intel_uncore_read_fw(uncore, GEN6_PCODE_MAILBOX)&GEN6_PCODE_READY)return -EAGAIN;So, for this reason we need to ensure that pcode is really ready before we wait.
Other options are to handle the EAGAIN return and then wait. Or even change the
__snb_pcode_rwto ensure that it is ready...Something like:
Review
Quoted 8 lines
(quoted diff) +6
-2- if (intel_uncore_read_fw(uncore, GEN6_PCODE_MAILBOX) & GEN6_PCODE_READY) - return -EAGAIN; + if (__intel_wait_for_register_fw(uncore, GEN6_PCODE_MAILBOX, + GEN6_PCODE_READY, GEN6_PCODE_READY, + fast_timeout_us, + slow_timeout_ms, + NULL)) + return -ETIMEDOUT;This works, but the difference is that you are putting it on any call to the
__snb_pcode_rw(), even when not necessary.Putting it in the schema we used in our offline chat, the original implementation was:
+-----------------------+ | r/w -> check if ready |-> r/w -> cir -> r/w -> cir -> r/w -> ... +-----------------------+probe time
(where cir stands for "check if ready"). Because in the first r/w we didn't check the status of the punit we added:
+-------------------+ | cir -> r/w -> cir |-> r/w -> cir -> r/w -> cir -> r/w -> ... +-------------------+probe time
So that we are sure that the first r/w works. What you are suggesting is:
+-------------------+ | cir -> r/w -> cir |-> cir -> r/w -> cir -> cir -> r/w -> cir ->--------------------+
probe timeAs you can se we have two "check if ready" in a raw, which might be a bit of an overkill, it doesn't hurt much but it would look like:
if (__intel_wait_for_register_fw()) return -EAGAIN; intel_uncore_write_fw...... if (__intel_wait_for_register_fw()) return -TIMEDOUT;and this for every single snb_pcode_r/w.
Besides some functions might need the first wait, some might not. Check, for example
icl_tc_cold_exit()where the-EAGAINis handled. With your version the retries would be quite ponderous.I'm sorry, but I can't take your suggestion as it can have major consequences along i915, it requires more changes and and it needs to be carefully tested.
On top of that I like Ashutosh refactoring that is quite an elegant way to gain more flexibility at boot time without any further impact.
Quoted 8 lines
In the best regular case it will not change anything since the wait for will return 0 immediatelly... in the worst case we would double the timeout value, but this would be only in this insane case of 180 seconds anyway and if it goes really really nasty...
But anyway, now the motivation and the source of the issue is pretty clear, I can add my rv-b there, but I really want a better commit msg at least...
I definitely need to make a better commit message :)
Thanks a lot,
AndiQuoted 123 lines
Quoted 123 lines
Quoted 123 lines
Thanks, Rodrigo!
AndiQuoted 118 lines
Quoted 118 lines
Andi
Quoted 115 lines
/* write value to
GEN6_PCODE_DATAregister */
intel_uncore_write_fw(uncore, GEN6_PCODE_DATA, *val);
intel_uncore_write_fw(uncore, GEN6_PCODE_DATA1, val1 ? *val1 : 0);/* In this scenario, the value
"DG1_PCODE_STATUS|GEN6_PCODE_READY"
is written to theGEN6_PCODE_MAILBOXregister,
so that the Busy status of theGEN6_PCODE_MAILBOXregister can be checked later.
(When the value of theGEN6_PCODE_READYbit of the
GEN6_PCODE_MAILBOXregister changes to 0, the operation can be considered completed.) */
intel_uncore_write_fw(uncore,
GEN6_PCODE_MAILBOX,GEN6_PCODE_READY| mbox);/* In this scenario, verify that the BUSY status bit in the
GEN6_PCODE_MAILBOXregister turns off for up to 500us. */
if (__intel_wait_for_register_fw(uncore,
GEN6_PCODE_MAILBOX,GEN6_PCODE_READY, 0,fast_timeout_us,slow_timeout_ms, &mbox))
return-ETIMEDOUT;
/* If there is a failure here, it may be considered that the
"DG1_PCODE_STATUS|GEN6_PCODE_READY" operation was not completed within 500us */
...
}int skl_pcode_request(struct
intel_uncore*uncore, u32 mbox, u32 request,
u32reply_mask, u32 reply, inttimeout_base_ms){ u32 status; int ret; mutex_lock(&uncore->i915->sb_lock);#define COND \ skl_pcode_try_request(uncore, mbox, request, reply_mask, reply, &status) /* the first trial for skl_pcode_try_request() can return -EAGAIN or -ETIMEDOUT. And the code did not check the error code here, so we don't know how far the __snb_pcode_rw() function went. It is not known whether the pcode_mailbox status was busy before writing the value to the GEN6_PCODE_DATA register or after.*/ if (COND) { ret = 0; goto out; } /* In this scenario, skl_pcode_try_request() is invoked every 10us for 180 seconds. When skl_pcode_try_request() returns -EAGAIN and -ETIMEDOUT by _wait_for(), -ETIMEDOUT is returned to a variable ret. */ ret = _wait_for(COND, timeout_base_ms * 1000, 10, 10); if (!ret) goto out; /* * The above can time out if the number of requests was low (2 in the * worst case) _and_ PCODE was busy for some reason even after a * (queued) request and @timeout_base_ms delay. As a workaround retry * the poll with preemption disabled to maximize the number of * requests. Increase the timeout from @timeout_base_ms to 50ms to * account for interrupts that could reduce the number of these * requests, and for any quirks of the PCODE firmware that delays * the request completion. */ drm_dbg_kms(&uncore->i915->drm, "PCODE timeout, retrying with preemption disabled\n"); drm_WARN_ON_ONCE(&uncore->i915->drm, timeout_base_ms > 3); preempt_disable(); ret = wait_for_atomic(COND, 50); preempt_enable();out:
mutex_unlock(&uncore->i915->sb_lock); return status ? status : ret;#undef COND
}---------------------------------------------------------
If you try
skl_pcode_try_request()for 180 seconds inskl_pcode_request(), and the first "intel_uncore_read_fw(uncore, GEN6_PCODE_MAILBOX)&GEN6_PCODE_READY)" call in__snb_pcode_rw()thatskl_pcode_try_request()invokes always fails. if then it does not make sense to me why this patch fixes it by just waiting 10 seconds.This is because if it was called with the flow described above, 180 seconds is longer than 10 seconds, so the scenario you mentioned is also covered in the existing code.To describe in more detail the second scenario I previously commented on:
skl_pcode_request()triesskl_pcode_try_request()for 180 seconds
1) Inskl_pcode_try_request(), theGEN6_PCODE_MAILBOXregister bit is not
BUSY, so the value is written to theGEN6_PCODE_DATA1register.
2)skl_pcode_try_request()returns-ETIMEDOUTvalue because the operation
of 1) does not complete within 500us.
3) Scenario in which theGEN6_PCODE_MAILBOXregister bit is checked as BUSY
and returns-EAGAINin the last call ofskl_pcode_try_request()invoked byskl_pcode_request()If the reason why this problem occurred is because of this scenario, shouldn't there be an attempt to increase
fast_timeout_usused as an argument of__snb_pcode_rw()to 500us or more whenskl_pcode_try_request()returns-ETIMEDOUT?Br,
G.G.
Hi GG,
Quoted 22 lines · Gwan-gyeong Mun
Hi Andi,
You gave a lot of explanations, and confirmed that this patch solves the problem, but the root cause of this problem still seems to be unclear.
In the logs where this problem was reported, the logs were output in the following order.
[log]
<7> [268.276586] i915 0000:4d:00.0: [drm:skl_pcode_request [i915]] PCODE timeout, retrying with preemption disabled
...
<4> [268.276942] WARNING: CPU: 1 PID: 1678 atdrivers/gpu/drm/i915/intel_pcode.c:196skl_pcode_request+0x242/0x260[i915]
...
<3> [268.329058] i915 0000:4d:00.0: [drm] *ERROR* gt0:intel_pcode_initfailed -11
Quoted 4 lines
In other words,
-EAGAINwas not returned when the firstskl_pcode_try_request()was executed in theskl_pcode_request()function, but the logs confirmed that the_wait_for(COND ...)of theskl_pcode_request()function was executed.
I'm not following... how can you say that -EAGAIN was not returned, it's written explecetly that pcode_init failed with -11, i.e. -EAGAIN.
Besides if we get as far as to receive "PCODE timeout, retrying with preemption disabled" isn't it because we failed twice? And apparently with -EAGAIN?
How can you say from the logs above that this
if (COND) {
ret = 0;
goto out;
}has not failed with -EAGAIN? The logs are not telling us how the calls to __skl_pcode_try_request() failed. I am telling you after I tested it.
I hope I understood what you are saying. Otherwise, let's have an offline chat.
Andi
Quoted 397 lines
And, I reviewed bpsec in more detail to see if I missed anything related to the status and restrictions of the initial
GTDRIVER_MAILBOX_INTERFACE(PCODE).I've reviewed bpsec in detail for the two below where
GTDRIVER_MAILBOX_INTERFACEis mentioned, but I haven't identified any additional constraints other than those described below.Pcode Driver Mailbox Spec: 34151
Driver Initialization Programming Sequence: 33149
RUN_BUSY: SW may write to the two mailbox registers only whenRUN_BUSYis cleared (0b)My suggestion is that the HW should explain and analyze the root cause of the situation where it takes more than 180s for the Busy bit of
GTDRIVER_MAILBOX_INTERFACEto change to 0.And the method of the proposed patch, the newly added line "
__intel_wait_for_register_fw(uncore, GEN6_PCODE_MAILBOX, GEN6_PCODE_READY, 0, 500, 10000, NULL))" succeeded, but the existing method of trying 180s with checkingintel_uncore_read_fw(uncore, GEN6_PCODE_MAILBOX)&GEN6_PCODE_READY) failed. Therefore please review it again.
(Checking the code, it appears that both callusleep_range(wait__, wait__ * 2); in__wait_for().)Br,
G.G.
int skl_pcode_request(struct
intel_uncore*uncore, u32 mbox, u32 request,
u32reply_mask, u32 reply, inttimeout_base_ms){ u32 status; int ret; mutex_lock(&uncore->i915->sb_lock);#define COND \ skl_pcode_try_request(uncore, mbox, request, reply_mask, reply, &status) /* * Prime the PCODE by doing a request first. Normally it guarantees * that a subsequent request, at most @timeout_base_ms later, succeeds. * _wait_for() doesn't guarantee when its passed condition is evaluated * first, so send the first request explicitly. */ if (COND) { ret = 0; goto out; } ret = _wait_for(COND, timeout_base_ms * 1000, 10, 10); if (!ret) goto out; /* * The above can time out if the number of requests was low (2 in the * worst case) _and_ PCODE was busy for some reason even after a * (queued) request and @timeout_base_ms delay. As a workaround retry * the poll with preemption disabled to maximize the number of * requests. Increase the timeout from @timeout_base_ms to 50ms to * account for interrupts that could reduce the number of these * requests, and for any quirks of the PCODE firmware that delays * the request completion. */ drm_dbg_kms(&uncore->i915->drm, "PCODE timeout, retrying with preemption disabled\n"); drm_WARN_ON_ONCE(&uncore->i915->drm, timeout_base_ms > 3); preempt_disable(); ret = wait_for_atomic(COND, 50); preempt_enable();out:
mutex_unlock(&uncore->i915->sb_lock); return status ? status : ret;#undef COND
}Quoted 311 lines · Andi Shyti
Hi Rodrigo,
first of all, thanks for looking into this!
Quoted 108 lines
Quoted 77 lines
Quoted 62 lines
Quoted 42 lines
Quoted 42 lines
Quoted 19 lines
In the call flow invoked by
intel_pcode_init(), I've added brief comments where further clarification is needed in this scenario, and a description of the suspicious scenario at the bottom.-------------------------------------------------------------------------
intel_pcode_init()
|
+-> skl_pcode_request(uncore,DG1_PCODE_STATUS,DG1_UNCORE_GET_INIT_STATUS, DG1_UNCORE_INIT_STATUS_COMPLETE, DG1_UNCORE_INIT_STATUS_COMPLETE, 180000); | +-> skl_pcode_try_request() | +-> *status = __snb_pcode_rw(uncore, mbox, &request, NULL, 500, 0, true);-------------------------------------------------------------------------
static int __snb_pcode_rw(structintel_uncore*uncore, u32 mbox,Quoted 10 lines
u32 *val, u32 *val1, int fast_timeout_us, int slow_timeout_ms, bool is_read) { ... /* Before writing a value to the GEN6_PCODE_DATA register, check if the bit in the GEN6_PCODE_MAILBOX register indicates BUSY. */ if (intel_uncore_read_fw(uncore, GEN6_PCODE_MAILBOX) & GEN6_PCODE_READY) return -EAGAIN;what if we fail here because the punit is still initializing and will be ready, say, in 10 seconds?
GG, without going any further, we fail here! The
-EAGAINwe receive from the test comes from this point. We don't fail with-ETIMEDOUT, but with-EAGAINand the reason is because the punit is not ready to perform the very fist communication and we fail the probing.It doesn't mean, though, that there is anything wrong, we just need to wait a bit before "taking drastic decisions"!
Quoted 6 lines
Quoted 2 lines
This patch is suggesting to wait up to 10s for the punit to be ready and eventually try to probe again... and, indeed, it works!
As GG, what I still don't understand is how this extra 10 seconds wait helps... have you tried to simple add the 10 to the 180 and make the code 190 sec instead?
maybe I haven't been able to explain the issue properly.
I can even set that timer to 2hrs and a half and nothing changes because we fail before.
Here it's not a matter of how much do I wait but when do I check the pcode readiness (i.e. signalled by the
GEN6_PCODE_READYbit in theGEN6_PCODE_MAILBOXregister).During a normal run we are always sure that communicating with the punit works, because we made it sure during the previous transaction.
Quoted 8 lines
During probe there is no previous transaction and we start communicating with the punit without making sure that it is ready. And indeed some times it is not, so that we suppress the probing on purpose instead of giving it another chance.
I admit that the commit message is not written properly and rather misleading, but here it's not at all a matter of how much do I wait.
The commit message was initially confused because it looks like we are just adding a wait, without doing anything....
But looking to the code we can see that it will wait until pcode is ready with a timeout of 10 seconds.
Quoted 7 lines
But if pcode is ready in 10 seconds, why pcode is not ready in 190 seconds. We are doing absolutely nothing more that could make pcode ready in 10 seconds that won't be in 190.
This is what we are missing here... The code as is doesn't make a lot of sense to us and it looks like it is solving the issue by the 10 extra seconds and not by some special status checking.
Okay, after an offline talk I am convinced now that we need some check like this in some place.
But the commit message needs be be fully re-written.
It needs to be clear that underneath, the pcode communication functions will do a check for ready without any wait, what will make desired timeout to never really wait for the pcode done and prematurely return.
at
__snb_pcode_rw():if (
intel_uncore_read_fw(uncore, GEN6_PCODE_MAILBOX)&GEN6_PCODE_READY)return -EAGAIN;So, for this reason we need to ensure that pcode is really ready before we wait.
Other options are to handle the EAGAIN return and then wait. Or even change the
__snb_pcode_rwto ensure that it is ready...Something like:
Review
Quoted 8 lines
(quoted diff) +6
-2- if (intel_uncore_read_fw(uncore, GEN6_PCODE_MAILBOX) & GEN6_PCODE_READY) - return -EAGAIN; + if (__intel_wait_for_register_fw(uncore, GEN6_PCODE_MAILBOX, + GEN6_PCODE_READY, GEN6_PCODE_READY, + fast_timeout_us, + slow_timeout_ms, + NULL)) + return -ETIMEDOUT;This works, but the difference is that you are putting it on any call to the
__snb_pcode_rw(), even when not necessary.Putting it in the schema we used in our offline chat, the original implementation was:
+-----------------------+ | r/w -> check if ready |-> r/w -> cir -> r/w -> cir -> r/w -> ... +-----------------------+probe time
(where cir stands for "check if ready"). Because in the first r/w we didn't check the status of the punit we added:
+-------------------+ | cir -> r/w -> cir |-> r/w -> cir -> r/w -> cir -> r/w -> ... +-------------------+probe time
So that we are sure that the first r/w works. What you are suggesting is:
+-------------------+ | cir -> r/w -> cir |-> cir -> r/w -> cir -> cir -> r/w -> cir ->--------------------+
probe timeAs you can se we have two "check if ready" in a raw, which might be a bit of an overkill, it doesn't hurt much but it would look like:
if (__intel_wait_for_register_fw()) return -EAGAIN; intel_uncore_write_fw...... if (__intel_wait_for_register_fw()) return -TIMEDOUT;and this for every single snb_pcode_r/w.
Besides some functions might need the first wait, some might not. Check, for example
icl_tc_cold_exit()where the-EAGAINis handled. With your version the retries would be quite ponderous.I'm sorry, but I can't take your suggestion as it can have major consequences along i915, it requires more changes and and it needs to be carefully tested.
On top of that I like Ashutosh refactoring that is quite an elegant way to gain more flexibility at boot time without any further impact.
Quoted 8 lines
In the best regular case it will not change anything since the wait for will return 0 immediatelly... in the worst case we would double the timeout value, but this would be only in this insane case of 180 seconds anyway and if it goes really really nasty...
But anyway, now the motivation and the source of the issue is pretty clear, I can add my rv-b there, but I really want a better commit msg at least...
I definitely need to make a better commit message :)
Thanks a lot,
AndiQuoted 123 lines
Quoted 123 lines
Quoted 123 lines
Thanks, Rodrigo!
AndiQuoted 118 lines
Quoted 118 lines
Andi
Quoted 115 lines
/* write value to
GEN6_PCODE_DATAregister */
intel_uncore_write_fw(uncore, GEN6_PCODE_DATA, *val);
intel_uncore_write_fw(uncore, GEN6_PCODE_DATA1, val1 ? *val1 : 0);/* In this scenario, the value
"DG1_PCODE_STATUS|GEN6_PCODE_READY"
is written to theGEN6_PCODE_MAILBOXregister,
so that the Busy status of theGEN6_PCODE_MAILBOXregister can be checked later.
(When the value of theGEN6_PCODE_READYbit of the
GEN6_PCODE_MAILBOXregister changes to 0, the operation can be considered completed.) */
intel_uncore_write_fw(uncore,
GEN6_PCODE_MAILBOX,GEN6_PCODE_READY| mbox);/* In this scenario, verify that the BUSY status bit in the
GEN6_PCODE_MAILBOXregister turns off for up to 500us. */
if (__intel_wait_for_register_fw(uncore,
GEN6_PCODE_MAILBOX,GEN6_PCODE_READY, 0,fast_timeout_us,slow_timeout_ms, &mbox))
return-ETIMEDOUT;
/* If there is a failure here, it may be considered that the
"DG1_PCODE_STATUS|GEN6_PCODE_READY" operation was not completed within 500us */
...
}int skl_pcode_request(struct
intel_uncore*uncore, u32 mbox, u32 request,
u32reply_mask, u32 reply, inttimeout_base_ms){ u32 status; int ret; mutex_lock(&uncore->i915->sb_lock);#define COND \ skl_pcode_try_request(uncore, mbox, request, reply_mask, reply, &status) /* the first trial for skl_pcode_try_request() can return -EAGAIN or -ETIMEDOUT. And the code did not check the error code here, so we don't know how far the __snb_pcode_rw() function went. It is not known whether the pcode_mailbox status was busy before writing the value to the GEN6_PCODE_DATA register or after.*/ if (COND) { ret = 0; goto out; } /* In this scenario, skl_pcode_try_request() is invoked every 10us for 180 seconds. When skl_pcode_try_request() returns -EAGAIN and -ETIMEDOUT by _wait_for(), -ETIMEDOUT is returned to a variable ret. */ ret = _wait_for(COND, timeout_base_ms * 1000, 10, 10); if (!ret) goto out; /* * The above can time out if the number of requests was low (2 in the * worst case) _and_ PCODE was busy for some reason even after a * (queued) request and @timeout_base_ms delay. As a workaround retry * the poll with preemption disabled to maximize the number of * requests. Increase the timeout from @timeout_base_ms to 50ms to * account for interrupts that could reduce the number of these * requests, and for any quirks of the PCODE firmware that delays * the request completion. */ drm_dbg_kms(&uncore->i915->drm, "PCODE timeout, retrying with preemption disabled\n"); drm_WARN_ON_ONCE(&uncore->i915->drm, timeout_base_ms > 3); preempt_disable(); ret = wait_for_atomic(COND, 50); preempt_enable();out:
mutex_unlock(&uncore->i915->sb_lock); return status ? status : ret;#undef COND
}---------------------------------------------------------
If you try
skl_pcode_try_request()for 180 seconds inskl_pcode_request(), and the first "intel_uncore_read_fw(uncore, GEN6_PCODE_MAILBOX)&GEN6_PCODE_READY)" call in__snb_pcode_rw()thatskl_pcode_try_request()invokes always fails. if then it does not make sense to me why this patch fixes it by just waiting 10 seconds.This is because if it was called with the flow described above, 180 seconds is longer than 10 seconds, so the scenario you mentioned is also covered in the existing code.To describe in more detail the second scenario I previously commented on:
skl_pcode_request()triesskl_pcode_try_request()for 180 seconds
1) Inskl_pcode_try_request(), theGEN6_PCODE_MAILBOXregister bit is not
BUSY, so the value is written to theGEN6_PCODE_DATA1register.
2)skl_pcode_try_request()returns-ETIMEDOUTvalue because the operation
of 1) does not complete within 500us.
3) Scenario in which theGEN6_PCODE_MAILBOXregister bit is checked as BUSY
and returns-EAGAINin the last call ofskl_pcode_try_request()invoked byskl_pcode_request()If the reason why this problem occurred is because of this scenario, shouldn't there be an attempt to increase
fast_timeout_usused as an argument of__snb_pcode_rw()to 500us or more whenskl_pcode_try_request()returns-ETIMEDOUT?Br,
G.G.
Quoted 38 lines · Andi Shyti
Hi GG,
Quoted 22 lines · Gwan-gyeong Mun
Hi Andi,
You gave a lot of explanations, and confirmed that this patch solves the problem, but the root cause of this problem still seems to be unclear.
In the logs where this problem was reported, the logs were output in the following order.
[log]
<7> [268.276586] i915 0000:4d:00.0: [drm:skl_pcode_request [i915]] PCODE timeout, retrying with preemption disabled
...
<4> [268.276942] WARNING: CPU: 1 PID: 1678 atdrivers/gpu/drm/i915/intel_pcode.c:196skl_pcode_request+0x242/0x260[i915]
...
<3> [268.329058] i915 0000:4d:00.0: [drm] *ERROR* gt0:intel_pcode_initfailed -11Quoted 4 lines
In other words,
-EAGAINwas not returned when the firstskl_pcode_try_request()was executed in theskl_pcode_request()function, but the logs confirmed that the_wait_for(COND ...)of theskl_pcode_request()function was executed.I'm not following... how can you say that
-EAGAINwas not returned, it's written explecetly thatpcode_initfailed with -11, i.e.-EAGAIN.Besides if we get as far as to receive "PCODE timeout, retrying with preemption disabled" isn't it because we failed twice? And apparently with
-EAGAIN?
Quoted 6 lines
How can you say from the logs above that this
if (COND) { ret = 0; goto out; }
Hi Andi,
Because, the log outputs "<7> [268.276586] i915 0000:4d:00.0: [drm:skl_pcode_request [i915]] PCODE timeout, retrying with preemption disabled"
Br,
G.G.
Quoted 407 lines
has not failed with
-EAGAIN? The logs are not telling us how the calls to__skl_pcode_try_request()failed. I am telling you after I tested it.I hope I understood what you are saying. Otherwise, let's have an offline chat.
Andi
Quoted 397 lines
And, I reviewed bpsec in more detail to see if I missed anything related to the status and restrictions of the initial
GTDRIVER_MAILBOX_INTERFACE(PCODE).I've reviewed bpsec in detail for the two below where
GTDRIVER_MAILBOX_INTERFACEis mentioned, but I haven't identified any additional constraints other than those described below.Pcode Driver Mailbox Spec: 34151
Driver Initialization Programming Sequence: 33149
RUN_BUSY: SW may write to the two mailbox registers only whenRUN_BUSYis cleared (0b)My suggestion is that the HW should explain and analyze the root cause of the situation where it takes more than 180s for the Busy bit of
GTDRIVER_MAILBOX_INTERFACEto change to 0.And the method of the proposed patch, the newly added line "
__intel_wait_for_register_fw(uncore, GEN6_PCODE_MAILBOX, GEN6_PCODE_READY, 0, 500, 10000, NULL))" succeeded, but the existing method of trying 180s with checkingintel_uncore_read_fw(uncore, GEN6_PCODE_MAILBOX)&GEN6_PCODE_READY) failed. Therefore please review it again.
(Checking the code, it appears that both callusleep_range(wait__, wait__ * 2); in__wait_for().)Br,
G.G.
int skl_pcode_request(struct
intel_uncore*uncore, u32 mbox, u32 request,
u32reply_mask, u32 reply, inttimeout_base_ms){ u32 status; int ret; mutex_lock(&uncore->i915->sb_lock);#define COND \ skl_pcode_try_request(uncore, mbox, request, reply_mask, reply, &status) /* * Prime the PCODE by doing a request first. Normally it guarantees * that a subsequent request, at most @timeout_base_ms later, succeeds. * _wait_for() doesn't guarantee when its passed condition is evaluated * first, so send the first request explicitly. */ if (COND) { ret = 0; goto out; } ret = _wait_for(COND, timeout_base_ms * 1000, 10, 10); if (!ret) goto out; /* * The above can time out if the number of requests was low (2 in the * worst case) _and_ PCODE was busy for some reason even after a * (queued) request and @timeout_base_ms delay. As a workaround retry * the poll with preemption disabled to maximize the number of * requests. Increase the timeout from @timeout_base_ms to 50ms to * account for interrupts that could reduce the number of these * requests, and for any quirks of the PCODE firmware that delays * the request completion. */ drm_dbg_kms(&uncore->i915->drm, "PCODE timeout, retrying with preemption disabled\n"); drm_WARN_ON_ONCE(&uncore->i915->drm, timeout_base_ms > 3); preempt_disable(); ret = wait_for_atomic(COND, 50); preempt_enable();out:
mutex_unlock(&uncore->i915->sb_lock); return status ? status : ret;#undef COND
}Quoted 311 lines · Andi Shyti
Hi Rodrigo,
first of all, thanks for looking into this!
Quoted 108 lines
Quoted 77 lines
Quoted 62 lines
Quoted 42 lines
Quoted 42 lines
Quoted 19 lines
In the call flow invoked by
intel_pcode_init(), I've added brief comments where further clarification is needed in this scenario, and a description of the suspicious scenario at the bottom.-------------------------------------------------------------------------
intel_pcode_init()
|
+-> skl_pcode_request(uncore,DG1_PCODE_STATUS,DG1_UNCORE_GET_INIT_STATUS, DG1_UNCORE_INIT_STATUS_COMPLETE, DG1_UNCORE_INIT_STATUS_COMPLETE, 180000); | +-> skl_pcode_try_request() | +-> *status = __snb_pcode_rw(uncore, mbox, &request, NULL, 500, 0, true);-------------------------------------------------------------------------
static int __snb_pcode_rw(structintel_uncore*uncore, u32 mbox,Quoted 10 lines
u32 *val, u32 *val1, int fast_timeout_us, int slow_timeout_ms, bool is_read) { ... /* Before writing a value to the GEN6_PCODE_DATA register, check if the bit in the GEN6_PCODE_MAILBOX register indicates BUSY. */ if (intel_uncore_read_fw(uncore, GEN6_PCODE_MAILBOX) & GEN6_PCODE_READY) return -EAGAIN;what if we fail here because the punit is still initializing and will be ready, say, in 10 seconds?
GG, without going any further, we fail here! The
-EAGAINwe receive from the test comes from this point. We don't fail with-ETIMEDOUT, but with-EAGAINand the reason is because the punit is not ready to perform the very fist communication and we fail the probing.It doesn't mean, though, that there is anything wrong, we just need to wait a bit before "taking drastic decisions"!
Quoted 6 lines
Quoted 2 lines
This patch is suggesting to wait up to 10s for the punit to be ready and eventually try to probe again... and, indeed, it works!
As GG, what I still don't understand is how this extra 10 seconds wait helps... have you tried to simple add the 10 to the 180 and make the code 190 sec instead?
maybe I haven't been able to explain the issue properly.
I can even set that timer to 2hrs and a half and nothing changes because we fail before.
Here it's not a matter of how much do I wait but when do I check the pcode readiness (i.e. signalled by the
GEN6_PCODE_READYbit in theGEN6_PCODE_MAILBOXregister).During a normal run we are always sure that communicating with the punit works, because we made it sure during the previous transaction.
Quoted 8 lines
During probe there is no previous transaction and we start communicating with the punit without making sure that it is ready. And indeed some times it is not, so that we suppress the probing on purpose instead of giving it another chance.
I admit that the commit message is not written properly and rather misleading, but here it's not at all a matter of how much do I wait.
The commit message was initially confused because it looks like we are just adding a wait, without doing anything....
But looking to the code we can see that it will wait until pcode is ready with a timeout of 10 seconds.
Quoted 7 lines
But if pcode is ready in 10 seconds, why pcode is not ready in 190 seconds. We are doing absolutely nothing more that could make pcode ready in 10 seconds that won't be in 190.
This is what we are missing here... The code as is doesn't make a lot of sense to us and it looks like it is solving the issue by the 10 extra seconds and not by some special status checking.
Okay, after an offline talk I am convinced now that we need some check like this in some place.
But the commit message needs be be fully re-written.
It needs to be clear that underneath, the pcode communication functions will do a check for ready without any wait, what will make desired timeout to never really wait for the pcode done and prematurely return.
at
__snb_pcode_rw():if (
intel_uncore_read_fw(uncore, GEN6_PCODE_MAILBOX)&GEN6_PCODE_READY)return -EAGAIN;So, for this reason we need to ensure that pcode is really ready before we wait.
Other options are to handle the EAGAIN return and then wait. Or even change the
__snb_pcode_rwto ensure that it is ready...Something like:
Review
Quoted 8 lines
(quoted diff) +6
-2- if (intel_uncore_read_fw(uncore, GEN6_PCODE_MAILBOX) & GEN6_PCODE_READY) - return -EAGAIN; + if (__intel_wait_for_register_fw(uncore, GEN6_PCODE_MAILBOX, + GEN6_PCODE_READY, GEN6_PCODE_READY, + fast_timeout_us, + slow_timeout_ms, + NULL)) + return -ETIMEDOUT;This works, but the difference is that you are putting it on any call to the
__snb_pcode_rw(), even when not necessary.Putting it in the schema we used in our offline chat, the original implementation was:
+-----------------------+ | r/w -> check if ready |-> r/w -> cir -> r/w -> cir -> r/w -> ... +-----------------------+probe time
(where cir stands for "check if ready"). Because in the first r/w we didn't check the status of the punit we added:
+-------------------+ | cir -> r/w -> cir |-> r/w -> cir -> r/w -> cir -> r/w -> ... +-------------------+probe time
So that we are sure that the first r/w works. What you are suggesting is:
+-------------------+ | cir -> r/w -> cir |-> cir -> r/w -> cir -> cir -> r/w -> cir ->--------------------+
probe timeAs you can se we have two "check if ready" in a raw, which might be a bit of an overkill, it doesn't hurt much but it would look like:
if (__intel_wait_for_register_fw()) return -EAGAIN; intel_uncore_write_fw...... if (__intel_wait_for_register_fw()) return -TIMEDOUT;and this for every single snb_pcode_r/w.
Besides some functions might need the first wait, some might not. Check, for example
icl_tc_cold_exit()where the-EAGAINis handled. With your version the retries would be quite ponderous.I'm sorry, but I can't take your suggestion as it can have major consequences along i915, it requires more changes and and it needs to be carefully tested.
On top of that I like Ashutosh refactoring that is quite an elegant way to gain more flexibility at boot time without any further impact.
Quoted 8 lines
In the best regular case it will not change anything since the wait for will return 0 immediatelly... in the worst case we would double the timeout value, but this would be only in this insane case of 180 seconds anyway and if it goes really really nasty...
But anyway, now the motivation and the source of the issue is pretty clear, I can add my rv-b there, but I really want a better commit msg at least...
I definitely need to make a better commit message :)
Thanks a lot,
AndiQuoted 123 lines
Quoted 123 lines
Quoted 123 lines
Thanks, Rodrigo!
AndiQuoted 118 lines
Quoted 118 lines
Andi
Quoted 115 lines
/* write value to
GEN6_PCODE_DATAregister */
intel_uncore_write_fw(uncore, GEN6_PCODE_DATA, *val);
intel_uncore_write_fw(uncore, GEN6_PCODE_DATA1, val1 ? *val1 : 0);/* In this scenario, the value
"DG1_PCODE_STATUS|GEN6_PCODE_READY"
is written to theGEN6_PCODE_MAILBOXregister,
so that the Busy status of theGEN6_PCODE_MAILBOXregister can be checked later.
(When the value of theGEN6_PCODE_READYbit of the
GEN6_PCODE_MAILBOXregister changes to 0, the operation can be considered completed.) */
intel_uncore_write_fw(uncore,
GEN6_PCODE_MAILBOX,GEN6_PCODE_READY| mbox);/* In this scenario, verify that the BUSY status bit in the
GEN6_PCODE_MAILBOXregister turns off for up to 500us. */
if (__intel_wait_for_register_fw(uncore,
GEN6_PCODE_MAILBOX,GEN6_PCODE_READY, 0,fast_timeout_us,slow_timeout_ms, &mbox))
return-ETIMEDOUT;
/* If there is a failure here, it may be considered that the
"DG1_PCODE_STATUS|GEN6_PCODE_READY" operation was not completed within 500us */
...
}int skl_pcode_request(struct
intel_uncore*uncore, u32 mbox, u32 request,
u32reply_mask, u32 reply, inttimeout_base_ms){ u32 status; int ret; mutex_lock(&uncore->i915->sb_lock);#define COND \ skl_pcode_try_request(uncore, mbox, request, reply_mask, reply, &status) /* the first trial for skl_pcode_try_request() can return -EAGAIN or -ETIMEDOUT. And the code did not check the error code here, so we don't know how far the __snb_pcode_rw() function went. It is not known whether the pcode_mailbox status was busy before writing the value to the GEN6_PCODE_DATA register or after.*/ if (COND) { ret = 0; goto out; } /* In this scenario, skl_pcode_try_request() is invoked every 10us for 180 seconds. When skl_pcode_try_request() returns -EAGAIN and -ETIMEDOUT by _wait_for(), -ETIMEDOUT is returned to a variable ret. */ ret = _wait_for(COND, timeout_base_ms * 1000, 10, 10); if (!ret) goto out; /* * The above can time out if the number of requests was low (2 in the * worst case) _and_ PCODE was busy for some reason even after a * (queued) request and @timeout_base_ms delay. As a workaround retry * the poll with preemption disabled to maximize the number of * requests. Increase the timeout from @timeout_base_ms to 50ms to * account for interrupts that could reduce the number of these * requests, and for any quirks of the PCODE firmware that delays * the request completion. */ drm_dbg_kms(&uncore->i915->drm, "PCODE timeout, retrying with preemption disabled\n"); drm_WARN_ON_ONCE(&uncore->i915->drm, timeout_base_ms > 3); preempt_disable(); ret = wait_for_atomic(COND, 50); preempt_enable();out:
mutex_unlock(&uncore->i915->sb_lock); return status ? status : ret;#undef COND
}---------------------------------------------------------
If you try
skl_pcode_try_request()for 180 seconds inskl_pcode_request(), and the first "intel_uncore_read_fw(uncore, GEN6_PCODE_MAILBOX)&GEN6_PCODE_READY)" call in__snb_pcode_rw()thatskl_pcode_try_request()invokes always fails. if then it does not make sense to me why this patch fixes it by just waiting 10 seconds.This is because if it was called with the flow described above, 180 seconds is longer than 10 seconds, so the scenario you mentioned is also covered in the existing code.To describe in more detail the second scenario I previously commented on:
skl_pcode_request()triesskl_pcode_try_request()for 180 seconds
1) Inskl_pcode_try_request(), theGEN6_PCODE_MAILBOXregister bit is not
BUSY, so the value is written to theGEN6_PCODE_DATA1register.
2)skl_pcode_try_request()returns-ETIMEDOUTvalue because the operation
of 1) does not complete within 500us.
3) Scenario in which theGEN6_PCODE_MAILBOXregister bit is checked as BUSY
and returns-EAGAINin the last call ofskl_pcode_try_request()invoked byskl_pcode_request()If the reason why this problem occurred is because of this scenario, shouldn't there be an attempt to increase
fast_timeout_usused as an argument of__snb_pcode_rw()to 500us or more whenskl_pcode_try_request()returns-ETIMEDOUT?Br,
G.G.
Rodrigo Vivi
maintainerQuoted 10 lines · Andi Shyti
From: Aravind Iddamsetty <[email protected]>
During module load not all the punit transaction have completed and we might end up timing out, as shown by the following warning:
i915 0000:4d:00.0: drm_WARN_ON_ONCE(timeout_base_ms > 3)Wait 10 seconds for the punit to settle and complete any outstanding transactions upon module load.
10 *SECONDS* ?!
Quoted 65 lines
Signed-off-by Aravind Iddamsetty <[email protected]>Co-developed-by Chris Wilson <[email protected]>Cc Rodrigo Vivi <[email protected]>Signed-off-by Andi Shyti <[email protected]>--- drivers/gpu/drm/i915/intel_pcode.c | 35 ++++++++++++++++++++++++++---- 1 file changed, 31 insertions(+), 4 deletions(-)drivers/gpu/drm/i915/intel_pcode.c +31
-4@@ -204,15 +204,42 @@ int skl_pcode_request(struct intel_uncore *uncore, u32 mbox, u32 request,#undef COND } +static int pcode_init_wait (struct intel_uncore *uncore,int timeout_ms) +{ +if (__intel_wait_for_register_fw (uncore, +GEN6_PCODE_MAILBOX , +GEN6_PCODE_READY ,0 , +500 , timeout_ms, +NULL )) +return -EPROBE_DEFER ; + +return skl_pcode_request (uncore, +DG1_PCODE_STATUS , +DG1_UNCORE_GET_INIT_STATUS , +DG1_UNCORE_INIT_STATUS_COMPLETE , +DG1_UNCORE_INIT_STATUS_COMPLETE , timeout_ms); +} +int intel_pcode_init (struct intel_uncore *uncore) { +int err; +if (!IS_DGFX (uncore->i915))return 0 ; -return skl_pcode_request (uncore,DG1_PCODE_STATUS , -DG1_UNCORE_GET_INIT_STATUS , -DG1_UNCORE_INIT_STATUS_COMPLETE , -DG1_UNCORE_INIT_STATUS_COMPLETE ,180000 ); +/* +* Wait 10 seconds so that the punit to settle and complete +* any outstanding transactions upon module load +*/ + err =pcode_init_wait (uncore,10000 ); + +if (err) { +drm_notice (&uncore->i915->drm, +"Waiting for HW initialisation...\n" ); + err =pcode_init_wait (uncore,180000 ); + } + +return err; }int snb_pcode_read_p (struct intel_uncore *uncore,u32 mbcmd,u32 p1,u32 p2,u32 *val)Signature · 1 line
2.34.1
Hi Rodrigo,
Quoted 7 lines · Rodrigo Vivi
Quoted 6 lines · Andi Shyti
From: Aravind Iddamsetty <[email protected]>
During module load not all the punit transaction have completed and we might end up timing out, as shown by the following warning:
Quoted 6 lines
Quoted 4 lines
i915 0000:4d:00.0: drm_WARN_ON_ONCE(timeout_base_ms > 3)Wait 10 seconds for the punit to settle and complete any outstanding transactions upon module load.
10 *SECONDS* ?!
Don't be alarmed :)
It's up to 10 seconds, otherwise we would end up waiting up to 3 minutes.
And I've seen a version (and you did as well) where those 3 minutes were raised to 6 (for the PVC particular case).
Thanks for checking this,
Andi
Quoted 65 lines
Quoted 65 lines
Signed-off-by Aravind Iddamsetty <[email protected]>Co-developed-by Chris Wilson <[email protected]>Cc Rodrigo Vivi <[email protected]>Signed-off-by Andi Shyti <[email protected]>--- drivers/gpu/drm/i915/intel_pcode.c | 35 ++++++++++++++++++++++++++---- 1 file changed, 31 insertions(+), 4 deletions(-)drivers/gpu/drm/i915/intel_pcode.c +31
-4@@ -204,15 +204,42 @@ int skl_pcode_request(struct intel_uncore *uncore, u32 mbox, u32 request,#undef COND } +static int pcode_init_wait (struct intel_uncore *uncore,int timeout_ms) +{ +if (__intel_wait_for_register_fw (uncore, +GEN6_PCODE_MAILBOX , +GEN6_PCODE_READY ,0 , +500 , timeout_ms, +NULL )) +return -EPROBE_DEFER ; + +return skl_pcode_request (uncore, +DG1_PCODE_STATUS , +DG1_UNCORE_GET_INIT_STATUS , +DG1_UNCORE_INIT_STATUS_COMPLETE , +DG1_UNCORE_INIT_STATUS_COMPLETE , timeout_ms); +} +int intel_pcode_init (struct intel_uncore *uncore) { +int err; +if (!IS_DGFX (uncore->i915))return 0 ; -return skl_pcode_request (uncore,DG1_PCODE_STATUS , -DG1_UNCORE_GET_INIT_STATUS , -DG1_UNCORE_INIT_STATUS_COMPLETE , -DG1_UNCORE_INIT_STATUS_COMPLETE ,180000 ); +/* +* Wait 10 seconds so that the punit to settle and complete +* any outstanding transactions upon module load +*/ + err =pcode_init_wait (uncore,10000 ); + +if (err) { +drm_notice (&uncore->i915->drm, +"Waiting for HW initialisation...\n" ); + err =pcode_init_wait (uncore,180000 ); + } + +return err; }int snb_pcode_read_p (struct intel_uncore *uncore,u32 mbcmd,u32 p1,u32 p2,u32 *val)Signature · 1 line
2.34.1
Rodrigo Vivi
maintainerQuoted 17 lines · Andi Shyti
Hi Rodrigo,
Quoted 7 lines · Rodrigo Vivi
Quoted 6 lines · Andi Shyti
From: Aravind Iddamsetty <[email protected]>
During module load not all the punit transaction have completed and we might end up timing out, as shown by the following warning:
Quoted 6 lines
Quoted 4 lines
i915 0000:4d:00.0: drm_WARN_ON_ONCE(timeout_base_ms > 3)Wait 10 seconds for the punit to settle and complete any outstanding transactions upon module load.
10 *SECONDS* ?!
Quoted 7 lines
Don't be alarmed :)
It's up to 10 seconds, otherwise we would end up waiting up to 3 minutes.
And I've seen a version (and you did as well) where those 3 minutes were raised to 6 (for the PVC particular case).
Oh yeap! and that case is funny! Because the indication from PCODE is that 10 seconds is enough, but there are some rare cases where it gets stuck and end up taking a very long time. Then they multiplied the bad rare case to 3, and no idea why how that become 6.
But anyway, thanks for refreshing my memory. When I first noticed this patch I thought it was in all the platforms, where this wouldn't make sense. But on discrete where the pcode needs to bring the memory and gt up before we can really use it, it makes sense.
And then I noticed that your patch is indeed for dgfx only, so it would be okay. And 10 seconds is okay.
However I also noticed that you do this before the other pcode_init check that we were told by pcode folks to use. So, I don't understand how your patch is helping now... you wait for 10 seconds and then you will wait more 10 seconds on the pcode_ready... why the pcode_ready check that we have in case already doesn't cover yours?
And why that return?
Quoted 69 lines
Thanks for checking this,
AndiQuoted 65 lines
Quoted 65 lines
Signed-off-by Aravind Iddamsetty <[email protected]>Co-developed-by Chris Wilson <[email protected]>Cc Rodrigo Vivi <[email protected]>Signed-off-by Andi Shyti <[email protected]>--- drivers/gpu/drm/i915/intel_pcode.c | 35 ++++++++++++++++++++++++++---- 1 file changed, 31 insertions(+), 4 deletions(-)drivers/gpu/drm/i915/intel_pcode.c +31
-4@@ -204,15 +204,42 @@ int skl_pcode_request(struct intel_uncore *uncore, u32 mbox, u32 request,#undef COND } +static int pcode_init_wait (struct intel_uncore *uncore,int timeout_ms) +{ +if (__intel_wait_for_register_fw (uncore, +GEN6_PCODE_MAILBOX , +GEN6_PCODE_READY ,0 , +500 , timeout_ms, +NULL )) +return -EPROBE_DEFER ; + +return skl_pcode_request (uncore, +DG1_PCODE_STATUS , +DG1_UNCORE_GET_INIT_STATUS , +DG1_UNCORE_INIT_STATUS_COMPLETE , +DG1_UNCORE_INIT_STATUS_COMPLETE , timeout_ms); +} +int intel_pcode_init (struct intel_uncore *uncore) { +int err; +if (!IS_DGFX (uncore->i915))return 0 ; -return skl_pcode_request (uncore,DG1_PCODE_STATUS , -DG1_UNCORE_GET_INIT_STATUS , -DG1_UNCORE_INIT_STATUS_COMPLETE , -DG1_UNCORE_INIT_STATUS_COMPLETE ,180000 ); +/* +* Wait 10 seconds so that the punit to settle and complete +* any outstanding transactions upon module load +*/ + err =pcode_init_wait (uncore,10000 ); + +if (err) { +drm_notice (&uncore->i915->drm, +"Waiting for HW initialisation...\n" ); + err =pcode_init_wait (uncore,180000 ); + } + +return err; }int snb_pcode_read_p (struct intel_uncore *uncore,u32 mbcmd,u32 p1,u32 p2,u32 *val)Signature · 1 line
2.34.1
Hi Rodrigo,
Quoted 31 lines · Rodrigo Vivi
Quoted 17 lines · Andi Shyti
Hi Rodrigo,
Quoted 7 lines · Rodrigo Vivi
Quoted 6 lines · Andi Shyti
From: Aravind Iddamsetty <[email protected]>
During module load not all the punit transaction have completed and we might end up timing out, as shown by the following warning:
Quoted 6 lines
Quoted 4 lines
i915 0000:4d:00.0: drm_WARN_ON_ONCE(timeout_base_ms > 3)Wait 10 seconds for the punit to settle and complete any outstanding transactions upon module load.
10 *SECONDS* ?!
Quoted 7 lines
Don't be alarmed :)
It's up to 10 seconds, otherwise we would end up waiting up to 3 minutes.
And I've seen a version (and you did as well) where those 3 minutes were raised to 6 (for the PVC particular case).
Oh yeap! and that case is funny! Because the indication from PCODE is that 10 seconds is enough, but there are some rare cases where it gets stuck and end up taking a very long time. Then they multiplied the bad rare case to 3, and no idea why how that become 6.
Quoted 7 lines
But anyway, thanks for refreshing my memory. When I first noticed this patch I thought it was in all the platforms, where this wouldn't make sense. But on discrete where the pcode needs to bring the memory and gt up before we can really use it, it makes sense.
And then I noticed that your patch is indeed for dgfx only, so it would be okay. And 10 seconds is okay.
yes... :)
Thanks for the recap!
Quoted 5 lines
However I also noticed that you do this before the other
pcode_initcheck that we were told by pcode folks to use. So, I don't understand how your patch is helping now... you wait for 10 seconds and then you will wait more 10 seconds on thepcode_ready... why thepcode_readycheck that we have in case already doesn't cover yours?
The difference is that the first timeout ensures that the punit is ready during boot time. The need to wait is only in boot, as we check if it's ready and then start communicating.
The second 10s wait comes after the read/write has actually happened. I expect in this case to wait just a little because it's just a test write done in boot to make sure the punit is really responding and ready for the next writes.
But because skl_pcode_request() is used in other contexts, as well, I assume that the punit is ready and I don't need to wait anymore before communicating with it. Thus I wait only after sending the command: I wait for it to complete and be ready again for the next command.
In other words each wait makes sure that the punit is ready for the next command. That's why we need a first 10 seconds wait as at the very first write we weren't sure 100% the punit was ready.
Quoted 1 line
And why that return?
Becase the driver is not really failing. Most probably the hardware is completing the boot routines thus we need to try probing again later.
I hope I was able to explain myself.
Thanks for the comments and for looking into this, Andi
Quoted 69 lines
Quoted 69 lines
Thanks for checking this,
AndiQuoted 65 lines
Quoted 65 lines
Signed-off-by Aravind Iddamsetty <[email protected]>Co-developed-by Chris Wilson <[email protected]>Cc Rodrigo Vivi <[email protected]>Signed-off-by Andi Shyti <[email protected]>--- drivers/gpu/drm/i915/intel_pcode.c | 35 ++++++++++++++++++++++++++---- 1 file changed, 31 insertions(+), 4 deletions(-)drivers/gpu/drm/i915/intel_pcode.c +31
-4@@ -204,15 +204,42 @@ int skl_pcode_request(struct intel_uncore *uncore, u32 mbox, u32 request,#undef COND } +static int pcode_init_wait (struct intel_uncore *uncore,int timeout_ms) +{ +if (__intel_wait_for_register_fw (uncore, +GEN6_PCODE_MAILBOX , +GEN6_PCODE_READY ,0 , +500 , timeout_ms, +NULL )) +return -EPROBE_DEFER ; + +return skl_pcode_request (uncore, +DG1_PCODE_STATUS , +DG1_UNCORE_GET_INIT_STATUS , +DG1_UNCORE_INIT_STATUS_COMPLETE , +DG1_UNCORE_INIT_STATUS_COMPLETE , timeout_ms); +} +int intel_pcode_init (struct intel_uncore *uncore) { +int err; +if (!IS_DGFX (uncore->i915))return 0 ; -return skl_pcode_request (uncore,DG1_PCODE_STATUS , -DG1_UNCORE_GET_INIT_STATUS , -DG1_UNCORE_INIT_STATUS_COMPLETE , -DG1_UNCORE_INIT_STATUS_COMPLETE ,180000 ); +/* +* Wait 10 seconds so that the punit to settle and complete +* any outstanding transactions upon module load +*/ + err =pcode_init_wait (uncore,10000 ); + +if (err) { +drm_notice (&uncore->i915->drm, +"Waiting for HW initialisation...\n" ); + err =pcode_init_wait (uncore,180000 ); + } + +return err; }int snb_pcode_read_p (struct intel_uncore *uncore,u32 mbcmd,u32 p1,u32 p2,u32 *val)Signature · 1 line
2.34.1