drm/xe/mmio: Make xe_mmio_wait32() aware of interrupts
Subsystems (2)
v2 cover drm/xe/mmio: Make xe_mmio_wait32() aware of interrupts
The current implementation of xe_mmio_wait32() is susceptible to preemption/interrupts causing false timeouts. This series proposes a fix for that.
This is v2 of [1].
Changes in v2:
- Add a patch moving xe_mmio_wait32() to xe_mmio.c
- Use a barrier()-free implementation.
[1] https://lore.kernel.org/r/[email protected]
Gustavo Sousa (2):
drm/xe/mmio: Move xe_mmio_wait32() to xe_mmio.c drm/xe/mmio: Make xe_mmio_wait32() aware of interrupts
drivers/gpu/drm/xe/xe_mmio.c | 59 ++++++++++++++++++++++++++++++++++++ drivers/gpu/drm/xe/xe_mmio.h | 39 ++---------------------- 2 files changed, 61 insertions(+), 37 deletions(-)
Signature · 1 line
2.42.0
PATCH v2 1/2 drm/xe/mmio: Move xe_mmio_wait32() to xe_mmio.c
This function is big enough, let's move it to a shared compilation unit.
While at it, document it.
--- drivers/gpu/drm/xe/xe_mmio.c | 53 ++++++++++++++++++++++++++++++++++++ drivers/gpu/drm/xe/xe_mmio.h | 39 ++------------------------ 2 files changed, 55 insertions(+), 37 deletions(-)
drivers/gpu/drm/xe/xe_mmio.c +53 -0
@@ -488,3 +488,56 @@ u64 xe_mmio_read64_2x32(struct xe_gt *gt, struct xe_reg reg)return (u64 )udw <<32 | ldw; } +/** +* xe_mmio_wait32() - Wait for a register to match the desired masked value +* @gt: MMIO target GT +* @reg: register to read value from +* @mask: mask to be applied to the value read from the register +* @val: desired value after applying the mask +* @timeout_us: time out after this period of time +* @out_val: if not NULL, points where to store the last unmasked value +* @atomic: needs to be true if calling from an atomic context +* +* This function polls for the desired masked value and returns zero on success +* or -ETIMEDOUT if timed out. +* +* Note that @timeout_us represents the minimum amount of time to wait before +* giving up. The actual time taken by this function can be a little more than +* @timeout_us for different reasons, specially in non-atomic contexts. Thus, +* it is possible that this function succeeds even after @timeout_us has passed. +*/ +int xe_mmio_wait32 (struct xe_gt *gt,struct xe_reg reg,u32 mask,u32 val,u32 timeout_us, +u32 *out_val,bool atomic) +{ +ktime_t cur =ktime_get_raw (); +const ktime_t end =ktime_add_us (cur, timeout_us); +int ret = -ETIMEDOUT ; +s64 wait =10 ; +u32 read; + +for (;;) { + read =xe_mmio_read32 (gt, reg); +if ((read & mask) == val) { + ret =0 ; +break ; + } + + cur =ktime_get_raw (); +if (!ktime_before (cur, end)) +break ; + +if (ktime_after (ktime_add_us (cur, wait), end)) + wait =ktime_us_delta (end, cur); + +if (atomic) +udelay (wait); +else +usleep_range (wait, wait <<1 ); + wait <<=1 ; + } + +if (out_val) + *out_val = read; + +return ret; +}
drivers/gpu/drm/xe/xe_mmio.h +2 -37
@@ -87,43 +87,6 @@ static inline int xe_mmio_write32_and_verify(struct xe_gt *gt,return (reg_val & mask) != eval ? -EINVAL :0 ; } -static inline int xe_mmio_wait32 (struct xe_gt *gt,struct xe_reg reg,u32 mask, -u32 val,u32 timeout_us,u32 *out_val, -bool atomic) -{ -ktime_t cur =ktime_get_raw (); -const ktime_t end =ktime_add_us (cur, timeout_us); -int ret = -ETIMEDOUT ; -s64 wait =10 ; -u32 read; - -for (;;) { - read =xe_mmio_read32 (gt, reg); -if ((read & mask) == val) { - ret =0 ; -break ; - } - - cur =ktime_get_raw (); -if (!ktime_before (cur, end)) -break ; - -if (ktime_after (ktime_add_us (cur, wait), end)) - wait =ktime_us_delta (end, cur); - -if (atomic) -udelay (wait); -else -usleep_range (wait, wait <<1 ); - wait <<=1 ; - } - -if (out_val) - *out_val = read; - -return ret; -} -int xe_mmio_ioctl (struct drm_device *dev,void *data,struct drm_file *file);
@@ -140,5 +103,7 @@ static inline bool xe_mmio_in_range(const struct xe_gt *gt,int xe_mmio_probe_vram (struct xe_device *xe);int xe_mmio_tile_vram_size (struct xe_tile *tile,u64 *vram_size,u64 *tile_size,u64 *tile_base);u64 xe_mmio_read64_2x32 (struct xe_gt *gt,struct xe_reg reg); +int xe_mmio_wait32 (struct xe_gt *gt,struct xe_reg reg,u32 mask,u32 val,u32 timeout_us, +u32 *out_val,bool atomic);#endif
Signature · 1 line
2.42.0
Re: [Intel-xe] [PATCH v2 1/2] drm/xe/mmio: Move xe_mmio_wait32() to xe_mmio.c
Quoted 3 lines · Gustavo Sousa
This function is big enough, let's move it to a shared compilation unit.
While at it, document it.
14.7k may seem a small gain, but IMO there's no benefit to inline this function.
$ ./scripts/bloat-o-meter build64/drivers/gpu/drm/xe/xe.ko{.old,}
add/remove: 2/0 grow/shrink: 0/58 up/down: 554/-15645 (-15091)
Function old new delta
xe_mmio_wait32 - 538 +538
__pfx_xe_mmio_wait32 - 16 +16
_intel_hdcp_enable 1390 1327 -63
xe_guc_reset 1339 1258 -81
__xe_guc_upload 2122 1984 -138
xe_uc_fw_upload 2815 2662 -153
intel_mpllb_enable 1986 1813 -173
intel_dp_aux_xfer 3028 2852 -176
xe_huc_auth 1294 1116 -178
_intel_hdcp_disable 1850 1672 -178
intel_ddi_set_idle_link_train 666 482 -184
icl_pll_power_enable 724 537 -187
icl_pll_enable 724 537 -187
chv_dpio_cmn_power_well_enable 884 697 -187
skl_ddi_pll_enable 1197 1009 -188
intel_write_sha_text 423 234 -189
i8xx_fbc_deactivate 575 385 -190
xe2lpd_pica_power_well_enable 517 323 -194
intel_mpllb_disable 924 729 -195
intel_cx0_bus_reset 556 360 -196
xe2lpd_pica_power_well_disable 514 317 -197
intel_cx0_wait_for_ack 901 702 -199
mcr_lock 414 213 -201
gt_reset_worker 1475 1271 -204
intel_disable_transcoder 1690 1485 -205
wait_for_act_sent 365 159 -206
vlv_wait_port_ready 785 576 -209
gen11_dsi_enable 1607 1398 -209
assert_chv_phy_status 1255 1045 -210
intel_psr_work 1105 894 -211
_intel_hdcp2_disable 2323 2112 -211
hsw_wait_for_power_well_enable 764 552 -212
gen11_dsi_post_disable 4765 4551 -214
intel_psr_wait_exit_locked 596 380 -216
chv_enable_pll 2212 1996 -216
gen9_wait_for_power_well_fuses 382 164 -218
skl_set_cdclk 3455 3235 -220
intel_snps_phy_wait_for_calibration 405 183 -222
intel_dp_mst_hdcp_stream_encryption 932 700 -232
intel_vrr_disable 926 690 -236
pcode_mailbox_rw 1376 1138 -238
vlv_enable_pll 1784 1543 -241
intel_dp_mst_hdcp2_stream_encryption 1169 927 -242
_intel_hdcp2_enable 7533 7281 -252
intel_mtl_pll_disable 2785 2531 -254
wait_panel_status 2347 2090 -257
gmbus_wait_idle 929 672 -257
__intel_cx0_read 1236 965 -271
xe_force_wake_get 999 707 -292
xe_force_wake_put 981 676 -305
intel_pmdemand_check_prev_transaction 520 132 -388
icl_pll_disable 1340 944 -396
xe_guc_mmio_send_recv 4920 4522 -398
intel_cx0_powerdown_change_sequence.constprop 1281 877 -404
__intel_cx0_write 1564 1104 -460
intel_psr_wait_for_idle_locked 1304 836 -468
xe_driver_flr_fini 1431 885 -546
_bxt_set_cdclk.isra 4301 3668 -633
intel_hdcp_auth 10233 9523 -710
intel_mtl_pll_enable 9114 7766 -1348
Total: Before=2181322, After=2166231, chg -0.69%What could make sense if we are worried about the extra function call would be to inline the first part and let the wait logic in a function in this compilation unit... because if we are going to spin/sleep, then the extra call is negligible. However looking at the timeouts being passed, I think we can just leave it here.
Maybe add the "add/remove" and "Total" lines to the commit message?
Quoted 12 lines
Signed-off-by Gustavo Sousa <[email protected]>---
drivers/gpu/drm/xe/xe_mmio.c| 53 ++++++++++++++++++++++++++++++++++++drivers/gpu/drm/xe/xe_mmio.h| 39 ++------------------------2 files changed, 55 insertions(+), 37 deletions(-)drivers/gpu/drm/xe/xe_mmio.c +0
-0
drivers/gpu/drm/xe/xe_mmio.c u64 xe_mmio_read64_2x32(struct xe_gt *gt, struct xe_reg reg) · line 497
Quoted 10 lines
drivers/gpu/drm/xe/xe_mmio.c +7
-0@@ -488,3 +488,56 @@ u64 xe_mmio_read64_2x32(struct xe_gt *gt, struct xe_reg reg)return (u64 )udw <<32 | ldw; } +/** +* xe_mmio_wait32() - Wait for a register to match the desired masked value +* @gt: MMIO target GT +* @reg: register to read value from +* @mask: mask to be applied to the value read from the register +* @val: desired value after applying the mask +* @timeout_us: time out after this period of time
@timeout_us: time out after this period of time. Wait logic tries to be smart, applying a exponential backoff until @timeout_us is reached.
besides the nits above, this looks good.
thanks
Lucas De Marchi
Quoted 105 lines
drivers/gpu/drm/xe/xe_mmio.c +46
-0+* @out_val: if not NULL, points where to store the last unmasked value +* @atomic: needs to be true if calling from an atomic context +* +* This function polls for the desired masked value and returns zero on success +* or -ETIMEDOUT if timed out. +* +* Note that @timeout_us represents the minimum amount of time to wait before +* giving up. The actual time taken by this function can be a little more than +* @timeout_us for different reasons, specially in non-atomic contexts. Thus, +* it is possible that this function succeeds even after @timeout_us has passed. +*/ +int xe_mmio_wait32 (struct xe_gt *gt,struct xe_reg reg,u32 mask,u32 val,u32 timeout_us, +u32 *out_val,bool atomic) +{ +ktime_t cur =ktime_get_raw (); +const ktime_t end =ktime_add_us (cur, timeout_us); +int ret = -ETIMEDOUT ; +s64 wait =10 ; +u32 read; + +for (;;) { + read =xe_mmio_read32 (gt, reg); +if ((read & mask) == val) { + ret =0 ; +break ; + } + + cur =ktime_get_raw (); +if (!ktime_before (cur, end)) +break ; + +if (ktime_after (ktime_add_us (cur, wait), end)) + wait =ktime_us_delta (end, cur); + +if (atomic) +udelay (wait); +else +usleep_range (wait, wait <<1 ); + wait <<=1 ; + } + +if (out_val) + *out_val = read; + +return ret; +}drivers/gpu/drm/xe/xe_mmio.h +2
-37@@ -87,43 +87,6 @@ static inline int xe_mmio_write32_and_verify(struct xe_gt *gt,return (reg_val & mask) != eval ? -EINVAL :0 ; } -static inline int xe_mmio_wait32 (struct xe_gt *gt,struct xe_reg reg,u32 mask, -u32 val,u32 timeout_us,u32 *out_val, -bool atomic) -{ -ktime_t cur =ktime_get_raw (); -const ktime_t end =ktime_add_us (cur, timeout_us); -int ret = -ETIMEDOUT ; -s64 wait =10 ; -u32 read; - -for (;;) { - read =xe_mmio_read32 (gt, reg); -if ((read & mask) == val) { - ret =0 ; -break ; - } - - cur =ktime_get_raw (); -if (!ktime_before (cur, end)) -break ; - -if (ktime_after (ktime_add_us (cur, wait), end)) - wait =ktime_us_delta (end, cur); - -if (atomic) -udelay (wait); -else -usleep_range (wait, wait <<1 ); - wait <<=1 ; - } - -if (out_val) - *out_val = read; - -return ret; -} -int xe_mmio_ioctl (struct drm_device *dev,void *data,struct drm_file *file);@@ -140,5 +103,7 @@ static inline bool xe_mmio_in_range(const struct xe_gt *gt,int xe_mmio_probe_vram (struct xe_device *xe);int xe_mmio_tile_vram_size (struct xe_tile *tile,u64 *vram_size,u64 *tile_size,u64 *tile_base);u64 xe_mmio_read64_2x32 (struct xe_gt *gt,struct xe_reg reg); +int xe_mmio_wait32 (struct xe_gt *gt,struct xe_reg reg,u32 mask,u32 val,u32 timeout_us, +u32 *out_val,bool atomic);#endif Signature · 1 line
2.42.0
PATCH v2 2/2 drm/xe/mmio: Make xe_mmio_wait32() aware of interrupts
In reply to Gustavo SousaWith the current implementation, a preemption or other kind of interrupt might happen between xe_mmio_read32() and ktime_get_raw(). Such an interruption (specially in the case of preemption) might be long enough to cause a timeout without giving a chance of a new check on the register value on a next iteration, which would have happened otherwise.
This issue causes some sporadic timeouts in some code paths. As an example, we were experiencing some rare timeouts when waiting for PLL unlock for C10/C20 PHYs (see intel_cx0pll_disable()). After debugging, we found out that the PLL unlock was happening within the expected time period (20us), which suggested a bug in xe_mmio_wait32().
To fix the issue, ensure that we do a last check out of the loop if necessary.
This change was tested with the aforementioned PLL unlocking code path. Experiments showed that, before this change, we observed reported timeouts in 54 of 5000 runs; and, after this change, no timeouts were reported in 5000 runs.
v2:
- Prefer an implementation without a barrier (v1 switched the order of xe_mmio_read32() and ktime_get_raw() calls and added a barrier() in between). (Lucas, Rodrigo)
--- drivers/gpu/drm/xe/xe_mmio.c | 6 ++++++ 1 file changed, 6 insertions(+)
drivers/gpu/drm/xe/xe_mmio.c +6 -0
@@ -536,6 +536,12 @@ int xe_mmio_wait32(struct xe_gt *gt, struct xe_reg reg, u32 mask, u32 val, u32 t wait <<=1 ; } +if (ret !=0 ) { + read =xe_mmio_read32 (gt, reg); +if ((read & mask) == val) + ret =0 ; + } +if (out_val) *out_val = read;
Signature · 1 line
2.42.0
Quoting Gustavo Sousa (2023-11-16 12:39:02-03:00)
Quoted 12 lines
With the current implementation, a preemption or other kind of interrupt might happen between
xe_mmio_read32()andktime_get_raw(). Such an interruption (specially in the case of preemption) might be long enough to cause a timeout without giving a chance of a new check on the register value on a next iteration, which would have happened otherwise.This issue causes some sporadic timeouts in some code paths. As an example, we were experiencing some rare timeouts when waiting for PLL unlock for C10/C20 PHYs (see
intel_cx0pll_disable()). After debugging, we found out that the PLL unlock was happening within the expected time period (20us), which suggested a bug inxe_mmio_wait32().
Quoted 7 lines
To fix the issue, ensure that we do a last check out of the loop if necessary.
This change was tested with the aforementioned PLL unlocking code path. Experiments showed that, before this change, we observed reported timeouts in 54 of 5000 runs; and, after this change, no timeouts were reported in 5000 runs.
Well, I'm using a crystal ball here :-)
This is under test as I type this (I just checked and it is at run #1053 of 5000), I'll come back later once the test is fully finished.
--
Gustavo Sousa
Quoted 33 lines
v2:
- Prefer an implementation without a barrier (v1 switched the order ofxe_mmio_read32()andktime_get_raw()calls and added abarrier()in between). (Lucas, Rodrigo)Cc Lucas De Marchi <[email protected]>Cc Rodrigo Vivi <[email protected]>Signed-off-by Gustavo Sousa <[email protected]>---
drivers/gpu/drm/xe/xe_mmio.c| 6 ++++++1 file changed, 6 insertions(+)drivers/gpu/drm/xe/xe_mmio.c +6
-0@@ -536,6 +536,12 @@ int xe_mmio_wait32(struct xe_gt *gt, struct xe_reg reg, u32 mask, u32 val, u32 t wait <<=1 ; } +if (ret !=0 ) { + read =xe_mmio_read32 (gt, reg); +if ((read & mask) == val) + ret =0 ; + } +if (out_val) *out_val = read;Signature · 1 line
2.42.0
Quoting Gustavo Sousa (2023-11-16 12:44:02-03:00)
Quoted 21 lines
Quoting Gustavo Sousa (2023-11-16 12:39:02-03:00)
Quoted 12 lines
With the current implementation, a preemption or other kind of interrupt might happen between
xe_mmio_read32()andktime_get_raw(). Such an interruption (specially in the case of preemption) might be long enough to cause a timeout without giving a chance of a new check on the register value on a next iteration, which would have happened otherwise.This issue causes some sporadic timeouts in some code paths. As an example, we were experiencing some rare timeouts when waiting for PLL unlock for C10/C20 PHYs (see
intel_cx0pll_disable()). After debugging, we found out that the PLL unlock was happening within the expected time period (20us), which suggested a bug inxe_mmio_wait32().Quoted 7 lines
To fix the issue, ensure that we do a last check out of the loop if necessary.
This change was tested with the aforementioned PLL unlocking code path. Experiments showed that, before this change, we observed reported timeouts in 54 of 5000 runs; and, after this change, no timeouts were reported in 5000 runs.
Quoted 4 lines
Well, I'm using a crystal ball here :-)
This is under test as I type this (I just checked and it is at run #1053 of 5000), I'll come back later once the test is fully finished.
As predicted :-)
--
Gustavo Sousa
Quoted 37 lines
--
Gustavo SousaQuoted 33 lines
v2:
- Prefer an implementation without a barrier (v1 switched the order ofxe_mmio_read32()andktime_get_raw()calls and added abarrier()in between). (Lucas, Rodrigo)Cc Lucas De Marchi <[email protected]>Cc Rodrigo Vivi <[email protected]>Signed-off-by Gustavo Sousa <[email protected]>---
drivers/gpu/drm/xe/xe_mmio.c| 6 ++++++1 file changed, 6 insertions(+)drivers/gpu/drm/xe/xe_mmio.c +6
-0@@ -536,6 +536,12 @@ int xe_mmio_wait32(struct xe_gt *gt, struct xe_reg reg, u32 mask, u32 val, u32 t wait <<=1 ; } +if (ret !=0 ) { + read =xe_mmio_read32 (gt, reg); +if ((read & mask) == val) + ret =0 ; + } +if (out_val) *out_val = read;Signature · 1 line
2.42.0
Quoted 20 lines · Gustavo Sousa
With the current implementation, a preemption or other kind of interrupt might happen between
xe_mmio_read32()andktime_get_raw(). Such an interruption (specially in the case of preemption) might be long enough to cause a timeout without giving a chance of a new check on the register value on a next iteration, which would have happened otherwise.This issue causes some sporadic timeouts in some code paths. As an example, we were experiencing some rare timeouts when waiting for PLL unlock for C10/C20 PHYs (see
intel_cx0pll_disable()). After debugging, we found out that the PLL unlock was happening within the expected time period (20us), which suggested a bug inxe_mmio_wait32().To fix the issue, ensure that we do a last check out of the loop if necessary.
This change was tested with the aforementioned PLL unlocking code path. Experiments showed that, before this change, we observed reported timeouts in 54 of 5000 runs; and, after this change, no timeouts were reported in 5000 runs.
Quoted 8 lines
v2:
- Prefer an implementation without a barrier (v1 switched the order ofxe_mmio_read32()andktime_get_raw()calls and added abarrier()in between). (Lucas, Rodrigo)Cc Lucas De Marchi <[email protected]>Cc Rodrigo Vivi <[email protected]>Signed-off-by Gustavo Sousa <[email protected]>
thanks
Lucas De Marchi
Quoted 24 lines
---
drivers/gpu/drm/xe/xe_mmio.c| 6 ++++++1 file changed, 6 insertions(+)drivers/gpu/drm/xe/xe_mmio.c +6
-0@@ -536,6 +536,12 @@ int xe_mmio_wait32(struct xe_gt *gt, struct xe_reg reg, u32 mask, u32 val, u32 t wait <<=1 ; } +if (ret !=0 ) { + read =xe_mmio_read32 (gt, reg); +if ((read & mask) == val) + ret =0 ; + } +if (out_val) *out_val = read;Signature · 1 line
2.42.0
Patchwork
CI[Intel-xe] ✗ CI.Patch_applied: failure for drm/xe/mmio: Make xe_mmio_wait32() aware of interrupts (rev4)
In reply to Gustavo Sousa== Series Details ==
== Series Details ==
Series: drm/xe/mmio: Make xe_mmio_wait32() aware of interrupts (rev4)
URL : https://patchwork.freedesktop.org/series/126428/
State : failure
== Summary ==
=== Applying kernel patches on branch 'drm-xe-next' with base: === Base commit: 33775eb43 drm/xe/guc: Promote guc_to_gt/xe helpers to .h === git am output follows ===
Applying: drm/xe/mmio: Move xe_mmio_wait32() to xe_mmio.c
Applying: drm/xe/mmio: Make xe_mmio_wait32() aware of interrupts
Re: [Intel-xe] ✗ CI.Patch_applied: failure for drm/xe/mmio: Make xe_mmio_wait32() aware of interrupts (rev4)
Quoting Patchwork (2023-11-16 12:42:35-03:00)
Quoted 6 lines
== Series Details ==
Series: drm/xe/mmio: Make
xe_mmio_wait32()aware of interrupts (rev4)
URL : https://patchwork.freedesktop.org/series/126428/
State : failure
Quoted 7 lines
== Summary ==
=== Applying kernel patches on branch 'drm-xe-next' with base: === Base commit:
33775eb43drm/xe/guc: Promote guc_to_gt/xe helpers to .h === git am output follows ===
Applying: drm/xe/mmio: Movexe_mmio_wait32()toxe_mmio.c
Applying: drm/xe/mmio: Makexe_mmio_wait32()aware of interrupts
Not sure what went wrong here. I just tested and I was able to apply the series successfully on top of 33775eb43.
--
Gustavo Sousa
Patchwork
CI[Intel-xe] ✗ CI.Patch_applied: failure for drm/xe/mmio: Make xe_mmio_wait32() aware of interrupts (rev4)
In reply to Gustavo Sousa== Series Details ==
== Series Details ==
Series: drm/xe/mmio: Make xe_mmio_wait32() aware of interrupts (rev4)
URL : https://patchwork.freedesktop.org/series/126428/
State : failure
== Summary ==
=== Applying kernel patches on branch 'drm-xe-next' with base: === Base commit: cb9b702d4 drm/xe/dg2: Wa_18028616096 now applies to all DG2 === git am output follows ===
error: patch failed: drivers/gpu/drm/xe/xe_mmio.h:140
error: drivers/gpu/drm/xe/xe_mmio.h: patch does not apply
hint: Use 'git am --show-current-patch' to see the failed patch
Applying: drm/xe/mmio: Move xe_mmio_wait32() to xe_mmio.c Patch failed at 0001 drm/xe/mmio: Move xe_mmio_wait32() to xe_mmio.c When you have resolved this problem, run "git am --continue". If you prefer to skip this patch, run "git am --skip" instead. To restore the original branch and stop patching, run "git am --abort".