drm/i915: fix race when clearing RPS IIR bits
Subsystems (2)
PATCH 1/2 drm/i915: fix race when clearing RPS IIR bits
When disabling RPS interrupts there is a race where we disable RPS inerrupts while the interrupt handler is running and the handler has already latched the pending RPS interrupt from the master IIR register. Afterwards the disabling path clears the PM IIR bits, making the state of pending interrupts inconsistent from the interrupt handler's point of view. This triggers the following warning: "The master control interrupt lied (PM)!".
To fix this make sure that any running interrupt handler (which may have already latched the master IIR) finishes before clearing the IIR bits.
Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=87347
--- drivers/gpu/drm/i915/i915_irq.c | 7 +++++++ 1 file changed, 7 insertions(+)
drivers/gpu/drm/i915/i915_irq.c +7 -0
@@ -330,6 +330,13 @@ void gen6_disable_rps_interrupts(struct drm_device *dev)__gen6_disable_pm_irq (dev_priv, dev_priv->pm_rps_events);I915_WRITE (gen6_pm_ier (dev_priv),I915_READ (gen6_pm_ier (dev_priv)) & ~dev_priv->pm_rps_events); + +spin_unlock_irq (&dev_priv->irq_lock); + +synchronize_irq (dev->irq); + +spin_lock_irq (&dev_priv->irq_lock); +I915_WRITE (gen6_pm_iir (dev_priv), dev_priv->pm_rps_events);I915_WRITE (gen6_pm_iir (dev_priv), dev_priv->pm_rps_events);
Signature · 6 lines
2.1.0 _______________________________________________ Intel-gfx mailing list [email protected] http://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH 2/2 drm/i915: move clearing of RPS interrupt bits from disable to reset time
The logical place for clearing the RPS latched interrupt bits is when resetting the RPS interrupts, so move the corresponding part from the RPS disable function to the reset function. During resetting we already cleared the IIR bits, so the only thing missing there was clearing pm_iir.
Note that we call gen6_disable_rps_interrupts() also during driver load and resume time via intel_uncore_sanitize() when i915 interrupts are still not installed. If there are any pending RPS bits at this point (which after this patch wouldn't be cleared) they will be cleared by the reset code via the interrupt preinstall hooks.
--- drivers/gpu/drm/i915/i915_irq.c | 10 +--------- 1 file changed, 1 insertion(+), 9 deletions(-)
drivers/gpu/drm/i915/i915_irq.c +1 -9
@@ -277,6 +277,7 @@ void gen6_reset_rps_interrupts(struct drm_device *dev)I915_WRITE (reg, dev_priv->pm_rps_events);I915_WRITE (reg, dev_priv->pm_rps_events);POSTING_READ (reg); + dev_priv->rps.pm_iir =0 ;spin_unlock_irq (&dev_priv->irq_lock); }
@@ -334,15 +335,6 @@ void gen6_disable_rps_interrupts(struct drm_device *dev)spin_unlock_irq (&dev_priv->irq_lock);synchronize_irq (dev->irq); - -spin_lock_irq (&dev_priv->irq_lock); - -I915_WRITE (gen6_pm_iir (dev_priv), dev_priv->pm_rps_events); -I915_WRITE (gen6_pm_iir (dev_priv), dev_priv->pm_rps_events); - - dev_priv->rps.pm_iir =0 ; - -spin_unlock_irq (&dev_priv->irq_lock); }/**
Signature · 6 lines
2.1.0 _______________________________________________ Intel-gfx mailing list [email protected] http://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [PATCH 2/2] drm/i915: move clearing of RPS interrupt bits from disable to reset time
Quoted 5 lines · Imre Deak
The logical place for clearing the RPS latched interrupt bits is when resetting the RPS interrupts, so move the corresponding part from the RPS disable function to the reset function. During resetting we already cleared the IIR bits, so the only thing missing there was clearing
pm_iir.
Quoted 7 lines
Note that we call
gen6_disable_rps_interrupts()also during driver load and resume time viaintel_uncore_sanitize()when i915 interrupts are still not installed. If there are any pending RPS bits at this point (which after this patch wouldn't be cleared) they will be cleared by the reset code via the interrupt preinstall hooks.Signed-off-by Imre Deak <[email protected]>
Both merged to dinq, thanks.
-Daniel
Signature · 7 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch _______________________________________________ Intel-gfx mailing list [email protected] http://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [PATCH 2/2] drm/i915: move clearing of RPS interrupt bits from disable to reset time
In reply to Imre DeakTask id: 6030 -------------------------------------Summary-------------------------------------
Platform Delta drm-intel-nightly Series Applied
PNV -1 275/275 274/275
ILK 303/303 303/303 SNB 304/304 304/304 IVB 339/339 339/339 BYT 287/287 287/287 HSW 361/361 361/361 BDW 310/310 310/310
-------------------------------------Detailed-------------------------------------
Platform Test drm-intel-nightly Series Applied
*PNV igt@gem_userptr_blits@minor-sync-interruptible PASS(2) DMESG_WARN(1)PASS(1)
Note: You need to pay more attention to line start with '*'
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
Quoted 8 lines · Imre Deak
When disabling RPS interrupts there is a race where we disable RPS inerrupts while the interrupt handler is running and the handler has already latched the pending RPS interrupt from the master IIR register. Afterwards the disabling path clears the PM IIR bits, making the state of pending interrupts inconsistent from the interrupt handler's point of view. This triggers the following warning: "The master control interrupt lied (PM)!".
Quoted 4 lines
To fix this make sure that any running interrupt handler (which may have already latched the master IIR) finishes before clearing the IIR bits.
Isn't this overkill for what is just a bogus WARN? If the WARN is a logical consequence of the code, let's just remove the WARN.
Or iow can you not find a cheaper way to fix this? -Chris
Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre _______________________________________________ Intel-gfx mailing list [email protected] http://lists.freedesktop.org/mailman/listinfo/intel-gfx
Quoted 14 lines · Chris Wilson
Quoted 8 lines · Imre Deak
When disabling RPS interrupts there is a race where we disable RPS inerrupts while the interrupt handler is running and the handler has already latched the pending RPS interrupt from the master IIR register. Afterwards the disabling path clears the PM IIR bits, making the state of pending interrupts inconsistent from the interrupt handler's point of view. This triggers the following warning: "The master control interrupt lied (PM)!".
Quoted 4 lines
To fix this make sure that any running interrupt handler (which may have already latched the master IIR) finishes before clearing the IIR bits.
Quoted 4 lines
Isn't this overkill for what is just a bogus WARN? If the WARN is a logical consequence of the code, let's just remove the WARN.
Or iow can you not find a cheaper way to fix this?
We only run this on suspend/resume afaik, overhead should be acceptable. And we've had that much overhead before we've done all the runtime pm unification, it's still less synchronization than disabling interrupts completely.
-Daniel
Signature · 7 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch _______________________________________________ Intel-gfx mailing list [email protected] http://lists.freedesktop.org/mailman/listinfo/intel-gfx
Quoted 18 lines · Daniel Vetter
Quoted 17 lines · Chris Wilson
Quoted 8 lines · Imre Deak
When disabling RPS interrupts there is a race where we disable RPS inerrupts while the interrupt handler is running and the handler has already latched the pending RPS interrupt from the master IIR register. Afterwards the disabling path clears the PM IIR bits, making the state of pending interrupts inconsistent from the interrupt handler's point of view. This triggers the following warning: "The master control interrupt lied (PM)!".
Quoted 4 lines
To fix this make sure that any running interrupt handler (which may have already latched the master IIR) finishes before clearing the IIR bits.
Isn't this overkill for what is just a bogus WARN? If the WARN is a logical consequence of the code, let's just remove the WARN.
Quoted 6 lines
Quoted 1 line
Or iow can you not find a cheaper way to fix this?
We only run this on suspend/resume afaik, overhead should be acceptable. And we've had that much overhead before we've done all the runtime pm unification, it's still less synchronization than disabling interrupts completely.
Hmm, I thought this was in conjunction with RPS pm masking (i.e. fired everytime we no longer expect to receive RPS interrupts). If it is only the infrequent, then yeah I can't complain too much - I still think it is slightly fishy, but I can accept that it is just a quirk of the buffering the interrupt does.
-Chris
Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre _______________________________________________ Intel-gfx mailing list [email protected] http://lists.freedesktop.org/mailman/listinfo/intel-gfx
Quoted 21 lines · Chris Wilson
Quoted 20 lines · Daniel Vetter
Quoted 18 lines · Chris Wilson
Quoted 8 lines · Imre Deak
When disabling RPS interrupts there is a race where we disable RPS inerrupts while the interrupt handler is running and the handler has already latched the pending RPS interrupt from the master IIR register. Afterwards the disabling path clears the PM IIR bits, making the state of pending interrupts inconsistent from the interrupt handler's point of view. This triggers the following warning: "The master control interrupt lied (PM)!".
Quoted 4 lines
To fix this make sure that any running interrupt handler (which may have already latched the master IIR) finishes before clearing the IIR bits.
Isn't this overkill for what is just a bogus WARN? If the WARN is a logical consequence of the code, let's just remove the WARN.
Or iow can you not find a cheaper way to fix this?
Quoted 7 lines
Quoted 4 lines
We only run this on suspend/resume afaik, overhead should be acceptable. And we've had that much overhead before we've done all the runtime pm unification, it's still less synchronization than disabling interrupts completely.
Hmm, I thought this was in conjunction with RPS pm masking (i.e. fired everytime we no longer expect to receive RPS interrupts).
Yes, it affects only the driver loading (where it's a nop) and the runtime/system suspend path.
Quoted 3 lines
If it is only the infrequent, then yeah I can't complain too much - I still think it is slightly fishy, but I can accept that it is just a quirk of the buffering the interrupt does.
The reason for sync_irq is to prevent any subsequent gen6_enable_rps_interrupts() call to trigger spurious interrupts. Without it we could have:
CPU 0 CPU 1gen8_gt_irq_handler()
tmp = I915_READ(GEN8_GT_IIR(2))
<tmp has pending RPS bits>
gen6_disable_rps_interrupts()
gen6_reset_rps_interrupts()
<clear GEN8_GT_IIR(2)>
gen6_enable_rps_interrupts()gen6_rps_irq_handler(tmp)
<handle the now stale RPS events>
Now I admit the above is quite unlikely, or even impossible due to what happens between gen6_disable_rps_interrupts() and gen6_enable_rps_interrupts() in our code atm. I still thought it's safer not to rely on those.
One alternative would have been to extend the irq_lock in the interrupt handler, so that we read GEN8_GT_IIR(2) and handle any events in it in one critical section, but that would add the overhead where it actually matters.
--Imre
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx