Testing PREEMPT_RT with disabling interrupts in the most critical section.
cover Testing PREEMPT_RT with disabling interrupts in the most critical section.
*** BLURB HERE ***
Maarten Lankhorst (3):
drm/xe: Bump xe_device_l2_flush even higher drm/i915/display: Disable interrupts on xe for realtime preemption [FOR-CI] PREEMPT_RT injection
Mike Galbraith (2):
drm/i915: Use preempt_disable/enable_rt() where recommended drm/i915: Don't disable interrupts on PREEMPT_RT during atomic updates
drivers/gpu/drm/i915/Kconfig.debug | 15 ------- drivers/gpu/drm/i915/display/intel_crtc.c | 13 ++++-- drivers/gpu/drm/i915/display/intel_cursor.c | 9 ++-- drivers/gpu/drm/i915/display/intel_vblank.c | 49 ++++++++++++++++----- drivers/gpu/drm/i915/display/intel_vblank.h | 9 ++++ drivers/gpu/drm/xe/Kconfig.debug | 5 +++ drivers/gpu/drm/xe/xe_device.c | 4 +- kernel/Kconfig.preempt | 3 +- 8 files changed, 69 insertions(+), 38 deletions(-)
Signature · 1 line
2.51.0
PATCH 1/5 drm/xe: Bump xe_device_l2_flush even higher
It turns out for some applications 1 ms is not enough, and 2 ms is needed.
Failing to flush causes a catastrophic display update failure, so we should prevent it if possible.
Add an even bigger margin of 5 ms, and complain loudly if we ever exceed it.
--- drivers/gpu/drm/xe/xe_device.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/gpu/drm/xe/xe_device.c +2 -2
@@ -1077,8 +1077,8 @@ void xe_device_l2_flush(struct xe_device *xe)spin_lock (>->global_invl_lock);xe_mmio_write32 (>->mmio,XE2_GLOBAL_INVAL ,0x1 ); -if (xe_mmio_wait32 (>->mmio,XE2_GLOBAL_INVAL ,0x1 ,0x0 ,1000 ,NULL , true)) -xe_gt_err_once (gt,"Global invalidation timeout\n" ); +if (xe_mmio_wait32 (>->mmio,XE2_GLOBAL_INVAL ,0x1 ,0x0 ,5000 ,NULL , true)) +xe_gt_err (gt,"Global invalidation timeout\n" );spin_unlock (>->global_invl_lock);
Signature · 1 line
2.51.0
PATCH 2/5 drm/i915: Use preempt_disable/enable_rt() where recommended
In reply to Maarten LankhorstFrom: Mike Galbraith <[email protected]>
Mario Kleiner suggest in commit
ad3543ede630f ("drm/intel: Push get_scanout_position() timestamping into kms driver.")
a spots where preemption should be disabled on PREEMPT_RT. The difference is that on PREEMPT_RT the intel_uncore::lock disables neither preemption nor interrupts and so region remains preemptible.
The area covers only register reads and writes. The part that worries me is:
- __intel_get_crtc_scanline() the worst case is 100us if no match is found.
- intel_crtc_scanlines_since_frame_timestamp() not sure how long this may take in the worst case.
It was in the RT queue for a while and nobody complained. Disable preemption on PREEPMPT_RT during timestamping.
[bigeasy: patch description.]
--- drivers/gpu/drm/i915/display/intel_vblank.c | 43 ++++++++++++++++----- 1 file changed, 33 insertions(+), 10 deletions(-)
drivers/gpu/drm/i915/display/intel_vblank.c +33 -10
@@ -315,6 +315,20 @@ static void intel_vblank_section_exit(struct intel_display *display)struct drm_i915_private *i915 =to_i915 (display->drm);spin_unlock (&i915->uncore.lock); } + +static void intel_vblank_section_enter_irqf (struct intel_display *display,unsigned long *flags) + __acquires(i915->uncore.lock) +{ +struct drm_i915_private *i915 =to_i915 (display->drm); +spin_lock_irqsave (&i915->uncore.lock, *flags); +} + +static void intel_vblank_section_exit_irqf (struct intel_display *display,unsigned long flags) + __releases(i915->uncore.lock) +{ +struct drm_i915_private *i915 =to_i915 (display->drm); +spin_unlock_irqrestore (&i915->uncore.lock, flags); +}#else static void intel_vblank_section_enter (struct intel_display *display) {
@@ -323,6 +337,17 @@ static void intel_vblank_section_enter(struct intel_display *display)static void intel_vblank_section_exit (struct intel_display *display) { } + +static void intel_vblank_section_enter_irqf (struct intel_display *display,unsigned long *flags) +{ + *flags =0 ; +} + +static void intel_vblank_section_exit_irqf (struct intel_display *display,unsigned long flags) +{ +if (flags) +return ; +}#endif static bool i915_get_crtc_scanoutpos(struct drm_crtc *_crtc,
@@ -359,10 +384,10 @@ static bool i915_get_crtc_scanoutpos(struct drm_crtc *_crtc,* timing critical raw register reads, potentially with * preemption disabled, so the following code must not block. */ -local_irq_save (irqflags); -intel_vblank_section_enter (display); +intel_vblank_section_enter_irqf (display, &irqflags); -/* preempt_disable_rt() should go right here in PREEMPT_RT patchset. */ +if (IS_ENABLED (CONFIG_PREEMPT_RT )) +preempt_disable ();/* Get optional system timestamp before query. */ if (stime)
@@ -426,10 +451,10 @@ static bool i915_get_crtc_scanoutpos(struct drm_crtc *_crtc,if (etime) *etime =ktime_get (); -/* preempt_enable_rt() should go right here in PREEMPT_RT patchset. */ +if (IS_ENABLED (CONFIG_PREEMPT_RT )) +preempt_enable (); -intel_vblank_section_exit (display); -local_irq_restore (irqflags); +intel_vblank_section_exit_irqf (display, irqflags);/* * While in vblank, position will be negative
@@ -467,13 +492,11 @@ int intel_get_crtc_scanline(struct intel_crtc *crtc)unsigned long irqflags;int position; -local_irq_save (irqflags); -intel_vblank_section_enter (display); +intel_vblank_section_enter_irqf (display, &irqflags); position =__intel_get_crtc_scanline (crtc); -intel_vblank_section_exit (display); -local_irq_restore (irqflags); +intel_vblank_section_exit_irqf (display, irqflags);return position; }
Signature · 1 line
2.51.0
PATCH 3/5 drm/i915: Don't disable interrupts on PREEMPT_RT during atomic updates
In reply to Maarten LankhorstFrom: Mike Galbraith <[email protected]>
Commit
8d7849db3eab7 ("drm/i915: Make sprite updates atomic")
started disabling interrupts across atomic updates. This breaks on PREEMPT_RT because within this section the code attempt to acquire spinlock_t locks which are sleeping locks on PREEMPT_RT.
According to the comment the interrupts are disabled to avoid random delays and not required for protection or synchronisation.
If this needs to happen with disabled interrupts on PREEMPT_RT, and the whole section is restricted to register access then all sleeping locks need to be acquired before interrupts are disabled and some function maybe moved after enabling interrupts again.
This includes:
- prepare_to_wait() + finish_wait() due its wake queue.
- drm_crtc_vblank_put() -> vblank_disable_fn() drm_device::vbl_lock.
- skl_pfit_enable(), intel_update_plane(), vlv_atomic_update_fifo() and maybe others due to intel_uncore::lock
- drm_crtc_arm_vblank_event() due to drm_device::event_lock and drm_device::vblank_time_lock.
Don't disable interrupts on PREEMPT_RT during atomic updates.
[bigeasy: drop local locks, commit message]
--- drivers/gpu/drm/i915/display/intel_crtc.c | 9 ++++++--- drivers/gpu/drm/i915/display/intel_cursor.c | 9 ++++++--- drivers/gpu/drm/i915/display/intel_vblank.c | 6 ++++-- 3 files changed, 16 insertions(+), 8 deletions(-)
drivers/gpu/drm/i915/display/intel_crtc.c +6 -3
@@ -567,7 +567,8 @@ void intel_pipe_update_start(struct intel_atomic_state *state,*/ intel_psr_wait_for_idle_locked (new_crtc_state); -local_irq_disable (); +if (!IS_ENABLED (CONFIG_PREEMPT_RT )) +local_irq_disable (); crtc->debug.min_vbl = evade.min; crtc->debug.max_vbl = evade.max;
@@ -585,7 +586,8 @@ void intel_pipe_update_start(struct intel_atomic_state *state,return ; irq_disable: -local_irq_disable (); +if (!IS_ENABLED (CONFIG_PREEMPT_RT )) +local_irq_disable (); }#if IS_ENABLED (CONFIG_DRM_I915_DEBUG_VBLANK_EVADE )
@@ -731,7 +733,8 @@ void intel_pipe_update_end(struct intel_atomic_state *state,if (!state->base.legacy_cursor_update)intel_vrr_send_push (NULL , new_crtc_state); -local_irq_enable (); +if (!IS_ENABLED (CONFIG_PREEMPT_RT )) +local_irq_enable ();if (intel_vgpu_active (dev_priv)) goto out;
drivers/gpu/drm/i915/display/intel_cursor.c +6 -3
@@ -919,13 +919,15 @@ intel_legacy_cursor_update(struct drm_plane *_plane,*/ intel_psr_wait_for_idle_locked (crtc_state); -local_irq_disable (); +if (!IS_ENABLED (CONFIG_PREEMPT_RT )) +local_irq_disable ();intel_vblank_evade (&evade);drm_crtc_vblank_put (&crtc->base); }else { -local_irq_disable (); +if (!IS_ENABLED (CONFIG_PREEMPT_RT )) +local_irq_disable (); }if (new_plane_state->uapi.visible) {
@@ -935,7 +937,8 @@ intel_legacy_cursor_update(struct drm_plane *_plane,intel_plane_disable_arm (NULL , plane, crtc_state); } -local_irq_enable (); +if (!IS_ENABLED (CONFIG_PREEMPT_RT )) +local_irq_enable ();intel_psr_unlock (crtc_state);
drivers/gpu/drm/i915/display/intel_vblank.c +4 -2
@@ -761,11 +761,13 @@ int intel_vblank_evade(struct intel_vblank_evade_ctx *evade)break ; } -local_irq_enable (); +if (!IS_ENABLED (CONFIG_PREEMPT_RT )) +local_irq_enable (); timeout =schedule_timeout (timeout); -local_irq_disable (); +if (!IS_ENABLED (CONFIG_PREEMPT_RT )) +local_irq_disable (); }finish_wait (wq, &wait);
Signature · 1 line
2.51.0
PATCH 4/5 drm/i915/display: Disable interrupts on xe for realtime preemption
In reply to Maarten LankhorstCheck if it works at all, and then see what the timing results are.
--- drivers/gpu/drm/i915/display/intel_crtc.c | 12 +++++++----- drivers/gpu/drm/i915/display/intel_vblank.c | 4 ++-- drivers/gpu/drm/i915/display/intel_vblank.h | 9 +++++++++ 3 files changed, 18 insertions(+), 7 deletions(-)
drivers/gpu/drm/i915/display/intel_crtc.c +7 -5
@@ -567,7 +567,7 @@ void intel_pipe_update_start(struct intel_atomic_state *state,*/ intel_psr_wait_for_idle_locked (new_crtc_state); -if (!IS_ENABLED (CONFIG_PREEMPT_RT )) +if (intel_disable_vblank_irqs ())local_irq_disable (); crtc->debug.min_vbl = evade.min;
@@ -586,7 +586,7 @@ void intel_pipe_update_start(struct intel_atomic_state *state,return ; irq_disable: -if (!IS_ENABLED (CONFIG_PREEMPT_RT )) +if (intel_disable_vblank_irqs ())local_irq_disable (); }
@@ -686,6 +686,10 @@ void intel_pipe_update_end(struct intel_atomic_state *state, intel_crtc_has_type(new_crtc_state, INTEL_OUTPUT_DSI))icl_dsi_frame_update (new_crtc_state); +preempt_disable (); +if (intel_disable_vblank_irqs ()) +local_irq_enable (); +/* We're still in the vblank-evade critical section, this can't race. * Would be slightly nice to just grab the vblank count and arm the * event outside of the critical section - the spinlock might spin for a
@@ -733,9 +737,7 @@ void intel_pipe_update_end(struct intel_atomic_state *state,if (!state->base.legacy_cursor_update)intel_vrr_send_push (NULL , new_crtc_state); -if (!IS_ENABLED (CONFIG_PREEMPT_RT )) -local_irq_enable (); - +preempt_enable ();if (intel_vgpu_active (dev_priv)) goto out;
drivers/gpu/drm/i915/display/intel_vblank.c +2 -2
@@ -761,12 +761,12 @@ int intel_vblank_evade(struct intel_vblank_evade_ctx *evade)break ; } -if (!IS_ENABLED (CONFIG_PREEMPT_RT )) +if (intel_disable_vblank_irqs ())local_irq_enable (); timeout =schedule_timeout (timeout); -if (!IS_ENABLED (CONFIG_PREEMPT_RT )) +if (intel_disable_vblank_irqs ())local_irq_disable (); }
drivers/gpu/drm/i915/display/intel_vblank.h +9 -0
@@ -50,4 +50,13 @@ intel_pre_commit_crtc_state(struct intel_atomic_state *state,int intel_crtc_vblank_length (const struct intel_crtc_state *crtc_state); +static inline int intel_disable_vblank_irqs (void ) +{ +#if !defined(I915 ) && !IS_ENABLED (CONFIG_PREEMPT_RT ) +return true; +#else +return false; +#endif +} +#endif /* __INTEL_VBLANK_H__ */
Signature · 1 line
2.51.0
PATCH 5/5 PREEMPT_RT injection
In reply to Maarten Lankhorst--- drivers/gpu/drm/i915/Kconfig.debug | 15 --------------- drivers/gpu/drm/xe/Kconfig.debug | 5 +++++ kernel/Kconfig.preempt | 3 +-- 3 files changed, 6 insertions(+), 17 deletions(-)
drivers/gpu/drm/i915/Kconfig.debug +0 -15
@@ -233,21 +233,6 @@ config DRM_I915_LOW_LEVEL_TRACEPOINTS If in doubt, say "N". -config DRM_I915_DEBUG_VBLANK_EVADE - bool "Enable extra debug warnings for vblank evasion" - depends on DRM_I915 - default n - help - Choose this option to turn on extra debug warnings for the - vblank evade mechanism. This gives a warning every time the - the deadline allotted for the vblank evade critical section - is exceeded, even if there isn't an actual risk of missing - the vblank. - - Recommended for driver developers only. - - If in doubt, say "N". - config DRM_I915_DEBUG_RUNTIME_PM bool "Enable extra state checking for runtime PM" depends on DRM_I915
drivers/gpu/drm/xe/Kconfig.debug +5 -0
@@ -30,6 +30,11 @@ config DRM_XE_DEBUG If in doubt, say "N". +config DRM_I915_DEBUG_VBLANK_EVADE + def_bool y + depends on DRM_XE + + config DRM_XE_DEBUG_VM bool "Enable extra VM debugging info" default n
kernel/Kconfig.preempt +1 -2
@@ -87,8 +87,7 @@ config PREEMPT_LAZY endchoice config PREEMPT_RT - bool "Fully Preemptible Kernel (Real-Time)" - depends on EXPERT && ARCH_SUPPORTS_RT && !COMPILE_TEST + def_bool y select PREEMPTION help This option turns the kernel into a real-time kernel by replacing
Signature · 1 line
2.51.0
Patchwork
CI✗ CI.checkpatch: warning for Testing PREEMPT_RT with disabling interrupts in the most critical section.
In reply to Maarten Lankhorst== Series Details ==
== Series Details ==
Series: Testing PREEMPT_RT with disabling interrupts in the most critical section.
URL : https://patchwork.freedesktop.org/series/156134/
State : warning
== Summary ==
+ KERNEL=/kernel
+ git clone https://gitlab.freedesktop.org/drm/maintainer-tools mt Cloning into 'mt'...
warning: redirecting to https://gitlab.freedesktop.org/drm/maintainer-tools.git/
+ git -C mt rev-list -n1 origin/master fbd08a78c3a3bb17964db2a326514c69c1dca660
+ cd /kernel
+ git config --global --add safe.directory /kernel
+ git log -n1
commit 750bd54e8a8317d7e60916aa89ca790d949a4ee6
Author: Maarten Lankhorst <[email protected]> Date: Fri Oct 17 17:37:48 2025 +0200
PREEMPT_RT injection
+ /mt/dim checkpatch b5e5976a35cb0e8e45aea836b42ecccf22df803f drm-intel1f74491ecafc drm/xe: Bump xe_device_l2_flush even highercd53ffd9ad75 drm/i915: Use preempt_disable/enable_rt() where recommended
-:7: WARNING:COMMIT_LOG_LONG_LINE: Prefer a maximum 75 chars per line (possible unwrapped commit description?) #7:
ad3543ede630f ("drm/intel: Push get_scanout_position() timestamping into kms driver.")
-:45: WARNING:LINE_SPACING: Missing a blank line after declarations #45: FILE: drivers/gpu/drm/i915/display/intel_vblank.c:323:
+ struct drm_i915_private *i915 = to_i915(display->drm);
+ spin_lock_irqsave(&i915->uncore.lock, *flags);
-:52: WARNING:LINE_SPACING: Missing a blank line after declarations #52: FILE: drivers/gpu/drm/i915/display/intel_vblank.c:330:
+ struct drm_i915_private *i915 = to_i915(display->drm);
+ spin_unlock_irqrestore(&i915->uncore.lock, flags);
total: 0 errors, 3 warnings, 0 checks, 78 lines checkedd0c62abf4505 drm/i915: Don't disable interrupts on PREEMPT_RT during atomic updates
-:10: WARNING:COMMIT_LOG_LONG_LINE: Prefer a maximum 75 chars per line (possible unwrapped commit description?) #10:
started disabling interrupts across atomic updates. This breaks on PREEMPT_RT
total: 0 errors, 1 warnings, 0 checks, 68 lines checked 25cad8154ec0 drm/i915/display: Disable interrupts on xe for realtime preemption -:38: ERROR:CODE_INDENT: code indent should use tabs where possible #38: FILE: drivers/gpu/drm/i915/display/intel_crtc.c:690:
+ if (intel_disable_vblank_irqs())$
-:38: WARNING:LEADING_SPACE: please, no spaces at the start of a line #38: FILE: drivers/gpu/drm/i915/display/intel_crtc.c:690:
+ if (intel_disable_vblank_irqs())$
total: 1 errors, 1 warnings, 0 checks, 63 lines checked 750bd54e8a83 PREEMPT_RT injection
-:7: WARNING:COMMIT_MESSAGE: Missing commit description - Add an appropriate one
total: 0 errors, 1 warnings, 0 checks, 41 lines checked
Patchwork
CI✗ CI.KUnit: failure for Testing PREEMPT_RT with disabling interrupts in the most critical section.
In reply to Maarten Lankhorst== Series Details ==
== Series Details ==
Series: Testing PREEMPT_RT with disabling interrupts in the most critical section.
URL : https://patchwork.freedesktop.org/series/156134/
State : failure
== Summary ==
+ trap cleanup EXIT
+ /kernel/tools/testing/kunit/kunit.py run --kunitconfig /kernel/drivers/gpu/drm/xe/.kunitconfig
ERROR:root:Not all Kconfig options selected in kunitconfig were in the generated .config.
This is probably due to unsatisfied dependencies.
Missing: CONFIG_DEBUG_MUTEXES=y, # CONFIG_DRM_XE_DISPLAY is not set
Note: many Kconfig options aren't available on UML. You can try running on a different architecture with something like "--arch=x86_64". [16:37:35] Configuring KUnit Kernel ...
Generating .config ...
Populating config with:
$ make ARCH=um O=.kunit olddefconfig + cleanup ++ stat -c %u:%g /kernel + chown -R 1003:1003 /kernel