Test PREEMPT_RT with irqs disabled.
cover Test PREEMPT_RT with irqs disabled.
Just a quick check to see if PREEMPT_RT runs into any issues without IRQs.
Maarten Lankhorst (3):
drm/xe: Bump xe_device_l2_flush even higher drm/i915/display: Disable interrupts on xe for realtime preemption [FOR-CI] PREEMPT_RT injection
Mike Galbraith (2):
drm/i915: Use preempt_disable/enable_rt() where recommended drm/i915: Don't disable interrupts on PREEMPT_RT during atomic updates
drivers/gpu/drm/i915/Kconfig.debug | 15 ------- drivers/gpu/drm/i915/display/intel_crtc.c | 13 ++++-- drivers/gpu/drm/i915/display/intel_cursor.c | 9 ++-- drivers/gpu/drm/i915/display/intel_vblank.c | 49 ++++++++++++++++----- drivers/gpu/drm/i915/display/intel_vblank.h | 9 ++++ drivers/gpu/drm/xe/Kconfig.debug | 5 +++ drivers/gpu/drm/xe/xe_device.c | 4 +- kernel/Kconfig.preempt | 3 +- 8 files changed, 69 insertions(+), 38 deletions(-)
Signature · 1 line
2.51.0
PATCH 1/5 drm/xe: Bump xe_device_l2_flush even higher
It turns out for some applications 1 ms is not enough, and 2 ms is needed.
Failing to flush causes a catastrophic display update failure, so we should prevent it if possible.
Add an even bigger margin of 5 ms, and complain loudly if we ever exceed it.
--- drivers/gpu/drm/xe/xe_device.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/gpu/drm/xe/xe_device.c +2 -2
@@ -1077,8 +1077,8 @@ void xe_device_l2_flush(struct xe_device *xe)spin_lock (>->global_invl_lock);xe_mmio_write32 (>->mmio,XE2_GLOBAL_INVAL ,0x1 ); -if (xe_mmio_wait32 (>->mmio,XE2_GLOBAL_INVAL ,0x1 ,0x0 ,1000 ,NULL , true)) -xe_gt_err_once (gt,"Global invalidation timeout\n" ); +if (xe_mmio_wait32 (>->mmio,XE2_GLOBAL_INVAL ,0x1 ,0x0 ,5000 ,NULL , true)) +xe_gt_err (gt,"Global invalidation timeout\n" );spin_unlock (>->global_invl_lock);
Signature · 1 line
2.51.0
PATCH 2/5 drm/i915: Use preempt_disable/enable_rt() where recommended
In reply to Maarten LankhorstFrom: Mike Galbraith <[email protected]>
Mario Kleiner suggest in commit
ad3543ede630f ("drm/intel: Push get_scanout_position() timestamping into kms driver.")
a spots where preemption should be disabled on PREEMPT_RT. The difference is that on PREEMPT_RT the intel_uncore::lock disables neither preemption nor interrupts and so region remains preemptible.
The area covers only register reads and writes. The part that worries me is:
- __intel_get_crtc_scanline() the worst case is 100us if no match is found.
- intel_crtc_scanlines_since_frame_timestamp() not sure how long this may take in the worst case.
It was in the RT queue for a while and nobody complained. Disable preemption on PREEPMPT_RT during timestamping.
[bigeasy: patch description.]
--- drivers/gpu/drm/i915/display/intel_vblank.c | 43 ++++++++++++++++----- 1 file changed, 33 insertions(+), 10 deletions(-)
drivers/gpu/drm/i915/display/intel_vblank.c +33 -10
@@ -315,6 +315,20 @@ static void intel_vblank_section_exit(struct intel_display *display)struct drm_i915_private *i915 =to_i915 (display->drm);spin_unlock (&i915->uncore.lock); } + +static void intel_vblank_section_enter_irqf (struct intel_display *display,unsigned long *flags) + __acquires(i915->uncore.lock) +{ +struct drm_i915_private *i915 =to_i915 (display->drm); +spin_lock_irqsave (&i915->uncore.lock, *flags); +} + +static void intel_vblank_section_exit_irqf (struct intel_display *display,unsigned long flags) + __releases(i915->uncore.lock) +{ +struct drm_i915_private *i915 =to_i915 (display->drm); +spin_unlock_irqrestore (&i915->uncore.lock, flags); +}#else static void intel_vblank_section_enter (struct intel_display *display) {
@@ -323,6 +337,17 @@ static void intel_vblank_section_enter(struct intel_display *display)static void intel_vblank_section_exit (struct intel_display *display) { } + +static void intel_vblank_section_enter_irqf (struct intel_display *display,unsigned long *flags) +{ + *flags =0 ; +} + +static void intel_vblank_section_exit_irqf (struct intel_display *display,unsigned long flags) +{ +if (flags) +return ; +}#endif static bool i915_get_crtc_scanoutpos(struct drm_crtc *_crtc,
@@ -359,10 +384,10 @@ static bool i915_get_crtc_scanoutpos(struct drm_crtc *_crtc,* timing critical raw register reads, potentially with * preemption disabled, so the following code must not block. */ -local_irq_save (irqflags); -intel_vblank_section_enter (display); +intel_vblank_section_enter_irqf (display, &irqflags); -/* preempt_disable_rt() should go right here in PREEMPT_RT patchset. */ +if (IS_ENABLED (CONFIG_PREEMPT_RT )) +preempt_disable ();/* Get optional system timestamp before query. */ if (stime)
@@ -426,10 +451,10 @@ static bool i915_get_crtc_scanoutpos(struct drm_crtc *_crtc,if (etime) *etime =ktime_get (); -/* preempt_enable_rt() should go right here in PREEMPT_RT patchset. */ +if (IS_ENABLED (CONFIG_PREEMPT_RT )) +preempt_enable (); -intel_vblank_section_exit (display); -local_irq_restore (irqflags); +intel_vblank_section_exit_irqf (display, irqflags);/* * While in vblank, position will be negative
@@ -467,13 +492,11 @@ int intel_get_crtc_scanline(struct intel_crtc *crtc)unsigned long irqflags;int position; -local_irq_save (irqflags); -intel_vblank_section_enter (display); +intel_vblank_section_enter_irqf (display, &irqflags); position =__intel_get_crtc_scanline (crtc); -intel_vblank_section_exit (display); -local_irq_restore (irqflags); +intel_vblank_section_exit_irqf (display, irqflags);return position; }
Signature · 1 line
2.51.0
PATCH 3/5 drm/i915: Don't disable interrupts on PREEMPT_RT during atomic updates
In reply to Maarten LankhorstFrom: Mike Galbraith <[email protected]>
Commit
8d7849db3eab7 ("drm/i915: Make sprite updates atomic")
started disabling interrupts across atomic updates. This breaks on PREEMPT_RT because within this section the code attempt to acquire spinlock_t locks which are sleeping locks on PREEMPT_RT.
According to the comment the interrupts are disabled to avoid random delays and not required for protection or synchronisation.
If this needs to happen with disabled interrupts on PREEMPT_RT, and the whole section is restricted to register access then all sleeping locks need to be acquired before interrupts are disabled and some function maybe moved after enabling interrupts again.
This includes:
- prepare_to_wait() + finish_wait() due its wake queue.
- drm_crtc_vblank_put() -> vblank_disable_fn() drm_device::vbl_lock.
- skl_pfit_enable(), intel_update_plane(), vlv_atomic_update_fifo() and maybe others due to intel_uncore::lock
- drm_crtc_arm_vblank_event() due to drm_device::event_lock and drm_device::vblank_time_lock.
Don't disable interrupts on PREEMPT_RT during atomic updates.
[bigeasy: drop local locks, commit message]
--- drivers/gpu/drm/i915/display/intel_crtc.c | 9 ++++++--- drivers/gpu/drm/i915/display/intel_cursor.c | 9 ++++++--- drivers/gpu/drm/i915/display/intel_vblank.c | 6 ++++-- 3 files changed, 16 insertions(+), 8 deletions(-)
drivers/gpu/drm/i915/display/intel_crtc.c +6 -3
@@ -567,7 +567,8 @@ void intel_pipe_update_start(struct intel_atomic_state *state,*/ intel_psr_wait_for_idle_locked (new_crtc_state); -local_irq_disable (); +if (!IS_ENABLED (CONFIG_PREEMPT_RT )) +local_irq_disable (); crtc->debug.min_vbl = evade.min; crtc->debug.max_vbl = evade.max;
@@ -585,7 +586,8 @@ void intel_pipe_update_start(struct intel_atomic_state *state,return ; irq_disable: -local_irq_disable (); +if (!IS_ENABLED (CONFIG_PREEMPT_RT )) +local_irq_disable (); }#if IS_ENABLED (CONFIG_DRM_I915_DEBUG_VBLANK_EVADE )
@@ -731,7 +733,8 @@ void intel_pipe_update_end(struct intel_atomic_state *state,if (!state->base.legacy_cursor_update)intel_vrr_send_push (NULL , new_crtc_state); -local_irq_enable (); +if (!IS_ENABLED (CONFIG_PREEMPT_RT )) +local_irq_enable ();if (intel_vgpu_active (dev_priv)) goto out;
drivers/gpu/drm/i915/display/intel_cursor.c +6 -3
@@ -919,13 +919,15 @@ intel_legacy_cursor_update(struct drm_plane *_plane,*/ intel_psr_wait_for_idle_locked (crtc_state); -local_irq_disable (); +if (!IS_ENABLED (CONFIG_PREEMPT_RT )) +local_irq_disable ();intel_vblank_evade (&evade);drm_crtc_vblank_put (&crtc->base); }else { -local_irq_disable (); +if (!IS_ENABLED (CONFIG_PREEMPT_RT )) +local_irq_disable (); }if (new_plane_state->uapi.visible) {
@@ -935,7 +937,8 @@ intel_legacy_cursor_update(struct drm_plane *_plane,intel_plane_disable_arm (NULL , plane, crtc_state); } -local_irq_enable (); +if (!IS_ENABLED (CONFIG_PREEMPT_RT )) +local_irq_enable ();intel_psr_unlock (crtc_state);
drivers/gpu/drm/i915/display/intel_vblank.c +4 -2
@@ -761,11 +761,13 @@ int intel_vblank_evade(struct intel_vblank_evade_ctx *evade)break ; } -local_irq_enable (); +if (!IS_ENABLED (CONFIG_PREEMPT_RT )) +local_irq_enable (); timeout =schedule_timeout (timeout); -local_irq_disable (); +if (!IS_ENABLED (CONFIG_PREEMPT_RT )) +local_irq_disable (); }finish_wait (wq, &wait);
Signature · 1 line
2.51.0
PATCH 4/5 drm/i915/display: Disable interrupts on xe for realtime preemption
In reply to Maarten LankhorstCheck if it works at all, and then see what the timing results are.
--- drivers/gpu/drm/i915/display/intel_crtc.c | 12 +++++++----- drivers/gpu/drm/i915/display/intel_vblank.c | 4 ++-- drivers/gpu/drm/i915/display/intel_vblank.h | 9 +++++++++ 3 files changed, 18 insertions(+), 7 deletions(-)
drivers/gpu/drm/i915/display/intel_crtc.c +7 -5
@@ -567,7 +567,7 @@ void intel_pipe_update_start(struct intel_atomic_state *state,*/ intel_psr_wait_for_idle_locked (new_crtc_state); -if (!IS_ENABLED (CONFIG_PREEMPT_RT )) +if (intel_disable_vblank_irqs ())local_irq_disable (); crtc->debug.min_vbl = evade.min;
@@ -586,7 +586,7 @@ void intel_pipe_update_start(struct intel_atomic_state *state,return ; irq_disable: -if (!IS_ENABLED (CONFIG_PREEMPT_RT )) +if (intel_disable_vblank_irqs ())local_irq_disable (); }
@@ -686,6 +686,10 @@ void intel_pipe_update_end(struct intel_atomic_state *state, intel_crtc_has_type(new_crtc_state, INTEL_OUTPUT_DSI))icl_dsi_frame_update (new_crtc_state); +preempt_disable (); +if (intel_disable_vblank_irqs ()) +local_irq_enable (); +/* We're still in the vblank-evade critical section, this can't race. * Would be slightly nice to just grab the vblank count and arm the * event outside of the critical section - the spinlock might spin for a
@@ -733,9 +737,7 @@ void intel_pipe_update_end(struct intel_atomic_state *state,if (!state->base.legacy_cursor_update)intel_vrr_send_push (NULL , new_crtc_state); -if (!IS_ENABLED (CONFIG_PREEMPT_RT )) -local_irq_enable (); - +preempt_enable ();if (intel_vgpu_active (dev_priv)) goto out;
drivers/gpu/drm/i915/display/intel_vblank.c +2 -2
@@ -761,12 +761,12 @@ int intel_vblank_evade(struct intel_vblank_evade_ctx *evade)break ; } -if (!IS_ENABLED (CONFIG_PREEMPT_RT )) +if (intel_disable_vblank_irqs ())local_irq_enable (); timeout =schedule_timeout (timeout); -if (!IS_ENABLED (CONFIG_PREEMPT_RT )) +if (intel_disable_vblank_irqs ())local_irq_disable (); }
drivers/gpu/drm/i915/display/intel_vblank.h +9 -0
@@ -50,4 +50,13 @@ intel_pre_commit_crtc_state(struct intel_atomic_state *state,int intel_crtc_vblank_length (const struct intel_crtc_state *crtc_state); +static inline int intel_disable_vblank_irqs (void ) +{ +#if !defined(I915 ) && !IS_ENABLED (CONFIG_PREEMPT_RT ) +return true; +#else +return false; +#endif +} +#endif /* __INTEL_VBLANK_H__ */
Signature · 1 line
2.51.0
PATCH 5/5 PREEMPT_RT injection
In reply to Maarten Lankhorst--- drivers/gpu/drm/i915/Kconfig.debug | 15 --------------- drivers/gpu/drm/xe/Kconfig.debug | 5 +++++ kernel/Kconfig.preempt | 3 +-- 3 files changed, 6 insertions(+), 17 deletions(-)
drivers/gpu/drm/i915/Kconfig.debug +0 -15
@@ -233,21 +233,6 @@ config DRM_I915_LOW_LEVEL_TRACEPOINTS If in doubt, say "N". -config DRM_I915_DEBUG_VBLANK_EVADE - bool "Enable extra debug warnings for vblank evasion" - depends on DRM_I915 - default n - help - Choose this option to turn on extra debug warnings for the - vblank evade mechanism. This gives a warning every time the - the deadline allotted for the vblank evade critical section - is exceeded, even if there isn't an actual risk of missing - the vblank. - - Recommended for driver developers only. - - If in doubt, say "N". - config DRM_I915_DEBUG_RUNTIME_PM bool "Enable extra state checking for runtime PM" depends on DRM_I915
drivers/gpu/drm/xe/Kconfig.debug +5 -0
@@ -30,6 +30,11 @@ config DRM_XE_DEBUG If in doubt, say "N". +config DRM_I915_DEBUG_VBLANK_EVADE + def_bool y + depends on DRM_XE + + config DRM_XE_DEBUG_VM bool "Enable extra VM debugging info" default n
kernel/Kconfig.preempt +1 -2
@@ -87,8 +87,7 @@ config PREEMPT_LAZY endchoice config PREEMPT_RT - bool "Fully Preemptible Kernel (Real-Time)" - depends on EXPERT && ARCH_SUPPORTS_RT && !COMPILE_TEST + def_bool y select PREEMPTION help This option turns the kernel into a real-time kernel by replacing
Signature · 1 line
2.51.0
Patchwork
CI✗ CI.checkpatch: warning for Test PREEMPT_RT with irqs disabled.
In reply to Maarten Lankhorst== Series Details ==
== Series Details ==
Series: Test PREEMPT_RT with irqs disabled.
URL : https://patchwork.freedesktop.org/series/156133/
State : warning
== Summary ==
+ KERNEL=/kernel
+ git clone https://gitlab.freedesktop.org/drm/maintainer-tools mt Cloning into 'mt'...
warning: redirecting to https://gitlab.freedesktop.org/drm/maintainer-tools.git/
+ git -C mt rev-list -n1 origin/master fbd08a78c3a3bb17964db2a326514c69c1dca660
+ cd /kernel
+ git config --global --add safe.directory /kernel
+ git log -n1
commit 97088f2ad46b0ca01f2e89d12fe95e33e9eca794
Author: Maarten Lankhorst <[email protected]> Date: Fri Oct 17 13:57:51 2025 +0200
PREEMPT_RT injection
+ /mt/dim checkpatch b5e5976a35cb0e8e45aea836b42ecccf22df803f drm-intel18a585fb5ae1 drm/xe: Bump xe_device_l2_flush even higherb30fb2cc3886 drm/i915: Use preempt_disable/enable_rt() where recommended
-:7: WARNING:COMMIT_LOG_LONG_LINE: Prefer a maximum 75 chars per line (possible unwrapped commit description?) #7:
ad3543ede630f ("drm/intel: Push get_scanout_position() timestamping into kms driver.")
-:45: WARNING:LINE_SPACING: Missing a blank line after declarations #45: FILE: drivers/gpu/drm/i915/display/intel_vblank.c:323:
+ struct drm_i915_private *i915 = to_i915(display->drm);
+ spin_lock_irqsave(&i915->uncore.lock, *flags);
-:52: WARNING:LINE_SPACING: Missing a blank line after declarations #52: FILE: drivers/gpu/drm/i915/display/intel_vblank.c:330:
+ struct drm_i915_private *i915 = to_i915(display->drm);
+ spin_unlock_irqrestore(&i915->uncore.lock, flags);
total: 0 errors, 3 warnings, 0 checks, 78 lines checkedc70c3113ccf1 drm/i915: Don't disable interrupts on PREEMPT_RT during atomic updates
-:10: WARNING:COMMIT_LOG_LONG_LINE: Prefer a maximum 75 chars per line (possible unwrapped commit description?) #10:
started disabling interrupts across atomic updates. This breaks on PREEMPT_RT
total: 0 errors, 1 warnings, 0 checks, 68 lines checked b2d54cca85bf drm/i915/display: Disable interrupts on xe for realtime preemption -:38: ERROR:CODE_INDENT: code indent should use tabs where possible #38: FILE: drivers/gpu/drm/i915/display/intel_crtc.c:690:
+ if (intel_disable_vblank_irqs())$
-:38: WARNING:LEADING_SPACE: please, no spaces at the start of a line #38: FILE: drivers/gpu/drm/i915/display/intel_crtc.c:690:
+ if (intel_disable_vblank_irqs())$
total: 1 errors, 1 warnings, 0 checks, 63 lines checked 97088f2ad46b PREEMPT_RT injection
-:7: WARNING:COMMIT_MESSAGE: Missing commit description - Add an appropriate one
total: 0 errors, 1 warnings, 0 checks, 41 lines checked
Patchwork
CI✗ CI.KUnit: failure for Test PREEMPT_RT with irqs disabled.
In reply to Maarten Lankhorst== Series Details ==
== Series Details ==
Series: Test PREEMPT_RT with irqs disabled.
URL : https://patchwork.freedesktop.org/series/156133/
State : failure
== Summary ==
+ trap cleanup EXIT
+ /kernel/tools/testing/kunit/kunit.py run --kunitconfig /kernel/drivers/gpu/drm/xe/.kunitconfig
ERROR:root:Not all Kconfig options selected in kunitconfig were in the generated .config.
This is probably due to unsatisfied dependencies.
Missing: CONFIG_DEBUG_MUTEXES=y, # CONFIG_DRM_XE_DISPLAY is not set
Note: many Kconfig options aren't available on UML. You can try running on a different architecture with something like "--arch=x86_64". [16:32:08] Configuring KUnit Kernel ...
Generating .config ...
Populating config with:
$ make ARCH=um O=.kunit olddefconfig + cleanup ++ stat -c %u:%g /kernel + chown -R 1003:1003 /kernel