3.6.11.4-rt36
Subsystems (2)
Dear RT Folks,
I'm pleased to announce the 3.6.11.4-rt36 stable release.
This release is just an update to the new stable 3.6.11.4 version and no RT specific changes have been made.
You can get this release via the git tree at:
git://git.kernel.org/pub/scm/linux/kernel/git/rt/linux-stable-rt.git
branch: v3.6-rt
Head SHA1: 605e3eba4fbf83ad32032cec4ca4c1d2fa665793
Or to build 3.6.11.4-rt36 directly, the following patches should be applied:
http://www.kernel.org/pub/linux/kernel/v3.x/linux-3.6.tar.xz
http://www.kernel.org/pub/linux/kernel/v3.x/patch-3.6.11.xz
http://www.kernel.org/pub/linux/kernel/projects/rt/3.6/stable/patch-3.6.11.4.xz
http://www.kernel.org/pub/linux/kernel/projects/rt/3.6/patch-3.6.11.4-rt36.patch.xz
Enjoy,
-- Steve
Just did a quick "smoketest" with cyclictest. This release spikes to over 600us when opening other gnome-terminals or switching to a VTY etc. I checked with 3.6.11.3-rt35, and the problem does not occur there.
Christoph
Quoted 44 lines · Steven Rostedt
Dear RT Folks,
I'm pleased to announce the 3.6.11.4-rt36 stable release.
This release is just an update to the new stable 3.6.11.4 version and no RT specific changes have been made.
You can get this release via the git tree at:
git://git.kernel.org/pub/scm/linux/kernel/git/rt/linux-stable-rt.gitbranch: v3.6-rt
Head SHA1:605e3eba4fbf83ad32032cec4ca4c1d2fa665793Or to build 3.6.11.4-rt36 directly, the following patches should be applied:
http://www.kernel.org/pub/linux/kernel/v3.x/linux-3.6.tar.xz
http://www.kernel.org/pub/linux/kernel/v3.x/patch-3.6.11.xz
http://www.kernel.org/pub/linux/kernel/projects/rt/3.6/stable/patch-3.6.11.4.xz
http://www.kernel.org/pub/linux/kernel/projects/rt/3.6/patch-3.6.11.4-rt36.patch.xz
Enjoy,
-- Steve
--
To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Sorry, I did not provide any info at all:
- Intel i7 sandybridge with 64b kernel
- Onboard i915 with two screens is in use
As a lucky guess I tried reverting a2f03d58cb16cdcdbbe83e28a00d03254a7840ba: drm/i915: Workaround incoherence between fences and LLC across multiple CPUs.
The resulting kernel did not show the problems above and cyclictest is back to around 10us worst case on an idle system.
Christoph
A message that is not in the archive (sent off-list or not yet received): CALqGcGop=cpgSvcdmwE6QOSjo-JHBDGYpe2qyy3cxULfamgy+w@mail.gmail.com
Quoted 4 lines · Christoph Mathys
Just did a quick "smoketest" with cyclictest. This release spikes to over 600us when opening other gnome-terminals or switching to a VTY etc. I checked with 3.6.11.3-rt35, and the problem does not occur there.
I'm not able to reproduce this. Can you send me your config.
Thanks!
-- Steve
Carsten Emde
Tested-by Jon BloomfieldQuoted 6 lines · Steven Rostedt
Quoted 4 lines · Christoph Mathys
Just did a quick "smoketest" with cyclictest. This release spikes to over 600us when opening other gnome-terminals or switching to a VTY etc. I checked with 3.6.11.3-rt35, and the problem does not occur there.
I'm not able to reproduce this.
I am -> https://www.osadl.org/?id=1543#c7602. It is similarly reproducible on current 3.6 and 3.8 RT kernels. You probably won't be able to reproduce the regression unless you use an impacted graphics adapter.
Please revert until Chris Wilson (or anybody else) finds a better solution for the problem the patch wanted to fix.
<rogue>
It generally is not a good idea to unconditionally invalidate and flush the entire cache, since this will finally get rid of any remaining determinism. A mechanism must be used to ensure that only affected cache lines are treated, if any. If this is not possible, then we simply need to go without the original patch.
</rogue>
-Carsten.
Subject: drm/i915: Revert workaround incoherence between fences and LLC across multiple CPUs
Originally from: Chris Wilson <[email protected]>
Original commit 25ff1195f8a0b3724541ae7bbe331b4296de9c06 upstream.
In order to fully serialize access to the fenced region and the update to the fence register we need to take extreme measures on SNB+, and manually flush writes to memory prior to writing the fence register in conjunction with the memory barriers placed around the register write.
Fixes i-g-t/gem_fence_thrash
v2: Bring a bigger gun
v3: Switch the bigger gun for heavier bullets (Arjan van de Ven)
v4: Remove changes for working generations.
v5: Reduce to a per-cpu wbinvd() call prior to updating the fences.
v6: Rewrite comments to ellide forgotten history.
Revert because it introduces long latencies of up to 2 milliseconds in RT kernels until we find a better solution. No guns please.
Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=62191
--- i915_gem.c | 26 ++++---------------------- 1 file changed, 4 insertions(+), 22 deletions(-)
drivers/gpu/drm/i915/i915_gem.c +4 -22
@@ -2656,35 +2656,17 @@ static inline int fence_number(struct drreturn fence - dev_priv->fence_regs; } -static void i915_gem_write_fence__ipi (void *data) -{ -wbinvd (); -} -static void i915_gem_object_update_fence (struct drm_i915_gem_object *obj,struct drm_i915_fence_reg *fence,bool enable) { -struct drm_device *dev = obj->base.dev; -struct drm_i915_private *dev_priv = dev->dev_private; -int fence_reg =fence_number (dev_priv, fence); +struct drm_i915_private *dev_priv = obj->base.dev->dev_private; +int reg =fence_number (dev_priv, fence); -/* In order to fully serialize access to the fenced region and -* the update to the fence register we need to take extreme -* measures on SNB+. In theory, the write to the fence register -* flushes all memory transactions before, and coupled with the -* mb() placed around the register write we serialise all memory -* operations with respect to the changes in the tiler. Yet, on -* SNB+ we need to take a step further and emit an explicit wbinvd() -* on each processor in order to manually flush all memory -* transactions before updating the fence register. -*/ -if (HAS_LLC (obj->base.dev)) -on_each_cpu (i915_gem_write_fence__ipi,NULL ,1 ); -i915_gem_write_fence (dev, fence_reg, enable ? obj :NULL ); +i915_gem_write_fence (obj->base.dev, reg, enable ? obj :NULL );if (enable) { - obj->fence_reg = fence_reg; + obj->fence_reg = reg; fence->obj = obj;list_move_tail (&fence->lru_list, &dev_priv->mm.fence_list); }else {
PATCH v2 1/1 drm/i915: Do not flush caches on RT, print a warning instead
Invalidating and flushing all caches may introduce long latencies of up to several milliseconds. Do not execute it in PREEMPT_RT_FULL kernels, warn once instead and propose to pin all GPU renderering tasks to a single CPU, if possible.
Original commit:25ff1195f8a0b3724541ae7bbe331b4296de9c06 upstream.
Original log:
In order to fully serialize access to the fenced region and the update to the fence register we need to take extreme measures on SNB+, and manually flush writes to memory prior to writing the fence register in conjunction with the memory barriers placed around the register write.
--- i915_gem.c | 26 ++++---------------------- 1 file changed, 4 insertions(+), 22 deletions(-)
drivers/gpu/drm/i915/i915_gem.c +12 -0
@@ -2656,10 +2656,12 @@ static inline int fence_number(struct drreturn fence - dev_priv->fence_regs; } +#ifndef CONFIG_PREEMPT_RT_FULL static void i915_gem_write_fence__ipi (void *data) {wbinvd (); } +#endif static void i915_gem_object_update_fence(struct drm_i915_gem_object *obj, struct drm_i915_fence_reg *fence,
@@ -2679,8 +2681,18 @@ static void i915_gem_object_update_fence* on each processor in order to manually flush all memory * transactions before updating the fence register. */ +#ifdef CONFIG_PREEMPT_RT_FULL +if (HAS_LLC (obj->base.dev)) { + WARN_ONCE(1, "Cannot flush caches on RT" +#ifdef CONFIG_SMP + ", please pin rendering tasks to a single CPU" +#endif +"\n" ); + } +#else if (HAS_LLC (obj->base.dev))on_each_cpu (i915_gem_write_fence__ipi,NULL ,1 ); +#endif i915_gem_write_fence (dev, fence_reg, enable ? obj :NULL );if (enable) {
Re: [PATCH v2] drm/i915: Do not flush caches on RT, print a warning instead
Quoted 8 lines · Carsten Emde
Invalidating and flushing all caches may introduce long latencies of up to several milliseconds. Do not execute it in
PREEMPT_RT_FULLkernels, warn once instead and propose to pin all GPU renderering tasks to a single CPU, if possible.Original commit:
25ff1195f8a0b3724541ae7bbe331b4296de9c06upstream.
Quoted 8 lines
Original log:
In order to fully serialize access to the fenced region and the update to the fence register we need to take extreme measures on SNB+, and manually flush writes to memory prior to writing the fence register in conjunction with the memory barriers placed around the register write.Cc Chris Wilson <[email protected]>Signed-off-by Carsten Emde <[email protected]>
This fixes the problem for me on 3.6.11.5-rt37. Thanks, Carsten!
Christoph
Re: [PATCH v2] drm/i915: Do not flush caches on RT, print a warning instead
Quoted 15 lines · [Re: [PATCH v2] drm/i915: Do not flush caches on RT, print a warning instead]…
Quoted 14 lines · Carsten Emde
Invalidating and flushing all caches may introduce long latencies of up to several milliseconds. Do not execute it in
PREEMPT_RT_FULLkernels, warn once instead and propose to pin all GPU renderering tasks to a single CPU, if possible.Original commit:
25ff1195f8a0b3724541ae7bbe331b4296de9c06upstream.Original log:
In order to fully serialize access to the fenced region and the update to the fence register we need to take extreme measures on SNB+, and manually flush writes to memory prior to writing the fence register in conjunction with the memory barriers placed around the register write.
Quoted 4 lines
Quoted 2 lines
Cc Chris Wilson <[email protected]>Signed-off-by Carsten Emde <[email protected]>This fixes the problem for me on 3.6.11.5-rt37. Thanks, Carsten!
This thread reminded me of the i915 compile warning I'd seen earlier today. Fixing the warning does actually change the code produced.
I'll let you guys who have the actual hardware have the joy of reading the asm and deciding whether the change is significant or not...
Paul.
--
From 8343a1b04ce4a9462697f584d67e06cd5431fe9b Mon Sep 17 00:00:00 2001
From: Paul Gortmaker <[email protected]>
Date: Mon, 10 Jun 2013 17:55:44 -0400
Subject: [PATCH RT-3.6] i915_gem: properly annotate use of completion locks as raw
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Currently when compiling drivers/gpu/drm/i915/i915_gem.c we will get the following warnings:
drivers/gpu/drm/i915/i915_gem.c:118:3: warning: passing argument 1 of ‘rt_spin_lock’ from incompatible pointer type [enabled by default] drivers/gpu/drm/i915/i915_gem.c:1890:3: warning: passing argument 1 of ‘rt_spin_lock’ from incompatible pointer type [enabled by default]
[...]include/linux/spinlock_rt.h:21:24: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
This happens because the i915 code is going after the lock contained within a completion -- and in RT, we have the commit "completion: Use simple wait queues", which contains:
struct completion {
unsigned int done; - wait_queue_head_t wait;
+ struct swait_head wait;
and a swait_head is defined as:
struct swait_head {
raw_spinlock_t lock;
struct list_head list;
};Noting that the wait_queue_head_t's lock was not a raw lock, we have thus converted the i915 code to use a raw lock, hence the compile warnings.
These appear to be more than just cosmetic warnings, as the assembly dump of before this commit and after show some level of change:
--- /tmp/before
+++ /tmp/after
@@ -551,27 +551,27 @@
82c: e8 00 00 00 00 callq 831 <i915_mutex_lock_interruptible+0x51>
831: 48 89 c2 mov %rax,%rdx
834: 83 fa 00 cmp $0x0,%edx
- 837: 74 36 je 86f <i915_mutex_lock_interruptible+0x8f>
+ 837: 74 2f je 868 <i915_mutex_lock_interruptible+0x88>
839: 7c d7 jl 812 <i915_mutex_lock_interruptible+0x32>
83b: 8b 83 f0 2c 00 00 mov 0x2cf0(%rbx),%eax
841: 85 c0 test %eax,%eax
843: 74 c3 je 808 <i915_mutex_lock_interruptible+0x28>
845: 4c 8d ab 88 1d 00 00 lea 0x1d88(%rbx),%r13
- 84c: e8 00 00 00 00 callq 851 <i915_mutex_lock_interruptible+0x71>
- 851: 4c 89 ef mov %r13,%rdi
- 854: e8 00 00 00 00 callq 859 <i915_mutex_lock_interruptible+0x79>
- 859: 83 83 80 1d 00 00 01 addl $0x1,0x1d80(%rbx)
- 860: 4c 89 ef mov %r13,%rdi
- 863: e8 00 00 00 00 callq 868 <i915_mutex_lock_interruptible+0x88>
- 868: e8 00 00 00 00 callq 86d <i915_mutex_lock_interruptible+0x8d>
- 86d: eb 99 jmp 808 <i915_mutex_lock_interruptible+0x28>
- 86f: 48 c7 c6 00 00 00 00 mov $0x0,%rsi
- 876: 48 c7 c7 00 00 00 00 mov $0x0,%rdi
- 87d: 31 c0 xor %eax,%eax
- 87f: e8 00 00 00 00 callq 884 <i915_mutex_lock_interruptible+0xa4>
- 884: b8 fb ff ff ff mov $0xfffffffb,%eax
- 889: eb 87 jmp 812 <i915_mutex_lock_interruptible+0x32>
- 88b: 0f 1f 44 00 00 nopl 0x0(%rax,%rax,1)
+ 84c: 4c 89 ef mov %r13,%rdi
+ 84f: e8 00 00 00 00 callq 854 <i915_mutex_lock_interruptible+0x74>
+ 854: 83 83 80 1d 00 00 01 addl $0x1,0x1d80(%rbx)
+ 85b: 48 89 c6 mov %rax,%rsi
+ 85e: 4c 89 ef mov %r13,%rdi
+ 861: e8 00 00 00 00 callq 866 <i915_mutex_lock_interruptible+0x86>
+ 866: eb a0 jmp 808 <i915_mutex_lock_interruptible+0x28>
+ 868: 48 c7 c6 00 00 00 00 mov $0x0,%rsi
+ 86f: 48 c7 c7 00 00 00 00 mov $0x0,%rdi
+ 876: 31 c0 xor %eax,%eax
+ 878: e8 00 00 00 00 callq 87d <i915_mutex_lock_interruptible+0x9d>
+ 87d: b8 fb ff ff ff mov $0xfffffffb,%eax
+ 882: eb 8e jmp 812 <i915_mutex_lock_interruptible+0x32> + 884: 66 66 66 2e 0f 1f 84 data32 data32 nopw %cs:0x0(%rax,%rax,1)
+ 88b: 00 00 00 00 00
[Similar instance of 2nd lock/unlock disassembly not shown]
drivers/gpu/drm/i915/i915_gem.c +4 -4
@@ -115,9 +115,9 @@ i915_gem_wait_for_error(struct drm_device *dev)* end up waiting upon a subsequent completion event that * will never happen. */ -spin_lock_irqsave (&x->wait.lock, flags); +raw_spin_lock_irqsave (&x->wait.lock, flags); x->done++; -spin_unlock_irqrestore (&x->wait.lock, flags); +raw_spin_unlock_irqrestore (&x->wait.lock, flags); }return 0 ; }
@@ -1887,9 +1887,9 @@ i915_gem_check_wedge(struct drm_i915_private *dev_priv,unsigned long flags;/* Give the error handler a chance to run. */ -spin_lock_irqsave (&x->wait.lock, flags); +raw_spin_lock_irqsave (&x->wait.lock, flags); recovery_complete = x->done >0 ; -spin_unlock_irqrestore (&x->wait.lock, flags); +raw_spin_unlock_irqrestore (&x->wait.lock, flags);/* Non-interruptible callers can't handle -EAGAIN, hence return * -EIO unconditionally for these. */
Signature · 6 lines
1.8.1.2 -- To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2] drm/i915: Do not flush caches on RT, print a warning instead
Quoted 5 lines · Paul Gortmaker
This thread reminded me of the i915 compile warning I'd seen earlier today. Fixing the warning does actually change the code produced.
I'll let you guys who have the actual hardware have the joy of reading the asm and deciding whether the change is significant or not...
This warning looks like the open-coded some of the functions which are different on RT. In the v3.8 RT I removed the open-coded crap.
Quoted 2 lines
Paul.
Sebastian
Re: [PATCH v2] drm/i915: Do not flush caches on RT, print a warning instead
In reply to Carsten EmdeQuoted 8 lines · Carsten Emde
Invalidating and flushing all caches may introduce long latencies of up to several milliseconds. Do not execute it in
PREEMPT_RT_FULLkernels, warn once instead and propose to pin all GPU renderering tasks to a single CPU, if possible.Original commit:
25ff1195f8a0b3724541ae7bbe331b4296de9c06upstream.
Quoted 8 lines
Original log:
In order to fully serialize access to the fenced region and the update to the fence register we need to take extreme measures on SNB+, and manually flush writes to memory prior to writing the fence register in conjunction with the memory barriers placed around the register write.Cc Chris Wilson <[email protected]>Signed-off-by Carsten Emde <[email protected]>
Oh boy.
Chris, I have a few questions:
- is the wbinvd() required even on the local CPU or just the remote? According to bugzilla non-smp works fine. If so, you open code wbinvd_on_all_cpus()
- is it possible to replace the wbinvd() with clflush() ?
- is the problem going away if every process doing graphics is pinned to single CPU and the wbindv() call is avoided?
Sebastian
Re: [PATCH v2] drm/i915: Do not flush caches on RT, print a warning instead
Quoted 18 lines · Sebastian Andrzej Siewior
Quoted 8 lines · Carsten Emde
Invalidating and flushing all caches may introduce long latencies of up to several milliseconds. Do not execute it in
PREEMPT_RT_FULLkernels, warn once instead and propose to pin all GPU renderering tasks to a single CPU, if possible.Original commit:
25ff1195f8a0b3724541ae7bbe331b4296de9c06upstream.Quoted 8 lines
Original log:
In order to fully serialize access to the fenced region and the update to the fence register we need to take extreme measures on SNB+, and manually flush writes to memory prior to writing the fence register in conjunction with the memory barriers placed around the register write.Cc Chris Wilson <[email protected]>Signed-off-by Carsten Emde <[email protected]>
Quoted 7 lines
Oh boy.
Chris, I have a few questions:
- is thewbinvd()required even on the local CPU or just the remote? According to bugzilla non-smp works fine. If so, you open codewbinvd_on_all_cpus()
- is it possible to replace thewbinvd()withclflush()?
-next has an extended version of this w/a where we also bang a special register after the wbinvd - the current one isn't good enough on baytrail platforms. Thus far we haven't found anything less offensive that still works (hey, we've started with machine_stop!), but we're working together with hw engineers trying to figure out what's going on.
Quoted 2 lines
- is the problem going away if every process doing graphics is pinned to single CPU and the
wbindv()call is avoided?
Yep, the issue only happens when threads migrate while they're accessing tiled buffers thru the gtt.
-Daniel
--
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch
Re: [ANNOUNCE] 3.6.11.4-rt36 - Kernel Bug
In reply to Steven RostedtHello Steven,
with 3.6.11.4-rt36 I got on a x86 machine with a long time test very sporadical the following kernel oops. I cannot reproduce it reliable, but it occurs about once a week. Do you have any ideas?
Stack: c014c56d 00000001 f3df5c04 f3df5c08 c014c56d 00000001 f3df5c14 c004fc6e f6ff7030 f3df5c20 00200046 f6ff7030 f3df5c34 c018844a f6ff7030 f3df5c38 00000000 00000001 f3df5c01 f6ff87b4 f63c2490 f3df07fc f3df5c50 c044d1c0 Call Trace: [<c014c56d>] ? get_parent_ip+0xb/0x31 [<c014c56d>] ? get_parent_ip+0xb/0x31 [<c044fc6e>] ? add_preempt_count+0x90/0xa0 [<c018844a>] ? __call_rcu_core+0x45/0xed [<c044d1c0>] __rt_spin_lock+0x1e/0x20 [<c0147ef9>] lg_local_lock+0x20/0x23 [<c01eb5e8>] mntput_no_expire+0x15/0xfb [<c01eb6ec>] mntput+0x1e/0x20 [<c01dd2e0>] path_put+0x15/0x18 [<c01f6fdf>] free_fs_struct+0xe/0x25 [<c01f7062>] exit_fs+0x6c/0x72 [<c012aa9b>] do_exit+0x26d/0x352 [<c044e13f>] oops_end+0x94/09c [<c043f909>] no_context+0x15e/0x168 [<c043fa34>] __bad_area_nosemaphore+0x121/0x12b [<c044f689>] ? spurious_fault+0xa0/0xa0 [<c043fa83>] bad_area+0x35/0x3b [<c044f933>] do_page_fault+0x2aa/0x429 [<c044d395>] ? _raw_spin_unlock_irq+0x12/0x30 [<c014ab12>] ? finish_task_switch0x45/0xb4 [<c044c333>] ? __schedule+0x642/0x679 [<c044f689>] ? spurious_fault+0xa0/0xa0 [<c044db4a>] error_code+0x5a/0x60 [<c044f689>] ? spurious_fault+0xa0/0xa0 [<c02ba5b2>] ? selinux_inode_permission+0xb1/0x131 [<c044c7d3>] ? preempt_schedule_irq+0x34/0x4d [<c044d608>] ? resume_kernel+0x38/0x50 [<c02b7069>] security_inode_permission+0x1b/0x1d [<c01de2e4>] __inode_permission+0x91/0x96 [<c01de329>] inode_permission+0x40/0x42 [<c01de36e>] link_path_walk+0x43/0x67f [<c01e0a2b>] path_openat+0x75/0x34d [<c01e0f0a>] do_filp_open+0x21/0x5d [<c014d77a>] ? migrate_enable+0x133/0x174 [<c014d77a>] ? migrate_enable+0x133/0x174 [<c01ea5d2>] ? alloc_fd+0xc0/0xcb [<c01d4c81>] do_sys_open+0xed/x16c [<c01d4d21>] sys_open+0x21/0x29 [<c0452210>] sysenter_do_call+0x12/0x22 Code: 75 09 8d 43 04 89 43 04 89 4308 31 c9 89 f2 6a 01 89 d8 e8 8f 58 d1 ff 5f 85 c0 0f 85 73 01 00 00 8b 43 0c 83 e0 fe 39 c6 75 02 <0f> 0b 8d be a4 04 00 00 89 f8 e8 6d 07 00 00 8b 06 89 46 04 64 EIP: [<c044ccc9>] rt_spin_lock_slowlock+0x51/0x1ec SS:ESP 0068:f3df5bf0
Maybe there are typos in the call stack, because I have only a "real" screenshot.
Andreas
------------------------------------------------------------------------------------------------------
Registergericht: Traunstein / Registry Court: HRB 275 - Sitz / Head Office: Traunreut Aufsichtsratsvorsitzender / Chairman of Supervisory Board: Rainer Burkhard Geschäftsführung / Management Board: Thomas Sesselmann (Vorsitzender / Chairman), Michael Grimm, Matthias Fauser, Sebastian Tondorf
E-Mail Haftungsausschluss / E-Mail Disclaimer: http://www.heidenhain.de/disclaimer