i915_hangcheck_hung problem with 3.8-rc2+ (Linus's latest tree)
Hi all,
I've hit this 3 times today on Linus's latest 3.8-rc2+ tree:
[11868.414648] [drm:i915_hangcheck_hung] *ERROR* Hangcheck timer elapsed... GPU hung [11868.414655] [drm] capturing error event; look for more information in /debug/dri/0/i915_error_state [11870.408342] [drm:i915_hangcheck_hung] *ERROR* Hangcheck timer elapsed... GPU hung [11870.408412] [drm:i915_reset] *ERROR* GPU hanging too fast, declaring wedged! [11870.408414] [drm:i915_reset] *ERROR* Failed to reset chip. [11883.083225] gnome-shell[19396]: segfault at 218 ip 00007feef5f32333 sp 00007ffffc1dc930 error 4 in i965_dri.so[7feef5ecb000+d0000]
When it happens, gnome-shell dies a horrible death and it requires a reboot in order to get xorg working properly again (probably because gnome-shell is hosed.)
The machine does still work to do other things from a text console (I'm writing this on the machine after the last time this happened.)
It seems to happen when doing a "stressful" thing on the machine (i.e. multiple kernel builds at the same time).
I also seem to be able to hit this on 3.7.1, but not as regularly, and not at all on 3.6.y.
Any hints or ideas of what to try out?
thanks,
greg k-h
Quoted 10 lines · Greg KH
Hi all,
I've hit this 3 times today on Linus's latest 3.8-rc2+ tree:
[11868.414648] [drm:i915_hangcheck_hung] *ERROR* Hangcheck timer elapsed... GPU hung [11868.414655] [drm] capturing error event; look for more information in /debug/dri/0/i915_error_state [11870.408342] [drm:i915_hangcheck_hung] *ERROR* Hangcheck timer elapsed... GPU hung [11870.408412] [drm:i915_reset] *ERROR* GPU hanging too fast, declaring wedged! [11870.408414] [drm:i915_reset] *ERROR* Failed to reset chip. [11883.083225] gnome-shell[19396]: segfault at 218 ip 00007feef5f32333 sp 00007ffffc1dc930 error 4 in i965_dri.so[7feef5ecb000+d0000]
I just hit this again. And, as the kernel was asking for it, attached is the i915_error_state file, compressed due to the size of it.
thanks,
greg k-h
Quoted 11 lines
Quoted 10 lines
Hi all,
I've hit this 3 times today on Linus's latest 3.8-rc2+ tree:
[11868.414648] [drm:i915_hangcheck_hung] *ERROR* Hangcheck timer elapsed... GPU hung [11868.414655] [drm] capturing error event; look for more information in /debug/dri/0/i915_error_state [11870.408342] [drm:i915_hangcheck_hung] *ERROR* Hangcheck timer elapsed... GPU hung [11870.408412] [drm:i915_reset] *ERROR* GPU hanging too fast, declaring wedged! [11870.408414] [drm:i915_reset] *ERROR* Failed to reset chip. [11883.083225] gnome-shell[19396]: segfault at 218 ip 00007feef5f32333 sp 00007ffffc1dc930 error 4 in i965_dri.so[7feef5ecb000+d0000]
Quoted 3 lines
I just hit this again. And, as the kernel was asking for it, attached is the
i915_error_statefile, compressed due to the size of it.
Welcome to sink hole that is
https://bugs.freedesktop.org/show_bug.cgi?id=55984
3 months and ticking, Intel guys are all running away from it saying they can't reproduce, everyone else on planet seems to reproduce quite easily.
Its generally considered a bug in the relocation/shrinker/no idea category,
Assuming you have an Ironlake machine which I'm going to guess you do.
Dave.
Quoted 14 lines · Dave Airlie
Quoted 11 lines
Quoted 10 lines
Hi all,
I've hit this 3 times today on Linus's latest 3.8-rc2+ tree:
[11868.414648] [drm:i915_hangcheck_hung] *ERROR* Hangcheck timer elapsed... GPU hung [11868.414655] [drm] capturing error event; look for more information in /debug/dri/0/i915_error_state [11870.408342] [drm:i915_hangcheck_hung] *ERROR* Hangcheck timer elapsed... GPU hung [11870.408412] [drm:i915_reset] *ERROR* GPU hanging too fast, declaring wedged! [11870.408414] [drm:i915_reset] *ERROR* Failed to reset chip. [11883.083225] gnome-shell[19396]: segfault at 218 ip 00007feef5f32333 sp 00007ffffc1dc930 error 4 in i965_dri.so[7feef5ecb000+d0000]Quoted 3 lines
I just hit this again. And, as the kernel was asking for it, attached is the
i915_error_statefile, compressed due to the size of it.
Quoted 8 lines
Welcome to sink hole that is
https://bugs.freedesktop.org/show_bug.cgi?id=559843 months and ticking, Intel guys are all running away from it saying they can't reproduce, everyone else on planet seems to reproduce quite easily.
Its generally considered a bug in the relocation/shrinker/no idea category,
Ugh, what a mess.
Quoted 1 line
Assuming you have an Ironlake machine which I'm going to guess you do.
I don't know, it's an old i5 machine that has never had any video problems for many years now. How do I tell?
thanks,
greg k-h
Quoted 22 lines · Greg KH
Quoted 21 lines · Dave Airlie
Quoted 11 lines
Quoted 10 lines
Hi all,
I've hit this 3 times today on Linus's latest 3.8-rc2+ tree:
[11868.414648] [drm:i915_hangcheck_hung] *ERROR* Hangcheck timer elapsed... GPU hung [11868.414655] [drm] capturing error event; look for more information in /debug/dri/0/i915_error_state [11870.408342] [drm:i915_hangcheck_hung] *ERROR* Hangcheck timer elapsed... GPU hung [11870.408412] [drm:i915_reset] *ERROR* GPU hanging too fast, declaring wedged! [11870.408414] [drm:i915_reset] *ERROR* Failed to reset chip. [11883.083225] gnome-shell[19396]: segfault at 218 ip 00007feef5f32333 sp 00007ffffc1dc930 error 4 in i965_dri.so[7feef5ecb000+d0000]Quoted 3 lines
I just hit this again. And, as the kernel was asking for it, attached is the
i915_error_statefile, compressed due to the size of it.Welcome to sink hole that is
https://bugs.freedesktop.org/show_bug.cgi?id=559843 months and ticking, Intel guys are all running away from it saying they can't reproduce, everyone else on planet seems to reproduce quite easily.
Quoted 8 lines
Quoted 1 line
Its generally considered a bug in the relocation/shrinker/no idea category,
Ugh, what a mess.
Quoted 1 line
Assuming you have an Ironlake machine which I'm going to guess you do.
I don't know, it's an old i5 machine that has never had any video problems for many years now. How do I tell?
lspci -nn probably an 8086:0046 device.
Old i5 probably means original i5 which means ironlake.
Dave.
Quoted 27 lines · Dave Airlie
Quoted 26 lines · Greg KH
Quoted 14 lines · Dave Airlie
Quoted 11 lines
Quoted 10 lines
Hi all,
I've hit this 3 times today on Linus's latest 3.8-rc2+ tree:
[11868.414648] [drm:i915_hangcheck_hung] *ERROR* Hangcheck timer elapsed... GPU hung [11868.414655] [drm] capturing error event; look for more information in /debug/dri/0/i915_error_state [11870.408342] [drm:i915_hangcheck_hung] *ERROR* Hangcheck timer elapsed... GPU hung [11870.408412] [drm:i915_reset] *ERROR* GPU hanging too fast, declaring wedged! [11870.408414] [drm:i915_reset] *ERROR* Failed to reset chip. [11883.083225] gnome-shell[19396]: segfault at 218 ip 00007feef5f32333 sp 00007ffffc1dc930 error 4 in i965_dri.so[7feef5ecb000+d0000]Quoted 3 lines
I just hit this again. And, as the kernel was asking for it, attached is the
i915_error_statefile, compressed due to the size of it.Quoted 8 lines
Welcome to sink hole that is
https://bugs.freedesktop.org/show_bug.cgi?id=559843 months and ticking, Intel guys are all running away from it saying they can't reproduce, everyone else on planet seems to reproduce quite easily.
Its generally considered a bug in the relocation/shrinker/no idea category,
Ugh, what a mess.
Quoted 9 lines
Quoted 4 lines
Quoted 1 line
Assuming you have an Ironlake machine which I'm going to guess you do.
I don't know, it's an old i5 machine that has never had any video problems for many years now. How do I tell?
lspci -nn probably an 8086:0046 device.
Old i5 probably means original i5 which means ironlake.
I have also seen this a couple of times on 3.7 and 3.8-rc1.
Most of the times I was watching youtube video in chrome. Nothing crashed though(I am not running gnome shell). System recovered after few seconds.
I didn't see this on 3.8-rc2 yet, probably because I haven't watched any video.
-lijo
Quoted 46 lines · Lijo Antony
Quoted 35 lines · Dave Airlie
Quoted 33 lines · Greg KH
Quoted 23 lines · Dave Airlie
Quoted 17 lines
Quoted 14 lines
Hi all,
I've hit this 3 times today on Linus's latest 3.8-rc2+ tree:
[11868.414648] [drm:i915_hangcheck_hung] *ERROR* Hangcheck timer elapsed... GPU hung [11868.414655] [drm] capturing error event; look for more information in /debug/dri/0/i915_error_state [11870.408342] [drm:i915_hangcheck_hung] *ERROR* Hangcheck timer elapsed... GPU hung [11870.408412] [drm:i915_reset] *ERROR* GPU hanging too fast, declaring wedged! [11870.408414] [drm:i915_reset] *ERROR* Failed to reset chip. [11883.083225] gnome-shell[19396]: segfault at 218 ipQuoted 2 lines
00007feef5f32333sp00007ffffc1dc930error 4 in
i965_dri.so[7feef5ecb000+d0000]Quoted 3 lines
I just hit this again. And, as the kernel was asking for it, attached is the
i915_error_statefile, compressed due to the size of it.Welcome to sink hole that is
https://bugs.freedesktop.org/show_bug.cgi?id=55984Quoted 6 lines
3 months and ticking, Intel guys are all running away from it saying they can't reproduce, everyone else on planet seems to reproduce quite easily.
Its generally considered a bug in the relocation/shrinker/no idea category,
Ugh, what a mess.
Quoted 9 lines
Quoted 4 lines
Quoted 1 line
Assuming you have an Ironlake machine which I'm going to guess you do.
I don't know, it's an old i5 machine that has never had any video problems for many years now. How do I tell?
lspci -nn probably an 8086:0046 device.
Old i5 probably means original i5 which means ironlake.
Quoted 7 lines
I have also seen this a couple of times on 3.7 and 3.8-rc1. Most of the times I was watching youtube video in chrome. Nothing crashed though(I am not running gnome shell). System recovered after few seconds.
I didn't see this on 3.8-rc2 yet, probably because I haven't watched any video.
I can easily reproduce it running glxgears on 3.8-rc1 or 3.8-rc2.
00:02.0 VGA compatible controller [0300]: Intel Corporation Core Processor Integrated Graphics Controller [8086:0046] (rev 02)
Thinkpad T410
Shaggy
Quoted 5 lines
-lijo
Quoted 9 lines · Dave Kleikamp
I can easily reproduce it running glxgears on 3.8-rc1 or 3.8-rc2.
00:02.0 VGA compatible controller [0300]: Intel Corporation Core Processor Integrated Graphics Controller [8086:0046] (rev 02)
Thinkpad T410
Shaggy
Daniel's patch:
drm/i915: Revert shrinker changes from "Track unbound pages"
fixes the problem for me.
Thanks,
Shaggy
Quoted 9 lines · Dave Kleikamp
Quoted 8 lines · Dave Kleikamp
I can easily reproduce it running glxgears on 3.8-rc1 or 3.8-rc2.
00:02.0 VGA compatible controller [0300]: Intel Corporation Core Processor Integrated Graphics Controller [8086:0046] (rev 02)
Thinkpad T410
Quoted 7 lines
Quoted 1 line
Shaggy
Daniel's patch:
drm/i915: Revert shrinker changes from "Track unbound pages"
fixes the problem for me.
Thanks for the hint, I'll go try that right now...
greg k-h
Quoted 9 lines · Dave Kleikamp
Quoted 8 lines · Dave Kleikamp
I can easily reproduce it running glxgears on 3.8-rc1 or 3.8-rc2.
00:02.0 VGA compatible controller [0300]: Intel Corporation Core Processor Integrated Graphics Controller [8086:0046] (rev 02)
Thinkpad T410
Quoted 7 lines
Quoted 1 line
Shaggy
Daniel's patch:
drm/i915: Revert shrinker changes from "Track unbound pages"
fixes the problem for me.
After an afternoon of multiple kernel builds and other stressful things, it looks like it fixes it for me as well. Chris, this will be going to Linus soon, right?
thanks,
greg k-h
Quoted 14 lines · Greg KH
Quoted 13 lines · Dave Kleikamp
Quoted 9 lines · Dave Kleikamp
I can easily reproduce it running glxgears on 3.8-rc1 or 3.8-rc2.
00:02.0 VGA compatible controller [0300]: Intel Corporation Core Processor Integrated Graphics Controller [8086:0046] (rev 02)
Thinkpad T410
Shaggy
Daniel's patch:
Quoted 7 lines
Quoted 3 lines
drm/i915: Revert shrinker changes from "Track unbound pages"
fixes the problem for me.
After an afternoon of multiple kernel builds and other stressful things, it looks like it fixes it for me as well. Chris, this will be going to Linus soon, right?
Daniel will send it on. I hope before he does so, he will clarify the changelog to note that it is just papering over the issue. If the conjecture is right, it will not prevent that path from triggering the hang, nor does it prevent other eviction paths from potentially causing the same issue.
-Chris
Signature · 1 line
Chris Wilson, Intel Open Source Technology Centre
Quoted 23 lines · Chris Wilson
Quoted 14 lines · Greg KH
Quoted 13 lines · Dave Kleikamp
Quoted 9 lines · Dave Kleikamp
I can easily reproduce it running glxgears on 3.8-rc1 or 3.8-rc2.
00:02.0 VGA compatible controller [0300]: Intel Corporation Core Processor Integrated Graphics Controller [8086:0046] (rev 02)
Thinkpad T410
Shaggy
Daniel's patch:
Quoted 7 lines
Quoted 3 lines
drm/i915: Revert shrinker changes from "Track unbound pages"
fixes the problem for me.
After an afternoon of multiple kernel builds and other stressful things, it looks like it fixes it for me as well. Chris, this will be going to Linus soon, right?
Quoted 5 lines
Daniel will send it on. I hope before he does so, he will clarify the changelog to note that it is just papering over the issue. If the conjecture is right, it will not prevent that path from triggering the hang, nor does it prevent other eviction paths from potentially causing the same issue.
In this case since the issue was papered over all the kernel up until 3.7, I think repapering is the answer for now. I have a novel idea maybe someone could spend some time working out what is broken in private on a test box instead of making everyone who runs 3.7 and 3.8 on ILK deal with it. I of course know this won't happen and I'll be reverting patches from you guys that cause Ironlake flakyness for ever.
Dave.
Quoted 9 lines · Dave Kleikamp
Quoted 8 lines · Dave Kleikamp
I can easily reproduce it running glxgears on 3.8-rc1 or 3.8-rc2.
00:02.0 VGA compatible controller [0300]: Intel Corporation Core Processor Integrated Graphics Controller [8086:0046] (rev 02)
Thinkpad T410
Quoted 7 lines
Quoted 1 line
Shaggy
Daniel's patch:
drm/i915: Revert shrinker changes from "Track unbound pages"
fixes the problem for me.
bug still kicking even w/ (drm/i915: Revert shrinker changes from "Track unbound pages")
$ glxgears
[ 429.656459] [drm:i915_hangcheck_hung] *ERROR* Hangcheck timer elapsed... GPU hung [ 429.656463] [drm] capturing error event; look for more information in /debug/dri/0/i915_error_state [ 429.665762] [drm:kick_ring] *ERROR* Kicking stuck wait on render ring
Signature · 1 line
Nikola
Quoted 2 lines · Nikola Pajkovsky
bug still kicking even w/ (drm/i915: Revert shrinker changes from "Track unbound pages")
Could be a different bug, can you please attach the error_state somewhere? -Daniel
Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation +41 (0) 79 365 57 48 - http://blog.ffwll.ch
Quoted 5 lines · Daniel Vetter
Quoted 2 lines · Nikola Pajkovsky
bug still kicking even w/ (drm/i915: Revert shrinker changes from "Track unbound pages")
Could be a different bug, can you please attach the
error_statesomewhere?
yep, i915_error_state is attached. btw, I'm going to bisect kernel, so hopefully I will bring some commit.
Signature · 2 lines
-- Nikola
Quoted 10 lines · Nikola Pajkovsky
Quoted 5 lines · Daniel Vetter
Quoted 2 lines · Nikola Pajkovsky
bug still kicking even w/ (drm/i915: Revert shrinker changes from "Track unbound pages")
Could be a different bug, can you please attach the
error_statesomewhere?yep,
i915_error_stateis attached. btw, I'm going to bisect kernel, so hopefully I will bring some commit.
Different bug, on a quick lock this could be a dupe of
https://bugzilla.kernel.org/show_bug.cgi?id=52311
Chris should know the details.
-Daniel
Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation +41 (0) 79 365 57 48 - http://blog.ffwll.ch
Quoted 7 lines · Daniel Vetter
Quoted 6 lines · Nikola Pajkovsky
Quoted 4 lines · Daniel Vetter
Quoted 2 lines · Nikola Pajkovsky
bug still kicking even w/ (drm/i915: Revert shrinker changes from "Track unbound pages")
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
Could be a different bug, can you please attach the
error_statesomewhere?yep,
i915_error_stateis attached. btw, I'm going to bisect kernel, so hopefully I will bring some commit.Different bug, on a quick lock this could be a dupe of
https://bugzilla.kernel.org/show_bug.cgi?id=52311
ok
Quoted 1 line
Chris should know the details.
thanks, bisection leads me to commit d7d4eed ("drm/i915: Allow DRM_ROOT_ONLY|DRM_MASTER to submit privileged batchbuffers). It's not possible to simply revert/test commit and I have no idea how i915 works.
Chris any ideas?
Signature · 1 line
Nikola
Quoted 19 lines · Nikola Pajkovsky
Quoted 7 lines · Daniel Vetter
Quoted 6 lines · Nikola Pajkovsky
Quoted 4 lines · Daniel Vetter
Quoted 2 lines · Nikola Pajkovsky
bug still kicking even w/ (drm/i915: Revert shrinker changes from "Track unbound pages")
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
Could be a different bug, can you please attach the
error_statesomewhere?yep,
i915_error_stateis attached. btw, I'm going to bisect kernel, so hopefully I will bring some commit.Different bug, on a quick lock this could be a dupe of
https://bugzilla.kernel.org/show_bug.cgi?id=52311ok
Quoted 7 lines
Quoted 1 line
Chris should know the details.
thanks, bisection leads me to commit d7d4eed ("drm/i915: Allow DRM_ROOT_ONLY|DRM_MASTER to submit privileged batchbuffers). It's not possible to simply revert/test commit and I have no idea how i915 works.
Chris any ideas?
Userspace is failing to prepare the GPU to execute a WAIT_FOR_EVENT command, which it can only try if the kernel allows execution of privileged batch buffers.
Option "SwapbuffersWait" "false" in xorg.conf will prevent the ddx from issuing the hanging command sequence. It is not clear yet what the missing ingredient is, I suspect the ddx needs to be more careful about not setting conditions that can never be met.
-Chris
Signature · 1 line
Chris Wilson, Intel Open Source Technology Centre