Skip to contentSkip to conversation

capturing oopses

DISCUSSION Folkert van Heusden 9 participants 12 messages Nov 22, 2005 – Nov 28, 2005

lore
Share & tools
lore
#0 lore raw

Hi,

My 2.6.14 system occasionally crashes; gives a kernel panic. Of course I would like to report it. Now the system locks up hard so I can't copy the stacktrace. The crash dump patches mentioned in oops-tracing.txt all don't work for 2.6.14 it seems. So: what should I do? Get my digicam and take a picture of the display?

Folkert van Heusden

Signature · 6 lines
Try MultiTail! Multiple windows with logfiles, filtered with regular
expressions, colored output, etc. etc. www.vanheusden.com/multitail/
----------------------------------------------------------------------
Get your PGP/GPG key signed at www.biglumber.com!
----------------------------------------------------------------------
Phone: +31-6-41278122, PGP-key: 1F28D8AE, www.vanheusden.com
#1 lore raw
Quoted 5 lines · Folkert van Heusden

My 2.6.14 system occasionally crashes; gives a kernel panic. Of course I would like to report it. Now the system locks up hard so I can't copy the stacktrace. The crash dump patches mentioned in oops-tracing.txt all don't work for 2.6.14 it seems. So: what should I do? Get my digicam and take a picture of the display?

Try the serial port- you can get a copy of the console on ttyS0.

Signature · 2 lines
http://www.PowerDNS.com      Open source, database driven DNS Software 
http://netherlabs.nl              Open and Closed source services
#2 lore raw
Quoted 11 lines · bert hubert
Quoted 7 lines · Folkert van Heusden

My 2.6.14 system occasionally crashes; gives a kernel panic. Of course I would like to report it. Now the system locks up hard so I can't copy the stacktrace. The crash dump patches mentioned in oops-tracing.txt all don't work for 2.6.14 it seems. So: what should I do? Get my digicam and take a picture of the display?

Quoted 4 lines

Try the serial port- you can get a copy of the console on ttyS0.

netconsole works great too.

Brice

#3 lore raw
In reply to bert hubert
Quoted 6 lines
Quoted 5 lines

My 2.6.14 system occasionally crashes; gives a kernel panic. Of course I would like to report it. Now the system locks up hard so I can't copy the stacktrace. The crash dump patches mentioned in oops-tracing.txt all don't work for 2.6.14 it seems. So: what should I do? Get my digicam and take a picture of the display?

Try the serial port- you can get a copy of the console on ttyS0.

No serial ports free unfortunately.

Folkert van Heusden

Signature · 6 lines
Try MultiTail! Multiple windows with logfiles, filtered with regular
expressions, colored output, etc. etc. www.vanheusden.com/multitail/
----------------------------------------------------------------------
Get your PGP/GPG key signed at www.biglumber.com!
----------------------------------------------------------------------
Phone: +31-6-41278122, PGP-key: 1F28D8AE, www.vanheusden.com
#4 lore raw
Quoted 3 lines · Folkert van Heusden
Quoted 1 line

Try the serial port- you can get a copy of the console on ttyS0.

No serial ports free unfortunately.

Then I suggest you whip out your trusty pencil.

Signature · 2 lines
http://www.PowerDNS.com      Open source, database driven DNS Software 
http://netherlabs.nl              Open and Closed source services
#5 lore raw
In reply to Folkert van Heusden
Quoted 1 line · Folkert van Heusden

Hi,

Hi Folkert,

Quoted 5 lines

My 2.6.14 system occasionally crashes; gives a kernel panic. Of course I would like to report it. Now the system locks up hard so I can't copy the stacktrace. The crash dump patches mentioned in oops-tracing.txt all don't work for 2.6.14 it seems. So: what should I do? Get my digicam and take a picture of the display?

yes, digicams have become a common tool for reporting Oops'es.

Quoted 1 line

Folkert van Heusden

cu
Adrian

Signature · 4 lines
       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed
#6 lore raw
Quoted 6 lines · you [Adrian Bunk]
Quoted 1 line · Folkert van Heusden

Hi,

Hi Folkert,

Quoted 7 lines
Quoted 5 lines

My 2.6.14 system occasionally crashes; gives a kernel panic. Of course I would like to report it. Now the system locks up hard so I can't copy the stacktrace. The crash dump patches mentioned in oops-tracing.txt all don't work for 2.6.14 it seems. So: what should I do? Get my digicam and take a picture of the display?

yes, digicams have become a common tool for reporting Oops'es.

Speaking of which, does anybody know a feasible (as in "not too much harder than manually typing it in manually") way to OCR characters from vga text mode screen captures - or even digican shots?

The vga text mode captures are from a remote administration interface (such as HP RILOE or vmware gsx console) so they are pixel perfect and OCR should be doable. The digican shots on the other hand... Well at least it would have hack value :).

(My personal opinion is that Linus' unwillingness to include anything like kmsgdump (http://www.xenotime.net/linux/kmsgdump/) is somewhat unfortunate.)

-- v --

[email protected]

#7 lore raw
Quoted 19 lines · On Sat, 26 Nov 2005 21:33:58 +0200 Ville Herva wrote:
Quoted 6 lines · you [Adrian Bunk]
Quoted 1 line · Folkert van Heusden

Hi,

Hi Folkert,

Quoted 7 lines
Quoted 5 lines

My 2.6.14 system occasionally crashes; gives a kernel panic. Of course I would like to report it. Now the system locks up hard so I can't copy the stacktrace. The crash dump patches mentioned in oops-tracing.txt all don't work for 2.6.14 it seems. So: what should I do? Get my digicam and take a picture of the display?

yes, digicams have become a common tool for reporting Oops'es.

Speaking of which, does anybody know a feasible (as in "not too much harder than manually typing it in manually") way to OCR characters from vga text mode screen captures - or even digican shots?

Quoted 7 lines

The vga text mode captures are from a remote administration interface (such as HP RILOE or vmware gsx console) so they are pixel perfect and OCR should be doable. The digican shots on the other hand... Well at least it would have hack value :).

(My personal opinion is that Linus' unwillingness to include anything like kmsgdump (http://www.xenotime.net/linux/kmsgdump/) is somewhat unfortunate.)

BTW, status of that: it needs a little work to be more reliable. (It hangs sometimes when switching from protected to real mode.) I'm hoping that some of the APIC/IOAPIC/PIC patches that are being done for kdump will also help kmsgdump. I'll be working more on it in the next few weeks/months.

so yes, when it's working, it's very useful IMO.

---
~Randy

#8 lore raw
Quoted 28 lines · Randy.Dunlap
Quoted 18 lines · On Sat, 26 Nov 2005 21:33:58 +0200 Ville Herva wrote:
Quoted 5 lines · you [Adrian Bunk]
Quoted 1 line · Folkert van Heusden

Hi,

Hi Folkert,

Quoted 7 lines
Quoted 5 lines

My 2.6.14 system occasionally crashes; gives a kernel panic. Of course I would like to report it. Now the system locks up hard so I can't copy the stacktrace. The crash dump patches mentioned in oops-tracing.txt all don't work for 2.6.14 it seems. So: what should I do? Get my digicam and take a picture of the display?

yes, digicams have become a common tool for reporting Oops'es.

Speaking of which, does anybody know a feasible (as in "not too much harder than manually typing it in manually") way to OCR characters from vga text mode screen captures - or even digican shots?

Quoted 8 lines

The vga text mode captures are from a remote administration interface (such as HP RILOE or vmware gsx console) so they are pixel perfect and OCR should be doable. The digican shots on the other hand... Well at least it would have hack value :).

(My personal opinion is that Linus' unwillingness to include anything like kmsgdump (http://www.xenotime.net/linux/kmsgdump/) is somewhat unfortunate.)

Quoted 8 lines

BTW, status of that: it needs a little work to be more reliable. (It hangs sometimes when switching from protected to real mode.) I'm hoping that some of the APIC/IOAPIC/PIC patches that are being done for kdump will also help kmsgdump. I'll be working more on it in the next few weeks/months.

so yes, when it's working, it's very useful IMO.

You know some platforms that perserve the memory above some addresses across warm boots. For such platforms, one could reserve a buffer in that area can copy the sys log buffer to it on panic along with a bit pattern that could be searched for upon the next boot.

Additionally some platforms have flash parts that could be used as a persistant store for such data in the same manner. I've done this on an XScale platform using an ancient (2.4.10) kernel. I've always thought it would be a handy thing for the newer kernels for PC's, if we could come up with a semi-platform independent way of identifying a few pages that would survive a warm boot.

--mgross

#9 lore raw
Quoted 1 line · mgross

You know some platforms that perserve the memory above some addresses across warm boots. For such platforms, one could reserve a buffer in that area can copy the sys log buffer to it on panic along with a bit pattern that could be searched for upon the next boot.

I've worked on a few blades that had this. Serial console wasn't going to be available in the field, so being able to just dump debug information to persistent memory was really nice.

Chris

#10 lore raw
In reply to mgross
Quoted 5 lines · you [mgross]

You know some platforms that perserve the memory above some addresses across warm boots. For such platforms, one could reserve a buffer in that area can copy the sys log buffer to it on panic along with a bit pattern that could be searched for upon the next boot.

I think there was a patch to do this a couple of years ago.

The only references I can find now are:

http://groups.google.com/groups?q=%22Utility+module+to+capture+OOPS+output+over+reboot%22&hl=en&selm=fa.fbd0l7v.14hau3n%40ifi.uio.no&rnum=1)
http://marc.theaimsgroup.com/?l=linux-kernel&m=93077831203565&w=2

-- v --

[email protected]

#11 lore raw
In reply to Folkert van Heusden
Quoted 2 lines · Folkert van Heusden

the stacktrace. The crash dump patches mentioned in oops-tracing.txt all don't work for 2.6.14 it seems. So: what should I do? Get my digicam and

could you try kdump (Documentation/kdump/kdump.txt) to capture the crash dump. It is part of mainline kernel atleast for i386 architecture. It should work for 2.6.14 also.

Thanks
Maneesh

Signature · 6 lines
Maneesh Soni
Linux Technology Center, 
IBM India Software Labs,
Bangalore, India
email: [email protected]
Phone: 91-80-25044990