Skip to contentSkip to conversation

/var/log/messages doesn't have crash info when kernel gets panic/oops/crash

DISCUSSION Dharmosoth Seetharam 4 participants 16 messages Jun 12, 2009 – Jun 16, 2009 2 messages not in the archive

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

Hi,

I have encounter a problem called /var/log/messages does not contain any information on kernel crash dump.

Prior to enabling KEXEC we saw stack traces and bug info etc in /var/log/messages. After enabling KEXEC feature those info was missed.

How can we get those missed info in /var/log/messages even when KEXEC feature is enabled.

Any help will appreciate.

Thanks and Regards
Seetharam

____________________________________________________________
Receive Notifications of Incoming Messages
Easily monitor multiple email accounts & access them with a click. Visit http://www.inbox.com/notifier and check it out!

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#1 lore raw
Quoted 5 lines · Dharmosoth Seetharam

Hi,

I have encounter a problem called /var/log/messages does not contain any information on kernel crash dump.

Quoted 7 lines

Prior to enabling KEXEC we saw stack traces and bug info etc in /var/log/messages. After enabling KEXEC feature those info was missed.

How can we get those missed info in /var/log/messages even when KEXEC feature is enabled.

Any help will appreciate.

run crash on the resultant vmcore file that kexec can produce/save and get the dmesg log out of it.
alternatively you can use makedumpfile with the --dump-dmesg log

Neil

Quoted 9 lines

Thanks and Regards
Seetharam

____________________________________________________________
Receive Notifications of Incoming Messages
Easily monitor multiple email accounts & access them with a click. Visit http://www.inbox.com/notifier and check it out!

Quoted 4 lines

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#2 lore raw

Hi Neil,

Thanks for your quick reply.

What you have suggested is nice when we got/save crash dump (i.e vmcore)
What if we failed to save/capture vmcore info or vmcore is corrupted etc. then we dont get any information regarding the panic/crash/oops at anywhere?

Thanks and Regards

Seetharam

Quoted 21 lines

-----Original Message-----
From: [email protected]
Sent: Fri, 12 Jun 2009 16:05:13 -0400
To: [email protected]
Subject: Re: /var/log/messages doesn't have crash info when kernel gets panic/oops/crash

Quoted 13 lines · Dharmosoth Seetharam

Hi,

I have encounter a problem called /var/log/messages does not contain any information
on kernel crash dump.

Prior to enabling KEXEC we saw stack traces and bug info etc in /var/log/messages.
After enabling KEXEC feature those info was missed.

How can we get those missed info in /var/log/messages even when KEXEC feature is enabled.

Quoted 22 lines
Quoted 2 lines

Any help will appreciate.

run crash on the resultant vmcore file that kexec can produce/save and get the
dmesg log out of it.
alternatively you can use makedumpfile with the --dump-dmesg log

Neil

Quoted 13 lines

Thanks and Regards
Seetharam

____________________________________________________________
Receive Notifications of Incoming Messages
Easily monitor multiple email accounts & access them with a click. Visit http://www.inbox.com/notifier and check it out!

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

____________________________________________________________
GET FREE SMILEYS FOR YOUR IM & EMAIL - Learn more at http://www.inbox.com/smileys
Works with AIM®, MSN® Messenger, Yahoo!® Messenger, ICQ®, Google Talk™ and most webmails

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#3 lore raw
Quoted 7 lines · Dharmosoth Seetharam

Hi Neil,

Thanks for your quick reply.

What you have suggested is nice when we got/save crash dump (i.e vmcore)
What if we failed to save/capture vmcore info or vmcore is corrupted etc. then we dont get any information regarding the panic/crash/oops at anywhere?

Basic questions.
1) Do you have panic on oops set?
I think that setting more than anything else will be the difference in what shows up in /var/log/messages.
2) Have you tried a dry run and confirmed you can get a crash dump?
3) Are you using kvm

Eric

Quoted 44 lines

Thanks and Regards

Seetharam

Quoted 39 lines

-----Original Message-----
From: [email protected]
Sent: Fri, 12 Jun 2009 16:05:13 -0400
To: [email protected]
Subject: Re: /var/log/messages doesn't have crash info when kernel gets panic/oops/crash

Quoted 6 lines · Dharmosoth Seetharam

Hi,

I have encounter a problem called /var/log/messages does not contain any information
on kernel crash dump.

Quoted 9 lines

Prior to enabling KEXEC we saw stack traces and bug info etc in /var/log/messages.
After enabling KEXEC feature those info was missed.

How can we get those missed info in /var/log/messages even when KEXEC feature is enabled.

Any help will appreciate.

run crash on the resultant vmcore file that kexec can produce/save and get the
dmesg log out of it.
alternatively you can use makedumpfile with the --dump-dmesg log

Neil

Quoted 9 lines

Thanks and Regards
Seetharam

____________________________________________________________
Receive Notifications of Incoming Messages
Easily monitor multiple email accounts & access them with a click. Visit http://www.inbox.com/notifier and check it out!

Quoted 8 lines
Quoted 4 lines
Quoted 4 lines

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

____________________________________________________________
GET FREE SMILEYS FOR YOUR IM & EMAIL - Learn more at http://www.inbox.com/smileys
Works with AIM®, MSN® Messenger, Yahoo!® Messenger, ICQ®, Google Talk™ and most webmails

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#4 lore raw
Quoted 4 lines
Quoted 18 lines

-----Original Message-----
From: [email protected]
Sent: Sat, 13 Jun 2009 00:11:25 -0700
To: [email protected]
Subject: Re: /var/log/messages doesn't have crash info when kernel gets panic/oops/crash

Quoted 8 lines · Dharmosoth Seetharam

Hi Neil,

Thanks for your quick reply.

What you have suggested is nice when we got/save crash dump (i.e vmcore) What if we failed to save/capture vmcore info or vmcore is corrupted etc. then we dont get any information
regarding the panic/crash/oops at anywhere?

Hi Eric,

Quoted 4 lines

Basic questions.
1) Do you have panic on oops set?
I think that setting more than anything else will be the difference in what shows up in /var/log/messages.

Yes, in my case panic_on_oops was set with 1

Quoted 1 line

2) Have you tried a dry run and confirmed you can get a crash dump?

No, I haven't tried this.

Quoted 1 line

3) Are you using kvm

kvm? kernel virtual memory?

thanks,
Seetharam

Quoted 60 lines

Eric

Quoted 55 lines

Thanks and Regards

Seetharam

Quoted 22 lines

-----Original Message-----
From: [email protected]
Sent: Fri, 12 Jun 2009 16:05:13 -0400
To: [email protected]
Subject: Re: /var/log/messages doesn't have crash info when kernel gets panic/oops/crash

Quoted 14 lines · Dharmosoth Seetharam

Hi,

I have encounter a problem called /var/log/messages does not contain any
information
on kernel crash dump.

Prior to enabling KEXEC we saw stack traces and bug info etc in /var/log/messages.
After enabling KEXEC feature those info was missed.

How can we get those missed info in /var/log/messages even when KEXEC feature is enabled.

Quoted 22 lines
Quoted 2 lines

Any help will appreciate.

run crash on the resultant vmcore file that kexec can produce/save and get the
dmesg log out of it.
alternatively you can use makedumpfile with the --dump-dmesg log

Neil

Quoted 13 lines

Thanks and Regards
Seetharam

____________________________________________________________
Receive Notifications of Incoming Messages
Easily monitor multiple email accounts & access them with a click. Visit http://www.inbox.com/notifier and check it out!

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

____________________________________________________________
GET FREE SMILEYS FOR YOUR IM & EMAIL - Learn more at
http://www.inbox.com/smileys
Works with AIM®, MSN® Messenger, Yahoo!® Messenger, ICQ®, Google Talk™ and most webmails

____________________________________________________________
Receive Notifications of Incoming Messages
Easily monitor multiple email accounts & access them with a click. Visit http://www.inbox.com/notifier and check it out!

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#5 lore raw
Quoted 18 lines · Dharmosoth Seetharam
Quoted 7 lines

-----Original Message-----
From: [email protected]
Sent: Sat, 13 Jun 2009 00:11:25 -0700
To: [email protected]
Subject: Re: /var/log/messages doesn't have crash info when kernel gets panic/oops/crash

Quoted 11 lines
Quoted 8 lines · Dharmosoth Seetharam

Hi Neil,

Thanks for your quick reply.

What you have suggested is nice when we got/save crash dump (i.e vmcore) What if we failed to save/capture vmcore info or vmcore is corrupted etc. then we dont get any information
regarding the panic/crash/oops at anywhere?

Quoted 8 lines

Hi Eric,

Quoted 4 lines

Basic questions.
1) Do you have panic on oops set?
I think that setting more than anything else will be the difference in what shows up in /var/log/messages.

Yes, in my case panic_on_oops was set with 1

That is the reason thing don't show up in /var/log/messages because you panic before syslog has a chance to write the messages.

Quoted 3 lines
Quoted 1 line

2) Have you tried a dry run and confirmed you can get a crash dump?

No, I haven't tried this.

It sounds like you have not been getting the coredumps when problems happen. Running a simple test run to Alt-sysrq-c to confirm that things are basically setup ok is a good idea.

Quoted 2 lines
Quoted 1 line

3) Are you using kvm

kvm? kernel virtual memory?

Kernel virtual machine. In particular.
Hardware acceleration for virtual machines.

Eric

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#6 lore raw
Quoted 37 lines

-----Original Message-----
From: [email protected]
Sent: Sat, 13 Jun 2009 05:51:25 -0700
To: [email protected]
Subject: Re: /var/log/messages doesn't have crash info when kernel gets panic/oops/crash

Quoted 26 lines · Dharmosoth Seetharam
Quoted 7 lines

-----Original Message-----
From: [email protected]
Sent: Sat, 13 Jun 2009 00:11:25 -0700
To: [email protected]
Subject: Re: /var/log/messages doesn't have crash info when kernel gets panic/oops/crash

Quoted 12 lines
Quoted 9 lines · Dharmosoth Seetharam

Hi Neil,

Thanks for your quick reply.

What you have suggested is nice when we got/save crash dump (i.e vmcore)
What if we failed to save/capture vmcore info or vmcore is corrupted etc. then we dont get any information
regarding the panic/crash/oops at anywhere?

Hi Eric,

Quoted 4 lines

Basic questions.
1) Do you have panic on oops set?
I think that setting more than anything else will be the difference in what shows up in /var/log/messages.

Quoted 4 lines
Quoted 1 line

Yes, in my case panic_on_oops was set with 1

That is the reason thing don't show up in /var/log/messages because you panic before syslog has a chance to write the messages.

Thanks.
So, if panic_on_oops set with 0(default), we will get all the needed info like stack traces, bug info etc. in /var/log/messages then reboots.

Or do we need to apply any specific patches to get info in /var/log/messages?

Quoted 9 lines
Quoted 4 lines
Quoted 2 lines

2) Have you tried a dry run and confirmed you can get a crash dump?

No, I haven't tried this.

It sounds like you have not been getting the coredumps when problems happen. Running a simple test run to Alt-sysrq-c to confirm that things are basically setup ok is a good idea.

Sorry, I misunderstood your question.
I have gave dry run and confirmed that the dumps are getting saved in particular dir and able to analyze.

I did in both the ways like
1 - echo c > /proc/sysrq-trigger
2 -Alt -sysrq -c

thanks,
Seetharam

Quoted 7 lines
Quoted 2 lines
Quoted 1 line

3) Are you using kvm

kvm? kernel virtual memory?

Kernel virtual machine. In particular.
Hardware acceleration for virtual machines.

Eric

____________________________________________________________
FREE ONLINE PHOTOSHARING - Share your photos online with your friends and family! Visit http://www.inbox.com/photosharing to find out more!

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#7 lore raw
Quoted 25 lines · Dharmosoth Seetharam
Quoted 9 lines
Quoted 6 lines
Quoted 4 lines

Basic questions.
1) Do you have panic on oops set?
I think that setting more than anything else will be the difference in what shows up in /var/log/messages.

Yes, in my case panic_on_oops was set with 1

That is the reason thing don't show up in /var/log/messages because you panic before syslog has a chance to write the messages.

Thanks.
So, if panic_on_oops set with 0(default), we will get all the needed info like stack traces, bug info etc. in /var/log/messages then reboots.

Or do we need to apply any specific patches to get info in /var/log/messages?

Quoted 9 lines
Quoted 4 lines
Quoted 2 lines

2) Have you tried a dry run and confirmed you can get a crash dump?

No, I haven't tried this.

It sounds like you have not been getting the coredumps when problems happen. Running a simple test run to Alt-sysrq-c to confirm that things are basically setup ok is a good idea.

Quoted 6 lines

Sorry, I misunderstood your question.
I have gave dry run and confirmed that the dumps are getting saved in particular dir and able to analyze.

I did in both the ways like
1 - echo c > /proc/sysrq-trigger
2 -Alt -sysrq -c

Then unless you are having problems capturing core dumps in real failure situations it sounds like all is well with the world.

A kernel oops is normally not fatal and the kernel tries to limp along allowing for better diagnostics etc. This allows klogd to read the kernels message buffer and pass the kernel messages to syslog which writes the messages to /var/log/messages.

After a kernel panic nothing is allowed to happen which prevents user space from writing to /var/log/messages in the usual way.

If you want data in /var/log/messages it appears you have two choices.
1) Post process a core dump as Neil suggested and feed the kernel log buffer to syslog.
2) disable panic on oops. The system will continue to limp along allowing user space to write to /var/log/messages.

Eric

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#8 lore raw
Quoted 56 lines

-----Original Message-----
From: [email protected]
Sent: Sat, 13 Jun 2009 07:08:50 -0700
To: [email protected]
Subject: Re: /var/log/messages doesn't have crash info when kernel gets panic/oops/crash

Quoted 27 lines · Dharmosoth Seetharam
Quoted 9 lines
Quoted 6 lines
Quoted 4 lines

Basic questions.
1) Do you have panic on oops set?
I think that setting more than anything else will be the difference in what shows up in /var/log/messages.

Yes, in my case panic_on_oops was set with 1

That is the reason thing don't show up in /var/log/messages because you panic before syslog has a chance to write the messages.

Thanks.
So, if panic_on_oops set with 0(default), we will get all the needed info like stack traces, bug info etc. in /var/log/messages then reboots.

Or do we need to apply any specific patches to get info in /var/log/messages?

Quoted 9 lines
Quoted 4 lines
Quoted 2 lines

2) Have you tried a dry run and confirmed you can get a crash dump?

No, I haven't tried this.

It sounds like you have not been getting the coredumps when problems happen. Running a simple test run to Alt-sysrq-c to confirm that things are basically setup ok is a good idea.

Quoted 7 lines

Sorry, I misunderstood your question.
I have gave dry run and confirmed that the dumps are getting saved in particular dir and able to analyze.

I did in both the ways like
1 - echo c > /proc/sysrq-trigger
2 -Alt -sysrq -c

Then unless you are having problems capturing core dumps in real failure situations it sounds like all is well with the world.

A kernel oops is normally not fatal and the kernel tries to limp along allowing for better diagnostics etc. This allows klogd to read the kernels message buffer and pass the kernel messages to syslog which writes the messages to /var/log/messages.

After a kernel panic nothing is allowed to happen which prevents user space from writing to /var/log/messages in the usual way.

Quoted 8 lines

If you want data in /var/log/messages it appears you have two choices.
1) Post process a core dump as Neil suggested and feed the kernel log buffer to syslog.
2) disable panic on oops. The system will continue to limp along allowing
user space to write to /var/log/messages.

Eric

Thanks Eric.

I will try all the ways as you suggested and let you know the outcome.

regards,
Seetharam

____________________________________________________________
FREE ONLINE PHOTOSHARING - Share your photos online with your friends and family! Visit http://www.inbox.com/photosharing to find out more!

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#9 lore raw
In reply to Dharmosoth Seetharam
Quoted 9 lines · Dharmosoth Seetharam

Hi Neil,

Thanks for your quick reply.

What you have suggested is nice when we got/save crash dump (i.e vmcore)
What if we failed to save/capture vmcore info or vmcore is corrupted etc. then we dont get any information regarding the panic/crash/oops at anywhere?

If the vmcore is corrupted to the point where makedumpfile can't extract the dmesg log, then its certain that syslog wouldn't have been able to write anything to /var/log/messages either. As a last resort you can try run strings on the corrupted vmcore and see if the dmesg log is still in there somewhere.

If you forgot to capture the vmcore, theres not much more help I can give you, Im afraid. You'll just have to make sure you capture it next time. If you only need the backtrace, and don't want the rest of the vmcore, you can make kdump just run makedumpfile --dump-dmesg /proc/vmcore /var/log/oops.txt

Neil

Quoted 47 lines

Thanks and Regards

Seetharam

Quoted 39 lines

-----Original Message-----
From: [email protected]
Sent: Fri, 12 Jun 2009 16:05:13 -0400
To: [email protected]
Subject: Re: /var/log/messages doesn't have crash info when kernel gets panic/oops/crash

Quoted 6 lines · Dharmosoth Seetharam

Hi,

I have encounter a problem called /var/log/messages does not contain any information
on kernel crash dump.

Quoted 9 lines

Prior to enabling KEXEC we saw stack traces and bug info etc in /var/log/messages.
After enabling KEXEC feature those info was missed.

How can we get those missed info in /var/log/messages even when KEXEC feature is enabled.

Any help will appreciate.

run crash on the resultant vmcore file that kexec can produce/save and get the
dmesg log out of it.
alternatively you can use makedumpfile with the --dump-dmesg log

Neil

Quoted 9 lines

Thanks and Regards
Seetharam

____________________________________________________________
Receive Notifications of Incoming Messages
Easily monitor multiple email accounts & access them with a click. Visit http://www.inbox.com/notifier and check it out!

Quoted 8 lines
Quoted 4 lines
Quoted 4 lines

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

____________________________________________________________
GET FREE SMILEYS FOR YOUR IM & EMAIL - Learn more at http://www.inbox.com/smileys
Works with AIM®, MSN® Messenger, Yahoo!® Messenger, ICQ®, Google Talk™ and most webmails

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#10 lore raw

Hi,

I have experimented few things and results are below.

Case 1) echo 0 > /proc/sys/kernel/panic_on_oops
When BUG/oops occurred it is able to log the diagnostics to /var/log/messages and then continue(not switching to dump-saving kernel).

What I am looking is - I want both the things to be happen. that is - it should log the appropriate messages in /var/log/messsages and should be able to take kernel dump.

case 2) In general, When panic has occurred we won't log any type of messages in
/var/log/messages ... am I right ?
If KEXEC is enabled and dump-saving kernel is loaded then will switch to it and save the kernel dump.

I have added the dump_stack() function call in the panic() function before calling crash_kexec with few seconds of delay. It is behaving like randomly (ie. some time stack traces are logged in /var/log/messages and not for rest of the time)

What I am looking is - I want to log the appropriate messages in /var/log/messages and then switch to dump-saving kernel.

case 3) In general, When die/nmi_die called we used to log the appropriate messages in /var/log/message
and then saving kernel dump will be depends on KEXEC enable and loading dump-saving kernel. --- am I right ?

I have added the delay in die() function with few seconds before calling crash_kexec() But I never seen messages logged in /var/log/messages, I can see those on serial console.

Here also my intention is want both, ie. log the appropriate messages in /var/log/messages and then take the action as per KEXEC enabled.

Please advice me, What we can do to achieve those things.

Thanks for helping me here.

regards,
Seetharam

Quoted 34 lines

-----Original Message-----
From: [email protected]
Sent: Sat, 13 Jun 2009 14:07:36 -0400
To: [email protected]
Subject: Re: /var/log/messages doesn't have crash info when kernel gets panic/oops/crash

Quoted 10 lines · Dharmosoth Seetharam

Hi Neil,

Thanks for your quick reply.

What you have suggested is nice when we got/save crash dump (i.e vmcore) What if we failed to save/capture vmcore info or vmcore is corrupted etc. then we dont get any information
regarding the panic/crash/oops at anywhere?

If the vmcore is corrupted to the point where makedumpfile can't extract the
dmesg log, then its certain that syslog wouldn't have been able to write anything to /var/log/messages either. As a last resort you can try run strings
on the corrupted vmcore and see if the dmesg log is still in there somewhere.

If you forgot to capture the vmcore, theres not much more help I can give you,
Im afraid. You'll just have to make sure you capture it next time. If you only
need the backtrace, and don't want the rest of the vmcore, you can make kdump
just run makedumpfile --dump-dmesg /proc/vmcore /var/log/oops.txt

Quoted 62 lines

Neil

Quoted 58 lines

Thanks and Regards

Seetharam

Quoted 22 lines

-----Original Message-----
From: [email protected]
Sent: Fri, 12 Jun 2009 16:05:13 -0400
To: [email protected]
Subject: Re: /var/log/messages doesn't have crash info when kernel gets panic/oops/crash

Quoted 14 lines · Dharmosoth Seetharam

Hi,

I have encounter a problem called /var/log/messages does not contain any
information
on kernel crash dump.

Prior to enabling KEXEC we saw stack traces and bug info etc in /var/log/messages.
After enabling KEXEC feature those info was missed.

How can we get those missed info in /var/log/messages even when KEXEC feature is enabled.

Quoted 22 lines
Quoted 2 lines

Any help will appreciate.

run crash on the resultant vmcore file that kexec can produce/save and get the
dmesg log out of it.
alternatively you can use makedumpfile with the --dump-dmesg log

Neil

Quoted 13 lines

Thanks and Regards
Seetharam

____________________________________________________________
Receive Notifications of Incoming Messages
Easily monitor multiple email accounts & access them with a click. Visit http://www.inbox.com/notifier and check it out!

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

____________________________________________________________
GET FREE SMILEYS FOR YOUR IM & EMAIL - Learn more at
http://www.inbox.com/smileys
Works with AIM®, MSN® Messenger, Yahoo!® Messenger, ICQ®, Google Talk™ and most webmails

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#11 lore raw

[email protected] wrote on 06/15/2009 09:43:02 PM:

Quoted 11 lines

Dharmosoth Seetharam <[email protected]>

Hi,

I have experimented few things and results are below.

Case 1) echo 0 > /proc/sys/kernel/panic_on_oops
When BUG/oops occurred it is able to log the diagnostics to
/var/log/messages
and then continue(not switching to dump-saving kernel).

Quoted 4 lines

What I am looking is - I want both the things to be happen. that is - it should log the appropriate messages in /var/
log/messsages and
should be able to take kernel dump.

when panic_on_oops is not set, it does not call crash_kexec() - change in kexec_should_crash() in kernel/kexec.c

Quoted 8 lines

case 2) In general, When panic has occurred we won't log any type of messages in
/var/log/messages ... am I right ?
If KEXEC is enabled and dump-saving kernel is loaded then will switch to it and save the kernel dump.

I have added the dump_stack() function call in the panic()

function

Quoted 1 line

before calling crash_kexec with few seconds of delay. It is

behaving

Quoted 8 lines

like randomly (ie. some time stack traces are logged in /
var/log/messages and
not for rest of the time)

What I am looking is - I want to log the appropriate
messages in /var/log/messages
and then switch to dump-saving kernel.

printk messages in /var/log/messages is depends on how fast klogd flushes printk buffer.

Quoted 6 lines

case 3) In general, When die/nmi_die called we used to log the appropriate messages in /var/log/message
and then saving kernel dump will be depends on KEXEC enable
and loading dump-saving kernel.
--- am I right ?

Quoted 8 lines

I have added the delay in die() function with few seconds
before calling crash_kexec()
But I never seen messages logged in /var/log/messages, I can
see those on serial console.

Here also my intention is want both, ie. log the appropriate
messages in /var/log/messages
and then take the action as per KEXEC enabled.

printk write to the console and its buffer, As I said, depends on time for klogd flushes printk buffer.

Quoted 29 lines

Please advice me, What we can do to achieve those things.

Thanks for helping me here.

regards,
Seetharam

Quoted 14 lines

-----Original Message-----
From: [email protected]
Sent: Sat, 13 Jun 2009 14:07:36 -0400
To: [email protected]
Subject: Re: /var/log/messages doesn't have crash info when kernel gets panic/oops/crash

Quoted 6 lines · Dharmosoth Seetharam

Hi Neil,

Thanks for your quick reply.

What you have suggested is nice when we got/save crash dump (i.e

vmcore)

Quoted 5 lines
Quoted 5 lines
Quoted 4 lines

What if we failed to save/capture vmcore info or vmcore is corrupted etc. then we dont get any information
regarding the panic/crash/oops at anywhere?

If the vmcore is corrupted to the point where makedumpfile can't

extract

Quoted 2 lines
Quoted 2 lines

the
dmesg log, then its certain that syslog wouldn't have been able to

write

Quoted 6 lines
Quoted 6 lines

anything to /var/log/messages either. As a last resort you can try run strings
on the corrupted vmcore and see if the dmesg log is still in there somewhere.

If you forgot to capture the vmcore, theres not much more help I can

give

Quoted 16 lines
Quoted 16 lines

you,
Im afraid. You'll just have to make sure you capture it next time. If you only
need the backtrace, and don't want the rest of the vmcore, you can make kdump
just run makedumpfile --dump-dmesg /proc/vmcore /var/log/oops.txt

Neil

Quoted 5 lines

Thanks and Regards

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

Seetharam

Quoted 5 lines

-----Original Message-----
From: [email protected]
Sent: Fri, 12 Jun 2009 16:05:13 -0400
To: [email protected]
Subject: Re: /var/log/messages doesn't have crash info when kernel

gets

Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines

panic/oops/crash

Quoted 7 lines · Dharmosoth Seetharam

Hi,

I have encounter a problem called /var/log/messages does not contain any
information
on kernel crash dump.

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

Prior to enabling KEXEC we saw stack traces and bug info etc in /var/log/messages.
After enabling KEXEC feature those info was missed.

How can we get those missed info in /var/log/messages even when

KEXEC

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 4 lines

feature is enabled.

Any help will appreciate.

run crash on the resultant vmcore file that kexec can produce/save

and

Quoted 25 lines
Quoted 20 lines
Quoted 20 lines
Quoted 19 lines

get the
dmesg log out of it.
alternatively you can use makedumpfile with the --dump-dmesg log

Neil

Quoted 13 lines

Thanks and Regards
Seetharam

____________________________________________________________
Receive Notifications of Incoming Messages
Easily monitor multiple email accounts & access them with a click. Visit http://www.inbox.com/notifier and check it out!

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

Quoted 5 lines
Quoted 5 lines

____________________________________________________________
GET FREE SMILEYS FOR YOUR IM & EMAIL - Learn more at
http://www.inbox.com/smileys
Works with AIM®, MSN® Messenger, Yahoo!® Messenger, ICQ®, Google Talk™ and most webmails

Quoted 4 lines

_______________________________________________
fastboot mailing list
[email protected]
https://lists.linux-foundation.org/mailman/listinfo/fastboot

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#12 lore raw
In reply to Dharmosoth Seetharam
Quoted 11 lines · Dharmosoth Seetharam

Hi,

I have experimented few things and results are below.

Case 1) echo 0 > /proc/sys/kernel/panic_on_oops
When BUG/oops occurred it is able to log the diagnostics to /var/log/messages and then continue(not switching to dump-saving kernel).

What I am looking is - I want both the things to be happen. that is - it should log the appropriate messages in /var/log/messsages and should be able to take kernel dump.

You can't have it. Sorry.
Keeping user space alive is something that simply can not be done reliably.

Quoted 1 line

Please advice me, What we can do to achieve those things.

The only scenario I see possible is to take the dump. Use makedumpfile or the crash log command to extract the kernel messages and put them in the /var/log/messages.

Eric

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#13 lore raw
In reply to Dharmosoth Seetharam
Quoted 11 lines · Dharmosoth Seetharam

Hi,

I have experimented few things and results are below.

Case 1) echo 0 > /proc/sys/kernel/panic_on_oops
When BUG/oops occurred it is able to log the diagnostics to /var/log/messages and then continue(not switching to dump-saving kernel).

What I am looking is - I want both the things to be happen. that is - it should log the appropriate messages in /var/log/messsages and should be able to take kernel dump.

You can't have it. Sorry.
Keeping user space alive is something that simply can not be done reliably.

Quoted 1 line

Please advice me, What we can do to achieve those things.

The only scenario I see possible is to take the dump. Use makedumpfile or the crash log command to extract the kernel messages and put them in the /var/log/messages.

Eric

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#14 lore raw
In reply to Dharmosoth Seetharam
Quoted 17 lines · Dharmosoth Seetharam

Hi,

I have experimented few things and results are below.

Case 1) echo 0 > /proc/sys/kernel/panic_on_oops
When BUG/oops occurred it is able to log the diagnostics to /var/log/messages and then continue(not switching to dump-saving kernel).

What I am looking is - I want both the things to be happen. that is - it should log the appropriate messages in /var/log/messsages and should be able to take kernel dump.

case 2) In general, When panic has occurred we won't log any type of messages in
/var/log/messages ... am I right ?
If KEXEC is enabled and dump-saving kernel is loaded then will switch to it and save the kernel dump.

Quoted 8 lines

I have added the dump_stack() function call in the panic() function before calling crash_kexec with few seconds of delay. It is behaving like randomly (ie. some time stack traces are logged in /var/log/messages and not for rest of the time)

What I am looking is - I want to log the appropriate messages in /var/log/messages and then switch to dump-saving kernel.

Then what you want to do is, enable kdump, and set your kernel up to panic_on_oops. Then, configure kdump to extract the dmesg log from /proc/vmcore, and write that to a file (either a file you create, or to the end of /var/log/messages). Most distributions kdump configurations allow for something simmilar to this. The new config method I've got in rawhide should make this very easy.

The problem is that when you panic the box (which is what has to happen for kdump to start), the syslog daemon doesn't get a chance to run first, so theres no way to write the oops backtrace to var/log/messages. You'll need to tell kdump to do that for you after the fact.

Quoted 7 lines

case 3) In general, When die/nmi_die called we used to log the appropriate messages in /var/log/message
and then saving kernel dump will be depends on KEXEC enable and loading dump-saving kernel. --- am I right ?

I have added the delay in die() function with few seconds before calling crash_kexec() But I never seen messages logged in /var/log/messages, I can see those on serial console.

See above, when we printk those messages, klogd/syslogd needs to run to get them from the console buffer and write them to /var/log/messages. Theres no guarantee that will ever happen.

Quoted 7 lines

Here also my intention is want both, ie. log the appropriate messages in /var/log/messages and then take the action as per KEXEC enabled.

Please advice me, What we can do to achieve those things.

See above, use makedumpfile to extract the dmesg log from /proc/vmcore in kdump, then use a custom script to append the log to /var/log/messages. In fedora that will be equivalent to using the core_collector and kdump_post options in /etc/kdump.conf.

Neil

Quoted 106 lines

Thanks for helping me here.

regards,
Seetharam

Quoted 96 lines

-----Original Message-----
From: [email protected]
Sent: Sat, 13 Jun 2009 14:07:36 -0400
To: [email protected]
Subject: Re: /var/log/messages doesn't have crash info when kernel gets panic/oops/crash

Quoted 10 lines · Dharmosoth Seetharam

Hi Neil,

Thanks for your quick reply.

What you have suggested is nice when we got/save crash dump (i.e vmcore) What if we failed to save/capture vmcore info or vmcore is corrupted etc. then we dont get any information
regarding the panic/crash/oops at anywhere?

If the vmcore is corrupted to the point where makedumpfile can't extract the
dmesg log, then its certain that syslog wouldn't have been able to write anything to /var/log/messages either. As a last resort you can try run strings
on the corrupted vmcore and see if the dmesg log is still in there somewhere.

If you forgot to capture the vmcore, theres not much more help I can give you,
Im afraid. You'll just have to make sure you capture it next time. If you only
need the backtrace, and don't want the rest of the vmcore, you can make kdump
just run makedumpfile --dump-dmesg /proc/vmcore /var/log/oops.txt

Neil

Quoted 58 lines

Thanks and Regards

Seetharam

Quoted 22 lines

-----Original Message-----
From: [email protected]
Sent: Fri, 12 Jun 2009 16:05:13 -0400
To: [email protected]
Subject: Re: /var/log/messages doesn't have crash info when kernel gets panic/oops/crash

Quoted 14 lines · Dharmosoth Seetharam

Hi,

I have encounter a problem called /var/log/messages does not contain any
information
on kernel crash dump.

Prior to enabling KEXEC we saw stack traces and bug info etc in /var/log/messages.
After enabling KEXEC feature those info was missed.

How can we get those missed info in /var/log/messages even when KEXEC feature is enabled.

Quoted 22 lines
Quoted 2 lines

Any help will appreciate.

run crash on the resultant vmcore file that kexec can produce/save and get the
dmesg log out of it.
alternatively you can use makedumpfile with the --dump-dmesg log

Neil

Quoted 13 lines

Thanks and Regards
Seetharam

____________________________________________________________
Receive Notifications of Incoming Messages
Easily monitor multiple email accounts & access them with a click. Visit http://www.inbox.com/notifier and check it out!

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

____________________________________________________________
GET FREE SMILEYS FOR YOUR IM & EMAIL - Learn more at
http://www.inbox.com/smileys
Works with AIM®, MSN® Messenger, Yahoo!® Messenger, ICQ®, Google Talk™ and most webmails

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#15 lore raw

Thanks a lot.
Seetharam

Quoted 81 lines

-----Original Message-----
From: [email protected]
Sent: Tue, 16 Jun 2009 07:24:25 -0400
To: [email protected]
Subject: Re: /var/log/messages doesn't have crash info when kernel gets panic/oops/crash

Quoted 28 lines · Dharmosoth Seetharam

Hi,

I have experimented few things and results are below.

Case 1) echo 0 > /proc/sys/kernel/panic_on_oops
When BUG/oops occurred it is able to log the diagnostics to
/var/log/messages
and then continue(not switching to dump-saving kernel).

What I am looking is - I want both the things to be happen. that is - it should log the appropriate messages in
/var/log/messsages and
should be able to take kernel dump.

case 2) In general, When panic has occurred we won't log any type of messages in
/var/log/messages ... am I right ?
If KEXEC is enabled and dump-saving kernel is loaded then will switch to it and save the kernel dump.

I have added the dump_stack() function call in the panic()
function
before calling crash_kexec with few seconds of delay. It is
behaving
like randomly (ie. some time stack traces are logged in
/var/log/messages and
not for rest of the time)

Quoted 4 lines

What I am looking is - I want to log the appropriate messages
in /var/log/messages
and then switch to dump-saving kernel.

Then what you want to do is, enable kdump, and set your kernel up to panic_on_oops. Then, configure kdump to extract the dmesg log from /proc/vmcore, and write that to a file (either a file you create, or to the end
of /var/log/messages). Most distributions kdump configurations allow for something simmilar to this. The new config method I've got in rawhide should
make this very easy.

The problem is that when you panic the box (which is what has to happen for
kdump to start), the syslog daemon doesn't get a chance to run first, so theres
no way to write the oops backtrace to var/log/messages. You'll need to tell
kdump to do that for you after the fact.

Quoted 11 lines

case 3) In general, When die/nmi_die called we used to log the appropriate messages in /var/log/message
and then saving kernel dump will be depends on KEXEC enable and
loading dump-saving kernel.
--- am I right ?

I have added the delay in die() function with few seconds before
calling crash_kexec()
But I never seen messages logged in /var/log/messages, I can see
those on serial console.

See above, when we printk those messages, klogd/syslogd needs to run to get them
from the console buffer and write them to /var/log/messages. Theres no guarantee that will ever happen.

Quoted 8 lines

Here also my intention is want both, ie. log the appropriate
messages in /var/log/messages
and then take the action as per KEXEC enabled.

Please advice me, What we can do to achieve those things.

Quoted 122 lines

See above, use makedumpfile to extract the dmesg log from /proc/vmcore in kdump,
then use a custom script to append the log to /var/log/messages. In fedora that
will be equivalent to using the core_collector and kdump_post options in /etc/kdump.conf.

Neil

Quoted 113 lines

Thanks for helping me here.

regards,
Seetharam

Quoted 103 lines

-----Original Message-----
From: [email protected]
Sent: Sat, 13 Jun 2009 14:07:36 -0400
To: [email protected]
Subject: Re: /var/log/messages doesn't have crash info when kernel gets panic/oops/crash

Quoted 11 lines · Dharmosoth Seetharam

Hi Neil,

Thanks for your quick reply.

What you have suggested is nice when we got/save crash dump (i.e vmcore)
What if we failed to save/capture vmcore info or vmcore is corrupted etc. then we dont get any information
regarding the panic/crash/oops at anywhere?

If the vmcore is corrupted to the point where makedumpfile can't extract
the
dmesg log, then its certain that syslog wouldn't have been able to write
anything to /var/log/messages either. As a last resort you can try run strings
on the corrupted vmcore and see if the dmesg log is still in there somewhere.

If you forgot to capture the vmcore, theres not much more help I can give
you,
Im afraid. You'll just have to make sure you capture it next time. If you only
need the backtrace, and don't want the rest of the vmcore, you can make kdump
just run makedumpfile --dump-dmesg /proc/vmcore /var/log/oops.txt

Neil

Quoted 61 lines

Thanks and Regards

Seetharam

Quoted 26 lines

-----Original Message-----
From: [email protected]
Sent: Fri, 12 Jun 2009 16:05:13 -0400
To: [email protected]
Subject: Re: /var/log/messages doesn't have crash info when kernel gets
panic/oops/crash

Quoted 11 lines · Dharmosoth Seetharam

Hi,

I have encounter a problem called /var/log/messages does not contain any
information
on kernel crash dump.

Prior to enabling KEXEC we saw stack traces and bug info etc in /var/log/messages.
After enabling KEXEC feature those info was missed.

Quoted 6 lines

How can we get those missed info in /var/log/messages even when KEXEC
feature is enabled.

Any help will appreciate.

Quoted 21 lines

run crash on the resultant vmcore file that kexec can produce/save and
get the
dmesg log out of it.
alternatively you can use makedumpfile with the --dump-dmesg log

Neil

Quoted 13 lines

Thanks and Regards
Seetharam

____________________________________________________________
Receive Notifications of Incoming Messages
Easily monitor multiple email accounts & access them with a click. Visit http://www.inbox.com/notifier and check it out!

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

____________________________________________________________
GET FREE SMILEYS FOR YOUR IM & EMAIL - Learn more at
http://www.inbox.com/smileys
Works with AIM®, MSN® Messenger, Yahoo!® Messenger, ICQ®, Google Talk™ and most webmails

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec