Skip to contentSkip to conversation

Slimdump framework using CRASH_REASON - v2

RFC PATCH In review K.Prasad 3 participants 15 messages Nov 21, 2011 – Nov 30, 2011

Subsystems (2)
lore
Share & tools
lore

K.Prasad

author
#0 lore raw

cover Slimdump framework using CRASH_REASON - v2

Hi All,
In furtherance of the previous discussion regarding 'slimdump'
(refer: http://article.gmane.org/gmane.linux.kernel/1204967), it was decided that,

- An entry in VMCOREINFO elf-note be added to denote the cause of crash, instead of creating a new elf-note.

- Upstream tools such as 'makedumpfile' and 'crash' be modified to recognise this string and inform the user accordingly.

Accordingly, this new version of the patchset makes the following changes

Changelog - version 2
-----------------------
(First version posted here:
http://article.gmane.org/gmane.linux.kernel/1198435)

- Append VMCOREINFO elf-note with a new variable CRASH_REASON whose value will be populated using arch_add_crash_reason() function.

- Define arch_add_crash_reason() in the x86 MCE path to return "PANIC_MCE" in the panic path of MCE.

- 'makedumpfile' tool is taught to recognise PANIC_MCE string as one value of CRASH_REASON for which 'slimdump' must be captured.

- Changes to 'crash' tool are not included, and is deferred till there's consensus to the kernel and makedumpfile patches.

Let me know your comments on this.

Thanks,
K.Prasad

author
#1 lore raw

RFC PATCH 1/2 Append CRASH_REASON to VMCOREINFO elf-note

Allow various crash paths to append the reason of crash into the VMCOREINFO elf-note through the field CRASH_REASON. We also make the fatal machine check exceptions append "PANIC_MCE" as the crash reason. This string will be recognised by upstream tools like makedumpfile and crash to generate slimdump.

With increased usage of the CRASH_REASON field, the crash strings can be encoded for better usage.

---
 arch/x86/kernel/cpu/mcheck/mce.c |    8 ++++++++
 kernel/kexec.c                   |    6 ++++++
 2 files changed, 14 insertions(+), 0 deletions(-)
arch/x86/kernel/cpu/mcheck/mce.c +8 -0
index 362056a..5b2cb6a 100644
@@ -38,6 +38,7 @@
 #include <linux/debugfs.h>
 #include <linux/irq_work.h>
 #include <linux/export.h>
+#include <linux/kexec.h>
 
 #include <asm/processor.h>
 #include <asm/mce.h>
@@ -240,6 +241,13 @@ static atomic_t mce_paniced;
 static int fake_panic;
 static atomic_t mce_fake_paniced;
 
+char *arch_add_crash_reason(void)
+{
+	static char crash_reason[] = "PANIC_MCE";
+
+	return crash_reason;
+}
+
 /* Panic in progress. Enable interrupts and wait for final IPI */
 static void wait_for_panic(void)
 {
kernel/kexec.c +6 -0
index dc7bc08..a731693 100644
@@ -1080,6 +1080,11 @@ asmlinkage long compat_sys_kexec_load(unsigned long entry,
 }
 #endif
 
+__weak char *arch_add_crash_reason(void)
+{
+	return (char *)NULL;
+}
+
 void crash_kexec(struct pt_regs *regs)
 {
 	/* Take the kexec_mutex here to prevent sys_kexec_load
@@ -1411,6 +1416,7 @@ static void update_vmcoreinfo_note(void)
 void crash_save_vmcoreinfo(void)
 {
 	vmcoreinfo_append_str("CRASHTIME=%ld", get_seconds());
+	vmcoreinfo_append_str("\nCRASH_REASON=%s\n", arch_add_crash_reason());
 	update_vmcoreinfo_note();
 }
 
#2 lore raw

Re: [RFC Patch 1/2][slimdump] Append CRASH_REASON to VMCOREINFO elf-note

Quoted 58 lines · K.Prasad

Allow various crash paths to append the reason of crash into the VMCOREINFO elf-note through the field CRASH_REASON. We also make the fatal machine check exceptions append "PANIC_MCE" as the crash reason. This string will be recognised by upstream tools like makedumpfile and crash to generate slimdump.

With increased usage of the CRASH_REASON field, the crash strings can be encoded for better usage.

---
 arch/x86/kernel/cpu/mcheck/mce.c |    8 ++++++++
 kernel/kexec.c                   |    6 ++++++
 2 files changed, 14 insertions(+), 0 deletions(-)
arch/x86/kernel/cpu/mcheck/mce.c +8 -0
index 362056a..5b2cb6a 100644
@@ -38,6 +38,7 @@
 #include <linux/debugfs.h>
 #include <linux/irq_work.h>
 #include <linux/export.h>
+#include <linux/kexec.h>
 
 #include <asm/processor.h>
 #include <asm/mce.h>
@@ -240,6 +241,13 @@ static atomic_t mce_paniced;
 static int fake_panic;
 static atomic_t mce_fake_paniced;
 
+char *arch_add_crash_reason(void)
+{
+	static char crash_reason[] = "PANIC_MCE";
+
+	return crash_reason;
+}
+
 /* Panic in progress. Enable interrupts and wait for final IPI */
 static void wait_for_panic(void)
 {
kernel/kexec.c +5 -0
index dc7bc08..a731693 100644
@@ -1080,6 +1080,11 @@ asmlinkage long compat_sys_kexec_load(unsigned long entry,
 }
 #endif
 
+__weak char *arch_add_crash_reason(void)
+{
+	return (char *)NULL;
+}
+
 void crash_kexec(struct pt_regs *regs)
 {
 	/* Take the kexec_mutex here to prevent sys_kexec_load

kernel/kexec.c static void update_vmcoreinfo_note(void) · line 1419

Quoted 4 lines
kernel/kexec.c +1 -0
@@ -1411,6 +1416,7 @@ static void update_vmcoreinfo_note(void)
 void crash_save_vmcoreinfo(void)
 {
 	vmcoreinfo_append_str("CRASHTIME=%ld", get_seconds());
+	vmcoreinfo_append_str("\nCRASH_REASON=%s\n", arch_add_crash_reason());

I think don't even create a CRASH_REASON= entry if arch returns a NULL string.

Vivek

author
#3 lore raw

Re: [RFC Patch 1/2][slimdump] Append CRASH_REASON to VMCOREINFO elf-note

Quoted 10 lines · Vivek Goyal
Quoted 9 lines · K.Prasad

Allow various crash paths to append the reason of crash into the VMCOREINFO elf-note through the field CRASH_REASON. We also make the fatal machine check exceptions append "PANIC_MCE" as the crash reason. This string will be recognised by upstream tools like makedumpfile and crash to generate slimdump.

With increased usage of the CRASH_REASON field, the crash strings can be encoded for better usage.

[snipped]

Quoted 8 lines
Quoted 8 lines
kernel/kexec.c +0 -0
index dc7bc08..a731693 100644
@@ -1080,6 +1080,11 @@ asmlinkage long compat_sys_kexec_load(unsigned long entry,
 }
 #endif
 
Quoted 17 lines

kernel/kexec.c static void update_vmcoreinfo_note(void) · line 1419

Quoted 13 lines
kernel/kexec.c +6 -0
+__weak char *arch_add_crash_reason(void)
+{
+	return (char *)NULL;
+}
+
 void crash_kexec(struct pt_regs *regs)
 {
 	/* Take the kexec_mutex here to prevent sys_kexec_load
@@ -1411,6 +1416,7 @@ static void update_vmcoreinfo_note(void)
 void crash_save_vmcoreinfo(void)
 {
 	vmcoreinfo_append_str("CRASHTIME=%ld", get_seconds());
+	vmcoreinfo_append_str("\nCRASH_REASON=%s\n", arch_add_crash_reason());

I think don't even create a CRASH_REASON= entry if arch returns a NULL string.

Yes, we could do that. I'll change the code accordingly in the next revision of the patchset.

Thanks,
K.Prasad

#4 lore raw

Re: [RFC Patch 1/2][slimdump] Append CRASH_REASON to VMCOREINFO elf-note

In reply to K.Prasad

----- Original Message -----

Quoted 5 lines

Allow various crash paths to append the reason of crash into the VMCOREINFO elf-note through the field CRASH_REASON. We also make the fatal machine check exceptions append "PANIC_MCE" as the crash reason. This string will be recognised by upstream tools like makedumpfile and crash to generate slimdump.

I don't understand -- how could "various paths" append a reason? The patch below seems to return "PANIC_MCE" for every x86 crash. What am I missing?

Dave

Quoted 63 lines

With increased usage of the CRASH_REASON field, the crash strings can be encoded for better usage.

---
 arch/x86/kernel/cpu/mcheck/mce.c |    8 ++++++++
 kernel/kexec.c                   |    6 ++++++
 2 files changed, 14 insertions(+), 0 deletions(-)
arch/x86/kernel/cpu/mcheck/mce.c +8 -0
index 362056a..5b2cb6a 100644
@@ -38,6 +38,7 @@
 #include <linux/debugfs.h>
 #include <linux/irq_work.h>
 #include <linux/export.h>
+#include <linux/kexec.h>
 
 #include <asm/processor.h>
 #include <asm/mce.h>
@@ -240,6 +241,13 @@ static atomic_t mce_paniced;
 static int fake_panic;
 static atomic_t mce_fake_paniced;
 
+char *arch_add_crash_reason(void)
+{
+	static char crash_reason[] = "PANIC_MCE";
+
+	return crash_reason;
+}
+
 /* Panic in progress. Enable interrupts and wait for final IPI */
 static void wait_for_panic(void)
 {
kernel/kexec.c +5 -0
index dc7bc08..a731693 100644
@@ -1080,6 +1080,11 @@ asmlinkage long compat_sys_kexec_load(unsigned
 long entry,
 }
 #endif
 
+__weak char *arch_add_crash_reason(void)
+{
+	return (char *)NULL;
+}
+
 void crash_kexec(struct pt_regs *regs)
 {
kernel/kexec.c +1 -0
 	/* Take the kexec_mutex here to prevent sys_kexec_load
@@ -1411,6 +1416,7 @@ static void update_vmcoreinfo_note(void)
 void crash_save_vmcoreinfo(void)
 {
 	vmcoreinfo_append_str("CRASHTIME=%ld", get_seconds());
+	vmcoreinfo_append_str("\nCRASH_REASON=%s\n", arch_add_crash_reason());
 	update_vmcoreinfo_note();
 }
 
author
#5 lore raw

Re: [RFC Patch 1/2][slimdump] Append CRASH_REASON to VMCOREINFO elf-note

Quoted 9 lines · Dave Anderson

----- Original Message -----

Quoted 5 lines

Allow various crash paths to append the reason of crash into the VMCOREINFO elf-note through the field CRASH_REASON. We also make the fatal machine check exceptions append "PANIC_MCE" as the crash reason. This string will be recognised by upstream tools like makedumpfile and crash to generate slimdump.

Quoted 6 lines

I don't understand -- how could "various paths" append a reason? The patch below seems to return "PANIC_MCE" for every x86 crash. What am I missing?

Dave

Yes, presently it can only be "PANIC_MCE" for MCE crashes in x86 (not for every crash though).

With increased usage, we should move this code to a generic location and let each of these crash paths return a string to be appended. In fact it doesn't have to be a string for CRASH_REASON but just an encoding of the various crash types into numbers. User-space tools could then do a lookup for getting the right crash string.

Thanks,
K.Prasad

#6 lore raw

Re: [RFC Patch 1/2][slimdump] Append CRASH_REASON to VMCOREINFO elf-note

Quoted 17 lines · K.Prasad
Quoted 9 lines · Dave Anderson

----- Original Message -----

Quoted 5 lines

Allow various crash paths to append the reason of crash into the VMCOREINFO elf-note through the field CRASH_REASON. We also make the fatal machine check exceptions append "PANIC_MCE" as the crash reason. This string will be recognised by upstream tools like makedumpfile and crash to generate slimdump.

Quoted 6 lines

I don't understand -- how could "various paths" append a reason? The patch below seems to return "PANIC_MCE" for every x86 crash. What am I missing?

Dave

Quoted 8 lines

Yes, presently it can only be "PANIC_MCE" for MCE crashes in x86 (not for every crash though).

With increased usage, we should move this code to a generic location and let each of these crash paths return a string to be appended. In fact it doesn't have to be a string for CRASH_REASON but just an encoding of the various crash types into numbers. User-space tools could then do a lookup for getting the right crash string.

Probably string is a better idea? Where do we do lookup to find out what maps to what? This would require kernel exporting this info in a header and then comes the issue of having same kernel version info. Or the issue of analyzing kernel dumps on the same machine.

Storing a string will atleast help that one does not have to worry about mapping error code. makedumpfile shall have to grep for exact same string though.

Thanks
Vivek

author
#7 lore raw

Re: [RFC Patch 1/2][slimdump] Append CRASH_REASON to VMCOREINFO elf-note

In reply to Dave Anderson
Quoted 9 lines · Dave Anderson

----- Original Message -----

Quoted 5 lines

Allow various crash paths to append the reason of crash into the VMCOREINFO elf-note through the field CRASH_REASON. We also make the fatal machine check exceptions append "PANIC_MCE" as the crash reason. This string will be recognised by upstream tools like makedumpfile and crash to generate slimdump.

Quoted 6 lines

I don't understand -- how could "various paths" append a reason? The patch below seems to return "PANIC_MCE" for every x86 crash. What am I missing?

Dave

Yes, presently it can only be "PANIC_MCE" for MCE crashes in x86 (not for every crash though).

With increased usage, we should move this code to a generic location and let each of these crash paths return a string to be appended. In fact it doesn't have to be a string for CRASH_REASON but just an encoding of the various crash types into numbers. User-space tools could then do a lookup for getting the right crash string.

Thanks,
K.Prasad

#8 lore raw

Re: [RFC Patch 1/2][slimdump] Append CRASH_REASON to VMCOREINFO elf-note

----- Original Message -----

Quoted 14 lines · Dave Anderson
Quoted 13 lines

----- Original Message -----

Quoted 5 lines

Allow various crash paths to append the reason of crash into the VMCOREINFO elf-note through the field CRASH_REASON. We also make the fatal machine check exceptions append "PANIC_MCE" as the crash reason. This string will be recognised by upstream tools like makedumpfile and crash to generate slimdump.

I don't understand -- how could "various paths" append a reason? The patch below seems to return "PANIC_MCE" for every x86 crash. What am I missing?

Quoted 5 lines
Quoted 2 lines

Dave

Yes, presently it can only be "PANIC_MCE" for MCE crashes in x86 (not for every crash though).

Why only MCE crashes? If your arch/x86/kernel/cpu/mcheck/mce.c is compiled into any x86 kernel, then its arch_add_crash_reason() will always override the weak version in kernel/kexec.c, right?

Dave

author
#9 lore raw

Re: [RFC Patch 1/2][slimdump] Append CRASH_REASON to VMCOREINFO elf-note

Quoted 20 lines · Dave Anderson

----- Original Message -----

Quoted 17 lines · Dave Anderson
Quoted 15 lines

----- Original Message -----

Quoted 5 lines

Allow various crash paths to append the reason of crash into the VMCOREINFO elf-note through the field CRASH_REASON. We also make the fatal machine check exceptions append "PANIC_MCE" as the crash reason. This string will be recognised by upstream tools like makedumpfile and crash to generate slimdump.

I don't understand -- how could "various paths" append a reason? The patch below seems to return "PANIC_MCE" for every x86 crash. What am I missing?

Dave

Quoted 7 lines
Quoted 2 lines

Yes, presently it can only be "PANIC_MCE" for MCE crashes in x86 (not for every crash though).

Why only MCE crashes? If your arch/x86/kernel/cpu/mcheck/mce.c is compiled into any x86 kernel, then its arch_add_crash_reason() will always override the weak version in kernel/kexec.c, right?

(Sorry for the delayed reply...there was a multiple-day outage in my email server).

Yes, I think it should be better by using a variable to allow various crash paths of each architecture populate the reason (the function approach doesn't work, we can use a function pointer that just returns a pointer to a char array, but that's less preferable and convoluted).

I'll change the code to contain a char pointer in a generic file, say arch/x86/kernel/crash.c which will be populated by the MCE crash path. The code in kernel/kexec.c can just use this.

Thanks for your comments.

--K.Prasad

author
#10 lore raw

RFC PATCH 2/2 Recognise PANIC_MCE crashes to generate slimdu

In reply to K.Prasad

Given that the kernel indicates the cause of crash through a new field CRASH_REASON in the VMCOREINFO elf-note, recognise the same. For crashes caused by PANIC_MCE, avoid capture of kernel memory, instead generate only a slimdump.

Since 'slimdump' will be of very small size (containing only elf-headers and elf-notes section), the resultant coredump will be of ELF type (and not kdump-compressed format).

---
 elf_info.c     |   67 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 elf_info.h     |    2 +
 makedumpfile.c |   13 ++++++++++-
 makedumpfile.h |    1 +
 4 files changed, 82 insertions(+), 1 deletions(-)
elf_info.c +67 -0
index 114dd05..a925484 100644
@@ -287,6 +287,73 @@ offset_note_desc(void *note)
 	return offset;
 }
 
+#define CRASH_REASON_PANIC_MCE	"CRASH_REASON=PANIC_MCE"
+
+/*
+ * This function checks if the vmcoreinfo note has its CRASH_REASON set as
+ * PANIC_MCE. This is added if the crash is due to a hardware error and
+ * when it makes no sense to read/store the crashing kernel's memory. In
+ * such a case, only a 'slimdump' is captured.
+ */
+int
+is_crash_by_mce(void)
+{
+	int note_size, ret = FALSE;
+	off_t offset;
+	char buf[VMCOREINFO_XEN_NOTE_NAME_BYTES];
+	char note[MAX_SIZE_NHDR];
+	void *vmcoreinfo_note = NULL;
+
+	offset = offset_pt_note_memory;
+	while (offset < offset_pt_note_memory + size_pt_note_memory) {
+		if (lseek(fd_memory, offset, SEEK_SET) < 0) {
+			ERRMSG("Can't seek the dump memory(%s). %s\n",
+			    name_memory, strerror(errno));
+			return FALSE;
+		}
+		if (read(fd_memory, note, sizeof(note)) != sizeof(note)) {
+			ERRMSG("Can't read the dump memory(%s). %s\n",
+			    name_memory, strerror(errno));
+			return FALSE;
+		}
+
+		if (read(fd_memory, &buf, sizeof(buf)) != sizeof(buf)) {
+			ERRMSG("Can't read the dump memory(%s). %s\n",
+			    name_memory, strerror(errno));
+			return FALSE;
+		}
+		if (strncmp(VMCOREINFO_NOTE_NAME, buf,
+				VMCOREINFO_NOTE_NAME_BYTES)) {
+			offset += offset_next_note(note);
+			continue;
+		}
+
+		/*
+		 * Now copy VMCOREINFO_NOTE to examine its contents.
+		 * We need to parse it to check if the CRASH_REASON=PANIC_MCE.
+		 */
+		note_size = offset_next_note(note);
+
+		vmcoreinfo_note = malloc(note_size);
+		if(!vmcoreinfo_note) {
+			ERRMSG("Can't allocate memory for the vmcoreinfo note."
+				"%s\n", strerror(errno));
+			return FALSE;
+		}
+		if (read(fd_memory, vmcoreinfo_note, note_size) != note_size) {
+			ERRMSG("Can't read the dump memory(%s). %s\n",
+			    name_memory, strerror(errno));
+			goto exit;
+		}
+		if(strstr(vmcoreinfo_note, CRASH_REASON_PANIC_MCE))
+			ret = TRUE;
+			break;
+	}
+exit:
+	free(vmcoreinfo_note);
+	return ret;
+}
+
 static int
 get_pt_note_info(void)
 {
elf_info.h +2 -0
index 4dff9c1..0437481 100644
@@ -34,6 +34,8 @@ unsigned long long get_max_paddr(void);
 int get_elf64_ehdr(int fd, char *filename, Elf64_Ehdr *ehdr);
 int get_elf32_ehdr(int fd, char *filename, Elf32_Ehdr *ehdr);
 int get_elf_info(int fd, char *filename);
+int is_crash_by_mce(void);
+
 void free_elf_info(void);
 
 int is_elf64_memory(void);
makedumpfile.c +12 -1
index 7b7c266..15efa90 100644
@@ -4173,7 +4173,11 @@ write_elf_pages(struct cache_data *cd_header, struct cache_data *cd_page)
 		if (!get_phdr_memory(i, &load))
 			return FALSE;
 
-		if (load.p_type != PT_LOAD)
+		/*
+		 * Do not capture the kernel's memory if flag_nocoredump is
+		 * turned on. This may be dangerous to the system stability.
+		 */
+		if ((load.p_type != PT_LOAD) || (info->flag_nocoredump))
 			continue;
 
 		off_memory= load.p_offset;
@@ -5760,6 +5764,13 @@ create_dumpfile(void)
 		if (!get_elf_info(info->fd_memory, info->name_memory))
 			return FALSE;
 	}
+	/*
+	 * If NT_NOCOREDUMP elf-note is present, indicate the same through
+	 * 'flag_nocoredump' flag. The resultant slimdump will always be in ELF
+	 * format, irrespective of the user options.
+	 */
+	info->flag_nocoredump = info->flag_elf_dumpfile = is_crash_by_mce();
+
 	if (is_xen_memory()) {
 		if (!initial_xen())
 			return FALSE;
makedumpfile.h +1 -0
index f0e5da8..faf1c65 100644
@@ -778,6 +778,7 @@ struct DumpInfo {
 	int		flag_exclude_xen_dom;/* exclude Domain-U from xen-kdump */
 	int             flag_dmesg;          /* dump the dmesg log out of the vmcore file */
 	int		flag_nospace;	     /* the flag of "No space on device" error */
+	int		flag_nocoredump;	/* coredump not collected */
 	unsigned long	vaddr_for_vtop;      /* virtual address for debugging */
 	long		page_size;           /* size of page */
 	long		page_shift;
#11 lore raw
In reply to K.Prasad
Quoted 19 lines · K.Prasad

Hi All,
In furtherance of the previous discussion regarding 'slimdump'
(refer: http://article.gmane.org/gmane.linux.kernel/1204967), it was decided that,

- An entry in VMCOREINFO elf-note be added to denote the cause of crash, instead of creating a new elf-note.

- Upstream tools such as 'makedumpfile' and 'crash' be modified to recognise this string and inform the user accordingly.

Accordingly, this new version of the patchset makes the following changes

Changelog - version 2
-----------------------
(First version posted here:
http://article.gmane.org/gmane.linux.kernel/1198435)

Quoted 8 lines

- Append VMCOREINFO elf-note with a new variable CRASH_REASON whose value will be populated using arch_add_crash_reason() function.

- Define arch_add_crash_reason() in the x86 MCE path to return "PANIC_MCE" in the panic path of MCE.

- 'makedumpfile' tool is taught to recognise PANIC_MCE string as one value of CRASH_REASON for which 'slimdump' must be captured.

So again, what is slimdump? I mean, what information is now being captured in the case of slimdump? Are you capturing atleast the kernel message buffers? I am assuming that any register info emitted on console will make into kernel buffers and that should be useful to figure out what MCE happened.

Thanks
Vivek

author
#12 lore raw
Quoted 26 lines · Vivek Goyal
Quoted 25 lines · K.Prasad

Hi All,
In furtherance of the previous discussion regarding 'slimdump'
(refer: http://article.gmane.org/gmane.linux.kernel/1204967), it was decided that,

- An entry in VMCOREINFO elf-note be added to denote the cause of crash, instead of creating a new elf-note.

- Upstream tools such as 'makedumpfile' and 'crash' be modified to recognise this string and inform the user accordingly.

Accordingly, this new version of the patchset makes the following changes

Changelog - version 2
-----------------------
(First version posted here:
http://article.gmane.org/gmane.linux.kernel/1198435)

- Append VMCOREINFO elf-note with a new variable CRASH_REASON whose value will be populated using arch_add_crash_reason() function.

- Define arch_add_crash_reason() in the x86 MCE path to return "PANIC_MCE" in the panic path of MCE.

Quoted 9 lines
Quoted 2 lines

- 'makedumpfile' tool is taught to recognise PANIC_MCE string as one value of CRASH_REASON for which 'slimdump' must be captured.

So again, what is slimdump? I mean, what information is now being captured in the case of slimdump? Are you capturing atleast the kernel message buffers? I am assuming that any register info emitted on console will make into kernel buffers and that should be useful to figure out what MCE happened.

The kernel message buffers can be obtained by using the --dump-dmesg option of makedumpfile but again that's risky. We wouldn't know if it'll cause access to the faulty memory (which is how the previous method of having a new elf-notes in a pristine location is much safer).

The method in this patch is quite primitive in that informs the user nothing more than a one-line cause of crash. One should take help from other tools (such as service processor/firmware/ACPI logs, or previous corrected error logs) to infer the location of bad memory. It would have been helpful if we could provide a dump of related information such as contents of "struct mce" (like in the very first iteration of the patch,
refer: http://article.gmane.org/gmane.linux.kernel/1146215), but that would need some memory (possibly in the form of a new elf-note).

Thanks,
K.Prasad

#13 lore raw
Quoted 42 lines · K.Prasad
Quoted 26 lines · Vivek Goyal
Quoted 25 lines · K.Prasad

Hi All,
In furtherance of the previous discussion regarding 'slimdump'
(refer: http://article.gmane.org/gmane.linux.kernel/1204967), it was decided that,

- An entry in VMCOREINFO elf-note be added to denote the cause of crash, instead of creating a new elf-note.

- Upstream tools such as 'makedumpfile' and 'crash' be modified to recognise this string and inform the user accordingly.

Accordingly, this new version of the patchset makes the following changes

Changelog - version 2
-----------------------
(First version posted here:
http://article.gmane.org/gmane.linux.kernel/1198435)

- Append VMCOREINFO elf-note with a new variable CRASH_REASON whose value will be populated using arch_add_crash_reason() function.

- Define arch_add_crash_reason() in the x86 MCE path to return "PANIC_MCE" in the panic path of MCE.

Quoted 9 lines
Quoted 2 lines

- 'makedumpfile' tool is taught to recognise PANIC_MCE string as one value of CRASH_REASON for which 'slimdump' must be captured.

So again, what is slimdump? I mean, what information is now being captured in the case of slimdump? Are you capturing atleast the kernel message buffers? I am assuming that any register info emitted on console will make into kernel buffers and that should be useful to figure out what MCE happened.

The kernel message buffers can be obtained by using the --dump-dmesg option of makedumpfile but again that's risky. We wouldn't know if it'll cause access to the faulty memory (which is how the previous method of having a new elf-notes in a pristine location is much safer).

Quoted 4 lines

The method in this patch is quite primitive in that informs the user nothing more than a one-line cause of crash. One should take help from other tools (such as service processor/firmware/ACPI logs, or previous corrected error logs) to infer the location of bad memory.

And how does one get to firmware/ACPI logs? Many system don't have service processor also.

I think extracting kernel buffers by default in case of MCE is reasonable. This should allow somebody to figure out some MCE related information.

You might want to modify makedumpfile so that it does not try to access pages marked poisoned.

Thanks
Vivek

author
#14 lore raw
Quoted 3 lines · Vivek Goyal
Quoted 2 lines · K.Prasad
Quoted 1 line · Vivek Goyal

On Mon, Nov 21, 2011 at 03:24:05PM +0530, K.Prasad wrote:

[snipped]

Quoted 11 lines
Quoted 10 lines

The kernel message buffers can be obtained by using the --dump-dmesg option of makedumpfile but again that's risky. We wouldn't know if it'll cause access to the faulty memory (which is how the previous method of having a new elf-notes in a pristine location is much safer).

The method in this patch is quite primitive in that informs the user nothing more than a one-line cause of crash. One should take help from other tools (such as service processor/firmware/ACPI logs, or previous corrected error logs) to infer the location of bad memory.

Quoted 9 lines

And how does one get to firmware/ACPI logs? Many system don't have service processor also.

I think extracting kernel buffers by default in case of MCE is reasonable. This should allow somebody to figure out some MCE related information.

You might want to modify makedumpfile so that it does not try to access pages marked poisoned.

I'm not sure how easy or difficult it would be to skip hw-poisoned pages from user-space i.e. makedumpfile. I'll start working on the relevant changes though, and keep the community posted with the patches.

Thanks,
K.Prasad