crash failure with 2.6.36-rc3 vmcore
A message that is not in the archive (sent off-list or not yet received): 837631540.119231283612666731.JavaMail.root@zmail06.collab.prod.int.phx2.redhat.com
Sorry for the confusion. Here is some background. The test here was to use the crash utility to analyse a vmcore generated from kdump. This commit caused its subcommand mod -S failed to gather the module section data from the in-kernel data structure as it looks like it still depends on this field. I am not sure if that the crash utility just need to be adjusted to cope with this kernel change.
Quoted 1 line
From http://people.redhat.com/anderson/crash_sources/symbols.c
/*
* Gather the module section data from the in-kernel data structures. */
static intadd_symbol_file_kallsyms(struct load_module *lm, struct gnu_request *req)
{
int len, buflen, done, nsections, retval;
ulong vaddr, array_entry, attribute, owner, name, address;
long name_type;
char buf[BUFSIZE];
char section_name[BUFSIZE];
ulong section_vaddr;
if (!(st->flags & (MODSECT_V1|MODSECT_V2|MODSECT_V3|MODSECT_UNKNOWN))) {
STRUCT_SIZE_INIT(module_sect_attr, "module_sect_attr");
MEMBER_OFFSET_INIT(module_sect_attrs,
"module", "sect_attrs");
MEMBER_OFFSET_INIT(module_sect_attrs_attrs,
"module_sect_attrs", "attrs");
MEMBER_OFFSET_INIT(module_sect_attrs_nsections,
"module_sect_attrs", "nsections");
MEMBER_OFFSET_INIT(module_sect_attr_mattr,
"module_sect_attr", "mattr");
MEMBER_OFFSET_INIT(module_sect_attr_name,
"module_sect_attr", "name");
MEMBER_OFFSET_INIT(module_sect_attr_address,
"module_sect_attr", "address");
MEMBER_OFFSET_INIT(module_attribute_attr,
"module_attribute", "attr");
MEMBER_OFFSET_INIT(module_sect_attr_attr,
"module_sect_attr", "attr");
MEMBER_OFFSET_INIT(module_sections_attrs,
"module_sections", "attrs");
MEMBER_OFFSET_INIT(attribute_owner,
"attribute", "owner");
...
owner = attribute + OFFSET(attribute_owner);Quoted 70 lines · ----- "Guenter Roeck
Hi,
Quoted 5 lines · CAI Qian
Quoted 5 lines
crash> mod -S
mod: invalid structure member offset:
attribute_owner
FILE:symbols.cLINE: 8577 FUNCTION:
add_symbol_file_kallsyms()Quoted 4 lines
Quoted 4 lines
MODULE NAME SIZE OBJECT FILE
ffffffffa000de60dm_mod76230
/lib/modules/2.6.36-rc2-mm1-wqfix-mkdfix+/kernel/drivers/md/dm-mod.koQuoted 1 line
Quoted 1 line
[
/usr/bin/crash] error trace: 4affb0 => 4f3236 => 4f12b5 =>4e587a
Quoted 8 lines
Quoted 8 lines
4e587a: OFFSET_verify.clone.4+186
4f12b5: add_symbol_file+933
4f3236: load_module_symbols+566
4affb0: do_module_cmd+1264mod: invalid structure member offset:
attribute_owner
FILE:symbols.cLINE: 8577 FUNCTION:
add_symbol_file_kallsyms()What do I have to do to reproduce this crash ?
Was the module in question compiled w/ the kernel, or separately ? Did it use the correct kernel header files for compilation ? Was the kernel version -rc3, or some other patched version ?
Just wondering, since the above message says 2.6.36-rc2-mm1-wqfix-mkdfix+,
which seems to indicate that it included some modifications.Thanks,
GuenterQuoted 31 lines
This failure was due to this commit,
commit
6fd69dc578fa0b1bbc3aad70ae3af9a137211707Author: Guenter Roeck <[email protected]> Date: Wed Jul 28 22:09:26 2010 -0700sysfs: Remove owner field from sysfs struct attribute
Signed-off-by Guenter Roeck <[email protected]>Acked-by Tejun Heo <[email protected]>Signed-off-by Greg Kroah-Hartman <[email protected]>include/linux/sysfs.h +0
-6@@ -22,14 +22,8 @@ struct kobject;struct module ;enum kobj_ns_type ; -/* FIXME -* The *owner field is no longer used. -* x86 tree has been cleaned up. The owner -* attribute is still left for other arches. -*/ struct attribute {const char *name; -struct module *owner;mode_t mode;#ifdef CONFIG_DEBUG_LOCK_ALLOC struct lock_class_key *key;
Quoted 2 lines · On Sat, Sep 04, 2010 at 11:07:43AM -0400, [email protected] wrote:
Sorry for the confusion. Here is some background. The test here was to use the crash utility to analyse a vmcore generated from kdump. This commit caused its subcommand mod -S failed to gather the module section data from the in-kernel data structure as it looks like it still depends on this field. I am not sure if that the crash utility just need to be adjusted to cope with this kernel change.
That would be the proper fix.
Question is what version of the crash utility you are using. The one included in my distribution (reporting to be version 4.1.0) doesn't include the code below.
Thanks,
Guenter
Quoted 112 lines
From http://people.redhat.com/anderson/crash_sources/symbols.c
/*
* Gather the module section data from the in-kernel data structures. */
static intadd_symbol_file_kallsyms(struct load_module *lm, struct gnu_request *req){ int len, buflen, done, nsections, retval; ulong vaddr, array_entry, attribute, owner, name, address; long name_type; char buf[BUFSIZE]; char section_name[BUFSIZE]; ulong section_vaddr; if (!(st->flags & (MODSECT_V1|MODSECT_V2|MODSECT_V3|MODSECT_UNKNOWN))) { STRUCT_SIZE_INIT(module_sect_attr, "module_sect_attr"); MEMBER_OFFSET_INIT(module_sect_attrs, "module", "sect_attrs"); MEMBER_OFFSET_INIT(module_sect_attrs_attrs, "module_sect_attrs", "attrs"); MEMBER_OFFSET_INIT(module_sect_attrs_nsections, "module_sect_attrs", "nsections"); MEMBER_OFFSET_INIT(module_sect_attr_mattr, "module_sect_attr", "mattr"); MEMBER_OFFSET_INIT(module_sect_attr_name, "module_sect_attr", "name"); MEMBER_OFFSET_INIT(module_sect_attr_address, "module_sect_attr", "address"); MEMBER_OFFSET_INIT(module_attribute_attr, "module_attribute", "attr"); MEMBER_OFFSET_INIT(module_sect_attr_attr, "module_sect_attr", "attr"); MEMBER_OFFSET_INIT(module_sections_attrs, "module_sections", "attrs"); MEMBER_OFFSET_INIT(attribute_owner, "attribute", "owner"); ... owner = attribute + OFFSET(attribute_owner);Quoted 70 lines · ----- "Guenter Roeck
Hi,
Quoted 5 lines · CAI Qian
Quoted 5 lines
crash> mod -S
mod: invalid structure member offset:
attribute_owner
FILE:symbols.cLINE: 8577 FUNCTION:
add_symbol_file_kallsyms()Quoted 4 lines
Quoted 4 lines
MODULE NAME SIZE OBJECT FILE
ffffffffa000de60dm_mod76230
/lib/modules/2.6.36-rc2-mm1-wqfix-mkdfix+/kernel/drivers/md/dm-mod.koQuoted 1 line
Quoted 1 line
[
/usr/bin/crash] error trace: 4affb0 => 4f3236 => 4f12b5 =>4e587a
Quoted 8 lines
Quoted 8 lines
4e587a: OFFSET_verify.clone.4+186
4f12b5: add_symbol_file+933
4f3236: load_module_symbols+566
4affb0: do_module_cmd+1264mod: invalid structure member offset:
attribute_owner
FILE:symbols.cLINE: 8577 FUNCTION:
add_symbol_file_kallsyms()What do I have to do to reproduce this crash ?
Was the module in question compiled w/ the kernel, or separately ? Did it use the correct kernel header files for compilation ? Was the kernel version -rc3, or some other patched version ?
Just wondering, since the above message says 2.6.36-rc2-mm1-wqfix-mkdfix+,
which seems to indicate that it included some modifications.Thanks,
GuenterQuoted 31 lines
This failure was due to this commit,
commit
6fd69dc578fa0b1bbc3aad70ae3af9a137211707Author: Guenter Roeck <[email protected]> Date: Wed Jul 28 22:09:26 2010 -0700sysfs: Remove owner field from sysfs struct attribute
Signed-off-by Guenter Roeck <[email protected]>Acked-by Tejun Heo <[email protected]>Signed-off-by Greg Kroah-Hartman <[email protected]>include/linux/sysfs.h +0
-6@@ -22,14 +22,8 @@ struct kobject;struct module ;enum kobj_ns_type ; -/* FIXME -* The *owner field is no longer used. -* x86 tree has been cleaned up. The owner -* attribute is still left for other arches. -*/ struct attribute {const char *name; -struct module *owner;mode_t mode;#ifdef CONFIG_DEBUG_LOCK_ALLOC struct lock_class_key *key;
Quoted 8 lines · ----- "Guenter Roeck
Quoted 1 line · On Sat, Sep 04, 2010 at 11:07:43AM -0400, [email protected] wrote:
Sorry for the confusion. Here is some background. The test here was
to use the crash utility to analyse a vmcore generated from kdump. This commit caused its subcommand mod -S failed to gather the module section data from the in-kernel data structure as it looks like it still depends on this field. I am not sure if that the crash utility just need to be adjusted to cope with this kernel change.
Quoted 1 line
Quoted 5 lines
That would be the proper fix.
Question is what version of the crash utility you are using. The one included in my distribution (reporting to be version 4.1.0) doesn't include the code below.
The latest upstream version 5.0.7, and can be found here,
http://people.redhat.com/anderson/
Quoted 122 lines
Thanks,
GuenterQuoted 3 lines
From http://people.redhat.com/anderson/crash_sources/symbols.c
/*
* Gather the module section data from the in-kernel datastructures.
Quoted 3 lines
*/
static int
add_symbol_file_kallsyms(structload_module*lm, structgnu_request*req)
Quoted 9 lines
{ int len, buflen, done, nsections, retval; ulong vaddr, array_entry, attribute, owner, name, address; long name_type; char buf[BUFSIZE]; char section_name[BUFSIZE]; ulong section_vaddr; if (!(st->flags &(MODSECT_V1|MODSECT_V2|MODSECT_V3|MODSECT_UNKNOWN))) {
Quoted 1 line
STRUCT_SIZE_INIT(module_sect_attr,
"
module_sect_attr");Quoted 24 lines
MEMBER_OFFSET_INIT(module_sect_attrs, "module", "sect_attrs"); MEMBER_OFFSET_INIT(module_sect_attrs_attrs, "module_sect_attrs", "attrs"); MEMBER_OFFSET_INIT(module_sect_attrs_nsections, "module_sect_attrs", "nsections"); MEMBER_OFFSET_INIT(module_sect_attr_mattr, "module_sect_attr", "mattr"); MEMBER_OFFSET_INIT(module_sect_attr_name, "module_sect_attr", "name"); MEMBER_OFFSET_INIT(module_sect_attr_address, "module_sect_attr", "address"); MEMBER_OFFSET_INIT(module_attribute_attr, "module_attribute", "attr"); MEMBER_OFFSET_INIT(module_sect_attr_attr, "module_sect_attr", "attr"); MEMBER_OFFSET_INIT(module_sections_attrs, "module_sections", "attrs"); MEMBER_OFFSET_INIT(attribute_owner, "attribute", "owner"); ... owner = attribute + OFFSET(attribute_owner);Quoted 16 lines
Quoted 14 lines · ----- "Guenter Roeck
Hi,
Quoted 5 lines · CAI Qian
Quoted 5 lines
crash> mod -S
mod: invalid structure member offset:
attribute_owner
FILE:symbols.cLINE: 8577 FUNCTION:
add_symbol_file_kallsyms()Quoted 4 lines
Quoted 4 lines
MODULE NAME SIZE OBJECT FILE
ffffffffa000de60dm_mod76230
/lib/modules/2.6.36-rc2-mm1-wqfix-mkdfix+/kernel/drivers/md/dm-mod.koQuoted 8 lines
Quoted 8 lines
Quoted 1 line
Quoted 1 line
[
/usr/bin/crash] error trace: 4affb0 => 4f3236 => 4f12b5 =>4e587a
Quoted 6 lines
Quoted 6 lines
4e587a: OFFSET_verify.clone.4+186
4f12b5: add_symbol_file+933
4f3236: load_module_symbols+566
4affb0: do_module_cmd+1264Quoted 7 lines
Quoted 7 lines
Quoted 2 lines
Quoted 2 lines
mod: invalid structure member offset:
attribute_owner
FILE:symbols.cLINE: 8577 FUNCTION:
add_symbol_file_kallsyms()What do I have to do to reproduce this crash ?
Was the module in question compiled w/ the kernel, or separately
?
Quoted 41 lines
Quoted 41 lines
Did it use the correct kernel header files for compilation ? Was the kernel version -rc3, or some other patched version ?
Just wondering, since the above message says 2.6.36-rc2-mm1-wqfix-mkdfix+,
which seems to indicate that it included some modifications.Thanks,
GuenterQuoted 31 lines
This failure was due to this commit,
commit
6fd69dc578fa0b1bbc3aad70ae3af9a137211707Author: Guenter Roeck <[email protected]> Date: Wed Jul 28 22:09:26 2010 -0700sysfs: Remove owner field from sysfs struct attribute
Signed-off-by Guenter Roeck <[email protected]>Acked-by Tejun Heo <[email protected]>Signed-off-by Greg Kroah-Hartman <[email protected]>include/linux/sysfs.h +0
-6@@ -22,14 +22,8 @@ struct kobject;struct module ;enum kobj_ns_type ; -/* FIXME -* The *owner field is no longer used. -* x86 tree has been cleaned up. The owner -* attribute is still left for other arches. -*/ struct attribute {const char *name; -struct module *owner;mode_t mode;#ifdef CONFIG_DEBUG_LOCK_ALLOC struct lock_class_key *key;
Quoted 11 lines · CAI Qian
Quoted 8 lines · ----- "Guenter Roeck
Quoted 1 line · On Sat, Sep 04, 2010 at 11:07:43AM -0400, [email protected] wrote:
Sorry for the confusion. Here is some background. The test here was
to use the crash utility to analyse a vmcore generated from kdump. This commit caused its subcommand mod -S failed to gather the module section data from the in-kernel data structure as it looks like it still depends on this field. I am not sure if that the crash utility just need to be adjusted to cope with this kernel change.
Quoted 1 line
Quoted 7 lines
Quoted 5 lines
That would be the proper fix.
Question is what version of the crash utility you are using. The one included in my distribution (reporting to be version 4.1.0) doesn't include the code below.
The latest upstream version 5.0.7, and can be found here,
http://people.redhat.com/anderson/
I browsed through the code a bit. Looks like the module owner will need to make the necessary changes. The code now seems to depend on kernel internal structure elements. Not sure if that is a good idea in the first place. At least for the attribute owner it will have to be changed to not depend on its existence.
Of course, one could also argue that the field should be re-introduced in the kernel to make the utility work w/o changes. However, I don't think it would be a good idea to hold the kernel hostage for such problems.
Thanks,
Guenter
Quoted 122 lines
Quoted 122 lines
Thanks,
GuenterQuoted 3 lines
From http://people.redhat.com/anderson/crash_sources/symbols.c
/*
* Gather the module section data from the in-kernel datastructures.
Quoted 3 lines
*/
static int
add_symbol_file_kallsyms(structload_module*lm, structgnu_request*req)
Quoted 9 lines
{ int len, buflen, done, nsections, retval; ulong vaddr, array_entry, attribute, owner, name, address; long name_type; char buf[BUFSIZE]; char section_name[BUFSIZE]; ulong section_vaddr; if (!(st->flags &(MODSECT_V1|MODSECT_V2|MODSECT_V3|MODSECT_UNKNOWN))) {
Quoted 1 line
STRUCT_SIZE_INIT(module_sect_attr,
"
module_sect_attr");Quoted 24 lines
MEMBER_OFFSET_INIT(module_sect_attrs, "module", "sect_attrs"); MEMBER_OFFSET_INIT(module_sect_attrs_attrs, "module_sect_attrs", "attrs"); MEMBER_OFFSET_INIT(module_sect_attrs_nsections, "module_sect_attrs", "nsections"); MEMBER_OFFSET_INIT(module_sect_attr_mattr, "module_sect_attr", "mattr"); MEMBER_OFFSET_INIT(module_sect_attr_name, "module_sect_attr", "name"); MEMBER_OFFSET_INIT(module_sect_attr_address, "module_sect_attr", "address"); MEMBER_OFFSET_INIT(module_attribute_attr, "module_attribute", "attr"); MEMBER_OFFSET_INIT(module_sect_attr_attr, "module_sect_attr", "attr"); MEMBER_OFFSET_INIT(module_sections_attrs, "module_sections", "attrs"); MEMBER_OFFSET_INIT(attribute_owner, "attribute", "owner"); ... owner = attribute + OFFSET(attribute_owner);Quoted 16 lines
Quoted 14 lines · ----- "Guenter Roeck
Hi,
Quoted 5 lines · CAI Qian
Quoted 5 lines
crash> mod -S
mod: invalid structure member offset:
attribute_owner
FILE:symbols.cLINE: 8577 FUNCTION:
add_symbol_file_kallsyms()Quoted 4 lines
Quoted 4 lines
MODULE NAME SIZE OBJECT FILE
ffffffffa000de60dm_mod76230
/lib/modules/2.6.36-rc2-mm1-wqfix-mkdfix+/kernel/drivers/md/dm-mod.koQuoted 8 lines
Quoted 8 lines
Quoted 1 line
Quoted 1 line
[
/usr/bin/crash] error trace: 4affb0 => 4f3236 => 4f12b5 =>4e587a
Quoted 6 lines
Quoted 6 lines
4e587a: OFFSET_verify.clone.4+186
4f12b5: add_symbol_file+933
4f3236: load_module_symbols+566
4affb0: do_module_cmd+1264Quoted 7 lines
Quoted 7 lines
Quoted 2 lines
Quoted 2 lines
mod: invalid structure member offset:
attribute_owner
FILE:symbols.cLINE: 8577 FUNCTION:
add_symbol_file_kallsyms()What do I have to do to reproduce this crash ?
Was the module in question compiled w/ the kernel, or separately
?
Quoted 41 lines
Quoted 41 lines
Did it use the correct kernel header files for compilation ? Was the kernel version -rc3, or some other patched version ?
Just wondering, since the above message says 2.6.36-rc2-mm1-wqfix-mkdfix+,
which seems to indicate that it included some modifications.Thanks,
GuenterQuoted 31 lines
This failure was due to this commit,
commit
6fd69dc578fa0b1bbc3aad70ae3af9a137211707Author: Guenter Roeck <[email protected]> Date: Wed Jul 28 22:09:26 2010 -0700sysfs: Remove owner field from sysfs struct attribute
Signed-off-by Guenter Roeck <[email protected]>Acked-by Tejun Heo <[email protected]>Signed-off-by Greg Kroah-Hartman <[email protected]>include/linux/sysfs.h +0
-6@@ -22,14 +22,8 @@ struct kobject;struct module ;enum kobj_ns_type ; -/* FIXME -* The *owner field is no longer used. -* x86 tree has been cleaned up. The owner -* attribute is still left for other arches. -*/ struct attribute {const char *name; -struct module *owner;mode_t mode;#ifdef CONFIG_DEBUG_LOCK_ALLOC struct lock_class_key *key;
Quoted 8 lines · Guenter Roeck
I browsed through the code a bit. Looks like the module owner will need to make the necessary changes. The code now seems to depend on kernel internal structure elements. Not sure if that is a good idea in the first place. At least for the attribute owner it will have to be changed to not depend on its existence.
Of course, one could also argue that the field should be re-introduced in the kernel to make the utility work w/o changes. However, I don't think it would be a good idea to hold the kernel hostage for such problems.
You're kidding, right? You should have tools flexable to handle the fact that the internal kernel structures change all the time like this. If not, your tools are broken.
good luck,
greg k-h