Skip to contentSkip to conversation

crash failure with 2.6.36-rc3 vmcore

DISCUSSION [email protected] 3 participants 5 messages Sep 4, 2010 – Sep 5, 2010 1 message not in the archive

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): 837631540.119231283612666731.JavaMail.root@zmail06.collab.prod.int.phx2.redhat.com

#1 lore raw

Sorry for the confusion. Here is some background. The test here was to use the crash utility to analyse a vmcore generated from kdump. This commit caused its subcommand mod -S failed to gather the module section data from the in-kernel data structure as it looks like it still depends on this field. I am not sure if that the crash utility just need to be adjusted to cope with this kernel change.

Quoted 1 line

From http://people.redhat.com/anderson/crash_sources/symbols.c

/*
* Gather the module section data from the in-kernel data structures. */
static int
add_symbol_file_kallsyms(struct load_module *lm, struct gnu_request *req)

{
        int len, buflen, done, nsections, retval;
        ulong vaddr, array_entry, attribute, owner, name, address;
        long name_type;
        char buf[BUFSIZE];
        char section_name[BUFSIZE];
        ulong section_vaddr;

        if (!(st->flags & (MODSECT_V1|MODSECT_V2|MODSECT_V3|MODSECT_UNKNOWN))) {
                STRUCT_SIZE_INIT(module_sect_attr, "module_sect_attr");
                MEMBER_OFFSET_INIT(module_sect_attrs,
                        "module", "sect_attrs");
                MEMBER_OFFSET_INIT(module_sect_attrs_attrs,
                        "module_sect_attrs", "attrs");
                MEMBER_OFFSET_INIT(module_sect_attrs_nsections,
                        "module_sect_attrs", "nsections");
                MEMBER_OFFSET_INIT(module_sect_attr_mattr,
                        "module_sect_attr", "mattr");
                MEMBER_OFFSET_INIT(module_sect_attr_name,
                        "module_sect_attr", "name");
                MEMBER_OFFSET_INIT(module_sect_attr_address,
                        "module_sect_attr", "address");
                MEMBER_OFFSET_INIT(module_attribute_attr,
                        "module_attribute", "attr");
                MEMBER_OFFSET_INIT(module_sect_attr_attr,
                        "module_sect_attr", "attr");
                MEMBER_OFFSET_INIT(module_sections_attrs,
                        "module_sections", "attrs");
                MEMBER_OFFSET_INIT(attribute_owner,
                        "attribute", "owner");
...
                owner = attribute + OFFSET(attribute_owner);
Quoted 70 lines · ----- "Guenter Roeck

Hi,

Quoted 5 lines · CAI Qian
Quoted 5 lines

crash> mod -S

mod: invalid structure member offset: attribute_owner
FILE: symbols.c LINE: 8577 FUNCTION:

add_symbol_file_kallsyms()

Quoted 4 lines
Quoted 4 lines

MODULE NAME SIZE OBJECT FILE
ffffffffa000de60 dm_mod 76230

/lib/modules/2.6.36-rc2-mm1-wqfix-mkdfix+/kernel/drivers/md/dm-mod.ko

Quoted 1 line
Quoted 1 line

[/usr/bin/crash] error trace: 4affb0 => 4f3236 => 4f12b5 =>

4e587a

Quoted 8 lines
Quoted 8 lines

4e587a: OFFSET_verify.clone.4+186
4f12b5: add_symbol_file+933
4f3236: load_module_symbols+566
4affb0: do_module_cmd+1264

mod: invalid structure member offset: attribute_owner
FILE: symbols.c LINE: 8577 FUNCTION:

add_symbol_file_kallsyms()

What do I have to do to reproduce this crash ?

Was the module in question compiled w/ the kernel, or separately ? Did it use the correct kernel header files for compilation ? Was the kernel version -rc3, or some other patched version ?

Just wondering, since the above message says 2.6.36-rc2-mm1-wqfix-mkdfix+,
which seems to indicate that it included some modifications.

Thanks,
Guenter

Quoted 31 lines

This failure was due to this commit,

commit 6fd69dc578fa0b1bbc3aad70ae3af9a137211707

Author: Guenter Roeck <[email protected]>
Date:   Wed Jul 28 22:09:26 2010 -0700

sysfs: Remove owner field from sysfs struct attribute

include/linux/sysfs.h +0 -6
index 8bf06b6..3c92121 100644
@@ -22,14 +22,8 @@ struct kobject;
 struct module;
 enum kobj_ns_type;
 
-/* FIXME
- * The *owner field is no longer used.
- * x86 tree has been cleaned up. The owner
- * attribute is still left for other arches.
- */
 struct attribute {
        const char              *name;
-       struct module           *owner;
        mode_t                  mode;
 #ifdef CONFIG_DEBUG_LOCK_ALLOC
        struct lock_class_key   *key;
#2 lore raw
Quoted 2 lines · On Sat, Sep 04, 2010 at 11:07:43AM -0400, [email protected] wrote:

Sorry for the confusion. Here is some background. The test here was to use the crash utility to analyse a vmcore generated from kdump. This commit caused its subcommand mod -S failed to gather the module section data from the in-kernel data structure as it looks like it still depends on this field. I am not sure if that the crash utility just need to be adjusted to cope with this kernel change.

That would be the proper fix.

Question is what version of the crash utility you are using. The one included in my distribution (reporting to be version 4.1.0) doesn't include the code below.

Thanks,
Guenter

Quoted 112 lines

From http://people.redhat.com/anderson/crash_sources/symbols.c
/*
* Gather the module section data from the in-kernel data structures. */
static int
add_symbol_file_kallsyms(struct load_module *lm, struct gnu_request *req)

{
        int len, buflen, done, nsections, retval;
        ulong vaddr, array_entry, attribute, owner, name, address;
        long name_type;
        char buf[BUFSIZE];
        char section_name[BUFSIZE];
        ulong section_vaddr;

        if (!(st->flags & (MODSECT_V1|MODSECT_V2|MODSECT_V3|MODSECT_UNKNOWN))) {
                STRUCT_SIZE_INIT(module_sect_attr, "module_sect_attr");
                MEMBER_OFFSET_INIT(module_sect_attrs,
                        "module", "sect_attrs");
                MEMBER_OFFSET_INIT(module_sect_attrs_attrs,
                        "module_sect_attrs", "attrs");
                MEMBER_OFFSET_INIT(module_sect_attrs_nsections,
                        "module_sect_attrs", "nsections");
                MEMBER_OFFSET_INIT(module_sect_attr_mattr,
                        "module_sect_attr", "mattr");
                MEMBER_OFFSET_INIT(module_sect_attr_name,
                        "module_sect_attr", "name");
                MEMBER_OFFSET_INIT(module_sect_attr_address,
                        "module_sect_attr", "address");
                MEMBER_OFFSET_INIT(module_attribute_attr,
                        "module_attribute", "attr");
                MEMBER_OFFSET_INIT(module_sect_attr_attr,
                        "module_sect_attr", "attr");
                MEMBER_OFFSET_INIT(module_sections_attrs,
                        "module_sections", "attrs");
                MEMBER_OFFSET_INIT(attribute_owner,
                        "attribute", "owner");
...
                owner = attribute + OFFSET(attribute_owner);
Quoted 70 lines · ----- "Guenter Roeck

Hi,

Quoted 5 lines · CAI Qian
Quoted 5 lines

crash> mod -S

mod: invalid structure member offset: attribute_owner
FILE: symbols.c LINE: 8577 FUNCTION:

add_symbol_file_kallsyms()

Quoted 4 lines
Quoted 4 lines

MODULE NAME SIZE OBJECT FILE
ffffffffa000de60 dm_mod 76230

/lib/modules/2.6.36-rc2-mm1-wqfix-mkdfix+/kernel/drivers/md/dm-mod.ko

Quoted 1 line
Quoted 1 line

[/usr/bin/crash] error trace: 4affb0 => 4f3236 => 4f12b5 =>

4e587a

Quoted 8 lines
Quoted 8 lines

4e587a: OFFSET_verify.clone.4+186
4f12b5: add_symbol_file+933
4f3236: load_module_symbols+566
4affb0: do_module_cmd+1264

mod: invalid structure member offset: attribute_owner
FILE: symbols.c LINE: 8577 FUNCTION:

add_symbol_file_kallsyms()

What do I have to do to reproduce this crash ?

Was the module in question compiled w/ the kernel, or separately ? Did it use the correct kernel header files for compilation ? Was the kernel version -rc3, or some other patched version ?

Just wondering, since the above message says 2.6.36-rc2-mm1-wqfix-mkdfix+,
which seems to indicate that it included some modifications.

Thanks,
Guenter

Quoted 31 lines

This failure was due to this commit,

commit 6fd69dc578fa0b1bbc3aad70ae3af9a137211707

Author: Guenter Roeck <[email protected]>
Date:   Wed Jul 28 22:09:26 2010 -0700

sysfs: Remove owner field from sysfs struct attribute

include/linux/sysfs.h +0 -6
index 8bf06b6..3c92121 100644
@@ -22,14 +22,8 @@ struct kobject;
 struct module;
 enum kobj_ns_type;
 
-/* FIXME
- * The *owner field is no longer used.
- * x86 tree has been cleaned up. The owner
- * attribute is still left for other arches.
- */
 struct attribute {
        const char              *name;
-       struct module           *owner;
        mode_t                  mode;
 #ifdef CONFIG_DEBUG_LOCK_ALLOC
        struct lock_class_key   *key;
author
#3 lore raw
Quoted 8 lines · ----- "Guenter Roeck
Quoted 1 line · On Sat, Sep 04, 2010 at 11:07:43AM -0400, [email protected] wrote:

Sorry for the confusion. Here is some background. The test here was

to use the crash utility to analyse a vmcore generated from kdump. This commit caused its subcommand mod -S failed to gather the module section data from the in-kernel data structure as it looks like it still depends on this field. I am not sure if that the crash utility just need to be adjusted to cope with this kernel change.

Quoted 1 line
Quoted 5 lines

That would be the proper fix.

Question is what version of the crash utility you are using. The one included in my distribution (reporting to be version 4.1.0) doesn't include the code below.

The latest upstream version 5.0.7, and can be found here,
http://people.redhat.com/anderson/

Quoted 122 lines

Thanks,
Guenter

Quoted 3 lines

From http://people.redhat.com/anderson/crash_sources/symbols.c
/*
* Gather the module section data from the in-kernel data

structures.

Quoted 3 lines

*/
static int
add_symbol_file_kallsyms(struct load_module *lm, struct gnu_request

*req)

Quoted 9 lines
{
        int len, buflen, done, nsections, retval;
        ulong vaddr, array_entry, attribute, owner, name, address;
        long name_type;
        char buf[BUFSIZE];
        char section_name[BUFSIZE];
        ulong section_vaddr;

        if (!(st->flags &

(MODSECT_V1|MODSECT_V2|MODSECT_V3|MODSECT_UNKNOWN))) {

Quoted 1 line

STRUCT_SIZE_INIT(module_sect_attr,

"module_sect_attr");

Quoted 24 lines
                MEMBER_OFFSET_INIT(module_sect_attrs,
                        "module", "sect_attrs");
                MEMBER_OFFSET_INIT(module_sect_attrs_attrs,
                        "module_sect_attrs", "attrs");
                MEMBER_OFFSET_INIT(module_sect_attrs_nsections,
                        "module_sect_attrs", "nsections");
                MEMBER_OFFSET_INIT(module_sect_attr_mattr,
                        "module_sect_attr", "mattr");
                MEMBER_OFFSET_INIT(module_sect_attr_name,
                        "module_sect_attr", "name");
                MEMBER_OFFSET_INIT(module_sect_attr_address,
                        "module_sect_attr", "address");
                MEMBER_OFFSET_INIT(module_attribute_attr,
                        "module_attribute", "attr");
                MEMBER_OFFSET_INIT(module_sect_attr_attr,
                        "module_sect_attr", "attr");
                MEMBER_OFFSET_INIT(module_sections_attrs,
                        "module_sections", "attrs");
                MEMBER_OFFSET_INIT(attribute_owner,
                        "attribute", "owner");
...
                owner = attribute + OFFSET(attribute_owner);
Quoted 16 lines
Quoted 14 lines · ----- "Guenter Roeck

Hi,

Quoted 5 lines · CAI Qian
Quoted 5 lines

crash> mod -S

mod: invalid structure member offset: attribute_owner
FILE: symbols.c LINE: 8577 FUNCTION:

add_symbol_file_kallsyms()

Quoted 4 lines
Quoted 4 lines

MODULE NAME SIZE OBJECT FILE
ffffffffa000de60 dm_mod 76230

/lib/modules/2.6.36-rc2-mm1-wqfix-mkdfix+/kernel/drivers/md/dm-mod.ko

Quoted 8 lines
Quoted 8 lines
Quoted 1 line
Quoted 1 line

[/usr/bin/crash] error trace: 4affb0 => 4f3236 => 4f12b5 =>

4e587a

Quoted 6 lines
Quoted 6 lines

4e587a: OFFSET_verify.clone.4+186
4f12b5: add_symbol_file+933
4f3236: load_module_symbols+566
4affb0: do_module_cmd+1264

Quoted 7 lines
Quoted 7 lines
Quoted 2 lines
Quoted 2 lines

mod: invalid structure member offset: attribute_owner
FILE: symbols.c LINE: 8577 FUNCTION:

add_symbol_file_kallsyms()

What do I have to do to reproduce this crash ?

Was the module in question compiled w/ the kernel, or separately

?

Quoted 41 lines
Quoted 41 lines

Did it use the correct kernel header files for compilation ? Was the kernel version -rc3, or some other patched version ?

Just wondering, since the above message says 2.6.36-rc2-mm1-wqfix-mkdfix+,
which seems to indicate that it included some modifications.

Thanks,
Guenter

Quoted 31 lines

This failure was due to this commit,

commit 6fd69dc578fa0b1bbc3aad70ae3af9a137211707

Author: Guenter Roeck <[email protected]>
Date:   Wed Jul 28 22:09:26 2010 -0700

sysfs: Remove owner field from sysfs struct attribute

include/linux/sysfs.h +0 -6
index 8bf06b6..3c92121 100644
@@ -22,14 +22,8 @@ struct kobject;
 struct module;
 enum kobj_ns_type;
 
-/* FIXME
- * The *owner field is no longer used.
- * x86 tree has been cleaned up. The owner
- * attribute is still left for other arches.
- */
 struct attribute {
        const char              *name;
-       struct module           *owner;
        mode_t                  mode;
 #ifdef CONFIG_DEBUG_LOCK_ALLOC
        struct lock_class_key   *key;
#4 lore raw
Quoted 11 lines · CAI Qian
Quoted 8 lines · ----- "Guenter Roeck
Quoted 1 line · On Sat, Sep 04, 2010 at 11:07:43AM -0400, [email protected] wrote:

Sorry for the confusion. Here is some background. The test here was

to use the crash utility to analyse a vmcore generated from kdump. This commit caused its subcommand mod -S failed to gather the module section data from the in-kernel data structure as it looks like it still depends on this field. I am not sure if that the crash utility just need to be adjusted to cope with this kernel change.

Quoted 1 line
Quoted 7 lines
Quoted 5 lines

That would be the proper fix.

Question is what version of the crash utility you are using. The one included in my distribution (reporting to be version 4.1.0) doesn't include the code below.

The latest upstream version 5.0.7, and can be found here,
http://people.redhat.com/anderson/

I browsed through the code a bit. Looks like the module owner will need to make the necessary changes. The code now seems to depend on kernel internal structure elements. Not sure if that is a good idea in the first place. At least for the attribute owner it will have to be changed to not depend on its existence.

Of course, one could also argue that the field should be re-introduced in the kernel to make the utility work w/o changes. However, I don't think it would be a good idea to hold the kernel hostage for such problems.

Thanks,
Guenter

Quoted 122 lines
Quoted 122 lines

Thanks,
Guenter

Quoted 3 lines

From http://people.redhat.com/anderson/crash_sources/symbols.c
/*
* Gather the module section data from the in-kernel data

structures.

Quoted 3 lines

*/
static int
add_symbol_file_kallsyms(struct load_module *lm, struct gnu_request

*req)

Quoted 9 lines
{
        int len, buflen, done, nsections, retval;
        ulong vaddr, array_entry, attribute, owner, name, address;
        long name_type;
        char buf[BUFSIZE];
        char section_name[BUFSIZE];
        ulong section_vaddr;

        if (!(st->flags &

(MODSECT_V1|MODSECT_V2|MODSECT_V3|MODSECT_UNKNOWN))) {

Quoted 1 line

STRUCT_SIZE_INIT(module_sect_attr,

"module_sect_attr");

Quoted 24 lines
                MEMBER_OFFSET_INIT(module_sect_attrs,
                        "module", "sect_attrs");
                MEMBER_OFFSET_INIT(module_sect_attrs_attrs,
                        "module_sect_attrs", "attrs");
                MEMBER_OFFSET_INIT(module_sect_attrs_nsections,
                        "module_sect_attrs", "nsections");
                MEMBER_OFFSET_INIT(module_sect_attr_mattr,
                        "module_sect_attr", "mattr");
                MEMBER_OFFSET_INIT(module_sect_attr_name,
                        "module_sect_attr", "name");
                MEMBER_OFFSET_INIT(module_sect_attr_address,
                        "module_sect_attr", "address");
                MEMBER_OFFSET_INIT(module_attribute_attr,
                        "module_attribute", "attr");
                MEMBER_OFFSET_INIT(module_sect_attr_attr,
                        "module_sect_attr", "attr");
                MEMBER_OFFSET_INIT(module_sections_attrs,
                        "module_sections", "attrs");
                MEMBER_OFFSET_INIT(attribute_owner,
                        "attribute", "owner");
...
                owner = attribute + OFFSET(attribute_owner);
Quoted 16 lines
Quoted 14 lines · ----- "Guenter Roeck

Hi,

Quoted 5 lines · CAI Qian
Quoted 5 lines

crash> mod -S

mod: invalid structure member offset: attribute_owner
FILE: symbols.c LINE: 8577 FUNCTION:

add_symbol_file_kallsyms()

Quoted 4 lines
Quoted 4 lines

MODULE NAME SIZE OBJECT FILE
ffffffffa000de60 dm_mod 76230

/lib/modules/2.6.36-rc2-mm1-wqfix-mkdfix+/kernel/drivers/md/dm-mod.ko

Quoted 8 lines
Quoted 8 lines
Quoted 1 line
Quoted 1 line

[/usr/bin/crash] error trace: 4affb0 => 4f3236 => 4f12b5 =>

4e587a

Quoted 6 lines
Quoted 6 lines

4e587a: OFFSET_verify.clone.4+186
4f12b5: add_symbol_file+933
4f3236: load_module_symbols+566
4affb0: do_module_cmd+1264

Quoted 7 lines
Quoted 7 lines
Quoted 2 lines
Quoted 2 lines

mod: invalid structure member offset: attribute_owner
FILE: symbols.c LINE: 8577 FUNCTION:

add_symbol_file_kallsyms()

What do I have to do to reproduce this crash ?

Was the module in question compiled w/ the kernel, or separately

?

Quoted 41 lines
Quoted 41 lines

Did it use the correct kernel header files for compilation ? Was the kernel version -rc3, or some other patched version ?

Just wondering, since the above message says 2.6.36-rc2-mm1-wqfix-mkdfix+,
which seems to indicate that it included some modifications.

Thanks,
Guenter

Quoted 31 lines

This failure was due to this commit,

commit 6fd69dc578fa0b1bbc3aad70ae3af9a137211707

Author: Guenter Roeck <[email protected]>
Date:   Wed Jul 28 22:09:26 2010 -0700

sysfs: Remove owner field from sysfs struct attribute

include/linux/sysfs.h +0 -6
index 8bf06b6..3c92121 100644
@@ -22,14 +22,8 @@ struct kobject;
 struct module;
 enum kobj_ns_type;
 
-/* FIXME
- * The *owner field is no longer used.
- * x86 tree has been cleaned up. The owner
- * attribute is still left for other arches.
- */
 struct attribute {
        const char              *name;
-       struct module           *owner;
        mode_t                  mode;
 #ifdef CONFIG_DEBUG_LOCK_ALLOC
        struct lock_class_key   *key;
#5 lore raw
Quoted 8 lines · Guenter Roeck

I browsed through the code a bit. Looks like the module owner will need to make the necessary changes. The code now seems to depend on kernel internal structure elements. Not sure if that is a good idea in the first place. At least for the attribute owner it will have to be changed to not depend on its existence.

Of course, one could also argue that the field should be re-introduced in the kernel to make the utility work w/o changes. However, I don't think it would be a good idea to hold the kernel hostage for such problems.

You're kidding, right? You should have tools flexable to handle the fact that the internal kernel structures change all the time like this. If not, your tools are broken.

good luck,

greg k-h