borked sysfs system devices in 2.5.72
The per-node numa meminfo files in 2.5.72 are broken, the only display node0's information. The devices are being properly registered:
Registering sys device 'node0':c0423844 id:0 kobj:c042384c Registering sys device 'node1':c0423888 id:1 kobj:c0423890 Registering sys device 'node2':c04238cc id:2 kobj:c04238d4 Registering sys device 'node3':c0423910 id:3 kobj:c0423918
When I look at the 4 nodes files with:
"cat /sys/devices/system/node/*/meminfo", I printed out some information:subsys_attr_show(kobj: c042384c, attr: c033ea30, page: e76ba000) subsys_attr_show(kobj: c0423890, attr: c033ea30, page: e76ba000) subsys_attr_show(kobj: c04238d4, attr: c033ea30, page: e76ba000) subsys_attr_show(kobj: c0423918, attr: c033ea30, page: e76ba000)
As you can see, the kobj is the one which belongs to the sys device, yet you do a to_subsys() on it. Why?
struct subsystem * s = to_subsys(kobj);
I'm getting a 0 as the node ID out of pure dumb luck. Is the NUMA code broken or is sysfs?
Signature · 2 lines
Dave Hansen [email protected]
Quoted 15 lines · Dave Hansen
The per-node numa meminfo files in 2.5.72 are broken, the only display node0's information. The devices are being properly registered:
Registering sys device 'node0':c0423844 id:0 kobj:c042384c Registering sys device 'node1':c0423888 id:1 kobj:c0423890 Registering sys device 'node2':c04238cc id:2 kobj:c04238d4 Registering sys device 'node3':c0423910 id:3 kobj:c0423918When I look at the 4 nodes files with:
"cat/sys/devices/system/node/*/meminfo", I printed out some information:subsys_attr_show(kobj: c042384c, attr: c033ea30, page: e76ba000)subsys_attr_show(kobj: c0423890, attr: c033ea30, page: e76ba000)subsys_attr_show(kobj: c04238d4, attr: c033ea30, page: e76ba000)subsys_attr_show(kobj: c0423918, attr: c033ea30, page: e76ba000)
Quoted 3 lines
As you can see, the kobj is the one which belongs to the sys device, yet you do a
to_subsys()on it. Why?
struct subsystem * s =to_subsys(kobj);
Where exactly is that happening?
Quoted 2 lines
I'm getting a 0 as the node ID out of pure dumb luck. Is the NUMA code broken or is sysfs?
It's taking the ID from the system device of the node that's passed to ->show(). That is set in register_node(), so I'm not sure how they could get out of sync, unless I'm missing something obvious.
BTW, I did request that the NUMA topology people take a look at these patches when I sent them a couple of weeks ago, so as to avoid this. :)
-pat
Quoted 17 lines · Patrick Mochel
Quoted 15 lines · Dave Hansen
The per-node numa meminfo files in 2.5.72 are broken, the only display node0's information. The devices are being properly registered:
Registering sys device 'node0':c0423844 id:0 kobj:c042384c Registering sys device 'node1':c0423888 id:1 kobj:c0423890 Registering sys device 'node2':c04238cc id:2 kobj:c04238d4 Registering sys device 'node3':c0423910 id:3 kobj:c0423918When I look at the 4 nodes files with:
"cat/sys/devices/system/node/*/meminfo", I printed out some information:subsys_attr_show(kobj: c042384c, attr: c033ea30, page: e76ba000)subsys_attr_show(kobj: c0423890, attr: c033ea30, page: e76ba000)subsys_attr_show(kobj: c04238d4, attr: c033ea30, page: e76ba000)subsys_attr_show(kobj: c0423918, attr: c033ea30, page: e76ba000)
Quoted 5 lines
Quoted 3 lines
As you can see, the kobj is the one which belongs to the sys device, yet you do a
to_subsys()on it. Why?
struct subsystem * s =to_subsys(kobj);Where exactly is that happening?
Look in subsys_attr_show(). It is being passed a kobject, which is a member of a "struct sys_device". We can tell this because I printed out the address of the sys device in sys_device_register(). A to_subsys() is being performed on that object, which is wrong, because the kobject is not a member of a "struct subsystem".
Quoted 6 lines
Quoted 2 lines
I'm getting a 0 as the node ID out of pure dumb luck. Is the NUMA code broken or is sysfs?
It's taking the ID from the system device of the node that's passed to ->show(). That is set in
register_node(), so I'm not sure how they could get out of sync, unless I'm missing something obvious.
They're not out of sync, it's getting garbage because of the bogus to_subsys().
Quoted 2 lines
BTW, I did request that the NUMA topology people take a look at these patches when I sent them a couple of weeks ago, so as to avoid this. :)
I'll cc him to make him feel bad :)
Signature · 2 lines
Dave Hansen [email protected]
Quoted 6 lines
Look in
subsys_attr_show(). It is being passed a kobject, which is a member of a "structsys_device". We can tell this because I printed out the address of the sys device insys_device_register(). Ato_subsys()is being performed on that object, which is wrong, because the kobject is not a member of a "struct subsystem".
My question was how the hell it was getting there in the first place, and I see that the type of the object isn't getting set properly, so it defaults to treat it as a struct subsystem.
Could you please try the following patch, and let me know if it works?
Thanks,
-pat
===== drivers/base/sys.c 1.25 vs edited =====
drivers/base/sys.c +3 -0
@@ -170,6 +172,9 @@/* Make sure the kset is set */ sysdev->kobj.kset = &cls->kset; + +/* But make sure we point to the right type for sysfs translation */ + sysdev->kobj.ktype = &ktype_sysdev;/* set the kobject name */ snprintf(sysdev->kobj.name,KOBJ_NAME_LEN,"%s%d",
Quoted 9 lines · Patrick Mochel
Quoted 5 lines
Look in
subsys_attr_show(). It is being passed a kobject, which is a member of a "structsys_device". We can tell this because I printed out the address of the sys device insys_device_register(). Ato_subsys()is being performed on that object, which is wrong, because the kobject is not a member of a "struct subsystem".My question was how the hell it was getting there in the first place, and I see that the type of the object isn't getting set properly, so it defaults to treat it as a struct subsystem.
Stack dump from si_meminfo_node():
Call Trace: [<c0133f39>] si_meminfo_node+0x4d/0x54 [<c02029ac>] node_read_meminfo+0x1c/0x80 [<c013385a>] __alloc_pages+0x82/0x2b4 [<c011c8fb>] release_console_sem+0x9b/0xa4 [<c0175ead>] subsys_attr_show+0x1d/0x28 [<c0175f7a>] fill_read_buffer+0x96/0xb4 [<c01ebf5e>] opost_block+0x18e/0x19c [<c01eed92>] pty_write+0x156/0x168 [<c0154d8c>] do_lookup+0x18/0x8c [<c0151a2f>] cp_new_stat64+0xe7/0x100 [<c017604b>] sysfs_read_file+0x1b/0x3c [<c014960c>] vfs_read+0x9c/0xcc [<c01497ed>] sys_read+0x31/0x4c [<c0108bd7>] syscall_call+0x7/0xb
Quoted 1 line
Could you please try the following patch, and let me know if it works?
That fixed it, thanks.
Signature · 2 lines
Dave Hansen [email protected]