Skip to contentSkip to conversation

BUG: using smp_processor_id() in preemptible (VFS-related)

git head f4b87dee9 Rafael J. Wysocki 4 participants 5 messages May 22, 2010 – May 24, 2010

lore
Share & tools
lore
#0 lore raw

Hi,

I get this every time on boot with current -git:

[    6.906133] EXT4-fs (sda7): mounted filesystem with ordered data mode
[    7.014359] BUG: using smp_processor_id() in preemptible [00000000] code: mount/1501
[    7.020570] caller is dqstats_inc+0x19/0x2c
[    7.026658] Pid: 1501, comm: mount Not tainted 2.6.34-tst #25
[    7.032792] Call Trace:
[    7.038859]  [<ffffffff8117c12b>] debug_smp_processor_id+0xcb/0xdc
[    7.045025]  [<ffffffff8111dfb5>] dqstats_inc+0x19/0x2c
[    7.051201]  [<ffffffff81120fac>] vfs_quota_sync+0x144/0x210
[    7.057349]  [<ffffffff810eccfa>] ? __shrink_dcache_sb+0x284/0x293
[    7.063574]  [<ffffffff810fc48d>] __sync_filesystem+0x3a/0x7e
[    7.069730]  [<ffffffff810fc552>] sync_filesystem+0x36/0x4c
[    7.075867]  [<ffffffff810deef0>] do_remount_sb+0x65/0x13c
[    7.081960]  [<ffffffff810f4359>] do_mount+0x251/0x78a
[    7.088039]  [<ffffffff810f2c3e>] ? copy_mount_options+0xda/0x146
[    7.094086]  [<ffffffff810f4911>] sys_mount+0x7f/0xb8
[    7.100085]  [<ffffffff81002aeb>] system_call_fastpath+0x16/0x1b
[    7.124208] BUG: using smp_processor_id() in preemptible [00000000] code: mount/1501
[    7.130307] caller is dqstats_inc+0x19/0x2c
[    7.136386] Pid: 1501, comm: mount Not tainted 2.6.34-tst #25
[    7.142512] Call Trace:
[    7.148629]  [<ffffffff8117c12b>] debug_smp_processor_id+0xcb/0xdc
[    7.154846]  [<ffffffff8111dfb5>] dqstats_inc+0x19/0x2c
[    7.161019]  [<ffffffff81120fac>] vfs_quota_sync+0x144/0x210
[    7.167171]  [<ffffffff810fc48d>] __sync_filesystem+0x3a/0x7e
[    7.173364]  [<ffffffff810fc563>] sync_filesystem+0x47/0x4c
[    7.179522]  [<ffffffff810deef0>] do_remount_sb+0x65/0x13c
[    7.185616]  [<ffffffff810f4359>] do_mount+0x251/0x78a
[    7.191663]  [<ffffffff810f2c3e>] ? copy_mount_options+0xda/0x146
[    7.197626]  [<ffffffff810f4911>] sys_mount+0x7f/0xb8
[    7.203493]  [<ffffffff81002aeb>] system_call_fastpath+0x16/0x1b
[   11.643596] udev: starting version 153

This is after reverting commit a7cf4145b (anon_inode: set S_IFREG on the anon_inode) that the system is not really useable with.

Thanks,
Rafael

#1 lore raw
Quoted 36 lines · On Sat, 22 May 2010 23:36:59 +0200 "Rafael J. Wysocki" <[email protected]> wrote:

Hi,

I get this every time on boot with current -git:

[    6.906133] EXT4-fs (sda7): mounted filesystem with ordered data mode
[    7.014359] BUG: using smp_processor_id() in preemptible [00000000] code: mount/1501
[    7.020570] caller is dqstats_inc+0x19/0x2c
[    7.026658] Pid: 1501, comm: mount Not tainted 2.6.34-tst #25
[    7.032792] Call Trace:
[    7.038859]  [<ffffffff8117c12b>] debug_smp_processor_id+0xcb/0xdc
[    7.045025]  [<ffffffff8111dfb5>] dqstats_inc+0x19/0x2c
[    7.051201]  [<ffffffff81120fac>] vfs_quota_sync+0x144/0x210
[    7.057349]  [<ffffffff810eccfa>] ? __shrink_dcache_sb+0x284/0x293
[    7.063574]  [<ffffffff810fc48d>] __sync_filesystem+0x3a/0x7e
[    7.069730]  [<ffffffff810fc552>] sync_filesystem+0x36/0x4c
[    7.075867]  [<ffffffff810deef0>] do_remount_sb+0x65/0x13c
[    7.081960]  [<ffffffff810f4359>] do_mount+0x251/0x78a
[    7.088039]  [<ffffffff810f2c3e>] ? copy_mount_options+0xda/0x146
[    7.094086]  [<ffffffff810f4911>] sys_mount+0x7f/0xb8
[    7.100085]  [<ffffffff81002aeb>] system_call_fastpath+0x16/0x1b
[    7.124208] BUG: using smp_processor_id() in preemptible [00000000] code: mount/1501
[    7.130307] caller is dqstats_inc+0x19/0x2c
[    7.136386] Pid: 1501, comm: mount Not tainted 2.6.34-tst #25
[    7.142512] Call Trace:
[    7.148629]  [<ffffffff8117c12b>] debug_smp_processor_id+0xcb/0xdc
[    7.154846]  [<ffffffff8111dfb5>] dqstats_inc+0x19/0x2c
[    7.161019]  [<ffffffff81120fac>] vfs_quota_sync+0x144/0x210
[    7.167171]  [<ffffffff810fc48d>] __sync_filesystem+0x3a/0x7e
[    7.173364]  [<ffffffff810fc563>] sync_filesystem+0x47/0x4c
[    7.179522]  [<ffffffff810deef0>] do_remount_sb+0x65/0x13c
[    7.185616]  [<ffffffff810f4359>] do_mount+0x251/0x78a
[    7.191663]  [<ffffffff810f2c3e>] ? copy_mount_options+0xda/0x146
[    7.197626]  [<ffffffff810f4911>] sys_mount+0x7f/0xb8
[    7.203493]  [<ffffffff81002aeb>] system_call_fastpath+0x16/0x1b
[   11.643596] udev: starting version 153
Quoted 2 lines

This is after reverting commit a7cf4145b (anon_inode: set S_IFREG on the anon_inode) that the system is not really useable with.

Caused by "quota: Make quota stat accounting lockless".

Guys, the code this patch adds utterly duplicates the interface provided by include/linux/percpu_counter.h, only the quota code does it wrongly and badly. Please, can we use the nice library code?

#2 lore raw
Quoted 38 lines · Andrew Morton
Quoted 36 lines · On Sat, 22 May 2010 23:36:59 +0200 "Rafael J. Wysocki" <[email protected]> wrote:

Hi,

I get this every time on boot with current -git:

[    6.906133] EXT4-fs (sda7): mounted filesystem with ordered data mode
[    7.014359] BUG: using smp_processor_id() in preemptible [00000000] code: mount/1501
[    7.020570] caller is dqstats_inc+0x19/0x2c
[    7.026658] Pid: 1501, comm: mount Not tainted 2.6.34-tst #25
[    7.032792] Call Trace:
[    7.038859]  [<ffffffff8117c12b>] debug_smp_processor_id+0xcb/0xdc
[    7.045025]  [<ffffffff8111dfb5>] dqstats_inc+0x19/0x2c
[    7.051201]  [<ffffffff81120fac>] vfs_quota_sync+0x144/0x210
[    7.057349]  [<ffffffff810eccfa>] ? __shrink_dcache_sb+0x284/0x293
[    7.063574]  [<ffffffff810fc48d>] __sync_filesystem+0x3a/0x7e
[    7.069730]  [<ffffffff810fc552>] sync_filesystem+0x36/0x4c
[    7.075867]  [<ffffffff810deef0>] do_remount_sb+0x65/0x13c
[    7.081960]  [<ffffffff810f4359>] do_mount+0x251/0x78a
[    7.088039]  [<ffffffff810f2c3e>] ? copy_mount_options+0xda/0x146
[    7.094086]  [<ffffffff810f4911>] sys_mount+0x7f/0xb8
[    7.100085]  [<ffffffff81002aeb>] system_call_fastpath+0x16/0x1b
[    7.124208] BUG: using smp_processor_id() in preemptible [00000000] code: mount/1501
[    7.130307] caller is dqstats_inc+0x19/0x2c
[    7.136386] Pid: 1501, comm: mount Not tainted 2.6.34-tst #25
[    7.142512] Call Trace:
[    7.148629]  [<ffffffff8117c12b>] debug_smp_processor_id+0xcb/0xdc
[    7.154846]  [<ffffffff8111dfb5>] dqstats_inc+0x19/0x2c
[    7.161019]  [<ffffffff81120fac>] vfs_quota_sync+0x144/0x210
[    7.167171]  [<ffffffff810fc48d>] __sync_filesystem+0x3a/0x7e
[    7.173364]  [<ffffffff810fc563>] sync_filesystem+0x47/0x4c
[    7.179522]  [<ffffffff810deef0>] do_remount_sb+0x65/0x13c
[    7.185616]  [<ffffffff810f4359>] do_mount+0x251/0x78a
[    7.191663]  [<ffffffff810f2c3e>] ? copy_mount_options+0xda/0x146
[    7.197626]  [<ffffffff810f4911>] sys_mount+0x7f/0xb8
[    7.203493]  [<ffffffff81002aeb>] system_call_fastpath+0x16/0x1b
[   11.643596] udev: starting version 153
Quoted 4 lines
Quoted 2 lines

This is after reverting commit a7cf4145b (anon_inode: set S_IFREG on the anon_inode) that the system is not really useable with.

Caused by "quota: Make quota stat accounting lockless".

FWIW, reverting that commit fixes the issue.

Thanks,
Rafael

#3 lore raw
In reply to Andrew Morton

Hi,

Quoted 33 lines · On Sat, 22 May 2010 23:36:59 +0200 "Rafael J. Wysocki" <[email protected]> wrote:
Quoted 32 lines
[    6.906133] EXT4-fs (sda7): mounted filesystem with ordered data mode
[    7.014359] BUG: using smp_processor_id() in preemptible [00000000] code: mount/1501
[    7.020570] caller is dqstats_inc+0x19/0x2c
[    7.026658] Pid: 1501, comm: mount Not tainted 2.6.34-tst #25
[    7.032792] Call Trace:
[    7.038859]  [<ffffffff8117c12b>] debug_smp_processor_id+0xcb/0xdc
[    7.045025]  [<ffffffff8111dfb5>] dqstats_inc+0x19/0x2c
[    7.051201]  [<ffffffff81120fac>] vfs_quota_sync+0x144/0x210
[    7.057349]  [<ffffffff810eccfa>] ? __shrink_dcache_sb+0x284/0x293
[    7.063574]  [<ffffffff810fc48d>] __sync_filesystem+0x3a/0x7e
[    7.069730]  [<ffffffff810fc552>] sync_filesystem+0x36/0x4c
[    7.075867]  [<ffffffff810deef0>] do_remount_sb+0x65/0x13c
[    7.081960]  [<ffffffff810f4359>] do_mount+0x251/0x78a
[    7.088039]  [<ffffffff810f2c3e>] ? copy_mount_options+0xda/0x146
[    7.094086]  [<ffffffff810f4911>] sys_mount+0x7f/0xb8
[    7.100085]  [<ffffffff81002aeb>] system_call_fastpath+0x16/0x1b
[    7.124208] BUG: using smp_processor_id() in preemptible [00000000] code: mount/1501
[    7.130307] caller is dqstats_inc+0x19/0x2c
[    7.136386] Pid: 1501, comm: mount Not tainted 2.6.34-tst #25
[    7.142512] Call Trace:
[    7.148629]  [<ffffffff8117c12b>] debug_smp_processor_id+0xcb/0xdc
[    7.154846]  [<ffffffff8111dfb5>] dqstats_inc+0x19/0x2c
[    7.161019]  [<ffffffff81120fac>] vfs_quota_sync+0x144/0x210
[    7.167171]  [<ffffffff810fc48d>] __sync_filesystem+0x3a/0x7e
[    7.173364]  [<ffffffff810fc563>] sync_filesystem+0x47/0x4c
[    7.179522]  [<ffffffff810deef0>] do_remount_sb+0x65/0x13c
[    7.185616]  [<ffffffff810f4359>] do_mount+0x251/0x78a
[    7.191663]  [<ffffffff810f2c3e>] ? copy_mount_options+0xda/0x146
[    7.197626]  [<ffffffff810f4911>] sys_mount+0x7f/0xb8
[    7.203493]  [<ffffffff81002aeb>] system_call_fastpath+0x16/0x1b
[   11.643596] udev: starting version 153
Quoted 8 lines
Quoted 2 lines

This is after reverting commit a7cf4145b (anon_inode: set S_IFREG on the anon_inode) that the system is not really useable with.

Caused by "quota: Make quota stat accounting lockless".

Guys, the code this patch adds utterly duplicates the interface provided by include/linux/percpu_counter.h, only the quota code does it wrongly and badly. Please, can we use the nice library code?

I was thinking about this when merging the code and I thought that I'll save
some memory (~200 bytes) because stats are fine with 32-bit counters (instead of 64-bit ones) and I'm interested only in rough counts, not exact ones. Since the code Dmitry added was quite simple, I thought the duplication won't be so bad.
But the code is probably more subtle than I thought so those 200 bytes aren't
worth it. So let's convert it to generic per-cpu counters. Dmitry, will you do it?
Hmm, I'm also wondering why I didn't see the warning Rafael was hitting in my
test runs... Ah, my kernel config has CONFIG_PREEMPT_NONE! I guess using CONFIG_PREEMPT is likely to give a better test coverage, right?

Honza

Signature · 2 lines
Jan Kara <[email protected]>
SuSE CR Labs
#4 lore raw
Quoted 43 lines · Jan Kara

Hi,

Quoted 33 lines · On Sat, 22 May 2010 23:36:59 +0200 "Rafael J. Wysocki" <[email protected]> wrote:
Quoted 32 lines
[    6.906133] EXT4-fs (sda7): mounted filesystem with ordered data mode
[    7.014359] BUG: using smp_processor_id() in preemptible [00000000] code: mount/1501
[    7.020570] caller is dqstats_inc+0x19/0x2c
[    7.026658] Pid: 1501, comm: mount Not tainted 2.6.34-tst #25
[    7.032792] Call Trace:
[    7.038859]  [<ffffffff8117c12b>] debug_smp_processor_id+0xcb/0xdc
[    7.045025]  [<ffffffff8111dfb5>] dqstats_inc+0x19/0x2c
[    7.051201]  [<ffffffff81120fac>] vfs_quota_sync+0x144/0x210
[    7.057349]  [<ffffffff810eccfa>] ? __shrink_dcache_sb+0x284/0x293
[    7.063574]  [<ffffffff810fc48d>] __sync_filesystem+0x3a/0x7e
[    7.069730]  [<ffffffff810fc552>] sync_filesystem+0x36/0x4c
[    7.075867]  [<ffffffff810deef0>] do_remount_sb+0x65/0x13c
[    7.081960]  [<ffffffff810f4359>] do_mount+0x251/0x78a
[    7.088039]  [<ffffffff810f2c3e>] ? copy_mount_options+0xda/0x146
[    7.094086]  [<ffffffff810f4911>] sys_mount+0x7f/0xb8
[    7.100085]  [<ffffffff81002aeb>] system_call_fastpath+0x16/0x1b
[    7.124208] BUG: using smp_processor_id() in preemptible [00000000] code: mount/1501
[    7.130307] caller is dqstats_inc+0x19/0x2c
[    7.136386] Pid: 1501, comm: mount Not tainted 2.6.34-tst #25
[    7.142512] Call Trace:
[    7.148629]  [<ffffffff8117c12b>] debug_smp_processor_id+0xcb/0xdc
[    7.154846]  [<ffffffff8111dfb5>] dqstats_inc+0x19/0x2c
[    7.161019]  [<ffffffff81120fac>] vfs_quota_sync+0x144/0x210
[    7.167171]  [<ffffffff810fc48d>] __sync_filesystem+0x3a/0x7e
[    7.173364]  [<ffffffff810fc563>] sync_filesystem+0x47/0x4c
[    7.179522]  [<ffffffff810deef0>] do_remount_sb+0x65/0x13c
[    7.185616]  [<ffffffff810f4359>] do_mount+0x251/0x78a
[    7.191663]  [<ffffffff810f2c3e>] ? copy_mount_options+0xda/0x146
[    7.197626]  [<ffffffff810f4911>] sys_mount+0x7f/0xb8
[    7.203493]  [<ffffffff81002aeb>] system_call_fastpath+0x16/0x1b
[   11.643596] udev: starting version 153
Quoted 8 lines
Quoted 2 lines

This is after reverting commit a7cf4145b (anon_inode: set S_IFREG on the anon_inode) that the system is not really useable with.

Caused by "quota: Make quota stat accounting lockless".

Guys, the code this patch adds utterly duplicates the interface provided by include/linux/percpu_counter.h, only the quota code does it wrongly and badly. Please, can we use the nice library code?

Quoted 6 lines

I was thinking about this when merging the code and I thought that I'll save
some memory (~200 bytes) because stats are fine with 32-bit counters (instead of 64-bit ones) and I'm interested only in rough counts, not exact ones. Since the code Dmitry added was quite simple, I thought the duplication won't be so bad.
But the code is probably more subtle than I thought so those 200 bytes aren't
worth it. So let's convert it to generic per-cpu counters. Dmitry, will you do it?

Yes. I will.
BTW we are working on generic scalable resource counter which is based on percpu counters and lockless on fast-paths. Resource counter examples: quota limits, bean-counters, etc. The core idea in most situations resource has three main points:
zero, soft-barrier, hard-barrier where exact accuracy is required.

Quoted 3 lines

Hmm, I'm also wondering why I didn't see the warning Rafael was hitting in my
test runs... Ah, my kernel config has CONFIG_PREEMPT_NONE! I guess using CONFIG_PREEMPT is likely to give a better test coverage, right?

Unfortunately preempt_debug was disabled on my testing hosts too.

Quoted 2 lines

Honza