kref refcnt and false positives
Quoted 29 lines · Venkatesh Pallipadi
With
WARN_ONaddition tokobject_init()
[ http://kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.19/2.6.19-mm1/dont-use/broken-out/gregkh-driver-kobject-warn.patch ]I started seeing following WARNING on CPU offline followed by online on my
x86_64system.WARNING at
lib/kobject.c:172kobject_init()Call Trace: [<ffffffff8020ab45>] dump_trace+0xaa/0x3ef [<ffffffff8020aec4>] show_trace+0x3a/0x50 [<ffffffff8020b0f6>] dump_stack+0x15/0x17 [<ffffffff80350abc>] kobject_init+0x3f/0x8a [<ffffffff80350be1>] kobject_register+0x1a/0x3e [<ffffffff803bbd89>] sysdev_register+0x5b/0xf9 [<ffffffff80211d0b>] mce_create_device+0x77/0xf4 [<ffffffff80211dc2>] mce_cpu_callback+0x3a/0xe5 [<ffffffff805632fd>] notifier_call_chain+0x26/0x3b [<ffffffff8023f6f3>] raw_notifier_call_chain+0x9/0xb [<ffffffff802519bf>] _cpu_up+0xb4/0xdc [<ffffffff80251a12>] cpu_up+0x2b/0x42 [<ffffffff803bef00>] store_online+0x4a/0x72 [<ffffffff803bb6ce>] sysdev_store+0x24/0x26 [<ffffffff802baaa2>] sysfs_write_file+0xcf/0xfc [<ffffffff8027fc6f>] vfs_write+0xae/0x154 [<ffffffff80280418>] sys_write+0x47/0x6f [<ffffffff8020963e>] system_call+0x7e/0x83 DWARF2 unwinder stuck at system_call+0x7e/0x83
Quoted 4 lines
Leftover inexact backtrace:
This is a false positive as
mce.cis unregistering/registering sysfs interfaces cleanly on hotplug.
The warning above tends to look like this is not true.
Quoted 2 lines
kref_put()and conditional decrement of refcnt seems to be the root cause for this and the patch below resolves the issue for me.
Why?
Are you properly initializing your kref to null before you register it with the driver core? Or is it a static object?
Quoted 2 lines
Original comment seemed to indicate that this conditional thing was performance related. Is it really? If not, we should consider the below patch.
Yes, it's a performance gain and I don't see how this patch would change the above warning.
thanks,
greg k-h
With WARN_ON addition to kobject_init()
[ http://kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.19/2.6.19-mm1/dont-use/broken-out/gregkh-driver-kobject-warn.patch ]
I started seeing following WARNING on CPU offline followed by online on my x86_64 system.
WARNING at lib/kobject.c:172 kobject_init()
Call Trace: [<ffffffff8020ab45>] dump_trace+0xaa/0x3ef [<ffffffff8020aec4>] show_trace+0x3a/0x50 [<ffffffff8020b0f6>] dump_stack+0x15/0x17 [<ffffffff80350abc>] kobject_init+0x3f/0x8a [<ffffffff80350be1>] kobject_register+0x1a/0x3e [<ffffffff803bbd89>] sysdev_register+0x5b/0xf9 [<ffffffff80211d0b>] mce_create_device+0x77/0xf4 [<ffffffff80211dc2>] mce_cpu_callback+0x3a/0xe5 [<ffffffff805632fd>] notifier_call_chain+0x26/0x3b [<ffffffff8023f6f3>] raw_notifier_call_chain+0x9/0xb [<ffffffff802519bf>] _cpu_up+0xb4/0xdc [<ffffffff80251a12>] cpu_up+0x2b/0x42 [<ffffffff803bef00>] store_online+0x4a/0x72 [<ffffffff803bb6ce>] sysdev_store+0x24/0x26 [<ffffffff802baaa2>] sysfs_write_file+0xcf/0xfc [<ffffffff8027fc6f>] vfs_write+0xae/0x154 [<ffffffff80280418>] sys_write+0x47/0x6f [<ffffffff8020963e>] system_call+0x7e/0x83 DWARF2 unwinder stuck at system_call+0x7e/0x83
Leftover inexact backtrace:
This is a false positive as mce.c is unregistering/registering sysfs interfaces cleanly on hotplug.
kref_put() and conditional decrement of refcnt seems to be the root cause for this and the patch below resolves the issue for me.
Original comment seemed to indicate that this conditional thing was performance related. Is it really? If not, we should consider the below patch.
Thanks,
Venki
Now that kobject_init has a WARN_ON for refcnt, change below is needed to avoid false positives.
lib/kref.c +1 -6
@@ -52,12 +52,7 @@ int kref_put(struct kref *kref, void (*rWARN_ON (release ==NULL );WARN_ON (release == (void (*)(struct kref *))kfree); -/* -* if current count is one, we are the last user and can release object -* right now, avoiding an atomic operation on 'refcount' -*/ -if ((atomic_read (&kref->refcount) ==1 ) || - (atomic_dec_and_test (&kref->refcount))) { +if (atomic_dec_and_test (&kref->refcount)) {release (kref);return 1 ; }
Quoted 5 lines · On Wed, 13 Dec 2006 16:12:46 -0800 Greg KH <[email protected]> wrote:
Quoted 2 lines
Original comment seemed to indicate that this conditional thing was performance related. Is it really? If not, we should consider the below patch.
Yes, it's a performance gain and I don't see how this patch would change the above warning.
I suspect it's a false optimisation.
int kref_put(struct kref *kref, void (*release)(struct kref *kref))
{
WARN_ON(release == NULL);
WARN_ON(release == (void (*)(struct kref *))kfree);
/*
* if current count is one, we are the last user and can release object
* right now, avoiding an atomic operation on 'refcount'
*/
if ((atomic_read(&kref->refcount) == 1) ||
(atomic_dec_and_test(&kref->refcount))) {
release(kref);
return 1;
}
return 0;
}The only time we avoid the atomic_dec_and_test() is when the object is about to be freed. ie: once in its entire lifetime. And freeing the object is part of an expensive (and rare) operation anyway.
otoh, we've gone and added a test-n-branch to the common case: those cases where the object will not be freed.
Quoted 15 lines · Greg KH
Quoted 11 lines · Venkatesh Pallipadi
With
WARN_ONaddition tokobject_init()
[ http://kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.19/2.6.19-mm1/dont-use/broken-out/gregkh-driver-kobject-warn.patch ]I started seeing following WARNING on CPU offline followed by online on my
x86_64system.WARNING at
lib/kobject.c:172kobject_init()This is a false positive as
mce.cis unregistering/registering sysfs interfaces cleanly on hotplug.The warning above tends to look like this is not true.
Quoted 8 lines
Quoted 2 lines
kref_put()and conditional decrement of refcnt seems to be the root cause for this and the patch below resolves the issue for me.Why?
Are you properly initializing your kref to null before you register it with the driver core? Or is it a static object?
Yes. arch/x86_64/kernel/mce.c is calling sysdev_register and sysdev_unregister for cpu hot_add and hot_remove respectively.
The problem is that due to the perf optimization, refcnt remains at 1 even after unregister (as it never gets decremented to 0 in kref_put()).
So, when we try to register next time for same sysdev (The object is percpu, so previously set refcnt will remain 1), init sees that refcnt is already 1 and print out the WARNing.
Quoted 6 lines
Quoted 2 lines
Original comment seemed to indicate that this conditional thing was performance related. Is it really? If not, we should consider the below patch.
Yes, it's a performance gain and I don't see how this patch would change the above warning.
In that case, I think we should change the WARN_ON in question to check for > 1.
Thanks,
Venki
Quoted 27 lines · Andrew Morton
Quoted 4 lines · On Wed, 13 Dec 2006 16:12:46 -0800 Greg KH <[email protected]> wrote:
Quoted 2 lines
Original comment seemed to indicate that this conditional thing was performance related. Is it really? If not, we should consider the below patch.
Yes, it's a performance gain and I don't see how this patch would change the above warning.
I suspect it's a false optimisation.
int
kref_put(struct kref *kref, void (*release)(struct kref *kref)){ WARN_ON(release == NULL); WARN_ON(release == (void (*)(struct kref *))kfree); /* * if current count is one, we are the last user and can release object * right now, avoiding an atomic operation on 'refcount' */ if ((atomic_read(&kref->refcount) == 1) || (atomic_dec_and_test(&kref->refcount))) { release(kref); return 1; } return 0; }
Quoted 7 lines
The only time we avoid the
atomic_dec_and_test()is when the object is about to be freed. ie: once in its entire lifetime. And freeing the object is part of an expensive (and rare) operation anyway.otoh, we've gone and added a test-n-branch to the common case: those cases where the object will not be freed.
I agree this 'optimization' is not "good" (I was the guy who suggested it http://lkml.org/lkml/2006/1/30/4 )
After Eric Biederman message (http://lkml.org/lkml/2006/1/30/292) I remember adding some stat counters and telling Greg to not put the patch in because kref_put() was mostly called with refcount=1. But the patch did its way. I *did* ask Greg to revert it, but cannot find this mail archived somewhere...
But I believe Venkatesh problem comes from its release() function : It is supposed to free the object. If not, it should properly setup it so that further uses are OK.
ie doing in release(kref)atomic_set(&kref->count, 0);
Eric