Skip to contentSkip to conversation

kfree(NULL)

DISCUSSION Hua Zhong 5 participants 11 messages Apr 21, 2006 – Apr 22, 2006 Sparse checker

lore
Share & tools
lore

Hua Zhong

author
#0 lore raw

Maybe something like this might be a useful debug option to detect unwise likely()/unlikely() usage?

This is a quick hack (not submitted for inclusion), but it works on my box and detected certain points after boot. Anyone likes this idea?

The reason I had to add printk_debug_likely() is that using printk directly gives compile error. It seems not to like asmlinkage for some reason..I guess likely/unlikely are too fundamental macros and the dependency gets too messy. Or maybe it could be fixed in a way I've not found.

It increases kernel size by about 10%, but hey, it's a debugg option. :-)

Sample print:

possible (un)likely misuse at include/asm/mmu_context.h:32/switch_mm()
<c011b644> printk_debug_likely+0x25/0x31 <c02b3092> schedule+0xb04/0xefa
<c02b457b> __mutex_lock_slowpath+0x35a/0x89e <c016c0fd> real_lookup+0x1f/0xb3
<c016c0fd> real_lookup+0x1f/0xb3 <c016c512> do_lookup+0x50/0xe8
<c016d14b> __link_path_walk+0xba1/0x125f <c016d8a6> link_path_walk+0x9d/0x166
<c014b911> __handle_mm_fault+0x2fd/0x35b <c01ca29e> _atomic_dec_and_lock+0x22/0x2c
<c016d5c2> __link_path_walk+0x1018/0x125f <c016d8a6> link_path_walk+0x9d/0x166
<c014b911> __handle_mm_fault+0x2fd/0x35b <c01ce9fe> strncpy_from_user+0x94/0xb3
<c016dee7> do_path_lookup+0x35c/0x4d2 <c016e385> __user_walk_fd+0x84/0x9b
<c0167ae1> vfs_stat_fd+0x15/0x3c <c014b911> __handle_mm_fault+0x2fd/0x35b
<c0168091> sys_stat64+0xf/0x23 <c0113fea> do_page_fault+0x30d/0x753
<c01ceefd> copy_to_user+0x113/0x11c <c0126ee6> sys_rt_sigprocmask+0xae/0xc1
<c01035cb> sysenter_past_esp+0x54/0x75 possible (un)likely misuse at kernel/sched.c:1638/context_switch()
<c011b644> printk_debug_likely+0x25/0x31 <c02b3025> schedule+0xa97/0xefa
<c011ddcd> do_exit+0x75b/0x765 <c011dec4> sys_exit_group+0x0/0xd
<c01035cb> sysenter_past_esp+0x54/0x75 possible (un)likely misuse at kernel/softlockup.c:59/softlockup_tick()
<c011b644> printk_debug_likely+0x25/0x31 <c0138f78> softlockup_tick+0x88/0xf5
<c0123dcd> update_process_times+0x35/0x57 <c0106133> timer_interrupt+0x3d/0x60
<c013919c> handle_IRQ_event+0x21/0x4a <c01392f7> __do_IRQ+0x132/0x1e7
<c0104d86> do_IRQ+0x9c/0xab <c01037fe> common_interrupt+0x1a/0x20
possible (un)likely misuse at kernel/sched.c:1645/context_switch()
<c011b644> printk_debug_likely+0x25/0x31 <c02b3236> schedule+0xca8/0xefa
<c0130d33> enqueue_hrtimer+0x5b/0x80 <c02b57f5> do_nanosleep+0x3b/0xc0
<c0131092> hrtimer_nanosleep+0x45/0xe6 <c01680b4> sys_lstat64+0xf/0x23
<c013102a> hrtimer_wakeup+0x0/0x18 <c01cf021> copy_from_user+0x11b/0x142
<c0131179> sys_nanosleep+0x46/0x4e <c01035cb> sysenter_past_esp+0x54/0x75

include/linux/compiler.h +25 -2
index f23d3c6..a6df5f7 100644
@@ -58,9 +58,32 @@ #endif
 * build go below this comment. Actual compiler/compiler version
 * specific implementations come from the above header files
 */
+#define CONFIG_DEBUG_LIKELY
+#ifdef CONFIG_DEBUG_LIKELY
+extern int printk_debug_likely(const char *fmt, ...);
+extern int debug_likely_count_min_thresh;
+extern int debug_likely_print_max_thresh;
+#define __check_likely(x, v, uv)                                \
+  ({ static int _ckl_print_nr = 0;                              \
+     static unsigned int _ckl_s[2];                             \
+     int _ckl_r = !!(x);                                        \
+     _ckl_s[_ckl_r]++;                                          \
+     if ((_ckl_s[v] == _ckl_s[uv]) && (_ckl_s[v] > debug_likely_count_min_thresh) \
+          && (_ckl_print_nr < debug_likely_print_max_thresh)) { \
+         _ckl_print_nr++;                                       \
+         printk_debug_likely("possible (un)likely misuse at %s:%d/%s()\n",        \
+                             __FILE__, __LINE__, __FUNCTION__); \
+     }                                                          \
+     _ckl_r; })
+#else
+#define __check_likely(x, v, uv)       __builtin_expect(!!(x), v)
+#endif
+
+#define likely(x)      __check_likely(x, 1, 0)
+#define unlikely(x)    __check_likely(x, 0, 1)
 
-#define likely(x)      __builtin_expect(!!(x), 1)
-#define unlikely(x)    __builtin_expect(!!(x), 0)
+//#define likely(x)    __builtin_expect(!!(x), 1)
+//#define unlikely(x)  __builtin_expect(!!(x), 0)
 

/* Optimization barrier */
#ifndef barrier

kernel/printk.c +0 -0
index c056f33..cc09dca 100644
@@ -602,6 +602,30 @@ out:

EXPORT_SYMBOL(printk);
EXPORT_SYMBOL(vprintk);

+#ifdef CONFIG_DEBUG_LIKELY
+int debug_likely_count_min_thresh = 100;
+int debug_likely_print_max_thresh = 1;
+int printk_debug_likely(const char *fmt, ...)
+{

+       int r = 0;
+       static atomic_t recurse = ATOMIC_INIT(1); /* as a mutex */
+       if (atomic_dec_and_test(&recurse)) {
+               va_list args;
+
+               va_start(args, fmt);
+               r = vprintk(fmt, args);
+               va_end(args);
+               dump_stack();

+ }
+ atomic_inc(&recurse);
+
+ return r;
+}
+EXPORT_SYMBOL(debug_likely_count_min_thresh);
+EXPORT_SYMBOL(debug_likely_print_max_thresh);
+EXPORT_SYMBOL(printk_debug_likely);
+#endif
+
#else

asmlinkage long sys_syslog(int type, char __user *buf, int len)

#1 lore raw

I've been working on some code that records all the data for output to a proc entry . It's pretty light weight .. I'll send it once it's finished ..

Daniel

Quoted 115 lines · Hua Zhong

Maybe something like this might be a useful debug option to detect unwise likely()/unlikely() usage?

This is a quick hack (not submitted for inclusion), but it works on my box and detected certain points after boot. Anyone likes this idea?

The reason I had to add printk_debug_likely() is that using printk directly gives compile error. It seems not to like asmlinkage for some reason..I guess likely/unlikely are too fundamental macros and the dependency gets too messy. Or maybe it could be fixed in a way I've not found.

It increases kernel size by about 10%, but hey, it's a debugg option. :-)

Sample print:

possible (un)likely misuse at include/asm/mmu_context.h:32/switch_mm()
<c011b644> printk_debug_likely+0x25/0x31 <c02b3092> schedule+0xb04/0xefa
<c02b457b> __mutex_lock_slowpath+0x35a/0x89e <c016c0fd> real_lookup+0x1f/0xb3
<c016c0fd> real_lookup+0x1f/0xb3 <c016c512> do_lookup+0x50/0xe8
<c016d14b> __link_path_walk+0xba1/0x125f <c016d8a6> link_path_walk+0x9d/0x166
<c014b911> __handle_mm_fault+0x2fd/0x35b <c01ca29e> _atomic_dec_and_lock+0x22/0x2c
<c016d5c2> __link_path_walk+0x1018/0x125f <c016d8a6> link_path_walk+0x9d/0x166
<c014b911> __handle_mm_fault+0x2fd/0x35b <c01ce9fe> strncpy_from_user+0x94/0xb3
<c016dee7> do_path_lookup+0x35c/0x4d2 <c016e385> __user_walk_fd+0x84/0x9b
<c0167ae1> vfs_stat_fd+0x15/0x3c <c014b911> __handle_mm_fault+0x2fd/0x35b
<c0168091> sys_stat64+0xf/0x23 <c0113fea> do_page_fault+0x30d/0x753
<c01ceefd> copy_to_user+0x113/0x11c <c0126ee6> sys_rt_sigprocmask+0xae/0xc1
<c01035cb> sysenter_past_esp+0x54/0x75
possible (un)likely misuse at kernel/sched.c:1638/context_switch()
<c011b644> printk_debug_likely+0x25/0x31 <c02b3025> schedule+0xa97/0xefa
<c011ddcd> do_exit+0x75b/0x765 <c011dec4> sys_exit_group+0x0/0xd
<c01035cb> sysenter_past_esp+0x54/0x75
possible (un)likely misuse at kernel/softlockup.c:59/softlockup_tick()
<c011b644> printk_debug_likely+0x25/0x31 <c0138f78> softlockup_tick+0x88/0xf5
<c0123dcd> update_process_times+0x35/0x57 <c0106133> timer_interrupt+0x3d/0x60
<c013919c> handle_IRQ_event+0x21/0x4a <c01392f7> __do_IRQ+0x132/0x1e7
<c0104d86> do_IRQ+0x9c/0xab <c01037fe> common_interrupt+0x1a/0x20
possible (un)likely misuse at kernel/sched.c:1645/context_switch()
<c011b644> printk_debug_likely+0x25/0x31 <c02b3236> schedule+0xca8/0xefa
<c0130d33> enqueue_hrtimer+0x5b/0x80 <c02b57f5> do_nanosleep+0x3b/0xc0
<c0131092> hrtimer_nanosleep+0x45/0xe6 <c01680b4> sys_lstat64+0xf/0x23
<c013102a> hrtimer_wakeup+0x0/0x18 <c01cf021> copy_from_user+0x11b/0x142
<c0131179> sys_nanosleep+0x46/0x4e <c01035cb> sysenter_past_esp+0x54/0x75

include/linux/compiler.h +25 -2
index f23d3c6..a6df5f7 100644
@@ -58,9 +58,32 @@ #endif
  * build go below this comment. Actual compiler/compiler version
  * specific implementations come from the above header files
  */
+#define CONFIG_DEBUG_LIKELY
+#ifdef CONFIG_DEBUG_LIKELY
+extern int printk_debug_likely(const char *fmt, ...);
+extern int debug_likely_count_min_thresh;
+extern int debug_likely_print_max_thresh;
+#define __check_likely(x, v, uv)                                \
+  ({ static int _ckl_print_nr = 0;                              \
+     static unsigned int _ckl_s[2];                             \
+     int _ckl_r = !!(x);                                        \
+     _ckl_s[_ckl_r]++;                                          \
+     if ((_ckl_s[v] == _ckl_s[uv]) && (_ckl_s[v] > debug_likely_count_min_thresh) \
+          && (_ckl_print_nr < debug_likely_print_max_thresh)) { \
+         _ckl_print_nr++;                                       \
+         printk_debug_likely("possible (un)likely misuse at %s:%d/%s()\n",        \
+                             __FILE__, __LINE__, __FUNCTION__); \
+     }                                                          \
+     _ckl_r; })
+#else
+#define __check_likely(x, v, uv)       __builtin_expect(!!(x), v)
+#endif
+
+#define likely(x)      __check_likely(x, 1, 0)
+#define unlikely(x)    __check_likely(x, 0, 1)
 
-#define likely(x)      __builtin_expect(!!(x), 1)
-#define unlikely(x)    __builtin_expect(!!(x), 0)
+//#define likely(x)    __builtin_expect(!!(x), 1)
+//#define unlikely(x)  __builtin_expect(!!(x), 0)
 
 /* Optimization barrier */
 #ifndef barrier
kernel/printk.c +24 -0
index c056f33..cc09dca 100644
@@ -602,6 +602,30 @@ out:
 EXPORT_SYMBOL(printk);
 EXPORT_SYMBOL(vprintk);
 
+#ifdef CONFIG_DEBUG_LIKELY
+int debug_likely_count_min_thresh = 100;
+int debug_likely_print_max_thresh = 1;
+int printk_debug_likely(const char *fmt, ...)
+{
+       int r = 0;
+       static atomic_t recurse = ATOMIC_INIT(1); /* as a mutex */
+       if (atomic_dec_and_test(&recurse)) {
+               va_list args;
+
+               va_start(args, fmt);
+               r = vprintk(fmt, args);
+               va_end(args);
+               dump_stack();
+       }
+       atomic_inc(&recurse);
+
+       return r;
+}
+EXPORT_SYMBOL(debug_likely_count_min_thresh);
+EXPORT_SYMBOL(debug_likely_print_max_thresh);
+EXPORT_SYMBOL(printk_debug_likely);
+#endif
+
 #else
 
 asmlinkage long sys_syslog(int type, char __user *buf, int len)
#2 lore raw
Quoted 4 lines · you

I've been working on some code that records all the data for output to a proc entry . It's pretty light weight .. I'll send it once it's finished ..

What about debugfs instead of proc? It has a much cleaner and easier to use interface. And such debugging things, ... that is what debugfs is for.

Signature · 1 line
Greetings Michael.
#3 lore raw
In reply to Hua Zhong
Quoted 2 lines · Hua Zhong

+#define __check_likely(x, v, uv) \

those single-char identifiers are not nice.

include/linux/compiler.h #endif · line 77

Quoted 11 lines
include/linux/compiler.h +11 -0
+  ({ static int _ckl_print_nr = 0;                              \
+     static unsigned int _ckl_s[2];                             \
+     int _ckl_r = !!(x);                                        \
+     _ckl_s[_ckl_r]++;                                          \
+     if ((_ckl_s[v] == _ckl_s[uv]) && (_ckl_s[v] > debug_likely_count_min_thresh) \
+          && (_ckl_print_nr < debug_likely_print_max_thresh)) { \
+         _ckl_print_nr++;                                       \
+         printk_debug_likely("possible (un)likely misuse at %s:%d/%s()\n",        \
+                             __FILE__, __LINE__, __FUNCTION__); \
+     }                                                          \
+     _ckl_r; })

Kinda. It would be better to put all the counters into a linked list

struct likeliness {
	char *file;
	int line;
	atomic_t true_count;
	atomic_t false_count;
	struct likeliness *next;
};

then

#define __check_likely(expr, value)
	({
		static struct likeliness likeliness;
		do_check_likely(__FILE__, __LINE__, &likeliness, value);
	})

...

static struct likeliness *likeliness_list;

void do_check_likely(char *file, int line, struct likeliness *likeliness,
int value)

{
	static DEFINE_SPINLOCK(lock);
	unsigned long flags;

	if (!likeliness->next) {
		if (!spin_trylock_irqsave(&lock, flags))
			return;		/* Can be called from NMI */
		if (!likeliness->next) {
			likeliness->next = likeliness_list;
			likeliness_list = likeliness;
			likeliness->file = file;
			likeliness->line = line;
		}
		spin_unlock_irqsave(&lock, flags);
	}

	if (value)
		atomic_inc(&likeliness->true_count);
	else
		atomic_inc(&likeliness->false_count);
}

EXPORT_SYMBOL(do_check_likely);

then write a thingy to dump the linked list (sysrq&printk if it's a hack, /proc handler if not).

It'll crash if a module gets registered then unloaded. CONFIG_MODULE_UNLOAD=n would be required.

#4 lore raw
Quoted 7 lines · Andrew Morton
#define __check_likely(expr, value)
	({
		static struct likeliness likeliness;
		do_check_likely(__FILE__, __LINE__, &likeliness, value);
	})
#define __check_likely(expr, value)
	({
		static struct likeliness likeliness = {
			.file = __FILE__,
			.line = __LINE__,
		};
		do_check_likely(&likeliness, value);
	})
author
#5 lore raw
In reply to Andrew Morton
Quoted 7 lines
struct likeliness {
	char *file;
	int line;
	atomic_t true_count;
	atomic_t false_count;
	struct likeliness *next;
};

It seems including atomic.h inside compiler.h is a bit tricky (might have interdependency).

Can we just live with int instead of atomic_t? We don't really care about losing a count occasionally..

Quoted 2 lines

It'll crash if a module gets registered then unloaded. CONFIG_MODULE_UNLOAD=n would be required.

What about init data that is thrown away after boot?

Hua

#6 lore raw
Quoted 11 lines · Hua Zhong
Quoted 7 lines
struct likeliness {
	char *file;
	int line;
	atomic_t true_count;
	atomic_t false_count;
	struct likeliness *next;
};

It seems including atomic.h inside compiler.h is a bit tricky (might have interdependency).

Can we just live with int instead of atomic_t? We don't really care about losing a count occasionally..

It's nice so you don't have to fool around with locking .. The atomic_t structure is pretty simple thought . I think it boils down to just an int anyway .

Daniel

author
#7 lore raw
Quoted 9 lines
Quoted 1 line

It seems including atomic.h inside compiler.h is a bit

tricky (might have interdependency).

Quoted 2 lines

Can we just live with int instead of atomic_t? We don't

really care about losing a count occasionally..

It's nice so you don't have to fool around with locking .. The atomic_t structure is pretty simple thought . I think it boils down to just an int anyway .

We could move atomic_t into a separate atomic_type.h? I just want to make sure before I mess with the file structure..

Hua

#8 lore raw
In reply to Hua Zhong
Quoted 9 lines · Hua Zhong
Quoted 8 lines
struct likeliness {
	char *file;
	int line;
	atomic_t true_count;
	atomic_t false_count;
	struct likeliness *next;
};
Quoted 3 lines

It seems including atomic.h inside compiler.h is a bit tricky (might have interdependency).

Can we just live with int instead of atomic_t? We don't really care about losing a count occasionally..

Sure, int or long would work OK.

Quoted 4 lines
Quoted 2 lines

It'll crash if a module gets registered then unloaded. CONFIG_MODULE_UNLOAD=n would be required.

What about init data that is thrown away after boot?

Good point. Putting #ifndef CONFIG_foo around init/main.c:init()'s call to free_initmem would fix that.

#9 lore raw
In reply to Hua Zhong

btw, I don't think a patch like this would be appropriate to mainline - it's the sort of thing which just one or two people would use once or twice per year.

So I'd be inclined to host it permanently in -mm, along with the other 37 mm-only debugging patches.

Consequently it doesn't have to be an especially gorgeous piece of code ;)

#10 lore raw
In reply to Andrew Morton
Quoted 1 line

Kinda. It would be better to put all the counters into a linked list

I'd prefer a binary tree which sorts on the caller address and maps these addrs to struct someinfo;

Quoted 7 lines
struct likeliness {
	char *file;
	int line;
	atomic_t true_count;
	atomic_t false_count;
	struct likeliness *next;
};

Oh, and if it is going to stay linked list, why not use struct list_head.

Since we are at it, non-NULL counting could also be done, which could give an overview who unnecessarily calls kfree too often :>

Jan Engelhardt