Skip to contentSkip to conversation

Doubled stack dumps during locking testsuite

PATCH In review Matthew Wilcox 3 participants 8 messages Dec 16, 2006 – Dec 19, 2006

lore
Share & tools
lore
#0 lore raw

Hi Ingo,

I built a parisc kernel with CONFIG_DEBUG_LOCKING_API_SELFTESTS enabled recently, and got some interesting results:

double unlock: ok | ok |failed|WARNING at /home/willy/merge-2.6/kernel/mutex-debug.c:83 debug_mutex_unlock()

Backtrace:
 [<0000000040144b38>] printk+0x40/0x50
 [<000000004028d5f4>] init_class_Z+0xa4/0xc0
 [<0000000040144b38>] printk+0x40/0x50
 [<0000000040171fbc>] register_cpu_notifier+0x5c/0x78
 [<0000000040340212>] lpfc_sli_issue_mbox_wait+0x138/0x1c8
 [<0000000040204d01>] sg_grt_trans+0x150/0x168
 [<0000000040285700>] kobject_uevent+0x8/0x20
 [<0000000040304d65>] scsi_error_handler+0x9f4/0xa48
 [<0000000040207450>] dump_write+0x40/0x58
 [<000000004044c67c>] packet_getsockopt+0x144/0x150
 [<000000004044b05c>] packet_ioctl+0x1a4/0x1b0
 [<00000000404466d4>] unix_ioctl+0x154/0x168
 [<000000004042d5dc>] ipv4_sysctl_forward_strategy+0x12c/0x138
 [<000000004042d558>] ipv4_sysctl_forward_strategy+0xa8/0x138
 [<000000004042c1d4>] ip_mc_msfget+0x16c/0x1d0
 [<00000000404230a8>] ipv4_doint_and_flush_strategy+0x110/0x138

WARNING at /home/willy/merge-2.6/kernel/mutex-debug.c:86 debug_mutex_unlock()

Backtrace:
 [<0000000040144b38>] printk+0x40/0x50
 [<000000004028d5f4>] init_class_Z+0xa4/0xc0
 [<0000000040144b38>] printk+0x40/0x50
 [<0000000040171fbc>] register_cpu_notifier+0x5c/0x78
 [<0000000040340212>] lpfc_sli_issue_mbox_wait+0x138/0x1c8
 [<0000000040204d01>] sg_grt_trans+0x150/0x168
 [<0000000040285700>] kobject_uevent+0x8/0x20
 [<0000000040304d65>] scsi_error_handler+0x9f4/0xa48
 [<0000000040207450>] dump_write+0x40/0x58
 [<000000004044c67c>] packet_getsockopt+0x144/0x150
 [<000000004044b05c>] packet_ioctl+0x1a4/0x1b0
 [<00000000404466d4>] unix_ioctl+0x154/0x168
 [<000000004042d5dc>] ipv4_sysctl_forward_strategy+0x12c/0x138
 [<000000004042d558>] ipv4_sysctl_forward_strategy+0xa8/0x138
 [<000000004042c1d4>] ip_mc_msfget+0x16c/0x1d0
 [<00000000404230a8>] ipv4_doint_and_flush_strategy+0x110/0x138

failed|failed|failed|

Now, lines 83 to 86 of mutex-debug.c are:

        DEBUG_LOCKS_WARN_ON(lock->owner != current_thread_info());
        DEBUG_LOCKS_WARN_ON(lock->magic != lock);
        DEBUG_LOCKS_WARN_ON(!lock->wait_list.prev && !lock->wait_list.next);
        DEBUG_LOCKS_WARN_ON(lock->owner != current_thread_info());

Do we really need to test the same thing twice and produce the same stack dump?

Moreover, do we want to get stack dumps while running the locking testsuite in the first place? From various comments, it looks like it's supposed to be turned off, but it looks like the sense of debug_locks_silent is inverted in the definition of DEBUG_LOCKS_WARN_ON:

        if (unlikely(c)) {                                              \
                if (debug_locks_silent || debug_locks_off())            \
                        WARN_ON(1);                                     \

Surely that should be:

		if (!debug_locks_silent && debug_locks_off())
			WARN_ON(1);
#1 lore raw

PATCH 1/1 lock debugging: fix DEBUG_LOCKS_WARN_ON() & debug_locks_silent

Quoted 6 lines · Matthew Wilcox

Moreover, do we want to get stack dumps while running the locking testsuite in the first place? From various comments, it looks like it's supposed to be turned off, but it looks like the sense of debug_locks_silent is inverted in the definition of DEBUG_LOCKS_WARN_ON:

Quoted 8 lines
        if (unlikely(c)) {                                              \
                if (debug_locks_silent || debug_locks_off())            \
                        WARN_ON(1);                                     \

Surely that should be:

		if (!debug_locks_silent && debug_locks_off())
			WARN_ON(1);

oops, indeed! Fix below.

Ingo

------------->
Subject: [patch] lock debugging: fix DEBUG_LOCKS_WARN_ON() & debug_locks_silent
From: Ingo Molnar <[email protected]>

Matthew Wilcox noticed that the debug_locks_silent use should be inverted in DEBUG_LOCKS_WARN_ON(). This bug was causing spurious stacktraces and incorrect failures in the locking self-test on the parisc kernel.

---
 include/linux/debug_locks.h |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
include/linux/debug_locks.h +1 -1
Index: linux/include/linux/debug_locks.h
===================================================================
@@ -24,7 +24,7 @@ extern int debug_locks_off(void);
 	int __ret = 0;							\
 									\
 	if (unlikely(c)) {						\
-		if (debug_locks_silent || debug_locks_off())		\
+		if (!debug_locks_silent && debug_locks_off())		\
 			WARN_ON(1);					\
 		__ret = 1;						\
 	}								\
#2 lore raw

Re: [patch] lock debugging: fix DEBUG_LOCKS_WARN_ON() & debug_locks_silent

Quoted 1 line · Ingo Molnar

* Matthew Wilcox <[email protected]> wrote:

...

Quoted 11 lines
---
 include/linux/debug_locks.h |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

Index: linux/include/linux/debug_locks.h
===================================================================

include/linux/debug_locks.h extern int debug_locks_off(void); · line 30

Quoted 8 lines
include/linux/debug_locks.h +1 -1
@@ -24,7 +24,7 @@ extern int debug_locks_off(void);
 	int __ret = 0;							\
 									\
 	if (unlikely(c)) {						\
-		if (debug_locks_silent || debug_locks_off())		\
+		if (!debug_locks_silent && debug_locks_off())		\
 			WARN_ON(1);					\
 		__ret = 1;						\
 	}								\

I wonder why doing debug_locks_off depends here on debug_lock_silent state which is only "esthetical" flag. And debug_locks_off() takes into consideration debug_lock_silent after all. So IMHO:

	if (unlikely(c)) {						\
		if (debug_locks_off())					\
			WARN_ON(1);					\
		__ret = 1;						\
	}								\

Jarek P.

#3 lore raw

Re: [patch] lock debugging: fix DEBUG_LOCKS_WARN_ON() & debug_locks_silent

Quoted 4 lines · Jarek Poplawski

I wonder why doing debug_locks_off depends here on debug_lock_silent state which is only "esthetical" flag. And debug_locks_off() takes into consideration debug_lock_silent after all. So IMHO:

It's not 'aesthetic' at all. It's used to say "We are about to cause a locking failure deliberately as part of the test suite". It would be wrong to disable lock debugging as a result of running the test suite.

#4 lore raw

Re: [patch] lock debugging: fix DEBUG_LOCKS_WARN_ON() & debug_locks_silent

Quoted 9 lines · Matthew Wilcox
Quoted 4 lines · Jarek Poplawski

I wonder why doing debug_locks_off depends here on debug_lock_silent state which is only "esthetical" flag. And debug_locks_off() takes into consideration debug_lock_silent after all. So IMHO:

It's not 'aesthetic' at all. It's used to say "We are about to cause a locking failure deliberately as part of the test suite". It would be wrong to disable lock debugging as a result of running the test suite.

So it's probably something with my English...

Quoted 1 line

From lib/debug_locks.c:

"/*
* The locking-testsuite uses <debug_locks_silent> to get a
* 'silent failure': nothing is printed to the console when
* a locking bug is detected.
*/
int debug_locks_silent;"

Jarek P.

#5 lore raw

Re: [patch] lock debugging: fix DEBUG_LOCKS_WARN_ON() & debug_locks_silent

In reply to Jarek Poplawski
Quoted 3 lines · Jarek Poplawski

Review

Quoted 3 lines
(quoted diff) +1 -1
 	if (unlikely(c)) {						\
-		if (debug_locks_silent || debug_locks_off())		\
+		if (!debug_locks_silent && debug_locks_off())		\

btw., updated patch is below - the right order is to first do debug_locks_off(), then debug_locks_silent.

[ btw., could you fix your mailer so that a reply to your mails doesnt
put all participants into the 'To:' line? You can do it via adding "unset followup_to" to your $HOME/.muttrc file. ]

------------->
Subject: [patch] lock debugging: fix DEBUG_LOCKS_WARN_ON() & debug_locks_silent
From: Ingo Molnar <[email protected]>

Matthew Wilcox noticed that the debug_locks_silent use should be inverted in DEBUG_LOCKS_WARN_ON(). This bug was causing spurious stacktraces and incorrect failures in the locking self-test on the parisc kernel.

---
 include/linux/debug_locks.h |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
include/linux/debug_locks.h +1 -1
Index: linux/include/linux/debug_locks.h
===================================================================
@@ -24,7 +24,7 @@ extern int debug_locks_off(void);
 	int __ret = 0;							\
 									\
 	if (unlikely(c)) {						\
-		if (debug_locks_silent || debug_locks_off())		\
+		if (debug_locks_off() && !debug_locks_silent)		\
 			WARN_ON(1);					\
 		__ret = 1;						\
 	}								\
#6 lore raw

Re: [patch] lock debugging: fix DEBUG_LOCKS_WARN_ON() & debug_locks_silent

Quoted 9 lines · Ingo Molnar
Quoted 3 lines · Jarek Poplawski

include/linux/debug_locks.h extern int debug_locks_off(void);

Quoted 3 lines
include/linux/debug_locks.h +1 -1
 	if (unlikely(c)) {						\
-		if (debug_locks_silent || debug_locks_off())		\
+		if (!debug_locks_silent && debug_locks_off())		\

btw., updated patch is below - the right order is to first do debug_locks_off(), then debug_locks_silent.

Then how does one re-enable lock debugging after running the locking testsuite?

#7 lore raw

Re: [patch] lock debugging: fix DEBUG_LOCKS_WARN_ON() & debug_locks_silent

Quoted 11 lines · Matthew Wilcox
Quoted 9 lines · Ingo Molnar
Quoted 3 lines · Jarek Poplawski

include/linux/debug_locks.h extern int debug_locks_off(void);

Quoted 3 lines
include/linux/debug_locks.h +1 -1
 	if (unlikely(c)) {						\
-		if (debug_locks_silent || debug_locks_off())		\
+		if (!debug_locks_silent && debug_locks_off())		\

btw., updated patch is below - the right order is to first do debug_locks_off(), then debug_locks_silent.

Quoted 2 lines

Then how does one re-enable lock debugging after running the locking testsuite?

see the lib/locking-selftest.c:locking_selftest() function, if all testcases pass then it re-enables lock debugging. If a testcase turns off lock debugging because it triggers a bug (as many of them legitimately do), then reset_locks()->lockdep_reset() will set debug_locks back to 1.

Ingo