ath5k : ath5k_config_interface deadlock fix

PATCH In review Dave Young 6 participants 15 messages Aug 1, 2008 – Aug 4, 2008 Networking drivers (wireless)

lore

Share & tools

lore

Dave Young

author

2008-08-01 07:45 UTC #0 lore raw

PATCH 1/1 ath5k : ath5k_config_interface deadlock fix

In the drivers/net/wireless/ath5k/base.c, there's recursive locking of sc->lock This casue the kernel stuck

Bug report please see:
http://lkml.org/lkml/2008/7/29/32

Fixed it by remove the lock in sub routine "ath5k_beacon_update", The ath5k_config_interface is the only caller to it.

[  171.430207] =============================================
[  171.432140] [ INFO: possible recursive locking detected ]
[  171.433113] 2.6.27-rc1-smp #4
[  171.434079] ---------------------------------------------
[  171.435039] ath5k_pci/2447 is trying to acquire lock:
[  171.435990]  (&sc->lock){--..}, at: [<f89ee9b5>] ath5k_config_interface+0xd5/0x340 [ath5k]
[  171.437046] 
[  171.437048] but task is already holding lock:
[  171.438903]  (&sc->lock){--..}, at: [<f89ee91d>] ath5k_config_interface+0x3d/0x340 [ath5k]
[  171.439953] 
[  171.439954] other info that might help us debug this:
[  171.441795] 3 locks held by ath5k_pci/2447:
[  171.442729]  #0:  ((name)){--..}, at: [<c013a122>] run_workqueue+0x102/0x1d0
[  171.443800]  #1:  (&(&local->scan_work)->work){--..}, at: [<c013a122>] run_workqueue+0x102/0x1d0
[  171.444859]  #2:  (&sc->lock){--..}, at: [<f89ee91d>] ath5k_config_interface+0x3d/0x340 [ath5k]
[  171.445941] 
[  171.445942] stack backtrace:
[  171.447791] Pid: 2447, comm: ath5k_pci Not tainted 2.6.27-rc1-smp #4
[  171.448732]  [<c014d69f>] __lock_acquire+0xa8f/0x11b0
[  171.449698]  [<c014de41>] lock_acquire+0x81/0xa0
[  171.450655]  [<f89ee9b5>] ? ath5k_config_interface+0xd5/0x340 [ath5k]
[  171.451651]  [<c046073b>] mutex_lock_nested+0x9b/0x280
[  171.452609]  [<f89ee9b5>] ? ath5k_config_interface+0xd5/0x340 [ath5k]
[  171.453588]  [<f89ee9b5>] ? ath5k_config_interface+0xd5/0x340 [ath5k]
[  171.454555]  [<f89ee9b5>] ath5k_config_interface+0xd5/0x340 [ath5k]
[  171.455485]  [<c0184082>] ? check_object+0xe2/0x1e0
[  171.456428]  [<c014c7cb>] ? trace_hardirqs_on+0xb/0x10
[  171.457377]  [<c014c73d>] ? trace_hardirqs_on_caller+0xbd/0x140
[  171.458334]  [<c014c7cb>] ? trace_hardirqs_on+0xb/0x10
[  171.459304]  [<c03eb31c>] ? dev_alloc_skb+0x1c/0x30
[  171.460276]  [<f89a3747>] ieee80211_if_config+0x97/0x160 [mac80211]
[  171.461237]  [<f89a95bc>] ieee80211_sta_join_ibss+0x25c/0x3e0 [mac80211]
[  171.462196]  [<c012f070>] ? local_bh_enable_ip+0x80/0xc0
[  171.463171]  [<f89ab325>] ieee80211_sta_find_ibss+0x365/0x4b0 [mac80211]
[  171.464134]  [<f89ab8ab>] ieee80211_scan_completed+0x2cb/0x320 [mac80211]
[  171.465097]  [<f89ab7de>] ? ieee80211_scan_completed+0x1fe/0x320 [mac80211]
[  171.466084]  [<f89af3b3>] ieee80211_sta_scan_work+0xa3/0x1f0 [mac80211]
[  171.467054]  [<c013a122>] ? run_workqueue+0x102/0x1d0
[  171.468026]  [<c013a175>] run_workqueue+0x155/0x1d0
[  171.468978]  [<c013a122>] ? run_workqueue+0x102/0x1d0
[  171.469947]  [<c014c7cb>] ? trace_hardirqs_on+0xb/0x10
[  171.470925]  [<f89af310>] ? ieee80211_sta_scan_work+0x0/0x1f0 [mac80211]
[  171.471924]  [<c013ac68>] worker_thread+0x88/0xe0
[  171.472880]  [<c013da70>] ? autoremove_wake_function+0x0/0x40
[  171.473862]  [<c013abe0>] ? worker_thread+0x0/0xe0
[  171.474835]  [<c013d782>] kthread+0x42/0x70
[  171.475789]  [<c013d740>] ? kthread+0x0/0x70
[  171.476765]  [<c01048e7>] kernel_thread_helper+0x7/0x10

Signed-off-by Dave Young <[email protected]>

drivers/net/wireless/ath5k/base.c | 6 +++---

1 file changed, 3 insertions(+), 3 deletions(-)

drivers/net/wireless/ath5k/base.c +3 -3

Index: linux-2.6.26/drivers/net/wireless/ath5k/base.c
===================================================================
@@ -3024,6 +3024,9 @@
 		ath5k_hw_reset_tsf(sc->ah);
 }
 
+/*
+ * caller need lock the hw->sc->lock for us.
+ */
 static int
 ath5k_beacon_update(struct ieee80211_hw *hw, struct sk_buff *skb)
 {

@@ -3032,8 +3035,6 @@
 
 	ath5k_debug_dump_skb(sc, skb, "BC  ", 1);
 
-	mutex_lock(&sc->lock);
-
 	if (sc->opmode != IEEE80211_IF_TYPE_IBSS) {
 		ret = -EIO;
 		goto end;

@@ -3048,7 +3049,6 @@
 		ath5k_beacon_config(sc);
 
 end:
-	mutex_unlock(&sc->lock);
 	return ret;
 }

Jiri Slaby

2008-08-01 08:01 UTC #1 lore raw

Dave Young napsal(a):

Quoted 9 lines

In the drivers/net/wireless/ath5k/base.c, there's recursive locking of sc->lock This casue the kernel stuck
Bug report please see:
http://lkml.org/lkml/2008/7/29/32
Fixed it by remove the lock in sub routine "ath5k_beacon_update", The ath5k_config_interface is the only caller to it.

Quoted 13 lines

[  171.430207] =============================================
[  171.432140] [ INFO: possible recursive locking detected ]
[  171.433113] 2.6.27-rc1-smp #4
[  171.434079] ---------------------------------------------
[  171.435039] ath5k_pci/2447 is trying to acquire lock:
[  171.435990]  (&sc->lock){--..}, at: [<f89ee9b5>] ath5k_config_interface+0xd5/0x340 [ath5k]
[  171.437046] [  171.437048] but task is already holding lock:
[  171.438903]  (&sc->lock){--..}, at: [<f89ee91d>] ath5k_config_interface+0x3d/0x340 [ath5k]
[  171.439953] [  171.439954] other info that might help us debug this:
[  171.441795] 3 locks held by ath5k_pci/2447:
[  171.442729]  #0:  ((name)){--..}, at: [<c013a122>] run_workqueue+0x102/0x1d0
[  171.443800]  #1:  (&(&local->scan_work)->work){--..}, at: [<c013a122>] run_workqueue+0x102/0x1d0
[  171.444859]  #2:  (&sc->lock){--..}, at: [<f89ee91d>] ath5k_config_interface+0x3d/0x340 [ath5k]

Should be fixed already:
http://git.kernel.org/?p=linux/kernel/git/linville/wireless-2.6.git;a=commitdiff_plain;h=bc05116ab33d30342e2b4b1bcc6d6e1184e9df97

Pekka Enberg

2008-08-01 08:06 UTC #2 lore raw

Quoted 24 lines · Jiri Slaby

Dave Young napsal(a):

Quoted 9 lines

In the drivers/net/wireless/ath5k/base.c, there's recursive locking of sc->lock This casue the kernel stuck
Bug report please see:
http://lkml.org/lkml/2008/7/29/32
Fixed it by remove the lock in sub routine "ath5k_beacon_update", The ath5k_config_interface is the only caller to it.

Quoted 13 lines

[  171.430207] =============================================
[  171.432140] [ INFO: possible recursive locking detected ]
[  171.433113] 2.6.27-rc1-smp #4
[  171.434079] ---------------------------------------------
[  171.435039] ath5k_pci/2447 is trying to acquire lock:
[  171.435990]  (&sc->lock){--..}, at: [<f89ee9b5>] ath5k_config_interface+0xd5/0x340 [ath5k]
[  171.437046] [  171.437048] but task is already holding lock:
[  171.438903]  (&sc->lock){--..}, at: [<f89ee91d>] ath5k_config_interface+0x3d/0x340 [ath5k]
[  171.439953] [  171.439954] other info that might help us debug this:
[  171.441795] 3 locks held by ath5k_pci/2447:
[  171.442729]  #0:  ((name)){--..}, at: [<c013a122>] run_workqueue+0x102/0x1d0
[  171.443800]  #1:  (&(&local->scan_work)->work){--..}, at: [<c013a122>] run_workqueue+0x102/0x1d0
[  171.444859]  #2:  (&sc->lock){--..}, at: [<f89ee91d>] ath5k_config_interface+0x3d/0x340 [ath5k]

Quoted 2 lines

Should be fixed already:
http://git.kernel.org/?p=linux/kernel/git/linville/wireless-2.6.git;a=commitdiff_plain;h=bc05116ab33d30342e2b4b1bcc6d6e1184e9df97

I guess that didn't make it to -stable?

Andrew Morton

2008-08-01 08:16 UTC #3 lore raw

Quoted 31 lines · On Fri, 01 Aug 2008 11:03:37 +0300 Pekka Enberg <[email protected]> wrote:

Quoted 30 lines · Jiri Slaby

Dave Young napsal(a):

Quoted 28 lines

In the drivers/net/wireless/ath5k/base.c, there's recursive locking of sc->lock
This casue the kernel stuck

Bug report please see:
http://lkml.org/lkml/2008/7/29/32

Fixed it by remove the lock in sub routine "ath5k_beacon_update", The ath5k_config_interface is the only caller to it.

[  171.430207] =============================================
[  171.432140] [ INFO: possible recursive locking detected ]
[  171.433113] 2.6.27-rc1-smp #4
[  171.434079] ---------------------------------------------
[  171.435039] ath5k_pci/2447 is trying to acquire lock:
[  171.435990]  (&sc->lock){--..}, at: [<f89ee9b5>] 
ath5k_config_interface+0xd5/0x340 [ath5k]
[  171.437046] [  171.437048] but task is already holding lock:
[  171.438903]  (&sc->lock){--..}, at: [<f89ee91d>] 
ath5k_config_interface+0x3d/0x340 [ath5k]
[  171.439953] [  171.439954] other info that might help us debug this:
[  171.441795] 3 locks held by ath5k_pci/2447:
[  171.442729]  #0:  ((name)){--..}, at: [<c013a122>] 
run_workqueue+0x102/0x1d0
[  171.443800]  #1:  (&(&local->scan_work)->work){--..}, at: 
[<c013a122>] run_workqueue+0x102/0x1d0
[  171.444859]  #2:  (&sc->lock){--..}, at: [<f89ee91d>]

ath5k_config_interface+0x3d/0x340 [ath5k]

Quoted 4 lines

Quoted 2 lines
Should be fixed already:
http://git.kernel.org/?p=linux/kernel/git/linville/wireless-2.6.git;a=commitdiff_plain;h=bc05116ab33d30342e2b4b1bcc6d6e1184e9df97
I guess that didn't make it to -stable?

(cc stable!)

Bob Copeland

2008-08-01 13:43 UTC #4 lore raw

Quoted 7 lines · Andrew Morton

Quoted 5 lines · On Fri, 01 Aug 2008 11:03:37 +0300 Pekka Enberg <[email protected]> wrote:
Quoted 4 lines · Jiri Slaby
Dave Young napsal(a):
Quoted 2 lines
In the drivers/net/wireless/ath5k/base.c, there's recursive locking of sc->lock

Quoted 6 lines

Quoted 4 lines
Quoted 2 lines
Should be fixed already:
http://git.kernel.org/?p=linux/kernel/git/linville/wireless-2.6.git;a=commitdiff_plain;h=bc05116ab33d30342e2b4b1bcc6d6e1184e9df97
I guess that didn't make it to -stable?
(cc stable!)

Not to worry, the commit that introduced it was 9d139c810a2aa17365cc548d0cd2a189d8433c65, which as far as I can tell came in after 2.6.26.

Signature · 1 line

Bob Copeland %% www.bobcopeland.com

Jiri Slaby

2008-08-01 15:02 UTC #5 lore raw

Bob Copeland napsal(a):

Quoted 7 lines · Andrew Morton

Quoted 6 lines · On Fri, 01 Aug 2008 11:03:37 +0300 Pekka Enberg <[email protected]> wrote:
Quoted 5 lines
Quoted 3 lines · Jiri Slaby
Dave Young napsal(a):
Quoted 2 lines
In the drivers/net/wireless/ath5k/base.c, there's recursive locking of sc->lock

Quoted 8 lines

Quoted 4 lines
Quoted 3 lines
Quoted 2 lines
Should be fixed already:
http://git.kernel.org/?p=linux/kernel/git/linville/wireless-2.6.git;a=commitdiff_plain;h=bc05116ab33d30342e2b4b1bcc6d6e1184e9df97
I guess that didn't make it to -stable?
(cc stable!)
Not to worry, the commit that introduced it was 9d139c810a2aa17365cc548d0cd2a189d8433c65, which as far as I can tell came in after 2.6.26.

git-describe 9d139c810a2aa17365cc548d0cd2a189d8433c65 v2.6.26-rc8-1219-g9d139c8

Alan Jenkins

2008-08-01 15:21 UTC #6 lore raw

Quoted 30 lines · Jiri Slaby

Bob Copeland napsal(a):
Quoted 15 lines · Andrew Morton
Quoted 14 lines · On Fri, 01 Aug 2008 11:03:37 +0300 Pekka Enberg <[email protected]> wrote:
Quoted 12 lines
Quoted 8 lines · Jiri Slaby
Dave Young napsal(a):
Quoted 3 lines
In the drivers/net/wireless/ath5k/base.c, there's recursive locking of sc->lock
Should be fixed already:
http://git.kernel.org/?p=linux/kernel/git/linville/wireless-2.6.git;a=commitdiff_plain;h=bc05116ab33d30342e2b4b1bcc6d6e1184e9df97
Quoted 8 lines
Quoted 4 lines
Quoted 2 lines
I guess that didn't make it to -stable?
(cc stable!)
Not to worry, the commit that introduced it was 9d139c810a2aa17365cc548d0cd2a189d8433c65, which as far as I can tell came in after 2.6.26.
git-describe 9d139c810a2aa17365cc548d0cd2a189d8433c65 v2.6.26-rc8-1219-g9d139c8

Quoted 21 lines · Jiri Slaby

Bob Copeland napsal(a):
Quoted 19 lines · Andrew Morton
Quoted 18 lines · On Fri, 01 Aug 2008 11:03:37 +0300 Pekka Enberg <[email protected]> wrote:
Quoted 6 lines
Quoted 2 lines · Jiri Slaby
Dave Young napsal(a):
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
In the drivers/net/wireless/ath5k/base.c, there's recursive locking of sc->lock
Should be fixed already:
http://git.kernel.org/?p=linux/kernel/git/linville/wireless-2.6.git;a=commitdiff_plain;h=bc05116ab33d30342e2b4b1bcc6d6e1184e9df97
I guess that didn't make it to -stable?
(cc stable!)

Quoted 8 lines

Quoted 4 lines
Not to worry, the commit that introduced it was 9d139c810a2aa17365cc548d0cd2a189d8433c65, which as far as I can tell came in after 2.6.26.
git-describe 9d139c810a2aa17365cc548d0cd2a189d8433c65 v2.6.26-rc8-1219-g9d139c8

Unfortunately git-describe can be misleading.

All this tells you is that the commit in question was based on (a descendant of) v2.6.26-rc8. It doesn't tell you whether the patch is present in v2.6.26.

There must be a better way (for efficient merges, right?). But all I can think of is comparing the files in question against the diff. I checked myself and the changes don't appear to have been included in v2.6.26.

Alan

Jiri Slaby

2008-08-01 15:25 UTC #7 lore raw

git-describe [Was: ath5k_config_interface deadlock fix]

Alan Jenkins napsal(a):

Quoted 4 lines

There must be a better way (for efficient merges, right?). But all I can think of is comparing the files in question against the diff. I checked myself and the changes don't appear to have been included in v2.6.26.

Ah, I see. Is this a git-describe bug or my misunderstanding of the tool?

Sorry for the noise, it's not in 2.6.26.

Alan Jenkins

2008-08-01 16:11 UTC #8 lore raw

Re: git-describe [Was: ath5k_config_interface deadlock fix]

Quoted 7 lines · Jiri Slaby

Alan Jenkins napsal(a):
Quoted 4 lines
There must be a better way (for efficient merges, right?). But all I can think of is comparing the files in question against the diff. I checked myself and the changes don't appear to have been included in v2.6.26.
Ah, I see. Is this a git-describe bug or my misunderstanding of the tool?

It's not an implementation bug. It's just very easy to misunderstand.

git-describe only looks _back_ in time to what the current commit is based on. In this case the commit was based on v2.6.26-rc8 - but it _wasn't_ merged into 2.6.26. It was presumably merged in for v2.6.27-rc1. This is a perfectly legal history in GIT.

It's probably most often encountered during git-bisect. If you bisect e.g. between v2.6.26 and v.2.6.27-rc1, you're likely to see commits which were based on v2.6.26-rc8.

If that doesn't help, try finding an introduction to GIT with some good pictures. I think it's easier to understand it visually.

Alan

Jiri Slaby

2008-08-01 15:21 UTC #9 lore raw

In reply to Jiri Slaby

Jiri Slaby napsal(a):

Quoted 12 lines

Bob Copeland napsal(a):
Quoted 11 lines · Andrew Morton
Quoted 10 lines · On Fri, 01 Aug 2008 11:03:37 +0300 Pekka Enberg <[email protected]> wrote:
Quoted 9 lines
Quoted 6 lines · Jiri Slaby
Dave Young napsal(a):
Quoted 2 lines
In the drivers/net/wireless/ath5k/base.c, there's recursive locking of sc->lock
Should be fixed already:
http://git.kernel.org/?p=linux/kernel/git/linville/wireless-2.6.git;a=commitdiff_plain;h=bc05116ab33d30342e2b4b1bcc6d6e1184e9df97
I guess that didn't make it to -stable?

Quoted 8 lines

Quoted 5 lines
Quoted 1 line
(cc stable!)
Not to worry, the commit that introduced it was 9d139c810a2aa17365cc548d0cd2a189d8433c65, which as far as I can tell came in after 2.6.26.
git-describe 9d139c810a2aa17365cc548d0cd2a189d8433c65 v2.6.26-rc8-1219-g9d139c8

Fingers faster than brain. Why such a change went into -rc8? How long has this been in linux-next? It appeared in mmotm between 2008-07-15-15-39 and 2008-07-23-02-07 which I think correspond to -next appearance -- isn't it way too fast?

Bob Copeland

2008-08-01 15:24 UTC #10 lore raw

In reply to Jiri Slaby

Quoted 7 lines · Jiri Slaby

Quoted 4 lines
Not to worry, the commit that introduced it was 9d139c810a2aa17365cc548d0cd2a189d8433c65, which as far as I can tell came in after 2.6.26.
git-describe 9d139c810a2aa17365cc548d0cd2a189d8433c65 v2.6.26-rc8-1219-g9d139c8

I'm not that expert in git, what does that mean? Some other commits for 2.6.27 also show a 2.6.26-rc from git-describe (e.g. my 3a078876caee).

For my own investigation, I did a 'git-checkout v2.6.26' and looked at the code to see the patch had not been applied yet. Is there an easier way to definitively say commit X is/is not in tagged release Y?

Signature · 1 line

Bob Copeland %% www.bobcopeland.com

Alan Jenkins

2008-08-01 16:28 UTC #11 lore raw

Quoted 10 lines · Bob Copeland

Quoted 7 lines · Jiri Slaby
Quoted 4 lines
Not to worry, the commit that introduced it was 9d139c810a2aa17365cc548d0cd2a189d8433c65, which as far as I can tell came in after 2.6.26.
git-describe 9d139c810a2aa17365cc548d0cd2a189d8433c65 v2.6.26-rc8-1219-g9d139c8

Quoted 9 lines

I'm not that expert in git, what does that mean? Some other commits for 2.6.27 also show a 2.6.26-rc from git-describe (e.g. my 3a078876caee).
For my own investigation, I did a 'git-checkout v2.6.26' and looked at the code to see the patch had not been applied yet. Is there an easier way to definitively say commit X is/is not in tagged release Y?

Aha! I think we want git-describe --contains.

$ git-describe --contains 2b142900784c6e38c8d39fa57d5f95ef08e735d8
v2.6.27-rc1~55^2~1

Alan

Dave Young

author

2008-08-04 03:02 UTC #12 lore raw

In reply to Bob Copeland

Quoted 11 lines · Bob Copeland

Quoted 10 lines · Andrew Morton
Quoted 8 lines · On Fri, 01 Aug 2008 11:03:37 +0300 Pekka Enberg <[email protected]> wrote:
Quoted 6 lines · Jiri Slaby
Dave Young napsal(a):
Quoted 2 lines
In the drivers/net/wireless/ath5k/base.c, there's recursive locking of sc->lock
Should be fixed already:
http://git.kernel.org/?p=linux/kernel/git/linville/wireless-2.6.git;a=commitdiff_plain;h=bc05116ab33d30342e2b4b1bcc6d6e1184e9df97

Quoted 7 lines

Quoted 3 lines
Quoted 1 line
I guess that didn't make it to -stable?
(cc stable!)
Not to worry, the commit that introduced it was 9d139c810a2aa17365cc548d0cd2a189d8433c65, which as far as I can tell came in after 2.6.26.

Strange, I get system stuck in 2.6.26 as well, so there might be other bugs.

Quoted 4 lines

--
Bob Copeland %% www.bobcopeland.com

Signature · 2 lines

Regards
dave

Jiri Slaby

2008-08-04 08:06 UTC #13 lore raw

Dave Young napsal(a):

Quoted 10 lines · Bob Copeland

Quoted 9 lines · Andrew Morton
Quoted 8 lines · On Fri, 01 Aug 2008 11:03:37 +0300 Pekka Enberg <[email protected]> wrote:
Quoted 7 lines
Quoted 5 lines · Jiri Slaby
Dave Young napsal(a):
Quoted 2 lines
In the drivers/net/wireless/ath5k/base.c, there's recursive locking of sc->lock
Should be fixed already:
http://git.kernel.org/?p=linux/kernel/git/linville/wireless-2.6.git;a=commitdiff_plain;h=bc05116ab33d30342e2b4b1bcc6d6e1184e9df97

Quoted 7 lines

Quoted 5 lines
Quoted 2 lines
Quoted 1 line
I guess that didn't make it to -stable?
(cc stable!)
Not to worry, the commit that introduced it was 9d139c810a2aa17365cc548d0cd2a189d8433c65, which as far as I can tell came in after 2.6.26.
Strange, I get system stuck in 2.6.26 as well, so there might be other bugs.

Could you provide any logs regarding this, please?

Dave Young

author

2008-08-04 09:02 UTC #14 lore raw

Quoted 26 lines · Jiri Slaby

Dave Young napsal(a):
Quoted 25 lines · Bob Copeland
Quoted 23 lines
Quoted 19 lines · Andrew Morton
Quoted 8 lines · On Fri, 01 Aug 2008 11:03:37 +0300 Pekka Enberg <[email protected]> wrote:
Quoted 7 lines · Jiri Slaby
Dave Young napsal(a):
Quoted 4 lines
In the drivers/net/wireless/ath5k/base.c, there's recursive locking of
sc->lock
Quoted 5 lines
Quoted 3 lines
Should be fixed already:
http://git.kernel.org/?p=linux/kernel/git/linville/wireless-2.6.git;a=commitdiff_plain;h=bc05116ab33d30342e2b4b1bcc6d6e1184e9df97
I guess that didn't make it to -stable?
(cc stable!)

Quoted 9 lines

Quoted 6 lines
Quoted 3 lines
Not to worry, the commit that introduced it was 9d139c810a2aa17365cc548d0cd2a189d8433c65, which as far as I can tell came in after 2.6.26.
Strange, I get system stuck in 2.6.26 as well, so there might be other bugs.
Could you provide any logs regarding this, please?

I will generate some new logs tomorrow.

There's some info in:
http://lkml.org/lkml/2008/7/29/32

Signature · 2 lines

Regards
dave