Skip to contentSkip to conversation

pmf_register_irq_client gives sleep with locks held warning

PATCH New Johannes Berg 2 participants 9 messages May 31, 2006 – Jun 13, 2006

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

Hi,

When testing headphone detection stuff I got this:

[  634.218762] BUG: sleeping function called from invalid context at mm/slab.c:2794
[  634.218774] in_atomic():0, irqs_disabled():1
[  634.218777] Call Trace:
[  634.218779] [D67ADC00] [C00085F8] show_stack+0x50/0x190 (unreliable)
[  634.218794] [D67ADC30] [C0024030] __might_sleep+0xcc/0xe8
[  634.218804] [D67ADC40] [C0076C04] __kmalloc+0xf4/0xf8
[  634.218815] [D67ADC60] [C00B37D4] proc_create+0x9c/0xe8
[  634.218827] [D67ADC90] [C00B3930] proc_mkdir_mode+0x2c/0x88
[  634.218833] [D67ADCB0] [C005227C] register_handler_proc+0xe4/0xfc
[  634.218844] [D67ADD50] [C0051D70] setup_irq+0x118/0x16c
[  634.218850] [D67ADD70] [C0051E68] request_irq+0xa4/0xb8
[  634.218855] [D67ADDA0] [C0020BA0] macio_do_gpio_irq_enable+0x40/0x50
[  634.218860] [D67ADDB0] [C00208BC] pmf_register_irq_client+0x88/0x9c

[..., not important]

The problem obviously is that request_irq via setup_irq and ... may sleep via kmalloc, while pmf_register_irq_client spin_lock_irqsave()s around the call.

The solution is probably to use spin_lock() instead of spin_lock_irqsave() here.

Comments?

johannes

#1 lore raw
Quoted 17 lines · Johannes Berg

Hi,

When testing headphone detection stuff I got this:

[  634.218762] BUG: sleeping function called from invalid context at mm/slab.c:2794
[  634.218774] in_atomic():0, irqs_disabled():1
[  634.218777] Call Trace:
[  634.218779] [D67ADC00] [C00085F8] show_stack+0x50/0x190 (unreliable)
[  634.218794] [D67ADC30] [C0024030] __might_sleep+0xcc/0xe8
[  634.218804] [D67ADC40] [C0076C04] __kmalloc+0xf4/0xf8
[  634.218815] [D67ADC60] [C00B37D4] proc_create+0x9c/0xe8
[  634.218827] [D67ADC90] [C00B3930] proc_mkdir_mode+0x2c/0x88
[  634.218833] [D67ADCB0] [C005227C] register_handler_proc+0xe4/0xfc
[  634.218844] [D67ADD50] [C0051D70] setup_irq+0x118/0x16c
[  634.218850] [D67ADD70] [C0051E68] request_irq+0xa4/0xb8
[  634.218855] [D67ADDA0] [C0020BA0] macio_do_gpio_irq_enable+0x40/0x50
[  634.218860] [D67ADDB0] [C00208BC] pmf_register_irq_client+0x88/0x9c
Quoted 1 line

[..., not important]

That's a bug in the PowerMac PMF code. What about this patch ?

This fixes request_irq() potentially called from atomic context.

arch/powerpc/platforms/powermac/pfunc_core.c +11 -7
Index: linux-work/arch/powerpc/platforms/powermac/pfunc_core.c
===================================================================
@@ -11,6 +11,7 @@
 #include <linux/kernel.h>
 #include <linux/spinlock.h>
 #include <linux/module.h>
+#include <linux/mutex.h>
 
 #include <asm/semaphore.h>
 #include <asm/prom.h>
@@ -546,6 +547,7 @@
 
 static LIST_HEAD(pmf_devices);
 static spinlock_t pmf_lock = SPIN_LOCK_UNLOCKED;
+static DEFINE_MUTEX(pmf_irq_mutex);
 
 static void pmf_release_device(struct kref *kref)
 {
@@ -864,15 +866,17 @@
 
 	spin_lock_irqsave(&pmf_lock, flags);
 	func = __pmf_find_function(target, name, PMF_FLAGS_INT_GEN);
-	if (func == NULL) {
-		spin_unlock_irqrestore(&pmf_lock, flags);
+	if (func)
+		func = pmf_get_function(func);
+	spin_unlock_irqrestore(&pmf_lock, flags);
+	if (func == NULL)
 		return -ENODEV;
-	}
+	mutex_lock(&pmf_irq_mutex);
 	if (list_empty(&func->irq_clients))
 		func->dev->handlers->irq_enable(func);
 	list_add(&client->link, &func->irq_clients);
 	client->func = func;
-	spin_unlock_irqrestore(&pmf_lock, flags);
+	mutex_unlock(&pmf_irq_mutex);
 
 	return 0;
 }
@@ -881,16 +885,16 @@
 void pmf_unregister_irq_client(struct pmf_irq_client *client)
 {
 	struct pmf_function *func = client->func;
-	unsigned long flags;
 
 	BUG_ON(func == NULL);
 
-	spin_lock_irqsave(&pmf_lock, flags);
+	mutex_lock(&pmf_irq_mutex);
 	client->func = NULL;
 	list_del(&client->link);
 	if (list_empty(&func->irq_clients))
 		func->dev->handlers->irq_disable(func);
-	spin_unlock_irqrestore(&pmf_lock, flags);
+	mutex_unlock(&pmf_irq_mutex);
+	pmf_put_function(func);
 }
 EXPORT_SYMBOL_GPL(pmf_unregister_irq_client);
 
#2 lore raw
Quoted 2 lines · Benjamin Herrenschmidt

This fixes request_irq() potentially called from atomic context. [...]

Sorry, almost forgot about this.

I don't think your patch is right, it seems to me that now pmf_unregister_irq_client races against pmf_do_irq: what happens when an interrupt comes in right in the middle of the list_del()?

It might actually not do any harm due to the way list_del() and pmf_do_irq() work and the fact that client->handler() must still be prepared to be called... But I'd rather make it explicit.

By the way, what do I do when like this I change a patch? Do I rely on your Signed-off-by and simply put mine instead (like I did now)?

Anyway, here's a changed patch that illustrates my point. I've tested it and it works and as expected fixes the problem :)
---
This fixes request_irq() being called with interrupts disabled in the powermac platform function code when registering the first irq client for a given platform function.

Cc Benjamin Herrenschmidt <[email protected]>
arch/powerpc/platforms/powermac/pfunc_core.c +24 -6
@@ -11,6 +11,7 @@ #include <linux/delay.h>
 #include <linux/kernel.h>
 #include <linux/spinlock.h>
 #include <linux/module.h>
+#include <linux/mutex.h>
 
 #include <asm/semaphore.h>
 #include <asm/prom.h>
@@ -546,6 +547,7 @@ struct pmf_device {
 
 static LIST_HEAD(pmf_devices);
 static spinlock_t pmf_lock = SPIN_LOCK_UNLOCKED;
+static DEFINE_MUTEX(pmf_irq_mutex);
 
 static void pmf_release_device(struct kref *kref)
 {
@@ -864,16 +866,25 @@ int pmf_register_irq_client(struct devic
 
 	spin_lock_irqsave(&pmf_lock, flags);
 	func = __pmf_find_function(target, name, PMF_FLAGS_INT_GEN);
-	if (func == NULL) {
-		spin_unlock_irqrestore(&pmf_lock, flags);
+	if (func)
+		func = pmf_get_function(func);
+	spin_unlock_irqrestore(&pmf_lock, flags);
+	if (func == NULL)
 		return -ENODEV;
-	}
+
+	/* guard against manipulations of list */
+	mutex_lock(&pmf_irq_mutex);
 	if (list_empty(&func->irq_clients))
 		func->dev->handlers->irq_enable(func);
+
+	/* guard against pmf_do_irq while changing list */
+	spin_lock_irqsave(&pmf_lock, flags);
 	list_add(&client->link, &func->irq_clients);
-	client->func = func;
 	spin_unlock_irqrestore(&pmf_lock, flags);
 
+	client->func = func;
+	mutex_unlock(&pmf_irq_mutex);
+
 	return 0;
 }
 EXPORT_SYMBOL_GPL(pmf_register_irq_client);
@@ -885,12 +896,19 @@ void pmf_unregister_irq_client(struct pm
 
 	BUG_ON(func == NULL);
 
-	spin_lock_irqsave(&pmf_lock, flags);
+	/* guard against manipulations of list */
+	mutex_lock(&pmf_irq_mutex);
 	client->func = NULL;
+
+	/* guard against pmf_do_irq while changing list */
+	spin_lock_irqsave(&pmf_lock, flags);
 	list_del(&client->link);
+	spin_unlock_irqrestore(&pmf_lock, flags);
+
 	if (list_empty(&func->irq_clients))
 		func->dev->handlers->irq_disable(func);
-	spin_unlock_irqrestore(&pmf_lock, flags);
+	mutex_unlock(&pmf_irq_mutex);
+	pmf_put_function(func);
 }
 EXPORT_SYMBOL_GPL(pmf_unregister_irq_client);
 
#3 lore raw
Quoted 10 lines · Johannes Berg
Quoted 2 lines · Benjamin Herrenschmidt

This fixes request_irq() potentially called from atomic context. [...]

Sorry, almost forgot about this.

I don't think your patch is right, it seems to me that now pmf_unregister_irq_client races against pmf_do_irq: what happens when an interrupt comes in right in the middle of the list_del()?

Yeah, possibly... too late for 2.6.17 tho.

Quoted 2 lines

By the way, what do I do when like this I change a patch? Do I rely on your Signed-off-by and simply put mine instead (like I did now)?

You add yours.

Quoted 2 lines

Anyway, here's a changed patch that illustrates my point. I've tested it and it works and as expected fixes the problem :)

Too late :)

Cheers,
Ben.

Quoted 79 lines

---
This fixes request_irq() being called with interrupts disabled in the powermac platform function code when registering the first irq client for a given platform function.

Cc Benjamin Herrenschmidt <[email protected]>
arch/powerpc/platforms/powermac/pfunc_core.c +24 -6
@@ -11,6 +11,7 @@ #include <linux/delay.h>
 #include <linux/kernel.h>
 #include <linux/spinlock.h>
 #include <linux/module.h>
+#include <linux/mutex.h>
 
 #include <asm/semaphore.h>
 #include <asm/prom.h>
@@ -546,6 +547,7 @@ struct pmf_device {
 
 static LIST_HEAD(pmf_devices);
 static spinlock_t pmf_lock = SPIN_LOCK_UNLOCKED;
+static DEFINE_MUTEX(pmf_irq_mutex);
 
 static void pmf_release_device(struct kref *kref)
 {
@@ -864,16 +866,25 @@ int pmf_register_irq_client(struct devic
 
 	spin_lock_irqsave(&pmf_lock, flags);
 	func = __pmf_find_function(target, name, PMF_FLAGS_INT_GEN);
-	if (func == NULL) {
-		spin_unlock_irqrestore(&pmf_lock, flags);
+	if (func)
+		func = pmf_get_function(func);
+	spin_unlock_irqrestore(&pmf_lock, flags);
+	if (func == NULL)
 		return -ENODEV;
-	}
+
+	/* guard against manipulations of list */
+	mutex_lock(&pmf_irq_mutex);
 	if (list_empty(&func->irq_clients))
 		func->dev->handlers->irq_enable(func);
+
+	/* guard against pmf_do_irq while changing list */
+	spin_lock_irqsave(&pmf_lock, flags);
 	list_add(&client->link, &func->irq_clients);
-	client->func = func;
 	spin_unlock_irqrestore(&pmf_lock, flags);
 
+	client->func = func;
+	mutex_unlock(&pmf_irq_mutex);
+
 	return 0;
 }
 EXPORT_SYMBOL_GPL(pmf_register_irq_client);
@@ -885,12 +896,19 @@ void pmf_unregister_irq_client(struct pm
 
 	BUG_ON(func == NULL);
 
-	spin_lock_irqsave(&pmf_lock, flags);
+	/* guard against manipulations of list */
+	mutex_lock(&pmf_irq_mutex);
 	client->func = NULL;
+
+	/* guard against pmf_do_irq while changing list */
+	spin_lock_irqsave(&pmf_lock, flags);
 	list_del(&client->link);
+	spin_unlock_irqrestore(&pmf_lock, flags);
+
 	if (list_empty(&func->irq_clients))
 		func->dev->handlers->irq_disable(func);
-	spin_unlock_irqrestore(&pmf_lock, flags);
+	mutex_unlock(&pmf_irq_mutex);
+	pmf_put_function(func);
 }
 EXPORT_SYMBOL_GPL(pmf_unregister_irq_client);
 
#4 lore raw
Quoted 5 lines · Benjamin Herrenschmidt
Quoted 3 lines

I don't think your patch is right, it seems to me that now pmf_unregister_irq_client races against pmf_do_irq: what happens when an interrupt comes in right in the middle of the list_del()?

Yeah, possibly... too late for 2.6.17 tho.

That's ok, we don't have any in-kernel users anyway. Alas the alsa people will be dissatisfied because they like to ship new drivers for old kernels or something. Oh well, I don't care.

johannes

#5 lore raw
Quoted 11 lines · Johannes Berg
Quoted 5 lines · Benjamin Herrenschmidt
Quoted 3 lines

I don't think your patch is right, it seems to me that now pmf_unregister_irq_client races against pmf_do_irq: what happens when an interrupt comes in right in the middle of the list_del()?

Yeah, possibly... too late for 2.6.17 tho.

That's ok, we don't have any in-kernel users anyway. Alas the alsa people will be dissatisfied because they like to ship new drivers for old kernels or something. Oh well, I don't care.

We still want people to build out-of-tree for 2.6.17 (please keep the git there for that), as it will take a while before 2.6.18 is here.

The remaining possible bug in the pmf irq code is probably harmless in 99.99% of the cases in practice :)

Ben.

#6 lore raw
Quoted 6 lines · Benjamin Herrenschmidt
Quoted 3 lines

That's ok, we don't have any in-kernel users anyway. Alas the alsa people will be dissatisfied because they like to ship new drivers for old kernels or something. Oh well, I don't care.

We still want people to build out-of-tree for 2.6.17 (please keep the git there for that), as it will take a while before 2.6.18 is here.

Sure, I'll continue maintaining snd-aoa in that repository.

Quoted 2 lines

The remaining possible bug in the pmf irq code is probably harmless in 99.99% of the cases in practice :)

I see your patch is in, I'll make a patch that fixes it then instead of replacing it, I guess.

johannes

#7 lore raw
Quoted 3 lines

I see your patch is in, I'll make a patch that fixes it then instead of replacing it, I guess.

Please do, thanks.

Ben.

#8 lore raw

PATCH 1/1 make pmf irq_client functions safe against pmf interrupts coming in

This fixes the pmf irq_client functions to be safe against pmf interrupts coming in while a client is registered/unregistered.

arch/powerpc/platforms/powermac/pfunc_core.c +14 -0
@@ -871,10 +871,17 @@ int pmf_register_irq_client(struct devic
 	spin_unlock_irqrestore(&pmf_lock, flags);
 	if (func == NULL)
 		return -ENODEV;
+
+	/* guard against manipulations of list */
 	mutex_lock(&pmf_irq_mutex);
 	if (list_empty(&func->irq_clients))
 		func->dev->handlers->irq_enable(func);
+
+	/* guard against pmf_do_irq while changing list */
+	spin_lock_irqsave(&pmf_lock, flags);
 	list_add(&client->link, &func->irq_clients);
+	spin_unlock_irqrestore(&pmf_lock, flags);
+
 	client->func = func;
 	mutex_unlock(&pmf_irq_mutex);
 
@@ -885,12 +892,19 @@ EXPORT_SYMBOL_GPL(pmf_register_irq_clien
 void pmf_unregister_irq_client(struct pmf_irq_client *client)
 {
 	struct pmf_function *func = client->func;
+	unsigned long flags;
 
 	BUG_ON(func == NULL);
 
+	/* guard against manipulations of list */
 	mutex_lock(&pmf_irq_mutex);
 	client->func = NULL;
+
+	/* guard against pmf_do_irq while changing list */
+	spin_lock_irqsave(&pmf_lock, flags);
 	list_del(&client->link);
+	spin_unlock_irqrestore(&pmf_lock, flags);
+
 	if (list_empty(&func->irq_clients))
 		func->dev->handlers->irq_disable(func);
 	mutex_unlock(&pmf_irq_mutex);