Skip to contentSkip to conversation

Sleeping function called from illegal context...

DISCUSSION Greg KH 9 participants 25 messages Sep 27, 2002 – Sep 30, 2002

Subsystems (3)
lore
Share & tools
lore

Greg KH

author
#0 lore raw

So I got bold and enabled CONFIG_PREEMPT in 2.5.39, and got the following message at boot time:

Sleeping function called from illegal context at slab.c:1374 c12a5ea8 c0117a26 c0296260 c0298202 0000055e c1283060 c013ab4f c0298202

       0000055e c03b668c 00000002 00000003 c01ff5ec c03b668c 00000042 c12838e0 
       c12838e0 c12838e0 00000246 cfdee214 c0207830 04000000 c03b65f0 c0109be2 
Call Trace:
 [<c0117a26>]__might_sleep+0x56/0x5d
 [<c013ab4f>]kmalloc+0x4f/0x330
 [<c01ff5ec>]piix_tune_chipset+0x33c/0x350
 [<c0207830>]ide_intr+0x0/0x320
 [<c0109be2>]request_irq+0x52/0xa0
 [<c0200a33>]init_irq+0x263/0x400
 [<c0207830>]ide_intr+0x0/0x320
 [<c0200edc>]hwif_init+0x10c/0x260
 [<c02006ad>]probe_hwif_init+0x1d/0x70
 [<c02121d1>]ide_setup_pci_device+0x41/0x70
 [<c01ff7a5>]piix_init_one+0x35/0x40
 [<c010511b>]init+0x8b/0x250
 [<c0105090>]init+0x0/0x250
 [<c01055f9>]kernel_thread_helper+0x5/0xc

kksymoops is very nice :)

The system still seems to be running ok, but I think I'll turn off CONFIG_PREEMPT just to be sure.

This is a SMP kernel running on a UP box. Other configuration options available if needed.

thanks,

greg k-h

#1 lore raw
Quoted 15 lines · Greg KH

So I got bold and enabled CONFIG_PREEMPT in 2.5.39, and got the following message at boot time:

Sleeping function called from illegal context at slab.c:1374 c12a5ea8 c0117a26 c0296260 c0298202 0000055e c1283060 c013ab4f c0298202

       0000055e c03b668c 00000002 00000003 c01ff5ec c03b668c 00000042 c12838e0
       c12838e0 c12838e0 00000246 cfdee214 c0207830 04000000 c03b65f0 c0109be2
Call Trace:
 [<c0117a26>]__might_sleep+0x56/0x5d
 [<c013ab4f>]kmalloc+0x4f/0x330
 [<c01ff5ec>]piix_tune_chipset+0x33c/0x350
 [<c0207830>]ide_intr+0x0/0x320
 [<c0109be2>]request_irq+0x52/0xa0
 [<c0200a33>]init_irq+0x263/0x400
Quoted 9 lines
 [<c0207830>]ide_intr+0x0/0x320
 [<c0200edc>]hwif_init+0x10c/0x260
 [<c02006ad>]probe_hwif_init+0x1d/0x70
 [<c02121d1>]ide_setup_pci_device+0x41/0x70
 [<c01ff7a5>]piix_init_one+0x35/0x40
 [<c010511b>]init+0x8b/0x250
 [<c0105090>]init+0x0/0x250
 [<c01055f9>]kernel_thread_helper+0x5/0xc

Everyone will get this. It's IDE's init_irq() function doing unsafe things inside ide_lock.

It'll be quite harmless at boot-time, but it'd be nice to get it fixed up.

#2 lore raw

Re: (more) Sleeping function called from illegal context...

With CONFIG_PREEMPT=y on an SMP AMD (2CPU):

Sleeping function called from illegal context at /kernel/l-2.5.39/include/asm/semaphore.h:119 c1b75e20 c0117094 c0280b00 c028d480 00000077 c1b41cf0 c016bcf9 c028d480

       00000077 c1b41c50 ffffffea c1b74000 00001000 c01937c6 c02e9578 c1b41cf0
       c1b41c50 f7f0b840 c1b75eda c018ad0a c1b41c50 c02e9578 00000020 00000000
Call Trace:
 [<c0117094>]__might_sleep+0x54/0x58
 [<c016bcf9>]driverfs_create_file+0x39/0xa0
 [<c01937c6>]device_create_file+0x26/0x40
 [<c018ad0a>]pci_pool_create+0xea/0x170
 [<c02034af>]hcd_buffer_create+0x3f/0x80
 [<c02039c3>]usb_hcd_pci_probe+0x253/0x370
 [<c01563d8>]alloc_inode+0x58/0x190
 [<c018b751>]pci_device_probe+0x41/0x60
 [<c0191b88>]probe+0x18/0x30
 [<c0191c2b>]found_match+0x2b/0x60
 [<c0191d57>]do_driver_attach+0x37/0x50
 [<c019296c>]bus_for_each_dev+0x9c/0x130
 [<c0191d83>]driver_attach+0x13/0x20
 [<c0191d20>]do_driver_attach+0x0/0x50
 [<c0192ee4>]driver_register+0x94/0xb0
 [<c018b856>]pci_register_driver+0x36/0x50
 [<c01050b7>]init+0x47/0x1c0
 [<c0105070>]init+0x0/0x1c0
 [<c010553d>]kernel_thread_helper+0x5/0x18
#3 lore raw

Re: (more) Sleeping function called from illegal context...

Quoted 19 lines · Luc Van Oostenryck

With CONFIG_PREEMPT=y on an SMP AMD (2CPU):

Sleeping function called from illegal context at /kernel/l-2.5.39/include/asm/semaphore.h:119 c1b75e20 c0117094 c0280b00 c028d480 00000077 c1b41cf0 c016bcf9 c028d480

        00000077 c1b41c50 ffffffea c1b74000 00001000 c01937c6 c02e9578 c1b41cf0
        c1b41c50 f7f0b840 c1b75eda c018ad0a c1b41c50 c02e9578 00000020 00000000
Call Trace:
  [<c0117094>]__might_sleep+0x54/0x58
  [<c016bcf9>]driverfs_create_file+0x39/0xa0
  [<c01937c6>]device_create_file+0x26/0x40
  [<c018ad0a>]pci_pool_create+0xea/0x170
  [<c02034af>]hcd_buffer_create+0x3f/0x80
  [<c02039c3>]usb_hcd_pci_probe+0x253/0x370
  [<c01563d8>]alloc_inode+0x58/0x190
  [<c018b751>]pci_device_probe+0x41/0x60
  [<c0191b88>]probe+0x18/0x30
  [<c0191c2b>]found_match+0x2b/0x60
  [<c0191d57>]do_driver_attach+0x37/0x50
Quoted 9 lines
  [<c019296c>]bus_for_each_dev+0x9c/0x130
  [<c0191d83>]driver_attach+0x13/0x20
  [<c0191d20>]do_driver_attach+0x0/0x50
  [<c0192ee4>]driver_register+0x94/0xb0
  [<c018b856>]pci_register_driver+0x36/0x50
  [<c01050b7>]init+0x47/0x1c0
  [<c0105070>]init+0x0/0x1c0
  [<c010553d>]kernel_thread_helper+0x5/0x18

pci_pool_create() is calling device_create_file() under pools_lock.

#4 lore raw

Re: (more) Sleeping function called from illegal context...

In reply to Andrew Morton

With CONFIG_PREEMPT=y on an SMP AMD (2CPU):

Sleeping function called from illegal context at /kernel/l-2.5.39/include/asm/semaphore.h:119 c1b4ff7c c0117094 c0280b00 c02bc680 00000077 f7f78540 c01ffc8c c02bc680

       00000077 c1b4e000 c1b4e000 00000001 c1b4ffdc c1b4ffc0 00000206 f7f78568
       c1b4e000 00000001 c1b4ffdc c01fff35 c01fff00 00000000 00000000 00000000
Call Trace:
 [<c0117094>]__might_sleep+0x54/0x58
 [<c01ffc8c>]usb_hub_events+0x6c/0x2e0
 [<c01fff35>]usb_hub_thread+0x35/0xe0
 [<c01fff00>]usb_hub_thread+0x0/0xe0
 [<c0115500>]default_wake_function+0x0/0x40
 [<c010553d>]kernel_thread_helper+0x5/0x18
#5 lore raw

Re: (more) Sleeping function called from illegal context...

Quoted 5 lines · Luc Van Oostenryck

With CONFIG_PREEMPT=y on an SMP AMD (2CPU):

Sleeping function called from illegal context at /kernel/l-2.5.39/include/asm/semaphore.h:119 c1b4ff7c c0117094 c0280b00 c02bc680 00000077 f7f78540 c01ffc8c c02bc680

Quoted 9 lines
        00000077 c1b4e000 c1b4e000 00000001 c1b4ffdc c1b4ffc0 00000206 f7f78568
        c1b4e000 00000001 c1b4ffdc c01fff35 c01fff00 00000000 00000000 00000000
Call Trace:
  [<c0117094>]__might_sleep+0x54/0x58
  [<c01ffc8c>]usb_hub_events+0x6c/0x2e0
  [<c01fff35>]usb_hub_thread+0x35/0xe0
  [<c01fff00>]usb_hub_thread+0x0/0xe0
  [<c0115500>]default_wake_function+0x0/0x40
  [<c010553d>]kernel_thread_helper+0x5/0x18

usb_hub_events() does down() inside hub_event_lock.

author
#6 lore raw

Re: (more) Sleeping function called from illegal context...

Quoted 16 lines · Andrew Morton
Quoted 5 lines · Luc Van Oostenryck

With CONFIG_PREEMPT=y on an SMP AMD (2CPU):

Sleeping function called from illegal context at /kernel/l-2.5.39/include/asm/semaphore.h:119 c1b4ff7c c0117094 c0280b00 c02bc680 00000077 f7f78540 c01ffc8c c02bc680

Quoted 9 lines
        00000077 c1b4e000 c1b4e000 00000001 c1b4ffdc c1b4ffc0 00000206 f7f78568
        c1b4e000 00000001 c1b4ffdc c01fff35 c01fff00 00000000 00000000 00000000
Call Trace:
  [<c0117094>]__might_sleep+0x54/0x58
  [<c01ffc8c>]usb_hub_events+0x6c/0x2e0
  [<c01fff35>]usb_hub_thread+0x35/0xe0
  [<c01fff00>]usb_hub_thread+0x0/0xe0
  [<c0115500>]default_wake_function+0x0/0x40
  [<c010553d>]kernel_thread_helper+0x5/0x18
Quoted 1 line

usb_hub_events() does down() inside hub_event_lock.

Yup, just got that one myself, along with a few other USB goodies :(

This is a very good debugging tool, thanks for doing it.

greg k-h

#7 lore raw

Re: (more) Sleeping function called from illegal context...

In reply to Andrew Morton

With CONFIG_PREEMPT=y on an SMP AMD (2CPU):

Sleeping function called from illegal context at slab.c:1374 f79dbe2c c0117094 c0280b00 c02847cb 0000055e f7817b40 c01328fd c02847cb

       0000055e f79da000 f7ede980 c03ecc58 f7817b40 c025af91 c18cf248 c0266b3e
       00000024 000001d0 f79da000 00000286 00000001 bffffc9c c02f63e0 c0266e20
Call Trace:
 [<c0117094>]__might_sleep+0x54/0x58
 [<c01328fd>]kmalloc+0x5d/0x1e0
 [<c025af91>]fib_add_ifaddr+0x61/0x110
 [<c0266b3e>]__sctp_get_local_addr_list+0x9e/0x140
 [<c0266e20>]sctp_netdev_event+0x30/0x60
 [<c01241ae>]notifier_call_chain+0x1e/0x40
 [<c02566f5>]inet_insert_ifa+0x1b5/0x1c0
 [<c02567b4>]inet_set_ifa+0xb4/0xc0
 [<c0257091>]devinet_ioctl+0x511/0x740
 [<c0259897>]inet_ioctl+0x157/0x1b0
 [<c021e276>]sock_ioctl+0x56/0x90
 [<c0150039>]sys_ioctl+0x289/0x2d8
 [<c0107d11>]error_code+0x2d/0x38
 [<c01072cf>]syscall_call+0x7/0xb
#8 lore raw

Re: (more) Sleeping function called from illegal context...

Quoted 14 lines · Luc Van Oostenryck

With CONFIG_PREEMPT=y on an SMP AMD (2CPU):

Sleeping function called from illegal context at slab.c:1374 f79dbe2c c0117094 c0280b00 c02847cb 0000055e f7817b40 c01328fd c02847cb

        0000055e f79da000 f7ede980 c03ecc58 f7817b40 c025af91 c18cf248 c0266b3e
        00000024 000001d0 f79da000 00000286 00000001 bffffc9c c02f63e0 c0266e20
Call Trace:
  [<c0117094>]__might_sleep+0x54/0x58
  [<c01328fd>]kmalloc+0x5d/0x1e0
  [<c025af91>]fib_add_ifaddr+0x61/0x110
  [<c0266b3e>]__sctp_get_local_addr_list+0x9e/0x140
  [<c0266e20>]sctp_netdev_event+0x30/0x60
  [<c01241ae>]notifier_call_chain+0x1e/0x40
Quoted 9 lines
  [<c02566f5>]inet_insert_ifa+0x1b5/0x1c0
  [<c02567b4>]inet_set_ifa+0xb4/0xc0
  [<c0257091>]devinet_ioctl+0x511/0x740
  [<c0259897>]inet_ioctl+0x157/0x1b0
  [<c021e276>]sock_ioctl+0x56/0x90
  [<c0150039>]sys_ioctl+0x289/0x2d8
  [<c0107d11>]error_code+0x2d/0x38
  [<c01072cf>]syscall_call+0x7/0xb

sctp_v4_get_local_addr_list():

                /* XXX BUG: sleeping allocation with lock held -DaveM */
                addr = t_new(struct sockaddr_storage_list, GFP_KERNEL);

Is true. We're holding dev_base_lock, inetdev_lock and in_dev->lock here.

#9 lore raw

Re: [Lksctp-developers] Re: (more) Sleeping function called from illegal context...

Thanks. We'll fix it up.

Best Regards,
Jon Grimm

Quoted 11 lines · Andrew Morton

sctp_v4_get_local_addr_list():

                /* XXX BUG: sleeping allocation with lock held -DaveM */
                addr = t_new(struct sockaddr_storage_list, GFP_KERNEL);

Is true. We're holding dev_base_lock, inetdev_lock and in_dev->lock here.

#10 lore raw

Re: [Lksctp-developers] Re: (more) Sleeping function called from illegal context...

In reply to Andrew Morton

See Patch below to stop any bleeding. I'll feed the patch in through
DaveM.

Thanks, Jon

Quoted 6 lines · Andrew Morton
Quoted 3 lines · Luc Van Oostenryck
Quoted 7 lines

sctp_v4_get_local_addr_list():

                /* XXX BUG: sleeping allocation with lock held -DaveM */
                addr = t_new(struct sockaddr_storage_list, GFP_KERNEL);

Is true. We're holding dev_base_lock, inetdev_lock and in_dev->lock here.

# This is a BitKeeper generated patch for the following project:
# Project Name: Linux kernel tree
# This patch format is intended for GNU patch command version 2.5 or higher.
# This patch includes the following deltas:
# ChangeSet 1.655 -> 1.656
# net/sctp/protocol.c 1.7 -> 1.8
#
# The following is the BitKeeper ChangeSet Log
# --------------------------------------------
# 02/09/30 [email protected] 1.656
# sctp: Fix GFP_KERNEL allocation with lock held.
# --------------------------------------------
#

net/sctp/protocol.c +2 -4
diff -Nru a/net/sctp/protocol.c b/net/sctp/protocol.c
@@ -119,8 +119,7 @@
 
 	for (ifa = in_dev->ifa_list; ifa; ifa = ifa->ifa_next) {
 		/* Add the address to the local list.  */
-		/* XXX BUG: sleeping allocation with lock held -DaveM */
-		addr = t_new(struct sockaddr_storage_list, GFP_KERNEL);
+		addr = t_new(struct sockaddr_storage_list, GFP_ATOMIC);
 		if (addr) {
 			INIT_LIST_HEAD(&addr->list);
 			addr->a.v4.sin_family = AF_INET;
@@ -157,8 +156,7 @@
 	read_lock_bh(&in6_dev->lock);
 	for (ifp = in6_dev->addr_list; ifp; ifp = ifp->if_next) {
 		/* Add the address to the local list.  */
-		/* XXX BUG: sleeping allocation with lock held -DaveM */
-		addr = t_new(struct sockaddr_storage_list, GFP_KERNEL);
+		addr = t_new(struct sockaddr_storage_list, GFP_ATOMIC);
 		if (addr) {
 			addr->a.v6.sin6_family = AF_INET6;
 			addr->a.v6.sin6_port = 0;
#11 lore raw
In reply to Andrew Morton

No SLEEPING under a KMALLOC on a spinlock while calling request_irq is not the brightest thing ever done!

So your finger pointing stinks!

See in 2.4-ac there is a stub to indicate the brokeness.

int init_irq (ide_hwif_t *hwif)

{
        unsigned long flags;
        unsigned int index;
        ide_hwgroup_t *hwgroup, *new_hwgroup;
        ide_hwif_t *match = NULL;
#if 0
        /* Allocate the buffer and no sleep allowed */
        new_hwgroup = kmalloc(sizeof(ide_hwgroup_t),GFP_ATOMIC);
#else
        /* Allocate the buffer and potentially sleep first */
        new_hwgroup = kmalloc(sizeof(ide_hwgroup_t),GFP_KERNEL);

#endif

#ifndef __IRQ_HELL_SPIN
        save_and_cli(flags);
#else
        spin_lock_irqsave(&io_request_lock, flags);

#endif

<snip>

                if (request_irq(hwif->irq,&ide_intr,sa,hwif->name,hwgroup)) {
                        if (!match)
                                kfree(hwgroup);
#ifndef __IRQ_HELL_SPIN
                        restore_flags(flags);
#else
                        spin_unlock_irqrestore(&io_request_lock, flags);
#endif
                        return 1;
                }

<snip>

        /* all CPUs; safe now that hwif->hwgroup is set up */
#ifndef __IRQ_HELL_SPIN
        restore_flags(flags);
#else
        spin_unlock_irqrestore(&io_request_lock, flags);

#endif

See in trying to move to a spinlock it goes totally south. So now that you know the where, and why ... please go fix. See I am off working with AC on the issues for 2.4.

Also with PREMPT, bah never mind.

Regards,

Andre Hedrick
LAD Storage Consulting Group

Quoted 37 lines · Andrew Morton
Quoted 15 lines · Greg KH

So I got bold and enabled CONFIG_PREEMPT in 2.5.39, and got the following message at boot time:

Sleeping function called from illegal context at slab.c:1374 c12a5ea8 c0117a26 c0296260 c0298202 0000055e c1283060 c013ab4f c0298202

       0000055e c03b668c 00000002 00000003 c01ff5ec c03b668c 00000042 c12838e0
       c12838e0 c12838e0 00000246 cfdee214 c0207830 04000000 c03b65f0 c0109be2
Call Trace:
 [<c0117a26>]__might_sleep+0x56/0x5d
 [<c013ab4f>]kmalloc+0x4f/0x330
 [<c01ff5ec>]piix_tune_chipset+0x33c/0x350
 [<c0207830>]ide_intr+0x0/0x320
 [<c0109be2>]request_irq+0x52/0xa0
 [<c0200a33>]init_irq+0x263/0x400
Quoted 9 lines
 [<c0207830>]ide_intr+0x0/0x320
 [<c0200edc>]hwif_init+0x10c/0x260
 [<c02006ad>]probe_hwif_init+0x1d/0x70
 [<c02121d1>]ide_setup_pci_device+0x41/0x70
 [<c01ff7a5>]piix_init_one+0x35/0x40
 [<c010511b>]init+0x8b/0x250
 [<c0105090>]init+0x0/0x250
 [<c01055f9>]kernel_thread_helper+0x5/0xc

Everyone will get this. It's IDE's init_irq() function doing unsafe things inside ide_lock.

It'll be quite harmless at boot-time, but it'd be nice to get it fixed up.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

#12 lore raw
Quoted 5 lines · Andre Hedrick

See in trying to move to a spinlock it goes totally south. So now that you know the where, and why ... please go fix. See I am off working with AC on the issues for 2.4.

Also with PREMPT, bah never mind.

Sigh... I do not want to start this but this problem has nothing to do with preemption and everything to do with you sleeping while holding a lock. It exists whether preempt is on or off.

Robert Love

#13 lore raw
Quoted 11 lines · Robert Love
Quoted 5 lines · Andre Hedrick

See in trying to move to a spinlock it goes totally south. So now that you know the where, and why ... please go fix. See I am off working with AC on the issues for 2.4.

Also with PREMPT, bah never mind.

Sigh... I do not want to start this but this problem has nothing to do with preemption and everything to do with you sleeping while holding a lock. It exists whether preempt is on or off.

Robert,

Glad we agree on the lock issue, thanks for confirming the point! There is an issue of interrupt acknowledgement and when one can pre-empt. I would like to resolve the issue, but I need a global caller/notifier api from you so I can block IO in a safe spot on the 'data transfer' state bar. Yeah, blah blah on underfined terms.

Some how I need to figure out how to address the pre-empt and keep the driver data stable. Initially I would suggest throttling back on the request size to maybe 4k or 8k regardless. I may not sound right but it will serve the purpose.

Cheers,

Andre Hedrick
LAD Storage Consulting Group

#14 lore raw
Quoted 1 line · Andre Hedrick

Glad we agree on the lock issue, thanks for confirming the point!

Great!

Quoted 4 lines

There is an issue of interrupt acknowledgement and when one can pre-empt. I would like to resolve the issue, but I need a global caller/notifier api from you so I can block IO in a safe spot on the 'data transfer' state bar. Yeah, blah blah on underfined terms.

Well, I do not know what the problem is (or what the hell you are talking about, to be honest). You really should not have any problems with preemption over regular SMP issues. If your code has a problem with other code running concurrently, then it should already hold a lock and thus be non-preemptive?

Also note we do not preempt interrupt handlers (obviously).

If you have a critical section in which you do not want to be preempted, do a:

	preempt_disable();
	/* critical section ... */
	preempt_enable();

This would have to be code that is in user-context and does not already hold a lock. There are very few explicit places that need this. You would be the first block driver, I believe.

Whatever this issue is, note it is entirely separate from the above locking issue. I also want to iterate that the locking problem (rescheduling while holding a lock) is a problem on UP even. Yes, think about it. Assuming the lock really needs to be held, it is protecting a critical region. If we reschedule, we can enter that region (or another one of the same data protected hopefully by the same lock). On SMP, we would deadlock. But on UP we will just silently corrupt the data. I.e., we can race on UP here.

Robert Love

#15 lore raw
In reply to Andre Hedrick
Quoted 9 lines · Andre Hedrick

There is an issue of interrupt acknowledgement and when one can pre-empt. I would like to resolve the issue, but I need a global caller/notifier api from you so I can block IO in a safe spot on the 'data transfer' state bar. Yeah, blah blah on underfined terms.

Some how I need to figure out how to address the pre-empt and keep the driver data stable. Initially I would suggest throttling back on the request size to maybe 4k or 8k regardless. I may not sound right but it will serve the purpose.

For things like old old broken PIO where interrupting the data stream screws up the data thats actually already covered. Pre-empt does actually do some things sensibly, and one of them is that when you hold a lock or disable irq you also disable pre-empt. That means hdparm -u0 PIO interface code is still going to do the right thing

Reminds me though Robert (and Jeff)

drivers/net/8390.c still needs ei_start_xmit fixing

pre-emption should be disabled between

/* Mask interrupts from the ethercard.
SMP: We have to grab the lock here otherwise the IRQ handler

and

       disable_irq_nosync(dev->irq);

        spin_lock(&ei_local->page_lock);

So that we don't leave the IRQ disabled due to pre-emption

(that code is wonderfully deranged but its the only way to make 8390 based chips not screw up things like serial I/O on a SMP box)

#16 lore raw
Quoted 6 lines · Alan Cox

Reminds me though Robert (and Jeff)

drivers/net/8390.c still needs ei_start_xmit fixing

pre-emption should be disabled between

Quoted 7 lines

/* Mask interrupts from the ethercard.
SMP: We have to grab the lock here otherwise the IRQ handler

and

       disable_irq_nosync(dev->irq);

        spin_lock(&ei_local->page_lock);

Sounds reasonable enough. What about the attached patch? If we flip the order of the disable_irq and spin_lock, we do not need to actually explicitly disable preemption... the lock will do that for us.

Is this safe?

This also has the general benefit of not spinning on the lock with the irq disabled (same sort of downside has preempting with the irq disabled).

Robert Love

drivers/net/8390.c +4 -5
diff -urN linux-2.5.39/drivers/net/8390.c linux/drivers/net/8390.c
@@ -243,9 +243,9 @@
 	}
 
 	/* Ugly but a reset can be slow, yet must be protected */
-		
-	disable_irq_nosync(dev->irq);
+
 	spin_lock(&ei_local->page_lock);
+	disable_irq_nosync(dev->irq);
 		
 	/* Try to restart the card.  Perhaps the user has fixed something. */
 	ei_reset_8390(dev);
@@ -286,10 +286,9 @@
 	/*
 	 *	Slow phase with lock held.
 	 */
-	 
-	disable_irq_nosync(dev->irq);
-	
+
 	spin_lock(&ei_local->page_lock);
+	disable_irq_nosync(dev->irq);
 	
 	ei_local->irqlock = 1;
 
#17 lore raw
In reply to Greg KH
Quoted 2 lines · Greg KH

The system still seems to be running ok, but I think I'll turn off CONFIG_PREEMPT just to be sure.

Note this has nothing to do with kernel preemption. IDE explicitly sleeps while purposely holding a lock.

It is just we do not have the ability to measure atomicity w/o preemption enabled - e.g. the debugging only works when it is enabled.

Robert Love

author
#18 lore raw
Quoted 10 lines · Robert Love
Quoted 2 lines · Greg KH

The system still seems to be running ok, but I think I'll turn off CONFIG_PREEMPT just to be sure.

Note this has nothing to do with kernel preemption. IDE explicitly sleeps while purposely holding a lock.

It is just we do not have the ability to measure atomicity w/o preemption enabled - e.g. the debugging only works when it is enabled.

Yes, you are correct. Sorry for stating that. It's shaking out lots of potential proplems :)

greg k-h

#19 lore raw
In reply to Robert Love
Quoted 5 lines · Robert Love

Note this has nothing to do with kernel preemption. IDE explicitly sleeps while purposely holding a lock.

It is just we do not have the ability to measure atomicity w/o preemption enabled - e.g. the debugging only works when it is enabled.

Would it be particularly difficult to separate this debug tool from the feature ? Surely we could make it so that CONFIG_PREEMPT depends on CONFIG_MIGHT_SLEEP or whatever, and just adds the actual ability to reschedule.

I have a bit of a problem with __might_sleep because I call sleepable stuff holding a spinlock (yes, it is justified, and yes, it is safe afaics, at least with PREEMPT=n)

regards
john

Signature · 2 lines
"When your name is Winner, that's it. You don't need a nickname."
	- Loser Lane
#20 lore raw
Quoted 6 lines · John Levon
Quoted 3 lines · Robert Love

Note this has nothing to do with kernel preemption. IDE explicitly sleeps while purposely holding a lock.

Quoted 7 lines
Quoted 2 lines

It is just we do not have the ability to measure atomicity w/o preemption enabled - e.g. the debugging only works when it is enabled.

Would it be particularly difficult to separate this debug tool from the feature ? Surely we could make it so that CONFIG_PREEMPT depends on CONFIG_MIGHT_SLEEP or whatever, and just adds the actual ability to reschedule.

We need a standalone CONFIG_MIGHT_SLEEP. I sinfully hooked it to CONFIG_DEBUG_KERNEL (it's not obvious why CONFIG_DEBUG_KERNEL exists actually).

So yes, you could make CONFIG_MIGHT_SLEEP mutually exclusive with CONFIG_OPROFILE. But that would make people look at you suspiciously.

Quoted 3 lines

I have a bit of a problem with __might_sleep because I call sleepable stuff holding a spinlock (yes, it is justified, and yes, it is safe afaics, at least with PREEMPT=n)

I'm looking at you suspiciously. How come?

#21 lore raw
Quoted 3 lines · Andrew Morton

We need a standalone CONFIG_MIGHT_SLEEP. I sinfully hooked it to CONFIG_DEBUG_KERNEL (it's not obvious why CONFIG_DEBUG_KERNEL exists actually).

Ah, OK.

Quoted 5 lines
Quoted 3 lines

I have a bit of a problem with __might_sleep because I call sleepable stuff holding a spinlock (yes, it is justified, and yes, it is safe afaics, at least with PREEMPT=n)

I'm looking at you suspiciously. How come?

NMI interrupt handler cannot block so it trylocks against a spinlock instead. Buffer processing code needs to block against concurrent NMI interrupts so takes the spinlock for them. All actual blocks on the spinlock are beneath a down() on another semaphore, so a sleep whilst holding the spinlock won't actually cause deadlock.

I don't know a way out of this that can safely ensure we've finished processing an NMI on the remote CPU the buffer processing code is about to look at.

[I'll post a new patch against 2.5.39 in a bit so you can see in context]

regards
john

Signature · 2 lines
"When your name is Winner, that's it. You don't need a nickname."
	- Loser Lane
#22 lore raw
Quoted 5 lines · John Levon

NMI interrupt handler cannot block so it trylocks against a spinlock instead. Buffer processing code needs to block against concurrent NMI interrupts so takes the spinlock for them. All actual blocks on the spinlock are beneath a down() on another semaphore, so a sleep whilst holding the spinlock won't actually cause deadlock.

If all accesses to the spinlock are taken under a semaphore, then the spinlock is not needed (i.e. the down'ed semaphore provides the same protection), or am I missing something?

If this is not the case - e.g. there are other accesses to these locks - then you cannot sleep, no?

I really can think of no case in which it is safe to sleep while holding a spinlock or otherwise atomic. If it is, then the atomicity is not needed, sort of by definition.

Robert Love

#23 lore raw
Quoted 6 lines · Robert Love
Quoted 5 lines

NMI interrupt handler cannot block so it trylocks against a spinlock instead. Buffer processing code needs to block against concurrent NMI interrupts so takes the spinlock for them. All actual blocks on the spinlock are beneath a down() on another semaphore, so a sleep whilst holding the spinlock won't actually cause deadlock.

Quoted 6 lines

If all accesses to the spinlock are taken under a semaphore, then the spinlock is not needed (i.e. the down'ed semaphore provides the same protection), or am I missing something?

If this is not the case - e.g. there are other accesses to these locks - then you cannot sleep, no?

The other accessors are spin_trylock()ers, as I mentioned. They will not block but they are not under the semaphore.

The spinlock cannot be a semaphore because NMI interrupts do not take to kindly to up()

regards
john

Signature · 2 lines
"When your name is Winner, that's it. You don't need a nickname."
	- Loser Lane
#24 lore raw
In reply to Robert Love
Quoted 5 lines · John Levon
Quoted 5 lines

NMI interrupt handler cannot block so it trylocks against a spinlock instead. Buffer processing code needs to block against concurrent NMI interrupts so takes the spinlock for them. All actual blocks on the spinlock are beneath a down() on another semaphore, so a sleep whilst holding the spinlock won't actually cause deadlock.

Quoted 8 lines · Robert Love

If all accesses to the spinlock are taken under a semaphore, then the spinlock is not needed (i.e. the down'ed semaphore provides the same protection), or am I missing something?
If this is not the case - e.g. there are other accesses to these locks - then you cannot sleep, no?
I really can think of no case in which it is safe to sleep while holding a spinlock or otherwise atomic. If it is, then the atomicity is not needed, sort of by definition.

Actually, though he may be using a spinlock_t, when used this way, it is not a spinlock, but rather a semaphore-like construct like PG_locked. Spinlocks include blocking via busywait semantics, which this usage does not have. It just happens to use the same data type. There are other interesting abuses of spinlock-like constructs in "advanced" locks, for instance, in non-sleeping handoff-scheduled queueing locks (e.g. MCS spinlocks and rwlocks) it's a common idiom for one waiter to set a "blocked" bit or lock word and then spin on it until another waiter and/or cpu manipulating the lock clears it.

Bill