Spinlocks and interrupts

When I readup on spinlocks, it seems like I need to choose between disabling interrupts and not. If a spinlock_t is never used during an interrupt, am I safe to leave interrupts enabled while I hold the lock? (Same question for read/write locks if it is different.)

Hi Kai,

Quoted 4 lines · Kai Meyer

When I readup on spinlocks, it seems like I need to choose between disabling interrupts and not. If a spinlock_t is never used during an interrupt, am I safe to leave interrupts enabled while I hold the lock? (Same question for read/write locks if it is different.)

So the intention behind using a spinlock is to provide mutual exclusion.

A spinlock by itself only really provides mutual exclusion between 2 cores, and not within the same core. To provide the mutual exclusion within the same core, you need to disable interrupts.

Normally, you would disable interrupts and acquire the spinlock to guarantee that mutual exclusion, and the only reason you would normally use the spinlock without disabling interrupts is when you know that interrupts are already disabled.

The danger of acquiring a spinlock with interrupts enabled is that if another interrupt fired (or the same interrupt fired again) and it tried to acquire the same spinlock, then you could have deadlock.

If no interrupts touch the spinlock, then you're probably using the wrong mutual exclusion mechanism. spinlocks are really intended to provide mutual exclsion between interrupt context and non-interrupt context.

Also remember, that on a non-SMP (aka UP) build, spinlocks become no-ops (except when certain debug checking code is enabled).

-----Original Message-----
From: kernelnewbies-bounces+jharan=bytemobile.com at kernelnewbies.org [mailto:kernelnewbies-
bounces+jharan=bytemobile.com at kernelnewbies.org] On Behalf Of Kai Meyer
Sent: Wednesday, November 09, 2011 3:12 PM
To: kernelnewbies at kernelnewbies.org
Subject: Re: Spinlocks and interrupts

Ok, I need mutual exclusion on a data structure regardless of

and core. It sounds like it can be done by using a spinlock and disabling interrupts, but you mention that "spinlocks are intended to provide mutual exclsion between interrupt context and non-interrupt context." Should I be using a semaphore (mutex) instead?

Perhaps I could explain my problem with some code:
struct my_struct *get_data(spinlock_t *mylock, int ALLOC_DATA)

{
     struct my_struct *mydata = NULL;
     spin_lock(mylock);
     if (test_bit(index, mybitmap))
             mydata = retrieve_data();
     if (!mydata && ALLOC_DATA) {
             mydata = alloc_data();
             set_bit(index, mybitmap);
     }
     spin_unlock(mylock);
     return mydata;
}

I need to prevent retrieve_data from being called if the index bit is set in mybitmap and alloc_data has not completed, so I use a bitmap to indicate that alloc_data has completed. I also need to protect alloc_data from being run multiple times, so I use the spin_lock to ensure that test_bit (and possibly retrieve_data) is not run while alloc_data is being run (because it runs while the bit is cleared).

-Kai Meyer

Ok, I need mutual exclusion on a data structure regardless of interrupts and core. It sounds like it can be done by using a spinlock and disabling interrupts, but you mention that "spinlocks are intended to provide mutual exclsion between interrupt context and non-interrupt context." Should I be using a semaphore (mutex) instead?

Perhaps I could explain my problem with some code:
struct my_struct *get_data(spinlock_t *mylock, int ALLOC_DATA) {
? ? struct my_struct *mydata = NULL;
? ? spin_lock(mylock);
? ? if (test_bit(index, mybitmap))
? ? ? ? ? ? mydata = retrieve_data();
? ? if (!mydata && ALLOC_DATA) {
? ? ? ? ? ? mydata = alloc_data();
? ? ? ? ? ? set_bit(index, mybitmap);
? ? }
? ? spin_unlock(mylock);
? ? return mydata;
}

I need to prevent retrieve_data from being called if the index bit is set in mybitmap and alloc_data has not completed, so I use a bitmap to indicate that alloc_data has completed. I also need to protect alloc_data from being run multiple times, so I use the spin_lock to ensure that test_bit (and possibly retrieve_data) is not run while alloc_data is being run (because it runs while the bit is cleared).

Hi Kai,

Quoted 4 lines · Kai Meyer

When I readup on spinlocks, it seems like I need to choose between disabling interrupts and not. If a spinlock_t is never used during an interrupt, am I safe to leave interrupts enabled while I hold the lock? (Same question for read/write locks if it is different.)

So the intention behind using a spinlock is to provide mutual exclusion.

A spinlock by itself only really provides mutual exclusion between 2 cores, and not within the same core. To provide the mutual exclusion within the same core, you need to disable interrupts.

Normally, you would disable interrupts and acquire the spinlock to guarantee that mutual exclusion, and the only reason you would normally use the spinlock without disabling interrupts is when you know that interrupts are already disabled.

The danger of acquiring a spinlock with interrupts enabled is that if another interrupt fired (or the same interrupt fired again) and it tried to acquire the same spinlock, then you could have deadlock.

If no interrupts touch the spinlock, then you're probably using the wrong mutual exclusion mechanism. spinlocks are really intended to provide mutual exclsion between interrupt context and non-interrupt context.

Also remember, that on a non-SMP (aka UP) build, spinlocks become no-ops (except when certain debug checking code is enabled).

--
Dave Hylands
Shuswap, BC, Canada
http://www.davehylands.com

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies at kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Also, remember that spin-locks are no-ops on a single processor machine, so as coded, you have no protection on a single-processor machine if you're calling from thread context.

Quoted 37 lines · Dave Hylands

Hi Kai,

Quoted 4 lines · Kai Meyer

When I readup on spinlocks, it seems like I need to choose between disabling interrupts and not. If a spinlock_t is never used during an interrupt, am I safe to leave interrupts enabled while I hold the lock? (Same question for read/write locks if it is different.)

So the intention behind using a spinlock is to provide mutual exclusion.

A spinlock by itself only really provides mutual exclusion between 2 cores, and not within the same core. To provide the mutual exclusion within the same core, you need to disable interrupts.

Normally, you would disable interrupts and acquire the spinlock to guarantee that mutual exclusion, and the only reason you would normally use the spinlock without disabling interrupts is when you know that interrupts are already disabled.

The danger of acquiring a spinlock with interrupts enabled is that if another interrupt fired (or the same interrupt fired again) and it tried to acquire the same spinlock, then you could have deadlock.

If no interrupts touch the spinlock, then you're probably using the wrong mutual exclusion mechanism. spinlocks are really intended to provide mutual exclsion between interrupt context and non-interrupt context.

Also remember, that on a non-SMP (aka UP) build, spinlocks become no-ops (except when certain debug checking code is enabled).

--
Dave Hylands
Shuswap, BC, Canada
http://www.davehylands.com

Quoted 4 lines

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies at kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Nice explanation Dave.

Regards,
Rohan Puri

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies at kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Hi Dave,

Quoted 3 lines

Also, remember that spin-locks are no-ops on a single processor machine, so as coded, you have no protection on a single-processor machine if you're calling from thread context.

Not completely true, spinlock can still provide protection even in this case i.e. two thread context sharing a resource on UP. Remember that for peemptive kernel, it disables preemption, so it is guranteed that scheduler does not through out this thread as long as spinlock is held.

I agree that a mutex can as well be used for this example but it again depends on situation, if you want to be quick about your task and are 100% sure that task does not sleep or takes too long processing cycles, disabling preemption (i.e. spinlock in this case) can be a better option.

Thanks,
Rajat

Quoted 47 lines · rohan puri

Quoted 37 lines · Dave Hylands

Hi Kai,

Quoted 4 lines · Kai Meyer

When I readup on spinlocks, it seems like I need to choose between disabling interrupts and not. If a spinlock_t is never used during an interrupt, am I safe to leave interrupts enabled while I hold the lock? (Same question for read/write locks if it is different.)

So the intention behind using a spinlock is to provide mutual exclusion.

A spinlock by itself only really provides mutual exclusion between 2 cores, and not within the same core. To provide the mutual exclusion within the same core, you need to disable interrupts.

Normally, you would disable interrupts and acquire the spinlock to guarantee that mutual exclusion, and the only reason you would normally use the spinlock without disabling interrupts is when you know that interrupts are already disabled.

The danger of acquiring a spinlock with interrupts enabled is that if another interrupt fired (or the same interrupt fired again) and it tried to acquire the same spinlock, then you could have deadlock.

If no interrupts touch the spinlock, then you're probably using the wrong mutual exclusion mechanism. spinlocks are really intended to provide mutual exclsion between interrupt context and non-interrupt context.

Also remember, that on a non-SMP (aka UP) build, spinlocks become no-ops (except when certain debug checking code is enabled).

--
Dave Hylands
Shuswap, BC, Canada
http://www.davehylands.com

Quoted 4 lines

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies at kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Nice explanation Dave.

Quoted 9 lines

Regards,
Rohan Puri

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies at kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies at kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Hi Kai,

Quoted 5 lines · Kai Meyer

Ok, I need mutual exclusion on a data structure regardless of interrupts and core. It sounds like it can be done by using a spinlock and disabling interrupts, but you mention that "spinlocks are intended to provide mutual exclsion between interrupt context and non-interrupt context." Should I be using a semaphore (mutex) instead?

It depends. If the function is only called from thread context, then you probably want to use a mutex. If there is a possibility that it might be called from interrupt context, then you can't use a mutex.

Also, remember that spin-locks are no-ops on a single processor machine, so as coded, you have no protection on a single-processor machine if you're calling from thread context.

Quoted 15 lines

Perhaps I could explain my problem with some code:
struct my_struct *get_data(spinlock_t *mylock, int ALLOC_DATA)

{
     struct my_struct *mydata = NULL;
     spin_lock(mylock);
     if (test_bit(index, mybitmap))
             mydata = retrieve_data();
     if (!mydata&&  ALLOC_DATA) {
             mydata = alloc_data();
             set_bit(index, mybitmap);
     }
     spin_unlock(mylock);
     return mydata;
}

Quoted 6 lines

I need to prevent retrieve_data from being called if the index bit is set in mybitmap and alloc_data has not completed, so I use a bitmap to indicate that alloc_data has completed. I also need to protect alloc_data from being run multiple times, so I use the spin_lock to ensure that test_bit (and possibly retrieve_data) is not run while alloc_data is being run (because it runs while the bit is cleared).

If alloc_data might block, then you can't disable interrupts and you definitely shouldn't be using spinlocks.

Quoted 8 lines · Dave Hylands

Hi Kai,

Quoted 5 lines · Kai Meyer

Ok, I need mutual exclusion on a data structure regardless of interrupts and core. It sounds like it can be done by using a spinlock and disabling interrupts, but you mention that "spinlocks are intended to provide mutual exclsion between interrupt context and non-interrupt context." Should I be using a semaphore (mutex) instead?

Quoted 8 lines

It depends. If the function is only called from thread context, then you probably want to use a mutex. If there is a possibility that it might be called from interrupt context, then you can't use a mutex.

Also, remember that spin-locks are no-ops on a single processor machine, so as coded, you have no protection on a single-processor machine if you're calling from thread context.

To make sure I understand you, it sounds like there's two contexts I need to be concerned about, thread context and interrupt context. As far as I can be sure, this code will only run in thread context. If you could verify for me that a block device's make request function is only reached in thread context, then that would make me doubly sure.

Quoted 15 lines

Quoted 15 lines

Perhaps I could explain my problem with some code:
struct my_struct *get_data(spinlock_t *mylock, int ALLOC_DATA)

{
      struct my_struct *mydata = NULL;
      spin_lock(mylock);
      if (test_bit(index, mybitmap))
              mydata = retrieve_data();
      if (!mydata&&   ALLOC_DATA) {
              mydata = alloc_data();
              set_bit(index, mybitmap);
      }
      spin_unlock(mylock);
      return mydata;
}

Quoted 9 lines

Quoted 6 lines

I need to prevent retrieve_data from being called if the index bit is set in mybitmap and alloc_data has not completed, so I use a bitmap to indicate that alloc_data has completed. I also need to protect alloc_data from being run multiple times, so I use the spin_lock to ensure that test_bit (and possibly retrieve_data) is not run while alloc_data is being run (because it runs while the bit is cleared).

If alloc_data might block, then you can't disable interrupts and you definitely shouldn't be using spinlocks.

alloc_data will call kmalloc(size, GFP_KERNEL), which I think may block, so disabling irqs is out.

Between thread context and kmalloc with GFP_KERNEL, it sounds like your suggestion would be to use a mutex. Is that correct?

-Kai Meyer

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies at kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Well, I changed my code to use a mutex instead of a spinlock, and now I get:

BUG: scheduling while atomic: swapper/0/0x10010000

All I changed was the spinlock_t to a struct mutex, and call mutex_init, mutex_lock, and mutex_unlock where I was previously calling the spin_lock variations. I'm confused. What does mutex_lock do besides set values in an atomic_t?

-Kai Meyer

Quoted 52 lines · Kai Meyer

Quoted 8 lines · Dave Hylands

Hi Kai,

Quoted 5 lines · Kai Meyer

Ok, I need mutual exclusion on a data structure regardless of interrupts and core. It sounds like it can be done by using a spinlock and disabling interrupts, but you mention that "spinlocks are intended to provide mutual exclsion between interrupt context and non-interrupt context." Should I be using a semaphore (mutex) instead?

Quoted 8 lines

It depends. If the function is only called from thread context, then you probably want to use a mutex. If there is a possibility that it might be called from interrupt context, then you can't use a mutex.

Also, remember that spin-locks are no-ops on a single processor machine, so as coded, you have no protection on a single-processor machine if you're calling from thread context.

To make sure I understand you, it sounds like there's two contexts I need to be concerned about, thread context and interrupt context. As far as I can be sure, this code will only run in thread context. If you could verify for me that a block device's make request function is only reached in thread context, then that would make me doubly sure.

Quoted 15 lines

Quoted 15 lines

Perhaps I could explain my problem with some code:
struct my_struct *get_data(spinlock_t *mylock, int ALLOC_DATA)

{
       struct my_struct *mydata = NULL;
       spin_lock(mylock);
       if (test_bit(index, mybitmap))
               mydata = retrieve_data();
       if (!mydata&&    ALLOC_DATA) {
               mydata = alloc_data();
               set_bit(index, mybitmap);
       }
       spin_unlock(mylock);
       return mydata;
}

Quoted 9 lines

Quoted 6 lines

I need to prevent retrieve_data from being called if the index bit is set in mybitmap and alloc_data has not completed, so I use a bitmap to indicate that alloc_data has completed. I also need to protect alloc_data from being run multiple times, so I use the spin_lock to ensure that test_bit (and possibly retrieve_data) is not run while alloc_data is being run (because it runs while the bit is cleared).

If alloc_data might block, then you can't disable interrupts and you definitely shouldn't be using spinlocks.

alloc_data will call kmalloc(size, GFP_KERNEL), which I think may block, so disabling irqs is out.

Between thread context and kmalloc with GFP_KERNEL, it sounds like your suggestion would be to use a mutex. Is that correct?

Quoted 6 lines

-Kai Meyer

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies at kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies at kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

I think I get it. I'm hitting the scheduling while atomic because I'm calling my function from a struct bio's endio function, which is probably running with a lock held somewhere else, and then my mutex sleeps, while the spin_lock functions do not sleep.

Perhaps I need to learn more about the context in which my endio function is being called.

-----Original Message-----
From: kernelnewbies-bounces+jharan=bytemobile.com at kernelnewbies.org [mailto:kernelnewbies-
bounces+jharan=bytemobile.com at kernelnewbies.org] On Behalf Of Dave Hylands
Sent: Thursday, November 10, 2011 11:07 AM
To: Kai Meyer
Cc: kernelnewbies at kernelnewbies.org
Subject: Re: Spinlocks and interrupts

Hi Kai,

Quoted 1 line · Kai Meyer

I think I get it. I'm hitting the scheduling while atomic because

Quoted 3 lines

calling my function from a struct bio's endio function, which is probably running with a lock held somewhere else, and then my mutex sleeps, while the spin_lock functions do not sleep.

Actually, just holding a lock doesn't create an atomic context.

Quoted 10 lines

-----Original Message-----
From: kernelnewbies-bounces+jharan=bytemobile.com at kernelnewbies.org [mailto:kernelnewbies-
bounces+jharan=bytemobile.com at kernelnewbies.org] On Behalf Of Dave Hylands
Sent: Thursday, November 10, 2011 11:07 AM
To: Kai Meyer
Cc: kernelnewbies at kernelnewbies.org
Subject: Re: Spinlocks and interrupts

Quoted 4 lines

Hi Kai,

Quoted 1 line · Kai Meyer

I think I get it. I'm hitting the scheduling while atomic because

I'm

Quoted 4 lines

Quoted 3 lines

calling my function from a struct bio's endio function, which is probably running with a lock held somewhere else, and then my mutex sleeps, while the spin_lock functions do not sleep.

Actually, just holding a lock doesn't create an atomic context.

I believe on kernels with kernel pre-emption enabled the act of taking the lock disables pre-emption. If it didn't work this way you could end up taking the lock in one process context and while the lock was held get pre-empted. Then another process tries to take the lock and you dead lock.

Jeff Haran

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies at kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

-----Original Message-----
From: kernelnewbies-bounces at kernelnewbies.org [mailto:kernelnewbies- bounces at kernelnewbies.org] On Behalf Of Kai Meyer
Sent: Thursday, November 10, 2011 1:55 PM
To: kernelnewbies at kernelnewbies.org
Subject: Re: Spinlocks and interrupts

Alright, to summarize, for my benefit mostly,

I'm writing a block device driver, which has 2 entry points into my

that will reach this critical section. It's either the make request function for the block device, or the resulting bio->bi_end_io

I do some waiting with msleep() (for now) from the make request

entry point, so I'm confident that entry point is not in an atomic context. I also only end up requesting the critical section to call kmalloc from this context, which is why I never ran into the

while atomic issue before.

I'm fairly certain the critical section executes in thread context not interrupt context from either entry point.

I'm certain that the spinlock_t is only ever used in one function (a I posted a simplified version of the critical section earlier).

It seems that the critical section is often called in an atomic

The spin_lock function sounds like it will only cause a second call to spin_lock to spin if it is called on a separate core.

But, since I'm certain the critical section is never called from interrupt context, only thread context, the fact that pre-emption is disabled on the core should provide the protection I need with out having to disable IRQs. Disabling IRQs would prevent an interrupt from occurring while the lock is acquired. I would like to avoid disabling interrupts if I don't need to.

So it sounds like spin_lock/spin_unlock is the correct choice?

In addition, I'd like to be more confident in my assumptions above.

I test for atomic context? For instance, I know that you can call irqs_disabled(), is there a similar is_atomic() function I can call? I would like to put a few calls in different places to learn what sort

context I'm.

-Kai Meyer

Quoted 2 lines · Jeff Haran

Quoted 2 lines

-----Original Message-----
From: kernelnewbies-

bounces+jharan=bytemobile.com at kernelnewbies.org

Quoted 2 lines

Quoted 2 lines

[mailto:kernelnewbies-
bounces+jharan=bytemobile.com at kernelnewbies.org] On Behalf Of

Dave

Quoted 13 lines

Quoted 10 lines

Hylands
Sent: Thursday, November 10, 2011 11:07 AM
To: Kai Meyer
Cc: kernelnewbies at kernelnewbies.org
Subject: Re: Spinlocks and interrupts

Hi Kai,

Quoted 1 line · Kai Meyer

I think I get it. I'm hitting the scheduling while atomic because

I'm

Quoted 2 lines

Quoted 2 lines

calling my function from a struct bio's endio function, which is probably running with a lock held somewhere else, and then my

Quoted 3 lines

Quoted 2 lines

Quoted 1 line

sleeps, while the spin_lock functions do not sleep.

Actually, just holding a lock doesn't create an atomic context.

I believe on kernels with kernel pre-emption enabled the act of

Quoted 1 line

the lock disables pre-emption. If it didn't work this way you could

Quoted 1 line

up taking the lock in one process context and while the lock was

Quoted 1 line

get pre-empted. Then another process tries to take the lock and you

Quoted 4 lines

lock.

Jeff Haran

Quoted 10 lines

-----Original Message-----
From: kernelnewbies-bounces at kernelnewbies.org [mailto:kernelnewbies- bounces at kernelnewbies.org] On Behalf Of Kai Meyer
Sent: Thursday, November 10, 2011 1:55 PM
To: kernelnewbies at kernelnewbies.org
Subject: Re: Spinlocks and interrupts

Alright, to summarize, for my benefit mostly,

I'm writing a block device driver, which has 2 entry points into my

code

Quoted 2 lines

that will reach this critical section. It's either the make request function for the block device, or the resulting bio->bi_end_io

function.

Quoted 1 line

I do some waiting with msleep() (for now) from the make request

function

Quoted 3 lines

entry point, so I'm confident that entry point is not in an atomic context. I also only end up requesting the critical section to call kmalloc from this context, which is why I never ran into the

scheduling

Quoted 9 lines

while atomic issue before.

I'm fairly certain the critical section executes in thread context not interrupt context from either entry point.

I'm certain that the spinlock_t is only ever used in one function (a I posted a simplified version of the critical section earlier).

It seems that the critical section is often called in an atomic

context.

Quoted 10 lines

The spin_lock function sounds like it will only cause a second call to spin_lock to spin if it is called on a separate core.

But, since I'm certain the critical section is never called from interrupt context, only thread context, the fact that pre-emption is disabled on the core should provide the protection I need with out having to disable IRQs. Disabling IRQs would prevent an interrupt from occurring while the lock is acquired. I would like to avoid disabling interrupts if I don't need to.

Quoted 3 lines

So it sounds like spin_lock/spin_unlock is the correct choice?

In addition, I'd like to be more confident in my assumptions above.

Can

Quoted 3 lines

I test for atomic context? For instance, I know that you can call irqs_disabled(), is there a similar is_atomic() function I can call? I would like to put a few calls in different places to learn what sort

of

Quoted 4 lines

context I'm.

-Kai Meyer

Quoted 20 lines

Quoted 2 lines · Jeff Haran

Quoted 2 lines

-----Original Message-----
From: kernelnewbies-

bounces+jharan=bytemobile.com at kernelnewbies.org

Quoted 2 lines

Quoted 2 lines

[mailto:kernelnewbies-
bounces+jharan=bytemobile.com at kernelnewbies.org] On Behalf Of

Dave

Quoted 13 lines

Quoted 10 lines

Hylands
Sent: Thursday, November 10, 2011 11:07 AM
To: Kai Meyer
Cc: kernelnewbies at kernelnewbies.org
Subject: Re: Spinlocks and interrupts

Hi Kai,

Quoted 1 line · Kai Meyer

I think I get it. I'm hitting the scheduling while atomic because

I'm

Quoted 2 lines

Quoted 2 lines

calling my function from a struct bio's endio function, which is probably running with a lock held somewhere else, and then my

mutex

Quoted 3 lines

Quoted 3 lines

Quoted 2 lines

Quoted 1 line

sleeps, while the spin_lock functions do not sleep.

Actually, just holding a lock doesn't create an atomic context.

I believe on kernels with kernel pre-emption enabled the act of

taking

Quoted 1 line

Quoted 1 line

the lock disables pre-emption. If it didn't work this way you could

end

Quoted 1 line

Quoted 1 line

up taking the lock in one process context and while the lock was

held

Quoted 1 line

Quoted 1 line

get pre-empted. Then another process tries to take the lock and you

dead

Quoted 4 lines

Quoted 4 lines

lock.

Jeff Haran

Kai, you might want to try bottom posting. It is the standard on these lists. It makes it easier for others to follow the thread.

I know of no kernel call that you can make to test for current execution context. There are the in_irq(), in_interrupt() and in_softirq() macros in hardirq.h, but when I've looked at the code that implements them I've come to the conclusion that they sometimes will lie. in_softirq() returns non-zero if you are in a software IRQ. Fair enough. But based on my reading in the past it's looked to me like it will also return non-zero if you've disabled bottom halves from process context with say a call to spin_lock_bh().

It would be nice if there were some way of asking the kernel what context you are in, for debugging if for no other reason, but if it's there I haven't found it.

I'd love to be proven wrong here, BTW. If others know better, please enlighten me.

Jeff Haran

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies at kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Quoted 111 lines · Jeff Haran

Quoted 10 lines

-----Original Message-----
From: kernelnewbies-bounces at kernelnewbies.org [mailto:kernelnewbies- bounces at kernelnewbies.org] On Behalf Of Kai Meyer
Sent: Thursday, November 10, 2011 1:55 PM
To: kernelnewbies at kernelnewbies.org
Subject: Re: Spinlocks and interrupts

Alright, to summarize, for my benefit mostly,

I'm writing a block device driver, which has 2 entry points into my

code

Quoted 2 lines

that will reach this critical section. It's either the make request function for the block device, or the resulting bio->bi_end_io

function.

Quoted 1 line

I do some waiting with msleep() (for now) from the make request

function

Quoted 3 lines

entry point, so I'm confident that entry point is not in an atomic context. I also only end up requesting the critical section to call kmalloc from this context, which is why I never ran into the

scheduling

Quoted 9 lines

while atomic issue before.

I'm fairly certain the critical section executes in thread context not interrupt context from either entry point.

I'm certain that the spinlock_t is only ever used in one function (a I posted a simplified version of the critical section earlier).

It seems that the critical section is often called in an atomic

context.

Quoted 10 lines

The spin_lock function sounds like it will only cause a second call to spin_lock to spin if it is called on a separate core.

But, since I'm certain the critical section is never called from interrupt context, only thread context, the fact that pre-emption is disabled on the core should provide the protection I need with out having to disable IRQs. Disabling IRQs would prevent an interrupt from occurring while the lock is acquired. I would like to avoid disabling interrupts if I don't need to.

Quoted 3 lines

So it sounds like spin_lock/spin_unlock is the correct choice?

In addition, I'd like to be more confident in my assumptions above.

Can

Quoted 3 lines

I test for atomic context? For instance, I know that you can call irqs_disabled(), is there a similar is_atomic() function I can call? I would like to put a few calls in different places to learn what sort

of

Quoted 4 lines

context I'm.

-Kai Meyer

Quoted 20 lines

Quoted 2 lines · Jeff Haran

Quoted 2 lines

-----Original Message-----
From: kernelnewbies-

bounces+jharan=bytemobile.com at kernelnewbies.org

Quoted 2 lines

Quoted 2 lines

[mailto:kernelnewbies-
bounces+jharan=bytemobile.com at kernelnewbies.org] On Behalf Of

Dave

Quoted 13 lines

Quoted 10 lines

Hylands
Sent: Thursday, November 10, 2011 11:07 AM
To: Kai Meyer
Cc: kernelnewbies at kernelnewbies.org
Subject: Re: Spinlocks and interrupts

Hi Kai,

Quoted 1 line · On Thu, Nov 10, 2011 at 10:14 AM, Kai Meyer<[email protected]> ? wrote:

I think I get it. I'm hitting the scheduling while atomic because

I'm

Quoted 2 lines

Quoted 2 lines

calling my function from a struct bio's endio function, which is probably running with a lock held somewhere else, and then my

mutex

Quoted 3 lines

Quoted 3 lines

Quoted 2 lines

Quoted 1 line

sleeps, while the spin_lock functions do not sleep.

Actually, just holding a lock doesn't create an atomic context.

I believe on kernels with kernel pre-emption enabled the act of

taking

Quoted 1 line

Quoted 1 line

the lock disables pre-emption. If it didn't work this way you could

end

Quoted 1 line

Quoted 1 line

up taking the lock in one process context and while the lock was

held

Quoted 1 line

Quoted 1 line

get pre-empted. Then another process tries to take the lock and you

dead

Quoted 4 lines

Quoted 4 lines

lock.

Jeff Haran

Kai, you might want to try bottom posting. It is the standard on these lists. It makes it easier for others to follow the thread.

I know of no kernel call that you can make to test for current execution context. There are the in_irq(), in_interrupt() and in_softirq() macros in hardirq.h, but when I've looked at the code that implements them I've come to the conclusion that they sometimes will lie. in_softirq() returns non-zero if you are in a software IRQ. Fair enough. But based on my reading in the past it's looked to me like it will also return non-zero if you've disabled bottom halves from process context with say a call to spin_lock_bh().

It would be nice if there were some way of asking the kernel what context you are in, for debugging if for no other reason, but if it's there I haven't found it.

I'd love to be proven wrong here, BTW. If others know better, please enlighten me.

Jeff Haran

Quoted 4 lines

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies at kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

I try to remember to bottom post on message lists, but obviously I've been negligent :)

Perhaps I'll just add some calls to msleep() at various places to help me identify when portions of my code are in an atomic context, just to help me learn what's going on.

-Kai Meyer

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies at kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

For most of the block drivers bio_endio runs in a context of its tasklet, so it is indeed atomic context.

-Rajat

Quoted 124 lines · Kai Meyer

Quoted 111 lines · Jeff Haran

Quoted 10 lines

-----Original Message-----
From: kernelnewbies-bounces at kernelnewbies.org [mailto:kernelnewbies- bounces at kernelnewbies.org] On Behalf Of Kai Meyer
Sent: Thursday, November 10, 2011 1:55 PM
To: kernelnewbies at kernelnewbies.org
Subject: Re: Spinlocks and interrupts

Alright, to summarize, for my benefit mostly,

I'm writing a block device driver, which has 2 entry points into my

code

Quoted 2 lines

that will reach this critical section. It's either the make request function for the block device, or the resulting bio->bi_end_io

function.

Quoted 1 line

I do some waiting with msleep() (for now) from the make request

function

Quoted 3 lines

entry point, so I'm confident that entry point is not in an atomic context. I also only end up requesting the critical section to call kmalloc from this context, which is why I never ran into the

scheduling

Quoted 9 lines

while atomic issue before.

I'm fairly certain the critical section executes in thread context not interrupt context from either entry point.

I'm certain that the spinlock_t is only ever used in one function (a I posted a simplified version of the critical section earlier).

It seems that the critical section is often called in an atomic

context.

Quoted 10 lines

The spin_lock function sounds like it will only cause a second call to spin_lock to spin if it is called on a separate core.

But, since I'm certain the critical section is never called from interrupt context, only thread context, the fact that pre-emption is disabled on the core should provide the protection I need with out having to disable IRQs. Disabling IRQs would prevent an interrupt from occurring while the lock is acquired. I would like to avoid disabling interrupts if I don't need to.

Quoted 3 lines

So it sounds like spin_lock/spin_unlock is the correct choice?

In addition, I'd like to be more confident in my assumptions above.

Can

Quoted 3 lines

I test for atomic context? For instance, I know that you can call irqs_disabled(), is there a similar is_atomic() function I can call? I would like to put a few calls in different places to learn what sort

of

Quoted 4 lines

context I'm.

-Kai Meyer

Quoted 20 lines

Quoted 2 lines · Jeff Haran

Quoted 2 lines

-----Original Message-----
From: kernelnewbies-

bounces+jharan=bytemobile.com at kernelnewbies.org

Quoted 2 lines

Quoted 2 lines

[mailto:kernelnewbies-
bounces+jharan=bytemobile.com at kernelnewbies.org] On Behalf Of

Dave

Quoted 13 lines

Quoted 10 lines

Hylands
Sent: Thursday, November 10, 2011 11:07 AM
To: Kai Meyer
Cc: kernelnewbies at kernelnewbies.org
Subject: Re: Spinlocks and interrupts

Hi Kai,

Quoted 1 line · Kai Meyer

I think I get it. I'm hitting the scheduling while atomic because

I'm

Quoted 2 lines

Quoted 2 lines

calling my function from a struct bio's endio function, which is probably running with a lock held somewhere else, and then my

mutex

Quoted 3 lines

Quoted 3 lines

Quoted 2 lines

Quoted 1 line

sleeps, while the spin_lock functions do not sleep.

Actually, just holding a lock doesn't create an atomic context.

I believe on kernels with kernel pre-emption enabled the act of

taking

Quoted 1 line

Quoted 1 line

the lock disables pre-emption. If it didn't work this way you could

end

Quoted 1 line

Quoted 1 line

up taking the lock in one process context and while the lock was

held

Quoted 1 line

Quoted 1 line

get pre-empted. Then another process tries to take the lock and you

dead

Quoted 4 lines

Quoted 4 lines

lock.

Jeff Haran

Kai, you might want to try bottom posting. It is the standard on these lists. It makes it easier for others to follow the thread.

I know of no kernel call that you can make to test for current execution context. There are the in_irq(), in_interrupt() and in_softirq() macros in hardirq.h, but when I've looked at the code that implements them I've come to the conclusion that they sometimes will lie. in_softirq() returns non-zero if you are in a software IRQ. Fair enough. But based on my reading in the past it's looked to me like it will also return non-zero if you've disabled bottom halves from process context with say a call to spin_lock_bh().

It would be nice if there were some way of asking the kernel what context you are in, for debugging if for no other reason, but if it's there I haven't found it.

I'd love to be proven wrong here, BTW. If others know better, please enlighten me.

Jeff Haran

Quoted 4 lines

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies at kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

I try to remember to bottom post on message lists, but obviously I've been negligent :)

Perhaps I'll just add some calls to msleep() at various places to help me identify when portions of my code are in an atomic context, just to help me learn what's going on.

Quoted 7 lines

-Kai Meyer

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies at kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies at kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies