"upping" a semaphore from interrupt context?
Hello,
I am writing a USB driver for some custom hardware, and I need to synchronize between the user-space and the USB subsystem. Can I create a semaphore and "down" it in the reader then "up" it in the completion handler?
I know the completion handler runs in interrupt context so you are not allowed to acquire any semaphores: but can you release them? Will the waiting tasks wake up after the handler and its caller returned - IOW will the waking up task run in interrupt context as well?
This is with Linux 2.4 (if it makes a difference).
Thanks,
florin
Signature · 2 lines
Bruce Schneier expects the Spanish Inquisition.
http://geekz.co.uk/schneierfacts/fact/163Quoted 4 lines · Florin Iucha
I am writing a USB driver for some custom hardware, and I need to synchronize between the user-space and the USB subsystem. Can I create a semaphore and "down" it in the reader then "up" it in the completion handler?
yes, but you should not. The use of semaphores is not recommended for new code, it should be replaced with either a mutex or a completion.
Quoted 4 lines
I know the completion handler runs in interrupt context so you are not allowed to acquire any semaphores: but can you release them? Will the waiting tasks wake up after the handler and its caller returned - IOW will the waking up task run in interrupt context as well?
What you should use is a 'struct completion', which you can wait for in process context and complete from interrupt context.
Arnd <><
Quoted 9 lines · Arnd Bergmann
Quoted 4 lines · Florin Iucha
I am writing a USB driver for some custom hardware, and I need to synchronize between the user-space and the USB subsystem. Can I create a semaphore and "down" it in the reader then "up" it in the completion handler?
yes, but you should not. The use of semaphores is not recommended for new code, it should be replaced with either a mutex or a completion.
can you clarify this? it sounds like you're saying that the current implementation of semaphores is entirely superfluous. but surely it isn't possible to replace all semaphores with either mutexes or completions, is it?
rday
Signature · 7 lines
======================================================================== Robert P. J. Day Linux Consulting, Training and Annoying Kernel Pedantry Waterloo, Ontario, CANADA http://fsdev.net/wiki/index.php?title=Main_Page ========================================================================
Quoted 10 lines · Robert P. J. Day
Quoted 4 lines · Arnd Bergmann
yes, but you should not. The use of semaphores is not recommended for new code, it should be replaced with either a mutex or a completion.
can you clarify this? it sounds like you're saying that the current implementation of semaphores is entirely superfluous. but surely it isn't possible to replace all semaphores with either mutexes or completions, is it?
No, not all of them, but the vast majority. There are multiple differences, the most important one being the 'counting' in semaphores. You can e.g. define a semaphore that can be held by N users at the same time, but not more. In a mutex, N is by definition 1, so only one thread can hold a mutex.
There are other subtle differences in the implementation, e.g. you cannot mutex_trylock at interrupt time.
Arnd <><
Quoted 7 lines · Arnd Bergmann
Quoted 6 lines · Robert P. J. Day
Quoted 4 lines · Arnd Bergmann
yes, but you should not. The use of semaphores is not recommended for new code, it should be replaced with either a mutex or a completion.
Quoted 8 lines
Quoted 4 lines
can you clarify this? it sounds like you're saying that the current implementation of semaphores is entirely superfluous. but surely it isn't possible to replace all semaphores with either mutexes or completions, is it?
No, not all of them, but the vast majority. There are multiple differences, the most important one being the 'counting' in semaphores.
right, that was exactly the feature i was thinking of. ok, i'm clear on this now -- while the *majority* of semaphores can be more properly replaced by mutexes or completions, there will always be a need for a general-purpose counting semaphore.
rday
Signature · 7 lines
======================================================================== Robert P. J. Day Linux Consulting, Training and Annoying Kernel Pedantry Waterloo, Ontario, CANADA http://fsdev.net/wiki/index.php?title=Main_Page ========================================================================
Hi Robert, Arnd,
Quoted 11 lines · Arnd Bergmann
Quoted 10 lines · Robert P. J. Day
Quoted 4 lines · Arnd Bergmann
yes, but you should not. The use of semaphores is not recommended for new code, it should be replaced with either a mutex or a completion.
can you clarify this? it sounds like you're saying that the current implementation of semaphores is entirely superfluous. but surely it isn't possible to replace all semaphores with either mutexes or completions, is it?
Semaphores being used as completions are superfluous, obsoleted by completion handlers. Semaphores that are not counted (hence binary) are superfluous, obsoleted by struct mutex. It's not that using semaphores for the above two usages would be "incorrect", it's just that the other options are precisely implemented for their specific purpose and hence are better (benchmark struct mutex against (binary) struct semaphore yourself, for example). So there's no good reason why a driver's design would want to use semaphores as above with the other available options.
A simple way to detect users who are still {mis}using semaphores as completions are those that will (thus) inevitably have to declare (or initialize) them as locked, say most (all?) users of DECLARE_MUTEX_LOCKED.
22-rc5 has 4 such users, one of which is dead code inside a #if 0, one declared a spurious semaphore variable without using it anywhere else in the code (deleted in -mm), one has been converted to completions (in -mm) already, and the last one (libusual.c) is the problematic one which still exists, because of the comment in that file that it wants the _completions_ to be also *counted*, and thus ostensibly wants to use semaphores instead of completions. However, that comment is totally wrong, and doesn't seem to know about the existence of the complete_all() function that precisely serves the purpose that libusual wants.
In short: there are _no_ valid excuses to use DECLARE_MUTEX_LOCKED (or semaphores as completions) ever.
[ libusual's {ab}use of that semaphore is *insane* for several other reasons, but then that will be the subject matter of another thread :-) ]
Quoted 8 lines
No, not all of them, but the vast majority. There are multiple differences, the most important one being the 'counting' in semaphores. You can e.g. define a semaphore that can be held by N users at the same time, but not more. In a mutex, N is by definition 1, so only one thread can hold a mutex.
There are other subtle differences in the implementation, e.g. you cannot
mutex_trylockat interrupt time.
IMHO, mutexes (or semaphores, be it binary ones or counted) are simply *not* the correct locking primitive to use from interrupt context in the first place. down_trylock() did work from interrupt contexts earlier (without producing any __might_sleep-induced warnings either) but mutex_trylock() does not work, and rightly so. Somebody using a down_trylock() from interrupt context is a sure-shot sign of design error (or massive ugliness) in the driver somewhere (which could, and should, be resolved in design).
To elaborate: what is the (only) valid way in which such code (that attempts to down_trylock() or mutex_trylock() from interrupt context) can be written?
*****
err = down_trylock(...);
if (err) {
/*
* Either:
* 1. proceed although someone else owns the lock anyway => insane
* 2. too bad, so ignore this interrupt just this once => WTF?
* 3. set up a timer and schedule another function to service the
* interrupt / do what needs to be done then, hopefully the mutex
* would be uncontended then => *gargh*
*/
}
/* do what needs to be done with the shared data to service this interrupt */
*****
Option 2 is nonsense. Option 3 (believe me, I've seen code like that) is even more so :-) [ note that timers run in softirq-context, so illegal to use mutex_lock() or down() there again, which means we'd *again* have to use _trylock() variant there, with the same ugly error handling I described above. ]
Satyam
Am Samstag, 23. Juni 2007 schrieb Satyam Sharma:
Quoted 3 lines
* 3. set up a timer and schedule another function to service the
* interrupt / do what needs to be done then, hopefully the mutex
* would be uncontended then => *gargh*
You could use schedule_work(). However then why not use it always. This would make sense if what you want to do is outright trivial.
Regards
Oliver
Quoted 7 lines · Oliver Neukum
Am Samstag, 23. Juni 2007 schrieb Satyam Sharma:
Quoted 3 lines
* 3. set up a timer and schedule another function to service the
* interrupt / do what needs to be done then, hopefully the mutex
* would be uncontended then => *gargh*You could use
schedule_work(). However then why not use it always. This would make sense if what you want to do is outright trivial.
If you use schedule_work() to pass off work from interrupt context to process context, then you wouldn't be calling down_trylock() from interrupt context in the first place (which is what is being discussed here). You would simply pass off the entire code that uses the shared data (and wraps a *proper* down() or mutex_lock() around it, not the _trylock() variant) to the workqueue.
Also, that is precisely my point too. What I'm saying is that it is generally poor design to be wanting to use the _trylock() variant of semaphore / mutex in interrupt context. Workqueues _are_ the preferred mechanism to use for (most) such cases where you need to do something that may require you to sleep.
Satyam
Quoted 10 lines · Satyam Sharma
Hi Robert, Arnd,
Quoted 7 lines · Arnd Bergmann
Quoted 6 lines · Robert P. J. Day
Quoted 4 lines · Arnd Bergmann
yes, but you should not. The use of semaphores is not recommended for new code, it should be replaced with either a mutex or a completion.
Quoted 8 lines
Quoted 4 lines
Quoted 4 lines
can you clarify this? it sounds like you're saying that the current implementation of semaphores is entirely superfluous. but surely it isn't possible to replace all semaphores with either mutexes or completions, is it?
Semaphores being used as completions are superfluous, obsoleted by completion handlers. Semaphores that are not counted (hence binary) are superfluous, obsoleted by struct mutex.
hang on, how is that true? as i read it, mutexes are more than just binary semaphores -- they have additional restrictions that regular semaphores don't. so i'm not convinced that binary semaphores can simply be replaced by mutexes, unless that's not what you meant here.
rday
Signature · 7 lines
======================================================================== Robert P. J. Day Linux Consulting, Training and Annoying Kernel Pedantry Waterloo, Ontario, CANADA http://fsdev.net/wiki/index.php?title=Main_Page ========================================================================
Quoted 17 lines · Robert P. J. Day
Quoted 15 lines · Satyam Sharma
Hi Robert, Arnd,
Quoted 11 lines · Arnd Bergmann
Quoted 10 lines · Robert P. J. Day
Quoted 4 lines · Arnd Bergmann
yes, but you should not. The use of semaphores is not recommended for new code, it should be replaced with either a mutex or a completion.
can you clarify this? it sounds like you're saying that the current implementation of semaphores is entirely superfluous. but surely it isn't possible to replace all semaphores with either mutexes or completions, is it?
Quoted 7 lines
Quoted 3 lines
Semaphores being used as completions are superfluous, obsoleted by completion handlers. Semaphores that are not counted (hence binary) are superfluous, obsoleted by struct mutex.
hang on, how is that true? as i read it, mutexes are more than just binary semaphores -- they have additional restrictions that regular semaphores don't.
Yes, they do have additional restrictions (mutex_trylock() illegal from contexts that cannot sleep, mutexes may only be unlocked by tasks that took them in the first place). But note that these are _implementation_ sanity checks that were introduced to catch nonsensical usage, which was possible (and not explicitly being guarded against, because of the generic-ness that was needed to be maintained for the counted case too) with the "semaphore"s.
Quoted 2 lines
so i'm not convinced that binary semaphores can
simply be replaced by mutexes, unless that's not what you meant here.
I do mean precisely that. I really cannot think of any sensible / normal usage case of binary semaphores that cannot be replaced with either mutexes (if that's the kind of locking you actually want) or completion handlers (if that's the kind of synchronization you actually want).
Satyam
Quoted 14 lines · Satyam Sharma
Quoted 3 lines · Robert P. J. Day
hang on, how is that true? as i read it, mutexes are more than just binary semaphores -- they have additional restrictions that regular semaphores don't.
Yes, they do have additional restrictions (
mutex_trylock()illegal from contexts that cannot sleep, mutexes may only be unlocked by tasks that took them in the first place). But note that these are _implementation_ sanity checks that were introduced to catch nonsensical usage, which was possible (and not explicitly being guarded against, because of the generic-ness that was needed to be maintained for the counted case too) with the "semaphore"s.
Quoted 8 lines
Quoted 2 lines
so i'm not convinced that binary semaphores can simply be replaced by mutexes, unless that's not what you meant here.
I do mean precisely that. I really cannot think of any sensible / normal usage case of binary semaphores that cannot be replaced with either mutexes (if that's the kind of locking you actually want) or completion handlers (if that's the kind of synchronization you actually want).
ok, i can see what you're getting at now. i'm just going to have to convince myself that, once you have mutexes and completions, you have no further need for even general counting semaphores.
rday
Hi Robert,
Quoted 25 lines · Robert P. J. Day
Quoted 14 lines · Satyam Sharma
Quoted 3 lines · Robert P. J. Day
hang on, how is that true? as i read it, mutexes are more than just binary semaphores -- they have additional restrictions that regular semaphores don't.
Yes, they do have additional restrictions (
mutex_trylock()illegal from contexts that cannot sleep, mutexes may only be unlocked by tasks that took them in the first place). But note that these are _implementation_ sanity checks that were introduced to catch nonsensical usage, which was possible (and not explicitly being guarded against, because of the generic-ness that was needed to be maintained for the counted case too) with the "semaphore"s.Quoted 8 lines
Quoted 2 lines
so i'm not convinced that binary semaphores can simply be replaced by mutexes, unless that's not what you meant here.
I do mean precisely that. I really cannot think of any sensible / normal usage case of binary semaphores that cannot be replaced with either mutexes (if that's the kind of locking you actually want) or completion handlers (if that's the kind of synchronization you actually want).
Quoted 3 lines
ok, i can see what you're getting at now. i'm just going to have to convince myself that, once you have mutexes and completions, you have no further need for even general counting semaphores.
Whoa, hold on. But I've been explicitly mentioning *binary* semaphores all along!
Of course users who want / allow multiple tasks (but only upto a specific maximum number, which is what counted semaphores are all about) to be present in a given critical section simultaneously would still want to use the _counted_ semaphores, which is why you won't see the old "struct semaphores" dying anytime soon.
Satyam
Quoted 8 lines · Satyam Sharma
Whoa, hold on. But I've been explicitly mentioning *binary* semaphores all along!
Of course users who want / allow multiple tasks (but only upto a specific maximum number, which is what counted semaphores are all about) to be present in a given critical section simultaneously would still want to use the _counted_ semaphores, which is why you won't see the old "struct semaphores" dying anytime soon.
ah, ok, i've been misreading all this, sorry. i'm wondering, then, if, given both mutexes and completions, whether the general semantics of semaphores can be tightened up at all. just curious.
rday
Signature · 7 lines
======================================================================== Robert P. J. Day Linux Consulting, Training and Annoying Kernel Pedantry Waterloo, Ontario, CANADA http://fsdev.net/wiki/index.php?title=Main_Page ========================================================================
Hi,
Quoted 11 lines · Robert P. J. Day
Quoted 8 lines · Satyam Sharma
Whoa, hold on. But I've been explicitly mentioning *binary* semaphores all along!
Of course users who want / allow multiple tasks (but only upto a specific maximum number, which is what counted semaphores are all about) to be present in a given critical section simultaneously would still want to use the _counted_ semaphores, which is why you won't see the old "struct semaphores" dying anytime soon.
Quoted 3 lines
ah, ok, i've been misreading all this, sorry. i'm wondering, then, if, given both mutexes and completions, whether the general semantics of semaphores can be tightened up at all. just curious.
[ Ok, I'll elaborate, just in case someone else interested regarding this is watching or reading the archives ... but this may be waaaaay too basic for you and others here, so those can safely ignore it. ]
First: By a semaphore being binary, I mean that the value of that semaphore (counter) variable can only be 0 or 1. Say we follow the rule that 1 == available (unlocked) and 0 == unavailable (locked). So down == lock == test-and-decrement, up == unlock == increment.
Also, note that there is nothing (I'm _not_ talking of implementation details here) that the "struct mutex" can do something that a *binary* "struct semaphore" can't, if usage ensures that its value is always in {0, 1}. So semantically speaking they are equivalent.
1. (Binary) semaphores used for locking:
========================================
Binary semaphores may be used to synchronize access to critical sections that can (must) only be executed by _only_ *one* task at any given time. That is, _no_ concurrent execution of that piece of code can be tolerated. This is often the most common case / usage to implement lock-protection of any shared data.
Now obviously, because only _one_ task can be in such critical sections at any given time, who (and only who) can be allowed to unlock the semaphore (i.e. increment the counter variable back to 1)?
The task that took this (binary) semaphore in the first place, isn't it? So this precisely one of those extra sanity checks that the "struct mutex" introduces. [ There are other sanity checks listed at the top of include/linux/mutex.h, but this is one of the important ones, IMHO. ]
Why is this check _not_ applicable to *counting* semaphores? Because _multiple_ tasks can be in that given critical section _concurrently_ so the task that _last_ down()'ed the semaphore *need not be* forced to be same as the task that _next_ wants to up() it!
2. Semaphores used to indicate completion:
==========================================
Ok, this one's again purely an implementation thing -- in fact if you look at the way "struct semaphore"s are currently implemented in the kernel they _do_ use waitqueues, which is precisely what "struct completion"s use as well. The difference is even less pronounced because all down()'s do become (in the slowpath i.e. when the test failed so we decide to schedule() ourselves out) _add_wait_queue_tail with callers marking themselves as exclusive waiters), and all up()'s also do call __wake_up_common with NR_EXCLUSIVE == 1 to ensure that at least one exclusively-waiting task _is_ woken up when the lock is released (i.e. counter incremented).
So this basically means that one _can_ use semaphores as completion notification primitives, and also (to again give the technically accurate answer to Florin's original question on this thread) "upping" is not illegal per-se from interrupt/atomic contexts.
But wait_for_completion() and complete() are to be still preferred in new code (although they have a very similar implementation) because they are specifically for this purpose, plus they avoid all the extra fastpath/lock-acquiring stuff that the semaphore implementation involves (and one less member in the struct if nothing else). [ I have never benchmarked, of course, but I'd expect use of the completion primitives to be microscopically faster than using semaphores in this manner too. ]
Satyam
Hi Florin,
Quoted 6 lines · Florin Iucha
Hello,
I am writing a USB driver for some custom hardware, and I need to synchronize between the user-space and the USB subsystem. Can I create a semaphore and "down" it in the reader then "up" it in the completion handler?
It's not exactly clear from your description what you are "synchronizing" exactly ... if there is some shared data you want accessed from process as well as interrupt context, the (only) safe primitives to use are spin_lock_irqsave / spin_unlock_irqrestore.
If you simply want the process context task to block _till_ it receives some kind of notification that some other job has been completed (seems to be from interrupt context in your case from your description) than I suspect "struct completion" and associated primitives are what you are looking for.
Quoted 4 lines
I know the completion handler runs in interrupt context so you are not allowed to acquire any semaphores: but can you release them? Will the waiting tasks wake up after the handler and its caller returned - IOW will the waking up task run in interrupt context as well?
The waiting task (say if it was blocked at wait_for_completion) would continue to execute in the same context it was from that point onwards.
Quoted 1 line
This is with Linux 2.4 (if it makes a difference).
Whoa, it does, I would expect. I'm not sure completion handlers exist in 2.4? If they do, they're the way to go. If not, sorry, I'm not well versed with the 2.4 kernel at all.
Satyam