mutex: make mutex_lock_nested an inline function
Subsystems (2)
PATCH 1/1 mutex: make mutex_lock_nested an inline function
The second argument of the mutex_lock_nested() helper is only evaluated if CONFIG_DEBUG_LOCK_ALLOC is set. Otherwise we get this build warning for the new regulator_lock_supply function:
drivers/regulator/core.c: In function 'regulator_lock_supply':drivers/regulator/core.c:142:6: warning: unused variable 'i' [-Wunused-variable]
To avoid the warning, this patch changes the definition of mutex_lock_nested() to be static inline function rather than a macro, which tells gcc that the variable is potentially used.
---
Second try, this time fixing the root of the problem rather than the code that triggered it, as suggested by Mark Brown.
I guess this means the fix should now probably go through Ingo's sched-locking branch or through Andrew's mm patches. The warning currently only shows up in Linux-next because of Mark's regulator branch that is destined for 4.4, so it's not needed for 4.3 but also harmless.
include/linux/mutex.h +4 -1
@@ -158,7 +158,10 @@ extern void mutex_lock(struct mutex *lock);extern int __must_check mutex_lock_interruptible (struct mutex *lock);extern int __must_check mutex_lock_killable (struct mutex *lock); -# define mutex_lock_nested (lock, subclass) mutex_lock(lock) +static inline void mutex_lock_nested (struct mutex *lock,unsigned int subclass) +{ +return mutex_lock (lock); +}# define mutex_lock_interruptible_nested (lock, subclass) mutex_lock_interruptible(lock)# define mutex_lock_killable_nested (lock, subclass) mutex_lock_killable(lock)# define mutex_lock_nest_lock (lock, nest_lock) mutex_lock(lock)
Peter Zijlstra
maintainerQuoted 13 lines · Arnd Bergmann
The second argument of the
mutex_lock_nested()helper is only evaluated ifCONFIG_DEBUG_LOCK_ALLOCis set. Otherwise we get this build warning for the newregulator_lock_supplyfunction:
drivers/regulator/core.c: In function 'regulator_lock_supply':drivers/regulator/core.c:142:6: warning: unused variable 'i' [-Wunused-variable]To avoid the warning, this patch changes the definition of
mutex_lock_nested()to be static inline function rather than a macro, which tells gcc that the variable is potentially used.
include/linux/mutex.h extern void mutex_lock(struct mutex *lock); · line 164
Quoted 5 lines
include/linux/mutex.h +4
-1-# define mutex_lock_nested (lock, subclass) mutex_lock(lock) +static inline void mutex_lock_nested (struct mutex *lock,unsigned int subclass) +{ +return mutex_lock (lock); +}
Can you verify that this results in an identical kernel?
Having this a proper argument results in the compiler having to actually evaluate the expression resulting in @subclass, this might have side effects and generate code.
A quick grep shows a large amount of trivial code that optimizers will still happily throw away, but it should be verified that this does not result in pointless code generation.
Quoted 22 lines · On Tuesday 13 October 2015 22:38:12 Peter Zijlstra wrote:
Quoted 13 lines · Arnd Bergmann
The second argument of the
mutex_lock_nested()helper is only evaluated ifCONFIG_DEBUG_LOCK_ALLOCis set. Otherwise we get this build warning for the newregulator_lock_supplyfunction:
drivers/regulator/core.c: In function 'regulator_lock_supply':drivers/regulator/core.c:142:6: warning: unused variable 'i' [-Wunused-variable]To avoid the warning, this patch changes the definition of
mutex_lock_nested()to be static inline function rather than a macro, which tells gcc that the variable is potentially used.Review
Quoted 5 lines
(quoted diff) +4
-1-# define mutex_lock_nested(lock, subclass) mutex_lock(lock) +static inline void mutex_lock_nested(struct mutex *lock, unsigned int subclass) +{ + return mutex_lock(lock); +}Can you verify that this results in an identical kernel?
Quoted 7 lines
Having this a proper argument results in the compiler having to actually evaluate the expression resulting in @subclass, this might have side effects and generate code.
A quick grep shows a large amount of trivial code that optimizers will still happily throw away, but it should be verified that this does not result in pointless code generation.
Indeed, I'm seeing a tiny code growth with ARM multi_v7_defconfig when my patch is applied, as the image (according to size -A) grows from 13740187 bytes to 13740283, all of it in .text of two drivers (i2c-core and three files of bluetooth.ko).
multi_v7_defconfig/vmlinux.o.size renamed from multi_v7_defconfig-before/vmlinux.o.size +5 -5
@@ -1,6 +1,6 @@ build/multi_v7_defconfig/vmlinux.o : section size addr -.text 8219408 0 +.text 8219504 0 --- build/multi_v7_defconfig-before/net/bluetooth/bluetooth.ko.size 2015-10-13 23:11:40.704382038 +0200 +++ build/multi_v7_defconfig/net/bluetooth/bluetooth.ko.size 2015-10-13 23:07:58.639116862 +0200
@@ -1,7 +1,7 @@ build/multi_v7_defconfig/net/bluetooth/bluetooth.ko : section size addr .note.gnu.build-id 36 0 -.text 241512 0 +.text 241696 0 --- build/multi_v7_defconfig-before/drivers/i2c/i2c-core.o.size 2015-10-13 23:11:40.636385326 +0200 +++ build/multi_v7_defconfig/drivers/i2c/i2c-core.o.size 2015-10-13 23:07:53.403369830 +0200
@@ -1,6 +1,6 @@ build/multi_v7_defconfig/drivers/i2c/i2c-core.o : section size addr -.text 12112 0 +.text 12208 0
The code in question is
a)
static ssize_t
i2c_sysfs_delete_device(struct device *dev, struct device_attribute *attr,
const char *buf, size_t count)
{
...
mutex_lock_nested(&adap->userspace_clients_lock,
i2c_adapter_depth(adap));...
}
and
b)
static inline void l2cap_chan_lock(struct l2cap_chan *chan)
{
mutex_lock_nested(&chan->lock, atomic_read(&chan->nesting));
} The first one has a small size impact but no performance change as it is only called during probe/release of i2c modules. The second one adds an extra pointer access (due to the volatile keyword in atomic_read()) for every caller of l2cap_chan_lock().
Arnd
Peter Zijlstra
maintainerQuoted 6 lines
Quoted 6 lines · Arnd Bergmann
Quoted 5 lines
The second argument of the
mutex_lock_nested()helper is only evaluated ifCONFIG_DEBUG_LOCK_ALLOCis set. Otherwise we get this build warning for the newregulator_lock_supplyfunction:
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
drivers/regulator/core.c: In function 'regulator_lock_supply':drivers/regulator/core.c:142:6: warning: unused variable 'i' [-Wunused-variable]To avoid the warning, this patch changes the definition of
mutex_lock_nested()to be static inline function rather than a macro, which tells gcc that the variable is potentially used.
Uuh, I just looked at next and saw this regulator_lock_supply() function. How is that limited? subclass must be <8 otherwise bad things happen.
Peter Zijlstra
maintainerQuoted 9 lines · Peter Zijlstra
Quoted 9 lines
Quoted 9 lines · Arnd Bergmann
Quoted 8 lines
The second argument of the
mutex_lock_nested()helper is only evaluated ifCONFIG_DEBUG_LOCK_ALLOCis set. Otherwise we get this build warning for the newregulator_lock_supplyfunction:
drivers/regulator/core.c: In function 'regulator_lock_supply':drivers/regulator/core.c:142:6: warning: unused variable 'i' [-Wunused-variable]
Quoted 8 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
To avoid the warning, this patch changes the definition of
mutex_lock_nested()to be static inline function rather than a macro, which tells gcc that the variable is potentially used.Uuh, I just looked at next and saw this
regulator_lock_supply()function. How is that limited? subclass must be <8 otherwise bad things happen.
Also, the function appears unused, just delete it ;-)
Quoted 15 lines · On Wednesday 14 October 2015 10:37:07 Peter Zijlstra wrote:
Quoted 14 lines · Peter Zijlstra
Quoted 13 lines
Quoted 13 lines · Arnd Bergmann
Quoted 12 lines
The second argument of the
mutex_lock_nested()helper is only evaluated ifCONFIG_DEBUG_LOCK_ALLOCis set. Otherwise we get this build warning for the newregulator_lock_supplyfunction:
drivers/regulator/core.c: In function 'regulator_lock_supply':drivers/regulator/core.c:142:6: warning: unused variable 'i' [-Wunused-variable]To avoid the warning, this patch changes the definition of
mutex_lock_nested()to be static inline function rather than a macro, which tells gcc that the variable is potentially used.
Quoted 5 lines
Quoted 3 lines
Uuh, I just looked at next and saw this
regulator_lock_supply()function. How is that limited? subclass must be <8 otherwise bad things happen.Also, the function appears unused, just delete it
That was my first suggestion as well when I ran into
drivers/regulator/core.c:139:13: warning: 'regulator_lock_supply' defined but not used
but apparently this is work-in-progress and the plan is to use it in 4.4 when the rest of the currently-under-review patches are merged.
Arnd
Peter Zijlstra
maintainerQuoted 6 lines · Arnd Bergmann
Quoted 4 lines · On Wednesday 14 October 2015 10:37:07 Peter Zijlstra wrote:
Quoted 3 lines
Uuh, I just looked at next and saw this
regulator_lock_supply()function. How is that limited? subclass must be <8 otherwise bad things happen.
Quoted 8 lines
Quoted 1 line
Also, the function appears unused, just delete it
That was my first suggestion as well when I ran into
drivers/regulator/core.c:139:13: warning: 'regulator_lock_supply' defined but not usedbut apparently this is work-in-progress and the plan is to use it in 4.4 when the rest of the currently-under-review patches are merged.
And here I thought we had a fairly strong rule that we should not merge unused code :/
In any case, whomever wrote that function had better first explain why its not broken.
And I'm not too keen on making mutex_lock_nested() an inline due it being too easy to generate code with it. I'd much rather work around the occasional warning than have to deal with silent but unintended code spills.
Quoted 8 lines · Peter Zijlstra
Quoted 4 lines · Arnd Bergmann
On Wednesday 14 October 2015 10:37:07 Peter Zijlstra wrote:
but apparently this is work-in-progress and the plan is to use it in 4.4 when the rest of the currently-under-review patches are merged.
And here I thought we had a fairly strong rule that we should not merge unused code :/
The revisions are very minor (and are sitting in my inbox for me to
look at just now), and keeping things out of tree just makes it more
difficult to get them merged - it's one thing if things are getting in
the way for a long time or have a high cost but that's not the case
here.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20151014/e475a529/attachment.sig>
Quoted 3 lines · Peter Zijlstra
Uuh, I just looked at next and saw this
regulator_lock_supply()function. How is that limited? subclass must be <8 otherwise bad things happen.
Can we please get some more discoverable documentation of the arbitrary limits in the lockdep code? I seem to keep seeing code that bumps into surprising limits like this and I'm not sure how I'm supposed to know about them except through finding out after the fact or trawling the code every time someone touches locking.
I would be very surprised to see a system that pushes over 8 locks,
while there's nothing actually preventing it system design
considerations mean that even four cascaded supplies are pretty
unlikely so we should be fine. Every time you add a new level of
regulation you're both increasing the load on regulators up the chain
(which means they need to be bigger and more expensive) and except in
the case of a DCDC supplying an LDO (which only works to one level)
you're going to be decreasing the efficiency of the system. If we get
to that point we can worry about what to do.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20151014/2eaedd02/attachment.sig>
Peter Zijlstra
maintainerQuoted 8 lines · Mark Brown
Quoted 3 lines · Peter Zijlstra
Uuh, I just looked at next and saw this
regulator_lock_supply()function. How is that limited? subclass must be <8 otherwise bad things happen.Can we please get some more discoverable documentation of the arbitrary limits in the lockdep code?
include/linux/lockdep.h:#define MAX_LOCKDEP_SUBCLASSES 8UL
Also, it will give a runtime warn if you ever do hit that.
Quoted 4 lines
I seem to keep seeing code that bumps into
surprising limits like this and I'm not sure how I'm supposed to know about them except through finding out after the fact or trawling the code every time someone touches locking.
Not knowing what other limits you've hit, I'm not entirely sure how to help out there.
Quoted 9 lines
I would be very surprised to see a system that pushes over 8 locks, while there's nothing actually preventing it system design considerations mean that even four cascaded supplies are pretty unlikely so we should be fine. Every time you add a new level of regulation you're both increasing the load on regulators up the chain (which means they need to be bigger and more expensive) and except in the case of a DCDC supplying an LDO (which only works to one level) you're going to be decreasing the efficiency of the system. If we get to that point we can worry about what to do.
OK I suppose.
Quoted 11 lines · Peter Zijlstra
Quoted 8 lines · Mark Brown
Quoted 3 lines · Peter Zijlstra
Uuh, I just looked at next and saw this
regulator_lock_supply()function. How is that limited? subclass must be <8 otherwise bad things happen.Can we please get some more discoverable documentation of the arbitrary limits in the lockdep code?
include/linux/lockdep.h:#define
MAX_LOCKDEP_SUBCLASSES8UL
Sure, but I don't really expect to have to trawl the implementation of an API to find out about this sort of thing (I hadn't even been aware that the subclasses were required to be small positive integers, never mind needing to check what the limit was). I think the main place I'd have expected to see it was in lockdep-design.txt or somewhere near that.
Quoted 7 lines
Quoted 4 lines
I seem to keep seeing code that bumps into
surprising limits like this and I'm not sure how I'm supposed to know about them except through finding out after the fact or trawling the code every time someone touches locking.Not knowing what other limits you've hit, I'm not entirely sure how to help out there.
The other big one that came up recently was that lockdep apparently
works out what a class is by looking at the point of allocation which
causes a lot of problems for regmap since it makes all regmap locks look
like a single class. That's fixed now by explicitly allocating a class
per regmap with some macro magic but it was a bit of a surprise. The
documentation doesn't make this obvious.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20151014/74474f5e/attachment.sig>
Peter Zijlstra
maintainerQuoted 2 lines · Mark Brown
Sure, but I don't really expect to have to trawl the implementation of an API to find out about this sort of thing (I hadn't even been aware
I more like view header files as 'documentation' and c files as implementation :-)
Quoted 4 lines
that the subclasses were required to be small positive integers, never mind needing to check what the limit was). I think the main place I'd have expected to see it was in lockdep-design.txt or somewhere near that.
Oh, you're one of them people that actually looks in Documentation/.
Quoted 6 lines
The other big one that came up recently was that lockdep apparently works out what a class is by looking at the point of allocation which causes a lot of problems for regmap since it makes all regmap locks look like a single class. That's fixed now by explicitly allocating a class per regmap with some macro magic but it was a bit of a surprise. The documentation doesn't make this obvious.
Yes, Documentation/locking/lockdep-design.txt needs help -- I'd even forgotten we had it.
Does the below work for you?
--- Documentation/locking/lockdep-design.txt | 8 ++++++++ 1 file changed, 8 insertions(+)
Documentation/locking/lockdep-design.txt +8 -0
@@ -24,6 +24,10 @@ a lock-class is used for the first time after bootup it gets registered, and all subsequent uses of that lock-class will be attached to this lock-class. +A class is typically associated with a lock's initialisation site; although +its possible to explicitly initialize a lock with a different class key -- +such class keys much come from static storage. + State -----
@@ -165,6 +169,10 @@ partition. The validator treats a lock that is taken in such a nested fashion as a separate (sub)class for the purposes of validation. +Since lock classes are associated with static addresses, the size of struct +lock_class_key determines the amount of sub classes that are possible -- +currently set to 8. + Note: When changing code to use the _nested() primitives, be careful and check really thoroughly that the hierarchy is correctly mapped; otherwise you can get false positives or false negatives.
Peter Zijlstra
maintainerQuoted 9 lines · Peter Zijlstra
--- Documentation/locking/lockdep-design.txt | 8 ++++++++ 1 file changed, 8 insertions(+)Documentation/locking/lockdep-design.txt +0
-0
Documentation/locking/lockdep-design.txt a lock-class is used for the first time after bootup it gets registered, · line 29
Quoted 6 lines
Documentation/locking/lockdep-design.txt +3
-0@@ -24,6 +24,10 @@ a lock-class is used for the first time after bootup it gets registered, and all subsequent uses of that lock-class will be attached to this lock-class. +A class is typically associated with a lock's initialisation site; although +its possible to explicitly initialize a lock with a different class key -- +such class keys much come from static storage.
Now if only I could actually type: s/much/must/
Quoted 13 lines · Peter Zijlstra
Quoted 12 lines · Peter Zijlstra
--- Documentation/locking/lockdep-design.txt | 8 ++++++++ 1 file changed, 8 insertions(+)Documentation/locking/lockdep-design.txt +0
-0@@ -24,6 +24,10 @@ a lock-class is used for the first time after bootup it gets registered, and all subsequent uses of that lock-class will be attached to this lock-class.
Quoted 5 lines
Documentation/locking/lockdep-design.txt a lock-class is used for the first time after bootup it gets registered, · line 29
Quoted 3 lines
Documentation/locking/lockdep-design.txt +3
-0+A class is typically associated with a lock's initialisation site; although +its possible to explicitly initialize a lock with a different class key -- +such class keys much come from static storage.Now if only I could actually type: s/much/must/
also:
s/its/it's
:-)
Thanks,
Ingo
Mark Brown
Reviewed-byQuoted 3 lines · Peter Zijlstra
On Wed, Oct 14, 2015 at 01:36:24PM +0100, Mark Brown wrote:
Does the below work for you?
Yes, that's helpful thanks!
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20151014/f9e33bbc/attachment.sig>
Quoted 9 lines · On Tuesday 13 October 2015 23:46:35 Arnd Bergmann wrote:
Quoted 3 lines · On Tuesday 13 October 2015 22:38:12 Peter Zijlstra wrote:
A quick grep shows a large amount of trivial code that optimizers will still happily throw away, but it should be verified that this does not result in pointless code generation.
Indeed, I'm seeing a tiny code growth with ARM
multi_v7_defconfigwhen my patch is applied, as the image (according to size -A) grows from 13740187 bytes to 13740283, all of it in .text of two drivers (i2c-core and three files of bluetooth.ko).
Did we actually reach any conclusion here? We still get the warnings in the regulator code in linux-next, and I'd like to see either this patch ("mutex: make mutex_lock_nested an inline function") or "regulator: core: avoid unused variable warning" get merged.
We could also remove the two functions again, as they are still unused and we are getting closer to the merge window.
Arnd
Quoted 53 lines
multi_v7_defconfig/vmlinux.o.size renamed from multi_v7_defconfig-before/vmlinux.o.size +1
-1@@ -1,6 +1,6 @@ build/multi_v7_defconfig/vmlinux.o : section size addr -.text 8219408 0 +.text 8219504 0multi_v7_defconfig/net/bluetooth/bluetooth.ko.size renamed from multi_v7_defconfig-before/net/bluetooth/bluetooth.ko.size +1
-1@@ -1,7 +1,7 @@ build/multi_v7_defconfig/net/bluetooth/bluetooth.ko : section size addr .note.gnu.build-id 36 0 -.text 241512 0 +.text 241696 0multi_v7_defconfig/drivers/i2c/i2c-core.o.size renamed from multi_v7_defconfig-before/drivers/i2c/i2c-core.o.size +1
-1@@ -1,6 +1,6 @@ build/multi_v7_defconfig/drivers/i2c/i2c-core.o : section size addr -.text 12112 0 +.text 12208 0The code in question is
a)
static
ssize_t
i2c_sysfs_delete_device(struct device *dev, structdevice_attribute*attr,
const char *buf,size_tcount)
{
...mutex_lock_nested(&adap->userspace_clients_lock, i2c_adapter_depth(adap));...
}and
b)
static inline void
l2cap_chan_lock(struct l2cap_chan *chan){ mutex_lock_nested(&chan->lock, atomic_read(&chan->nesting)); }The first one has a small size impact but no performance change as it is only called during probe/release of i2c modules. The second one adds an extra pointer access (due to the volatile keyword in
atomic_read()) for every caller ofl2cap_chan_lock().
Peter Zijlstra
maintainerQuoted 12 lines · Arnd Bergmann
Quoted 9 lines · On Tuesday 13 October 2015 23:46:35 Arnd Bergmann wrote:
Quoted 3 lines · On Tuesday 13 October 2015 22:38:12 Peter Zijlstra wrote:
A quick grep shows a large amount of trivial code that optimizers will still happily throw away, but it should be verified that this does not result in pointless code generation.
Indeed, I'm seeing a tiny code growth with ARM
multi_v7_defconfigwhen my patch is applied, as the image (according to size -A) grows from 13740187 bytes to 13740283, all of it in .text of two drivers (i2c-core and three files of bluetooth.ko).
Quoted 7 lines
Did we actually reach any conclusion here? We still get the warnings in the regulator code in linux-next, and I'd like to see either this patch ("mutex: make
mutex_lock_nestedan inline function") or "regulator: core: avoid unused variable warning" get merged.We could also remove the two functions again, as they are still unused and we are getting closer to the merge window.
Hmm, I was sure I send a reply, but I cannot even find it in my own sent folder so who knows.
My current preference is to keep the thing a macro and work around it in the usage site because while these warns are annoying, they're at least visible. Whereas, with an inline, code bloat is entirely silent. Even if the sites you found are harmless, there's no saying what the future will bring etc..
Quoted 8 lines · Peter Zijlstra
Hmm, I was sure I send a reply, but I cannot even find it in my own sent folder so who knows.
My current preference is to keep the thing a macro and work around it in the usage site because while these warns are annoying, they're at least visible. Whereas, with an inline, code bloat is entirely silent. Even if the sites you found are harmless, there's no saying what the future will bring etc..
I agree - we've got way too many inline functions already. My biggest annoyance in that respect is the asm-generic dma_map_single() implementation that we're now forced to use on ARM, which results in quite a large chunk of code at every callsite.
The problem there is that when you have drivers which do something like:
dma = dma_map_single(dev, page_address(page), size, dir);
you end up with code which converts the struct page to a virtual address, and then you end up with code in the dma_map_single() inline function which then converts it back to a struct page + offset - none of which, with modern ARM kernels, the compiler has a hope in hell of optimising.
So we end up with all that junk at every single dma_map_single() callsite. If dma_map_single() were a library function, it would be a lot smaller since we'd only have one copy of the complex virt->struct page conversion.
Signature · 2 lines
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up according to speedtest.net.
Quoted 19 lines · Russell King - ARM Linux
Quoted 8 lines · Peter Zijlstra
Hmm, I was sure I send a reply, but I cannot even find it in my own sent folder so who knows.
My current preference is to keep the thing a macro and work around it in the usage site because while these warns are annoying, they're at least visible. Whereas, with an inline, code bloat is entirely silent. Even if the sites you found are harmless, there's no saying what the future will bring etc..
I agree - we've got way too many inline functions already. My biggest annoyance in that respect is the asm-generic
dma_map_single()implementation that we're now forced to use on ARM, which results in quite a large chunk of code at every callsite.The problem there is that when you have drivers which do something like:
dma = dma_map_single(dev, page_address(page), size, dir);
Quoted 8 lines
you end up with code which converts the struct page to a virtual address, and then you end up with code in the
dma_map_single()inline function which then converts it back to a struct page + offset - none of which, with modern ARM kernels, the compiler has a hope in hell of optimising.So we end up with all that junk at every single
dma_map_single()callsite. Ifdma_map_single()were a library function, it would be a lot smaller since we'd only have one copy of the complexvirt->structpage conversion.
Should be pretty easy to fix, once you know which inline functions hurt.
Thanks,
Ingo