Increasing usage of mutex guards (with SmPL)?
Hello,
I constructed another small script variant for the semantic patch language.
@replacement disable decl_init@
expression action, e, value;
expression list el;
identifier rc;
@@
-int rc;
... when != rc = value
when any
-mutex_lock
+guard(mutex)
(&e);
-rc = action(el);
-mutex_unlock(&e);
return
- rc
+ action(el)
;
I tried this SmPL script example out on source code from the software “Linux next-20240523”.
425 source files were found where some function implementations could be adjusted also according to the application of mutual exclusion.
Will it become helpful to offer further SmPL script variants which can be used together with the tool “coccicheck”?
Regards,
Markus
Quoted 24 lines · Markus Elfring
Hello,
I constructed another small script variant for the semantic patch language.
@replacement disable decl_init@
expression action, e, value;
expression list el;
identifier rc;
@@(quoted diff) +2
-5-int rc; ... when != rc = value when any -mutex_lock +guard(mutex) (&e); -rc = action(el); -mutex_unlock(&e); return - rc + action(el) ;
Quoted 8 lines
I tried this SmPL script example out on source code from the software “Linux next-20240523”.
425 source files were found where some function implementations could be adjusted also according to the application of mutual exclusion.Will it become helpful to offer further SmPL script variants which can be used together with the tool “coccicheck”?
I'm not the one to decide whether we should use cleanup for mutex locks. Intuitively, my opinion would be that we should not. Something that can have so much impact on the execution of the system should be visible. But that's just my opinion.
julia
Quoted 6 lines
Quoted 2 lines
Will it become helpful to offer further SmPL script variants which can be used together with the tool “coccicheck”?
I'm not the one to decide whether we should use cleanup for mutex locks. Intuitively, my opinion would be that we should not. Something that can have so much impact on the execution of the system should be visible.
You are trying to influence software evolution also according to scope-based resource management for a while, don't you?
https://elixir.bootlin.com/linux/v6.9.2/source/include/linux/cleanup.h#L124
Regards,
Markus
Quoted 10 lines · Markus Elfring
Quoted 6 lines
Quoted 2 lines
Will it become helpful to offer further SmPL script variants which can be used together with the tool “coccicheck”?
I'm not the one to decide whether we should use cleanup for mutex locks. Intuitively, my opinion would be that we should not. Something that can have so much impact on the execution of the system should be visible.
You are trying to influence software evolution also according to scope-based resource management for a while, don't you?
https://elixir.bootlin.com/linux/v6.9.2/source/include/linux/cleanup.h#L124
I have only helped to automate what others have already shown interest in.
julia
Quoted 4 lines · Julia Lawall
I'm not the one to decide whether we should use cleanup for mutex locks. Intuitively, my opinion would be that we should not. Something that can have so much impact on the execution of the system should be visible. But that's just my opinion.
Different subsystems are taking different approaches to the cleanup.h code. Sound adopted it. Networking is taking it slow. Some GPIO cleanup.h code was merged and some wasn't.
It really does change how you review code, so mass conversion patches probably aren't welcome.
regards,
dan carpenter
Quoted 1 line
Different subsystems are taking different approaches to the
cleanup.hcode.
…
Quoted 2 lines
It really does change how you review code, so mass conversion patches probably aren't welcome.
* Can any contributors (with a better reputation) achieve corresponding tree-wide source code transformations?
* Would any software design “advertisements” help for adjustment chances?
* Will the support grow for the usage of SmPL script variants especially in combination with the tool “coccicheck”?
Regards,
Markus
Re: [cocci] Looking at guard usage (with SmPL)
In reply to Julia LawallQuoted 1 line
I'm not the one to decide whether we should use cleanup for mutex locks.
Would you get further development ideas from another bit of source code analysis which can be performed also with the help of the following small script variant for the semantic patch language?
@initialize:python@
@@
import sys
delimiter = "|"
def format_data(places, item, input):
for place in places:
sys.stdout.write(delimiter.join([item,
str(input),
place.current_element,
place.file,
place.line,
str(int(place.column) + 1)
]))
sys.stdout.write("\n")@find@
expression list el;
identifier item;
position p;
@@
guard@p (item) (el);
@script:python output@
input << find.el;
item << find.item;
places << find.p;
@@format_data(places, item, input)
Thus it seems that special “guards” are used at 678 source code places of the software “Linux next-20240529”.
https://elixir.bootlin.com/linux/v6.10-rc1/source/include/linux/cleanup.h#L124
Will interests accordingly grow for further collateral evolution?
Regards,
Markus
Re: [cocci] Looking at guard usage (with SmPL)
Quoted 5 lines · * Markus Elfring ([email protected]) wrote:
Quoted 1 line
I'm not the one to decide whether we should use cleanup for mutex locks.
Would you get further development ideas from another bit of source code analysis which can be performed also with the help of the following small script variant for the semantic patch language?
Is there a way to look for functions that have a lock, followed by unlocks on multiple error paths?
IMHO the nicest thing with guard is simplifying lots of error paths, and innevitably someone forgets to unlock in one of them.
Dave
Quoted 42 lines
@initialize:python@
@@
import sys
delimiter = "|"def
format_data(places, item, input):for place in places: sys.stdout.write(delimiter.join([item, str(input), place.current_element, place.file, place.line, str(int(place.column) + 1) ])) sys.stdout.write("\n")@find@
expression list el;
identifier item;
position p;
@@
guard@p (item) (el);@script:python output@
input << find.el;
item << find.item;
places << find.p;
@@format_data(places, item, input)Thus it seems that special “guards” are used at 678 source code places of the software “Linux next-20240529”.
https://elixir.bootlin.com/linux/v6.10-rc1/source/include/linux/cleanup.h#L124Will interests accordingly grow for further collateral evolution?
Regards,
Markus
Signature · 4 lines
-----Open up your eyes, open up your mind, open up your code ------- / Dr. David Alan Gilbert | Running GNU/Linux | Happy \ \ dave @ treblig.org | | In Hex / \ _________________________|_____ http://www.treblig.org |_______/
Re: [cocci] Looking at guard usage (with SmPL)
Quoted 2 lines
Is there a way to look for functions that have a lock, followed by unlocks on multiple error paths?
Probably, yes.
https://cwe.mitre.org/data/definitions/667.html
Quoted 2 lines
IMHO the nicest thing with guard is simplifying lots of error paths, and innevitably someone forgets to unlock in one of them.
Would you like to improve the support anyhow for more complete and succinct resource management in affected software areas?
Regards,
Markus
Re: [cocci] Looking at guard usage (with SmPL)
Quoted 7 lines · * Markus Elfring ([email protected]) wrote:
Quoted 2 lines
Is there a way to look for functions that have a lock, followed by unlocks on multiple error paths?
Probably, yes.
https://cwe.mitre.org/data/definitions/667.html
Quoted 5 lines
Quoted 2 lines
IMHO the nicest thing with guard is simplifying lots of error paths, and innevitably someone forgets to unlock in one of them.
Would you like to improve the support anyhow for more complete and succinct resource management in affected software areas?
I was just suggesting a perhaps more targeted approach rather than just looking for all cases of lock/unlock.
Dave
Quoted 3 lines
Regards,
Markus
Signature · 4 lines
-----Open up your eyes, open up your mind, open up your code ------- / Dr. David Alan Gilbert | Running GNU/Linux | Happy \ \ dave @ treblig.org | | In Hex / \ _________________________|_____ http://www.treblig.org |_______/
Re: [cocci] Looking at lock scopes (with SmPL)?
Quoted 7 lines
Quoted 7 lines
Quoted 2 lines
Is there a way to look for functions that have a lock, followed by unlocks on multiple error paths?
Probably, yes.
https://cwe.mitre.org/data/definitions/667.html
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
IMHO the nicest thing with guard is simplifying lots of error paths, and innevitably someone forgets to unlock in one of them.
Would you like to improve the support anyhow for more complete and succinct resource management in affected software areas?
I was just suggesting a perhaps more targeted approach rather than just looking for all cases of lock/unlock.
How would you like to handle source code analyses if lock scopes would occasionally be distributed over different function implementations?
Will development interests grow also for checking combinations according to other pairs of actions?
Regards,
Markus
Re: [cocci] Looking at lock scopes (with SmPL)?
Quoted 10 lines · * Markus Elfring ([email protected]) wrote:
Quoted 10 lines
Quoted 10 lines
Quoted 2 lines
Is there a way to look for functions that have a lock, followed by unlocks on multiple error paths?
Probably, yes.
https://cwe.mitre.org/data/definitions/667.htmlQuoted 2 lines
IMHO the nicest thing with guard is simplifying lots of error paths, and innevitably someone forgets to unlock in one of them.
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
Would you like to improve the support anyhow for more complete and succinct resource management in affected software areas?
I was just suggesting a perhaps more targeted approach rather than just looking for all cases of lock/unlock.
How would you like to handle source code analyses if lock scopes would occasionally be distributed over different function implementations?
It depends on the aim:
a) If you want to find some cases to replace with guard, then I'd only do the simple case where there is nothing distributed across functions.
b) If you're trying to prove/detect failures then that needs to look at the hard cases I guess; but I think there are other tools doing that??
Quoted 2 lines
Will development interests grow also for checking combinations according to other pairs of actions?
This is only a suggestion from me; I wouldn't drive too much off it.
Dave
Quoted 2 lines
Regards,
Markus
Signature · 4 lines
-----Open up your eyes, open up your mind, open up your code ------- / Dr. David Alan Gilbert | Running GNU/Linux | Happy \ \ dave @ treblig.org | | In Hex / \ _________________________|_____ http://www.treblig.org |_______/
Re: [cocci] Increasing usage of spinlock guards (with SmPL)?
In reply to Julia LawallQuoted 1 line
I'm not the one to decide whether we should use cleanup for mutex locks.
Would you become interested to support the conversion of another tiny script (like the following) for the semantic patch language into a corresponding SmPL script variant which can be used together with the tool “coccicheck” any more?
@replacement@
expression e;
@@
-raw_spin_lock
+guard(raw_spinlock_irq)
(&e);
... when != e
when any
-raw_spin_unlock(&e);
Regards,
Markus
Re: [cocci] Increasing usage of spinlock guards (with SmPL)?
Quoted 10 lines · Markus Elfring
Quoted 1 line
I'm not the one to decide whether we should use cleanup for mutex locks.
Would you become interested to support the conversion of another tiny script (like the following) for the semantic patch language into a corresponding SmPL script variant which can be used together with the tool “coccicheck” any more?
@replacement@
expression e;
@@
Review
Quoted 6 lines
(quoted diff) +1
-2-raw_spin_lock +guard(raw_spinlock_irq) (&e); ... when != e when any -raw_spin_unlock(&e);
What exactly is the problem? I tried this rule and it worked fine. There were a few issues of inconsistent control-flow paths, but it made 325 changes.
julia
Re: [cocci] Increasing usage of spinlock guards (with SmPL)?
Quoted 5 lines
Quoted 5 lines
Would you become interested to support the conversion of another tiny script (like the following) for the semantic patch language into a corresponding SmPL script variant which can be used together with the tool “coccicheck” any more?
Quoted 11 lines
Review
Quoted 9 lines
@replacement@
expression e;
@@(quoted diff) +1
-2-raw_spin_lock +guard(raw_spinlock_irq) (&e); ... when != e when any -raw_spin_unlock(&e);What exactly is the problem? I tried this rule and it worked fine.
* Such a simple SmPL script is working to some degree under the constraints of some known software limitations.
Thus there are challenges to consider for the rating of the corresponding application confidence.
* I would also appreciate further positive indications for the integration of coccicheck SmPL script variants.
Regards,
Markus
Re: [cocci] Increasing usage of spinlock guards (with SmPL)?
In reply to Julia LawallQuoted 11 lines
Review
Quoted 9 lines
@replacement@
expression e;
@@(quoted diff) +1
-2-raw_spin_lock +guard(raw_spinlock_irq) (&e); ... when != e when any -raw_spin_unlock(&e);What exactly is the problem?
There are several development challenges remaining to make the application of similar SmPL script variants really safe.
https://elixir.bootlin.com/linux/v6.10-rc2/source/include/linux/spinlock.h#L537
The discussed guards combine two actions.
The shown transformation pattern depends on the deletion of the first action identifier. This identifier should also be usable to determine a corresponding guard parameter and the companion action.
Action pairs can eventually be stored in special data structures so that a mapping can be automatically performed (on demand).
See also:
Working with variables for case match identification by SmPL disjunctions 2019-02-19
https://github.com/coccinelle/coccinelle/issues/159
Regards,
Markus
Re: [cocci] Determination of lock guard data (with SmPL)?
In reply to Julia LawallQuoted 1 line
What exactly is the problem?
Another development concern (for example):
I would like to benefit a bit more from information which is provided as parameters for macro calls.
https://elixir.bootlin.com/linux/v6.10-rc2/source/include/linux/cleanup.h#L179
Thus I hope that something can be achieved by “advanced” source code analyses also with help of the semantic patch language.
May I expect that some data should be presented according to the following tiny SmPL script?
@display@
@@*DEFINE_LOCK_GUARD_1(...)
Test result:
Markus_Elfring@Sonne:…/Projekte/Linux/next-analyses/include/linux> spatch --include-headers …/Projekte/Coccinelle/janitor/show_lock_guard_definitions.cocci spinlock.h …
No data are displayed by the software combination “Coccinelle 1.2-00008-g75525032” (because of the specified SmPL asterisk) according to the selected macro call.
See also another known information source for further development considerations:
https://elixir.bootlin.com/linux/v6.10-rc2/source/include/linux/spinlock.h#L537
Regards,
Markus