Skip to contentSkip to conversation

Increasing usage of mutex guards (with SmPL)?

cocci Markus Elfring 4 participants 17 messages May 26, 2024 – Jun 8, 2024 Kernel janitors

lore
Share & tools
lore
#0 lore raw

Hello,

I constructed another small script variant for the semantic patch language.

@replacement disable decl_init@
expression action, e, value;
expression list el;
identifier rc;
@@
-int rc;
... when != rc = value
when any
-mutex_lock
+guard(mutex)

           (&e);

-rc = action(el);
-mutex_unlock(&e);
return
- rc
+ action(el)
;

I tried this SmPL script example out on source code from the software “Linux next-20240523”.
425 source files were found where some function implementations could be adjusted also according to the application of mutual exclusion.

Will it become helpful to offer further SmPL script variants which can be used together with the tool “coccicheck”?

Regards,
Markus

#1 lore raw
Quoted 24 lines · Markus Elfring

Hello,

I constructed another small script variant for the semantic patch language.

@replacement disable decl_init@
expression action, e, value;
expression list el;
identifier rc;
@@

(quoted diff) +2 -5
-int rc;
 ... when != rc = value
     when any
-mutex_lock
+guard(mutex)
           (&e);
-rc = action(el);
-mutex_unlock(&e);
 return
-       rc
+       action(el)
 ;
Quoted 8 lines

I tried this SmPL script example out on source code from the software “Linux next-20240523”.
425 source files were found where some function implementations could be adjusted also according to the application of mutual exclusion.

Will it become helpful to offer further SmPL script variants which can be used together with the tool “coccicheck”?

I'm not the one to decide whether we should use cleanup for mutex locks. Intuitively, my opinion would be that we should not. Something that can have so much impact on the execution of the system should be visible. But that's just my opinion.

julia

#2 lore raw
Quoted 6 lines
Quoted 2 lines

Will it become helpful to offer further SmPL script variants which can be used together with the tool “coccicheck”?

I'm not the one to decide whether we should use cleanup for mutex locks. Intuitively, my opinion would be that we should not. Something that can have so much impact on the execution of the system should be visible.

You are trying to influence software evolution also according to scope-based resource management for a while, don't you?
https://elixir.bootlin.com/linux/v6.9.2/source/include/linux/cleanup.h#L124

Regards,
Markus

#3 lore raw
Quoted 10 lines · Markus Elfring
Quoted 6 lines
Quoted 2 lines

Will it become helpful to offer further SmPL script variants which can be used together with the tool “coccicheck”?

I'm not the one to decide whether we should use cleanup for mutex locks. Intuitively, my opinion would be that we should not. Something that can have so much impact on the execution of the system should be visible.

You are trying to influence software evolution also according to scope-based resource management for a while, don't you?
https://elixir.bootlin.com/linux/v6.9.2/source/include/linux/cleanup.h#L124

I have only helped to automate what others have already shown interest in.

julia

#4 lore raw
In reply to Julia Lawall
Quoted 4 lines · Julia Lawall

I'm not the one to decide whether we should use cleanup for mutex locks. Intuitively, my opinion would be that we should not. Something that can have so much impact on the execution of the system should be visible. But that's just my opinion.

Different subsystems are taking different approaches to the cleanup.h code. Sound adopted it. Networking is taking it slow. Some GPIO cleanup.h code was merged and some wasn't.

It really does change how you review code, so mass conversion patches probably aren't welcome.

regards,
dan carpenter

#5 lore raw
Quoted 1 line

Different subsystems are taking different approaches to the cleanup.h code.

…

Quoted 2 lines

It really does change how you review code, so mass conversion patches probably aren't welcome.

* Can any contributors (with a better reputation) achieve corresponding tree-wide source code transformations?

* Would any software design “advertisements” help for adjustment chances?

* Will the support grow for the usage of SmPL script variants especially in combination with the tool “coccicheck”?

Regards,
Markus

#6 lore raw

Re: [cocci] Looking at guard usage (with SmPL)

In reply to Julia Lawall
Quoted 1 line

I'm not the one to decide whether we should use cleanup for mutex locks.

Would you get further development ideas from another bit of source code analysis which can be performed also with the help of the following small script variant for the semantic patch language?

@initialize:python@
@@
import sys
delimiter = "|"

def format_data(places, item, input):
for place in places:

      sys.stdout.write(delimiter.join([item,
                                       str(input),
                                       place.current_element,
                                       place.file,
                                       place.line,
                                       str(int(place.column) + 1)
                                      ]))
      sys.stdout.write("\n")

@find@
expression list el;
identifier item;
position p;
@@
guard@p (item) (el);

@script:python output@
input << find.el;
item << find.item;
places << find.p;
@@
format_data(places, item, input)

Thus it seems that special “guards” are used at 678 source code places of the software “Linux next-20240529”.
https://elixir.bootlin.com/linux/v6.10-rc1/source/include/linux/cleanup.h#L124

Will interests accordingly grow for further collateral evolution?

Regards,
Markus

#7 lore raw

Re: [cocci] Looking at guard usage (with SmPL)

Quoted 5 lines · * Markus Elfring ([email protected]) wrote:
Quoted 1 line

I'm not the one to decide whether we should use cleanup for mutex locks.

Would you get further development ideas from another bit of source code analysis which can be performed also with the help of the following small script variant for the semantic patch language?

Is there a way to look for functions that have a lock, followed by unlocks on multiple error paths?
IMHO the nicest thing with guard is simplifying lots of error paths, and innevitably someone forgets to unlock in one of them.

Dave

Quoted 42 lines

@initialize:python@
@@
import sys
delimiter = "|"

def format_data(places, item, input):

   for place in places:
      sys.stdout.write(delimiter.join([item,
                                       str(input),
                                       place.current_element,
                                       place.file,
                                       place.line,
                                       str(int(place.column) + 1)
                                      ]))
      sys.stdout.write("\n")

@find@
expression list el;
identifier item;
position p;
@@
guard@p (item) (el);

@script:python output@
input << find.el;
item << find.item;
places << find.p;
@@
format_data(places, item, input)

Thus it seems that special “guards” are used at 678 source code places of the software “Linux next-20240529”.
https://elixir.bootlin.com/linux/v6.10-rc1/source/include/linux/cleanup.h#L124

Will interests accordingly grow for further collateral evolution?

Regards,
Markus

Signature · 4 lines
 -----Open up your eyes, open up your mind, open up your code -------   
/ Dr. David Alan Gilbert    |       Running GNU/Linux       | Happy  \ 
\        dave @ treblig.org |                               | In Hex /
 \ _________________________|_____ http://www.treblig.org   |_______/
#8 lore raw

Re: [cocci] Looking at guard usage (with SmPL)

Quoted 2 lines

Is there a way to look for functions that have a lock, followed by unlocks on multiple error paths?

Probably, yes.
https://cwe.mitre.org/data/definitions/667.html

Quoted 2 lines

IMHO the nicest thing with guard is simplifying lots of error paths, and innevitably someone forgets to unlock in one of them.

Would you like to improve the support anyhow for more complete and succinct resource management in affected software areas?

Regards,
Markus

#9 lore raw

Re: [cocci] Looking at guard usage (with SmPL)

Quoted 7 lines · * Markus Elfring ([email protected]) wrote:
Quoted 2 lines

Is there a way to look for functions that have a lock, followed by unlocks on multiple error paths?

Probably, yes.
https://cwe.mitre.org/data/definitions/667.html

Quoted 5 lines
Quoted 2 lines

IMHO the nicest thing with guard is simplifying lots of error paths, and innevitably someone forgets to unlock in one of them.

Would you like to improve the support anyhow for more complete and succinct resource management in affected software areas?

I was just suggesting a perhaps more targeted approach rather than just looking for all cases of lock/unlock.

Dave

Quoted 3 lines

Regards,
Markus

Signature · 4 lines
 -----Open up your eyes, open up your mind, open up your code -------   
/ Dr. David Alan Gilbert    |       Running GNU/Linux       | Happy  \ 
\        dave @ treblig.org |                               | In Hex /
 \ _________________________|_____ http://www.treblig.org   |_______/
#10 lore raw

Re: [cocci] Looking at lock scopes (with SmPL)?

Quoted 7 lines
Quoted 7 lines
Quoted 2 lines

Is there a way to look for functions that have a lock, followed by unlocks on multiple error paths?

Probably, yes.
https://cwe.mitre.org/data/definitions/667.html

Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

IMHO the nicest thing with guard is simplifying lots of error paths, and innevitably someone forgets to unlock in one of them.

Would you like to improve the support anyhow for more complete and succinct resource management in affected software areas?

I was just suggesting a perhaps more targeted approach rather than just looking for all cases of lock/unlock.

How would you like to handle source code analyses if lock scopes would occasionally be distributed over different function implementations?

Will development interests grow also for checking combinations according to other pairs of actions?

Regards,
Markus

#11 lore raw

Re: [cocci] Looking at lock scopes (with SmPL)?

Quoted 10 lines · * Markus Elfring ([email protected]) wrote:
Quoted 10 lines
Quoted 10 lines
Quoted 2 lines

Is there a way to look for functions that have a lock, followed by unlocks on multiple error paths?

Probably, yes.
https://cwe.mitre.org/data/definitions/667.html

Quoted 2 lines

IMHO the nicest thing with guard is simplifying lots of error paths, and innevitably someone forgets to unlock in one of them.

Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

Would you like to improve the support anyhow for more complete and succinct resource management in affected software areas?

I was just suggesting a perhaps more targeted approach rather than just looking for all cases of lock/unlock.

How would you like to handle source code analyses if lock scopes would occasionally be distributed over different function implementations?

It depends on the aim:
a) If you want to find some cases to replace with guard, then I'd only do the simple case where there is nothing distributed across functions.

b) If you're trying to prove/detect failures then that needs to look at the hard cases I guess; but I think there are other tools doing that??

Quoted 2 lines

Will development interests grow also for checking combinations according to other pairs of actions?

This is only a suggestion from me; I wouldn't drive too much off it.

Dave

Quoted 2 lines

Regards,
Markus

Signature · 4 lines
 -----Open up your eyes, open up your mind, open up your code -------   
/ Dr. David Alan Gilbert    |       Running GNU/Linux       | Happy  \ 
\        dave @ treblig.org |                               | In Hex /
 \ _________________________|_____ http://www.treblig.org   |_______/
#12 lore raw

Re: [cocci] Increasing usage of spinlock guards (with SmPL)?

In reply to Julia Lawall
Quoted 1 line

I'm not the one to decide whether we should use cleanup for mutex locks.

Would you become interested to support the conversion of another tiny script (like the following) for the semantic patch language into a corresponding SmPL script variant which can be used together with the tool “coccicheck” any more?

@replacement@
expression e;
@@
-raw_spin_lock
+guard(raw_spinlock_irq)

              (&e);

... when != e
when any
-raw_spin_unlock(&e);

Regards,
Markus

#13 lore raw

Re: [cocci] Increasing usage of spinlock guards (with SmPL)?

Quoted 10 lines · Markus Elfring
Quoted 1 line

I'm not the one to decide whether we should use cleanup for mutex locks.

Would you become interested to support the conversion of another tiny script (like the following) for the semantic patch language into a corresponding SmPL script variant which can be used together with the tool “coccicheck” any more?

@replacement@
expression e;
@@

Review

Quoted 6 lines
(quoted diff) +1 -2
-raw_spin_lock
+guard(raw_spinlock_irq)
              (&e);
 ... when != e
     when any
-raw_spin_unlock(&e);

What exactly is the problem? I tried this rule and it worked fine. There were a few issues of inconsistent control-flow paths, but it made 325 changes.

julia

#14 lore raw

Re: [cocci] Increasing usage of spinlock guards (with SmPL)?

Quoted 5 lines
Quoted 5 lines

Would you become interested to support the conversion of another tiny script (like the following) for the semantic patch language into a corresponding SmPL script variant which can be used together with the tool “coccicheck” any more?

Quoted 11 lines

Review

Quoted 9 lines

@replacement@
expression e;
@@

(quoted diff) +1 -2
-raw_spin_lock
+guard(raw_spinlock_irq)
              (&e);
 ... when != e
     when any
-raw_spin_unlock(&e);

What exactly is the problem? I tried this rule and it worked fine.

* Such a simple SmPL script is working to some degree under the constraints of some known software limitations.
Thus there are challenges to consider for the rating of the corresponding application confidence.

* I would also appreciate further positive indications for the integration of coccicheck SmPL script variants.

Regards,
Markus

#15 lore raw

Re: [cocci] Increasing usage of spinlock guards (with SmPL)?

In reply to Julia Lawall
Quoted 11 lines

Review

Quoted 9 lines

@replacement@
expression e;
@@

(quoted diff) +1 -2
-raw_spin_lock
+guard(raw_spinlock_irq)
              (&e);
 ... when != e
     when any
-raw_spin_unlock(&e);

What exactly is the problem?

There are several development challenges remaining to make the application of similar SmPL script variants really safe.
https://elixir.bootlin.com/linux/v6.10-rc2/source/include/linux/spinlock.h#L537

The discussed guards combine two actions.
The shown transformation pattern depends on the deletion of the first action identifier. This identifier should also be usable to determine a corresponding guard parameter and the companion action.

Action pairs can eventually be stored in special data structures so that a mapping can be automatically performed (on demand).

See also:
Working with variables for case match identification by SmPL disjunctions 2019-02-19
https://github.com/coccinelle/coccinelle/issues/159

Regards,
Markus

#16 lore raw

Re: [cocci] Determination of lock guard data (with SmPL)?

In reply to Julia Lawall
Quoted 1 line

What exactly is the problem?

Another development concern (for example):
I would like to benefit a bit more from information which is provided as parameters for macro calls.
https://elixir.bootlin.com/linux/v6.10-rc2/source/include/linux/cleanup.h#L179

Thus I hope that something can be achieved by “advanced” source code analyses also with help of the semantic patch language.
May I expect that some data should be presented according to the following tiny SmPL script?

@display@
@@
*DEFINE_LOCK_GUARD_1(...)

Test result:
Markus_Elfring@Sonne:…/Projekte/Linux/next-analyses/include/linux> spatch --include-headers …/Projekte/Coccinelle/janitor/show_lock_guard_definitions.cocci spinlock.h …

No data are displayed by the software combination “Coccinelle 1.2-00008-g75525032” (because of the specified SmPL asterisk) according to the selected macro call.

See also another known information source for further development considerations:
https://elixir.bootlin.com/linux/v6.10-rc2/source/include/linux/spinlock.h#L537

Regards,
Markus