Skip to contentSkip to conversation

coccinelle: api: add kfree_mismatch script

PATCH Markus Elfring 2 participants 6 messages Oct 16, 2020 – Oct 18, 2020 Kernel janitors

lore
Share & tools
lore
#0 lore raw

…

Review

Quoted 3 lines
(quoted diff) +3 -0
+    E = \(kmalloc\|kzalloc\|krealloc\|kcalloc\|
+          kmalloc_node\|kzalloc_node\|kmalloc_array\|
+          kmalloc_array_node\|kcalloc_node\)(...)@kok

…

How do you think about the possibility for any adjustments according to the order of the mentioned function names in proposed disjunctions for the semantic patch language?

Can any additional identifiers become relevant?

Regards,
Markus

_______________________________________________
Cocci mailing list
[email protected]
https://systeme.lip6.fr/mailman/listinfo/cocci

#1 lore raw
Quoted 8 lines · Markus Elfring

…

Review

Quoted 3 lines
(quoted diff) +3 -0
+    E = \(kmalloc\|kzalloc\|krealloc\|kcalloc\|
+          kmalloc_node\|kzalloc_node\|kmalloc_array\|
+          kmalloc_array_node\|kcalloc_node\)(...)@kok

…

How do you think about the possibility for any adjustments according to the order of the mentioned function names in proposed disjunctions for the semantic patch language?

Please think about this for 5 seconds. Maybe there are 2000 calls to these allocation functions, and maybe there are a million function calls in the files that contain these calls. Microscopically optimizing the treatment of 2000 calls is not going to do anything to help the overall runtime, which depends on matching all of the above function names against the one million overall calls.

Quoted 1 line

Can any additional identifiers become relevant?

If you have other names to suggest, please do. If you don't have other names to suggest, then please stop asking such rhetorical questions.

julia

_______________________________________________
Cocci mailing list
[email protected]
https://systeme.lip6.fr/mailman/listinfo/cocci

#2 lore raw
Quoted 6 lines
Quoted 2 lines

How do you think about the possibility for any adjustments according to the order of the mentioned function names in proposed disjunctions for the semantic patch language?

Please think about this for 5 seconds. Maybe there are 2000 calls to these allocation functions, and maybe there are a million function calls in the files that contain these calls.

Would you become interested to check the usage statistics in more detail?

Quoted 3 lines

Microscopically optimizing the treatment of 2000 calls is not going to do anything to help the overall runtime, which depends on matching all
of the above function names against the one million overall calls.

I got an other software understanding for the evaluation characteristics of discussed SmPL scripts.

Quoted 4 lines
Quoted 1 line

Can any additional identifiers become relevant?

If you have other names to suggest, please do. If you don't have other names to suggest, then please stop asking such rhetorical questions.

I suggest to look at further possibilities so that more function call combinations can be checked automatically.
How do you think about approaches to determine relevant properties
in systematic ways (besides listing involved identifiers explicitly)?

Regards,
Markus

_______________________________________________
Cocci mailing list
[email protected]
https://systeme.lip6.fr/mailman/listinfo/cocci

#3 lore raw

Re: [Cocci] Determination of an usage statistic for memory allocation calls

In reply to Markus Elfring
Quoted 8 lines

…

Review

Quoted 3 lines
(quoted diff) +3 -0
+    E = \(kmalloc\|kzalloc\|krealloc\|kcalloc\|
+          kmalloc_node\|kzalloc_node\|kmalloc_array\|
+          kmalloc_array_node\|kcalloc_node\)(...)@kok

…

How do you think about the possibility for any adjustments according to the order of the mentioned function names in proposed disjunctions for the semantic patch language?

I would like to share another source code analysis approach.
I hope that this contribution can trigger further helpful software development ideas.

@initialize:python@
@@
import sys

def write_identifier(source, call):

    names = []
    for x in source:
       names.append(call)

    sys.stdout.write("\n".join(names) + "\n")

@find1@
expression e;
identifier call, x;
position pos;
type rt;
@@
rt x(...)
{
<+...
e =@pos
(kzalloc@call

|kmalloc@call
|kcalloc@call
|kmalloc_array@call
|kmemdup@call
|kstrdup@call
|vmalloc@call
|vzalloc@call
|kzalloc_node@call
|kvmalloc@call
|krealloc@call
|kmalloc_node@call
|kcalloc_node@call
|__vmalloc@call
|vmalloc_user@call
|vzalloc_node@call
|vmalloc_32@call
|__vmalloc_node_range@call
|vmalloc_node@call
|kmalloc_array_node@call
|__vmalloc_node@call
|vmalloc_32_user@call
|vmalloc_exec@call

)(...)
...+>
}

@script:python collection1@
call << find1.call;
place << find1.pos;
@@
write_identifier(place, call)

@find2@
identifier call, var, x;
position pos;
type rt, vt;
@@
rt x(...)
{
<+...
vt var =@pos
(kzalloc@call

|kmalloc@call
|kcalloc@call
|kmalloc_array@call
|kmemdup@call
|kstrdup@call
|vmalloc@call
|vzalloc@call
|kzalloc_node@call
|kvmalloc@call
|krealloc@call
|kmalloc_node@call
|kcalloc_node@call
|__vmalloc@call
|vmalloc_user@call
|vzalloc_node@call
|vmalloc_32@call
|__vmalloc_node_range@call
|vmalloc_node@call
|kmalloc_array_node@call
|__vmalloc_node@call
|vmalloc_32_user@call
|vmalloc_exec@call

)(...);
...+>
}

@script:python collection2@
call << find2.call;
place << find2.pos;
@@
write_identifier(place, call)

Test result:
elfring@Sonne:~/Projekte/Linux/next-patched> git checkout next-20201016 && XX=$(date) && time spatch --timeout 23 --python python3 --jobs 4 --chunksize 1 --include-headers --no-includes --dir . ~/Projekte/Coccinelle/janitor/report_memory_allocation_calls4.cocci 2> ~/Projekte/Bau/Linux/scripts/Coccinelle/call_checks/20201016/report_memory_allocation_calls4-errors.txt | echo "$(echo 'call' && cat)" | csvsql --query 'select call, count(*) from stdin group by call order by count(*) desc'; YY=$(date) && echo "$XX | $YY" …
call,count(*)
kzalloc,12652
kmalloc,4902
kcalloc,2564
kmalloc_array,859
kmemdup,797
kstrdup,469
vmalloc,405
vzalloc,359
kzalloc_node,177
kvmalloc,154
krealloc,151
kmalloc_node,49
kcalloc_node,44
__vmalloc,34
vmalloc_user,28
vzalloc_node,21
vmalloc_32,9
__vmalloc_node_range,8
vmalloc_node,7
kmalloc_array_node,5
__vmalloc_node,4
vmalloc_32_user,1

real 22m25,049s
user 84m11,257s
sys 0m12,168s
So 18. Okt 16:55:08 CEST 2020 | So 18. Okt 17:17:33 CEST 2020

The log file contains the information “9211 files match”.
Can such facts influence the specification of efficient SmPL disjunctions another bit?

Regards,
Markus

_______________________________________________
Cocci mailing list
[email protected]
https://systeme.lip6.fr/mailman/listinfo/cocci

#4 lore raw

Re: [Cocci] Determination of an usage statistic for memory allocation calls

Quoted 138 lines · Markus Elfring
Quoted 8 lines

…

Review

Quoted 3 lines
(quoted diff) +3 -0
+    E = \(kmalloc\|kzalloc\|krealloc\|kcalloc\|
+          kmalloc_node\|kzalloc_node\|kmalloc_array\|
+          kmalloc_array_node\|kcalloc_node\)(...)@kok

…

How do you think about the possibility for any adjustments according to the order of the mentioned function names in proposed disjunctions for the semantic patch language?

I would like to share another source code analysis approach.
I hope that this contribution can trigger further helpful software development ideas.

@initialize:python@
@@
import sys

def write_identifier(source, call):

    names = []
    for x in source:
       names.append(call)

    sys.stdout.write("\n".join(names) + "\n")

@find1@
expression e;
identifier call, x;
position pos;
type rt;
@@
rt x(...)
{
<+...
e =@pos
(kzalloc@call

|kmalloc@call
|kcalloc@call
|kmalloc_array@call
|kmemdup@call
|kstrdup@call
|vmalloc@call
|vzalloc@call
|kzalloc_node@call
|kvmalloc@call
|krealloc@call
|kmalloc_node@call
|kcalloc_node@call
|__vmalloc@call
|vmalloc_user@call
|vzalloc_node@call
|vmalloc_32@call
|__vmalloc_node_range@call
|vmalloc_node@call
|kmalloc_array_node@call
|__vmalloc_node@call
|vmalloc_32_user@call
|vmalloc_exec@call

)(...)
...+>
}

@script:python collection1@
call << find1.call;
place << find1.pos;
@@
write_identifier(place, call)

@find2@
identifier call, var, x;
position pos;
type rt, vt;
@@
rt x(...)
{
<+...
vt var =@pos
(kzalloc@call

|kmalloc@call
|kcalloc@call
|kmalloc_array@call
|kmemdup@call
|kstrdup@call
|vmalloc@call
|vzalloc@call
|kzalloc_node@call
|kvmalloc@call
|krealloc@call
|kmalloc_node@call
|kcalloc_node@call
|__vmalloc@call
|vmalloc_user@call
|vzalloc_node@call
|vmalloc_32@call
|__vmalloc_node_range@call
|vmalloc_node@call
|kmalloc_array_node@call
|__vmalloc_node@call
|vmalloc_32_user@call
|vmalloc_exec@call

)(...);
...+>
}

@script:python collection2@
call << find2.call;
place << find2.pos;
@@
write_identifier(place, call)

Test result:
elfring@Sonne:~/Projekte/Linux/next-patched> git checkout next-20201016 && XX=$(date) && time spatch --timeout 23 --python python3 --jobs 4 --chunksize 1 --include-headers --no-includes --dir . ~/Projekte/Coccinelle/janitor/report_memory_allocation_calls4.cocci 2> ~/Projekte/Bau/Linux/scripts/Coccinelle/call_checks/20201016/report_memory_allocation_calls4-errors.txt | echo "$(echo 'call' && cat)" | csvsql --query 'select call, count(*) from stdin group by call order by count(*) desc'; YY=$(date) && echo "$XX | $YY" …
call,count(*)
kzalloc,12652
kmalloc,4902
kcalloc,2564
kmalloc_array,859
kmemdup,797
kstrdup,469
vmalloc,405
vzalloc,359
kzalloc_node,177
kvmalloc,154
krealloc,151
kmalloc_node,49
kcalloc_node,44
__vmalloc,34
vmalloc_user,28
vzalloc_node,21
vmalloc_32,9
__vmalloc_node_range,8
vmalloc_node,7
kmalloc_array_node,5
__vmalloc_node,4
vmalloc_32_user,1

Quoted 8 lines

real 22m25,049s
user 84m11,257s
sys 0m12,168s
So 18. Okt 16:55:08 CEST 2020 | So 18. Okt 17:17:33 CEST 2020

The log file contains the information “9211 files match”.
Can such facts influence the specification of efficient SmPL disjunctions another bit?

On my machine, putting the three functions that you have foudn to be the most frequent at the end of each disjunction has no impact on the performance. So what do you suggest?

julia

#5 lore raw

Re: [Cocci] Determination of an usage statistic for memory allocation calls

Quoted 4 lines
Quoted 1 line

Can such facts influence the specification of efficient SmPL disjunctions another bit?

On my machine, putting the three functions that you have foudn to be the most frequent at the end of each disjunction has no impact on the performance.

I propose to reconsider this view.

Quoted 1 line

So what do you suggest?

1. I would appreciate if you would share more technical details about your test environment for a safer comparison.

2. The observed software behaviour can be clarified further, can't it? Aspects like the following can trigger corresponding development considerations.
* “noticable” differences (depending on the run time environment)
* measurable effects
* mathematical properties of an algorithm

Regards,
Markus

_______________________________________________
Cocci mailing list
[email protected]
https://systeme.lip6.fr/mailman/listinfo/cocci