Recursive matches / returning lists from scripting code back to coccinelle
Hello everybody,
first of all thanks a lot for your very nice tool, it has already been proven very useful!
Sorry, if this is covered in the grammar documentation and I have overlooked it, I'm a new user to coccinelle.
Is it possible to define rules in a recursive way? I want to match a function where subsequent function calls call a certain function (in this case "malloc"). Using scripting or a different way would also be fine if I can achieve this.
The patch below the mail is programmed for just two levels, but what I'd really would want is going as deep as possible. If applied to the C code below the patch, the output on my system yields:
init_defs_builtins: /usr/lib/coccinelle/standard.h
HANDLING: src/a.c
ISR function ISR_func1 line 5 file src/a.c calling function other_func->malloc ISR function ISR_func1 line 5 file src/a.c calling function other_func->other_func2->malloc
On another note, is it possible to pass whole lists from python or OCaml script code back to coccinelle and generate several dependent rules on it? I know I can use the value of one variable from python as a metavariable for a following rule (Although the order of matching is not entirely clear to me: When the python code depends on one rule and another rule depends on the metavariable created by this python code, are there multiple passes over the code? Or is it possible to trigger this somehow? I'm lacking still some understanding of the internal mechanism of coccinelle).
Best regards and thanks!
Florian Pesth
Semantic patch:
@findisr@
expression E;
identifier isrfunction != NULL;
@@ISR_SetHandler(E, isrfunction)
@mallocfunc@
identifier malloc_func;
@@
malloc_func(...){
...
malloc(...);
...
}
@findmalloc extends findisr@
position p;
identifier mallocfunc.malloc_func;
@@
isrfunction(...){@p
...malloc_func(...);
...
}
@script:python@
isrfunction << findisr.isrfunction;
p << findmalloc.p;malloc_func << mallocfunc.malloc_func;
@@
print "ISR function %s line %s file %s calling function %s->malloc" % (isrfunction, p[0].line, p[0].file, malloc_func)
@cmallocfunc@
identifier mallocfunc.malloc_func;
identifier cmalloc_func;
@@
cmalloc_func(...){
...malloc_func(...);
...
}
@findcmalloc extends findisr@
position p;
identifier cmallocfunc.cmalloc_func;
@@
isrfunction(...){@p
...cmalloc_func(...);
...
}
@script:python@
isrfunction << findisr.isrfunction;
p << findcmalloc.p;cmalloc_func << cmallocfunc.cmalloc_func;malloc_func << mallocfunc.malloc_func;
@@
print "ISR function %s line %s file %s calling function %s->%s->malloc" % (isrfunction, p[0].line, p[0].file, cmalloc_func, malloc_func)
Example C File:
void other_func(int something);
void other_func2(int something);
void ISR_func1(int something)
{
other_func(something);
}void Init_ISR(void)
{
int hello;
ISR_SetHandler(hello, ISR_func1);
}void other_func(int something)
{
malloc(sizeof(double)*something);
other_func2(something);
}void other_func2(int something)
{
malloc(sizeof(double)*something);
}Quoted 7 lines · Pesth Florian
Hello everybody,
first of all thanks a lot for your very nice tool, it has already been proven very useful!
Sorry, if this is covered in the grammar documentation and I have overlooked it, I'm a new user to coccinelle.
Is it possible to define rules in a recursive way? I want to match a function where subsequent function calls call a certain function (in this case "malloc"). Using scripting or a different way would also be fine if I can achieve this.
Look at demos/iteration.cocci. This is discussed in "Advanced SmPL" at
http://coccinelle.lip6.fr/papers.php.
Quoted 8 lines
The patch below the mail is programmed for just two levels, but what I'd really would want is going as deep as possible. If applied to the C code below the patch, the output on my system yields:
init_defs_builtins:/usr/lib/coccinelle/standard.h
HANDLING:src/a.c
ISR functionISR_func1line 5 filesrc/a.ccalling functionother_func->mallocISR functionISR_func1line 5 filesrc/a.ccalling functionother_func->other_func2->malloc
Quoted 9 lines
On another note, is it possible to pass whole lists from python or OCaml script code back to coccinelle and generate several dependent rules on it? I know I can use the value of one variable from python as a metavariable for a following rule (Although the order of matching is not entirely clear to me: When the python code depends on one rule and another rule depends on the metavariable created by this python code, are there multiple passes over the code? Or is it possible to trigger this somehow? I'm lacking still some understanding of the internal mechanism of coccinelle).
No. There is no way to create several environments from a single invocation of a script rule. All you can do is extend the existing environment that caused the rule to be executed in the first place. Probably you just want a global list or hash table containing your information. You can also make metavariable constraints that involve scripting code that could involve your global list or hash table. For example:
identifier i : script:ocaml() { Hashtbl.mem some_functions i };
The script code should return true or false.
julia
Quoted 96 lines
Best regards and thanks!
Florian PesthSemantic patch:
@findisr@
expression E;
identifier isrfunction != NULL;
@@ISR_SetHandler(E, isrfunction)@mallocfunc@
identifiermalloc_func;
@@
malloc_func(...){
...
malloc(...);
...
}@findmalloc extends findisr@
position p;
identifier mallocfunc.malloc_func;
@@
isrfunction(...){@p
...malloc_func(...);
...
}@script:python@
isrfunction << findisr.isrfunction;
p << findmalloc.p;malloc_func<< mallocfunc.malloc_func;
@@
print "ISR function %s line %s file %s calling function %s->malloc" % (isrfunction, p[0].line, p[0].file,malloc_func)@cmallocfunc@
identifier mallocfunc.malloc_func;
identifiercmalloc_func;
@@
cmalloc_func(...){
...malloc_func(...);
...
}@findcmalloc extends findisr@
position p;
identifier cmallocfunc.cmalloc_func;
@@
isrfunction(...){@p
...cmalloc_func(...);
...
}@script:python@
isrfunction << findisr.isrfunction;
p << findcmalloc.p;cmalloc_func<< cmallocfunc.cmalloc_func;malloc_func<< mallocfunc.malloc_func;
@@
print "ISR function %s line %s file %s calling function %s->%s->malloc" % (isrfunction, p[0].line, p[0].file,cmalloc_func,malloc_func)Example C File:
voidother_func(int something);
voidother_func2(int something);void
ISR_func1(int something){ other_func(something); }void
Init_ISR(void){ int hello; ISR_SetHandler(hello, ISR_func1); }void
other_func(int something){ malloc(sizeof(double)*something); other_func2(something); }void
other_func2(int something){ malloc(sizeof(double)*something); }_______________________________________________
Cocci mailing list
Cocci at systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci
Von: Julia Lawall <[email protected]>
Gesendet: Mittwoch, 29. November 2017 12:21
Quoted 2 lines
Look at
demos/iteration.cocci.? This is discussed in "Advanced SmPL" at
http://coccinelle.lip6.fr/papers.php.
Thanks, this seems to be exactly what I need! However demos/iteration.cocci and demos/python_iteration.cocci should do the same with ocaml and python respectively, or is there a difference? With demos/iteration.cocci, I get a match on
...
Skipping: demos/interprocedural_adhoc.c
1 pending new file instances
0 pending original file instances
HANDLING: demos/python_iteration.c
1 pending new file instances
0 pending original file instances
HANDLING: demos/python_iteration.c
* TODO [[view:demos/python_iteration.c::face=ovl-face1::linb=11::colb=4::cole=5][def demos/python_iteration.c::11]] [[view:demos/python_iteration.c::face=ovl-face2::linb=12::colb=4::cole=5][ref]]
but with demos/python_iteration.cocci I don't get a match:
...
Skipping: demos/interprocedural_adhoc.c
1 pending new file instances
0 pending original file instances
HANDLING: demos/python_iteration.c
1 pending new file instances
0 pending original file instances
No rules apply. Perhaps your semantic patch doesn't contain any +/-/* code, or you have a failed dependency. If the problem is not clear, try --debug-parse-cocci or check whether any virtual rules (e.g., after_start) should be defined.
This was tested with the latest git code on OpenSuse and Ubuntu:
spatch version 1.0.6-00362-gce2ea2c compiled with OCaml version 4.02.3 Flags passed to the configure script: --prefix=/home/v6000d/ --disable-parmap OCaml scripting support: yes
Python scripting support: yes
Syntax of regular expresssions: PCRE
BTW on Ubuntu (16.04.2 LTS) I had some problems compiling coccinelle which could be solved by adding "--disable-parmap" to the configure script. Is this a known issue? I did not need to add this option on OpenSuse.
Quoted 4 lines
... There is no way to create several environments from a single invocation of a script rule.? All you can do is extend the existing environment that caused the rule to be executed in the first place. ... [solution with global list]
Thanks, if iteration works as above this will already be sufficient for me at the moment. Thanks a lot for your help!
Best regards,
Florian Pesth
Quoted 28 lines · Pesth Florian
Von: Julia Lawall <[email protected]>
Gesendet: Mittwoch, 29. November 2017 12:21Quoted 2 lines
Look at
demos/iteration.cocci.? This is discussed in "Advanced SmPL" at
http://coccinelle.lip6.fr/papers.php.Thanks, this seems to be exactly what I need! However
demos/iteration.coccianddemos/python_iteration.coccishould do the same with ocaml and python respectively, or is there a difference? Withdemos/iteration.cocci, I get a match on...
Skipping:demos/interprocedural_adhoc.c
1 pending new file instances
0 pending original file instancesHANDLING:
demos/python_iteration.c
1 pending new file instances
0 pending original file instancesHANDLING:
demos/python_iteration.c
* TODO [[view:demos/python_iteration.c::face=ovl-face1::linb=11::colb=4::cole=5][def demos/python_iteration.c::11]] [[view:demos/python_iteration.c::face=ovl-face2::linb=12::colb=4::cole=5][ref]]but with
demos/python_iteration.cocciI don't get a match:...
Skipping:demos/interprocedural_adhoc.c
1 pending new file instances
0 pending original file instances
Quoted 5 lines
HANDLING:
demos/python_iteration.c
1 pending new file instances
0 pending original file instancesNo rules apply. Perhaps your semantic patch doesn't contain any +/-/* code, or you have a failed dependency. If the problem is not clear, try --debug-parse-cocci or check whether any virtual rules (e.g.,
after_start) should be defined.
I have this behavior too. Could you live with the OCaml version in the short term? You only have to use OCaml when calling add_if_not_present. You can write python scripts if you need to do other things.
Quoted 10 lines
This was tested with the latest git code on OpenSuse and Ubuntu:
spatch version 1.0.6-00362-gce2ea2c compiled with OCaml version 4.02.3 Flags passed to the configure script: --prefix=/home/v6000d/ --disable-parmap OCaml scripting support: yes
Python scripting support: yes
Syntax of regular expresssions: PCREBTW on Ubuntu (16.04.2 LTS) I had some problems compiling coccinelle which could be solved by adding "--disable-parmap" to the configure script. Is this a known issue? I did not need to add this option on OpenSuse.
No, it's not a known issue. Do you have a parmap that was compiled with a different version of OCaml?
julia
Quoted 10 lines
Quoted 4 lines
... There is no way to create several environments from a single invocation of a script rule.? All you can do is extend the existing environment that caused the rule to be executed in the first place. ... [solution with global list]
Thanks, if iteration works as above this will already be sufficient for me at the moment. Thanks a lot for your help!
Best regards,
Florian Pesth
Von: Julia Lawall <[email protected]>
Gesendet: Montag, 4. Dezember 2017 15:58
Quoted 3 lines
Quoted 3 lines
I have this behavior too.? Could you live with the OCaml version in the short term?? You only have to use OCaml when calling
add_if_not_present. You can write python scripts if you need to do other things.
Yes, of course! I was just wondering about the python variant. For my purposes the Ocaml variant of add_if_not_present is fine.
Quoted 4 lines
BTW on Ubuntu (16.04.2 LTS) I had some problems compiling coccinelle which could be solved by adding "--disable-parmap" to the configure script. Is this a known issue? I did not need to add this option on OpenSuse.
Quoted 2 lines
No, it's not a known issue.? Do you have a parmap that was compiled with a different version of OCaml?
I'm not sure about that, this is what the package manager says about parmap (Looks like it is compiled against ocaml 4.02.3?):
~/coccinelle$ apt-cache show libparmap-ocaml
Package: libparmap-ocaml
Priority: optional
Section: universe/ocaml
Installed-Size: 159
Maintainer: Ubuntu Developers <[email protected]>
Original-Maintainer: Debian OCaml Maintainers <[email protected]>
Architecture: amd64
Source: parmap
Version: 1.0~rc7-1
Provides: libparmap-ocaml-d0ym9
Depends: libc6 (>= 2.14), ocaml-base-nox-4.02.3
Filename: pool/universe/p/parmap/libparmap-ocaml_1.0~rc7-1_amd64.deb
Size: 33006
MD5sum: 2517aa1fd6ae67876ccefc3c52521174
SHA1: 30b81be2dff5f51ee220f029f5015c864f3672ca
SHA256: 1825d791f74caf751a34704f996895e4af58e5c3a85be5be8d1a75c0039c12f1
Description-en: small OCaml library allowing to exploit multicore architectures
Parmap is a minimalistic library allowing to exploit multicore architectures for OCaml programs with minimal modifications.
.
This package provides the runtime library.
Description-md5: 2d2a4e90ce86065d78033253316fbc51
Homepage: https://rdicosmo.github.io/parmap/
Bugs: https://bugs.launchpad.net/ubuntu/+filebug
Origin: Ubuntu
And about ocaml:
~/coccinelle$ ocaml -version
The OCaml toplevel, version 4.02.3
./configure says:
...
checking for ocamlc... /usr/bin/ocamlc
OCaml version is 4.02.3
OCaml library path is /usr/lib/ocaml
checking for ocamlopt... /usr/bin/ocamlopt
checking for ocamlc.opt... /usr/bin/ocamlc.opt
checking for ocamlopt.opt... /usr/bin/ocamlopt.opt
checking for ocaml... /usr/bin/ocaml
checking for ocamldep... /usr/bin/ocamldep
checking for ocamlmktop... /usr/bin/ocamlmktop
checking for ocamlmklib... /usr/bin/ocamlmklib
checking for ocamldoc... /usr/bin/ocamldoc
checking that the OCaml version is at least 3.12... yes
checking that the OCaml version is at least 4.02... yes
checking for ocamllex... /usr/bin/ocamllex
checking for ocamllex.opt... /usr/bin/ocamllex.opt
checking for ocamlyacc... /usr/bin/ocamlyacc
checking for ocamlfind... /usr/bin/ocamlfind
checking for ocamlprof... /usr/bin/ocamlprof
configure: verifying basic ocaml modules
configure: coccinelle may use external ocaml libraries in /home/v6000d/coccinelle/bundles
...
configure: configuring package parmap
checking for OCaml findlib package parmap... found
make says:
...
OCAMLC parsing_c/includes.mli
OCAMLOPT parsing_c/includes.ml
File "parsing_c/includes.ml", line 159, characters 9-22:
Error: Unbound module Parmap
Makefile:390: recipe for target 'parsing_c/includes.cmx' failed
make: *** [parsing_c/includes.cmx] Error 2
rm parsing_cocci/lexer_cli.ml parsing_cocci/parser_cocci_menhir.ml.d parsing_cocci/parser_cocci_menhir.mli.d parsing_cocci/lexer_script.ml parsing_cocci/lexer_cocci.ml
Not sure, if any of this helps.
Best regards,
Florian Pesth