coccinelle 1.0.6: OpenMP #pragma directive leads to function ignore
Dear Prof. Lawall and Coccinelle team,
I'm an enthusiastical, however still beginner use of Coccinelle.
While running spatch 1.0.6 on a long OpenMP-enabled listing, I noticed that the presence of the C line:
#pragma omp parallel for reduction(+:I) private(I)
seems to prevent restructuring of the function containing it. If I simplify the pragma to something like
#pragma omp parallel
the problem disappears.
Is there anything I can do to restructure large codes containing many of such similar lines ?
In the attachment you find a test case and the output I experience.
Best regards,
Michele
-------------- next part --------------
+ spatch --version
spatch version 1.0.6 compiled with OCaml version 4.05.0
Flags passed to the configure script: --host=i586-suse-linux-gnu --build=i586-suse-linux-gnu --program-prefix= --disable-dependency-tracking --prefix=/usr --exec-prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin --sysconfdir=/etc --datadir=/usr/share --includedir=/usr/include --libdir=/usr/lib --libexecdir=/usr/lib --localstatedir=/var --sharedstatedir=/var/lib --mandir=/usr/share/man --infodir=/usr/share/info --disable-dependency-tracking Python scripting support: yes
Syntax of regular expresssions: PCRE
+ sed -i 's/^...pragma/#pragma /g' pragma_ignored.c
+ cat pragma_ignored.c
void good(void)
{
A[I]; /* not ignored */
}
void bad(void)
{
A[I]; /* if pragma uncommented ignored by spatch 1.0.6 */
#pragma omp parallel for reduction(+:I) private(I)
}
+ spatch --sp-file pragma_ignored.cocci pragma_ignored.cinit_defs_builtins: /usr/lib/coccinelle/standard.h
HANDLING: pragma_ignored.c
tmp/di36pef/cocci-output-200330-4f22e8-pragma_ignored.c renamed from pragma_ignored.c +1 -1
@@ -1,6 +1,6 @@void good (void ) { -A [I ];/* not ignored */ +A [J ];/* not ignored */ }void bad (void ) {
+ sed -i 's/^.pragma /\/\/#pragma/g' pragma_ignored.c
+ cat pragma_ignored.c
void good(void)
{
A[I]; /* not ignored */
}
void bad(void)
{
A[I]; /* if pragma uncommented ignored by spatch 1.0.6 */
//#pragma omp parallel for reduction(+:I) private(I) }
+ spatch --sp-file pragma_ignored.cocci pragma_ignored.cinit_defs_builtins: /usr/lib/coccinelle/standard.h
HANDLING: pragma_ignored.c
tmp/di36pef/cocci-output-200335-2965e3-pragma_ignored.c renamed from pragma_ignored.c +2 -2
@@ -1,9 +1,9 @@void good (void ) { -A [I ];/* not ignored */ +A [J ];/* not ignored */ }void bad (void ) { -A [I ];/* if pragma uncommented ignored by spatch 1.0.6 */ +A [J ];/* if pragma uncommented ignored by spatch 1.0.6 */ //#pragma omp parallel for reduction(+:I) private(I) }
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pragma_ignored.sh
Type: application/x-sh
Size: 345 bytes
Desc: not available
URL: <https://systeme.lip6.fr/pipermail/cocci/attachments/20180426/bb9498f0/attachment.sh>
-------------- next part --------------
void good(void)
{
A[I]; /* not ignored */
}
void bad(void)
{
A[I]; /* if pragma uncommented ignored by spatch 1.0.6 */
//#pragma omp parallel for reduction(+:I) private(I)
}
-------------- next part --------------
// pragma_ignored.cocci
//
// $ spatch --version
// spatch version 1.0.6 compiled with OCaml version 4.05.0
// Flags passed to the configure script: --host=i586-suse-linux-gnu --build=i586-suse-linux-gnu --program-prefix= --disable-dependency-tracking --prefix=/usr --exec-prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin --sysconfdir=/etc --datadir=/usr/share --includedir=/usr/include --libdir=/usr/lib --libexecdir=/usr/lib --localstatedir=/var --sharedstatedir=/var/lib --mandir=/usr/share/man --infodir=/usr/share/info --disable-dependency-tracking // Python scripting support: yes
// Syntax of regular expresssions: PCRE
//
//
// $ spatch --sp-file pragma_ignored.cocci pragma_i
//
// @@ -1,9 +1,9 @@
// void good(void)
// {
// - A[I]; /* not ignored */
// + A[J]; /* not ignored */
// }
// void bad(void)
// {// - A[I]; /* if pragma uncommented ignored by spatch 1.0.6 */ // + A[J]; /* if pragma uncommented ignored by spatch 1.0.6 */ // //#pragma omp parallel for reduction(+:I) private(I) // }
//
@@
symbol A,I,J;
@@
-A[I]
+A[J]
Quoted 12 lines · Michele Martone
Dear Prof. Lawall and Coccinelle team,
I'm an enthusiastical, however still beginner use of Coccinelle.
While running spatch 1.0.6 on a long OpenMP-enabled listing, I noticed that the presence of the C line:
#pragma omp parallel for reduction(+:I) private(I)
seems to prevent restructuring of the function containing it. If I simplify the pragma to something like
#pragma omp parallel
the problem disappears.
Quoted 4 lines
Is there anything I can do to restructure large codes containing many of such similar lines ?
In the attachment you find a test case and the output I experience.
Thanks for the report. We can try to make the parsing of #pragmas more flexible.
julia
Quoted 17 lines · Julia Lawall
Quoted 15 lines · Michele Martone
Dear Prof. Lawall and Coccinelle team,
I'm an enthusiastical, however still beginner use of Coccinelle.
While running spatch 1.0.6 on a long OpenMP-enabled listing, I noticed that the presence of the C line:
#pragma omp parallel for reduction(+:I) private(I)
seems to prevent restructuring of the function containing it. If I simplify the pragma to something like
#pragma omp parallel
the problem disappears.Is there anything I can do to restructure large codes containing many of such similar lines ?
Quoted 6 lines
Quoted 1 line
In the attachment you find a test case and the output I experience.
Thanks for the report. We can try to make the parsing of #pragmas more flexible.
julia
Dear Julia,
thanks for the quick response!
I'll be waiting for Coccinelle-sided developments then.
May I ask what would you recommend now to circumvent this ?
Quoted 1 line
From the SmPL v1.0.6 Grammar I am not aware of any way to
e.g. ignore pragma lines as they were comments.
So the best what comes to my mind is to:
- comment #pragma lines
- compute semantic patch
- apply semantic patch
- uncomment #pragma lines
If there is any better solution, please let me know.
Thanks,
Michele
Quoted 36 lines · Michele Martone
Quoted 17 lines · Julia Lawall
Quoted 15 lines · Michele Martone
Dear Prof. Lawall and Coccinelle team,
I'm an enthusiastical, however still beginner use of Coccinelle.
While running spatch 1.0.6 on a long OpenMP-enabled listing, I noticed that the presence of the C line:
#pragma omp parallel for reduction(+:I) private(I)
seems to prevent restructuring of the function containing it. If I simplify the pragma to something like
#pragma omp parallel
the problem disappears.Is there anything I can do to restructure large codes containing many of such similar lines ?
Quoted 6 lines
Quoted 1 line
In the attachment you find a test case and the output I experience.
Thanks for the report. We can try to make the parsing of #pragmas more flexible.
julia
Dear Julia,
thanks for the quick response!
I'll be waiting for Coccinelle-sided developments then.
May I ask what would you recommend now to circumvent this ?
From the SmPL v1.0.6 Grammar I am not aware of any way to e.g. ignore pragma lines as they were comments.
Quoted 8 lines
So the best what comes to my mind is to:
- comment #pragma lines
- compute semantic patch
- apply semantic patch
- uncomment #pragma linesIf there is any better solution, please let me know.
I don't see any other solution, sorry. For some other parsing issues, it might be possible to define a macro, but that is not possible here.
julia
Quoted 36 lines · Michele Martone
Quoted 17 lines · Julia Lawall
Quoted 15 lines · Michele Martone
Dear Prof. Lawall and Coccinelle team,
I'm an enthusiastical, however still beginner use of Coccinelle.
While running spatch 1.0.6 on a long OpenMP-enabled listing, I noticed that the presence of the C line:
#pragma omp parallel for reduction(+:I) private(I)
seems to prevent restructuring of the function containing it. If I simplify the pragma to something like
#pragma omp parallel
the problem disappears.Is there anything I can do to restructure large codes containing many of such similar lines ?
Quoted 6 lines
Quoted 1 line
In the attachment you find a test case and the output I experience.
Thanks for the report. We can try to make the parsing of #pragmas more flexible.
julia
Dear Julia,
thanks for the quick response!
I'll be waiting for Coccinelle-sided developments then.
May I ask what would you recommend now to circumvent this ?
From the SmPL v1.0.6 Grammar I am not aware of any way to e.g. ignore pragma lines as they were comments.
Quoted 8 lines
So the best what comes to my mind is to:
- comment #pragma lines
- compute semantic patch
- apply semantic patch
- uncomment #pragma linesIf there is any better solution, please let me know.
The problem is now fixed in the github version of Coccinelle.
julia
Quoted 45 lines · Julia Lawall
Quoted 43 lines · Michele Martone
Quoted 17 lines · Julia Lawall
Quoted 15 lines · Michele Martone
Dear Prof. Lawall and Coccinelle team,
I'm an enthusiastical, however still beginner use of Coccinelle.
While running spatch 1.0.6 on a long OpenMP-enabled listing, I noticed that the presence of the C line:
#pragma omp parallel for reduction(+:I) private(I)
seems to prevent restructuring of the function containing it. If I simplify the pragma to something like
#pragma omp parallel
the problem disappears.Is there anything I can do to restructure large codes containing many of such similar lines ?
Quoted 6 lines
Quoted 1 line
In the attachment you find a test case and the output I experience.
Thanks for the report. We can try to make the parsing of #pragmas more flexible.
julia
Dear Julia,
thanks for the quick response!
I'll be waiting for Coccinelle-sided developments then.
May I ask what would you recommend now to circumvent this ?
From the SmPL v1.0.6 Grammar I am not aware of any way to e.g. ignore pragma lines as they were comments.
So the best what comes to my mind is to:
- comment #pragma lines
- compute semantic patch
- apply semantic patch
- uncomment #pragma lines
Quoted 5 lines
Quoted 1 line
If there is any better solution, please let me know.
The problem is now fixed in the github version of Coccinelle.
julia
That is good news -- thank you :-)
Handling such pragma lines opens new ranges of use for Coccinelle!
On my side, I just followed install.txt:
#!/bin/bash
set -e
set -x
trap "read" EXIT # will block for input on error git clone https://github.com/coccinelle/coccinelle.git
cd coccinelle
./autogen
./configure
make # it stops here
# make install
and it reaches:
...
OCAMLOPT enter.ml OCAMLC main.mli OCAMLOPT main.ml OCAMLOPT -o spatch.opt
/usr/lib64/ocaml/libbigarray.a(bigarray_stubs.o): In function `caml_ba_deserialize':/home/abuild/rpmbuild/BUILD/ocaml-4.03.0/otherlibs/bigarray/bigarray_stubs.c:979: undefined reference to caml_umul_overflow' /home/abuild/rpmbuild/BUILD/ocaml-4.03.0/otherlibs/bigarray/bigarray_stubs.c:985: undefined reference to caml_umul_overflow'
collect2: error: ld returned 1 exit status
File "caml_startup", line 1:
Error: Error during linking
Makefile:609: recipe for target 'spatch.opt' failed
make: *** [spatch.opt] Error 2
rm parsing_cocci/lexer_cli.ml parsing_cocci/parser_cocci_menhir.ml.d parsing_cocci/parser_cocci_menhir.mli.d parsing_cocci/lexer_script.ml parsing_cocci/lexer_cocci.ml parsing_c/lexer_c.ml
+ read
It's my first experience building any ocaml-based software, so I would appreciate any hint how to proceed further, if possible.
Michele
Quoted 87 lines · Michele Martone
Quoted 45 lines · Julia Lawall
Quoted 43 lines · Michele Martone
Quoted 17 lines · Julia Lawall
Quoted 15 lines · Michele Martone
Dear Prof. Lawall and Coccinelle team,
I'm an enthusiastical, however still beginner use of Coccinelle.
While running spatch 1.0.6 on a long OpenMP-enabled listing, I noticed that the presence of the C line:
#pragma omp parallel for reduction(+:I) private(I)
seems to prevent restructuring of the function containing it. If I simplify the pragma to something like
#pragma omp parallel
the problem disappears.Is there anything I can do to restructure large codes containing many of such similar lines ?
Quoted 6 lines
Quoted 1 line
In the attachment you find a test case and the output I experience.
Thanks for the report. We can try to make the parsing of #pragmas more flexible.
julia
Dear Julia,
thanks for the quick response!
I'll be waiting for Coccinelle-sided developments then.
May I ask what would you recommend now to circumvent this ?
From the SmPL v1.0.6 Grammar I am not aware of any way to e.g. ignore pragma lines as they were comments.
So the best what comes to my mind is to:
- comment #pragma lines
- compute semantic patch
- apply semantic patch
- uncomment #pragma linesQuoted 5 lines
Quoted 1 line
If there is any better solution, please let me know.
The problem is now fixed in the github version of Coccinelle.
julia
That is good news -- thank you :-)
Handling such pragma lines opens new ranges of use for Coccinelle!
On my side, I just followed install.txt:
#!/bin/bash
set -e
set -x
trap "read" EXIT # will block for input on error git clone https://github.com/coccinelle/coccinelle.git
cd coccinelle
./autogen
./configure
make # it stops here# make installand it reaches:
...
OCAMLOPT enter.ml OCAMLC main.mli OCAMLOPT main.ml OCAMLOPT -o spatch.opt/usr/lib64/ocaml/libbigarray.a(bigarray_stubs.o): In function `caml_ba_deserialize':
/home/abuild/rpmbuild/BUILD/ocaml-4.03.0/otherlibs/bigarray/bigarray_stubs.c:979: undefined reference tocaml_umul_overflow' /home/abuild/rpmbuild/BUILD/ocaml-4.03.0/otherlibs/bigarray/bigarray_stubs.c:985: undefined reference tocaml_umul_overflow'
collect2: error: ld returned 1 exit status
File "caml_startup", line 1:
Error: Error during linking
Makefile:609: recipe for target 'spatch.opt' failed
make: *** [spatch.opt] Error 2
rm parsing_cocci/lexer_cli.ml parsing_cocci/parser_cocci_menhir.ml.d parsing_cocci/parser_cocci_menhir.mli.d parsing_cocci/lexer_script.ml parsing_cocci/lexer_cocci.ml parsing_c/lexer_c.ml
+ read
Quoted 2 lines
It's my first experience building any ocaml-based software, so I would appreciate any hint how to proceed further, if possible.
Oops, I'll forward this to my engineer, who knows about building issues. I'm not sure if he will answer before Wednesday, due to the holiday. What is your OS?
julia
Quoted 91 lines · Julia Lawall
Quoted 87 lines · Michele Martone
Quoted 45 lines · Julia Lawall
Quoted 43 lines · Michele Martone
Quoted 17 lines · Julia Lawall
Quoted 15 lines · Michele Martone
Dear Prof. Lawall and Coccinelle team,
I'm an enthusiastical, however still beginner use of Coccinelle.
While running spatch 1.0.6 on a long OpenMP-enabled listing, I noticed that the presence of the C line:
#pragma omp parallel for reduction(+:I) private(I)
seems to prevent restructuring of the function containing it. If I simplify the pragma to something like
#pragma omp parallel
the problem disappears.Is there anything I can do to restructure large codes containing many of such similar lines ?
Quoted 6 lines
Quoted 1 line
In the attachment you find a test case and the output I experience.
Thanks for the report. We can try to make the parsing of #pragmas more flexible.
julia
Dear Julia,
thanks for the quick response!
I'll be waiting for Coccinelle-sided developments then.
May I ask what would you recommend now to circumvent this ?
From the SmPL v1.0.6 Grammar I am not aware of any way to e.g. ignore pragma lines as they were comments.
So the best what comes to my mind is to:
- comment #pragma lines
- compute semantic patch
- apply semantic patch
- uncomment #pragma linesQuoted 5 lines
Quoted 1 line
If there is any better solution, please let me know.
The problem is now fixed in the github version of Coccinelle.
julia
That is good news -- thank you :-)
Handling such pragma lines opens new ranges of use for Coccinelle!
On my side, I just followed install.txt:
#!/bin/bash
set -e
set -x
trap "read" EXIT # will block for input on error git clone https://github.com/coccinelle/coccinelle.git
cd coccinelle
./autogen
./configure
make # it stops here# make installand it reaches:
...
OCAMLOPT enter.ml OCAMLC main.mli OCAMLOPT main.ml OCAMLOPT -o spatch.opt/usr/lib64/ocaml/libbigarray.a(bigarray_stubs.o): In function `caml_ba_deserialize':
/home/abuild/rpmbuild/BUILD/ocaml-4.03.0/otherlibs/bigarray/bigarray_stubs.c:979: undefined reference tocaml_umul_overflow' /home/abuild/rpmbuild/BUILD/ocaml-4.03.0/otherlibs/bigarray/bigarray_stubs.c:985: undefined reference tocaml_umul_overflow'
collect2: error: ld returned 1 exit status
File "caml_startup", line 1:
Error: Error during linking
Makefile:609: recipe for target 'spatch.opt' failed
make: *** [spatch.opt] Error 2
rm parsing_cocci/lexer_cli.ml parsing_cocci/parser_cocci_menhir.ml.d parsing_cocci/parser_cocci_menhir.mli.d parsing_cocci/lexer_script.ml parsing_cocci/lexer_cocci.ml parsing_c/lexer_c.ml
+ read
Quoted 6 lines
Quoted 2 lines
It's my first experience building any ocaml-based software, so I would appreciate any hint how to proceed further, if possible.
Oops, I'll forward this to my engineer, who knows about building issues. I'm not sure if he will answer before Wednesday, due to the holiday. What is your OS?
$ cat /etc/os-release NAME="openSUSE Leap" VERSION="42.3" ID=opensuse ID_LIKE="suse" VERSION_ID="42.3" PRETTY_NAME="openSUSE Leap 42.3" ANSI_COLOR="0;32" CPE_NAME="cpe:/o:opensuse:leap:42.3"
thanks!
[...]
Quoted 18 lines
#!/bin/bash
set -e
set -x
trap "read" EXIT # will block for input on error git clone https://github.com/coccinelle/coccinelle.git
cd coccinelle
./autogen
./configure
make # it stops here# make installand it reaches:
...
OCAMLOPT enter.ml OCAMLC main.mli OCAMLOPT main.ml OCAMLOPT -o spatch.opt
Quoted 10 lines
/usr/lib64/ocaml/libbigarray.a(bigarray_stubs.o): In function `caml_ba_deserialize':
/home/abuild/rpmbuild/BUILD/ocaml-4.03.0/otherlibs/bigarray/bigarray_stubs.c:979: undefined reference tocaml_umul_overflow' /home/abuild/rpmbuild/BUILD/ocaml-4.03.0/otherlibs/bigarray/bigarray_stubs.c:985: undefined reference tocaml_umul_overflow'
collect2: error: ld returned 1 exit status
File "caml_startup", line 1:
Error: Error during linking
Makefile:609: recipe for target 'spatch.opt' failed
make: *** [spatch.opt] Error 2
rm parsing_cocci/lexer_cli.ml parsing_cocci/parser_cocci_menhir.ml.d parsing_cocci/parser_cocci_menhir.mli.d parsing_cocci/lexer_script.ml parsing_cocci/lexer_cocci.ml parsing_c/lexer_c.ml
+ read
I usually ask the package manager to install build dependencies for me:
Fedora:
$ sudo dnf builddep coccinelle
Ubuntu/Debian:
$ sudo apt-get build-dep coccinelle
[...]
Signature · 1 line
Peter
Quoted 33 lines · Peter Senna Tschudin
[...]
Quoted 18 lines
#!/bin/bash
set -e
set -x
trap "read" EXIT # will block for input on error git clone https://github.com/coccinelle/coccinelle.git
cd coccinelle
./autogen
./configure
make # it stops here# make installand it reaches:
...
OCAMLOPT enter.ml OCAMLC main.mli OCAMLOPT main.ml OCAMLOPT -o spatch.optQuoted 10 lines
/usr/lib64/ocaml/libbigarray.a(bigarray_stubs.o): In function `caml_ba_deserialize':
/home/abuild/rpmbuild/BUILD/ocaml-4.03.0/otherlibs/bigarray/bigarray_stubs.c:979: undefined reference tocaml_umul_overflow' /home/abuild/rpmbuild/BUILD/ocaml-4.03.0/otherlibs/bigarray/bigarray_stubs.c:985: undefined reference tocaml_umul_overflow'
collect2: error: ld returned 1 exit status
File "caml_startup", line 1:
Error: Error during linking
Makefile:609: recipe for target 'spatch.opt' failed
make: *** [spatch.opt] Error 2
rm parsing_cocci/lexer_cli.ml parsing_cocci/parser_cocci_menhir.ml.d parsing_cocci/parser_cocci_menhir.mli.d parsing_cocci/lexer_script.ml parsing_cocci/lexer_cocci.ml parsing_c/lexer_c.ml
+ readI usually ask the package manager to install build dependencies for me:
Quoted 7 lines
Fedora:
$ sudo dnf builddep coccinelleUbuntu/Debian:
$ sudo apt-get build-dep coccinelle[...]
Dear Peter, thanks for the apt-get based recipe, it worked for me on debian buster with ocaml-4.0.5.
Dear Julia, thanks for the pragma patch which will enable coccinelle to support OpenMP-enabled listings too (I just tested it on the above setup) :-)
Michele
Quoted 10 lines · Michele Martone
Quoted 1 line · Peter Senna Tschudin
...
Dear Peter, thanks for the apt-get based recipe, it worked for me on debian buster with ocaml-4.0.5.
Dear Julia, thanks for the pragma patch which will enable coccinelle to support OpenMP-enabled listings too (I just tested it on the above setup) :-)
Michele
Dear Julia,
I have another similar minimal test spatch case failing, attached. It seems like using
#pragma omp parallel for private(I)
breaks the semantic patch, while having
#pragma omp parallel for
is still ok.
Please see attachment.
I'm using self-built ocaml (e407ecf11 2018-05-25)
and coccinelle (a4532f08 2018-05-17).
would be very useful to have full #pragma resilience!
Michele
-------------- next part --------------
// seems like this example breaks:
// spatch version 1.0.6-00440-ga4532f08 compiled with OCaml version 4.08.0+dev0-2018-04-09 // (a4532f08 2018-05-17)
/*pragma_ignored_again.cocci:
@@
symbol A,I,J;
@@
-A[I]
+A[J]
*/
void f(void)
{
int*A=NULL,I=0;
// pragma presence breaks semantic patch (remote private(i) to 'fix' it)
// comment it to have A[I] -> A[J]
#pragma omp parallel for private(I)
A[I]++;
}[Cocci] Coccinelle 1.0.6: OpenMP #pragma directive leads to function ignore
In reply to Julia LawallQuoted 1 line
We can try to make the parsing of #pragmas more flexible.
I am also curious on further software evolution in this area for a while.
https://github.com/coccinelle/coccinelle/issues/51
Regards,
Markus