Skip to contentSkip to conversation

coccinelle 1.0.6: OpenMP #pragma directive leads to function ignore

Cocci Michele Martone 4 participants 12 messages Apr 26, 2018 – May 25, 2018

lore
Share & tools
lore
#0 lore raw

Dear Prof. Lawall and Coccinelle team,

I'm an enthusiastical, however still beginner use of Coccinelle.

While running spatch 1.0.6 on a long OpenMP-enabled listing, I noticed that the presence of the C line:
#pragma omp parallel for reduction(+:I) private(I)
seems to prevent restructuring of the function containing it. If I simplify the pragma to something like
#pragma omp parallel
the problem disappears.

Is there anything I can do to restructure large codes containing many of such similar lines ?

In the attachment you find a test case and the output I experience.

Best regards,
Michele
-------------- next part --------------
+ spatch --version
spatch version 1.0.6 compiled with OCaml version 4.05.0
Flags passed to the configure script: --host=i586-suse-linux-gnu --build=i586-suse-linux-gnu --program-prefix= --disable-dependency-tracking --prefix=/usr --exec-prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin --sysconfdir=/etc --datadir=/usr/share --includedir=/usr/include --libdir=/usr/lib --libexecdir=/usr/lib --localstatedir=/var --sharedstatedir=/var/lib --mandir=/usr/share/man --infodir=/usr/share/info --disable-dependency-tracking Python scripting support: yes
Syntax of regular expresssions: PCRE
+ sed -i 's/^...pragma/#pragma /g' pragma_ignored.c
+ cat pragma_ignored.c
void good(void)
{
A[I]; /* not ignored */
}
void bad(void)
{
A[I]; /* if pragma uncommented ignored by spatch 1.0.6 */
#pragma omp parallel for reduction(+:I) private(I)
}
+ spatch --sp-file pragma_ignored.cocci pragma_ignored.c
init_defs_builtins: /usr/lib/coccinelle/standard.h
HANDLING: pragma_ignored.c

tmp/di36pef/cocci-output-200330-4f22e8-pragma_ignored.c renamed from pragma_ignored.c +1 -1
diff = 
@@ -1,6 +1,6 @@
 void good(void)
 {
- A[I]; /* not ignored */
+ A[J]; /* not ignored */
 }
 void bad(void)
 {

+ sed -i 's/^.pragma /\/\/#pragma/g' pragma_ignored.c
+ cat pragma_ignored.c
void good(void)
{
A[I]; /* not ignored */
}
void bad(void)
{
A[I]; /* if pragma uncommented ignored by spatch 1.0.6 */
//#pragma omp parallel for reduction(+:I) private(I) }
+ spatch --sp-file pragma_ignored.cocci pragma_ignored.c
init_defs_builtins: /usr/lib/coccinelle/standard.h
HANDLING: pragma_ignored.c

tmp/di36pef/cocci-output-200335-2965e3-pragma_ignored.c renamed from pragma_ignored.c +2 -2
diff = 
@@ -1,9 +1,9 @@
 void good(void)
 {
- A[I]; /* not ignored */
+ A[J]; /* not ignored */
 }
 void bad(void)
 {
- A[I]; /* if pragma uncommented ignored by spatch 1.0.6  */
+ A[J]; /* if pragma uncommented ignored by spatch 1.0.6  */
 //#pragma omp parallel for reduction(+:I) private(I)
 }

-------------- next part --------------
A non-text attachment was scrubbed...
Name: pragma_ignored.sh
Type: application/x-sh
Size: 345 bytes
Desc: not available
URL: <https://systeme.lip6.fr/pipermail/cocci/attachments/20180426/bb9498f0/attachment.sh>
-------------- next part --------------
void good(void)
{
A[I]; /* not ignored */
}
void bad(void)
{
A[I]; /* if pragma uncommented ignored by spatch 1.0.6 */
//#pragma omp parallel for reduction(+:I) private(I)
}
-------------- next part --------------
// pragma_ignored.cocci
//
// $ spatch --version
// spatch version 1.0.6 compiled with OCaml version 4.05.0
// Flags passed to the configure script: --host=i586-suse-linux-gnu --build=i586-suse-linux-gnu --program-prefix= --disable-dependency-tracking --prefix=/usr --exec-prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin --sysconfdir=/etc --datadir=/usr/share --includedir=/usr/include --libdir=/usr/lib --libexecdir=/usr/lib --localstatedir=/var --sharedstatedir=/var/lib --mandir=/usr/share/man --infodir=/usr/share/info --disable-dependency-tracking // Python scripting support: yes
// Syntax of regular expresssions: PCRE
//
//
// $ spatch --sp-file pragma_ignored.cocci pragma_i
//
// @@ -1,9 +1,9 @@
// void good(void)
// {
// - A[I]; /* not ignored */
// + A[J]; /* not ignored */

//  }
//  void bad(void)
//  {

// - A[I]; /* if pragma uncommented ignored by spatch 1.0.6 */ // + A[J]; /* if pragma uncommented ignored by spatch 1.0.6 */ // //#pragma omp parallel for reduction(+:I) private(I) // }
//
@@
symbol A,I,J;
@@

-A[I]
+A[J]

#1 lore raw
Quoted 12 lines · Michele Martone

Dear Prof. Lawall and Coccinelle team,

I'm an enthusiastical, however still beginner use of Coccinelle.

While running spatch 1.0.6 on a long OpenMP-enabled listing, I noticed that the presence of the C line:
#pragma omp parallel for reduction(+:I) private(I)
seems to prevent restructuring of the function containing it. If I simplify the pragma to something like
#pragma omp parallel
the problem disappears.

Quoted 4 lines

Is there anything I can do to restructure large codes containing many of such similar lines ?

In the attachment you find a test case and the output I experience.

Thanks for the report. We can try to make the parsing of #pragmas more flexible.

julia

#2 lore raw
Quoted 17 lines · Julia Lawall
Quoted 15 lines · Michele Martone

Dear Prof. Lawall and Coccinelle team,

I'm an enthusiastical, however still beginner use of Coccinelle.

While running spatch 1.0.6 on a long OpenMP-enabled listing, I noticed that the presence of the C line:
#pragma omp parallel for reduction(+:I) private(I)
seems to prevent restructuring of the function containing it. If I simplify the pragma to something like
#pragma omp parallel
the problem disappears.

Is there anything I can do to restructure large codes containing many of such similar lines ?

Quoted 6 lines
Quoted 1 line

In the attachment you find a test case and the output I experience.

Thanks for the report. We can try to make the parsing of #pragmas more flexible.

julia

Dear Julia,

thanks for the quick response!

I'll be waiting for Coccinelle-sided developments then.

May I ask what would you recommend now to circumvent this ?

Quoted 1 line

From the SmPL v1.0.6 Grammar I am not aware of any way to

e.g. ignore pragma lines as they were comments.

So the best what comes to my mind is to:

- comment #pragma lines
- compute semantic patch
- apply semantic patch
- uncomment #pragma lines

If there is any better solution, please let me know.

Thanks,
Michele

#3 lore raw
Quoted 36 lines · Michele Martone
Quoted 17 lines · Julia Lawall
Quoted 15 lines · Michele Martone

Dear Prof. Lawall and Coccinelle team,

I'm an enthusiastical, however still beginner use of Coccinelle.

While running spatch 1.0.6 on a long OpenMP-enabled listing, I noticed that the presence of the C line:
#pragma omp parallel for reduction(+:I) private(I)
seems to prevent restructuring of the function containing it. If I simplify the pragma to something like
#pragma omp parallel
the problem disappears.

Is there anything I can do to restructure large codes containing many of such similar lines ?

Quoted 6 lines
Quoted 1 line

In the attachment you find a test case and the output I experience.

Thanks for the report. We can try to make the parsing of #pragmas more flexible.

julia

Dear Julia,

thanks for the quick response!

I'll be waiting for Coccinelle-sided developments then.

May I ask what would you recommend now to circumvent this ?

From the SmPL v1.0.6 Grammar I am not aware of any way to e.g. ignore pragma lines as they were comments.

Quoted 8 lines

So the best what comes to my mind is to:

- comment #pragma lines
- compute semantic patch
- apply semantic patch
- uncomment #pragma lines

If there is any better solution, please let me know.

I don't see any other solution, sorry. For some other parsing issues, it might be possible to define a macro, but that is not possible here.

julia

#4 lore raw
In reply to Michele Martone
Quoted 36 lines · Michele Martone
Quoted 17 lines · Julia Lawall
Quoted 15 lines · Michele Martone

Dear Prof. Lawall and Coccinelle team,

I'm an enthusiastical, however still beginner use of Coccinelle.

While running spatch 1.0.6 on a long OpenMP-enabled listing, I noticed that the presence of the C line:
#pragma omp parallel for reduction(+:I) private(I)
seems to prevent restructuring of the function containing it. If I simplify the pragma to something like
#pragma omp parallel
the problem disappears.

Is there anything I can do to restructure large codes containing many of such similar lines ?

Quoted 6 lines
Quoted 1 line

In the attachment you find a test case and the output I experience.

Thanks for the report. We can try to make the parsing of #pragmas more flexible.

julia

Dear Julia,

thanks for the quick response!

I'll be waiting for Coccinelle-sided developments then.

May I ask what would you recommend now to circumvent this ?

From the SmPL v1.0.6 Grammar I am not aware of any way to e.g. ignore pragma lines as they were comments.

Quoted 8 lines

So the best what comes to my mind is to:

- comment #pragma lines
- compute semantic patch
- apply semantic patch
- uncomment #pragma lines

If there is any better solution, please let me know.

The problem is now fixed in the github version of Coccinelle.

julia

#5 lore raw
Quoted 45 lines · Julia Lawall
Quoted 43 lines · Michele Martone
Quoted 17 lines · Julia Lawall
Quoted 15 lines · Michele Martone

Dear Prof. Lawall and Coccinelle team,

I'm an enthusiastical, however still beginner use of Coccinelle.

While running spatch 1.0.6 on a long OpenMP-enabled listing, I noticed that the presence of the C line:
#pragma omp parallel for reduction(+:I) private(I)
seems to prevent restructuring of the function containing it. If I simplify the pragma to something like
#pragma omp parallel
the problem disappears.

Is there anything I can do to restructure large codes containing many of such similar lines ?

Quoted 6 lines
Quoted 1 line

In the attachment you find a test case and the output I experience.

Thanks for the report. We can try to make the parsing of #pragmas more flexible.

julia

Dear Julia,

thanks for the quick response!

I'll be waiting for Coccinelle-sided developments then.

May I ask what would you recommend now to circumvent this ?

From the SmPL v1.0.6 Grammar I am not aware of any way to e.g. ignore pragma lines as they were comments.

So the best what comes to my mind is to:

- comment #pragma lines
- compute semantic patch
- apply semantic patch
- uncomment #pragma lines

Quoted 5 lines
Quoted 1 line

If there is any better solution, please let me know.

The problem is now fixed in the github version of Coccinelle.

julia

That is good news -- thank you :-)

Handling such pragma lines opens new ranges of use for Coccinelle!

On my side, I just followed install.txt:

#!/bin/bash
set -e
set -x
trap "read" EXIT # will block for input on error git clone https://github.com/coccinelle/coccinelle.git
cd coccinelle
./autogen
./configure
make # it stops here

# make install

and it reaches:

...

OCAMLOPT  enter.ml
OCAMLC    main.mli
OCAMLOPT  main.ml
OCAMLOPT  -o spatch.opt

/usr/lib64/ocaml/libbigarray.a(bigarray_stubs.o): In function `caml_ba_deserialize':
/home/abuild/rpmbuild/BUILD/ocaml-4.03.0/otherlibs/bigarray/bigarray_stubs.c:979: undefined reference to caml_umul_overflow' /home/abuild/rpmbuild/BUILD/ocaml-4.03.0/otherlibs/bigarray/bigarray_stubs.c:985: undefined reference to caml_umul_overflow'
collect2: error: ld returned 1 exit status
File "caml_startup", line 1:
Error: Error during linking
Makefile:609: recipe for target 'spatch.opt' failed
make: *** [spatch.opt] Error 2
rm parsing_cocci/lexer_cli.ml parsing_cocci/parser_cocci_menhir.ml.d parsing_cocci/parser_cocci_menhir.mli.d parsing_cocci/lexer_script.ml parsing_cocci/lexer_cocci.ml parsing_c/lexer_c.ml
+ read

It's my first experience building any ocaml-based software, so I would appreciate any hint how to proceed further, if possible.

Michele

#6 lore raw
Quoted 87 lines · Michele Martone
Quoted 45 lines · Julia Lawall
Quoted 43 lines · Michele Martone
Quoted 17 lines · Julia Lawall
Quoted 15 lines · Michele Martone

Dear Prof. Lawall and Coccinelle team,

I'm an enthusiastical, however still beginner use of Coccinelle.

While running spatch 1.0.6 on a long OpenMP-enabled listing, I noticed that the presence of the C line:
#pragma omp parallel for reduction(+:I) private(I)
seems to prevent restructuring of the function containing it. If I simplify the pragma to something like
#pragma omp parallel
the problem disappears.

Is there anything I can do to restructure large codes containing many of such similar lines ?

Quoted 6 lines
Quoted 1 line

In the attachment you find a test case and the output I experience.

Thanks for the report. We can try to make the parsing of #pragmas more flexible.

julia

Dear Julia,

thanks for the quick response!

I'll be waiting for Coccinelle-sided developments then.

May I ask what would you recommend now to circumvent this ?

From the SmPL v1.0.6 Grammar I am not aware of any way to e.g. ignore pragma lines as they were comments.

So the best what comes to my mind is to:

- comment #pragma lines
- compute semantic patch
- apply semantic patch
- uncomment #pragma lines

Quoted 5 lines
Quoted 1 line

If there is any better solution, please let me know.

The problem is now fixed in the github version of Coccinelle.

julia

That is good news -- thank you :-)

Handling such pragma lines opens new ranges of use for Coccinelle!

On my side, I just followed install.txt:

#!/bin/bash
set -e
set -x
trap "read" EXIT # will block for input on error git clone https://github.com/coccinelle/coccinelle.git
cd coccinelle
./autogen
./configure
make # it stops here

# make install

and it reaches:

...

OCAMLOPT  enter.ml
OCAMLC    main.mli
OCAMLOPT  main.ml
OCAMLOPT  -o spatch.opt

/usr/lib64/ocaml/libbigarray.a(bigarray_stubs.o): In function `caml_ba_deserialize':
/home/abuild/rpmbuild/BUILD/ocaml-4.03.0/otherlibs/bigarray/bigarray_stubs.c:979: undefined reference to caml_umul_overflow' /home/abuild/rpmbuild/BUILD/ocaml-4.03.0/otherlibs/bigarray/bigarray_stubs.c:985: undefined reference to caml_umul_overflow'
collect2: error: ld returned 1 exit status
File "caml_startup", line 1:
Error: Error during linking
Makefile:609: recipe for target 'spatch.opt' failed
make: *** [spatch.opt] Error 2
rm parsing_cocci/lexer_cli.ml parsing_cocci/parser_cocci_menhir.ml.d parsing_cocci/parser_cocci_menhir.mli.d parsing_cocci/lexer_script.ml parsing_cocci/lexer_cocci.ml parsing_c/lexer_c.ml
+ read

Quoted 2 lines

It's my first experience building any ocaml-based software, so I would appreciate any hint how to proceed further, if possible.

Oops, I'll forward this to my engineer, who knows about building issues. I'm not sure if he will answer before Wednesday, due to the holiday. What is your OS?

julia

#7 lore raw
Quoted 91 lines · Julia Lawall
Quoted 87 lines · Michele Martone
Quoted 45 lines · Julia Lawall
Quoted 43 lines · Michele Martone
Quoted 17 lines · Julia Lawall
Quoted 15 lines · Michele Martone

Dear Prof. Lawall and Coccinelle team,

I'm an enthusiastical, however still beginner use of Coccinelle.

While running spatch 1.0.6 on a long OpenMP-enabled listing, I noticed that the presence of the C line:
#pragma omp parallel for reduction(+:I) private(I)
seems to prevent restructuring of the function containing it. If I simplify the pragma to something like
#pragma omp parallel
the problem disappears.

Is there anything I can do to restructure large codes containing many of such similar lines ?

Quoted 6 lines
Quoted 1 line

In the attachment you find a test case and the output I experience.

Thanks for the report. We can try to make the parsing of #pragmas more flexible.

julia

Dear Julia,

thanks for the quick response!

I'll be waiting for Coccinelle-sided developments then.

May I ask what would you recommend now to circumvent this ?

From the SmPL v1.0.6 Grammar I am not aware of any way to e.g. ignore pragma lines as they were comments.

So the best what comes to my mind is to:

- comment #pragma lines
- compute semantic patch
- apply semantic patch
- uncomment #pragma lines

Quoted 5 lines
Quoted 1 line

If there is any better solution, please let me know.

The problem is now fixed in the github version of Coccinelle.

julia

That is good news -- thank you :-)

Handling such pragma lines opens new ranges of use for Coccinelle!

On my side, I just followed install.txt:

#!/bin/bash
set -e
set -x
trap "read" EXIT # will block for input on error git clone https://github.com/coccinelle/coccinelle.git
cd coccinelle
./autogen
./configure
make # it stops here

# make install

and it reaches:

...

OCAMLOPT  enter.ml
OCAMLC    main.mli
OCAMLOPT  main.ml
OCAMLOPT  -o spatch.opt

/usr/lib64/ocaml/libbigarray.a(bigarray_stubs.o): In function `caml_ba_deserialize':
/home/abuild/rpmbuild/BUILD/ocaml-4.03.0/otherlibs/bigarray/bigarray_stubs.c:979: undefined reference to caml_umul_overflow' /home/abuild/rpmbuild/BUILD/ocaml-4.03.0/otherlibs/bigarray/bigarray_stubs.c:985: undefined reference to caml_umul_overflow'
collect2: error: ld returned 1 exit status
File "caml_startup", line 1:
Error: Error during linking
Makefile:609: recipe for target 'spatch.opt' failed
make: *** [spatch.opt] Error 2
rm parsing_cocci/lexer_cli.ml parsing_cocci/parser_cocci_menhir.ml.d parsing_cocci/parser_cocci_menhir.mli.d parsing_cocci/lexer_script.ml parsing_cocci/lexer_cocci.ml parsing_c/lexer_c.ml
+ read

Quoted 6 lines
Quoted 2 lines

It's my first experience building any ocaml-based software, so I would appreciate any hint how to proceed further, if possible.

Oops, I'll forward this to my engineer, who knows about building issues. I'm not sure if he will answer before Wednesday, due to the holiday. What is your OS?

 $ cat /etc/os-release 
 NAME="openSUSE Leap"
 VERSION="42.3"
 ID=opensuse
 ID_LIKE="suse"
 VERSION_ID="42.3"
 PRETTY_NAME="openSUSE Leap 42.3"
 ANSI_COLOR="0;32"
 CPE_NAME="cpe:/o:opensuse:leap:42.3"

thanks!

#8 lore raw
In reply to Michele Martone

[...]

Quoted 18 lines

#!/bin/bash
set -e
set -x
trap "read" EXIT # will block for input on error git clone https://github.com/coccinelle/coccinelle.git
cd coccinelle
./autogen
./configure
make # it stops here

# make install

and it reaches:

...

OCAMLOPT  enter.ml
OCAMLC    main.mli
OCAMLOPT  main.ml
OCAMLOPT  -o spatch.opt
Quoted 10 lines

/usr/lib64/ocaml/libbigarray.a(bigarray_stubs.o): In function `caml_ba_deserialize':
/home/abuild/rpmbuild/BUILD/ocaml-4.03.0/otherlibs/bigarray/bigarray_stubs.c:979: undefined reference to caml_umul_overflow' /home/abuild/rpmbuild/BUILD/ocaml-4.03.0/otherlibs/bigarray/bigarray_stubs.c:985: undefined reference to caml_umul_overflow'
collect2: error: ld returned 1 exit status
File "caml_startup", line 1:
Error: Error during linking
Makefile:609: recipe for target 'spatch.opt' failed
make: *** [spatch.opt] Error 2
rm parsing_cocci/lexer_cli.ml parsing_cocci/parser_cocci_menhir.ml.d parsing_cocci/parser_cocci_menhir.mli.d parsing_cocci/lexer_script.ml parsing_cocci/lexer_cocci.ml parsing_c/lexer_c.ml
+ read

I usually ask the package manager to install build dependencies for me:

Fedora:

$ sudo dnf builddep coccinelle

Ubuntu/Debian:

$ sudo apt-get build-dep coccinelle

[...]

Signature · 1 line
Peter
#9 lore raw
Quoted 33 lines · Peter Senna Tschudin

[...]

Quoted 18 lines

#!/bin/bash
set -e
set -x
trap "read" EXIT # will block for input on error git clone https://github.com/coccinelle/coccinelle.git
cd coccinelle
./autogen
./configure
make # it stops here

# make install

and it reaches:

...

OCAMLOPT  enter.ml
OCAMLC    main.mli
OCAMLOPT  main.ml
OCAMLOPT  -o spatch.opt
Quoted 10 lines

/usr/lib64/ocaml/libbigarray.a(bigarray_stubs.o): In function `caml_ba_deserialize':
/home/abuild/rpmbuild/BUILD/ocaml-4.03.0/otherlibs/bigarray/bigarray_stubs.c:979: undefined reference to caml_umul_overflow' /home/abuild/rpmbuild/BUILD/ocaml-4.03.0/otherlibs/bigarray/bigarray_stubs.c:985: undefined reference to caml_umul_overflow'
collect2: error: ld returned 1 exit status
File "caml_startup", line 1:
Error: Error during linking
Makefile:609: recipe for target 'spatch.opt' failed
make: *** [spatch.opt] Error 2
rm parsing_cocci/lexer_cli.ml parsing_cocci/parser_cocci_menhir.ml.d parsing_cocci/parser_cocci_menhir.mli.d parsing_cocci/lexer_script.ml parsing_cocci/lexer_cocci.ml parsing_c/lexer_c.ml
+ read

I usually ask the package manager to install build dependencies for me:

Quoted 7 lines

Fedora:

$ sudo dnf builddep coccinelle

Ubuntu/Debian:

$ sudo apt-get build-dep coccinelle

[...]

Dear Peter, thanks for the apt-get based recipe, it worked for me on debian buster with ocaml-4.0.5.

Dear Julia, thanks for the pragma patch which will enable coccinelle to support OpenMP-enabled listings too (I just tested it on the above setup) :-)

Michele

#10 lore raw
Quoted 10 lines · Michele Martone
Quoted 1 line · Peter Senna Tschudin

...

Dear Peter, thanks for the apt-get based recipe, it worked for me on debian buster with ocaml-4.0.5.

Dear Julia, thanks for the pragma patch which will enable coccinelle to support OpenMP-enabled listings too (I just tested it on the above setup) :-)

Michele

Dear Julia,

I have another similar minimal test spatch case failing, attached. It seems like using
#pragma omp parallel for private(I)
breaks the semantic patch, while having
#pragma omp parallel for
is still ok.
Please see attachment.

I'm using self-built ocaml (e407ecf11 2018-05-25)
and coccinelle (a4532f08 2018-05-17).

would be very useful to have full #pragma resilience!

Michele
-------------- next part --------------
// seems like this example breaks:
// spatch version 1.0.6-00440-ga4532f08 compiled with OCaml version 4.08.0+dev0-2018-04-09 // (a4532f08 2018-05-17)

/*
pragma_ignored_again.cocci:

@@
symbol A,I,J;
@@

-A[I]
+A[J]
*/
void f(void)

{
	int*A=NULL,I=0;
// pragma presence breaks semantic patch (remote private(i) to 'fix' it)
// comment it to have A[I] -> A[J]
#pragma omp parallel for private(I)
	A[I]++;
}