Skip to contentSkip to conversation

Checking build dependencies for "pyml"

Cocci SF Markus Elfring 3 participants 19 messages Jul 7, 2016 – Jul 11, 2016

lore
Share & tools
lore
#0 lore raw

Hello,

I have tried to generate the current software also on my openSUSE Tumbleweed system.

elfring at Sonne:~/Projekte/Coccinelle/20160205> ./configure --enable-release --enable-ocaml && LANG=C make ?
make[2]: *** @MAKE_pyml@: No such file or directory. Stop.
?

Unfortunately, I stumble on an error message.

How is the development status after the commit "Fixed: Bundles compilation and pyml update" on 2016-06-26?
https://github.com/coccinelle/coccinelle/commit/62f2df8c7d35967a1eca4dcd487d59a9918dd5aa

Regards,
Markus

#1 lore raw

Hello, Markus.

Markus:

Quoted 1 line

make[2]: *** @MAKE_pyml@: No such file or directory. Stop.

Thank you for the report. Does this error continue to happen if you delete the subdirectory autom4te.cache/ and then execute ./autogen, ./configure, etc.?

Signature · 3 lines
Thierry.

SF Markus Elfring (2016/07/07, 23:40 MSK)?:
Quoted 23 lines

Hello,

I have tried to generate the current software also on my openSUSE Tumbleweed system.

elfring at Sonne:~/Projekte/Coccinelle/20160205> ./configure --enable-release --enable-ocaml && LANG=C make ?
make[2]: *** @MAKE_pyml@: No such file or directory. Stop.
?

Unfortunately, I stumble on an error message.

How is the development status after the commit "Fixed: Bundles compilation and pyml update" on 2016-06-26?
https://github.com/coccinelle/coccinelle/commit/62f2df8c7d35967a1eca4dcd487d59a9918dd5aa

Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci at systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

#2 lore raw

[Cocci] Checking build dependencies for "pyml" (and Menhir)

Quoted 5 lines
Quoted 1 line

make[2]: *** @MAKE_pyml@: No such file or directory. Stop.

Thank you for the report. Does this error continue to happen if you delete the subdirectory autom4te.cache/ and then execute ./autogen, ./configure, etc.?

Did a configuration script become outdated? -rwxr-xr-x 1 elfring users 524178 Mar 15 15:30 configure

It seems that the desired software build can be continued after another update.

elfring at Sonne:~/Projekte/Coccinelle/20160205> ./autogen && ./configure && LANG=C make
?
/usr/bin/ocamlc.opt -unsafe -I ../commons -I ../commons/ocamlextra -I ../globals -I /home/elfring/Projekte/Coccinelle/20160205/bundles/menhirLib/ -c parser_cocci_menhir.mli File "parser_cocci_menhir.mli", line 262, characters 10-56:
Error: Unbound module MenhirLib.IncrementalEngine
make[6]: *** [Makefile:89: parser_cocci_menhir.cmi] Error 2
?

How should such a build dependency be resolved?

Regards,
Markus

#3 lore raw

[Cocci] Checking build dependencies for "pyml" (and Menhir)

Markus:

Quoted 2 lines

Did a configuration script become outdated? -rwxr-xr-x 1 elfring users 524178 Mar 15 15:30 configure

Yes, the configure.ac script has been changed to handle the change from Pycaml to Py.ml.

Quoted 3 lines

/usr/bin/ocamlc.opt -unsafe -I ../commons -I ../commons/ocamlextra -I ../globals -I /home/elfring/Projekte/Coccinelle/20160205/bundles/menhirLib/ -c parser_cocci_menhir.mli File "parser_cocci_menhir.mli", line 262, characters 10-56:
Error: Unbound module MenhirLib.IncrementalEngine

Your parser_cocci_menhir.{ml,mli} have been generated with a version of Menhir more recent than the one that is bundled by Coccinelle.

You can just safely remove parsing_cocci/parser_cocci_menhir.{ml,mli} files (or just run make distclean) and try to run make again.

Signature · 3 lines
Thierry.

SF Markus Elfring (2016/07/08, 08:21 MSK)?:
Quoted 25 lines
Quoted 5 lines
Quoted 1 line

make[2]: *** @MAKE_pyml@: No such file or directory. Stop.

Thank you for the report. Does this error continue to happen if you delete the subdirectory autom4te.cache/ and then execute ./autogen, ./configure, etc.?

Did a configuration script become outdated? -rwxr-xr-x 1 elfring users 524178 Mar 15 15:30 configure

It seems that the desired software build can be continued after another update.

elfring at Sonne:~/Projekte/Coccinelle/20160205> ./autogen && ./configure && LANG=C make
?
/usr/bin/ocamlc.opt -unsafe -I ../commons -I ../commons/ocamlextra -I ../globals -I /home/elfring/Projekte/Coccinelle/20160205/bundles/menhirLib/ -c parser_cocci_menhir.mli File "parser_cocci_menhir.mli", line 262, characters 10-56:
Error: Unbound module MenhirLib.IncrementalEngine
make[6]: *** [Makefile:89: parser_cocci_menhir.cmi] Error 2
?

How should such a build dependency be resolved?

Regards,
Markus

#4 lore raw

[Cocci] Checking build dependencies for "pyml" (and Menhir)

Quoted 11 lines · Thierry Martinez

Markus:

Quoted 2 lines

Did a configuration script become outdated? -rwxr-xr-x 1 elfring users 524178 Mar 15 15:30 configure

Yes, the configure.ac script has been changed to handle the change from Pycaml to Py.ml.

Quoted 3 lines

/usr/bin/ocamlc.opt -unsafe -I ../commons -I ../commons/ocamlextra -I ../globals -I /home/elfring/Projekte/Coccinelle/20160205/bundles/menhirLib/ -c parser_cocci_menhir.mli File "parser_cocci_menhir.mli", line 262, characters 10-56:
Error: Unbound module MenhirLib.IncrementalEngine

Quoted 5 lines

Your parser_cocci_menhir.{ml,mli} have been generated with a version of Menhir more recent than the one that is bundled by Coccinelle.

You can just safely remove parsing_cocci/parser_cocci_menhir.{ml,mli} files (or just run make distclean) and try to run make again.

Probably it will be necessary to run autogen and configure again as well.

julia

Quoted 36 lines

--
Thierry.

SF Markus Elfring (2016/07/08, 08:21 MSK)?:

Quoted 21 lines
Quoted 5 lines
Quoted 1 line

make[2]: *** @MAKE_pyml@: No such file or directory. Stop.

Thank you for the report. Does this error continue to happen if you delete the subdirectory autom4te.cache/ and then execute ./autogen, ./configure, etc.?

Did a configuration script become outdated? -rwxr-xr-x 1 elfring users 524178 Mar 15 15:30 configure

It seems that the desired software build can be continued after another update.

elfring at Sonne:~/Projekte/Coccinelle/20160205> ./autogen && ./configure && LANG=C make
?
/usr/bin/ocamlc.opt -unsafe -I ../commons -I ../commons/ocamlextra -I ../globals -I /home/elfring/Projekte/Coccinelle/20160205/bundles/menhirLib/ -c parser_cocci_menhir.mli File "parser_cocci_menhir.mli", line 262, characters 10-56:
Error: Unbound module MenhirLib.IncrementalEngine
make[6]: *** [Makefile:89: parser_cocci_menhir.cmi] Error 2
?

Quoted 4 lines

How should such a build dependency be resolved?

Regards,
Markus

_______________________________________________
Cocci mailing list
Cocci at systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

#5 lore raw

[Cocci] Checking build dependencies for "pyml" (and Menhir)

Julia:

Quoted 1 line

Probably it will be necessary to run autogen and configure again as well.

In the case of make distclean, yes.
If the files parsing_cocci/parser_cocci_menhir.{ml,mli} are just removed, it should be fine to run make with the current Makefile.

Signature · 3 lines
Thierry.

Julia Lawall (2016/07/08, 13:12 MSK)?:
Quoted 59 lines · Thierry Martinez
Quoted 12 lines

Markus:

Quoted 2 lines

Did a configuration script become outdated? -rwxr-xr-x 1 elfring users 524178 Mar 15 15:30 configure

Yes, the configure.ac script has been changed to handle the change from Pycaml to Py.ml.

Quoted 3 lines

/usr/bin/ocamlc.opt -unsafe -I ../commons -I ../commons/ocamlextra -I ../globals -I /home/elfring/Projekte/Coccinelle/20160205/bundles/menhirLib/ -c parser_cocci_menhir.mli File "parser_cocci_menhir.mli", line 262, characters 10-56:
Error: Unbound module MenhirLib.IncrementalEngine

Quoted 5 lines

Your parser_cocci_menhir.{ml,mli} have been generated with a version of Menhir more recent than the one that is bundled by Coccinelle.

You can just safely remove parsing_cocci/parser_cocci_menhir.{ml,mli} files (or just run make distclean) and try to run make again.

Probably it will be necessary to run autogen and configure again as well.

julia

Quoted 36 lines

--
Thierry.

SF Markus Elfring (2016/07/08, 08:21 MSK):

Quoted 21 lines
Quoted 5 lines
Quoted 1 line

make[2]: *** @MAKE_pyml@: No such file or directory. Stop.

Thank you for the report. Does this error continue to happen if you delete the subdirectory autom4te.cache/ and then execute ./autogen, ./configure, etc.?

Did a configuration script become outdated? -rwxr-xr-x 1 elfring users 524178 Mar 15 15:30 configure

It seems that the desired software build can be continued after another update.

elfring at Sonne:~/Projekte/Coccinelle/20160205> ./autogen && ./configure && LANG=C make
?
/usr/bin/ocamlc.opt -unsafe -I ../commons -I ../commons/ocamlextra -I ../globals -I /home/elfring/Projekte/Coccinelle/20160205/bundles/menhirLib/ -c parser_cocci_menhir.mli File "parser_cocci_menhir.mli", line 262, characters 10-56:
Error: Unbound module MenhirLib.IncrementalEngine
make[6]: *** [Makefile:89: parser_cocci_menhir.cmi] Error 2
?

Quoted 4 lines

How should such a build dependency be resolved?

Regards,
Markus

_______________________________________________
Cocci mailing list
Cocci at systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

#6 lore raw
In reply to Thierry Martinez
Quoted 2 lines

Yes, the configure.ac script has been changed to handle the change from Pycaml to Py.ml.

Has such a software evolution got any influence on the Python application programming interface by the semantic patch language?

Will the mapping (and corresponding data type conversion) will need another look for integer values in this use case?

Regards,
Markus

#7 lore raw

Markus:

Quoted 2 lines

Has such a software evolution got any influence on the Python application programming interface by the semantic patch language?

The same ./spatch executable can now be run in a Python 2 and Python 3 environment (before, with Pycaml, Coccinelle had to be compiled in a Python 2 environment to work with Python 2, and to be compiled in a Python 3 environment to work with Python 3).

Python error handling is better as well, and the Python interface is now only initialized when needed.

Quoted 2 lines

Will the mapping (and corresponding data type conversion) will need another look for integer values in this use case?

The mapping is unchanged. But the better error handling should help to debug scripts.

Signature · 3 lines
Thierry.

SF Markus Elfring (2016/07/08, 14:14 MSK)?:
Quoted 11 lines
Quoted 2 lines

Yes, the configure.ac script has been changed to handle the change from Pycaml to Py.ml.

Has such a software evolution got any influence on the Python application programming interface by the semantic patch language?

Will the mapping (and corresponding data type conversion) will need another look for integer values in this use case?

Regards,
Markus

#8 lore raw
Quoted 4 lines
Quoted 2 lines

Will the mapping (and corresponding data type conversion) will need another look for integer values in this use case?

The mapping is unchanged.

How do you think about to achieve that integers will be preserved (without conversion to strings) here?

Regards,
Markus

#9 lore raw
Quoted 7 lines · SF Markus Elfring
Quoted 4 lines
Quoted 2 lines

Will the mapping (and corresponding data type conversion) will need another look for integer values in this use case?

The mapping is unchanged.

How do you think about to achieve that integers will be preserved (without conversion to strings) here?

Could you explain what exactly you are talking about, perhaps with a small example? There are things like line numbers that are converted to strings. There is no technical reason for that, but the person who first implemented the python interface did that, and then it didn't seem like big enough of a deal to break backward compatability. But maybe you are referring to somrething else.

julia

#10 lore raw
Quoted 1 line

There are things like line numbers that are converted to strings.

I am affected by such an implementation detail in some SmPL scripts.

Quoted 3 lines

There is no technical reason for that, but the person who first implemented the python interface did that, and then it didn't seem like big enough of a deal to break backward compatability.

Do any more software developers care for this aspect?

Quoted 1 line

But maybe you are referring to somrething else.

I guess not. - I am just unsure if this will matter at more source code places.

Regards,
Markus

#11 lore raw

[Cocci] Checking build dependencies for "Menhir"

In reply to Thierry Martinez
Quoted 6 lines
Quoted 3 lines

/usr/bin/ocamlc.opt -unsafe -I ../commons -I ../commons/ocamlextra -I ../globals -I /home/elfring/Projekte/Coccinelle/20160205/bundles/menhirLib/ -c parser_cocci_menhir.mli File "parser_cocci_menhir.mli", line 262, characters 10-56:
Error: Unbound module MenhirLib.IncrementalEngine

Your parser_cocci_menhir.{ml,mli} have been generated with a version of Menhir more recent than the one that is bundled by Coccinelle.

Thanks for your explanation.

Is the bundled Menhir version currently required for a working Coccinelle software?

Quoted 2 lines

You can just safely remove parsing_cocci/parser_cocci_menhir.{ml,mli} files (or just run make distclean) and try to run make again.

I imagine that there are further challenges to consider for the safe dependency resolution of involved libraries.
* Which releases can be picked up and finally used from special directories?
* Will any more fine-tuning be needed for corresponding directory selection?

Regards,
Markus

#12 lore raw

[Cocci] Checking build dependencies for "Menhir"

Markus:

Quoted 2 lines

Is the bundled Menhir version currently required for a working Coccinelle software?

No, you can use a more recent version of Menhir if you want.

The reason of having an old version of Menhir in the bundles is that the newer versions of Menhir require OCaml 4.02 (for "match exception"), and we would like to keep the compatibility with OCaml 4.01 (or even 3.12).

Quoted 5 lines

I imagine that there are further challenges to consider for the safe dependency resolution of involved libraries.
* Which releases can be picked up and finally used from special directories?
* Will any more fine-tuning be needed for corresponding directory selection?

I don?t know if dependencies can be made more precise between the Coccinelle code and bundled libraries.

In the current settings, you may have to run "make clean" or even "make distclean" after a "git pull" to get a successful build (or else, there is effectively a bug). I think it is not too demanding, but I agree that we should write more clearly in the commit message when cleaning is required.

Signature · 3 lines
Thierry.

SF Markus Elfring (2016/07/08, 14:32 MSK)?:
Quoted 22 lines
Quoted 6 lines
Quoted 3 lines

/usr/bin/ocamlc.opt -unsafe -I ../commons -I ../commons/ocamlextra -I ../globals -I /home/elfring/Projekte/Coccinelle/20160205/bundles/menhirLib/ -c parser_cocci_menhir.mli File "parser_cocci_menhir.mli", line 262, characters 10-56:
Error: Unbound module MenhirLib.IncrementalEngine

Your parser_cocci_menhir.{ml,mli} have been generated with a version of Menhir more recent than the one that is bundled by Coccinelle.

Thanks for your explanation.

Is the bundled Menhir version currently required for a working Coccinelle software?

Quoted 2 lines

You can just safely remove parsing_cocci/parser_cocci_menhir.{ml,mli} files (or just run make distclean) and try to run make again.

I imagine that there are further challenges to consider for the safe dependency resolution of involved libraries.
* Which releases can be picked up and finally used from special directories?
* Will any more fine-tuning be needed for corresponding directory selection?

Regards,
Markus

#13 lore raw

[Cocci] Checking build dependencies for "Menhir"

Quoted 4 lines
Quoted 2 lines

Is the bundled Menhir version currently required for a working Coccinelle software?

No, you can use a more recent version of Menhir if you want.

I would appreciate that.

I am struggling to complete the needed dependencies for current releases of various OCaml software.

Quoted 3 lines

The reason of having an old version of Menhir in the bundles is that the newer versions of Menhir require OCaml 4.02 (for "match exception"), and we would like to keep the compatibility with OCaml 4.01 (or even 3.12).

Do such constraints need any more configuration checks in the build scripts?

Quoted 1 line

I don?t know if dependencies can be made more precise

I guess that it is generally possible to improve the build system safety.

Quoted 1 line

between the Coccinelle code and bundled libraries.

I am unsure on the efforts that we would like to invest here.

Quoted 2 lines

I think it is not too demanding, but I agree that we should write more clearly in the commit message when cleaning is required.

I hope that such an extra build script clean-up should not be needed usually.

Regards,
Markus

#14 lore raw

[Cocci] Checking build dependencies for "Menhir"

In reply to Thierry Martinez
Quoted 4 lines
Quoted 2 lines

Is the bundled Menhir version currently required for a working Coccinelle software?

No, you can use a more recent version of Menhir if you want.

?
checking for menhir... /usr/local/bin/menhir
configure: system menhir will be used
?
File "parser_cocci_menhir.mli", line 262, characters 10-56:
Error: Unbound module MenhirLib.IncrementalEngine make[6]: *** [Makefile:89: parser_cocci_menhir.cmi] Error 2 ?

A command like "make distclean" does not help so far.

Will there any more checks and settings be needed to get the desired build working together with the software "OCaml 4.03.0-1.1"?

Regards,
Markus

#15 lore raw

[Cocci] Checking build dependencies for "Menhir"

In reply to Thierry Martinez
Quoted 1 line

No, you can use a more recent version of Menhir if you want.

The following command succeeds.

elfring at Sonne:~/Projekte/Coccinelle/20160205/parsing_cocci> /usr/bin/ocamlc.opt -unsafe -I ../commons -I ../commons/ocamlextra -I ../globals -I /usr/local/share/menhir -c parser_cocci_menhir.mli

Now I wonder why the directory "bundles" was passed for the corresponding include parameter after the "system tool" was selected by the configuration script.
How should this detail be fixed?
https://github.com/coccinelle/coccinelle/blob/25c0b3278dff5e1ec4928724ff29ce977acc256b/configure.ac#L177

Regards,
Markus

#16 lore raw

[Cocci] Checking build dependencies for "Menhir"

I have gathered a bit more background information.

AC_REQ_COCCI_EXTPKG([menhirLib])
? AC_COCCI_OCAMLPKG()
? AC_CHECK_OCAML_PKG()
? $OCAMLFIND query

?
configure: configuring package menhirLib checking for OCaml findlib package menhirLib... not found ?

The mentioned steps from m4 scripts generated such a display after the command "make PREFIX=/usr/local install" succeeded for the software "Menhir 20160526". It showed also the message "Installing MenhirLib manually." which indicates a detail that I find strange at the moment.
I would appreciate a display of the message "Installing MenhirLib via ocamlfind." instead. Would you like to help to get a command "src/_stage2/menhir.native --suggest-ocamlfind" working in this direction?

Regards,
Markus

#17 lore raw

[Cocci] Checking build dependencies for "Menhir"

Markus:

Quoted 3 lines

Would you like to help to get
a command "src/_stage2/menhir.native --suggest-ocamlfind" working in this direction?

Thank you for your report. The last GitHub update fixes the problem you noticed and follows your suggestions.

menhir and menhirLib were detected independently, and the detection of menhirLib required ocamlfind: so, if menhir was installed on your system but if you didn't have ocamlfind, Coccinelle used the system menhir but the bundled menhirLib, hence the failure your noticed in case of version mismatch.

Now, the menhirLib is detected via menhir --suggest-menhirLib if available (this option is only available in recent versions of menhir), or via ocamlfind if available. Otherwise, the bundled versions of menhir and menhirLib are used, regardless of what is installed.

Signature · 3 lines
Thierry.

SF Markus Elfring (2016/07/09, 07:45 CEST)?:
Quoted 25 lines

I have gathered a bit more background information.

AC_REQ_COCCI_EXTPKG([menhirLib])
? AC_COCCI_OCAMLPKG()
? AC_CHECK_OCAML_PKG()
? $OCAMLFIND query

?
configure: configuring package menhirLib checking for OCaml findlib package menhirLib... not found ?

The mentioned steps from m4 scripts generated such a display after the command "make PREFIX=/usr/local install" succeeded for the software "Menhir 20160526". It showed also the message "Installing MenhirLib manually." which indicates a detail that I find strange at the moment.
I would appreciate a display of the message "Installing MenhirLib via ocamlfind." instead. Would you like to help to get a command "src/_stage2/menhir.native --suggest-ocamlfind" working in this direction?

Regards,
Markus

#18 lore raw

[Cocci] Checking build dependencies for "Menhir"

Quoted 3 lines

menhir and menhirLib were detected independently, and the detection of menhirLib required ocamlfind: so, if menhir was installed on your system but if you didn't have ocamlfind,

This software tool was available in my use case.

Quoted 2 lines

Coccinelle used the system menhir but the bundled menhirLib, hence the failure your noticed in case of version mismatch.

Did you consider also another implementation detail already?

* How do you think about the relevance of a make command parameter or setting in the environment like "USE_OCAMLFIND=1"?

* Fran?ois Pottier is thinking about improvements for the document "INSTALLATION".

Regards,
Markus