New: python-pyparsing: "No module named 'pyparsing'" on target / while building cantools

https://bugs.busybox.net/show_bug.cgi?id=11831

Bug ID: 11831
Summary: python-pyparsing: "No module named 'pyparsing'" on
target / while building cantools
Product: buildroot
Version: 2019.02.2
Hardware: All
OS: Linux
Status: NEW
Severity: normal
Priority: P5
Component: Other
Assignee: unassigned at buildroot.uclibc.org
Reporter: tistolz at outlook.de
CC: buildroot at uclibc.org
Target Milestone: ---

Created attachment 7991
--> https://bugs.busybox.net/attachment.cgi?id=7991&action=edit
python-cantools/Config.in

Hi all,

I'm trying to include the Python module "cantools"
<https://pypi.org/project/cantools/> into my Buildroot installation. The package files are given in the attachment. However, the build step for python-cantools fails with the message:
ModuleNotFoundError: No module named 'pyparsing'
in a nested import while processing the first line (import cantools) of setup.py.
I can confirm that the python-pyparsing package has been built before, so it should be available to python.

Trying to track this bug down, I first started output/host/bin/python3 and entered the command "import pyparsing", upon which I saw the same error message. However, another package I built was python-bitstruct (using utils/scanpypi), and this became available for "import bitstruct" in an output/host/bin/python3 session.

Furthermore, I rebuilt Buildroot 2019.02.2 (from a freshly extracted tar archive) with only the following packages selected:
Interpreter languages and scripting --> python3 SELECTED
core python3 modules: DESELECTED unicodedata External python modules: SELECTED python-can, python-spidev,
python-pyparsing

On the target, in a python3 session, I get:

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'pyparsing' >>> import spidev
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'spidev' >>>

So, the "python-can" package seems to install correctly, while "python-pyparsing" and "python-spidev" appear to be not present on the target.

Thanks for your attention,
Tibor Stolz

You are receiving this mail because:
You are on the CC list for the bug.

https://bugs.busybox.net/show_bug.cgi?id=11831

--- Comment #1 from Tibor Stolz <[email protected]> --- Created attachment 7996
--> https://bugs.busybox.net/attachment.cgi?id=7996&action=edit
python-cantools/python-cantools.hash

You are receiving this mail because:
You are on the CC list for the bug.

https://bugs.busybox.net/show_bug.cgi?id=11831

--- Comment #2 from Tibor Stolz <[email protected]> --- Created attachment 8001
--> https://bugs.busybox.net/attachment.cgi?id=8001&action=edit
python-cantools/python-cantools.mk

You are receiving this mail because:
You are on the CC list for the bug.

https://bugs.busybox.net/show_bug.cgi?id=11831

Tibor Stolz <[email protected]> changed:

What |Removed |Added
----------------------------------------------------------------------------

   Attachment #7991|0                           |1

is obsolete| |

--- Comment #3 from Tibor Stolz <[email protected]> --- Created attachment 8006
--> https://bugs.busybox.net/attachment.cgi?id=8006&action=edit
python-cantools/Config.in

same file, but set content type now correctly

You are receiving this mail because:
You are on the CC list for the bug.

https://bugs.busybox.net/show_bug.cgi?id=11831

Tibor Stolz <[email protected]> changed:

What |Removed |Added
----------------------------------------------------------------------------

   Attachment #7996|0                           |1

is obsolete| |

--- Comment #4 from Tibor Stolz <[email protected]> --- Created attachment 8011
--> https://bugs.busybox.net/attachment.cgi?id=8011&action=edit
python-cantools/python-cantools.hash

same file, but content type now set correctly

You are receiving this mail because:
You are on the CC list for the bug.

https://bugs.busybox.net/show_bug.cgi?id=11831

Tibor Stolz <[email protected]> changed:

What |Removed |Added
----------------------------------------------------------------------------

   Attachment #8001|0                           |1

is obsolete| |

--- Comment #5 from Tibor Stolz <[email protected]> --- Created attachment 8016
--> https://bugs.busybox.net/attachment.cgi?id=8016&action=edit
python-cantools/python-cantools.mk

same file, but content type set correctly

You are receiving this mail because:
You are on the CC list for the bug.

https://bugs.busybox.net/show_bug.cgi?id=11831

--- Comment #6 from Tibor Stolz <[email protected]> --- The build log for python-cantools in detail:

cantools-18.0.1.tar.gz: OK (sha256:
dc5d265103b8b11d4c7cc21996ad1954325b1f1930837505cabb0b5b137d37ee)

gzip -d -c /home/tibor/Buildroot/buildroot-2019.02.2/dl/python-cantools/cantools-18.0.1.tar.gz | tar --strip-components=1 -C /home/tibor/Buildroot/buildroot-2019.02.2/output/build/python-cantools-18.0.1 -xf -

(cd
/home/tibor/Buildroot/buildroot-2019.02.2/output/build/python-cantools-18.0.1//;
_PYTHON_SYSCONFIGDATA_NAME="_sysconfigdata_m_linux_arm-linux-gnueabihf" PATH="/home/tibor/Buildroot/buildroot-2019.02.2/output/host/bin:/home/tibor/Buildroot/buildroot-2019.02.2/output/host/sbin:/home/tibor/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin" PYTHONPATH="/home/tibor/Buildroot/buildroot-2019.02.2/output/target/usr/lib/python3.7/"
PYTHONNOUSERSITE=1
_python_sysroot=/home/tibor/Buildroot/buildroot-2019.02.2/output/host/arm-buildroot-linux-uclibcgnueabihf/sysroot
_python_prefix=/usr _python_exec_prefix=/usr
/home/tibor/Buildroot/buildroot-2019.02.2/output/host/bin/python setup.py build
)
Traceback (most recent call last):
File "setup.py", line 4, in <module>
import cantools
File
"/home/tibor/Buildroot/buildroot-2019.02.2/output/build/python-cantools-18.0.1/cantools/__init__.py",
line 6, in <module>
from . import db
File
"/home/tibor/Buildroot/buildroot-2019.02.2/output/build/python-cantools-18.0.1/cantools/db/__init__.py",
line 4, in <module>
from .database import Database
File
"/home/tibor/Buildroot/buildroot-2019.02.2/output/build/python-cantools-18.0.1/cantools/db/database.py",
line 3, in <module>
from .formats import dbc
File
"/home/tibor/Buildroot/buildroot-2019.02.2/output/build/python-cantools-18.0.1/cantools/db/formats/dbc.py",
line 4, in <module>
from pyparsing import Word
ModuleNotFoundError: No module named 'pyparsing'
package/pkg-generic.mk:238: recipe for target
'/home/tibor/Buildroot/buildroot-2019.02.2/output/build/python-cantools-18.0.1/.stamp_built'
failed
make[1]: ***
[/home/tibor/Buildroot/buildroot-2019.02.2/output/build/python-cantools-18.0.1/.stamp_built]
Error 1
Makefile:84: recipe for target '_all' failed
make: *** [_all] Error 2

You are receiving this mail because:
You are on the CC list for the bug.

https://bugs.busybox.net/show_bug.cgi?id=11831

--- Comment #7 from Tibor Stolz <[email protected]> --- Created attachment 8026
--> https://bugs.busybox.net/attachment.cgi?id=8026&action=edit
python_ok_bbb_defconfig

I'm sorry I cannot repeat the problem anymore. Using exactly the same buildroot version and menu settings (except "System hostname" and "System banner"), I get packages python-pyparsing and python-spidev that work fine and can be imported into a python3 session without any issue.

For your reference, I'm attaching the main config file, created using "make savedefconfig".

I'm closing the bug until I hit that behaviour again. (I hope not...)

You are receiving this mail because:
You are on the CC list for the bug.

https://bugs.busybox.net/show_bug.cgi?id=11831

Tibor Stolz <[email protected]> changed:

What |Removed |Added
----------------------------------------------------------------------------

         Resolution|---                         |WORKSFORME
           Assignee|unassigned at buildroot.uclibc |tistolz at outlook.de
                   |.org                        |
             Status|NEW                         |RESOLVED

You are receiving this mail because:
You are on the CC list for the bug.

https://bugs.busybox.net/show_bug.cgi?id=11831

--- Comment #8 from Tibor Stolz <[email protected]> --- Created attachment 8046
--> https://bugs.busybox.net/attachment.cgi?id=8046&action=edit
FIXED python-cantools/python-cantools.mk

Hi all,

The build error for python-cantools persisted, but now I figured out why I ran into it: In order to build correctly, the build-time dependencies must be installed on the *host*! Therefore, the package makefile (python-cantools.mk) must be changed:

- PYTHON_CANTOOLS_DEPENDENCIES = python-bitstruct python-pyparsing
+ PYTHON_CANTOOLS_DEPENDENCIES = host-python-bitstruct host-python-pyparsing

Doing a 'grep -r DEPENDENCIES package/python-*' in the buildroot-2019.02.2 directory revealed that most python library dependencies are in the host-python-* form. Maybe this should be mentioned in the Buildroot user manual?

Best regards,
Tibor Stolz

You are receiving this mail because:
You are on the CC list for the bug.

https://bugs.busybox.net/show_bug.cgi?id=11831

--- Comment #9 from Tibor Stolz <[email protected]> --- Created attachment 8051
--> https://bugs.busybox.net/attachment.cgi?id=8051&action=edit
FIXED python-cantools/Config.in

For anyone interested in building python-cantools v18.0.1 (16 Feb 2018), here is the Config.in file.

You also need packages python-bitstruct and python-pyparsing, which contain the following line at the end of the .mk file:

    $(eval $(host-python-package))

Besides that, the python-bitstruct package is easy to get with utils/scanpypi, and the python-pyparsing package is already available (with host-python-package missing).

You are receiving this mail because:
You are on the CC list for the bug.

https://bugs.busybox.net/show_bug.cgi?id=11831

Tibor Stolz <[email protected]> changed:

What |Removed |Added
----------------------------------------------------------------------------

            Summary|python-pyparsing: "No       |python-pyparsing: "No
                   |module named 'pyparsing'"   |module named 'pyparsing'"
                   |on target / while building  |while building cantools
                   |cantools                    |(SOLVED: use
                   |                            |host-python-pyparsing)

You are receiving this mail because:
You are on the CC list for the bug.