python3 cryptic build failures
Hi Trevor,
I thought this issue might be something you might be able to spot the solution with quickly.
I had a failing python3-iniconfig build in a TMPDIR I've reused heavily:
ERROR: python3-iniconfig-2.0.0-r0 do_compile: ExecutionError('/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/temp/run.do_compile.2391637', 1, None, None)
ERROR: Logfile of failure stored in: /media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/temp/log.do_compile.2391637
Log data follows:
| DEBUG: Executing shell function do_compile
| * Getting build dependencies for wheel...
|
| Traceback (most recent call last):
| File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", line 375, in main
| built = build_call(
| ^^^^^^^^^^^
| File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", line 208, in build_package
| out = _build(isolation, builder, outdir, distribution, config_settings, skip_dependency_check)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", line 147, in _build
| return _build_in_current_env(builder, outdir, distribution, config_settings, skip_dependency_check)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", line 127, in _build_in_current_env
| missing = builder.check_dependencies(distribution)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", line 369, in check_dependencies
| return {u for d in dependencies for u in check_dependency(d)}
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", line 369, in <setcomp>
| return {u for d in dependencies for u in check_dependency(d)}
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", line 185, in check_dependency
| yield from check_dependency(other_req_string, ancestral_req_strings + (normalised_req_string,), req.extras)
| File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", line 179, in check_dependency
| if req.specifier and not req.specifier.contains(dist.version, prereleases=True):
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/packaging/specifiers.py", line 905, in contains
| item = Version(item)
| ^^^^^^^^^^^^^
| File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/packaging/version.py", line 196, in __init__
| match = self._regex.search(version)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^
| TypeError: expected string or bytes-like object, got 'NoneType'
|
| ERROR expected string or bytes-like object, got 'NoneType'
| WARNING: exit code 1 from a shell command.ERROR: Task (/media/build/poky/meta/recipes-devtools/python/python3-iniconfig_2.0.0.bb:do_compile) failed with exit code '1'
I wondered why and I isolated it down to this:
richard@jet:/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages$ find -type d -empty ./hatchling-1.13.0.dist-info/licenses ./pathspec-0.11.0.dist-info ./importlib_metadata-6.0.0.dist-info ./setuptools-67.4.0.dist-info ./wheel-0.38.4.dist-info ./installer-0.6.0.dist-info ./setuptools-67.6.0.dist-info ./packaging-23.0.dist-info richard@jet:/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages$ find -type d -empty -delete
After which the compile succeeds. These empty dist-info directories really upset the build. They're left behind by the sstate cleanup code from when I have older versions of these python modules in play since the sstate code can race over directory removal. Is there a way to make python ignore them? Otherwise should we remove them somewhere?
I suspect you can recreate by adding these empty directories and running a -c compile -f. I've seen it with plenty of other python modules too.
Cheers,
Richard
Quoted 8 lines · Richard Purdie
Hi Trevor,
I thought this issue might be something you might be able to spot the solution with quickly.
I had a failing python3-iniconfig build in a TMPDIR I've reused heavily:
FWIW. I see this in about 6 or 7 python recipes in my extremely heavily reused tmpdir.
After I came back from vacation, I iterated through a -c clean on the python packages, and they always built the second time through.
Bruce
Quoted 80 lines
ERROR: python3-iniconfig-2.0.0-r0
do_compile: ExecutionError('/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/temp/run.do_compile.2391637', 1, None, None)
ERROR: Logfile of failure stored in: /media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/temp/log.do_compile.2391637
Log data follows:| DEBUG: Executing shell function do_compile | * Getting build dependencies for wheel... | | Traceback (most recent call last): | File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", line 375, in main | built = build_call( | ^^^^^^^^^^^ | File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", line 208, in build_package | out = _build(isolation, builder, outdir, distribution, config_settings, skip_dependency_check) | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", line 147, in _build | return _build_in_current_env(builder, outdir, distribution, config_settings, skip_dependency_check) | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", line 127, in _build_in_current_env | missing = builder.check_dependencies(distribution) | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", line 369, in check_dependencies | return {u for d in dependencies for u in check_dependency(d)} | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", line 369, in <setcomp> | return {u for d in dependencies for u in check_dependency(d)} | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", line 185, in check_dependency | yield from check_dependency(other_req_string, ancestral_req_strings + (normalised_req_string,), req.extras) | File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", line 179, in check_dependency | if req.specifier and not req.specifier.contains(dist.version, prereleases=True): | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/packaging/specifiers.py", line 905, in contains | item = Version(item) | ^^^^^^^^^^^^^ | File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/packaging/version.py", line 196, in __init__ | match = self._regex.search(version) | ^^^^^^^^^^^^^^^^^^^^^^^^^^^ | TypeError: expected string or bytes-like object, got 'NoneType' | | ERROR expected string or bytes-like object, got 'NoneType' | WARNING: exit code 1 from a shell command.ERROR: Task (/media/build/poky/meta/recipes-devtools/python/python3-iniconfig_2.0.0.bb:do_compile) failed with exit code '1'
I wondered why and I isolated it down to this:
richard@jet:/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages$ find -type d -empty ./hatchling-1.13.0.dist-info/licenses ./pathspec-0.11.0.dist-info ./importlib_metadata-6.0.0.dist-info ./setuptools-67.4.0.dist-info ./wheel-0.38.4.dist-info ./installer-0.6.0.dist-info ./setuptools-67.6.0.dist-info ./packaging-23.0.dist-info richard@jet:/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages$ find -type d -empty -deleteAfter which the compile succeeds. These empty dist-info directories really upset the build. They're left behind by the sstate cleanup code from when I have older versions of these python modules in play since the sstate code can race over directory removal. Is there a way to make python ignore them? Otherwise should we remove them somewhere?
I suspect you can recreate by adding these empty directories and running a -c compile -f. I've seen it with plenty of other python modules too.
Cheers,
Richard
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#182010): https://lists.openembedded.org/g/openembedded-core/message/182010
Mute This Topic: https://lists.openembedded.org/mt/99241972/1050810
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Signature · 3 lines
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end - "Use the force Harry" - Gandalf, Star Trek II
Quoted 7 lines · Bruce Ashfield
Quoted 5 lines · Richard Purdie
Hi Trevor,
I thought this issue might be something you might be able to spot the solution with quickly.
Quoted 7 lines
Quoted 2 lines
I had a failing python3-iniconfig build in a TMPDIR I've reused heavily:
FWIW. I see this in about 6 or 7 python recipes in my extremely heavily reused tmpdir.
After I came back from vacation, I iterated through a -c clean on the python packages, and they always built the second time through.
Do you recall which those are, and if it's the same ones each time it happens? I'm wondering if it may be an issue with some recipes.
I haven't been able to recreate the issue yet, but my tmpdir is not very old. When I attempt the suggested reproducer (creating the empty directories and rerunning do_compile), it is successful. I'll keep poking at it.
Trevor
Quoted 83 lines
Bruce
Quoted 79 lines
ERROR: python3-iniconfig-2.0.0-r0
do_compile: ExecutionError('/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/temp/run.do_compile.2391637', 1, None, None)
ERROR: Logfile of failure stored in: /media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/temp/log.do_compile.2391637
Log data follows:| DEBUG: Executing shell function do_compile | * Getting build dependencies for wheel... | | Traceback (most recent call last): | File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", line 375, in main | built = build_call( | ^^^^^^^^^^^ | File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", line 208, in build_package | out = _build(isolation, builder, outdir, distribution, config_settings, skip_dependency_check) | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", line 147, in _build | return _build_in_current_env(builder, outdir, distribution, config_settings, skip_dependency_check) | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", line 127, in _build_in_current_env | missing = builder.check_dependencies(distribution) | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", line 369, in check_dependencies | return {u for d in dependencies for u in check_dependency(d)} | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", line 369, in <setcomp> | return {u for d in dependencies for u in check_dependency(d)} | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", line 185, in check_dependency | yield from check_dependency(other_req_string, ancestral_req_strings + (normalised_req_string,), req.extras) | File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", line 179, in check_dependency | if req.specifier and not req.specifier.contains(dist.version, prereleases=True): | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/packaging/specifiers.py", line 905, in contains | item = Version(item) | ^^^^^^^^^^^^^ | File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/packaging/version.py", line 196, in __init__ | match = self._regex.search(version) | ^^^^^^^^^^^^^^^^^^^^^^^^^^^ | TypeError: expected string or bytes-like object, got 'NoneType' | | ERROR expected string or bytes-like object, got 'NoneType' | WARNING: exit code 1 from a shell command.ERROR: Task (/media/build/poky/meta/recipes-devtools/python/python3-iniconfig_2.0.0.bb:do_compile) failed with exit code '1'
I wondered why and I isolated it down to this:
richard@jet:/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages$ find -type d -empty ./hatchling-1.13.0.dist-info/licenses ./pathspec-0.11.0.dist-info ./importlib_metadata-6.0.0.dist-info ./setuptools-67.4.0.dist-info ./wheel-0.38.4.dist-info ./installer-0.6.0.dist-info ./setuptools-67.6.0.dist-info ./packaging-23.0.dist-info richard@jet:/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages$ find -type d -empty -deleteAfter which the compile succeeds. These empty dist-info directories really upset the build. They're left behind by the sstate cleanup code from when I have older versions of these python modules in play since the sstate code can race over directory removal. Is there a way to make python ignore them? Otherwise should we remove them somewhere?
I suspect you can recreate by adding these empty directories and running a -c compile -f. I've seen it with plenty of other python modules too.
Cheers,
Richard
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#182010): https://lists.openembedded.org/g/openembedded-core/message/182010
Mute This Topic: https://lists.openembedded.org/mt/99241972/1050810
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Quoted 12 lines · Trevor Gamblin
Quoted 9 lines · Bruce Ashfield
Quoted 7 lines · Richard Purdie
Hi Trevor,
I thought this issue might be something you might be able to spot the solution with quickly.
I had a failing python3-iniconfig build in a TMPDIR I've reused heavily:
Quoted 9 lines
Quoted 5 lines
FWIW. I see this in about 6 or 7 python recipes in my extremely heavily reused tmpdir.
After I came back from vacation, I iterated through a -c clean on the python packages, and they always built the second time through.
Do you recall which those are, and if it's the same ones each time it happens? I'm wondering if it may be an issue with some recipes.
roughly this set (as grepped from my history):
2783 bitbake -c clean python3-attrs; bitbake core-image-minimal 2828 bitbake -c clean python3-colorama; bitbake core-image-minimal 2829 bitbake -c clean python3-jsonschema; bitbake core-image-minimal 2943 bitbake -c clean python3-hatch-vcs python3-hatch-vcs-native; bitbake core-image-minimal 2944 bitbake -c clean python3-attrs; bitbake core-image-minimal 2945 bitbake -c clean python3-urllib3; bitbake core-image-minimal 2946 bitbake -c clean python3-colorama; bitbake core-image-minimal 2947 bitbake -c clean python3-jsonschema; bitbake core-image-minimal 2948 bitbake -c clean python3-iniconfig; bitbake core-image-minimal
Apparently I had particular issues with colorama :)
Bruce
Quoted 90 lines
I haven't been able to recreate the issue yet, but my tmpdir is not very old. When I attempt the suggested reproducer (creating the empty directories and rerunning
do_compile), it is successful. I'll keep poking at it.Trevor
Quoted 83 lines
Bruce
Quoted 79 lines
ERROR: python3-iniconfig-2.0.0-r0
do_compile: ExecutionError('/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/temp/run.do_compile.2391637', 1, None, None)
ERROR: Logfile of failure stored in: /media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/temp/log.do_compile.2391637
Log data follows:| DEBUG: Executing shell function do_compile | * Getting build dependencies for wheel... | | Traceback (most recent call last): | File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", line 375, in main | built = build_call( | ^^^^^^^^^^^ | File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", line 208, in build_package | out = _build(isolation, builder, outdir, distribution, config_settings, skip_dependency_check) | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", line 147, in _build | return _build_in_current_env(builder, outdir, distribution, config_settings, skip_dependency_check) | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", line 127, in _build_in_current_env | missing = builder.check_dependencies(distribution) | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", line 369, in check_dependencies | return {u for d in dependencies for u in check_dependency(d)} | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", line 369, in <setcomp> | return {u for d in dependencies for u in check_dependency(d)} | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", line 185, in check_dependency | yield from check_dependency(other_req_string, ancestral_req_strings + (normalised_req_string,), req.extras) | File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", line 179, in check_dependency | if req.specifier and not req.specifier.contains(dist.version, prereleases=True): | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/packaging/specifiers.py", line 905, in contains | item = Version(item) | ^^^^^^^^^^^^^ | File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/packaging/version.py", line 196, in __init__ | match = self._regex.search(version) | ^^^^^^^^^^^^^^^^^^^^^^^^^^^ | TypeError: expected string or bytes-like object, got 'NoneType' | | ERROR expected string or bytes-like object, got 'NoneType' | WARNING: exit code 1 from a shell command.ERROR: Task (/media/build/poky/meta/recipes-devtools/python/python3-iniconfig_2.0.0.bb:do_compile) failed with exit code '1'
I wondered why and I isolated it down to this:
richard@jet:/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages$ find -type d -empty ./hatchling-1.13.0.dist-info/licenses ./pathspec-0.11.0.dist-info ./importlib_metadata-6.0.0.dist-info ./setuptools-67.4.0.dist-info ./wheel-0.38.4.dist-info ./installer-0.6.0.dist-info ./setuptools-67.6.0.dist-info ./packaging-23.0.dist-info richard@jet:/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages$ find -type d -empty -deleteAfter which the compile succeeds. These empty dist-info directories really upset the build. They're left behind by the sstate cleanup code from when I have older versions of these python modules in play since the sstate code can race over directory removal. Is there a way to make python ignore them? Otherwise should we remove them somewhere?
I suspect you can recreate by adding these empty directories and running a -c compile -f. I've seen it with plenty of other python modules too.
Cheers,
Richard
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#182010): https://lists.openembedded.org/g/openembedded-core/message/182010
Mute This Topic: https://lists.openembedded.org/mt/99241972/1050810
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Signature · 3 lines
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end - "Use the force Harry" - Gandalf, Star Trek II
Quoted 33 lines · Bruce Ashfield
Quoted 11 lines · Trevor Gamblin
Quoted 9 lines · Bruce Ashfield
Quoted 7 lines · Richard Purdie
Hi Trevor,
I thought this issue might be something you might be able to spot the solution with quickly.
I had a failing python3-iniconfig build in a TMPDIR I've reused heavily:
Quoted 8 lines
Quoted 5 lines
FWIW. I see this in about 6 or 7 python recipes in my extremely heavily reused tmpdir.
After I came back from vacation, I iterated through a -c clean on the python packages, and they always built the second time through.
Do you recall which those are, and if it's the same ones each time it happens? I'm wondering if it may be an issue with some recipes.
roughly this set (as grepped from my history):
2783 bitbake -c clean python3-attrs; bitbake core-image-minimal 2828 bitbake -c clean python3-colorama; bitbake core-image-minimal 2829 bitbake -c clean python3-jsonschema; bitbake core-image-minimal 2943 bitbake -c clean python3-hatch-vcs python3-hatch-vcs-native; bitbake core-image-minimal 2944 bitbake -c clean python3-attrs; bitbake core-image-minimal 2945 bitbake -c clean python3-urllib3; bitbake core-image-minimal 2946 bitbake -c clean python3-colorama; bitbake core-image-minimal 2947 bitbake -c clean python3-jsonschema; bitbake core-image-minimal 2948 bitbake -c clean python3-iniconfig; bitbake core-image-minimal
Quoted 1 line
Apparently I had particular issues with colorama :)
It could possibly be an issue with recipes that depend on hatchling and/or hatch-vcs. A quick grep for "hatch" in oe-core/meta/recipes-devtools:
python3-jsonschema_4.17.3.bb
python3-attrs_23.1.0.bb
python3-urllib3_2.0.2.bb
python3-hatchling_1.17.0.bb
python3-hatch-fancy-pypi-readme_22.8.0.bb
python3-hatch-vcs_0.3.0.bb
python3-iniconfig_2.0.0.bb
and in meta-python:
python3-traitlets_5.9.0.bb
python3-prettytable_3.6.0.bb
python3-pyhamcrest_2.0.4.bb
python3-termcolor_2.3.0.bb
python3-platformdirs_3.5.1.bb
python3-soupsieve_2.4.1.bb
python3-colorama_0.4.6.bb
If it happens with traitlets, prettytable, soupsieve, platformdirs, termcolor, or pyhamcrest too then that's probably where the issue is. I'll keep trying to reproduce it.
Quoted 97 lines
Bruce
Quoted 5 lines
I haven't been able to recreate the issue yet, but my tmpdir is not very old. When I attempt the suggested reproducer (creating the empty directories and rerunning
do_compile), it is successful. I'll keep poking at it.Quoted 77 lines
Trevor
Quoted 75 lines
Bruce
Quoted 73 lines
ERROR: python3-iniconfig-2.0.0-r0
do_compile: ExecutionError('/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/temp/run.do_compile.2391637', 1, None, None)
ERROR: Logfile of failure stored in: /media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/temp/log.do_compile.2391637
Log data follows:| DEBUG: Executing shell function do_compile | * Getting build dependencies for wheel... | | Traceback (most recent call last): | File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", line 375, in main | built = build_call( | ^^^^^^^^^^^ | File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", line 208, in build_package | out = _build(isolation, builder, outdir, distribution, config_settings, skip_dependency_check) | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", line 147, in _build | return _build_in_current_env(builder, outdir, distribution, config_settings, skip_dependency_check) | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", line 127, in _build_in_current_env | missing = builder.check_dependencies(distribution) | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", line 369, in check_dependencies | return {u for d in dependencies for u in check_dependency(d)} | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", line 369, in <setcomp> | return {u for d in dependencies for u in check_dependency(d)} | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", line 185, in check_dependency | yield from check_dependency(other_req_string, ancestral_req_strings + (normalised_req_string,), req.extras) | File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", line 179, in check_dependency | if req.specifier and not req.specifier.contains(dist.version, prereleases=True): | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/packaging/specifiers.py", line 905, in contains | item = Version(item) | ^^^^^^^^^^^^^ | File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/packaging/version.py", line 196, in __init__ | match = self._regex.search(version) | ^^^^^^^^^^^^^^^^^^^^^^^^^^^ | TypeError: expected string or bytes-like object, got 'NoneType' | | ERROR expected string or bytes-like object, got 'NoneType' | WARNING: exit code 1 from a shell command.ERROR: Task (/media/build/poky/meta/recipes-devtools/python/python3-iniconfig_2.0.0.bb:do_compile) failed with exit code '1'
I wondered why and I isolated it down to this:
richard@jet:/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages$ find -type d -empty ./hatchling-1.13.0.dist-info/licenses ./pathspec-0.11.0.dist-info ./importlib_metadata-6.0.0.dist-info ./setuptools-67.4.0.dist-info ./wheel-0.38.4.dist-info ./installer-0.6.0.dist-info ./setuptools-67.6.0.dist-info ./packaging-23.0.dist-info richard@jet:/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages$ find -type d -empty -deleteAfter which the compile succeeds. These empty dist-info directories really upset the build. They're left behind by the sstate cleanup code from when I have older versions of these python modules in play since the sstate code can race over directory removal. Is there a way to make python ignore them? Otherwise should we remove them somewhere?
I suspect you can recreate by adding these empty directories and running a -c compile -f. I've seen it with plenty of other python modules too.
Cheers,
Richard
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#182013): https://lists.openembedded.org/g/openembedded-core/message/182013
Mute This Topic: https://lists.openembedded.org/mt/99241972/4619944
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Quoted 37 lines · Bruce Ashfield
Quoted 12 lines · Trevor Gamblin
Quoted 9 lines · Bruce Ashfield
Quoted 7 lines · Richard Purdie
Hi Trevor,
I thought this issue might be something you might be able to spot the solution with quickly.
I had a failing python3-iniconfig build in a TMPDIR I've reused heavily:
Quoted 9 lines
Quoted 5 lines
FWIW. I see this in about 6 or 7 python recipes in my extremely heavily reused tmpdir.
After I came back from vacation, I iterated through a -c clean on the python packages, and they always built the second time through.
Do you recall which those are, and if it's the same ones each time it happens? I'm wondering if it may be an issue with some recipes.
roughly this set (as grepped from my history):
2783 bitbake -c clean python3-attrs; bitbake core-image-minimal 2828 bitbake -c clean python3-colorama; bitbake core-image-minimal 2829 bitbake -c clean python3-jsonschema; bitbake core-image-minimal 2943 bitbake -c clean python3-hatch-vcs python3-hatch-vcs-native; bitbake core-image-minimal 2944 bitbake -c clean python3-attrs; bitbake core-image-minimal 2945 bitbake -c clean python3-urllib3; bitbake core-image-minimal 2946 bitbake -c clean python3-colorama; bitbake core-image-minimal 2947 bitbake -c clean python3-jsonschema; bitbake core-image-minimal 2948 bitbake -c clean python3-iniconfig; bitbake core-image-minimal
Quoted 5 lines
Apparently I had particular issues with colorama :)
Bruce
Do you recall when you started seeing this behavior? I’m wondering if it is related to some of the wheel build changes rburton made. But that’s just my spider sense…
Quoted 139 lines
Quoted 12 lines
I haven't been able to recreate the issue yet, but my tmpdir is not very old. When I attempt the suggested reproducer (creating the empty directories and rerunning
do_compile), it is successful. I'll keep poking at it.Trevor
Quoted 4 lines
Bruce
Quoted 1 line
ERROR: python3-iniconfig-2.0.0-r0
do_compile:ExecutionError('/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/temp/run.do_compile.2391637', 1, None, None)
Quoted 1 line
Quoted 1 line
Quoted 1 line
ERROR: Logfile of failure stored in:
/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/temp/log.do_compile.2391637
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Log data follows:
| DEBUG: Executing shell function do_compile | * Getting build dependencies for wheel... | | Traceback (most recent call last): | File"/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", line 375, in main
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
| built = build_call( | ^^^^^^^^^^^ | File"/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", line 208, in
build_packageQuoted 1 line
Quoted 1 line
Quoted 1 line
| out = _build(isolation, builder, outdir, distribution,
config_settings,skip_dependency_check)Quoted 1 line
Quoted 1 line
Quoted 1 line
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Quoted 1 line
Quoted 1 line
Quoted 1 line
| File
"/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", line 147, in
_buildQuoted 1 line
Quoted 1 line
Quoted 1 line
| return _build_in_current_env(builder, outdir, distribution,
config_settings,skip_dependency_check)Quoted 1 line
Quoted 1 line
Quoted 1 line
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Quoted 1 line
Quoted 1 line
Quoted 1 line
| File
"/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", line 127, in
_build_in_current_envQuoted 3 lines
Quoted 3 lines
Quoted 3 lines
| missing = builder.check_dependencies(distribution) | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | File"/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", line 369, in
check_dependenciesQuoted 3 lines
Quoted 3 lines
Quoted 3 lines
| return {u for d in dependencies for u in check_dependency(d)} | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | File"/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", line 369, in <setcomp>
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
| return {u for d in dependencies for u in check_dependency(d)} | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | File"/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", line 185, in
check_dependencyQuoted 1 line
Quoted 1 line
Quoted 1 line
| yield from check_dependency(other_req_string,
ancestral_req_strings+ (normalised_req_string,), req.extras)Quoted 1 line
Quoted 1 line
Quoted 1 line
| File
"/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", line 179, in
check_dependencyQuoted 1 line
Quoted 1 line
Quoted 1 line
| if req.specifier and not req.specifier.contains(dist.version,
prereleases=True):
Quoted 1 line
Quoted 1 line
Quoted 1 line
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Quoted 1 line
Quoted 1 line
Quoted 1 line
| File
"/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/packaging/specifiers.py", line 905, in contains
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
| item = Version(item) | ^^^^^^^^^^^^^ | File"/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/packaging/version.py", line 196, in
__init__Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
| match = self._regex.search(version) | ^^^^^^^^^^^^^^^^^^^^^^^^^^^ | TypeError: expected string or bytes-like object, got 'NoneType' | | ERROR expected string or bytes-like object, got 'NoneType' | WARNING: exit code 1 from a shell command.ERROR: Task
(/media/build/poky/meta/recipes-devtools/python/python3-iniconfig_2.0.0.bb:do_compile) failed with exit code '1'
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
I wondered why and I isolated it down to this:
richard@jet:/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages$
find -type d -empty
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
./hatchling-1.13.0.dist-info/licenses
./pathspec-0.11.0.dist-info
./importlib_metadata-6.0.0.dist-info
./setuptools-67.4.0.dist-info
./wheel-0.38.4.dist-info
./installer-0.6.0.dist-info
./setuptools-67.6.0.dist-info
./packaging-23.0.dist-info richard@jet:/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages$find -type d -empty -delete
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
After which the compile succeeds. These empty dist-info directories really upset the build. They're left behind by the sstate cleanup code from when I have older versions of these python modules in play since the sstate code can race over directory removal. Is there a way to
make
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
python ignore them? Otherwise should we remove them somewhere?
I suspect you can recreate by adding these empty directories and running a -c compile -f. I've seen it with plenty of other python modules too.
Cheers,
Quoted 7 lines
Quoted 7 lines
Quoted 6 lines
Richard
--
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
- "Use the force Harry" - Gandalf, Star Trek II-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#182013):
https://lists.openembedded.org/g/openembedded-core/message/182013
Mute This Topic: https://lists.openembedded.org/mt/99241972/924729
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ [email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Quoted 41 lines · Tim Orling
Quoted 37 lines · Bruce Ashfield
Quoted 12 lines · Trevor Gamblin
Quoted 9 lines · Bruce Ashfield
Quoted 7 lines · Richard Purdie
Hi Trevor,
I thought this issue might be something you might be able to spot the solution with quickly.
I had a failing python3-iniconfig build in a TMPDIR I've reused heavily:
Quoted 9 lines
Quoted 5 lines
FWIW. I see this in about 6 or 7 python recipes in my extremely heavily reused tmpdir.
After I came back from vacation, I iterated through a -c clean on the python packages, and they always built the second time through.
Do you recall which those are, and if it's the same ones each time it happens? I'm wondering if it may be an issue with some recipes.
roughly this set (as grepped from my history):
2783 bitbake -c clean python3-attrs; bitbake core-image-minimal 2828 bitbake -c clean python3-colorama; bitbake core-image-minimal 2829 bitbake -c clean python3-jsonschema; bitbake core-image-minimal 2943 bitbake -c clean python3-hatch-vcs python3-hatch-vcs-native; bitbake core-image-minimal 2944 bitbake -c clean python3-attrs; bitbake core-image-minimal 2945 bitbake -c clean python3-urllib3; bitbake core-image-minimal 2946 bitbake -c clean python3-colorama; bitbake core-image-minimal 2947 bitbake -c clean python3-jsonschema; bitbake core-image-minimal 2948 bitbake -c clean python3-iniconfig; bitbake core-image-minimal
Quoted 6 lines
Quoted 4 lines
Apparently I had particular issues with colorama :)
Bruce
Do you recall when you started seeing this behavior? I’m wondering if it is related to some of the wheel build changes rburton made. But that’s just my spider sense…
As we can all attest, timelines get blurry very quickly ... but I recall it popping up with the same / similar error message starting in the last 6 months.
Bruce
Quoted 102 lines
Quoted 102 lines
Quoted 86 lines
I haven't been able to recreate the issue yet, but my tmpdir is not very old. When I attempt the suggested reproducer (creating the empty directories and rerunning
do_compile), it is successful. I'll keep poking at it.Trevor
Quoted 77 lines
Bruce
Quoted 73 lines
ERROR: python3-iniconfig-2.0.0-r0
do_compile: ExecutionError('/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/temp/run.do_compile.2391637', 1, None, None)
ERROR: Logfile of failure stored in: /media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/temp/log.do_compile.2391637
Log data follows:| DEBUG: Executing shell function do_compile | * Getting build dependencies for wheel... | | Traceback (most recent call last): | File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", line 375, in main | built = build_call( | ^^^^^^^^^^^ | File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", line 208, in build_package | out = _build(isolation, builder, outdir, distribution, config_settings, skip_dependency_check) | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", line 147, in _build | return _build_in_current_env(builder, outdir, distribution, config_settings, skip_dependency_check) | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", line 127, in _build_in_current_env | missing = builder.check_dependencies(distribution) | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", line 369, in check_dependencies | return {u for d in dependencies for u in check_dependency(d)} | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", line 369, in <setcomp> | return {u for d in dependencies for u in check_dependency(d)} | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", line 185, in check_dependency | yield from check_dependency(other_req_string, ancestral_req_strings + (normalised_req_string,), req.extras) | File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", line 179, in check_dependency | if req.specifier and not req.specifier.contains(dist.version, prereleases=True): | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/packaging/specifiers.py", line 905, in contains | item = Version(item) | ^^^^^^^^^^^^^ | File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/packaging/version.py", line 196, in __init__ | match = self._regex.search(version) | ^^^^^^^^^^^^^^^^^^^^^^^^^^^ | TypeError: expected string or bytes-like object, got 'NoneType' | | ERROR expected string or bytes-like object, got 'NoneType' | WARNING: exit code 1 from a shell command.ERROR: Task (/media/build/poky/meta/recipes-devtools/python/python3-iniconfig_2.0.0.bb:do_compile) failed with exit code '1'
I wondered why and I isolated it down to this:
richard@jet:/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages$ find -type d -empty ./hatchling-1.13.0.dist-info/licenses ./pathspec-0.11.0.dist-info ./importlib_metadata-6.0.0.dist-info ./setuptools-67.4.0.dist-info ./wheel-0.38.4.dist-info ./installer-0.6.0.dist-info ./setuptools-67.6.0.dist-info ./packaging-23.0.dist-info richard@jet:/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages$ find -type d -empty -deleteAfter which the compile succeeds. These empty dist-info directories really upset the build. They're left behind by the sstate cleanup code from when I have older versions of these python modules in play since the sstate code can race over directory removal. Is there a way to make python ignore them? Otherwise should we remove them somewhere?
I suspect you can recreate by adding these empty directories and running a -c compile -f. I've seen it with plenty of other python modules too.
Cheers,
Richard
--
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
- "Use the force Harry" - Gandalf, Star Trek II-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#182013): https://lists.openembedded.org/g/openembedded-core/message/182013
Mute This Topic: https://lists.openembedded.org/mt/99241972/924729
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Signature · 3 lines
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end - "Use the force Harry" - Gandalf, Star Trek II
I’m regularly seeing this when updating various python recipes. It’s caused by empty versioned directories in sysroots that are leftovers from previous versions. Doesn’t happen in freshly made tmpdirs.
Alex
Quoted 213 lines · Bruce Ashfield
Quoted 4 lines · Tim Orling
On Wed, May 31, 2023 at 7:25 AM Bruce Ashfield <[email protected]>
Quoted 2 lines · wrote:
Quoted 2 lines
On Wed, May 31, 2023 at 10:20 AM Trevor Gamblin <[email protected]>
Quoted 8 lines · wrote:
Quoted 8 lines
Quoted 8 lines · Bruce Ashfield
Quoted 5 lines
Quoted 3 lines · Richard Purdie
Hi Trevor,
I thought this issue might be something you might be able to spot
the
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 4 lines
solution with quickly.
I had a failing python3-iniconfig build in a TMPDIR I've reused heavily:
FWIW. I see this in about 6 or 7 python recipes in my extremely heavily reused tmpdir.
After I came back from vacation, I iterated through a -c clean on
the
Quoted 19 lines
Quoted 19 lines
Quoted 5 lines
Quoted 1 line
python packages, and they always built the second time through.
Do you recall which those are, and if it's the same ones each time it happens? I'm wondering if it may be an issue with some recipes.
roughly this set (as grepped from my history):
2783 bitbake -c clean python3-attrs; bitbake core-image-minimal 2828 bitbake -c clean python3-colorama; bitbake core-image-minimal 2829 bitbake -c clean python3-jsonschema; bitbake core-image-minimal 2943 bitbake -c clean python3-hatch-vcs python3-hatch-vcs-native; bitbake core-image-minimal 2944 bitbake -c clean python3-attrs; bitbake core-image-minimal 2945 bitbake -c clean python3-urllib3; bitbake core-image-minimal 2946 bitbake -c clean python3-colorama; bitbake core-image-minimal 2947 bitbake -c clean python3-jsonschema; bitbake core-image-minimal 2948 bitbake -c clean python3-iniconfig; bitbake core-image-minimalQuoted 6 lines
Quoted 4 lines
Apparently I had particular issues with colorama :)
Bruce
Do you recall when you started seeing this behavior? I’m wondering if it
is related to some of the wheel build changes rburton made. But that’s just my spider sense…
As we can all attest, timelines get blurry very quickly ... but I recall it popping up with the same / similar error message starting in the last 6 months.
Bruce
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
I haven't been able to recreate the issue yet, but my tmpdir is not
very
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
old. When I attempt the suggested reproducer (creating the empty directories and rerunning
do_compile), it is successful. I'll keep poking at it.Trevor
Quoted 4 lines
Bruce
Quoted 1 line
ERROR: python3-iniconfig-2.0.0-r0
do_compile:ExecutionError('/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/temp/run.do_compile.2391637', 1, None, None)
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
ERROR: Logfile of failure stored in:
/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/temp/log.do_compile.2391637
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Log data follows:
| DEBUG: Executing shell function do_compile | * Getting build dependencies for wheel... | | Traceback (most recent call last): | File"/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", line 375, in main
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
| built = build_call( | ^^^^^^^^^^^ | File"/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", line 208, in
build_packageQuoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
| out = _build(isolation, builder, outdir, distribution,
config_settings,skip_dependency_check)Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
| File
"/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", line 147, in
_buildQuoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
| return _build_in_current_env(builder, outdir, distribution,
config_settings,skip_dependency_check)Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
| File
"/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", line 127, in
_build_in_current_envQuoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
| missing = builder.check_dependencies(distribution) | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | File"/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", line 369, in
check_dependenciesQuoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
| return {u for d in dependencies for u in check_dependency(d)} | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | File"/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", line 369, in <setcomp>
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
| return {u for d in dependencies for u in check_dependency(d)} | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | File"/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", line 185, in
check_dependencyQuoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
| yield from check_dependency(other_req_string,
ancestral_req_strings+ (normalised_req_string,), req.extras)Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
| File
"/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", line 179, in
check_dependencyQuoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
| if req.specifier and not
req.specifier.contains(dist.version, prereleases=True):Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
| File
"/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/packaging/specifiers.py", line 905, in contains
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
| item = Version(item) | ^^^^^^^^^^^^^ | File"/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/packaging/version.py", line 196, in
__init__Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
| match = self._regex.search(version) | ^^^^^^^^^^^^^^^^^^^^^^^^^^^ | TypeError: expected string or bytes-like object, got 'NoneType' | | ERROR expected string or bytes-like object, got 'NoneType' | WARNING: exit code 1 from a shell command.ERROR: Task
(/media/build/poky/meta/recipes-devtools/python/python3-iniconfig_2.0.0.bb:do_compile) failed with exit code '1'
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
I wondered why and I isolated it down to this:
richard@jet:/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages$
find -type d -empty
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
./hatchling-1.13.0.dist-info/licenses
./pathspec-0.11.0.dist-info
./importlib_metadata-6.0.0.dist-info
./setuptools-67.4.0.dist-info
./wheel-0.38.4.dist-info
./installer-0.6.0.dist-info
./setuptools-67.6.0.dist-info
./packaging-23.0.dist-info richard@jet:/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages$find -type d -empty -delete
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
After which the compile succeeds. These empty dist-info directories really upset the build. They're left behind by the sstate cleanup
code
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
from when I have older versions of these python modules in play
since
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
the sstate code can race over directory removal. Is there a way to
make
Quoted 18 lines
Quoted 18 lines
Quoted 15 lines
Quoted 15 lines
Quoted 14 lines
python ignore them? Otherwise should we remove them somewhere?
I suspect you can recreate by adding these empty directories and running a -c compile -f. I've seen it with plenty of other python modules too.
Cheers,
Richard
Quoted 7 lines
Quoted 7 lines
--
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
- "Use the force Harry" - Gandalf, Star Trek II--
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
- "Use the force Harry" - Gandalf, Star Trek II-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#182016):
https://lists.openembedded.org/g/openembedded-core/message/182016
Mute This Topic: https://lists.openembedded.org/mt/99241972/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ [email protected]]
-=-=-=-=-=-=-=-=-=-=-=-