Skip to contentSkip to conversation

python3 cryptic build failures

DISCUSSION Richard Purdie 6 participants 8 messages May 31, 2023

lore
Share & tools
lore
#0 lore raw

Hi Trevor,

I thought this issue might be something you might be able to spot the solution with quickly.

I had a failing python3-iniconfig build in a TMPDIR I've reused heavily:

ERROR: python3-iniconfig-2.0.0-r0 do_compile: ExecutionError('/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/temp/run.do_compile.2391637', 1, None, None)
ERROR: Logfile of failure stored in: /media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/temp/log.do_compile.2391637
Log data follows:

| DEBUG: Executing shell function do_compile
| * Getting build dependencies for wheel...
| 
| Traceback (most recent call last):
|   File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", line 375, in main
|     built = build_call(
|             ^^^^^^^^^^^
|   File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", line 208, in build_package
|     out = _build(isolation, builder, outdir, distribution, config_settings, skip_dependency_check)
|           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|   File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", line 147, in _build
|     return _build_in_current_env(builder, outdir, distribution, config_settings, skip_dependency_check)
|            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|   File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", line 127, in _build_in_current_env
|     missing = builder.check_dependencies(distribution)
|               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|   File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", line 369, in check_dependencies
|     return {u for d in dependencies for u in check_dependency(d)}
|            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|   File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", line 369, in <setcomp>
|     return {u for d in dependencies for u in check_dependency(d)}
|            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|   File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", line 185, in check_dependency
|     yield from check_dependency(other_req_string, ancestral_req_strings + (normalised_req_string,), req.extras)
|   File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", line 179, in check_dependency
|     if req.specifier and not req.specifier.contains(dist.version, prereleases=True):
|                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|   File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/packaging/specifiers.py", line 905, in contains
|     item = Version(item)
|            ^^^^^^^^^^^^^
|   File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/packaging/version.py", line 196, in __init__
|     match = self._regex.search(version)
|             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
| TypeError: expected string or bytes-like object, got 'NoneType'
| 
| ERROR expected string or bytes-like object, got 'NoneType'
| WARNING: exit code 1 from a shell command.

ERROR: Task (/media/build/poky/meta/recipes-devtools/python/python3-iniconfig_2.0.0.bb:do_compile) failed with exit code '1'

I wondered why and I isolated it down to this:

richard@jet:/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages$ find -type d -empty
./hatchling-1.13.0.dist-info/licenses
./pathspec-0.11.0.dist-info
./importlib_metadata-6.0.0.dist-info
./setuptools-67.4.0.dist-info
./wheel-0.38.4.dist-info
./installer-0.6.0.dist-info
./setuptools-67.6.0.dist-info
./packaging-23.0.dist-info
richard@jet:/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages$ find -type d -empty -delete

After which the compile succeeds. These empty dist-info directories really upset the build. They're left behind by the sstate cleanup code from when I have older versions of these python modules in play since the sstate code can race over directory removal. Is there a way to make python ignore them? Otherwise should we remove them somewhere?

I suspect you can recreate by adding these empty directories and running a -c compile -f. I've seen it with plenty of other python modules too.

Cheers,

Richard

#1 lore raw
Quoted 8 lines · Richard Purdie

Hi Trevor,

I thought this issue might be something you might be able to spot the solution with quickly.

I had a failing python3-iniconfig build in a TMPDIR I've reused heavily:

FWIW. I see this in about 6 or 7 python recipes in my extremely heavily reused tmpdir.

After I came back from vacation, I iterated through a -c clean on the python packages, and they always built the second time through.

Bruce

Quoted 80 lines

ERROR: python3-iniconfig-2.0.0-r0 do_compile: ExecutionError('/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/temp/run.do_compile.2391637', 1, None, None)
ERROR: Logfile of failure stored in: /media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/temp/log.do_compile.2391637
Log data follows:

| DEBUG: Executing shell function do_compile
| * Getting build dependencies for wheel...
|
| Traceback (most recent call last):
|   File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", line 375, in main
|     built = build_call(
|             ^^^^^^^^^^^
|   File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", line 208, in build_package
|     out = _build(isolation, builder, outdir, distribution, config_settings, skip_dependency_check)
|           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|   File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", line 147, in _build
|     return _build_in_current_env(builder, outdir, distribution, config_settings, skip_dependency_check)
|            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|   File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", line 127, in _build_in_current_env
|     missing = builder.check_dependencies(distribution)
|               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|   File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", line 369, in check_dependencies
|     return {u for d in dependencies for u in check_dependency(d)}
|            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|   File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", line 369, in <setcomp>
|     return {u for d in dependencies for u in check_dependency(d)}
|            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|   File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", line 185, in check_dependency
|     yield from check_dependency(other_req_string, ancestral_req_strings + (normalised_req_string,), req.extras)
|   File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", line 179, in check_dependency
|     if req.specifier and not req.specifier.contains(dist.version, prereleases=True):
|                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|   File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/packaging/specifiers.py", line 905, in contains
|     item = Version(item)
|            ^^^^^^^^^^^^^
|   File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/packaging/version.py", line 196, in __init__
|     match = self._regex.search(version)
|             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
| TypeError: expected string or bytes-like object, got 'NoneType'
|
| ERROR expected string or bytes-like object, got 'NoneType'
| WARNING: exit code 1 from a shell command.

ERROR: Task (/media/build/poky/meta/recipes-devtools/python/python3-iniconfig_2.0.0.bb:do_compile) failed with exit code '1'

I wondered why and I isolated it down to this:

richard@jet:/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages$ find -type d -empty
./hatchling-1.13.0.dist-info/licenses
./pathspec-0.11.0.dist-info
./importlib_metadata-6.0.0.dist-info
./setuptools-67.4.0.dist-info
./wheel-0.38.4.dist-info
./installer-0.6.0.dist-info
./setuptools-67.6.0.dist-info
./packaging-23.0.dist-info
richard@jet:/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages$ find -type d -empty -delete

After which the compile succeeds. These empty dist-info directories really upset the build. They're left behind by the sstate cleanup code from when I have older versions of these python modules in play since the sstate code can race over directory removal. Is there a way to make python ignore them? Otherwise should we remove them somewhere?

I suspect you can recreate by adding these empty directories and running a -c compile -f. I've seen it with plenty of other python modules too.

Cheers,

Richard

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#182010): https://lists.openembedded.org/g/openembedded-core/message/182010
Mute This Topic: https://lists.openembedded.org/mt/99241972/1050810
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

Signature · 3 lines
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II
#2 lore raw
Quoted 7 lines · Bruce Ashfield
Quoted 5 lines · Richard Purdie

Hi Trevor,

I thought this issue might be something you might be able to spot the solution with quickly.

Quoted 7 lines
Quoted 2 lines

I had a failing python3-iniconfig build in a TMPDIR I've reused heavily:

FWIW. I see this in about 6 or 7 python recipes in my extremely heavily reused tmpdir.

After I came back from vacation, I iterated through a -c clean on the python packages, and they always built the second time through.

Do you recall which those are, and if it's the same ones each time it happens? I'm wondering if it may be an issue with some recipes.

I haven't been able to recreate the issue yet, but my tmpdir is not very old. When I attempt the suggested reproducer (creating the empty directories and rerunning do_compile), it is successful. I'll keep poking at it.

Trevor

Quoted 83 lines

Bruce

Quoted 79 lines

ERROR: python3-iniconfig-2.0.0-r0 do_compile: ExecutionError('/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/temp/run.do_compile.2391637', 1, None, None)
ERROR: Logfile of failure stored in: /media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/temp/log.do_compile.2391637
Log data follows:

| DEBUG: Executing shell function do_compile
| * Getting build dependencies for wheel...
|
| Traceback (most recent call last):
|   File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", line 375, in main
|     built = build_call(
|             ^^^^^^^^^^^
|   File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", line 208, in build_package
|     out = _build(isolation, builder, outdir, distribution, config_settings, skip_dependency_check)
|           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|   File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", line 147, in _build
|     return _build_in_current_env(builder, outdir, distribution, config_settings, skip_dependency_check)
|            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|   File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", line 127, in _build_in_current_env
|     missing = builder.check_dependencies(distribution)
|               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|   File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", line 369, in check_dependencies
|     return {u for d in dependencies for u in check_dependency(d)}
|            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|   File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", line 369, in <setcomp>
|     return {u for d in dependencies for u in check_dependency(d)}
|            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|   File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", line 185, in check_dependency
|     yield from check_dependency(other_req_string, ancestral_req_strings + (normalised_req_string,), req.extras)
|   File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", line 179, in check_dependency
|     if req.specifier and not req.specifier.contains(dist.version, prereleases=True):
|                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|   File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/packaging/specifiers.py", line 905, in contains
|     item = Version(item)
|            ^^^^^^^^^^^^^
|   File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/packaging/version.py", line 196, in __init__
|     match = self._regex.search(version)
|             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
| TypeError: expected string or bytes-like object, got 'NoneType'
|
| ERROR expected string or bytes-like object, got 'NoneType'
| WARNING: exit code 1 from a shell command.

ERROR: Task (/media/build/poky/meta/recipes-devtools/python/python3-iniconfig_2.0.0.bb:do_compile) failed with exit code '1'

I wondered why and I isolated it down to this:

richard@jet:/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages$ find -type d -empty
./hatchling-1.13.0.dist-info/licenses
./pathspec-0.11.0.dist-info
./importlib_metadata-6.0.0.dist-info
./setuptools-67.4.0.dist-info
./wheel-0.38.4.dist-info
./installer-0.6.0.dist-info
./setuptools-67.6.0.dist-info
./packaging-23.0.dist-info
richard@jet:/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages$ find -type d -empty -delete

After which the compile succeeds. These empty dist-info directories really upset the build. They're left behind by the sstate cleanup code from when I have older versions of these python modules in play since the sstate code can race over directory removal. Is there a way to make python ignore them? Otherwise should we remove them somewhere?

I suspect you can recreate by adding these empty directories and running a -c compile -f. I've seen it with plenty of other python modules too.

Cheers,

Richard

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#182010): https://lists.openembedded.org/g/openembedded-core/message/182010
Mute This Topic: https://lists.openembedded.org/mt/99241972/1050810
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

#3 lore raw
Quoted 12 lines · Trevor Gamblin
Quoted 9 lines · Bruce Ashfield
Quoted 7 lines · Richard Purdie

Hi Trevor,

I thought this issue might be something you might be able to spot the solution with quickly.

I had a failing python3-iniconfig build in a TMPDIR I've reused heavily:

Quoted 9 lines
Quoted 5 lines

FWIW. I see this in about 6 or 7 python recipes in my extremely heavily reused tmpdir.

After I came back from vacation, I iterated through a -c clean on the python packages, and they always built the second time through.

Do you recall which those are, and if it's the same ones each time it happens? I'm wondering if it may be an issue with some recipes.

roughly this set (as grepped from my history):

 2783  bitbake -c clean python3-attrs; bitbake core-image-minimal
 2828  bitbake -c clean python3-colorama; bitbake core-image-minimal
 2829  bitbake -c clean python3-jsonschema; bitbake core-image-minimal
 2943  bitbake -c clean python3-hatch-vcs python3-hatch-vcs-native;
bitbake core-image-minimal
 2944  bitbake -c clean python3-attrs; bitbake core-image-minimal
 2945  bitbake -c clean python3-urllib3; bitbake core-image-minimal
 2946  bitbake -c clean python3-colorama; bitbake core-image-minimal
 2947  bitbake -c clean python3-jsonschema; bitbake core-image-minimal
 2948  bitbake -c clean python3-iniconfig; bitbake core-image-minimal

Apparently I had particular issues with colorama :)

Bruce

Quoted 90 lines

I haven't been able to recreate the issue yet, but my tmpdir is not very old. When I attempt the suggested reproducer (creating the empty directories and rerunning do_compile), it is successful. I'll keep poking at it.

Trevor

Quoted 83 lines

Bruce

Quoted 79 lines

ERROR: python3-iniconfig-2.0.0-r0 do_compile: ExecutionError('/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/temp/run.do_compile.2391637', 1, None, None)
ERROR: Logfile of failure stored in: /media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/temp/log.do_compile.2391637
Log data follows:

| DEBUG: Executing shell function do_compile
| * Getting build dependencies for wheel...
|
| Traceback (most recent call last):
|   File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", line 375, in main
|     built = build_call(
|             ^^^^^^^^^^^
|   File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", line 208, in build_package
|     out = _build(isolation, builder, outdir, distribution, config_settings, skip_dependency_check)
|           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|   File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", line 147, in _build
|     return _build_in_current_env(builder, outdir, distribution, config_settings, skip_dependency_check)
|            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|   File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", line 127, in _build_in_current_env
|     missing = builder.check_dependencies(distribution)
|               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|   File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", line 369, in check_dependencies
|     return {u for d in dependencies for u in check_dependency(d)}
|            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|   File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", line 369, in <setcomp>
|     return {u for d in dependencies for u in check_dependency(d)}
|            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|   File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", line 185, in check_dependency
|     yield from check_dependency(other_req_string, ancestral_req_strings + (normalised_req_string,), req.extras)
|   File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", line 179, in check_dependency
|     if req.specifier and not req.specifier.contains(dist.version, prereleases=True):
|                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|   File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/packaging/specifiers.py", line 905, in contains
|     item = Version(item)
|            ^^^^^^^^^^^^^
|   File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/packaging/version.py", line 196, in __init__
|     match = self._regex.search(version)
|             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
| TypeError: expected string or bytes-like object, got 'NoneType'
|
| ERROR expected string or bytes-like object, got 'NoneType'
| WARNING: exit code 1 from a shell command.

ERROR: Task (/media/build/poky/meta/recipes-devtools/python/python3-iniconfig_2.0.0.bb:do_compile) failed with exit code '1'

I wondered why and I isolated it down to this:

richard@jet:/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages$ find -type d -empty
./hatchling-1.13.0.dist-info/licenses
./pathspec-0.11.0.dist-info
./importlib_metadata-6.0.0.dist-info
./setuptools-67.4.0.dist-info
./wheel-0.38.4.dist-info
./installer-0.6.0.dist-info
./setuptools-67.6.0.dist-info
./packaging-23.0.dist-info
richard@jet:/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages$ find -type d -empty -delete

After which the compile succeeds. These empty dist-info directories really upset the build. They're left behind by the sstate cleanup code from when I have older versions of these python modules in play since the sstate code can race over directory removal. Is there a way to make python ignore them? Otherwise should we remove them somewhere?

I suspect you can recreate by adding these empty directories and running a -c compile -f. I've seen it with plenty of other python modules too.

Cheers,

Richard

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#182010): https://lists.openembedded.org/g/openembedded-core/message/182010
Mute This Topic: https://lists.openembedded.org/mt/99241972/1050810
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

Signature · 3 lines
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II
#4 lore raw
Quoted 33 lines · Bruce Ashfield
Quoted 11 lines · Trevor Gamblin
Quoted 9 lines · Bruce Ashfield
Quoted 7 lines · Richard Purdie

Hi Trevor,

I thought this issue might be something you might be able to spot the solution with quickly.

I had a failing python3-iniconfig build in a TMPDIR I've reused heavily:

Quoted 8 lines
Quoted 5 lines

FWIW. I see this in about 6 or 7 python recipes in my extremely heavily reused tmpdir.

After I came back from vacation, I iterated through a -c clean on the python packages, and they always built the second time through.

Do you recall which those are, and if it's the same ones each time it happens? I'm wondering if it may be an issue with some recipes.

roughly this set (as grepped from my history):

  2783  bitbake -c clean python3-attrs; bitbake core-image-minimal
  2828  bitbake -c clean python3-colorama; bitbake core-image-minimal
  2829  bitbake -c clean python3-jsonschema; bitbake core-image-minimal
  2943  bitbake -c clean python3-hatch-vcs python3-hatch-vcs-native;
bitbake core-image-minimal
  2944  bitbake -c clean python3-attrs; bitbake core-image-minimal
  2945  bitbake -c clean python3-urllib3; bitbake core-image-minimal
  2946  bitbake -c clean python3-colorama; bitbake core-image-minimal
  2947  bitbake -c clean python3-jsonschema; bitbake core-image-minimal
  2948  bitbake -c clean python3-iniconfig; bitbake core-image-minimal
Quoted 1 line

Apparently I had particular issues with colorama :)

It could possibly be an issue with recipes that depend on hatchling and/or hatch-vcs. A quick grep for "hatch" in oe-core/meta/recipes-devtools:

python3-jsonschema_4.17.3.bb
python3-attrs_23.1.0.bb
python3-urllib3_2.0.2.bb
python3-hatchling_1.17.0.bb
python3-hatch-fancy-pypi-readme_22.8.0.bb
python3-hatch-vcs_0.3.0.bb
python3-iniconfig_2.0.0.bb

and in meta-python:

python3-traitlets_5.9.0.bb
python3-prettytable_3.6.0.bb
python3-pyhamcrest_2.0.4.bb
python3-termcolor_2.3.0.bb
python3-platformdirs_3.5.1.bb
python3-soupsieve_2.4.1.bb
python3-colorama_0.4.6.bb

If it happens with traitlets, prettytable, soupsieve, platformdirs, termcolor, or pyhamcrest too then that's probably where the issue is. I'll keep trying to reproduce it.

Quoted 97 lines

Bruce

Quoted 5 lines

I haven't been able to recreate the issue yet, but my tmpdir is not very old. When I attempt the suggested reproducer (creating the empty directories and rerunning do_compile), it is successful. I'll keep poking at it.

Quoted 77 lines

Trevor

Quoted 75 lines

Bruce

Quoted 73 lines

ERROR: python3-iniconfig-2.0.0-r0 do_compile: ExecutionError('/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/temp/run.do_compile.2391637', 1, None, None)
ERROR: Logfile of failure stored in: /media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/temp/log.do_compile.2391637
Log data follows:

| DEBUG: Executing shell function do_compile
| * Getting build dependencies for wheel...
|
| Traceback (most recent call last):
|   File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", line 375, in main
|     built = build_call(
|             ^^^^^^^^^^^
|   File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", line 208, in build_package
|     out = _build(isolation, builder, outdir, distribution, config_settings, skip_dependency_check)
|           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|   File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", line 147, in _build
|     return _build_in_current_env(builder, outdir, distribution, config_settings, skip_dependency_check)
|            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|   File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", line 127, in _build_in_current_env
|     missing = builder.check_dependencies(distribution)
|               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|   File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", line 369, in check_dependencies
|     return {u for d in dependencies for u in check_dependency(d)}
|            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|   File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", line 369, in <setcomp>
|     return {u for d in dependencies for u in check_dependency(d)}
|            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|   File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", line 185, in check_dependency
|     yield from check_dependency(other_req_string, ancestral_req_strings + (normalised_req_string,), req.extras)
|   File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", line 179, in check_dependency
|     if req.specifier and not req.specifier.contains(dist.version, prereleases=True):
|                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|   File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/packaging/specifiers.py", line 905, in contains
|     item = Version(item)
|            ^^^^^^^^^^^^^
|   File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/packaging/version.py", line 196, in __init__
|     match = self._regex.search(version)
|             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
| TypeError: expected string or bytes-like object, got 'NoneType'
|
| ERROR expected string or bytes-like object, got 'NoneType'
| WARNING: exit code 1 from a shell command.

ERROR: Task (/media/build/poky/meta/recipes-devtools/python/python3-iniconfig_2.0.0.bb:do_compile) failed with exit code '1'

I wondered why and I isolated it down to this:

richard@jet:/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages$ find -type d -empty
./hatchling-1.13.0.dist-info/licenses
./pathspec-0.11.0.dist-info
./importlib_metadata-6.0.0.dist-info
./setuptools-67.4.0.dist-info
./wheel-0.38.4.dist-info
./installer-0.6.0.dist-info
./setuptools-67.6.0.dist-info
./packaging-23.0.dist-info
richard@jet:/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages$ find -type d -empty -delete

After which the compile succeeds. These empty dist-info directories really upset the build. They're left behind by the sstate cleanup code from when I have older versions of these python modules in play since the sstate code can race over directory removal. Is there a way to make python ignore them? Otherwise should we remove them somewhere?

I suspect you can recreate by adding these empty directories and running a -c compile -f. I've seen it with plenty of other python modules too.

Cheers,

Richard

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#182013): https://lists.openembedded.org/g/openembedded-core/message/182013
Mute This Topic: https://lists.openembedded.org/mt/99241972/4619944
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

#5 lore raw
In reply to Bruce Ashfield
Quoted 37 lines · Bruce Ashfield
Quoted 12 lines · Trevor Gamblin
Quoted 9 lines · Bruce Ashfield
Quoted 7 lines · Richard Purdie

Hi Trevor,

I thought this issue might be something you might be able to spot the solution with quickly.

I had a failing python3-iniconfig build in a TMPDIR I've reused heavily:

Quoted 9 lines
Quoted 5 lines

FWIW. I see this in about 6 or 7 python recipes in my extremely heavily reused tmpdir.

After I came back from vacation, I iterated through a -c clean on the python packages, and they always built the second time through.

Do you recall which those are, and if it's the same ones each time it happens? I'm wondering if it may be an issue with some recipes.

roughly this set (as grepped from my history):

 2783  bitbake -c clean python3-attrs; bitbake core-image-minimal
 2828  bitbake -c clean python3-colorama; bitbake core-image-minimal
 2829  bitbake -c clean python3-jsonschema; bitbake core-image-minimal
 2943  bitbake -c clean python3-hatch-vcs python3-hatch-vcs-native;
bitbake core-image-minimal
 2944  bitbake -c clean python3-attrs; bitbake core-image-minimal
 2945  bitbake -c clean python3-urllib3; bitbake core-image-minimal
 2946  bitbake -c clean python3-colorama; bitbake core-image-minimal
 2947  bitbake -c clean python3-jsonschema; bitbake core-image-minimal
 2948  bitbake -c clean python3-iniconfig; bitbake core-image-minimal
Quoted 5 lines

Apparently I had particular issues with colorama :)

Bruce

Do you recall when you started seeing this behavior? I’m wondering if it is related to some of the wheel build changes rburton made. But that’s just my spider sense…

Quoted 139 lines
Quoted 12 lines

I haven't been able to recreate the issue yet, but my tmpdir is not very old. When I attempt the suggested reproducer (creating the empty directories and rerunning do_compile), it is successful. I'll keep poking at it.

Trevor

Quoted 4 lines

Bruce

Quoted 1 line

ERROR: python3-iniconfig-2.0.0-r0 do_compile:

ExecutionError('/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/temp/run.do_compile.2391637', 1, None, None)

Quoted 1 line
Quoted 1 line
Quoted 1 line

ERROR: Logfile of failure stored in:

/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/temp/log.do_compile.2391637

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

Log data follows:

| DEBUG: Executing shell function do_compile
| * Getting build dependencies for wheel...
|
| Traceback (most recent call last):
|   File

"/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", line 375, in main

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
|     built = build_call(
|             ^^^^^^^^^^^
|   File

"/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", line 208, in build_package

Quoted 1 line
Quoted 1 line
Quoted 1 line

| out = _build(isolation, builder, outdir, distribution,

config_settings, skip_dependency_check)

Quoted 1 line
Quoted 1 line
Quoted 1 line

|

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Quoted 1 line
Quoted 1 line
Quoted 1 line

| File

"/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", line 147, in _build

Quoted 1 line
Quoted 1 line
Quoted 1 line

| return _build_in_current_env(builder, outdir, distribution,

config_settings, skip_dependency_check)

Quoted 1 line
Quoted 1 line
Quoted 1 line

|

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Quoted 1 line
Quoted 1 line
Quoted 1 line

| File

"/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", line 127, in _build_in_current_env

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
|     missing = builder.check_dependencies(distribution)
|               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|   File

"/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", line 369, in check_dependencies

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
|     return {u for d in dependencies for u in check_dependency(d)}
|            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|   File

"/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", line 369, in <setcomp>

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
|     return {u for d in dependencies for u in check_dependency(d)}
|            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|   File

"/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", line 185, in check_dependency

Quoted 1 line
Quoted 1 line
Quoted 1 line

| yield from check_dependency(other_req_string,

ancestral_req_strings + (normalised_req_string,), req.extras)

Quoted 1 line
Quoted 1 line
Quoted 1 line

| File

"/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", line 179, in check_dependency

Quoted 1 line
Quoted 1 line
Quoted 1 line

| if req.specifier and not req.specifier.contains(dist.version,

prereleases=True):

Quoted 1 line
Quoted 1 line
Quoted 1 line

|

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Quoted 1 line
Quoted 1 line
Quoted 1 line

| File

"/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/packaging/specifiers.py", line 905, in contains

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
|     item = Version(item)
|            ^^^^^^^^^^^^^
|   File

"/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/packaging/version.py", line 196, in __init__

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
|     match = self._regex.search(version)
|             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
| TypeError: expected string or bytes-like object, got 'NoneType'
|
| ERROR expected string or bytes-like object, got 'NoneType'
| WARNING: exit code 1 from a shell command.

ERROR: Task

(/media/build/poky/meta/recipes-devtools/python/python3-iniconfig_2.0.0.bb:do_compile) failed with exit code '1'

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

I wondered why and I isolated it down to this:

richard@jet:/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages$

find -type d -empty

Quoted 9 lines
Quoted 9 lines
Quoted 9 lines

./hatchling-1.13.0.dist-info/licenses
./pathspec-0.11.0.dist-info
./importlib_metadata-6.0.0.dist-info
./setuptools-67.4.0.dist-info
./wheel-0.38.4.dist-info
./installer-0.6.0.dist-info
./setuptools-67.6.0.dist-info
./packaging-23.0.dist-info richard@jet:/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages$

find -type d -empty -delete

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

After which the compile succeeds. These empty dist-info directories really upset the build. They're left behind by the sstate cleanup code from when I have older versions of these python modules in play since the sstate code can race over directory removal. Is there a way to

make

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

python ignore them? Otherwise should we remove them somewhere?

I suspect you can recreate by adding these empty directories and running a -c compile -f. I've seen it with plenty of other python modules too.

Cheers,

Quoted 7 lines
Quoted 7 lines
Quoted 6 lines

Richard

--
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#182013):
https://lists.openembedded.org/g/openembedded-core/message/182013
Mute This Topic: https://lists.openembedded.org/mt/99241972/924729
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ [email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

#6 lore raw
Quoted 41 lines · Tim Orling
Quoted 37 lines · Bruce Ashfield
Quoted 12 lines · Trevor Gamblin
Quoted 9 lines · Bruce Ashfield
Quoted 7 lines · Richard Purdie

Hi Trevor,

I thought this issue might be something you might be able to spot the solution with quickly.

I had a failing python3-iniconfig build in a TMPDIR I've reused heavily:

Quoted 9 lines
Quoted 5 lines

FWIW. I see this in about 6 or 7 python recipes in my extremely heavily reused tmpdir.

After I came back from vacation, I iterated through a -c clean on the python packages, and they always built the second time through.

Do you recall which those are, and if it's the same ones each time it happens? I'm wondering if it may be an issue with some recipes.

roughly this set (as grepped from my history):

 2783  bitbake -c clean python3-attrs; bitbake core-image-minimal
 2828  bitbake -c clean python3-colorama; bitbake core-image-minimal
 2829  bitbake -c clean python3-jsonschema; bitbake core-image-minimal
 2943  bitbake -c clean python3-hatch-vcs python3-hatch-vcs-native;
bitbake core-image-minimal
 2944  bitbake -c clean python3-attrs; bitbake core-image-minimal
 2945  bitbake -c clean python3-urllib3; bitbake core-image-minimal
 2946  bitbake -c clean python3-colorama; bitbake core-image-minimal
 2947  bitbake -c clean python3-jsonschema; bitbake core-image-minimal
 2948  bitbake -c clean python3-iniconfig; bitbake core-image-minimal
Quoted 6 lines
Quoted 4 lines

Apparently I had particular issues with colorama :)

Bruce

Do you recall when you started seeing this behavior? I’m wondering if it is related to some of the wheel build changes rburton made. But that’s just my spider sense…

As we can all attest, timelines get blurry very quickly ... but I recall it popping up with the same / similar error message starting in the last 6 months.

Bruce

Quoted 102 lines
Quoted 102 lines
Quoted 86 lines

I haven't been able to recreate the issue yet, but my tmpdir is not very old. When I attempt the suggested reproducer (creating the empty directories and rerunning do_compile), it is successful. I'll keep poking at it.

Trevor

Quoted 77 lines

Bruce

Quoted 73 lines

ERROR: python3-iniconfig-2.0.0-r0 do_compile: ExecutionError('/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/temp/run.do_compile.2391637', 1, None, None)
ERROR: Logfile of failure stored in: /media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/temp/log.do_compile.2391637
Log data follows:

| DEBUG: Executing shell function do_compile
| * Getting build dependencies for wheel...
|
| Traceback (most recent call last):
|   File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", line 375, in main
|     built = build_call(
|             ^^^^^^^^^^^
|   File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", line 208, in build_package
|     out = _build(isolation, builder, outdir, distribution, config_settings, skip_dependency_check)
|           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|   File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", line 147, in _build
|     return _build_in_current_env(builder, outdir, distribution, config_settings, skip_dependency_check)
|            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|   File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", line 127, in _build_in_current_env
|     missing = builder.check_dependencies(distribution)
|               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|   File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", line 369, in check_dependencies
|     return {u for d in dependencies for u in check_dependency(d)}
|            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|   File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", line 369, in <setcomp>
|     return {u for d in dependencies for u in check_dependency(d)}
|            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|   File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", line 185, in check_dependency
|     yield from check_dependency(other_req_string, ancestral_req_strings + (normalised_req_string,), req.extras)
|   File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", line 179, in check_dependency
|     if req.specifier and not req.specifier.contains(dist.version, prereleases=True):
|                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|   File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/packaging/specifiers.py", line 905, in contains
|     item = Version(item)
|            ^^^^^^^^^^^^^
|   File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/packaging/version.py", line 196, in __init__
|     match = self._regex.search(version)
|             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
| TypeError: expected string or bytes-like object, got 'NoneType'
|
| ERROR expected string or bytes-like object, got 'NoneType'
| WARNING: exit code 1 from a shell command.

ERROR: Task (/media/build/poky/meta/recipes-devtools/python/python3-iniconfig_2.0.0.bb:do_compile) failed with exit code '1'

I wondered why and I isolated it down to this:

richard@jet:/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages$ find -type d -empty
./hatchling-1.13.0.dist-info/licenses
./pathspec-0.11.0.dist-info
./importlib_metadata-6.0.0.dist-info
./setuptools-67.4.0.dist-info
./wheel-0.38.4.dist-info
./installer-0.6.0.dist-info
./setuptools-67.6.0.dist-info
./packaging-23.0.dist-info
richard@jet:/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages$ find -type d -empty -delete

After which the compile succeeds. These empty dist-info directories really upset the build. They're left behind by the sstate cleanup code from when I have older versions of these python modules in play since the sstate code can race over directory removal. Is there a way to make python ignore them? Otherwise should we remove them somewhere?

I suspect you can recreate by adding these empty directories and running a -c compile -f. I've seen it with plenty of other python modules too.

Cheers,

Richard

--
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#182013): https://lists.openembedded.org/g/openembedded-core/message/182013
Mute This Topic: https://lists.openembedded.org/mt/99241972/924729
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

Signature · 3 lines
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II
#7 lore raw

I’m regularly seeing this when updating various python recipes. It’s caused by empty versioned directories in sysroots that are leftovers from previous versions. Doesn’t happen in freshly made tmpdirs.

Alex

Quoted 213 lines · Bruce Ashfield
Quoted 4 lines · Tim Orling

On Wed, May 31, 2023 at 7:25 AM Bruce Ashfield <[email protected]>

Quoted 2 lines · wrote:
Quoted 2 lines

On Wed, May 31, 2023 at 10:20 AM Trevor Gamblin <[email protected]>

Quoted 8 lines · wrote:
Quoted 8 lines
Quoted 8 lines · Bruce Ashfield
Quoted 5 lines
Quoted 3 lines · Richard Purdie

Hi Trevor,

I thought this issue might be something you might be able to spot

the

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 4 lines

solution with quickly.

I had a failing python3-iniconfig build in a TMPDIR I've reused heavily:

FWIW. I see this in about 6 or 7 python recipes in my extremely heavily reused tmpdir.

After I came back from vacation, I iterated through a -c clean on

the

Quoted 19 lines
Quoted 19 lines
Quoted 5 lines
Quoted 1 line

python packages, and they always built the second time through.

Do you recall which those are, and if it's the same ones each time it happens? I'm wondering if it may be an issue with some recipes.

roughly this set (as grepped from my history):

 2783  bitbake -c clean python3-attrs; bitbake core-image-minimal
 2828  bitbake -c clean python3-colorama; bitbake core-image-minimal
 2829  bitbake -c clean python3-jsonschema; bitbake core-image-minimal
 2943  bitbake -c clean python3-hatch-vcs python3-hatch-vcs-native;
bitbake core-image-minimal
 2944  bitbake -c clean python3-attrs; bitbake core-image-minimal
 2945  bitbake -c clean python3-urllib3; bitbake core-image-minimal
 2946  bitbake -c clean python3-colorama; bitbake core-image-minimal
 2947  bitbake -c clean python3-jsonschema; bitbake core-image-minimal
 2948  bitbake -c clean python3-iniconfig; bitbake core-image-minimal
Quoted 6 lines
Quoted 4 lines

Apparently I had particular issues with colorama :)

Bruce

Do you recall when you started seeing this behavior? I’m wondering if it

is related to some of the wheel build changes rburton made. But that’s just my spider sense…

As we can all attest, timelines get blurry very quickly ... but I recall it popping up with the same / similar error message starting in the last 6 months.

Bruce

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

I haven't been able to recreate the issue yet, but my tmpdir is not

very

Quoted 10 lines
Quoted 10 lines
Quoted 10 lines

old. When I attempt the suggested reproducer (creating the empty directories and rerunning do_compile), it is successful. I'll keep poking at it.

Trevor

Quoted 4 lines

Bruce

Quoted 1 line

ERROR: python3-iniconfig-2.0.0-r0 do_compile:

ExecutionError('/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/temp/run.do_compile.2391637', 1, None, None)

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

ERROR: Logfile of failure stored in:

/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/temp/log.do_compile.2391637

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

Log data follows:

| DEBUG: Executing shell function do_compile
| * Getting build dependencies for wheel...
|
| Traceback (most recent call last):
|   File

"/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", line 375, in main

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
|     built = build_call(
|             ^^^^^^^^^^^
|   File

"/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", line 208, in build_package

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

| out = _build(isolation, builder, outdir, distribution,

config_settings, skip_dependency_check)

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

|

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

| File

"/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", line 147, in _build

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

| return _build_in_current_env(builder, outdir, distribution,

config_settings, skip_dependency_check)

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

|

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

| File

"/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", line 127, in _build_in_current_env

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
|     missing = builder.check_dependencies(distribution)
|               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|   File

"/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", line 369, in check_dependencies

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
|     return {u for d in dependencies for u in check_dependency(d)}
|            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|   File

"/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", line 369, in <setcomp>

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
|     return {u for d in dependencies for u in check_dependency(d)}
|            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|   File

"/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", line 185, in check_dependency

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

| yield from check_dependency(other_req_string,

ancestral_req_strings + (normalised_req_string,), req.extras)

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

| File

"/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", line 179, in check_dependency

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

| if req.specifier and not

req.specifier.contains(dist.version, prereleases=True):

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

|

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

| File

"/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/packaging/specifiers.py", line 905, in contains

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
|     item = Version(item)
|            ^^^^^^^^^^^^^
|   File

"/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/packaging/version.py", line 196, in __init__

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
|     match = self._regex.search(version)
|             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
| TypeError: expected string or bytes-like object, got 'NoneType'
|
| ERROR expected string or bytes-like object, got 'NoneType'
| WARNING: exit code 1 from a shell command.

ERROR: Task

(/media/build/poky/meta/recipes-devtools/python/python3-iniconfig_2.0.0.bb:do_compile) failed with exit code '1'

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

I wondered why and I isolated it down to this:

richard@jet:/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages$

find -type d -empty

Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines

./hatchling-1.13.0.dist-info/licenses
./pathspec-0.11.0.dist-info
./importlib_metadata-6.0.0.dist-info
./setuptools-67.4.0.dist-info
./wheel-0.38.4.dist-info
./installer-0.6.0.dist-info
./setuptools-67.6.0.dist-info
./packaging-23.0.dist-info richard@jet:/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages$

find -type d -empty -delete

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

After which the compile succeeds. These empty dist-info directories really upset the build. They're left behind by the sstate cleanup

code

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

from when I have older versions of these python modules in play

since

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

the sstate code can race over directory removal. Is there a way to

make

Quoted 18 lines
Quoted 18 lines
Quoted 15 lines
Quoted 15 lines
Quoted 14 lines

python ignore them? Otherwise should we remove them somewhere?

I suspect you can recreate by adding these empty directories and running a -c compile -f. I've seen it with plenty of other python modules too.

Cheers,

Richard

Quoted 7 lines
Quoted 7 lines

--
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

--
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#182016):
https://lists.openembedded.org/g/openembedded-core/message/182016
Mute This Topic: https://lists.openembedded.org/mt/99241972/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ [email protected]]
-=-=-=-=-=-=-=-=-=-=-=-