python3-native: Fix pip install issue due to unclean build directory
A message that is not in the archive (sent off-list or not yet received): [email protected]
PATCH 1/1 python3-native: Fix pip install issue due to unclean build directory
When installing python3-native sometimes pips default build directory (which is on the host and is user dependant) is left unclean, due to this, when python3-core is being installed it tries to use the same directory producing an error, this explicitly removes what the previous installation might have left behind, fixing the issue.
--- .../python/python3-native_3.4.2.bb | 1 + .../python3/pip_build_directory_unclean.patch | 28 ++++++++++++++++++++++ 2 files changed, 29 insertions(+) create mode 100644 meta/recipes-devtools/python/python3/pip_build_directory_unclean.patch
meta/recipes-devtools/python/python3-native_3.4.2.bb +1 -0
@@ -26,6 +26,7 @@ file://makerace.patch \ ${DISTRO_SRC_URI} \ file://sysconfig.py-add-_PYTHON_PROJECT_SRC.patch \ file://setup.py-check-cross_compiling-when-get-FLAGS.patch \ +file://pip_build_directory_unclean.patch \ " SRC_URI[md5sum] = "36fc7327c02c6f12fa24fc9ba78039e3" SRC_URI[sha256sum] = "1c6d9682d145c056537e477bbfa060ce727f9edd38df1827e0f970dcf04b2def"
meta/recipes-devtools/python/python3/pip_build_directory_unclean.patch new file +28 -0
@@ -0,0 +1,28 @@ +Index: Python-3.4.2/Lib/ensurepip/__init__.py +=================================================================== +--- Python-3.4.2.orig/Lib/ensurepip/__init__.py ++++ Python-3.4.2/Lib/ensurepip/__init__.py +@@ -3,6 +3,7 @@ import os.path + import pkgutil + import sys + import tempfile ++import pwd + + + __all__ = ["version", "bootstrap"] +@@ -36,6 +37,15 @@ def _run_pip(args, additional_paths=None + if additional_paths is not None: + sys.path = additional_paths + sys.path + ++ # Explicitly make sure pip build directory was left clean ++ user = pwd.getpwuid(os.getuid())[0] ++ if os.path.exists("/tmp/pip_build_" + user + "/pip-delete-this-directory.txt"): ++ os.remove("/tmp/pip_build_" + user + "/pip-delete-this-directory.txt") ++ try: ++ os.rmdir("/tmp/pip_build_" + user + "/") ++ except OSError: ++ print("Error: Pip build directory (%s) was left unclean, make sure it is clean before continuing\n" % ("/tmp/pip_build_" + user + "/")) ++ + # Install the bundled software + import pip + pip.main(args)
Signature · 1 line
1.8.4.5
Quoted 6 lines · Alejandro Hernandez
When installing python3-native sometimes pips default build directory (which is on the host and is user dependant) is left unclean, due to this, when python3-core is being installed it tries to use the same directory producing an error, this explicitly removes what the previous installation might have left behind, fixing the issue.
Quoted 6 lines
Signed-off-by Alejandro Hernandez <[email protected]>--- .../python/python3-native_3.4.2.bb | 1 + .../python3/pip_build_directory_unclean.patch | 28 ++++++++++++++++++++++ 2 files changed, 29 insertions(+) create mode 100644 meta/recipes-devtools/python/python3/pip_build_directory_unclean.patch
The problem here is that two builds of python3-native could be happening on the same system at the same time. In fact I'd bet that is why its breaking on the autobuilder. Can we not tell it to use something in WORKDIR as its temp directory instead?
Cheers,
Richard
Quoted 13 lines · Richard Purdie
Quoted 6 lines · Alejandro Hernandez
When installing python3-native sometimes pips default build directory (which is on the host and is user dependant) is left unclean, due to this, when python3-core is being installed it tries to use the same directory producing an error, this explicitly removes what the previous installation might have left behind, fixing the issue.
Quoted 6 lines
Signed-off-by Alejandro Hernandez <[email protected]>--- .../python/python3-native_3.4.2.bb | 1 + .../python3/pip_build_directory_unclean.patch | 28 ++++++++++++++++++++++ 2 files changed, 29 insertions(+)create mode 100644
meta/recipes-devtools/python/python3/pip_build_directory_unclean.patch
Quoted 8 lines
The problem here is that two builds of python3-native could be happening on the same system at the same time. In fact I'd bet that is why its breaking on the autobuilder. Can we not tell it to use something in WORKDIR as its temp directory instead?
Cheers,
Richard
Hmm, that's potentially what is happening, in fact, my first thought was to do that and use something within the WORKDIR, but I wasn't able to do that because they do something weird to install pip, they bootstrap the installation and they use python wheels, so the actual part of the code where one could specify a different build directory for pip is inside one of the wheels, making it complicated to modify. The only other thing that occurs to me would be to just make it sleep or make it wait somehow (we could modify the queue?) until the directory is empty.
I still think this should work, I only saw cases where the build directory is left with one empty file, another reason why I decided to explicitly delete it instead of forcing it to delete everything inside the directory, disabling pip is not an option because it is required since python 3.4 as I stated on the upgrade patch, we could try this, and if it doesn't work I'll find another workaround, but let me know what you think.
Alejandro
Quoted 32 lines · Alejandro Hernandez
Quoted 13 lines · Richard Purdie
Quoted 6 lines · Alejandro Hernandez
When installing python3-native sometimes pips default build directory (which is on the host and is user dependant) is left unclean, due to this, when python3-core is being installed it tries to use the same directory producing an error, this explicitly removes what the previous installation might have left behind, fixing the issue.
Quoted 6 lines
Signed-off-by Alejandro Hernandez <[email protected]>--- .../python/python3-native_3.4.2.bb | 1 + .../python3/pip_build_directory_unclean.patch | 28 ++++++++++++++++++++++ 2 files changed, 29 insertions(+)create mode 100644
meta/recipes-devtools/python/python3/pip_build_directory_unclean.patchQuoted 8 lines
The problem here is that two builds of python3-native could be happening on the same system at the same time. In fact I'd bet that is why its breaking on the autobuilder. Can we not tell it to use something in WORKDIR as its temp directory instead?
Cheers,
Richard
Hmm, that's potentially what is happening, in fact, my first thought was to do that and use something within the WORKDIR, but I wasn't able to do that because they do something weird to install pip, they bootstrap the installation and they use python wheels, so the actual part of the code where one could specify a different build directory for pip is inside one of the wheels, making it complicated to modify.
The only other thing that occurs to me would be to just make it sleep or make it wait somehow (we could modify the queue?) until the directory is empty.
Quoted 7 lines
I still think this should work, I only saw cases where the build directory is left with one empty file, another reason why I decided to explicitly delete it instead of forcing it to delete everything inside the directory, disabling pip is not an option because it is required since python 3.4 as I stated on the upgrade patch, we could try this, and if it doesn't work I'll find another workaround, but let me know what you think.
I tried it on the autobuilder and it failed again. I think part of the problem is that this part of the process runs under pseudo so it thinks its root, yet can't clear root's pip directory in /tmp.
I suspect we are going to have to tell pip to use WORKDIR somehow, even if we have to patch it to take some kind of prompt from the environment for example.
Cheers,
Richard
Quoted 19 lines · Richard Purdie
Quoted 8 lines · Alejandro Hernandez
Quoted 7 lines · Richard Purdie
Quoted 6 lines · Alejandro Hernandez
When installing python3-native sometimes pips default build directory (which is on the host and is user dependant) is left unclean, due to this, when python3-core is being installed it tries to use the same directory producing an error, this explicitly removes what the previous installation might have left behind, fixing the issue.
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Signed-off-by Alejandro Hernandez <[email protected]>--- .../python/python3-native_3.4.2.bb | 1 + .../python3/pip_build_directory_unclean.patch | 28 ++++++++++++++++++++++ 2 files changed, 29 insertions(+)create mode 100644
meta/recipes-devtools/python/python3/pip_build_directory_unclean.patchI tried it on the autobuilder and it failed again. I think part of the problem is that this part of the process runs under pseudo so it thinks its root, yet can't clear root's pip directory in /tmp.
Quoted 8 lines
I suspect we are going to have to tell pip to use WORKDIR somehow, even if we have to patch it to take some kind of prompt from the environment for example.
Cheers,
Richard
Yes, the problem does happen when it uses pseudo and it thinks its root, I thought removing the files inside the same python code would be able to do it, ok I will find another workaround.