Skip to contentSkip to conversation

python-pip: Improve reproducibility

PATCH New Richard Purdie 1 participant 1 message Feb 24, 2022

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 python-pip: Improve reproducibility

Pip installed wheels are not reproducible currently. The direct_url files encode an installation path and the installed wheels compile the python files at their location, not their final install location which is incorrect.

To fix this, simply disable the direct_urls and pass the "root" to the python compile function to strip that path out of the compiled files.

---
 .../python/python3-pip/reproducible.patch     | 74 +++++++++++++++++++
 .../python/python3-pip_22.0.3.bb              |  1 +
 2 files changed, 75 insertions(+)
 create mode 100644 meta/recipes-devtools/python/python3-pip/reproducible.patch
meta/recipes-devtools/python/python3-pip/reproducible.patch new file +74 -0
new file mode 100644
index 00000000000..538bb94f7ad
@@ -0,0 +1,74 @@
+Pip installed wheels are not reproducible currently. The direct_url
+files encode an installation path and the installed wheels compile
+the python files at their location, not their final install location
+which is incorrect.
+
+To fix this, simply disable the direct_urls and pass the "root" to
+the python compile function to strip that path out of the compiled
+files.
+
+A version of this patch, perhaps stripping root from the direct_urls
+may be something that could be considered by upstream.
+
+Signed-off-by: Richard Purdie <[email protected]>
+
+Upstream-Status: Pending
+
+Index: pip-22.0.3/src/pip/_internal/req/req_install.py
+===================================================================
+--- pip-22.0.3.orig/src/pip/_internal/req/req_install.py
++++ pip-22.0.3/src/pip/_internal/req/req_install.py
+@@ -758,7 +758,9 @@ class InstallRequirement:
+         if self.is_wheel:
+             assert self.local_file_path
+             direct_url = None
+-            if self.editable:
++            if '_PYTHON_SYSCONFIGDATA_NAME' in os.environ:
++                direct_url = None
++            elif self.editable:
+                 direct_url = direct_url_for_editable(self.unpacked_source_directory)
+             elif self.original_link:
+                 direct_url = direct_url_from_link(
+@@ -775,6 +777,7 @@ class InstallRequirement:
+                 warn_script_location=warn_script_location,
+                 direct_url=direct_url,
+                 requested=self.user_supplied,
++                root=root,
+             )
+             self.install_succeeded = True
+             return
+Index: pip-22.0.3/src/pip/_internal/operations/install/wheel.py
+===================================================================
+--- pip-22.0.3.orig/src/pip/_internal/operations/install/wheel.py
++++ pip-22.0.3/src/pip/_internal/operations/install/wheel.py
+@@ -436,6 +436,7 @@ def _install_wheel(
+     warn_script_location: bool = True,
+     direct_url: Optional[DirectUrl] = None,
+     requested: bool = False,
++    root: str = None,
+ ) -> None:
+     """Install a wheel.
+ 
+@@ -612,7 +613,7 @@ def _install_wheel(
+             with warnings.catch_warnings():
+                 warnings.filterwarnings("ignore")
+                 for path in pyc_source_file_paths():
+-                    success = compileall.compile_file(path, force=True, quiet=True)
++                    success = compileall.compile_file(path, force=True, quiet=True, stripdir=root)
+                     if success:
+                         pyc_path = pyc_output_path(path)
+                         assert os.path.exists(pyc_path)
+@@ -723,6 +724,7 @@ def install_wheel(
+     warn_script_location: bool = True,
+     direct_url: Optional[DirectUrl] = None,
+     requested: bool = False,
++    root: str = None,
+ ) -> None:
+     with ZipFile(wheel_path, allowZip64=True) as z:
+         with req_error_context(req_description):
+@@ -735,4 +737,5 @@ def install_wheel(
+                 warn_script_location=warn_script_location,
+                 direct_url=direct_url,
+                 requested=requested,
++                root=root,
+             )
meta/recipes-devtools/python/python3-pip_22.0.3.bb +1 -0
index e02ea5bd3d2..7eb9fb69ba4 100644
@@ -14,6 +14,7 @@ DEPENDS:append:class-native = " unzip-native"
 
 SRC_URI += "file://0001-change-shebang-to-python3.patch"
 SRC_URI += "file://no_shebang_mangling.patch"
+SRC_URI += "file://reproducible.patch"
 
 SRC_URI[sha256sum] = "f29d589df8c8ab99c060e68ad294c4a9ed896624f6368c5349d70aa581b333d0"
 
Signature · 1 line
2.32.0