Skip to contentSkip to conversation

classes-recipe/python_pep517: remove all RECORDS files

PATCH In review Ross Burton 3 participants 4 messages Nov 20, 2024

lore
Share & tools
lore
#0 lore raw

PATCH 1/2 classes-recipe/python_pep517: remove all RECORDS files

Python modules install metadata into a .dist-info directory, one of which is RECORD, which contains the files that were installed and their checksum[1]. This is typically used by pip to validate the install, or to know what files to remove when the module is uninstalled.

This is slightly problematic when we need to do patching of installed .py files in do_install(), as the RECORD file has already been written at that point.

However, the RECORD files only really have a use outside of a system- managed environment, which our python packages are. We already have commands to verify and remove modules (opkg, dpkg, rpm) and the RECORD file existing simply allows people to 'sudo pip' and alter the package- managed directories outside of the package manager.

This is not a good idea, and some other distros remove the RECORD file to stop this possibility:
- Debian[2]
- Fedora[3]
- Gentoo[4]

We can follow for all packages which inherit python_pep517, which is the majority of the Python packages now.

[1] https://peps.python.org/pep-0491/#the-dist-info-directory
[2] https://salsa.debian.org/python-team/tools/dh-python/-/blob/master/dhpython/fs.py?ref_type=heads#L185
[3] https://src.fedoraproject.org/rpms/pyproject-rpm-macros/blob/rawhide/f/macros.pyproject#_105
[4] https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=73c49f3c00415dee99407dabba8d3b22895c9d25

---
 meta/classes-recipe/python_pep517.bbclass | 2 ++
 1 file changed, 2 insertions(+)
meta/classes-recipe/python_pep517.bbclass +2 -0
index c30674c8ec8..e8cd1923ef2 100644
@@ -50,6 +50,8 @@ python_pep517_do_install () {
     fi
 
     nativepython3 -m installer ${INSTALL_WHEEL_COMPILE_BYTECODE} --interpreter "${USRBINPATH}/env ${PEP517_INSTALL_PYTHON}" --destdir=${D} ${PEP517_WHEEL_PATH}/*.whl
+
+    find ${D} -path *.dist-info/RECORD -delete
 }
 
 # A manual do_install that just uses unzip for bootstrapping purposes. Callers should DEPEND on unzip-native.
Signature · 1 line
2.34.1
#1 lore raw

PATCH 2/2 python3-cryptography-vectors: don't remove RECORD

We remove this file in the python_pep517 class now, so we don't need to do it here.

---
 .../python/python3-cryptography-vectors_43.0.1.bb            | 5 -----
 1 file changed, 5 deletions(-)
meta/recipes-devtools/python/python3-cryptography-vectors_43.0.1.bb +0 -5
index a1a10b7621b..1ad9a8ab867 100644
@@ -19,11 +19,6 @@ DEPENDS += " \
     python3-cryptography \
 "
 
-do_install:append () {
-	# Remove the sha256 checksum lines for pycache files
-	sed ${D}${PYTHON_SITEPACKAGES_DIR}/cryptography_vectors-${PV}.dist-info/RECORD -e '/__pycache__/d' -i
-}
-
 BBCLASSEXTEND = "native nativesdk"
 
 UPSTREAM_CHECK_REGEX = ""
Signature · 1 line
2.34.1
#2 lore raw
In reply to Ross Burton
Quoted 24 lines · Ross Burton via lists.openembedded.org

Python modules install metadata into a .dist-info directory, one of which is RECORD, which contains the files that were installed and their checksum[1]. This is typically used by pip to validate the install, or to know what files to remove when the module is uninstalled.

This is slightly problematic when we need to do patching of installed .py files in do_install(), as the RECORD file has already been written at that point.

However, the RECORD files only really have a use outside of a system- managed environment, which our python packages are. We already have commands to verify and remove modules (opkg, dpkg, rpm) and the RECORD file existing simply allows people to 'sudo pip' and alter the package- managed directories outside of the package manager.

This is not a good idea, and some other distros remove the RECORD file to stop this possibility:
- Debian[2]
- Fedora[3]
- Gentoo[4]

We can follow for all packages which inherit python_pep517, which is the majority of the Python packages now.

Quoted 6 lines

[1] https://peps.python.org/pep-0491/#the-dist-info-directory
[2] https://salsa.debian.org/python-team/tools/dh-python/-/blob/master/dhpython/fs.py?ref_type=heads#L185
[3] https://src.fedoraproject.org/rpms/pyproject-rpm-macros/blob/rawhide/f/macros.pyproject#_105
[4] https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=73c49f3c00415dee99407dabba8d3b22895c9d25

As discussed in IRC, LGTM.

Quoted 26 lines
---
  meta/classes-recipe/python_pep517.bbclass | 2 ++
  1 file changed, 2 insertions(+)
meta/classes-recipe/python_pep517.bbclass +2 -1
index c30674c8ec8..e8cd1923ef2 100644
@@ -50,6 +50,8 @@ python_pep517_do_install () {
      fi
 
      nativepython3 -m installer ${INSTALL_WHEEL_COMPILE_BYTECODE} --interpreter "${USRBINPATH}/env ${PEP517_INSTALL_PYTHON}" --destdir=${D} ${PEP517_WHEEL_PATH}/*.whl
+
+    find ${D} -path *.dist-info/RECORD -delete
  }
 
  # A manual do_install that just uses unzip for bootstrapping purposes. Callers should DEPEND on unzip-native.
 
-=-=-=-=-=-=-=-=-=-=-=-

Links: You receive all messages sent to this group.
View/Reply Online (#207479): https://lists.openembedded.org/g/openembedded-core/message/207479
Mute This Topic: https://lists.openembedded.org/mt/109691086/7611679
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

Reviewed-by
#3 lore raw
Quoted 28 lines · Trevor Gamblin via lists.openembedded.org
Quoted 21 lines · Ross Burton via lists.openembedded.org

Python modules install metadata into a .dist-info directory, one of which is RECORD, which contains the files that were installed and their checksum[1]. This is typically used by pip to validate the install, or to know what files to remove when the module is uninstalled.

This is slightly problematic when we need to do patching of installed .py files in do_install(), as the RECORD file has already been written at that point.

However, the RECORD files only really have a use outside of a system- managed environment, which our python packages are. We already have commands to verify and remove modules (opkg, dpkg, rpm) and the RECORD file existing simply allows people to 'sudo pip' and alter the package- managed directories outside of the package manager.

This is not a good idea, and some other distros remove the RECORD file to stop this possibility:
- Debian[2]
- Fedora[3]
- Gentoo[4]

Quoted 5 lines

We can follow for all packages which inherit python_pep517, which is the majority of the Python packages now.

[1] https://peps.python.org/pep-0491/#the-dist-info-directory
[2]

Quoted 8 lines

https://salsa.debian.org/python-team/tools/dh-python/-/blob/master/dhpython/fs.py?ref_type=heads#L185

Quoted 1 line

[3]

https://src.fedoraproject.org/rpms/pyproject-rpm-macros/blob/rawhide/f/macros.pyproject#_105

Quoted 1 line

[4]

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=73c49f3c00415dee99407dabba8d3b22895c9d25

Quoted 2 lines

As discussed in IRC, LGTM.

Let us keep sane by following the upstream distributions lead. Thank you Ross!

Quoted 37 lines

Review

Quoted 6 lines
---
  meta/classes-recipe/python_pep517.bbclass | 2 ++
  1 file changed, 2 insertions(+)
(quoted diff) +0 -0

b/meta/classes-recipe/python_pep517.bbclass

meta/classes-recipe/python_pep517.bbclass python_pep517_do_install () { · line 52

Quoted 7 lines

index c30674c8ec8..e8cd1923ef2 100644

meta/classes-recipe/python_pep517.bbclass +0 -0
@@ -50,6 +50,8 @@ python_pep517_do_install () {
      fi
 
      nativepython3 -m installer ${INSTALL_WHEEL_COMPILE_BYTECODE}

--interpreter "${USRBINPATH}/env ${PEP517_INSTALL_PYTHON}" --destdir=${D} ${PEP517_WHEEL_PATH}/*.whl

meta/classes-recipe/python_pep517.bbclass python_pep517_do_install () {

Quoted 5 lines
meta/classes-recipe/python_pep517.bbclass +2 -0
+
+    find ${D} -path *.dist-info/RECORD -delete
  }
 
  # A manual do_install that just uses unzip for bootstrapping purposes.

Callers should DEPEND on unzip-native.

Quoted 3 lines

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#207488):
https://lists.openembedded.org/g/openembedded-core/message/207488
Mute This Topic: https://lists.openembedded.org/mt/109691086/924729
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ [email protected]]
-=-=-=-=-=-=-=-=-=-=-=-