Skip to contentSkip to conversation

Remove dependency on the Python module distutils

PATCH Superseded James Carter 3 participants 9 messages Oct 28, 2022 – Nov 1, 2022 SELinux security module

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

cover Remove dependency on the Python module distutils

The distutils package is deprecated and scheduled to be removed in Python 3.12.

When building the SELinux userspace we currently get warnings like the following:
<string>:1: DeprecationWarning: The distutils.sysconfig module is deprecated, use sysconfig instead
<string>:1: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives

I am not a Python expert, but everything seems to work with these patches. I would appreciate other people testing this and any Python experts to let me know if this is the right way of replacing distutils.

James Carter (5):
libselinux: Remove dependency on the Python module distutils
libsemanage: Remove dependency on the Python module distutils
python: Remove dependency on the Python module distutils
scripts: Remove dependency on the Python module distutils README.md: Remove mention of python3-distutils dependency

 README.md                             | 1 -
 libselinux/src/Makefile               | 2 +-
 libselinux/src/setup.py               | 2 +-
 libsemanage/src/Makefile              | 2 +-
 python/semanage/Makefile              | 2 +-
 python/sepolgen/src/sepolgen/Makefile | 2 +-
 python/sepolicy/sepolicy/gui.py       | 2 +-
 python/sepolicy/setup.py              | 2 +-
 scripts/env_use_destdir               | 2 +-
 scripts/run-scan-build                | 2 +-
 10 files changed, 9 insertions(+), 10 deletions(-)
Signature · 1 line
2.37.3
#1 lore raw

PATCH 1/5 libselinux: Remove dependency on the Python module distutils

The distutils package is deprecated and scheduled to be removed in Python 3.12. Use the setuptools and sysconfig modules instead.

---
 libselinux/src/Makefile | 2 +-
 libselinux/src/setup.py | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
libselinux/src/Makefile +1 -1
index 49b7d60a..dc284832 100644
@@ -14,7 +14,7 @@ SHLIBDIR ?= /lib
 INCLUDEDIR ?= $(PREFIX)/include
 PYINC ?= $(shell $(PKG_CONFIG) --cflags $(PYPREFIX))
 PYLIBS ?= $(shell $(PKG_CONFIG) --libs $(PYPREFIX))
-PYTHONLIBDIR ?= $(shell $(PYTHON) -c "from distutils.sysconfig import *; print(get_python_lib(plat_specific=1, prefix='$(PREFIX)'))")
+PYTHONLIBDIR ?= $(shell $(PYTHON) -c "import sysconfig; print(sysconfig.get_path('platlib', vars={'platbase': '$(PREFIX)', 'base': '$(PREFIX)'}))")
 PYCEXT ?= $(shell $(PYTHON) -c 'import importlib.machinery;print(importlib.machinery.EXTENSION_SUFFIXES[0])')
 RUBYINC ?= $(shell $(RUBY) -e 'puts "-I" + RbConfig::CONFIG["rubyarchhdrdir"] + " -I" + RbConfig::CONFIG["rubyhdrdir"]')
 RUBYLIBS ?= $(shell $(RUBY) -e 'puts "-L" + RbConfig::CONFIG["libdir"] + " -L" + RbConfig::CONFIG["archlibdir"] + " " + RbConfig::CONFIG["LIBRUBYARG_SHARED"]')
libselinux/src/setup.py +1 -1
index 71e69a10..6cbe3a0e 100644
@@ -1,6 +1,6 @@
 #!/usr/bin/python3
 
-from distutils.core import Extension, setup
+from setuptools import Extension, setup
 
 setup(
     name="selinux",
Signature · 1 line
2.37.3
#2 lore raw

PATCH 2/5 libsemanage: Remove dependency on the Python module distutils

In reply to James Carter

The distutils package is deprecated and scheduled to be removed in Python 3.12. Use the sysconfig module instead.

---
 libsemanage/src/Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
libsemanage/src/Makefile +1 -1
index 01df0181..589e4a70 100644
@@ -14,7 +14,7 @@ INCLUDEDIR ?= $(PREFIX)/include
 SYSCONFDIR ?= /etc
 PYINC ?= $(shell $(PKG_CONFIG) --cflags $(PYPREFIX))
 PYLIBS ?= $(shell $(PKG_CONFIG) --libs $(PYPREFIX))
-PYTHONLIBDIR ?= $(shell $(PYTHON) -c "from distutils.sysconfig import *; print(get_python_lib(plat_specific=1, prefix='$(PREFIX)'))")
+PYTHONLIBDIR ?= $(shell $(PYTHON) -c "import sysconfig; print(sysconfig.get_path('platlib', vars={'platbase': '$(PREFIX)', 'base': '$(PREFIX)'}))")
 PYCEXT ?= $(shell $(PYTHON) -c 'import importlib.machinery;print(importlib.machinery.EXTENSION_SUFFIXES[0])')
 RUBYINC ?= $(shell $(RUBY) -e 'puts "-I" + RbConfig::CONFIG["rubyarchhdrdir"] + " -I" + RbConfig::CONFIG["rubyhdrdir"]')
 RUBYLIBS ?= $(shell $(RUBY) -e 'puts "-L" + RbConfig::CONFIG["libdir"] + " -L" + RbConfig::CONFIG["archlibdir"] + " " + RbConfig::CONFIG["LIBRUBYARG_SHARED"]')
Signature · 1 line
2.37.3
#3 lore raw

PATCH 3/5 python: Remove dependency on the Python module distutils

In reply to James Carter

The distutils package is deprecated and scheduled to be removed in Python 3.12. Use the setuptools and sysconfig modules instead.

---
 python/semanage/Makefile              | 2 +-
 python/sepolgen/src/sepolgen/Makefile | 2 +-
 python/sepolicy/sepolicy/gui.py       | 2 +-
 python/sepolicy/setup.py              | 2 +-
 4 files changed, 4 insertions(+), 4 deletions(-)
python/semanage/Makefile +1 -1
index 024e9640..b1f1bd3a 100644
@@ -5,7 +5,7 @@ LINGUAS ?= ru
 PREFIX ?= /usr
 SBINDIR ?= $(PREFIX)/sbin
 MANDIR = $(PREFIX)/share/man
-PYTHONLIBDIR ?= $(shell $(PYTHON) -c "from distutils.sysconfig import *; print(get_python_lib(prefix='$(PREFIX)'))")
+PYTHONLIBDIR ?= $(shell $(PYTHON) -c "import sysconfig; print(sysconfig.get_path('platlib', vars={'platbase': '$(PREFIX)', 'base': '$(PREFIX)'}))")
 PACKAGEDIR ?= $(PYTHONLIBDIR)
 BASHCOMPLETIONDIR ?= $(PREFIX)/share/bash-completion/completions
 
python/sepolgen/src/sepolgen/Makefile +1 -1
index cac8def7..9578af23 100644
@@ -1,6 +1,6 @@
 PREFIX ?= /usr
 PYTHON ?= python3
-PYTHONLIBDIR ?= $(shell $(PYTHON) -c "from distutils.sysconfig import *; print(get_python_lib(prefix='$(PREFIX)'))")
+PYTHONLIBDIR ?= $(shell $(PYTHON) -c "import sysconfig; print(sysconfig.get_path('platlib', vars={'platbase': '$(PREFIX)', 'base': '$(PREFIX)'}))")
 PACKAGEDIR ?= /$(PYTHONLIBDIR)/sepolgen
 
 all:
python/sepolicy/sepolicy/gui.py +1 -1
index 5bdbfeba..63f2371f 100644
@@ -77,7 +77,7 @@ def cmp(a, b):
         return 1
     return (a > b) - (a < b)
 
-import distutils.sysconfig
+import sysconfig
 ADVANCED_LABEL = (_("Advanced >>"), _("Advanced <<"))
 ADVANCED_SEARCH_LABEL = (_("Advanced Search >>"), _("Advanced Search <<"))
 OUTBOUND_PAGE = 0
python/sepolicy/setup.py +1 -1
index b0f9650d..c8220664 100644
@@ -2,7 +2,7 @@
 
 # Author: Thomas Liu <[email protected]>
 # Author: Dan Walsh <[email protected]>
-from distutils.core import setup
+from setuptools import setup
 
 setup(
     name="sepolicy",
Signature · 1 line
2.37.3
#4 lore raw

Re: [PATCH 3/5] python: Remove dependency on the Python module distutils

Quoted 16 lines · James Carter

The distutils package is deprecated and scheduled to be removed in Python 3.12. Use the setuptools and sysconfig modules instead.

---
 python/semanage/Makefile              | 2 +-
 python/sepolgen/src/sepolgen/Makefile | 2 +-
 python/sepolicy/sepolicy/gui.py       | 2 +-
 python/sepolicy/setup.py              | 2 +-
 4 files changed, 4 insertions(+), 4 deletions(-)
python/semanage/Makefile +0 -0
index 024e9640..b1f1bd3a 100644

python/semanage/Makefile LINGUAS ?= ru · line 8

Quoted 5 lines
python/semanage/Makefile +1 -1
@@ -5,7 +5,7 @@ LINGUAS ?= ru
 PREFIX ?= /usr
 SBINDIR ?= $(PREFIX)/sbin
 MANDIR = $(PREFIX)/share/man
-PYTHONLIBDIR ?= $(shell $(PYTHON) -c "from distutils.sysconfig import *; print(get_python_lib(prefix='$(PREFIX)'))")
+PYTHONLIBDIR ?= $(shell $(PYTHON) -c "import sysconfig; print(sysconfig.get_path('platlib', vars={'platbase': '$(PREFIX)', 'base': '$(PREFIX)'}))")
$ python3 -c "from distutils.sysconfig import *; print(get_python_lib(prefix='/usr'))"     
/usr/lib/python3.10/site-packages

vs

$ python3 -c "import sysconfig; print(sysconfig.get_path('platlib', vars={'platbase': '/usr', 'base': '/usr'}))"
/usr/lib64/python3.10/site-packages

Given that python/ is not platform specific this should use 'purelib' -
https://docs.python.org/3/library/sysconfig.html#installation-paths

$ python3 -c "import sysconfig; print(sysconfig.get_path('purelib', vars={'platbase': '/usr', 'base': '/usr'}))"
/usr/lib/python3.10/site-packages
Quoted 43 lines
python/semanage/Makefile +0 -0
 PACKAGEDIR ?= $(PYTHONLIBDIR)
 BASHCOMPLETIONDIR ?= $(PREFIX)/share/bash-completion/completions
 
python/sepolgen/src/sepolgen/Makefile +1 -1
index cac8def7..9578af23 100644
@@ -1,6 +1,6 @@
 PREFIX ?= /usr
 PYTHON ?= python3
-PYTHONLIBDIR ?= $(shell $(PYTHON) -c "from distutils.sysconfig import *; print(get_python_lib(prefix='$(PREFIX)'))")
+PYTHONLIBDIR ?= $(shell $(PYTHON) -c "import sysconfig; print(sysconfig.get_path('platlib', vars={'platbase': '$(PREFIX)', 'base': '$(PREFIX)'}))")
 PACKAGEDIR ?= /$(PYTHONLIBDIR)/sepolgen
 
 all:
python/sepolicy/sepolicy/gui.py +1 -1
index 5bdbfeba..63f2371f 100644
@@ -77,7 +77,7 @@ def cmp(a, b):
         return 1
     return (a > b) - (a < b)
 
-import distutils.sysconfig
+import sysconfig
 ADVANCED_LABEL = (_("Advanced >>"), _("Advanced <<"))
 ADVANCED_SEARCH_LABEL = (_("Advanced Search >>"), _("Advanced Search <<"))
 OUTBOUND_PAGE = 0
python/sepolicy/setup.py +1 -1
index b0f9650d..c8220664 100644
@@ -2,7 +2,7 @@
 
 # Author: Thomas Liu <[email protected]>
 # Author: Dan Walsh <[email protected]>
-from distutils.core import setup
+from setuptools import setup
 
 setup(
     name="sepolicy",
Signature · 1 line
2.37.3
#5 lore raw

Re: [PATCH 3/5] python: Remove dependency on the Python module distutils

Quoted 30 lines · Petr Lautrbach
Quoted 16 lines · James Carter

The distutils package is deprecated and scheduled to be removed in Python 3.12. Use the setuptools and sysconfig modules instead.

---
 python/semanage/Makefile              | 2 +-
 python/sepolgen/src/sepolgen/Makefile | 2 +-
 python/sepolicy/sepolicy/gui.py       | 2 +-
 python/sepolicy/setup.py              | 2 +-
 4 files changed, 4 insertions(+), 4 deletions(-)
python/semanage/Makefile +0 -0
index 024e9640..b1f1bd3a 100644

python/semanage/Makefile LINGUAS ?= ru · line 8

Quoted 5 lines
python/semanage/Makefile +1 -1
@@ -5,7 +5,7 @@ LINGUAS ?= ru
 PREFIX ?= /usr
 SBINDIR ?= $(PREFIX)/sbin
 MANDIR = $(PREFIX)/share/man
-PYTHONLIBDIR ?= $(shell $(PYTHON) -c "from distutils.sysconfig import *; print(get_python_lib(prefix='$(PREFIX)'))")
+PYTHONLIBDIR ?= $(shell $(PYTHON) -c "import sysconfig; print(sysconfig.get_path('platlib', vars={'platbase': '$(PREFIX)', 'base': '$(PREFIX)'}))")
$ python3 -c "from distutils.sysconfig import *; print(get_python_lib(prefix='/usr'))"
/usr/lib/python3.10/site-packages

vs

Quoted 10 lines
$ python3 -c "import sysconfig; print(sysconfig.get_path('platlib', vars={'platbase': '/usr', 'base': '/usr'}))"
/usr/lib64/python3.10/site-packages

Given that python/ is not platform specific this should use 'purelib' -
https://docs.python.org/3/library/sysconfig.html#installation-paths

$ python3 -c "import sysconfig; print(sysconfig.get_path('purelib', vars={'platbase': '/usr', 'base': '/usr'}))"
/usr/lib/python3.10/site-packages

Thanks. That was one of the things I was not sure about. I will change that. Jim

Quoted 46 lines
Quoted 45 lines
python/semanage/Makefile +0 -0
 PACKAGEDIR ?= $(PYTHONLIBDIR)
 BASHCOMPLETIONDIR ?= $(PREFIX)/share/bash-completion/completions
 
python/sepolgen/src/sepolgen/Makefile +1 -1
index cac8def7..9578af23 100644
@@ -1,6 +1,6 @@
 PREFIX ?= /usr
 PYTHON ?= python3
-PYTHONLIBDIR ?= $(shell $(PYTHON) -c "from distutils.sysconfig import *; print(get_python_lib(prefix='$(PREFIX)'))")
+PYTHONLIBDIR ?= $(shell $(PYTHON) -c "import sysconfig; print(sysconfig.get_path('platlib', vars={'platbase': '$(PREFIX)', 'base': '$(PREFIX)'}))")
 PACKAGEDIR ?= /$(PYTHONLIBDIR)/sepolgen
 
 all:
python/sepolicy/sepolicy/gui.py +1 -1
index 5bdbfeba..63f2371f 100644
@@ -77,7 +77,7 @@ def cmp(a, b):
         return 1
     return (a > b) - (a < b)
 
-import distutils.sysconfig
+import sysconfig
 ADVANCED_LABEL = (_("Advanced >>"), _("Advanced <<"))
 ADVANCED_SEARCH_LABEL = (_("Advanced Search >>"), _("Advanced Search <<"))
 OUTBOUND_PAGE = 0
python/sepolicy/setup.py +1 -2
index b0f9650d..c8220664 100644
@@ -2,7 +2,7 @@
 
 # Author: Thomas Liu <[email protected]>
 # Author: Dan Walsh <[email protected]>
-from distutils.core import setup
+from setuptools import setup
 
 setup(
     name="sepolicy",
--

2.37.3

#6 lore raw

PATCH 4/5 scripts: Remove dependency on the Python module distutils

In reply to James Carter

The distutils package is deprecated and scheduled to be removed in Python 3.12. Use the sysconfig module instead.

---
 scripts/env_use_destdir | 2 +-
 scripts/run-scan-build  | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
scripts/env_use_destdir +1 -1
index 8274013e..89d989a2 100755
@@ -43,7 +43,7 @@ if [ -n "${SBINDIR:-}" ] ; then
     PATH="$DESTDIR$SBINDIR:$PATH"
 fi
 
-NEW_PYTHONPATH="$DESTDIR$(${PYTHON:-python3} -c "from distutils.sysconfig import *;print(get_python_lib(plat_specific=1, prefix='/usr'))"):$DESTDIR$(${PYTHON:-python3} -c "from distutils.sysconfig import *;print(get_python_lib(prefix='/usr'))")"
+NEW_PYTHONPATH="$DESTDIR$(${PYTHON:-python3} -c "import sysconfig; print(sysconfig.get_path('platlib', vars={'platbase': '/usr', 'base': '/usr'}))"):$DESTDIR$(${PYTHON:-python3} -c "import sysconfig; print(sysconfig.get_path('purelib', vars={'platbase': '/usr', 'base': '/usr'}))")"
 if [ -n "${PYTHONPATH:-}" ] ; then
     # Prefix the PYTHONPATH with the new directories
     export PYTHONPATH="$NEW_PYTHONPATH:$PYTHONPATH"
scripts/run-scan-build +1 -1
index 77e02ca9..931ffd2a 100755
@@ -21,7 +21,7 @@ fi
 # Make sure to use the newly-installed libraries when running tests
 export LD_LIBRARY_PATH="$DESTDIR/usr/lib:$DESTDIR/lib"
 export PATH="$DESTDIR/usr/sbin:$DESTDIR/usr/bin:$DESTDIR/sbin:$DESTDIR/bin:$PATH"
-export PYTHONPATH="$DESTDIR$(${PYTHON:-python3} -c "from distutils.sysconfig import *;print(get_python_lib(prefix='/usr'))")"
+export PYTHONPATH="$DESTDIR$(${PYTHON:-python3} -c "import sysconfig; print(sysconfig.get_path('purelib', vars={'platbase': '/usr', 'base': '/usr'}))")"
 export RUBYLIB="$DESTDIR/$(${RUBY:-ruby} -e 'puts RbConfig::CONFIG["vendorlibdir"]'):$DESTDIR/$(${RUBY:-ruby} -e 'puts RbConfig::CONFIG["vendorarchdir"]')"
 
 if [ -f /etc/debian_version ] && [ -z "${IS_CIRCLE_CI:-}" ] ; then
Signature · 1 line
2.37.3
#7 lore raw

PATCH 5/5 README.md: Remove mention of python3-distutils dependency

In reply to James Carter

With the removal of any dependence on the python disutils module, Debian no longer depends on the python3-disutils package.

---
 README.md | 1 -
 1 file changed, 1 deletion(-)
README.md +0 -1
index 74b0a0c3..f91cb7d9 100644
@@ -81,7 +81,6 @@ apt-get install --no-install-recommends --no-install-suggests \
     libpcre2-dev \
     pkgconf \
     python3 \
-    python3-distutils \
     systemd \
     xmlto
 
Signature · 1 line
2.37.3
Acked-by on the v1 series
#8 lore raw
In reply to James Carter
Quoted 11 lines · James Carter

The distutils package is deprecated and scheduled to be removed in Python 3.12.

When building the SELinux userspace we currently get warnings like the following:
<string>:1: DeprecationWarning: The distutils.sysconfig module is deprecated, use sysconfig instead
<string>:1: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives

I am not a Python expert, but everything seems to work with these patches. I would appreciate other people testing this and any Python experts to let me know if this is the right way of replacing distutils.

Quoted 6 lines

James Carter (5):
libselinux: Remove dependency on the Python module distutils
libsemanage: Remove dependency on the Python module distutils
python: Remove dependency on the Python module distutils
scripts: Remove dependency on the Python module distutils README.md: Remove mention of python3-distutils dependency

Thanks!

Quoted 14 lines
 README.md                             | 1 -
 libselinux/src/Makefile               | 2 +-
 libselinux/src/setup.py               | 2 +-
 libsemanage/src/Makefile              | 2 +-
 python/semanage/Makefile              | 2 +-
 python/sepolgen/src/sepolgen/Makefile | 2 +-
 python/sepolicy/sepolicy/gui.py       | 2 +-
 python/sepolicy/setup.py              | 2 +-
 scripts/env_use_destdir               | 2 +-
 scripts/run-scan-build                | 2 +-
 10 files changed, 9 insertions(+), 10 deletions(-)
Signature · 1 line
2.37.3