python: Use and refer to the venv module rather than virtualenv
PATCH 1/1 python: Use and refer to the venv module rather than virtualenv
Using some form of sandbox with Python modules is a long standing best practice with the language. There are a number of ways to have a Python sandbox be created. At this point in time, it seems the Python community is moving towards using the "venv" module provided with Python rather than a separate tool. To match that we make the following changes:
- Refer to a "Python sandbox" rather than virtualenv in comments, etc.
- Install the python3-venv module in our container and not virtualenv.
- In our CI files, invoke "python -m venv" rather than "virtualenv".
- In documentation, tell users to install python3-venv and not virtualenv.
--- .azure-pipelines.yml | 10 +++++----- .gitlab-ci.yml | 10 +++++----- doc/build/gcc.rst | 2 +- doc/develop/py_testing.rst | 12 ++++++------ tools/buildman/test.py | 2 +- tools/buildman/toolchain.py | 2 +- tools/docker/Dockerfile | 5 ++--- tools/u_boot_pylib/test_util.py | 2 +- 8 files changed, 22 insertions(+), 23 deletions(-)
.azure-pipelines.yml +5 -5
@@ -90,7 +90,7 @@ stages: steps: - script: | set -e - virtualenv -p /usr/bin/python3 /tmp/venvhtml + python3 -m venv /tmp/venvhtml . /tmp/venvhtml/bin/activate pip install -r doc/sphinx/requirements.txt make htmldocs KDOC_WERROR=1
@@ -132,7 +132,7 @@ stages: git config --global user.email [email protected] git config --global --add safe.directory $(work_dir) export USER=azure - virtualenv -p /usr/bin/python3 /tmp/venv + python3 -m venv /tmp/venv . /tmp/venv/bin/activate pip install -r test/py/requirements.txt \ -r tools/binman/requirements.txt \
@@ -170,7 +170,7 @@ stages: - script: | git config --global --add safe.directory $(work_dir) export USER=azure - virtualenv -p /usr/bin/python3 /tmp/venv + python3 -m venv /tmp/venv . /tmp/venv/bin/activate pip install -r test/py/requirements.txt \ -r tools/binman/requirements.txt \
@@ -277,7 +277,7 @@ stages: if [ -n "\${BUILD_ENV}" ]; then export \${BUILD_ENV}; fi - virtualenv -p /usr/bin/python3 /tmp/venv + python3 -m venv /tmp/venv . /tmp/venv/bin/activate pip install -r tools/binman/requirements.txt \ -r tools/buildman/requirements.txt \
@@ -592,7 +592,7 @@ stages: # make environment variables available as tests are running inside a container export BUILDMAN="${BUILDMAN}" git config --global --add safe.directory ${WORK_DIR} - virtualenv -p /usr/bin/python3 /tmp/venv + python3 -m venv /tmp/venv . /tmp/venv/bin/activate pip install -r tools/binman/requirements.txt \ -r tools/buildman/requirements.txt
.gitlab-ci.yml +5 -5
@@ -57,7 +57,7 @@ stages: export BINMAN_INDIRS=/tmp; fi # Prepare python environment - - virtualenv -p /usr/bin/python3 /tmp/venv; + - python3 -m venv /tmp/venv; . /tmp/venv/bin/activate; pip install -r test/py/requirements.txt -r tools/binman/requirements.txt -r tools/buildman/requirements.txt -r tools/u_boot_pylib/requirements.txt
@@ -124,7 +124,7 @@ build all platforms in a single job: - ${HOST} script: # Prepare python environment - - virtualenv -p /usr/bin/python3 /tmp/venv; + - python3 -m venv /tmp/venv; . /tmp/venv/bin/activate; pip install -r tools/binman/requirements.txt -r tools/buildman/requirements.txt
@@ -155,7 +155,7 @@ check for new CONFIG symbols outside Kconfig: docs: extends: .testsuites script: - - virtualenv -p /usr/bin/python3 /tmp/venvhtml + - python3 -m venv /tmp/venvhtml - . /tmp/venvhtml/bin/activate - pip install -r doc/sphinx/requirements.txt - make htmldocs KDOC_WERROR=1
@@ -184,7 +184,7 @@ Run binman, buildman, dtoc, Kconfig and patman testsuites: git config --global user.email [email protected]; git config --global --add safe.directory "${CI_PROJECT_DIR}"; export USER=gitlab; - virtualenv -p /usr/bin/python3 /tmp/venv; + python3 -m venv /tmp/venv; . /tmp/venv/bin/activate; pip install -r test/py/requirements.txt -r tools/binman/requirements.txt -r tools/buildman/requirements.txt -r tools/patman/requirements.txt
@@ -210,7 +210,7 @@ Run pylint: extends: .testsuites script: - git config --global --add safe.directory "${CI_PROJECT_DIR}" - - virtualenv -p /usr/bin/python3 /tmp/venv + - python3 -m venv /tmp/venv - . /tmp/venv/bin/activate - pip install -r test/py/requirements.txt -r tools/binman/requirements.txt -r tools/buildman/requirements.txt -r tools/patman/requirements.txt
doc/build/gcc.rst +1 -1
@@ -31,7 +31,7 @@ Depending on the build targets further packages maybe needed python3-pkg-resources python3-pycryptodome python3-pyelftools \ python3-pytest python3-pytest-xdist python3-sphinxcontrib.apidoc \ python3-sphinx-rtd-theme python3-subunit python3-testtools \ - python3-virtualenv swig uuid-dev + python3-venv swig uuid-dev SUSE based ~~~~~~~~~~
doc/develop/py_testing.rst +6 -6
@@ -69,19 +69,19 @@ To create disk images we have helper functions located in `test/py/tests/fs_helper.py` which shall be used in any tests that require creating disk images. -Using `virtualenv` to provide requirements -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Using a Python sandbox to provide requirements +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The recommended way to run the test suite, in order to ensure reproducibility -is to use `virtualenv` to set up the necessary environment. This can be done -via the following commands: +is to use a Python sandbox such as `python -m venv` to set up the necessary +environment. This can be done via the following commands: .. code-block:: console $ cd /path/to/u-boot - $ sudo apt-get install python3 python3-virtualenv - $ virtualenv -p /usr/bin/python3 venv + $ sudo apt-get install python3 python3-venv + $ python3 -m venv venv $ . ./venv/bin/activate $ pip install -r test/py/requirements.txt
tools/buildman/test.py +1 -1
@@ -987,7 +987,7 @@ class TestBuild(unittest.TestCase): diff = self.call_make_environment(tchn, full_path=True)[0] self.assertEqual({b'LC_ALL': b'C'}, diff) - # Test that virtualenv is handled correctly + # Test that Python sandbox is handled correctly tchn.override_toolchain = False sys.prefix = '/some/venv' env = dict(os.environb)
tools/buildman/toolchain.py +1 -1
@@ -217,7 +217,7 @@ class Toolchain: elif self.cross: env[b'CROSS_COMPILE'] = tools.to_bytes(wrapper + self.cross) - # Detect a Python virtualenv and avoid defeating it + # Detect a Python sandbox and avoid defeating it if sys.prefix != sys.base_prefix: paths = env[b'PATH'].split(b':') new_paths = []
tools/docker/Dockerfile +2 -3
@@ -124,7 +124,7 @@ RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \ python3-dev \ python3-pip \ python3-sphinx \ - python3-virtualenv \ + python3-venv \ rpm2cpio \ sbsigntool \ socat \
@@ -136,7 +136,6 @@ RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \ texinfo \ util-linux \ uuid-dev \ - virtualenv \ vboot-kernel-utils \ vboot-utils \ xilinx-bootgen \
@@ -313,7 +312,7 @@ RUN wget -O /tmp/binman-requirements.txt https://source.denx.de/u-boot/u-boot/-/ RUN wget -O /tmp/buildman-requirements.txt https://source.denx.de/u-boot/u-boot/-/raw/master/tools/buildman/requirements.txt RUN wget -O /tmp/patman-requirements.txt https://source.denx.de/u-boot/u-boot/-/raw/master/tools/patman/requirements.txt RUN wget -O /tmp/u_boot_pylib-requirements.txt https://source.denx.de/u-boot/u-boot/-/raw/master/tools/u_boot_pylib/requirements.txt -RUN virtualenv -p /usr/bin/python3 /tmp/venv && \ +RUN python3 -m venv /tmp/venv && \ . /tmp/venv/bin/activate && \ pip install -r /tmp/pytest-requirements.txt \ -r /tmp/sphinx-requirements.txt \
tools/u_boot_pylib/test_util.py +1 -1
@@ -64,7 +64,7 @@ def run_test_coverage(prog, filter_fname, exclude_list, build_dir, if build_dir: prefix = 'PYTHONPATH=$PYTHONPATH:%s/sandbox_spl/tools ' % build_dir - # Detect a Python virtualenv and use 'coverage' instead + # Detect a Python sandbox and use 'coverage' instead covtool = ('python3-coverage' if sys.prefix == sys.base_prefix else 'coverage')
Signature · 1 line
2.43.0
Quoted 6 lines · Tom Rini
Using some form of sandbox with Python modules is a long standing best practice with the language. There are a number of ways to have a Python sandbox be created. At this point in time, it seems the Python community is moving towards using the "venv" module provided with Python rather than a separate tool. To match that we make the following changes:
Quoted 7 lines
- Refer to a "Python sandbox" rather than virtualenv in comments, etc.
- Install the python3-venv module in our container and not virtualenv.
- In our CI files, invoke "python -m venv" rather than "virtualenv".
- In documentation, tell users to install python3-venv and not virtualenv.[...]
Applied to u-boot/master, thanks!
[1/1] python: Use and refer to the venv module rather than virtualenv
commit: efd00b03455644b18eaf2b6dcd07f6d55b95b39e
Signature · 1 line
Tom