Updates to host requirements
v2 cover Updates to host requirements
---
Changes in v2:
- Rework this websockets addition, as it proved to be tricky to provide
on older Ubuntu/Debian distros.
- Add a patch to document the buildtools requirement on openSUSE Leap
15.6.
- Link to v1: https://patch.msgid.link/20260420-python3-websockets-requirement-v1-1-28c4fce2f766@bootlin.com
---
Antonin Godard (2):
ref-manual/system-requirements.rst: make python3-websockets a default host requirement ref-manual/system-requirements.rst: instruct to install a buildtools tarball
documentation/ref-manual/system-requirements.rst | 33 ++++++++++++++++++++++ .../host_packages_scripts/fedora_essential.sh | 2 +- .../opensuse_essential_16.0.sh | 2 +- 3 files changed, 35 insertions(+), 2 deletions(-)
---
base-commit: 2c12ec7bf29aedeacf82970a9d2eb262fde4670e
change-id: 20260420-python3-websockets-requirement-4594450f8d92
PATCH v2 1/2 ref-manual/system-requirements.rst: make python3-websockets a default host requirement
When configuring the project through bitbake-setup we can choose the "poky-with-sstate" configuration, which requires the python3-websockets package.
Add this package as a host requirement (under different names depending on the distro).
Alma, CentOS, and Rocky have a requirement on the buildtools tarball, which already contains this package, so we don't need to add it for them.
On Ubuntu 22.04 and Debian 11, the version of python3-websockets provided by the distribution is too old compared to what we need [1]. Add a separate set of instructions for these.
--- documentation/ref-manual/system-requirements.rst | 28 ++++++++++++++++++++++ .../host_packages_scripts/fedora_essential.sh | 2 +- .../opensuse_essential_16.0.sh | 2 +- 3 files changed, 30 insertions(+), 2 deletions(-)
documentation/ref-manual/system-requirements.rst +28 -0
@@ -166,6 +166,34 @@ Ubuntu and Debian .. include:: ubuntu-debian-host-packages-nodocs.rst +Additionally, on distributions other than Debian 11 and Ubuntu 22.04 (otherwise +see note below), install the following package: + +.. code-block:: console + + $ sudo apt install python3-websockets + +.. note:: + + On Debian 11 and Ubuntu 22.04, the requirement on the version of + ``python3-websockets`` is not satisfied by the host distribution. This + package is used for fetching pre-built :ref:`shared state + <overview-manual/concepts:shared state>` artifacts from upstream Yocto Project + servers (see :ref:`ref-fragments-core-yocto-sstate-mirror-cdn` for more + information). + + In order to use this feature, you can get the correct version of + ``python3-websockets`` by either: + + - Using a :term:`buildtools` tarball by following the + :ref:`system-requirements-buildtools` section below. + + - Installing ``python3-websockets`` using `Pip <https://pip.pypa.io/en/stable/>`__: + + .. code-block:: console + + $ pip install websockets + Here are the packages needed to build Project documentation manuals: .. literalinclude:: ../tools/host_packages_scripts/ubuntu_docs.sh
documentation/tools/host_packages_scripts/fedora_essential.sh +1 -1
@@ -1 +1 @@ -sudo dnf install bzip2 ccache chrpath cpio cpp diffstat diffutils file findutils gawk gcc gcc-c++ git glibc-devel glibc-langpack-en gzip hostname libacl make patch perl perl-Data-Dumper perl-File-Compare perl-File-Copy perl-FindBin perl-Text-ParseWords perl-Thread-Queue perl-bignum perl-locale python python3 python3-GitPython python3-jinja2 python3-pexpect python3-pip rpcgen socat tar texinfo unzip wget which xz zstd $([ $(rpm -E %fedora) -ge 42 ] && echo "util-linux-script") +sudo dnf install bzip2 ccache chrpath cpio cpp diffstat diffutils file findutils gawk gcc gcc-c++ git glibc-devel glibc-langpack-en gzip hostname libacl make patch perl perl-Data-Dumper perl-File-Compare perl-File-Copy perl-FindBin perl-Text-ParseWords perl-Thread-Queue perl-bignum perl-locale python python3 python3-GitPython python3-jinja2 python3-pexpect python3-pip python3-websockets rpcgen socat tar texinfo unzip wget which xz zstd $([ $(rpm -E %fedora) -ge 42 ] && echo "util-linux-script")
documentation/tools/host_packages_scripts/opensuse_essential_16.0.sh +1 -1
@@ -1 +1 @@ -sudo zypper install bzip2 chrpath diffstat gcc gcc-c++ git gzip hostname libacl1 make makeinfo patch python3 python3-GitPython python3-Jinja2 python3-curses python3-pexpect python3-pip python3-xml rpcgen socat tar wget which xz zstd +sudo zypper install bzip2 chrpath diffstat gcc gcc-c++ git gzip hostname libacl1 make makeinfo patch python3 python3-GitPython python3-Jinja2 python3-curses python3-pexpect python3-pip python313-websockets python3-xml rpcgen socat tar wget which xz zstd
Signature · 1 line
2.53.0
Re: [docs] [PATCH v2 1/2] ref-manual/system-requirements.rst: make python3-websockets a default host requirement
Hi Antonin,
Quoted 27 lines · Antonin Godard via lists.yoctoproject.org
When configuring the project through bitbake-setup we can choose the "poky-with-sstate" configuration, which requires the python3-websockets package.
Add this package as a host requirement (under different names depending on the distro).
Alma, CentOS, and Rocky have a requirement on the buildtools tarball, which already contains this package, so we don't need to add it for them.
On Ubuntu 22.04 and Debian 11, the version of python3-websockets provided by the distribution is too old compared to what we need [1]. Add a separate set of instructions for these.
Signed-off-by Antonin Godard <[email protected]>--- documentation/ref-manual/system-requirements.rst | 28 ++++++++++++++++++++++ .../host_packages_scripts/fedora_essential.sh | 2 +- .../opensuse_essential_16.0.sh | 2 +- 3 files changed, 30 insertions(+), 2 deletions(-)documentation/ref-manual/system-requirements.rst +0
-0
documentation/ref-manual/system-requirements.rst Ubuntu and Debian · line 175
Quoted 10 lines
documentation/ref-manual/system-requirements.rst +7
-0@@ -166,6 +166,34 @@ Ubuntu and Debian .. include:: ubuntu-debian-host-packages-nodocs.rst +Additionally, on distributions other than Debian 11 and Ubuntu 22.04 (otherwise +see note below), install the following package: + +.. code-block:: console + + $ sudo apt install python3-websockets +
Maybe use apt-get here? I know we have one or the other across the code-base. For some reason, I can't bring myself to use apt, it always results in weird behaviors I don't have using apt-get on servers (I don't run Debian-based on my own machines).
documentation/ref-manual/system-requirements.rst Ubuntu and Debian · line 178
Quoted 3 lines
documentation/ref-manual/system-requirements.rst +3
-0+.. note:: + + On Debian 11 and Ubuntu 22.04, the requirement on the version of
A bit awkward wording to me, would
the minimum version for ``python3-websockets`` is not satisfied...
work better?
Quoted 8 lines
documentation/ref-manual/system-requirements.rst +8
-0+ ``python3-websockets`` is not satisfied by the host distribution. This + package is used for fetching pre-built :ref:`shared state + <overview-manual/concepts:shared state>` artifacts from upstream Yocto Project + servers (see :ref:`ref-fragments-core-yocto-sstate-mirror-cdn` for more + information). + + In order to use this feature, you can get the correct version of + ``python3-websockets`` by either:
documentation/ref-manual/system-requirements.rst Ubuntu and Debian · line 196
Quoted 10 lines
documentation/ref-manual/system-requirements.rst +10
-0+ + - Using a :term:`buildtools` tarball by following the + :ref:`system-requirements-buildtools` section below. + + - Installing ``python3-websockets`` using `Pip <https://pip.pypa.io/en/stable/>`__: + + .. code-block:: console + + $ pip install websockets +
websockets 9.1 supports Python 3.9 which is available in Debian 11 and Ubuntu 22.04 so it'll get at least that version (it's not because a newer version of a package is available on PyPI that your system can install it; and the opposite is true, you may have a too new Python version for the last package version available (e.g. tensorflow-lite isn't compatible with Python 3.14)). Can you check what happens if the user already has python3-websockets installed on the system and runs this command? My hunch is it'll not get updated to >=9.1 without -U/--upgrade. Also, running -U will likely break their system if any of the hsot packages depend on that older version of websockets... Maybe we should only be using Python packages from a virtualenv?
Quoted 15 lines
documentation/ref-manual/system-requirements.rst +0
-0Here are the packages needed to build Project documentation manuals: .. literalinclude:: ../tools/host_packages_scripts/ubuntu_docs.shdocumentation/tools/host_packages_scripts/fedora_essential.sh +1
-1@@ -1 +1 @@ -sudo dnf install bzip2 ccache chrpath cpio cpp diffstat diffutils file findutils gawk gcc gcc-c++ git glibc-devel glibc-langpack-en gzip hostname libacl make patch perl perl-Data-Dumper perl-File-Compare perl-File-Copy perl-FindBin perl-Text-ParseWords perl-Thread-Queue perl-bignum perl-locale python python3 python3-GitPython python3-jinja2 python3-pexpect python3-pip rpcgen socat tar texinfo unzip wget which xz zstd $([ $(rpm -E %fedora) -ge 42 ] && echo "util-linux-script") +sudo dnf install bzip2 ccache chrpath cpio cpp diffstat diffutils file findutils gawk gcc gcc-c++ git glibc-devel glibc-langpack-en gzip hostname libacl make patch perl perl-Data-Dumper perl-File-Compare perl-File-Copy perl-FindBin perl-Text-ParseWords perl-Thread-Queue perl-bignum perl-locale python python3 python3-GitPython python3-jinja2 python3-pexpect python3-pip python3-websockets rpcgen socat tar texinfo unzip wget which xz zstd $([ $(rpm -E %fedora) -ge 42 ] && echo "util-linux-script")documentation/tools/host_packages_scripts/opensuse_essential_16.0.sh +0
-0
documentation/tools/host_packages_scripts/opensuse_essential_16.0.sh line 1
Quoted 2 lines
documentation/tools/host_packages_scripts/opensuse_essential_16.0.sh +1
-1@@ -1 +1 @@ -sudo zypper install bzip2 chrpath diffstat gcc gcc-c++ git gzip hostname libacl1 make makeinfo patch python3 python3-GitPython python3-Jinja2 python3-curses python3-pexpect python3-pip python3-xml rpcgen socat tar wget which xz zstd +sudo zypper install bzip2 chrpath diffstat gcc gcc-c++ git gzip hostname libacl1 make makeinfo patch python3 python3-GitPython python3-Jinja2 python3-curses python3-pexpect python3-pip python313-websockets python3-xml rpcgen socat tar wget which xz zstd
Can you rather use python3-websockets here? https://opensuse.pkgs.org/16.0/opensuse-oss/python313-websockets-14.2-160000.2.2.aarch64.rpm.html says it provides and replaces python3-websockets. That should make the upgrade to the next Leap version easier since I'm assuming it may switch to Python 3.14 by then?
Cheers,
Quentin
Re: [docs] [PATCH v2 1/2] ref-manual/system-requirements.rst: make python3-websockets a default host requirement
Hi,
Quoted 33 lines · Quentin Schulz
Hi Antonin,
Quoted 30 lines · Antonin Godard via lists.yoctoproject.org
When configuring the project through bitbake-setup we can choose the "poky-with-sstate" configuration, which requires the python3-websockets package.
Add this package as a host requirement (under different names depending on the distro).
Alma, CentOS, and Rocky have a requirement on the buildtools tarball, which already contains this package, so we don't need to add it for them.
On Ubuntu 22.04 and Debian 11, the version of python3-websockets provided by the distribution is too old compared to what we need [1]. Add a separate set of instructions for these.
Signed-off-by Antonin Godard <[email protected]>--- documentation/ref-manual/system-requirements.rst | 28 ++++++++++++++++++++++ .../host_packages_scripts/fedora_essential.sh | 2 +- .../opensuse_essential_16.0.sh | 2 +- 3 files changed, 30 insertions(+), 2 deletions(-)documentation/ref-manual/system-requirements.rst +0
-0@@ -166,6 +166,34 @@ Ubuntu and Debian .. include:: ubuntu-debian-host-packages-nodocs.rst
Quoted 12 lines
documentation/ref-manual/system-requirements.rst Ubuntu and Debian · line 175
Quoted 7 lines
documentation/ref-manual/system-requirements.rst +7
-0+Additionally, on distributions other than Debian 11 and Ubuntu 22.04 (otherwise +see note below), install the following package: + +.. code-block:: console + + $ sudo apt install python3-websockets +Maybe use apt-get here? I know we have one or the other across the code-base. For some reason, I can't bring myself to use apt, it always results in weird behaviors I don't have using apt-get on servers (I don't run Debian-based on my own machines).
Yes, that's better to use here and also aligns with the rest, thanks!
Quoted 9 lines
documentation/ref-manual/system-requirements.rst Ubuntu and Debian · line 178
Quoted 3 lines
documentation/ref-manual/system-requirements.rst +3
-0+.. note:: + + On Debian 11 and Ubuntu 22.04, the requirement on the version ofA bit awkward wording to me, would
the minimum version for ``python3-websockets`` is not satisfied...
work better?
Indeed, will rephrase
Quoted 22 lines
Quoted 8 lines
documentation/ref-manual/system-requirements.rst +8
-0+ ``python3-websockets`` is not satisfied by the host distribution. This + package is used for fetching pre-built :ref:`shared state + <overview-manual/concepts:shared state>` artifacts from upstream Yocto Project + servers (see :ref:`ref-fragments-core-yocto-sstate-mirror-cdn` for more + information). + + In order to use this feature, you can get the correct version of + ``python3-websockets`` by either:documentation/ref-manual/system-requirements.rst Ubuntu and Debian · line 196
Quoted 10 lines
documentation/ref-manual/system-requirements.rst +10
-0+ + - Using a :term:`buildtools` tarball by following the + :ref:`system-requirements-buildtools` section below. + + - Installing ``python3-websockets`` using `Pip <https://pip.pypa.io/en/stable/>`__: + + .. code-block:: console + + $ pip install websockets +websockets 9.1 supports Python 3.9 which is available in Debian 11 and Ubuntu 22.04 so it'll get at least that version (it's not because a newer version of a package is available on PyPI that your system can
Quoted 8 lines
install it; and the opposite is true, you may have a too new Python version for the last package version available (e.g. tensorflow-lite isn't compatible with Python 3.14)). Can you check what happens if the user already has python3-websockets installed on the system and runs this command? My hunch is it'll not get updated to >=9.1 without -U/--upgrade. Also, running -U will likely break their system if any of the hsot packages depend on that older version of websockets... Maybe we should only be using Python packages from a virtualenv?
Maybe I can state that this it is advised to install it in a virtual environment and avoid the possible gotchas you stated?
Quoted 15 lines
Quoted 15 lines
documentation/ref-manual/system-requirements.rst +0
-0Here are the packages needed to build Project documentation manuals: .. literalinclude:: ../tools/host_packages_scripts/ubuntu_docs.shdocumentation/tools/host_packages_scripts/fedora_essential.sh +1
-1@@ -1 +1 @@ -sudo dnf install bzip2 ccache chrpath cpio cpp diffstat diffutils file findutils gawk gcc gcc-c++ git glibc-devel glibc-langpack-en gzip hostname libacl make patch perl perl-Data-Dumper perl-File-Compare perl-File-Copy perl-FindBin perl-Text-ParseWords perl-Thread-Queue perl-bignum perl-locale python python3 python3-GitPython python3-jinja2 python3-pexpect python3-pip rpcgen socat tar texinfo unzip wget which xz zstd $([ $(rpm -E %fedora) -ge 42 ] && echo "util-linux-script") +sudo dnf install bzip2 ccache chrpath cpio cpp diffstat diffutils file findutils gawk gcc gcc-c++ git glibc-devel glibc-langpack-en gzip hostname libacl make patch perl perl-Data-Dumper perl-File-Compare perl-File-Copy perl-FindBin perl-Text-ParseWords perl-Thread-Queue perl-bignum perl-locale python python3 python3-GitPython python3-jinja2 python3-pexpect python3-pip python3-websockets rpcgen socat tar texinfo unzip wget which xz zstd $([ $(rpm -E %fedora) -ge 42 ] && echo "util-linux-script")documentation/tools/host_packages_scripts/opensuse_essential_16.0.sh +0
-0@@ -1 +1 @@
Quoted 8 lines
documentation/tools/host_packages_scripts/opensuse_essential_16.0.sh line 1
Quoted 2 lines
documentation/tools/host_packages_scripts/opensuse_essential_16.0.sh +1
-1-sudo zypper install bzip2 chrpath diffstat gcc gcc-c++ git gzip hostname libacl1 make makeinfo patch python3 python3-GitPython python3-Jinja2 python3-curses python3-pexpect python3-pip python3-xml rpcgen socat tar wget which xz zstd +sudo zypper install bzip2 chrpath diffstat gcc gcc-c++ git gzip hostname libacl1 make makeinfo patch python3 python3-GitPython python3-Jinja2 python3-curses python3-pexpect python3-pip python313-websockets python3-xml rpcgen socat tar wget which xz zstdCan you rather use python3-websockets here?
https://opensuse.pkgs.org/16.0/opensuse-oss/python313-websockets-14.2-160000.2.2.aarch64.rpm.html
says it provides and replaces python3-websockets. That should make the upgrade to the next Leap version easier since I'm assuming it may switch to Python 3.14 by then?
Ah good point, I did not spot that it was a provider of python3-websockets. I'll correct that.
Thanks,
Antonin
PATCH v2 2/2 ref-manual/system-requirements.rst: instruct to install a buildtools tarball
In reply to Antonin GodardI noticed that openSUSE Leap 15.6 provide Python 3.6, which is not supported by BitBake (which requires Python >3.9). Instruct to use a buildtools tarball, which is how this is done on the Autobuilder.
--- documentation/ref-manual/system-requirements.rst | 5 +++++ 1 file changed, 5 insertions(+)
documentation/ref-manual/system-requirements.rst +5 -0
@@ -243,6 +243,11 @@ with a supported openSUSE Leap 15.6 distribution: .. literalinclude:: ../tools/host_packages_scripts/opensuse_essential_15.6.sh :language: shell +Additionally, openSUSE Leap 15.6 requires a :term:`buildtools` tarball to be +installed and set up, as this distribution provides a version of Python that is +too old for :term:`BitBake`. Follow the :ref:`system-requirements-buildtools` +section below for more information. + Here are the packages needed to build Project documentation manuals: .. literalinclude:: ../tools/host_packages_scripts/opensuse_docs.sh
Signature · 1 line
2.53.0
Re: [docs] [PATCH v2 2/2] ref-manual/system-requirements.rst: instruct to install a buildtools tarball
Hi Antonin,
Quoted 14 lines · Antonin Godard via lists.yoctoproject.org
I noticed that openSUSE Leap 15.6 provide Python 3.6, which is not supported by BitBake (which requires Python >3.9). Instruct to use a buildtools tarball, which is how this is done on the Autobuilder.
Signed-off-by Antonin Godard <[email protected]>--- documentation/ref-manual/system-requirements.rst | 5 +++++ 1 file changed, 5 insertions(+)documentation/ref-manual/system-requirements.rst +0
-0
documentation/ref-manual/system-requirements.rst with a supported openSUSE Leap 15.6 distribution: · line 250
Quoted 8 lines
documentation/ref-manual/system-requirements.rst +5
-0@@ -243,6 +243,11 @@ with a supported openSUSE Leap 15.6 distribution: .. literalinclude:: ../tools/host_packages_scripts/opensuse_essential_15.6.sh :language: shell +Additionally, openSUSE Leap 15.6 requires a :term:`buildtools` tarball to be +installed and set up, as this distribution provides a version of Python that is +too old for :term:`BitBake`. Follow the :ref:`system-requirements-buildtools` +section below for more information. +
Shouldn't the buildtools cover all required packages to be present on the host? If so, do we actually need instructions without (e.g. the opensuse_essential_15.6.sh above)?
Cheers,
Quentin
Re: [docs] [PATCH v2 2/2] ref-manual/system-requirements.rst: instruct to install a buildtools tarball
Hi,
Quoted 20 lines · Quentin Schulz
Hi Antonin,
Quoted 17 lines · Antonin Godard via lists.yoctoproject.org
I noticed that openSUSE Leap 15.6 provide Python 3.6, which is not supported by BitBake (which requires Python >3.9). Instruct to use a buildtools tarball, which is how this is done on the Autobuilder.
Signed-off-by Antonin Godard <[email protected]>--- documentation/ref-manual/system-requirements.rst | 5 +++++ 1 file changed, 5 insertions(+)documentation/ref-manual/system-requirements.rst +0
-0@@ -243,6 +243,11 @@ with a supported openSUSE Leap 15.6 distribution: .. literalinclude:: ../tools/host_packages_scripts/opensuse_essential_15.6.sh :language: shell
Quoted 9 lines
documentation/ref-manual/system-requirements.rst with a supported openSUSE Leap 15.6 distribution: · line 250
Quoted 5 lines
documentation/ref-manual/system-requirements.rst +5
-0+Additionally, openSUSE Leap 15.6 requires a :term:`buildtools` tarball to be +installed and set up, as this distribution provides a version of Python that is +too old for :term:`BitBake`. Follow the :ref:`system-requirements-buildtools` +section below for more information. +Shouldn't the buildtools cover all required packages to be present on the host? If so, do we actually need instructions without (e.g. the
opensuse_essential_15.6.shabove)?
Probably, but to be fair I'm trying to match what is installed on the Autobuilder (and that is known to work), and I'd like to avoid dropping one requirement by mistake. That's a bit inefficient, but a bit safer. I wish the Autobuilder list of requirements was public and allow my to test things but I don't have such an environment :/
Antonin