Skip to contentSkip to conversation

migration-guides: prepare for release 3.5

PATCH Superseded Michael Opdenacker 5 participants 7 messages Dec 17, 2021 – Jan 18, 2022

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 migration-guides: prepare for release 3.5

---
 documentation/migration-guides/index.rst      |  1 +
 .../migration-guides/migration-3.5.rst        | 37 +++++++++++++++++++
 2 files changed, 38 insertions(+)
 create mode 100644 documentation/migration-guides/migration-3.5.rst
documentation/migration-guides/index.rst +1 -0
index 287b553195..6472cc2848 100644
@@ -12,6 +12,7 @@ to move to one release of the Yocto Project from the previous one.
 .. toctree::
 
    migration-general
+   migration-3.5
    migration-3.4
    migration-3.3
    migration-3.2
documentation/migration-guides/migration-3.5.rst new file +37 -0
new file mode 100644
index 0000000000..c63c9dff05
@@ -0,0 +1,37 @@
+Release 3.5 (kirkstone)
+=======================
+
+This section provides migration information for moving to the Yocto
+Project 3.5 Release (codename "kirkstone") from the prior release.
+
+Recipe changes
+--------------
+
+- Because of the uncertainty in future default branch names in git repositories,
+  it is now required to add a branch name to all URLs described
+  by ``git://`` and ``gitsm://`` :term:`SRC_URI` entries. For example::
+
+     SRC_URI = "git://git.denx.de/u-boot.git;branch=master"
+
+  A :oe_git:`convert-srcuri </openembedded-core/tree/scripts/contrib/convert-srcuri.py>`
+  script to convert your recipes is available in :term:`OpenEmbedded-Core (OE-Core)`
+  and in :term:`Poky`.
+
+- Because of `GitHub dropping support for the git:
+  protocol <https://github.blog/2021-09-01-improving-git-protocol-security-github/>`,
+  recipes now need to use ``;protocol=https`` at the end of GitHub
+  URLs. The same script as above can be used to convert the recipes.
+
+- The :term:`TOPDIR` variable and the current working directory are no longer modified
+  when parsing recipes. Any code depending on that behaviour will no longer worked as
+  expected (we don't know of any).
+
+- BitBake no longer supports ``append``, ``prepend`` and ``remove`` operators
+  combined with the ``+=``, ``=+``, ``.=``, ``=.`` and ``?=`` operators.
+  Such operations should be made using ``=`` alone. This means that
+  a leading space may need to be explicitly added.
+
+- :ref:`allarch <ref-classes-allarch>` packagegroups can no longer depend on packages
+  which use ``PKG:xxx`` renaming such as :ref:`debian.bbclass <ref-classes-debian>`.
+  Doing this will produce an error.
+
Signature · 1 line
2.25.1
#1 lore raw
Quoted 48 lines

-----Original Message-----
From: [email protected] <[email protected]> On Behalf Of Michael Opdenacker
Sent: den 17 december 2021 15:04
To: [email protected]
Cc: Michael Opdenacker <[email protected]>
Subject: [docs] [PATCH] migration-guides: prepare for release 3.5

---
 documentation/migration-guides/index.rst      |  1 +
 .../migration-guides/migration-3.5.rst        | 37 +++++++++++++++++++
 2 files changed, 38 insertions(+)
 create mode 100644 documentation/migration-guides/migration-3.5.rst
documentation/migration-guides/index.rst +1 -0
index 287b553195..6472cc2848 100644
@@ -12,6 +12,7 @@ to move to one release of the Yocto Project from the previous one.
 .. toctree::
 
    migration-general
+   migration-3.5
    migration-3.4
    migration-3.3
    migration-3.2
documentation/migration-guides/migration-3.5.rst new file +16 -0
new file mode 100644
index 0000000000..c63c9dff05
@@ -0,0 +1,37 @@
+Release 3.5 (kirkstone)
+=======================
+
+This section provides migration information for moving to the Yocto
+Project 3.5 Release (codename "kirkstone") from the prior release.
+
+Recipe changes
+--------------
+
+- Because of the uncertainty in future default branch names in git repositories,
+  it is now required to add a branch name to all URLs described
+  by ``git://`` and ``gitsm://`` :term:`SRC_URI` entries. For example::
+
+     SRC_URI = "git://git.denx.de/u-boot.git;branch=master"
+
+  A :oe_git:`convert-srcuri </openembedded-core/tree/scripts/contrib/convert-srcuri.py>`

documentation/migration-guides/migration-3.5.rst line 26

Quoted 10 lines
documentation/migration-guides/migration-3.5.rst new file +10 -0
+  script to convert your recipes is available in :term:`OpenEmbedded-Core (OE-Core)`
+  and in :term:`Poky`.
+
+- Because of `GitHub dropping support for the git:
+  protocol <https://github.blog/2021-09-01-improving-git-protocol-security-github/>`,
+  recipes now need to use ``;protocol=https`` at the end of GitHub
+  URLs. The same script as above can be used to convert the recipes.
+
+- The :term:`TOPDIR` variable and the current working directory are no longer modified
+  when parsing recipes. Any code depending on that behaviour will no longer worked as

Change "worked" to "work".

Quoted 12 lines
documentation/migration-guides/migration-3.5.rst new file +11 -1
+  expected (we don't know of any).
+
+- BitBake no longer supports ``append``, ``prepend`` and ``remove`` operators
+  combined with the ``+=``, ``=+``, ``.=``, ``=.`` and ``?=`` operators.
+  Such operations should be made using ``=`` alone. This means that
+  a leading space may need to be explicitly added.
+
+- :ref:`allarch <ref-classes-allarch>` packagegroups can no longer depend on packages
+  which use ``PKG:xxx`` renaming such as :ref:`debian.bbclass <ref-classes-debian>`.
+  Doing this will produce an error.
+
--
Quoted 1 line

2.25.1

//Peter

#2 lore raw
In reply to Michael Opdenacker

Hi Michael,

Quoted 43 lines · Michael Opdenacker
---
  documentation/migration-guides/index.rst      |  1 +
  .../migration-guides/migration-3.5.rst        | 37 +++++++++++++++++++
  2 files changed, 38 insertions(+)

create mode 100644 documentation/migration-guides/migration-3.5.rst

documentation/migration-guides/index.rst +1 -0
index 287b553195..6472cc2848 100644
@@ -12,6 +12,7 @@ to move to one release of the Yocto Project from the previous one.
  .. toctree::
 
     migration-general
+   migration-3.5
     migration-3.4
     migration-3.3
     migration-3.2
documentation/migration-guides/migration-3.5.rst new file +18 -0
new file mode 100644
index 0000000000..c63c9dff05
@@ -0,0 +1,37 @@
+Release 3.5 (kirkstone)
+=======================
+
+This section provides migration information for moving to the Yocto
+Project 3.5 Release (codename "kirkstone") from the prior release.
+
+Recipe changes
+--------------
+
+- Because of the uncertainty in future default branch names in git repositories,
+  it is now required to add a branch name to all URLs described
+  by ``git://`` and ``gitsm://`` :term:`SRC_URI` entries. For example::
+
+     SRC_URI = "git://git.denx.de/u-boot.git;branch=master"
+
+  A :oe_git:`convert-srcuri </openembedded-core/tree/scripts/contrib/convert-srcuri.py>`
+  script to convert your recipes is available in :term:`OpenEmbedded-Core (OE-Core)`
+  and in :term:`Poky`.

documentation/migration-guides/migration-3.5.rst line 28

Quoted 10 lines
documentation/migration-guides/migration-3.5.rst new file +10 -0
+
+- Because of `GitHub dropping support for the git:
+  protocol <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.blog_2021-2D09-2D01-2Dimproving-2Dgit-2Dprotocol-2Dsecurity-2Dgithub_&d=DwIDAg&c=_sEr5x9kUWhuk4_nFwjJtA&r=LYjLexDn7rXIzVmkNPvw5ymA1XTSqHGq8yBP6m6qZZ4njZguQhZhkI_-172IIy1t&m=BHtIscjgpnlAxrsnv58CSizo0w27BvdJhKyrnKN2iWSXMxSc6E9_hS4mbjYz1tJ7&s=dHvCcZgUMuYx5xpf5Vm0zUvn-LHw3OZ5u2E8xzrGUGM&e= >`,
+  recipes now need to use ``;protocol=https`` at the end of GitHub
+  URLs. The same script as above can be used to convert the recipes.
+
+- The :term:`TOPDIR` variable and the current working directory are no longer modified
+  when parsing recipes. Any code depending on that behaviour will no longer worked as
+  expected (we don't know of any).
+

s/will no longer worked as expected (we don't know of any)/will no longer work/

documentation/migration-guides/migration-3.5.rst line 30

Quoted 2 lines
documentation/migration-guides/migration-3.5.rst new file +2 -0
+- BitBake no longer supports ``append``, ``prepend`` and ``remove`` operators
+  combined with the ``+=``, ``=+``, ``.=``, ``=.`` and ``?=`` operators.

I haven't checked, but what about ??= operator?

documentation/migration-guides/migration-3.5.rst line 32

Quoted 2 lines
documentation/migration-guides/migration-3.5.rst new file +2 -0
+  Such operations should be made using ``=`` alone. This means that > +  a leading space may need to be explicitly added.
+

May I suggest the following instead? For the whole paragraph (provided ??= also isn't supported).

"""
The ``append``, ``prepend`` and ``remove`` operators can now only be combined with ``=`` and ``:=`` operators. To the exception of the ``append`` plus ``+=`` and ``prepend`` plus ``=+`` combinations, all combinations could be factored up to the ``append``, ``prepend`` or ``remove`` in the combination. This brought a lot of confusion on how the override style syntax operators work and should be used. Therefore, those combinations can simply be replaced by a single ``append``, ``prepend`` or ``remove`` operator without any additional change. For the ``append`` plus ``+=`` (and ``prepend`` plus ``=+``) combination, the content should be prefixed (respectively suffixed) by a space to maintain the same behavior.
You can learn more about override style syntax operators (``append``, ``prepend`` and ``remove``) in the :ref:BitBake documentation </don't know the path>.
"""

The path to the doc: https://docs.yoctoproject.org/bitbake/bitbake-user-manual/bitbake-user-manual-metadata.html#appending-and-prepending-override-style-syntax and https://docs.yoctoproject.org/bitbake/bitbake-user-manual/bitbake-user-manual-metadata.html#removal-override-style-syntax

Looking for those links, I also noticed that we haven't updated the bitbake docs yet to remove the explanation on += combination in https://docs.yoctoproject.org/bitbake/bitbake-user-manual/bitbake-user-manual-metadata.html#override-style-operation-advantages.

documentation/migration-guides/migration-3.5.rst line 35

Quoted 2 lines
documentation/migration-guides/migration-3.5.rst new file +2 -0
+- :ref:`allarch <ref-classes-allarch>` packagegroups can no longer depend on packages
+  which use ``PKG:xxx`` renaming such as :ref:`debian.bbclass <ref-classes-debian>`.

s/``PKG:xxx``/:term:PKG/ ?

documentation/migration-guides/migration-3.5.rst line 37

Quoted 2 lines
documentation/migration-guides/migration-3.5.rst new file +2 -0
+  Doing this will produce an error.
+

I would remove this sentence or give the kind of message to be expected when such an error happens.

Cheers,
Quentin

#3 lore raw
In reply to Michael Opdenacker
Quoted 33 lines · Michael Opdenacker
documentation/migration-guides/migration-3.5.rst new file +27 -0
new file mode 100644
index 0000000000..c63c9dff05
@@ -0,0 +1,37 @@
+Release 3.5 (kirkstone)
+=======================
+
+This section provides migration information for moving to the Yocto
+Project 3.5 Release (codename "kirkstone") from the prior release.
+
+Recipe changes
+--------------
+
+- Because of the uncertainty in future default branch names in git repositories,
+  it is now required to add a branch name to all URLs described
+  by ``git://`` and ``gitsm://`` :term:`SRC_URI` entries. For example::
+
+     SRC_URI = "git://git.denx.de/u-boot.git;branch=master"
+
+  A :oe_git:`convert-srcuri </openembedded-core/tree/scripts/contrib/convert-srcuri.py>`
+  script to convert your recipes is available in :term:`OpenEmbedded-Core (OE-Core)`
+  and in :term:`Poky`.
+
+- Because of `GitHub dropping support for the git:
+  protocol <https://github.blog/2021-09-01-improving-git-protocol-security-github/>`,
+  recipes now need to use ``;protocol=https`` at the end of GitHub
+  URLs. The same script as above can be used to convert the recipes.
+
+- The :term:`TOPDIR` variable and the current working directory are no longer modified
+  when parsing recipes. Any code depending on that behaviour will no longer worked as
+  expected (we don't know of any).

documentation/migration-guides/migration-3.5.rst line 37

Quoted 10 lines
documentation/migration-guides/migration-3.5.rst new file +10 -0
+
+- BitBake no longer supports ``append``, ``prepend`` and ``remove`` operators
+  combined with the ``+=``, ``=+``, ``.=``, ``=.`` and ``?=`` operators.
+  Such operations should be made using ``=`` alone. This means that
+  a leading space may need to be explicitly added.
+
+- :ref:`allarch <ref-classes-allarch>` packagegroups can no longer depend on packages
+  which use ``PKG:xxx`` renaming such as :ref:`debian.bbclass <ref-classes-debian>`.
+  Doing this will produce an error.
+

Thanks for starting this. We need to now add something like:

"""
Network access from tasks is now disabled by default on kernels which support this (most recent distros e.g. centos8 and debian11 onwards). This means that tasks accessing the network need to be marked as such with the network flag, e.g.:

do_mytask[network] = "1"

This is allowed by default from do_fetch but not from any of our other standard tasks. Recipes shouldn't be accessing the network outside of do_fetch as it that usually undermines fetcher source mirroring, image manifests and licence manifests/auditing.
"""

Cheers,

Richard

#4 lore raw

It undermines software supply chain security as well, an important point to make I think. It is not okay to pull random things from the internet quietly.

Alex

Quoted 96 lines · Richard Purdie

Review

Quoted 1 line · Michael Opdenacker
(quoted diff) +0 -0

b/documentation/migration-guides/migration-3.5.rst

Quoted 5 lines

new file mode 100644
index 0000000000..c63c9dff05

documentation/migration-guides/migration-3.5.rst line 10

Quoted 10 lines
documentation/migration-guides/migration-3.5.rst new file +10 -0
@@ -0,0 +1,37 @@
+Release 3.5 (kirkstone)
+=======================
+
+This section provides migration information for moving to the Yocto
+Project 3.5 Release (codename "kirkstone") from the prior release.
+
+Recipe changes
+--------------
+
+- Because of the uncertainty in future default branch names in git

repositories,

documentation/migration-guides/migration-3.5.rst

Quoted 6 lines
documentation/migration-guides/migration-3.5.rst new file +6 -0
+  it is now required to add a branch name to all URLs described
+  by ``git://`` and ``gitsm://`` :term:`SRC_URI` entries. For example::
+
+     SRC_URI = "git://git.denx.de/u-boot.git;branch=master"
+
+  A :oe_git:`convert-srcuri

</openembedded-core/tree/scripts/contrib/convert-srcuri.py>`

documentation/migration-guides/migration-3.5.rst line 17

Quoted 1 line
documentation/migration-guides/migration-3.5.rst new file +1 -0
+  script to convert your recipes is available in

:term:OpenEmbedded-Core (OE-Core)

documentation/migration-guides/migration-3.5.rst

Quoted 4 lines
documentation/migration-guides/migration-3.5.rst new file +4 -0
+  and in :term:`Poky`.
+
+- Because of `GitHub dropping support for the git:
+  protocol <

https://github.blog/2021-09-01-improving-git-protocol-security-github/>`,

documentation/migration-guides/migration-3.5.rst

Quoted 4 lines
documentation/migration-guides/migration-3.5.rst new file +4 -0
+  recipes now need to use ``;protocol=https`` at the end of GitHub
+  URLs. The same script as above can be used to convert the recipes.
+
+- The :term:`TOPDIR` variable and the current working directory are no

longer modified

documentation/migration-guides/migration-3.5.rst line 26

Quoted 1 line
documentation/migration-guides/migration-3.5.rst new file +1 -0
+  when parsing recipes. Any code depending on that behaviour will no

longer worked as

documentation/migration-guides/migration-3.5.rst

Quoted 3 lines
documentation/migration-guides/migration-3.5.rst new file +3 -0
+  expected (we don't know of any).
+
+- BitBake no longer supports ``append``, ``prepend`` and ``remove``

operators

documentation/migration-guides/migration-3.5.rst

Quoted 5 lines
documentation/migration-guides/migration-3.5.rst new file +5 -0
+  combined with the ``+=``, ``=+``, ``.=``, ``=.`` and ``?=`` operators.
+  Such operations should be made using ``=`` alone. This means that
+  a leading space may need to be explicitly added.
+
+- :ref:`allarch <ref-classes-allarch>` packagegroups can no longer

depend on packages

documentation/migration-guides/migration-3.5.rst line 35

Quoted 1 line
documentation/migration-guides/migration-3.5.rst new file +1 -0
+  which use ``PKG:xxx`` renaming such as :ref:`debian.bbclass

<ref-classes-debian>`.

documentation/migration-guides/migration-3.5.rst line 37

Quoted 2 lines
documentation/migration-guides/migration-3.5.rst new file +2 -0
+  Doing this will produce an error.
+

Thanks for starting this. We need to now add something like:

"""
Network access from tasks is now disabled by default on kernels which support
this (most recent distros e.g. centos8 and debian11 onwards). This means that
tasks accessing the network need to be marked as such with the network flag,
e.g.:

do_mytask[network] = "1"

This is allowed by default from do_fetch but not from any of our other standard
tasks. Recipes shouldn't be accessing the network outside of do_fetch as it that
usually undermines fetcher source mirroring, image manifests and licence manifests/auditing.
"""

Cheers,

Richard

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#2414):
https://lists.yoctoproject.org/g/docs/message/2414
Mute This Topic: https://lists.yoctoproject.org/mt/87789405/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.yoctoproject.org/g/docs/unsub [ [email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

#5 lore raw

PATCH 1/1 migration-3.5: mention task specific network access

---
 documentation/migration-guides/migration-3.5.rst | 12 ++++++++++++
 1 file changed, 12 insertions(+)
documentation/migration-guides/migration-3.5.rst +12 -0
index bd807eb8c7..df15487b10 100644
@@ -22,6 +22,18 @@ Recipe changes
   recipes now need to use ``;protocol=https`` at the end of GitHub
   URLs. The same script as above can be used to convert the recipes.
 
+- Network access from tasks is now disabled by default on kernels which support
+  this feature (on most recent distros such as CentOS 8 and Debian 11 onwards).
+  This means that tasks accessing the network need to be marked as such with the ``network``
+  flag. For example::
+
+     do_mytask[network] = "1"
+
+  This is allowed by default from ``do_fetch`` but not from any of our other standard
+  tasks. Recipes shouldn't be accessing the network outside of ``do_fetch`` as it
+  usually undermines fetcher source mirroring, image and licence manifests, software
+  auditing and supply chain security.
+
 - The :term:`TOPDIR` variable and the current working directory are no longer modified
   when parsing recipes. Any code depending on that behaviour will no longer work.
 
Signature · 1 line
2.25.1
#6 lore raw

Re: [docs] [PATCH] migration-3.5: mention task specific network access

Quoted 17 lines · Michael Opdenacker
---
 documentation/migration-guides/migration-3.5.rst | 12 ++++++++++++
 1 file changed, 12 insertions(+)
documentation/migration-guides/migration-3.5.rst +4 -0
index bd807eb8c7..df15487b10 100644
@@ -22,6 +22,18 @@ Recipe changes
   recipes now need to use ``;protocol=https`` at the end of GitHub
   URLs. The same script as above can be used to convert the recipes.
 
+- Network access from tasks is now disabled by default on kernels which support
+  this feature (on most recent distros such as CentOS 8 and Debian 11 onwards).
+  This means that tasks accessing the network need to be marked as such with the ``network``
+  flag. For example::

documentation/migration-guides/migration-3.5.rst Recipe changes · line 38

Quoted 10 lines
documentation/migration-guides/migration-3.5.rst +8 -0
+
+     do_mytask[network] = "1"
+
+  This is allowed by default from ``do_fetch`` but not from any of our other standard
+  tasks. Recipes shouldn't be accessing the network outside of ``do_fetch`` as it
+  usually undermines fetcher source mirroring, image and licence manifests, software
+  auditing and supply chain security.
+
 - The :term:`TOPDIR` variable and the current working directory are no longer modified
   when parsing recipes. Any code depending on that behaviour will no longer work.

Looks good to me, thanks!

Cheers,

Richard