Skip to contentSkip to conversation

migration-guides: update 4.2 migration and release notes

PATCH Superseded Michael Opdenacker 3 participants 7 messages Apr 5, 2023 – Apr 6, 2023 1 message not in the archive

v2 of 5 Superseded Latest is v5 →

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

#1 lore raw

Greetings,

I copied the OE-core mailing list to get the attention of contributors. If you want your new features to be featured in the release and migration notes for 4.2, it's time to speak!

What I wrote so far is a very early draft meant to help you detect what's missing and correct what I've written so far.

See my questions and comments below...

Quoted 4 lines · Michael Opdenacker via lists.openembedded.org
documentation/migration-guides/migration-4.2.rst +1 -0
index 1339411b3e..3b04e072cb 100644
...
+Rust: mandatory checksums for crates

documentation/migration-guides/migration-4.2.rst

Quoted 10 lines
documentation/migration-guides/migration-4.2.rst +10 -0
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+This release now supports checksums for Rust crates and make
+them mandatory for each crate in a recipe. See :yocto_git:`python3_bcrypt recipe changes
+</poky/commit/?h=mickledore&id=0dcb5ab3462fdaaf1646b05a00c7150eea711a9a>`
+for example.
+
+The ``cargo-update-recipe-crates`` utility
+:yocto_git:`has been extended </poky/commit/?h=mickledore&id=eef7fbea2c5bf59369390be4d5efa915591b7b22>`
+to include such checksums.

Any more Rust changes we should mention in the migration notes?

documentation/migration-guides/release-notes-4.2.rst

Quoted 4 lines
documentation/migration-guides/release-notes-4.2.rst +0 -0
index edafe378b0..1a60fe5490 100644

Here in the release notes, I didn't work on the detailed changes, such as version changes for the kernel and many recipes. I'm trying to help with the first part showing the big picture.

Quoted 6 lines
documentation/migration-guides/release-notes-4.2.rst +4 -1
-     addpylib ${LAYERDIR}/lib oe
+      addpylib ${LAYERDIR}/lib oe
+
+-  BitBake has seen multiple internal changes that may impact
+   memory and disk usage as well as parsing time, in particular:

documentation/migration-guides/release-notes-4.2.rst

Quoted 10 lines
documentation/migration-guides/release-notes-4.2.rst +10 -0
+
+   -  BitBake's Cooker server is now multithreaded.
+
+   -  BitBake's cache has been extended to include more hash
+      debugging data, but has also been optimized to :yocto_git:`compress
+      cache data <https://git.yoctoproject.org/poky/commit/?h=mickledore&id=7d010055e2af3294e17db862f42664ca689a9356>`.
+
+   -  BitBake's Cooker server :yocto_git:`can now be pinged
+      https://git.yoctoproject.org/poky/commit/?h=mickledore&id=26f255da09>`
+      from the UI.

More BitBake changes we should mention?

documentation/migration-guides/release-notes-4.2.rst

Quoted 8 lines
documentation/migration-guides/release-notes-4.2.rst +8 -0
+
+-  Rust improvements:
+
+   -  This release adds Cargo support on the target, and includes
+      automated QA tests for this functionality.
+
+   -  It also supports checksums for Rust crates and makes
+      them mandatory for each crate in a recipe.

More Rust changes we should highlight?

documentation/migration-guides/release-notes-4.2.rst

Quoted 4 lines

+- Miscellaneous changes:

documentation/migration-guides/release-notes-4.2.rst +3 -0
+
+   -  Supporting 64 bit dates on 32 bit platforms: several packages have been
+      updated to pass Y2038 tests.

By the way, what's the status of Y2038 support in Mickledore? Are all tests passing now?

To make the drafts easier to review, they have been compiled in the master-next branch:
https://docs.yoctoproject.org/next/migration-guides/release-4.2.html

Thanks in advance
Michael.

Signature · 3 lines
Michael Opdenacker, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
#2 lore raw
Quoted 29 lines · Michael Opdenacker

Greetings,

I copied the OE-core mailing list to get the attention of contributors. If you want your new features to be featured in the release and migration notes for 4.2, it's time to speak!

What I wrote so far is a very early draft meant to help you detect what's missing and correct what I've written so far.

See my questions and comments below...

Review

Quoted 1 line · Michael Opdenacker via lists.openembedded.org
(quoted diff) +0 -0

b/documentation/migration-guides/migration-4.2.rst

Review

Quoted 7 lines

index 1339411b3e..3b04e072cb 100644
...

(quoted diff) +5 -0
+Rust: mandatory checksums for crates
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+This release now supports checksums for Rust crates and make
+them mandatory for each crate in a recipe. See

:yocto_git:`python3_bcrypt recipe changes

Quoted 1 line

+</poky/commit/?h=mickledore&id=0dcb5ab3462fdaaf1646b05a00c7150eea711a9a>`

Review

Quoted 4 lines
(quoted diff) +4 -0
+for example.
+
+The ``cargo-update-recipe-crates`` utility
+:yocto_git:`has been extended

</poky/commit/?h=mickledore&id=eef7fbea2c5bf59369390be4d5efa915591b7b22>`

Quoted 6 lines
Quoted 1 line

+to include such checksums.

Any more Rust changes we should mention in the migration notes?

Maybe mention a specific procedure to update a cargo based recipe from scratch (i.e. a recipe that
didn't use the cargo-update-recipe-crates inheritance):

To bootstrap a cargo base recipe which have not the checksum list, below are the steps to make:
- make the recipe inherit cargo-update-recipe-crates
- remove all crate:// from the recipe
- create an empty ${BPN}-crates.inc file and make your recipe requires it
- execute "bitbake -c update_crates your_recipe"
- copy paste the output of bitbake about the missing checksums in your
${BPN}-crates.inc

The last step requires a bitbake patch to show all the missing checksum which I don't know
if it will be in 4.2.

Quoted 12 lines

Review

Quoted 8 lines
(quoted diff) +8 -0
+
+-  Rust improvements:
+
+   -  This release adds Cargo support on the target, and includes
+      automated QA tests for this functionality.
+
+   -  It also supports checksums for Rust crates and makes
+      them mandatory for each crate in a recipe.

More Rust changes we should highlight?

This one seems redundant with the previously mentioned rust changes, isn't it ?
Nevertheless, I'd maybe add a line about the fact that now we can develop
rust project which contains "private" crate (i.e. crates that would be hosted
in a git repository instead of crates.io registry), and by develop, it means we
can of course build such a package but perform devtool operation on it (at least devtool modify and devtool deploy-target)

#3 lore raw

Hi Frederic

Quoted 17 lines · Frederic Martinsons

Any more Rust changes we should mention in the migration notes?

Maybe mention a specific procedure to update a cargo based recipe from scratch (i.e. a recipe that didn't use the cargo-update-recipe-crates inheritance):

To bootstrap a cargo base recipe which have not the checksum list, below are the steps to make:
  - make the recipe inherit cargo-update-recipe-crates
  - remove all crate:// from the recipe
  - create an empty ${BPN}-crates.inc file and make your recipe requires it
  - execute "bitbake -c update_crates your_recipe"
  - copy paste the output of bitbake about the missing checksums in your ${BPN}-crates.inc

Quoted 2 lines

The last step requires a bitbake patch to show all the missing checksum which I don't know if it will be in 4.2.

Do you mean this one?
https://git.openembedded.org/bitbake/commit/?h=2.4&id=dafa07c080e05975b6319b5adf78a9691c6b6643

It just got merged.
Thanks for the tip.

Cheers
Michael.

Signature · 3 lines
Michael Opdenacker, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
#4 lore raw
Quoted 36 lines · Michael Opdenacker

Hi Frederic

Quoted 21 lines · Frederic Martinsons

Any more Rust changes we should mention in the migration notes?

Maybe mention a specific procedure to update a cargo based recipe from scratch (i.e. a recipe that
didn't use the cargo-update-recipe-crates inheritance):

To bootstrap a cargo base recipe which have not the checksum list, below are
the steps to make:
- make the recipe inherit cargo-update-recipe-crates
- remove all crate:// from the recipe
- create an empty ${BPN}-crates.inc file and make your recipe
requires it
- execute "bitbake -c update_crates your_recipe"
- copy paste the output of bitbake about the missing checksums in
your ${BPN}-crates.inc

Quoted 3 lines

The last step requires a bitbake patch to show all the missing checksum which I don't know
if it will be in 4.2.

Do you mean this one?

https://git.openembedded.org/bitbake/commit/?h=2.4&id=dafa07c080e05975b6319b5adf78a9691c6b6643

It just got merged.
Thanks for the tip.

Quoted 8 lines

Cheers
Michael.

--
Michael Opdenacker, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Yes that's the one. Great.

#5 lore raw

PATCH v2 1/1 migration-guides: update 4.2 migration and release notes

From: Michael Opdenacker <[email protected]>

CC Frederic Martinsons <[email protected]>
CC Marta Rybczynska <[email protected]>
CC Paul Eggleton <[email protected]>
CC Alex Kiernan <[email protected]>

---

Changes in V2:
- Add details about filling checksums using the 'cargo-update-recipe-crates' class, as suggested by Frederic Martinsons.

---
 .../migration-guides/migration-4.2.rst        | 84 ++++++++++++++++++-
 .../migration-guides/release-notes-4.2.rst    | 68 +++++++++++++--
 2 files changed, 143 insertions(+), 9 deletions(-)
documentation/migration-guides/migration-4.2.rst +83 -1
index 1339411b3e..3b60b9a807 100644
@@ -18,11 +18,78 @@ BitBake and OpenEmbedded-Core are now relying on Python 3.8,
 making it a requirement to use a distribution providing at least this
 version, or to use :term:`buildtools`.
 
+.. _migration-4.2-gcc-8.0:
+
+gcc 8.0 is now the minumum required GNU C compiler version
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+This version, released in 2018, is a minimum requirement
+to build the ``mesa-native`` recipe.
+
+.. _migration-4.2-new-nvd-api:
+
+Fetching the NVD vulnerability database through the 2.0 API
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+This new version adds a new fetcher for the NVD database using the 2.0 API,
+as the 1.0 API will be retired in 2023.
+
+The implementation changes as little as possible, keeping the current
+database format (but using a different database file for the transition
+period), with a notable exception of not using the META table.
+
+Here are minor changes that you may notice:
+
+-  The database starts in 1999 instead of 2002
+-  The complete fetch is longer (30 minutes typically)
+
+.. _migration-4.2-rust-crate-checksums:
+
+Rust: mandatory checksums for crates
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+This release now supports checksums for Rust crates and make
+them mandatory for each crate in a recipe. See :yocto_git:`python3_bcrypt recipe changes
+</poky/commit/?h=mickledore&id=0dcb5ab3462fdaaf1646b05a00c7150eea711a9a>`
+for example.
+
+The ``cargo-update-recipe-crates`` utility
+:yocto_git:`has been extended </poky/commit/?h=mickledore&id=eef7fbea2c5bf59369390be4d5efa915591b7b22>`
+to include such checksums. So, in case you need to add the list of checksums
+to a recipe just inheriting the :ref:`ref-classes-cargo` class so far, you can
+follow these steps:
+
+#.  Make the recipe inherit :ref:`ref-classes-cargo-update-recipe-crates`
+#.  Remove all ``crate://`` lines from the recipe
+#.  Create an empty ``${BPN}-crates.inc`` file and make your recipe require it
+#.  Execute ``bitbake -c update_crates your_recipe``
+#.  Copy and paste the output of BitBake about the missing checksums into the
+    ``${BPN}-crates.inc`` file.
+
 .. _migration-4.2-qa-checks:
 
 QA check changes
 ~~~~~~~~~~~~~~~~
 
+.. _migration-4.2-supported-distributions:
+
+Supported distributions
+~~~~~~~~~~~~~~~~~~~~~~~
+
+This release supports running BitBake on new GNU/Linux distributions:
+
+-  Fedora 36 and 37
+-  AlmaLinux 8.7 and 9.1
+-  OpenSuse 15.4
+
+On the other hand, some earlier distributions are no longer supported:
+
+-  Debian 10.x
+-  Fedora 34 and 35
+-  AlmaLinux 8.5
+
+See :ref:`all supported distributions <system-requirements-supported-distros>`.
+
 .. _migration-4.2-misc-changes:
 
 Miscellaneous changes
@@ -30,6 +97,7 @@ Miscellaneous changes
 
 -  The ``OEBasic`` signature handler (see :term:`BB_SIGNATURE_HANDLER`) has been
    removed.
+  
 
 .. _migration-4.2-removed-variables:
 
@@ -38,7 +106,7 @@ Removed variables
 
 The following variables have been removed:
 
--  ``SERIAL_CONSOLE``, deprecated since version 2.6, replaced by :term:``SERIAL_CONSOLES``.
+-  ``SERIAL_CONSOLE``, deprecated since version 2.6, replaced by :term:`SERIAL_CONSOLES`.
 
 .. _migration-4.2-removed-recipes:
 
@@ -47,3 +115,17 @@ Removed recipes
 
 The following recipes have been removed in this release:
 
+-  ``python3-picobuild``: after switching to ``python3-build``
+-  ``python3-strict-rfc3339``: unmaintained and not needed by anything in
+   :oe_git:`openembedded-core </openembedded-core>`
+   or :oe_git:`meta-openembedded </meta-openembedded>`.
+
+.. _migration-4.2-removed-classes:
+
+Removed classes
+~~~~~~~~~~~~~~~
+
+The following classes have been removed in this release:
+
+-  ``rust-bin``: no longer used in Poky
+
documentation/migration-guides/release-notes-4.2.rst +60 -8

83 lines not on this page · raw message

Signature · 1 line
2.34.1
#6 lore raw
Quoted 8 lines · Michael Opdenacker via lists.openembedded.org

From: Michael Opdenacker <[email protected]>

CC Frederic Martinsons <[email protected]>
CC Marta Rybczynska <[email protected]>
CC Paul Eggleton <[email protected]>
CC Alex Kiernan <[email protected]>
Quoted 5 lines

---

Changes in V2:
- Add details about filling checksums using the 'cargo-update-recipe-crates' class, as suggested by Frederic Martinsons.

We should add something about ptest changes:

"""
The ptest images have changed structure in this release. The underlying core-image-ptest recipe now uses BBCLASSEXTEND to create a variant for each ptest enabled recipe in OE-Core.

For example, this means that "core-image-ptest-bzip2", "core-image- ptest-lttng-tools" and many more image targets now exist and can be built/tested individually.

The core-image-ptest-all and core-image-ptest-fast targets are now wrappers that target groups of individual images and means that the tests can be executed in parallel during our automated testing. This also means the dependencies are more accurately tested. """

Cheers,

Richard

#7 lore raw
Quoted 4 lines · Richard Purdie

The ptest images have changed structure in this release. The underlying core-image-ptest recipe now uses BBCLASSEXTEND to create a variant for each ptest enabled recipe in OE-Core.

Quoted 8 lines

For example, this means that "core-image-ptest-bzip2", "core-image- ptest-lttng-tools" and many more image targets now exist and can be built/tested individually.

The core-image-ptest-all and core-image-ptest-fast targets are now wrappers that target groups of individual images and means that the tests can be executed in parallel during our automated testing. This also means the dependencies are more accurately tested.

Thanks Richard. This will be included in the next update I send. Cheers
Michael.

Signature · 3 lines
Michael Opdenacker, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com