Skip to contentSkip to conversation

docs/manual/migrating.txt: add section with general migrating tips

RFC In review Arnout Vandecappelle (Essensium/Mind) 4 participants 9 messages Aug 2, 2021 – Aug 4, 2021

lore
Share & tools
lore
#0 lore raw

RFC 1/2 docs/manual/migrating.txt: add section with general migrating tips

This is based on Arnout's experience with migrating Buildroot.

---
 docs/manual/migrating.txt | 28 ++++++++++++++++++++++++++++
 1 file changed, 28 insertions(+)
docs/manual/migrating.txt +28 -0
index 92e487c71e..9fd7d7e676 100644
@@ -8,6 +8,34 @@ Some versions have introduced backward incompatibilities. This section
 explains those incompatibilities, and for each explains what to do to
 complete the migration.
 
+[[migrating-approach]]
+=== General approach
+
+To migrate from an older Buildroot version, take the following steps.
+
+. For all your configurations, do a build in the old Buildroot
+  environment. Save the full +.config+ and
+  +build/packages-file-list.txt+.
+. Review the specific migration notes below and make the required
+  adaptations to external packages and custom build scripts.
+. In the new Buildroot environment, run +make menuconfig+ starting from
+  the existing +.config+.
+. If anything is enabled in the Legacy menu, check its help text,
+  unselect it, and save the configuration.
+. Review the CHANGES file to see if any of your packages and features
+  are affected by the changes.
+. Build in the new Buildroot environment.
+. Fix build issues in external packages (usually due to updated
+  dependencies).
+. Compare the new +packages-file-list.txt+ with the original one, to
+  check if no required files have disappeared.
+. For configuration (and other) files in a custom overlay that overwrite
+  files created by Buildroot, check if there are changes in the
+  Buildroot-generated file that need to be propagated to your custom
+  file.
+. Run +make savedefconfig+ and verify that what is selected really is
+  what you intended to enable.
+
 [[br2-external-converting]]
 === Migrating to 2016.11
 
Signature · 6 lines
2.31.1

_______________________________________________
buildroot mailing list
[email protected]
http://lists.busybox.net/mailman/listinfo/buildroot
#1 lore raw

RFC 2/2 docs/manual/migrating.txt: add "individual packages" section

Add a section to the migrating documentation to explain what changed in individual packages and how it may affect your build.

This section is organised per package, not per Buildroot release, because in general it's easier to check package per package if anything changed than to check release per release for all your packages.

The idea is that every time a package is changed in a way that is not immediately obvious (mainly: affects runtime but not build time), it is mentioned in this section. This may include:

- a new version of the package got a new name due to API incompatibilities;
- the default behaviour or config optoins of the package changed;
- the Buildroot integration (e.g. configuration file, init script) of the package changed;
- what is installed by the package is changed (e.g. executable is renamed or removed).

To start with, add the recent changes of the installation defaults of bluez5_utils.

---
 docs/manual/migrating.txt | 15 +++++++++++++--
 1 file changed, 13 insertions(+), 2 deletions(-)
docs/manual/migrating.txt +13 -2
index 9fd7d7e676..36e4aa4724 100644
@@ -22,8 +22,8 @@ To migrate from an older Buildroot version, take the following steps.
   the existing +.config+.
 . If anything is enabled in the Legacy menu, check its help text,
   unselect it, and save the configuration.
-. Review the CHANGES file to see if any of your packages and features
-  are affected by the changes.
+. Review the individual packages section below to see if any of your
+  packages are affected by the changes.
 . Build in the new Buildroot environment.
 . Fix build issues in external packages (usually due to updated
   dependencies).
@@ -36,6 +36,17 @@ To migrate from an older Buildroot version, take the following steps.
 . Run +make savedefconfig+ and verify that what is selected really is
   what you intended to enable.
 
+[[migrating-packages]]
+=== Individual packages
+
+This section describes changes to individual packages. For all packages
+that are built in your configuration, check below if changes are needed.
+
+==== bluez5_utils
+
+Since 2021.08, the tools and all plugins are optional and default off.
+Make sure to enable the plugins you require.
+
 [[br2-external-converting]]
 === Migrating to 2016.11
 
Signature · 6 lines
2.31.1

_______________________________________________
buildroot mailing list
[email protected]
http://lists.busybox.net/mailman/listinfo/buildroot
#2 lore raw

Re: [Buildroot] [RFC 2/2] docs/manual/migrating.txt: add "individual packages" section

Arnout, All,

On 2021-08-02 18:22 +0200, Arnout Vandecappelle (Essensium/Mind) spake thusly:

Quoted 11 lines

Add a section to the migrating documentation to explain what changed in individual packages and how it may affect your build.

This section is organised per package, not per Buildroot release, because in general it's easier to check package per package if anything changed than to check release per release for all your packages.

The idea is that every time a package is changed in a way that is not immediately obvious (mainly: affects runtime but not build time), it is mentioned in this section. This may include:

Quoted 7 lines

- a new version of the package got a new name due to API incompatibilities;
- the default behaviour or config optoins of the package changed;
- the Buildroot integration (e.g. configuration file, init script) of the package changed;
- what is installed by the package is changed (e.g. executable is renamed or removed).

No. Please, no.

We already have two locations were changes in packages are identified:

- git commit logs, the authoritative reference,

- the CHANGES files

I do not want to have to maintain yet a third, ever-growing and never-trimmed list. Also, we will always have to bikeshed whther such a change is disruptive enough or not to warrant being on that list...

Instead, I would just suggest that people run "git log --oneline" or "git log -p --stat" on the packages they are interested in.

And of course, that means we *need* well-written commit logs (but we're already pretty good I believe).

Having a section about the core internals, like the br2-defconfig stuff that is not obvious, or about the RPATH of host packages, is acceptable, because it is not easily discoverable.

Regards,
Yann E. MORIN.

Quoted 44 lines

To start with, add the recent changes of the installation defaults of bluez5_utils.

---
 docs/manual/migrating.txt | 15 +++++++++++++--
 1 file changed, 13 insertions(+), 2 deletions(-)
docs/manual/migrating.txt +13 -2
index 9fd7d7e676..36e4aa4724 100644
@@ -22,8 +22,8 @@ To migrate from an older Buildroot version, take the following steps.
   the existing +.config+.
 . If anything is enabled in the Legacy menu, check its help text,
   unselect it, and save the configuration.
-. Review the CHANGES file to see if any of your packages and features
-  are affected by the changes.
+. Review the individual packages section below to see if any of your
+  packages are affected by the changes.
 . Build in the new Buildroot environment.
 . Fix build issues in external packages (usually due to updated
   dependencies).
@@ -36,6 +36,17 @@ To migrate from an older Buildroot version, take the following steps.
 . Run +make savedefconfig+ and verify that what is selected really is
   what you intended to enable.
 
+[[migrating-packages]]
+=== Individual packages
+
+This section describes changes to individual packages. For all packages
+that are built in your configuration, check below if changes are needed.
+
+==== bluez5_utils
+
+Since 2021.08, the tools and all plugins are optional and default off.
+Make sure to enable the plugins you require.
+
 [[br2-external-converting]]
 === Migrating to 2016.11
 
Signature · 1 line
2.31.1
Signature · 10 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
[email protected]
http://lists.busybox.net/mailman/listinfo/buildroot
#3 lore raw

Re: [Buildroot] [RFC 2/2] docs/manual/migrating.txt: add "individual packages" section

Quoted 28 lines · Yann E. MORIN

Arnout, All,

On 2021-08-02 18:22 +0200, Arnout Vandecappelle (Essensium/Mind) spake thusly:

Quoted 11 lines

Add a section to the migrating documentation to explain what changed in individual packages and how it may affect your build.

This section is organised per package, not per Buildroot release, because in general it's easier to check package per package if anything changed than to check release per release for all your packages.

The idea is that every time a package is changed in a way that is not immediately obvious (mainly: affects runtime but not build time), it is mentioned in this section. This may include:

Quoted 7 lines

- a new version of the package got a new name due to API incompatibilities;
- the default behaviour or config optoins of the package changed;
- the Buildroot integration (e.g. configuration file, init script) of the package changed;
- what is installed by the package is changed (e.g. executable is renamed or removed).

No. Please, no.

We already have two locations were changes in packages are identified:

- git commit logs, the authoritative reference,

Quoted 8 lines

- the CHANGES files

I do not want to have to maintain yet a third, ever-growing and never-trimmed list. Also, we will always have to bikeshed whther such a change is disruptive enough or not to warrant being on that list...

Instead, I would just suggest that people run "git log --oneline" or "git log -p --stat" on the packages they are interested in.

That's good advice as well. Simple "git log" should be sufficient though, if
the commit log is OK. "git log --oneline" is definitely not OK, because important changes during a version bump will not appear in the oneline.

I'm going to add a bullet point for this and merge it into the first patch.

Regards,
Arnout

Quoted 52 lines

And of course, that means we *need* well-written commit logs (but we're already pretty good I believe).

Having a section about the core internals, like the br2-defconfig stuff that is not obvious, or about the RPATH of host packages, is acceptable, because it is not easily discoverable.

Regards,
Yann E. MORIN.

Quoted 41 lines

To start with, add the recent changes of the installation defaults of bluez5_utils.

---
 docs/manual/migrating.txt | 15 +++++++++++++--
 1 file changed, 13 insertions(+), 2 deletions(-)
docs/manual/migrating.txt +13 -2
index 9fd7d7e676..36e4aa4724 100644
@@ -22,8 +22,8 @@ To migrate from an older Buildroot version, take the following steps.
   the existing +.config+.
 . If anything is enabled in the Legacy menu, check its help text,
   unselect it, and save the configuration.
-. Review the CHANGES file to see if any of your packages and features
-  are affected by the changes.
+. Review the individual packages section below to see if any of your
+  packages are affected by the changes.
 . Build in the new Buildroot environment.
 . Fix build issues in external packages (usually due to updated
   dependencies).
@@ -36,6 +36,17 @@ To migrate from an older Buildroot version, take the following steps.
 . Run +make savedefconfig+ and verify that what is selected really is
   what you intended to enable.
 
+[[migrating-packages]]
+=== Individual packages
+
+This section describes changes to individual packages. For all packages
+that are built in your configuration, check below if changes are needed.
+
+==== bluez5_utils
+
+Since 2021.08, the tools and all plugins are optional and default off.
+Make sure to enable the plugins you require.
+
 [[br2-external-converting]]
 === Migrating to 2016.11
 
Quoted 4 lines
Quoted 3 lines
Signature · 1 line
2.31.1

_______________________________________________
buildroot mailing list
[email protected]
http://lists.busybox.net/mailman/listinfo/buildroot

#4 lore raw

Re: [Buildroot] [RFC 2/2] docs/manual/migrating.txt: add "individual packages" section

Hi,

Quoted 35 lines · Arnout Vandecappelle
Quoted 33 lines · Yann E. MORIN

Arnout, All,

On 2021-08-02 18:22 +0200, Arnout Vandecappelle (Essensium/Mind) spake thusly:

Quoted 11 lines

Add a section to the migrating documentation to explain what changed in individual packages and how it may affect your build.

This section is organised per package, not per Buildroot release, because in general it's easier to check package per package if anything changed than to check release per release for all your packages.

The idea is that every time a package is changed in a way that is not immediately obvious (mainly: affects runtime but not build time), it is mentioned in this section. This may include:

Quoted 7 lines

- a new version of the package got a new name due to API incompatibilities;
- the default behaviour or config optoins of the package changed;
- the Buildroot integration (e.g. configuration file, init script) of the package changed;
- what is installed by the package is changed (e.g. executable is renamed or removed).

No. Please, no.

We already have two locations were changes in packages are identified:

- git commit logs, the authoritative reference,

- the CHANGES files

I do not want to have to maintain yet a third, ever-growing and never-trimmed list. Also, we will always have to bikeshed whther such a change is disruptive enough or not to warrant being on that list...

Quoted 7 lines
Quoted 2 lines

Instead, I would just suggest that people run "git log --oneline" or "git log -p --stat" on the packages they are interested in.

That's good advice as well. Simple "git log" should be sufficient though, if
the commit log is OK. "git log --oneline" is definitely not OK, because important changes during a version bump will not appear in the oneline.

I'm going to add a bullet point for this and merge it into the first patch.

One thing I want to point out: The releases are tarballs. Should we assume the user has the git history available?

Regards,
Michael

_______________________________________________
buildroot mailing list
[email protected]
http://lists.busybox.net/mailman/listinfo/buildroot

#5 lore raw

Re: [Buildroot] [RFC 2/2] docs/manual/migrating.txt: add "individual packages" section

Micheal, All,

On 2021-08-03 20:56 +0200, Michael Nosthoff spake thusly:

Quoted 5 lines · Arnout Vandecappelle
Quoted 4 lines · Yann E. MORIN
Quoted 3 lines

On 2021-08-02 18:22 +0200, Arnout Vandecappelle (Essensium/Mind) spake thusly:

Quoted 2 lines

Add a section to the migrating documentation to explain what changed in individual packages and how it may affect your build.

[--SNIP--]

Quoted 10 lines
Quoted 7 lines
Quoted 2 lines

Instead, I would just suggest that people run "git log --oneline" or "git log -p --stat" on the packages they are interested in.

That's good advice as well. Simple "git log" should be sufficient though, if
the commit log is OK. "git log --oneline" is definitely not OK, because important changes during a version bump will not appear in the oneline.

I'm going to add a bullet point for this and merge it into the first patch.

One thing I want to point out: The releases are tarballs. Should we assume the user has the git history available?

People are really still using tarballs in these days? ;-) But OK, you've got a point.

However, I would still think that people who want to follow a process when updating are probably already using buildroot a git submodule of some sorts, so yes, they will have a git tree to look at.

People who use a released tarball are not really looking for a sustained or long-term vision, and are not planning on upgrading.

So, I think we can state something like:

- look at the CHANGES file, and run 'git log' on the packages of your interest; if you are not using git, clone the upstream repository for this.

Regards,
Yann E. MORIN.

Signature · 10 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
[email protected]
http://lists.busybox.net/mailman/listinfo/buildroot
#6 lore raw

Re: [Buildroot] [RFC 2/2] docs/manual/migrating.txt: add "individual packages" section

Hello,

Quoted 9 lines · On Tue, 3 Aug 2021 21:47:58 +0200 "Yann E. MORIN" <[email protected]>…

People are really still using tarballs in these days? ;-) But OK, you've got a point.

However, I would still think that people who want to follow a process when updating are probably already using buildroot a git submodule of some sorts, so yes, they will have a git tree to look at.

People who use a released tarball are not really looking for a sustained or long-term vision, and are not planning on upgrading.

I was also surprised, but the other day, a person working for a large IP vendor company complained that he was not able to download the Buildroot tarballs. For some reason, the IP of this company had been blocked in iptables rules of buildroot.org. When I suggested to use Git instead, and especially the Github or Gitlab mirrors as a workaround, it apparently wasn't considered really a good solution.

So it does seem like some people are still using the tarballs, indeed.

Thomas

Signature · 7 lines
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
_______________________________________________
buildroot mailing list
[email protected]
http://lists.busybox.net/mailman/listinfo/buildroot
#7 lore raw
In reply to Arnout Vandecappelle (Essensium/Mind)

Arnout, All,

On 2021-08-02 18:22 +0200, Arnout Vandecappelle (Essensium/Mind) spake thusly:

Quoted 1 line

This is based on Arnout's experience with migrating Buildroot.

Except for the details, this also quite closely matches my own experience.

Quoted 11 lines
---
 docs/manual/migrating.txt | 28 ++++++++++++++++++++++++++++
 1 file changed, 28 insertions(+)
docs/manual/migrating.txt +0 -0
index 92e487c71e..9fd7d7e676 100644
@@ -8,6 +8,34 @@ Some versions have introduced backward incompatibilities. This section
 explains those incompatibilities, and for each explains what to do to

docs/manual/migrating.txt Some versions have introduced backward incompatibilities. This section · line 18

Quoted 10 lines
docs/manual/migrating.txt +8 -0
 complete the migration.
 
+[[migrating-approach]]
+=== General approach
+
+To migrate from an older Buildroot version, take the following steps.
+
+. For all your configurations, do a build in the old Buildroot
+  environment. Save the full +.config+ and
+  +build/packages-file-list.txt+.

(read my comment below first, then come back here) Maybe run 'make clean' here?

docs/manual/migrating.txt Some versions have introduced backward incompatibilities. This section · line 27

Quoted 9 lines
docs/manual/migrating.txt +9 -0
+. Review the specific migration notes below and make the required
+  adaptations to external packages and custom build scripts.
+. In the new Buildroot environment, run +make menuconfig+ starting from
+  the existing +.config+.
+. If anything is enabled in the Legacy menu, check its help text,
+  unselect it, and save the configuration.
+. Review the CHANGES file to see if any of your packages and features
+  are affected by the changes.
+. Build in the new Buildroot environment.

Above, you said to start with a full build, so if folowing this sequence to the letter, we still have a full build. We need to run 'make clean' first (go back reading my comment, above).

docs/manual/migrating.txt Some versions have introduced backward incompatibilities. This section · line 31

Quoted 4 lines
docs/manual/migrating.txt +4 -0
+. Fix build issues in external packages (usually due to updated
+  dependencies).
+. Compare the new +packages-file-list.txt+ with the original one, to
+  check if no required files have disappeared.

Maybe also suggest graph-size et al. about investigating size changes, too?

docs/manual/migrating.txt Some versions have introduced backward incompatibilities. This section · line 37

Quoted 6 lines
docs/manual/migrating.txt +6 -0
+. For configuration (and other) files in a custom overlay that overwrite
+  files created by Buildroot, check if there are changes in the
+  Buildroot-generated file that need to be propagated to your custom
+  file.
+. Run +make savedefconfig+ and verify that what is selected really is
+  what you intended to enable.

I am always wary instructing people to look at defconfig files; I always consider them to be artefacts of the configuration.

Rather, I suggest people read the output of legal-info to check that all the packages there are acceptable, and see graph-depends to see how a package came to the build.

Regards,
Yann E. MORIN.

Quoted 6 lines
docs/manual/migrating.txt +0 -0
 [[br2-external-converting]]
 === Migrating to 2016.11
 
Signature · 1 line
2.31.1
Signature · 10 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
[email protected]
http://lists.busybox.net/mailman/listinfo/buildroot
#8 lore raw
Quoted 7 lines · Yann E. MORIN

Arnout, All,

On 2021-08-02 18:22 +0200, Arnout Vandecappelle (Essensium/Mind) spake thusly:

Quoted 1 line

This is based on Arnout's experience with migrating Buildroot.

Except for the details, this also quite closely matches my own experience.

Excellent. I'll remove the RFC bit from v2 then.

Quoted 14 lines
Quoted 14 lines
---
 docs/manual/migrating.txt | 28 ++++++++++++++++++++++++++++
 1 file changed, 28 insertions(+)
docs/manual/migrating.txt +0 -0
index 92e487c71e..9fd7d7e676 100644
@@ -8,6 +8,34 @@ Some versions have introduced backward incompatibilities. This section
 explains those incompatibilities, and for each explains what to do to
 complete the migration.
 
Quoted 11 lines

docs/manual/migrating.txt Some versions have introduced backward incompatibilities. This section · line 18

Quoted 8 lines
docs/manual/migrating.txt +8 -0
+[[migrating-approach]]
+=== General approach
+
+To migrate from an older Buildroot version, take the following steps.
+
+. For all your configurations, do a build in the old Buildroot
+  environment. Save the full +.config+ and
+  +build/packages-file-list.txt+.

(read my comment below first, then come back here) Maybe run 'make clean' here?

Mentally I assumed there would be an "old" buildroot directory with its output
directory, and a "new" buildroot directory with its distinct output directory. But I indeed didn't write that, and in fact I don't work like that myself. So 'make clean' it is!

Quoted 14 lines

docs/manual/migrating.txt Some versions have introduced backward incompatibilities. This section · line 27

Quoted 9 lines
docs/manual/migrating.txt +9 -0
+. Review the specific migration notes below and make the required
+  adaptations to external packages and custom build scripts.
+. In the new Buildroot environment, run +make menuconfig+ starting from
+  the existing +.config+.
+. If anything is enabled in the Legacy menu, check its help text,
+  unselect it, and save the configuration.
+. Review the CHANGES file to see if any of your packages and features
+  are affected by the changes.
+. Build in the new Buildroot environment.

Above, you said to start with a full build, so if folowing this sequence to the letter, we still have a full build. We need to run 'make clean' first (go back reading my comment, above).

Quoted 7 lines

docs/manual/migrating.txt Some versions have introduced backward incompatibilities. This section · line 31

Quoted 4 lines
docs/manual/migrating.txt +4 -0
+. Fix build issues in external packages (usually due to updated
+  dependencies).
+. Compare the new +packages-file-list.txt+ with the original one, to
+  check if no required files have disappeared.

Maybe also suggest graph-size et al. about investigating size changes, too?

Excellent idea. You can easily compare the csv files.

Quoted 9 lines

docs/manual/migrating.txt Some versions have introduced backward incompatibilities. This section · line 37

Quoted 6 lines
docs/manual/migrating.txt +6 -0
+. For configuration (and other) files in a custom overlay that overwrite
+  files created by Buildroot, check if there are changes in the
+  Buildroot-generated file that need to be propagated to your custom
+  file.
+. Run +make savedefconfig+ and verify that what is selected really is
+  what you intended to enable.

I am always wary instructing people to look at defconfig files; I always consider them to be artefacts of the configuration.

I agree with the principle. Unfortunately, I know of no better way of getting
the "top-level" packages.

Quoted 3 lines

Rather, I suggest people read the output of legal-info to check that all the packages there are acceptable, and see graph-depends to see how a package came to the build.

Theoretically, graph-depends gives the top-level packages as well (i.e. the
packages that ALL points to). However, for a reasonably-sized project with about 50 top-level packages, the graph is pretty much unreadable. 'grep -w _all graphs/graph-depends.dot' is an alternative but it's not exactly user-friendly - and it doesn't show the selected sub-options like the defconfig does.

The defconfig isn't great either, of course. For example, it doesn't show the
sub-options that default y, and also choices aren't expressed very clearly.

Actually, the main reason I use the defconfig is that it comes for free: 'make
savedefconfig; git diff'. So maybe I should mention that instead.

Since this is a bit controversial, however, I'll just drop this point.

Regarding legal-info: if it's just about getting a list of packages, graph-size
already gives that, so I think I will talk about graph-size but not about legal-info.

Regards,
Arnout

Quoted 11 lines

Regards,
Yann E. MORIN.

Quoted 6 lines
docs/manual/migrating.txt +0 -0
 [[br2-external-converting]]
 === Migrating to 2016.11
 
Signature · 1 line
2.31.1

_______________________________________________
buildroot mailing list
[email protected]
http://lists.busybox.net/mailman/listinfo/buildroot