Skip to contentSkip to conversation

continue sanitizing '<pkg>_VERSION' variable

PATCH New Victor Huesca 3 participants 7 messages Jul 9, 2019 – Jul 10, 2019 Arm/LPC32XX SoC support

lore
Share & tools
lore
#0 lore raw

cover continue sanitizing '<pkg>_VERSION' variable

This patch expose the '_DL_VERSION' variable to packages and allow packages to override it. The goal is to have a clean '_VERSION' variable while preserving the current operation. In particular, this does not add any new variable since it was used internaly and is pretty generic so it can handle multiple corner cases (and not only the git ones).

The 2 following commits uses this variable to sanitize the '_VERSION' variable for git-fetch packages.

Victor Huesca (3):
package/pkg-generic: allow to use <pkg>_DL_VERSION in packages .mk
files
package: remove common prefix/suffix from git-fetched packages
package: remove non-conventional prefix/suffix from git-fetched
packages

 boot/lpc32xxcdl/lpc32xxcdl.mk               | 3 ++-
 package/at/at.mk                            | 3 ++-
 package/brickd/brickd.mk                    | 3 ++-
 package/ctorrent/ctorrent.mk                | 3 ++-
 package/dt-utils/dt-utils.mk                | 3 ++-
 package/f2fs-tools/f2fs-tools.mk            | 3 ++-
 package/fio/fio.mk                          | 3 ++-
 package/fmc/fmc.mk                          | 3 ++-
 package/fmlib/fmlib.mk                      | 3 ++-
 package/gstreamer1/gst1-shark/gst1-shark.mk | 3 ++-
 package/linux-zigbee/linux-zigbee.mk        | 3 ++-
 package/netcat-openbsd/netcat-openbsd.mk    | 3 ++-
 package/pkg-generic.mk                      | 4 +++-
 package/rcw/rcw.mk                          | 3 ++-
 package/speexdsp/speexdsp.mk                | 3 ++-
 package/trace-cmd/trace-cmd.mk              | 3 ++-
 16 files changed, 33 insertions(+), 16 deletions(-)
Signature · 1 line
2.21.0
#1 lore raw

PATCH 1/3 package/pkg-generic: allow to use <pkg>_DL_VERSION in packages .mk files

This patch continue the work done here http://buildroot-busybox.2317881.n4.nabble.com/PATCH-0-4-Sanetize-packages-version-tp225554.html
By making this DL_VERSION public we should be able to handle the missing corner-cases
with package version sanitizeing.

---
 package/pkg-generic.mk | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
package/pkg-generic.mk +3 -1
index 9620dec524..c48f54a2b7 100644
@@ -447,7 +447,9 @@ $(2)_PKGDIR			=  $(pkgdir)
 # version control system branch or tag, for example remotes/origin/1_10_stable.
 # Similar for spaces and colons (:) that may appear in date-based revisions for
 # CVS.
-ifndef $(2)_VERSION
+ifdef $(2)_DL_VERSION
+ $(2)_DL_VERSION := $$(strip $$($(2)_DL_VERSION))
+else ifndef $(2)_VERSION
  ifdef $(3)_DL_VERSION
   $(2)_DL_VERSION := $$($(3)_DL_VERSION)
  else ifdef $(3)_VERSION
Signature · 1 line
2.21.0
#2 lore raw

PATCH 1/3 package/pkg-generic: allow to use <pkg>_DL_VERSION in packages .mk files

Hi Victor,

Quoted 1 line · Victor Huesca

This patch continue the work done here http://buildroot-busybox.2317881.n4.nabble.com/PATCH-0-4-Sanetize-packages-version-tp225554.html

It's generally nices to summarize the discussion and put the reference as a
footnote, e.g.:

The _VERSION values should correspond to the ones on release-monitoring.org, so we can easily check if there is an update for a package. For downloaded packages, we can just add any prefix/suffix as part of the _SOURCE and/or _SITE variables. However, for VCS-downloaded packages, the _VERSION is used directly by the download infra.

This patch makes the DL_VERSION public, i.e. packages can set it (to a different value than _VERSION). This way, we can handle the missing corner cases without changes all over the places. See [1] for the full discussion.

[1]
http://buildroot-busybox.2317881.n4.nabble.com/PATCH-0-4-Sanetize-packages-version-tp225554.html

However, I don't think the discussion said anything more than what I wrote
above, so I don't think adding the reference is very valuable. It's up to you.

Quoted 13 lines

By making this DL_VERSION public we should be able to handle the missing corner-cases with package version sanitizeing.

---
 package/pkg-generic.mk | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
package/pkg-generic.mk +0 -0
index 9620dec524..c48f54a2b7 100644

package/pkg-generic.mk $(2)_PKGDIR = $(pkgdir) · line 449

Quoted 3 lines
package/pkg-generic.mk +0 -0
@@ -447,7 +447,9 @@ $(2)_PKGDIR			=  $(pkgdir)
 # version control system branch or tag, for example remotes/origin/1_10_stable.
 # Similar for spaces and colons (:) that may appear in date-based revisions for
 # CVS.

Maybe it would be good to add a comment like this:

# Since _DL_VERSION and _VERSION are defined based on themselves, we need to use # := assignment. Also, since they may be calculated through functions, it's best

# to strip them - this makes it easier for the packages themselves.

package/pkg-generic.mk $(2)_PKGDIR = $(pkgdir) · line 451

Quoted 3 lines
package/pkg-generic.mk +2 -1
-ifndef $(2)_VERSION
+ifdef $(2)_DL_VERSION
+ $(2)_DL_VERSION := $$(strip $$($(2)_DL_VERSION))

This will not work as intended... A few lines below, we have:

$(2)_VERSION := $$(call sanitize,$$($(2)_DL_VERSION))

so the _VERSION defined by the package will not be used at all...

I'm surprised that you didn't notice this. Maybe I'm mistaken? Or is there some
situation where the correct _VERSION does end up getting used?

I think the only solution is something like:

ifdef $(2)_DL_VERSION
$(2)_DL_VERSION := $$(strip $$($(2)_DL_VERSION)) $(2)_VERSION := $$(call sanitize,$$($(2)_DL_VERSION))
else
ifndef $(2)_VERSION
ifdef $(3)_DL_VERSION
$(2)_DL_VERSION := $$($(3)_DL_VERSION)
else ifdef $(3)_VERSION
$(2)_DL_VERSION := $$($(3)_VERSION)
endif
else
$(2)_DL_VERSION := $$(strip $$($(2)_VERSION))
endif
endif

Regards,
Arnout

Quoted 5 lines
package/pkg-generic.mk +1 -0
+else ifndef $(2)_VERSION
  ifdef $(3)_DL_VERSION
   $(2)_DL_VERSION := $$($(3)_DL_VERSION)
  else ifdef $(3)_VERSION
#3 lore raw

PATCH 1/3 package/pkg-generic: allow to use <pkg>_DL_VERSION in packages .mk files

Hi Arnout,

Quoted 18 lines · Arnout Vandecappelle

Hi Victor,

Quoted 1 line · Victor Huesca

This patch continue the work done here http://buildroot-busybox.2317881.n4.nabble.com/PATCH-0-4-Sanetize-packages-version-tp225554.html

It's generally nices to summarize the discussion and put the reference as a
footnote, e.g.:

The _VERSION values should correspond to the ones on release-monitoring.org, so we can easily check if there is an update for a package. For downloaded packages, we can just add any prefix/suffix as part of the _SOURCE and/or _SITE variables. However, for VCS-downloaded packages, the _VERSION is used directly by the download infra.

This patch makes the DL_VERSION public, i.e. packages can set it (to a different value than _VERSION). This way, we can handle the missing corner cases without changes all over the places. See [1] for the full discussion.

Quoted 7 lines

[1]
http://buildroot-busybox.2317881.n4.nabble.com/PATCH-0-4-Sanetize-packages-version-tp225554.html

However, I don't think the discussion said anything more than what I wrote
above, so I don't think adding the reference is very valuable. It's up to you.

I see, thank you, I'll use this for the v2.

Quoted 30 lines
Quoted 13 lines

By making this DL_VERSION public we should be able to handle the missing corner-cases with package version sanitizeing.

---
 package/pkg-generic.mk | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
package/pkg-generic.mk +0 -0
index 9620dec524..c48f54a2b7 100644

package/pkg-generic.mk $(2)_PKGDIR = $(pkgdir) · line 449

Quoted 3 lines
package/pkg-generic.mk +0 -0
@@ -447,7 +447,9 @@ $(2)_PKGDIR			=  $(pkgdir)
 # version control system branch or tag, for example remotes/origin/1_10_stable.
 # Similar for spaces and colons (:) that may appear in date-based revisions for
 # CVS.

Maybe it would be good to add a comment like this:

# Since _DL_VERSION and _VERSION are defined based on themselves, we need to use # := assignment. Also, since they may be calculated through functions, it's best

# to strip them - this makes it easier for the packages themselves.

package/pkg-generic.mk $(2)_PKGDIR = $(pkgdir) · line 451

Quoted 3 lines
package/pkg-generic.mk +2 -1
-ifndef $(2)_VERSION
+ifdef $(2)_DL_VERSION
+ $(2)_DL_VERSION := $$(strip $$($(2)_DL_VERSION))

This will not work as intended... A few lines below, we have:

Quoted 6 lines

$(2)_VERSION := $$(call sanitize,$$($(2)_DL_VERSION))

so the _VERSION defined by the package will not be used at all...

I'm surprised that you didn't notice this. Maybe I'm mistaken? Or is there some
situation where the correct _VERSION does end up getting used?

You're right, I was focused on making the download work and forgot that now that _DL_VERSION and _VERSION can differs we should keep _VERSION rather than _DL_VERSION.

Quoted 15 lines

I think the only solution is something like:

ifdef $(2)_DL_VERSION
$(2)_DL_VERSION := $$(strip $$($(2)_DL_VERSION)) $(2)_VERSION := $$(call sanitize,$$($(2)_DL_VERSION))> else ifndef $(2)_VERSION
ifdef $(3)_DL_VERSION
$(2)_DL_VERSION := $$($(3)_DL_VERSION)
else ifdef $(3)_VERSION
$(2)_DL_VERSION := $$($(3)_VERSION)
endif
else
$(2)_DL_VERSION := $$(strip $$($(2)_VERSION))
endif
endif

Maybe I am mistaken but to me this code produce exactly the same results as mine...

Anyway, now that you pointed out to me that we should keep the original _VERSION I can make a v2 that take this in account.

Signature · 3 lines
Victor Huesca, Bootlin
Embedded Linux and kernel engineering
https://bootlin.com
#4 lore raw

PATCH 1/3 package/pkg-generic: allow to use <pkg>_DL_VERSION in packages .mk files

In reply to Victor Huesca

Hello Victor,

Thanks for the work on cleaning up those version variables!

Quoted 5 lines · On Tue, 9 Jul 2019 09:45:49 +0200 Victor Huesca <[email protected]>…

This patch continue the work done here http://buildroot-busybox.2317881.n4.nabble.com/PATCH-0-4-Sanetize-packages-version-tp225554.html
By making this DL_VERSION public we should be able to handle the missing corner-cases
with package version sanitizeing.

I just want to point out that this patch makes <pkg>_DL_VERSION a "public" variable, so it needs to be documented in the Buildroot manual. It should be done in a patch in the same patch series.

Also, nitpick: make sure to wrap your commit messages (and generally e-mails) at 72 characters.

Thanks!

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
#5 lore raw

PATCH 2/3 package: remove common prefix/suffix from git-fetched packages

In reply to Victor Huesca

While many project name their git tags <version> where <version> is just the version number, some project add other prefix/suffix to their version number. Common cases are:
- Add 'v' like '<pkg>_VERSION = v0.3.1'
- Add the package name (often separeted with a '/', '-' or '.') like '<pkg>_VERSION = <pkg>-1.5.2' or '<pkg>_VERSION = <pkg>/1.5.2'.
- Add a generic suffix like '-release', '-dev' or 'debian'
- Any combinaison of the those.

In Buildroot, we currently encode the full version (with its prefix/suffix) in the <pkg>_VERSION variable. However, that doesn't work well to match version numbers with the release-monitoring.org website, which filters such prefixes/suffixes.

This commit rely on the '<pkg>_DL_VERSION' variable.

Other corner-case witch very specific prefix/suffix will be handle by a separate commit.

---
 boot/lpc32xxcdl/lpc32xxcdl.mk               | 3 ++-
 package/at/at.mk                            | 3 ++-
 package/dt-utils/dt-utils.mk                | 3 ++-
 package/f2fs-tools/f2fs-tools.mk            | 3 ++-
 package/fio/fio.mk                          | 3 ++-
 package/gstreamer1/gst1-shark/gst1-shark.mk | 3 ++-
 package/linux-zigbee/linux-zigbee.mk        | 3 ++-
 package/netcat-openbsd/netcat-openbsd.mk    | 3 ++-
 package/speexdsp/speexdsp.mk                | 3 ++-
 package/trace-cmd/trace-cmd.mk              | 3 ++-
 10 files changed, 20 insertions(+), 10 deletions(-)
boot/lpc32xxcdl/lpc32xxcdl.mk +2 -1
index 30afed050b..6f3c929f0e 100644
@@ -4,7 +4,8 @@
 #
 ################################################################################
 
-LPC32XXCDL_VERSION = lpc32xx_cdl_v2.11
+LPC32XXCDL_VERSION = 2.11
+LPC32XXCDL_DL_VERSION = lpc32xx_cdl_v$(LPC32XXCDL_VERSION)
 LPC32XXCDL_SITE = http://git.lpcware.com/lpc3xxx_cdl.git
 LPC32XXCDL_SITE_METHOD = git
 
package/at/at.mk +2 -1
index 36aa8c7a42..93e33be19c 100644
@@ -4,7 +4,8 @@
 #
 ################################################################################
 
-AT_VERSION = release/3.1.23
+AT_VERSION = 3.1.23
+AT_DL_VERSION = release/$(AT_VERSION)
 AT_SITE = https://salsa.debian.org/debian/at.git
 AT_SITE_METHOD = git
 # Tried to add missing deps for parsetime.l but still parallel build fails
package/dt-utils/dt-utils.mk +2 -1
index 25c287d434..7b503f3b02 100644
@@ -4,7 +4,8 @@
 #
 ################################################################################
 
-DT_UTILS_VERSION = v2019.01.0
+DT_UTILS_VERSION = 2019.01.0
+DT_UTILS_DL_VERSION = v$(DT_UTILS_VERSION)
 DT_UTILS_SITE = https://git.pengutronix.de/git/tools/dt-utils
 DT_UTILS_SITE_METHOD = git
 DT_UTILS_LICENSE = GPL-2.0
package/f2fs-tools/f2fs-tools.mk +2 -1
index ba16218bda..424db55a63 100644
@@ -4,7 +4,8 @@
 #
 ################################################################################
 
-F2FS_TOOLS_VERSION = v1.11.0
+F2FS_TOOLS_VERSION = 1.11.0
+F2FS_TOOLS_DL_VERSION = v$(F2FS_TOOLS_VERSION)
 F2FS_TOOLS_SITE = http://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs-tools.git
 F2FS_TOOLS_SITE_METHOD = git
 F2FS_TOOLS_CONF_ENV = ac_cv_file__git=no
package/fio/fio.mk +2 -1
index 34e24e55de..f31c181b63 100644
@@ -4,7 +4,8 @@
 #
 ################################################################################
 
-FIO_VERSION = fio-3.9
+FIO_VERSION = 3.9
+FIO_DL_VERSION = fio-$(FIO_VERSION)
 FIO_SITE = git://git.kernel.dk/fio.git
 FIO_LICENSE = GPL-2.0
 FIO_LICENSE_FILES = COPYING MORAL-LICENSE
package/gstreamer1/gst1-shark/gst1-shark.mk +2 -1
index cdf3145429..3e3352d3e0 100644
@@ -4,7 +4,8 @@
 #
 ################################################################################
 
-GST1_SHARK_VERSION = v0.6.0
+GST1_SHARK_VERSION = 0.6.0
+GST1_SHARK_DL_VERSION = v$(GST1_SHARK_VERSION)
 GST1_SHARK_SITE =  https://github.com/RidgeRun/gst-shark.git
 GST1_SHARK_SITE_METHOD = git
 GST1_SHARK_GIT_SUBMODULES = YES
package/linux-zigbee/linux-zigbee.mk +2 -1
index 8d3248f2e1..3f1174492e 100644
@@ -4,7 +4,8 @@
 #
 ################################################################################
 
-LINUX_ZIGBEE_VERSION = v0.3.1
+LINUX_ZIGBEE_VERSION = 0.3.1
+LINUX_ZIGBEE_DL_VERSION = v$(LINUX_ZIGBEE_VERSION)
 LINUX_ZIGBEE_SITE = git://linux-zigbee.git.sourceforge.net/gitroot/linux-zigbee/linux-zigbee
 LINUX_ZIGBEE_LICENSE = GPL-2.0
 LINUX_ZIGBEE_LICENSE_FILES = COPYING
package/netcat-openbsd/netcat-openbsd.mk +2 -1
index a2333531c0..41a3035c42 100644
@@ -4,7 +4,8 @@
 #
 ################################################################################
 
-NETCAT_OPENBSD_VERSION = debian/1.195-2
+NETCAT_OPENBSD_VERSION = 1.195-2
+NETCAT_OPENBSD_DL_VERSION = debian/$(NETCAT_OPENBSD_VERSION)
 NETCAT_OPENBSD_SITE = https://salsa.debian.org/debian/netcat-openbsd.git
 NETCAT_OPENBSD_SITE_METHOD = git
 NETCAT_OPENBSD_LICENSE = BSD-3-Clause
package/speexdsp/speexdsp.mk +2 -1
index f5a46cd9de..a357258614 100644
@@ -4,7 +4,8 @@
 #
 ################################################################################
 
-SPEEXDSP_VERSION = SpeexDSP-1.2.0
+SPEEXDSP_VERSION = 1.2.0
+SPEEXDSP_DL_VERSION = SpeexDSP-$(SPEEXDSP_VERSION)
 SPEEXDSP_SITE = https://git.xiph.org/speexdsp.git
 SPEEXDSP_SITE_METHOD = git
 SPEEXDSP_LICENSE = BSD-3-Clause
package/trace-cmd/trace-cmd.mk +2 -1
index dfca6a170d..9440fafe17 100644
@@ -4,7 +4,8 @@
 #
 ################################################################################
 
-TRACE_CMD_VERSION = trace-cmd-v2.7
+TRACE_CMD_VERSION = 2.7
+TRACE_CMD_DL_VERSION = trace-cmd-v$(TRACE_CMD_VERSION)
 TRACE_CMD_SITE = https://git.kernel.org/pub/scm/linux/kernel/git/rostedt/trace-cmd.git
 TRACE_CMD_SITE_METHOD = git
 TRACE_CMD_INSTALL_STAGING = YES
Signature · 1 line
2.21.0
#6 lore raw

PATCH 3/3 package: remove non-conventional prefix/suffix from git-fetched packages

In reply to Victor Huesca

While many project name their git tags <version> where <version> is just the version number, some project add other prefix/suffix to their version number. In these laters, the suffix/suffix is often simply 'v', 'release' or the package name, but there are some weirder naming. This patch aims to fix these special cases.

In Buildroot, we currently encode the full version (with its prefix/suffix) in the <pkg>_VERSION variable. However, that doesn't work well to match version numbers with the release-monitoring.org website, which filters such prefixes/suffixes.

This commit rely on the '<pkg>_DL_VERSION' variable, so dl archive name are unchanged.

---
 package/brickd/brickd.mk     | 3 ++-
 package/ctorrent/ctorrent.mk | 3 ++-
 package/fmc/fmc.mk           | 3 ++-
 package/fmlib/fmlib.mk       | 3 ++-
 package/rcw/rcw.mk           | 3 ++-
 5 files changed, 10 insertions(+), 5 deletions(-)
package/brickd/brickd.mk +2 -1
index 7fe28daf78..66865f00e1 100644
@@ -4,7 +4,8 @@
 #
 ################################################################################
 
-BRICKD_VERSION = ev3dev-stretch/1.2.1
+BRICKD_VERSION = 1.2.1
+BRICKD_DL_VERSION = ev3dev-stretch/$(BRICKD_VERSION)
 BRICKD_SITE = https://github.com/ev3dev/brickd
 BRICKD_SITE_METHOD = git
 BRICKD_GIT_SUBMODULES = YES
package/ctorrent/ctorrent.mk +2 -1
index 959e543f7b..26bfcc335e 100644
@@ -4,7 +4,8 @@
 #
 ################################################################################
 
-CTORRENT_VERSION = dnh3.3.2
+CTORRENT_VERSION = 3.3.2
+CTORRENT_DL_VERSION = dnh$(CTORRENT_VERSION)
 CTORRENT_SITE = http://www.rahul.net/dholmes/ctorrent
 CTORRENT_LICENSE = GPL-2.0
 CTORRENT_LICENSE_FILES = COPYING
package/fmc/fmc.mk +2 -1
index 2a95fec0b7..db1ac8509a 100644
@@ -4,7 +4,8 @@
 #
 ################################################################################
 
-FMC_VERSION = fsl-sdk-v2.0
+FMC_VERSION = 2.0
+FMC_DL_VERSION = fsl-sdk-v$(FMC_VERSION)
 FMC_SITE = git://git.freescale.com/ppc/sdk/fmc.git
 FMC_LICENSE = MIT
 FMC_LICENSE_FILES = COPYING
package/fmlib/fmlib.mk +2 -1
index e67ab1e5de..a5813e948b 100644
@@ -4,7 +4,8 @@
 #
 ################################################################################
 
-FMLIB_VERSION = fsl-sdk-v2.0
+FMLIB_VERSION = 2.0
+FMLIB_DL_VERSION = fsl-sdk-v$(FMLIB_VERSION)
 FMLIB_SITE = git://git.freescale.com/ppc/sdk/fmlib.git
 FMLIB_LICENSE = BSD-3-Clause, GPL-2.0+
 FMLIB_LICENSE_FILES = COPYING
package/rcw/rcw.mk +2 -1
index f4570b9bde..53b4cc2a30 100644
@@ -4,7 +4,8 @@
 #
 ################################################################################
 
-RCW_VERSION = LSDK-18.12
+RCW_VERSION = 18.12
+RCW_DL_VERSION = LSDK-$(RCW_VERSION)
 RCW_SITE = https://source.codeaurora.org/external/qoriq/qoriq-components/rcw
 RCW_SITE_METHOD = git
 RCW_LICENSE = BSD-3-Clause
Signature · 1 line
2.21.0