continue sanitizing '<pkg>_VERSION' variable
cover continue sanitizing '<pkg>_VERSION' variable
This patch expose the '_DL_VERSION' variable to packages and allow packages to override it. The goal is to have a clean '_VERSION' variable while preserving the current operation. In particular, this does not add any new variable since it was used internaly and is pretty generic so it can handle multiple corner cases (and not only the git ones).
The 2 following commits uses this variable to sanitize the '_VERSION' variable for git-fetch packages.
Victor Huesca (3):
package/pkg-generic: allow to use <pkg>_DL_VERSION in packages .mk
files
package: remove common prefix/suffix from git-fetched packages
package: remove non-conventional prefix/suffix from git-fetched
packages
boot/lpc32xxcdl/lpc32xxcdl.mk | 3 ++- package/at/at.mk | 3 ++- package/brickd/brickd.mk | 3 ++- package/ctorrent/ctorrent.mk | 3 ++- package/dt-utils/dt-utils.mk | 3 ++- package/f2fs-tools/f2fs-tools.mk | 3 ++- package/fio/fio.mk | 3 ++- package/fmc/fmc.mk | 3 ++- package/fmlib/fmlib.mk | 3 ++- package/gstreamer1/gst1-shark/gst1-shark.mk | 3 ++- package/linux-zigbee/linux-zigbee.mk | 3 ++- package/netcat-openbsd/netcat-openbsd.mk | 3 ++- package/pkg-generic.mk | 4 +++- package/rcw/rcw.mk | 3 ++- package/speexdsp/speexdsp.mk | 3 ++- package/trace-cmd/trace-cmd.mk | 3 ++- 16 files changed, 33 insertions(+), 16 deletions(-)
Signature · 1 line
2.21.0
PATCH 1/3 package/pkg-generic: allow to use <pkg>_DL_VERSION in packages .mk files
This patch continue the work done here http://buildroot-busybox.2317881.n4.nabble.com/PATCH-0-4-Sanetize-packages-version-tp225554.html
By making this DL_VERSION public we should be able to handle the missing corner-cases
with package version sanitizeing.
--- package/pkg-generic.mk | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
package/pkg-generic.mk +3 -1
@@ -447,7 +447,9 @@ $(2)_PKGDIR = $(pkgdir) # version control system branch or tag, for example remotes/origin/1_10_stable. # Similar for spaces and colons (:) that may appear in date-based revisions for # CVS. -ifndef $(2)_VERSION +ifdef $(2)_DL_VERSION + $(2)_DL_VERSION := $$(strip $$($(2)_DL_VERSION)) +else ifndef $(2)_VERSION ifdef $(3)_DL_VERSION $(2)_DL_VERSION := $$($(3)_DL_VERSION) else ifdef $(3)_VERSION
Signature · 1 line
2.21.0
PATCH 1/3 package/pkg-generic: allow to use <pkg>_DL_VERSION in packages .mk files
Hi Victor,
Quoted 1 line · Victor Huesca
This patch continue the work done here http://buildroot-busybox.2317881.n4.nabble.com/PATCH-0-4-Sanetize-packages-version-tp225554.html
It's generally nices to summarize the discussion and put the reference as a
footnote, e.g.:
The _VERSION values should correspond to the ones on release-monitoring.org, so we can easily check if there is an update for a package. For downloaded packages, we can just add any prefix/suffix as part of the _SOURCE and/or _SITE variables. However, for VCS-downloaded packages, the _VERSION is used directly by the download infra.
This patch makes the DL_VERSION public, i.e. packages can set it (to a different value than _VERSION). This way, we can handle the missing corner cases without changes all over the places. See [1] for the full discussion.
[1]
http://buildroot-busybox.2317881.n4.nabble.com/PATCH-0-4-Sanetize-packages-version-tp225554.html
However, I don't think the discussion said anything more than what I wrote
above, so I don't think adding the reference is very valuable. It's up to you.
Quoted 13 lines
By making this
DL_VERSIONpublic we should be able to handle the missing corner-cases with package version sanitizeing.Signed-off-by Victor Huesca <[email protected]>--- package/pkg-generic.mk | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)package/pkg-generic.mk +0
-0
package/pkg-generic.mk $(2)_PKGDIR = $(pkgdir) · line 449
Quoted 3 lines
package/pkg-generic.mk +0
-0@@ -447,7 +447,9 @@ $(2)_PKGDIR = $(pkgdir) # version control system branch or tag, for example remotes/origin/1_10_stable. # Similar for spaces and colons (:) that may appear in date-based revisions for # CVS.
Maybe it would be good to add a comment like this:
# Since _DL_VERSION and _VERSION are defined based on themselves, we need to use # := assignment. Also, since they may be calculated through functions, it's best
# to strip them - this makes it easier for the packages themselves.
package/pkg-generic.mk $(2)_PKGDIR = $(pkgdir) · line 451
Quoted 3 lines
package/pkg-generic.mk +2
-1-ifndef $(2)_VERSION +ifdef $(2)_DL_VERSION + $(2)_DL_VERSION := $$(strip $$($(2)_DL_VERSION))
This will not work as intended... A few lines below, we have:
$(2)_VERSION := $$(call sanitize,$$($(2)_DL_VERSION))
so the _VERSION defined by the package will not be used at all...
I'm surprised that you didn't notice this. Maybe I'm mistaken? Or is there some
situation where the correct _VERSION does end up getting used?
I think the only solution is something like:
ifdef $(2)_DL_VERSION
$(2)_DL_VERSION := $$(strip $$($(2)_DL_VERSION)) $(2)_VERSION := $$(call sanitize,$$($(2)_DL_VERSION))
else
ifndef $(2)_VERSION
ifdef $(3)_DL_VERSION
$(2)_DL_VERSION := $$($(3)_DL_VERSION)
else ifdef $(3)_VERSION
$(2)_DL_VERSION := $$($(3)_VERSION)
endif
else
$(2)_DL_VERSION := $$(strip $$($(2)_VERSION))
endif
endif
Regards,
Arnout
Quoted 5 lines
package/pkg-generic.mk +1
-0+else ifndef $(2)_VERSION ifdef $(3)_DL_VERSION $(2)_DL_VERSION := $$($(3)_DL_VERSION) else ifdef $(3)_VERSION
PATCH 1/3 package/pkg-generic: allow to use <pkg>_DL_VERSION in packages .mk files
Hi Arnout,
Quoted 18 lines · Arnout Vandecappelle
Hi Victor,
Quoted 1 line · Victor Huesca
This patch continue the work done here http://buildroot-busybox.2317881.n4.nabble.com/PATCH-0-4-Sanetize-packages-version-tp225554.html
It's generally nices to summarize the discussion and put the reference as a
footnote, e.g.:The
_VERSIONvalues should correspond to the ones on release-monitoring.org, so we can easily check if there is an update for a package. For downloaded packages, we can just add any prefix/suffix as part of the_SOURCEand/or_SITEvariables. However, for VCS-downloaded packages, the_VERSIONis used directly by the download infra.This patch makes the
DL_VERSIONpublic, i.e. packages can set it (to a different value than_VERSION). This way, we can handle the missing corner cases without changes all over the places. See [1] for the full discussion.
Quoted 7 lines
[1]
http://buildroot-busybox.2317881.n4.nabble.com/PATCH-0-4-Sanetize-packages-version-tp225554.htmlHowever, I don't think the discussion said anything more than what I wrote
above, so I don't think adding the reference is very valuable. It's up to you.
I see, thank you, I'll use this for the v2.
Quoted 30 lines
Quoted 13 lines
By making this
DL_VERSIONpublic we should be able to handle the missing corner-cases with package version sanitizeing.Signed-off-by Victor Huesca <[email protected]>--- package/pkg-generic.mk | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)package/pkg-generic.mk +0
-0package/pkg-generic.mk $(2)_PKGDIR = $(pkgdir) · line 449
Quoted 3 lines
package/pkg-generic.mk +0
-0@@ -447,7 +447,9 @@ $(2)_PKGDIR = $(pkgdir) # version control system branch or tag, for example remotes/origin/1_10_stable. # Similar for spaces and colons (:) that may appear in date-based revisions for # CVS.Maybe it would be good to add a comment like this:
# Since
_DL_VERSIONand_VERSIONare defined based on themselves, we need to use # := assignment. Also, since they may be calculated through functions, it's best# to strip them - this makes it easier for the packages themselves.package/pkg-generic.mk $(2)_PKGDIR = $(pkgdir) · line 451
Quoted 3 lines
package/pkg-generic.mk +2
-1-ifndef $(2)_VERSION +ifdef $(2)_DL_VERSION + $(2)_DL_VERSION := $$(strip $$($(2)_DL_VERSION))This will not work as intended... A few lines below, we have:
Quoted 6 lines
$(2)_VERSION := $$(call sanitize,$$($(2)_DL_VERSION))
so the
_VERSIONdefined by the package will not be used at all...I'm surprised that you didn't notice this. Maybe I'm mistaken? Or is there some
situation where the correct_VERSIONdoes end up getting used?
You're right, I was focused on making the download work and forgot that now that _DL_VERSION and _VERSION can differs we should keep _VERSION rather than _DL_VERSION.
Quoted 15 lines
I think the only solution is something like:
ifdef $(2)_DL_VERSION
$(2)_DL_VERSION := $$(strip $$($(2)_DL_VERSION)) $(2)_VERSION := $$(call sanitize,$$($(2)_DL_VERSION))> else ifndef $(2)_VERSION
ifdef $(3)_DL_VERSION
$(2)_DL_VERSION := $$($(3)_DL_VERSION)
else ifdef $(3)_VERSION
$(2)_DL_VERSION := $$($(3)_VERSION)
endif
else
$(2)_DL_VERSION := $$(strip $$($(2)_VERSION))
endif
endif
Maybe I am mistaken but to me this code produce exactly the same results as mine...
Anyway, now that you pointed out to me that we should keep the original _VERSION I can make a v2 that take this in account.
Signature · 3 lines
Victor Huesca, Bootlin Embedded Linux and kernel engineering https://bootlin.com
PATCH 1/3 package/pkg-generic: allow to use <pkg>_DL_VERSION in packages .mk files
In reply to Victor HuescaHello Victor,
Thanks for the work on cleaning up those version variables!
Quoted 5 lines · On Tue, 9 Jul 2019 09:45:49 +0200 Victor Huesca <[email protected]>…
This patch continue the work done here http://buildroot-busybox.2317881.n4.nabble.com/PATCH-0-4-Sanetize-packages-version-tp225554.html
By making thisDL_VERSIONpublic we should be able to handle the missing corner-cases
with package version sanitizeing.Signed-off-by Victor Huesca <[email protected]>
I just want to point out that this patch makes <pkg>_DL_VERSION a "public" variable, so it needs to be documented in the Buildroot manual. It should be done in a patch in the same patch series.
Also, nitpick: make sure to wrap your commit messages (and generally e-mails) at 72 characters.
Thanks!
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
PATCH 2/3 package: remove common prefix/suffix from git-fetched packages
In reply to Victor HuescaWhile many project name their git tags <version> where <version> is just the version number, some project add other prefix/suffix to their version number. Common cases are:
- Add 'v' like '<pkg>_VERSION = v0.3.1'
- Add the package name (often separeted with a '/', '-' or '.') like '<pkg>_VERSION = <pkg>-1.5.2' or '<pkg>_VERSION = <pkg>/1.5.2'.
- Add a generic suffix like '-release', '-dev' or 'debian'
- Any combinaison of the those.
In Buildroot, we currently encode the full version (with its prefix/suffix) in the <pkg>_VERSION variable. However, that doesn't work well to match version numbers with the release-monitoring.org website, which filters such prefixes/suffixes.
This commit rely on the '<pkg>_DL_VERSION' variable.
Other corner-case witch very specific prefix/suffix will be handle by a separate commit.
--- boot/lpc32xxcdl/lpc32xxcdl.mk | 3 ++- package/at/at.mk | 3 ++- package/dt-utils/dt-utils.mk | 3 ++- package/f2fs-tools/f2fs-tools.mk | 3 ++- package/fio/fio.mk | 3 ++- package/gstreamer1/gst1-shark/gst1-shark.mk | 3 ++- package/linux-zigbee/linux-zigbee.mk | 3 ++- package/netcat-openbsd/netcat-openbsd.mk | 3 ++- package/speexdsp/speexdsp.mk | 3 ++- package/trace-cmd/trace-cmd.mk | 3 ++- 10 files changed, 20 insertions(+), 10 deletions(-)
boot/lpc32xxcdl/lpc32xxcdl.mk +2 -1
@@ -4,7 +4,8 @@ # ################################################################################ -LPC32XXCDL_VERSION = lpc32xx_cdl_v2.11 +LPC32XXCDL_VERSION = 2.11 +LPC32XXCDL_DL_VERSION = lpc32xx_cdl_v$(LPC32XXCDL_VERSION) LPC32XXCDL_SITE = http://git.lpcware.com/lpc3xxx_cdl.git LPC32XXCDL_SITE_METHOD = git
package/at/at.mk +2 -1
@@ -4,7 +4,8 @@ # ################################################################################ -AT_VERSION = release/3.1.23 +AT_VERSION = 3.1.23 +AT_DL_VERSION = release/$(AT_VERSION) AT_SITE = https://salsa.debian.org/debian/at.git AT_SITE_METHOD = git # Tried to add missing deps for parsetime.l but still parallel build fails
package/dt-utils/dt-utils.mk +2 -1
@@ -4,7 +4,8 @@ # ################################################################################ -DT_UTILS_VERSION = v2019.01.0 +DT_UTILS_VERSION = 2019.01.0 +DT_UTILS_DL_VERSION = v$(DT_UTILS_VERSION) DT_UTILS_SITE = https://git.pengutronix.de/git/tools/dt-utils DT_UTILS_SITE_METHOD = git DT_UTILS_LICENSE = GPL-2.0
package/f2fs-tools/f2fs-tools.mk +2 -1
@@ -4,7 +4,8 @@ # ################################################################################ -F2FS_TOOLS_VERSION = v1.11.0 +F2FS_TOOLS_VERSION = 1.11.0 +F2FS_TOOLS_DL_VERSION = v$(F2FS_TOOLS_VERSION) F2FS_TOOLS_SITE = http://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs-tools.git F2FS_TOOLS_SITE_METHOD = git F2FS_TOOLS_CONF_ENV = ac_cv_file__git=no
package/fio/fio.mk +2 -1
@@ -4,7 +4,8 @@ # ################################################################################ -FIO_VERSION = fio-3.9 +FIO_VERSION = 3.9 +FIO_DL_VERSION = fio-$(FIO_VERSION) FIO_SITE = git://git.kernel.dk/fio.git FIO_LICENSE = GPL-2.0 FIO_LICENSE_FILES = COPYING MORAL-LICENSE
package/gstreamer1/gst1-shark/gst1-shark.mk +2 -1
@@ -4,7 +4,8 @@ # ################################################################################ -GST1_SHARK_VERSION = v0.6.0 +GST1_SHARK_VERSION = 0.6.0 +GST1_SHARK_DL_VERSION = v$(GST1_SHARK_VERSION) GST1_SHARK_SITE = https://github.com/RidgeRun/gst-shark.git GST1_SHARK_SITE_METHOD = git GST1_SHARK_GIT_SUBMODULES = YES
package/linux-zigbee/linux-zigbee.mk +2 -1
@@ -4,7 +4,8 @@ # ################################################################################ -LINUX_ZIGBEE_VERSION = v0.3.1 +LINUX_ZIGBEE_VERSION = 0.3.1 +LINUX_ZIGBEE_DL_VERSION = v$(LINUX_ZIGBEE_VERSION) LINUX_ZIGBEE_SITE = git://linux-zigbee.git.sourceforge.net/gitroot/linux-zigbee/linux-zigbee LINUX_ZIGBEE_LICENSE = GPL-2.0 LINUX_ZIGBEE_LICENSE_FILES = COPYING
package/netcat-openbsd/netcat-openbsd.mk +2 -1
@@ -4,7 +4,8 @@ # ################################################################################ -NETCAT_OPENBSD_VERSION = debian/1.195-2 +NETCAT_OPENBSD_VERSION = 1.195-2 +NETCAT_OPENBSD_DL_VERSION = debian/$(NETCAT_OPENBSD_VERSION) NETCAT_OPENBSD_SITE = https://salsa.debian.org/debian/netcat-openbsd.git NETCAT_OPENBSD_SITE_METHOD = git NETCAT_OPENBSD_LICENSE = BSD-3-Clause
package/speexdsp/speexdsp.mk +2 -1
@@ -4,7 +4,8 @@ # ################################################################################ -SPEEXDSP_VERSION = SpeexDSP-1.2.0 +SPEEXDSP_VERSION = 1.2.0 +SPEEXDSP_DL_VERSION = SpeexDSP-$(SPEEXDSP_VERSION) SPEEXDSP_SITE = https://git.xiph.org/speexdsp.git SPEEXDSP_SITE_METHOD = git SPEEXDSP_LICENSE = BSD-3-Clause
package/trace-cmd/trace-cmd.mk +2 -1
@@ -4,7 +4,8 @@ # ################################################################################ -TRACE_CMD_VERSION = trace-cmd-v2.7 +TRACE_CMD_VERSION = 2.7 +TRACE_CMD_DL_VERSION = trace-cmd-v$(TRACE_CMD_VERSION) TRACE_CMD_SITE = https://git.kernel.org/pub/scm/linux/kernel/git/rostedt/trace-cmd.git TRACE_CMD_SITE_METHOD = git TRACE_CMD_INSTALL_STAGING = YES
Signature · 1 line
2.21.0
PATCH 3/3 package: remove non-conventional prefix/suffix from git-fetched packages
In reply to Victor HuescaWhile many project name their git tags <version> where <version> is just the version number, some project add other prefix/suffix to their version number. In these laters, the suffix/suffix is often simply 'v', 'release' or the package name, but there are some weirder naming. This patch aims to fix these special cases.
In Buildroot, we currently encode the full version (with its prefix/suffix) in the <pkg>_VERSION variable. However, that doesn't work well to match version numbers with the release-monitoring.org website, which filters such prefixes/suffixes.
This commit rely on the '<pkg>_DL_VERSION' variable, so dl archive name are unchanged.
--- package/brickd/brickd.mk | 3 ++- package/ctorrent/ctorrent.mk | 3 ++- package/fmc/fmc.mk | 3 ++- package/fmlib/fmlib.mk | 3 ++- package/rcw/rcw.mk | 3 ++- 5 files changed, 10 insertions(+), 5 deletions(-)
package/brickd/brickd.mk +2 -1
@@ -4,7 +4,8 @@ # ################################################################################ -BRICKD_VERSION = ev3dev-stretch/1.2.1 +BRICKD_VERSION = 1.2.1 +BRICKD_DL_VERSION = ev3dev-stretch/$(BRICKD_VERSION) BRICKD_SITE = https://github.com/ev3dev/brickd BRICKD_SITE_METHOD = git BRICKD_GIT_SUBMODULES = YES
package/ctorrent/ctorrent.mk +2 -1
@@ -4,7 +4,8 @@ # ################################################################################ -CTORRENT_VERSION = dnh3.3.2 +CTORRENT_VERSION = 3.3.2 +CTORRENT_DL_VERSION = dnh$(CTORRENT_VERSION) CTORRENT_SITE = http://www.rahul.net/dholmes/ctorrent CTORRENT_LICENSE = GPL-2.0 CTORRENT_LICENSE_FILES = COPYING
package/fmc/fmc.mk +2 -1
@@ -4,7 +4,8 @@ # ################################################################################ -FMC_VERSION = fsl-sdk-v2.0 +FMC_VERSION = 2.0 +FMC_DL_VERSION = fsl-sdk-v$(FMC_VERSION) FMC_SITE = git://git.freescale.com/ppc/sdk/fmc.git FMC_LICENSE = MIT FMC_LICENSE_FILES = COPYING
package/fmlib/fmlib.mk +2 -1
@@ -4,7 +4,8 @@ # ################################################################################ -FMLIB_VERSION = fsl-sdk-v2.0 +FMLIB_VERSION = 2.0 +FMLIB_DL_VERSION = fsl-sdk-v$(FMLIB_VERSION) FMLIB_SITE = git://git.freescale.com/ppc/sdk/fmlib.git FMLIB_LICENSE = BSD-3-Clause, GPL-2.0+ FMLIB_LICENSE_FILES = COPYING
package/rcw/rcw.mk +2 -1
@@ -4,7 +4,8 @@ # ################################################################################ -RCW_VERSION = LSDK-18.12 +RCW_VERSION = 18.12 +RCW_DL_VERSION = LSDK-$(RCW_VERSION) RCW_SITE = https://source.codeaurora.org/external/qoriq/qoriq-components/rcw RCW_SITE_METHOD = git RCW_LICENSE = BSD-3-Clause
Signature · 1 line
2.21.0