Skip to contentSkip to conversation

Cleanup trailing slashed from FOO_SITE

PATCH Superseded Luca Ceresoli 4 participants 14 messages Sep 28, 2015 – Oct 3, 2015

v3 of 4 Superseded Latest is v4 →

lore
Share & tools
lore

Luca Ceresoli

author
#0 lore raw

v3 cover Cleanup trailing slashed from FOO_SITE

Hi,

after several months since v1 [0], here's the respin of my patch set to clean up the form of FOO_SITE variables by removing all trailing slashes.

Those slashes are useless and potentially armful, which led to introducing a workaround to strip them:

commit 1cbffbd015106ea90fe49e27433375769dc1035b
Author: Shawn J. Goff <[email protected]>
Date: Fri Apr 12 09:40:30 2013 +0000

eliminate double slashes caused by FOO_SITE ending in a slash

When a FOO_SITE variable ends in a slash and gets joined with a FOO_SOURCE variable like $(FOO_SITE)/$(FOO_SOURCE), the resulting URI has a double slash. While double-slashes are fine in unix paths, they are reserved in URIs - the part following '//' must be an authority.

Signed-off-by Shawn J. Goff <[email protected]>
Signed-off-by Peter Korsgaard <[email protected]>

v1 removed all these slashes but did not add a check to prevent some more ones from slipping in new pagkages, and of course they did. So I took Thomas' suggestion to add a check, and here it is.

In this patchset first I remove the slashes that are currently in Buildroot, then I add a check so it doesn't happen again unnoticed. Finally I remove the workaround that is not needed anymore and explicitly state in the manual that trailing slashes are incorrect.

This work has been inspired by this comment from Arnout Vandecappelle [1]:

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

Review

Quoted 3 lines
(quoted diff) +3 -0
+else
+ifneq ($$($(2)_ACTUAL_SOURCE_TARBALL),$$($(2)_SOURCE))
+          $(call DOWNLOAD,$$($(2)_ACTUAL_SOURCE_SITE:/=)/$$($(2)$($(PKG)_SITE:/=)_ACTUAL_SOURCE_TARBALL))

I think the $($(PKG)_SITE:/=) construct was just introduced because for some
packages, the _SITE ends with a / and that should be stripped, and we were too lazy to fix the packages. Hm, looks like all the the external toolchain _SITEs end with a /...

[0] http://lists.busybox.net/pipermail/buildroot/2015-March/121502.html
[1] https://patchwork.ozlabs.org/patch/424980/

Luca Ceresoli (4):
Remove trailing slash from all package site URLs
pkg-generic: prevent _SITE URLs with a trailing slash docs/manual: FOO_SITE must not have a trailing slash
download: get rid of trailing slash removal hack for FOO_SITE

 docs/manual/adding-packages-generic.txt          | 6 ++++--
 package/angularjs/angularjs.mk                   | 2 +-
 package/audit/audit.mk                           | 2 +-
 package/c-icap-modules/c-icap-modules.mk         | 2 +-
 package/c-icap/c-icap.mk                         | 2 +-
 package/doxygen/doxygen.mk                       | 2 +-
 package/drbd-utils/drbd-utils.mk                 | 2 +-
 package/ebtables/ebtables.mk                     | 2 +-
 package/ibrcommon/ibrcommon.mk                   | 2 +-
 package/ibrdtn-tools/ibrdtn-tools.mk             | 2 +-
 package/ibrdtn/ibrdtn.mk                         | 2 +-
 package/ibrdtnd/ibrdtnd.mk                       | 2 +-
 package/ipmiutil/ipmiutil.mk                     | 2 +-
 package/libcddb/libcddb.mk                       | 2 +-
 package/libftdi1/libftdi1.mk                     | 2 +-
 package/libpam-radius-auth/libpam-radius-auth.mk | 2 +-
 package/lirc-tools/lirc-tools.mk                 | 2 +-
 package/monkey/monkey.mk                         | 2 +-
 package/mono-gtksharp3/mono-gtksharp3.mk         | 2 +-
 package/mosh/mosh.mk                             | 2 +-
 package/openipmi/openipmi.mk                     | 2 +-
 package/pkg-generic.mk                           | 4 +++-
 package/powertop/powertop.mk                     | 2 +-
 package/python-can/python-can.mk                 | 2 +-
 package/python-pycli/python-pycli.mk             | 2 +-
 package/python-pyxml/python-pyxml.mk             | 2 +-
 package/python-pyyaml/python-pyyaml.mk           | 2 +-
 package/quazip/quazip.mk                         | 2 +-
 package/rapidxml/rapidxml.mk                     | 2 +-
 package/setools/setools.mk                       | 2 +-
 package/sp-oops-extract/sp-oops-extract.mk       | 2 +-
 package/stress-ng/stress-ng.mk                   | 2 +-
 32 files changed, 37 insertions(+), 33 deletions(-)
Signature · 1 line
1.9.1
author
#1 lore raw

PATCH v3 1/4 Remove trailing slash from all package site URLs

The recommended form is without the trailing slash, and will become mandatory in a coming commit.

This avoids the need for the $$($(2)_SITE:/=) magic in package/pkg-generic.mk to avoid double slashes in download URLs, like "https://mosh.mit.edu//mosh-1.2.5.tar.gz".

                     ^^

Note: this work has already been done in b0b9606530dfc6de4030 a few months ago and earlier in c7f4b964718bc5a3329b and 4a9eb20de817fa64, but no check has been added at that time to avoid new slashes to slip in, and so they did. This time a patch will follow immediately to prevent future mistakes from being unnoticed.

Mass-replaced with the following command:

git grep -l '_SITE.*/$' | xargs sed -i '/_SITE.*=/s|/$||'

Signed-off-by Luca Ceresoli <[email protected]>

---

Changes v2 -> v3:
- rebased on current master.

---
 package/angularjs/angularjs.mk                   | 2 +-
 package/audit/audit.mk                           | 2 +-
 package/c-icap-modules/c-icap-modules.mk         | 2 +-
 package/c-icap/c-icap.mk                         | 2 +-
 package/doxygen/doxygen.mk                       | 2 +-
 package/drbd-utils/drbd-utils.mk                 | 2 +-
 package/ebtables/ebtables.mk                     | 2 +-
 package/ibrcommon/ibrcommon.mk                   | 2 +-
 package/ibrdtn-tools/ibrdtn-tools.mk             | 2 +-
 package/ibrdtn/ibrdtn.mk                         | 2 +-
 package/ibrdtnd/ibrdtnd.mk                       | 2 +-
 package/ipmiutil/ipmiutil.mk                     | 2 +-
 package/libcddb/libcddb.mk                       | 2 +-
 package/libftdi1/libftdi1.mk                     | 2 +-
 package/libpam-radius-auth/libpam-radius-auth.mk | 2 +-
 package/lirc-tools/lirc-tools.mk                 | 2 +-
 package/monkey/monkey.mk                         | 2 +-
 package/mono-gtksharp3/mono-gtksharp3.mk         | 2 +-
 package/mosh/mosh.mk                             | 2 +-
 package/openipmi/openipmi.mk                     | 2 +-
 package/powertop/powertop.mk                     | 2 +-
 package/python-can/python-can.mk                 | 2 +-
 package/python-pycli/python-pycli.mk             | 2 +-
 package/python-pyxml/python-pyxml.mk             | 2 +-
 package/python-pyyaml/python-pyyaml.mk           | 2 +-
 package/quazip/quazip.mk                         | 2 +-
 package/rapidxml/rapidxml.mk                     | 2 +-
 package/setools/setools.mk                       | 2 +-
 package/sp-oops-extract/sp-oops-extract.mk       | 2 +-
 package/stress-ng/stress-ng.mk                   | 2 +-
 30 files changed, 30 insertions(+), 30 deletions(-)
package/angularjs/angularjs.mk +1 -1
index 1dbd7a7..60dbdb6 100644
@@ -6,7 +6,7 @@
 
 ANGULARJS_VERSION = 1.4.3
 ANGULARJS_SOURCE = angular-$(ANGULARJS_VERSION).zip
-ANGULARJS_SITE = https://code.angularjs.org/$(ANGULARJS_VERSION)/
+ANGULARJS_SITE = https://code.angularjs.org/$(ANGULARJS_VERSION)
 ANGULARJS_LICENSE = MIT
 # There's no separate license file in the archive, so use angular.js instead.
 ANGULARJS_LICENSE_FILES = angular.js
package/audit/audit.mk +1 -1
index 61e876b..6ff0986 100644
@@ -5,7 +5,7 @@
 ################################################################################
 
 AUDIT_VERSION = 2.4.3
-AUDIT_SITE = http://people.redhat.com/sgrubb/audit/
+AUDIT_SITE = http://people.redhat.com/sgrubb/audit
 AUDIT_LICENSE = GPLv2
 AUDIT_LICENSE_FILES = COPYING
package/c-icap-modules/c-icap-modules.mk +1 -1
index aeab580..5252a81 100644
@@ -6,7 +6,7 @@
 
 C_ICAP_MODULES_VERSION = 0.3.2
 C_ICAP_MODULES_SOURCE = c_icap_modules-$(C_ICAP_MODULES_VERSION).tar.gz
-C_ICAP_MODULES_SITE = http://downloads.sourceforge.net/c-icap/
+C_ICAP_MODULES_SITE = http://downloads.sourceforge.net/c-icap
 C_ICAP_MODULES_LICENSE = GPLv2+
 C_ICAP_MODULES_LICENSE_FILES = COPYING
 C_ICAP_MODULES_DEPENDENCIES = c-icap
package/c-icap/c-icap.mk +1 -1
index 0e27cc4..b4defb8 100644
@@ -6,7 +6,7 @@
 
 C_ICAP_VERSION = 0.3.5
 C_ICAP_SOURCE = c_icap-$(C_ICAP_VERSION).tar.gz
-C_ICAP_SITE = http://downloads.sourceforge.net/c-icap/
+C_ICAP_SITE = http://downloads.sourceforge.net/c-icap
 C_ICAP_LICENSE = LGPLv2.1+
 C_ICAP_LICENSE_FILES = COPYING
 C_ICAP_INSTALL_STAGING = YES
package/doxygen/doxygen.mk +1 -1
index 6f81a56..14dd090 100644
@@ -6,7 +6,7 @@
 
 DOXYGEN_VERSION = 1.8.9.1
 DOXYGEN_SOURCE = doxygen-$(DOXYGEN_VERSION).src.tar.gz
-DOXYGEN_SITE = http://ftp.stack.nl/pub/users/dimitri/
+DOXYGEN_SITE = http://ftp.stack.nl/pub/users/dimitri
 DOXYGEN_LICENSE = GPLv2
 DOXYGEN_LICENSE_FILES = LICENSE
 DOXYGEN_DEPENDENCIES = host-flex host-bison
package/drbd-utils/drbd-utils.mk +1 -1
index 75bc854..88d2729 100644
@@ -5,7 +5,7 @@
 ################################################################################
 
 DRBD_UTILS_VERSION = 8.9.4
-DRBD_UTILS_SITE = http://oss.linbit.com/drbd/
+DRBD_UTILS_SITE = http://oss.linbit.com/drbd
 DRBD_UTILS_LICENSE = GPLv2+
 DRBD_UTILS_LICENSE_FILES = COPYING
 DRBD_UTILS_DEPENDENCIES = host-flex
package/ebtables/ebtables.mk +1 -1
index e6cdd26..6395930 100644
@@ -6,7 +6,7 @@
 
 EBTABLES_VERSION = 2.0.10-4
 EBTABLES_SOURCE = ebtables-v$(EBTABLES_VERSION).tar.gz
-EBTABLES_SITE = http://ftp.netfilter.org/pub/ebtables/
+EBTABLES_SITE = http://ftp.netfilter.org/pub/ebtables
 EBTABLES_LICENSE = GPLv2+
 EBTABLES_LICENSE_FILES = COPYING
 EBTABLES_STATIC = $(if $(BR2_STATIC_LIBS),static)
package/ibrcommon/ibrcommon.mk +1 -1
index 97eced3..5fa2cb5 100644
@@ -6,7 +6,7 @@
 
 IBRCOMMON_VERSION = 1.0.1
 IBRCOMMON_SOURCE = ibrcommon-$(IBRCOMMON_VERSION).tar.gz
-IBRCOMMON_SITE = https://www.ibr.cs.tu-bs.de/projects/ibr-dtn/releases/
+IBRCOMMON_SITE = https://www.ibr.cs.tu-bs.de/projects/ibr-dtn/releases
 IBRCOMMON_INSTALL_STAGING = YES
 IBRCOMMON_LICENSE = Apache-2.0
 IBRCOMMON_LICENSE_FILES = COPYING README
package/ibrdtn-tools/ibrdtn-tools.mk +1 -1
index c9a585d..34f05a6 100644
@@ -5,7 +5,7 @@
 ################################################################################
 
 IBRDTN_TOOLS_VERSION = 1.0.1
-IBRDTN_TOOLS_SITE = https://www.ibr.cs.tu-bs.de/projects/ibr-dtn/releases/
+IBRDTN_TOOLS_SITE = https://www.ibr.cs.tu-bs.de/projects/ibr-dtn/releases
 IBRDTN_TOOLS_LICENSE = Apache-2.0
 IBRDTN_TOOLS_LICENSE_FILES = COPYING
 IBRDTN_TOOLS_DEPENDENCIES = ibrcommon ibrdtn host-pkgconf
package/ibrdtn/ibrdtn.mk +1 -1
index f899843..7bd9b9b 100644
@@ -6,7 +6,7 @@
 
 IBRDTN_VERSION = 1.0.1
 IBRDTN_SOURCE = ibrdtn-$(IBRDTN_VERSION).tar.gz
-IBRDTN_SITE = https://www.ibr.cs.tu-bs.de/projects/ibr-dtn/releases/
+IBRDTN_SITE = https://www.ibr.cs.tu-bs.de/projects/ibr-dtn/releases
 IBRDTN_INSTALL_STAGING = YES
 IBRDTN_LICENSE = Apache-2.0
 IBRDTN_LICENSE_FILES = COPYING
package/ibrdtnd/ibrdtnd.mk +1 -1
index 8a37fa3..e9fa43d 100644
@@ -6,7 +6,7 @@
 
 IBRDTND_VERSION = 1.0.1
 IBRDTND_SOURCE = ibrdtnd-$(IBRDTND_VERSION).tar.gz
-IBRDTND_SITE = https://www.ibr.cs.tu-bs.de/projects/ibr-dtn/releases/
+IBRDTND_SITE = https://www.ibr.cs.tu-bs.de/projects/ibr-dtn/releases
 IBRDTND_LICENSE = Apache-2.0
 IBRDTND_LICENSE_FILES = COPYING
 IBRDTND_DEPENDENCIES = ibrdtn ibrcommon
package/ipmiutil/ipmiutil.mk +1 -1
index 6f462b9..4861c81 100644
@@ -5,7 +5,7 @@
 ################################################################################
 
 IPMIUTIL_VERSION = 2.9.5
-IPMIUTIL_SITE = http://sourceforge.net/projects/ipmiutil/files/
+IPMIUTIL_SITE = http://sourceforge.net/projects/ipmiutil/files
 IPMIUTIL_LICENSE = BSD-3c
 IPMIUTIL_LICENSE_FILES = COPYING
 # We're patching configure.ac , lib/Makefile.am and lib/lanplus/Makefile.am
package/libcddb/libcddb.mk +1 -1
index 7ddb1e7..96b265a 100644
@@ -6,7 +6,7 @@
 
 LIBCDDB_VERSION = 1.3.2
 LIBCDDB_SOURCE = libcddb-$(LIBCDDB_VERSION).tar.bz2
-LIBCDDB_SITE = http://downloads.sourceforge.net/libcddb/
+LIBCDDB_SITE = http://downloads.sourceforge.net/libcddb
 LIBCDDB_LICENSE = LGPLv2+
 LIBCDDB_LICENSE_FILES = COPYING
 LIBCDDB_INSTALL_STAGING = YES
package/libftdi1/libftdi1.mk +1 -1
index 201feb7..e77f37b 100644
@@ -6,7 +6,7 @@
 
 LIBFTDI1_VERSION = 1.2
 LIBFTDI1_SOURCE = libftdi1-$(LIBFTDI1_VERSION).tar.bz2
-LIBFTDI1_SITE = http://www.intra2net.com/en/developer/libftdi/download/
+LIBFTDI1_SITE = http://www.intra2net.com/en/developer/libftdi/download
 LIBFTDI1_INSTALL_STAGING = YES
 LIBFTDI1_DEPENDENCIES = libusb
 LIBFTDI1_LICENSE = LGPLv2 (libftdi1), GPLv2 with exception (ftdipp1)
package/libpam-radius-auth/libpam-radius-auth.mk +1 -1
index 42acffd..5ed5953 100644
@@ -5,7 +5,7 @@
 ################################################################################
 
 LIBPAM_RADIUS_AUTH_VERSION = 1.4.0
-LIBPAM_RADIUS_AUTH_SITE = ftp://ftp.freeradius.org/pub/radius/
+LIBPAM_RADIUS_AUTH_SITE = ftp://ftp.freeradius.org/pub/radius
 LIBPAM_RADIUS_AUTH_SOURCE = pam_radius-$(LIBPAM_RADIUS_AUTH_VERSION).tar.gz
 LIBPAM_RADIUS_AUTH_DEPENDENCIES = linux-pam
 LIBPAM_RADIUS_AUTH_INSTALL_STAGING = YES
package/lirc-tools/lirc-tools.mk +1 -1
index 6837042..2052b72 100644
@@ -6,7 +6,7 @@
 
 LIRC_TOOLS_VERSION = 0.9.2
 LIRC_TOOLS_SOURCE = lirc-$(LIRC_TOOLS_VERSION).tar.bz2
-LIRC_TOOLS_SITE = http://downloads.sourceforge.net/project/lirc/LIRC/$(LIRC_TOOLS_VERSION)/
+LIRC_TOOLS_SITE = http://downloads.sourceforge.net/project/lirc/LIRC/$(LIRC_TOOLS_VERSION)
 LIRC_TOOLS_LICENSE = GPLv2+
 LIRC_TOOLS_LICENSE_FILES = COPYING
 LIRC_TOOLS_DEPENDENCIES = host-pkgconf
package/monkey/monkey.mk +1 -1
index 73fbd62..67c8f35 100644
@@ -6,7 +6,7 @@
 
 MONKEY_VERSION_MAJOR = 1.5
 MONKEY_VERSION = $(MONKEY_VERSION_MAJOR).6
-MONKEY_SITE = http://monkey-project.com/releases/$(MONKEY_VERSION_MAJOR)/
+MONKEY_SITE = http://monkey-project.com/releases/$(MONKEY_VERSION_MAJOR)
 MONKEY_LICENSE = Apache-2.0
 MONKEY_LICENSE_FILES = LICENSE
package/mono-gtksharp3/mono-gtksharp3.mk +1 -1
index fa63d66..cccfcde 100644
@@ -6,7 +6,7 @@
 
 MONO_GTKSHARP3_VERSION_MAJOR = 2.99
 MONO_GTKSHARP3_VERSION = $(MONO_GTKSHARP3_VERSION_MAJOR).3
-MONO_GTKSHARP3_SITE = http://ftp.gnome.org/pub/gnome/sources/gtk-sharp/$(MONO_GTKSHARP3_VERSION_MAJOR)/
+MONO_GTKSHARP3_SITE = http://ftp.gnome.org/pub/gnome/sources/gtk-sharp/$(MONO_GTKSHARP3_VERSION_MAJOR)
 MONO_GTKSHARP3_SOURCE = gtk-sharp-$(MONO_GTKSHARP3_VERSION).tar.xz
 MONO_GTKSHARP3_LICENSE = LGPLv2, MIT (cairo)
 MONO_GTKSHARP3_LICENSE_FILES = COPYING
package/mosh/mosh.mk +1 -1
index 02954f9..f8c45e8 100644
@@ -5,7 +5,7 @@
 ################################################################################
 
 MOSH_VERSION = 1.2.5
-MOSH_SITE = https://mosh.mit.edu/
+MOSH_SITE = https://mosh.mit.edu
 MOSH_DEPENDENCIES = zlib ncurses protobuf openssl host-pkgconf
 MOSH_LICENSE = GPLv3+ with exception
 MOSH_LICENSE_FILES = COPYING COPYING.iOS
package/openipmi/openipmi.mk +1 -1
index 4a813ad..85bd6b4 100644
@@ -5,7 +5,7 @@
 ################################################################################
 
 OPENIPMI_VERSION = 2.0.21
-OPENIPMI_SITE = http://sourceforge.net/projects/openipmi/files/OpenIPMI%202.0%20Library/
+OPENIPMI_SITE = http://sourceforge.net/projects/openipmi/files/OpenIPMI%202.0%20Library
 OPENIPMI_SOURCE = OpenIPMI-2.0.21.tar.gz
 OPENIPMI_LICENSE = LGPLv2+, GPLv2+, BSD-3c
 OPENIPMI_LICENSE_FILES = COPYING.LIB COPYING COPYING.BSD
package/powertop/powertop.mk +1 -1
index 87da5a4..53be483 100644
@@ -5,7 +5,7 @@
 ################################################################################
 
 POWERTOP_VERSION = 2.7
-POWERTOP_SITE = https://01.org/sites/default/files/downloads/powertop/
+POWERTOP_SITE = https://01.org/sites/default/files/downloads/powertop
 POWERTOP_DEPENDENCIES = pciutils ncurses libnl host-gettext host-pkgconf
 POWERTOP_LICENSE = GPLv2
 POWERTOP_LICENSE_FILES = COPYING
package/python-can/python-can.mk +1 -1
index 53d5e0f..c7b06a5 100644
@@ -5,7 +5,7 @@
 ################################################################################
 
 PYTHON_CAN_VERSION = ae5b6cf
-PYTHON_CAN_SITE = https://bitbucket.org/hardbyte/python-can/get/
+PYTHON_CAN_SITE = https://bitbucket.org/hardbyte/python-can/get
 PYTHON_CAN_SOURCE = $(PYTHON_CAN_VERSION).tar.bz2
 PYTHON_CAN_LICENSE = LGPLv3
 PYTHON_CAN_LICENSE_FILES = LICENSE.txt
package/python-pycli/python-pycli.mk +1 -1
index 2d83acd..8e48a06 100644
@@ -6,7 +6,7 @@
 
 PYTHON_PYCLI_VERSION = devel
 PYTHON_PYCLI_SOURCE = pyCLI-$(PYTHON_PYCLI_VERSION).tar.gz
-PYTHON_PYCLI_SITE = https://pypi.python.org/packages/source/p/pyCLI/
+PYTHON_PYCLI_SITE = https://pypi.python.org/packages/source/p/pyCLI
 PYTHON_PYCLI_LICENSE = ISC-like
 PYTHON_PYCLI_LICENSE_FILES = lib/cli/__init__.py
 PYTHON_PYCLI_SETUP_TYPE = setuptools
package/python-pyxml/python-pyxml.mk +1 -1
index 40435f4..7ccce8e 100644
@@ -6,7 +6,7 @@
 
 PYTHON_PYXML_VERSION = 0.8.4
 PYTHON_PYXML_SOURCE = PyXML-$(PYTHON_PYXML_VERSION).tar.gz
-PYTHON_PYXML_SITE = http://downloads.sourceforge.net/project/pyxml/pyxml/$(PYTHON_PYXML_VERSION)/
+PYTHON_PYXML_SITE = http://downloads.sourceforge.net/project/pyxml/pyxml/$(PYTHON_PYXML_VERSION)
 PYTHON_PYXML_LICENSE = BSD-3c
 PYTHON_PYXML_LICENSE_FILES = LICENCE
 PYTHON_PYXML_SETUP_TYPE = distutils
package/python-pyyaml/python-pyyaml.mk +1 -1
index ef10dac..f3d7283 100644
@@ -6,7 +6,7 @@
 
 PYTHON_PYYAML_VERSION = 3.11
 PYTHON_PYYAML_SOURCE = PyYAML-$(PYTHON_PYYAML_VERSION).tar.gz
-PYTHON_PYYAML_SITE = https://pypi.python.org/packages/source/P/PyYAML/
+PYTHON_PYYAML_SITE = https://pypi.python.org/packages/source/P/PyYAML
 PYTHON_PYYAML_SETUP_TYPE = distutils
 PYTHON_PYYAML_LICENSE = Python software foundation license v2
 PYTHON_PYYAML_LICENSE_FILES = LICENSE
package/quazip/quazip.mk +1 -1
index 178a531..adccf90 100644
@@ -5,7 +5,7 @@
 ################################################################################
 
 QUAZIP_VERSION = 0.7.1
-QUAZIP_SITE = http://sourceforge.net/projects/quazip/files/quazip/$(QUAZIP_VERSION)/
+QUAZIP_SITE = http://sourceforge.net/projects/quazip/files/quazip/$(QUAZIP_VERSION)
 QUAZIP_INSTALL_STAGING = YES
 QUAZIP_DEPENDENCIES = \
 	zlib \
package/rapidxml/rapidxml.mk +1 -1
index 4356d8d..a76a21b 100644
@@ -6,7 +6,7 @@
 
 RAPIDXML_VERSION = 1.13
 RAPIDXML_SOURCE = rapidxml-$(RAPIDXML_VERSION).zip
-RAPIDXML_SITE = http://downloads.sourceforge.net/project/rapidxml/rapidxml/rapidxml%20$(RAPIDXML_VERSION)/
+RAPIDXML_SITE = http://downloads.sourceforge.net/project/rapidxml/rapidxml/rapidxml%20$(RAPIDXML_VERSION)
 RAPIDXML_LICENSE = Boost Software License 1.0 or MIT
 RAPIDXML_LICENSE_FILES = license.txt
package/setools/setools.mk +1 -1
index 71301c7..c993171 100644
@@ -6,7 +6,7 @@
 
 SETOOLS_VERSION = 3.3.8
 SETOOLS_SOURCE = setools-$(SETOOLS_VERSION).tar.bz2
-SETOOLS_SITE = https://raw.githubusercontent.com/wiki/TresysTechnology/setools3/files/dists/setools-$(SETOOLS_VERSION)/
+SETOOLS_SITE = https://raw.githubusercontent.com/wiki/TresysTechnology/setools3/files/dists/setools-$(SETOOLS_VERSION)
 SETOOLS_DEPENDENCIES = libselinux libsepol sqlite libxml2 bzip2 host-bison
 SETOOLS_INSTALL_STAGING = YES
 SETOOLS_LICENSE = GPLv2+ LGPLv2.1+
package/sp-oops-extract/sp-oops-extract.mk +1 -1
index 5882431..9919807 100644
@@ -5,7 +5,7 @@
 ################################################################################
 
 SP_OOPS_EXTRACT_VERSION = 0.0.7-1
-SP_OOPS_EXTRACT_SITE = http://repository.maemo.org/pool/maemo5.0/free/s/sp-oops-extract/
+SP_OOPS_EXTRACT_SITE = http://repository.maemo.org/pool/maemo5.0/free/s/sp-oops-extract
 SP_OOPS_EXTRACT_SOURCE = sp-oops-extract_$(SP_OOPS_EXTRACT_VERSION).tar.gz
 SP_OOPS_EXTRACT_LICENSE = GPLv2
 SP_OOPS_EXTRACT_LICENSE_FILES = COPYING
package/stress-ng/stress-ng.mk +1 -1
index 7169c2b..114c427 100644
@@ -5,7 +5,7 @@
 ################################################################################
 
 STRESS_NG_VERSION = 0.04.16
-STRESS_NG_SITE = http://kernel.ubuntu.com/~cking/tarballs/stress-ng/
+STRESS_NG_SITE = http://kernel.ubuntu.com/~cking/tarballs/stress-ng
 STRESS_NG_LICENSE = GPLv2+
 STRESS_NG_LICENSE_FILES = COPYING
Signature · 1 line
1.9.1
Applied
#2 lore raw

PATCH v3 1/4 Remove trailing slash from all package site URLs

Dear Luca Ceresoli,

Quoted 14 lines · Luca Ceresoli

The recommended form is without the trailing slash, and will become mandatory in a coming commit.

This avoids the need for the $$($(2)_SITE:/=) magic in package/pkg-generic.mk to avoid double slashes in download URLs, like "https://mosh.mit.edu//mosh-1.2.5.tar.gz".

                     ^^

Note: this work has already been done in b0b9606530dfc6de4030 a few months ago and earlier in c7f4b964718bc5a3329b and 4a9eb20de817fa64, but no check has been added at that time to avoid new slashes to slip in, and so they did. This time a patch will follow immediately to prevent future mistakes from being unnoticed.

Quoted 7 lines

Mass-replaced with the following command:

git grep -l '_SITE.*/$' | xargs sed -i '/_SITE.*=/s|/$||'

Signed-off-by Luca Ceresoli <[email protected]>

---

Applied, thanks.

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com
author
#3 lore raw

PATCH v3 2/4 pkg-generic: prevent _SITE URLs with a trailing slash

In reply to Luca Ceresoli

A trailing slash in FOO_SITE is useless, since Buildroot automatically adds a slash between FOO_SITE and the filename as appropriate.

Moreover they are potentially armful, which led to introducing a workaround to strip them:

commit 1cbffbd015106ea90fe49e27433375769dc1035b
Author: Shawn J. Goff <[email protected]>
Date: Fri Apr 12 09:40:30 2013 +0000

eliminate double slashes caused by FOO_SITE ending in a slash

When a FOO_SITE variable ends in a slash and gets joined with a FOO_SOURCE variable like $(FOO_SITE)/$(FOO_SOURCE), the resulting URI has a double slash. While double-slashes are fine in unix paths, they are reserved in URIs - the part following '//' must be an authority.

So let's ban trailing slashes entirely. They have all been removed in a recent commit, now add a check to error out loudly in case a new one is added.

Example commands to test this check:

  $ make busybox-dirclean busybox-source
  rm -Rf /home/murray/devel/buildroot/output/build/busybox-1.23.2
  busybox-1.23.2.tar.bz2: OK (md5: 7925683d7dd105aabe9b6b618d48cc73)
  busybox-1.23.2.tar.bz2: OK (sha1: 7f37193cb249f27630e0b2a2c6c9bbb7b1d24c16)
  $
  $ make BUSYBOX_SITE=http://www.busybox.net/downloads/ busybox-dirclean busybox-source
  rm -Rf /home/murray/devel/buildroot/output/build/busybox-1.23.2
  BUSYBOX_SITE (http://www.busybox.net/downloads/) cannot have a trailing slash
  make[1]: *** [/home/murray/devel/buildroot/output/build/busybox-1.23.2/.stamp_downloaded] Error 1
  make: *** [_all] Error 2
  $
Signed-off-by Luca Ceresoli <[email protected]>

---

Changed v2 -> v3:
- Simplify the check logic removing a useless "&& true".
- Expand commit log message. This is a pretty important change, so it should be well justified.

---
 package/pkg-generic.mk | 2 ++
 1 file changed, 2 insertions(+)
package/pkg-generic.mk +2 -0
index 5201fca..73353aa 100644
@@ -70,6 +70,8 @@ endif
 
 # Retrieve the archive
 $(BUILD_DIR)/%/.stamp_downloaded:
+	@(echo "$($(PKG)_SITE)" | grep -Eq "[^/]$$" || \
+		(echo "$(PKG)_SITE ($($(PKG)_SITE)) cannot have a trailing slash" && false))
 	$(foreach hook,$($(PKG)_PRE_DOWNLOAD_HOOKS),$(call $(hook))$(sep))
 # Only show the download message if it isn't already downloaded
 	$(Q)for p in $($(PKG)_ALL_DOWNLOADS); do \
Signature · 1 line
1.9.1
#4 lore raw

PATCH v3 2/4 pkg-generic: prevent _SITE URLs with a trailing slash

Hi Luca,

Quoted 4 lines · Luca Ceresoli

A trailing slash in FOO_SITE is useless, since Buildroot automatically adds a slash between FOO_SITE and the filename as appropriate.

Moreover they are potentially armful, which led to introducing a workaround

harmful?

Quoted 1 line

to strip them:

baruch

Signature · 3 lines
     http://baruch.siach.name/blog/                  ~. .~   Tk Open Systems
=}------------------------------------------------ooO--U--Ooo------------{=
   - baruch at tkos.co.il - tel: +972.2.679.5364, http://www.tkos.co.il -
author
#5 lore raw

PATCH v3 2/4 pkg-generic: prevent _SITE URLs with a trailing slash

Dear Baruch,

Quoted 9 lines · Baruch Siach

Hi Luca,

Quoted 4 lines · Luca Ceresoli

A trailing slash in FOO_SITE is useless, since Buildroot automatically adds a slash between FOO_SITE and the filename as appropriate.

Moreover they are potentially armful, which led to introducing a workaround

harmful?

Of course!
Thanks for spotting it.

Signature · 1 line
Luca
#6 lore raw

PATCH v3 2/4 pkg-generic: prevent _SITE URLs with a trailing slash

In reply to Luca Ceresoli

Dear Luca Ceresoli,

package/pkg-generic.mk endif · line 74

Quoted 4 lines · Luca Ceresoli

# Retrieve the archive
$(BUILD_DIR)/%/.stamp_downloaded:

package/pkg-generic.mk +2 -0
+	@(echo "$($(PKG)_SITE)" | grep -Eq "[^/]$$" || \
+		(echo "$(PKG)_SITE ($($(PKG)_SITE)) cannot have a trailing slash" && false))

You could use $(error ...) instead:

Review

Quoted 2 lines
(quoted diff) +2 -0
+	@(echo "$($(PKG)_SITE)" | grep -Eq "[^/]$$" || \
+		$(error "$(PKG)_SITE ($($(PKG)_SITE)) cannot have a trailing slash"))

I am wondering if there are indeed no places left where a final / could sneak in. For example, when you specify a custom tarball location for packages like U-Boot, the Linux kernel and so on.

For Linux and U-Boot, it is taken care of:

UBOOT_SITE = $(patsubst %/,%,$(dir $(UBOOT_TARBALL)))

LINUX_SITE = $(patsubst %/,%,$(dir $(LINUX_TARBALL)))

But for example, for the external toolchain site:

TOOLCHAIN_EXTERNAL_SITE = $(dir $(call qstrip,$(BR2_TOOLCHAIN_EXTERNAL_URL)))

I believe this means that the TOOLCHAIN_EXTERNAL_SITE variable will contain a final /. But that's indeed the only case I could find from a quick inspection.

Best regards,

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com
#7 lore raw

PATCH v3 2/4 pkg-generic: prevent _SITE URLs with a trailing slash

Quoted 11 lines · Thomas Petazzoni

Dear Luca Ceresoli,

package/pkg-generic.mk endif · line 74

Quoted 4 lines · Luca Ceresoli

# Retrieve the archive
$(BUILD_DIR)/%/.stamp_downloaded:

package/pkg-generic.mk +2 -0
+	@(echo "$($(PKG)_SITE)" | grep -Eq "[^/]$$" || \
+		(echo "$(PKG)_SITE ($($(PKG)_SITE)) cannot have a trailing slash" && false))

You could use $(error ...) instead:

Quoted 2 lines

Review

Quoted 2 lines
(quoted diff) +2 -0
+	@(echo "$($(PKG)_SITE)" | grep -Eq "[^/]$$" || \
+		$(error "$(PKG)_SITE ($($(PKG)_SITE)) cannot have a trailing slash"))

That's not going to work AFAIK. The whole thing would have to be converted to a
make condition.

BTW, Luca, you should indicate in your commit message why this has to be part
of .stamp_downloaded and not a static check in inner-generic-package.

Quoted 11 lines

I am wondering if there are indeed no places left where a final / could sneak in. For example, when you specify a custom tarball location for packages like U-Boot, the Linux kernel and so on.

For Linux and U-Boot, it is taken care of:

UBOOT_SITE = $(patsubst %/,%,$(dir $(UBOOT_TARBALL)))

LINUX_SITE = $(patsubst %/,%,$(dir $(LINUX_TARBALL)))

Quoted 7 lines

But for example, for the external toolchain site:

TOOLCHAIN_EXTERNAL_SITE = $(dir $(call qstrip,$(BR2_TOOLCHAIN_EXTERNAL_URL)))

I believe this means that the TOOLCHAIN_EXTERNAL_SITE variable will contain a final /. But that's indeed the only case I could find from a quick inspection.

Well, to be consistent, the removal should not be done for U-Boot or Linux
either, and instead the user should get this error message when he builds.

Regards,
Arnout

Quoted 5 lines

Best regards,

Thomas

Signature · 6 lines
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF
#8 lore raw

PATCH v3 2/4 pkg-generic: prevent _SITE URLs with a trailing slash

Quoted 12 lines · Arnout Vandecappelle
Quoted 11 lines · Thomas Petazzoni
Quoted 11 lines

Dear Luca Ceresoli,

Quoted 4 lines · Luca Ceresoli

package/pkg-generic.mk endif · line 74

Quoted 4 lines

# Retrieve the archive
$(BUILD_DIR)/%/.stamp_downloaded:

package/pkg-generic.mk +2 -0
+	@(echo "$($(PKG)_SITE)" | grep -Eq "[^/]$$" || \
+		(echo "$(PKG)_SITE ($($(PKG)_SITE)) cannot have a trailing slash" && false))

You could use $(error ...) instead:

Quoted 4 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

Review

Quoted 2 lines
(quoted diff) +2 -0
+	@(echo "$($(PKG)_SITE)" | grep -Eq "[^/]$$" || \
+		$(error "$(PKG)_SITE ($($(PKG)_SITE)) cannot have a trailing slash"))

That's not going to work AFAIK. The whole thing would have to be converted to a
make condition.

E.g.:

ifeq ($(patsubst %/,ERROR,$($(PKG)_SITE)),ERROR)
$(error ...)
endif

Or if that doesn't work:

ifeq ($(shell echo "$($(PKG)_SITE)" | grep -Eq "[^/]$$" || echo n),n) $(error ...)
endif

Regards,
Arnout

Signature · 6 lines
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF
author
#9 lore raw

PATCH v3 2/4 pkg-generic: prevent _SITE URLs with a trailing slash

In reply to Arnout Vandecappelle

Hi Arnout, Thomas,

Arnout Vandecappelle wrote:
[...]

Quoted 14 lines
Quoted 14 lines

I am wondering if there are indeed no places left where a final / could sneak in. For example, when you specify a custom tarball location for packages like U-Boot, the Linux kernel and so on.

For Linux and U-Boot, it is taken care of:

UBOOT_SITE = $(patsubst %/,%,$(dir $(UBOOT_TARBALL)))

LINUX_SITE = $(patsubst %/,%,$(dir $(LINUX_TARBALL)))

But for example, for the external toolchain site:

Quoted 8 lines
Quoted 5 lines

TOOLCHAIN_EXTERNAL_SITE = $(dir $(call qstrip,$(BR2_TOOLCHAIN_EXTERNAL_URL)))

I believe this means that the TOOLCHAIN_EXTERNAL_SITE variable will contain a final /. But that's indeed the only case I could find from a quick inspection.

Well, to be consistent, the removal should not be done for U-Boot or Linux
either, and instead the user should get this error message when he builds.

But FOO_SITE is generated internally when the user chooses to fetch from a custom URL. The user has no access to FOO_SITE in these cases, so Buildroot should strip them.

So I guess I'll add a patsubst to toolchain-external too, and hunt for other places where it might be needed.

Signature · 1 line
Luca
#10 lore raw

PATCH v3 2/4 pkg-generic: prevent _SITE URLs with a trailing slash

Quoted 24 lines · Luca Ceresoli

Hi Arnout, Thomas,

Arnout Vandecappelle wrote:
[...]

Quoted 20 lines
Quoted 19 lines

I am wondering if there are indeed no places left where a final / could sneak in. For example, when you specify a custom tarball location for packages like U-Boot, the Linux kernel and so on.

For Linux and U-Boot, it is taken care of:

UBOOT_SITE = $(patsubst %/,%,$(dir $(UBOOT_TARBALL)))

LINUX_SITE = $(patsubst %/,%,$(dir $(LINUX_TARBALL)))

But for example, for the external toolchain site:

TOOLCHAIN_EXTERNAL_SITE = $(dir $(call qstrip,$(BR2_TOOLCHAIN_EXTERNAL_URL)))

I believe this means that the TOOLCHAIN_EXTERNAL_SITE variable will contain a final /. But that's indeed the only case I could find from a quick inspection.

Quoted 6 lines
Quoted 2 lines

Well, to be consistent, the removal should not be done for U-Boot or Linux
either, and instead the user should get this error message when he builds.

But FOO_SITE is generated internally when the user chooses to fetch from a custom URL. The user has no access to FOO_SITE in these cases, so Buildroot should strip them.

You're completely right, sorry!

Quoted 2 lines

So I guess I'll add a patsubst to toolchain-external too, and hunt for other places where it might be needed.

I can't think of an easy grep for that :-(

Regards,
Arnout

Signature · 6 lines
Arnout Vandecappelle      arnout dot vandecappelle at essensium dot com
Senior Embedded Software Architect . . . . . . +32-478-010353 (mobile)
Essensium, Mind division . . . . . . . . . . . . . . http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium . . . . . BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF
author
#11 lore raw

PATCH v3 3/4 docs/manual: FOO_SITE must not have a trailing slash

In reply to Luca Ceresoli

This has been the best practice since a long time, but it was not clearly stated in the manual. Now it is mandatory, so mention it explicitly.

Signed-off-by Luca Ceresoli <[email protected]>

---

Changed v2 -> v3:
- Improve commit message.

---
 docs/manual/adding-packages-generic.txt | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
docs/manual/adding-packages-generic.txt +4 -2
index 4c12b42..40798c0 100644
@@ -227,7 +227,9 @@ information is (assuming the package name is +libfoo+) :
 
 * +LIBFOO_SITE+ provides the location of the package, which can be a
   URL or a local filesystem path. HTTP, FTP and SCP are supported URL
-  types for retrieving package tarballs. Git, Subversion, Mercurial,
+  types for retrieving package tarballs. In these cases don't include a
+  trailing slash: it will be added by Buildroot between the directory
+  and the filename as appropriate. Git, Subversion, Mercurial,
   and Bazaar are supported URL types for retrieving packages directly
   from source code management systems. There is a helper function to make
   it easier to download source tarballs from GitHub (refer to

@@ -246,7 +248,7 @@ information is (assuming the package name is +libfoo+) :
     +LIBFOO_SITE=http://www.libfoosoftware.org/libfoo+ +
     +LIBFOO_SITE=http://svn.xiph.org/trunk/Tremor+ +
     +LIBFOO_SITE=/opt/software/libfoo.tar.gz+ +
-    +LIBFOO_SITE=$(TOPDIR)/../src/libfoo/+
+    +LIBFOO_SITE=$(TOPDIR)/../src/libfoo+
 
 * +LIBFOO_EXTRA_DOWNLOADS+ is a space-separated list of additional
   files that Buildroot should download. If an entry contains +://+
Signature · 1 line
1.9.1
Applied
#12 lore raw

PATCH v3 3/4 docs/manual: FOO_SITE must not have a trailing slash

Dear Luca Ceresoli,

Quoted 7 lines · Luca Ceresoli

This has been the best practice since a long time, but it was not clearly stated in the manual. Now it is mandatory, so mention it explicitly.

Signed-off-by Luca Ceresoli <[email protected]>

---

Applied, thanks.

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com
author
#13 lore raw

PATCH v3 4/4 download: get rid of trailing slash removal hack for FOO_SITE

In reply to Luca Ceresoli

Not needed anymore since the URLs have been cleared, the manual states they are illegal and a check has been introduced to notice future mistakes.

Signed-off-by Luca Ceresoli <[email protected]>

---

Changes v2 -> v3:
- move after the commit on docs/manual, update commit message.

Changes v1 -> v2:
- redo on current master.

---
 package/pkg-generic.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
package/pkg-generic.mk +1 -1
index 73353aa..8491a4f 100644
@@ -390,7 +390,7 @@ endif
 $(2)_ALL_DOWNLOADS = \
 	$$(foreach p,$$($(2)_SOURCE) $$($(2)_PATCH) $$($(2)_EXTRA_DOWNLOADS),\
 		$$(if $$(findstring ://,$$(p)),$$(p),\
-			$$($(2)_SITE:/=)/$$(p)))
+			$$($(2)_SITE)/$$(p)))
 
 ifndef $(2)_SITE
  ifdef $(3)_SITE
Signature · 1 line
1.9.1