fetch2/wget.py: use default wget user-agent in upstream version checks
PATCH 1/2 fetch2/wget.py: use default wget user-agent in upstream version checks
From: Alexander Kanavin <[email protected]>
This has been set to various mozilla versions from this point onwards without explanation or specific examples:
https://git.yoctoproject.org/poky/commit/?h=master-next&id=ba06cc8943d021f406c3d292ce415b77319ca5c6
However, sourceforge has recently banned the current 'fake mozilla' version we've been supplying, resulting in broken version checks:
alex@Zen2:/srv/storage/alex/yocto/build-test-sourceforge$ devtool check-upgrade-status gptfdisk ... --2024-10-12 11:53:23-- https://sourceforge.net/projects/gptfdisk/files/gptfdisk/ Resolving sourceforge.net (sourceforge.net)... 172.64.150.145, 104.18.37.111, 2606:4700:4400::6812:256f, ... Connecting to sourceforge.net (sourceforge.net)|172.64.150.145|:443... connected. HTTP request sent, awaiting response... 403 Forbidden 2024-10-12 11:53:24 ERROR 403: Forbidden.
gptfdisk 1.0.10 UNKNOWN_BROKEN Alexander Kanavin <[email protected]>
On the other hand, using wget's default user-agent works just fine:
--2024-10-12 12:16:06-- https://sourceforge.net/projects/gptfdisk/files/gptfdisk/
Resolving sourceforge.net (sourceforge.net)... 104.18.37.111, 172.64.150.145, 2606:4700:4400::6812:256f, ... Connecting to sourceforge.net (sourceforge.net)|104.18.37.111|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
--- bitbake/lib/bb/fetch2/wget.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
bitbake/lib/bb/fetch2/wget.py +1 -1
@@ -463,7 +463,7 @@ class Wget(FetchMethod): f = tempfile.NamedTemporaryFile() with tempfile.TemporaryDirectory(prefix="wget-index-") as workdir, tempfile.NamedTemporaryFile(dir=workdir, prefix="wget-listing-") as f: fetchcmd = self.basecmd - fetchcmd += " -O " + f.name + " --user-agent='" + self.user_agent + "' '" + uri + "'" + fetchcmd += " -O " + f.name + " '" + uri + "'" try: self._runwget(ud, d, fetchcmd, True, workdir=workdir) fetchresult = f.read()
Signature · 1 line
2.39.5
PATCH 2/2 meta: use explicit uri for all sourceforge upstream version checks
From: Alexander Kanavin <[email protected]>
Previously the check for some recipes relied on sourceforge redirecting from downloads.sourceforge.net (SRC_URI is set to that) into the actual project page.
Sourceforge does this for interactive browsers, but not for wget.
With the check no longer mimicking a browser, and being truthful about coming from wget we need to explicitly fetch
the project page in all cases, which is what this commit does. (many recipes already set this explicitly and don't need to be tweaked)
--- meta/classes-recipe/sourceforge-releases.bbclass | 2 ++ meta/recipes-bsp/acpid/acpid.inc | 5 +++-- meta/recipes-bsp/gnu-efi/gnu-efi_3.0.18.bb | 2 ++ meta/recipes-bsp/setserial/setserial_2.17.bb | 2 +- meta/recipes-devtools/cdrtools/cdrtools-native_3.01.bb | 2 +- meta/recipes-extended/cwautomacros/cwautomacros_20110201.bb | 5 ++++- meta/recipes-extended/hdparm/hdparm_9.65.bb | 1 + meta/recipes-extended/mingetty/mingetty_1.08.bb | 2 +- meta/recipes-extended/rpcbind/rpcbind_1.2.7.bb | 2 +- meta/recipes-multimedia/lame/lame_3.100.bb | 2 +- meta/recipes-support/libfm/libfm-extra_1.3.2.bb | 5 +++-- meta/recipes-support/libfm/libfm_1.3.2.bb | 5 +++-- meta/recipes-support/libpcre/libpcre_8.45.bb | 5 +++-- 13 files changed, 26 insertions(+), 14 deletions(-) create mode 100644 meta/classes-recipe/sourceforge-releases.bbclass
meta/classes-recipe/sourceforge-releases.bbclass new file +2 -0
@@ -0,0 +1,2 @@ +SOURCEFORGE_PROJECT ?= "${BPN}" +UPSTREAM_CHECK_URI = "https://sourceforge.net/projects/${SOURCEFORGE_PROJECT}/files/"
meta/recipes-bsp/acpid/acpid.inc +3 -2
@@ -10,7 +10,8 @@ BUGTRACKER = "http://sourceforge.net/p/acpid2/tickets/?source=navbar" SECTION = "base" LICENSE = "GPL-2.0-or-later" -SRC_URI = "${SOURCEFORGE_MIRROR}/acpid2/acpid-${PV}.tar.xz \ +SOURCEFORGE_PROJECT = "acpid2" +SRC_URI = "${SOURCEFORGE_MIRROR}/${SOURCEFORGE_PROJECT}/acpid-${PV}.tar.xz \ file://init \ file://acpid.service \ file://0001-Replace-stat64-with-stat.patch \
@@ -18,7 +19,7 @@ SRC_URI = "${SOURCEFORGE_MIRROR}/acpid2/acpid-${PV}.tar.xz \ CVE_PRODUCT = "acpid2" -inherit autotools update-rc.d systemd +inherit autotools update-rc.d systemd sourceforge-releases INITSCRIPT_NAME = "acpid" INITSCRIPT_PARAMS = "defaults"
meta/recipes-bsp/gnu-efi/gnu-efi_3.0.18.bb +2 -0
@@ -18,6 +18,8 @@ SRC_URI = "${SOURCEFORGE_MIRROR}/${BPN}/files/${BP}.tar.bz2 \ " SRC_URI[sha256sum] = "7f212c96ee66547eeefb531267b641e5473d7d8529f0bd8ccdefd33cf7413f5c" +inherit sourceforge-releases + COMPATIBLE_HOST = "(x86_64.*|i.86.*|aarch64.*|arm.*|riscv64.*)-linux" COMPATIBLE_HOST:armv4 = 'null'
meta/recipes-bsp/setserial/setserial_2.17.bb +1 -1
@@ -8,7 +8,7 @@ LIC_FILES_CHKSUM = "file://version.h;beginline=1;endline=6;md5=2e7c59cb9e57e356a DEPENDS += "groff-native" -inherit autotools-brokensep +inherit autotools-brokensep sourceforge-releases SRC_URI = "${SOURCEFORGE_MIRROR}/setserial/${BPN}-${PV}.tar.gz \ file://add_stdlib.patch \
meta/recipes-devtools/cdrtools/cdrtools-native_3.01.bb +1 -1
@@ -25,7 +25,7 @@ EXTRA_OEMAKE = "-e MAKEFLAGS= CPPOPTX='${CPPFLAGS}' COPTX='${CFLAGS}' C++OPTX='$ # Stop failures when 'cc' can't be found export ac_cv_prog_CC = "${CC}" -inherit native +inherit sourceforge-releases native # Use -std=gnu89 to build with gcc-14 (https://bugs.gentoo.org/903876) # this needs to be after native inherit (which sets CFLAGS to BUILD_CFLAGS)
meta/recipes-extended/cwautomacros/cwautomacros_20110201.bb +4 -1
@@ -5,10 +5,13 @@ DESCRIPTION = "A collection of autoconf macros, plus an autogen.sh script that c LICENSE = "GPL-2.0-only" LIC_FILES_CHKSUM = "file://LICENSE;md5=eb723b61539feef013de476e68b5c50a" -SRC_URI = "${SOURCEFORGE_MIRROR}/${BPN}.berlios/${BP}.tar.bz2" +SOURCEFORGE_PROJECT = "${BPN}.berlios" +SRC_URI = "${SOURCEFORGE_MIRROR}/${SOURCEFORGE_PROJECT}/${BP}.tar.bz2" SRC_URI[md5sum] = "074afcb50d0a8bff10786a2954b2b02d" SRC_URI[sha256sum] = "3115603b891f3a163c0bbb5fea2f3742113a183fa6745ee5e89e5f6d0e9f6121" +inherit sourceforge-releases + do_configure() { : }
meta/recipes-extended/hdparm/hdparm_9.65.bb +1 -0
@@ -26,6 +26,7 @@ SRC_URI = "${SOURCEFORGE_MIRROR}/hdparm/${BP}.tar.gz \ " SRC_URI[sha256sum] = "d14929f910d060932e717e9382425d47c2e7144235a53713d55a94f7de535a4b" +UPSTREAM_CHECK_URI = "https://sourceforge.net/projects/hdparm/files/hdparm/" EXTRA_OEMAKE = 'STRIP="echo" LDFLAGS="${LDFLAGS}"'
meta/recipes-extended/mingetty/mingetty_1.08.bb +1 -1
@@ -21,7 +21,7 @@ do_install(){ oe_runmake install DESTDIR=${D} } -inherit update-alternatives +inherit update-alternatives sourceforge-releases ALTERNATIVE:${PN} = "getty" ALTERNATIVE_LINK_NAME[getty] = "${base_sbindir}/getty"
meta/recipes-extended/rpcbind/rpcbind_1.2.7.bb +1 -1
@@ -20,7 +20,7 @@ SRC_URI = "${SOURCEFORGE_MIRROR}/rpcbind/rpcbind-${PV}.tar.bz2 \ " SRC_URI[sha256sum] = "f6edf8cdf562aedd5d53b8bf93962d61623292bfc4d47eedd3f427d84d06f37e" -inherit autotools update-rc.d systemd pkgconfig update-alternatives +inherit autotools update-rc.d systemd pkgconfig update-alternatives sourceforge-releases PACKAGECONFIG ??= "tcp-wrappers" PACKAGECONFIG[tcp-wrappers] = "--enable-libwrap,--disable-libwrap,tcp-wrappers"
meta/recipes-multimedia/lame/lame_3.100.bb +1 -1
@@ -17,7 +17,7 @@ SRC_URI = "${SOURCEFORGE_MIRROR}/lame/lame-${PV}.tar.gz \ SRC_URI[md5sum] = "83e260acbe4389b54fe08e0bdbf7cddb" SRC_URI[sha256sum] = "ddfe36cab873794038ae2c1210557ad34857a4b6bdc515785d1da9e175b1da1e" -inherit autotools pkgconfig +inherit autotools pkgconfig sourceforge-releases PACKAGES += "libmp3lame libmp3lame-dev" FILES:${PN} = "${bindir}/lame"
meta/recipes-support/libfm/libfm-extra_1.3.2.bb +3 -2
@@ -8,7 +8,8 @@ LIC_FILES_CHKSUM = "file://src/fm-extra.h;beginline=8;endline=21;md5=ef1f84da64b SECTION = "x11/libs" DEPENDS = "glib-2.0-native glib-2.0 intltool-native" -SRC_URI = "${SOURCEFORGE_MIRROR}/pcmanfm/libfm-${PV}.tar.xz \ +SOURCEFORGE_PROJECT = "pcmanfm" +SRC_URI = "${SOURCEFORGE_MIRROR}/${SOURCEFORGE_PROJECT}/libfm-${PV}.tar.xz \ " SRC_URI[sha256sum] = "a5042630304cf8e5d8cff9d565c6bd546f228b48c960153ed366a34e87cad1e5"
@@ -17,6 +18,6 @@ S = "${WORKDIR}/libfm-${PV}" EXTRA_OECONF = "--with-extra-only --with-gtk=no" -inherit autotools pkgconfig gtk-doc gettext +inherit autotools pkgconfig gtk-doc gettext sourceforge-releases do_configure[dirs] =+ "${S}/m4"
meta/recipes-support/libfm/libfm_1.3.2.bb +3 -2
@@ -13,14 +13,15 @@ LIC_FILES_CHKSUM = "file://COPYING;md5=4641e94ec96f98fabc56ff9cc48be14b \ SECTION = "x11/libs" DEPENDS = "glib-2.0 glib-2.0-native pango gtk+3 menu-cache intltool-native libexif libfm-extra" -SRC_URI = "${SOURCEFORGE_MIRROR}/pcmanfm/libfm-${PV}.tar.xz \ +SOURCEFORGE_PROJECT = "pcmanfm" +SRC_URI = "${SOURCEFORGE_MIRROR}/${SOURCEFORGE_PROJECT}/libfm-${PV}.tar.xz \ file://0001-Correctly-check-the-stamp-file-that-indicates-if-we-.patch \ file://0001-Do-not-add-library-path-to-avoid-host-contamination.patch \ " SRC_URI[sha256sum] = "a5042630304cf8e5d8cff9d565c6bd546f228b48c960153ed366a34e87cad1e5" -inherit autotools pkgconfig gtk-doc gettext features_check mime mime-xdg +inherit autotools pkgconfig gtk-doc gettext features_check mime mime-xdg sourceforge-releases ANY_OF_DISTRO_FEATURES = "${GTK3DISTROFEATURES}" EXTRA_OECONF = "--with-gtk=3"
meta/recipes-support/libpcre/libpcre_8.45.bb +3 -2
@@ -7,7 +7,8 @@ HOMEPAGE = "http://www.pcre.org" SECTION = "devel" LICENSE = "BSD-3-Clause" LIC_FILES_CHKSUM = "file://LICENCE;md5=b5d5d1a69a24ea2718263f1ff85a1c58" -SRC_URI = "${SOURCEFORGE_MIRROR}/pcre/pcre-${PV}.tar.bz2 \ +SOURCEFORGE_PROJECT = "pcre" +SRC_URI = "${SOURCEFORGE_MIRROR}/${SOURCEFORGE_PROJECT}/pcre-${PV}.tar.bz2 \ file://run-ptest \ file://Makefile \ "
@@ -32,7 +33,7 @@ PACKAGECONFIG[jit] = "--enable-jit=auto,--disable-jit" BINCONFIG = "${bindir}/pcre-config" -inherit autotools binconfig-disabled ptest +inherit autotools binconfig-disabled ptest sourceforge-releases EXTRA_OECONF = "--enable-utf"
Signature · 1 line
2.39.5
Patchtest results for [PATCH 1/2] fetch2/wget.py: use default wget user-agent in upstream version checks
In reply to Alexander KanavinThank you for your submission. Patchtest identified one or more issues with the patch. Please see the log below for more information:
---
Testing patch /home/patchtest/share/mboxes/1-2-fetch2-wget.py-use-default-wget-user-agent-in-upstream-version-checks.patch
FAIL: test target mailing list: Series sent to the wrong mailing list or some patches from the series correspond to different mailing lists (test_mbox.TestMbox.test_target_mailing_list)
PASS: pretest pylint (test_python_pylint.PyLint.pretest_pylint)
PASS: test Signed-off-by presence (test_mbox.TestMbox.test_signed_off_by_presence)
PASS: test author valid (test_mbox.TestMbox.test_author_valid)
PASS: test commit message presence (test_mbox.TestMbox.test_commit_message_presence)
PASS: test max line length (test_metadata.TestMetadata.test_max_line_length)
PASS: test mbox format (test_mbox.TestMbox.test_mbox_format)
PASS: test non-AUH upgrade (test_mbox.TestMbox.test_non_auh_upgrade)
PASS: test pylint (test_python_pylint.PyLint.test_pylint)
PASS: test shortlog format (test_mbox.TestMbox.test_shortlog_format)
PASS: test shortlog length (test_mbox.TestMbox.test_shortlog_length)
SKIP: pretest src uri left files: No modified recipes, skipping pretest (test_metadata.TestMetadata.pretest_src_uri_left_files)
SKIP: test CVE check ignore: No modified recipes or older target branch, skipping test (test_metadata.TestMetadata.test_cve_check_ignore)
SKIP: test CVE tag format: No new CVE patches introduced (test_patch.TestPatch.test_cve_tag_format)
SKIP: test Signed-off-by presence: No new CVE patches introduced (test_patch.TestPatch.test_signed_off_by_presence)
SKIP: test Upstream-Status presence: No new CVE patches introduced (test_patch.TestPatch.test_upstream_status_presence_format)
SKIP: test bugzilla entry format: No bug ID found (test_mbox.TestMbox.test_bugzilla_entry_format)
SKIP: test lic files chksum modified not mentioned: No modified recipes, skipping test (test_metadata.TestMetadata.test_lic_files_chksum_modified_not_mentioned)
SKIP: test lic files chksum presence: No added recipes, skipping test (test_metadata.TestMetadata.test_lic_files_chksum_presence)
SKIP: test license presence: No added recipes, skipping test (test_metadata.TestMetadata.test_license_presence)
SKIP: test series merge on head: Merge test is disabled for now (test_mbox.TestMbox.test_series_merge_on_head)
SKIP: test src uri left files: No modified recipes, skipping pretest (test_metadata.TestMetadata.test_src_uri_left_files)
SKIP: test summary presence: No added recipes, skipping test (test_metadata.TestMetadata.test_summary_presence)
---
Please address the issues identified and
submit a new revision of the patch, or alternatively, reply to this email with an explanation of why the patch should be accepted. If you believe these results are due to an error in patchtest, please submit a bug at https://bugzilla.yoctoproject.org/ (use the 'Patchtest' category under 'Yocto Project Subprojects'). For more information on specific failures, see: https://wiki.yoctoproject.org/wiki/Patchtest. Thank you!