Skip to contentSkip to conversation

zeus -next review

PATCH In review Anuj Mittal 2 participants 15 messages Apr 6, 2020 – Apr 9, 2020

lore
Share & tools
lore
#0 lore raw

cover zeus -next review

Next set of changes for zeus. Please review.

Thanks,

Anuj

The following changes since commit 0743dcd0cd70dee87a49d3bcd017168352e60982:

wic/direct: reserve 2 sector for extended partition (2020-03-26 07:33:07 +0800)

are available in the Git repository at:

git://push.openembedded.org/openembedded-core-contrib anujm/zeus

Jan Luebbe (5):
lib/oe/package_manager: make sure to not remove packages in apt
install
lib/oe/package_manager: fix handling of last package lib/oe/package_manager: collect provided package names when using debs lib/oe/package_manager: avoid installing provided packages via apt lib/oe/package_manager: don't try to rm /var/lib/opkg

Jeremy Puhlman (1):
busybox: on upgrade save busybox if it is the last shell

Joshua Watt (1):
u-boot-tools: Split out inc file

Marek Vasut (2):
waffle: no need to depend on target python3
lttng-modules: update to 2.10.14

Paul Barker (3):
kernel-yocto.bbclass: Support config fragments with externalsrc
perf: Fix externalsrc support
kernelsrc.bbclass: Fix externalsrc support

haiqing (1):
glib-2.0: fix CVE-2020-6750

 meta/classes/kernel-yocto.bbclass             |   3 +-
 meta/classes/kernelsrc.bbclass                |   2 +-
 meta/lib/oe/package_manager.py                |  37 +-
 meta/recipes-bsp/u-boot/u-boot-tools.inc      |  65 ++
 .../u-boot/u-boot-tools_2019.07.bb            |  67 +-
 meta/recipes-core/busybox/busybox.inc         |  43 +
 .../glib-2.0/glib-2.0/CVE-2020-6750.patch     | 741 ++++++++++++++++++
 meta/recipes-core/glib-2.0/glib-2.0_2.60.7.bb |   1 +
 meta/recipes-graphics/waffle/waffle_1.6.0.bb  |   5 +
 ...ops-when-trace-sunrpc_task-events-in.patch |  94 ---
 ...rpc_clnt-dereference-in-rpc_task_que.patch |  44 --
 ...rpc-use-signed-integer-for-client-id.patch | 105 ---
 ...nrpc-introduce-lttng_get_clid-helper.patch | 130 ---
 ...es_2.10.11.bb => lttng-modules_2.10.14.bb} |  12 +-
 meta/recipes-kernel/perf/perf.bb              |   2 +-
 15 files changed, 887 insertions(+), 464 deletions(-)
 create mode 100644 meta/recipes-bsp/u-boot/u-boot-tools.inc
 create mode 100644 meta/recipes-core/glib-2.0/glib-2.0/CVE-2020-6750.patch
 delete mode 100644 meta/recipes-kernel/lttng/lttng-modules/0001-Fix-SUNRPC-Fix-oops-when-trace-sunrpc_task-events-in.patch
 delete mode 100644 meta/recipes-kernel/lttng/lttng-modules/0002-Fix-sunrpc-null-rpc_clnt-dereference-in-rpc_task_que.patch
 delete mode 100644 meta/recipes-kernel/lttng/lttng-modules/0003-Fix-sunrpc-use-signed-integer-for-client-id.patch
 delete mode 100644 meta/recipes-kernel/lttng/lttng-modules/0004-sunrpc-introduce-lttng_get_clid-helper.patch
 rename meta/recipes-kernel/lttng/{lttng-modules_2.10.11.bb => lttng-modules_2.10.14.bb} (75%)
Signature · 1 line
2.25.1
#1 lore raw

PATCH 1/13 u-boot-tools: Split out inc file

From: Joshua Watt <[email protected]>

Spilts out the inc file for u-boot-tools so that BSPs can include it for their forked version of the tools, just like u-boot.inc.

As an example, this recipe [1] builds mkimage u-boot as part of the u-boot recipe itself. This is incorrect because u-boot is a target recipe, which means that the uninative loader transformations do not get applied when the recipe is restored from sstate, breaking the mkimage command breaks. The correct solution is to have a u-boot-tools-imx recipe that pulls in their custom u-boot source; this inc file makes that much easier to maintain.

[1]: https://github.com/Freescale/meta-freescale/blob/aa27fc62095cfd6c147c8f6865296ddc29bf0fa6/recipes-bsp/u-boot/u-boot-imx_2019.04.bb

(cherry picked from commit 2f3bd198d9b1e85f17f7a587b1690ab196ab0300)
---
 meta/recipes-bsp/u-boot/u-boot-tools.inc      | 65 ++++++++++++++++++
 .../u-boot/u-boot-tools_2019.07.bb            | 67 +------------------
 2 files changed, 66 insertions(+), 66 deletions(-)
 create mode 100644 meta/recipes-bsp/u-boot/u-boot-tools.inc
meta/recipes-bsp/u-boot/u-boot-tools.inc new file +65 -0
new file mode 100644
index 0000000000..35894e1a8f
@@ -0,0 +1,65 @@
+SUMMARY = "U-Boot bootloader tools"
+DEPENDS += "openssl"
+
+PROVIDES = "${MLPREFIX}u-boot-mkimage ${MLPREFIX}u-boot-mkenvimage"
+PROVIDES_class-native = "u-boot-mkimage-native u-boot-mkenvimage-native"
+
+PACKAGES += "${PN}-mkimage ${PN}-mkenvimage"
+
+# Required for backward compatibility with "u-boot-mkimage-xxx.bb"
+RPROVIDES_${PN}-mkimage = "u-boot-mkimage"
+RREPLACES_${PN}-mkimage = "u-boot-mkimage"
+RCONFLICTS_${PN}-mkimage = "u-boot-mkimage"
+
+EXTRA_OEMAKE_class-target = 'CROSS_COMPILE="${TARGET_PREFIX}" CC="${CC} ${CFLAGS} ${LDFLAGS}" HOSTCC="${BUILD_CC} ${BUILD_CFLAGS} ${BUILD_LDFLAGS}" STRIP=true V=1'
+EXTRA_OEMAKE_class-native = 'CC="${BUILD_CC} ${BUILD_CFLAGS} ${BUILD_LDFLAGS}" HOSTCC="${BUILD_CC} ${BUILD_CFLAGS} ${BUILD_LDFLAGS}" STRIP=true V=1'
+EXTRA_OEMAKE_class-nativesdk = 'CROSS_COMPILE="${HOST_PREFIX}" CC="${CC} ${CFLAGS} ${LDFLAGS}" HOSTCC="${BUILD_CC} ${BUILD_CFLAGS} ${BUILD_LDFLAGS}" STRIP=true V=1'
+
+SED_CONFIG_EFI = '-e "s/CONFIG_EFI_LOADER=.*/# CONFIG_EFI_LOADER is not set/"'
+SED_CONFIG_EFI_x86 = ''
+SED_CONFIG_EFI_x86-64 = ''
+SED_CONFIG_EFI_arm = ''
+SED_CONFIG_EFI_armeb = ''
+SED_CONFIG_EFI_aarch64 = ''
+
+do_compile () {
+	oe_runmake sandbox_defconfig
+
+	# Disable CONFIG_CMD_LICENSE, license.h is not used by tools and
+	# generating it requires bin2header tool, which for target build
+	# is built with target tools and thus cannot be executed on host.
+	sed -i -e "s/CONFIG_CMD_LICENSE=.*/# CONFIG_CMD_LICENSE is not set/" ${SED_CONFIG_EFI} .config
+
+	oe_runmake cross_tools NO_SDL=1
+}
+
+do_install () {
+	install -d ${D}${bindir}
+
+	# mkimage
+	install -m 0755 tools/mkimage ${D}${bindir}/uboot-mkimage
+	ln -sf uboot-mkimage ${D}${bindir}/mkimage
+
+	# mkenvimage
+	install -m 0755 tools/mkenvimage ${D}${bindir}/uboot-mkenvimage
+	ln -sf uboot-mkenvimage ${D}${bindir}/mkenvimage
+
+	# dumpimage
+	install -m 0755 tools/dumpimage ${D}${bindir}/uboot-dumpimage
+	ln -sf uboot-dumpimage ${D}${bindir}/dumpimage
+
+	# fit_check_sign
+	install -m 0755 tools/fit_check_sign ${D}${bindir}/uboot-fit_check_sign
+	ln -sf uboot-fit_check_sign ${D}${bindir}/fit_check_sign
+}
+
+ALLOW_EMPTY_${PN} = "1"
+FILES_${PN} = ""
+FILES_${PN}-mkimage = "${bindir}/uboot-mkimage ${bindir}/mkimage ${bindir}/uboot-dumpimage ${bindir}/dumpimage ${bindir}/uboot-fit_check_sign ${bindir}/fit_check_sign"
+FILES_${PN}-mkenvimage = "${bindir}/uboot-mkenvimage ${bindir}/mkenvimage"
+
+RDEPENDS_${PN}-mkimage += "dtc"
+RDEPENDS_${PN} += "${PN}-mkimage ${PN}-mkenvimage"
+RDEPENDS_${PN}_class-native = ""
+
+BBCLASSEXTEND = "native nativesdk"
meta/recipes-bsp/u-boot/u-boot-tools_2019.07.bb +1 -66

68 lines not on this page · raw message

Signature · 1 line
2.25.1
#2 lore raw

PATCH 2/13 waffle: no need to depend on target python3

In reply to Anuj Mittal

From: Marek Vasut <[email protected]>

---
 meta/recipes-graphics/waffle/waffle_1.6.0.bb | 5 +++++
 1 file changed, 5 insertions(+)
meta/recipes-graphics/waffle/waffle_1.6.0.bb +5 -0
index 8a1d5748f6..82cead9ad1 100644
@@ -35,3 +35,8 @@ PACKAGECONFIG[x11-egl] = "-Dx11_egl=enabled,-Dx11_egl=disabled,virtual/${MLPREFI
 PACKAGECONFIG[surfaceless-egl] = "-Dsurfaceless_egl=enabled,-Dsurfaceless_egl=disabled,virtual/${MLPREFIX}libgl"
 
 # TODO: optionally build manpages and examples
+
+# Unset these to stop python trying to report the target Python setup
+_PYTHON_SYSCONFIGDATA_NAME[unexport] = "1"
+STAGING_INCDIR[unexport] = "1"
+STAGING_LIBDIR[unexport] = "1"
Signature · 1 line
2.25.1
#3 lore raw

PATCH 3/13 lttng-modules: update to 2.10.14

In reply to Anuj Mittal

From: Marek Vasut <[email protected]>

Upgrade to version 2.10.14 in order to fix some build errors with latest 5.4 kernel.

- conflicting types for 'trace_rcu_dyntick'
Reproductible on kernel greater than v5.4.22, starting with commit 6cf539a87a ("rcu: Fix data-race due to atomic_t copy-by-value")

Also drop patches which are no longer required.

Cc Armin Kuster <[email protected]>
Cc Daniel Dragomir <[email protected]>
Cc Richard Purdie <[email protected]>
---
 ...ops-when-trace-sunrpc_task-events-in.patch |  94 -------------
 ...rpc_clnt-dereference-in-rpc_task_que.patch |  44 ------
 ...rpc-use-signed-integer-for-client-id.patch | 105 --------------
 ...nrpc-introduce-lttng_get_clid-helper.patch | 130 ------------------
 ...es_2.10.11.bb => lttng-modules_2.10.14.bb} |  12 +-
 5 files changed, 4 insertions(+), 381 deletions(-)
 delete mode 100644 meta/recipes-kernel/lttng/lttng-modules/0001-Fix-SUNRPC-Fix-oops-when-trace-sunrpc_task-events-in.patch
 delete mode 100644 meta/recipes-kernel/lttng/lttng-modules/0002-Fix-sunrpc-null-rpc_clnt-dereference-in-rpc_task_que.patch
 delete mode 100644 meta/recipes-kernel/lttng/lttng-modules/0003-Fix-sunrpc-use-signed-integer-for-client-id.patch
 delete mode 100644 meta/recipes-kernel/lttng/lttng-modules/0004-sunrpc-introduce-lttng_get_clid-helper.patch
 rename meta/recipes-kernel/lttng/{lttng-modules_2.10.11.bb => lttng-modules_2.10.14.bb} (75%)
meta/recipes-kernel/lttng/lttng-modules/0001-Fix-SUNRPC-Fix-oops-when-trace-sunrpc_task-events-in.patch deleted +0 -94
deleted file mode 100644
index bdbc4f811e..0000000000
@@ -1,94 +0,0 @@
-From 1ff7013bcf7f068cf4371d12d758f9c0fd16a619 Mon Sep 17 00:00:00 2001
-From: Quanyang Wang <[email protected]>
-Date: Thu, 5 Dec 2019 15:35:32 +0800
-Subject: [PATCH 1/4] Fix: SUNRPC: Fix oops when trace sunrpc_task events in
- nfs client
-
-See upstream commit :
-
-	commit 2ca310fc4160ed0420da65534a21ae77b24326a8
-	Author: Ditang Chen <[email protected]>
-	Date: Fri, 7 Mar 2014 13:27:57 +0800
-	Subject: SUNRPC: Fix oops when trace sunrpc_task events in nfs client
-
-	When tracking sunrpc_task events in nfs client, the clnt pointer may be NULL.
-
-	[  139.269266] BUG: unable to handle kernel NULL pointer dereference at 0000000000000004
-	[  139.269915] IP: [<ffffffffa026f216>] ftrace_raw_event_rpc_task_running+0x86/0xf0 [sunrpc]
-	[  139.269915] PGD 1d293067 PUD 1d294067 PMD 0
-	[  139.269915] Oops: 0000 [#1] SMP
-	[  139.269915] Modules linked in: nfsv4 dns_resolver nfs lockd sunrpc fscache sg ppdev e1000
-	serio_raw pcspkr parport_pc parport i2c_piix4 i2c_core microcode xfs libcrc32c sd_mod sr_mod
-	cdrom ata_generic crc_t10dif crct10dif_common pata_acpi ahci libahci ata_piix libata dm_mirror
-	dm_region_hash dm_log dm_mod
-	[  139.269915] CPU: 0 PID: 59 Comm: kworker/0:2 Not tainted 3.10.0-84.el7.x86_64 #1
-	[  139.269915] Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
-	[  139.269915] Workqueue: rpciod rpc_async_schedule [sunrpc]
-	[  139.269915] task: ffff88001b598000 ti: ffff88001b632000 task.ti: ffff88001b632000
-	[  139.269915] RIP: 0010:[<ffffffffa026f216>]  [<ffffffffa026f216>] ftrace_raw_event_rpc_task_running+0x86/0xf0 [sunrpc]
-	[  139.269915] RSP: 0018:ffff88001b633d70  EFLAGS: 00010206
-	[  139.269915] RAX: ffff88001dfc5338 RBX: ffff88001cc37a00 RCX: ffff88001dfc5334
-	[  139.269915] RDX: ffff88001dfc5338 RSI: 0000000000000000 RDI: ffff88001dfc533c
-	[  139.269915] RBP: ffff88001b633db0 R08: 000000000000002c R09: 000000000000000a
-	[  139.269915] R10: 0000000000062180 R11: 00000020759fb9dc R12: ffffffffa0292c20
-	[  139.269915] R13: ffff88001dfc5334 R14: 0000000000000000 R15: 0000000000000000
-	[  139.269915] FS:  0000000000000000(0000) GS:ffff88001fc00000(0000) knlGS:0000000000000000
-	[  139.269915] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
-	[  139.269915] CR2: 0000000000000004 CR3: 000000001d290000 CR4: 00000000000006f0
-	[  139.269915] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
-	[  139.269915] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
-	[  139.269915] Stack:
-	[  139.269915]  000000001b633d98 0000000000000246 ffff88001df1dc00 ffff88001cc37a00
-	[  139.269915]  ffff88001bc35e60 0000000000000000 ffff88001ffa0a48 ffff88001bc35ee0
-	[  139.269915]  ffff88001b633e08 ffffffffa02704b5 0000000000010000 ffff88001cc37a70
-	[  139.269915] Call Trace:
-	[  139.269915]  [<ffffffffa02704b5>] __rpc_execute+0x1d5/0x400 [sunrpc]
-	[  139.269915]  [<ffffffffa0270706>] rpc_async_schedule+0x26/0x30 [sunrpc]
-	[  139.269915]  [<ffffffff8107867b>] process_one_work+0x17b/0x460
-	[  139.269915]  [<ffffffff8107942b>] worker_thread+0x11b/0x400
-	[  139.269915]  [<ffffffff81079310>] ? rescuer_thread+0x3e0/0x3e0
-	[  139.269915]  [<ffffffff8107fc80>] kthread+0xc0/0xd0
-	[  139.269915]  [<ffffffff8107fbc0>] ? kthread_create_on_node+0x110/0x110
-	[  139.269915]  [<ffffffff815d122c>] ret_from_fork+0x7c/0xb0
-	[  139.269915]  [<ffffffff8107fbc0>] ? kthread_create_on_node+0x110/0x110
-	[  139.269915] Code: 4c 8b 45 c8 48 8d 7d d0 89 4d c4 41 89 c9 b9 28 00 00 00 e8 9d b4 e9
-	e0 48 85 c0 49 89 c5 74 a2 48 89 c7 e8 9d 3f e9 e0 48 89 c2 <41> 8b 46 04 48 8b 7d d0 4c
-	89 e9 4c 89 e6 89 42 0c 0f b7 83 d4
-	[  139.269915] RIP  [<ffffffffa026f216>] ftrace_raw_event_rpc_task_running+0x86/0xf0 [sunrpc]
-	[  139.269915]  RSP <ffff88001b633d70>
-	[  139.269915] CR2: 0000000000000004
-	[  140.946406] ---[ end trace ba486328b98d7622 ]---
-
-Upstream-Status: Backport [https://github.com/lttng/lttng-modules/commit/2b228b503cad10bf0c5a99b42a908ca906eab5b9]
-
-Signed-off-by: Quanyang Wang <[email protected]>
-Signed-off-by: Mathieu Desnoyers <[email protected]>
----
- instrumentation/events/lttng-module/rpc.h | 4 ++--
- 1 file changed, 2 insertions(+), 2 deletions(-)
-
-diff --git a/instrumentation/events/lttng-module/rpc.h b/instrumentation/events/lttng-module/rpc.h
-index 3798e8e..fb13106 100644
---- a/instrumentation/events/lttng-module/rpc.h
-+++ b/instrumentation/events/lttng-module/rpc.h
-@@ -139,7 +139,7 @@ LTTNG_TRACEPOINT_EVENT_CLASS(rpc_task_running,
- 
- 	TP_FIELDS(
- 		ctf_integer(unsigned int, task_id, task->tk_pid)
--		ctf_integer(unsigned int, client_id, task->tk_client->cl_clid)
-+		ctf_integer(unsigned int, client_id, task->tk_client ? task->tk_client->cl_clid : -1)
- 		ctf_integer_hex(const void *, action, action)
- 		ctf_integer(unsigned long, runstate, task->tk_runstate)
- 		ctf_integer(int, status, task->tk_status)
-@@ -208,7 +208,7 @@ LTTNG_TRACEPOINT_EVENT_CLASS(rpc_task_running,
- 
- 	TP_FIELDS(
- 		ctf_integer(unsigned int, task_id, task->tk_pid)
--		ctf_integer(unsigned int, client_id, task->tk_client->cl_clid)
-+		ctf_integer(unsigned int, client_id, task->tk_client ? task->tk_client->cl_clid : -1)
- 		ctf_integer_hex(const void *, action, action)
- 		ctf_integer(unsigned long, runstate, task->tk_runstate)
- 		ctf_integer(int, status, task->tk_status)
--- 
-2.17.1
-
meta/recipes-kernel/lttng/lttng-modules/0002-Fix-sunrpc-null-rpc_clnt-dereference-in-rpc_task_que.patch deleted +0 -44

44 lines not on this page · raw message

meta/recipes-kernel/lttng/lttng-modules/0003-Fix-sunrpc-use-signed-integer-for-client-id.patch deleted +0 -105

105 lines not on this page · raw message

meta/recipes-kernel/lttng/lttng-modules/0004-sunrpc-introduce-lttng_get_clid-helper.patch deleted +0 -130

130 lines not on this page · raw message

meta/recipes-kernel/lttng/lttng-modules_2.10.14.bb renamed from meta/recipes-kernel/lttng/lttng-modules_2.10.11.bb +4 -8
similarity index 75%
rename from meta/recipes-kernel/lttng/lttng-modules_2.10.11.bb
rename to meta/recipes-kernel/lttng/lttng-modules_2.10.14.bb
index cc4f44519a..1c24e94902 100644
@@ -14,14 +14,10 @@ COMPATIBLE_HOST = '(x86_64|i.86|powerpc|aarch64|mips|nios2|arm|riscv).*-linux'
 SRC_URI = "https://lttng.org/files/${BPN}/${BPN}-${PV}.tar.bz2 \
            file://Makefile-Do-not-fail-if-CONFIG_TRACEPOINTS-is-not-en.patch \
            file://BUILD_RUNTIME_BUG_ON-vs-gcc7.patch \
-           file://0001-Fix-SUNRPC-Fix-oops-when-trace-sunrpc_task-events-in.patch \
-           file://0002-Fix-sunrpc-null-rpc_clnt-dereference-in-rpc_task_que.patch \
-           file://0003-Fix-sunrpc-use-signed-integer-for-client-id.patch \
-           file://0004-sunrpc-introduce-lttng_get_clid-helper.patch \
            "
 
-SRC_URI[md5sum] = "c618fb646514dfc1bf910cfd7cda4256"
-SRC_URI[sha256sum] = "7f91e39b2e8e46d8bbba2b4c8c1614f1fb380611cd1a1fccc1d1859be26112f1"
+SRC_URI[md5sum] = "3e9ed67a2da17edf93194f8a5e75a246"
+SRC_URI[sha256sum] = "d0ba614a9cac3daf8ac034837f8b786e6be2ce0242aeecef7096bed5e03b762c"
 
 export INSTALL_MOD_DIR="kernel/lttng-modules"
 
@@ -44,7 +40,7 @@ SRC_URI_class-devupstream = "git://git.lttng.org/lttng-modules;branch=stable-2.1
            file://Makefile-Do-not-fail-if-CONFIG_TRACEPOINTS-is-not-en.patch \
            file://BUILD_RUNTIME_BUG_ON-vs-gcc7.patch \
            "
-SRCREV_class-devupstream = "624aca5d7507fbd11ea4a1a474c3aa1031bd9a31"
-PV_class-devupstream = "2.10.10+git${SRCPV}"
+SRCREV_class-devupstream = "b34304f146ea234ea764580d7ce1b03d05a215f9"
+PV_class-devupstream = "2.10.14+git${SRCPV}"
 S_class-devupstream = "${WORKDIR}/git"
 SRCREV_FORMAT ?= "lttng_git"
Signature · 1 line
2.25.1
#4 lore raw

PATCH 4/13 glib-2.0: fix CVE-2020-6750

In reply to Anuj Mittal

From: haiqing <[email protected]>

GSocketClient in GNOME GLib through 2.62.4 may occasionally connect directly to a target address instead of connecting via a proxy server when configured to do so, because the proxy_addr field is mishandled. This bug is timing-dependent and may occur only sporadically depending on network delays. The greatest security relevance is in use cases where a proxy is used to help with privacy/anonymity, even though there is no technical barrier to a direct connection.

---
 .../glib-2.0/glib-2.0/CVE-2020-6750.patch     | 741 ++++++++++++++++++
 meta/recipes-core/glib-2.0/glib-2.0_2.60.7.bb |   1 +
 2 files changed, 742 insertions(+)
 create mode 100644 meta/recipes-core/glib-2.0/glib-2.0/CVE-2020-6750.patch
meta/recipes-core/glib-2.0/glib-2.0/CVE-2020-6750.patch new file +741 -0
new file mode 100644
index 0000000000..6db3934978
@@ -0,0 +1,741 @@
+From 747f2c646f5a86ac58ad59be08036e81388e971d Mon Sep 17 00:00:00 2001
+From: Patrick Griffis <[email protected]>
+Date: Thu, 23 Jan 2020 19:58:41 -0800
+Subject: [PATCH] Refactor g_socket_client_connect_async()
+
+This is a fairly large refactoring. The highlights are:
+
+- Removing in-progress connections/addresses from GSocketClientAsyncConnectData:
+
+  This caused issues where multiple ConnectionAttempt's would step over eachother
+  and modify shared state causing bugs like accidentally bypassing a set proxy.
+
+  Fixes #1871
+  Fixes #1989
+  Fixes #1902
+
+- Cancelling address enumeration on error/completion
+
+- Queuing successful TCP connections and doing application layer work serially:
+
+  This is more in the spirit of Happy Eyeballs but it also greatly simplifies
+  the flow of connection handling so fewer tasks are happening in parallel
+  when they don't need to be.
+
+  The behavior also should more closely match that of g_socket_client_connect().
+
+- Better track the state of address enumeration:
+
+  Previously we were over eager to treat enumeration finishing as an error.
+
+  Fixes #1872
+  See also #1982
+
+- Add more detailed documentation and logging.
+
+Closes #1995
+
+CVE: CVE-2020-6750
+
+Upstream-Status: Backport [ https://gitlab.gnome.org/GNOME/glib.git;
+commit=2722620e3291b930a3a228100d7c0e07b69534e3 ]
+
+Signed-off-by: Haiqing Bai <[email protected]>
+---
+ gio/gsocketclient.c | 459 ++++++++++++++++++++++++++++----------------
+ 1 file changed, 296 insertions(+), 163 deletions(-)
+
+diff --git a/gio/gsocketclient.c b/gio/gsocketclient.c
+index 81767c0..b1d5f6c 100644
+--- a/gio/gsocketclient.c
++++ b/gio/gsocketclient.c
+@@ -1332,13 +1332,15 @@ typedef struct
+ 
+   GSocketConnectable *connectable;
+   GSocketAddressEnumerator *enumerator;
+-  GProxyAddress *proxy_addr;
+-  GSocket *socket;
+-  GIOStream *connection;
++  GCancellable *enumeration_cancellable;
+ 
+   GSList *connection_attempts;
++  GSList *successful_connections;
+   GError *last_error;
+ 
++  gboolean enumerated_at_least_once;
++  gboolean enumeration_completed;
++  gboolean connection_in_progress;
+   gboolean completed;
+ } GSocketClientAsyncConnectData;
+ 
+@@ -1350,10 +1352,9 @@ g_socket_client_async_connect_data_free (GSocketClientAsyncConnectData *data)
+   data->task = NULL;
+   g_clear_object (&data->connectable);
+   g_clear_object (&data->enumerator);
+-  g_clear_object (&data->proxy_addr);
+-  g_clear_object (&data->socket);
+-  g_clear_object (&data->connection);
++  g_clear_object (&data->enumeration_cancellable);
+   g_slist_free_full (data->connection_attempts, connection_attempt_unref);
++  g_slist_free_full (data->successful_connections, connection_attempt_unref);
+ 
+   g_clear_error (&data->last_error);
+ 
+@@ -1365,6 +1366,7 @@ typedef struct
+   GSocketAddress *address;
+   GSocket *socket;
+   GIOStream *connection;
++  GProxyAddress *proxy_addr;
+   GSocketClientAsyncConnectData *data; /* unowned */
+   GSource *timeout_source;
+   GCancellable *cancellable;
+@@ -1396,6 +1398,7 @@ connection_attempt_unref (gpointer pointer)
+       g_clear_object (&attempt->socket);
+       g_clear_object (&attempt->connection);
+       g_clear_object (&attempt->cancellable);
++      g_clear_object (&attempt->proxy_addr);
+       if (attempt->timeout_source)
+         {
+           g_source_destroy (attempt->timeout_source);
+@@ -1413,37 +1416,59 @@ connection_attempt_remove (ConnectionAttempt *attempt)
+ }
+ 
+ static void
+-g_socket_client_async_connect_complete (GSocketClientAsyncConnectData *data)
++cancel_all_attempts (GSocketClientAsyncConnectData *data)
+ {
+-  g_assert (data->connection);
++  GSList *l;
+ 
+-  if (!G_IS_SOCKET_CONNECTION (data->connection))
++  for (l = data->connection_attempts; l; l = g_slist_next (l))
+     {
+-      GSocketConnection *wrapper_connection;
+-
+-      wrapper_connection = g_tcp_wrapper_connection_new (data->connection, data->socket);
+-      g_object_unref (data->connection);
+-      data->connection = (GIOStream *)wrapper_connection;
++      ConnectionAttempt *attempt_entry = l->data;
++      g_cancellable_cancel (attempt_entry->cancellable);
++      connection_attempt_unref (attempt_entry);
+     }
++  g_slist_free (data->connection_attempts);
++  data->connection_attempts = NULL;
+ 
+-  if (!data->completed)
++  g_slist_free_full (data->successful_connections, connection_attempt_unref);
++  data->successful_connections = NULL;
++
++  g_cancellable_cancel (data->enumeration_cancellable);
++}
++
++static void
++g_socket_client_async_connect_complete (ConnectionAttempt *attempt)
++{
++  GSocketClientAsyncConnectData *data = attempt->data;
++  GError *error = NULL;
++  g_assert (attempt->connection);
++  g_assert (!data->completed);
++
++  if (!G_IS_SOCKET_CONNECTION (attempt->connection))
+     {
+-      GError *error = NULL;
++      GSocketConnection *wrapper_connection;
+ 
+-      if (g_cancellable_set_error_if_cancelled (g_task_get_cancellable (data->task), &error))
+-        {
+-          g_socket_client_emit_event (data->client, G_SOCKET_CLIENT_COMPLETE, data->connectable, NULL);
+-          g_task_return_error (data->task, g_steal_pointer (&error));
+-        }
+-      else
+-        {
+-          g_socket_client_emit_event (data->client, G_SOCKET_CLIENT_COMPLETE, data->connectable, data->connection);
+-          g_task_return_pointer (data->task, g_steal_pointer (&data->connection), g_object_unref);
+-        }
++      wrapper_connection = g_tcp_wrapper_connection_new (attempt->connection, attempt->socket);
++      g_object_unref (attempt->connection);
++      attempt->connection = (GIOStream *)wrapper_connection;
++    }
+ 
+-      data->completed = TRUE;
++  data->completed = TRUE;
++  cancel_all_attempts (data);
++
++  if (g_cancellable_set_error_if_cancelled (g_task_get_cancellable (data->task), &error))
++    {
++      g_debug ("GSocketClient: Connection cancelled!");
++      g_socket_client_emit_event (data->client, G_SOCKET_CLIENT_COMPLETE, data->connectable, NULL);
++      g_task_return_error (data->task, g_steal_pointer (&error));
++    }
++  else
++    {
++      g_debug ("GSocketClient: Connection successful!");
++      g_socket_client_emit_event (data->client, G_SOCKET_CLIENT_COMPLETE, data->connectable, attempt->connection);
++      g_task_return_pointer (data->task, g_steal_pointer (&attempt->connection), g_object_unref);
+     }
+ 
++  connection_attempt_unref (attempt);
+   g_object_unref (data->task);
+ }
+ 
+@@ -1465,59 +1490,63 @@ static void
+ enumerator_next_async (GSocketClientAsyncConnectData *data,
+                        gboolean                       add_task_ref)
+ {
+-  /* We need to cleanup the state */
+-  g_clear_object (&data->socket);
+-  g_clear_object (&data->proxy_addr);
+-  g_clear_object (&data->connection);
+-
+   /* Each enumeration takes a ref. This arg just avoids repeated unrefs when
+      an enumeration starts another enumeration */
+   if (add_task_ref)
+     g_object_ref (data->task);
+ 
+   g_socket_client_emit_event (data->client, G_SOCKET_CLIENT_RESOLVING, data->connectable, NULL);
++  g_debug ("GSocketClient: Starting new address enumeration");
+   g_socket_address_enumerator_next_async (data->enumerator,
+-					  g_task_get_cancellable (data->task),
++					  data->enumeration_cancellable,
+ 					  g_socket_client_enumerator_callback,
+ 					  data);
+ }
+ 
++static void try_next_connection_or_finish (GSocketClientAsyncConnectData *, gboolean);
++
+ static void
+ g_socket_client_tls_handshake_callback (GObject      *object,
+ 					GAsyncResult *result,
+ 					gpointer      user_data)
+ {
+-  GSocketClientAsyncConnectData *data = user_data;
++  ConnectionAttempt *attempt = user_data;
++  GSocketClientAsyncConnectData *data = attempt->data;
+ 
+   if (g_tls_connection_handshake_finish (G_TLS_CONNECTION (object),
+ 					 result,
+ 					 &data->last_error))
+     {
+-      g_object_unref (data->connection);
+-      data->connection = G_IO_STREAM (object);
++      g_object_unref (attempt->connection);
++      attempt->connection = G_IO_STREAM (object);
+ 
+-      g_socket_client_emit_event (data->client, G_SOCKET_CLIENT_TLS_HANDSHAKED, data->connectable, data->connection);
+-      g_socket_client_async_connect_complete (data);
++      g_debug ("GSocketClient: TLS handshake succeeded");
++      g_socket_client_emit_event (data->client, G_SOCKET_CLIENT_TLS_HANDSHAKED, data->connectable, attempt->connection);
++      g_socket_client_async_connect_complete (attempt);
+     }
+   else
+     {
+       g_object_unref (object);
+-      enumerator_next_async (data, FALSE);
++      connection_attempt_unref (attempt);
++      g_debug ("GSocketClient: TLS handshake failed: %s", data->last_error->message);
++      try_next_connection_or_finish (data, TRUE);
+     }
+ }
+ 
+ static void
+-g_socket_client_tls_handshake (GSocketClientAsyncConnectData *data)
++g_socket_client_tls_handshake (ConnectionAttempt *attempt)
+ {
++  GSocketClientAsyncConnectData *data = attempt->data;
+   GIOStream *tlsconn;
+ 
+   if (!data->client->priv->tls)
+     {
+-      g_socket_client_async_connect_complete (data);
++      g_socket_client_async_connect_complete (attempt);
+       return;
+     }
+ 
+-  tlsconn = g_tls_client_connection_new (data->connection,
++  g_debug ("GSocketClient: Starting TLS handshake");
++  tlsconn = g_tls_client_connection_new (attempt->connection,
+ 					 data->connectable,
+ 					 &data->last_error);
+   if (tlsconn)
+@@ -1529,11 +1558,12 @@ g_socket_client_tls_handshake (GSocketClientAsyncConnectData *data)
+ 					G_PRIORITY_DEFAULT,
+ 					g_task_get_cancellable (data->task),
+ 					g_socket_client_tls_handshake_callback,
+-					data);
++					attempt);
+     }
+   else
+     {
+-      enumerator_next_async (data, FALSE);
++      connection_attempt_unref (attempt);
++      try_next_connection_or_finish (data, TRUE);
+     }
+ }
+ 
+@@ -1542,23 +1572,38 @@ g_socket_client_proxy_connect_callback (GObject      *object,
+ 					GAsyncResult *result,
+ 					gpointer      user_data)
+ {
+-  GSocketClientAsyncConnectData *data = user_data;
++  ConnectionAttempt *attempt = user_data;
++  GSocketClientAsyncConnectData *data = attempt->data;
+ 
+-  g_object_unref (data->connection);
+-  data->connection = g_proxy_connect_finish (G_PROXY (object),
+-					     result,
+-					     &data->last_error);
+-  if (data->connection)
++  g_object_unref (attempt->connection);
++  attempt->connection = g_proxy_connect_finish (G_PROXY (object),
++                                                result,
++                                                &data->last_error);
++  if (attempt->connection)
+     {
+-      g_socket_client_emit_event (data->client, G_SOCKET_CLIENT_PROXY_NEGOTIATED, data->connectable, data->connection);
++      g_socket_client_emit_event (data->client, G_SOCKET_CLIENT_PROXY_NEGOTIATED, data->connectable, attempt->connection);
+     }
+   else
+     {
+-      enumerator_next_async (data, FALSE);
++      connection_attempt_unref (attempt);
++      try_next_connection_or_finish (data, TRUE);
+       return;
+     }
+ 
+-  g_socket_client_tls_handshake (data);
++  g_socket_client_tls_handshake (attempt);
++}
++
++static void
++complete_connection_with_error (GSocketClientAsyncConnectData *data,
++                                GError                        *error)
++{
++  g_debug ("GSocketClient: Connection failed: %s", error->message);
++  g_assert (!data->completed);
++
++  g_socket_client_emit_event (data->client, G_SOCKET_CLIENT_COMPLETE, data->connectable, NULL);
++  data->completed = TRUE;
++  cancel_all_attempts (data);
++  g_task_return_error (data->task, error);
+ }
+ 
+ static gboolean
+@@ -1572,15 +1617,114 @@ task_completed_or_cancelled (GSocketClientAsyncConnectData *data)
+     return TRUE;
+   else if (g_cancellable_set_error_if_cancelled (cancellable, &error))
+     {
+-      g_socket_client_emit_event (data->client, G_SOCKET_CLIENT_COMPLETE, data->connectable, NULL);
+-      g_task_return_error (task, g_steal_pointer (&error));
+-      data->completed = TRUE;
++      complete_connection_with_error (data, g_steal_pointer (&error));
+       return TRUE;
+     }
+   else
+     return FALSE;
+ }
+ 
++static gboolean
++try_next_successful_connection (GSocketClientAsyncConnectData *data)
++{
++  ConnectionAttempt *attempt;
++  const gchar *protocol;
++  GProxy *proxy;
++
++  if (data->connection_in_progress)
++    return FALSE;
++
++  g_assert (data->successful_connections != NULL);
++  attempt = data->successful_connections->data;
++  g_assert (attempt != NULL);
++  data->successful_connections = g_slist_remove (data->successful_connections, attempt);
++  data->connection_in_progress = TRUE;
++
++  g_debug ("GSocketClient: Starting application layer connection");
++
++  if (!attempt->proxy_addr)
++    {
++      g_socket_client_tls_handshake (g_steal_pointer (&attempt));
++      return TRUE;
++    }
++
++  protocol = g_proxy_address_get_protocol (attempt->proxy_addr);
++
++  /* The connection should not be anything other than TCP,
++   * but let's put a safety guard in case
++   */
++  if (!G_IS_TCP_CONNECTION (attempt->connection))
++    {
++      g_critical ("Trying to proxy over non-TCP connection, this is "
++          "most likely a bug in GLib IO library.");
++
++      g_set_error_literal (&data->last_error,
++          G_IO_ERROR, G_IO_ERROR_NOT_SUPPORTED,
++          _("Proxying over a non-TCP connection is not supported."));
++    }
++  else if (g_hash_table_contains (data->client->priv->app_proxies, protocol))
++    {
++      /* Simply complete the connection, we don't want to do TLS handshake
++       * as the application proxy handling may need proxy handshake first */
++      g_socket_client_async_connect_complete (g_steal_pointer (&attempt));
++      return TRUE;
++    }
++  else if ((proxy = g_proxy_get_default_for_protocol (protocol)))
++    {
++      GIOStream *connection = attempt->connection;
++      GProxyAddress *proxy_addr = attempt->proxy_addr;
++
++      g_socket_client_emit_event (data->client, G_SOCKET_CLIENT_PROXY_NEGOTIATING, data->connectable, attempt->connection);
++      g_debug ("GSocketClient: Starting proxy connection");
++      g_proxy_connect_async (proxy,
++                             connection,
++                             proxy_addr,
++                             g_task_get_cancellable (data->task),
++                             g_socket_client_proxy_connect_callback,
++                             g_steal_pointer (&attempt));
++      g_object_unref (proxy);
++      return TRUE;
++    }
++  else
++    {
++      g_clear_error (&data->last_error);
++
++      g_set_error (&data->last_error, G_IO_ERROR, G_IO_ERROR_NOT_SUPPORTED,
++          _("Proxy protocol “%s” is not supported."),
++          protocol);
++    }
++
++  data->connection_in_progress = FALSE;
++  g_clear_pointer (&attempt, connection_attempt_unref);
++  return FALSE; /* All non-return paths are failures */
++}
++
++static void
++try_next_connection_or_finish (GSocketClientAsyncConnectData *data,
++                               gboolean                       end_current_connection)
++{
++  if (end_current_connection)
++    data->connection_in_progress = FALSE;
++
++  if (data->connection_in_progress)
++    return;
++
++  /* Keep trying successful connections until one works, each iteration pops one */
++  while (data->successful_connections)
++    {
++      if (try_next_successful_connection (data))
++        return;
++    }
++
++  if (!data->enumeration_completed)
++    {
++      enumerator_next_async (data, FALSE);
++      return;
++    }
++
++  complete_connection_with_error (data, data->last_error);
++}
++
+ static void
+ g_socket_client_connected_callback (GObject      *source,
+ 				    GAsyncResult *result,
+@@ -1588,10 +1732,7 @@ g_socket_client_connected_callback (GObject      *source,
+ {
+   ConnectionAttempt *attempt = user_data;
+   GSocketClientAsyncConnectData *data = attempt->data;
+-  GSList *l;
+   GError *error = NULL;
+-  GProxy *proxy;
+-  const gchar *protocol;
+ 
+   if (task_completed_or_cancelled (data) || g_cancellable_is_cancelled (attempt->cancellable))
+     {
+@@ -1613,11 +1754,12 @@ g_socket_client_connected_callback (GObject      *source,
+         {
+           clarify_connect_error (error, data->connectable, attempt->address);
+           set_last_error (data, error);
++          g_debug ("GSocketClient: Connection attempt failed: %s", error->message);
+           connection_attempt_remove (attempt);
+-          enumerator_next_async (data, FALSE);
+           connection_attempt_unref (attempt);
++          try_next_connection_or_finish (data, FALSE);
+         }
+-      else
++      else /* Silently ignore cancelled attempts */
+         {
+           g_clear_error (&error);
+           g_object_unref (data->task);
+@@ -1627,74 +1769,21 @@ g_socket_client_connected_callback (GObject      *source,
+       return;
+     }
+ 
+-  data->socket = g_steal_pointer (&attempt->socket);
+-  data->connection = g_steal_pointer (&attempt->connection);
+-
+-  for (l = data->connection_attempts; l; l = g_slist_next (l))
+-    {
+-      ConnectionAttempt *attempt_entry = l->data;
+-      g_cancellable_cancel (attempt_entry->cancellable);
+-      connection_attempt_unref (attempt_entry);
+-    }
+-  g_slist_free (data->connection_attempts);
+-  data->connection_attempts = NULL;
+-  connection_attempt_unref (attempt);
+-
+-  g_socket_connection_set_cached_remote_address ((GSocketConnection*)data->connection, NULL);
+-  g_socket_client_emit_event (data->client, G_SOCKET_CLIENT_CONNECTED, data->connectable, data->connection);
++  g_socket_connection_set_cached_remote_address ((GSocketConnection*)attempt->connection, NULL);
++  g_debug ("GSocketClient: TCP connection successful");
++  g_socket_client_emit_event (data->client, G_SOCKET_CLIENT_CONNECTED, data->connectable, attempt->connection);
+ 
+   /* wrong, but backward compatible */
+-  g_socket_set_blocking (data->socket, TRUE);
++  g_socket_set_blocking (attempt->socket, TRUE);
+ 
+-  if (!data->proxy_addr)
+-    {
+-      g_socket_client_tls_handshake (data);
+-      return;
+-    }
+-
+-  protocol = g_proxy_address_get_protocol (data->proxy_addr);
+-
+-  /* The connection should not be anything other than TCP,
+-   * but let's put a safety guard in case
++  /* This ends the parallel "happy eyeballs" portion of connecting.
++     Now that we have a successful tcp connection we will attempt to connect
++     at the TLS/Proxy layer. If those layers fail we will move on to the next
++     connection.
+    */
+-  if (!G_IS_TCP_CONNECTION (data->connection))
+-    {
+-      g_critical ("Trying to proxy over non-TCP connection, this is "
+-          "most likely a bug in GLib IO library.");
+-
+-      g_set_error_literal (&data->last_error,
+-          G_IO_ERROR, G_IO_ERROR_NOT_SUPPORTED,
+-          _("Proxying over a non-TCP connection is not supported."));
+-
+-      enumerator_next_async (data, FALSE);
+-    }
+-  else if (g_hash_table_contains (data->client->priv->app_proxies, protocol))
+-    {
+-      /* Simply complete the connection, we don't want to do TLS handshake
+-       * as the application proxy handling may need proxy handshake first */
+-      g_socket_client_async_connect_complete (data);
+-    }
+-  else if ((proxy = g_proxy_get_default_for_protocol (protocol)))
+-    {
+-      g_socket_client_emit_event (data->client, G_SOCKET_CLIENT_PROXY_NEGOTIATING, data->connectable, data->connection);
+-      g_proxy_connect_async (proxy,
+-                             data->connection,
+-                             data->proxy_addr,
+-                             g_task_get_cancellable (data->task),
+-                             g_socket_client_proxy_connect_callback,
+-                             data);
+-      g_object_unref (proxy);
+-    }
+-  else
+-    {
+-      g_clear_error (&data->last_error);
+-
+-      g_set_error (&data->last_error, G_IO_ERROR, G_IO_ERROR_NOT_SUPPORTED,
+-          _("Proxy protocol “%s” is not supported."),
+-          protocol);
+-
+-      enumerator_next_async (data, FALSE);
+-    }
++  connection_attempt_remove (attempt);
++  data->successful_connections = g_slist_append (data->successful_connections, g_steal_pointer (&attempt));
++  try_next_connection_or_finish (data, FALSE);
+ }
+ 
+ static gboolean
+@@ -1702,7 +1791,11 @@ on_connection_attempt_timeout (gpointer data)
+ {
+   ConnectionAttempt *attempt = data;
+ 
+-  enumerator_next_async (attempt->data, TRUE);
++  if (!attempt->data->enumeration_completed)
++    {
++      g_debug ("GSocketClient: Timeout reached, trying another enumeration");
++      enumerator_next_async (attempt->data, TRUE);
++    }
+ 
+   g_clear_pointer (&attempt->timeout_source, g_source_unref);
+   return G_SOURCE_REMOVE;
+@@ -1712,9 +1805,9 @@ static void
+ on_connection_cancelled (GCancellable *cancellable,
+                          gpointer      data)
+ {
+-  GCancellable *attempt_cancellable = data;
++  GCancellable *linked_cancellable = G_CANCELLABLE (data);
+ 
+-  g_cancellable_cancel (attempt_cancellable);
++  g_cancellable_cancel (linked_cancellable);
+ }
+ 
+ static void
+@@ -1738,39 +1831,49 @@ g_socket_client_enumerator_callback (GObject      *object,
+ 						     result, &error);
+   if (address == NULL)
+     {
+-      if (data->connection_attempts)
++      if (G_UNLIKELY (data->enumeration_completed))
++        return;
++
++      data->enumeration_completed = TRUE;
++      g_debug ("GSocketClient: Address enumeration completed (out of addresses)");
++
++      /* As per API docs: We only care about error if its the first call,
++         after that the enumerator is done.
++
++         Note that we don't care about cancellation errors because
++         task_completed_or_cancelled() above should handle that.
++
++         If this fails and nothing is in progress then we will complete task here.
++       */
++      if ((data->enumerated_at_least_once && !data->connection_attempts && !data->connection_in_progress) ||
++          !data->enumerated_at_least_once)
+         {
+-          g_object_unref (data->task);
+-          return;
++          g_debug ("GSocketClient: Address enumeration failed: %s", error ? error->message : NULL);
++          if (data->last_error)
++            {
++              g_clear_error (&error);
++              error = data->last_error;
++              data->last_error = NULL;
++            }
++          else if (!error)
++            {
++              g_set_error_literal (&error, G_IO_ERROR, G_IO_ERROR_FAILED,
++                _("Unknown error on connect"));
++            }
++
++          complete_connection_with_error (data, error);
+         }
+ 
+-      g_socket_client_emit_event (data->client, G_SOCKET_CLIENT_COMPLETE, data->connectable, NULL);
+-      data->completed = TRUE;
+-      if (!error)
+-	{
+-	  if (data->last_error)
+-	    {
+-	      error = data->last_error;
+-	      data->last_error = NULL;
+-	    }
+-	  else
+-	    {
+-	      g_set_error_literal (&error, G_IO_ERROR, G_IO_ERROR_FAILED,
+-				   _("Unknown error on connect"));
+-	    }
+-	}
+-      g_task_return_error (data->task, error);
++      /* Enumeration should never trigger again, drop our ref */
+       g_object_unref (data->task);
+       return;
+     }
+ 
++  data->enumerated_at_least_once = TRUE;
++  g_debug ("GSocketClient: Address enumeration succeeded");
+   g_socket_client_emit_event (data->client, G_SOCKET_CLIENT_RESOLVED,
+ 			      data->connectable, NULL);
+ 
+-  if (G_IS_PROXY_ADDRESS (address) &&
+-      data->client->priv->enable_proxy)
+-    data->proxy_addr = g_object_ref (G_PROXY_ADDRESS (address));
+-
+   g_clear_error (&data->last_error);
+ 
+   socket = create_socket (data->client, address, &data->last_error);
+@@ -1788,6 +1891,10 @@ g_socket_client_enumerator_callback (GObject      *object,
+   attempt->cancellable = g_cancellable_new ();
+   attempt->connection = (GIOStream *)g_socket_connection_factory_create_connection (socket);
+   attempt->timeout_source = g_timeout_source_new (HAPPY_EYEBALLS_CONNECTION_ATTEMPT_TIMEOUT_MS);
++
++  if (G_IS_PROXY_ADDRESS (address) && data->client->priv->enable_proxy)
++    attempt->proxy_addr = g_object_ref (G_PROXY_ADDRESS (address));
++
+   g_source_set_callback (attempt->timeout_source, on_connection_attempt_timeout, attempt, NULL);
+   g_source_attach (attempt->timeout_source, g_main_context_get_thread_default ());
+   data->connection_attempts = g_slist_append (data->connection_attempts, attempt);
+@@ -1797,6 +1904,7 @@ g_socket_client_enumerator_callback (GObject      *object,
+                            g_object_ref (attempt->cancellable), g_object_unref);
+ 
+   g_socket_connection_set_cached_remote_address ((GSocketConnection *)attempt->connection, address);
++  g_debug ("GSocketClient: Starting TCP connection attempt");
+   g_socket_client_emit_event (data->client, G_SOCKET_CLIENT_CONNECTING, data->connectable, attempt->connection);
+   g_socket_connection_connect_async (G_SOCKET_CONNECTION (attempt->connection),
+ 				     address,
+@@ -1849,24 +1957,48 @@ g_socket_client_connect_async (GSocketClient       *client,
+   else
+     data->enumerator = g_socket_connectable_enumerate (connectable);
+ 
+-  /* The flow and ownership here isn't quite obvious:
+-    - The task starts an async attempt to connect.
+-      - Each attempt holds a single ref on task.
+-      - Each attempt may create new attempts by timing out (not a failure) so
+-        there are multiple attempts happening in parallel.
+-      - Upon failure an attempt will start a new attempt that steals its ref
+-        until there are no more attempts left and it drops its ref.
+-      - Upon success it will cancel all other attempts and continue on
+-        to the rest of the connection (tls, proxies, etc) which do not
+-        happen in parallel and at the very end drop its ref.
+-      - Upon cancellation an attempt drops its ref.
+-   */
++  /* This function tries to match the behavior of g_socket_client_connect ()
++     which is simple enough but much of it is done in parallel to be as responsive
++     as possible as per Happy Eyeballs (RFC 8305). This complicates flow quite a
++     bit but we can describe it in 3 sections:
++
++     Firstly we have address enumeration (DNS):
++       - This may be triggered multiple times by enumerator_next_async().
++       - It also has its own cancellable (data->enumeration_cancellable).
++       - Enumeration is done lazily because GNetworkAddressAddressEnumerator
++         also does work in parallel and may lazily add new addresses.
++       - If the first enumeration errors then the task errors. Otherwise all enumerations
++         will potentially be used (until task or enumeration is cancelled).
++
++      Then we start attempting connections (TCP):
++        - Each connection is independent and kept in a ConnectionAttempt object.
++          - They each hold a ref on the main task and have their own cancellable.
++        - Multiple attempts may happen in parallel as per Happy Eyeballs.
++        - Upon failure or timeouts more connection attempts are made.
++          - If no connections succeed the task errors.
++        - Upon success they are kept in a list of successful connections.
++
++      Lastly we connect at the application layer (TLS, Proxies):
++        - These are done in serial.
++          - The reasoning here is that Happy Eyeballs is about making bad connections responsive
++            at the IP/TCP layers. Issues at the application layer are generally not due to
++            connectivity issues but rather misconfiguration.
++        - Upon failure it will try the next TCP connection until it runs out and
++          the task errors.
++        - Upon success it cancels everything remaining (enumeration and connections)
++          and returns the connection.
++  */
+ 
+   data->task = g_task_new (client, cancellable, callback, user_data);
+   g_task_set_check_cancellable (data->task, FALSE); /* We handle this manually */
+   g_task_set_source_tag (data->task, g_socket_client_connect_async);
+   g_task_set_task_data (data->task, data, (GDestroyNotify)g_socket_client_async_connect_data_free);
+ 
++  data->enumeration_cancellable = g_cancellable_new ();
++  if (cancellable)
++    g_cancellable_connect (cancellable, G_CALLBACK (on_connection_cancelled),
++                           g_object_ref (data->enumeration_cancellable), g_object_unref);
++
+   enumerator_next_async (data, FALSE);
+ }
+ 
+@@ -1985,6 +2117,7 @@ g_socket_client_connect_to_uri_async (GSocketClient        *client,
+     }
+   else
+     {
++      g_debug("g_socket_client_connect_to_uri_async");
+       g_socket_client_connect_async (client,
+ 				     connectable, cancellable,
+ 				     callback, user_data);
+-- 
+2.23.0
+
meta/recipes-core/glib-2.0/glib-2.0_2.60.7.bb +1 -0
index 5aefa6ad8b..5be81a8f31 100644
@@ -16,6 +16,7 @@ SRC_URI = "${GNOME_MIRROR}/glib/${SHRT_VER}/glib-${PV}.tar.xz \
            file://0001-Do-not-write-bindir-into-pkg-config-files.patch \
            file://0001-meson.build-do-not-hardcode-linux-as-the-host-system.patch \
            file://0001-meson-do-a-build-time-check-for-strlcpy-before-attem.patch \
+           file://CVE-2020-6750.patch \
            "
 
 SRC_URI_append_class-native = " file://relocate-modules.patch"
Signature · 1 line
2.25.1
#5 lore raw

PATCH 5/13 busybox: on upgrade save busybox if it is the last shell

In reply to Anuj Mittal

From: Jeremy Puhlman <[email protected]>

During a busybox upgrade on a ipk based system, it is possible that busybox is the only shell in the system. During the uninstall the alternative for /bin/sh is removed and everything after that goes down hill.

* Add a check to verify if busybox is the shell, and save it to the busyboxrm directory created in tmp. Then add an alternative for /bin/sh that points to that busybox at the lowest priority.

* Add PATH to the busyboxrm directory using shell(as during an upgrade busybox and its links are missing).

* When install over remove extra busybox if present.

deb and rpm are uneffected by the bug because they both drag in bash, however neither upgrade seemed to have issue with the changes.

[YOCTO 13850]

(cherry picked from commit a9d2af8f5b3da8239cf00a52883ca596a19ea23a)
---
 meta/recipes-core/busybox/busybox.inc | 43 +++++++++++++++++++++++++++
 1 file changed, 43 insertions(+)
meta/recipes-core/busybox/busybox.inc +43 -0
index bf6ddae7d1..33c84bc2c1 100644
@@ -431,6 +431,32 @@ fi
     d.prependVar('pkg_postinst_%s' % pkg, postinst)
 }
 
+pkg_postinst_${PN}_prepend () {
+        # Need path to saved utils, but they may have be removed on upgrade of busybox
+        # Only use shell to get paths. Also capture if busybox was saved.
+        BUSYBOX=""
+        if [ "x$D" = "x" ] ; then 
+           for busybox_rmdir in /tmp/busyboxrm-*; do
+               if [ "$busybox_rmdir" != '/tmp/busyboxrm-*' ] ; then
+                  export PATH=$busybox_rmdir:$PATH
+                  if [ -e $busybox_rmdir/busybox* ] ; then
+                    BUSYBOX="$busybox_rmdir/busybox*"
+                  fi
+               fi
+           done
+        fi
+}
+
+pkg_postinst_${PN}_append () {
+        # If busybox exists in the remove directory it is because it was the only shell left.
+        if [ "x$D" = "x" ] ; then
+           if [ "x$BUSYBOX" != "x" ] ; then
+              update-alternatives --remove sh $BUSYBOX
+              rm -f $BUSYBOX
+           fi
+        fi
+} 
+
 pkg_prerm_${PN} () {
 	# This is so you can make busybox commit suicide - removing busybox with no other packages
 	# providing its files, this will make update-alternatives work, but the update-rc.d part
@@ -451,9 +477,26 @@ pkg_prerm_${PN} () {
 	ln -s ${base_bindir}/busybox $tmpdir/grep
 	ln -s ${base_bindir}/busybox $tmpdir/tail
 	export PATH=$PATH:$tmpdir
+
+        # If busybox is the shell, we need to save it since its the lowest priority shell
+        # Register saved bitbake as the lowest priority shell possible as back up.
+        if [ -n "$(readlink -f /bin/sh | grep busybox)" ] ; then
+           BUSYBOX=$(readlink -f /bin/sh)
+           cp $BUSYBOX $tmpdir/$(basename $BUSYBOX)
+           update-alternatives --install /bin/sh sh $tmpdir/$(basename $BUSYBOX) 1 
+        fi
 }
 
 pkg_postrm_${PN} () {
+        # Add path to remove dir in case we removed our only grep
+        if [ "x$D" = "x" ] ; then
+           for busybox_rmdir in /tmp/busyboxrm-*; do
+               if [ "$busybox_rmdir" != '/tmp/busyboxrm-*' ] ; then
+                  export PATH=$busybox_rmdir:$PATH
+               fi
+           done
+        fi
+
 	if grep -q "^${base_bindir}/bash$" $D${sysconfdir}/busybox.links* && [ ! -e $D${base_bindir}/bash ]; then
 		printf "$(grep -v "^${base_bindir}/bash$" $D${sysconfdir}/shells)\n" > $D${sysconfdir}/shells
 	fi
Signature · 1 line
2.25.1
#6 lore raw

PATCH 6/13 lib/oe/package_manager: make sure to not remove packages in apt install

In reply to Anuj Mittal

From: Jan Luebbe <[email protected]>

apt install can decide to remove already installed packages if there are conflicts. Avoid this by explicitly specifying --no-remove. This will then cause a "E: Packages need to be removed but remove is disabled." message.

(cherry picked from commit 9605a488b55042add012e9aeef13ab3f4e70e6e5)
---
 meta/lib/oe/package_manager.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/lib/oe/package_manager.py +1 -1
index 7c373715ad..61206daeaa 100644
@@ -1619,7 +1619,7 @@ class DpkgPM(OpkgDpkgPM):
 
         os.environ['APT_CONFIG'] = self.apt_conf_file
 
-        cmd = "%s %s install --force-yes --allow-unauthenticated %s" % \
+        cmd = "%s %s install --force-yes --allow-unauthenticated --no-remove %s" % \
               (self.apt_get_cmd, self.apt_args, ' '.join(pkgs))
 
         try:
Signature · 1 line
2.25.1
#7 lore raw

PATCH 7/13 lib/oe/package_manager: fix handling of last package

In reply to Anuj Mittal

From: Jan Luebbe <[email protected]>

In commit 7d214b34e11dc57316ed5c1c7747c4601286f6d2, only the code in the for loop was modified to store the pkgarch value. The code used if there was no empty line at the end was not modified.

Instead of fixing the duplicated code, remove it and just make sure that a final empty line is processed.

(cherry picked from commit a7b93c695b23d015607b179d98526b9b14c03d45)
---
 meta/lib/oe/package_manager.py | 8 +-------
 1 file changed, 1 insertion(+), 7 deletions(-)
meta/lib/oe/package_manager.py +1 -7
index 61206daeaa..27b2f17f95 100644
@@ -41,7 +41,7 @@ def opkg_query(cmd_output):
     filename = ""
     dep = []
     pkgarch = ""
-    for line in cmd_output.splitlines():
+    for line in cmd_output.splitlines()+['']:
         line = line.rstrip()
         if ':' in line:
             if line.startswith("Package: "):
@@ -80,12 +80,6 @@ def opkg_query(cmd_output):
             dep = []
             pkgarch = ""
 
-    if pkg:
-        if not filename:
-            filename = "%s_%s_%s.ipk" % (pkg, ver, arch)
-        output[pkg] = {"arch":arch, "ver":ver,
-                "filename":filename, "deps": dep }
-
     return output
 
 def failed_postinsts_abort(pkgs, log_path):
Signature · 1 line
2.25.1
#8 lore raw

PATCH 8/13 lib/oe/package_manager: collect provided package names when using debs

In reply to Anuj Mittal

From: Jan Luebbe <[email protected]>

This is needed for a later change to avoid installing packages which are already provided by an installed package.

(cherry picked from commit ad72dfaaa2caf2c39d033dc1682f0bbbbe45dbbd)
---
 meta/lib/oe/package_manager.py | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)
meta/lib/oe/package_manager.py +8 -2
index 27b2f17f95..e6319dc3b8 100644
@@ -40,6 +40,7 @@ def opkg_query(cmd_output):
     ver = ""
     filename = ""
     dep = []
+    prov = []
     pkgarch = ""
     for line in cmd_output.splitlines()+['']:
         line = line.rstrip()
@@ -64,6 +65,10 @@ def opkg_query(cmd_output):
                     dep.append("%s [REC]" % recommend)
             elif line.startswith("PackageArch: "):
                 pkgarch = line.split(": ")[1]
+            elif line.startswith("Provides: "):
+                provides = verregex.sub('', line.split(": ")[1])
+                for provide in provides.split(", "):
+                    prov.append(provide)
 
         # When there is a blank line save the package information
         elif not line:
@@ -72,12 +77,13 @@ def opkg_query(cmd_output):
                 filename = "%s_%s_%s.ipk" % (pkg, ver, arch)
             if pkg:
                 output[pkg] = {"arch":arch, "ver":ver,
-                        "filename":filename, "deps": dep, "pkgarch":pkgarch }
+                        "filename":filename, "deps": dep, "pkgarch":pkgarch, "provs": prov}
             pkg = ""
             arch = ""
             ver = ""
             filename = ""
             dep = []
+            prov = []
             pkgarch = ""
 
     return output
@@ -354,7 +360,7 @@ class DpkgPkgsList(PkgsList):
                "--admindir=%s/var/lib/dpkg" % self.rootfs_dir,
                "-W"]
 
-        cmd.append("-f=Package: ${Package}\nArchitecture: ${PackageArch}\nVersion: ${Version}\nFile: ${Package}_${Version}_${Architecture}.deb\nDepends: ${Depends}\nRecommends: ${Recommends}\n\n")
+        cmd.append("-f=Package: ${Package}\nArchitecture: ${PackageArch}\nVersion: ${Version}\nFile: ${Package}_${Version}_${Architecture}.deb\nDepends: ${Depends}\nRecommends: ${Recommends}\nProvides: ${Provides}\n\n")
 
         try:
             cmd_output = subprocess.check_output(cmd, stderr=subprocess.STDOUT).strip().decode("utf-8")
Signature · 1 line
2.25.1
#9 lore raw

PATCH 9/13 lib/oe/package_manager: avoid installing provided packages via apt

In reply to Anuj Mittal

From: Jan Luebbe <[email protected]>

If there already is a package providing (and conflicting against) packages what should be installed, apt will try remove the conflicting package (target-sdk-provides-dummy) and any that depend on it (like apt and dpkg). This usually fails because of the protection of essential packages. In that case, no -dev/-dbg packages are installed to the SDK.

Avoid this problem by checking which packages are already provided and removing them from the list to be installed. Also sort the list to make it easier to read when debugging.

(cherry picked from commit 3ffb339dd55f8ca7c952fd3390608510f772e19f)
---
 meta/lib/oe/package_manager.py | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)
meta/lib/oe/package_manager.py +12 -2
index e6319dc3b8..4d9e501f0e 100644
@@ -578,6 +578,11 @@ class PackageManager(object, metaclass=ABCMeta):
         # oe-pkgdata-util reads it from a file
         with tempfile.NamedTemporaryFile(mode="w+", prefix="installed-pkgs") as installed_pkgs:
             pkgs = self.list_installed()
+
+            provided_pkgs = set()
+            for pkg in pkgs.values():
+                provided_pkgs |= set(pkg.get('provs', []))
+
             output = oe.utils.format_pkg_list(pkgs, "arch")
             installed_pkgs.write(output)
             installed_pkgs.flush()
@@ -589,10 +594,15 @@ class PackageManager(object, metaclass=ABCMeta):
             if exclude:
                 cmd.extend(['--exclude=' + '|'.join(exclude.split())])
             try:
-                bb.note("Installing complementary packages ...")
                 bb.note('Running %s' % cmd)
                 complementary_pkgs = subprocess.check_output(cmd, stderr=subprocess.STDOUT).decode("utf-8")
-                self.install(complementary_pkgs.split(), attempt_only=True)
+                complementary_pkgs = set(complementary_pkgs.split())
+                skip_pkgs = sorted(complementary_pkgs & provided_pkgs)
+                install_pkgs = sorted(complementary_pkgs - provided_pkgs)
+                bb.note("Installing complementary packages ... %s (skipped already provided packages %s)" % (
+                    ' '.join(install_pkgs),
+                    ' '.join(skip_pkgs)))
+                self.install(install_pkgs, attempt_only=True)
             except subprocess.CalledProcessError as e:
                 bb.fatal("Could not compute complementary packages list. Command "
                          "'%s' returned %d:\n%s" %
Signature · 1 line
2.25.1
#10 lore raw

PATCH 10/13 lib/oe/package_manager: don't try to rm /var/lib/opkg

In reply to Anuj Mittal

From: Jan Luebbe <[email protected]>

As opkglibdir starts with a /, os.path.join will ignore self.target_rootfs, leading to an attempt to remove /var/lib/opkg.

This only fails if it exists on the host, explaining why this remained undiscovered for long.

(cherry picked from commit fc974977cea389f54e7fc7de7b1c8fd3d8bafe58)
---
 meta/lib/oe/package_manager.py | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
meta/lib/oe/package_manager.py +1 -2
index 4d9e501f0e..e0b15dc9b4 100644
@@ -1791,8 +1791,7 @@ class DpkgPM(OpkgDpkgPM):
             open(os.path.join(target_dpkg_dir, "available"), "w+").close()
 
     def remove_packaging_data(self):
-        bb.utils.remove(os.path.join(self.target_rootfs,
-                                     self.d.getVar('opkglibdir')), True)
+        bb.utils.remove(self.target_rootfs + self.d.getVar('opkglibdir'), True)
         bb.utils.remove(self.target_rootfs + "/var/lib/dpkg/", True)
 
     def fix_broken_dependencies(self):
Signature · 1 line
2.25.1
#11 lore raw

PATCH 11/13 kernel-yocto.bbclass: Support config fragments with externalsrc

In reply to Anuj Mittal

From: Paul Barker <[email protected]>

The merging of config fragments is performend in the do_kernel_configme task and so config fragments will not be supported when this task is removed from the dependency tree.

kernel-yocto adds additional tasks which may modify the source directory to SRCTREECOVEREDTASKS so that they are removed when using externalsrc. However, do_kernel_configme should be safe to use, the only modification to the source tree is the potential creation of the '.kernel-meta' directory and the '.metadir' file.

(cherry picked from commit 44f04c039a4d61dd18666e42b9b9865cbc3ada9e)
---
 meta/classes/kernel-yocto.bbclass | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
meta/classes/kernel-yocto.bbclass +2 -1
index ed9bcfa57c..ab05ac91f4 100644
@@ -1,5 +1,5 @@
 # remove tasks that modify the source tree in case externalsrc is inherited
-SRCTREECOVEREDTASKS += "do_kernel_configme do_validate_branches do_kernel_configcheck do_kernel_checkout do_fetch do_unpack do_patch"
+SRCTREECOVEREDTASKS += "do_validate_branches do_kernel_configcheck do_kernel_checkout do_fetch do_unpack do_patch"
 PATCH_GIT_USER_EMAIL ?= "kernel-yocto@oe"
 PATCH_GIT_USER_NAME ?= "OpenEmbedded"
 
@@ -301,6 +301,7 @@ do_validate_branches[depends] = "kern-tools-native:do_populate_sysroot"
 do_kernel_configme[depends] += "virtual/${TARGET_PREFIX}binutils:do_populate_sysroot"
 do_kernel_configme[depends] += "virtual/${TARGET_PREFIX}gcc:do_populate_sysroot"
 do_kernel_configme[depends] += "bc-native:do_populate_sysroot bison-native:do_populate_sysroot"
+do_kernel_configme[depends] += "kern-tools-native:do_populate_sysroot"
 do_kernel_configme[dirs] += "${S} ${B}"
 do_kernel_configme() {
 	set +e
Signature · 1 line
2.25.1
#12 lore raw

PATCH 12/13 perf: Fix externalsrc support

In reply to Anuj Mittal

From: Paul Barker <[email protected]>

When the externalsrc class is used the tasks listed in SRCTREECOVEREDTASKS are deleted to prevent them being executed. If externalsrc is used for the kernel then this will include virtual/kernel:do_patch.

We can depend on do_shared_workdir instead as this will survive when externalsrc is used.

(cherry picked from commit eab605bab94552046ec4adae5debe026cc03bb4c)
---
 meta/recipes-kernel/perf/perf.bb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/recipes-kernel/perf/perf.bb +1 -1
index 8201c0cb60..90f05c0e62 100644
@@ -51,7 +51,7 @@ export PYTHON_SITEPACKAGES_DIR
 #kernel 3.1+ supports WERROR to disable warnings as errors
 export WERROR = "0"
 
-do_populate_lic[depends] += "virtual/kernel:do_patch"
+do_populate_lic[depends] += "virtual/kernel:do_shared_workdir"
 
 # needed for building the tools/perf Perl binding
 include ${@bb.utils.contains('PACKAGECONFIG', 'scripting', 'perf-perl.inc', '', d)}
Signature · 1 line
2.25.1
#13 lore raw

PATCH 13/13 kernelsrc.bbclass: Fix externalsrc support

In reply to Anuj Mittal

From: Paul Barker <[email protected]>

When the externalsrc class is used the tasks listed in SRCTREECOVEREDTASKS are deleted to prevent them being executed. If externalsrc is used for the kernel then this will include virtual/kernel:do_patch.

We can depend on do_shared_workdir instead as this will survive when externalsrc is used.

(cherry picked from commit 2c17d35cc7b9c5e01fd5829858d2f0234e7ac8d6)
---
 meta/classes/kernelsrc.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/classes/kernelsrc.bbclass +1 -1
index 675d40ec9a..a951ba3325 100644
@@ -1,7 +1,7 @@
 S = "${STAGING_KERNEL_DIR}"
 deltask do_fetch
 deltask do_unpack
-do_patch[depends] += "virtual/kernel:do_patch"
+do_patch[depends] += "virtual/kernel:do_shared_workdir"
 do_patch[noexec] = "1"
 do_package[depends] += "virtual/kernel:do_populate_sysroot"
 KERNEL_VERSION = "${@get_kernelversion_file("${STAGING_KERNEL_BUILDDIR}")}"
Signature · 1 line
2.25.1
#14 lore raw

✗ patchtest: failure for zeus -next review (rev4)

In reply to Anuj Mittal

== Series Details ==

Series: zeus -next review (rev4)
Revision: 4
URL : https://patchwork.openembedded.org/series/21451/
State : failure

== Summary ==

Thank you for submitting this patch series to OpenEmbedded Core. This is an automated response. Several tests have been executed on the proposed series by patchtest resulting in the following failures:

* Patch            [zeus, 05/13] busybox: on upgrade save busybox if it is the last shell
 Issue             Yocto Project bugzilla tag is not correctly formatted [test_bugzilla_entry_format] 
  Suggested fix    Specify bugzilla ID in commit description with format: "[YOCTO #<bugzilla ID>]"

If you believe any of these test results are incorrect, please reply to the mailing list ([email protected]) raising your concerns. Otherwise we would appreciate you correcting the issues and submitting a new version of the patchset if applicable. Please ensure you add/increment the version number when sending the new version (i.e. [PATCH] -> [PATCH v2] -> [PATCH v3] -> ...).

---

Guidelines:     https://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines
Test framework: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest
Test suite:     http://git.yoctoproject.org/cgit/cgit.cgi/patchtest-oe