Skip to contentSkip to conversation

Patch review

PATCH In review Steve Sakoman 2 participants 12 messages May 2, 2022 – May 3, 2022

lore
Share & tools
lore
#0 lore raw

cover Patch review

Please review this set of patches for dunfell and have comments back by end of day Wednesday.

Passed a-full on autobuilder:

https://autobuilder.yoctoproject.org/typhoon/#/builders/83/builds/3600

with the exception of the meta-virtualization test which was just added to a-full:

https://autobuilder.yoctoproject.org/typhoon/#/builders/128/builds/19

Note that the test passed for qemuarm and qemuarm64, but failed for qemux86-64.

I tried to refrain from commenting that the test was added by someone with an arm.com address, but I couldn't help myself ;-) (looking at you Ross!)

I'm not going to hold up the review process on this, since this is a newly added test.

Any help fixing this for qemux86-64 would be much appreciated.

Steve

The following changes since commit bb3fc61f0d7f7bcd77ef194b76f4fdd8a7ff6aa5:

scripts/contrib/oe-build-perf-report-email.py: remove obsolete check for phantomjs and optipng (2022-04-27 05:00:00 -1000)

are available in the Git repository at:

git://git.openembedded.org/openembedded-core-contrib stable/dunfell-nut
http://cgit.openembedded.org/openembedded-core-contrib/log/?h=stable/dunfell-nut

Chen Qi (1):
cases/buildepoxy.py: fix typo

Khem Raj (1):
busybox: Use base_bindir instead of hardcoding /bin path

Paul Gortmaker (1):
install/devshell: Introduce git intercept script due to fakeroot
issues

Peter Kjellerstedt (1):
devshell.bbclass: Allow devshell & pydevshell to use the network

Rahul Kumar (1):
neard: Switch SRC_URI to git repo

Richard Purdie (2):
base: Drop git intercept
uninative: Upgrade to 3.6 with gcc 12 support

Ross Burton (2):
python3: ignore CVE-2015-20107
bitbake.conf: mark all directories as safe for git to read

 meta/classes/devshell.bbclass                 |  4 ++++
 meta/conf/bitbake.conf                        |  8 ++++++++
 meta/conf/distro/include/yocto-uninative.inc  |  8 ++++----
 meta/lib/oeqa/sdk/cases/buildepoxy.py         |  2 +-
 meta/recipes-connectivity/neard/neard_0.16.bb | 13 +++++++------
 meta/recipes-core/busybox/busybox.inc         |  2 +-
 .../recipes-devtools/python/python3_3.8.13.bb |  3 +++
 scripts/git-intercept/git                     | 19 +++++++++++++++++++
 8 files changed, 47 insertions(+), 12 deletions(-)
 create mode 100755 scripts/git-intercept/git
Signature · 1 line
2.25.1
#1 lore raw

PATCH 1/9 python3: ignore CVE-2015-20107

From: Ross Burton <[email protected]>

CVE-2015-20107 describes an arbitrary command execution in the mailcap module, but this is by design in mailcap and needs to be worked around by the calling application.

Upstream Python will be documenting this flaw in the library reference, and it is likely that the mailcap module will be deprecated and removed in the future.

(cherry picked from commit 85fac8408baf92d8b71946f5bfea92952b7eab01)
---
 meta/recipes-devtools/python/python3_3.8.13.bb | 3 +++
 1 file changed, 3 insertions(+)
meta/recipes-devtools/python/python3_3.8.13.bb +3 -0
index d7f6e9155d..040bacf97c 100644
@@ -57,6 +57,9 @@ CVE_CHECK_WHITELIST += "CVE-2019-18348"
 
 # This is windows only issue.
 CVE_CHECK_WHITELIST += "CVE-2020-15523 CVE-2022-26488"
+# The mailcap module is insecure by design, so this can't be fixed in a meaningful way.
+# The module will be removed in the future and flaws documented.
+CVE_CHECK_WHITELIST += "CVE-2015-20107"
 
 PYTHON_MAJMIN = "3.8"
 
Signature · 1 line
2.25.1
#2 lore raw

PATCH 2/9 busybox: Use base_bindir instead of hardcoding /bin path

In reply to Steve Sakoman

From: Khem Raj <[email protected]>

This symlink is not valid when using usrmerge and ptest packaging would fail

Exception: FileExistsError: [Errno 17] File exists: '/usr/bin/busybox.suid' -> '/mnt/b/yoe/master/build/tmp/work/ppc64p9le-yoe-linux-musl/busybox/1.35.0-r0/package/usr/lib/busybox/ptest/bin/login'

(cherry picked from commit 238fd30689054c7b44176dce7180fb6dac4e1b6f)
---
 meta/recipes-core/busybox/busybox.inc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/recipes-core/busybox/busybox.inc +1 -1
index e0522be729..3553376582 100644
@@ -348,7 +348,7 @@ do_install_ptest () {
         # These access the internet which is not guaranteed to work on machines running the tests
         rm -rf ${D}${PTEST_PATH}/testsuite/wget
 	sort ${B}/.config > ${D}${PTEST_PATH}/.config
-	ln -s /bin/busybox   ${D}${PTEST_PATH}/busybox
+	ln -s ${base_bindir}/busybox   ${D}${PTEST_PATH}/busybox
 }
 
 inherit update-alternatives
Signature · 1 line
2.25.1
#3 lore raw

PATCH 3/9 devshell.bbclass: Allow devshell & pydevshell to use the network

In reply to Steve Sakoman

From: Peter Kjellerstedt <[email protected]>

Otherwise it will fail if using OE_TERMINAL = "xterm" with the not so helpful error:

xterm: Xt error: Can't open display: localhost:0.0

(cherry picked from commit ba53fc3bcecfe32401471dc1008c7ead96504150)
---
 meta/classes/devshell.bbclass | 2 ++
 1 file changed, 2 insertions(+)
meta/classes/devshell.bbclass +2 -0
index 76dd0b42ee..ad9f267848 100644
@@ -21,6 +21,7 @@ addtask devshell after do_patch do_prepare_recipe_sysroot
 DEVSHELL_STARTDIR ?= "${S}"
 do_devshell[dirs] = "${DEVSHELL_STARTDIR}"
 do_devshell[nostamp] = "1"
+do_devshell[network] = "1"
 
 # devshell and fakeroot/pseudo need careful handling since only the final
 # command should run under fakeroot emulation, any X connection should
@@ -154,3 +155,4 @@ python do_devpyshell() {
 addtask devpyshell after do_patch
 
 do_devpyshell[nostamp] = "1"
+do_devpyshell[network] = "1"
Signature · 1 line
2.25.1
#6 lore raw

PATCH 4/9 cases/buildepoxy.py: fix typo

In reply to Steve Sakoman

From: Chen Qi <[email protected]>

(cherry picked from commit 3a9b6e71d1e7e8e2ebc0ed047841e36f09300387)
---
 meta/lib/oeqa/sdk/cases/buildepoxy.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/lib/oeqa/sdk/cases/buildepoxy.py +1 -1
index 385f8ccca8..f69f720cd6 100644
@@ -17,7 +17,7 @@ class EpoxyTest(OESDKTestCase):
     """
     def setUp(self):
         if not (self.tc.hasHostPackage("nativesdk-meson")):
-            raise unittest.SkipTest("GalculatorTest class: SDK doesn't contain Meson")
+            raise unittest.SkipTest("EpoxyTest class: SDK doesn't contain Meson")
 
     def test_epoxy(self):
         with tempfile.TemporaryDirectory(prefix="epoxy", dir=self.tc.sdk_dir) as testdir:
Signature · 1 line
2.25.1
#7 lore raw

PATCH 5/9 install/devshell: Introduce git intercept script due to fakeroot issues

In reply to Steve Sakoman

From: Paul Gortmaker <[email protected]>

In a devshell, recent versions of git will complain if the repo is owned by someone other than the current UID - consider this example:

------
bitbake -c devshell linux-yocto

[...]

kernel-source#git branch
fatal: unsafe repository ('/home/paul/poky/build-qemuarm64/tmp/work-shared/qemuarm64/kernel-source' is owned by someone else) To add an exception for this directory, call:

        git config --global --add safe.directory /home/paul/poky/build-qemuarm64/tmp/work-shared/qemuarm64/kernel-source

kernel-source#
------

Of course the devshell has UID zero and the "real" UID is for "paul" in this case. And so recent git versions complain.

As the whole purpose of the devshell is to invoke a shell where development can take place, having a non-functional git is clearly unacceptable.

Richard suggested we could use PSEUDO_UNLOAD=1 to evade this issue, and I suggested we probably will see other similar instances like this and should make use of PATH to intercept via devshell wrappers - conveniently we already have examples of this.

Here, we copy the existing "ar" example and tune it to the needs of git to combine Richard's suggestion and mine.

As such we now also can store commit logs and use send-email with our user specific settings, instead of "root", so in additon to fixing basic commands like "git branch" it should also increase general usefulness.

RP: Tweaked the patch so the PATH change only applies to the devshell task and is a generic git intercept rather than devshell specific.

RP: Also apply the PATH change to do_install tasks since that also runs under fakeroot and several software projects inject "git describe" output into their binaries (systemd, iputils, llvm, ipt-gpu-tools at least) causing reproducibility issues from systems with different git versions.

(cherry picked from commit 3266c327dfa186791e0f1e2ad63c6f5d39714814)
---
 meta/classes/base.bbclass     |  1 +
 meta/classes/devshell.bbclass |  2 ++
 scripts/git-intercept/git     | 19 +++++++++++++++++++
 3 files changed, 22 insertions(+)
 create mode 100755 scripts/git-intercept/git
meta/classes/base.bbclass +1 -0
index 9ed736b0e1..398b098651 100644
@@ -335,6 +335,7 @@ addtask install after do_compile
 do_install[dirs] = "${B}"
 # Remove and re-create ${D} so that is it guaranteed to be empty
 do_install[cleandirs] = "${D}"
+PATH:prepend:task-install = "${COREBASE}/scripts/git-intercept:"
 
 base_do_install() {
 	:
meta/classes/devshell.bbclass +2 -0
index ad9f267848..114a50b20e 100644
@@ -2,6 +2,8 @@ inherit terminal
 
 DEVSHELL = "${SHELL}"
 
+PATH:prepend:task-devshell = "${COREBASE}/scripts/git-intercept:"
+
 python do_devshell () {
     if d.getVarFlag("do_devshell", "manualfakeroot"):
        d.prependVar("DEVSHELL", "pseudo ")
scripts/git-intercept/git new file +19 -0
new file mode 100755
index 0000000000..8adf5c9ecb
@@ -0,0 +1,19 @@
+#!/usr/bin/env python3
+#
+# Wrapper around 'git' that doesn't think we are root
+
+import os
+import shutil
+import sys
+
+os.environ['PSEUDO_UNLOAD'] = '1'
+
+# calculate path to the real 'git'
+path = os.environ['PATH']
+path = path.replace(os.path.dirname(sys.argv[0]), '')
+real_git = shutil.which('git', path=path)
+
+if len(sys.argv) == 1:
+    os.execl(real_git, 'git')
+
+os.execv(real_git, sys.argv)
Signature · 1 line
2.25.1
#8 lore raw

PATCH 6/9 base: Drop git intercept

In reply to Steve Sakoman

From: Richard Purdie <[email protected]>

We're going to use the environment approach for solving this issue.

(cherry picked from commit 0982977dc052ad4e65608f6853f930121d08837a)
---
 meta/classes/base.bbclass | 1 -
 1 file changed, 1 deletion(-)
meta/classes/base.bbclass +0 -1
index 398b098651..9ed736b0e1 100644
@@ -335,7 +335,6 @@ addtask install after do_compile
 do_install[dirs] = "${B}"
 # Remove and re-create ${D} so that is it guaranteed to be empty
 do_install[cleandirs] = "${D}"
-PATH:prepend:task-install = "${COREBASE}/scripts/git-intercept:"
 
 base_do_install() {
 	:
Signature · 1 line
2.25.1
#9 lore raw

PATCH 7/9 bitbake.conf: mark all directories as safe for git to read

In reply to Steve Sakoman

From: Ross Burton <[email protected]>

Recent git releases containing [1] have an ownership check when opening repositories, and refuse to open a repository if it is owned by a different user.

This breaks any use of git in do_install, as that is executed by the (fake) root user. Whilst not common, this does happen.

Setting the git configuration safe.directories=* disables this check, so that git is usable in fakeroot tasks. This can be set globally via the internal environment variable GIT_CONFIG_PARAMETERS, we can't use GIT_CONFIG_*_KEY/VALUE as that isn't present in all the releases which have the ownership check.

We already set GIT_CEILING_DIRECTORIES to ensure that git doesn't recurse up out of the work directory, so this isn't a security issue.

[1] https://github.com/git/git/commit/8959555cee7ec045958f9b6dd62e541affb7e7d9

(cherry picked from commit 8bed8e6993e7297bdcd68940aa0d47ef47120117)
---
 meta/conf/bitbake.conf | 8 ++++++++
 1 file changed, 8 insertions(+)
meta/conf/bitbake.conf +8 -0
index 91f003d6dd..2b94e37861 100644
@@ -726,10 +726,18 @@ export PKG_CONFIG_DISABLE_UNINSTALLED = "yes"
 export PKG_CONFIG_SYSTEM_LIBRARY_PATH = "${base_libdir}:${libdir}"
 export PKG_CONFIG_SYSTEM_INCLUDE_PATH = "${includedir}"
 
+# Git configuration
+
 # Don't allow git to chdir up past WORKDIR so that it doesn't detect the OE
 # repository when building a recipe
 export GIT_CEILING_DIRECTORIES = "${WORKDIR}"
 
+# Treat all directories are safe, as during fakeroot tasks git will run as
+# root so recent git releases (eg 2.30.3) will refuse to work on repositories. See
+# https://github.com/git/git/commit/8959555cee7ec045958f9b6dd62e541affb7e7d9 for
+# further details.
+export GIT_CONFIG_PARAMETERS="'safe.directory=*'"
+
 ###
 ### Config file processing
 ###
Signature · 1 line
2.25.1
#10 lore raw

PATCH 8/9 neard: Switch SRC_URI to git repo

In reply to Steve Sakoman

From: Rahul Kumar <[email protected]>

The tarball (neard-0.16.tar.xz) fetched by the recipe is incomplete. Few plugins (e.g. tizen) and tests scripts (e.g. Test-channel, test-see, neard-ui.py, ndef-agent etc) are missing.

Since neard did not release latest tarballs, so as per community recommendation switching the recipe SRC_URI to git repo.

Community Discussion:
https://lists.openembedded.org/g/openembedded-core/topic/90058043#163681

(cherry-picked from b563f40ebf4461d9c35df72bd7599ea11e97da9c)

---
 meta/recipes-connectivity/neard/neard_0.16.bb | 13 +++++++------
 1 file changed, 7 insertions(+), 6 deletions(-)
meta/recipes-connectivity/neard/neard_0.16.bb +7 -6
index 7c124a3c0b..dd0742f792 100644
@@ -2,21 +2,22 @@ SUMMARY = "Linux NFC daemon"
 DESCRIPTION = "A daemon for the Linux Near Field Communication stack"
 HOMEPAGE = "http://01.org/linux-nfc"
 LICENSE = "GPLv2"
+LIC_FILES_CHKSUM = "file://COPYING;md5=12f884d2ae1ff87c09e5b7ccc2c4ca7e \
+                    file://src/near.h;beginline=1;endline=20;md5=358e4deefef251a4761e1ffacc965d13 \
+                   "
 
 DEPENDS = "dbus glib-2.0 libnl"
 
-SRC_URI = "${KERNELORG_MIRROR}/linux/network/nfc/${BP}.tar.xz \
+SRC_URI = "git://git.kernel.org/pub/scm/network/nfc/neard.git;protocol=git;branch=master \
            file://neard.in \
            file://Makefile.am-fix-parallel-issue.patch \
            file://Makefile.am-do-not-ship-version.h.patch \
            file://0001-Add-header-dependency-to-nciattach.o.patch \
           "
-SRC_URI[md5sum] = "5c691fb7872856dc0d909c298bc8cb41"
-SRC_URI[sha256sum] = "eae3b11c541a988ec11ca94b7deab01080cd5b58cfef3ced6ceac9b6e6e65b36"
 
-LIC_FILES_CHKSUM = "file://COPYING;md5=12f884d2ae1ff87c09e5b7ccc2c4ca7e \
- file://src/near.h;beginline=1;endline=20;md5=358e4deefef251a4761e1ffacc965d13 \
- "
+SRCREV = "949795024f7625420e93e288c56e194cb9a3e74a"
+
+S = "${WORKDIR}/git"
 
 inherit autotools pkgconfig systemd update-rc.d
 
Signature · 1 line
2.25.1
#11 lore raw

PATCH 9/9 uninative: Upgrade to 3.6 with gcc 12 support

In reply to Steve Sakoman

From: Richard Purdie <[email protected]>

There are reports of issues with the new libstdc++ from gcc 12. This upgrades to a gcc 12 version of uninative to allow builds on those systems. Gcc 12 isn't finalised so we may need to add a new version of this if/as appropriate when it is.

(cherry picked from commit e3da4da7e5da5bb9e1d360e2be2fdd5132e69320)
---
 meta/conf/distro/include/yocto-uninative.inc | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
meta/conf/distro/include/yocto-uninative.inc +4 -4
index bfe05ce1eb..411fe45a24 100644
@@ -7,9 +7,9 @@
 #
 
 UNINATIVE_MAXGLIBCVERSION = "2.35"
-UNINATIVE_VERSION = "3.5"
+UNINATIVE_VERSION = "3.6"
 
 UNINATIVE_URL ?= "http://downloads.yoctoproject.org/releases/uninative/${UNINATIVE_VERSION}/"
-UNINATIVE_CHECKSUM[aarch64] ?= "6de0771bd21e0fcb5e80388e5b561a8023b24083bcbf46e056a089982aff75d7"
-UNINATIVE_CHECKSUM[i686] ?= "8c8745becbfa1c341bae839c7eab56ddf17ce36c303bcd73d3b2f2f788b631c2"
-UNINATIVE_CHECKSUM[x86_64] ?= "e8047a5748e6f266165da141eb6d08b23674f30e477b0e5505b6403d50fbc4b2"
+UNINATIVE_CHECKSUM[aarch64] ?= "d64831cf2792c8e470c2e42230660e1a8e5de56a579cdd59978791f663c2f3ed"
+UNINATIVE_CHECKSUM[i686] ?= "2f0ee9b66b1bb2c85e2b592fb3c9c7f5d77399fa638d74961330cdb8de34ca3b"
+UNINATIVE_CHECKSUM[x86_64] ?= "9bfc4c970495b3716b2f9e52c4df9f968c02463a9a95000f6657fbc3fde1f098"
Signature · 1 line
2.25.1
#4 lore raw

RE: [OE-core][dunfell 3/9] devshell.bbclass: Allow devshell & pydevshell to use the network

In reply to Steve Sakoman
Quoted 43 lines

-----Original Message-----
From: [email protected] <openembedded- [email protected]> On Behalf Of Steve Sakoman
Sent: den 3 maj 2022 01:03
To: [email protected]
Subject: [OE-core][dunfell 3/9] devshell.bbclass: Allow devshell & pydevshell to use the network

From: Peter Kjellerstedt <[email protected]>

Otherwise it will fail if using OE_TERMINAL = "xterm" with the not so helpful error:

xterm: Xt error: Can't open display: localhost:0.0

(cherry picked from commit ba53fc3bcecfe32401471dc1008c7ead96504150)
---
 meta/classes/devshell.bbclass | 2 ++
 1 file changed, 2 insertions(+)
meta/classes/devshell.bbclass +0 -0
index 76dd0b42ee..ad9f267848 100644
@@ -21,6 +21,7 @@ addtask devshell after do_patch

do_prepare_recipe_sysroot
DEVSHELL_STARTDIR ?= "${S}"
do_devshell[dirs] = "${DEVSHELL_STARTDIR}"
do_devshell[nostamp] = "1"

meta/classes/devshell.bbclass +2 -1
+do_devshell[network] = "1"
 
 # devshell and fakeroot/pseudo need careful handling since only the final
 # command should run under fakeroot emulation, any X connection should
@@ -154,3 +155,4 @@ python do_devpyshell() {
 addtask devpyshell after do_patch
 
 do_devpyshell[nostamp] = "1"
+do_devpyshell[network] = "1"
--
Quoted 1 line

2.25.1

This shouldn't be needed for Dunfell, should it? I would assume the support for blocking network operations per task hasn't been backported.

//Peter

#5 lore raw

Re: [OE-core][dunfell 3/9] devshell.bbclass: Allow devshell & pydevshell to use the network

Quoted 41 lines · Peter Kjellerstedt
Quoted 41 lines

-----Original Message-----
From: [email protected] <openembedded- [email protected]> On Behalf Of Steve Sakoman
Sent: den 3 maj 2022 01:03
To: [email protected]
Subject: [OE-core][dunfell 3/9] devshell.bbclass: Allow devshell & pydevshell to use the network

From: Peter Kjellerstedt <[email protected]>

Otherwise it will fail if using OE_TERMINAL = "xterm" with the not so helpful error:

xterm: Xt error: Can't open display: localhost:0.0

(cherry picked from commit ba53fc3bcecfe32401471dc1008c7ead96504150)
---
 meta/classes/devshell.bbclass | 2 ++
 1 file changed, 2 insertions(+)
meta/classes/devshell.bbclass +0 -0
index 76dd0b42ee..ad9f267848 100644
@@ -21,6 +21,7 @@ addtask devshell after do_patch

do_prepare_recipe_sysroot
DEVSHELL_STARTDIR ?= "${S}"
do_devshell[dirs] = "${DEVSHELL_STARTDIR}"
do_devshell[nostamp] = "1"

meta/classes/devshell.bbclass +1 -0
+do_devshell[network] = "1"
 
 # devshell and fakeroot/pseudo need careful handling since only the final
 # command should run under fakeroot emulation, any X connection should
@@ -154,3 +155,4 @@ python do_devpyshell() {
 addtask devpyshell after do_patch
 
Quoted 7 lines
Quoted 4 lines
meta/classes/devshell.bbclass +1 -1
 do_devpyshell[nostamp] = "1"
+do_devpyshell[network] = "1"
--

2.25.1

This shouldn't be needed for Dunfell, should it? I would assume the support for blocking network operations per task hasn't been backported.

Thanks so much for the review Peter!

Clearly I haven't yet perfected my mode switching between kirkstone and dunfell :-(

Steve