Skip to contentSkip to conversation

make: 4.2.1 -> 4.3

PATCH In review Jens Rehsack 4 participants 6 messages Feb 24, 2020 – Feb 29, 2020

lore
Share & tools
lore
#0 lore raw

PATCH v3 1/1 make: 4.2.1 -> 4.3

Announcement: https://lists.gnu.org/archive/html/bug-make/2020-01/msg00057.html

1) Remove upstream provided patches 0001-glob-Do-not-assume-glibc-glob-internals.patch and 0002-glob-Do-not-assume-glibc-glob-internals.patch.

2) License has been changed to GPLv3 only

3) Important bug-fix is
* https://lists.gnu.org/archive/html/bug-make/2018-09/msg00006.html

4) Backward-incompatibilities:
* Number signs (#) appearing inside a macro reference or function invocation no longer introduce comments and should not be escaped with backslashes
* Previously appending using '+=' to an empty variable would result in a value starting with a space. Now the initial space is only added if the variable already contains some value. Similarly, appending an empty string does not add a trailing space.

Fix incompatibility issues between gnulib bundled with updated make fix issues in w32 compat sources.

---
 meta/recipes-devtools/make/make.inc           |  4 +-
 ...b-Do-not-assume-glibc-glob-internals.patch | 70 ----------------
 ...m4-restrict-AIX-specific-test-on-AIX.patch | 38 +++++++++
 ...rc-dir.c-fix-buffer-overflow-warning.patch | 41 ++++++++++
 ...b-Do-not-assume-glibc-glob-internals.patch | 38 ---------
 ...low-being-detected-by-importing-proj.patch | 33 ++++++++
 ...02-w32-compat-dirent.c-follow-header.patch | 36 +++++++++
 ...-posixfcn-fcntl-gnulib-make-emulated.patch | 79 +++++++++++++++++++
 meta/recipes-devtools/make/make_4.2.1.bb      | 12 ---
 meta/recipes-devtools/make/make_4.3.bb        | 18 +++++
 10 files changed, 246 insertions(+), 123 deletions(-)
 delete mode 100644 meta/recipes-devtools/make/make/0001-glob-Do-not-assume-glibc-glob-internals.patch
 create mode 100644 meta/recipes-devtools/make/make/0001-m4-getloadavg.m4-restrict-AIX-specific-test-on-AIX.patch
 create mode 100644 meta/recipes-devtools/make/make/0001-src-dir.c-fix-buffer-overflow-warning.patch
 delete mode 100644 meta/recipes-devtools/make/make/0002-glob-Do-not-assume-glibc-glob-internals.patch
 create mode 100644 meta/recipes-devtools/make/make/0002-modules-fcntl-allow-being-detected-by-importing-proj.patch
 create mode 100644 meta/recipes-devtools/make/make/0002-w32-compat-dirent.c-follow-header.patch
 create mode 100644 meta/recipes-devtools/make/make/0003-posixfcn-fcntl-gnulib-make-emulated.patch
 delete mode 100644 meta/recipes-devtools/make/make_4.2.1.bb
 create mode 100644 meta/recipes-devtools/make/make_4.3.bb
meta/recipes-devtools/make/make.inc +1 -3
index b8905bc6d3..4142cf23d3 100644
@@ -5,9 +5,7 @@ called the makefile, which lists each of the non-source files and how to compute
 HOMEPAGE = "http://www.gnu.org/software/make/"
 SECTION = "devel"
 
-SRC_URI = "${GNU_MIRROR}/make/make-${PV}.tar.bz2 \
-           file://0001-glob-Do-not-assume-glibc-glob-internals.patch \
-           file://0002-glob-Do-not-assume-glibc-glob-internals.patch \
+SRC_URI = "${GNU_MIRROR}/make/make-${PV}.tar.lz \
            "
 
 inherit autotools gettext pkgconfig texinfo
meta/recipes-devtools/make/make/0001-glob-Do-not-assume-glibc-glob-internals.patch deleted +0 -70

70 lines not on this page · raw message

meta/recipes-devtools/make/make/0001-m4-getloadavg.m4-restrict-AIX-specific-test-on-AIX.patch new file +38 -0
new file mode 100644
index 0000000000..096bcfdf78
@@ -0,0 +1,38 @@
+From 8309601775d9442416329a77f7dcfd8aa799e9a6 Mon Sep 17 00:00:00 2001
+From: Jens Rehsack <[email protected]>
+Date: Fri, 21 Feb 2020 17:39:56 +0100
+Subject: [PATCH 1/2] m4/getloadavg.m4: restrict AIX specific test on AIX
+
+When cross compiling for a system without getloadavg, do not try add
+additional linker paths unless it's absolutely necessary.
+
+Signed-off-by: Jens Rehsack <[email protected]>
+---
+Upstream-Status: Pending
+ m4/getloadavg.m4 | 3 +++
+ 1 file changed, 3 insertions(+)
+
+diff --git a/m4/getloadavg.m4 b/m4/getloadavg.m4
+index 3bd2a14..696c5de 100644
+--- a/m4/getloadavg.m4
++++ b/m4/getloadavg.m4
+@@ -42,6 +42,8 @@ AC_CHECK_FUNC([getloadavg], [],
+    fi
+ 
+    if test $gl_func_getloadavg_done = no; then
++     AS_CASE([$host_os],
++             [aix*], [
+      # There is a commonly available library for RS/6000 AIX.
+      # Since it is not a standard part of AIX, it might be installed locally.
+      gl_getloadavg_LIBS=$LIBS
+@@ -49,6 +51,7 @@ AC_CHECK_FUNC([getloadavg], [],
+      AC_CHECK_LIB([getloadavg], [getloadavg],
+                   [LIBS="-lgetloadavg $LIBS" gl_func_getloadavg_done=yes],
+                   [LIBS=$gl_getloadavg_LIBS])
++       ], [:])
+    fi
+ 
+    # Set up the replacement function if necessary.
+-- 
+2.17.1
+
meta/recipes-devtools/make/make/0001-src-dir.c-fix-buffer-overflow-warning.patch new file +41 -0

41 lines not on this page · raw message

meta/recipes-devtools/make/make/0002-glob-Do-not-assume-glibc-glob-internals.patch deleted +0 -38
deleted file mode 100644
index d49acd9f6e..0000000000
@@ -1,38 +0,0 @@
-From 9858702dbd1e137262c06765919937660879f63c Mon Sep 17 00:00:00 2001
-From: Paul Eggert <[email protected]>
-Date: Sun, 24 Sep 2017 09:12:58 -0400
-Subject: [PATCH 2/2] glob: Do not assume glibc glob internals.
-
-It has been proposed that glibc glob start using gl_lstat,
-which the API allows it to do.  GNU 'make' should not get in
-the way of this.  See:
-https://sourceware.org/ml/libc-alpha/2017-09/msg00409.html
-
-* dir.c (local_lstat): New function, like local_stat.
-(dir_setup_glob): Use it to initialize gl_lstat too, as the API
-requires.
----
-Upstream-Status: Backport
-
- configure.ac | 3 +--
- 1 file changed, 1 insertion(+), 2 deletions(-)
-
-diff --git a/configure.ac b/configure.ac
-index 64ec870..e87901c 100644
---- a/configure.ac
-+++ b/configure.ac
-@@ -399,10 +399,9 @@ AC_CACHE_CHECK([if system libc has GNU glob], [make_cv_sys_gnu_glob],
- #include <glob.h>
- #include <fnmatch.h>
- 
--#define GLOB_INTERFACE_VERSION 1
- #if !defined _LIBC && defined __GNU_LIBRARY__ && __GNU_LIBRARY__ > 1
- # include <gnu-versions.h>
--# if _GNU_GLOB_INTERFACE_VERSION == GLOB_INTERFACE_VERSION
-+if _GNU_GLOB_INTERFACE_VERSION == 1 || _GNU_GLOB_INTERFACE_VERSION == 2
-    gnu glob
- # endif
- #endif],
--- 
-2.16.1
-
meta/recipes-devtools/make/make/0002-modules-fcntl-allow-being-detected-by-importing-proj.patch new file +33 -0
new file mode 100644
index 0000000000..b3d97f9a3a
@@ -0,0 +1,33 @@
+From fb8aaed3b040e589cd880fd714dda5ec00687217 Mon Sep 17 00:00:00 2001
+From: Jens Rehsack <[email protected]>
+Date: Mon, 24 Feb 2020 12:10:06 +0100
+Subject: [PATCH 2/2] modules: fcntl: allow being detected by importing
+ projects
+
+GNU project `make` relies on gnulib but provides some own compatibility
+functions - including an `fcntl`, which fails on mingw.
+The intension of gnulib is providing these functions and being wider tested,
+but silently injecting a function opens battle of compatibility layers.
+
+So adding a hint into target `config.h` to allow deciding whether using
+an own compatibility implementation or not.
+
+Signed-off-by: Jens Rehsack <[email protected]>
+---
+Upstream-Status: Pending
+
+ m4/gnulib-comp.m4 | 1 +
+ 1 file changed, 1 insertion(+)
+
+diff --git a/m4/gnulib-comp.m4 b/m4/gnulib-comp.m4
+index 3ee0811..cf75541 100644
+--- a/m4/gnulib-comp.m4
++++ b/m4/gnulib-comp.m4
+@@ -147,6 +147,7 @@
+   gl_FUNC_FCNTL
+   if test $HAVE_FCNTL = 0 || test $REPLACE_FCNTL = 1; then
+     AC_LIBOBJ([fcntl])
++    AC_DEFINE(HAVE_GNULIB_FCNTL, 1, [Define to 1 if you have the `fcntl' function via gnulib.])
+   fi
+   gl_FCNTL_MODULE_INDICATOR([fcntl])
+   gl_FCNTL_H
meta/recipes-devtools/make/make/0002-w32-compat-dirent.c-follow-header.patch new file +36 -0
new file mode 100644
index 0000000000..9ecc44543e
@@ -0,0 +1,36 @@
+From 4dd8b4f43aa0078707ad9a7932f4e137bc4383ed Mon Sep 17 00:00:00 2001
+From: Jens Rehsack <[email protected]>
+Date: Mon, 24 Feb 2020 11:12:43 +0100
+Subject: [PATCH 2/3] w32: compat: dirent.c: follow header
+
+src/w32/include/dirent.h completely delegates to mingw dirent implementation,
+gnulib detects it as fine and completely usable - trust in that.
+
+Signed-off-by: Jens Rehsack <[email protected]>
+---
+Upstream-Status: Pending (https://savannah.gnu.org/bugs/?57888)
+
+ src/w32/compat/dirent.c | 3 ++-
+ 1 file changed, 2 insertions(+), 1 deletion(-)
+
+diff --git a/src/w32/compat/dirent.c b/src/w32/compat/dirent.c
+index b8ec615..de80f72 100644
+--- a/src/w32/compat/dirent.c
++++ b/src/w32/compat/dirent.c
+@@ -23,7 +23,7 @@ this program.  If not, see <http://www.gnu.org/licenses/>.  */
+ #include <stdlib.h>
+ #include "dirent.h"
+ 
+-
++#ifndef __MINGW32__
+ DIR*
+ opendir(const char* pDirName)
+ {
+@@ -193,3 +193,4 @@ seekdir(DIR* pDir, long nPosition)
+ 
+         return;
+ }
++#endif  /* !__MINGW32__ */
+-- 
+2.17.1
+
meta/recipes-devtools/make/make/0003-posixfcn-fcntl-gnulib-make-emulated.patch new file +79 -0

79 lines not on this page · raw message

meta/recipes-devtools/make/make_4.2.1.bb deleted +0 -12
deleted file mode 100644
index c6e6a0cd58..0000000000
@@ -1,12 +0,0 @@
-LICENSE = "GPLv3 & LGPLv2"
-LIC_FILES_CHKSUM = "file://COPYING;md5=d32239bcb673463ab874e80d47fae504 \
-                    file://tests/COPYING;md5=d32239bcb673463ab874e80d47fae504 \
-                    file://glob/COPYING.LIB;md5=4a770b67e6be0f60da244beb2de0fce4"
-require make.inc
-
-EXTRA_OECONF += "--without-guile"
-
-SRC_URI[md5sum] = "15b012617e7c44c0ed482721629577ac"
-SRC_URI[sha256sum] = "d6e262bf3601b42d2b1e4ef8310029e1dcf20083c5446b4b7aa67081fdffc589"
-
-BBCLASSEXTEND = "native nativesdk"
meta/recipes-devtools/make/make_4.3.bb new file +18 -0
new file mode 100644
index 0000000000..70caf0ae16
@@ -0,0 +1,18 @@
+LICENSE = "GPLv3"
+LIC_FILES_CHKSUM = "file://COPYING;md5=d32239bcb673463ab874e80d47fae504"
+require make.inc
+
+SRC_URI += "\
+	file://0001-m4-getloadavg.m4-restrict-AIX-specific-test-on-AIX.patch \
+	file://0002-modules-fcntl-allow-being-detected-by-importing-proj.patch \
+	file://0001-src-dir.c-fix-buffer-overflow-warning.patch \
+	file://0002-w32-compat-dirent.c-follow-header.patch \
+	file://0003-posixfcn-fcntl-gnulib-make-emulated.patch \
+"
+
+EXTRA_OECONF += "--without-guile"
+
+SRC_URI[md5sum] = "d5c40e7bd1e97a7404f5d3be982f479a"
+SRC_URI[sha256sum] = "de1a441c4edf952521db30bfca80baae86a0ff1acd0a00402999344f04c45e82"
+
+BBCLASSEXTEND = "native nativesdk"
Signature · 1 line
2.17.1
#1 lore raw

meta/recipes-devtools/make/make.inc called the makefile, which lists each of the non-source files and how to compute · line 8

Quoted 5 lines · Jens Rehsack

...

meta/recipes-devtools/make/make.inc +1 -1
-SRC_URI = "${GNU_MIRROR}/make/make-${PV}.tar.bz2 \
...
+SRC_URI = "${GNU_MIRROR}/make/make-${PV}.tar.lz \
...

Please use the .tar.gz instead.

Building lzip-native just for being able to build make is not worth the small space savings, especially since this creates a bottleneck that can slow down the whole build.

cu
Adrian

#2 lore raw
In reply to Jens Rehsack
Quoted 18 lines · Jens Rehsack

Announcement: https://lists.gnu.org/archive/html/bug-make/2020-01/msg00057.html

1) Remove upstream provided patches 0001-glob-Do-not-assume-glibc-glob-internals.patch and 0002-glob-Do-not-assume-glibc-glob-internals.patch.

2) License has been changed to GPLv3 only

3) Important bug-fix is
* https://lists.gnu.org/archive/html/bug-make/2018-09/msg00006.html

4) Backward-incompatibilities:
* Number signs (#) appearing inside a macro reference or function invocation no longer introduce comments and should not be escaped with backslashes
* Previously appending using '+=' to an empty variable would result in a value starting with a space. Now the initial space is only added if the variable already contains some value. Similarly, appending an empty string does not add a trailing space.

Quoted 4 lines

Fix incompatibility issues between gnulib bundled with updated make fix issues in w32 compat sources.

Unfortunately, crazy as this sounds, this is causing mips problems on target with kernel module building. We only see the following failures when this patch is included:

https://autobuilder.yoctoproject.org/typhoon/#/builders/60/builds/1612
https://autobuilder.yoctoproject.org/typhoon/#/builders/102/builds/315
https://autobuilder.yoctoproject.org/typhoon/#/builders/74/builds/1616

Cheers,

Richard

#3 lore raw

[sorry for the top post: bad email client at work]

Hi Richard, Jens,

I was able to reproduce and debugged the mips new make issue. Workaround patch attached.

In summary: kernelmodule.KernelModuleTest.test_kernel_module
fails with 'fatal error: opening dependency file scripts/basic/.fixdep.d: Permission denied'

New make-4.3 started using posix_spawn glibc call,
that happens to be broken on mips. On mips during posix_spawn call __spawni_child function that is executed after clone makes setresuid call with spurious/wrong value of ruid. Instead of -1 it passes 127, effectively switching real user id since test runs under root. Subsequent gcc runs under wrong ruid and it fails to write into /usr/src/kernel/scripts/basic directory.

__spawni_child uses direct system call invocation for setresuid rather calling glibc function. And there is wrong code generation issue somewhere in inline assembly for direct system call and/or way how compiler processes it. I did not dig into it yet, it looks as quite hard problem and needs more time.

For now I've made workaround patch that just through configure ac_cv_func_posix_spawn=no parameter tells make configure not to use posix_spawn on mips.

I will also send suggested patch through 'git send-mail'

Tested workaround on qemumips64, did not have time for qemumips full test, verified build only and made sure that posix_spawn is not used. But I believe it is the same issue.

Detailed debugging notes
=================

Running 'bitbake core-image-sato-sdk -c testimage' fails on qemumips64
----------------------------------------------------------------------

RESULTS - kernelmodule.KernelModuleTest.test_kernel_module: FAILED (36.71s)

Reproducing issue manually
--------------------------

root@qemumips64:~# cd /usr/src/kernel && make scripts prepare
  HOSTCC  scripts/basic/fixdep
scripts/basic/fixdep.c:410:1: fatal error: opening dependency file scripts/basic/.fixdep.d: Permission denied
  410 | }
      | ^
compilation terminated.
make[2]: *** [scripts/Makefile.host:107: scripts/basic/fixdep] Error 1
make[1]: *** [Makefile:500: scripts_basic] Error 2
make: *** [Makefile:678: include/config/auto.conf.cmd] Error 2
make: *** [include/config/auto.conf.cmd] Deleting file 'include/config/tristate.conf'
root@qemumips64:/usr/src/kernel# ls -al scripts/basic/.fixdep.d
ls: cannot access 'scripts/basic/.fixdep.d': No such file or directory

Calling the same gcc command manually from shell
------------------------------------------------

It works, no problem.

root@qemumips64:/usr/src/kernel# gcc -Wp,-MD,scripts/basic/.fixdep.d -Wall -Wmissing-prototypes -Wstrict-prototypes -O2 -fomit-frame-pointer -std=gnu89 -o scripts/basic/fixdep scripts/basic/fixdep.c
root@qemumips64:/usr/src/kernel# 

Means make changes context in such way that prevents gcc command execution.

Looking at strace result
------------------------

Run the same make command under strace. In log see that getuid return 127! I.e it is different user. It does not have permission to write into scripts/basic/.fixdep.d

1144  getuid()                          = 127
1144  getgid()                          = 127

and earlier:

1144  getuid()                          = 0
1144  setresuid(127, 0, -1)             = 0
1144  getgid()                          = 0
1144  setresgid(127, 0, -1)             = 0
1144  dup2(6, 1)                        = 1
1144  rt_sigprocmask(SIG_SETMASK, [], NULL, 16) = 0
1144  execve("/bin/sh", ["/bin/sh", "-c", "uname -m | sed -e s/i.86/x86/ -e"...], 0xfffbb2e1d0 /* 16 vars */ <unfinished ...>

There is call to setresuid that sets real user id to
127. Why?

Running make under gdb
----------------------

(gdb) set args scripts
(gdb) b main
Breakpoint 1 at 0x9d80: file ../make-4.3/src/main.c, line 1056. (gdb) set follow-fork-mode child
(gdb) run
Starting program: /usr/bin/make scripts

Breakpoint 1, main (argc=2, argv=0xffffffac78, envp=0xffffffac90) at ../make-4.3/src/main.c:1056
1056 ../make-4.3/src/main.c: No such file or directory.
(gdb) b getuid
Breakpoint 2 at 0xfff7eb5a00: file ../sysdeps/unix/syscall-template.S, line 59. (gdb) b setresuid
Breakpoint 3 at 0xfff7eb5eb0: file ../sysdeps/unix/sysv/linux/setresuid.c, line 29. (gdb) b execve
Breakpoint 4 at 0xfff7eb4f10: file ../sysdeps/unix/syscall-template.S, line 78. (gdb) b clone
Breakpoint 5 at 0xfff7ef3c90: file ../sysdeps/unix/sysv/linux/mips/clone.S, line 45. (gdb) b posix_spawn
Breakpoint 6 at 0xfff7edd790: posix_spawn. (2 locations)
(gdb) b __spawni_child
Breakpoint 7 at 0xfff7eddac0: file ../sysdeps/unix/sysv/linux/spawni.c, line 123. (gdb) c
Continuing.

Breakpoint 2, getuid () at ../sysdeps/unix/syscall-template.S:59
59 ../sysdeps/unix/syscall-template.S: No such file or directory.
(gdb) bt

#0  getuid () at ../sysdeps/unix/syscall-template.S:59
#1  0x000000fff7edfa7c in __euidaccess (path=0xaaaab21b30 "/bin/sh", mode=1) at ../sysdeps/posix/euidaccess.c:159
#2  0x000000aaaaadecdc in find_in_given_path (progname=0xaaaab25440 "/bin/sh", path=<optimized out>, optimize_for_exec=<optimized out>)
    at ../../make-4.3/lib/findprog-in.c:137
#3  0x000000aaaaac7680 in child_execute_job (child=0xffffff8770, good_stdin=<optimized out>, argv=0xaaaab26cb0) at ../make-4.3/src/job.c:2384
#4  0x000000aaaaac2678 in func_shell_base (o=0xaaaab1e350 "", argv=<optimized out>, trim_newlines=<optimized out>) at ../make-4.3/src/function.c:1790
#5  0x000000aaaaac2c38 in handle_function (op=0xffffff8930, stringp=<optimized out>) at ../make-4.3/src/function.c:2544
#6  0x000000aaaaabaa10 in variable_expand_string (line=<optimized out>, string=<optimized out>, length=<optimized out>) at ../make-4.3/src/expand.c:262
#7  0x000000aaaaabb5b4 in allocated_variable_expand_for_file (line=<optimized out>, file=<optimized out>) at ../make-4.3/src/expand.c:566
#8  0x000000aaaaadc11c in do_variable_definition (flocp=0xffffff8d70, varname=0xaaaab1fca0 "SUBARCH", 
    value=0xaaaab24e6b "$(shell uname -m | sed -e s/i.86/x86/ -e s/x86_64/x86/ -e s/sun4u/sparc64/ -e s/arm.*/arm/ -e s/sa110/arm/ -e s/s390x/s390/ -e s/parisc64/parisc/ -e s/ppc.*/powerpc/ -e s/mips.*/mips/ -e s/sh[234].*/s"..., origin=<optimized out>, flavor=<optimized out>, target_var=<optimized out>) at ../make-4.3/src/variable.c:1187
#9  0x000000aaaaadccac in try_variable_definition (flocp=0xffffff8d70, line=<optimized out>, origin=<optimized out>, target_var=<optimized out>)
    at ../make-4.3/src/variable.c:1633

#10 0x000000aaaaad2dfc in eval (ebuf=<optimized out>, set_default=<optimized out>) at ../make-4.3/src/read.c:750 #11 0x000000aaaaad451c in eval_makefile (filename=<optimized out>, flags=<optimized out>) at ../make-4.3/src/read.c:436 #12 0x000000aaaaad3cc4 in eval (ebuf=<optimized out>, set_default=<optimized out>) at ../make-4.3/src/read.c:908 #13 0x000000aaaaad451c in eval_makefile (filename=<optimized out>, flags=<optimized out>) at ../make-4.3/src/read.c:436 #14 0x000000aaaaad4a70 in read_all_makefiles (makefiles=<optimized out>) at ../make-4.3/src/read.c:258
#15 0x000000aaaaab4f9c in main (argc=<optimized out>, argv=<optimized out>, envp=<optimized out>) at ../make-4.3/src/main.c:1945 (gdb) c
Continuing.

Breakpoint 6, __GI___posix_spawn (pid=0xffffff84b4, path=0xaaaab25440 "/bin/sh", file_actions=0xffffff84b8, attrp=0xffffff8588, argv=0xaaaab26cb0, envp=0xffffffac90)

    at spawn.c:30
30	spawn.c: No such file or directory.
(gdb) bt
#0  __GI___posix_spawn (pid=0xffffff84b4, path=0xaaaab25440 "/bin/sh", file_actions=0xffffff84b8, attrp=0xffffff8588, argv=0xaaaab26cb0, envp=0xffffffac90) at spawn.c:30
#1  0x000000aaaaac76b0 in child_execute_job (child=0xffffff8770, good_stdin=<optimized out>, argv=0xaaaab26cb0) at ../make-4.3/src/job.c:2396
#2  0x000000aaaaac2678 in func_shell_base (o=0xaaaab1e350 "", argv=<optimized out>, trim_newlines=<optimized out>) at ../make-4.3/src/function.c:1790
#3  0x000000aaaaac2c38 in handle_function (op=0xffffff8930, stringp=<optimized out>) at ../make-4.3/src/function.c:2544
#4  0x000000aaaaabaa10 in variable_expand_string (line=<optimized out>, string=<optimized out>, length=<optimized out>) at ../make-4.3/src/expand.c:262
#5  0x000000aaaaabb5b4 in allocated_variable_expand_for_file (line=<optimized out>, file=<optimized out>) at ../make-4.3/src/expand.c:566
#6  0x000000aaaaadc11c in do_variable_definition (flocp=0xffffff8d70, varname=0xaaaab1fca0 "SUBARCH", 
    value=0xaaaab24e6b "$(shell uname -m | sed -e s/i.86/x86/ -e s/x86_64/x86/ -e s/sun4u/sparc64/ -e s/arm.*/arm/ -e s/sa110/arm/ -e s/s390x/s390/ -e s/parisc64/parisc/ -e s/ppc.*/powerpc/ -e s/mips.*/mips/ -e s/sh[234].*/s"..., origin=<optimized out>, flavor=<optimized out>, target_var=<optimized out>) at ../make-4.3/src/variable.c:1187
#7  0x000000aaaaadccac in try_variable_definition (flocp=0xffffff8d70, line=<optimized out>, origin=<optimized out>, target_var=<optimized out>)
    at ../make-4.3/src/variable.c:1633

#8 0x000000aaaaad2dfc in eval (ebuf=<optimized out>, set_default=<optimized out>) at ../make-4.3/src/read.c:750 #9 0x000000aaaaad451c in eval_makefile (filename=<optimized out>, flags=<optimized out>) at ../make-4.3/src/read.c:436 #10 0x000000aaaaad3cc4 in eval (ebuf=<optimized out>, set_default=<optimized out>) at ../make-4.3/src/read.c:908 #11 0x000000aaaaad451c in eval_makefile (filename=<optimized out>, flags=<optimized out>) at ../make-4.3/src/read.c:436 #12 0x000000aaaaad4a70 in read_all_makefiles (makefiles=<optimized out>) at ../make-4.3/src/read.c:258
#13 0x000000aaaaab4f9c in main (argc=<optimized out>, argv=<optimized out>, envp=<optimized out>) at ../make-4.3/src/main.c:1945 (gdb) c
Continuing.

Breakpoint 5, clone () at ../sysdeps/unix/sysv/linux/mips/clone.S:45
45 ../sysdeps/unix/sysv/linux/mips/clone.S: No such file or directory.
(gdb) bt
#0 clone () at ../sysdeps/unix/sysv/linux/mips/clone.S:45
#1 0x000000fff7edd938 in __spawnix (pid=0xffffff84b4, file=0xaaaab25440 "/bin/sh", file_actions=<optimized out>, attrp=<optimized out>, argv=0xaaaab26cb0,

    envp=0xffffffac90, xflags=<optimized out>, exec=0xfff7eb4f10 <execve>) at ../sysdeps/unix/sysv/linux/spawni.c:382
#2  0x000000aaaaac76b0 in child_execute_job (child=0xffffff8770, good_stdin=<optimized out>, argv=0xaaaab26cb0) at ../make-4.3/src/job.c:2396
#3  0x000000aaaaac2678 in func_shell_base (o=0xaaaab1e350 "", argv=<optimized out>, trim_newlines=<optimized out>) at ../make-4.3/src/function.c:1790
#4  0x000000aaaaac2c38 in handle_function (op=0xffffff8930, stringp=<optimized out>) at ../make-4.3/src/function.c:2544
#5  0x000000aaaaabaa10 in variable_expand_string (line=<optimized out>, string=<optimized out>, length=<optimized out>) at ../make-4.3/src/expand.c:262
#6  0x000000aaaaabb5b4 in allocated_variable_expand_for_file (line=<optimized out>, file=<optimized out>) at ../make-4.3/src/expand.c:566
#7  0x000000aaaaadc11c in do_variable_definition (flocp=0xffffff8d70, varname=0xaaaab1fca0 "SUBARCH", 
    value=0xaaaab24e6b "$(shell uname -m | sed -e s/i.86/x86/ -e s/x86_64/x86/ -e s/sun4u/sparc64/ -e s/arm.*/arm/ -e s/sa110/arm/ -e s/s390x/s390/ -e s/parisc64/parisc/ -e s/ppc.*/powerpc/ -e s/mips.*/mips/ -e s/sh[234].*/s"..., origin=<optimized out>, flavor=<optimized out>, target_var=<optimized out>) at ../make-4.3/src/variable.c:1187
#8  0x000000aaaaadccac in try_variable_definition (flocp=0xffffff8d70, line=<optimized out>, origin=<optimized out>, target_var=<optimized out>)
    at ../make-4.3/src/variable.c:1633

#9 0x000000aaaaad2dfc in eval (ebuf=<optimized out>, set_default=<optimized out>) at ../make-4.3/src/read.c:750 #10 0x000000aaaaad451c in eval_makefile (filename=<optimized out>, flags=<optimized out>) at ../make-4.3/src/read.c:436 #11 0x000000aaaaad3cc4 in eval (ebuf=<optimized out>, set_default=<optimized out>) at ../make-4.3/src/read.c:908 #12 0x000000aaaaad451c in eval_makefile (filename=<optimized out>, flags=<optimized out>) at ../make-4.3/src/read.c:436 #13 0x000000aaaaad4a70 in read_all_makefiles (makefiles=<optimized out>) at ../make-4.3/src/read.c:258
#14 0x000000aaaaab4f9c in main (argc=<optimized out>, argv=<optimized out>, envp=<optimized out>) at ../make-4.3/src/main.c:1945 (gdb) c
Continuing.
[Attaching after process 3441 vfork to child process 3444]
[New inferior 2 (process 3444)]
[Switching to process 3444]

Thread 2.1 "make" hit Breakpoint 7, __spawni_child (arguments=0xffffff8350) at ../sysdeps/unix/sysv/linux/spawni.c:123
123 ../sysdeps/unix/sysv/linux/spawni.c: No such file or directory.
(gdb) bt
#0 __spawni_child (arguments=0xffffff8350) at ../sysdeps/unix/sysv/linux/spawni.c:123
#1 0x000000fff7ef3d28 in __thread_start () at ../sysdeps/unix/sysv/linux/mips/clone.S:135
(gdb) c
Continuing.

Thread 2.1 "make" hit Breakpoint 2, getuid () at ../sysdeps/unix/syscall-template.S:59
59 ../sysdeps/unix/syscall-template.S: No such file or directory.
(gdb) bt

#0  getuid () at ../sysdeps/unix/syscall-template.S:59
#1  0x000000fff7eddedc in __spawni_child (arguments=0xffffff8350) at ../sysdeps/unix/sysv/linux/spawni.c:189
#2  0x000000fff7ef3d28 in __thread_start () at ../sysdeps/unix/sysv/linux/mips/clone.S:135

(gdb) c
Continuing.

Thread 2.1 "make" hit Breakpoint 4, 0x000000fff7eb4f10 in execve () at ../sysdeps/unix/syscall-template.S:78
78 ../sysdeps/unix/syscall-template.S: No such file or directory.
(gdb) bt

#0  0x000000fff7eb4f10 in execve () at ../sysdeps/unix/syscall-template.S:78
#1  0x000000fff7eddd10 in __spawni_child (arguments=0xffffff8350) at ../sysdeps/unix/sysv/linux/spawni.c:292
#2  0x000000fff7ef3d28 in __thread_start () at ../sysdeps/unix/sysv/linux/mips/clone.S:135

It looks like we did not hit setresuid breakpoint
but reached execve ... strange

Looking at glibc __spawni_child code
------------------------------------

From cat -n ./2.31-r0/git/sysdeps/unix/sysv/linux/spawni.c

   187    /* Set the effective user and group IDs.  */
   188    if ((attr->__flags & POSIX_SPAWN_RESETIDS) != 0
   189        && (local_seteuid (__getuid ()) != 0
   190            || local_setegid (__getgid ()) != 0))
   191      goto fail;
   192  

What is local_seteuid
---------------------

From cat -n 2.31-r0/git/sysdeps/unix/sysv/linux/local-setxid.h

8 # define local_seteuid(id) INLINE_SYSCALL (setresuid, 3, -1, id, -1)

15 # define local_setegid(id) INLINE_SYSCALL (setresgid, 3, -1, id, -1)

That explains why setresuid breakpoint was not hit. Code calls directly system call.

What is internal_syscall3
-------------------------

From cat -n 2.31-r0/git/sysdeps/unix/sysv/linux/mips/mips64/n64/sysdep.h

   181  #define internal_syscall3(v0_init, input, number, err,                  \
   182                            arg1, arg2, arg3)                             \
   183  ({                                                                      \
   184          long _sys_result;                                               \
   185                                                                          \
   186          {                                                               \
   187          register long __s0 asm ("$16") __attribute__ ((unused))         \
   188            = (number);                                                   \
   189          register long __v0 asm ("$2");                                  \
   190          register long __a0 asm ("$4") = (long) (arg1);                  \
   191          register long __a1 asm ("$5") = (long) (arg2);                  \
   192          register long __a2 asm ("$6") = (long) (arg3);                  \
   193          register long __a3 asm ("$7");                                  \
   194          __asm__ volatile (                                              \
   195          ".set\tnoreorder\n\t"                                           \
   196          v0_init                                                         \
   197          "syscall\n\t"                                                   \
   198          ".set\treorder"                                                 \
   199          : "=r" (__v0), "=r" (__a3)                                      \
   200          : input, "r" (__a0), "r" (__a1), "r" (__a2)                     \
   201          : __SYSCALL_CLOBBERS);                                          \
   202          err = __a3;                                                     \
   203          _sys_result = __v0;                                             \
   204          }                                                               \
   205          _sys_result;                                                    \

206 })

In gdb again
------------

(gdb) bt

#0  getuid () at ../sysdeps/unix/syscall-template.S:59
#1  0x000000fff7eddedc in __spawni_child (arguments=0xffffff8350) at ../sysdeps/unix/sysv/linux/spawni.c:189
#2  0x000000fff7ef3d28 in __thread_start () at ../sysdeps/unix/sysv/linux/mips/clone.S:135

(gdb) up
#1 0x000000fff7eddedc in __spawni_child (arguments=0xffffff8350) at ../sysdeps/unix/sysv/linux/spawni.c:189
189 ../sysdeps/unix/sysv/linux/spawni.c: No such file or directory.
(gdb) x /20i $pc - 20
0xfff7eddec8 <__spawni_child+1032>: nop
0xfff7eddecc <__spawni_child+1036>: b 0xfff7eddcc0 <__spawni_child+512>
0xfff7edded0 <__spawni_child+1040>: lw a0,4(s1)
0xfff7edded4 <__spawni_child+1044>: jalr t9
0xfff7edded8 <__spawni_child+1048>: nop
=> 0xfff7eddedc <__spawni_child+1052>: li a2,-1
0xfff7eddee0 <__spawni_child+1056>: dext a1,v0,0x0,0x20
0xfff7eddee4 <__spawni_child+1060>: li v0,5115
0xfff7eddee8 <__spawni_child+1064>: syscall
0xfff7eddeec <__spawni_child+1068>: bnez a3,0xfff7ede000 <__spawni_child+1344>
0xfff7eddef0 <__spawni_child+1072>: nop
0xfff7eddef4 <__spawni_child+1076>: bnez v0,0xfff7edde00 <__spawni_child+832>
0xfff7eddef8 <__spawni_child+1080>: ld t9,-26712(gp)
0xfff7eddefc <__spawni_child+1084>: jalr t9
0xfff7eddf00 <__spawni_child+1088>: nop
0xfff7eddf04 <__spawni_child+1092>: li a2,-1
0xfff7eddf08 <__spawni_child+1096>: dext a1,v0,0x0,0x20
0xfff7eddf0c <__spawni_child+1100>: li v0,5117
0xfff7eddf10 <__spawni_child+1104>: syscall
0xfff7eddf14 <__spawni_child+1108>: bnez a3,0xfff7ede000 <__spawni_child+1344>
0xfff7eddf18 <__spawni_child+1112>: nop
(gdb) display /i $pc
1: x/i $pc
=> 0xfff7eddedc <__spawni_child+1052>: li a2,-1
(gdb) si
0x000000fff7eb5a04 59 ../sysdeps/unix/syscall-template.S: No such file or directory.
1: x/i $pc
=> 0xfff7eb5a04 <getuid+4>: syscall
(gdb) info reg
zero at v0 v1
R0 0000000000000000 000000001400a4e1 00000000000013ec 0000000000000000
a0 a1 a2 a3
R4 000000000000007f 000000fff7ff2d80 0000000000000000 0000000000000000
a4 a5 a6 a7
R8 0000000000000000 0000000000000000 0000000000000000 0000000000000000
t0 t1 t2 t3
R12 0000000000000000 ffffffff84080018 ffffffff801554b8 0000000000000000
s0 s1 s2 s3
R16 000000ffffff8588 000000ffffff84b8 0000000000000001 000000ffffff8350
s4 s5 s6 s7
R20 000000fff7e2ef80 000000fff7eb4f10 000000000000007f 000000000000007f
t8 t9 k0 k1
R24 0000000000000000 000000fff7eb5a00 0000000000000010 0000000000000000
gp sp s8 ra
R28 000000fff7f9f5e0 000000fff7ff2e20 000000fff7ff2e20 000000fff7eddedc

                status               lo               hi         badvaddr
      000000000400a4f3 1b6ee11164e1e246 eb05325e15c10102 000000fff7ff2fe0 
                 cause               pc
      0000000010800024 000000fff7eb5a04 
                  fcsr              fir          restart
              00000000         007f0000 0000000000000000 

(gdb) p /d $a0
$1 = 127

a0 value is 127! But it should be -1 according to local_seteuid call. Why is that?

This code looks quite wrong:

0xfff7edded0 <__spawni_child+1040>: lw a0,4(s1)
0xfff7edded4 <__spawni_child+1044>: jalr t9
0xfff7edded8 <__spawni_child+1048>: nop
=> 0xfff7eddedc <__spawni_child+1052>: li a2,-1
0xfff7eddee0 <__spawni_child+1056>: dext a1,v0,0x0,0x20
0xfff7eddee4 <__spawni_child+1060>: li v0,5115
0xfff7eddee8 <__spawni_child+1064>: syscall

There is not load of -1 into a0 before jumping to setresuid system call.

In disassembly at certain point one can see:

0x000000fff7edde1c <+860>: li a0,127

It seems either internal_syscall3 is wrong or compiler does not process it correctly ... quite bad ...

Looking at make make-4.3/src/job.c
----------------------------------

It seems that posix_spawn would be used only if USE_POSIX_SPAWN defined.

Test for USE_POSIX_SPAWN is in configure.ac
-------------------------------------------

From cat -n make-4.3/configure.ac

   382  AS_IF([test "$make_cv_posix_spawn" = yes],
   383    AC_CACHE_CHECK([for posix_spawn that fails synchronously],
   384      [make_cv_synchronous_posix_spawn],
   385      [make_cv_synchronous_posix_spawn=no
   386       AC_RUN_IFELSE([AC_LANG_SOURCE([[
   387         #include <spawn.h>
   388         #include <string.h>
   389  
   390         extern char **environ;
   391  
   392         int main() {
   393           char* path = strdup("./non-existent");
   394           char *argv[[2]];
   395           argv[[0]] = path;
   396           argv[[1]] =  0;
   397           return posix_spawn(0, path, 0, 0, argv, environ);
   398         }]])],
   399         [make_cv_synchronous_posix_spawn=no],
   400         [make_cv_synchronous_posix_spawn=yes],
   401         [make_cv_synchronous_posix_spawn="no (cross-compiling)"])]))
   402  
   403  AS_CASE([/$user_posix_spawn/$make_cv_posix_spawn/$make_cv_synchronous_posix_spawn/],
   404    [*/no/*], [make_cv_posix_spawn=no],
   405    [AC_DEFINE(USE_POSIX_SPAWN, 1, [Define to 1 to use posix_spawn().])
   406    ])

Note USE_POSIX_SPAWN code was not present in make-4.2.1
-------------------------------------------------------

Workaround
-----------

In case of mips disable use of posix_spawn through ac_cv_func_posix_spawn=no configure option.

Testing
-------

Tested 'bitbake core-image-sato-sdk -c testimage' all tested are passed.

Thanks,
Victor

________________________________________
From: Richard Purdie <[email protected]>
Sent: Thursday, February 27, 2020 11:45 PM
To: Jens Rehsack; [email protected]
Cc: Bruce Ashfield; Victor Kamensky (kamensky)
Subject: Re: [OE-core] [PATCH v3] make: 4.2.1 -> 4.3

Quoted 18 lines · Jens Rehsack

Announcement: https://lists.gnu.org/archive/html/bug-make/2020-01/msg00057.html

1) Remove upstream provided patches 0001-glob-Do-not-assume-glibc-glob-internals.patch and 0002-glob-Do-not-assume-glibc-glob-internals.patch.

2) License has been changed to GPLv3 only

3) Important bug-fix is
* https://lists.gnu.org/archive/html/bug-make/2018-09/msg00006.html

4) Backward-incompatibilities:
* Number signs (#) appearing inside a macro reference or function invocation no longer introduce comments and should not be escaped with backslashes
* Previously appending using '+=' to an empty variable would result in a value starting with a space. Now the initial space is only added if the variable already contains some value. Similarly, appending an empty string does not add a trailing space.

Quoted 4 lines

Fix incompatibility issues between gnulib bundled with updated make fix issues in w32 compat sources.

Unfortunately, crazy as this sounds, this is causing mips problems on target with kernel module building. We only see the following failures when this patch is included:

https://autobuilder.yoctoproject.org/typhoon/#/builders/60/builds/1612
https://autobuilder.yoctoproject.org/typhoon/#/builders/102/builds/315
https://autobuilder.yoctoproject.org/typhoon/#/builders/74/builds/1616

Cheers,

Richard

From 088ebd7b1c50babcf131cb0364cf8841b697863b Mon Sep 17 00:00:00 2001
From: Victor Kamensky <[email protected]>
Date: Fri, 28 Feb 2020 22:26:11 -0800
Subject: [PATCH] make: disable use of posix_spawn on mips

After make-4.3 migration child_execute_job function started using posix_spawn function, which happens to be broken on mips.

It manifests itself as when make executed by root, it switches real user id to wrong value because of some issues with direct setresuid system call done in glibc __spawni_child function through inline assemble and/or gcc compiling it produces wrong code. I.e instead of passing -1 posix_spawn function incorrectly passes 127 as ruid. Subsequently job started by make can fail with permission issue because they run under wrong user.

For now workaround is used by explicitly disabling posix_spawn call use by make on mips through configure variable.

---
 meta/recipes-devtools/make/make_4.3.bb | 3 +++
 1 file changed, 3 insertions(+)
meta/recipes-devtools/make/make_4.3.bb +3 -0
index 70caf0ae16..5fa7059169 100644
@@ -12,6 +12,9 @@ SRC_URI += "\
 
 EXTRA_OECONF += "--without-guile"
 
+EXTRA_OECONF_append_mips=" ac_cv_func_posix_spawn=no"
+EXTRA_OECONF_append_mips64=" ac_cv_func_posix_spawn=no"
+
 SRC_URI[md5sum] = "d5c40e7bd1e97a7404f5d3be982f479a"
 SRC_URI[sha256sum] = "de1a441c4edf952521db30bfca80baae86a0ff1acd0a00402999344f04c45e82"
 
Signature · 1 line
2.21.1
#4 lore raw

Hi Victor,

Quoted 25 lines · Victor Kamensky (kamensky)

I was able to reproduce and debugged the mips new make issue. Workaround patch attached.

In summary: kernelmodule.KernelModuleTest.test_kernel_module fails with 'fatal error: opening dependency file scripts/basic/.fixdep.d: Permission denied'

New make-4.3 started using posix_spawn glibc call,
that happens to be broken on mips. On mips during posix_spawn call __spawni_child function that is executed after clone makes setresuid call with spurious/wrong value of ruid. Instead of -1 it passes 127, effectively switching real user id since test runs under root. Subsequent gcc runs under wrong ruid and it fails to write into /usr/src/kernel/scripts/basic directory.

__spawni_child uses direct system call invocation for setresuid rather calling glibc function. And there is wrong code generation issue somewhere in inline assembly for direct system call and/or way how compiler processes it. I did not dig into it yet, it looks as quite hard problem and needs more time.

For now I've made workaround patch that just through configure ac_cv_func_posix_spawn=no parameter tells make configure not to use posix_spawn on mips.

Quoted 5 lines

I will also send suggested patch through 'git send-mail'

Tested workaround on qemumips64, did not have time for qemumips full test, verified build only and made sure that posix_spawn is not used. But I believe it is the same issue.

Thanks for the quick turnaround on this, sounds like good progress on a workaround! It does massively help with the preparation for 3.1 and is much appreciated.

I've put the patch in for testing. It sounds like there may be an upstream glibc patch which might be the fix of the underlying issue. I'll probably look to merge the workaround and then we could look at the glibc upgrade next.

I'll report back on how the testing goes.

Cheers,

Richard

#5 lore raw
Quoted 40 lines · Richard Purdie

Hi Victor,

Quoted 25 lines · Victor Kamensky (kamensky)

I was able to reproduce and debugged the mips new make issue. Workaround patch attached.

In summary: kernelmodule.KernelModuleTest.test_kernel_module fails with 'fatal error: opening dependency file scripts/basic/.fixdep.d: Permission denied'

New make-4.3 started using posix_spawn glibc call,
that happens to be broken on mips. On mips during posix_spawn call __spawni_child function that is executed after clone makes setresuid call with spurious/wrong value of ruid. Instead of -1 it passes 127, effectively switching real user id since test runs under root. Subsequent gcc runs under wrong ruid and it fails to write into /usr/src/kernel/scripts/basic directory.

__spawni_child uses direct system call invocation for setresuid rather calling glibc function. And there is wrong code generation issue somewhere in inline assembly for direct system call and/or way how compiler processes it. I did not dig into it yet, it looks as quite hard problem and needs more time.

For now I've made workaround patch that just through configure ac_cv_func_posix_spawn=no parameter tells make configure not to use posix_spawn on mips.

Quoted 5 lines

I will also send suggested patch through 'git send-mail'

Tested workaround on qemumips64, did not have time for qemumips full test, verified build only and made sure that posix_spawn is not used. But I believe it is the same issue.

Thanks for the quick turnaround on this, sounds like good progress on a
workaround! It does massively help with the preparation for 3.1 and is
much appreciated.

Quoted 6 lines

I've put the patch in for testing. It sounds like there may be an upstream glibc patch which might be the fix of the underlying issue. I'll probably look to merge the workaround and then we could look at the glibc upgrade next.

I'll report back on how the testing goes.

Looks good, thanks!

I'll merge with a comment about the issue and likely fix when we can drop it.

Cheers,

Richard