Skip to contentSkip to conversation

glibc: fix create thread failed in old unprivileged docker

PATCH In review hongxu 4 participants 7 messages Aug 30, 2021 – Sep 1, 2021 2 messages not in the archive

lore
Share & tools
lore

hongxu

author
#1 lore raw

PATCH v3 1/1 glibc: fix create thread failed in old unprivileged docker

In reply to a message not in the archive

# Changed in V3: add missing Upstream-Status tag

Since upstream commit [d8ea0d0168 Add an internal wrapper for clone, clone2 and clone3] applied, start a unprivileged container (docker run without --privileged), it creates a thread failed in container.

In commit d8ea0d0168, it calls __clone3 if HAVE_CLONE3_WAPPER is defined. If __clone3 returns -1 with ENOSYS, fall back to clone or clone2.

The newest docker has fixed the issue in commit [1], but it was applied only on master, to backward compatibility with old docker, discussed with glibc maintainer [2], explicitly disable clone3 wrapper work by removing macro definition of HAVE_CLONE3_WRAPPER

[1] https://github.com/moby/moby/commit/9f6b562dd12ef7b1f9e2f8e6f2ab6477790a6594
[2] https://sourceware.org/pipermail/libc-alpha/2021-August/130591.html

---
 ...hread-failed-in-unprivileged-process.patch | 94 +++++++++++++++++++
 meta/recipes-core/glibc/glibc_2.34.bb         |  1 +
 2 files changed, 95 insertions(+)
 create mode 100644 meta/recipes-core/glibc/glibc/0001-fix-create-thread-failed-in-unprivileged-process.patch
meta/recipes-core/glibc/glibc/0001-fix-create-thread-failed-in-unprivileged-process.patch new file +94 -0
new file mode 100644
index 0000000000..d3316c8c42
@@ -0,0 +1,94 @@
+From 116fcbcbf5edbd7692f48280e996884c3df0e993 Mon Sep 17 00:00:00 2001
+From: Hongxu Jia <[email protected]>
+Date: Sun, 29 Aug 2021 20:49:16 +0800
+Subject: [PATCH] fix create thread failed in unprivileged process
+
+Since commit [d8ea0d0168 Add an internal wrapper for clone, clone2 and clone3]
+applied, start a unprivileged container (docker run without --privileged),
+it creates a thread failed in container.
+
+In commit d8ea0d0168, it calls __clone3 if HAVE_CLONE3_WAPPER is defined.  If
+__clone3 returns -1 with ENOSYS, fall back to clone or clone2.
+
+The newest docker has fixed the issue in commit [1], but it was applied
+only on master, to backward compatibility with old docker, we explicitly
+disable clone3 wrapper work by removing macro definition of HAVE_CLONE3_WRAPPER
+
+[1] https://github.com/moby/moby/commit/9f6b562dd12ef7b1f9e2f8e6f2ab6477790a6594
+
+Here are the test steps:
+
+1) Prepare test code
+cat > conftest.c <<ENDOF
+ #include <pthread.h>
+ #include <stdio.h>
+
+int check_me = 0;
+void* func(void* data) {check_me = 42; printf("start thread: check_me %d\n", check_me); return &check_me;}
+int main()
+{
+  pthread_t t;
+  void *ret;
+  pthread_create (&t, 0, func, 0);
+  pthread_join (t, &ret);
+  printf("check_me %d, p %p\n", check_me, &ret);
+  return (check_me != 42 || ret != &check_me);
+}
+
+ENDOF
+
+2) Compile
+gcc -o conftest -pthread conftest.c
+
+3) Start a container with glibc 2.34 installed
+[skip details]
+docker run -it <container-image-name> bash
+
+4) Run conftest without this patch
+$ ./conftest
+check_me 0, p 0x7ffd91ccd400
+
+5) Run conftest with this patch
+$ ./conftest
+start thread: check_me 42
+check_me 42, p 0x7ffe253c6f20
+
+Upstream-Status: Inappropriate [oe specific]
+
+Signed-off-by: Hongxu Jia <[email protected]>
+---
+ sysdeps/unix/sysv/linux/i386/sysdep.h   | 3 ++-
+ sysdeps/unix/sysv/linux/x86_64/sysdep.h | 3 ++-
+ 2 files changed, 4 insertions(+), 2 deletions(-)
+
+diff --git a/sysdeps/unix/sysv/linux/i386/sysdep.h b/sysdeps/unix/sysv/linux/i386/sysdep.h
+index 8a9911b7ac..60d5cb2d9a 100644
+--- a/sysdeps/unix/sysv/linux/i386/sysdep.h
++++ b/sysdeps/unix/sysv/linux/i386/sysdep.h
+@@ -291,7 +291,8 @@ struct libc_do_syscall_args
+ # define HAVE_TIME_VSYSCALL             "__vdso_time"
+ # define HAVE_CLOCK_GETRES_VSYSCALL     "__vdso_clock_getres"
+ 
+-# define HAVE_CLONE3_WRAPPER		1
++/* Disable it to backward compatibility with old docker */
++//# define HAVE_CLONE3_WRAPPER		1
+ 
+ # undef HAVE_INTERNAL_BRK_ADDR_SYMBOL
+ # define HAVE_INTERNAL_BRK_ADDR_SYMBOL 1
+diff --git a/sysdeps/unix/sysv/linux/x86_64/sysdep.h b/sysdeps/unix/sysv/linux/x86_64/sysdep.h
+index 327e59388b..a7bc2cc686 100644
+--- a/sysdeps/unix/sysv/linux/x86_64/sysdep.h
++++ b/sysdeps/unix/sysv/linux/x86_64/sysdep.h
+@@ -377,7 +377,8 @@
+ # define HAVE_GETCPU_VSYSCALL		"__vdso_getcpu"
+ # define HAVE_CLOCK_GETRES64_VSYSCALL   "__vdso_clock_getres"
+ 
+-# define HAVE_CLONE3_WRAPPER			1
++/* Disable it to backward compatibility with old docker */
++//# define HAVE_CLONE3_WRAPPER			1
+ 
+ # define SINGLE_THREAD_BY_GLOBAL		1
+ 
+-- 
+2.27.0
+
meta/recipes-core/glibc/glibc_2.34.bb +1 -0
index eafc0216ff..46de1dae43 100644
@@ -57,6 +57,7 @@ SRC_URI =  "${GLIBC_GIT_URI};branch=${SRCBRANCH};name=glibc \
            file://0030-powerpc-Do-not-ask-compiler-for-finding-arch.patch \
            file://0001-CVE-2021-38604.patch \
            file://0002-CVE-2021-38604.patch \
+           file://0001-fix-create-thread-failed-in-unprivileged-process.patch \
            "
 S = "${WORKDIR}/git"
 B = "${WORKDIR}/build-${TARGET_SYS}"
Signature · 1 line
2.30.2
author
#5 lore raw

Upgrade uninative to fix multiple native build failures in old unprivileged docker

In reply to a message not in the archive

Hi Richard & Michael,

The reason why I submitted patch [glibc: fix create thread failed in old unprivileged docker] is there are build failures with uninative 3.3 under old unprivileged docker. If the glibc fix is OK, would you please upgrade a new uninative that contains the fix.

Here are the steps to reproduce the failures
1) Run a container without privileged

$ docker pull resin/yocto-build-env
$ docker run -it -e HOST_UID="$(id -u)" -e DISTRO="poky" -v $PWD:/mnt -w /mnt resin/yocto-build-env bash

2) Poky build preparation

$ apt update && apt install -y liblz4-tool zstd
$ useradd -u $HOST_UID -m -d /mnt/test -s /bin/bash test
$ su test
$ cd ~

3) Setup a Poky project with uninative enabled

$ git clone --branch master --single-branch git://git.yoctoproject.org/poky
$ cd poky && . ./oe-init-build-env
$ echo 'INHERIT += "uninative"' >> conf/local.conf

4) Build failures
...

$ bitbake ninja-native
|ninja: fatal: posix_spawn: Operation not permitted
$ bitbake go-cross-core2-64
|runtime/cgo: pthread_create failed: Operation not permitted
$ bitbake pkgconfig-native
| configure: error: I can't find the libraries for the thread implementation
...

After applying the glibc fix, I built a uninative tarball on my local project to replace uninative 3.3. Then repeat above steps, no above failures any more

//Hongxu

#6 lore raw

Re: [OE-core] Upgrade uninative to fix multiple native build failures in old unprivileged docker

Hi,

I just experienced this problem in Docker version 19.03.13, build 4484c46d9d

Quoted 4 lines
$ bitbake pkgconfig-native
| configure: error: I can't find the libraries for the thread
implementation
...

Can you please tell me what patches I need to apply to get this to work?

How will this be handled upstream?

I guess people would want to run a build in an unprivileged container.

Regards,

Robert

#7 lore raw

Re: [OE-core] Upgrade uninative to fix multiple native build failures in old unprivileged docker

Quoted 5 lines · Robert Berger

Hi,

I just experienced this problem in Docker version 19.03.13, build 4484c46d9d

Quoted 7 lines
Quoted 4 lines
$ bitbake pkgconfig-native
| configure: error: I can't find the libraries for the thread
implementation
...

Can you please tell me what patches I need to apply to get this to work?

https://lists.openembedded.org/g/openembedded-core/message/155465

and rebuild new uninative tarball with this applied.

Quoted 2 lines

How will this be handled upstream?

Hopefully v20.10.9 docker will be released soon with:
https://github.com/moby/moby/commit/9f6b562dd12ef7b1f9e2f8e6f2ab6477790a6594

#8 lore raw

Re: [OE-core] Upgrade uninative to fix multiple native build failures in old unprivileged docker

In reply to Robert Berger
Quoted 9 lines · Robert Berger

Hi,

I just experienced this problem in Docker version 19.03.13, build 4484c46d9d

Quoted 4 lines
$ bitbake pkgconfig-native
| configure: error: I can't find the libraries for the thread
implementation
...
Quoted 5 lines

Can you please tell me what patches I need to apply to get this to work?

How will this be handled upstream?

I guess people would want to run a build in an unprivileged container.

You could also revert the last uninative upgrade and use an older glibc uninative for now.

We'll get a fix merged soon and a new uninative built which can work around the issue.

Cheers,

Richard

#2 lore raw
In reply to hongxu
Quoted 14 lines · Hongxu Jia

# Changed in V3: add missing Upstream-Status tag

Since upstream commit [d8ea0d0168 Add an internal wrapper for clone, clone2 and clone3] applied, start a unprivileged container (docker run without --privileged), it creates a thread failed in container.

In commit d8ea0d0168, it calls __clone3 if HAVE_CLONE3_WAPPER is defined. If __clone3 returns -1 with ENOSYS, fall back to clone or clone2.

The newest docker has fixed the issue in commit [1], but it was applied only on master, to backward compatibility with old docker, discussed with glibc maintainer [2], explicitly disable clone3 wrapper work by removing macro definition of HAVE_CLONE3_WRAPPER

Quoted 4 lines

[1] https://github.com/moby/moby/commit/9f6b562dd12ef7b1f9e2f8e6f2ab6477790a6594
[2] https://sourceware.org/pipermail/libc-alpha/2021-August/130591.html

Just for reference, I'm going with v1 of this fix.

The reason is that I do want clone3 to work on target and I do want the wrapper to be available. Falling back in the EPERM case is therefore easiest.

I did wonder about making the fix nativesdk specific however I decided that was going to complicate upgrades and so on a bit too much. The v1 fix is simple enough it can be used everywhere without too many side effects.

Cheers,

Richard

author
#3 lore raw
Quoted 21 lines · Richard Purdie

[Please note: This e-mail is from an EXTERNAL e-mail address]

Quoted 14 lines · Hongxu Jia

# Changed in V3: add missing Upstream-Status tag

Since upstream commit [d8ea0d0168 Add an internal wrapper for clone, clone2 and clone3] applied, start a unprivileged container (docker run without --privileged), it creates a thread failed in container.

In commit d8ea0d0168, it calls __clone3 if HAVE_CLONE3_WAPPER is defined. If __clone3 returns -1 with ENOSYS, fall back to clone or clone2.

The newest docker has fixed the issue in commit [1], but it was applied only on master, to backward compatibility with old docker, discussed with glibc maintainer [2], explicitly disable clone3 wrapper work by removing macro definition of HAVE_CLONE3_WRAPPER

Quoted 4 lines

[1] https://github.com/moby/moby/commit/9f6b562dd12ef7b1f9e2f8e6f2ab6477790a6594
[2] https://sourceware.org/pipermail/libc-alpha/2021-August/130591.html

Quoted 8 lines

Just for reference, I'm going with v1 of this fix.

The reason is that I do want clone3 to work on target and I do want the wrapper to be available. Falling back in the EPERM case is therefore easiest.

I did wonder about making the fix nativesdk specific however I decided that was going to complicate upgrades and so on a bit too much. The v1 fix is simple enough it can be used everywhere without too many side effects.

Absolutely agree

//Hongxu

Quoted 4 lines

Cheers,

Richard