Skip to contentSkip to conversation

uninative: Update to 2.7 release

PATCH In review Michael Halstead 4 participants 5 messages Oct 7, 2019 – Oct 11, 2019

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 uninative: Update to 2.7 release

The 2.7 release updates glibc to version 2.30. Recently added to openSUSE Tumbleweed and needed for Fedora Core 31.

---
 meta/conf/distro/include/yocto-uninative.inc | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)
meta/conf/distro/include/yocto-uninative.inc +5 -5
index df24346855..ad75d3e2a3 100644
@@ -6,9 +6,9 @@
 # to the distro running on the build machine.
 #
 
-UNINATIVE_MAXGLIBCVERSION = "2.29"
+UNINATIVE_MAXGLIBCVERSION = "2.30"
 
-UNINATIVE_URL ?= "http://downloads.yoctoproject.org/releases/uninative/2.6/"
-UNINATIVE_CHECKSUM[aarch64] ?= "a37118fc8b423f48146120707b81dd15017512c3e8ef9e6ca2cb3a033f4f4046"
-UNINATIVE_CHECKSUM[i686] ?= "3234fc3ded810225071f23a0e9a99f4f8c2480059945a848eff076ce78122ade"
-UNINATIVE_CHECKSUM[x86_64] ?= "133387753a9acf3e1b788103c59fac91e968e2ee331d7a4b9498e926ada7be57"
+UNINATIVE_URL ?= "http://downloads.yoctoproject.org/releases/uninative/2.7/"
+UNINATIVE_CHECKSUM[aarch64] ?= "e76a45886ee8a0b3904b761c17ac8ff91edf9811ee455f1832d10763ba794dfc"
+UNINATIVE_CHECKSUM[i686] ?= "810d027dfb1c7675226afbcec07808770516c969ee7378f6d8240281083f8924"
+UNINATIVE_CHECKSUM[x86_64] ?= "9498d8bba047499999a7310ac2576d0796461184965351a56f6d32c888a1f216"
Signature · 1 line
2.21.0
#1 lore raw

this helps archlinux too

Quoted 36 lines · Michael Halstead

The 2.7 release updates glibc to version 2.30. Recently added to openSUSE Tumbleweed and needed for Fedora Core 31.

---
 meta/conf/distro/include/yocto-uninative.inc | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)
meta/conf/distro/include/yocto-uninative.inc +5 -6
index df24346855..ad75d3e2a3 100644
@@ -6,9 +6,9 @@
 # to the distro running on the build machine.
 #
 
-UNINATIVE_MAXGLIBCVERSION = "2.29"
+UNINATIVE_MAXGLIBCVERSION = "2.30"
 
-UNINATIVE_URL ?= "http://downloads.yoctoproject.org/releases/uninative/2.6/"
-UNINATIVE_CHECKSUM[aarch64] ?= "a37118fc8b423f48146120707b81dd15017512c3e8ef9e6ca2cb3a033f4f4046"
-UNINATIVE_CHECKSUM[i686] ?= "3234fc3ded810225071f23a0e9a99f4f8c2480059945a848eff076ce78122ade"
-UNINATIVE_CHECKSUM[x86_64] ?= "133387753a9acf3e1b788103c59fac91e968e2ee331d7a4b9498e926ada7be57"
+UNINATIVE_URL ?= "http://downloads.yoctoproject.org/releases/uninative/2.7/"
+UNINATIVE_CHECKSUM[aarch64] ?= "e76a45886ee8a0b3904b761c17ac8ff91edf9811ee455f1832d10763ba794dfc"
+UNINATIVE_CHECKSUM[i686] ?= "810d027dfb1c7675226afbcec07808770516c969ee7378f6d8240281083f8924"
+UNINATIVE_CHECKSUM[x86_64] ?= "9498d8bba047499999a7310ac2576d0796461184965351a56f6d32c888a1f216"
--

2.21.0

--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core

#2 lore raw
In reply to Michael Halstead
Quoted 2 lines · Michael Halstead

The 2.7 release updates glibc to version 2.30. Recently added to openSUSE Tumbleweed and needed for Fedora Core 31.

Since we updated master to include this commit we see regularly the following issues:

WARNING: Disabling uninative as unable to install uninative tarball: Command 'mkdir -p /workdir/oe/tmp/sysroots-uninative; cd /workdir/oe/tmp/sysroots-uninative; tar -xJf
/workdir/downloads/uninative//9498d8bba047499999a7310ac2576d0796461184965351a56f6d32c888a1f216/x86_64-nativesdk-libc.tar.xz; /workdir/oe/tmp/sysroots-uninative/relocate_sdk.py /workdir/oe/tmp/sysroots-uninative/x86_64-linux
/workdir/oe/tmp/sysroots-uninative/x86_64-linux/lib/ld-linux-x86-64.so.2 /workdir/oe/tmp/sysroots-uninative/x86_64-linux/lib/ld-linux-x86-64.so.2 /workdir/oe/tmp/sysroots-uninative/x86_64-linux//usr/bin/patchelf-uninative
/workdir/oe/tmp/sysroots-uninative/x86_64-linux/lib/libc*.so' returned non-zero exit status 1.
WARNING: To build your own uninative loader, please bitbake uninative-tarball and set UNINATIVE_TARBALL appropriately.

[...]

ERROR: ldconfig-native-2.12.1-r2 do_populate_sysroot_setscene: Error executing a python function in exec_python_func() autogenerated:

The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>

     0001:
 *** 0002:uninative_changeinterp(d)
     0003:

File: '/workdir/oe/build/conf/../../layers/openembedded-core/meta/classes/uninative.bbclass', lineno: 165, function: uninative_changeinterp
0161: continue
0162: if not elf.isDynamic():
0163: continue
0164:
*** 0165: subprocess.check_output(("patchelf-uninative", "--set-interpreter", d.getVar("UNINATIVE_LOADER"), f), stderr=subprocess.STDOUT)

     0166:}

File: '/usr/lib/python3.6/subprocess.py', lineno: 336, function: check_output
0332: # empty string. That is maintained here for backwards compatibility.
0333: kwargs['input'] = '' if kwargs.get('universal_newlines', False) else b''
0334:
0335: return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
*** 0336: **kwargs).stdout

     0337:
     0338:
     0339:class CompletedProcess(object):
     0340:    """A process that has finished running.

File: '/usr/lib/python3.6/subprocess.py', lineno: 403, function: run
0399: if 'stdin' in kwargs:
0400: raise ValueError('stdin and input arguments may not both be used.')
0401: kwargs['stdin'] = PIPE
0402:
*** 0403: with Popen(*popenargs, **kwargs) as process:
0404: try:
0405: stdout, stderr = process.communicate(input, timeout=timeout)
0406: except TimeoutExpired:
0407: process.kill()
File: '/usr/lib/python3.6/subprocess.py', lineno: 709, function: __init__

     0705:                                startupinfo, creationflags, shell,
     0706:                                p2cread, p2cwrite,
     0707:                                c2pread, c2pwrite,
     0708:                                errread, errwrite,
 *** 0709:                                restore_signals, start_new_session)

0710: except:
0711: # Cleanup if the child failed starting.
0712: for f in filter(None, (self.stdin, self.stdout, self.stderr)):
0713: try:
File: '/usr/lib/python3.6/subprocess.py', lineno: 1344, function: _execute_child

     1340:                    if errno_num != 0:
     1341:                        err_msg = os.strerror(errno_num)
     1342:                        if errno_num == errno.ENOENT:
     1343:                            err_msg += ': ' + repr(err_filename)

*** 1344: raise child_exception_type(errno_num, err_msg, err_filename)
1345: raise child_exception_type(err_msg)
1346:
1347:
1348: def _handle_exitstatus(self, sts, _WIFSIGNALED=os.WIFSIGNALED,
Exception: FileNotFoundError: [Errno 2] No such file or directory: 'patchelf-uninative': 'patchelf-uninative'

ERROR: Logfile of failure stored in: /workdir/oe/tmp/work/x86_64-linux/ldconfig-native/2.12.1-r2/temp/log.do_populate_sysroot_setscene.14967

Looking into the directory shows that uninative only got extracted half-way:

$ find /workdir/oe/tmp/sysroots-uninative
/workdir/oe/tmp/sysroots-uninative
/workdir/oe/tmp/sysroots-uninative/x86_64-linux
/workdir/oe/tmp/sysroots-uninative/x86_64-linux/lib
/workdir/oe/tmp/sysroots-uninative/x86_64-linux/lib/libresolv-2.30.so
/workdir/oe/tmp/sysroots-uninative/x86_64-linux/lib/libnsl.so.1
/workdir/oe/tmp/sysroots-uninative/x86_64-linux/lib/libanl.so.1
/workdir/oe/tmp/sysroots-uninative/x86_64-linux/lib/libnss_compat-2.30.so
/workdir/oe/tmp/sysroots-uninative/x86_64-linux/lib/libpthread-2.30.so
/workdir/oe/tmp/sysroots-uninative/x86_64-linux/lib/ld-2.30.so
/workdir/oe/tmp/sysroots-uninative/x86_64-linux/lib/libutil-2.30.so
/workdir/oe/tmp/sysroots-uninative/x86_64-linux/lib/libnss_compat.so.2
/workdir/oe/tmp/sysroots-uninative/x86_64-linux/lib/libdl.so.2
/workdir/oe/tmp/sysroots-uninative/x86_64-linux/lib/libc-2.30.so
/workdir/oe/tmp/sysroots-uninative/x86_64-linux/lib/ld-linux-x86-64.so.2
/workdir/oe/tmp/sysroots-uninative/x86_64-linux/lib/librt.so.1
/workdir/oe/tmp/sysroots-uninative/x86_64-linux/lib/libBrokenLocale.so.1
/workdir/oe/tmp/sysroots-uninative/x86_64-linux/var
/workdir/oe/tmp/sysroots-uninative/x86_64-linux/var/lib
/workdir/oe/tmp/sysroots-uninative/x86_64-linux/var/lib/rpm
/workdir/oe/tmp/sysroots-uninative/relocate_sdk.py
<end of output...>

We run several build in parallel using the same downloads folder, it seems to me that there is a race condition. The history shows the warning several times already with exit code 127, but in those instances the bulid succeeded

What I don't understand is how the update to 2.7 release could affect the behavior such that it suddenly leads to issues. Any idea?

--
Stefan

Quoted 34 lines
---
 meta/conf/distro/include/yocto-uninative.inc | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)
meta/conf/distro/include/yocto-uninative.inc +1 -3
index df24346855..ad75d3e2a3 100644
@@ -6,9 +6,9 @@
 # to the distro running on the build machine.
 #
 
-UNINATIVE_MAXGLIBCVERSION = "2.29"
+UNINATIVE_MAXGLIBCVERSION = "2.30"
 
-UNINATIVE_URL ?= "http://downloads.yoctoproject.org/releases/uninative/2.6/"
-UNINATIVE_CHECKSUM[aarch64] ?=

"a37118fc8b423f48146120707b81dd15017512c3e8ef9e6ca2cb3a033f4f4046" -UNINATIVE_CHECKSUM[i686] ?= "3234fc3ded810225071f23a0e9a99f4f8c2480059945a848eff076ce78122ade" -UNINATIVE_CHECKSUM[x86_64] ?= "133387753a9acf3e1b788103c59fac91e968e2ee331d7a4b9498e926ada7be57"

meta/conf/distro/include/yocto-uninative.inc +2 -0
+UNINATIVE_URL ?= "http://downloads.yoctoproject.org/releases/uninative/2.7/"
+UNINATIVE_CHECKSUM[aarch64] ?=

"e76a45886ee8a0b3904b761c17ac8ff91edf9811ee455f1832d10763ba794dfc" +UNINATIVE_CHECKSUM[i686] ?= "810d027dfb1c7675226afbcec07808770516c969ee7378f6d8240281083f8924" +UNINATIVE_CHECKSUM[x86_64] ?= "9498d8bba047499999a7310ac2576d0796461184965351a56f6d32c888a1f216"

Signature · 1 line
2.21.0
#3 lore raw
Quoted 9 lines · Stefan Agner

We run several build in parallel using the same downloads folder, it seems to me that
there is a race condition. The history shows the warning several times already with
exit code 127, but in those instances the bulid succeeded

What I don't understand is how the update to 2.7 release could affect the behavior
such that it suddenly leads to issues. Any idea?

We use the bitbake fetcher for uninative so there should be locking, the same as any other download.

Are these builds in parallel on different machines with a shared network filesystem for downloads or the same physical machine?

Are you using any special MIRRORS/PREMIRRORS?

FWIW we run many builds on the autobuilder and don't see this. I also haven't seen any other reports of it.

I'm therefore at a loss as to why the locking would seem to be breaking for you :(

Cheers,

Richard

#4 lore raw
In reply to Stefan Agner
Quoted 130 lines · Stefan Agner
Quoted 2 lines · Michael Halstead

The 2.7 release updates glibc to version 2.30. Recently added to openSUSE Tumbleweed and needed for Fedora Core 31.

Since we updated master to include this commit we see regularly the following issues:

WARNING: Disabling uninative as unable to install uninative tarball:
Command 'mkdir -p /workdir/oe/tmp/sysroots-uninative; cd
/workdir/oe/tmp/sysroots-uninative; tar -xJf /workdir/downloads/uninative//9498d8bba047499999a7310ac2576d0796461184965351a56f6d32c888a1f216/x86_64-nativesdk-libc.tar.xz; /workdir/oe/tmp/sysroots-uninative/relocate_sdk.py
/workdir/oe/tmp/sysroots-uninative/x86_64-linux /workdir/oe/tmp/sysroots-uninative/x86_64-linux/lib/ld-linux-x86-64.so.2

/workdir/oe/tmp/sysroots-uninative/x86_64-linux/lib/ld-linux-x86-64.so.2

/workdir/oe/tmp/sysroots-uninative/x86_64-linux//usr/bin/patchelf-uninative

/workdir/oe/tmp/sysroots-uninative/x86_64-linux/lib/libc*.so' returned non-zero exit status 1.
WARNING: To build your own uninative loader, please bitbake uninative-tarball and set UNINATIVE_TARBALL appropriately.

[...]

ERROR: ldconfig-native-2.12.1-r2 do_populate_sysroot_setscene: Error executing a python function in exec_python_func() autogenerated:

The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>

     0001:
 *** 0002:uninative_changeinterp(d)
     0003:

File:
'/workdir/oe/build/conf/../../layers/openembedded-core/meta/classes/uninative.bbclass',
lineno: 165, function: uninative_changeinterp
0161: continue
0162: if not elf.isDynamic():
0163: continue
0164:
*** 0165: subprocess.check_output(("patchelf-uninative",
"--set-interpreter", d.getVar("UNINATIVE_LOADER"), f), stderr=subprocess.STDOUT)

     0166:}
File: '/usr/lib/python3.6/subprocess.py', lineno: 336, function: check_output
     0332:        # empty string. That is maintained here for
backwards compatibility.
     0333:        kwargs['input'] = '' if

kwargs.get('universal_newlines', False) else b''
0334:
0335: return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
*** 0336: **kwargs).stdout

     0337:
     0338:
     0339:class CompletedProcess(object):
     0340:    """A process that has finished running.

File: '/usr/lib/python3.6/subprocess.py', lineno: 403, function: run
0399: if 'stdin' in kwargs:
0400: raise ValueError('stdin and input arguments may
not both be used.')

     0401:        kwargs['stdin'] = PIPE
     0402:

*** 0403: with Popen(*popenargs, **kwargs) as process:
0404: try:
0405: stdout, stderr = process.communicate(input,
timeout=timeout)
0406: except TimeoutExpired:
0407: process.kill()
File: '/usr/lib/python3.6/subprocess.py', lineno: 709, function: __init__

     0705:                                startupinfo, creationflags, shell,
     0706:                                p2cread, p2cwrite,
     0707:                                c2pread, c2pwrite,
     0708:                                errread, errwrite,
 *** 0709:                                restore_signals, start_new_session)

0710: except:
0711: # Cleanup if the child failed starting.
0712: for f in filter(None, (self.stdin, self.stdout,
self.stderr)):
0713: try:
File: '/usr/lib/python3.6/subprocess.py', lineno: 1344, function: _execute_child

     1340:                    if errno_num != 0:
     1341:                        err_msg = os.strerror(errno_num)
     1342:                        if errno_num == errno.ENOENT:
     1343:                            err_msg += ': ' + repr(err_filename)

*** 1344: raise child_exception_type(errno_num,
err_msg, err_filename)
1345: raise child_exception_type(err_msg)
1346:
1347:
1348: def _handle_exitstatus(self, sts, _WIFSIGNALED=os.WIFSIGNALED,
Exception: FileNotFoundError: [Errno 2] No such file or directory:
'patchelf-uninative': 'patchelf-uninative'

ERROR: Logfile of failure stored in:
/workdir/oe/tmp/work/x86_64-linux/ldconfig-native/2.12.1-r2/temp/log.do_populate_sysroot_setscene.14967

Looking into the directory shows that uninative only got extracted half-way:

$ find /workdir/oe/tmp/sysroots-uninative
/workdir/oe/tmp/sysroots-uninative
/workdir/oe/tmp/sysroots-uninative/x86_64-linux
/workdir/oe/tmp/sysroots-uninative/x86_64-linux/lib
/workdir/oe/tmp/sysroots-uninative/x86_64-linux/lib/libresolv-2.30.so
/workdir/oe/tmp/sysroots-uninative/x86_64-linux/lib/libnsl.so.1
/workdir/oe/tmp/sysroots-uninative/x86_64-linux/lib/libanl.so.1
/workdir/oe/tmp/sysroots-uninative/x86_64-linux/lib/libnss_compat-2.30.so
/workdir/oe/tmp/sysroots-uninative/x86_64-linux/lib/libpthread-2.30.so
/workdir/oe/tmp/sysroots-uninative/x86_64-linux/lib/ld-2.30.so
/workdir/oe/tmp/sysroots-uninative/x86_64-linux/lib/libutil-2.30.so
/workdir/oe/tmp/sysroots-uninative/x86_64-linux/lib/libnss_compat.so.2
/workdir/oe/tmp/sysroots-uninative/x86_64-linux/lib/libdl.so.2
/workdir/oe/tmp/sysroots-uninative/x86_64-linux/lib/libc-2.30.so
/workdir/oe/tmp/sysroots-uninative/x86_64-linux/lib/ld-linux-x86-64.so.2
/workdir/oe/tmp/sysroots-uninative/x86_64-linux/lib/librt.so.1
/workdir/oe/tmp/sysroots-uninative/x86_64-linux/lib/libBrokenLocale.so.1
/workdir/oe/tmp/sysroots-uninative/x86_64-linux/var
/workdir/oe/tmp/sysroots-uninative/x86_64-linux/var/lib
/workdir/oe/tmp/sysroots-uninative/x86_64-linux/var/lib/rpm
/workdir/oe/tmp/sysroots-uninative/relocate_sdk.py
<end of output...>

We run several build in parallel using the same downloads folder, it seems to me that
there is a race condition. The history shows the warning several times already with
exit code 127, but in those instances the bulid succeeded

Quoted 3 lines

What I don't understand is how the update to 2.7 release could affect the behavior
such that it suddenly leads to issues. Any idea?

There seems to be definitely a race in uninative handling, this is from build/bitbake-cookerdaemon.log on the failing build:
Running command ['buildTargets', ['torizon-core-docker'], 'build']
xz: (stdin): Unexpected end of input
tar: Unexpected EOF in archive
tar: Unexpected EOF in archive
tar: Error is not recoverable: exiting now
Traceback (most recent call last):
File "/workdir/oe/tmp/sysroots-uninative/relocate_sdk.py", line 219,
in <module>
perms = os.stat(e)[stat.ST_MODE]
FileNotFoundError: [Errno 2] No such file or directory:
'/workdir/oe/tmp/sysroots-uninative/x86_64-linux//usr/bin/patchelf-uninative'

It seems that the recipe checks only for existence of the tar.xz, which of course could be partway downloaded by a parallel running build. I will send a patch to fix this. But what I still don't quite understand is why we see that all of the sudden as the race seems to be there already since a while.

--
Stefan

Quoted 39 lines

--
Stefan

Quoted 34 lines
---
 meta/conf/distro/include/yocto-uninative.inc | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)
meta/conf/distro/include/yocto-uninative.inc +1 -3
index df24346855..ad75d3e2a3 100644
@@ -6,9 +6,9 @@
 # to the distro running on the build machine.
 #
 
-UNINATIVE_MAXGLIBCVERSION = "2.29"
+UNINATIVE_MAXGLIBCVERSION = "2.30"
 
-UNINATIVE_URL ?= "http://downloads.yoctoproject.org/releases/uninative/2.6/"
-UNINATIVE_CHECKSUM[aarch64] ?=

"a37118fc8b423f48146120707b81dd15017512c3e8ef9e6ca2cb3a033f4f4046" -UNINATIVE_CHECKSUM[i686] ?= "3234fc3ded810225071f23a0e9a99f4f8c2480059945a848eff076ce78122ade" -UNINATIVE_CHECKSUM[x86_64] ?= "133387753a9acf3e1b788103c59fac91e968e2ee331d7a4b9498e926ada7be57"

meta/conf/distro/include/yocto-uninative.inc +2 -0
+UNINATIVE_URL ?= "http://downloads.yoctoproject.org/releases/uninative/2.7/"
+UNINATIVE_CHECKSUM[aarch64] ?=

"e76a45886ee8a0b3904b761c17ac8ff91edf9811ee455f1832d10763ba794dfc" +UNINATIVE_CHECKSUM[i686] ?= "810d027dfb1c7675226afbcec07808770516c969ee7378f6d8240281083f8924" +UNINATIVE_CHECKSUM[x86_64] ?= "9498d8bba047499999a7310ac2576d0796461184965351a56f6d32c888a1f216" --
2.21.0