Skip to contentSkip to conversation

Toolchain test results

OE-core Richard Purdie 3 participants 6 messages Jul 20, 2023 – Jul 27, 2023 2 messages not in the archive

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

#1 lore raw
Quoted 9 lines · Richard Purdie via lists.openembedded.org

qemuarm has ~350 failures
qemuarm64 has ~350 failures
qemux86-64 has ~4000 (3900 in glibc)
qemux86 has ~4000 (3500 in glibc)
qemuppc has ~600 failures
qemumips64 has ~5000 failures (all over)
qemumips has ~1600 failures

Anuj: Can Intel look into the glibc test failures on x86?

I realised the glibc issues were due to the network being disabled for the tests and have sent a patch to fix that. That reduces the failures from ~3900 to ~330. We should really try and reduce that further but it is a start!

Cheers,

Richard

#2 lore raw
Quoted 12 lines · Richard Purdie
Quoted 9 lines · Richard Purdie via lists.openembedded.org

qemuarm has ~350 failures
qemuarm64 has ~350 failures
qemux86-64 has ~4000 (3900 in glibc)
qemux86 has ~4000 (3500 in glibc)
qemuppc has ~600 failures
qemumips64 has ~5000 failures (all over)
qemumips has ~1600 failures

Anuj: Can Intel look into the glibc test failures on x86?

Quoted 8 lines

I realised the glibc issues were due to the network being disabled for
the tests and have sent a patch to fix that. That reduces the failures
from ~3900 to ~330. We should really try and reduce that further but it
is a start!

A lot of locale/iconv tests seemed to be failing when calling write with large buffers/files over NFS. Some of others were triggering OOM.

I ran the tests again after making a few changes:

https://autobuilder.yocto.io/pub/non-release/20230726-11/testresults/qemux86-64-tc/

After switching NFS mount to TCP and increasing the memory available to 1024, the number of tests failed came down to 69.

=======================================================================
=======================================
qemux86-64 PTest Result Summary
=======================================================================
=======================================
-----------------------------------------------------------------------
---------------------------------------
Recipe | Passed | Failed | Skipped | Time(s)
-----------------------------------------------------------------------
---------------------------------------

binutils         | 289          | 0        | 8           | -
binutils-gas     | 1582         | 4        | 1           | -
binutils-ld      | 1622         | 6        | 111         | -
gcc              | 149696       | 28       | 27599       | -
gcc-g++          | 219260       | 30       | 21108       | -
gcc-libatomic    | 27           | 1        | 27          | -
gcc-libgomp      | 3426         | 1        | 1962        | -
gcc-libitm       | 24           | 1        | 24          | -
gcc-libstdc++-v3 | 9695         | 32       | 5459        | -
glibc            | 4984         | 69       | 147         | -
rust             | 15768        | 0        | 554         | -

-----------------------------------------------------------------------
---------------------------------------

Thanks,

Anuj

#3 lore raw
Quoted 26 lines · Anuj Mittal
Quoted 12 lines · Richard Purdie
Quoted 9 lines · Richard Purdie via lists.openembedded.org

qemuarm has ~350 failures
qemuarm64 has ~350 failures
qemux86-64 has ~4000 (3900 in glibc)
qemux86 has ~4000 (3500 in glibc)
qemuppc has ~600 failures
qemumips64 has ~5000 failures (all over)
qemumips has ~1600 failures

Anuj: Can Intel look into the glibc test failures on x86?

Quoted 8 lines

I realised the glibc issues were due to the network being disabled for
the tests and have sent a patch to fix that. That reduces the failures
from ~3900 to ~330. We should really try and reduce that further but it
is a start!

A lot of locale/iconv tests seemed to be failing when calling write with large buffers/files over NFS. Some of others were triggering OOM.

Quoted 7 lines

I ran the tests again after making a few changes:

https://autobuilder.yocto.io/pub/non-release/20230726-11/testresults/qemux86-64-tc/

After switching NFS mount to TCP and increasing the memory available to 1024, the number of tests failed came down to 69.

This is a nice, thanks for doing it. I looked quickly at your results especially glibc part
and it seems some of remaining failures are in nss module. I could not see detail logs
why those tests were failing but few things to check is if we install

libnss-db and glibc-extra-nss packages into image or not. some of these tests edit
/etc/nsswitch.conf, so are these tests running in parallel and racing.

Quoted 36 lines

=======================================================================
=======================================
qemux86-64 PTest Result Summary
=======================================================================
=======================================
-----------------------------------------------------------------------
---------------------------------------
Recipe | Passed | Failed | Skipped | Time(s)
-----------------------------------------------------------------------
---------------------------------------

binutils         | 289          | 0        | 8           | -
binutils-gas     | 1582         | 4        | 1           | -
binutils-ld      | 1622         | 6        | 111         | -
gcc              | 149696       | 28       | 27599       | -
gcc-g++          | 219260       | 30       | 21108       | -
gcc-libatomic    | 27           | 1        | 27          | -
gcc-libgomp      | 3426         | 1        | 1962        | -
gcc-libitm       | 24           | 1        | 24          | -
gcc-libstdc++-v3 | 9695         | 32       | 5459        | -
glibc            | 4984         | 69       | 147         | -
rust             | 15768        | 0        | 554         | -

-----------------------------------------------------------------------
---------------------------------------

Thanks,

Anuj

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#184858): https://lists.openembedded.org/g/openembedded-core/message/184858
Mute This Topic: https://lists.openembedded.org/mt/100212267/1997914
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

#4 lore raw
Quoted 40 lines · Khem Raj
Quoted 29 lines · Anuj Mittal
Quoted 12 lines · Richard Purdie
Quoted 9 lines · Richard Purdie via lists.openembedded.org

qemuarm has ~350 failures
qemuarm64 has ~350 failures
qemux86-64 has ~4000 (3900 in glibc)
qemux86 has ~4000 (3500 in glibc)
qemuppc has ~600 failures
qemumips64 has ~5000 failures (all over)
qemumips has ~1600 failures

Anuj: Can Intel look into the glibc test failures on x86?

Quoted 10 lines

I realised the glibc issues were due to the network being disabled
for
the tests and have sent a patch to fix that. That reduces the failures
from ~3900 to ~330. We should really try and reduce that further but
it
is a start!

A lot of locale/iconv tests seemed to be failing when calling write with large buffers/files over NFS. Some of others were triggering OOM.

Quoted 8 lines

I ran the tests again after making a few changes:

https://autobuilder.yocto.io/pub/non-release/20230726-11/testresults/qemux86-64-tc/

After switching NFS mount to TCP and increasing the memory available to
1024, the number of tests failed came down to 69.

Quoted 6 lines

This is a nice, thanks for doing it. I looked quickly at your results especially glibc part
and it seems some of remaining failures are in nss module. I could not
see detail logs
why those tests were failing but few things to check is if we install

All the nss tests are failing here:

https://github.com/bminor/glibc/blob/master/support/test-container.c#L842

Could be a NFS problem as well. I have not looked at it in much detail yet.

Thanks,

Anuj

A message that is not in the archive (sent off-list or not yet received): [email protected]

#6 lore raw
Quoted 53 lines · Anuj Mittal
Quoted 43 lines · Khem Raj
Quoted 31 lines · Anuj Mittal
Quoted 12 lines · Richard Purdie
Quoted 9 lines · Richard Purdie via lists.openembedded.org

qemuarm has ~350 failures
qemuarm64 has ~350 failures
qemux86-64 has ~4000 (3900 in glibc)
qemux86 has ~4000 (3500 in glibc)
qemuppc has ~600 failures
qemumips64 has ~5000 failures (all over)
qemumips has ~1600 failures

Anuj: Can Intel look into the glibc test failures on x86?

Quoted 11 lines

I realised the glibc issues were due to the network being disabled
for
the tests and have sent a patch to fix that. That reduces the failures
from ~3900 to ~330. We should really try and reduce that further
but
it
is a start!

A lot of locale/iconv tests seemed to be failing when calling write
with large buffers/files over NFS. Some of others were triggering OOM.

Quoted 8 lines

I ran the tests again after making a few changes:

https://autobuilder.yocto.io/pub/non-release/20230726-11/testresults/qemux86-64-tc/

After switching NFS mount to TCP and increasing the memory available to
1024, the number of tests failed came down to 69.

Quoted 8 lines

This is a nice, thanks for doing it. I looked quickly at your results
especially glibc part
and it seems some of remaining failures are in nss module. I could not
see detail logs
why those tests were failing but few things to check is if we install

Quoted 4 lines

All the nss tests are failing here:

https://github.com/bminor/glibc/blob/master/support/test-container.c#L842

I took a look at this and these are failing because of two reasons. We're looking at the wrong path when creating that file because of OBJDIR_PATH we set here:

https://git.yoctoproject.org/poky/tree/meta/recipes-core/glibc/glibc/0022-Avoid-hardcoded-build-time-paths-in-the-output-binar.patch

That path is not correct for glibc-testsuite.

Next, the test tries to acquire a lock to a file on NFS which times out because the user space server we use doesn't support file locking with NLM.

After reverting this patch for testing and disabling locking, all the tests that need the test container and some others pass:

https://autobuilder.yocto.io/pub/non-release/20230727-9/testresults/qemux86-64-tc/

glibc | 5016 | 40 | 144 | -

Thanks,

Anuj

#7 lore raw
In reply to Mittal, Anuj
Quoted 53 lines · Anuj
Quoted 12 lines · Richard Purdie
Quoted 9 lines · Richard Purdie via lists.openembedded.org

qemuarm has ~350 failures
qemuarm64 has ~350 failures
qemux86-64 has ~4000 (3900 in glibc)
qemux86 has ~4000 (3500 in glibc)
qemuppc has ~600 failures
qemumips64 has ~5000 failures (all over)
qemumips has ~1600 failures

Anuj: Can Intel look into the glibc test failures on x86?

Quoted 8 lines

I realised the glibc issues were due to the network being disabled for
the tests and have sent a patch to fix that. That reduces the failures
from ~3900 to ~330. We should really try and reduce that further but it
is a start!

A lot of locale/iconv tests seemed to be failing when calling write with large buffers/files over NFS. Some of others were triggering OOM.

I ran the tests again after making a few changes:

https://autobuilder.yocto.io/pub/non-release/20230726-11/testresults/qemux86-64-tc/

After switching NFS mount to TCP and increasing the memory available to 1024, the number of tests failed came down to 69.

=======================================================================
=======================================
qemux86-64 PTest Result Summary
=======================================================================
=======================================
-----------------------------------------------------------------------
---------------------------------------
Recipe | Passed | Failed | Skipped | Time(s)
-----------------------------------------------------------------------
---------------------------------------

binutils         | 289          | 0        | 8           | -
binutils-gas     | 1582         | 4        | 1           | -
binutils-ld      | 1622         | 6        | 111         | -
gcc              | 149696       | 28       | 27599       | -
gcc-g++          | 219260       | 30       | 21108       | -
gcc-libatomic    | 27           | 1        | 27          | -
gcc-libgomp      | 3426         | 1        | 1962        | -
gcc-libitm       | 24           | 1        | 24          | -
gcc-libstdc++-v3 | 9695         | 32       | 5459        | -
glibc            | 4984         | 69       | 147         | -
rust             | 15768        | 0        | 554         | -
Quoted 2 lines

-----------------------------------------------------------------------
---------------------------------------

That looks like really good progress, thanks Anuj!

I think we might rebuild M2 to allow for some of the resulttool, ltp and other test fixes to make it in. I want to look into this 32 bit arm regression first to see what is going on there. If you're able to send some of these out we should pull them in and improve the failures for glibc too.

Cheers,

Richard