Skip to contentSkip to conversation

❌ FAIL: Test report for kernel 5.4.12-100.fc30 (fedora-30)

LTP CKI Project 3 participants 3 messages Jan 15, 2020 – Jan 16, 2020

lore
Share & tools
lore
#0 lore raw

Hello jcline,

We ran automated tests on a kernel build that you sent to Koji:

    Kernel package: kernel-5.4.12-100.fc30
         Koji task: https://koji.fedoraproject.org/koji/taskinfo?taskID=40542166

The results of these automated tests are provided below.

Overall result: FAILED (see details below)
Tests: FAILED

One or more kernel tests failed:

    ppc64le:
     ? LTP


    Pipeline: https://xci32.lab.eng.rdu2.redhat.com/cki-project/brew-pipeline/pipelines/384229

We hope that these logs can help you find the problem quickly. For the full detail on our testing procedures, please scroll to the bottom of this message.

Please reply to this email if you have any questions about the tests that we ran or if you have any suggestions on how to make future tests more effective.

        ,-.   ,-.
       ( C ) ( K )  Continuous
        `-',-.`-'   Kernel
          ( I )     Integration
           `-'

______________________________________________________________________________

Hardware testing
----------------
All the testing jobs are listed here:

https://beaker.engineering.redhat.com/jobs/?jobsearch-0.table=Whiteboard&jobsearch-0.operation=contains&jobsearch-0.value=cki%40gitlab%3A384229

We booted each kernel and ran the following tests:

aarch64:
Host 1: https://beaker.engineering.redhat.com/recipes/7797420

? Internal infrastructure issues prevented one or more tests (marked with ???) from running on this architecture.
This is not the fault of the kernel that was tested.

??? Boot test
??? xfstests: ext4
??? xfstests: xfs
??? lvm thinp sanity
??? stress: stress-ng
? ??? Storage blktests

Host 2: https://beaker.engineering.redhat.com/recipes/7797419
? Boot test
? LTP
? Loopdev Sanity
? Memory function: memfd_create
? AMTU (Abstract Machine Test Utility)
? Ethernet drivers sanity
? ? CIFS Connectathon
? ? LTP: openposix test suite

Host 3: https://beaker.engineering.redhat.com/recipes/7797482
? Boot test
? xfstests: ext4
? xfstests: xfs
? lvm thinp sanity
? stress: stress-ng
? ? Storage blktests

ppc64le:
Host 1: https://beaker.engineering.redhat.com/recipes/7797422
? Boot test
? xfstests: ext4
? xfstests: xfs
? lvm thinp sanity
? ? Storage blktests

Host 2: https://beaker.engineering.redhat.com/recipes/7797421
? Boot test
? LTP
? Loopdev Sanity
? Memory function: memfd_create
? AMTU (Abstract Machine Test Utility)
? Ethernet drivers sanity
? ? CIFS Connectathon
? ? LTP: openposix test suite

x86_64:
Host 1: https://beaker.engineering.redhat.com/recipes/7797418
? Boot test
? xfstests: ext4
? xfstests: xfs
? lvm thinp sanity
? stress: stress-ng
? ? Storage blktests

Host 2: https://beaker.engineering.redhat.com/recipes/7797417
? Boot test
? LTP
? Loopdev Sanity
? Memory function: memfd_create
? AMTU (Abstract Machine Test Utility)
? Ethernet drivers sanity
? ? CIFS Connectathon
? ? LTP: openposix test suite

Test sources: https://github.com/CKI-project/tests-beaker
? Pull requests are welcome for new tests or improvements to existing tests!

Waived tests
------------
If the test run included waived tests, they are marked with ?. Such tests are executed but their results are not taken into account. Tests are waived when their results are not reliable enough, e.g. when they're just introduced or are being fixed.

Testing timeout
---------------
We aim to provide a report within reasonable timeframe. Tests that haven't finished running are marked with ?. Reports for non-upstream kernels have a Beaker recipe linked to next to each host.

Reproducing results
-------------------
Click on a link below to access a web page that allows you to adjust the Beaker job and re-run any failed tests. These links are generated for failed or aborted tests that are not waived. Please adjust the Beaker job whiteboard string in the web page so that it is easy for you to find and so that it is not confused with regular CKI job.

After clicking the "Submit the job!" button, a dialog with an url will open. Opening that url will show you the progress of submitting your job. At the end it should say "Submitted job J:XXXXX", meaning the job was really sent to Beaker.

http://respin-service-ark.cloud.paas.psi.redhat.com/respin.html?whiteboard=respin_job_4015280&recipe_id=7797421&job_id=4015280

#2 lore raw

[LTP] ??? FAIL: Test report for kernel 5.4.12-100.fc30 (fedora-30)

Quoted 5 lines · Cyril Hrubis

Hi!

Quoted 1 line
     Pipeline: https://xci32.lab.eng.rdu2.redhat.com/cki-project/brew-pipeline/pipelines/384229

Looks like unreachable URL again.

Oops sorry about that, since the Fedora maintainers had access to internal logs we used the regular template, only after did we start cc'ing LTP for upstream failures. I just changed it to the correct email template to include links to our external logs.

The failure was a dmesg check probably related to running on a slow kvm guest:

[ 5646.612562] watchdog: BUG: soft lockup - CPU#15 stuck for 31s! [systemd-journal:559]
[ 5840.828707] watchdog: BUG: soft lockup - CPU#10 stuck for 23s! [systemd:1]
[ 6018.881344] watchdog: BUG: soft lockup - CPU#2 stuck for 50s! [systemd-coredum:819905]
[ 6047.933641] watchdog: BUG: soft lockup - CPU#15 stuck for 21s! [systemd:1]
[ 6100.825887] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [systemd-coredum:819905]
[ 6369.103978] watchdog: BUG: soft lockup - CPU#5 stuck for 23s! [diotest6:819897]
[ 6780.415986] watchdog: BUG: soft lockup - CPU#10 stuck for 22s! [khvcd:223]
[ 6831.956434] watchdog: BUG: soft lockup - CPU#3 stuck for 70s! [migration/3:25]
[ 6831.956436] watchdog: BUG: soft lockup - CPU#2 stuck for 52s! [kworker/2:2:1997]
[ 6864.394439] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [rcu_sched:10]
[ 7004.381847] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [rcu_sched:10]
[ 7084.688008] watchdog: BUG: soft lockup - CPU#4 stuck for 67s! [kworker/4:3:818401]
[ 7108.389938] watchdog: BUG: soft lockup - CPU#4 stuck for 22s! [rcu_sched:10]
[ 7303.256283] watchdog: BUG: soft lockup - CPU#7 stuck for 110s! [diotest6:819970]

-Rachel