Skip to contentSkip to conversation

test: move debug_autotest out of fast suite

PATCH In review [email protected] 3 participants 5 messages Feb 25, 2021 – Mar 2, 2021 Arm/Amlogic Meson SoC support

lore
Share & tools
lore

[email protected]

author
#0 lore raw

PATCH 1/1 test: move debug_autotest out of fast suite

From: Luca Boccassi <[email protected]>

It consistently fails when ran on a build machine within the reproducible build environment, so move it out of there, so that we can run the fast suite at build time when packaging.

Repro test environment:

https://salsa.debian.org/salsa-ci-team/pipeline/raw/master/salsa-ci.yml
https://salsa.debian.org/salsa-ci-team/pipeline/

Cc [email protected]
Signed-off-by Luca Boccassi <[email protected]>
---
 app/test/meson.build | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
app/test/meson.build +1 -1
index 561e493a29..6230d33944 100644
@@ -190,7 +190,6 @@ fast_tests = [
         ['common_autotest', true],
         ['cpuflags_autotest', true],
         ['cycles_autotest', true],
-        ['debug_autotest', true],
         ['eal_flags_c_opt_autotest', false],
         ['eal_flags_main_opt_autotest', false],
         ['eal_flags_n_opt_autotest', false],

@@ -305,6 +304,7 @@ perf_test_names = [
         'hash_readwrite_lf_perf_autotest',
         'trace_perf_autotest',
 	'ipsec_perf_autotest',
+        'debug_autotest',
 ]
 
 driver_test_names = [
Signature · 1 line
2.29.2
#1 lore raw
Quoted 6 lines · On Thu, Feb 25, 2021 at 8:49 PM <[email protected]> wrote:

From: Luca Boccassi <[email protected]>

It consistently fails when ran on a build machine within the reproducible build environment, so move it out of there, so that we can run the fast suite at build time when packaging.

This test is quite simple.
Surprising that it consistently fails.

Quoted 5 lines

Repro test environment:

https://salsa.debian.org/salsa-ci-team/pipeline/raw/master/salsa-ci.yml
https://salsa.debian.org/salsa-ci-team/pipeline/

I am not familiar with this CI.
Where can we find the logs of such a failure?

Signature · 1 line
David Marchand
#2 lore raw
Quoted 9 lines · David Marchand
Quoted 5 lines · On Thu, Feb 25, 2021 at 8:49 PM <[email protected]> wrote:

From: Luca Boccassi <[email protected]>

It consistently fails when ran on a build machine within the reproducible build environment, so move it out of there, so that we can run the fast suite at build time when packaging.

This test is quite simple.
Surprising that it consistently fails.

It is very weird. I think it times out after the fork().

Quoted 7 lines
Quoted 4 lines

Repro test environment:

https://salsa.debian.org/salsa-ci-team/pipeline/raw/master/salsa-ci.yml
https://salsa.debian.org/salsa-ci-team/pipeline/

I am not familiar with this CI.
Where can we find the logs of such a failure?

It does various scramblings to ensure the produced binaries are bit-by- bit identical despite changes like number of CPUs, time of the day, timezone, filesystem, locale, etc.

Not much useful info in the logs unfortunately, despite the verbose run.

https://salsa.debian.org/bluca/dpdk/-/jobs/1453534

RTE>>EAL: Detected 2 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: Multi-process socket /var/run/dpdk/debug_autotest/mp_socket
EAL: Selected IOVA mode 'VA'
EAL: Probing VFIO support...
APP: HPET is not enabled, using TSC as default timer
RTE>>debug_autotest
10: [/tmp/reprotest.YBma9j/build-experiment-1/build-experiment-1/obj-x86_64-linux-gnu/app/test/dpdk-test(+0x61c6a) [0x5562b61f7c6a]]
9: [/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) [0x7f9793188d0a]]
8: [/tmp/reprotest.YBma9j/build-experiment-1/build-experiment-1/obj-x86_64-linux-gnu/app/test/dpdk-test(+0x386ae) [0x5562b61ce6ae]]
7: [/tmp/reprotest.YBma9j/build-experiment-1/build-experiment-1/obj-x86_64-linux-gnu/app/test/../../lib/librte_cmdline.so.21(cmdline_in+0x71) [0x7f9793925841]] 6: [/tmp/reprotest.YBma9j/build-experiment-1/build-experiment-1/obj-x86_64-linux-gnu/app/test/../../lib/librte_cmdline.so.21(rdline_char_in+0x36b) [0x7f9793928abb]] 5: [/tmp/reprotest.YBma9j/build-experiment-1/build-experiment-1/obj-x86_64-linux-gnu/app/test/../../lib/librte_cmdline.so.21(+0x3770) [0x7f9793925770]] 4: [/tmp/reprotest.YBma9j/build-experiment-1/build-experiment-1/obj-x86_64-linux-gnu/app/test/../../lib/librte_cmdline.so.21(cmdline_parse+0x1af) [0x7f979392664f]] 3: [/tmp/reprotest.YBma9j/build-experiment-1/build-experiment-1/obj-x86_64-linux-gnu/app/test/dpdk-test(+0x61d5b) [0x5562b61f7d5b]]
2: [/tmp/reprotest.YBma9j/build-experiment-1/build-experiment-1/obj-x86_64-linux-gnu/app/test/dpdk-test(+0xc2c13) [0x5562b6258c13]]
1: [/tmp/reprotest.YBma9j/build-experiment-1/build-experiment-1/obj-x86_64-linux-gnu/app/test/../../lib/librte_eal.so.21(rte_dump_stack+0x2e) [0x7f9793a931be]] PANIC in test_panic():
Test Debug
11: [/tmp/reprotest.YBma9j/build-experiment-1/build-experiment-1/obj-x86_64-linux-gnu/app/test/dpdk-test(+0x61c6a) [0x5562b61f7c6a]]
10: [/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) [0x7f9793188d0a]]
9: [/tmp/reprotest.YBma9j/build-experiment-1/build-experiment-1/obj-x86_64-linux-gnu/app/test/dpdk-test(+0x386ae) [0x5562b61ce6ae]]
8: [/tmp/reprotest.YBma9j/build-experiment-1/build-experiment-1/obj-x86_64-linux-gnu/app/test/../../lib/librte_cmdline.so.21(cmdline_in+0x71) [0x7f9793925841]] 7: [/tmp/reprotest.YBma9j/build-experiment-1/build-experiment-1/obj-x86_64-linux-gnu/app/test/../../lib/librte_cmdline.so.21(rdline_char_in+0x36b) [0x7f9793928abb]] 6: [/tmp/reprotest.YBma9j/build-experiment-1/build-experiment-1/obj-x86_64-linux-gnu/app/test/../../lib/librte_cmdline.so.21(+0x3770) [0x7f9793925770]] 5: [/tmp/reprotest.YBma9j/build-experiment-1/build-experiment-1/obj-x86_64-linux-gnu/app/test/../../lib/librte_cmdline.so.21(cmdline_parse+0x1af) [0x7f979392664f]] 4: [/tmp/reprotest.YBma9j/build-experiment-1/build-experiment-1/obj-x86_64-linux-gnu/app/test/dpdk-test(+0x61d5b) [0x5562b61f7d5b]]
3: [/tmp/reprotest.YBma9j/build-experiment-1/build-experiment-1/obj-x86_64-linux-gnu/app/test/dpdk-test(+0x3d671) [0x5562b61d3671]]
2: [/tmp/reprotest.YBma9j/build-experiment-1/build-experiment-1/obj-x86_64-linux-gnu/app/test/../../lib/librte_eal.so.21(__rte_panic+0xc1) [0x7f9793a7187f]] 1: [/tmp/reprotest.YBma9j/build-experiment-1/build-experiment-1/obj-x86_64-linux-gnu/app/test/../../lib/librte_eal.so.21(rte_dump_stack+0x2e) [0x7f9793a931be]] <...>
debug_autotest time out (After 30.0 seconds)

Signature · 2 lines
Kind regards,
Luca Boccassi
#3 lore raw
Quoted 24 lines · Luca Boccassi

RTE>>EAL: Detected 2 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: Multi-process socket /var/run/dpdk/debug_autotest/mp_socket
EAL: Selected IOVA mode 'VA'
EAL: Probing VFIO support...
APP: HPET is not enabled, using TSC as default timer
RTE>>debug_autotest
10: [/tmp/reprotest.YBma9j/build-experiment-1/build-experiment-1/obj-x86_64-linux-gnu/app/test/dpdk-test(+0x61c6a) [0x5562b61f7c6a]]
9: [/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) [0x7f9793188d0a]]
8: [/tmp/reprotest.YBma9j/build-experiment-1/build-experiment-1/obj-x86_64-linux-gnu/app/test/dpdk-test(+0x386ae) [0x5562b61ce6ae]]
7: [/tmp/reprotest.YBma9j/build-experiment-1/build-experiment-1/obj-x86_64-linux-gnu/app/test/../../lib/librte_cmdline.so.21(cmdline_in+0x71) [0x7f9793925841]] 6: [/tmp/reprotest.YBma9j/build-experiment-1/build-experiment-1/obj-x86_64-linux-gnu/app/test/../../lib/librte_cmdline.so.21(rdline_char_in+0x36b) [0x7f9793928abb]] 5: [/tmp/reprotest.YBma9j/build-experiment-1/build-experiment-1/obj-x86_64-linux-gnu/app/test/../../lib/librte_cmdline.so.21(+0x3770) [0x7f9793925770]] 4: [/tmp/reprotest.YBma9j/build-experiment-1/build-experiment-1/obj-x86_64-linux-gnu/app/test/../../lib/librte_cmdline.so.21(cmdline_parse+0x1af) [0x7f979392664f]] 3: [/tmp/reprotest.YBma9j/build-experiment-1/build-experiment-1/obj-x86_64-linux-gnu/app/test/dpdk-test(+0x61d5b) [0x5562b61f7d5b]]
2: [/tmp/reprotest.YBma9j/build-experiment-1/build-experiment-1/obj-x86_64-linux-gnu/app/test/dpdk-test(+0xc2c13) [0x5562b6258c13]]
1: [/tmp/reprotest.YBma9j/build-experiment-1/build-experiment-1/obj-x86_64-linux-gnu/app/test/../../lib/librte_eal.so.21(rte_dump_stack+0x2e) [0x7f9793a931be]] PANIC in test_panic():
Test Debug
11: [/tmp/reprotest.YBma9j/build-experiment-1/build-experiment-1/obj-x86_64-linux-gnu/app/test/dpdk-test(+0x61c6a) [0x5562b61f7c6a]]
10: [/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) [0x7f9793188d0a]]
9: [/tmp/reprotest.YBma9j/build-experiment-1/build-experiment-1/obj-x86_64-linux-gnu/app/test/dpdk-test(+0x386ae) [0x5562b61ce6ae]]
8: [/tmp/reprotest.YBma9j/build-experiment-1/build-experiment-1/obj-x86_64-linux-gnu/app/test/../../lib/librte_cmdline.so.21(cmdline_in+0x71) [0x7f9793925841]] 7: [/tmp/reprotest.YBma9j/build-experiment-1/build-experiment-1/obj-x86_64-linux-gnu/app/test/../../lib/librte_cmdline.so.21(rdline_char_in+0x36b) [0x7f9793928abb]]

Quoted 8 lines

6: [/tmp/reprotest.YBma9j/build-experiment-1/build-experiment-1/obj-x86_64-linux-gnu/app/test/../../lib/librte_cmdline.so.21(+0x3770) [0x7f9793925770]] 5: [/tmp/reprotest.YBma9j/build-experiment-1/build-experiment-1/obj-x86_64-linux-gnu/app/test/../../lib/librte_cmdline.so.21(cmdline_parse+0x1af) [0x7f979392664f]] 4: [/tmp/reprotest.YBma9j/build-experiment-1/build-experiment-1/obj-x86_64-linux-gnu/app/test/dpdk-test(+0x61d5b) [0x5562b61f7d5b]]
3: [/tmp/reprotest.YBma9j/build-experiment-1/build-experiment-1/obj-x86_64-linux-gnu/app/test/dpdk-test(+0x3d671) [0x5562b61d3671]]
2: [/tmp/reprotest.YBma9j/build-experiment-1/build-experiment-1/obj-x86_64-linux-gnu/app/test/../../lib/librte_eal.so.21(__rte_panic+0xc1) [0x7f9793a7187f]] 1: [/tmp/reprotest.YBma9j/build-experiment-1/build-experiment-1/obj-x86_64-linux-gnu/app/test/../../lib/librte_eal.so.21(rte_dump_stack+0x2e) [0x7f9793a931be]] <...>
debug_autotest time out (After 30.0 seconds)

Ah a timeout.
Could it be that coredumps (of the children processes here) are being captured?

Can you test with this patch:
https://patchwork.dpdk.org/project/dpdk/patch/[email protected]/
?
It disables coredump generation in the autotests when failure is expected.

Signature · 1 line
David Marchand
#4 lore raw
Quoted 34 lines · David Marchand
Quoted 24 lines · Luca Boccassi

RTE>>EAL: Detected 2 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: Multi-process socket /var/run/dpdk/debug_autotest/mp_socket
EAL: Selected IOVA mode 'VA'
EAL: Probing VFIO support...
APP: HPET is not enabled, using TSC as default timer
RTE>>debug_autotest
10: [/tmp/reprotest.YBma9j/build-experiment-1/build-experiment-1/obj-x86_64-linux-gnu/app/test/dpdk-test(+0x61c6a) [0x5562b61f7c6a]]
9: [/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) [0x7f9793188d0a]]
8: [/tmp/reprotest.YBma9j/build-experiment-1/build-experiment-1/obj-x86_64-linux-gnu/app/test/dpdk-test(+0x386ae) [0x5562b61ce6ae]]
7: [/tmp/reprotest.YBma9j/build-experiment-1/build-experiment-1/obj-x86_64-linux-gnu/app/test/../../lib/librte_cmdline.so.21(cmdline_in+0x71) [0x7f9793925841]] 6: [/tmp/reprotest.YBma9j/build-experiment-1/build-experiment-1/obj-x86_64-linux-gnu/app/test/../../lib/librte_cmdline.so.21(rdline_char_in+0x36b) [0x7f9793928abb]] 5: [/tmp/reprotest.YBma9j/build-experiment-1/build-experiment-1/obj-x86_64-linux-gnu/app/test/../../lib/librte_cmdline.so.21(+0x3770) [0x7f9793925770]] 4: [/tmp/reprotest.YBma9j/build-experiment-1/build-experiment-1/obj-x86_64-linux-gnu/app/test/../../lib/librte_cmdline.so.21(cmdline_parse+0x1af) [0x7f979392664f]] 3: [/tmp/reprotest.YBma9j/build-experiment-1/build-experiment-1/obj-x86_64-linux-gnu/app/test/dpdk-test(+0x61d5b) [0x5562b61f7d5b]]
2: [/tmp/reprotest.YBma9j/build-experiment-1/build-experiment-1/obj-x86_64-linux-gnu/app/test/dpdk-test(+0xc2c13) [0x5562b6258c13]]
1: [/tmp/reprotest.YBma9j/build-experiment-1/build-experiment-1/obj-x86_64-linux-gnu/app/test/../../lib/librte_eal.so.21(rte_dump_stack+0x2e) [0x7f9793a931be]] PANIC in test_panic():
Test Debug
11: [/tmp/reprotest.YBma9j/build-experiment-1/build-experiment-1/obj-x86_64-linux-gnu/app/test/dpdk-test(+0x61c6a) [0x5562b61f7c6a]]
10: [/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) [0x7f9793188d0a]]
9: [/tmp/reprotest.YBma9j/build-experiment-1/build-experiment-1/obj-x86_64-linux-gnu/app/test/dpdk-test(+0x386ae) [0x5562b61ce6ae]]
8: [/tmp/reprotest.YBma9j/build-experiment-1/build-experiment-1/obj-x86_64-linux-gnu/app/test/../../lib/librte_cmdline.so.21(cmdline_in+0x71) [0x7f9793925841]] 7: [/tmp/reprotest.YBma9j/build-experiment-1/build-experiment-1/obj-x86_64-linux-gnu/app/test/../../lib/librte_cmdline.so.21(rdline_char_in+0x36b) [0x7f9793928abb]]

Quoted 8 lines

6: [/tmp/reprotest.YBma9j/build-experiment-1/build-experiment-1/obj-x86_64-linux-gnu/app/test/../../lib/librte_cmdline.so.21(+0x3770) [0x7f9793925770]] 5: [/tmp/reprotest.YBma9j/build-experiment-1/build-experiment-1/obj-x86_64-linux-gnu/app/test/../../lib/librte_cmdline.so.21(cmdline_parse+0x1af) [0x7f979392664f]] 4: [/tmp/reprotest.YBma9j/build-experiment-1/build-experiment-1/obj-x86_64-linux-gnu/app/test/dpdk-test(+0x61d5b) [0x5562b61f7d5b]]
3: [/tmp/reprotest.YBma9j/build-experiment-1/build-experiment-1/obj-x86_64-linux-gnu/app/test/dpdk-test(+0x3d671) [0x5562b61d3671]]
2: [/tmp/reprotest.YBma9j/build-experiment-1/build-experiment-1/obj-x86_64-linux-gnu/app/test/../../lib/librte_eal.so.21(__rte_panic+0xc1) [0x7f9793a7187f]] 1: [/tmp/reprotest.YBma9j/build-experiment-1/build-experiment-1/obj-x86_64-linux-gnu/app/test/../../lib/librte_eal.so.21(rte_dump_stack+0x2e) [0x7f9793a931be]] <...>
debug_autotest time out (After 30.0 seconds)

Quoted 7 lines

Ah a timeout.
Could it be that coredumps (of the children processes here) are being captured?

Can you test with this patch:
https://patchwork.dpdk.org/project/dpdk/patch/[email protected]/
?
It disables coredump generation in the autotests when failure is expected.

That works, thank you!

Please mark it for backporting, so I can include it in 20.11

Signature · 2 lines
Kind regards,
Luca Boccassi