unit_tests_mempool/test_mempool_perf: timeout on mempool_perf_autotest

Bug 937 [email protected] 3 participants 6 messages Feb 17, 2022 – Feb 18, 2022

Share & tools

[email protected]

author

2022-02-17 09:10 UTC #0 lore raw

https://bugs.dpdk.org/show_bug.cgi?id=937

            Bug ID: 937
           Summary: [dpdk-22.03] unit_tests_mempool/test_mempool_perf:
                    timeout on mempool_perf_autotest
           Product: DPDK
           Version: unspecified
          Hardware: All
                OS: All
            Status: UNCONFIRMED
          Severity: normal
          Priority: Normal
         Component: other
          Assignee: [email protected]
          Reporter: [email protected]

Target Milestone: ---

dpdk-22.03-rc1: 2e07139b66a810883871ff20a5f31e4c222e5b40

Reproduce Step:

1.launch dpdk-test x86_64-native-linuxapp-gcc/app/test/dpdk-test -l 1-111 -n 4
2. RTE>>mempool_perf_autotest

Expect results: mempool_perf_autotest timeout in 2000s.

Actual results: mempool_perf_autotest need run 1 or more hours.

Is this issue a regression: Y
Version the regression was introduced: Specify git id if known.
First bad commit:ed579e50c66db90485593c6c79f5366a57145f2a
Author: Morten Brørup <[email protected]>
Date: Mon Jan 24 15:59:53 2022 +0100

mempool: test performance with constant n

"What gets measured gets done."

This patch adds mempool performance tests where the number of objects to put and get is constant at compile time, which may significantly improve the performance of these functions. [*]

Also, it is ensured that the array holding the object used for testing is cache line aligned, for maximum performance.

And finally, the following entries are added to the list of tests:
- Number of kept objects: 512
- Number of objects to get and to put: The number of pointers fitting into a cache line, i.e. 8 or 16

[*] Some example performance test (with cache) results:

get_bulk=4 put_bulk=4 keep=128 constant_n=false rate_persec=280480972 get_bulk=4 put_bulk=4 keep=128 constant_n=true rate_persec=622159462

get_bulk=8 put_bulk=8 keep=128 constant_n=false rate_persec=477967155 get_bulk=8 put_bulk=8 keep=128 constant_n=true rate_persec=917582643

get_bulk=32 put_bulk=32 keep=32 constant_n=false rate_persec=871248691 get_bulk=32 put_bulk=32 keep=32 constant_n=true rate_persec=1134021836

Signed-off-by Morten Brørup <[email protected]>

Signed-off-by Olivier Matz <[email protected]>

Signature · 2 lines

You are receiving this mail because:
You are the assignee for the bug.

Morten Brørup

2022-02-17 10:16 UTC #1 lore raw

Quoted 5 lines

From: [email protected] [mailto:[email protected]]
Sent: Thursday, 17 February 2022 10.11
https://bugs.dpdk.org/show_bug.cgi?id=937

[...]

Quoted 29 lines

dpdk-22.03-rc1: 2e07139b66a810883871ff20a5f31e4c222e5b40
Reproduce Step:
1.launch dpdk-test x86_64-native-linuxapp-gcc/app/test/dpdk-test -l 1-111 -n 4
2. RTE>>mempool_perf_autotest
Expect results: mempool_perf_autotest timeout in 2000s.
Actual results: mempool_perf_autotest need run 1 or more hours.
Is this issue a regression: Y
Version the regression was introduced: Specify git id if known.
First bad commit:ed579e50c66db90485593c6c79f5366a57145f2a
Author: Morten Brørup <[email protected]>
Date: Mon Jan 24 15:59:53 2022 +0100
mempool: test performance with constant n
"What gets measured gets done."
This patch adds mempool performance tests where the number of objects to
put and get is constant at compile time, which may significantly improve
the performance of these functions. [*]

Quoted 8 lines

Also, it is ensured that the array holding the object used for testing is cache line aligned, for maximum performance.
And finally, the following entries are added to the list of tests:
- Number of kept objects: 512
- Number of objects to get and to put: The number of pointers fitting into a cache line, i.e. 8 or 16

A few more test cases were added, and each test is now run in two variants, one variant with variable N, and one variant with constant N.

To reduce the very long test running time, we could consider removing some of the tests. However, a few undocumented magic numbers are used throughout DPDK, e.g. a burst size of 32. These magic numbers do not have names, but are just used numerically. So I am not sure if we can safely remove the tests with burst size 4 and/or 128 kept objects. E.g. I suspect that 4 could be a magic number in some NIC drivers, and thus still relevant to test.

Quoted 17 lines

[*] Some example performance test (with cache) results:
get_bulk=4 put_bulk=4 keep=128 constant_n=false rate_persec=280480972 get_bulk=4 put_bulk=4 keep=128 constant_n=true rate_persec=622159462
get_bulk=8 put_bulk=8 keep=128 constant_n=false rate_persec=477967155 get_bulk=8 put_bulk=8 keep=128 constant_n=true rate_persec=917582643
get_bulk=32 put_bulk=32 keep=32 constant_n=false rate_persec=871248691 get_bulk=32 put_bulk=32 keep=32 constant_n=true rate_persec=1134021836
Signed-off-by Morten Brørup <[email protected]>
Signed-off-by Olivier Matz <[email protected]>
--
You are receiving this mail because:
You are the assignee for the bug.

Jiang, YuX

2022-02-17 10:30 UTC #2 lore raw

Quoted 56 lines

-----Original Message-----
From: Morten Brørup <[email protected]>
Sent: Thursday, February 17, 2022 6:16 PM
To: [email protected]; [email protected]
Subject: RE: [Bug 937] [dpdk-22.03]
unit_tests_mempool/test_mempool_perf: timeout on
mempool_perf_autotest
Quoted 5 lines
From: [email protected] [mailto:[email protected]]
Sent: Thursday, 17 February 2022 10.11
https://bugs.dpdk.org/show_bug.cgi?id=937
[...]
Quoted 27 lines
dpdk-22.03-rc1: 2e07139b66a810883871ff20a5f31e4c222e5b40
Reproduce Step:
1.launch dpdk-test x86_64-native-linuxapp-gcc/app/test/dpdk-test -l 1-111 -n 4 2. RTE>>mempool_perf_autotest
Expect results: mempool_perf_autotest timeout in 2000s.
Actual results: mempool_perf_autotest need run 1 or more hours.
Is this issue a regression: Y
Version the regression was introduced: Specify git id if known.
First bad commit:ed579e50c66db90485593c6c79f5366a57145f2a
Author: Morten Brørup <[email protected]>
Date: Mon Jan 24 15:59:53 2022 +0100
mempool: test performance with constant n
"What gets measured gets done."
This patch adds mempool performance tests where the number of objects to put and get is constant at compile time, which may significantly improve the performance of these functions. [*]
Quoted 8 lines
Also, it is ensured that the array holding the object used for testing is cache line aligned, for maximum performance.
And finally, the following entries are added to the list of tests:
- Number of kept objects: 512
- Number of objects to get and to put: The number of pointers fitting into a cache line, i.e. 8 or 16
A few more test cases were added, and each test is now run in two variants, one variant with variable N, and one variant with constant N.

Quoted 8 lines

To reduce the very long test running time, we could consider removing some of the tests. However, a few undocumented magic numbers are used throughout DPDK, e.g. a burst size of 32. These magic numbers do not have names, but are just used numerically. So I am not sure if we can safely remove the tests with burst size 4 and/or 128 kept objects. E.g. I suspect that 4 could be a magic number in some NIC drivers, and thus still relevant to test.

Do you mean that this is normal behavior due to this merged patch? How long this mempool_perf_autotest will take? 1 more hours or several hours?

Quoted 21 lines

Quoted 3 lines
[*] Some example performance test (with cache) results:
get_bulk=4 put_bulk=4 keep=128 constant_n=false
rate_persec=280480972
Quoted 3 lines
get_bulk=4 put_bulk=4 keep=128 constant_n=true rate_persec=622159462
get_bulk=8 put_bulk=8 keep=128 constant_n=false
rate_persec=477967155
Quoted 3 lines
get_bulk=8 put_bulk=8 keep=128 constant_n=true rate_persec=917582643
get_bulk=32 put_bulk=32 keep=32 constant_n=false
rate_persec=871248691
Quoted 1 line
get_bulk=32 put_bulk=32 keep=32 constant_n=true
rate_persec=1134021836
Quoted 7 lines
Signed-off-by Morten Brørup <[email protected]>
Signed-off-by Olivier Matz <[email protected]>
--
You are receiving this mail because:
You are the assignee for the bug.

Morten Brørup

2022-02-18 08:10 UTC #3 lore raw

Quoted 66 lines

From: Jiang, YuX [mailto:[email protected]]
Sent: Thursday, 17 February 2022 11.31
Quoted 36 lines
From: Morten Brørup <[email protected]>
Sent: Thursday, February 17, 2022 6:16 PM
Quoted 5 lines
From: [email protected] [mailto:[email protected]]
Sent: Thursday, 17 February 2022 10.11
https://bugs.dpdk.org/show_bug.cgi?id=937
[...]
Quoted 24 lines
dpdk-22.03-rc1: 2e07139b66a810883871ff20a5f31e4c222e5b40
Reproduce Step:
1.launch dpdk-test x86_64-native-linuxapp-gcc/app/test/dpdk-test -l 1-111 -n 4 2. RTE>>mempool_perf_autotest
Expect results: mempool_perf_autotest timeout in 2000s.
Actual results: mempool_perf_autotest need run 1 or more hours.
Is this issue a regression: Y
Version the regression was introduced: Specify git id if known.
First bad commit:ed579e50c66db90485593c6c79f5366a57145f2a
Author: Morten Brørup <[email protected]>
Date: Mon Jan 24 15:59:53 2022 +0100
mempool: test performance with constant n
"What gets measured gets done."
This patch adds mempool performance tests where the number of
objects
Quoted 4 lines
Quoted 4 lines
to put and get is constant at compile time, which may significantly improve the performance of these functions. [*]
Also, it is ensured that the array holding the object used for
testing
Quoted 5 lines
Quoted 5 lines
is cache line aligned, for maximum performance.
And finally, the following entries are added to the list of tests:
- Number of kept objects: 512
- Number of objects to get and to put: The number of pointers
fitting
Quoted 4 lines
Quoted 2 lines
into a cache line, i.e. 8 or 16
A few more test cases were added, and each test is now run in two
variants,
Quoted 3 lines
one variant with variable N, and one variant with constant N.
To reduce the very long test running time, we could consider removing
some
Quoted 2 lines
of the tests. However, a few undocumented magic numbers are used throughout DPDK, e.g. a burst size of 32. These magic numbers do not
have
Quoted 1 line
names, but are just used numerically. So I am not sure if we can
safely
Quoted 1 line
remove the tests with burst size 4 and/or 128 kept objects. E.g. I

Quoted 8 lines

suspect that
Quoted 1 line
4 could be a magic number in some NIC drivers, and thus still
relevant to test.
Quoted 2 lines
Do you mean that this is normal behavior due to this merged patch? How long this mempool_perf_autotest will take? 1 more hours or several hours?

It was running for 2-3 hours on our server.

-Morten

Jiang, YuX

2022-02-18 09:51 UTC #4 lore raw

Quoted 80 lines

-----Original Message-----
From: Morten Brørup <[email protected]>
Sent: Friday, February 18, 2022 4:10 PM
To: Jiang, YuX <[email protected]>; [email protected]; [email protected]
Subject: RE: [Bug 937] [dpdk-22.03] unit_tests_mempool/test_mempool_perf: timeout on mempool_perf_autotest
Quoted 72 lines
From: Jiang, YuX [mailto:[email protected]]
Sent: Thursday, 17 February 2022 11.31
Quoted 36 lines
From: Morten Brørup <[email protected]>
Sent: Thursday, February 17, 2022 6:16 PM
Quoted 5 lines
From: [email protected] [mailto:[email protected]]
Sent: Thursday, 17 February 2022 10.11
https://bugs.dpdk.org/show_bug.cgi?id=937
[...]
Quoted 24 lines
dpdk-22.03-rc1: 2e07139b66a810883871ff20a5f31e4c222e5b40
Reproduce Step:
1.launch dpdk-test x86_64-native-linuxapp-gcc/app/test/dpdk-test -l 1-111 -n 4 2. RTE>>mempool_perf_autotest
Expect results: mempool_perf_autotest timeout in 2000s.
Actual results: mempool_perf_autotest need run 1 or more hours.
Is this issue a regression: Y
Version the regression was introduced: Specify git id if known.
First bad commit:ed579e50c66db90485593c6c79f5366a57145f2a
Author: Morten Brørup <[email protected]>
Date: Mon Jan 24 15:59:53 2022 +0100
mempool: test performance with constant n
"What gets measured gets done."
This patch adds mempool performance tests where the number of
objects
Quoted 4 lines
Quoted 4 lines
to put and get is constant at compile time, which may significantly improve the performance of these functions. [*]
Also, it is ensured that the array holding the object used for
testing
Quoted 5 lines
Quoted 5 lines
is cache line aligned, for maximum performance.
And finally, the following entries are added to the list of tests:
- Number of kept objects: 512
- Number of objects to get and to put: The number of pointers
fitting
Quoted 4 lines
Quoted 2 lines
into a cache line, i.e. 8 or 16
A few more test cases were added, and each test is now run in two
variants,
Quoted 4 lines
one variant with variable N, and one variant with constant N.
To reduce the very long test running time, we could consider removing
some
Quoted 2 lines
of the tests. However, a few undocumented magic numbers are used throughout DPDK, e.g. a burst size of 32. These magic numbers do not
have
Quoted 1 line
names, but are just used numerically. So I am not sure if we can
safely
Quoted 1 line
remove the tests with burst size 4 and/or 128 kept objects. E.g. I
suspect that
Quoted 1 line
4 could be a magic number in some NIC drivers, and thus still
relevant to test.
Quoted 2 lines

Quoted 7 lines

Quoted 3 lines
Do you mean that this is normal behavior due to this merged patch? How long this mempool_perf_autotest will take? 1 more hours or several hours?
It was running for 2-3 hours on our server.
-Morten

Thanks Morten, so you mean this is normal behavior, right?

Morten Brørup

2022-02-18 09:53 UTC #5 lore raw

Quoted 82 lines

From: Jiang, YuX [mailto:[email protected]]
Sent: Friday, 18 February 2022 10.52
Quoted 51 lines
From: Morten Brørup <[email protected]>
Sent: Friday, February 18, 2022 4:10 PM
Quoted 48 lines
From: Jiang, YuX [mailto:[email protected]]
Sent: Thursday, 17 February 2022 11.31
Quoted 36 lines
From: Morten Brørup <[email protected]>
Sent: Thursday, February 17, 2022 6:16 PM
Quoted 5 lines
From: [email protected] [mailto:[email protected]]
Sent: Thursday, 17 February 2022 10.11
https://bugs.dpdk.org/show_bug.cgi?id=937
[...]
Quoted 24 lines
dpdk-22.03-rc1: 2e07139b66a810883871ff20a5f31e4c222e5b40
Reproduce Step:
1.launch dpdk-test x86_64-native-linuxapp-gcc/app/test/dpdk-test -l 1-111 -n 4 2. RTE>>mempool_perf_autotest
Expect results: mempool_perf_autotest timeout in 2000s.
Actual results: mempool_perf_autotest need run 1 or more hours.
Is this issue a regression: Y
Version the regression was introduced: Specify git id if known.
First bad commit:ed579e50c66db90485593c6c79f5366a57145f2a
Author: Morten Brørup <[email protected]>
Date: Mon Jan 24 15:59:53 2022 +0100
mempool: test performance with constant n
"What gets measured gets done."
This patch adds mempool performance tests where the number of
objects
Quoted 4 lines
Quoted 4 lines
to put and get is constant at compile time, which may significantly improve the performance of these functions. [*]
Also, it is ensured that the array holding the object used for
testing
Quoted 3 lines
Quoted 3 lines
is cache line aligned, for maximum performance.
And finally, the following entries are added to the list of
tests:
Quoted 7 lines
Quoted 7 lines
Quoted 2 lines
Quoted 2 lines
- Number of kept objects: 512
- Number of objects to get and to put: The number of pointers
fitting
Quoted 4 lines
Quoted 2 lines
into a cache line, i.e. 8 or 16
A few more test cases were added, and each test is now run in two
Quoted 8 lines
Quoted 8 lines
variants,
Quoted 4 lines
one variant with variable N, and one variant with constant N.
To reduce the very long test running time, we could consider removing
some
Quoted 2 lines
of the tests. However, a few undocumented magic numbers are used throughout DPDK, e.g. a burst size of 32. These magic numbers do
not
Quoted 4 lines
Quoted 4 lines
have
Quoted 1 line
names, but are just used numerically. So I am not sure if we can
safely
Quoted 1 line
remove the tests with burst size 4 and/or 128 kept objects. E.g.
I
Quoted 6 lines
Quoted 6 lines
suspect that
Quoted 1 line
4 could be a magic number in some NIC drivers, and thus still
relevant to test.
Quoted 2 lines
Do you mean that this is normal behavior due to this merged patch?

Quoted 8 lines

How
Quoted 6 lines
Quoted 2 lines
long this mempool_perf_autotest will take? 1 more hours or several hours?
It was running for 2-3 hours on our server.
-Morten
Thanks Morten, so you mean this is normal behavior, right?

Yes, unfortunately.