app/test: make perf tests usable on wider range of systems
Get series with b4
b4 am [email protected]Versions newest first · linked by author and title
- v2 Latest5 patches 1 replyIn review
- v17 patches 6 repliesSuperseded
Changes in v2 from the cover letter
- Patch 5 no longer drops the constant-values replay. Morten pointed out that mempools are commonly used with compile time constant request sizes, so those paths are real regression signal, and NAKed the removal. The replay is kept but restricted to a subset of the bulk sizes rather than the whole get/put diagonal.
- The bulk size tables keep 64 and RTE_MEMPOOL_CACHE_MAX_SIZE / 2. The latter is the bounce buffer limit in Morten's mempool optimization work, above which requests bypass the cache, so it marks a real change in behaviour.
- The mempool patches are squashed into one per test file. The result contains considerably more than was reviewed in v1, so the acks given on the v1 mempool patches have been dropped rather than carried forward.
- Added the test_loop() bound and the reduced bulk table described above, plus per-point elapsed time and a flush before measuring, so a run killed by a timeout shows where it got to. That is how the 50 second points were found.
Patches in v2 No applies recorded yet
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/5 | app/test/reciprocal_division: make it a fast testapp/ | +157 | 0 replies | |
| 2/5 | app/test/reciprocal_division_perf: reduce test timeapp/ | +2 | 0 replies | |
| 3/5 | app/test/test_rcu_qsbr_perf: call quiescent more oftenapp/ | +1 | 0 replies | |
| 4/5 | app/test/test_pmd_perf: skip if no device availableapp/ | +1 | 0 replies | |
| 5/5 | app/test/mempool_perf: adjust test to work with many coresapp/ | +120 | 1 reply |
Reviews trailers per patch
1 of 5 patches has a Reviewed-by or Acked-by.
| Patch | Morten Brørup |
|---|---|
| 1/5 | |
| 2/5 | |
| 3/5 | |
| 4/5 | |
| 5/5 | A-b |
Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.
Cover letter [PATCH v2 0/5] app/test: make perf tests usable on wider range of systems
Several of the perf/autotests assume server-class machines: they run for billions of iterations, or size hugepage allocations by total lcore count, which makes them time out or fail to allocate on smaller systems and on machines with high core counts but modest memory.
This series trims runtime and memory use without losing meaningful test coverage, and reports resource shortfalls as skips rather than failures. The goal is a perf suite that can be run regularly to catch large regressions, so finishing reliably matters more than fine resolution.
Measured with "meson test --suite=perf-tests" on a 32 lcore system with 10 GB of hugepages, same machine and same session before and after:
main this series
Ok 41 44
Fail 1 0
Timeout 4 0
mempool_perf_autotest timeout 414s
mempool_perf_autotest_1core timeout 83s
mempool_perf_autotest_2cores timeout 84s
mempool_perf_autotest_allcores timeout 200s
rcu_qsbr_perf_autotest 495s 32s
reciprocal_division_perf 252s 1s
reciprocal_division 104s moved to fast-tests
pmd_perf_autotest FAIL skip
total suite 65.5 min 24.8 min
The 65.5 minutes understates it: four of those tests were killed at the 600 second timeout rather than finishing, so the real figure is unknown and larger.
Note that with enough memory for the largest mempool, all four mempool variants time out on main, including the single core one. Memory is not the only problem; the test also has no working bound on its own runtime.
Two distinct problems were behind that, and both needed fixing:
- test_loop() did a fixed number of objects per call, and the elapsed time is only checked between calls. A small bulk size needs many more mempool operations for the same objects, and each operation is much more expensive when many cores contend, so with bulk size 1 on 32 cores a single call took over 50 seconds and TIME_MS could not bound anything. The work per call is now limited by mempool operations.
- Each test point launches and joins every lcore. That costs about 2 ms on one core but over a second on 32, so the runtime is set by the number of points rather than by the time spent measuring. Above two cores a reduced set of get/put bulk sizes is used.
On a machine with less memory the mempool sizing matters too: sizing the pools by the cores actually exercised lets the one and two core variants run where they previously could not allocate at all, and a shortfall is now reported as a skip rather than a failure.
v2 changes:
- Patch 5 no longer drops the constant-values replay. Morten pointed out that mempools are commonly used with compile time constant request sizes, so those paths are real regression signal, and NAKed the removal. The replay is kept but restricted to a subset of the bulk sizes rather than the whole get/put diagonal.
- The bulk size tables keep 64 and RTE_MEMPOOL_CACHE_MAX_SIZE / 2. The latter is the bounce buffer limit in Morten's mempool optimization work, above which requests bypass the cache, so it marks a real change in behaviour.
- The mempool patches are squashed into one per test file. The result contains considerably more than was reviewed in v1, so the acks given on the v1 mempool patches have been dropped rather than carried forward.
- Added the test_loop() bound and the reduced bulk table described above, plus per-point elapsed time and a flush before measuring, so a run killed by a timeout shows where it got to. That is how the 50 second points were found.
Stephen Hemminger (5):
app/test/reciprocal_division: make it a fast test app/test/reciprocal_division_perf: reduce test time app/test/test_rcu_qsbr_perf: call quiescent more often app/test/test_pmd_perf: skip if no device available app/test/mempool_perf: adjust test for large core counts
app/test/test_mempool_perf.c | 179 ++++++++++----- app/test/test_pmd_perf.c | 2 +- app/test/test_rcu_qsbr_perf.c | 3 +- app/test/test_reciprocal_division.c | 279 +++++++++++++---------- app/test/test_reciprocal_division_perf.c | 4 +- 5 files changed, 281 insertions(+), 186 deletions(-)
Signature · 1 line
2.53.0