benchmarks/gem_blt: fix baseline estimation
PATCH 1/1 benchmarks/gem_blt: fix baseline estimation
From: Mauro Carvalho Chehab <[email protected]>
This test is expected to run under a certain time defined by a command line parameter (by default, 2s).
In order to achive the specified time, the baseline() function tries to estimate the value of a counter that will get the minimal amount of interaction to wait for > 0.1s.
However, currently, the baseline estimation is broken, returning a too high number, as it is measuring the memcpy() time, instead of actually measuring each loop's interaction.
Due to that, a default test without passing any command line parameter would take more than 1.5 years to output the first benchmark data!
Fix it by using the same logic that it is inside run() at the baseline time estimation.
On ADL-P, before this patch, baseline would return 399242693. After it, it now return 20 - meaning an interval of about 120 ms, which should be good enough to ensure that the tests won't take too long.
--- benchmarks/gem_blt.c | 18 +++++------------- 1 file changed, 5 insertions(+), 13 deletions(-)
benchmarks/gem_blt.c +5 -13
@@ -57,29 +57,21 @@ elapsed(const struct timespec *start, const struct timespec *end)return (end->tv_sec - start->tv_sec) +1e-9 *(end->tv_nsec - start->tv_nsec); } -static int baseline (uint64_t bytes,int milliseconds) +static int baseline (int fd,struct drm_i915_gem_execbuffer2 execbuf,int milliseconds) {struct timespec start, end; -const int size =64 *1024 *1024 ;int count =0 ; -void *mem; - - mem =malloc (size); -if (mem ==NULL ) -return 1 ;clock_gettime (CLOCK_MONOTONIC , &start);do { -memset (mem, count, size); +gem_execbuf (fd, &execbuf); count++;clock_gettime (CLOCK_MONOTONIC , &end); -if (elapsed (&start, &end) >0.1 ) +if (elapsed (&start, &end) > (milliseconds /1000. ))break ; }while (1 ); -free (mem); - -return ceil (1e-3 *milliseconds/elapsed (&start, &end) * count * size / bytes); +return count; } static int gem_linear_blt(int fd,
@@ -260,7 +252,7 @@ static int run(int object, int batch, int time, int reps, int ncpus, unsigned fl execbuf.flags |=I915_EXEC_NO_RELOC ;/* Guess how many loops we need for 0.1s */ - count =baseline ((uint64_t )object * batch,100 ) / ncpus; + count =baseline (fd, execbuf,100 ) / ncpus;if (flags &SYNC ) { time *= count /2 ; count =1 ;
Signature · 1 line
2.36.1
Patchwork
CI[igt-dev] ✗ Fi.CI.BAT: failure for benchmarks/gem_blt: fix baseline estimation
== Series Details ==
== Series Details ==
Series: benchmarks/gem_blt: fix baseline estimation
URL : https://patchwork.freedesktop.org/series/104163/
State : failure
== Summary ==
CI Bug Log - changes from IGT_6481 -> IGTPW_7141
====================================================
Summary
-------
**FAILURE**
Serious unknown changes coming with IGTPW_7141 absolutely need to be verified manually.
If you think the reported changes have nothing to do with the changes introduced in IGTPW_7141, please notify your bug team to allow them to document this new failure mode, which will reduce false positives in CI.
External URL: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7141/index.html
Participating hosts (45 -> 45)
------------------------------
Additional (2): fi-icl-u2 fi-pnv-d510
Missing (2): bat-dg2-8 bat-dg2-9
Possible new issues
-------------------
Here are the unknown changes that may have been introduced in IGTPW_7141:
### IGT changes ###
#### Possible regressions ####
* igt@i915_selftest@live@workarounds:
- fi-bdw-5557u: [PASS][1] -> [INCOMPLETE][2]
[1]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6481/fi-bdw-5557u/igt@i915_selftest@[email protected]
[2]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7141/fi-bdw-5557u/igt@i915_selftest@[email protected]
* igt@i915_suspend@basic-s3-without-i915:
- fi-cfl-8109u: [PASS][3] -> [INCOMPLETE][4]
[3]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6481/fi-cfl-8109u/igt@[email protected]
[4]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7141/fi-cfl-8109u/igt@[email protected]
Known issues
------------
Here are the changes found in IGTPW_7141 that come from known issues:
### IGT changes ###
#### Issues hit ####
* igt@gem_huc_copy@huc-copy:
- fi-icl-u2: NOTRUN -> [SKIP][5] ([i915#2190])
[5]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7141/fi-icl-u2/igt@[email protected]
* igt@gem_lmem_swapping@parallel-random-engines:
- fi-icl-u2: NOTRUN -> [SKIP][6] ([i915#4613]) +3 similar issues
[6]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7141/fi-icl-u2/igt@[email protected]
* igt@i915_pm_rpm@basic-rte:
- fi-cfl-8109u: [PASS][7] -> [DMESG-WARN][8] ([i915#1888] / [i915#62])
[7]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6481/fi-cfl-8109u/igt@[email protected]
[8]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7141/fi-cfl-8109u/igt@[email protected]
* igt@i915_pm_rpm@module-reload:
- fi-cfl-8109u: [PASS][9] -> [DMESG-FAIL][10] ([i915#62]) +1 similar issue
[9]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6481/fi-cfl-8109u/igt@[email protected]
[10]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7141/fi-cfl-8109u/igt@[email protected]
* igt@i915_pm_rps@basic-api:
- fi-glk-j4005: [PASS][11] -> [DMESG-WARN][12] ([i915#118])
[11]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6481/fi-glk-j4005/igt@[email protected]
[12]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7141/fi-glk-j4005/igt@[email protected]
* igt@i915_selftest@live@hangcheck:
- fi-hsw-g3258: [PASS][13] -> [INCOMPLETE][14] ([i915#4785])
[13]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6481/fi-hsw-g3258/igt@i915_selftest@[email protected]
[14]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7141/fi-hsw-g3258/igt@i915_selftest@[email protected]
* igt@i915_selftest@live@requests:
- fi-pnv-d510: NOTRUN -> [DMESG-FAIL][15] ([i915#4528])
[15]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7141/fi-pnv-d510/igt@i915_selftest@[email protected]
* igt@i915_suspend@basic-s2idle-without-i915:
- bat-dg1-5: NOTRUN -> [INCOMPLETE][16] ([i915#6011])
[16]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7141/bat-dg1-5/igt@[email protected]
* igt@i915_suspend@basic-s3-without-i915:
- fi-icl-u2: NOTRUN -> [SKIP][17] ([i915#5903])
[17]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7141/fi-icl-u2/igt@[email protected]
* igt@kms_busy@basic@flip:
- fi-tgl-u2: [PASS][18] -> [DMESG-WARN][19] ([i915#402]) +2 similar issues
[18]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6481/fi-tgl-u2/igt@kms_busy@[email protected]
[19]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7141/fi-tgl-u2/igt@kms_busy@[email protected]
* igt@kms_chamelium@common-hpd-after-suspend:
- fi-bsw-nick: NOTRUN -> [SKIP][20] ([fdo#109271] / [fdo#111827])
[20]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7141/fi-bsw-nick/igt@[email protected]
* igt@kms_chamelium@hdmi-hpd-fast:
- fi-icl-u2: NOTRUN -> [SKIP][21] ([fdo#111827]) +8 similar issues
[21]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7141/fi-icl-u2/igt@[email protected]
* igt@kms_cursor_legacy@basic-busy-flip-before-cursor-legacy:
- fi-icl-u2: NOTRUN -> [SKIP][22] ([fdo#109278]) +2 similar issues
[22]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7141/fi-icl-u2/igt@[email protected]
* igt@kms_flip@basic-flip-vs-modeset@b-edp1:
- bat-adlp-4: [PASS][23] -> [DMESG-WARN][24] ([i915#3576])
[23]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6481/bat-adlp-4/igt@kms_flip@[email protected]
[24]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7141/bat-adlp-4/igt@kms_flip@[email protected]
* igt@kms_force_connector_basic@force-load-detect:
- fi-icl-u2: NOTRUN -> [SKIP][25] ([fdo#109285])
[25]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7141/fi-icl-u2/igt@[email protected]
* igt@kms_pipe_crc_basic@compare-crc-sanitycheck-pipe-b:
- fi-cfl-8109u: [PASS][26] -> [DMESG-WARN][27] ([i915#62]) +12 similar issues
[26]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6481/fi-cfl-8109u/igt@[email protected]
[27]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7141/fi-cfl-8109u/igt@[email protected]
* igt@kms_pipe_crc_basic@suspend-read-crc-pipe-a:
- fi-bsw-nick: NOTRUN -> [SKIP][28] ([fdo#109271])
[28]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7141/fi-bsw-nick/igt@[email protected]
* igt@kms_setmode@basic-clone-single-crtc:
- fi-icl-u2: NOTRUN -> [SKIP][29] ([i915#3555])
[29]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7141/fi-icl-u2/igt@[email protected]
* igt@prime_vgem@basic-userptr:
- fi-pnv-d510: NOTRUN -> [SKIP][30] ([fdo#109271]) +39 similar issues
[30]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7141/fi-pnv-d510/igt@[email protected]
- fi-icl-u2: NOTRUN -> [SKIP][31] ([i915#3301])
[31]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7141/fi-icl-u2/igt@[email protected]
* igt@runner@aborted:
- fi-pnv-d510: NOTRUN -> [FAIL][32] ([fdo#109271] / [i915#2403] / [i915#4312])
[32]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7141/fi-pnv-d510/igt@[email protected]
- fi-cfl-8109u: NOTRUN -> [FAIL][33] ([i915#4312])
[33]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7141/fi-cfl-8109u/igt@[email protected]
- fi-hsw-g3258: NOTRUN -> [FAIL][34] ([fdo#109271] / [i915#4312])
[34]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7141/fi-hsw-g3258/igt@[email protected]
#### Possible fixes ####
* igt@i915_selftest@live@execlists:
- fi-bsw-nick: [INCOMPLETE][35] ([i915#5801]) -> [PASS][36]
[35]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6481/fi-bsw-nick/igt@i915_selftest@[email protected]
[36]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7141/fi-bsw-nick/igt@i915_selftest@[email protected]
* igt@i915_selftest@live@gt_engines:
- bat-dg1-5: [INCOMPLETE][37] ([i915#4418]) -> [PASS][38]
[37]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6481/bat-dg1-5/igt@i915_selftest@live@gt_engines.html
[38]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7141/bat-dg1-5/igt@i915_selftest@live@gt_engines.html
* igt@kms_flip@basic-flip-vs-modeset@a-edp1:
- {bat-adlp-6}: [DMESG-WARN][39] ([i915#3576]) -> [PASS][40] +1 similar issue
[39]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6481/bat-adlp-6/igt@kms_flip@[email protected]
[40]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7141/bat-adlp-6/igt@kms_flip@[email protected]
* igt@kms_pipe_crc_basic@compare-crc-sanitycheck-pipe-d:
- fi-tgl-u2: [DMESG-WARN][41] ([i915#402]) -> [PASS][42]
[41]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6481/fi-tgl-u2/igt@[email protected]
[42]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7141/fi-tgl-u2/igt@[email protected]
{name}: This element is suppressed. This means it is ignored when computing
the status of the difference (SUCCESS, WARNING, or FAILURE).
[fdo#109271]: https://bugs.freedesktop.org/show_bug.cgi?id=109271
[fdo#109278]: https://bugs.freedesktop.org/show_bug.cgi?id=109278
[fdo#109285]: https://bugs.freedesktop.org/show_bug.cgi?id=109285
[fdo#111827]: https://bugs.freedesktop.org/show_bug.cgi?id=111827
[i915#118]: https://gitlab.freedesktop.org/drm/intel/issues/118
[i915#1888]: https://gitlab.freedesktop.org/drm/intel/issues/1888
[i915#1982]: https://gitlab.freedesktop.org/drm/intel/issues/1982
[i915#2190]: https://gitlab.freedesktop.org/drm/intel/issues/2190
[i915#2403]: https://gitlab.freedesktop.org/drm/intel/issues/2403
[i915#2411]: https://gitlab.freedesktop.org/drm/intel/issues/2411
[i915#3301]: https://gitlab.freedesktop.org/drm/intel/issues/3301
[i915#3555]: https://gitlab.freedesktop.org/drm/intel/issues/3555
[i915#3576]: https://gitlab.freedesktop.org/drm/intel/issues/3576
[i915#402]: https://gitlab.freedesktop.org/drm/intel/issues/402
[i915#4312]: https://gitlab.freedesktop.org/drm/intel/issues/4312
[i915#4391]: https://gitlab.freedesktop.org/drm/intel/issues/4391
[i915#4418]: https://gitlab.freedesktop.org/drm/intel/issues/4418
[i915#4528]: https://gitlab.freedesktop.org/drm/intel/issues/4528
[i915#4613]: https://gitlab.freedesktop.org/drm/intel/issues/4613
[i915#4785]: https://gitlab.freedesktop.org/drm/intel/issues/4785
[i915#5801]: https://gitlab.freedesktop.org/drm/intel/issues/5801
[i915#5903]: https://gitlab.freedesktop.org/drm/intel/issues/5903
[i915#5950]: https://gitlab.freedesktop.org/drm/intel/issues/5950
[i915#6011]: https://gitlab.freedesktop.org/drm/intel/issues/6011
[i915#62]: https://gitlab.freedesktop.org/drm/intel/issues/62
Build changes
-------------
* CI: CI-20190529 -> None
* IGT: IGT_6481 -> IGTPW_7141
CI-20190529: 20190529
CI_DRM_11675: c31582c636f4c6ac8c0f822c39b250db2c5cd437 @ git://anongit.freedesktop.org/gfx-ci/linux
IGTPW_7141: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7141/index.html
IGT_6481: f2a9c2e6f6f7aa97e5d92274f20aa698087359c4 @ https://gitlab.freedesktop.org/drm/igt-gpu-tools.git
== Logs ==
For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7141/index.html
Hi Mauro,
small nitpicks on commit message.
Quoted 10 lines · Mauro Carvalho Chehab
From: Mauro Carvalho Chehab <[email protected]>
This test is expected to run under a certain time defined by a command line parameter (by default, 2s).
In order to achive the specified time, the
baseline()function tries to estimate the value of a counter that will get the minimal amount of interaction to wait for > 0.1s.However, currently, the baseline estimation is broken, returning a too high
------------------------------------------------------------------------ ^ s/high/big/
Quoted 2 lines
number, as it is measuring the
memcpy()time, instead of actually measuring each loop's interaction.
/each loop's interaction./gem_execbuf()./
Quoted 8 lines
Due to that, a default test without passing any command line parameter would take more than 1.5 years to output the first benchmark data!
Fix it by using the same logic that it is inside
run()at the baseline time estimation.On ADL-P, before this patch, baseline would return 399242693. After
s/On ADL-P, before/Before/
s/would/could/
Quoted 1 line
it, it now return 20 - meaning an interval of about 120 ms, which
Well, both numbers means the same interval _if_ used for the same syscall.
With that fixed you can add my r-b tag.
--
Kamil
Quoted 57 lines
should be good enough to ensure that the tests won't take too long.
Signed-off-by Mauro Carvalho Chehab <[email protected]>--- benchmarks/gem_blt.c | 18 +++++------------- 1 file changed, 5 insertions(+), 13 deletions(-)benchmarks/gem_blt.c +5
-13@@ -57,29 +57,21 @@ elapsed(const struct timespec *start, const struct timespec *end)return (end->tv_sec - start->tv_sec) +1e-9 *(end->tv_nsec - start->tv_nsec); } -static int baseline (uint64_t bytes,int milliseconds) +static int baseline (int fd,struct drm_i915_gem_execbuffer2 execbuf,int milliseconds) {struct timespec start, end; -const int size =64 *1024 *1024 ;int count =0 ; -void *mem; - - mem =malloc (size); -if (mem ==NULL ) -return 1 ;clock_gettime (CLOCK_MONOTONIC , &start);do { -memset (mem, count, size); +gem_execbuf (fd, &execbuf); count++;clock_gettime (CLOCK_MONOTONIC , &end); -if (elapsed (&start, &end) >0.1 ) +if (elapsed (&start, &end) > (milliseconds /1000. ))break ; }while (1 ); -free (mem); - -return ceil (1e-3 *milliseconds/elapsed (&start, &end) * count * size / bytes); +return count; } static int gem_linear_blt(int fd,@@ -260,7 +252,7 @@ static int run(int object, int batch, int time, int reps, int ncpus, unsigned fl execbuf.flags |=I915_EXEC_NO_RELOC ;/* Guess how many loops we need for 0.1s */ - count =baseline ((uint64_t )object * batch,100 ) / ncpus; + count =baseline (fd, execbuf,100 ) / ncpus;if (flags &SYNC ) { time *= count /2 ; count =1 ;Signature · 1 line
2.36.1