Skip to content

Flight report performance improvements

Ian Jackson xen-devel 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 Latest14 patches 11 repliesIn review

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/14sg-report-flight: Add a comment re same-flight search narrowingsg-report-flight+18 -00 replies
2/14sg-report-flight: Sort failures by job name as last resortsg-report-flight+1 -00 replies
3/14schema: Provide indices for sg-report-flightschema/runvars-built-index.sql schema/runvars-revision-index.sql schema/steps-job-index.sql+21 -00 replies
4/14sg-report-flight: Ask the db for flights of interestschema/runvars-built-index.sql sg-report-flight+62 -42 replies
5/14sg-report-flight: Use WITH to use best index use for $flightsqsg-report-flight+24 -152 replies
6/14sg-report-flight: Use WITH clause to use index for $anypassqschema/steps-job-index.sql sg-report-flight+13 -32 replies
7/14sg-report-flight: Use the job row from the intitial querysg-report-flight+3 -80 replies
8/14Executive: Use index for report__find_testOsstest/Executive.pm schema/runvars-revision-index.sql+9 -132 replies
9/14duration_estimator: Ignore truncated jobs unless we know the stepOsstest/Executive.pm+6 -20 replies
10/14duration_estimator: Introduce some _qtxt variablesOsstest/Executive.pm+4 -20 replies
11/14duration_estimator: Explicitly provide null in general host qOsstest/Executive.pm+2 -00 replies
12/14duration_estimator: Return job column in first queryOsstest/Executive.pm+3 -10 replies
13/14duration_estimator: Move $uptincl_testid to separate @x_paramsOsstest/Executive.pm+4 -10 replies
14/14duration_estimator: Move duration query loop into databaseOsstest/Executive.pm+20 -113 replies

Reviews trailers per patch

1 of 14 patches has a Reviewed-by or Acked-by.

PatchGeorge Dunlap
1/14
2/14
3/14
4/14
5/14
6/14R-b
7/14
8/14
9/14
10/14
11/14
12/14
13/14
14/14

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v1 thread

Cover letter [OSSTEST PATCH 00/14] Flight report performance improvements

osstest was taking far too long calculating what test failures were regressions, and generating the email and web reports. The slow part was analysing the test history, mostly because it ended up doing a lot of dumb scans of large tables.

In this series I fix this problem for sg-report-flight: I add some indexes, and reorganise some of the queries so that they can make good use of them.

I suspect there may still be problems with sg-report-host-history and cs-bisection-step. I haven't investigated those yet.

George: you volunteered to review my SQL. I hope the information in the commit messages is useful for that. Thanks!

Ian Jackson (14):
sg-report-flight: Add a comment re same-flight search narrowing
sg-report-flight: Sort failures by job name as last resort
schema: Provide indices for sg-report-flight
sg-report-flight: Ask the db for flights of interest
sg-report-flight: Use WITH to use best index use for $flightsq
sg-report-flight: Use WITH clause to use index for $anypassq
sg-report-flight: Use the job row from the intitial query
Executive: Use index for report__find_test
duration_estimator: Ignore truncated jobs unless we know the step
duration_estimator: Introduce some _qtxt variables
duration_estimator: Explicitly provide null in general host q
duration_estimator: Return job column in first query
duration_estimator: Move $uptincl_testid to separate @x_params
duration_estimator: Move duration query loop into database

 Osstest/Executive.pm              |  70 ++++++++++------
 schema/runvars-built-index.sql    |   7 ++
 schema/runvars-revision-index.sql |   7 ++
 schema/steps-job-index.sql        |   7 ++
 sg-report-flight                  | 127 +++++++++++++++++++++++++-----
 5 files changed, 174 insertions(+), 44 deletions(-)
 create mode 100644 schema/runvars-built-index.sql
 create mode 100644 schema/runvars-revision-index.sql
 create mode 100644 schema/steps-job-index.sql
Signature · 1 line
2.20.1