Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.0 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E17B2C433E6 for ; Tue, 21 Jul 2020 18:42:56 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id BA4A320717 for ; Tue, 21 Jul 2020 18:42:56 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BA4A320717 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=eu.citrix.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jxxDz-00026f-4m; Tue, 21 Jul 2020 18:42:47 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jxxDy-0001xV-N5 for xen-devel@lists.xenproject.org; Tue, 21 Jul 2020 18:42:46 +0000 X-Inumbo-ID: e926bf38-cb81-11ea-85a2-bc764e2007e4 Received: from chiark.greenend.org.uk (unknown [2001:ba8:1e3::]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id e926bf38-cb81-11ea-85a2-bc764e2007e4; Tue, 21 Jul 2020 18:42:21 +0000 (UTC) Received: from [172.18.45.5] (helo=zealot.relativity.greenend.org.uk) by chiark.greenend.org.uk (Debian Exim 4.84_2 #1) with esmtp (return-path ijackson@chiark.greenend.org.uk) id 1jxxDY-0001u7-Nz; Tue, 21 Jul 2020 19:42:20 +0100 From: Ian Jackson To: xen-devel@lists.xenproject.org Subject: [OSSTEST PATCH 06/14] sg-report-flight: Use WITH clause to use index for $anypassq Date: Tue, 21 Jul 2020 19:41:57 +0100 Message-Id: <20200721184205.15232-7-ian.jackson@eu.citrix.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200721184205.15232-1-ian.jackson@eu.citrix.com> References: <20200721184205.15232-1-ian.jackson@eu.citrix.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Ian Jackson , George Dunlap Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" Perf: runtime of my test case now ~11s Example query before (from the Perl DBI trace): SELECT * FROM flights JOIN steps USING (flight) WHERE (branch='xen-unstable') AND job=? and testid=? and status='pass' AND ( (TRUE AND flight <= 151903) AND (blessing='real') ) LIMIT 1 After: WITH s AS ( SELECT * FROM steps WHERE job=? and testid=? and status='pass' ) SELECT * FROM flights JOIN s USING (flight) WHERE (branch='xen-unstable') AND ( (TRUE AND flight <= 151903) AND (blessing='real') ) LIMIT 1 In both cases with bind vars: "test-amd64-i386-xl-pvshim" "guest-start" Diff to the query: - SELECT * FROM flights JOIN steps USING (flight) + WITH s AS + ( + SELECT * FROM steps + WHERE job=? and testid=? and status='pass' + ) + SELECT * FROM flights JOIN s USING (flight) WHERE (branch='xen-unstable') - AND job=? and testid=? and status='pass' AND ( (TRUE AND flight <= 151903) AND (blessing='real') ) LIMIT 1 CC: George Dunlap Signed-off-by: Ian Jackson --- schema/steps-job-index.sql | 2 +- sg-report-flight | 14 ++++++++++++-- 2 files changed, 13 insertions(+), 3 deletions(-) diff --git a/schema/steps-job-index.sql b/schema/steps-job-index.sql index 07dc5a30..2c33af72 100644 --- a/schema/steps-job-index.sql +++ b/schema/steps-job-index.sql @@ -1,4 +1,4 @@ --- ##OSSTEST## 006 Preparatory +-- ##OSSTEST## 006 Needed -- -- This index helps sg-report-flight find if a test ever passed. diff --git a/sg-report-flight b/sg-report-flight index b5398573..b8d948da 100755 --- a/sg-report-flight +++ b/sg-report-flight @@ -849,10 +849,20 @@ sub justifyfailures ($;$) { my @failures= values %{ $fi->{Failures} }; + # In psql 9.6 this WITH clause makes postgresql do the steps query + # first. This is good because if this test never passed we can + # determine that really quickly using the new index, without + # having to scan the flights table. (If the test passed we will + # probably not have to look at many flights to find one, so in + # that case this is not much worse.) my $anypassq= <