Received: from list by lists.gnu.org with archive (Exim 4.90_1) id 1jCpeF-0004mn-FX for mharc-grub-devel@gnu.org; Fri, 13 Mar 2020 15:07:07 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:59182) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jCpeB-0004jz-P9 for grub-devel@gnu.org; Fri, 13 Mar 2020 15:07:06 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1jCpeA-0001Wh-1N for grub-devel@gnu.org; Fri, 13 Mar 2020 15:07:03 -0400 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:27912 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1jCpe9-0001Ul-TH for grub-devel@gnu.org; Fri, 13 Mar 2020 15:07:01 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1584126421; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=CVitHJ5S0VOo2pTBc/zk0bDINlXDqO1m1bx8SPC4JHo=; b=I58uyRkE/8dbT5zmZz+xB8XW4SZrSstE9v5kBSEUMQ9TYDsnOhnGE6vSU8An38EywZttmv edn0DdMxTduP+2RtKzdfEbIHrrST7VoGanAoQlgda/O0IFyTqt1qmB4KKJKk/487ESoQI1 LELzjJ73gfKCL/6vKLsm9VzJKNYlO+U= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-313-0mv-qBb0NiezZ5FTc6wTAQ-1; Fri, 13 Mar 2020 15:06:57 -0400 X-MC-Unique: 0mv-qBb0NiezZ5FTc6wTAQ-1 Received: by mail-wm1-f69.google.com with SMTP id y7so3809001wmd.4 for ; Fri, 13 Mar 2020 12:06:57 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=YWD2jbIh5j59rko883Uym9bC1z7XcI1jkLznSxgVRbI=; b=INjHDk9pw+7P4ICh1s0LCXKVT0BSUUE4FFsOtHYt4Ct57ea+JJYgEXq+F+8xDGtsX1 g43pZQHTaCO2LiSpn7tnFmDinfZ0CuZrcoZHc8yDBOhxK9cjEOXFLTXKtShNgJndE8js 09zu9oeoQcj9Wo94xahk5sfoODHYKBwCvevHJASpex6ymDjPocZlqcwF8472c7/9Xs6C zWSQJUgbfd/IhLbVE6z2jIUI3O0xxQISdLUFYggsN33B17iaFMClv9uFf2mHXzZ+OfBW B5uYfh8wsJHqgmkla5o74HefrILTVaU5xvZZ1V8CyavWshyNMSsODAaSPSQIlDZGHo7L Om0w== X-Gm-Message-State: ANhLgQ2MoBjOEYh4Jn+V/hvpMweq9GPPPQN7oRLCtMOHnnomfyKzIT+/ bPgtPyL/qPgMhVmEWpv82UbwvgaLaEPcr26/DronzVr/us0LfVfIUd+AlzoDE9Qtom1u5ZN4ZBh yXk0F1Y0U/TY= X-Received: by 2002:a7b:cb44:: with SMTP id v4mr11823068wmj.29.1584126414354; Fri, 13 Mar 2020 12:06:54 -0700 (PDT) X-Google-Smtp-Source: ADFU+vtD2UmqrZ2UV7p8mnNwVIwdQ9PABa7ACxJVRqXV4DAQa3ML01XIKwxSb1GQkh79f67Zx8FahQ== X-Received: by 2002:a7b:cb44:: with SMTP id v4mr11822983wmj.29.1584126413034; Fri, 13 Mar 2020 12:06:53 -0700 (PDT) Received: from minerva.redhat.com ([2a01:c50f:ab80:6700:527f:85b3:5615:ff0b]) by smtp.gmail.com with ESMTPSA id z11sm1919763wmc.30.2020.03.13.12.06.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Mar 2020 12:06:51 -0700 (PDT) From: Javier Martinez Canillas To: grub-devel@gnu.org Cc: Daniel Kiper , Peter Jones , Javier Martinez Canillas Subject: [PATCH 01/10] i386: Make pmtimer tsc calibration not take 51 seconds to fail Date: Fri, 13 Mar 2020 20:06:33 +0100 Message-Id: <20200313190642.822157-2-javierm@redhat.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200313190642.822157-1-javierm@redhat.com> References: <20200313190642.822157-1-javierm@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 207.211.31.120 X-BeenThere: grub-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: The development of GNU GRUB List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Mar 2020 19:07:06 -0000 From: Peter Jones On my laptop running at 2.4GHz, if I run a VM where tsc calibration using pmtimer will fail presuming a broken pmtimer, it takes ~51 seconds to do so (as measured with the stopwatch on my phone), with a tsc delta of 0x1cd1c85300, or around 125 billion cycles. If instead of trying to wait for 5-200ms to show up on the pmtimer, we try to wait for 5-200us, it decides it's broken in ~0x2626aa0 TSCs, aka ~2.4 million cycles, or more or less instantly. Additionally, this reading the pmtimer was returning 0xffffffff anyway, and that's obviously an invalid return. I've added a check for that and 0 so we don't bother waiting for the test if what we're seeing is dead pins with no response at all. If "debug" is includes "pmtimer", you will see one of the following three outcomes. If pmtimer gives all 0 or all 1 bits, you will see: pmtimer: 0xffffff bad_reads: 1 pmtimer: 0xffffff bad_reads: 2 pmtimer: 0xffffff bad_reads: 3 pmtimer: 0xffffff bad_reads: 4 pmtimer: 0xffffff bad_reads: 5 pmtimer: 0xffffff bad_reads: 6 pmtimer: 0xffffff bad_reads: 7 pmtimer: 0xffffff bad_reads: 8 pmtimer: 0xffffff bad_reads: 9 pmtimer: 0xffffff bad_reads: 10 timer is broken; giving up. This outcome was tested using qemu+kvm with UEFI (OVMF) firmware and these options: -machine pc-q35-2.10 -cpu Broadwell-noTSX If pmtimer gives any other bit patterns but is not actually marching forward fast enough to use for clock calibration, you will see: pmtimer delta is 0x0 (1904 iterations) tsc delta is implausible: 0x2626aa0 This outcome was tested using grub compiled with GRUB_PMTIMER_IGNORE_BAD_RE= ADS defined (so as not to trip the bad read test) using qemu+kvm with UEFI (OVMF) firmware, and these options: -machine pc-q35-2.10 -cpu Broadwell-noT= SX If pmtimer actually works, you'll see something like: pmtimer delta is 0xdff tsc delta is 0x278756 This outcome was tested using qemu+kvm with UEFI (OVMF) firmware, and these options: -machine pc-i440fx-2.4 -cpu Broadwell-noTSX I've also tested this outcome on a real Intel Xeon E3-1275v3 on an Intel Server Board S1200V3RPS using the SDV.RP.B8 "Release" build here: https://firmware.intel.com/sites/default/files/UEFIDevKit_S1200RP_vB8.zip Signed-off-by: Peter Jones Signed-off-by: Javier Martinez Canillas --- grub-core/kern/i386/tsc_pmtimer.c | 109 ++++++++++++++++++++++++------ 1 file changed, 89 insertions(+), 20 deletions(-) diff --git a/grub-core/kern/i386/tsc_pmtimer.c b/grub-core/kern/i386/tsc_pm= timer.c index c9c36169978..412f5fe3c83 100644 --- a/grub-core/kern/i386/tsc_pmtimer.c +++ b/grub-core/kern/i386/tsc_pmtimer.c @@ -28,40 +28,101 @@ #include #include =20 +/* + * Define GRUB_PMTIMER_IGNORE_BAD_READS if you're trying to test a timer t= hat's + * present but doesn't keep time well. + */ +// #define GRUB_PMTIMER_IGNORE_BAD_READS + grub_uint64_t grub_pmtimer_wait_count_tsc (grub_port_t pmtimer, =09=09=09 grub_uint16_t num_pm_ticks) { grub_uint32_t start; - grub_uint32_t last; - grub_uint32_t cur, end; + grub_uint64_t cur, end; grub_uint64_t start_tsc; grub_uint64_t end_tsc; - int num_iter =3D 0; + unsigned int num_iter =3D 0; +#ifndef GRUB_PMTIMER_IGNORE_BAD_READS + int bad_reads =3D 0; +#endif =20 - start =3D grub_inl (pmtimer) & 0xffffff; - last =3D start; + /* + * Some timers are 24-bit and some are 32-bit, but it doesn't make much + * difference to us. Caring which one we have isn't really worth it sin= ce + * the low-order digits will give us enough data to calibrate TSC. So j= ust + * mask the top-order byte off. + */ + cur =3D start =3D grub_inl (pmtimer) & 0x00ffffffUL; end =3D start + num_pm_ticks; start_tsc =3D grub_get_tsc (); while (1) { - cur =3D grub_inl (pmtimer) & 0xffffff; - if (cur < last) -=09cur |=3D 0x1000000; - num_iter++; + cur &=3D 0xffffffffff000000ULL; + cur |=3D grub_inl (pmtimer) & 0x00ffffffUL; + + end_tsc =3D grub_get_tsc(); + +#ifndef GRUB_PMTIMER_IGNORE_BAD_READS + /* + * If we get 10 reads in a row that are obviously dead pins, there's= no + * reason to do this thousands of times. + */ + if (cur =3D=3D 0xffffffUL || cur =3D=3D 0) +=09{ +=09 bad_reads++; +=09 grub_dprintf ("pmtimer", +=09=09=09"pmtimer: 0x%"PRIxGRUB_UINT64_T" bad_reads: %d\n", +=09=09=09cur, bad_reads); +=09 grub_dprintf ("pmtimer", "timer is broken; giving up.\n"); + +=09 if (bad_reads =3D=3D 10) +=09 return 0; +=09} +#endif + + if (cur < start) +=09cur +=3D 0x1000000; + if (cur >=3D end) =09{ -=09 end_tsc =3D grub_get_tsc (); +=09 grub_dprintf ("pmtimer", "pmtimer delta is 0x%"PRIxGRUB_UINT64_T"\n", +=09=09=09cur - start); +=09 grub_dprintf ("pmtimer", "tsc delta is 0x%"PRIxGRUB_UINT64_T"\n", +=09=09=09end_tsc - start_tsc); =09 return end_tsc - start_tsc; =09} - /* Check for broken PM timer. -=09 50000000 TSCs is between 5 ms (10GHz) and 200 ms (250 MHz) -=09 if after this time we still don't have 1 ms on pmtimer, then -=09 pmtimer is broken. + + /* + * Check for broken PM timer. 1ms at 10GHz should be 1E+7 TSCs; at + * 250MHz it should be 2.5E5. So if after 4E+7 TSCs on a 10GHz mach= ine, + * we should have seen pmtimer show 4ms of change (i.e. cur =3D~ + * start+14320); on a 250MHz machine that should be 160ms (start+572= 800). + * If after this a time we still don't have 1ms on pmtimer, then pmt= imer + * is broken. + * + * Likewise, if our code is perfectly efficient and introduces no de= lays + * whatsoever, on a 10GHz system we should see a TSC delta of 3580 i= n + * ~3580 iterations. On a 250MHz machine that should be ~900 iterat= ions. + * + * With those factors in mind, there are two limits here. There's a= hard + * limit here at 8x our desired pm timer delta, picked as an arbitra= rily + * large value that's still not a lot of time to humans, because if = we + * get that far this is either an implausibly fast machine or the pm= timer + * is not running. And there's another limit on 4x our 10GHz tsc de= lta + * without seeing cur converge on our target value. */ - if ((num_iter & 0xffffff) =3D=3D 0 && grub_get_tsc () - start_tsc > = 5000000) { -=09return 0; - } + if ((++num_iter > (grub_uint32_t)num_pm_ticks << 3UL) || +=09 end_tsc - start_tsc > 40000000) +=09{ +=09 grub_dprintf ("pmtimer", +=09=09=09"pmtimer delta is 0x%"PRIxGRUB_UINT64_T" (%u iterations)\n", +=09=09=09cur - start, num_iter); +=09 grub_dprintf ("pmtimer", +=09=09=09"tsc delta is implausible: 0x%"PRIxGRUB_UINT64_T"\n", +=09=09=09end_tsc - start_tsc); +=09 return 0; +=09} } } =20 @@ -74,12 +135,20 @@ grub_tsc_calibrate_from_pmtimer (void) =20 fadt =3D grub_acpi_find_fadt (); if (!fadt) - return 0; + { + grub_dprintf ("pmtimer", "No FADT found; not using pmtimer.\n"); + return 0; + } pmtimer =3D fadt->pmtimer; if (!pmtimer) - return 0; + { + grub_dprintf ("pmtimer", "FADT does not specify pmtimer; skipping.\n= "); + return 0; + } =20 - /* It's 3.579545 MHz clock. Wait 1 ms. */ + /* + * It's 3.579545 MHz clock. Wait 1 ms. + */ tsc_diff =3D grub_pmtimer_wait_count_tsc (pmtimer, 3580); if (tsc_diff =3D=3D 0) return 0; --=20 2.24.1