Return-Path: Received: from mail.linutronix.de (146.0.238.70:993) by crypto-ml.lab.linutronix.de with IMAP4-SSL for ; 29 Jun 2018 19:17:37 -0000 Received: from mga04.intel.com ([192.55.52.120]) by Galois.linutronix.de with esmtps (TLS1.2:DHE_RSA_AES_256_CBC_SHA256:256) (Exim 4.80) (envelope-from ) id 1fYytk-0005bi-0N for speck@linutronix.de; Fri, 29 Jun 2018 21:17:36 +0200 Date: Fri, 29 Jun 2018 12:17:33 -0700 From: "Luck, Tony" Subject: [MODERATED] Re: [patch 2/2] cpu/hotplug: Boot HT siblings at least once Message-ID: <20180629191733.GA21690@agluck-desk> References: <20180629140546.075961202@linutronix.de> <20180629140611.033996006@linutronix.de> MIME-Version: 1.0 In-Reply-To: <20180629140611.033996006@linutronix.de> Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit To: speck@linutronix.de List-ID: On Fri, Jun 29, 2018 at 04:05:48PM +0200, speck for Thomas Gleixner wrote: > From: Thomas Gleixner > Subject: cpu/hotplug: Boot HT siblings at least once > > Due to the way Machine Check Exceptions work on X86 hyperthreads it's > required to boot up _all_ logical cores at least once in order to set the > CR4.MCE bit. > > So instead of ignoring the sibling threads right away, let them boot up > once so they can configure themself. After they came out of the initial > boot stage check whether its a "secondary" sibling and cancel the operation > which puts the CPU back into offline state. Having just joined the list. I don't have the full tree to apply this to. I hacked just this patch on top of v4.18-rc2 and booted with nosmt. [Is there a place to securely pick up a tarball of patches? Some of the problems below might be a result of my hacking not covering some patch bits I don't have.] Basics are working. dmesg during boot looked like: [ 0.158114] x86: Booting SMP configuration: [ 0.159002] .... node #0, CPUs: #1 #2 #3 #4 #5 #6 #7 #8 #9 #10 #11 #12 #13 #14 #15 #16 #17 #18 #19 #20 #21 #22 #23 [ 0.388003] .... node #1, CPUs: #24 #25 #26 #27 #28 #29 #30 #31 #32 #33 #34 #35 #36 #37 #38 #39 #40 #41 #42 #43 #44 #45 #46 #47 [ 0.754003] .... node #2, CPUs: #48 #49 #50 #51 #52 #53 #54 #55 #56 #57 #58 #59 #60 #61 #62 #63 #64 #65 #66 #67 #68 #69 #70 #71 [ 1.121003] .... node #3, CPUs: #72 #73 #74 #75 #76 #77 #78 #79 #80 #81 #82 #83 #84 #85 #86 #87 #88 #89 #90 #91 #92 #93 #94 #95 [ 1.430003] .... node #0, CPUs: #96 #97 #98 #99 #100 #101 #102 #103 #104 #105 #106 #107 #108 #109 #110 #111 #112 #113 #114 #115 #116 #117 #118 #119 [ 1.537008] .... node #1, CPUs: #120 #121 #122 #123 #124 #125 #126 #127 #128 #129 #130 #131 #132 #133 #134 #135 #136 #137 #138 #139 #140 #141 #142 #143 [ 1.659002] .... node #2, CPUs: #144 #145 #146 #147 #148 #149 #150 #151 #152 #153 #154 #155 #156 #157 #158 #159 #160 #161 #162 #163 #164 #165 #166 #167 [ 1.784002] .... node #3, CPUs: #168 #169 #170 #171 #172 #173 #174 #175 #176 #177 #178 #179 #180 #181 #182 #183 #184 #185 #186 #187 #188 #189 #190 #191 [ 1.889073] smp: Brought up 4 nodes, 96 CPUs so we started bringing up all CPUs, but only half ended up online. Awesome! Confusingly only five of the 96 offline CPUs reported they were offline: [ 26.528492] smpboot: CPU 100 is now offline [ 35.081182] smpboot: CPU 97 is now offline [ 35.122864] smpboot: CPU 99 is now offline [ 35.170483] smpboot: CPU 96 is now offline [ 35.239174] smpboot: CPU 98 is now offline Also a bit odd that these messages were delayed so long. It might be better if we didn't print this for any of them ... just useless noise? I injected an uncorrected error and consumed it to generate a broadcast machine check. That worked perfectly. Hurrah! I tried to bring CPU 96 online: # echo 1 > /sys/devices/system/cpu/cpu96/online but it tried, and failed: [ 318.998632] smpboot: Booting Node 0 Processor 96 APIC 0x1 [ 319.023475] smpboot: CPU 96 is now offline -Tony