Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 37959C2BBCF for ; Fri, 18 Dec 2020 13:37:05 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9E8B1235FA for ; Fri, 18 Dec 2020 13:37:04 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9E8B1235FA Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=irrelevant.dk Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:38222 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kqFwN-0005iT-E2 for qemu-devel@archiver.kernel.org; Fri, 18 Dec 2020 08:37:03 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:55610) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kqFp0-0003Vw-Jg; Fri, 18 Dec 2020 08:29:26 -0500 Received: from wout5-smtp.messagingengine.com ([64.147.123.21]:54887) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kqFoq-0002vO-UZ; Fri, 18 Dec 2020 08:29:25 -0500 Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailout.west.internal (Postfix) with ESMTP id E4C34438; Fri, 18 Dec 2020 08:29:14 -0500 (EST) Received: from mailfrontend1 ([10.202.2.162]) by compute2.internal (MEProxy); Fri, 18 Dec 2020 08:29:15 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=irrelevant.dk; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; s=fm1; bh=FZ2cT2+OPXRaa CjWDHktlCGJWTp/4N3joJPKRgShwQw=; b=CdbhXJANcbZbfb8vG71x/vNJTa2V/ hOc2kkg5tNDZ7D8NZcUzelJ21Lylz52cNBeuV8jq+yOqLYrSBBSRRcBnzCixbU/9 E1kebWf8lJksbs8BiwtksQUR01ZOMRB9zQIxPIsEHsblARdclIs8P+X0R8R9CjAe MSNsK/69Q96B/1/+vOV5afPd9zCwCU1p5ICn4wB9a5BA8bJHtKsw3NJyiSJncwd1 u1HvOYw3/VIjOmtteRWusfT0RyQtJbquUw4WFw5DevUxfvQNukoLnKp4DQ1jtUPE 8dBUuyDhIOYOGo1g2TwtteEvIZknxjdx2mwzFmtfkvhu/SyGOATBon1dg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:date:from :in-reply-to:message-id:mime-version:references:subject:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; bh=FZ2cT2+OPXRaaCjWDHktlCGJWTp/4N3joJPKRgShwQw=; b=I6bD4wcA jbDa/PnQN8ed//Ety2QKEy/o9AsTi84LJReDNzbghtuk42mwkL5KHYBTQeojqmVd LTF/VMcfdAbHNNZKNd6sho/VyEnp4clJdwQjpER4qfarnzwHK3nVrAtC+EBlkAsm 0c1izCevsrSpumL37YlMVyRALPwVTKCMAf/o7mDPVrHRRHUgmrp0nMhlBMGLVF4Y rbAMUaPdL7nhJxsTOQixi7DNIrAIeRh7dUpJoXHSqiyACFRu5nb60nV7jP+C0qsp vx33VBvI6cEtH3qNT/RKmVZiL0DMHZdVssMosKVmDOi89Ugix4q5il141B0YliWA rcqAtJgl6lyY9A== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedujedrudeliedgheefucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvufffkffojghfggfgsedtkeertdertddtnecuhfhrohhmpefmlhgruhhs ucflvghnshgvnhcuoehithhssehirhhrvghlvghvrghnthdrughkqeenucggtffrrghtth gvrhhnpeeuleetgeeiuefhgfekfefgveejiefgteekiedtgfdtieefhfdthfefueffvefg keenucfkphepkedtrdduieejrdelkedrudeltdenucevlhhushhtvghrufhiiigvpedune curfgrrhgrmhepmhgrihhlfhhrohhmpehithhssehirhhrvghlvghvrghnthdrughk X-ME-Proxy: Received: from apples.local (80-167-98-190-cable.dk.customer.tdc.net [80.167.98.190]) by mail.messagingengine.com (Postfix) with ESMTPA id 3F2E624005B; Fri, 18 Dec 2020 08:29:13 -0500 (EST) From: Klaus Jensen To: qemu-devel@nongnu.org Subject: [PATCH 4/8] hw/block/nvme: fix controller reset/shutdown logic Date: Fri, 18 Dec 2020 14:29:01 +0100 Message-Id: <20201218132905.967326-5-its@irrelevant.dk> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201218132905.967326-1-its@irrelevant.dk> References: <20201218132905.967326-1-its@irrelevant.dk> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Received-SPF: pass client-ip=64.147.123.21; envelope-from=its@irrelevant.dk; helo=wout5-smtp.messagingengine.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Kevin Wolf , qemu-block@nongnu.org, Klaus Jensen , Max Reitz , Keith Busch , Andrzej Jakowski , Stefan Hajnoczi , Klaus Jensen Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Klaus Jensen Differentiate between Controller Level Reset (i.e. CC.EN transitions from 1 to 0) and Controller Shutdown (i.e. host sets CC.SHN to 01b or 10b). Prior to this patch the controller would perform the equivalent of a Controller Level Reset when the host set CC.SHN to 01b or 10b. This erroneously entails clearing CC. For a Shutdown, it is the host that *should* delete any I/O queues. Setting CC.EN to 0 is not part of the Shutdown procedure and it is specifically recommended that the host does NOT do this. The host should also stop issuing new I/O commands and allow any outstanding ones to complete before setting CC.SHN. The only guarantee the controller must give is that it is safe to power off the controller when CSTS.SHST indicates that shutdown processing is complete. Do so by issuing flushes for the attached namespaces and the PMR (if in use) and set CSTS.SHST to 10b. Specifically do not care about draining since the host *should* not have left any outstanding commands around. If it did, behavior is undefined. On the other hand, a Controller Level Reset is not about being safe for shutdown - so only drain here, do not explicitly flush. Signed-off-by: Klaus Jensen --- hw/block/nvme.c | 37 ++++++++++++++++++++++++------------- 1 file changed, 24 insertions(+), 13 deletions(-) diff --git a/hw/block/nvme.c b/hw/block/nvme.c index 69f2fe4bd4ff..96c9f9be3dc4 100644 --- a/hw/block/nvme.c +++ b/hw/block/nvme.c @@ -2262,7 +2262,26 @@ static void nvme_process_sq(void *opaque) } } -static void nvme_clear_ctrl(NvmeCtrl *n) +static void nvme_ctrl_shutdown(NvmeCtrl *n) +{ + NvmeNamespace *ns; + int i; + + if (n->pmrdev) { + memory_region_msync(&n->pmrdev->mr, 0, n->pmrdev->size); + } + + for (i = 1; i <= n->num_namespaces; i++) { + ns = nvme_ns(n, i); + if (!ns) { + continue; + } + + nvme_ns_flush(ns); + } +} + +static void nvme_ctrl_reset(NvmeCtrl *n) { NvmeNamespace *ns; int i; @@ -2297,15 +2316,6 @@ static void nvme_clear_ctrl(NvmeCtrl *n) n->outstanding_aers = 0; n->qs_created = false; - for (i = 1; i <= n->num_namespaces; i++) { - ns = nvme_ns(n, i); - if (!ns) { - continue; - } - - nvme_ns_flush(ns); - } - n->bar.cc = 0; } @@ -2473,12 +2483,12 @@ static void nvme_write_bar(NvmeCtrl *n, hwaddr offset, uint64_t data, } } else if (!NVME_CC_EN(data) && NVME_CC_EN(n->bar.cc)) { trace_pci_nvme_mmio_stopped(); - nvme_clear_ctrl(n); + nvme_ctrl_reset(n); n->bar.csts &= ~NVME_CSTS_READY; } if (NVME_CC_SHN(data) && !(NVME_CC_SHN(n->bar.cc))) { trace_pci_nvme_mmio_shutdown_set(); - nvme_clear_ctrl(n); + nvme_ctrl_shutdown(n); n->bar.cc = data; n->bar.csts |= NVME_CSTS_SHST_COMPLETE; } else if (!NVME_CC_SHN(data) && NVME_CC_SHN(n->bar.cc)) { @@ -3110,7 +3120,8 @@ static void nvme_exit(PCIDevice *pci_dev) { NvmeCtrl *n = NVME(pci_dev); - nvme_clear_ctrl(n); + nvme_ctrl_reset(n); + g_free(n->cq); g_free(n->sq); g_free(n->aer_reqs); -- 2.29.2