Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id F1D4CEB64D7 for ; Tue, 13 Jun 2023 20:41:09 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 723DC81F2D; Tue, 13 Jun 2023 20:41:09 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 723DC81F2D DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=osuosl.org; s=default; t=1686688869; bh=7nIGPeu8ZtPS28mUh6qYzmwi9tgIcFTHfKWr6vanxrQ=; h=From:To:Date:Subject:List-Id:List-Unsubscribe:List-Archive: List-Post:List-Help:List-Subscribe:From; b=9PE5E8mLTHUZVahUmTuQlgxeBdtZ6DG9kDjnft4nLWkC993rRVraN2SgL7iGC84Vi 3fsreJTC72kZg1zSvDGgwZ2X2SUmk4pdd/M28vMwgIJGud/tDKzQDxrQ2+EiE+fAFf oxhyJ6Dk8MaGUN7w0W35a9xFiDwf/XsNrkeuFF/x85EM2+xIDLDt4mJVmtBQfPz0vU GTjxODZEkB3fUAemNrWBPpq3rHIPpRLZPaPO7aXTzTiQCtdUao2RLtONF6RUMaO+uS O3n0W1WCxhb8n89KR0PPBMwdNGGW29msYQ65C2FvrdOG0gu46EyCCMyj4cVq3zWJ2f 0fsh/G4IQtv3g== X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id VGdS9_g0l78m; Tue, 13 Jun 2023 20:41:08 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by smtp1.osuosl.org (Postfix) with ESMTP id 1768481F34; Tue, 13 Jun 2023 20:41:08 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 1768481F34 Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by ash.osuosl.org (Postfix) with ESMTP id C52F91BF31A for ; Tue, 13 Jun 2023 20:41:06 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 92D2360E83 for ; Tue, 13 Jun 2023 20:41:06 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 92D2360E83 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id PkK4vjp272nP for ; Tue, 13 Jun 2023 20:41:00 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org BFB0060B1B Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by smtp3.osuosl.org (Postfix) with ESMTPS id BFB0060B1B for ; Tue, 13 Jun 2023 20:41:00 +0000 (UTC) X-IronPort-AV: E=McAfee;i="6600,9927,10740"; a="348105447" X-IronPort-AV: E=Sophos;i="6.00,240,1681196400"; d="scan'208";a="348105447" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Jun 2023 13:40:59 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10740"; a="781831986" X-IronPort-AV: E=Sophos;i="6.00,240,1681196400"; d="scan'208";a="781831986" Received: from jekeller-desk.amr.corp.intel.com (HELO jekeller-desk.jekeller.internal) ([10.166.241.1]) by fmsmga004-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Jun 2023 13:40:58 -0700 From: Jacob Keller To: Intel Wired LAN , Anthony Nguyen , Michal Schmidt Date: Tue, 13 Jun 2023 13:40:53 -0700 Message-ID: <20230613204053.3581328-1-jacob.e.keller@intel.com> X-Mailer: git-send-email 2.40.0.471.gbd7f14d9353b MIME-Version: 1.0 X-Mailman-Original-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1686688860; x=1718224860; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=2V/fPMSwXJhrcuCmNVAMs+rqj/DoUG0I24TmvnTF5Fo=; b=XOPSwXibiEjGBaHVq6dWuRyABCjcV2E61ASyVYZke2ikpar6znlOKSIs eAkm/vmCOquF0W70w+o3xb7k2l5DpVcPVOmGV+t/RJdTmtq2RXGjlULp+ xUEQPwzjD6zHhgDy4HrxsqOC6ce/htiok8/QfG5sFDlvsl6+Sn/ib8Fm6 ZlAwiZ4d6kfGKmeceBEF6Lvtab3IuWLEodbohbKLgYuHomj2xU4/cxUZZ b0Gnn5IxfuV6cW/nY+6ludUzqF168E/wQDmzquT3zQte2zF4Fnb15JeAp wWPyXw/l+pnQQPO3eizv8pwNx3jTBHSIiLJh7k7z0hNR9CP620CrAdRhA Q==; X-Mailman-Original-Authentication-Results: smtp3.osuosl.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=XOPSwXib Subject: [Intel-wired-lan] [PATCH iwl-next] ice: reduce initial wait for control queue messages X-BeenThere: intel-wired-lan@osuosl.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel Wired Ethernet Linux Kernel Driver Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: intel-wired-lan-bounces@osuosl.org Sender: "Intel-wired-lan" The ice_sq_send_cmd() function is used to send messages to the control queues used to communicate with firmware, virtual functions, and even some hardware. When sending a control queue message, the driver is designed to synchronously wait for a response from the queue. Currently it waits between checks for 100 to 150 microseconds. Commit f86d6f9c49f6 ("ice: sleep, don't busy-wait, for ICE_CTL_Q_SQ_CMD_TIMEOUT") did recently change the behavior from an unnecessary delay into a sleep which is a significant improvement over the old behavior of polling using udelay. Because of the nature of PCIe transactions, the hardware won't be informed about a new message until the write to the tail register posts. This is only guaranteed to occur at the next register read. In ice_sq_send_cmd(), this happens at the ice_sq_done() call. Because of this, the driver essentially forces a minimum of one full wait time regardless of how fast the response is. For the hardware-based sideband queue, this is especially slow. It is expected that the hardware will respond within 2 or 3 microseconds, an order of magnitude faster than the 100-150 microsecond sleep. Allow such fast completions to occur without delay by introducing a small 5 microsecond delay first before entering the sleeping timeout loop. Ensure the tail write has been posted by using ice_flush(hw) first. While at it, lets also remove the ICE_CTL_Q_SQ_CMD_USEC macro as it obscures the sleep time in the inner loop. It was likely introduced to avoid "magic numbers", but in practice sleep and delay values are easier to read and understand when using actual numbers instead of a named constant. This change should allow the fast hardware based control queue messages to complete quickly without delay, while slower firmware queue response times will sleep while waiting for the response. Signed-off-by: Jacob Keller --- @Michal, do you think this initial 5 microsecond delay would have significant impact on the use cases that you fixed in the mentioned commit? I don't want to break those, but do want to make sure that the fast response hardware queues don't wait unnecessarily. One alternative I considered was only doing this delay if we are on the appropriate queue type. I'd appreciate your thoughts. drivers/net/ethernet/intel/ice/ice_controlq.c | 9 +++++++-- drivers/net/ethernet/intel/ice/ice_controlq.h | 1 - 2 files changed, 7 insertions(+), 3 deletions(-) diff --git a/drivers/net/ethernet/intel/ice/ice_controlq.c b/drivers/net/ethernet/intel/ice/ice_controlq.c index d2faf1baad2f..385fd88831db 100644 --- a/drivers/net/ethernet/intel/ice/ice_controlq.c +++ b/drivers/net/ethernet/intel/ice/ice_controlq.c @@ -1056,14 +1056,19 @@ ice_sq_send_cmd(struct ice_hw *hw, struct ice_ctl_q_info *cq, if (cq->sq.next_to_use == cq->sq.count) cq->sq.next_to_use = 0; wr32(hw, cq->sq.tail, cq->sq.next_to_use); + ice_flush(hw); + + /* Wait a short time before initial ice_sq_done() check, to allow + * hardware time for completion. + */ + udelay(5); timeout = jiffies + ICE_CTL_Q_SQ_CMD_TIMEOUT; do { if (ice_sq_done(hw, cq)) break; - usleep_range(ICE_CTL_Q_SQ_CMD_USEC, - ICE_CTL_Q_SQ_CMD_USEC * 3 / 2); + usleep_range(100, 150); } while (time_before(jiffies, timeout)); /* if ready, copy the desc back to temp */ diff --git a/drivers/net/ethernet/intel/ice/ice_controlq.h b/drivers/net/ethernet/intel/ice/ice_controlq.h index 950b7f4a7a05..8f2fd1613a95 100644 --- a/drivers/net/ethernet/intel/ice/ice_controlq.h +++ b/drivers/net/ethernet/intel/ice/ice_controlq.h @@ -35,7 +35,6 @@ enum ice_ctl_q { /* Control Queue timeout settings - max delay 1s */ #define ICE_CTL_Q_SQ_CMD_TIMEOUT HZ /* Wait max 1s */ -#define ICE_CTL_Q_SQ_CMD_USEC 100 /* Check every 100usec */ #define ICE_CTL_Q_ADMIN_INIT_TIMEOUT 10 /* Count 10 times */ #define ICE_CTL_Q_ADMIN_INIT_MSEC 100 /* Check every 100msec */ -- 2.40.0.471.gbd7f14d9353b _______________________________________________ Intel-wired-lan mailing list Intel-wired-lan@osuosl.org https://lists.osuosl.org/mailman/listinfo/intel-wired-lan