Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 54BD5C4332F for ; Thu, 9 Nov 2023 15:06:06 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 9CD6788EFE; Thu, 9 Nov 2023 15:06:05 +0000 (UTC) Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.20]) by gabe.freedesktop.org (Postfix) with ESMTPS id AD12510E8C4 for ; Thu, 9 Nov 2023 15:05:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1699542356; x=1731078356; h=message-id:date:mime-version:subject:to:references:from: in-reply-to:content-transfer-encoding; bh=dBhZtYupHQt42uFc548rxxliTyHYOawwekhp4y2N6dk=; b=DeWuL2wBwnjHIeeP1ZILcB7e2xofcJmVKgmB91uU/lDOI+6IxHvg41OD z7Wf2TUt8MP08whL9WaXrO6YAclQ4VxFC64hKWu7lWKc8j/PLb8TJkVEM GSiaJGmdSIR4ext0rBxhNF114uzJSm0c/G7PqWLe6HVgu5q50jQTnjLsZ TAuzaqcXDDOau3frcMl/iltOBCmHFby0j4rYbd1Sy2lj4GMj0rwRYZgM1 bgpnvNJ162R6uPKzoFfUDjyiagss77Qe98eFLJyOKYo/QCbLH82x9xPY1 IN2n2amP9qli7I9D0HtL56QIfG2mi925PgzIh3+Avxewq8Te6RhFh/dhV A==; X-IronPort-AV: E=McAfee;i="6600,9927,10889"; a="380393264" X-IronPort-AV: E=Sophos;i="6.03,289,1694761200"; d="scan'208";a="380393264" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Nov 2023 07:05:33 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10889"; a="887034378" X-IronPort-AV: E=Sophos;i="6.03,289,1694761200"; d="scan'208";a="887034378" Received: from bbegley-mobl1.ger.corp.intel.com (HELO [10.252.4.12]) ([10.252.4.12]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Nov 2023 07:05:28 -0800 Message-ID: Date: Thu, 9 Nov 2023 15:05:24 +0000 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0 Thunderbird/102.15.1 To: "Milczarek, Slawomir" , Tomasz Mistat , "intel-xe@lists.freedesktop.org" , "Arteaga Molina, Jaime A" , "Cetnerowski, Adam" , "Chodor, Jaroslaw" , "Dunajski, Bartosz" , "Harasimiuk, Artur" , "Hoppe, Mateusz" , "Jablonski, Mateusz" , "Jobczyk, Lukasz" , "Kasper, Kacper K" , "Mrozek, Michal" , "Rozenfeld, Piotr" , "Wilma, Pawel" , "Wolny, Bartlomiej" , "Yates, Brandon" , "Zdanowicz, Zbigniew" References: <0c8104df-3c51-95a4-aec1-392b4b92efb9@intel.com> Content-Language: en-GB From: Matthew Auld In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Intel-xe] FW: [PATCH v9 0/3] PAT and cache coherency support X-BeenThere: intel-xe@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel Xe graphics driver List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-xe-bounces@lists.freedesktop.org Sender: "Intel-xe" On 09/11/2023 14:58, Milczarek, Slawomir wrote: > Hi Matthew, > > In the compute driver we call GEM_MMAP_OFFSET with flags to specify cpu cache mode (wb/wc): > GEM_CREATE_EXT() and mmap() + GEM_MMAP_OFFSET(flags=wb/wc) > > Starting from Xe KMD we should be able to achieve the same CPU caching policies with GEM_CREATE_EXT: > GEM_CREATE_EXT(flags=wb/wc) and mmap() + GEM_MMAP_OFFSET(flags=0) > Am I correct? Yes, that's correct. > > Thanks, > Slawek > > -----Original Message----- > From: Auld, Matthew > Sent: Wednesday, November 8, 2023 1:04 PM > To: Tomasz Mistat ; intel-xe@lists.freedesktop.org; Arteaga Molina, Jaime A ; Cetnerowski, Adam ; Chodor, Jaroslaw ; Dunajski, Bartosz ; Harasimiuk, Artur ; Hoppe, Mateusz ; Jablonski, Mateusz ; Jobczyk, Lukasz ; Kasper, Kacper K ; Milczarek, Slawomir ; Mrozek, Michal ; Rozenfeld, Piotr ; Wilma, Pawel ; Wolny, Bartlomiej ; Yates, Brandon ; Zdanowicz, Zbigniew ; Dunajski, Bartosz > Subject: Re: FW: [Intel-xe] [PATCH v9 0/3] PAT and cache coherency support > > On 08/11/2023 11:30, Tomasz Mistat wrote: >> compute-umd_uapi_review_team@intel.com,bartosz.dunajski@intel.com,saur >> abhg.gupta@intel.com,ulisses.furquim@intel.com,michal.mrozek@intel.com >> ,tomasz.mistat@intel.com,piotr.rozenfeld@intel.com,ayaz.siddiqui@intel >> .com,luis.strano@intel.com,filip.hazubski@intel.com,filip.hazubski@int >> el.com,mateusz.naklicki@intel.com,kamil.kopryk@intel.com >> Bcc: >> Subject: Re: [Intel-xe] [PATCH v9 0/3] PAT and cache coherency support >> Reply-To: >> In-Reply-To: <20231103154309.402159-5-matthew.auld@intel.com> >> >> On 2023-11-03 at 15:43:10 +0000, Matthew Auld wrote: >>> Branch available here: >>> https://gitlab.freedesktop.org/mwa/kernel/-/tree/xe-pat-index?ref_typ >>> e=heads >>> >>> IGT changes: >>> https://patchwork.freedesktop.org/series/124667/ >>> >>> Mesa: >>> https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25462 >>> >>> Goal here is to allow userspace to directly control the pat_index >>> when mapping memory via the ppGTT, in addtion to the CPU caching >>> mode. This is very much needed on newer igpu platforms which allow >>> incoherent GT access, where the choice over the cache level and >>> expected coherency is best left to userspace depending on their >>> usecase. In the future there may also be other stuff encoded in the pat_index, so giving userspace direct control will also be needed there. >>> >>> To support this we added new gem_create uAPI for selecting the CPU >>> cache mode to use for system memory, including the expected GPU >>> coherency mode. There are various restrictions here for the selected >>> coherency mode and compatible CPU cache modes. With that in place >>> the actual pat_index can now be provided as part of vm_bind. The only >>> restriction is that the coherency mode of the pat_index must be at >>> least as coherent as the gem_create coherency mode. There are also some special cases like with userptr and dma-buf. >>> >>> v2: >>> - Loads of improvements/tweaks. Main changes are to now allow >>> gem_create.coh_mode <= coh_mode(pat_index), rather than it needing to match >>> exactly. This simplifies the dma-buf policy from userspace pov. Also we now >>> only consider COH_NONE and COH_AT_LEAST_1WAY. >>> v3: >>> - Rebase. Split the pte_encode() refactoring, plus various smaller tweaks and >>> fixes. >>> v4: >>> - Rebase on Lucas' new series. >>> - Drop UC cache mode. >>> - s/smem_cpu_caching/cpu_caching/. Idea is to make VRAM WC explicit in the >>> uapi, plus make it more future proof. >>> v5: >>> - Rebase, plus some small tweaks and fixes. >>> v6: >>> - CI hooks fixes + checkpatch. >>> v7: >>> - Some small tweaks >>> v8: >>> - Rebase on Xe2 PAT table additions. >>> v9: >>> - Drop coh_mode. >>> >>> -- >>> 2.41.0 >>> >> >> I'm sharing down here questions passed by bartosz.dunajski@intel.com >> lets continue review here at respective ML >> >> Sent: Wednesday, November 8, 2023 09:05 >> Subject: RE: [Intel-xe] [PATCH v9 0/3] PAT and cache coherency support >> >> Just to confirm my understanding. We will have 2 separate APIs? >> 1. For gem_create to explicitly set coherency/caching according to API enums > > Yeah, you just need to provide the cpu_caching mode (wc or wb). If you later mmap() the object this same caching mode is used. Also any KMD internal things like swap-in will also respect this caching mode. The coh_mode is now removed. > >> 2. For vm_bind to set PAT index that is aligned with Bspec and controls more than #1. It can also override coherency/caching that was previously set? > > Right, as part of vm_bind you provide whatever PAT index control (as per > Bspec) you want for each mapping of the object (you could map it multiple times). > > When choosing a PAT index the cpu_caching only matters if you choose an incoherent PAT index (coh_none in Bspec), since we don't want to allow cpu_caching=wb and coh_none, since that leads to some complications in the KMD. > >> >>