Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.3 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5275AC49EA3 for ; Mon, 28 Jun 2021 10:56:06 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C052361A2B for ; Mon, 28 Jun 2021 10:56:04 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C052361A2B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.intel.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id AC7396E0A8; Mon, 28 Jun 2021 10:56:03 +0000 (UTC) Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by gabe.freedesktop.org (Postfix) with ESMTPS id 178B26E0A8; Mon, 28 Jun 2021 10:56:02 +0000 (UTC) X-IronPort-AV: E=McAfee;i="6200,9189,10028"; a="271787905" X-IronPort-AV: E=Sophos;i="5.83,305,1616482800"; d="scan'208";a="271787905" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Jun 2021 03:56:01 -0700 X-IronPort-AV: E=Sophos;i="5.83,305,1616482800"; d="scan'208";a="454464450" Received: from danielmi-mobl2.ger.corp.intel.com (HELO [10.249.254.242]) ([10.249.254.242]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Jun 2021 03:55:59 -0700 Subject: Re: [PATCH v2 2/2] drm/i915/gem: only allow WB for smem only placements To: Matthew Auld , intel-gfx@lists.freedesktop.org References: <20210625122751.590289-1-matthew.auld@intel.com> <20210625122751.590289-2-matthew.auld@intel.com> <55e1981f-8945-1370-8753-d5749e0d69ba@intel.com> From: =?UTF-8?Q?Thomas_Hellstr=c3=b6m?= Message-ID: <03645a57-f044-ed25-61b4-a95a40a3f714@linux.intel.com> Date: Mon, 28 Jun 2021 12:55:57 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.10.1 MIME-Version: 1.0 In-Reply-To: <55e1981f-8945-1370-8753-d5749e0d69ba@intel.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Daniel Vetter , dri-devel@lists.freedesktop.org Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" On 6/28/21 12:20 PM, Matthew Auld wrote: > On 28/06/2021 10:38, Thomas Hellström wrote: >> Hi, >> >> On 6/28/21 11:12 AM, Matthew Auld wrote: >>> On 28/06/2021 08:41, Thomas Hellström wrote: >>>> >>>> On 6/25/21 2:27 PM, Matthew Auld wrote: >>>>> We only support single mode and this should be immutable. For smem >>>>> only >>>>> placements on DGFX this should be WB. On DG1 everything is snooped, >>>>> always, and so should be coherent. >>>>> >>>>> I915_GEM_DOMAIN_GTT looks like it's for the aperture which is now >>>>> gone >>>>> for DGFX, so hopefully can also be safely rejected. >>>>> >>>>> Signed-off-by: Matthew Auld >>>>> Cc: Thomas Hellström >>>>> Cc: Maarten Lankhorst >>>>> Cc: Daniel Vetter >>>>> --- >>>>>   drivers/gpu/drm/i915/gem/i915_gem_domain.c |  7 +++++++ >>>>>   drivers/gpu/drm/i915/gem/i915_gem_mman.c   | 10 ++++++++++ >>>>>   2 files changed, 17 insertions(+) >>>>> >>>>> diff --git a/drivers/gpu/drm/i915/gem/i915_gem_domain.c >>>>> b/drivers/gpu/drm/i915/gem/i915_gem_domain.c >>>>> index d0c91697bb22..e3459a524e64 100644 >>>>> --- a/drivers/gpu/drm/i915/gem/i915_gem_domain.c >>>>> +++ b/drivers/gpu/drm/i915/gem/i915_gem_domain.c >>>>> @@ -577,6 +577,13 @@ i915_gem_set_domain_ioctl(struct drm_device >>>>> *dev, void *data, >>>>>           goto out_unpin; >>>>>       } >>>>> +    if (IS_DGFX(to_i915(obj->base.dev)) && obj->mm.n_placements >>>>> == 1 && >>>>> +        i915_gem_object_placements_contain_type(obj, >>>>> INTEL_MEMORY_SYSTEM) && >>>>> +        read_domains != I915_GEM_DOMAIN_CPU) { >>>>> +        err = -EINVAL; >>>>> +        goto out_unpin; >>>>> +    } >>>>> + >>>>>       if (read_domains & I915_GEM_DOMAIN_WC) >>>>>           err = i915_gem_object_set_to_wc_domain(obj, write_domain); >>>>>       else if (read_domains & I915_GEM_DOMAIN_GTT) >>>>> diff --git a/drivers/gpu/drm/i915/gem/i915_gem_mman.c >>>>> b/drivers/gpu/drm/i915/gem/i915_gem_mman.c >>>>> index f3586b36dd53..afc9f3dc38b9 100644 >>>>> --- a/drivers/gpu/drm/i915/gem/i915_gem_mman.c >>>>> +++ b/drivers/gpu/drm/i915/gem/i915_gem_mman.c >>>>> @@ -673,6 +673,7 @@ __assign_mmap_offset(struct >>>>> drm_i915_gem_object *obj, >>>>>                enum i915_mmap_type mmap_type, >>>>>                u64 *offset, struct drm_file *file) >>>>>   { >>>>> +    struct drm_i915_private *i915 = to_i915(obj->base.dev); >>>>>       struct i915_mmap_offset *mmo; >>>>>       if (i915_gem_object_never_mmap(obj)) >>>>> @@ -697,6 +698,15 @@ __assign_mmap_offset(struct >>>>> drm_i915_gem_object *obj, >>>>>           i915_gem_object_placements_contain_type(obj, >>>>> INTEL_MEMORY_LOCAL)) >>>>>           return -ENODEV; >>>>> +    /* >>>>> +     * For smem only placements on DGFX we need to default to WB. >>>>> On DG1 >>>>> +     * everything is snooped always, so should always be coherent. >>>>> +     */ >>>>> +     if (IS_DGFX(i915) && >>>>> +         mmap_type != I915_MMAP_TYPE_WB && obj->mm.n_placements >>>>> == 1 && >>>>> +         i915_gem_object_placements_contain_type(obj, >>>>> INTEL_MEMORY_SYSTEM)) >>>>> +        return -ENODEV; >>>>> + >>>> >>>> Same thing here as in the previous patch. >>>> >>>> Also do we need to modify i915_coherent_map_type() to also include >>>> HAS_SNOOP()? >>>> >>>> While we're at it, that "always_coherent" argument to >>>> i915_coherent_map_type() appears scary to me and probably needs >>>> some documentation. It seems used for page-tables. Is it because we >>>> know those are always snooped? >>> >>> Yeah, it's either because the caller has/will mark the pages as >>> coherent(which translates to some special ppGTT bits), or we >>> manually flush ourselves. In i915_coherent_map_type() we should >>> account for DG1 somehow. >>> >>> Historically I don't think we enabled snooping by default since it's >>> considered slow compared to shared LLC. On DG1 this is a different >>> story though. >>> >>> Also the pin_map() interface is pretty much only for kernel internal >>> objects, so I don't think we have any users which try to map >>> userspace objects with that interface. Ok, except for vm_access it >>> seems, but that should hopefully be a simple fix to use the correct >>> caching mode? We can maybe add some sanity checking there if someone >>> tries to map a userspace object? >> I'm not fully sure that's sufficient, see below. >>> >>> For all the other callers of pin_map() which should all be kernel >>> internal do we still need to force WB for system memory? By design >>> we only support a single mm.mapping there. For lmem we already use >>> WC only. >> >> We're only allowed to map with the same caching mode as the linear >> kernel mapping for discrete. Otherwise things may blow up on >> non-intel architectures. We can probably update >> 195_ttm_select_tt_caching to always use WB for system pages for >> kernel objects, but then we must make sure we don't try to map these WC. > > Ok, do you think that should be a separate series? It looks like our > internal objects don't use ttm(?). Should it? If so should we make a > region for it, or can we just make create_internal use the ttm system > region? It should be pretty much the same, except we don't want > swapping, clearing or eviction, and ideally we would have some way of > marking the pages as volatile(I think we can just keep IS_SHRINKABLE > for that). > > Or can we keep create_internal as is and then it's just a case of > dealing with all the pin_map() callers? > For now, I think, the internal objects don't really need to use ttm, and if we want to move them over, as you say, it's just a matter of reusing the SYSTEM region with suitable flags. I think we should restrict this to deal with pin_map() for now, and a separate series would work fine. Although there seem to be callers, at least in selftests, that use pin_map with create_shmem() which maps to TTM system on discrete. /Thomas >> >> /Thomas >> >> >>> >>>> >>>> /Thomas >>>> >>>> >>>>>       mmo = mmap_offset_attach(obj, mmap_type, file); >>>>>       if (IS_ERR(mmo)) >>>>>           return PTR_ERR(mmo);