Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7D6CEC38159 for ; Wed, 18 Jan 2023 14:11:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D8E256B0074; Wed, 18 Jan 2023 09:11:21 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D3DC96B0075; Wed, 18 Jan 2023 09:11:21 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BDEAC6B0078; Wed, 18 Jan 2023 09:11:21 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id ACA126B0074 for ; Wed, 18 Jan 2023 09:11:21 -0500 (EST) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 5FD8F120C6D for ; Wed, 18 Jan 2023 14:11:21 +0000 (UTC) X-FDA: 80368107162.19.94F6E77 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf23.hostedemail.com (Postfix) with ESMTP id 60527140031 for ; Wed, 18 Jan 2023 14:11:11 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=KoyTo1Jh; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf23.hostedemail.com: domain of david@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=david@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1674051076; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Sr1/LUQHwubaC/6+yigReObSaZRNqB3QQDgWoJAi3TM=; b=rfTZ3X8hKPvqRwhCuQQzvCEbOJZKsNcmv+DzlaEemK6zGUaH8jJWrtYBZhJ2y/mVb1F9cN fze/P7Qm/x0I/NbOW/+7HUGjEyx1DVtupxE0QDmYwDRFMkx/QO/1pifDvYlADe8FSaJbRY 8/JSkPy+ARFagPyCFDE8PipnKsFk3BY= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=KoyTo1Jh; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf23.hostedemail.com: domain of david@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=david@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1674051076; a=rsa-sha256; cv=none; b=xjiIFQNG+6dRlgrslhW3I3/oPAVLl6aBiULwkjn2WUTMR4JGAzfEGHjVrTZcnWtLyNvY40 8FYK8murZ+utm/DsiqcNJ/l9+NlTfdrCn1Rg2vWYxRpSfUqfgWp0qhQFinNFYTUPkN429f RnTFOzdwDk5p7ay9xYo9WBUcew3SEj4= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1674051058; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Sr1/LUQHwubaC/6+yigReObSaZRNqB3QQDgWoJAi3TM=; b=KoyTo1Jhif8HVOUxFGPqz1VlsQ1Iwj2S0//ZA8S+Qdx2k29XguUWqQ8x7wLd4AiUqsTCOQ /MR8XybAxnUrp7Mtt4P+6ytRr0VC23YITKZqfzD97qoYDtsD6wSUckrt8h8CFFIRC+25ha mRgse0/l24Cl9U+9ipD3BZ8Q5uUH/WI= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-428-VFMwKa7ZPYS1PLKwscAguw-1; Wed, 18 Jan 2023 09:10:56 -0500 X-MC-Unique: VFMwKa7ZPYS1PLKwscAguw-1 Received: by mail-wm1-f69.google.com with SMTP id r15-20020a05600c35cf00b003d9a14517b2so1278941wmq.2 for ; Wed, 18 Jan 2023 06:10:56 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:in-reply-to:subject:organization:from :references:cc:to:content-language:user-agent:mime-version:date :message-id:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=Sr1/LUQHwubaC/6+yigReObSaZRNqB3QQDgWoJAi3TM=; b=B4itAWAmB9WlN6Ch1IcKoLyeu0wvjPZkxdttvQYEaoZqZRlwMT5UANvV7Sdvjgs6iL c39TfeYSWQjs73YGoGs0U8m5JwAIHlzcnKqL5gm4yxLVvmHAbrNNibmOi8nucNUWnmKH eNUbUPsziEFUg4MjbhAdipHDCF/uaHZ2iMIM9kYU6896PaCibJSqpv/lI3trfNi57SeF WYKYwXUIXtYhDH95OQM0RkSPDAq6vmho8pPcZhzHwHRAlRjXYTSh1aIjXl24YOfaSMUf SwcglxNfgyLOD942KTGDPXIiabrT3MXGeFq1Bb00D2iboJWcLijk1qXM8HwEoqY9OWvw /ewA== X-Gm-Message-State: AFqh2kpS+5pROvPq+qzY/fnl7LgmvAlbtQIN364jMD2ergbALxRqD1xX 8IKoBlvbT+T2yGKaeqt28xDVtbFEzQoresrIBAOjF9O2ISEFAcap+mnbvsszuYFJ3jDJlhZjuCb sq/zQ+NdPadg= X-Received: by 2002:a5d:6350:0:b0:2bc:7d12:7400 with SMTP id b16-20020a5d6350000000b002bc7d127400mr6531111wrw.36.1674051055642; Wed, 18 Jan 2023 06:10:55 -0800 (PST) X-Google-Smtp-Source: AMrXdXsbFV5oKkK70OyPh+biK8wDKD2aSk+ulJZm3rqGI4w5U+Q9tyC10pkOd75ZN+xb/8QbhovsEQ== X-Received: by 2002:a5d:6350:0:b0:2bc:7d12:7400 with SMTP id b16-20020a5d6350000000b002bc7d127400mr6531095wrw.36.1674051055273; Wed, 18 Jan 2023 06:10:55 -0800 (PST) Received: from ?IPV6:2003:cb:c705:800:1a88:f98a:d223:c454? (p200300cbc70508001a88f98ad223c454.dip0.t-ipconnect.de. [2003:cb:c705:800:1a88:f98a:d223:c454]) by smtp.gmail.com with ESMTPSA id c6-20020adffb06000000b002423620d356sm11286393wrr.35.2023.01.18.06.10.54 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 18 Jan 2023 06:10:54 -0800 (PST) Message-ID: <60125c55-9ab7-7531-e124-cb676943fea6@redhat.com> Date: Wed, 18 Jan 2023 15:10:53 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.6.0 To: yang.yang29@zte.com.cn, akpm@linux-foundation.org Cc: imbrenda@linux.ibm.com, jiang.xuexin@zte.com.cn, linux-kernel@vger.kernel.org, linux-mm@kvack.org, ran.xiaokai@zte.com.cn, xu.xin.sc@gmail.com, xu.xin16@zte.com.cn References: <202212300913573751808@zte.com.cn> From: David Hildenbrand Organization: Red Hat Subject: Re: [PATCH v5 2/6] ksm: support unsharing zero pages placed by KSM In-Reply-To: <202212300913573751808@zte.com.cn> X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 60527140031 X-Rspamd-Server: rspam09 X-Rspam-User: X-Stat-Signature: 1n4h3rwb6s3k6z1ginuhhdk1n93t5orj X-HE-Tag: 1674051071-471760 X-HE-Meta: U2FsdGVkX1+Y7GNV9cpKcteSUwrFmBNw6kSNn9jjxeJgu/CvAj32e4f7jMCN4+GOvxUFwXWkwfrwUlz/AiJJX/ZrLqNXkXRTfMcViNmgIBaZ9ulswwMATYFHl32V5mwkmcCFWVY5ksd8BEvqSAZpKubF/oyOLSuDKTcdGyvPF26I9VCC/kMkJjTBnhLZrGwbtc8b9wM5E0qb1kb2qM6zPX3f2BqtKh/FwCwtKwcU98bKZxfqzuxM6yw1mNxIETdzYJ+FVOCdNvzaeQ2E3AWLibxXf2hZFH2yNPNfaQP6JHjeN8t9AFepfeAHkCxL+q5j22Qe2ftouYo4zdUpK1LsjICZ+1w/NcIqszCB9XerGhKTz//rlNiRzvGpvLptIfqs5pXSRMYMLhomgPuZcXrO+HtczbY3XFwRMcR3YlN0Ti02x8YA2g0ev+NV2CkzPaLFNXoTkPu3eFw0llAhFEQ2LohdE0iRZddEHMQt6j0UjtoMyRd8ISakIGJPQNngoTdiwKNu/W6Uv5yfJ1wU31G9icKZl6enJGyAscfOVYwrltdCEVk4uvjIOMGCAswYzs9gRSb8BxM5bwobnCP3TqsE2+riV6zf7GZzdg/haGCexrwde7KCfHPtG9Ag27s8yeS0N+D0pron1LoC2bhS8+qI5X5TtP3deALme7Tm+5+E9K90xe44EUy5Q0mqepfQrwuSdW8ubC4/e5vYB43eKPWjSuMbw5XePxiwCFeXKyFCyV4edePRrtwh3zCFb/3oB42Z606OMYDyPbKS/6WEEqFZIRUFeisF7drPXNDcK7Xc6vw1KkkjLeMTd0Mau14fdnZhRQk2lxTOnZMzltP63+SyP+AYc4q2Y5GQabgg8aPcees04O2My3SE+aruf3w6i6ITDNx12tWgOOyVHJm1DPokdoQTDjUzX/IJxaGJsBNlgyfTxqoVQ3EmLcNpQH3Hvap7m6YG/8ZH1m5sElzhSTx V0QqE3Lm nkjlwtODgL4Q0sXKtjJUWRtl8vuWrLYIkN9X7W79ORnjriDxdiRj9ppdA94Kpu1Vm5ZlbovntVG6C4zuKGN55ucUJrzPHzMpbr7dK39WGTdlrdEnGQXj/xUGEPhKY9zk0jycxJSXdhrELcZbZcyW4n3lq2FjnWJYGKel7a/8JzXjPLVzBBvdFosC8/PnUZL5z/OvIKc1TX4tixwqlIH+/L3G+UDRQk76gUZQLEXMsVyn5FPho57sfhsAn0qgYY+Oo5NV5Kwr7m3rdXo6iP5d8BWMT71+tMMX7O3Mrh/lpZcD0pgoWcLFV5zToMog/7B9/rdiRARb4NHtkGoFmKF2OCCAoxZga8IcJ6Gaz6mK1GViSFkMPs2zDtgU6RBA5QojYIJruaUQ5GLNMyHP1hNPNwQEca+RcIbN3QPoJRO19O6fU4qqh9N5Y0xBbRlZ1TcpPCIr9bI8R87uy9B66LEh06G6Uje84j7U3X52B9uIxD0ae3iFpJGejbohqyAM8llRT25ptGm2gvhIoHR9NRXTjwTs6U6hv4lulESAZzJ/NXna+YjzZaZZ89aQf033PI73bTJi/j7jP4CoRrNfFAGW+x0PQS2Fncq3hrLcUgC86rZjV47U= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 30.12.22 02:13, yang.yang29@zte.com.cn wrote: > From: xu xin > > use_zero_pages may be very useful, not just because of cache colouring > as described in doc, but also because use_zero_pages can accelerate > merging empty pages when there are plenty of empty pages (full of zeros) > as the time of page-by-page comparisons (unstable_tree_search_insert) is > saved. > > But when enabling use_zero_pages, madvise(addr, len, MADV_UNMERGEABLE) and > other ways (like write 2 to /sys/kernel/mm/ksm/run) to trigger unsharing > will *not* actually unshare the shared zeropage as placed by KSM (which is > against the MADV_UNMERGEABLE documentation). As these KSM-placed zero pages > are out of the control of KSM, the related counts of ksm pages don't expose > how many zero pages are placed by KSM (these special zero pages are different > from those initially mapped zero pages, because the zero pages mapped to > MADV_UNMERGEABLE areas are expected to be a complete and unshared page) > > To not blindly unshare all shared zero_pages in applicable VMAs, the patch > introduces a dedicated flag ZERO_PAGE_FLAG to mark the rmap_items of those > shared zero_pages. and guarantee that these rmap_items will be not freed > during the time of zero_pages not being writing, so we can only unshare > the *KSM-placed* zero_pages. > > The patch will not degrade the performance of use_zero_pages as it doesn't > change the way of merging empty pages in use_zero_pages's feature. > > Signed-off-by: xu xin > Reported-by: David Hildenbrand > Cc: Claudio Imbrenda > Cc: Xuexin Jiang > Reviewed-by: Xiaokai Ran > Reviewed-by: Yang Yang [...] > > /* The stable and unstable tree heads */ > static struct rb_root one_stable_tree[1] = { RB_ROOT }; > @@ -420,6 +421,11 @@ static inline bool ksm_test_exit(struct mm_struct *mm) > return atomic_read(&mm->mm_users) == 0; > } > > +enum break_ksm_pmd_entry_return_flag { > + HAVE_KSM_PAGE = 1, > + HAVE_ZERO_PAGE > +}; Why use flags if they both conditions are mutually exclusive? > + > static int break_ksm_pmd_entry(pmd_t *pmd, unsigned long addr, unsigned long next, > struct mm_walk *walk) > { > @@ -427,6 +433,7 @@ static int break_ksm_pmd_entry(pmd_t *pmd, unsigned long addr, unsigned long nex > spinlock_t *ptl; > pte_t *pte; > int ret; > + bool is_zero_page = false; > > if (pmd_leaf(*pmd) || !pmd_present(*pmd)) > return 0; > @@ -434,6 +441,8 @@ static int break_ksm_pmd_entry(pmd_t *pmd, unsigned long addr, unsigned long nex > pte = pte_offset_map_lock(walk->mm, pmd, addr, &ptl); > if (pte_present(*pte)) { > page = vm_normal_page(walk->vma, addr, *pte); > + if (!page) > + is_zero_page = is_zero_pfn(pte_pfn(*pte)); > } else if (!pte_none(*pte)) { > swp_entry_t entry = pte_to_swp_entry(*pte); > > @@ -444,7 +453,14 @@ static int break_ksm_pmd_entry(pmd_t *pmd, unsigned long addr, unsigned long nex > if (is_migration_entry(entry)) > page = pfn_swap_entry_to_page(entry); > } > - ret = page && PageKsm(page); > + > + if (page && PageKsm(page)) > + ret = HAVE_KSM_PAGE; > + else if (is_zero_page) > + ret = HAVE_ZERO_PAGE; > + else > + ret = 0; > + > pte_unmap_unlock(pte, ptl); > return ret; > } > @@ -466,19 +482,22 @@ static const struct mm_walk_ops break_ksm_ops = { > * of the process that owns 'vma'. We also do not want to enforce > * protection keys here anyway. > */ > -static int break_ksm(struct vm_area_struct *vma, unsigned long addr) > +static int break_ksm(struct vm_area_struct *vma, unsigned long addr, > + bool unshare_zero_page) > { > vm_fault_t ret = 0; > > do { > - int ksm_page; > + int walk_result; > > cond_resched(); > - ksm_page = walk_page_range_vma(vma, addr, addr + 1, > + walk_result = walk_page_range_vma(vma, addr, addr + 1, > &break_ksm_ops, NULL); > - if (WARN_ON_ONCE(ksm_page < 0)) > - return ksm_page; > - if (!ksm_page) > + if (WARN_ON_ONCE(walk_result < 0)) > + return walk_result; > + if (!walk_result) > + return 0; > + if (walk_result == HAVE_ZERO_PAGE && !unshare_zero_page) > return 0; > ret = handle_mm_fault(vma, addr, > FAULT_FLAG_UNSHARE | FAULT_FLAG_REMOTE, > @@ -539,7 +558,7 @@ static void break_cow(struct ksm_rmap_item *rmap_item) > mmap_read_lock(mm); > vma = find_mergeable_vma(mm, addr); > if (vma) > - break_ksm(vma, addr); > + break_ksm(vma, addr, false); > mmap_read_unlock(mm); > } > > @@ -764,6 +783,30 @@ static struct page *get_ksm_page(struct ksm_stable_node *stable_node, > return NULL; > } > > +/* > + * Cleaning the rmap_item's ZERO_PAGE_FLAG > + * This function will be called when unshare or writing on zero pages. > + */ > +static inline void clean_rmap_item_zero_flag(struct ksm_rmap_item *rmap_item) > +{ > + if (rmap_item->address & ZERO_PAGE_FLAG) > + rmap_item->address &= PAGE_MASK; > +} > + > +/* Only called when rmap_item is going to be freed */ > +static inline void unshare_zero_pages(struct ksm_rmap_item *rmap_item) > +{ > + struct vm_area_struct *vma; > + > + if (rmap_item->address & ZERO_PAGE_FLAG) { > + vma = vma_lookup(rmap_item->mm, rmap_item->address); > + if (vma && !ksm_test_exit(rmap_item->mm)) > + break_ksm(vma, rmap_item->address, true); > + } > + /* Put at last. */ > + clean_rmap_item_zero_flag(rmap_item); > +} > + > /* > * Removing rmap_item from stable or unstable tree. > * This function will clean the information from the stable/unstable tree. > @@ -824,6 +867,7 @@ static void remove_trailing_rmap_items(struct ksm_rmap_item **rmap_list) > struct ksm_rmap_item *rmap_item = *rmap_list; > *rmap_list = rmap_item->rmap_list; > remove_rmap_item_from_tree(rmap_item); > + unshare_zero_pages(rmap_item); > free_rmap_item(rmap_item); > } > } > @@ -853,7 +897,7 @@ static int unmerge_ksm_pages(struct vm_area_struct *vma, > if (signal_pending(current)) > err = -ERESTARTSYS; > else > - err = break_ksm(vma, addr); > + err = break_ksm(vma, addr, false); > } MADV_UNMERGEABLE -> unmerge_ksm_pages() will never unshare the shared zeropage? I thought the patch description mentions that that is one of the goals? -- Thanks, David / dhildenb