Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 29EFDC4332F for ; Wed, 1 Nov 2023 10:31:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A65FB8D005C; Wed, 1 Nov 2023 06:31:27 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A16A18D0001; Wed, 1 Nov 2023 06:31:27 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 905678D005C; Wed, 1 Nov 2023 06:31:27 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 821438D0001 for ; Wed, 1 Nov 2023 06:31:27 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 50E6D803D8 for ; Wed, 1 Nov 2023 10:31:27 +0000 (UTC) X-FDA: 81409018614.19.96673A1 Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) by imf04.hostedemail.com (Postfix) with ESMTP id A709F40013 for ; Wed, 1 Nov 2023 10:31:24 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=infradead.org header.s=desiato.20200630 header.b=OaeoBP70; dmarc=none; spf=none (imf04.hostedemail.com: domain of peterz@infradead.org has no SPF policy when checking 90.155.92.199) smtp.mailfrom=peterz@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1698834685; a=rsa-sha256; cv=none; b=Q0HGCMYtzQwFC/2mJjxmETJ2UMY9lOP/TVQpjSNSH1B4xpUM1ufH0/Z1+Yr9kqKCYr22jH wKxsZ44E6kYhSxWkLW/Pa2wsJRpUuw0KU6HcUNmBxqzRA7zTyMb950G7hBuHm9eOiiIhe1 DUfB4g1dBaEwLlhaOtrmiIvgFWSXEMc= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=infradead.org header.s=desiato.20200630 header.b=OaeoBP70; dmarc=none; spf=none (imf04.hostedemail.com: domain of peterz@infradead.org has no SPF policy when checking 90.155.92.199) smtp.mailfrom=peterz@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1698834685; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=AdMoMRnJbhIexa+O8wRz6i1mgPwE7z0trzuc+nyuQEM=; b=k8Yue8w6k79E2LVapkuA9T/A3EeDzu1Xd02/oQ5PINhy5yJEiHD75Nrh3N6Ie29IEpmlMc Gl+3uFovwQn3JPiBpML6n7c2dTRV42qLv53xxO1VWJhzeSZb2IPiFbclfKAR7gR2sytLJu 2JGxMec8pnf9uGw/uZsAhh577vK2uLU= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=In-Reply-To:Content-Type:MIME-Version: References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=AdMoMRnJbhIexa+O8wRz6i1mgPwE7z0trzuc+nyuQEM=; b=OaeoBP70Q7nlTJAgaaMOGRrjDW z+4bitIwzxirHaaGd5ByozfhOnj1GAeYtZTy9N1e9rr4b/KT3AqQD2LjErjc7w7ed+z182W7QWHb6 GWQtLP+3k44ZkDueVccvTJ/uTTU1ipfWtz//sgC9DmFFCuYtGOxwQmRPqAHYRCWXtvmevxvXhzRxX zEbkMi0Z1NCtAymKmkhBcqoBmx7g/zb39F+CksTsNQ6Lmlgl23WbqW5HfOpvf/DsJMcEcB9tE92GG I+/Tg4TGAxyRu8CGLrHjf+iVQoBEbpQn+YrK4vx+EGBgiDxTmqHB81QrYRHAfYEDDcAkWR6JnH5xX mhZcUOuQ==; Received: from j130084.upc-j.chello.nl ([24.132.130.84] helo=noisy.programming.kicks-ass.net) by desiato.infradead.org with esmtpsa (Exim 4.96 #2 (Red Hat Linux)) id 1qy8VG-005dj6-2F; Wed, 01 Nov 2023 10:31:14 +0000 Received: by noisy.programming.kicks-ass.net (Postfix, from userid 1000) id 5A59B3002AF; Wed, 1 Nov 2023 11:31:14 +0100 (CET) Date: Wed, 1 Nov 2023 11:31:14 +0100 From: Peter Zijlstra To: Mel Gorman Cc: Raghavendra K T , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Ingo Molnar , Andrew Morton , David Hildenbrand , rppt@kernel.org, Juri Lelli , Vincent Guittot , Bharata B Rao , Aithal Srikanth , kernel test robot , Sapkal Swapnil , K Prateek Nayak Subject: Re: [PATCH V1 1/1] sched/numa: Fix mm numa_scan_seq based unconditional scan Message-ID: <20231101103114.GK15024@noisy.programming.kicks-ass.net> References: <2ea7cbce80ac7c62e90cbfb9653a7972f902439f.1697816692.git.raghavendra.kt@amd.com> <20231101092101.i562axvfqjdvw2rx@suse.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20231101092101.i562axvfqjdvw2rx@suse.de> X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: A709F40013 X-Stat-Signature: gemzijdnzfqzamn1t5xy7frm867fojb1 X-HE-Tag: 1698834684-807707 X-HE-Meta: U2FsdGVkX1+5fxbTzRtgR21NQ/XOgZRHD9kRsVwIGq5a4VAbpDyCIJScNHXZ1Eq2x/lfANA2yHzx+WahZ+F4//HvEvOJoUtnlShF8sN91KkMaPQLkSLT+f0FUK2ajaEoZP0/RpymJHsByXy5ZJPcLUtgkmAQQVWLD4OlnIliDHborGJDb1btZlt4ZhC+VDTZyU25MtuRLf+PKJVJfF8+aKEjmBb8DoPXTvjel5ykNfrLRV8zXit8p7yi5wwDRNppU0zbc2LbCiH/JtbONuq9MZsCkbnaqoseNtHmxda4YSI6fJLUPwXA7941Xyznr0aP20jlbw/rB45lfJjtD47o6ciM8Qi5/b7woLEHBTsqWqcy0CJyLlwN+df1h21ZwKxmo0i02e+eQ+84xXS6/uSvWNxrRGg+Ec8OAoDcRWkyYX+MkXFxPZP4wdCRwb7K6G0Q+XRgvLuYHzWlqyFdmefSNjlb0zJoqlTRTpHu2Wiy1rNo5aNkWeR14zVazvP42lZSf+SgZCRtT+t4my4PLjk1MvZPuTzlY8Kb8FLPLPz2aikiOQdeUnvHbv0TkcyPnlM3F+ASMNSTs5Rj1lY21ib3bqYop3fsmO9moCxH12u+hVnEJ1PQ5ga7LdFbRCZlD5rRsuwSFbYUuSpLR/uodsoHpdWNjTXaI0WyT2mvhx71xDhqjP4/EoVrsY1OHcc+Spkib8mDqMLgc+INbPykvUxPQq+mjMYWPkc0FGwS3KRVH0aJRmCjDrTulrlHuKuOOPNHvsclhQYgh+01M9jGD61Pjk93hA3FzTL3Ia+F4b4GzIR3BKTxkz3qM6+oGbf1sP8DLKwaoIRXtnQst9EkRhWzA73Vrqnzea0+Q0cn4MTtJ4VevTmo+nPh24PiWP8GLEwNpz2pfztN9FMIbnDmeaCXbsdqRxupp5qa9m+Q8O3MwQmhR32ah0OxcsSlENLzrS6MdQzMdyRZNTIsDe2RTcg TZcT05wm so8V+Nxlb3rWXJE00dutvD8Zwrh5hB7zEDmoAar86J5+4TBaNERdxkJ2XWqrlKInBugBTJmQIl5YQmk4ifS+dDwLLkfJrbWyA9mAa686tbOsjLqoTAxKMZ8CwE7/PXPC7P3nU2WGJ2r9Nxc2V7GWkkwvgjyJu0tvzCHylRCF2LVAhF+V9f5alrvnRhkOSQe3AdsTa2xYH2KupEe76z6915wxyvnyWx0b6TPdcs36jktzzcK4OWSDY6b9Niktbxq3oKm4caKIgdMCTYGUF/zn+TgPSiKwaa/rRF+b7 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, Nov 01, 2023 at 09:21:01AM +0000, Mel Gorman wrote: > On Fri, Oct 20, 2023 at 09:27:46PM +0530, Raghavendra K T wrote: > > Since commit fc137c0ddab2 ("sched/numa: enhance vma scanning logic") > > > > NUMA Balancing allows updating PTEs to trap NUMA hinting faults if the > > task had previously accessed VMA. However unconditional scan of VMAs are > > allowed during initial phase of VMA creation until process's > > mm numa_scan_seq reaches 2 even though current task had not accessed VMA. > > > > Rationale: > > - Without initial scan subsequent PTE update may never happen. > > - Give fair opportunity to all the VMAs to be scanned and subsequently > > understand the access pattern of all the VMAs. > > > > But it has a corner case where, if a VMA is created after some time, > > process's mm numa_scan_seq could be already greater than 2. > > > > For e.g., values of mm numa_scan_seq when VMAs are created by running > > mmtest autonuma benchmark briefly looks like: > > start_seq=0 : 459 > > start_seq=2 : 138 > > start_seq=3 : 144 > > start_seq=4 : 8 > > start_seq=8 : 1 > > start_seq=9 : 1 > > This results in no unconditional PTE updates for those VMAs created after > > some time. > > > > Fix: > > - Note down the initial value of mm numa_scan_seq in per VMA start_seq. > > - Allow unconditional scan till start_seq + 2. > > > > Result: > > SUT: AMD EPYC Milan with 2 NUMA nodes 256 cpus. > > base kernel: upstream 6.6-rc6 with Mels patches [1] applied. > > > > kernbench > > ========== base patched %gain > > Amean elsp-128 165.09 ( 0.00%) 164.78 * 0.19%* > > > > Duration User 41404.28 41375.08 > > Duration System 9862.22 9768.48 > > Duration Elapsed 519.87 518.72 > > > > Ops NUMA PTE updates 1041416.00 831536.00 > > Ops NUMA hint faults 263296.00 220966.00 > > Ops NUMA pages migrated 258021.00 212769.00 > > Ops AutoNUMA cost 1328.67 1114.69 > > > > autonumabench > > > > NUMA01_THREADLOCAL > > ================== > > Amean elsp-NUMA01_THREADLOCAL 81.79 (0.00%) 67.74 * 17.18%* > > > > Duration User 54832.73 47379.67 > > Duration System 75.00 185.75 > > Duration Elapsed 576.72 476.09 > > > > Ops NUMA PTE updates 394429.00 11121044.00 > > Ops NUMA hint faults 1001.00 8906404.00 > > Ops NUMA pages migrated 288.00 2998694.00 > > Ops AutoNUMA cost 7.77 44666.84 > > > > Link: https://lore.kernel.org/linux-mm/ZSXF3AFZgIld1meX@gmail.com/T/ > > > > Signed-off-by: Raghavendra K T > > Acked-by: Mel Gorman Thanks, will queue for the next merge window (6.8 I think that is) once 6.7-rc1 comes around.