Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6F2CAC7EE23 for ; Wed, 7 Jun 2023 12:24:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AAF768E0001; Wed, 7 Jun 2023 08:24:55 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A601A6B0072; Wed, 7 Jun 2023 08:24:55 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 927268E0001; Wed, 7 Jun 2023 08:24:55 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 801216B0071 for ; Wed, 7 Jun 2023 08:24:55 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 4FFB91A02A8 for ; Wed, 7 Jun 2023 12:24:55 +0000 (UTC) X-FDA: 80875870950.20.376A539 Received: from outbound-smtp04.blacknight.com (outbound-smtp04.blacknight.com [81.17.249.35]) by imf15.hostedemail.com (Postfix) with ESMTP id 70799A000A for ; Wed, 7 Jun 2023 12:24:52 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf15.hostedemail.com: domain of mgorman@techsingularity.net designates 81.17.249.35 as permitted sender) smtp.mailfrom=mgorman@techsingularity.net ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686140692; a=rsa-sha256; cv=none; b=wrKb/EaeCiKhIZeJC9DDo7UF4+wZ5UN1N5ODtxW3v7HMeiNa4HsQ8+LFbCfsn2BU4kLtcg Tqvr27u8/vWkTcRLjL62r5tpAqCHXGLvJw7rE5tnTcyL+poQabmxiJpwnzDFGEo3xkziLa ysKVE3CH1xnFAPf0U2s22IrfayxOz40= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf15.hostedemail.com: domain of mgorman@techsingularity.net designates 81.17.249.35 as permitted sender) smtp.mailfrom=mgorman@techsingularity.net ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686140692; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=M+Ywc4POHn3/io2A04g9yIxAt0ysWkDlhsZ6aENIgSo=; b=IPOWp5WCFF8uCHCfswLEQ6tkmpXcm6XjTwj4/WvLPFB9zsHrfwG2mz8BnkkHKe13NL448w sDWQ5LP9TPSGahR0ITEZiNgakVssy2570k9CAvci+2Gxnr4uF6zvyumq+ACSdvMKpzrCpN +sG6YRTNq1b1g0bindLG9DNs6hy7MDM= Received: from mail.blacknight.com (pemlinmail04.blacknight.ie [81.17.254.17]) by outbound-smtp04.blacknight.com (Postfix) with ESMTPS id 77631BECD6 for ; Wed, 7 Jun 2023 13:24:50 +0100 (IST) Received: (qmail 5691 invoked from network); 7 Jun 2023 12:24:50 -0000 Received: from unknown (HELO techsingularity.net) (mgorman@techsingularity.net@[84.203.21.103]) by 81.17.254.9 with ESMTPSA (AES256-SHA encrypted, authenticated); 7 Jun 2023 12:24:50 -0000 Date: Wed, 7 Jun 2023 13:24:48 +0100 From: Mel Gorman To: Vlastimil Babka Cc: Andrew Morton , Jiri Slaby , Maxim Levitsky , Michal Hocko , Pedro Falcato , Paolo Bonzini , Chuyi Zhou , Linux-MM , LKML Subject: Re: [PATCH 3/4] mm: compaction: Update pageblock skip when first migration candidate is not at the start Message-ID: <20230607122448.nvyxtviyuawk3rfy@techsingularity.net> References: <20230515113344.6869-1-mgorman@techsingularity.net> <20230515113344.6869-4-mgorman@techsingularity.net> <20230529103342.esek6r5fvmft2nky@techsingularity.net> <6695b7e5-9fa5-fae8-8a66-cc5985b0baaf@suse.cz> <20230602111622.swtxhn6lu2qwgrwq@techsingularity.net> <152e0730-0ddc-a1f8-7122-275d51741a1d@suse.cz> <20230602124825.24a775kwwuf4rs6v@techsingularity.net> <2c802986-3726-f79c-6383-cc03adb9fb0c@suse.cz> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-15 Content-Disposition: inline In-Reply-To: <2c802986-3726-f79c-6383-cc03adb9fb0c@suse.cz> X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 70799A000A X-Stat-Signature: m675a37gmu3b7smtz7y9adoes9a65agx X-Rspam-User: X-HE-Tag: 1686140692-47986 X-HE-Meta: U2FsdGVkX1+5v57+zNaulv0P22xI7I5uN1iXQ4IPVl6tnriqeRc67Js0xPBeXzYPNVA2Am/rE1Zwn5BYVz6TBk+UngzVjeXQ56DflbzOEqpQI7HlVDuAn+R9pNfQ4A3qsjPFO8KgKB31FQ0CIu0ZaCq2lswB0OPbu5pGnMsjO8VgMi3a9hvmtTJwhW/i1L2bdYP32fo68dME93jcUvQaQn23XkDwd9NiWFqTscvTZvQQXzJkUH9AsC6YA1misgptKAUaZLrmj1hBkBmxUd5Y7yGdHQdbgSey/SbUBLLMfVIaePdeQXH6u4veeKqA6pZ3VpyB47iMGags01v+J+lScGrPvcqnQ47XJWKMAvv/D0FGbNOEXGdNvYgmQPvJWq9qVtOYVbYMS4eamqFCX6ux33MQzSacgVKpZVUJ0/DNXsUgJbIaVQlv1AOuDIUDXxEsFj0Yjr59zOhzFQpWpiwPavCwJP9Sy9yn3nTOv8DhxTIWh+fsOTG8kelcnOUkAljdwKFmdtThCOLRxFy24FfbQn6/gQKDahs27+Bd9ivD9H0fvBWD7WPKRma25XsgwSX/evgHR01Ub0WaXuBE06LMrrGUB9BP/WPlCktsZY6Sqy0zAdjhzIz7ABycPjPN7mLh4I1FBAeye4DMgLl6nbbBiZzRbSaUh/bN9sgwN8efUGVSXSXPehvEwFlKVmNHhS0oQjDgTbMxjECvKbUMOwoFxewDP+3CrdIpeauZ5RDOPyO7gUtnL8FU6IxyEqVg1DE3KUKD8sr0cb1LusD8a4EbOn/PoWjoH1r60SLlRxMHE6oQBgJPPsy9Z96UyAL68KAWo7VgBEHXJw8tphCYfJlOgmY0mAxp7WIAtsMr8u6O4DJXS9lC8R03przyUj9JQE2UeqsdO+9m7QF5Hv/X5eJj8l4xrRUypEVGOj4ns9kGJtMIdwL8dvLGEngZtBSDgkQUtGNG4TjMmdWKJg1hH0E H4/ZzMVD HZTdcZpg5hGLIWERilxAWRVKeeuFOOBuBcwYDym1HbvJjohWXww9Dz8OYLZQltcagM3clcXXrSA6PUbqvzMtRQQ4wxO2eD/ewXgPQn2ry1770gjPBh/zO+qW30GBww57OlaQi40ucQJF1XwYmXVymZxANmzTiru8g5yR3utjlkmuF1SptMBtVcen2PFxf6N1wqeZfmAQ1TYN2r9bK7+TKTSLE6pQ+tltcLxrbuerDlXYuqmBLerJAVbngl7qS+VW2xgOy4kFSkjRJS+3U04zgk6p9be/AG/XXMBqBog96k6gHm1E= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Jun 06, 2023 at 03:11:27PM +0200, Vlastimil Babka wrote: > On 6/2/23 14:48, Mel Gorman wrote: > > On Fri, Jun 02, 2023 at 02:19:00PM +0200, Vlastimil Babka wrote: > >> On 6/2/23 13:16, Mel Gorman wrote: > >> > On Mon, May 29, 2023 at 02:43:48PM +0200, Vlastimil Babka wrote: > >> >> On 5/29/23 12:33, Mel Gorman wrote: > >> >> > On Thu, May 25, 2023 at 03:37:43PM +0200, Vlastimil Babka wrote: > >> >> >> On 5/15/23 13:33, Mel Gorman wrote: > >> >> >> > isolate_migratepages_block should mark a pageblock as skip if scanning > >> >> >> > started on an aligned pageblock boundary but it only updates the skip > >> >> >> > flag if the first migration candidate is also aligned. Tracing during > >> >> >> > a compaction stress load (mmtests: workload-usemem-stress-numa-compact) > >> >> >> > that many pageblocks are not marked skip causing excessive scanning of > >> >> >> > blocks that had been recently checked. Update pageblock skip based on > >> >> >> > "valid_page" which is set if scanning started on a pageblock boundary. > >> >> >> > > >> >> >> > Signed-off-by: Mel Gorman > >> >> >> > >> >> >> I wonder if this has an unintended side-effect that if we resume > >> >> >> isolate_migratepages_block() of a partially compacted pageblock to finish > >> >> >> it, test_and_set_skip() will now tell us to abort, because we already set > >> >> >> the skip bit in the previous call. This would include the > >> >> >> cc->finish_pageblock rescan cases. > >> >> >> > >> >> >> So unless I miss something that already prevents that, I agree we should not > >> >> >> tie setting the skip bit to pageblock_aligned(pfn), but maybe if we are not > >> >> >> pageblock aligned, we should ignore the already-set skip bit, as it was most > >> >> >> likely being set by us in the previous iteration and should not prevent us > >> >> >> from finishing the pageblock? > >> >> >> > >> >> > > >> >> > Hmm, I think you're right. While it should not hit the original bug, > >> >> > migration candidates are missed until the next compaction scan which > >> >> > could be tricky to detect. Something like this as a separate patch? > >> >> > Build tested only but the intent is for an unaligned start to set the skip > >> >> > bet if already unset but otherwise complete the scan. Like earlier fixes, > >> >> > this might overscan some pageblocks in a given context but we are probably > >> >> > hitting the limits on how compaction can run efficiently in the current > >> >> > scheme without causing other side-effects :( > >> >> > >> >> Yeah that should work! I think it should be even folded to 3/4 but if you > >> >> want separate, fine too. > >> >> > >> > > >> > I was not happy with the test results so limited the scope of the patch > >> > which performed much better both in terms of absolute performance and > >> > compaction activity. > >> > >> That's surprising. Does that mean that if we isolate COMPACT_CLUSTER_MAX > >> pages, migrate them without failing, but it's not enough to succeed (i.e. > >> there are more pages we need to migrate to free up a whole pageblock), it's > >> better to give up on the rest of the pageblock rather than continue? > > > > I don't have precise enough data to answer that with certainty but probably > > yes, at least in terms of scan activity. The first version had spikes of > > pages scanned for migration that are not always reproducible and not on > > all machines. > > Well, that kinda sucks. So for the patch (with adding the missing NOT below). > > Acked-by: Vlastimil Babka > > But in raises a question whether we should terminate compaction under the > right conditions after a successful migration immediately, rather than > invoke another iteration of isolate_migratepages_block() where we could skip > over some pages uselessly only to abort at first valid page due to the skip bit. > It would save some cycles and be much more obvious than now, where anyone > trying to understand how it works in detail might conclude it's an oversight? > It sounds like a solid idea and would be a good standalone patch with the usual supporting data. At a quick glance, the check for a page stored on compact_control happens surprisingly late which makes me think we probably over-compact in direct compaction in particular. It would need supporting data because it probably means that compaction cost gets spread over multiple tasks requiring high-order pages instead of one unlucky task doing compaction works that unrelated tasks indirectly benefit from. It's probably more sensible behaviour that tasks requiring high-order pages pay the cost if kcompactd cannot keep up but supporting data would tell us one way or the other. -- Mel Gorman SUSE Labs