Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 55F46C4332F for ; Wed, 23 Nov 2022 16:30:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BB1376B0071; Wed, 23 Nov 2022 11:30:22 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B3A006B0073; Wed, 23 Nov 2022 11:30:22 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9DB1C6B0074; Wed, 23 Nov 2022 11:30:22 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 8A44B6B0071 for ; Wed, 23 Nov 2022 11:30:22 -0500 (EST) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 0F491AB7BB for ; Wed, 23 Nov 2022 16:30:22 +0000 (UTC) X-FDA: 80165244684.25.9519940 Received: from mail-qk1-f173.google.com (mail-qk1-f173.google.com [209.85.222.173]) by imf23.hostedemail.com (Postfix) with ESMTP id 602DD140014 for ; Wed, 23 Nov 2022 16:30:20 +0000 (UTC) Received: by mail-qk1-f173.google.com with SMTP id z1so12777915qkl.9 for ; Wed, 23 Nov 2022 08:30:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20210112.gappssmtp.com; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=vt2TI2s/ONwTSTFJrVl4zFA/mn26DOjDlB7TgOSLANM=; b=gelHH9WcdQSeyZW2TipTpvkwMV0+VGmqtodQTG4dG4qWCEMZ0y6SfoZMmBl3+zrMtA G363HAcjLxF8bBafLmHrj+o5EFaSR8ugndv1YzyCOG9h8h/aZutl+9Iu1KkfIicqCeTw 3ibKaXSzagLIYS7cRG0gYCgFmLar08h/dl/q9xNKz0eCjblvOjI4NwN0JvWBdTyMEyss 0sti78XhZOdFhaBE4JGus6TOalVsifJT0qvMUmOMqAQhxGYsLzOlJGV8RJ//PWOgID+8 BaafvpBE9mQl4zLXwbaM9JVOIHFSR5iWnBrk4RN+bEimvHWnuAP58PfpjMUBqOrbZ/dj 6VwA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=vt2TI2s/ONwTSTFJrVl4zFA/mn26DOjDlB7TgOSLANM=; b=J/upCXanKp8Np/e6FG3y/YeFGjiVYVVpRFSUQlic1caiTm2ezHwsU3yMmmygJ2pQ2v tBy5CCx6zFcBJ179cPcVAc7ZQnRXUiCXMnD/FRI14E5YXpWeO8opBO2aR2sQG7hEXlTs b99e2GCqfuiSBGaRRRpTwWxioelEZ3dRih1Bmmdzqty7NMJLPhKWK3NgnmVKNW6IsTce /rP7VJJA+muZzgPsURDbquH/sWV/ajpRdeWihGzeea6fj4vfuoqoHO+/xbKeZhx+fHd/ G/cmyPCm9rGHOG4pYGsiVLjFz6vv26THYqYx4Y0QIuFn6k7Ry4PZbU1BPvqfSEI6JFyj yXCA== X-Gm-Message-State: ANoB5pkUSyVHWWaIf4W5FrqmEKi0Wa5yDerMiyYV+uRAyPk5k3pWgKC5 XE7mBOSzzDCUs739EZFOa/fljg== X-Google-Smtp-Source: AA0mqf62dokmczKsgWdcoJf2uWNfAwNh6i6+iKFz76k1kXUxub6M2QE9vG8qJF2eIjihESqQ9ABfSA== X-Received: by 2002:a37:aa81:0:b0:6fa:12a9:b327 with SMTP id t123-20020a37aa81000000b006fa12a9b327mr8788431qke.149.1669221019413; Wed, 23 Nov 2022 08:30:19 -0800 (PST) Received: from localhost ([2620:10d:c091:480::1:bc4]) by smtp.gmail.com with ESMTPSA id g10-20020ac8124a000000b00399b73d06f0sm9894595qtj.38.2022.11.23.08.30.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Nov 2022 08:30:19 -0800 (PST) Date: Wed, 23 Nov 2022 11:30:45 -0500 From: Johannes Weiner To: Yosry Ahmed Cc: Sergey Senozhatsky , Nhat Pham , akpm@linux-foundation.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, minchan@kernel.org, ngupta@vflare.org, sjenning@redhat.com, ddstreet@ieee.org, vitaly.wool@konsulko.com Subject: Re: [PATCH v6 4/6] zsmalloc: Add a LRU to zs_pool to keep track of zspages in LRU order Message-ID: References: <20221119001536.2086599-1-nphamcs@gmail.com> <20221119001536.2086599-5-nphamcs@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1669221020; a=rsa-sha256; cv=none; b=fk2vslRymxPUaOvWS8XJ+EgIpExz7Ebz25ak/+FPQuaHlNPERDKK90sCnh13GAY35wI+tH emJ8bQkpSjPHHV43XVVThnZ1/TZ6Pj62QDvJiaucgnbTklMnGZDdZm04ThKHhf6xH6T//Y i99hLQyoytC1UdlJcqUbNdMZ624y13U= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=cmpxchg-org.20210112.gappssmtp.com header.s=20210112 header.b=gelHH9Wc; dmarc=pass (policy=none) header.from=cmpxchg.org; spf=pass (imf23.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.222.173 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1669221020; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=vt2TI2s/ONwTSTFJrVl4zFA/mn26DOjDlB7TgOSLANM=; b=0KLg007hy9ZlUvmocqwQwsUXq2miP/zOi+J3lCw/9yK/FidQC2i81Gy9vnYLlBcRoWLmPa o1LZpTFs/51YjSvEv6R2IpimuwJvDZVh1iEfowl1H+jWxcRT6IQk2LCxXkAWmpoC8JuF5B vvcqw6r0sgwGPE4KLT1C+UDm8+P2Pxs= Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=cmpxchg-org.20210112.gappssmtp.com header.s=20210112 header.b=gelHH9Wc; dmarc=pass (policy=none) header.from=cmpxchg.org; spf=pass (imf23.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.222.173 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org X-Stat-Signature: juxq1gxebz863w4xi5ydc198csnyifkt X-Rspamd-Queue-Id: 602DD140014 X-Rspam-User: X-Rspamd-Server: rspam11 X-HE-Tag: 1669221020-357793 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Nov 23, 2022 at 12:11:24AM -0800, Yosry Ahmed wrote: > On Wed, Nov 23, 2022 at 12:02 AM Yosry Ahmed wrote: > > On Tue, Nov 22, 2022 at 7:50 PM Sergey Senozhatsky > > > There are no accesses to swapped out pages yes, but zspage holds multiple > > > objects, which are compressed swapped out pages in this particular case. > > > For example, zspage in class size 176 (bytes) can hold 93 objects per-zspage, > > > that is 93 compressed swapped out pages. Consider ZS_FULL zspages which > > > is at the tail of the LRU list. Suppose that we page-faulted 20 times and > > > read 20 objects from that zspage, IOW zspage has been in use 20 times very > > > recently, while writeback still considers it to be "not-used" and will > > > evict it. > > > > > > So if this works for you then I'm fine. But we probably, like you suggested, > > > can document a couple of things here - namely why WRITE access to zspage > > > counts as "zspage is in use" but READ access to the same zspage does not > > > count as "zspage is in use". > Nonetheless, the fact that we refaulted an object in a zspage does not > necessarily mean that other objects on the same are hotter than > objects in other zspages IIUC. Yes. On allocation, we know that there is at least one hot object in the page. On refault, the connection between objects in a page is weak. And it's weaker on zsmalloc than with other backends due to the many size classes making temporal grouping less likely. So I think you're quite right, Segey, that a per-class LRU would be more accurate. It's no-LRU < zspage-LRU < class-LRU < object-LRU. Like Yosry said, the plan is to implement an object-LRU next as part of the generalized LRU for zsmalloc, zbud and z3fold. For now, the zspage LRU is an improvement to no-LRU. Our production experiments confirmed that.