Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 59C0BC83038 for ; Tue, 1 Jul 2025 13:23:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F38ED6B00A7; Tue, 1 Jul 2025 09:23:54 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id F10D46B00AD; Tue, 1 Jul 2025 09:23:54 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DFF196B00AE; Tue, 1 Jul 2025 09:23:54 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id C94F96B00A7 for ; Tue, 1 Jul 2025 09:23:54 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 6CEC45918D for ; Tue, 1 Jul 2025 13:23:53 +0000 (UTC) X-FDA: 83615763546.26.13FC403 Received: from techbitestudio.com (techbitestudio.com [75.119.147.106]) by imf02.hostedemail.com (Postfix) with ESMTP id 9281980008 for ; Tue, 1 Jul 2025 13:23:51 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=kenip.in header.s=mail header.b=KaN8hsF1; dmarc=pass (policy=none) header.from=kenip.in; spf=pass (imf02.hostedemail.com: domain of siddhartha@kenip.in designates 75.119.147.106 as permitted sender) smtp.mailfrom=siddhartha@kenip.in ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1751376231; a=rsa-sha256; cv=none; b=f4uQ2lrL2mulyOk+SgBcj8/uhQquve1H+ny6JPgYpltcSarC3T8wDBtgMyO+j29IefVnuG Fk88MB6Z17mTXoIPNKzDslmG3Zw0LgX0jBoU/Bo+JGK5vvEgq2OvtjQHfO2puvSSa0slH/ ENgSqCkmh0LvIWWCrM6X2rsgH5nGxrs= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=kenip.in header.s=mail header.b=KaN8hsF1; dmarc=pass (policy=none) header.from=kenip.in; spf=pass (imf02.hostedemail.com: domain of siddhartha@kenip.in designates 75.119.147.106 as permitted sender) smtp.mailfrom=siddhartha@kenip.in ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1751376231; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=WYc6BWDDB6pjMcrs1k9Wn+s7y4QT7W16n4fHbowUlO8=; b=fnlNA5HLaCT8e4NjXtIbiU6HshTPKTpKsivbma4ZoMm3cICsF55Q4bLQzFdD/qKRIk1Lbt KX5pOUB/+VCucUh0EwvBGcamozfqFIQfWBdATYBjkchwZ+y+mXVzTjmvTm3HUR+UPccGSA UrUZnbZs3sQ88WIQcYuZ4BTKs915e5M= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=kenip.in; s=mail; h=Content-Transfer-Encoding:Content-Type:Message-ID:References: In-Reply-To:Subject:Cc:To:From:Date:MIME-Version:Sender:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=WYc6BWDDB6pjMcrs1k9Wn+s7y4QT7W16n4fHbowUlO8=; b=KaN8hsF1Io0Buz77jFukgLk0vJ z+8u7x8xSAebY2Uv4qCztDZG71bezDxWmw6OVB8lizdyktcCtBEkOuGVP2eKu2dGbMthknfQaIady xUe3Kf39jD5GkcKBXbLSDP4fGeInKBuAHcDQs17uRWrYvDFw9BZbmXkvPosTXUyPRbQw=; Received: from localhost ([127.0.0.1] helo=kenip.in) by techbitestudio.com with esmtpa (Exim 4.93) (envelope-from ) id 1uWaxf-00075f-NT; Tue, 01 Jul 2025 18:53:47 +0530 MIME-Version: 1.0 Date: Tue, 01 Jul 2025 18:53:47 +0530 From: siddhartha@kenip.in To: Lorenzo Stoakes Cc: Dev Jain , linux-mm@kvack.org, linux-kernel@vger.kernel.org, mgorman@suse.de Subject: =?UTF-8?Q?Re=3A_=5BPATCH=5D_mm=3A_limit_THP_alignment_=E2=80=93_?= =?UTF-8?Q?performance_gain_observed_in_AI_inference_workloads?= In-Reply-To: <5c3d307f-d303-48c3-b730-99a83d4815ec@lucifer.local> References: <5816677a-705e-4a8f-b598-d74ff6198a02@arm.com> <80b849d4-faf3-47a9-8b8c-e8053299cfb2@arm.com> <2e99712b-8dac-4762-9fc5-fe3ef569b65e@lucifer.local> <787639a1e6a27c0f3b0e3ae658e1b8e7@kenip.in> <5c3d307f-d303-48c3-b730-99a83d4815ec@lucifer.local> Message-ID: X-Sender: siddhartha@kenip.in X-Priority: 1 (Highest) Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 9281980008 X-Stat-Signature: 69ygriqgdxws9noyis13pwodtbaa83zc X-Rspam-User: X-HE-Tag: 1751376231-737314 X-HE-Meta: U2FsdGVkX1+0xC8/Q6rRJv8wUx8qMtGKDpqdDkhkPkNYX0AS6Bkdg1aXlPXtg+QYhy8M7weHdEVGkTGXA1B4ciB+nSHtaIcIj8viGITQcQm+cI2CE3cbq8VnRn0+VDmA5nmlLRDzu66C+yYwgcP7lzzSNlYt1vnTyf9JnzXUBsy4pUEALCQQHDhWpsRqYGaPxR5fQkWK/rtdTApLBwgTmzl+qbOBJipHy4o4URlAp0bArbYX514FhDHMKWQSd1NZ6YSdwNYXKxsyJJRigw9gZzxm40a6xnFpIaFzOEBlqjBxPHvUG+yt4cKXv8bAu7WywmBNPR2DO3nW/Z3YeUmMPBx6NEMorhDzV6g9i4IG4yYhWJpMWJwRjPoBJWRBZ6to4+8wo52njI26OiHBF6vLyRxDNRApfg+CTs27lLMVzXZv5Zm+TjrDotu9P3eukeenVxzHw+1d29KF3yqxWWWaEZA49v+h00kpFZuytJohVr8DGryBYMQIJngUPeAa9jl28523nKPkopR7Qj4Tbcuz7jS76UvaehCoD4A7ixDDogAL75P0kP9I8dtEZ7QzInz39fnnpDmsjyfrKC537JXl+HgopPxt80gkNqo43cT1kz2L0XcmO8NRP28D4FdAhQ9XI0fsjpCSxlBGBEFJ8DwBPx0NmEeZLSohenEq2UxNrG8ZHY4qNUayhyy7oxEfBED8IeEJge557IfS1BL6PVkrum2aGVHKMyEifQ8BpI38hMtmFbxi9roWwiI4jxwS1CPByEOni0CchRvvUlbkirt22JxE/+FSlyw5ThUf02S7nQSzcLe8lrhdVzO+KcGgi8An5Fe22rjoCz2kxJdc5VmapnHPdHEDevJfuLVAwjGQk4lGw+DIn07mvLRDd82PXJdg0fDb4cBMYWiZ5mcmPELGXvvrCzA2+56KDGskGfpX4waTtYbGBBTw8MkkMDVlAOGS1tHhykLlhEa/VK4m0uP aafnf4US LiAm9Rf89tKF/2JL//n7mxEPexBcbI3aLPoIowEPKB83TQ6nltOYTrqfDukxQgz59vZpEyZ3JD7W4t+U2VFJc7URocPmjMSNaNmUcOu7H2CQ5Dz9L6TWRIGUZoIfayzqhvkVCkqSnB3JQTMVsZ+NDD8pI09MyMoLToAUIf/hNy7U1o1Fiz3PEfVwJgulcn/kBTMXS1Z6BooCoMXXVRTKhTIOPjF/NMNblwWL4yyZNtJcn7+iEI63Moe4fltqWDKl4NoaY0jS7PAcw21DWSk1v7rVVC7ieJqPEKgMX7roG9VtZsjRQ/T9YcQ/q4PNFeEMFm7nXVQwaxyZe4XKNf0ASfMN7aaRRDlhHqZDRqgPKGkwTTS/I+ngLCR/fzPiwKREupvpuOxH2bBEjr9w9YbETRAtaMxjkqQQWW8YvbbgzZQ5BcJY6hzc6w6NWjW6tyg5ufUYeEfS1D/6Tkw4= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 2025-07-01 18:09, Lorenzo Stoakes wrote: > On Tue, Jul 01, 2025 at 05:45:51PM +0530, siddhartha@kenip.in wrote: >> 🧩 1. Does the patch cause VMAs to be merged eventually? >> You're correct: VMA merging only happens at mmap() time (via >> __mmap_region()). What the patch affects is the behavior of >> thp_get_unmapped_area_vmflags() before the mmap is placed. > > [...] > >> >> 📐 2. Why aren’t the VMAs mergeable before the patch? >> Great question. Even if the VMA flags are identical, gaps introduced >> by >> forced alignment from get_unmapped_area() break the precondition for >> merging: > > [...] > >> 💡 4. Why this patch complements Rik’s rather than contradicts it: > > I'm really perplexed as to why you felt the need to (seemingly via LLM) > reply with the explanation I've already provided here?... > > There's errors in things you say here too. > > With respect, please don't do this. > > (I'm the co-maintainer of pretty much all the relevant code here and > wrote > the VMA merge logic you're referring to.) > >> >> 🤖 3. How does this impact AI workloads like Hugging Face Transformers? >> Tokenization and dynamic batching create non-deterministic memory >> allocation >> patterns: >> >> Models like BERT and T5 dynamically allocate intermediate buffers per >> token-length, batch size, and attention window. >> >> Hugging Face + ONNX Runtime uses multiple small-ish anonymous mmap()s, >> often >> 512KB–1.8MB. >> >> These allocations come in bursts — but due to forced alignment, the >> kernel >> was placing them with artificial gaps, defeating THP eligibility >> entirely. >> >> By not force-aligning non-PMD-sized mappings, we avoid injecting gaps. >> The >> result is that: >> >> a. VMAs remain adjacent → mergeable >> >> b. Physical memory is contiguous → eligible for khugepaged collapse >> >> c. THP utilization increases → fewer TLB misses → lower latency → >> higher >> throughput >> > > This is very useful information and it's appreciated! Let's not drown > this > out with restatements of stuff already covered. > >> >> ⚙️ 5. mTHP note >> Although this patch doesn’t target mTHP directly, I believe a similar >> logic >> tweak could apply there too — especially with shmem-backed workloads >> (common >> in model servers using shared tensor memory). I’d be happy to help >> test any >> changes proposed there to derive the consequent results. > > Dev - could we hold off on any effort to do something like this until > I've > had a chance to refactor THP somewhat? This is already a mess and I'd > like > to avoid us piling on more complexity. > > We can revisit this at a later stage. > >> >> Thanks again for the detailed discussion. Let me know if you’d like a >> trace >> or VMA map from a Hugging Face benchmarked run (happy to generate one >> locally). >> > > Thanks! Much appreciated. > > Cheers, Lorenzo Hi Lorenzo, Thanks for your clarification, and I appreciate your patience — especially given your role in maintaining and designing the VMA merge logic. I understand now that my earlier phrasing may have repeated your explanation for VMA adjacency, and I regret unintentionally restating it. I’ll make sure to be more careful and direct going forward. As for the THP alignment condition now being `IS_ALIGNED(len, PMD_SIZE)`, I agree this resolves the regressions by removing alignment for non-aligned sizes, which was exactly what broke workloads like cactusBSSN or some AI inference loads. Thanks again for the guidance — I’m learning a lot from this thread. Best Regards, Siddhartha Sharma