Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 17ADCC0218D for ; Wed, 29 Jan 2025 14:43:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 60080280056; Wed, 29 Jan 2025 09:43:26 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 5B1E8280055; Wed, 29 Jan 2025 09:43:26 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4518C280056; Wed, 29 Jan 2025 09:43:26 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 27530280055 for ; Wed, 29 Jan 2025 09:43:26 -0500 (EST) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id A8B251A0A91 for ; Wed, 29 Jan 2025 14:43:25 +0000 (UTC) X-FDA: 83060757570.11.257B532 Received: from mail-wr1-f74.google.com (mail-wr1-f74.google.com [209.85.221.74]) by imf16.hostedemail.com (Postfix) with ESMTP id 00931180009 for ; Wed, 29 Jan 2025 14:43:23 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=RTy4QXSS; spf=pass (imf16.hostedemail.com: domain of 3ij6aZwgKCLMcTVdfTgUZhhZeX.Vhfebgnq-ffdoTVd.hkZ@flex--jackmanb.bounces.google.com designates 209.85.221.74 as permitted sender) smtp.mailfrom=3ij6aZwgKCLMcTVdfTgUZhhZeX.Vhfebgnq-ffdoTVd.hkZ@flex--jackmanb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738161804; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=+RBp7dmX/tLnUznf+aWw739z2lK3UAY8L4NTkuE8UPo=; b=NoA2ojqK+PcSOJfVnLrWqAtLMAigEhObR2aP5VYqAsqprsCN7QyyF25zx57q8J+in6XVPH TsAcNXJSqk2pos9ksR373rgGEoADZ1RtGQhJSqXGN5x6M+IYDG1d8CkWyxAkWLn+70tHMX 874roEC/j2yMRHiZhhcBwe4mAqahIfE= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738161804; a=rsa-sha256; cv=none; b=Z5Ff6tIbCLWBpBXEhgt7ucmpOIGod9VLqBLrYuEk6ooCygW7aZ59G8pN65QOlYRaPpAOMk 0l+YnS/vh3/LCaxM4W4PUvOTKVx+GpBNcoDXy2nNUe8/rBrM1xXT+rxGhTb6R1bq0+COs7 P0MJNXMwTQ0U5+YfZuo8hkXf0BBWtVY= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=RTy4QXSS; spf=pass (imf16.hostedemail.com: domain of 3ij6aZwgKCLMcTVdfTgUZhhZeX.Vhfebgnq-ffdoTVd.hkZ@flex--jackmanb.bounces.google.com designates 209.85.221.74 as permitted sender) smtp.mailfrom=3ij6aZwgKCLMcTVdfTgUZhhZeX.Vhfebgnq-ffdoTVd.hkZ@flex--jackmanb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-wr1-f74.google.com with SMTP id ffacd0b85a97d-385e49efd59so2664837f8f.0 for ; Wed, 29 Jan 2025 06:43:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1738161802; x=1738766602; darn=kvack.org; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=+RBp7dmX/tLnUznf+aWw739z2lK3UAY8L4NTkuE8UPo=; b=RTy4QXSSy3s8iCzIwW9s0Jt7DrttUUv8zADEK5v0+VaTLTd8v5nDhM4RmC2+jRwZDB RahRsF2Na9ORraz2L8soWc+E2jj3HZCfL3Pgk5oXJoj5Xa4CGYFzhNWEQe+g97UkdBFO w/sLIECtKe/Or4gUT9SRdxzk2sYrd72WKZWfrP91zdEK6sftAJlI8cGqpkQycnPdntmn TGc3onSRcrzyswuEdDksUgE6z8j6p1TRWeCW1isMx2OGJP86q+OXWJBweHa6jAU6C6kb UsnKDH/QYioO2ptS9o0KNo7/jWIfvlO90hvDn2Eq/hWm84JL847cH21n3fdPK04D61Um JKsA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738161802; x=1738766602; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=+RBp7dmX/tLnUznf+aWw739z2lK3UAY8L4NTkuE8UPo=; b=SbLhmSMmgyJhWaPPxpBEVQDMVeKyuHxPcF0zJ41tMG997DyCdsDgdrRG/yrI2WgWfz GASY5hf+dMFcelTgQa/X27Y/DzClH4JxtcTBXGzfhHvH9OptNwAhyqJDMsF1Q8d/pe54 tQid1Eciwb4IAHts4d5d3/n2lJE0NCO8uMSV31d+9eL9LSOk2wKXApwqMXRZTP9J7osn EkqLB5p3EC1dFM9InSvq/jHe+fXNpRaujVfu6YOi8rUoDYsve6Lh+gkFCNo9V7T1DhPd sHJ44RSNnRVzlBEbWV43zVfn/mR+eBS2j2/S8Ri06YvBJghuJB/RRavqyVj0AgamOEwU itBw== X-Gm-Message-State: AOJu0Yy90Xjo36Zw7mimLnQ50LDB5MRZZZNtbmiXqDmJnp1uHxJtsXEY lbgpLtc4imVGpFALXvL1EiygTRj12fEXXCRFp6vozgFPrtd1VqeGlCOtSqJEJ6mZX8cxAd5DLJY 3q31DEQv+zQ== X-Google-Smtp-Source: AGHT+IFAga87gEZEX7h/7CO3N7mEpooDvFdrBzOHseVP3j1LLMe4vgJcKGukZP5yWqPJIQ5SUybEeuwYIMNUqA== X-Received: from wmqe5.prod.google.com ([2002:a05:600c:4e45:b0:436:1abf:b8fe]) (user=jackmanb job=prod-delivery.src-stubby-dispatcher) by 2002:a5d:58e9:0:b0:385:eecb:6f02 with SMTP id ffacd0b85a97d-38c519748c5mr2130516f8f.28.1738161802271; Wed, 29 Jan 2025 06:43:22 -0800 (PST) Date: Wed, 29 Jan 2025 14:43:20 +0000 Mime-Version: 1.0 X-Mailer: git-send-email 2.48.1.262.g85cc9f2d1e-goog Message-ID: <20250129144320.2675822-1-jackmanb@google.com> Subject: [LSF/MM/BPF TOPIC] ASI's page cache problem From: Brendan Jackman To: lsf-pc@lists.linux-foundation.org Cc: linux-mm@kvack.org Content-Type: text/plain; charset="UTF-8" X-Stat-Signature: h7o998hmdx4hfw34zoxwp3a3zosg9c5b X-Rspamd-Queue-Id: 00931180009 X-Rspam-User: X-Rspamd-Server: rspam06 X-HE-Tag: 1738161803-608551 X-HE-Meta: U2FsdGVkX19MCfG64Kn6HcLzOpIRa/vEcZtTjV3AQH5yvbOKB/MueRQQJeY6Vj4Yp3b7zTlX0Pgpww73M1Hcyrir87M5q5YKFdwYkJg3MVQxUjjAzYwQaQdbklkxLOMvRUOOy67HHfeq7eUavlf4WB/rssaCcNlY2CACgEHiknuWppw35JDWAZPGH06zrCM7jXLZlD7Pq2dHROOvjWTYeUwhSPoVUBSmOf4ySlxifRvupVcO7IOzhwUcInNHyMalUbc/EHHYBMkYK3kFvxVbazmq02yAGDmhpFltMwlzfQlGkZVZP6lolizl4brH1FgZu2AXL0QA8jn5NqKRcMzvXbdlUJ7yfAV1WQJUPAvVRobe/BgW9W+gBVlPFu2hY+l/5b+KDU+gvqO5EXRLJahixJA/nsZtHvqkK5vM+9hpuGZJEdUTiRE+NehM89vrG9TZarL3nj9eTmRxr0vEnXtYLURYwwvHB0DgL9eurVKXFJcPAUDYCYLfc9s3z95uPnglQzpZRbVv35wYHn+qK1m1neqizLE14AV6mPd/qg481tvUy1q/44mnQ+Fv0yQJUTLvIDNzC7/zrx1/8QO5zm/iIhVeLShOQH+f4UVf8xS83EWIJEDGOn+cOqotO/xm1TeaAXjfN4vIlPiCh2pfOUTaNklkxHZNiVCdpG1yJd9ntpUPdjjXThvN7FLfrXsHVB8R1/BjfpVMrx09AudZIS+MAA32zlFR9drVKvhLcnLTOgkbXAw4STLjj5th3xKuWfQpQ1bFOWNB0O6aiQzC11VZZKMQqh1jiHVF47DuSH3/C2bCQHQfm6y/UYLpbPAjjENxIK3bgg4H3s16PQpPhSE1BnhEnMKIqBW7Bgo6lvSDyGPmsrmHR9jIbQZsguy4R+WMcxEBaYtZxfoC/W3YzVUeWgyJfQfHXCreIBZRnuSbtCzL1q9/WZzYGc8e7fLUrCZQZCvkZpbz4Spk4lhkGSO h/cd1uXK pzwRv+DGcqgeA3hh9D+V8i2TPtutforaVtAW8JQONLr5eWm2BgNRAGBs2qFmERxJGD/y9dfkJug5K4DnCG2jNR/fgrMMh5zMzPhN2DukGLo/wS+DEpCqymaiFpR728vGAsgHcXerpOUuPeTJEtfgNH3IytUFJ5oyLX3fdfWMlrlJVWUJ3QtWmgLB/C64/2arLwvaImkSswQUdFhbOZcdjVxjOrP/JVXydn7c+cml9feQiz9F4dHcsAUUpCuwEXB/O/lLNSm+MB+dU7YQzNHqxEFHsqmgDOlg1GphY2vtrMTQ9iGmHrFVju7Gar44Ga3Ehzv79LOeOzkPrTbzk9POItUZf0OKtymEOCUMjR3koqcbKOGqGQPxa1mKSgR8z4KiESHu1s9YAyJufB2CRWzVDMt8rv5IJr3wSXdfCFhQpVZtbPlo6ap2qEu4IioWEPVIHpXjzJELY1D/tWZ11f48ScaePuKERjj4r2xmvWKEXCaaqEW8hWbbhUGk+YgJQSG26DAJ0a31n5UE18At0Nu/0dzi/c2Wz6Z6psVpCgQyFq4XbacPsBxQ95w4NahLBA1hah/deAHrQqPX/KUQ= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000871, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This is a "lower priority" topic that I would like to discuss if there are unused slots, but it shouldn't be scheduled in favour of other sessions as I have not started properly researching it and I don't expect to be very well prepared by the time of the conference. You'll get an idea of this lack of research from the vague hand-wavy ideas discussed at the bottom of this mail. My main topic proposal is [0], there's more context about ASI there. In the RFCv2 [1] for ASI, I added the capability to halt attacks from malicious bare-metal processes. This is the major missing piece that's required before ASI offers a way for maintainers stop developing bespoke per-CPU-vuln mitigations. As I discussed in the cover letter though, it exposes us to a major performance issue: we have no way to map file pages into the restricted address space. This means that whenever a process accesses a file via read(), an ASI page fault is triggered when the kernel accesses the page in the direct map. This is very expensive: I measured a 70% degradation on a 4k fio randread benchmark. And it's totally pointless as the process is about to get architectural access to the data we are "protecting" anyway. The basic issue at play here is that ASI decides on "sensitivity" (whether to map into the restricted address space) at allocation time, but whether ASI is required to protect file data from a given process is not generally known at the time when the physical pages that will hold it are allocated. The most obvious direction to search is a solution that maps these file pages at some later time, and ensure they are unmapped before the process loses logical access to the data. For this session I'd like to discuss ways of doing that without creating intolerably TLB management pain. The physmap is global, but "sensitivity" of file data is obviously relative to the process that wants to access it. Thus this addition to the restricted address space has to be process-local. I haven't properly explored it, but I suspect mixing global and local elements together in the ASI physmap is not practical. (Note: Junaid's earlier ASI RFC [2] included support for process-local sensitivity but still required deciding on the sensitivity at allocation time). So two ideas come to mind: - Create a new process-local vmalloc-like area, where file pages can be mapped as the process gains access to the underlying file. I don't yet have a mental picture of whether this is possible without creating overheads that grow linearly with the number of processes that can read a file, or how bad such overheads would be. - Create a new CPU-local region of the kernel address space. When reading file pages, ephemerally map them into this region with preemption off, and tear down these mappings before re-enabling preemption. Since they are CPU-local, that teardown requires no cross-thread communication and "should be pretty fast". At best, this means incurring a TLB miss on every file access; I don't know how bad that would be. I also don't know how costly it would be to create per-CPU virtual memory regions (meaning the PGD must be per-CPU). And I'd like to discuss: - Reasons people might see why these ideas are total non-starters. - Totally different ideas for solving the page-cache issue. - Other problems that might overlap with this one, and benefit from some new shared virtual memory facility. [0] https://lore.kernel.org/all/20250129124034.2612562-1-jackmanb@google.com/ [1] https://lore.kernel.org/linux-mm/20250110-asi-rfc-v2-v2-0-8419288bc805@google.com/ [2] https://lore.kernel.org/all/20220223052223.1202152-1-junaids@google.com/