Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E5A97C0218D for ; Wed, 29 Jan 2025 12:40:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5E6EF280025; Wed, 29 Jan 2025 07:40:40 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 5707F280004; Wed, 29 Jan 2025 07:40:40 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3EA9B280025; Wed, 29 Jan 2025 07:40:40 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 19F98280004 for ; Wed, 29 Jan 2025 07:40:40 -0500 (EST) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id BB592C01F1 for ; Wed, 29 Jan 2025 12:40:39 +0000 (UTC) X-FDA: 83060448198.19.84F94EA Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) by imf26.hostedemail.com (Postfix) with ESMTP id DE54A140004 for ; Wed, 29 Jan 2025 12:40:37 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=vEcj5npT; spf=pass (imf26.hostedemail.com: domain of 3xCGaZwgKCLMcTVdfTgUZhhZeX.Vhfebgnq-ffdoTVd.hkZ@flex--jackmanb.bounces.google.com designates 209.85.128.74 as permitted sender) smtp.mailfrom=3xCGaZwgKCLMcTVdfTgUZhhZeX.Vhfebgnq-ffdoTVd.hkZ@flex--jackmanb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738154438; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=81tjbVSk+Hf3eaxrqH3kSiUKX76knkCpttty+I9TDK4=; b=xWAIme+9gLY2kw8mqTy0h8ELddsiktD19lb0wH0jwjxXf432INdR9f80aMq3tvKzx4TUen hR4ColHTwQKVPydCVy+2JXZTUD6nXE19msVvdAw+mNj5dsorb4UHhaeVXUqxCkWWokb7cQ o5cOOSjzO6e8qVV6pommCJj/FEqTnH4= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738154438; a=rsa-sha256; cv=none; b=ejAS1mGxKM8YCgrx/+wyJIKhtsbBxmayT4T/xZR5EKYj09v1iSGD3drBw0W9uJffpdUmqh c6mu8ms6x0/0bkcOiLopUYS+Glg9sDDG2raGerchbOCNpA8JwqblSljVt6dbCT1AZe87fV z6gqyRQp3sbESOIXCK43yN1BKxsnAJM= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=vEcj5npT; spf=pass (imf26.hostedemail.com: domain of 3xCGaZwgKCLMcTVdfTgUZhhZeX.Vhfebgnq-ffdoTVd.hkZ@flex--jackmanb.bounces.google.com designates 209.85.128.74 as permitted sender) smtp.mailfrom=3xCGaZwgKCLMcTVdfTgUZhhZeX.Vhfebgnq-ffdoTVd.hkZ@flex--jackmanb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-436328fcfeeso52722905e9.1 for ; Wed, 29 Jan 2025 04:40:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1738154436; x=1738759236; darn=kvack.org; h=content-transfer-encoding:cc:to:from:subject:message-id :mime-version:date:from:to:cc:subject:date:message-id:reply-to; bh=81tjbVSk+Hf3eaxrqH3kSiUKX76knkCpttty+I9TDK4=; b=vEcj5npTTKvT1X7Ir6F8qwDikOCOGK65wzgDogHK6nvJLHQFywbr9UHeCSq3GMN/G2 4bKR/OkvmBugUQAHqoB71t4oXX9yO4sB3LAedb9FGVrCL4/4tumV7WOPPYQncjUzWgEG iN1hFuQ9kF11LrL/Q1JsmhhgJHI3TMlyQKvVDiXoPR8oMiNtmd9gzfs8yea9XUV2YXl/ L4OuEU2rLaR0Q/B3eKFaFI6MzSq3lYJ1rg5yx9YDllaTsdL95dru0nW9nBUWVi7UhBg/ eG5NlI5/Y81IwwN0QIDvGYy2Poi3h9No5MwIoDRipso24YGNYzw9aePmOxrpfNriFhdc xXuw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738154436; x=1738759236; h=content-transfer-encoding:cc:to:from:subject:message-id :mime-version:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=81tjbVSk+Hf3eaxrqH3kSiUKX76knkCpttty+I9TDK4=; b=AgmmP5FFbC4NF3Cmcb+NEUyXUOyho/FYxx6kJwLSyL6k8JCHql4CSNr3mspKnj7/iD dc0n368GLQEadlWwGKZOb3PrlmOkxL3zOpY0AgdM5i6UN2YZ7FJNItiwMXf5XR05Us4K FOk9V3Pt/0sQGs5KQ46Gy4o4GTEkbCxnxjuh926wJGXPe0cXYClAvWL6I4kRj1YqgtKN B3hCkJz0cYVXGEtaSBN8OqfmiUd+lhs4BBFKGQHsQpNVmOGBK35S8JogrwdfIv9FGd5X wvAiqGRcaiP02zfa0s33IDQqUuteDqE4zOVDNF+hR5E0p85OrCET3Hi5Rp3SGbCcMaQw oE2A== X-Gm-Message-State: AOJu0YzhcIdUyF27K/do9mpIgZJy8Pd64EOdvkeFjxi04rSAD8cLg9id 5vb6pggD8sgJO1cA8QqeI2W6EEHNBrR41Rr9n1BSz6k/9RNabjGdGUdKBGixaDaq7nLXZhBYDVd 1sn6sE3wTSA== X-Google-Smtp-Source: AGHT+IEMtEivCRktckmNRZVEf8pWnNirqBruxQ3QZOsHimU7mOPWuNKzYE/9ZAxJ2BbST2bRyri+fwgSIYQcKg== X-Received: from wmbbh11.prod.google.com ([2002:a05:600c:3d0b:b0:434:f7e5:bd2f]) (user=jackmanb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:1c1e:b0:432:7c08:d0ff with SMTP id 5b1f17b1804b1-438dc40b8e6mr24063645e9.23.1738154436521; Wed, 29 Jan 2025 04:40:36 -0800 (PST) Date: Wed, 29 Jan 2025 12:40:33 +0000 Mime-Version: 1.0 X-Mailer: git-send-email 2.48.1.262.g85cc9f2d1e-goog Message-ID: <20250129124034.2612562-1-jackmanb@google.com> Subject: [LSF/MM/BPF TOPIC] Page allocation for ASI From: Brendan Jackman To: lsf-pc@lists.linux-foundation.org Cc: linux-mm@kvack.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Stat-Signature: 1pjfer4w439gfjpuh7tb9dmesgpbpntc X-Rspamd-Queue-Id: DE54A140004 X-Rspam-User: X-Rspamd-Server: rspam06 X-HE-Tag: 1738154437-992375 X-HE-Meta: U2FsdGVkX1+3XDSpP7KPgG+RFzKuE+YP6ssLMtnYGw+3R8hHUF6nTlFY+kIlzeA4XVQKybZenlEEpTTmBhT2Ejh1Gxi6XiiOvG5WSKFvbhSDGCXQklQihxNOF0/+FIR32+xmYvRMopHCaPnctG7KdzPE61RGF9lgJeFTAs77gjIY4nateUl1fDaxaiR+BUGcg+HmTUL6Lh+2C/xuUI1/7pCbla0DqA3b8A79BLPKvVBuisUAn+EF9W+qppxThz5WOF3DOKM+w79WCnYSARSWW1VkAeDDn7o6p33870jvrjlTbeqdQiREQDke05CsvXtTKXw+C6IdFoKnEB/9rrWgryKHUBMdxZ9lDzqG3TgeySBgqHixPRLBIBJhYgsE9v4UFquh1xQqYJr5UCWl/6/7CfNWp6G9rJlHfX0wubyJVe2V1F+a3Y5+vBVY0NTVciYjhhiX8wePlwQ5gfrr8TwBi4JuUrdecgQwoV61A1Bbk2r3vCcmf/UJufc037kR5oguYgp7tb+EFY1Ct8Oi1bN5yBNeR9pGeezE+AjDrBEM4U0bY3yNWbEnqyVEf3kuewaxDSm9QqeDR26qP9D7BEn59QbY05PuDbNDiV1NDsKwpcJvaxdgAOflMzDvXc9r/HoX53Cgc5chIVrpoWgCCwwcaF40chp6o6ClQIFK5Lx3tJuquGjDujYPGuwkwlVUFAlTDEaEOf3vL+lXBwqgl9OhCWrUWrd24EY8yG7fgHhvZx1ycn69gO/sf4MmrWXzgWDddvSJOmZzRxVqYrUxloApHhJjsOOzB2eLAqXnSFXkisyY0dZZnGkbWfNQGsUU2gRQA9QW7Roums0bR0QvCr0HMqD7Z9y+nRoY0Orb3tkoYqi0PLSDy9hgC0D+c7Hm0WVS3Q2I7GsxopU4wgveZIqQTrONUtcM4jD2FXGRIzAuF3ACxB13G3sekTMVhAugSGn4Le4Q25nZFvBhInFtjE7 /hzWRo1F 6bCK+b4Md1c279LXd4t2REUj4GElEM6IxMOmN5Zw1K67nalnUOgpvSublNn/dKXNGKrtjtgDjqslrQaxtVQm1R5liFBAYCNLb8maFpIqRYWU9+1H9LzbtH2LCEoAzJ35ThuR4hX9uZe4TCINyC3b8qHlZgEPjTTEeWut0jOVtaB9W5StrVs2i2DaEQUYsFwelyp5GvJWn8Wa1Q5QqDpfRHoFHAtUEOKe7y9gYoKGYojuIxKsiNgd1oIA8kiZ6gXBXs92Y6Qa9UUnvP+MP2RaqmQpgy18s2K+geLCPWMIeEtZK+oKpLf8x3IU8hXhSvTJOTp1aZ85js33mUnNq9+RHQ/RS1GbZUpZzGepaHiwlfBD+dw/Endxt/TCGcePc7ltL4Jd02cuXs12qgTuoDHXWMXIODPqSo61emmSWXqa7tTVQkC70twBvzy6r0lQX07AsRsJBkzazYoZa/huTsW4mfC191YJWP7oxyfKPBKgDYJlmAZ3ub7xAXlrBKP4Zq7Ops7CcZuMddmLGo5PDv0AsRQFLYmzMaKQMeKqMxM+il5vU4Gjf8xaKXcqyNcD9awATH2fvASUIlOWGSg9A+lUsmfvy9LLCI0b3tvqUK7ry7JZPkmyaG8H4Hk7qpP4SRUVikSA4H/OgbrzREzDArQCmTNkL61WbCG3bjtHP X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: At last year=E2=80=99s LSF/MM/BPF I presented Address Space Isolation (ASI)= [0]. ASI is a mitigation for a broad class of CPU vulnerabilities that works by creatin= g a second =E2=80=9Crestricted=E2=80=9D kernel address space which has =E2=80= =9Csensitive=E2=80=9D data unmapped. If you=E2=80=99re unfamiliar with ASI, the first 10-15 minutes of that talk pr= ovide a broad overview of the whole system. The v1 of my RFC [2] also has some explanator= y discussion in the cover letter. Last year my talk was pretty high-level, taking the temperature of the MM community about how to integrate this into the broader kernel and whether t= here are any major roadblocks. Since then, I=E2=80=99ve posted a new RFC [1] and Google=E2=80=99s internal= implementation has continued to expand its footprint in production - it=E2=80=99s now a corner= stone of our CPU security strategy. Nonetheless, as noted in the RFCv2 cover-letter ther= e are a few hurdles to overcome, at least in a proof-of-concept, before I=E2=80= =99ll be making actual requests to merge ASI upstream. The one I=E2=80=99d like to talk about at this session is how to best integ= rate ASI into the page allocator. =E2=80=9CSensitivty=E2=80=9D of memory in ASI is curren= tly all decided at the allocation site. This means when allocating pages we need to alter the pagetables for the restricted address space. This is a little tricky from t= he page allocator: 1. In the most general case, adding pages to the restricted address space r= equires allocating pagetables. Allocating while you allocate requires some thoug= ht to avoid spaghetti code/deadlock risk. 2. Removing them requires a TLB flush, which can=E2=80=99t be done from all page-freeing/allocating contexts. In the RFCs, we=E2=80=99ve simply kept all free pages unmapped from the res= tricted address space. The allocator itself is largely unchanged; at the very end o= f allocation we map pages (if appropriate), allocating pagetables via totally separate allocation calls. When ASI-mapped pages are freed, they go onto a = queue that is then freed asynchronously from a context that=E2=80=99s able to bat= ch up the TLB flushes before making them available for re-allocation. Reclaim is then mad= e aware of this asynchronous process so that __GFP_DIRECT_RECLAIM allocations= can block on it where necessary. Although we=E2=80=99ve been able to hammer this approach into a viable shap= e for the Google workloads we=E2=80=99ve been concerned with so far, it=E2=80=99s not= a general solution. Some concrete reasons include: a. It leads to pointless TLB shootdowns; there must be pathological cases w= here lots of pages get un-mapped only to get immediately re-allocated and map= ped again. b. The asynchronous worker creates CPU jitter. v. It provides no ability to prioritise re-allocating pages with the same sensitivity as prior allocations. As well as TLB issues this creates pag= e zeroing costs as pages that were formerly sensitive need to be zeroed be= fore they can be mapped into the restricted address space. d. This all creates unnecessary allocation latency and extra work to free p= ages. At last year=E2=80=99s session I touched on the idea of instead using somet= hing akin to migratetypes to track sensitivity (more accurately: presence in ASI=E2=80= =99s restricted pagetables) of free pages/pageblocks. The feedback on that idea was basical= ly =E2=80=9Cdunno, we would need more details=E2=80=9D. I=E2=80=99m now workin= g on a design based on this approach and I=E2=80=99d like to use this session to go over such details. = I don=E2=80=99t have a prototype yet, but by March I hope to have shared some illustrative code. Some questions I=E2=80=99m currently investigating that I=E2=80=99d like to= discuss details of (hopefully, with proposed answers by the time of the conference!): - Can we totally avoid the need to allocate pagetables during allocation, b= y keeping ASI=E2=80=99s restricted copy of the physmap in-sync with the unr= estricted one, different only in _PAGE_PRESENT? - If not, what=E2=80=99s the best way to allocate while we allocate? - When a TLB shootdown would let us satisfy an allocation that is getting i= nto the deeper end of the slowpath, how is that prioritised and structured wrt. d= irect compact/reclaim/other fallbacks etc? - How do we maintain a balance of sensitivities among free pages, and what = does that desired balance look like? - (Note: if no page-table-allocation is needed to map nonsensitive pages,= the second question goes away: since mapping is cheap but unmapping is expensive, we would mostly just want to minimize the number of free pag= es mapped into the restricted address space). [0] https://lwn.net/Articles/974390/ https://www.youtube.com/watch?v=3DDxaN6X_fdlI [1] https://lore.kernel.org/linux-mm/20250110-asi-rfc-v2-v2-0-8419288bc805@= google.com/ [2] https://lore.kernel.org/linux-mm/20240712-asi-rfc-24-v1-0-144b319a40d8@= google.com/