Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 503F7C0218B for ; Fri, 24 Jan 2025 21:04:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AFEED2800A4; Fri, 24 Jan 2025 16:04:08 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id AAEF02800A2; Fri, 24 Jan 2025 16:04:08 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9C60F2800A4; Fri, 24 Jan 2025 16:04:08 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 7F8712800A2 for ; Fri, 24 Jan 2025 16:04:08 -0500 (EST) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 459BF14184B for ; Fri, 24 Jan 2025 21:04:08 +0000 (UTC) X-FDA: 83043572976.13.8EC3E03 Received: from out-177.mta1.migadu.com (out-177.mta1.migadu.com [95.215.58.177]) by imf12.hostedemail.com (Postfix) with ESMTP id 4887340006 for ; Fri, 24 Jan 2025 21:04:06 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=f3SVysBv; spf=pass (imf12.hostedemail.com: domain of yanjun.zhu@linux.dev designates 95.215.58.177 as permitted sender) smtp.mailfrom=yanjun.zhu@linux.dev; dmarc=pass (policy=none) header.from=linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1737752646; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=JvNcD1380VUSk+WbNCAjzR0rjQqe8vcJXl2ZALVuyvQ=; b=Z+Yvgt8LEMR9g5lxUJDShFtcmemVwnjXbkSh39UutY63mx1YGako+vjl9y8woYoNS+kd5q xmI/LrdmxJaUJrZKB2Gc9f1bxmfpcQJNiT36XCFdyag8Xqs8MNW3Dh6aX0fqHmM3YXhZBA 8ePjZ7QQgfMbzEd+ZeuZoaDq4ilz6nE= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1737752646; a=rsa-sha256; cv=none; b=VFHMGu3KZa3LlDXbeYqRKj1wtn1B3XfwwfGzv0m+0/3WHrUEL3+0UkBck0Q7P3+p6eEBsV Zo+ygsaJOwhq3GmTADtSDT6GTAHxT3O+7RxLnWr7aKysSwZ13Ms4IMvUewCSKPzizB8y1T URAzlKcEZBunUKVEald+HSLoxP3uo4c= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=f3SVysBv; spf=pass (imf12.hostedemail.com: domain of yanjun.zhu@linux.dev designates 95.215.58.177 as permitted sender) smtp.mailfrom=yanjun.zhu@linux.dev; dmarc=pass (policy=none) header.from=linux.dev Message-ID: <153dc5bb-f82d-4863-aba4-7a5c15059ee6@linux.dev> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1737752644; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=JvNcD1380VUSk+WbNCAjzR0rjQqe8vcJXl2ZALVuyvQ=; b=f3SVysBvvQMOO64e5ebZiOH0nwskHjXcpWsUHIcMqBVl1gCyOKpmCBgWRcA49yoghSjCZ1 AqW1/u3XUoR9zVq4f+oh3H4mm3BPEROnezs9+DjV17wBANXNmRZLYYVOuH7n7keHj/X19z 3mVAeb49GlikdmdZTGGAWTOWeahTCKU= Date: Fri, 24 Jan 2025 22:03:57 +0100 MIME-Version: 1.0 Subject: Re: [LSF/MM/BPF TOPIC] memory persistence over kexec To: David Rientjes , Jason Gunthorpe Cc: Mike Rapoport , lsf-pc@lists.linux-foundation.org, Alexander Graf , "Gowans, James" , linux-mm@kvack.org, Pasha Tatashin References: <20250120141427.GK674319@ziepe.ca> X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Zhu Yanjun In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Migadu-Flow: FLOW_OUT X-Rspamd-Queue-Id: 4887340006 X-Stat-Signature: umnqhbjzar6pu4hyrkuj5fp4eb73btgr X-Rspam-User: X-Rspamd-Server: rspam12 X-HE-Tag: 1737752646-879350 X-HE-Meta: U2FsdGVkX19LNDlfBmRaM7TOlJq1Sph5H/s6YasYzLnCLCckJCPABS7bTMn5yVt7N90OQ0SldE7TQZE5Zx/z5XGRKKLzGs2Xolwe5UakhJHu+rChVvOY5wBMVdXwiuJO6gv1sAPMnDVxgl0K1ExcTs3L0rSdcL5xnmg72Icoj/rQFH+tVGrCI4yp0bzCAMN5RftEy1nyi5is1BMP5cAEVb37g/fIjkMBbzD3hj7meUDSe2k5tcDS57oCrK94iYQc816olHGmbB3p8jQUUqKSDsnrdzVIoyqwzK7cF2sUuEu0NTXvcdfT2UJCdRHE0fhkuoV8Fu8mfcshsjU80EMMTyr5Xl2jh9xPZ0nUhdWkI4orevQrfXprPNzuJtZvMY0yAMaY9POYRaN9QTA11gXqalN1owRjCfqM6tDFVjc6SodIj6K752f370AuK725P4mZdrdgOGf9I60ZU8fw7mB9a+HTLXuPiGUgM1nO0LyuZJUVq5bcHupdpplSJYpMXKAcqq1SNpYTw5WxaEIC/sFagPWidtj4R9Xib6UTM/lBrQxnnVGcx5aN77SbvbjAoi8QSSljyeSKiSg91IzSK2XZ/Mxf8hK5RVx5wFTP1CLH9Gr/ecfwxtPWLa7uYgQArNrOKeZYkXcc/U56K23TKXXuWw7KrowCy6xc5DUPKHbSnPxZ736yA/6xk4wNx15tc+R7rJJpJZk+DgNn9CVe7VXpd+6+mL6z6RsIDRJkBWiIwbwWGFr9VdSNQgwYBYC/EBGWZK9yBXyB20wbm2jUqTipP1PeTz3xo8Z6hKGffTc4XBqVrWLknNvztLBW+w1PyBotpq3VaQXXdVQtJyXTrUy02ICMdIzobrg3As4sNMoacuCrSlF2LBd2DT9p7kXo15SbHtMM2twNkxu/5zWl4GVQku15pqy/0fO9rZnr7awTgU0fKs/aPV9QjhMM74GtVFha7Qaaz/TefRK/IUwSZEN FPE8T91k ztWO7yni5OAmqcmEuufI6lsTfTAOdGnisCQlrhVjTa2CgdtSB5Nxxmq/IUlntbbJ8BbHtPVvFPwH6NvNQU3G8sNc9aDfOA/ASbgbbqyd1wuu38sGwCM6ICtV6cW6Ek5eS4TL3MSpNDQ451L/m4t0SYimGLcvmfrPIu/u1S61oV3uNW1c9ho/OGE2cKeH0P2wk6IfDtGi3GGVJ584PX5amzyCly/8asam857gqEsWvJeZf8EDP7y8KECKSVdZ51GO7bJeo X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: 在 2025/1/20 20:42, David Rientjes 写道: > On Mon, 20 Jan 2025, Jason Gunthorpe wrote: > >> On Mon, Jan 20, 2025 at 09:54:15AM +0200, Mike Rapoport wrote: >>> Hi, >>> >>> I'd like to discuss memory persistence across kexec. >>> >>> Currently there is ongoing work on Kexec HandOver (KHO) [1] that allows >>> serialization and deserialization of kernel data as well as preserving >>> arbitrary memory ranges across kexec. >>> >>> In addition, KHO keeps a physically contiguous memory regions that are >>> guaranteed to not have any memory that KHO would preserve, but still can be >>> used by the system. The kexeced kernel bootstraps itself using those >>> regions and sets all handed over memory as in use. KHO users then can >>> recover their state from the preserved data. This includes memory >>> reservations, where the user can either discard or claim reservations. >>> >>> KHO can be used as the base layer for implementation of persistence-aware >>> memory allocator and persistent in-memory filesystem. >>> >>> Aside from status update on KHO progress there are a few topics that I would >>> like to discuss: >>> * Is it feasible and desirable to enable KHO support in tmpfs and hugetlbfs? > > This is a very timely discussion since the last Linux MM Alignment Session > on the topic since some use cases, at least for tmpfs, have emerged. Not > necessarily a requirement, but more out of convenience. > >>> * Or is it better to implement yet another in-memory filesystem dedicated >>> for persistence? >>> * What is the best way to ensure that the memory we want to persist is not >>> scattered all over the place? >> >> There is alot of talk about taking *drivers* and having them survive >> kexec, meaning the driver has to put alot of its state into KHO and >> then get it back out again. >> >> I've been hoping for a model where a driver can be told to "go to KHO" >> and the KHO code can be largely contained in the driver and regulated >> to recording the driver state. This implies the state may be >> fragmented all over memory. >> > > This sounds fantastic if it is doable! > >> The other direction is that the driver has to start up in some special >> KHO mode and KHO becomes invasive on all driver paths to use special >> KHO allocations. This seems like a PITA. >> >> You can see this difference just in the discussion around the iommu >> serialization where one idea was to have KHO be an integral (and >> invasive!) part of the page table operations from time zero vs some >> later serialization at kexec time. >> >> Regardless, I'm interested in this discussion to bring some >> concreteness about how drivers work.. >> > > +1, I'm also interested in this discussion. +1, Hope I can join the meeting to listen to this presentation. Zhu Yanjun > > As previously mentioned[1], we'll also start a biweekly on hypervisor live > update to accelerate progress. The first instance of that meeting will be > next week, Monday, January 27 at 8am PST (UTC-8). Calendar invites will > go out later today for everybody on that email thread, if anybody else is > interested in attending on a regular basis please email me. Hoping this > can be leveraged as well to build up to LSF/MM/BPF. > > [1] > https://lore.kernel.org/kexec/2908e4ab-abc4-ddd0-b191-fe820856cfb4@google.com/T/#u >