Make KHO Stateless
Subsystems (6)
v5 cover Make KHO Stateless
This series transitions KHO from an xarray-based metadata tracking system with serialization to a radix tree data structure that can be passed directly to the next kernel.
The key motivations for this change are to:
- Eliminate the need for data serialization before kexec.
- Remove the KHO finalize state.
- Pass preservation metadata more directly to the next kernel via the FDT.
The new approach uses a radix tree to mark preserved pages. A page's physical address and its order are encoded into a single value. The tree is composed of multiple levels of page-sized tables, with leaf nodes being bitmaps where each set bit represents a preserved page. The physical address of the radix tree's root is passed in the FDT, allowing the next kernel to reconstruct the preserved memory map.
This series is broken down into the following patches:
1. kho: Adopt radix tree for preserved memory tracking:
Replaces the xarray-based tracker with the new radix tree implementation and increments the ABI version.
2. kho: Remove finalize state and clients:
Removes the now-obsolete kho_finalize() function and its usage from client code and debugfs.
---
Changelog since v4 [1]:
- Removed the 'level' and 'start' arguments from the KHO radix tree API kho_radix_walk_tree().
- Removed locking of the KHO radix tree root during module initialization, as it is the sole user of the tree at that stage.
- Renamed kho_radix_get_index() to kho_radix_get_table_index() and updated callers to invoke kho_radix_get_bitmap_index() explicitly for the bitmap (leaf) level.
- Updated documentation to clarify that "page offset" is now referred to as "shifted physical address".
[1] https://lore.kernel.org/lkml/[email protected]/
---
Jason Miu (2):
kho: Adopt radix tree for preserved memory tracking
kho: Remove finalize state and clients
Documentation/admin-guide/mm/kho.rst | 53 +- Documentation/core-api/kho/abi.rst | 6 + Documentation/core-api/kho/index.rst | 18 +- include/linux/kho/abi/kexec_handover.h | 141 +++- include/linux/kho_radix_tree.h | 70 ++ kernel/liveupdate/kexec_handover.c | 682 ++++++++++---------- kernel/liveupdate/kexec_handover_debugfs.c | 23 - kernel/liveupdate/kexec_handover_internal.h | 3 - kernel/liveupdate/luo_core.c | 12 +- kernel/liveupdate/luo_flb.c | 2 +- tools/testing/selftests/kho/init.c | 20 - 11 files changed, 557 insertions(+), 473 deletions(-) create mode 100644 include/linux/kho_radix_tree.h
base-commit: f417b7ffcbef7d76b0d8860518f50dae0e7e5eda
Signature · 1 line
2.52.0.457.g6b5491de43-goog
PATCH v5 1/2 kho: Adopt radix tree for preserved memory tracking
Introduce a radix tree implementation for tracking preserved memory pages and switch the KHO memory tracking mechanism to use it. This lays the groundwork for a stateless KHO implementation that eliminates the need for serialization and the associated "finalize" state.
This patch introduces the core radix tree data structures and constants to the KHO ABI. It adds the radix tree node and leaf structures, along with documentation for the radix tree key encoding scheme that combines a page's physical address and order.
To support broader use by other kernel subsystems, such as hugetlb preservation, the core radix tree manipulation functions are exported as a public API.
The xarray-based memory tracking is replaced with this new radix tree implementation. The core KHO preservation and unpreservation functions are wired up to use the radix tree helpers. On boot, the second kernel restores the preserved memory map by walking the radix tree whose root physical address is passed via the FDT.
The ABI compatible version is bumped to "kho-v2" to reflect the structural changes in the preserved memory map and sub-FDT property names. This includes renaming "fdt" to "preserved-data" to better reflect that preserved state may use formats other than FDT.
--- Documentation/core-api/kho/abi.rst | 6 + Documentation/core-api/kho/index.rst | 12 + include/linux/kho/abi/kexec_handover.h | 141 +++++- include/linux/kho_radix_tree.h | 70 +++ kernel/liveupdate/kexec_handover.c | 661 +++++++++++++------------ 5 files changed, 546 insertions(+), 344 deletions(-) create mode 100644 include/linux/kho_radix_tree.h
Documentation/core-api/kho/abi.rst +6 -0
@@ -22,6 +22,12 @@ memblock preservation ABI .. kernel-doc:: include/linux/kho/abi/memblock.h :doc: memblock kexec handover ABI +KHO persistent memory tracker ABI +================================= + +.. kernel-doc:: include/linux/kho/abi/kexec_handover.h + :doc: KHO persistent memory tracker + See Also ========
Documentation/core-api/kho/index.rst +12 -0
@@ -83,6 +83,18 @@ called serialization. When the FDT is generated, some properties of the system may become immutable because they are already written down in the FDT. That state is called the KHO finalization phase. +Kexec Handover Radix Tree +========================= + +.. kernel-doc:: include/linux/kho_radix_tree.h + :doc: Kexec Handover Radix Tree + +Public API +========== + +.. kernel-doc:: kernel/liveupdate/kexec_handover.c + :export: + See Also ========
include/linux/kho/abi/kexec_handover.h +126 -15
188 lines not on this page · raw message
include/linux/kho_radix_tree.h new file +70 -0
70 lines not on this page · raw message
kernel/liveupdate/kexec_handover.c +332 -329
864 lines not on this page · raw message
Signature · 1 line
2.52.0.457.g6b5491de43-goog
PATCH v5 2/2 kho: Remove finalize state and clients
In reply to Jason MiuEliminate the kho_finalize() function and its associated state from the KHO subsystem. The transition to a radix tree for memory tracking makes the explicit "finalize" state and its serialization step obsolete.
Remove the kho_finalize() and kho_finalized() APIs and their stub implementations. Update KHO client code and the debugfs interface to no longer call or depend on the kho_finalize() mechanism.
Complete the move towards a stateless KHO, simplifying the overall design by removing unnecessary state management.
--- Documentation/admin-guide/mm/kho.rst | 53 +++++---------------- Documentation/core-api/kho/index.rst | 12 ----- kernel/liveupdate/kexec_handover.c | 21 +------- kernel/liveupdate/kexec_handover_debugfs.c | 23 --------- kernel/liveupdate/kexec_handover_internal.h | 3 -- kernel/liveupdate/luo_core.c | 12 +---- kernel/liveupdate/luo_flb.c | 2 +- tools/testing/selftests/kho/init.c | 20 -------- 8 files changed, 14 insertions(+), 132 deletions(-)
Documentation/admin-guide/mm/kho.rst +11 -42
@@ -28,20 +28,10 @@ per NUMA node scratch regions on boot. Perform a KHO kexec =================== -First, before you perform a KHO kexec, you need to move the system into -the :ref:`KHO finalization phase <kho-finalization-phase>` :: - - $ echo 1 > /sys/kernel/debug/kho/out/finalize - -After this command, the KHO FDT is available in -``/sys/kernel/debug/kho/out/fdt``. Other subsystems may also register -their own preserved sub FDTs under -``/sys/kernel/debug/kho/out/sub_fdts/``. - -Next, load the target payload and kexec into it. It is important that you -use the ``-s`` parameter to use the in-kernel kexec file loader, as user -space kexec tooling currently has no support for KHO with the user space -based file loader :: +To perform a KHO kexec, load the target payload and kexec into it. It +is important that you use the ``-s`` parameter to use the in-kernel +kexec file loader, as user space kexec tooling currently has no +support for KHO with the user space based file loader :: # kexec -l /path/to/bzImage --initrd /path/to/initrd -s # kexec -e
@@ -52,40 +42,19 @@ For example, if you used ``reserve_mem`` command line parameter to create an early memory reservation, the new kernel will have that memory at the same physical address as the old kernel. -Abort a KHO exec -================ - -You can move the system out of KHO finalization phase again by calling :: - - $ echo 0 > /sys/kernel/debug/kho/out/active - -After this command, the KHO FDT is no longer available in -``/sys/kernel/debug/kho/out/fdt``. - debugfs Interfaces ================== +These debugfs interfaces are available when the kernel is compiled with +``CONFIG_KEXEC_HANDOVER_DEBUGFS`` enabled. + Currently KHO creates the following debugfs interfaces. Notice that these interfaces may change in the future. They will be moved to sysfs once KHO is stabilized. -``/sys/kernel/debug/kho/out/finalize`` - Kexec HandOver (KHO) allows Linux to transition the state of - compatible drivers into the next kexec'ed kernel. To do so, - device drivers will instruct KHO to preserve memory regions, - which could contain serialized kernel state. - While the state is serialized, they are unable to perform - any modifications to state that was serialized, such as - handed over memory allocations. - - When this file contains "1", the system is in the transition - state. When contains "0", it is not. To switch between the - two states, echo the respective number into this file. - ``/sys/kernel/debug/kho/out/fdt`` - When KHO state tree is finalized, the kernel exposes the - flattened device tree blob that carries its current KHO - state in this file. Kexec user space tooling can use this + The kernel exposes the flattened device tree blob that carries its + current KHO state in this file. Kexec user space tooling can use this as input file for the KHO payload image. ``/sys/kernel/debug/kho/out/scratch_len``
@@ -100,8 +69,8 @@ stabilized. it should place its payload images. ``/sys/kernel/debug/kho/out/sub_fdts/`` - In the KHO finalization phase, KHO producers register their own - FDT blob under this directory. + KHO producers can register their own FDT blob another binary blob under + this directory. ``/sys/kernel/debug/kho/in/fdt`` When the kernel was booted with Kexec HandOver (KHO),
Documentation/core-api/kho/index.rst +0 -12
@@ -71,18 +71,6 @@ for boot memory allocations and as target memory for kexec blobs, some parts of that memory region may be reserved. These reservations are irrelevant for the next KHO, because kexec can overwrite even the original kernel. -.. _kho-finalization-phase: - -KHO finalization phase -====================== - -To enable user space based kexec file loader, the kernel needs to be able to -provide the FDT that describes the current kernel's state before -performing the actual kexec. The process of generating that FDT is -called serialization. When the FDT is generated, some properties -of the system may become immutable because they are already written down -in the FDT. That state is called the KHO finalization phase. - Kexec Handover Radix Tree =========================
kernel/liveupdate/kexec_handover.c +1 -20
@@ -69,8 +69,7 @@ early_param("kho", kho_parse_enable);struct kho_out {void *fdt; -bool finalized; -struct mutex lock;/* protects KHO FDT finalization */ +struct mutex lock;/* protects KHO FDT */ struct kho_radix_tree radix_tree;struct kho_debugfs dbg;
@@ -81,7 +80,6 @@ static struct kho_out kho_out = { .radix_tree = { .lock = __MUTEX_INITIALIZER(kho_out.radix_tree.lock), }, - .finalized = false, };/**
@@ -1196,23 +1194,6 @@ void kho_restore_free(void *mem) }EXPORT_SYMBOL_GPL (kho_restore_free); -int kho_finalize (void ) -{ -if (!kho_enable) -return -EOPNOTSUPP ; - -guard (mutex)(&kho_out.lock); - kho_out.finalized = true; - -return 0 ; -} - -bool kho_finalized (void ) -{ -guard (mutex)(&kho_out.lock); -return kho_out.finalized; -} -struct kho_in {phys_addr_t fdt_phys;phys_addr_t scratch_phys;
kernel/liveupdate/kexec_handover_debugfs.c +0 -23
@@ -75,24 +75,6 @@ void kho_debugfs_fdt_remove(struct kho_debugfs *dbg, void *fdt) } } -static int kho_out_finalize_get (void *data,u64 *val) -{ - *val =kho_finalized (); - -return 0 ; -} - -static int kho_out_finalize_set (void *data,u64 val) -{ -if (val) -return kho_finalize (); -else -return -EINVAL ; -} - -DEFINE_DEBUGFS_ATTRIBUTE (kho_out_finalize_fops, kho_out_finalize_get, - kho_out_finalize_set,"%llu\n" ); -static int scratch_phys_show (struct seq_file *m,void *v) {for (int i =0 ; i < kho_scratch_cnt; i++)
@@ -198,11 +180,6 @@ __init int kho_out_debugfs_init(struct kho_debugfs *dbg)if (IS_ERR (f)) goto err_rmdir; - f =debugfs_create_file ("finalize" ,0600 , dir,NULL , - &kho_out_finalize_fops); -if (IS_ERR (f)) - goto err_rmdir; - dbg->dir = dir; dbg->sub_fdt_dir = sub_fdt_dir;return 0 ;
kernel/liveupdate/kexec_handover_internal.h +0 -3
@@ -22,9 +22,6 @@ struct kho_debugfs {};extern struct kho_scratch *kho_scratch;extern unsigned int kho_scratch_cnt; -bool kho_finalized (void ); -int kho_finalize (void ); -#ifdef CONFIG_KEXEC_HANDOVER_DEBUGFS int kho_debugfs_init (void );void kho_in_debugfs_init (struct kho_debugfs *dbg,const void *fdt);
kernel/liveupdate/luo_core.c +1 -11
@@ -230,17 +230,7 @@ int liveupdate_reboot(void)luo_flb_serialize (); - err =kho_finalize (); -if (err) { -pr_err ("kho_finalize failed %d\n" , err); -/* -* kho_finalize() may return libfdt errors, to aboid passing to -* userspace unknown errors, change this to EAGAIN. -*/ - err = -EAGAIN ; - } - -return err; +return 0 ; }/**
kernel/liveupdate/luo_flb.c +1 -1
@@ -630,7 +630,7 @@ int __init luo_flb_setup_incoming(void *fdt_in)* data handle, and the final reference count. This allows the new kernel to * find the appropriate handler and reconstruct the FLB's state. * -* Context: Called from liveupdate_reboot() just before kho_finalize(). +* Context: Called from liveupdate_reboot(). */ void luo_flb_serialize (void ) {
tools/testing/selftests/kho/init.c +0 -20
@@ -11,7 +11,6 @@/* from arch/x86/include/asm/setup.h */ #define COMMAND_LINE_SIZE 2048 -#define KHO_FINALIZE "/debugfs/kho/out/finalize"#define KERNEL_IMAGE "/kernel"static int mount_filesystems (void )
@@ -22,22 +21,6 @@ static int mount_filesystems(void)return mount ("proc" ,"/proc" ,"proc" ,0 ,NULL ); } -static int kho_enable (void ) -{ -const char enable[] ="1" ; -int fd; - - fd =open (KHO_FINALIZE ,O_RDWR ); -if (fd <0 ) -return -1 ; - -if (write (fd, enable,sizeof (enable)) !=sizeof (enable)) -return 1 ; - -close (fd); -return 0 ; -} -static long kexec_file_load (int kernel_fd,int initrd_fd,unsigned long cmdline_len,const char *cmdline,unsigned long flags)
@@ -78,9 +61,6 @@ int main(int argc, char *argv[])if (mount_filesystems ()) goto err_reboot; -if (kho_enable ()) - goto err_reboot; -if (kexec_load ()) goto err_reboot;
Signature · 1 line
2.52.0.457.g6b5491de43-goog