btf/kallsyms based makedumpfile extension for mm page filtering
Subsystems (3)
v5 cover btf/kallsyms based makedumpfile extension for mm page filtering
A) This patchset will introduce the following features to makedumpfile:
1) Add .so extension support to makedumpfile
2) Enable btf and kallsyms for symbol type and address resolving.
B) The purpose of the features are:
1) Currently makedumpfile filters mm pages based on page flags, because flags can help to determine one page's usage. But this page-flag-checking method lacks of flexibility in certain cases, e.g. if we want to filter those mm pages occupied by GPU during vmcore dumping due to:
a) GPU may be taking a large memory and contains sensitive data;
b) GPU mm pages have no relations to kernel crash and useless for vmcore analysis.
But there is no GPU mm page specific flags, and apparently we don't need to create one just for kdump use. A programmable filtering tool is more suitable for such cases. In addition, different GPU vendors may use different ways for mm pages allocating, programmable filtering is better than hard coding these GPU specific logics into makedumpfile in this case.
2) Currently makedumpfile already contains a programmable filtering tool, aka eppic script, which allows user to write customized code for data erasing. However it has the following drawbacks:
a) cannot do mm page filtering.
b) need to access to debuginfo of both kernel and modules, which is not applicable in the 2nd kernel.
c) eppic library has memory leaks which are not all resolved [1]. This is not acceptable in 2nd kernel.
makedumpfile need to resolve the dwarf data from debuginfo, to get symbols types and addresses. In recent kernel there are dwarf alternatives such as btf/kallsyms which can be used for this purpose. And btf/kallsyms info are already packed within vmcore, so we can use it directly.
With these, this patchset introduces makedumpfile extensions, which is based on btf/kallsyms symbol resolving, and is programmable for mm page filtering. The following section shows its usage and performance, please note the tests are performed in 1st kernel.
3) Compile and run makedumpfile extensions:
$ make LINKTYPE=dynamic USELZO=on USESNAPPY=on USEZSTD=on EXTENSION=on $ make extensions
$ /usr/bin/time -v ./makedumpfile -d 31 -l /var/crash/127.0.0.1-2025-06-10-18\:03\:12/vmcore
/tmp/extension.out --extension amdgpu_filter.so
Loaded extension: ./extensions/amdgpu_filter.so
makedumpfile Completed.
User time (seconds): 5.08
System time (seconds): 0.84
Percent of CPU this job got: 99%
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:05.95
Maximum resident set size (kbytes): 17360
...To contrast with eppic script of v2 [2]:
$ /usr/bin/time -v ./makedumpfile -d 31 -l /var/crash/127.0.0.1-2025-06-10-18\:03\:12/vmcore
/tmp/eppic.out --eppic eppic_scripts/filter_amdgpu_mm_pages.c
makedumpfile Completed.
User time (seconds): 8.23
System time (seconds): 0.88
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:09.16
Maximum resident set size (kbytes): 57128
... -rw------- 1 root root 367475074 Jan 19 19:01 /tmp/extension.out
-rw------- 1 root root 367475074 Jan 19 19:48 /tmp/eppic.out
-rw------- 1 root root 387181418 Jun 10 18:03 /var/crash/127.0.0.1-2025-06-10-18:03:12/vmcore
C) Discussion:
1) GPU types: Currently only tested with amdgpu's mm page filtering, others are not tested.
2) OS: The code can work on rhel-10+/rhel9.5+ on x86_64/arm64/s390/ppc64. rhel8.x is not supported, others are not tested.
D) Testing:
If you don't want to create your vmcore, you can find a vmcore which I created with amdgpu mm pages unfiltered [3], the amdgpu mm pages are allocated by program [4]. You can use the vmcore in 1st kernel to filter the amdgpu mm pages by the previous performance testing cmdline. To verify the pages are filtered in crash:
Unfiltered:
crash> search -c "!QAZXSW@#EDC"
ffff96b7fa800000: !QAZXSW@#EDCXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
ffff96b87c800000: !QAZXSW@#EDCXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX crash> rd ffff96b7fa800000
ffff96b7fa800000: 405753585a415121 !QAZXSW@
crash> rd ffff96b87c800000
ffff96b87c800000: 405753585a415121 !QAZXSW@
Filtered:
crash> search -c "!QAZXSW@#EDC"
crash> rd ffff96b7fa800000
rd: page excluded: kernel virtual address: ffff96b7fa800000 type: "64-bit KVADDR" crash> rd ffff96b87c800000
rd: page excluded: kernel virtual address: ffff96b87c800000 type: "64-bit KVADDR"
[1]: https://github.com/lucchouina/eppic/pull/32
[2]: https://lore.kernel.org/kexec/[email protected]/
[3]: https://people.redhat.com/~ltao/core/vmcore
[4]: https://gist.github.com/liutgnu/a8cbce1c666452f1530e1410d1f352df
v5 -> v4:
1) Add "make EXTENSION=on" switch to customize the extension feature.
2) Clean up macros within btf_info.h.
3) Updated doc and a sample extension to demo the extension usage.
4) Use MSG()/ERRMSG() rather than fprintf().
5) Add return value check for readmem().
6) Allow "makedumpfile -d 1 --extension ext.so" to enter extension.
7) The patches are organized as follows:
--- <customization specific> ---
9. Add amdgpu mm pages filtering extension
--- <code should be merged> ---
8. Doc: Add --extension option to makedumpfile manual
7. Add sample extension as an example reference
6. Add makedumpfile extensions support
5. Implement kernel module's btf resolving
4. Implement kernel module's kallsyms resolving
3. Implement kernel btf resolving
2. Implement kernel kallsyms resolving
1. Reserve sections for makedumpfile and extenions
Patch 9 is customization specific, merging depends on the strategy of maintenance.
Patch 1 ~ 8 are common code which should be merged with makedumpfile.
Link to v4: https://lore.kernel.org/kexec/[email protected]/
Link to v3: https://lore.kernel.org/kexec/[email protected]/
Link to v2: https://lore.kernel.org/kexec/[email protected]/
Link to v1: https://lore.kernel.org/kexec/[email protected]/
Tao Liu (9):
Reserve sections for makedumpfile and extenions Implement kernel kallsyms resolving Implement kernel btf resolving
Implement kernel module's kallsyms resolving Implement kernel module's btf resolving Add makedumpfile extensions support
Add sample extension as an example reference
Doc: Add --extension option to makedumpfile manual Add amdgpu mm pages filtering extension
Makefile | 15 +- README | 6 + btf_info.c | 375 +++++++++++++++++++++++++ btf_info.h | 77 ++++++ extension.c | 338 ++++++++++++++++++++++ extension.h | 16 ++ extensions/Makefile | 13 + extensions/amdgpu_filter.c | 221 +++++++++++++++ extensions/maple_tree.c | 328 ++++++++++++++++++++++ extensions/maple_tree.h | 7 + extensions/sample.c | 69 +++++ kallsyms.c | 554 +++++++++++++++++++++++++++++++++++++ kallsyms.h | 87 ++++++ makedumpfile.8.in | 11 +- makedumpfile.c | 44 ++- makedumpfile.h | 12 + makedumpfile.ld | 16 ++ 17 files changed, 2180 insertions(+), 9 deletions(-) create mode 100644 btf_info.c create mode 100644 btf_info.h create mode 100644 extension.c create mode 100644 extension.h create mode 100644 extensions/Makefile create mode 100644 extensions/amdgpu_filter.c create mode 100644 extensions/maple_tree.c create mode 100644 extensions/maple_tree.h create mode 100644 extensions/sample.c create mode 100644 kallsyms.c create mode 100644 kallsyms.h create mode 100644 makedumpfile.ld
Signature · 1 line
2.47.0
PATCH v5 1/9 Reserve sections for makedumpfile and extenions
This patch makes preparation for btf/kallsyms support of makedumpfile and extensions. Any needed kernel symbols/types will be reserved within a special section, .init_ksyms for kallsyms symbols and .init_ktypes for kernel types. During makedumpfile kallsyms/btf initialization, those missing info will be resolved. A makedumpfile.ld script is introduced for the purpose.
--- Makefile | 2 +- makedumpfile.ld | 16 ++++++++++++++++ 2 files changed, 17 insertions(+), 1 deletion(-) create mode 100644 makedumpfile.ld
Makefile +1 -1
@@ -113,7 +113,7 @@ $(OBJ_ARCH): $(SRC_ARCH) $(CC) $(CFLAGS_ARCH) -c -o ./$@ $(VPATH)$(@:.o=.c) makedumpfile: $(SRC_BASE) $(OBJ_PART) $(OBJ_ARCH) - $(CC) $(CFLAGS) $(LDFLAGS) $(OBJ_PART) $(OBJ_ARCH) -rdynamic -o $@ $< $(LIBS) + $(CC) $(CFLAGS) $(LDFLAGS) $(OBJ_PART) $(OBJ_ARCH) -rdynamic -Wl,-T,makedumpfile.ld -o $@ $< $(LIBS) @sed -e "s/@DATE@/$(DATE)/" \ -e "s/@VERSION@/$(VERSION)/" \ $(VPATH)makedumpfile.8.in > $(VPATH)makedumpfile.8
makedumpfile.ld new file +16 -0
@@ -0,0 +1,16 @@ +SECTIONS +{ + .init_ksyms ALIGN(8) : { + __start_init_ksyms = .; + KEEP(*(.init_ksyms*)) + __stop_init_ksyms = .; + } + + .init_ktypes ALIGN(8) : { + __start_init_ktypes = .; + KEEP(*(.init_ktypes*)) + __stop_init_ktypes = .; + } +} +INSERT AFTER .data; +
Signature · 1 line
2.47.0
PATCH v5 2/9 Implement kernel kallsyms resolving
In reply to Tao LiuThis patch will parse kernel's kallsyms data. During the parsing process, the .init_ksyms sections of makedumpfile and the extensions will be iterated, so the kallsyms symbols which belongs to vmlinux can be resolved at this moment.
--- Makefile | 2 +- kallsyms.c | 396 +++++++++++++++++++++++++++++++++++++++++++++++++ kallsyms.h | 84 +++++++++++ makedumpfile.c | 3 + makedumpfile.h | 11 ++ 5 files changed, 495 insertions(+), 1 deletion(-) create mode 100644 kallsyms.c create mode 100644 kallsyms.h
Makefile +1 -1
@@ -45,7 +45,7 @@ CFLAGS_ARCH += -m32 endif SRC_BASE = makedumpfile.c makedumpfile.h diskdump_mod.h sadump_mod.h sadump_info.h -SRC_PART = print_info.c dwarf_info.c elf_info.c erase_info.c sadump_info.c cache.c tools.c printk.c detect_cycle.c +SRC_PART = print_info.c dwarf_info.c elf_info.c erase_info.c sadump_info.c cache.c tools.c printk.c detect_cycle.c kallsyms.c OBJ_PART=$(patsubst %.c,%.o,$(SRC_PART)) SRC_ARCH = arch/arm.c arch/arm64.c arch/x86.c arch/x86_64.c arch/ia64.c arch/ppc64.c arch/s390x.c arch/ppc.c arch/sparc64.c arch/mips64.c arch/loongarch64.c arch/riscv64.c OBJ_ARCH=$(patsubst %.c,%.o,$(SRC_ARCH))
kallsyms.c new file +396 -0
396 lines not on this page · raw message
kallsyms.h new file +84 -0
84 lines not on this page · raw message
makedumpfile.c +3 -0
@@ -27,6 +27,7 @@#include <limits.h> #include <assert.h> #include <zlib.h> +#include "kallsyms.h" struct symbol_table symbol_table;struct size_table size_table;
@@ -3105,6 +3106,8 @@ read_vmcoreinfo_from_vmcore(off_t offset, unsigned long size, int flag_xen_hv)if (!read_vmcoreinfo ()) goto out; } +read_vmcoreinfo_kallsyms (); +close_vmcoreinfo (); ret = TRUE;
makedumpfile.h +11 -0
@@ -259,6 +259,7 @@ static inline int string_exists(char *s) { return (s ? TRUE : FALSE); }#define UINT (ADDR ) *((unsigned int *)(ADDR))#define ULONG (ADDR ) *((unsigned long *)(ADDR))#define ULONGLONG (ADDR ) *((unsigned long long *)(ADDR)) +#define VOID_PTR (ADDR ) *((void **)(ADDR))/*
@@ -1919,6 +1920,16 @@ struct symbol_table {* symbols on sparc64 arch */ unsigned long long vmemmap_table; + +/* +* kallsyms related +*/ +unsigned long long kallsyms_names; +unsigned long long kallsyms_num_syms; +unsigned long long kallsyms_token_table; +unsigned long long kallsyms_token_index; +unsigned long long kallsyms_offsets; +unsigned long long kallsyms_relative_base; };struct size_table {
Signature · 1 line
2.47.0
Re: [PATCH v5][makedumpfile 2/9] Implement kernel kallsyms resolving
Hi Tao,
thank you for the update, almost finished.
there are a few of slight comments.
Quoted 13 lines · Tao Liu
kallsyms.h new file +13
-0+#define REGISTER_SECTION (T ) \ +bool register_##T##_section(char *start, char *stop) \ +{ \ + struct section_range *new_sr; \ + struct T##_info **p; \ + bool ret = false; \ + \ + if (!start || !stop) { \ + fprintf(stderr, "%s: Invalid section start/stop\n", \ + __func__); \ + goto out; \ + } \ + \
kallsyms.h line 63
Quoted 10 lines
kallsyms.h new file +10
-0+ for (p = (struct T##_info **)start; \ + p < (struct T##_info **)stop; \ + p++) { \ + if (!add_##T##_modname((*p)->modname)) \ + goto out; \ + } \ + \ + new_sr = malloc(sizeof(struct section_range)); \ + if (!new_sr) { \ + fprintf(stderr, "%s: Not enough memory!\n", __func__); \
fprintf()s still exist in this macro.
Thanks,
Kazu
Re: [PATCH v5][makedumpfile 2/9] Implement kernel kallsyms resolving
Hi Kazu,
Ah, you are right, I missed those. I will fix the fprintf()s in the macro and send v6.
Thanks,
Tao Liu
Quoted 36 lines · HAGIO KAZUHITO(萩尾 一仁)
Hi Tao,
thank you for the update, almost finished.
there are a few of slight comments.Quoted 13 lines · Tao Liu
kallsyms.h new file +13
-0+#define REGISTER_SECTION (T ) \ +bool register_##T##_section(char *start, char *stop) \ +{ \ + struct section_range *new_sr; \ + struct T##_info **p; \ + bool ret = false; \ + \ + if (!start || !stop) { \ + fprintf(stderr, "%s: Invalid section start/stop\n", \ + __func__); \ + goto out; \ + } \ + \kallsyms.h line 63
Quoted 10 lines
kallsyms.h new file +10
-0+ for (p = (struct T##_info **)start; \ + p < (struct T##_info **)stop; \ + p++) { \ + if (!add_##T##_modname((*p)->modname)) \ + goto out; \ + } \ + \ + new_sr = malloc(sizeof(struct section_range)); \ + if (!new_sr) { \ + fprintf(stderr, "%s: Not enough memory!\n", __func__); \fprintf()s still exist in this macro.
Thanks,
Kazu
PATCH v5 3/9 Implement kernel btf resolving
In reply to Tao LiuThis patch will parse kernel's btf data using libbpf. The kernel's btf data is located between __start_BTF and __stop_BTF symbols which are resolved by kallsyms of the previous patch. Same as the previous one, the .init_ktypes section of makedumpfile and the extensions will be iterated, and any types which belongs to vmlinux can be resolved at this time.
Another primary function implemented in this patch, is recursively diving into anonymous struct/union when encountered any, to find a member by given its name.
--- Makefile | 2 +- btf_info.c | 242 +++++++++++++++++++++++++++++++++++++++++++++++++++++ btf_info.h | 75 +++++++++++++++++ 3 files changed, 318 insertions(+), 1 deletion(-) create mode 100644 btf_info.c create mode 100644 btf_info.h
Makefile +1 -1
@@ -45,7 +45,7 @@ CFLAGS_ARCH += -m32 endif SRC_BASE = makedumpfile.c makedumpfile.h diskdump_mod.h sadump_mod.h sadump_info.h -SRC_PART = print_info.c dwarf_info.c elf_info.c erase_info.c sadump_info.c cache.c tools.c printk.c detect_cycle.c kallsyms.c +SRC_PART = print_info.c dwarf_info.c elf_info.c erase_info.c sadump_info.c cache.c tools.c printk.c detect_cycle.c kallsyms.c btf_info.c OBJ_PART=$(patsubst %.c,%.o,$(SRC_PART)) SRC_ARCH = arch/arm.c arch/arm64.c arch/x86.c arch/x86_64.c arch/ia64.c arch/ppc64.c arch/s390x.c arch/ppc.c arch/sparc64.c arch/mips64.c arch/loongarch64.c arch/riscv64.c OBJ_ARCH=$(patsubst %.c,%.o,$(SRC_ARCH))
btf_info.c new file +242 -0
242 lines not on this page · raw message
btf_info.h new file +75 -0
75 lines not on this page · raw message
Signature · 1 line
2.47.0
PATCH v5 4/9 Implement kernel module's kallsyms resolving
In reply to Tao LiuWith kernel's kallsyms and btf ready, we can get any kernel types and symbol addresses. So we can iterate kernel modules' linked list, and parse each one of kernel module's structure to get its kallsyms data. At this time, kernel modules' kallsyms symbol defined within .init_ksyms section will be resolved.
--- kallsyms.c | 158 +++++++++++++++++++++++++++++++++++++++++++++++++++++ kallsyms.h | 3 + 2 files changed, 161 insertions(+)
kallsyms.c +158 -0
@@ -6,6 +6,7 @@#include <string.h> #include "makedumpfile.h" #include "kallsyms.h" +#include "btf_info.h" static uint32_t *kallsyms_offsets =NULL ;static uint16_t *kallsyms_token_index =NULL ;
@@ -385,6 +386,163 @@ out: }return ret; } + +INIT_MOD_SYM (vmlinux, modules); + +INIT_MOD_STRUCT_MEMBER (vmlinux, list_head, next); +INIT_MOD_STRUCT_MEMBER (vmlinux, module, list); +INIT_MOD_STRUCT_MEMBER (vmlinux, module, name); +INIT_MOD_STRUCT_MEMBER (vmlinux, module, core_kallsyms); +INIT_MOD_STRUCT_MEMBER (vmlinux, mod_kallsyms, symtab); +INIT_MOD_STRUCT_MEMBER (vmlinux, mod_kallsyms, num_symtab); +INIT_MOD_STRUCT_MEMBER (vmlinux, mod_kallsyms, strtab); +INIT_MOD_STRUCT_MEMBER (vmlinux, elf64_sym, st_name); +INIT_MOD_STRUCT_MEMBER (vmlinux, elf64_sym, st_value); + +#define MEMBER_OFF (S ,M ) \ + GET_MOD_STRUCT_MEMBER_MOFF(vmlinux, S, M) / 8 +#define GET_KERN_STRUCT_MEMBER_MSIZE (S ,M ) \ + GET_MOD_STRUCT_MEMBER_MSIZE(vmlinux, S, M) +#define KERN_STRUCT_MEMBER_EXIST (S ,M ) \ + MOD_STRUCT_MEMBER_EXIST(vmlinux, S, M) +#define GET_KERN_STRUCT_MEMBER_SSIZE (S ,M ) \ + GET_MOD_STRUCT_MEMBER_SSIZE(vmlinux, S, M) +#define GET_KERN_SYM (SYM ) GET_MOD_SYM(vmlinux, SYM) +#define KERN_SYM_EXIST (SYM ) MOD_SYM_EXIST(vmlinux, SYM) + +uint64_t next_list (uint64_t list) +{ +uint64_t next =0 ; + +if (!readmem (VADDR , list +MEMBER_OFF (list_head, next), + &next,GET_KERN_STRUCT_MEMBER_MSIZE (list_head, next))) { +ERRMSG ("Can't get next list!\n" ); + } +return next; +} + +bool init_module_kallsyms (void ) +{ +uint64_t modules, list, value =0 , symtab =0 , strtab =0 ; +uint32_t st_name =0 ; +int num_symtab, i, j; +struct ksym_info **p; +char symname[512 ], ch; +char *modname =NULL ; +bool ret = false; + + modules =GET_KERN_SYM (modules); +if (!KERN_SYM_EXIST (modules)) { +/* Not a failure if no module enabled */ + ret = true; + goto out; + } + +if (!KERN_STRUCT_MEMBER_EXIST (list_head, next) || + !KERN_STRUCT_MEMBER_EXIST (module, list) || + !KERN_STRUCT_MEMBER_EXIST (module, name) || + !KERN_STRUCT_MEMBER_EXIST (module, core_kallsyms) || + !KERN_STRUCT_MEMBER_EXIST (mod_kallsyms, symtab) || + !KERN_STRUCT_MEMBER_EXIST (mod_kallsyms, num_symtab) || + !KERN_STRUCT_MEMBER_EXIST (mod_kallsyms, strtab) || + !KERN_STRUCT_MEMBER_EXIST (elf64_sym, st_name) || + !KERN_STRUCT_MEMBER_EXIST (elf64_sym, st_value)) { +/* Fail when module enabled but any required types not found */ +ERRMSG ("Missing required module syms/types!\n" ); + goto out; + } + + modname = (char *)malloc (GET_KERN_STRUCT_MEMBER_MSIZE (module, name)); +if (!modname) + goto no_mem; + +for (list =next_list (modules); list != modules; list =next_list (list)) { +if (!readmem (VADDR , list -MEMBER_OFF (module, list) + +MEMBER_OFF (module, name), + modname,GET_KERN_STRUCT_MEMBER_MSIZE (module, name))) { +ERRMSG ("Can't get module modname!\n" ); + goto out; + } +if (!check_ksyms_require_modname (modname,NULL )) +continue ; +if (!readmem (VADDR , list -MEMBER_OFF (module, list) + +MEMBER_OFF (module, core_kallsyms) + +MEMBER_OFF (mod_kallsyms, num_symtab), + &num_symtab,GET_KERN_STRUCT_MEMBER_MSIZE (mod_kallsyms, num_symtab))) { +ERRMSG ("Can't get module num_symtab!\n" ); + goto out; + } +if (!readmem (VADDR , list -MEMBER_OFF (module, list) + +MEMBER_OFF (module, core_kallsyms) + +MEMBER_OFF (mod_kallsyms, symtab), + &symtab,GET_KERN_STRUCT_MEMBER_MSIZE (mod_kallsyms, symtab))) { +ERRMSG ("Can't get module symtab!\n" ); + goto out; + } +if (!readmem (VADDR , list -MEMBER_OFF (module, list) + +MEMBER_OFF (module, core_kallsyms) + +MEMBER_OFF (mod_kallsyms, strtab), + &strtab,GET_KERN_STRUCT_MEMBER_MSIZE (mod_kallsyms, strtab))) { +ERRMSG ("Can't get module strtab!\n" ); + goto out; + } +for (i =0 ; i < num_symtab; i++) { + j =0 ; +if (!readmem (VADDR , symtab + i *GET_KERN_STRUCT_MEMBER_SSIZE (elf64_sym, st_value) + +MEMBER_OFF (elf64_sym, st_value), + &value,GET_KERN_STRUCT_MEMBER_MSIZE (elf64_sym, st_value))) { +ERRMSG ("Can't get module st_value!\n" ); + goto out; + } +if (!readmem (VADDR , symtab + i *GET_KERN_STRUCT_MEMBER_SSIZE (elf64_sym, st_name) + +MEMBER_OFF (elf64_sym, st_name), + &st_name,GET_KERN_STRUCT_MEMBER_MSIZE (elf64_sym, st_name))) { +ERRMSG ("Can't get module st_name!\n" ); + goto out; + } +do { +if (!readmem (VADDR , strtab + st_name + j++, &ch,1 )) { +ERRMSG ("Can't get module symname's next char!\n" ); + goto out; + } + }while (ch !='\0' ); +if (j ==1 || j >sizeof (symname)) +/* Skip empty or too long string */ +continue ; +if (!readmem (VADDR , strtab + st_name, symname, j)) { +ERRMSG ("Can't get module symname!\n" ); + goto out; + } + +for (j =0 ; j < sr_len; j++) { +for (p = (struct ksym_info **)(sr[j]->start); + p < (struct ksym_info **)(sr[j]->stop); + p++) { +if (!strcmp ((*p)->modname, modname) && + !strcmp ((*p)->symname, symname)) { + (*p)->value = value; + (*p)->index = i; + } + } + } + } + } + ret = true; + goto out; +no_mem: +ERRMSG ("Not enough memory!\n" ); +out: +if (modname) +free (modname); +return ret; +} + +void cleanup_kallsyms (void ) +{ +cleanup_ksyms_section_range (); +cleanup_ksyms_modname (); +} +#else /* EXTENSION */ bool read_vmcoreinfo_kallsyms (void )
kallsyms.h +3 -0
@@ -80,5 +80,8 @@ bool check_ksyms_require_modname(char *modname, int *total);bool register_ksym_section (char *start,char *stop);bool read_vmcoreinfo_kallsyms (void );bool init_kernel_kallsyms (void ); +uint64_t next_list (uint64_t list); +bool init_module_kallsyms (void ); +void cleanup_kallsyms (void );#endif /* _KALLSYMS_H */
Signature · 1 line
2.47.0
PATCH v5 5/9 Implement kernel module's btf resolving
In reply to Tao LiuSame as the previous patch, with kernel's kallsyms and btf ready, we can locate and iterate all kernel modules' btf data. So kernel modules' types specified within .init_ksyms section will be resolved.
--- btf_info.c | 133 +++++++++++++++++++++++++++++++++++++++++++++++++++++ btf_info.h | 2 + 2 files changed, 135 insertions(+)
btf_info.c +133 -0
@@ -238,5 +238,138 @@ out:free (buf);return ret; } + +INIT_MOD_SYM (vmlinux, btf_modules); + +INIT_MOD_STRUCT_MEMBER (vmlinux, btf_module, list); +INIT_MOD_STRUCT_MEMBER (vmlinux, btf_module, btf); +INIT_MOD_STRUCT_MEMBER (vmlinux, btf_module, module); +DECLARE_MOD_STRUCT_MEMBER (vmlinux, module, name); +INIT_MOD_STRUCT_MEMBER (vmlinux, btf, data); +INIT_MOD_STRUCT_MEMBER (vmlinux, btf, data_size); + +#define KERN_STRUCT_MEMBER_EXIST (S ,M ) MOD_STRUCT_MEMBER_EXIST(vmlinux, S, M) +#define MEMBER_OFF (S ,M ) GET_MOD_STRUCT_MEMBER_MOFF(vmlinux, S, M) / 8 +#define GET_KERN_STRUCT_MEMBER_MSIZE (S ,M ) GET_MOD_STRUCT_MEMBER_MSIZE(vmlinux, S, M) +#define GET_KERN_SYM (SYM ) GET_MOD_SYM(vmlinux, SYM) + +bool init_module_btf (void ) +{ +struct btf *btf_mod; +uint64_t btf_modules, list; +uint64_t btf =0 , data =0 , module =0 ; +int data_size =0 ; +bool ret = false; +char *btf_buf =NULL ; +char *modname =NULL ; +struct ktype_info **p; + + btf_modules =GET_KERN_SYM (btf_modules); +if (!KERN_SYM_EXIST (btf_modules)) +/* Maybe module is not enabled, this is not an error */ +return true; + +if (!KERN_STRUCT_MEMBER_EXIST (btf_module, list) || + !KERN_STRUCT_MEMBER_EXIST (btf_module, btf) || + !KERN_STRUCT_MEMBER_EXIST (btf_module, module) || + !KERN_STRUCT_MEMBER_EXIST (btf, data) || + !KERN_STRUCT_MEMBER_EXIST (btf, data_size)) { +/* Fail when module enabled but any required types not found */ +ERRMSG ("Missing required btf syms/types!\n" ); + goto out; + } + + modname = (char *)malloc (GET_KERN_STRUCT_MEMBER_MSIZE (module, name)); +if (!modname) + goto no_mem; + +for (list =next_list (btf_modules); list != btf_modules; list =next_list (list)) { +if (!readmem (VADDR , list -MEMBER_OFF (btf_module, list) + +MEMBER_OFF (btf_module, btf), + &btf,GET_KERN_STRUCT_MEMBER_MSIZE (btf_module, btf))) { +ERRMSG ("Can't get btf_module member btf!\n" ); + goto out; + } +if (!readmem (VADDR , list -MEMBER_OFF (btf_module, list) + +MEMBER_OFF (btf_module, module), + &module,GET_KERN_STRUCT_MEMBER_MSIZE (btf_module, module))) { +ERRMSG ("Can't get btf_module member module!\n" ); + goto out; + } +if (!readmem (VADDR , module +MEMBER_OFF (module, name), + modname,GET_KERN_STRUCT_MEMBER_MSIZE (module, name))) { +ERRMSG ("Can't get module modname!\n" ); + goto out; + } +if (!check_ktypes_require_modname (modname,NULL )) { +continue ; + } +if (!readmem (VADDR , btf +MEMBER_OFF (btf, data), + &data,GET_KERN_STRUCT_MEMBER_MSIZE (btf, data))) { +ERRMSG ("Can't get module btf address!\n" ); + goto out; + } +if (!readmem (VADDR , btf +MEMBER_OFF (btf, data_size), + &data_size,GET_KERN_STRUCT_MEMBER_MSIZE (btf, data_size))) { +ERRMSG ("Can't get module btf data size!\n" ); + goto out; + } + btf_buf = (char *)malloc (data_size); +if (!btf_buf) + goto no_mem; +if (!readmem (VADDR , data, btf_buf, data_size)) { +ERRMSG ("Can't get module btf data!\n" ); + goto out; + } + btf_mod =btf__new_split (btf_buf, data_size, btf_arr[0 ]->btf); +free (btf_buf); +if (libbpf_get_error (btf_mod) !=0 || +add_to_btf_arr (btf_mod,strdup (modname)) == false) { +ERRMSG ("init %s btf fail\n" , modname); + goto out; + } + } + +/* OK, we have loaded all needed modules's btf, now resolve the types */ +for (int i =0 ; i < sr_len; i++) { +for (p = (struct ktype_info **)(sr[i]->start); + p < (struct ktype_info **)(sr[i]->stop); + p++) +get_ktype_info (*p,NULL ); + } + + ret = true; + goto out; + +no_mem: +ERRMSG ("Not enough memory!\n" ); +out: +if (modname) +free (modname); +return ret; +} + +static void cleanup_btf_arr (void ) +{ +for (int i =0 ; i < btf_arr_len; i++) { +free (btf_arr[i]->module); +btf__free (btf_arr[i]->btf); +free (btf_arr[i]); + } +if (btf_arr) { +free (btf_arr); + btf_arr =NULL ; + } + btf_arr_len =0 ; + btf_arr_cap =0 ; +} + +void cleanup_btf (void ) +{ +cleanup_btf_arr (); +cleanup_ktypes_section_range (); +cleanup_ktypes_modname (); +} +#endif /* EXTENSION */
btf_info.h +2 -0
@@ -22,6 +22,8 @@ struct ktype_info {bool check_ktypes_require_modname (char *modname,int *total);bool register_ktype_section (char *start,char *stop);bool init_kernel_btf (void ); +bool init_module_btf (void ); +void cleanup_btf (void );#define _GEN_NAME_PTR_IMPL (PTR ,NAME ) PTR##NAME#define _GEN_NAME_PTR (PTR ,NAME ) _GEN_NAME_PTR_IMPL(PTR, NAME)
Signature · 1 line
2.47.0
PATCH v5 6/9 Add makedumpfile extensions support
In reply to Tao LiuThe extensions can be specified by makedumpfile cmdline parameter as "--extension", followed by extension's filename or absolute path. If filename is give, then "./", "./extenisons" and "/usr/lib64/makedumpfile/extensions/" will be searched.
The procedures of extensions are as follows:
Step 0: Every extensions will declare which kernel symbol/types they needed during programming. This info will be stored within .init_ksyms/ktypes section. Also extension will have a callback function for makedumpfile to call.
Step 1: Register .init_ksyms and .init_ktypes sections of makedumpfile itself and extension's .so files, then tell kallsyms/btf subcomponent that which kernel symbols/types will be resolved. And callbacks are also registered.
Step 2: Init kernel/module's btf/kallsyms on demand. Any un-needed kenrel modules will be skipped.
Step 3: During btf/kallsyms parsing, the needed info will be filled. For syms/types which are defined via INIT_MOD_OPT(...) macro, these are optinal syms/types, it won't fail at parsing step if any are missing, instead, they need to be checked within extension_init() of each extensions; Otherwise for syms/types which defined via INIT_MOD(...) macro, these are must-have syms/types, if any missing, the extension will fail at this step and as a result this extension will be skipped.
After this step, required kernel symbol value and kernel types size/offset are resolved, the extensions are ready to go.
Step 4: When makedumpfile doing page filtering, in addition to its original filtering mechanism, it will call extensions callbacks for advices whether the page should be included/excluded.
--- Makefile | 13 +- extension.c | 338 ++++++++++++++++++++++++++++++++++++++++++++ extension.h | 16 +++ extensions/Makefile | 11 ++ makedumpfile.c | 41 +++++- makedumpfile.h | 1 + 6 files changed, 413 insertions(+), 7 deletions(-) create mode 100644 extension.c create mode 100644 extension.h create mode 100644 extensions/Makefile
Makefile +12 -1
@@ -45,7 +45,7 @@ CFLAGS_ARCH += -m32 endif SRC_BASE = makedumpfile.c makedumpfile.h diskdump_mod.h sadump_mod.h sadump_info.h -SRC_PART = print_info.c dwarf_info.c elf_info.c erase_info.c sadump_info.c cache.c tools.c printk.c detect_cycle.c kallsyms.c btf_info.c +SRC_PART = print_info.c dwarf_info.c elf_info.c erase_info.c sadump_info.c cache.c tools.c printk.c detect_cycle.c kallsyms.c btf_info.c extension.c OBJ_PART=$(patsubst %.c,%.o,$(SRC_PART)) SRC_ARCH = arch/arm.c arch/arm64.c arch/x86.c arch/x86_64.c arch/ia64.c arch/ppc64.c arch/s390x.c arch/ppc.c arch/sparc64.c arch/mips64.c arch/loongarch64.c arch/riscv64.c OBJ_ARCH=$(patsubst %.c,%.o,$(SRC_ARCH))
@@ -73,6 +73,11 @@ LIBS := -lzstd $(LIBS) CFLAGS += -DUSEZSTD endif +ifeq ($(EXTENSION), on) +LIBS := -lbpf $(LIBS) +CFLAGS += -DEXTENSION +endif + ifeq ($(DEBUG), on) # Requires libasan CFLAGS += -fsanitize=address
@@ -126,6 +131,7 @@ eppic_makedumpfile.so: extension_eppic.c clean: rm -f $(OBJ) $(OBJ_PART) $(OBJ_ARCH) makedumpfile makedumpfile.8 makedumpfile.conf.5 + $(MAKE) -C extensions clean install: install -m 755 -d ${DESTDIR}/${SBINDIR} ${DESTDIR}/usr/share/man/man5 ${DESTDIR}/usr/share/man/man8
@@ -135,3 +141,8 @@ install: mkdir -p ${DESTDIR}/usr/share/makedumpfile/eppic_scripts install -m 644 -D $(VPATH)makedumpfile.conf ${DESTDIR}/usr/share/makedumpfile/makedumpfile.conf.sample install -m 644 -t ${DESTDIR}/usr/share/makedumpfile/eppic_scripts/ $(VPATH)eppic_scripts/* + +.PHONY: extensions +extensions: + $(MAKE) -C extensions CC=$(CC) +
extension.c new file +338 -0
338 lines not on this page · raw message
extension.h new file +16 -0
@@ -0,0 +1,16 @@ +#ifndef _EXTENSION_H +#define _EXTENSION_H +#include <stdbool.h> + +enum { +PG_INCLUDE ,// Exntesion will keep the page +PG_EXCLUDE ,// Exntesion will discard the page +PG_UNDECID ,// Exntesion makes no decision +}; +int run_extension_callback (unsigned long pfn,const void *pcache); +void init_extensions (void ); +void cleanup_extensions (void ); +bool add_extension_opts (char *opt); +bool has_extension_loaded (void ); +#endif /* _EXTENSION_H */ +
extensions/Makefile new file +11 -0
@@ -0,0 +1,11 @@ +CC ?= gcc +CONTRIB_SO := + +all: $(CONTRIB_SO) + +$(CONTRIB_SO): %.so: %.c + $(CC) -O2 -g -fPIC -shared -Wl,-T,../makedumpfile.ld -o $@ $^ + +clean: + rm -f $(CONTRIB_SO) +
makedumpfile.c +35 -6
136 lines not on this page · raw message
makedumpfile.h +1 -0
@@ -2747,6 +2747,7 @@ struct elf_prstatus {#define OPT_CHECK_PARAMS OPT_START+18#define OPT_DRY_RUN OPT_START+19#define OPT_SHOW_STATS OPT_START+20 +#define OPT_EXTENSION OPT_START+21/* * Function Prototype.
Signature · 1 line
2.47.0
Re: [PATCH v5][makedumpfile 6/9] Add makedumpfile extensions support
extension.c line 58
Quoted 9 lines · Tao Liu
extension.c new file +9
-0+static bool init_kallsyms_btf (void ) +{ +int count; +bool ret = false; +/* We will load module's btf/kallsyms on demand */ +bool init_ksyms_module = false; +bool init_ktypes_module = false; + +if (check_ksyms_require_modname ("vmlinux" , &count)) {
Thank you for the explanation [1].
so which code path adds "vmlinux" to the mods array before this line? I could not find it.
[1] https://lore.kernel.org/kexec/CAO7dBbUa_8H0fe+9bTNTQWjMVy9qCPiu0fo3g8XamiDS_Mr8eA@mail.gmail.com/
extension.c line 314
Quoted 4 lines
extension.c new file +4
-0+ +bool has_extension_loaded (void ) +{ +return extension_opts_len >0 ;
If an extension is skipped, this looks > 0 with no extention. handle_cbs_len is better?
Thanks,
Kazu
Re: [PATCH v5][makedumpfile 6/9] Add makedumpfile extensions support
Quoted 14 lines · HAGIO KAZUHITO(萩尾 一仁)
extension.c line 58
Quoted 9 lines · Tao Liu
extension.c new file +9
-0+static bool init_kallsyms_btf (void ) +{ +int count; +bool ret = false; +/* We will load module's btf/kallsyms on demand */ +bool init_ksyms_module = false; +bool init_ktypes_module = false; + +if (check_ksyms_require_modname ("vmlinux" , &count)) {Thank you for the explanation [1].
Quoted 2 lines
so which code path adds "vmlinux" to the mods array before this line? I could not find it.
Hello Kazu,
In kallsyms.c from patch 2, there is the line:
INIT_MOD_SYM(vmlinux, _stext);
This adds a struct ksym_info to the main makedumpfile executable's array, which requests symbol "_stext" from vmlinux.
In kallsyms.c "init_kernel_kallsyms()", again from patch 2, the first step is to call:
register_ksym_section((char *)__start_init_ksyms,
(char *)__stop_init_ksyms))
Which is a function generated by the macro REGISTER_SECTION(). The implementation will use add_ksym_modname() whens it encounters this symbol, ensuring that "vmlinux" is part of the modname array.
Thanks,
Stephen
Quoted 14 lines
[1] https://lore.kernel.org/kexec/CAO7dBbUa_8H0fe+9bTNTQWjMVy9qCPiu0fo3g8XamiDS_Mr8eA@mail.gmail.com/
extension.c line 314
Quoted 4 lines
extension.c new file +4
-0+ +bool has_extension_loaded (void ) +{ +return extension_opts_len >0 ;If an extension is skipped, this looks > 0 with no extention.
handle_cbs_lenis better?Thanks,
Kazu
Re: [PATCH v5][makedumpfile 6/9] Add makedumpfile extensions support
Hi Stephen & Kazu,
Quoted 26 lines · Stephen Brennan
Quoted 14 lines · HAGIO KAZUHITO(萩尾 一仁)
extension.c line 58
Quoted 9 lines · Tao Liu
extension.c new file +9
-0+static bool init_kallsyms_btf (void ) +{ +int count; +bool ret = false; +/* We will load module's btf/kallsyms on demand */ +bool init_ksyms_module = false; +bool init_ktypes_module = false; + +if (check_ksyms_require_modname ("vmlinux" , &count)) {Thank you for the explanation [1].
Quoted 2 lines
so which code path adds "vmlinux" to the mods array before this line? I could not find it.
Hello Kazu,
In
kallsyms.cfrom patch 2, there is the line:
INIT_MOD_SYM(vmlinux, _stext);
This adds a structksym_infoto the main makedumpfile executable's array, which requests symbol "_stext" from vmlinux.
Quoted 7 lines
In
kallsyms.c"init_kernel_kallsyms()", again from patch 2, the first step is to call:register_ksym_section((char *)__start_init_ksyms, (char *)__stop_init_ksyms))Which is a function generated by the macro
REGISTER_SECTION(). The implementation will useadd_ksym_modname()whens it encounters this symbol, ensuring that "vmlinux" is part of the modname array.
Thanks for the detailed info, yes, it is exactly how "vmlinux" is added to the array.
Quoted 14 lines
Thanks,
StephenQuoted 10 lines
[1] https://lore.kernel.org/kexec/CAO7dBbUa_8H0fe+9bTNTQWjMVy9qCPiu0fo3g8XamiDS_Mr8eA@mail.gmail.com/
extension.c line 314
Quoted 4 lines
extension.c new file +4
-0+ +bool has_extension_loaded (void ) +{ +return extension_opts_len >0 ;If an extension is skipped, this looks > 0 with no extention.
handle_cbs_lenis better?
Yeah, I agree, has_extension_loaded() should also be updated to has_extension_callback(). I will fix it in v6.
Thanks,
Tao Liu
Quoted 5 lines
Quoted 4 lines
Thanks,
Kazu
Re: [PATCH v5][makedumpfile 6/9] Add makedumpfile extensions support
Quoted 38 lines · Tao Liu
Hi Stephen & Kazu,
Quoted 26 lines · Stephen Brennan
Quoted 14 lines · HAGIO KAZUHITO(萩尾 一仁)
extension.c line 58
Quoted 9 lines · Tao Liu
extension.c new file +9
-0+static bool init_kallsyms_btf (void ) +{ +int count; +bool ret = false; +/* We will load module's btf/kallsyms on demand */ +bool init_ksyms_module = false; +bool init_ktypes_module = false; + +if (check_ksyms_require_modname ("vmlinux" , &count)) {Thank you for the explanation [1].
Quoted 2 lines
so which code path adds "vmlinux" to the mods array before this line? I could not find it.
Hello Kazu,
In
kallsyms.cfrom patch 2, there is the line:INIT_MOD_SYM(vmlinux, _stext);This adds a struct
ksym_infoto the main makedumpfile executable's array, which requests symbol "_stext" from vmlinux.Quoted 7 lines
In
kallsyms.c"init_kernel_kallsyms()", again from patch 2, the first step is to call:register_ksym_section((char *)__start_init_ksyms, (char *)__stop_init_ksyms))Which is a function generated by the macro
REGISTER_SECTION(). The implementation will useadd_ksym_modname()whens it encounters this symbol, ensuring that "vmlinux" is part of the modname array.
Quoted 2 lines
Thanks for the detailed info, yes, it is exactly how "vmlinux" is added to the array.
hmm, maybe I still misread it though..
The init_kernel_kallsyms() is called _after_ check_ksyms_require_modname(). so I've meant that if no extension adds "vmlinux" to the modname array, no one calls it. is it ok?
if (check_ksyms_require_modname("vmlinux", &count)) {
if (!init_kernel_kallsyms())Thanks,
Kazu
Quoted 27 lines
Quoted 15 lines
Thanks,
StephenQuoted 10 lines
[1] https://lore.kernel.org/kexec/CAO7dBbUa_8H0fe+9bTNTQWjMVy9qCPiu0fo3g8XamiDS_Mr8eA@mail.gmail.com/
extension.c line 314
Quoted 4 lines
extension.c new file +4
-0+ +bool has_extension_loaded (void ) +{ +return extension_opts_len >0 ;If an extension is skipped, this looks > 0 with no extention.
handle_cbs_lenis better?Yeah, I agree,
has_extension_loaded()should also be updated tohas_extension_callback(). I will fix it in v6.Thanks,
Tao LiuQuoted 5 lines
Quoted 4 lines
Thanks,
Kazu
Re: [PATCH v5][makedumpfile 6/9] Add makedumpfile extensions support
Quoted 43 lines · HAGIO KAZUHITO(萩尾 一仁)
Quoted 38 lines · Tao Liu
Hi Stephen & Kazu,
Quoted 26 lines · Stephen Brennan
Quoted 14 lines · HAGIO KAZUHITO(萩尾 一仁)
extension.c line 58
Quoted 9 lines · Tao Liu
extension.c new file +9
-0+static bool init_kallsyms_btf (void ) +{ +int count; +bool ret = false; +/* We will load module's btf/kallsyms on demand */ +bool init_ksyms_module = false; +bool init_ktypes_module = false; + +if (check_ksyms_require_modname ("vmlinux" , &count)) {Thank you for the explanation [1].
Quoted 2 lines
so which code path adds "vmlinux" to the mods array before this line? I could not find it.
Hello Kazu,
In
kallsyms.cfrom patch 2, there is the line:INIT_MOD_SYM(vmlinux, _stext);This adds a struct
ksym_infoto the main makedumpfile executable's array, which requests symbol "_stext" from vmlinux.Quoted 7 lines
In
kallsyms.c"init_kernel_kallsyms()", again from patch 2, the first step is to call:register_ksym_section((char *)__start_init_ksyms, (char *)__stop_init_ksyms))Which is a function generated by the macro
REGISTER_SECTION(). The implementation will useadd_ksym_modname()whens it encounters this symbol, ensuring that "vmlinux" is part of the modname array.Quoted 2 lines
Thanks for the detailed info, yes, it is exactly how "vmlinux" is added to the array.
Quoted 8 lines
hmm, maybe I still misread it though..
The
init_kernel_kallsyms()is called _after_check_ksyms_require_modname(). so I've meant that if no extension adds "vmlinux" to the modname array, no one calls it. is it ok?if (check_ksyms_require_modname("vmlinux", &count)) { if (!init_kernel_kallsyms())
No, even if you create a extension which doesn't require symbols from "vmlinux", once you load it, the check_ksyms_require_modname() will still be true and init_kernel_kallsyms() is called.
The reason is, with the introduction of makedumpfile.ld, and the line "INIT_MOD_SYM(vmlinux, _stext);" in kallsyms.c, makedumpfile and extensions have a similar section structure, you can regard makedumpfile itself as an extension:
$ readelf -S makedumpfile | grep -E 'ksyms|ktypes' [25] .init_ksyms PROGBITS 000000000046ee90 0006de90 [26] .init_ktypes PROGBITS 000000000046eeb8 0006deb8 $ readelf -S extensions/sample.so | grep -E 'ksyms|ktypes' [24] .init_ksyms PROGBITS 0000000000003108 00002108 [25] .init_ktypes PROGBITS 0000000000003110 00002110
If makedumpfile doesn't load any extensions, then makedumpfile doesn't need to initialize its .init_ksyms/ktypes section, the behaviour stays the same as before;
If makedumpfile load at least one extensions, then the extension subsystem will register all .init_ksyms/ktypes sections, both extenison's and makedumpfile's.
This is like, extension say "I need sym1 & sym2 to be ready before I can run", then makedumpfile say "OK, I got it, but before I can resolve your sym1 & sym2, I need to resolve some symbol(e.g. vmlinux's _stext) formyself first". So check_ksyms_require_modname("vmlinux", &count) will be true.
Hope this helps.
Thanks,
Tao Liu
Quoted 31 lines
Thanks,
KazuQuoted 27 lines
Quoted 15 lines
Thanks,
StephenQuoted 10 lines
[1] https://lore.kernel.org/kexec/CAO7dBbUa_8H0fe+9bTNTQWjMVy9qCPiu0fo3g8XamiDS_Mr8eA@mail.gmail.com/
extension.c line 314
Quoted 4 lines
extension.c new file +4
-0+ +bool has_extension_loaded (void ) +{ +return extension_opts_len >0 ;If an extension is skipped, this looks > 0 with no extention.
handle_cbs_lenis better?Yeah, I agree,
has_extension_loaded()should also be updated tohas_extension_callback(). I will fix it in v6.Thanks,
Tao LiuQuoted 5 lines
Quoted 4 lines
Thanks,
Kazu
Re: [PATCH v5][makedumpfile 6/9] Add makedumpfile extensions support
Quoted 77 lines · Tao Liu
Quoted 43 lines · HAGIO KAZUHITO(萩尾 一仁)
Quoted 38 lines · Tao Liu
Hi Stephen & Kazu,
Quoted 26 lines · Stephen Brennan
Quoted 14 lines · HAGIO KAZUHITO(萩尾 一仁)
extension.c line 58
Quoted 9 lines · Tao Liu
extension.c new file +9
-0+static bool init_kallsyms_btf (void ) +{ +int count; +bool ret = false; +/* We will load module's btf/kallsyms on demand */ +bool init_ksyms_module = false; +bool init_ktypes_module = false; + +if (check_ksyms_require_modname ("vmlinux" , &count)) {Thank you for the explanation [1].
Quoted 2 lines
so which code path adds "vmlinux" to the mods array before this line? I could not find it.
Hello Kazu,
In
kallsyms.cfrom patch 2, there is the line:INIT_MOD_SYM(vmlinux, _stext);This adds a struct
ksym_infoto the main makedumpfile executable's array, which requests symbol "_stext" from vmlinux.Quoted 7 lines
In
kallsyms.c"init_kernel_kallsyms()", again from patch 2, the first step is to call:register_ksym_section((char *)__start_init_ksyms, (char *)__stop_init_ksyms))Which is a function generated by the macro
REGISTER_SECTION(). The implementation will useadd_ksym_modname()whens it encounters this symbol, ensuring that "vmlinux" is part of the modname array.Quoted 2 lines
Thanks for the detailed info, yes, it is exactly how "vmlinux" is added to the array.
Quoted 8 lines
hmm, maybe I still misread it though..
The
init_kernel_kallsyms()is called _after_check_ksyms_require_modname(). so I've meant that if no extension adds "vmlinux" to the modname array, no one calls it. is it ok?if (check_ksyms_require_modname("vmlinux", &count)) { if (!init_kernel_kallsyms())No, even if you create a extension which doesn't require symbols from "vmlinux", once you load it, the
check_ksyms_require_modname()will still be true andinit_kernel_kallsyms()is called.The reason is, with the introduction of makedumpfile.ld, and the line "
INIT_MOD_SYM(vmlinux, _stext);" inkallsyms.c, makedumpfile and extensions have a similar section structure, you can regard makedumpfile itself as an extension:$ readelf -S makedumpfile | grep -E 'ksyms|ktypes' [25] .init_ksyms PROGBITS 000000000046ee90 0006de90 [26] .init_ktypes PROGBITS 000000000046eeb8 0006deb8 $ readelf -S extensions/sample.so | grep -E 'ksyms|ktypes' [24] .init_ksyms PROGBITS 0000000000003108 00002108 [25] .init_ktypes PROGBITS 0000000000003110 00002110If makedumpfile doesn't load any extensions, then makedumpfile doesn't need to initialize its .init_ksyms/ktypes section, the behaviour stays the same as before;
If makedumpfile load at least one extensions, then the extension subsystem will register all .init_ksyms/ktypes sections, both extenison's and makedumpfile's.
Quoted 5 lines
This is like, extension say "I need sym1 & sym2 to be ready before I can run", then makedumpfile say "OK, I got it, but before I can resolve your sym1 & sym2, I need to resolve some symbol(e.g. vmlinux's
_stext) formyself first". Socheck_ksyms_require_modname("vmlinux", &count)will be true.
hmm, but as far as I've tested with "extensions/sample.c" on RHEL10.2:
1) add INIT_MOD_SYM(xfs, xfsstats) and GET_MOD_SYM(xfs, xfsstats)
ok
Loaded extension: /share/tmp/makedumpfile/extensions/sample.so
sample.so: The address of xfsstats is: ffffffffc0c0b540 <<--- ok
sample.so: The address of init_task is: ffffffffb4412940 sample.so: The size of task_struct is: 10240 bytes sample.so: The offset of member mm within task_struct is: 2704 bytes sample.so: The size of member mm within task_struct is: 8 bytes sample.so: Your kernel is using maple tree in mm_struct
2) remove all "vmlinux" entries (only INIT_MOD_SYM(xfs, xfsstats))
skipped
Loaded extension: /share/tmp/makedumpfile/extensions/sample.so check_required_ksyms_all_resolved: Symbol xfsstats in xfs not foundinit_extensions: Skip 1th extension
3) restore INIT_MOD_SYM(vmlinux, init_task) and GET_MOD_SYM(vmlinux, init_task)
still skipped
Loaded extension: /share/tmp/makedumpfile/extensions/sample.so check_required_ksyms_all_resolved: Symbol xfsstats in xfs not foundinit_extensions: Skip 1th extension
4) also restore INIT_MOD_STRUCT_MEMBER(vmlinux, task_struct, mm) and GET_MOD_STRUCT_MEMBER_SSIZE(vmlinux, task_struct, mm)
ok
Loaded extension: /share/tmp/makedumpfile/extensions/sample.so sample.so: The address of xfsstats is: ffffffffc0c0b540 sample.so: The address of init_task is: ffffffffb4412940 sample.so: The size of task_struct is: 10240 bytes
So, it looks like, with the current condition check in init_kallsyms_btf() (check_ksyms_require_modname and check_ktypes_require_modname), a module needs to use INIT_* for vmlinux's kallsyms and BTF.
I'm still missing something..?
Thanks,
Kazu
Re: [PATCH v5][makedumpfile 6/9] Add makedumpfile extensions support
Hi Kazu,
Quoted 98 lines · HAGIO KAZUHITO(萩尾 一仁)
Quoted 77 lines · Tao Liu
Quoted 43 lines · HAGIO KAZUHITO(萩尾 一仁)
Quoted 38 lines · Tao Liu
Hi Stephen & Kazu,
Quoted 26 lines · Stephen Brennan
Quoted 14 lines · HAGIO KAZUHITO(萩尾 一仁)
Review
Quoted 9 lines · Tao Liu
(quoted diff) +9
-0+static bool init_kallsyms_btf(void) +{ + int count; + bool ret = false; + /* We will load module's btf/kallsyms on demand */ + bool init_ksyms_module = false; + bool init_ktypes_module = false; + + if (check_ksyms_require_modname("vmlinux", &count)) {Thank you for the explanation [1].
Quoted 2 lines
so which code path adds "vmlinux" to the mods array before this line? I could not find it.
Hello Kazu,
In
kallsyms.cfrom patch 2, there is the line:INIT_MOD_SYM(vmlinux, _stext);This adds a struct
ksym_infoto the main makedumpfile executable's array, which requests symbol "_stext" from vmlinux.Quoted 7 lines
In
kallsyms.c"init_kernel_kallsyms()", again from patch 2, the first step is to call:register_ksym_section((char *)__start_init_ksyms, (char *)__stop_init_ksyms))Which is a function generated by the macro
REGISTER_SECTION(). The implementation will useadd_ksym_modname()whens it encounters this symbol, ensuring that "vmlinux" is part of the modname array.Quoted 2 lines
Thanks for the detailed info, yes, it is exactly how "vmlinux" is added to the array.
Quoted 8 lines
hmm, maybe I still misread it though..
The
init_kernel_kallsyms()is called _after_check_ksyms_require_modname(). so I've meant that if no extension adds "vmlinux" to the modname array, no one calls it. is it ok?if (check_ksyms_require_modname("vmlinux", &count)) { if (!init_kernel_kallsyms())No, even if you create a extension which doesn't require symbols from "vmlinux", once you load it, the
check_ksyms_require_modname()will still be true andinit_kernel_kallsyms()is called.The reason is, with the introduction of makedumpfile.ld, and the line "
INIT_MOD_SYM(vmlinux, _stext);" inkallsyms.c, makedumpfile and extensions have a similar section structure, you can regard makedumpfile itself as an extension:$ readelf -S makedumpfile | grep -E 'ksyms|ktypes' [25] .init_ksyms PROGBITS 000000000046ee90 0006de90 [26] .init_ktypes PROGBITS 000000000046eeb8 0006deb8 $ readelf -S extensions/sample.so | grep -E 'ksyms|ktypes' [24] .init_ksyms PROGBITS 0000000000003108 00002108 [25] .init_ktypes PROGBITS 0000000000003110 00002110If makedumpfile doesn't load any extensions, then makedumpfile doesn't need to initialize its .init_ksyms/ktypes section, the behaviour stays the same as before;
If makedumpfile load at least one extensions, then the extension subsystem will register all .init_ksyms/ktypes sections, both extenison's and makedumpfile's.Quoted 5 lines
This is like, extension say "I need sym1 & sym2 to be ready before I can run", then makedumpfile say "OK, I got it, but before I can resolve your sym1 & sym2, I need to resolve some symbol(e.g. vmlinux's
_stext) formyself first". Socheck_ksyms_require_modname("vmlinux", &count)will be true.hmm, but as far as I've tested with "
extensions/sample.c" on RHEL10.2:1) add
INIT_MOD_SYM(xfs, xfsstats)andGET_MOD_SYM(xfs, xfsstats)ok
Loaded extension: /share/tmp/makedumpfile/extensions/sample.so
sample.so: The address of xfsstats is:ffffffffc0c0b540<<--- ok
sample.so: The address ofinit_taskis:ffffffffb4412940sample.so: The size oftask_structis: 10240 bytes sample.so: The offset of member mm withintask_structis: 2704 bytes sample.so: The size of member mm withintask_structis: 8 bytes sample.so: Your kernel is using maple tree inmm_struct
Quoted 8 lines
2) remove all "vmlinux" entries (only
INIT_MOD_SYM(xfs, xfsstats))skipped
Loaded extension: /share/tmp/makedumpfile/extensions/sample.so
check_required_ksyms_all_resolved: Symbol xfsstats in xfs not foundinit_extensions: Skip 1th extension
This is a bug which shouldn't happen, thanks for pointing it out.
Quoted 8 lines
3) restore
INIT_MOD_SYM(vmlinux, init_task)andGET_MOD_SYM(vmlinux, init_task)still skipped
Loaded extension: /share/tmp/makedumpfile/extensions/sample.so
check_required_ksyms_all_resolved: Symbol xfsstats in xfs not foundinit_extensions: Skip 1th extension
Also a bug. I made a code modification based on the v5 to fix, could you please give a try? If no problem I will integrate it with v6.
btf_info.c +0 -4
@@ -202,10 +202,6 @@ bool init_kernel_btf(void) goto out; } -if (!register_ktype_section ((char *)__start_init_ktypes, - (char *)__stop_init_ktypes)) -return ret; - size = stop_btf - start_btf; buf = (char *)malloc (size);if (!buf) {
extension.c +14 -2
@@ -153,7 +153,7 @@ static void load_extensions(void) } } -static bool register_extension_sections (void ) +static bool register_ksyms_ktypes_sections (void ) {char *start, *stop;int i;
@@ -170,6 +170,18 @@ static bool register_extension_sections(void)if (!register_ktype_section (start, stop)) goto out; } +/* If no extensions, don't register makedumpfile's section */ +if (handle_cbs_len >0 ) { + start =dlsym (NULL ,"__start_init_ksyms" ); + stop =dlsym (NULL ,"__stop_init_ksyms" ); +if (!register_ksym_section (start, stop)) + goto out; + + start =dlsym (NULL ,"__start_init_ktypes" ); + stop =dlsym (NULL ,"__stop_init_ktypes" ); +if (!register_ktype_section (start, stop)) + goto out; + } ret = true; out:return ret;
@@ -260,7 +272,7 @@ void init_extensions(void)void (*init)(void );load_extensions (); -if (!register_extension_sections ()) +if (!register_ksyms_ktypes_sections ()) goto fail;if (!init_kallsyms_btf ()) goto fail;
kallsyms.c +0 -4
@@ -276,10 +276,6 @@ bool init_kernel_kallsyms(void)return ret; } -if (!register_ksym_section ((char *)__start_init_ksyms, - (char *)__stop_init_ksyms)) -return ret; -if (!readmem (VADDR ,SYMBOL (kallsyms_num_syms), &kallsyms_num_syms,sizeof (kallsyms_num_syms))) {ERRMSG ("Can't get kallsyms_num_syms!\n" );
Quoted 19 lines
4) also restore
INIT_MOD_STRUCT_MEMBER(vmlinux, task_struct, mm)andGET_MOD_STRUCT_MEMBER_SSIZE(vmlinux, task_struct, mm)ok
Loaded extension: /share/tmp/makedumpfile/extensions/sample.so sample.so: The address of xfsstats is:
ffffffffc0c0b540sample.so: The address ofinit_taskis:ffffffffb4412940sample.so: The size oftask_structis: 10240 bytesSo, it looks like, with the current condition check in
init_kallsyms_btf()(check_ksyms_require_modnameandcheck_ktypes_require_modname), a module needs to use INIT_* for vmlinux's kallsyms and BTF.I'm still missing something..?
Thanks,
Kazu
Re: [PATCH v5][makedumpfile 6/9] Add makedumpfile extensions support
Quoted 116 lines · Tao Liu
Hi Kazu,
Quoted 98 lines · HAGIO KAZUHITO(萩尾 一仁)
Quoted 77 lines · Tao Liu
Quoted 43 lines · HAGIO KAZUHITO(萩尾 一仁)
Quoted 38 lines · Tao Liu
Hi Stephen & Kazu,
Quoted 26 lines · Stephen Brennan
Quoted 14 lines · HAGIO KAZUHITO(萩尾 一仁)
Review
Quoted 9 lines · Tao Liu
(quoted diff) +9
-0+static bool init_kallsyms_btf(void) +{ + int count; + bool ret = false; + /* We will load module's btf/kallsyms on demand */ + bool init_ksyms_module = false; + bool init_ktypes_module = false; + + if (check_ksyms_require_modname("vmlinux", &count)) {Thank you for the explanation [1].
Quoted 2 lines
so which code path adds "vmlinux" to the mods array before this line? I could not find it.
Hello Kazu,
In
kallsyms.cfrom patch 2, there is the line:INIT_MOD_SYM(vmlinux, _stext);This adds a struct
ksym_infoto the main makedumpfile executable's array, which requests symbol "_stext" from vmlinux.Quoted 7 lines
In
kallsyms.c"init_kernel_kallsyms()", again from patch 2, the first step is to call:register_ksym_section((char *)__start_init_ksyms, (char *)__stop_init_ksyms))Which is a function generated by the macro
REGISTER_SECTION(). The implementation will useadd_ksym_modname()whens it encounters this symbol, ensuring that "vmlinux" is part of the modname array.Quoted 2 lines
Thanks for the detailed info, yes, it is exactly how "vmlinux" is added to the array.
Quoted 8 lines
hmm, maybe I still misread it though..
The
init_kernel_kallsyms()is called _after_check_ksyms_require_modname(). so I've meant that if no extension adds "vmlinux" to the modname array, no one calls it. is it ok?if (check_ksyms_require_modname("vmlinux", &count)) { if (!init_kernel_kallsyms())No, even if you create a extension which doesn't require symbols from "vmlinux", once you load it, the
check_ksyms_require_modname()will still be true andinit_kernel_kallsyms()is called.The reason is, with the introduction of makedumpfile.ld, and the line "
INIT_MOD_SYM(vmlinux, _stext);" inkallsyms.c, makedumpfile and extensions have a similar section structure, you can regard makedumpfile itself as an extension:$ readelf -S makedumpfile | grep -E 'ksyms|ktypes' [25] .init_ksyms PROGBITS 000000000046ee90 0006de90 [26] .init_ktypes PROGBITS 000000000046eeb8 0006deb8 $ readelf -S extensions/sample.so | grep -E 'ksyms|ktypes' [24] .init_ksyms PROGBITS 0000000000003108 00002108 [25] .init_ktypes PROGBITS 0000000000003110 00002110If makedumpfile doesn't load any extensions, then makedumpfile doesn't need to initialize its .init_ksyms/ktypes section, the behaviour stays the same as before;
If makedumpfile load at least one extensions, then the extension subsystem will register all .init_ksyms/ktypes sections, both extenison's and makedumpfile's.Quoted 5 lines
This is like, extension say "I need sym1 & sym2 to be ready before I can run", then makedumpfile say "OK, I got it, but before I can resolve your sym1 & sym2, I need to resolve some symbol(e.g. vmlinux's
_stext) formyself first". Socheck_ksyms_require_modname("vmlinux", &count)will be true.hmm, but as far as I've tested with "
extensions/sample.c" on RHEL10.2:1) add
INIT_MOD_SYM(xfs, xfsstats)andGET_MOD_SYM(xfs, xfsstats)ok
Loaded extension: /share/tmp/makedumpfile/extensions/sample.so
sample.so: The address of xfsstats is:ffffffffc0c0b540<<--- ok
sample.so: The address ofinit_taskis:ffffffffb4412940sample.so: The size oftask_structis: 10240 bytes sample.so: The offset of member mm withintask_structis: 2704 bytes sample.so: The size of member mm withintask_structis: 8 bytes sample.so: Your kernel is using maple tree inmm_structQuoted 8 lines
2) remove all "vmlinux" entries (only
INIT_MOD_SYM(xfs, xfsstats))skipped
Loaded extension: /share/tmp/makedumpfile/extensions/sample.so
check_required_ksyms_all_resolved: Symbol xfsstats in xfs not foundinit_extensions: Skip 1th extensionThis is a bug which shouldn't happen, thanks for pointing it out.
Quoted 4 lines
3) restore
INIT_MOD_SYM(vmlinux, init_task)andGET_MOD_SYM(vmlinux, init_task)still skipped
Quoted 7 lines
Quoted 4 lines
Loaded extension: /share/tmp/makedumpfile/extensions/sample.so
check_required_ksyms_all_resolved: Symbol xfsstats in xfs not foundinit_extensions: Skip 1th extensionAlso a bug. I made a code modification based on the v5 to fix, could you please give a try? If no problem I will integrate it with v6.
This patch tested ok and looks good and consistent with the condition checks in init_kallsyms_btf().
Thanks,
Kazu
Quoted 93 lines
btf_info.c +0
-4@@ -202,10 +202,6 @@ bool init_kernel_btf(void) goto out; } -if (!register_ktype_section ((char *)__start_init_ktypes, - (char *)__stop_init_ktypes)) -return ret; - size = stop_btf - start_btf; buf = (char *)malloc (size);if (!buf) {extension.c +14
-2@@ -153,7 +153,7 @@ static void load_extensions(void) } } -static bool register_extension_sections (void ) +static bool register_ksyms_ktypes_sections (void ) {char *start, *stop;int i;@@ -170,6 +170,18 @@ static bool register_extension_sections(void)if (!register_ktype_section (start, stop)) goto out; } +/* If no extensions, don't register makedumpfile's section */ +if (handle_cbs_len >0 ) { + start =dlsym (NULL ,"__start_init_ksyms" ); + stop =dlsym (NULL ,"__stop_init_ksyms" ); +if (!register_ksym_section (start, stop)) + goto out; + + start =dlsym (NULL ,"__start_init_ktypes" ); + stop =dlsym (NULL ,"__stop_init_ktypes" ); +if (!register_ktype_section (start, stop)) + goto out; + } ret = true; out:return ret;@@ -260,7 +272,7 @@ void init_extensions(void)void (*init)(void );load_extensions (); -if (!register_extension_sections ()) +if (!register_ksyms_ktypes_sections ()) goto fail;if (!init_kallsyms_btf ()) goto fail;kallsyms.c +0
-4@@ -276,10 +276,6 @@ bool init_kernel_kallsyms(void)return ret; } -if (!register_ksym_section ((char *)__start_init_ksyms, - (char *)__stop_init_ksyms)) -return ret; -if (!readmem (VADDR ,SYMBOL (kallsyms_num_syms), &kallsyms_num_syms,sizeof (kallsyms_num_syms))) {ERRMSG ("Can't get kallsyms_num_syms!\n" );Quoted 21 lines
4) also restore
INIT_MOD_STRUCT_MEMBER(vmlinux, task_struct, mm)andGET_MOD_STRUCT_MEMBER_SSIZE(vmlinux, task_struct, mm)ok
Loaded extension: /share/tmp/makedumpfile/extensions/sample.so sample.so: The address of xfsstats is:
ffffffffc0c0b540sample.so: The address ofinit_taskis:ffffffffb4412940sample.so: The size oftask_structis: 10240 bytesSo, it looks like, with the current condition check in
init_kallsyms_btf()(check_ksyms_require_modnameandcheck_ktypes_require_modname), a module needs to use INIT_* for vmlinux's kallsyms and BTF.I'm still missing something..?
Thanks,
Kazu
Re: [PATCH v5][makedumpfile 6/9] Add makedumpfile extensions support
Hi Kazu,
Quoted 123 lines · HAGIO KAZUHITO(萩尾 一仁)
Quoted 121 lines · Tao Liu
Hi Kazu,
Quoted 98 lines · HAGIO KAZUHITO(萩尾 一仁)
Quoted 77 lines · Tao Liu
Quoted 43 lines · HAGIO KAZUHITO(萩尾 一仁)
Quoted 38 lines · Tao Liu
Hi Stephen & Kazu,
Quoted 26 lines · Stephen Brennan
Quoted 14 lines · HAGIO KAZUHITO(萩尾 一仁)
Review
Quoted 9 lines · Tao Liu
(quoted diff) +9
-0+static bool init_kallsyms_btf(void) +{ + int count; + bool ret = false; + /* We will load module's btf/kallsyms on demand */ + bool init_ksyms_module = false; + bool init_ktypes_module = false; + + if (check_ksyms_require_modname("vmlinux", &count)) {Thank you for the explanation [1].
Quoted 2 lines
so which code path adds "vmlinux" to the mods array before this line? I could not find it.
Hello Kazu,
In
kallsyms.cfrom patch 2, there is the line:INIT_MOD_SYM(vmlinux, _stext);This adds a struct
ksym_infoto the main makedumpfile executable's array, which requests symbol "_stext" from vmlinux.Quoted 7 lines
In
kallsyms.c"init_kernel_kallsyms()", again from patch 2, the first step is to call:register_ksym_section((char *)__start_init_ksyms, (char *)__stop_init_ksyms))Which is a function generated by the macro
REGISTER_SECTION(). The implementation will useadd_ksym_modname()whens it encounters this symbol, ensuring that "vmlinux" is part of the modname array.Quoted 2 lines
Thanks for the detailed info, yes, it is exactly how "vmlinux" is added to the array.
Quoted 8 lines
hmm, maybe I still misread it though..
The
init_kernel_kallsyms()is called _after_check_ksyms_require_modname(). so I've meant that if no extension adds "vmlinux" to the modname array, no one calls it. is it ok?if (check_ksyms_require_modname("vmlinux", &count)) { if (!init_kernel_kallsyms())No, even if you create a extension which doesn't require symbols from "vmlinux", once you load it, the
check_ksyms_require_modname()will still be true andinit_kernel_kallsyms()is called.The reason is, with the introduction of makedumpfile.ld, and the line "
INIT_MOD_SYM(vmlinux, _stext);" inkallsyms.c, makedumpfile and extensions have a similar section structure, you can regard makedumpfile itself as an extension:$ readelf -S makedumpfile | grep -E 'ksyms|ktypes' [25] .init_ksyms PROGBITS 000000000046ee90 0006de90 [26] .init_ktypes PROGBITS 000000000046eeb8 0006deb8 $ readelf -S extensions/sample.so | grep -E 'ksyms|ktypes' [24] .init_ksyms PROGBITS 0000000000003108 00002108 [25] .init_ktypes PROGBITS 0000000000003110 00002110If makedumpfile doesn't load any extensions, then makedumpfile doesn't need to initialize its .init_ksyms/ktypes section, the behaviour stays the same as before;
If makedumpfile load at least one extensions, then the extension subsystem will register all .init_ksyms/ktypes sections, both extenison's and makedumpfile's.Quoted 5 lines
This is like, extension say "I need sym1 & sym2 to be ready before I can run", then makedumpfile say "OK, I got it, but before I can resolve your sym1 & sym2, I need to resolve some symbol(e.g. vmlinux's
_stext) formyself first". Socheck_ksyms_require_modname("vmlinux", &count)will be true.hmm, but as far as I've tested with "
extensions/sample.c" on RHEL10.2:1) add
INIT_MOD_SYM(xfs, xfsstats)andGET_MOD_SYM(xfs, xfsstats)ok
Loaded extension: /share/tmp/makedumpfile/extensions/sample.so
sample.so: The address of xfsstats is:ffffffffc0c0b540<<--- ok
sample.so: The address ofinit_taskis:ffffffffb4412940sample.so: The size oftask_structis: 10240 bytes sample.so: The offset of member mm withintask_structis: 2704 bytes sample.so: The size of member mm withintask_structis: 8 bytes sample.so: Your kernel is using maple tree inmm_structQuoted 8 lines
2) remove all "vmlinux" entries (only
INIT_MOD_SYM(xfs, xfsstats))skipped
Loaded extension: /share/tmp/makedumpfile/extensions/sample.so
check_required_ksyms_all_resolved: Symbol xfsstats in xfs not foundinit_extensions: Skip 1th extensionThis is a bug which shouldn't happen, thanks for pointing it out.
Quoted 8 lines
3) restore
INIT_MOD_SYM(vmlinux, init_task)andGET_MOD_SYM(vmlinux, init_task)still skipped
Loaded extension: /share/tmp/makedumpfile/extensions/sample.so
check_required_ksyms_all_resolved: Symbol xfsstats in xfs not foundinit_extensions: Skip 1th extension
Quoted 5 lines
Quoted 2 lines
Also a bug. I made a code modification based on the v5 to fix, could you please give a try? If no problem I will integrate it with v6.
This patch tested ok and looks good and consistent with the condition checks in
init_kallsyms_btf().
Glad to know that! I'm working on v6 and will post it upstream later.
Thanks,
Tao Liu
Quoted 98 lines
Thanks,
KazuQuoted 93 lines
btf_info.c +0
-4@@ -202,10 +202,6 @@ bool init_kernel_btf(void) goto out; } -if (!register_ktype_section ((char *)__start_init_ktypes, - (char *)__stop_init_ktypes)) -return ret; - size = stop_btf - start_btf; buf = (char *)malloc (size);if (!buf) {extension.c +14
-2@@ -153,7 +153,7 @@ static void load_extensions(void) } } -static bool register_extension_sections (void ) +static bool register_ksyms_ktypes_sections (void ) {char *start, *stop;int i;@@ -170,6 +170,18 @@ static bool register_extension_sections(void)if (!register_ktype_section (start, stop)) goto out; } +/* If no extensions, don't register makedumpfile's section */ +if (handle_cbs_len >0 ) { + start =dlsym (NULL ,"__start_init_ksyms" ); + stop =dlsym (NULL ,"__stop_init_ksyms" ); +if (!register_ksym_section (start, stop)) + goto out; + + start =dlsym (NULL ,"__start_init_ktypes" ); + stop =dlsym (NULL ,"__stop_init_ktypes" ); +if (!register_ktype_section (start, stop)) + goto out; + } ret = true; out:return ret;@@ -260,7 +272,7 @@ void init_extensions(void)void (*init)(void );load_extensions (); -if (!register_extension_sections ()) +if (!register_ksyms_ktypes_sections ()) goto fail;if (!init_kallsyms_btf ()) goto fail;kallsyms.c +0
-4@@ -276,10 +276,6 @@ bool init_kernel_kallsyms(void)return ret; } -if (!register_ksym_section ((char *)__start_init_ksyms, - (char *)__stop_init_ksyms)) -return ret; -if (!readmem (VADDR ,SYMBOL (kallsyms_num_syms), &kallsyms_num_syms,sizeof (kallsyms_num_syms))) {ERRMSG ("Can't get kallsyms_num_syms!\n" );Quoted 21 lines
4) also restore
INIT_MOD_STRUCT_MEMBER(vmlinux, task_struct, mm)andGET_MOD_STRUCT_MEMBER_SSIZE(vmlinux, task_struct, mm)ok
Loaded extension: /share/tmp/makedumpfile/extensions/sample.so sample.so: The address of xfsstats is:
ffffffffc0c0b540sample.so: The address ofinit_taskis:ffffffffb4412940sample.so: The size oftask_structis: 10240 bytesSo, it looks like, with the current condition check in
init_kallsyms_btf()(check_ksyms_require_modnameandcheck_ktypes_require_modname), a module needs to use INIT_* for vmlinux's kallsyms and BTF.I'm still missing something..?
Thanks,
Kazu
PATCH v5 7/9 Add sample extension as an example reference
In reply to Tao LiuThis patch will add a sample.c as an example extension for code reference. The extension will do nothing related to page filtering, but print a few kernel symbol/type information.
--- extensions/Makefile | 2 +- extensions/sample.c | 69 +++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 70 insertions(+), 1 deletion(-) create mode 100644 extensions/sample.c
extensions/Makefile +1 -1
@@ -1,5 +1,5 @@ CC ?= gcc -CONTRIB_SO := +CONTRIB_SO := sample.so all: $(CONTRIB_SO)
extensions/sample.c new file +69 -0
69 lines not on this page · raw message
Signature · 1 line
2.47.0
PATCH v5 8/9 Doc: Add --extension option to makedumpfile manual
In reply to Tao Liu--- README | 6 ++++++ makedumpfile.8.in | 11 ++++++++++- 2 files changed, 16 insertions(+), 1 deletion(-)
README +6 -0
@@ -65,6 +65,12 @@ # make USEZSTD=on ; make install The user has to prepare zstd library. + - Build with extension support: + # make EXTENSION=on ; make install + The user has to prepare bpf library with version(>=v1.4.0). + Note: the extension will only work for kernels with DEBUG_INFO_BTF + & KALLSYMS enabled. + - Build the extension module for --eppic option. # make eppic_makedumpfile.so The user has to prepare eppic library from the following site:
makedumpfile.8.in +10 -1
@@ -2,7 +2,7 @@ .SH NAME makedumpfile \- make a small dumpfile of kdump .SH SYNOPSIS -\fBmakedumpfile\fR [\fIOPTION\fR] [\-x \fIVMLINUX\fR|\-i \fIVMCOREINFO\fR] \fIVMCORE\fR \fIDUMPFILE\fR +\fBmakedumpfile\fR [\fIOPTION\fR] [\-x \fIVMLINUX\fR|\-i \fIVMCOREINFO\fR] [--extension \fIEXTENSION1.SO\fR [--extension \fIEXTENSION2.SO\fR ..]] \fIVMCORE\fR \fIDUMPFILE\fR .br \fBmakedumpfile\fR \-F [\fIOPTION\fR] [\-x \fIVMLINUX\fR|\-i \fIVMCOREINFO\fR] \fIVMCORE\fR .br
@@ -664,6 +664,15 @@ This option cannot be used with the --dump-dmesg, --reassemble and -g options. Display report messages. This is an alternative to enabling bit 4 in the level provided to --message-level. +.TP +\fB\-\-extension\fR \fIEXTENSION.SO\fR +Load makedumpfile extensions. By using extensions, users can programmably +customize mm page keeping / discard, alone with the traditional page-flag +(-d dump_level) based filtering. The option can be used multiple times to +select various extensions. NOTE: this option needs EXTENSION=on when build +makedumpfile with libbpf(>=v1.4.0). Also this option only works for kernels +with DEBUG_INFO_BTF & KALLSYMS enabled. + .SH ENVIRONMENT VARIABLES .TP 8
Signature · 1 line
2.47.0
PATCH v5 9/9 Add amdgpu mm pages filtering extension
In reply to Tao LiuThis patch will introduce maple_tree & amdgpu mm page filtering extension, those mm pages allocated to amdgpu will be discarded from vmcore, in order to shrink vmcore size since mm pages allocated to amdgpu are useless to kernel crash and may contain sensitive data.
--- extensions/Makefile | 4 +- extensions/amdgpu_filter.c | 221 +++++++++++++++++++++++++ extensions/maple_tree.c | 328 +++++++++++++++++++++++++++++++++++++ extensions/maple_tree.h | 7 + 4 files changed, 559 insertions(+), 1 deletion(-) create mode 100644 extensions/amdgpu_filter.c create mode 100644 extensions/maple_tree.c create mode 100644 extensions/maple_tree.h
extensions/Makefile +3 -1
@@ -1,8 +1,10 @@ CC ?= gcc -CONTRIB_SO := sample.so +CONTRIB_SO := sample.so amdgpu_filter.so all: $(CONTRIB_SO) +amdgpu_filter.so: maple_tree.c + $(CONTRIB_SO): %.so: %.c $(CC) -O2 -g -fPIC -shared -Wl,-T,../makedumpfile.ld -o $@ $^
extensions/amdgpu_filter.c new file +221 -0
221 lines not on this page · raw message
extensions/maple_tree.c new file +328 -0
328 lines not on this page · raw message
extensions/maple_tree.h new file +7 -0
@@ -0,0 +1,7 @@ +#ifndef _MAPLE_TREE_H +#define _MAPLE_TREE_H +#include <stdbool.h> +unsigned long *mt_dump (unsigned long mt,int *array_len); +bool maple_init (void ); +#endif /* _MAPLE_TREE_H */ +
Signature · 1 line
2.47.0
Kindly ping, Any comments for this patchset?
Thanks,
Tao Liu
Quoted 187 lines · Tao Liu
A) This patchset will introduce the following features to makedumpfile:
1) Add .so extension support to makedumpfile
2) Enable btf and kallsyms for symbol type and address resolving.B) The purpose of the features are:
1) Currently makedumpfile filters mm pages based on page flags, because flags can help to determine one page's usage. But this page-flag-checking method lacks of flexibility in certain cases, e.g. if we want to filter those mm pages occupied by GPU during vmcore dumping due to:
a) GPU may be taking a large memory and contains sensitive data;
b) GPU mm pages have no relations to kernel crash and useless for vmcore analysis.But there is no GPU mm page specific flags, and apparently we don't need to create one just for kdump use. A programmable filtering tool is more suitable for such cases. In addition, different GPU vendors may use different ways for mm pages allocating, programmable filtering is better than hard coding these GPU specific logics into makedumpfile in this case.
2) Currently makedumpfile already contains a programmable filtering tool, aka eppic script, which allows user to write customized code for data erasing. However it has the following drawbacks:
a) cannot do mm page filtering.
b) need to access to debuginfo of both kernel and modules, which is not applicable in the 2nd kernel.
c) eppic library has memory leaks which are not all resolved [1]. This is not acceptable in 2nd kernel.makedumpfile need to resolve the dwarf data from debuginfo, to get symbols types and addresses. In recent kernel there are dwarf alternatives such as btf/kallsyms which can be used for this purpose. And btf/kallsyms info are already packed within vmcore, so we can use it directly.
With these, this patchset introduces makedumpfile extensions, which is based on btf/kallsyms symbol resolving, and is programmable for mm page filtering. The following section shows its usage and performance, please note the tests are performed in 1st kernel.
3) Compile and run makedumpfile extensions:
$ make LINKTYPE=dynamic USELZO=on USESNAPPY=on USEZSTD=on EXTENSION=on $ make extensions$ /usr/bin/time -v ./makedumpfile -d 31 -l /var/crash/127.0.0.1-2025-06-10-18\:03\:12/vmcore /tmp/extension.out --extension amdgpu_filter.so Loaded extension: ./extensions/amdgpu_filter.so makedumpfile Completed. User time (seconds): 5.08 System time (seconds): 0.84 Percent of CPU this job got: 99% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:05.95 Maximum resident set size (kbytes): 17360 ...To contrast with eppic script of v2 [2]:
$ /usr/bin/time -v ./makedumpfile -d 31 -l /var/crash/127.0.0.1-2025-06-10-18\:03\:12/vmcore /tmp/eppic.out --eppic eppic_scripts/filter_amdgpu_mm_pages.c makedumpfile Completed. User time (seconds): 8.23 System time (seconds): 0.88 Elapsed (wall clock) time (h:mm:ss or m:ss): 0:09.16 Maximum resident set size (kbytes): 57128 ...-rw------- 1 root root 367475074 Jan 19 19:01
/tmp/extension.out
-rw------- 1 root root 367475074 Jan 19 19:48/tmp/eppic.out
-rw------- 1 root root 387181418 Jun 10 18:03 /var/crash/127.0.0.1-2025-06-10-18:03:12/vmcoreC) Discussion:
1) GPU types: Currently only tested with amdgpu's mm page filtering, others are not tested.
2) OS: The code can work on rhel-10+/rhel9.5+ on x86_64/arm64/s390/ppc64. rhel8.x is not supported, others are not tested.D) Testing:
If you don't want to create your vmcore, you can find a vmcore which I created with amdgpu mm pages unfiltered [3], the amdgpu mm pages are allocated by program [4]. You can use the vmcore in 1st kernel to filter the amdgpu mm pages by the previous performance testing cmdline. To verify the pages are filtered in crash:
Unfiltered:
crash> search -c "!QAZXSW@#EDC"
ffff96b7fa800000: !QAZXSW@#EDCXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
ffff96b87c800000: !QAZXSW@#EDCXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX crash> rdffff96b7fa800000
ffff96b7fa800000:405753585a415121!QAZXSW@
crash> rdffff96b87c800000
ffff96b87c800000:405753585a415121!QAZXSW@Filtered:
crash> search -c "!QAZXSW@#EDC"
crash> rdffff96b7fa800000
rd: page excluded: kernel virtual address:ffff96b7fa800000type: "64-bit KVADDR" crash> rdffff96b87c800000
rd: page excluded: kernel virtual address:ffff96b87c800000type: "64-bit KVADDR"[1]: https://github.com/lucchouina/eppic/pull/32
[2]: https://lore.kernel.org/kexec/[email protected]/
[3]: https://people.redhat.com/~ltao/core/vmcore
[4]: https://gist.github.com/liutgnu/a8cbce1c666452f1530e1410d1f352dfv5 -> v4:
1) Add "make EXTENSION=on" switch to customize the extension feature.
2) Clean up macros withinbtf_info.h.
3) Updated doc and a sample extension to demo the extension usage.
4) Use MSG()/ERRMSG() rather thanfprintf().
5) Add return value check forreadmem().
6) Allow "makedumpfile -d 1 --extension ext.so" to enter extension.
7) The patches are organized as follows:--- <customization specific> ---
9. Add amdgpu mm pages filtering extension--- <code should be merged> ---
8. Doc: Add --extension option to makedumpfile manual
7. Add sample extension as an example reference
6. Add makedumpfile extensions support
5. Implement kernel module's btf resolving
4. Implement kernel module's kallsyms resolving
3. Implement kernel btf resolving
2. Implement kernel kallsyms resolving
1. Reserve sections for makedumpfile and extenionsPatch 9 is customization specific, merging depends on the strategy of maintenance.
Patch 1 ~ 8 are common code which should be merged with makedumpfile.Link to v4: https://lore.kernel.org/kexec/[email protected]/
Link to v3: https://lore.kernel.org/kexec/[email protected]/
Link to v2: https://lore.kernel.org/kexec/[email protected]/
Link to v1: https://lore.kernel.org/kexec/[email protected]/Tao Liu (9):
Reserve sections for makedumpfile and extenions Implement kernel kallsyms resolving Implement kernel btf resolving
Implement kernel module's kallsyms resolving Implement kernel module's btf resolving Add makedumpfile extensions support
Add sample extension as an example reference
Doc: Add --extension option to makedumpfile manual Add amdgpu mm pages filtering extensionMakefile | 15 +- README | 6 + btf_info.c | 375 +++++++++++++++++++++++++ btf_info.h | 77 ++++++ extension.c | 338 ++++++++++++++++++++++ extension.h | 16 ++ extensions/Makefile | 13 + extensions/amdgpu_filter.c | 221 +++++++++++++++ extensions/maple_tree.c | 328 ++++++++++++++++++++++ extensions/maple_tree.h | 7 + extensions/sample.c | 69 +++++ kallsyms.c | 554 +++++++++++++++++++++++++++++++++++++ kallsyms.h | 87 ++++++ makedumpfile.8.in | 11 +- makedumpfile.c | 44 ++- makedumpfile.h | 12 + makedumpfile.ld | 16 ++ 17 files changed, 2180 insertions(+), 9 deletions(-) create mode 100644 btf_info.c create mode 100644 btf_info.h create mode 100644 extension.c create mode 100644 extension.h create mode 100644 extensions/Makefile create mode 100644 extensions/amdgpu_filter.c create mode 100644 extensions/maple_tree.c create mode 100644 extensions/maple_tree.h create mode 100644 extensions/sample.c create mode 100644 kallsyms.c create mode 100644 kallsyms.h create mode 100644 makedumpfile.ld--
2.47.0
Quoted 132 lines · Tao Liu
A) This patchset will introduce the following features to makedumpfile:
1) Add .so extension support to makedumpfile
2) Enable btf and kallsyms for symbol type and address resolving.B) The purpose of the features are:
1) Currently makedumpfile filters mm pages based on page flags, because flags can help to determine one page's usage. But this page-flag-checking method lacks of flexibility in certain cases, e.g. if we want to filter those mm pages occupied by GPU during vmcore dumping due to:
a) GPU may be taking a large memory and contains sensitive data;
b) GPU mm pages have no relations to kernel crash and useless for vmcore analysis.But there is no GPU mm page specific flags, and apparently we don't need to create one just for kdump use. A programmable filtering tool is more suitable for such cases. In addition, different GPU vendors may use different ways for mm pages allocating, programmable filtering is better than hard coding these GPU specific logics into makedumpfile in this case.
2) Currently makedumpfile already contains a programmable filtering tool, aka eppic script, which allows user to write customized code for data erasing. However it has the following drawbacks:
a) cannot do mm page filtering.
b) need to access to debuginfo of both kernel and modules, which is not applicable in the 2nd kernel.
c) eppic library has memory leaks which are not all resolved [1]. This is not acceptable in 2nd kernel.makedumpfile need to resolve the dwarf data from debuginfo, to get symbols types and addresses. In recent kernel there are dwarf alternatives such as btf/kallsyms which can be used for this purpose. And btf/kallsyms info are already packed within vmcore, so we can use it directly.
With these, this patchset introduces makedumpfile extensions, which is based on btf/kallsyms symbol resolving, and is programmable for mm page filtering. The following section shows its usage and performance, please note the tests are performed in 1st kernel.
3) Compile and run makedumpfile extensions:
$ make LINKTYPE=dynamic USELZO=on USESNAPPY=on USEZSTD=on EXTENSION=on $ make extensions$ /usr/bin/time -v ./makedumpfile -d 31 -l /var/crash/127.0.0.1-2025-06-10-18\:03\:12/vmcore /tmp/extension.out --extension amdgpu_filter.so Loaded extension: ./extensions/amdgpu_filter.so makedumpfile Completed. User time (seconds): 5.08 System time (seconds): 0.84 Percent of CPU this job got: 99% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:05.95 Maximum resident set size (kbytes): 17360 ...To contrast with eppic script of v2 [2]:
$ /usr/bin/time -v ./makedumpfile -d 31 -l /var/crash/127.0.0.1-2025-06-10-18\:03\:12/vmcore /tmp/eppic.out --eppic eppic_scripts/filter_amdgpu_mm_pages.c makedumpfile Completed. User time (seconds): 8.23 System time (seconds): 0.88 Elapsed (wall clock) time (h:mm:ss or m:ss): 0:09.16 Maximum resident set size (kbytes): 57128 ...-rw------- 1 root root 367475074 Jan 19 19:01
/tmp/extension.out
-rw------- 1 root root 367475074 Jan 19 19:48/tmp/eppic.out
-rw------- 1 root root 387181418 Jun 10 18:03 /var/crash/127.0.0.1-2025-06-10-18:03:12/vmcoreC) Discussion:
1) GPU types: Currently only tested with amdgpu's mm page filtering, others are not tested.
2) OS: The code can work on rhel-10+/rhel9.5+ on x86_64/arm64/s390/ppc64. rhel8.x is not supported, others are not tested.D) Testing:
If you don't want to create your vmcore, you can find a vmcore which I created with amdgpu mm pages unfiltered [3], the amdgpu mm pages are allocated by program [4]. You can use the vmcore in 1st kernel to filter the amdgpu mm pages by the previous performance testing cmdline. To verify the pages are filtered in crash:
Unfiltered:
crash> search -c "!QAZXSW@#EDC"
ffff96b7fa800000: !QAZXSW@#EDCXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
ffff96b87c800000: !QAZXSW@#EDCXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX crash> rdffff96b7fa800000
ffff96b7fa800000:405753585a415121!QAZXSW@
crash> rdffff96b87c800000
ffff96b87c800000:405753585a415121!QAZXSW@Filtered:
crash> search -c "!QAZXSW@#EDC"
crash> rdffff96b7fa800000
rd: page excluded: kernel virtual address:ffff96b7fa800000type: "64-bit KVADDR" crash> rdffff96b87c800000
rd: page excluded: kernel virtual address:ffff96b87c800000type: "64-bit KVADDR"[1]: https://github.com/lucchouina/eppic/pull/32
[2]: https://lore.kernel.org/kexec/[email protected]/
[3]: https://people.redhat.com/~ltao/core/vmcore
[4]: https://gist.github.com/liutgnu/a8cbce1c666452f1530e1410d1f352dfv5 -> v4:
1) Add "make EXTENSION=on" switch to customize the extension feature.
2) Clean up macros withinbtf_info.h.
3) Updated doc and a sample extension to demo the extension usage.
4) Use MSG()/ERRMSG() rather thanfprintf().
5) Add return value check forreadmem().
6) Allow "makedumpfile -d 1 --extension ext.so" to enter extension.
7) The patches are organized as follows:--- <customization specific> ---
9. Add amdgpu mm pages filtering extension--- <code should be merged> ---
8. Doc: Add --extension option to makedumpfile manual
7. Add sample extension as an example reference
6. Add makedumpfile extensions support
5. Implement kernel module's btf resolving
4. Implement kernel module's kallsyms resolving
3. Implement kernel btf resolving
2. Implement kernel kallsyms resolving
1. Reserve sections for makedumpfile and extenions
Quoted 3 lines
Patch 9 is customization specific, merging depends on the strategy of maintenance.
Patch 1 ~ 8 are common code which should be merged with makedumpfile.
Hi Tao,
Just to be thorough, I did re-read the code of each patch, in addition to reviewing the diff from v4 to v5.
I've also re-based my userspace stack extension on this new version. I think it is ready to go.
Thanks,
Stephen
Quoted 49 lines
Link to v4: https://lore.kernel.org/kexec/[email protected]/
Link to v3: https://lore.kernel.org/kexec/[email protected]/
Link to v2: https://lore.kernel.org/kexec/[email protected]/
Link to v1: https://lore.kernel.org/kexec/[email protected]/Tao Liu (9):
Reserve sections for makedumpfile and extenions Implement kernel kallsyms resolving Implement kernel btf resolving
Implement kernel module's kallsyms resolving Implement kernel module's btf resolving Add makedumpfile extensions support
Add sample extension as an example reference
Doc: Add --extension option to makedumpfile manual Add amdgpu mm pages filtering extensionMakefile | 15 +- README | 6 + btf_info.c | 375 +++++++++++++++++++++++++ btf_info.h | 77 ++++++ extension.c | 338 ++++++++++++++++++++++ extension.h | 16 ++ extensions/Makefile | 13 + extensions/amdgpu_filter.c | 221 +++++++++++++++ extensions/maple_tree.c | 328 ++++++++++++++++++++++ extensions/maple_tree.h | 7 + extensions/sample.c | 69 +++++ kallsyms.c | 554 +++++++++++++++++++++++++++++++++++++ kallsyms.h | 87 ++++++ makedumpfile.8.in | 11 +- makedumpfile.c | 44 ++- makedumpfile.h | 12 + makedumpfile.ld | 16 ++ 17 files changed, 2180 insertions(+), 9 deletions(-) create mode 100644 btf_info.c create mode 100644 btf_info.h create mode 100644 extension.c create mode 100644 extension.h create mode 100644 extensions/Makefile create mode 100644 extensions/amdgpu_filter.c create mode 100644 extensions/maple_tree.c create mode 100644 extensions/maple_tree.h create mode 100644 extensions/sample.c create mode 100644 kallsyms.c create mode 100644 kallsyms.h create mode 100644 makedumpfile.ldSignature · 1 line
2.47.0
Hi Stephen,
Thanks a lot for your code review, I really appreciate it! And I have enjoyed the process of cooperating with you on the code discussion, design, and implementation. Thanks again for your help on this! :)
Hi maintainers,
Just a kind ask: any comments on the v5 patchset? I'm happy to make further modifications. Thanks!
Thanks,
Tao Liu
Quoted 201 lines · Stephen Brennan
Quoted 132 lines · Tao Liu
A) This patchset will introduce the following features to makedumpfile:
1) Add .so extension support to makedumpfile
2) Enable btf and kallsyms for symbol type and address resolving.B) The purpose of the features are:
1) Currently makedumpfile filters mm pages based on page flags, because flags can help to determine one page's usage. But this page-flag-checking method lacks of flexibility in certain cases, e.g. if we want to filter those mm pages occupied by GPU during vmcore dumping due to:
a) GPU may be taking a large memory and contains sensitive data;
b) GPU mm pages have no relations to kernel crash and useless for vmcore analysis.But there is no GPU mm page specific flags, and apparently we don't need to create one just for kdump use. A programmable filtering tool is more suitable for such cases. In addition, different GPU vendors may use different ways for mm pages allocating, programmable filtering is better than hard coding these GPU specific logics into makedumpfile in this case.
2) Currently makedumpfile already contains a programmable filtering tool, aka eppic script, which allows user to write customized code for data erasing. However it has the following drawbacks:
a) cannot do mm page filtering.
b) need to access to debuginfo of both kernel and modules, which is not applicable in the 2nd kernel.
c) eppic library has memory leaks which are not all resolved [1]. This is not acceptable in 2nd kernel.makedumpfile need to resolve the dwarf data from debuginfo, to get symbols types and addresses. In recent kernel there are dwarf alternatives such as btf/kallsyms which can be used for this purpose. And btf/kallsyms info are already packed within vmcore, so we can use it directly.
With these, this patchset introduces makedumpfile extensions, which is based on btf/kallsyms symbol resolving, and is programmable for mm page filtering. The following section shows its usage and performance, please note the tests are performed in 1st kernel.
3) Compile and run makedumpfile extensions:
$ make LINKTYPE=dynamic USELZO=on USESNAPPY=on USEZSTD=on EXTENSION=on $ make extensions$ /usr/bin/time -v ./makedumpfile -d 31 -l /var/crash/127.0.0.1-2025-06-10-18\:03\:12/vmcore /tmp/extension.out --extension amdgpu_filter.so Loaded extension: ./extensions/amdgpu_filter.so makedumpfile Completed. User time (seconds): 5.08 System time (seconds): 0.84 Percent of CPU this job got: 99% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:05.95 Maximum resident set size (kbytes): 17360 ...To contrast with eppic script of v2 [2]:
$ /usr/bin/time -v ./makedumpfile -d 31 -l /var/crash/127.0.0.1-2025-06-10-18\:03\:12/vmcore /tmp/eppic.out --eppic eppic_scripts/filter_amdgpu_mm_pages.c makedumpfile Completed. User time (seconds): 8.23 System time (seconds): 0.88 Elapsed (wall clock) time (h:mm:ss or m:ss): 0:09.16 Maximum resident set size (kbytes): 57128 ...-rw------- 1 root root 367475074 Jan 19 19:01
/tmp/extension.out
-rw------- 1 root root 367475074 Jan 19 19:48/tmp/eppic.out
-rw------- 1 root root 387181418 Jun 10 18:03 /var/crash/127.0.0.1-2025-06-10-18:03:12/vmcoreC) Discussion:
1) GPU types: Currently only tested with amdgpu's mm page filtering, others are not tested.
2) OS: The code can work on rhel-10+/rhel9.5+ on x86_64/arm64/s390/ppc64. rhel8.x is not supported, others are not tested.D) Testing:
If you don't want to create your vmcore, you can find a vmcore which I created with amdgpu mm pages unfiltered [3], the amdgpu mm pages are allocated by program [4]. You can use the vmcore in 1st kernel to filter the amdgpu mm pages by the previous performance testing cmdline. To verify the pages are filtered in crash:
Unfiltered:
crash> search -c "!QAZXSW@#EDC"
ffff96b7fa800000: !QAZXSW@#EDCXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
ffff96b87c800000: !QAZXSW@#EDCXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX crash> rdffff96b7fa800000
ffff96b7fa800000:405753585a415121!QAZXSW@
crash> rdffff96b87c800000
ffff96b87c800000:405753585a415121!QAZXSW@Filtered:
crash> search -c "!QAZXSW@#EDC"
crash> rdffff96b7fa800000
rd: page excluded: kernel virtual address:ffff96b7fa800000type: "64-bit KVADDR" crash> rdffff96b87c800000
rd: page excluded: kernel virtual address:ffff96b87c800000type: "64-bit KVADDR"[1]: https://github.com/lucchouina/eppic/pull/32
[2]: https://lore.kernel.org/kexec/[email protected]/
[3]: https://people.redhat.com/~ltao/core/vmcore
[4]: https://gist.github.com/liutgnu/a8cbce1c666452f1530e1410d1f352dfv5 -> v4:
1) Add "make EXTENSION=on" switch to customize the extension feature.
2) Clean up macros withinbtf_info.h.
3) Updated doc and a sample extension to demo the extension usage.
4) Use MSG()/ERRMSG() rather thanfprintf().
5) Add return value check forreadmem().
6) Allow "makedumpfile -d 1 --extension ext.so" to enter extension.
7) The patches are organized as follows:--- <customization specific> ---
9. Add amdgpu mm pages filtering extension--- <code should be merged> ---
8. Doc: Add --extension option to makedumpfile manual
7. Add sample extension as an example reference
6. Add makedumpfile extensions support
5. Implement kernel module's btf resolving
4. Implement kernel module's kallsyms resolving
3. Implement kernel btf resolving
2. Implement kernel kallsyms resolving
1. Reserve sections for makedumpfile and extenionsQuoted 3 lines
Patch 9 is customization specific, merging depends on the strategy of maintenance.
Patch 1 ~ 8 are common code which should be merged with makedumpfile.Hi Tao,
Just to be thorough, I did re-read the code of each patch, in addition to reviewing the diff from v4 to v5.
I've also re-based my userspace stack extension on this new version. I think it is ready to go.
Reviewed-by Stephen Brennan <[email protected]>Thanks,
StephenQuoted 49 lines
Link to v4: https://lore.kernel.org/kexec/[email protected]/
Link to v3: https://lore.kernel.org/kexec/[email protected]/
Link to v2: https://lore.kernel.org/kexec/[email protected]/
Link to v1: https://lore.kernel.org/kexec/[email protected]/Tao Liu (9):
Reserve sections for makedumpfile and extenions Implement kernel kallsyms resolving Implement kernel btf resolving
Implement kernel module's kallsyms resolving Implement kernel module's btf resolving Add makedumpfile extensions support
Add sample extension as an example reference
Doc: Add --extension option to makedumpfile manual Add amdgpu mm pages filtering extensionMakefile | 15 +- README | 6 + btf_info.c | 375 +++++++++++++++++++++++++ btf_info.h | 77 ++++++ extension.c | 338 ++++++++++++++++++++++ extension.h | 16 ++ extensions/Makefile | 13 + extensions/amdgpu_filter.c | 221 +++++++++++++++ extensions/maple_tree.c | 328 ++++++++++++++++++++++ extensions/maple_tree.h | 7 + extensions/sample.c | 69 +++++ kallsyms.c | 554 +++++++++++++++++++++++++++++++++++++ kallsyms.h | 87 ++++++ makedumpfile.8.in | 11 +- makedumpfile.c | 44 ++- makedumpfile.h | 12 + makedumpfile.ld | 16 ++ 17 files changed, 2180 insertions(+), 9 deletions(-) create mode 100644 btf_info.c create mode 100644 btf_info.h create mode 100644 extension.c create mode 100644 extension.h create mode 100644 extensions/Makefile create mode 100644 extensions/amdgpu_filter.c create mode 100644 extensions/maple_tree.c create mode 100644 extensions/maple_tree.h create mode 100644 extensions/sample.c create mode 100644 kallsyms.c create mode 100644 kallsyms.h create mode 100644 makedumpfile.ld--
2.47.0
Quoted 42 lines · Tao Liu
A) This patchset will introduce the following features to makedumpfile:
1) Add .so extension support to makedumpfile
2) Enable btf and kallsyms for symbol type and address resolving.B) The purpose of the features are:
1) Currently makedumpfile filters mm pages based on page flags, because flags can help to determine one page's usage. But this page-flag-checking method lacks of flexibility in certain cases, e.g. if we want to filter those mm pages occupied by GPU during vmcore dumping due to:
a) GPU may be taking a large memory and contains sensitive data;
b) GPU mm pages have no relations to kernel crash and useless for vmcore analysis.But there is no GPU mm page specific flags, and apparently we don't need to create one just for kdump use. A programmable filtering tool is more suitable for such cases. In addition, different GPU vendors may use different ways for mm pages allocating, programmable filtering is better than hard coding these GPU specific logics into makedumpfile in this case.
2) Currently makedumpfile already contains a programmable filtering tool, aka eppic script, which allows user to write customized code for data erasing. However it has the following drawbacks:
a) cannot do mm page filtering.
b) need to access to debuginfo of both kernel and modules, which is not applicable in the 2nd kernel.
c) eppic library has memory leaks which are not all resolved [1]. This is not acceptable in 2nd kernel.makedumpfile need to resolve the dwarf data from debuginfo, to get symbols types and addresses. In recent kernel there are dwarf alternatives such as btf/kallsyms which can be used for this purpose. And btf/kallsyms info are already packed within vmcore, so we can use it directly.
With these, this patchset introduces makedumpfile extensions, which is based on btf/kallsyms symbol resolving, and is programmable for mm page filtering. The following section shows its usage and performance, please note the tests are performed in 1st kernel.
Quoted 4 lines
3) Compile and run makedumpfile extensions:
$ make LINKTYPE=dynamic USELZO=on USESNAPPY=on USEZSTD=on EXTENSION=on $ make extensions
I love this idea. Do you have time to take it further, and if not are you open to making the extension framework more modular so that we could add others in the future?
Could the btf lookups be extended to cover the symbol lookups used by eppic and the erase filters so that the -x option is unnecessary for kernels that have BTF support?
The current extension implementation is focused just on skipping pages, but it would be great to be able to use this to erase data in structures like the config filters and eppic, but without having to provide a vmlinux at dump time. What do you think about adding the ability to use the extensions to also erase parts of data structures, in addition to filtering whole pages?
Would you be willing to modify the extension registration options to allow an extension to specify what kind it is? That way, in the future we could register multiple different kinds without breaking existing ones. One for filtering pages, one for erasing / modifying dump content, and others based upon whatever additional use cases develop.
Thanks,
-K
Hi Krister,
Thanks a lot for your suggestions and comments!
Quoted 44 lines · Krister Johansen
Quoted 42 lines · Tao Liu
A) This patchset will introduce the following features to makedumpfile:
1) Add .so extension support to makedumpfile
2) Enable btf and kallsyms for symbol type and address resolving.B) The purpose of the features are:
1) Currently makedumpfile filters mm pages based on page flags, because flags can help to determine one page's usage. But this page-flag-checking method lacks of flexibility in certain cases, e.g. if we want to filter those mm pages occupied by GPU during vmcore dumping due to:
a) GPU may be taking a large memory and contains sensitive data;
b) GPU mm pages have no relations to kernel crash and useless for vmcore analysis.But there is no GPU mm page specific flags, and apparently we don't need to create one just for kdump use. A programmable filtering tool is more suitable for such cases. In addition, different GPU vendors may use different ways for mm pages allocating, programmable filtering is better than hard coding these GPU specific logics into makedumpfile in this case.
2) Currently makedumpfile already contains a programmable filtering tool, aka eppic script, which allows user to write customized code for data erasing. However it has the following drawbacks:
a) cannot do mm page filtering.
b) need to access to debuginfo of both kernel and modules, which is not applicable in the 2nd kernel.
c) eppic library has memory leaks which are not all resolved [1]. This is not acceptable in 2nd kernel.makedumpfile need to resolve the dwarf data from debuginfo, to get symbols types and addresses. In recent kernel there are dwarf alternatives such as btf/kallsyms which can be used for this purpose. And btf/kallsyms info are already packed within vmcore, so we can use it directly.
With these, this patchset introduces makedumpfile extensions, which is based on btf/kallsyms symbol resolving, and is programmable for mm page filtering. The following section shows its usage and performance, please note the tests are performed in 1st kernel.
Quoted 8 lines
Quoted 4 lines
3) Compile and run makedumpfile extensions:
$ make LINKTYPE=dynamic USELZO=on USESNAPPY=on USEZSTD=on EXTENSION=on $ make extensionsI love this idea. Do you have time to take it further, and if not are you open to making the extension framework more modular so that we could add others in the future?
The purpose of extension is to make the framework modular. My original thought is, we can implement several makedumpfile extensions, each restricted to one specific function. Like one extension deals with AMD gpu mm filtering only, one deals with Intel gpu only etc. For distros we can ship all extensions along with makedumpfile once, but the respective extensions will only take effect if the machine has AMD / Intel gpu. This is the same case if you'd like to add other customized functions while the makedumpfile core remains unchanged.
Quoted 4 lines
Could the btf lookups be extended to cover the symbol lookups used by eppic and the erase filters so that the -x option is unnecessary for kernels that have BTF support?
Yes, from my view it is doable and not difficult to implement.
Quoted 7 lines
The current extension implementation is focused just on skipping pages, but it would be great to be able to use this to erase data in structures like the config filters and eppic, but without having to provide a vmlinux at dump time. What do you think about adding the ability to use the extensions to also erase parts of data structures, in addition to filtering whole pages?
That's the step 2 for the BTF/kallsyms work of makedumpfile, and I have planed to work on this once the patchset(step 1) is accepted. The reason for the task dividing is, the GPU mm page filtering is more urgent than data erasing from my view. For data erasing, at least we can do the erasing in 1st kernel with the help of dwarf, cumbersome but working; For GPU mm filtering, as far as I know, there are no handy tools in 2nd kernel.
I think erasing the data is doable upon the current page filtering code.
Quoted 3 lines
Would you be willing to modify the extension registration options to allow an extension to specify what kind it is? That way, in the future
I'm not sure what you mean by "what kind". Do you mean an extension needs to tell makedumpfile what purpose it is for when loading?
Quoted 3 lines
we could register multiple different kinds without breaking existing ones. One for filtering pages, one for erasing / modifying dump content, and others based upon whatever additional use cases develop.
That's the goal of extensions, each extension deals with its own business. Could you point out the code that doesn't match the goal? I'm happy to correct it in v6.
Thanks,
Tao Liu
Quoted 5 lines
Thanks,
-K
Hi Tao,
Thanks for the response! I've put the followups below.
Quoted 7 lines · Tao Liu
Quoted 4 lines · Krister Johansen
I love this idea. Do you have time to take it further, and if not are you open to making the extension framework more modular so that we could add others in the future?
Quoted 8 lines
The purpose of extension is to make the framework modular. My original thought is, we can implement several makedumpfile extensions, each restricted to one specific function. Like one extension deals with AMD gpu mm filtering only, one deals with Intel gpu only etc. For distros we can ship all extensions along with makedumpfile once, but the respective extensions will only take effect if the machine has AMD / Intel gpu. This is the same case if you'd like to add other customized functions while the makedumpfile core remains unchanged.
Makes sense.
Quoted 5 lines
Quoted 3 lines
Could the btf lookups be extended to cover the symbol lookups used by eppic and the erase filters so that the -x option is unnecessary for kernels that have BTF support?
Yes, from my view it is doable and not difficult to implement.
In some environments, the size of the vmlinux + modules can be fairly substantial to leave on disk. It's attractive to have the option to omit it and still filter dumps.
Quoted 7 lines
Quoted 6 lines
The current extension implementation is focused just on skipping pages, but it would be great to be able to use this to erase data in structures like the config filters and eppic, but without having to provide a vmlinux at dump time. What do you think about adding the ability to use the extensions to also erase parts of data structures, in addition to filtering whole pages?
Quoted 7 lines
That's the step 2 for the BTF/kallsyms work of makedumpfile, and I have planed to work on this once the patchset(step 1) is accepted. The reason for the task dividing is, the GPU mm page filtering is more urgent than data erasing from my view. For data erasing, at least we can do the erasing in 1st kernel with the help of dwarf, cumbersome but working; For GPU mm filtering, as far as I know, there are no handy tools in 2nd kernel.
Excited to hear that you have something already planned for erasing. My apologies if I missed a more comprehensive write-up about the longer term goals for the work.
Quoted 1 line
I think erasing the data is doable upon the current page filtering code.
I wondered about this, but for data-structures that are smaller than a page, wouldn't that mean that we're erasing other content? The "erase" plugins memset the output data to a chosen value (or 0), whereas the filtering just drops the page. Couldn't this also lead to a situation where the debugger can't find the page at all, versus giving us one that's sanitized? (I do understand why you want to drop the pages for the GPU cases)
Quoted 5 lines
Quoted 2 lines
Would you be willing to modify the extension registration options to allow an extension to specify what kind it is? That way, in the future
I'm not sure what you mean by "what kind". Do you mean an extension needs to tell makedumpfile what purpose it is for when loading?
Yes, sorry I wasn't clear in writing the question. Stating this differently, if we want to allow the ability for different extensions to do different things, how do the extensions declare to makedumpfile what they can do, so that it knows where to invoke their callbacks, and what callbacks of theirs to invoke.
Looking at patch 6/9, right now run_extension_callback() is involved from __exclude_unncessary_pages and always calls the "extension_callback" symbol in the module. This makes sense for a single extension type that's focused on filtering pages. However, if we wanted to have multiple different extensions, this might be more difficult.
If we could determine what type of functionality the module implements in load_extensions, then we could tell if this is a page filtering extension, an erase extension, or some other kind of extension.
For example, for an erase filter, perhaps we would want two callbacks:
one to set up the ranges to filter "extension_gather_callback" and another to actuallyf check the address range to see if it is filtered, "extension_filter_data_callback"
I'm not sure about the names. "extension_callback" seems generic, but this has a specific purpose. It's a "extension_filter_page_callback"
I may be overengineering this a bit, but having makedumpfile pass an ops vector to the extension in a load function could help here. Then the module's load function fills out the vector with the functions it supports. Depending on what's implemented, these can be placed into different callback lists to get invoked at different points in the program (e.g. one at pfn filter time, another in filter_data_buffer, etc).
It sounds like you had a plan here, though. Were you thinking of adding new extension types a different way?
Quoted 7 lines
Quoted 3 lines
we could register multiple different kinds without breaking existing ones. One for filtering pages, one for erasing / modifying dump content, and others based upon whatever additional use cases develop.
That's the goal of extensions, each extension deals with its own business. Could you point out the code that doesn't match the goal? I'm happy to correct it in v6.
Yes, I attempted to elaborate on this in the preceding paragraphs. Basically wondering how we can add new extension functionality without breaking existing extensions, and then get the code to invoke the right if there are multiple types that need to be used at different times.
Thanks,
-K
Hi Krister,
Quoted 3 lines · Krister Johansen
Hi Tao,
Thanks for the response! I've put the followups below.
Thanks for your in-depth explanation, it's very helpful to me for designing the data erasing function.
Quoted 24 lines · Tao Liu
Quoted 7 lines
Quoted 4 lines · Krister Johansen
I love this idea. Do you have time to take it further, and if not are you open to making the extension framework more modular so that we could add others in the future?
Quoted 8 lines
The purpose of extension is to make the framework modular. My original thought is, we can implement several makedumpfile extensions, each restricted to one specific function. Like one extension deals with AMD gpu mm filtering only, one deals with Intel gpu only etc. For distros we can ship all extensions along with makedumpfile once, but the respective extensions will only take effect if the machine has AMD / Intel gpu. This is the same case if you'd like to add other customized functions while the makedumpfile core remains unchanged.
Makes sense.
Quoted 4 lines
Quoted 3 lines
Could the btf lookups be extended to cover the symbol lookups used by eppic and the erase filters so that the -x option is unnecessary for kernels that have BTF support?
Quoted 5 lines
Quoted 1 line
Yes, from my view it is doable and not difficult to implement.
In some environments, the size of the vmlinux + modules can be fairly substantial to leave on disk. It's attractive to have the option to omit it and still filter dumps.
Yes, I totally agree.
Quoted 16 lines
Quoted 8 lines
Quoted 7 lines
The current extension implementation is focused just on skipping pages, but it would be great to be able to use this to erase data in structures like the config filters and eppic, but without having to provide a vmlinux at dump time. What do you think about adding the ability to use the extensions to also erase parts of data structures, in addition to filtering whole pages?
Quoted 7 lines
That's the step 2 for the BTF/kallsyms work of makedumpfile, and I have planed to work on this once the patchset(step 1) is accepted. The reason for the task dividing is, the GPU mm page filtering is more urgent than data erasing from my view. For data erasing, at least we can do the erasing in 1st kernel with the help of dwarf, cumbersome but working; For GPU mm filtering, as far as I know, there are no handy tools in 2nd kernel.
Quoted 3 lines
Excited to hear that you have something already planned for erasing. My apologies if I missed a more comprehensive write-up about the longer term goals for the work.
No worries, I didn't post the goals upstream; I only had internal discussions within my team regarding the next steps for BTF/kallsyms in makedumpfile.
Quoted 10 lines
Quoted 2 lines
I think erasing the data is doable upon the current page filtering code.
I wondered about this, but for data-structures that are smaller than a page, wouldn't that mean that we're erasing other content? The "erase" plugins memset the output data to a chosen value (or 0), whereas the filtering just drops the page. Couldn't this also lead to a situation where the debugger can't find the page at all, versus giving us one that's sanitized? (I do understand why you want to drop the pages for the GPU cases)
Frankly I didn't consider the data erasing as in-depth as you did. I think you are right, makedumpfile needs to know which extensions handle data erasing and which handle mm page filtering. I guess the mm page filtering extensions will need to perform a "dry-run" filter first, in case the "data erasing" extensions break any useful data structure. In this step, "dry-run" will only record pfn numbers of the pages that will be filtered. Then "data erasing" extensions are called, so all the sensitive data is memset to 0. Finally, all desired pages are filtered out based on the previous recording.
With this, "data erase" and "page filtering" will not interfere with each other. What do you think?
Quoted 40 lines
Quoted 6 lines
Quoted 3 lines
Would you be willing to modify the extension registration options to allow an extension to specify what kind it is? That way, in the future
I'm not sure what you mean by "what kind". Do you mean an extension needs to tell makedumpfile what purpose it is for when loading?
Yes, sorry I wasn't clear in writing the question. Stating this differently, if we want to allow the ability for different extensions to do different things, how do the extensions declare to makedumpfile what they can do, so that it knows where to invoke their callbacks, and what callbacks of theirs to invoke.
Looking at patch 6/9, right now
run_extension_callback()is involved from__exclude_unncessary_pagesand always calls the "extension_callback" symbol in the module. This makes sense for a single extension type that's focused on filtering pages. However, if we wanted to have multiple different extensions, this might be more difficult.If we could determine what type of functionality the module implements in
load_extensions, then we could tell if this is a page filtering extension, an erase extension, or some other kind of extension.For example, for an erase filter, perhaps we would want two callbacks:
one to set up the ranges to filter "extension_gather_callback" and another to actuallyf check the address range to see if it is filtered, "extension_filter_data_callback"I'm not sure about the names. "
extension_callback" seems generic, but this has a specific purpose. It's a "extension_filter_page_callback"I may be overengineering this a bit, but having makedumpfile pass an ops vector to the extension in a load function could help here. Then the module's load function fills out the vector with the functions it supports. Depending on what's implemented, these can be placed into different callback lists to get invoked at different points in the program (e.g. one at pfn filter time, another in
filter_data_buffer, etc).
Quoted 2 lines
It sounds like you had a plan here, though. Were you thinking of adding new extension types a different way?
I see your idea: makedumpfile predefines a few hook points at different stages, and extensions can register their callbacks to these hook points. For now I think 2 hook points are enough, one for page filtering and other one for resiger the data erasing, which definitely shouldn't be within __exclude_unnecessary_pages().
I'm willing to modify the code. Such as implementing a hooking point registration/management. But since I haven't work on the data erasing functions so far, the design might be superficial, personally I'd prefer to do this along with the data erasing functions in the next independent patchset, considering current patchset we already includes plenty of code/function implementations. @maintainers, What's your opinion?
Quoted 5 lines
Quoted 5 lines
Quoted 4 lines
we could register multiple different kinds without breaking existing ones. One for filtering pages, one for erasing / modifying dump content, and others based upon whatever additional use cases develop.
Quoted 8 lines
Quoted 3 lines
That's the goal of extensions, each extension deals with its own business. Could you point out the code that doesn't match the goal? I'm happy to correct it in v6.
Yes, I attempted to elaborate on this in the preceding paragraphs. Basically wondering how we can add new extension functionality without breaking existing extensions, and then get the code to invoke the right if there are multiple types that need to be used at different times.
Agreed.
Thanks,
Tao Liu
Quoted 5 lines
Thanks,
-K
Hi Tao,
Quoted 4 lines · Tao Liu
On Tue, Jun 2, 2026 at 12:47 PM Krister Johansen <[email protected]> wrote:
Thanks for your in-depth explanation, it's very helpful to me for designing the data erasing function.
Thanks for the great discussion.
Quoted 20 lines
Quoted 9 lines · Tao Liu
Quoted 1 line
On Sat, May 30, 2026 at 9:11 AM Krister Johansen
I wondered about this, but for data-structures that are smaller than a page, wouldn't that mean that we're erasing other content? The "erase" plugins memset the output data to a chosen value (or 0), whereas the filtering just drops the page. Couldn't this also lead to a situation where the debugger can't find the page at all, versus giving us one that's sanitized? (I do understand why you want to drop the pages for the GPU cases)
Frankly I didn't consider the data erasing as in-depth as you did. I think you are right, makedumpfile needs to know which extensions handle data erasing and which handle mm page filtering. I guess the mm page filtering extensions will need to perform a "dry-run" filter first, in case the "data erasing" extensions break any useful data structure. In this step, "dry-run" will only record pfn numbers of the pages that will be filtered. Then "data erasing" extensions are called, so all the sensitive data is memset to 0. Finally, all desired pages are filtered out based on the previous recording.
Quoted 2 lines
With this, "data erase" and "page filtering" will not interfere with each other. What do you think?
This is a great point. It's probably worth documenting the precedence order in which these callbacks are expected to be applied. Naively, I might expect filtering pages to take precedence over erasing data structures. For the GPU cases, these are orthogonal. However, for something where a user might be both trying to filter the page and erase matching content, we don't have any rules defined. It's probably less surprising to allow pages to be filtered first. (I think it is this way in the code.) It also prevents the page filtering from completely filtering a page.
Quoted 48 lines
Quoted 39 lines
Quoted 5 lines
Quoted 2 lines
Would you be willing to modify the extension registration options to allow an extension to specify what kind it is? That way, in the future
I'm not sure what you mean by "what kind". Do you mean an extension needs to tell makedumpfile what purpose it is for when loading?
Yes, sorry I wasn't clear in writing the question. Stating this differently, if we want to allow the ability for different extensions to do different things, how do the extensions declare to makedumpfile what they can do, so that it knows where to invoke their callbacks, and what callbacks of theirs to invoke.
Looking at patch 6/9, right now
run_extension_callback()is involved from__exclude_unncessary_pagesand always calls the "extension_callback" symbol in the module. This makes sense for a single extension type that's focused on filtering pages. However, if we wanted to have multiple different extensions, this might be more difficult.If we could determine what type of functionality the module implements in
load_extensions, then we could tell if this is a page filtering extension, an erase extension, or some other kind of extension.For example, for an erase filter, perhaps we would want two callbacks:
one to set up the ranges to filter "extension_gather_callback" and another to actuallyf check the address range to see if it is filtered, "extension_filter_data_callback"I'm not sure about the names. "
extension_callback" seems generic, but this has a specific purpose. It's a "extension_filter_page_callback"I may be overengineering this a bit, but having makedumpfile pass an ops vector to the extension in a load function could help here. Then the module's load function fills out the vector with the functions it supports. Depending on what's implemented, these can be placed into different callback lists to get invoked at different points in the program (e.g. one at pfn filter time, another in
filter_data_buffer, etc).Quoted 2 lines
It sounds like you had a plan here, though. Were you thinking of adding new extension types a different way?
I see your idea: makedumpfile predefines a few hook points at different stages, and extensions can register their callbacks to these hook points. For now I think 2 hook points are enough, one for page filtering and other one for resiger the data erasing, which definitely shouldn't be within
__exclude_unnecessary_pages().
Quoted 7 lines
I'm willing to modify the code. Such as implementing a hooking point registration/management. But since I haven't work on the data erasing functions so far, the design might be superficial, personally I'd prefer to do this along with the data erasing functions in the next independent patchset, considering current patchset we already includes plenty of code/function implementations. @maintainers, What's your opinion?
Just to clarify, I'm not asking that you implement any erase functionality in the current patchset. Rather, asking if there's a way to implement the current functionality such that the extension modules won't need recompilation when a new extension type is introduced. I think there are a number of different ways to do this, but I didn't want to be overly prescriptive in my feedback.
Thanks again,
-K
Hi Krister,
I took some time to think about the data erase functions as we discussed previously, and I have implemented one sample data eraser based on the current v5 extension framework in [1]. According to my test, this works well and has no conflict with the mm page filtering extensions. With this, people can do data erasure as well as page filtering in both 1st/2nd kernel as long as btf/kallsyms info is accessiable.
As for btf/kallsyms support for makedumpfile.conf based erasure, I forgot to mention there is one critical info missing compare with "-x vmlinux" dwarfinfo, is the type of global variables, please see the patch [2]. As far as I know, currently kernel hasn't enabled the btf's ability of including global variable's types. E.g. symbol "init_task", we can only know its address by lookup kallsyms, but btf have no record if its type, because it is a global variable. If we create makedumpfile.conf as follows:
[vmlinux]
erase init_task
It can work with no problem for "-x vmlinux" because dwarfinfo contains all info. But it will fail to btf/kallsyms, for btf doesn't know init_task's type is task_struct. However this is not a problem for makedumpfile extensions, because when programming, we can cast it as task_struct by "init_task = GET_KERN_SYM(init_task); task_list = init_task + KERN_MEMBER_OFF(task_struct, tasks); ...", but we cannot do the same to makedumpfile.conf.
In summary, if we still want to use makedumpfile.conf for data erasure, "-x vmlinux"'s dwarfinfo is unavoidable for now.
Thanks,
Tao Liu
[1]: https://lore.kernel.org/kexec/[email protected]/
[2]: https://lore.kernel.org/all/[email protected]/
Quoted 111 lines · Krister Johansen
Hi Tao,
Quoted 4 lines · Tao Liu
On Tue, Jun 2, 2026 at 12:47 PM Krister Johansen <[email protected]> wrote:
Thanks for your in-depth explanation, it's very helpful to me for designing the data erasing function.Thanks for the great discussion.
Quoted 20 lines
Quoted 9 lines · Tao Liu
Quoted 1 line
On Sat, May 30, 2026 at 9:11 AM Krister Johansen
I wondered about this, but for data-structures that are smaller than a page, wouldn't that mean that we're erasing other content? The "erase" plugins memset the output data to a chosen value (or 0), whereas the filtering just drops the page. Couldn't this also lead to a situation where the debugger can't find the page at all, versus giving us one that's sanitized? (I do understand why you want to drop the pages for the GPU cases)
Frankly I didn't consider the data erasing as in-depth as you did. I think you are right, makedumpfile needs to know which extensions handle data erasing and which handle mm page filtering. I guess the mm page filtering extensions will need to perform a "dry-run" filter first, in case the "data erasing" extensions break any useful data structure. In this step, "dry-run" will only record pfn numbers of the pages that will be filtered. Then "data erasing" extensions are called, so all the sensitive data is memset to 0. Finally, all desired pages are filtered out based on the previous recording.
Quoted 2 lines
With this, "data erase" and "page filtering" will not interfere with each other. What do you think?
This is a great point. It's probably worth documenting the precedence order in which these callbacks are expected to be applied. Naively, I might expect filtering pages to take precedence over erasing data structures. For the GPU cases, these are orthogonal. However, for something where a user might be both trying to filter the page and erase matching content, we don't have any rules defined. It's probably less surprising to allow pages to be filtered first. (I think it is this way in the code.) It also prevents the page filtering from completely filtering a page.
Quoted 48 lines
Quoted 39 lines
Quoted 5 lines
Quoted 2 lines
Would you be willing to modify the extension registration options to allow an extension to specify what kind it is? That way, in the future
I'm not sure what you mean by "what kind". Do you mean an extension needs to tell makedumpfile what purpose it is for when loading?
Yes, sorry I wasn't clear in writing the question. Stating this differently, if we want to allow the ability for different extensions to do different things, how do the extensions declare to makedumpfile what they can do, so that it knows where to invoke their callbacks, and what callbacks of theirs to invoke.
Looking at patch 6/9, right now
run_extension_callback()is involved from__exclude_unncessary_pagesand always calls the "extension_callback" symbol in the module. This makes sense for a single extension type that's focused on filtering pages. However, if we wanted to have multiple different extensions, this might be more difficult.If we could determine what type of functionality the module implements in
load_extensions, then we could tell if this is a page filtering extension, an erase extension, or some other kind of extension.For example, for an erase filter, perhaps we would want two callbacks:
one to set up the ranges to filter "extension_gather_callback" and another to actuallyf check the address range to see if it is filtered, "extension_filter_data_callback"I'm not sure about the names. "
extension_callback" seems generic, but this has a specific purpose. It's a "extension_filter_page_callback"I may be overengineering this a bit, but having makedumpfile pass an ops vector to the extension in a load function could help here. Then the module's load function fills out the vector with the functions it supports. Depending on what's implemented, these can be placed into different callback lists to get invoked at different points in the program (e.g. one at pfn filter time, another in
filter_data_buffer, etc).Quoted 2 lines
It sounds like you had a plan here, though. Were you thinking of adding new extension types a different way?
I see your idea: makedumpfile predefines a few hook points at different stages, and extensions can register their callbacks to these hook points. For now I think 2 hook points are enough, one for page filtering and other one for resiger the data erasing, which definitely shouldn't be within
__exclude_unnecessary_pages().Quoted 7 lines
I'm willing to modify the code. Such as implementing a hooking point registration/management. But since I haven't work on the data erasing functions so far, the design might be superficial, personally I'd prefer to do this along with the data erasing functions in the next independent patchset, considering current patchset we already includes plenty of code/function implementations. @maintainers, What's your opinion?
Just to clarify, I'm not asking that you implement any erase functionality in the current patchset. Rather, asking if there's a way to implement the current functionality such that the extension modules won't need recompilation when a new extension type is introduced. I think there are a number of different ways to do this, but I didn't want to be overly prescriptive in my feedback.
Thanks again,
-K
Hi Tao,
I appreciate your additional investigation on these questions.
Quoted 9 lines · Tao Liu
Hi Krister,
I took some time to think about the data erase functions as we discussed previously, and I have implemented one sample data eraser based on the current v5 extension framework in [1]. According to my test, this works well and has no conflict with the mm page filtering extensions. With this, people can do data erasure as well as page filtering in both 1st/2nd kernel as long as btf/kallsyms info is accessiable.
I took a look at the example in your subsequent e-mail. I think this would work for what I'm trying to. I didn't realize that the extension callbacks you built had the ability to call into the other erase code that's also available to eppic.
Quoted 12 lines
As for btf/kallsyms support for makedumpfile.conf based erasure, I forgot to mention there is one critical info missing compare with "-x vmlinux" dwarfinfo, is the type of global variables, please see the patch [2]. As far as I know, currently kernel hasn't enabled the btf's ability of including global variable's types. E.g. symbol "
init_task", we can only know its address by lookup kallsyms, but btf have no record if its type, because it is a global variable. If we create makedumpfile.conf as follows:[vmlinux]
eraseinit_task
Quoted 7 lines
It can work with no problem for "-x vmlinux" because dwarfinfo contains all info. But it will fail to btf/kallsyms, for btf doesn't know init_task's type is
task_struct. However this is not a problem for makedumpfile extensions, because when programming, we can cast it astask_structby "init_task=GET_KERN_SYM(init_task);task_list=init_task+KERN_MEMBER_OFF(task_struct, tasks); ...", but we cannot do the same to makedumpfile.conf.
Got it. It wouldn't work with the current configuration file syntax because it needs more information to pass to the APIs that perform the btf lookup? Trying to understand if GET_KERN_SYM can be dynamically filled in, or if its a macro that's evaluated at compile time and isn't dynamically changable.
Thanks again,
-K
Hi Krister,
Quoted 15 lines · Krister Johansen
Hi Tao,
I appreciate your additional investigation on these questions.Quoted 9 lines · Tao Liu
Hi Krister,
I took some time to think about the data erase functions as we discussed previously, and I have implemented one sample data eraser based on the current v5 extension framework in [1]. According to my test, this works well and has no conflict with the mm page filtering extensions. With this, people can do data erasure as well as page filtering in both 1st/2nd kernel as long as btf/kallsyms info is accessiable.
Quoted 4 lines
I took a look at the example in your subsequent e-mail. I think this would work for what I'm trying to. I didn't realize that the extension callbacks you built had the ability to call into the other erase code that's also available to eppic.
Thank you for raising the data erasure issue, to be honest I hadn't thought about it in depth before your message.
Quoted 21 lines
Quoted 13 lines
As for btf/kallsyms support for makedumpfile.conf based erasure, I forgot to mention there is one critical info missing compare with "-x vmlinux" dwarfinfo, is the type of global variables, please see the patch [2]. As far as I know, currently kernel hasn't enabled the btf's ability of including global variable's types. E.g. symbol "
init_task", we can only know its address by lookup kallsyms, but btf have no record if its type, because it is a global variable. If we create makedumpfile.conf as follows:[vmlinux]
eraseinit_taskQuoted 7 lines
It can work with no problem for "-x vmlinux" because dwarfinfo contains all info. But it will fail to btf/kallsyms, for btf doesn't know init_task's type is
task_struct. However this is not a problem for makedumpfile extensions, because when programming, we can cast it astask_structby "init_task=GET_KERN_SYM(init_task);task_list=init_task+KERN_MEMBER_OFF(task_struct, tasks); ...", but we cannot do the same to makedumpfile.conf.
Quoted 3 lines
Got it. It wouldn't work with the current configuration file syntax because it needs more information to pass to the APIs that perform the btf lookup? Trying to understand if
GET_KERN_SYMcan be dynamically
Yes, because currently, no BTF info is integrated for global kernel variables. That is, we can only know global variable's address by kallsyms, but no type info from BTF. If we try to build makedumpfile.conf erasure upon btf, it should be something like:
[vmlinux]
erase (struct task_struct)init_task
To be honest, the syntax will be ugly, and I'd prefer the issue fixed on the kernel side rather than in makedumpfile.conf side. As for why kernel hasn't enable the global variable's btf info, that's another story and should be addressed individually.
Quoted 2 lines
filled in, or if its a macro that's evaluated at compile time and isn't dynamically changable.
When macros such as INIT_MOD_SYM(vmlinux, init_task) are used, a special elf section will be reserved for the extension .so at compile time, this indicates the extension needs this kernel symbol before it can run. When makedumpfile parses the kernel's btf/kallsyms, these special sections will be filled with their respective values. After this, the extension is runnable, by using GET_KERN_SYM() macro, the values will be retrived from the special section and developers can use it directly.
Thanks,
Tao Liu
Quoted 5 lines
Thanks again,
-K
PATCH 1/1 Add erase_sample extension as an data erase reference
In reply to Tao LiuThis patch will add a erase_sample.c as a data erase reference. It will erase all tasks & threads's comm[] member within their task_struct, as a result, you will see the ps info outputted by crash utility similiar as follows:
crash> ps
...
1 0 10 ffff96b7c0980000 IN 0.0 25504 15704 XXXXXXXXXXXXXXX 2 0 15 ffff96b7c0982f40 IN 0.0 0 0 [XXXXXXXXXXXXXXX] 3 2 2 ffff96b7c09a0000 IN 0.0 0 0 [XXXXXXXXXXXXXXX] 4 2 0 ffff96b7c09a2f40 ID 0.0 0 0 [XXXXXXXXXXXXXXX] 5 2 0 ffff96b7c09a8000 ID 0.0 0 0 [XXXXXXXXXXXXXXX]
The comm[] erasure doesn't cause critical info loss and has slight impact on vmcore analyze, so it is safe to take it as an example for developers.
---
Hi maintainers,
This patch should be applied upon "[PATCH v5][makedumpfile 9/9] Add amdgpu mm pages filtering extension". Since this patch doesn't do modifications on any previous patches, and you are reviewing v5, so I post appended to v5. If this causes any confusing, I'm happy to post a v6 with a cleaner format. Thanks!
--- extensions/Makefile | 2 +- extensions/erase_sample.c | 78 +++++++++++++++++++++++++++++++++++++++ 2 files changed, 79 insertions(+), 1 deletion(-) create mode 100644 extensions/erase_sample.c
extensions/Makefile +1 -1
@@ -1,5 +1,5 @@ CC ?= gcc -CONTRIB_SO := sample.so amdgpu_filter.so +CONTRIB_SO := sample.so erase_sample.so amdgpu_filter.so all: $(CONTRIB_SO)
extensions/erase_sample.c new file +78 -0
78 lines not on this page · raw message
Signature · 1 line
2.47.0