btf/kallsyms based eppic extension for mm page filtering
Subsystems (3)
v2 cover btf/kallsyms based eppic extension for mm page filtering
A) This patchset will introduce the following features to makedumpfile:
1) Enable eppic script for memory pages filtering.
2) Enable btf and kallsyms for symbol type and address resolving.
B) The purpose of the features are:
1) Currently makedumpfile filters mm pages based on page flags, because flags can help to determine one page's usage. But this page-flag-checking method lacks of flexibility in certain cases, e.g. if we want to filter those mm pages occupied by GPU during vmcore dumping due to:
a) GPU may be taking a large memory and contains sensitive data;
b) GPU mm pages have no relations to kernel crash and useless for vmcore analysis.
But there is no GPU mm page specific flags, and apparently we don't need to create one just for kdump use. A programmable filtering tool is more suitable for such cases. In addition, different GPU vendors may use different ways for mm pages allocating, programmable filtering is better than hard coding these GPU specific logics into makedumpfile in this case.
2) Currently makedumpfile already contains a programmable filtering tool, aka eppic script, which allows user to write customized code for data erasing. However it has the following drawbacks:
a) cannot do mm page filtering.
b) need to access to debuginfo of both kernel and modules, which is not applicable in the 2nd kernel.
c) Poor performance, making vmcore dumping time unacceptable (See the following performance testing).
makedumpfile need to resolve the dwarf data from debuginfo, to get symbols types and addresses. In recent kernel there are dwarf alternatives such as btf/kallsyms which can be used for this purpose. And btf/kallsyms info are already packed within vmcore, so we can use it directly.
With these, this patchset introduces an upgraded eppic, which is based on btf/kallsyms symbol resolving, and is programmable for mm page filtering. The following info shows its usage and performance, please note the tests are performed in 1st kernel:
$ time ./makedumpfile -d 31 -l /var/crash/127.0.0.1-2025-06-10-18\:03\:12/vmcore
/tmp/dwarf.out -x /lib/debug/lib/modules/6.11.8-300.fc41.x86_64/vmlinux
--eppic eppic_scripts/filter_amdgpu_mm_pages.c
real 14m6.894s
user 4m16.900s
sys 9m44.695s $ time ./makedumpfile -d 31 -l /var/crash/127.0.0.1-2025-06-10-18\:03\:12/vmcore
/tmp/btf.out --eppic eppic_scripts/filter_amdgpu_mm_pages.c
real 0m10.672s
user 0m9.270s
sys 0m1.130s -rw------- 1 root root 367475074 Jun 10 18:06 btf.out
-rw------- 1 root root 367475074 Jun 10 21:05 dwarf.out
-rw-rw-rw- 1 root root 387181418 Jun 10 18:03 /var/crash/127.0.0.1-2025-06-10-18:03:12/vmcore
C) Discussion:
1) GPU types: Currently only tested with amdgpu's mm page filtering, others are not tested.
2) OS: The code can work on rhel-10+/rhel9.5+ on x86_64/arm64/s390/ppc64. Others are not tested.
D) Testing:
1) If you don't want to create your vmcore, you can find a vmcore which I created with amdgpu mm pages unfiltered [1], the amdgpu mm pages are allocated by program [2]. You can use the vmcore in 1st kernel to filter the amdgpu mm pages by the previous performance testing cmdline. To verify the pages are filtered in crash:
Unfiltered:
crash> search -c "!QAZXSW@#EDC"
ffff96b7fa800000: !QAZXSW@#EDCXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
ffff96b87c800000: !QAZXSW@#EDCXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX crash> rd ffff96b7fa800000
ffff96b7fa800000: 405753585a415121 !QAZXSW@
crash> rd ffff96b87c800000
ffff96b87c800000: 405753585a415121 !QAZXSW@
Filtered:
crash> search -c "!QAZXSW@#EDC"
crash> rd ffff96b7fa800000
rd: page excluded: kernel virtual address: ffff96b7fa800000 type: "64-bit KVADDR" crash> rd ffff96b87c800000
rd: page excluded: kernel virtual address: ffff96b87c800000 type: "64-bit KVADDR"
2) You can use eppic_scripts/print_all_vma.c against an ordinary vmcore to test only btf/kallsyms functions by output all VMAs if no amdgpu vmcores/machine avaliable.
[1]: https://people.redhat.com/~ltao/core/
[2]: https://gist.github.com/liutgnu/a8cbce1c666452f1530e1410d1f352df
v2 -> v1:
1) Moved maple tree related code(for VMA iteration) into eppic script, so we don't need to port maple tree code to makedumpfile.
2) Reorganized the patchset as follows:
--- <common modification> ---
1.Add page filtering function 2.Supporting main() as the entry of eppic script
--- <dwarf related modification> --- 3.dwarf_info: Support kernel address randomization 4.dwarf_info: Fix a infinite recursion bug for rust 5.eppic dwarf: support anonymous structs member resolving 6.Enable page filtering for dwarf eppic
--- <btf & kallsyms related modification> --- 7.Implement kernel kallsyms resolving
8.Implement kernel btf resolving
9.Implement kernel module's kallsyms resolving 10.Implement kernel module's btf resolving 11.Export necessary btf/kallsyms functions to eppic extension 12.Enable page filtering for btf/kallsyms eppic 13.Docs: Update eppic related entries
--- <only for test purpose, don't merge> --- 14.Introducing 2 eppic scripts to test the dwarf/btf eppic extension
The modification on dwarf is primary for comparision purpose, that for the same eppic program, mm page filtering should get exact same outputs for dwarf & kallsyms/btf based approaches. If outputs unmatch, this indicates bugs. In fact, we will never take dwarf mm pages filtering in real use, due to its poor performance as well as inaccessibility of debuginfo during kdump in 2nd kernel. So patch 3/4/5 won't affect the function of btf/kallsyms eppic mm page filtering, but there are functions shared in patch 6, so it is a must-have one. Patch 14 is only for test purpose, to demonstrate how to write eppic script for mm page filtering, so it isn't a must-have patch.
Please note, in patch 14, I have deliberately converted all array operation into pointer operation, e.g. modified "node->slot[i]" into "*((unsigned long *)&(node->slot) + i)". This is because there are bugs for array operation support in extension_eppic.c. I didn't have effort to test and fix them all because as I mentioned previously, mm page filtering in dwarf side is only for comparision and will never be used in real use. There is no such issue for kallsyms/btf eppic side.
3) Since we ported maple tree code to eppic script, several bugs found both for eppic library & eppic btf support. Please use master branch of eppic library to co-compile with this patchset.
Tao Liu (14):
Add page filtering function
Supporting main() as the entry of eppic script
dwarf_info: Support kernel address randomization
dwarf_info: Fix a infinite recursion bug for rust eppic dwarf: support anonymous structs member resolving Enable page filtering for dwarf eppic
Implement kernel kallsyms resolving
Implement kernel btf resolving
Implement kernel module's kallsyms resolving
Implement kernel module's btf resolving
Export necessary btf/kallsyms functions to eppic extension Enable page filtering for btf/kallsyms eppic
Docs: Update eppic related entries
Introducing 2 eppic scripts to test the dwarf/btf eppic extension
Makefile | 6 +- btf.c | 919 +++++++++++++++++++++++++ btf.h | 177 +++++ dwarf_info.c | 7 + eppic_scripts/filter_amdgpu_mm_pages.c | 255 +++++++ eppic_scripts/print_all_vma.c | 239 +++++++ erase_info.c | 120 +++- erase_info.h | 19 + extension_btf.c | 258 +++++++ extension_eppic.c | 106 ++- extension_eppic.h | 6 +- kallsyms.c | 392 +++++++++++ kallsyms.h | 41 ++ makedumpfile.8.in | 24 +- makedumpfile.c | 21 +- makedumpfile.h | 11 + print_info.c | 11 +- 17 files changed, 2550 insertions(+), 62 deletions(-) create mode 100644 btf.c create mode 100644 btf.h create mode 100644 eppic_scripts/filter_amdgpu_mm_pages.c create mode 100644 eppic_scripts/print_all_vma.c create mode 100644 extension_btf.c create mode 100644 kallsyms.c create mode 100644 kallsyms.h
Signature · 1 line
2.47.0
PATCH v2 1/14 Add page filtering function
pfn and num is the data which eppic scritps give to makedumpfile for mm page filtering, so any page within (pfn ~ pfn + num) can be filtered. Since makedumpfile will iterate the pfn in an ascending order in __exclude_unnecessary_pages(), the linked lists formed by ft_page_info will also be organized in an ascending order by pfn, so if one pfn
is hit by one list, the following pfn is most likely to be hit either by this list again, or the one follows, so a cur variable is used for saving the current list position to speedup the pfn checking process.
--- erase_info.c | 69 ++++++++++++++++++++++++++++++++++++++++++++++++++ erase_info.h | 8 ++++++ makedumpfile.c | 18 ++++++++++--- 3 files changed, 91 insertions(+), 4 deletions(-)
erase_info.c +69 -0
@@ -2466,3 +2466,72 @@ get_size_eraseinfo(void)return size_eraseinfo; } +static struct ft_page_info *ft_head =NULL ; + +/* +* Insert the ft_page_info blocks into ft_head by ascending pfn. +*/ +int +update_filter_pages_info (unsigned long pfn,unsigned long num) +{ +struct ft_page_info *p; +struct ft_page_info *new_p =malloc (sizeof (struct ft_page_info )); + +if (!new_p) { +ERRMSG ("Can't allocate memory for ft_page_info at %lx\n" , pfn); +return 1 ; + } + new_p->pfn = pfn; + new_p->num = num; + new_p->next =NULL ; + +if (!ft_head || ft_head->pfn > new_p->pfn) { + new_p->next = ft_head; + ft_head = new_p; +return 0 ; + } + + p = ft_head; +while (p->next !=NULL && p->next->pfn < new_p->pfn) { + p = p->next; + } + + new_p->next = p->next; + p->next = new_p; +return 0 ; +} + +/* +* Check if the pfn page should be filtered. +* +* pfn and ft_head are in ascending order, so save the current ft_page_info +* block into **p because it is likely to hit again next time. +*/ +int +filter_page (unsigned long pfn,struct ft_page_info **p) +{ +if (ft_head ==NULL ) +return 0 ; + +if (*p ==NULL ) + *p = ft_head; + +/* Handle the 1st gap */ +if (pfn >=0 && pfn < ft_head->pfn) +return 0 ; + +/* Handle ft_page_info blocks and following gaps */ +while ((*p)->next) { +if (pfn >= (*p)->pfn && pfn < (*p)->pfn + (*p)->num) +return 1 ;// filter this page +if (pfn >= (*p)->pfn + (*p)->num && pfn < (*p)->next->pfn) +return 0 ;// save this page + *p = (*p)->next; + } + +/* Handle the last gap */ +if (pfn >= (*p)->pfn + (*p)->num) +return 0 ; +else +return 1 ; +}
erase_info.h +8 -0
@@ -64,6 +64,14 @@ void filter_data_buffer_parallel(unsigned char *buf, unsigned long long paddr, size_t size, pthread_mutex_t *mutex);unsigned long get_size_eraseinfo (void );int update_filter_info_raw (unsigned long long ,int ,int ); +int update_filter_pages_info (unsigned long ,unsigned long ); + +struct ft_page_info { +unsigned long pfn; +unsigned long num; +struct ft_page_info *next; +}; +int filter_page (unsigned long ,struct ft_page_info **p);#endif /* _ERASE_INFO_H */
makedumpfile.c +14 -4
@@ -100,6 +100,7 @@ mdf_pfn_t pfn_user;mdf_pfn_t pfn_free;mdf_pfn_t pfn_hwpoison;mdf_pfn_t pfn_offline; +mdf_pfn_t pfn_eppic;mdf_pfn_t pfn_elf_excluded;mdf_pfn_t num_dumped;
@@ -6453,6 +6454,7 @@ __exclude_unnecessary_pages(unsigned long mem_map,unsigned int order_offset, dtor_offset;unsigned long flags, mapping, private =0 ;unsigned long compound_dtor, compound_head =0 ; +struct ft_page_info *cur =NULL ;/* * If a multi-page exclusion is pending, do it first
@@ -6670,6 +6672,13 @@ check_order:else if (isOffline (flags, _mapcount)) { pfn_counter = &pfn_offline; } +/* +* Exclude pages that specified by eppic script +*/ +else if (filter_page (pfn, &cur)) { + nr_pages =1 ; + pfn_counter = &pfn_eppic; + }/* * Unexcludable page */
@@ -8217,7 +8226,7 @@ write_elf_pages_cyclic(struct cache_data *cd_header, struct cache_data *cd_page)*/ if (info->flag_cyclic) { pfn_zero = pfn_cache = pfn_cache_private =0 ; - pfn_user = pfn_free = pfn_hwpoison = pfn_offline =0 ; + pfn_user = pfn_free = pfn_hwpoison = pfn_offline = pfn_eppic =0 ; pfn_memhole = info->max_mapnr; }
@@ -9555,7 +9564,7 @@ write_kdump_pages_and_bitmap_cyclic(struct cache_data *cd_header, struct cache_d* Reset counter for debug message. */ pfn_zero = pfn_cache = pfn_cache_private =0 ; - pfn_user = pfn_free = pfn_hwpoison = pfn_offline =0 ; + pfn_user = pfn_free = pfn_hwpoison = pfn_offline = pfn_eppic =0 ; pfn_memhole = info->max_mapnr;/*
@@ -10504,7 +10513,7 @@ print_report(void) pfn_original = info->max_mapnr - pfn_memhole; pfn_excluded = pfn_zero + pfn_cache + pfn_cache_private - + pfn_user + pfn_free + pfn_hwpoison + pfn_offline; + + pfn_user + pfn_free + pfn_hwpoison + pfn_offline + pfn_eppic;REPORT_MSG ("\n" );REPORT_MSG ("Original pages : 0x%016llx\n" , pfn_original);
@@ -10520,6 +10529,7 @@ print_report(void)REPORT_MSG (" Free pages : 0x%016llx\n" , pfn_free);REPORT_MSG (" Hwpoison pages : 0x%016llx\n" , pfn_hwpoison);REPORT_MSG (" Offline pages : 0x%016llx\n" , pfn_offline); +REPORT_MSG (" Eppic filtered pages : 0x%016llx\n" , pfn_eppic);REPORT_MSG (" Remaining pages : 0x%016llx\n" , pfn_original - pfn_excluded);
@@ -10560,7 +10570,7 @@ print_mem_usage(void) pfn_original = info->max_mapnr - pfn_memhole; pfn_excluded = pfn_zero + pfn_cache + pfn_cache_private - + pfn_user + pfn_free + pfn_hwpoison + pfn_offline; + + pfn_user + pfn_free + pfn_hwpoison + pfn_offline + pfn_eppic; shrinking = (pfn_original - pfn_excluded) *100 ; shrinking = shrinking / pfn_original; total_size = info->page_size * pfn_original;
Signature · 1 line
2.47.0
PATCH v2 2/14 Supporting main() as the entry of eppic script
In reply to Tao LiuPreviously only a function with usage or help functions is regarded as the entry of eppic script. This constraint makes no sense because main() is widely accepted, and people can easily get confused when writing eppic script and blocked by the constraint. This patch will support main() as an entry.
--- extension_eppic.c | 7 +++++++ 1 file changed, 7 insertions(+)
extension_eppic.c +7 -0
@@ -49,6 +49,8 @@ static int apigetctype(int, char *, type_t *);* entry point and user will not have any option to execute the usage * or help functions. However they are required to identify the entry * points in the eppic macro. +* +* "main" can also work as the entry point of eppic macro. */ void reg_callback (char *name,int load)
@@ -59,6 +61,11 @@ reg_callback(char *name, int load)if (!load)return ; +if (!strcmp (name,"main" )) { +eppic_cmd (name,NULL ,0 ); +return ; + } +snprintf (fname,sizeof (fname),"%s_help" , name);if (eppic_chkfname (fname,0 )) {snprintf (fname,sizeof (fname),"%s_usage" , name);
Signature · 1 line
2.47.0
PATCH v2 3/14 dwarf_info: Support kernel address randomization
In reply to Tao LiuThere is a bug of not supporting randomized kernel address, this patch fix it.
--- erase_info.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
erase_info.c +4 -4
@@ -1881,7 +1881,7 @@ get_symbol_addr_all(char *name) {if (!strcmp (get_dwarf_module_name (),"vmlinux" )) { symbol_addr =get_symbol_addr (name);if (symbol_addr) -return symbol_addr; +return symbol_addr + info->kaslr_offset; vmlinux_searched =1 ; }
@@ -1942,9 +1942,9 @@ get_symbol_addr_all(char *name) {* this function is called with debuginfo set to a particular * kernel module and we are looking for symbol in vmlinux */ -if (!vmlinux_searched) -return get_symbol_addr (name); -else +if (!vmlinux_searched && !!(symbol_addr =get_symbol_addr (name))) { +return symbol_addr + info->kaslr_offset; + }else return NOT_FOUND_SYMBOL ; }
Signature · 1 line
2.47.0
Re: [PATCH v2][makedumpfile 03/14] dwarf_info: Support kernel address randomization
Quoted 3 lines · Tao Liu
There is a bug of not supporting randomized kernel address, this patch fix it.
Signed-off-by Tao Liu <[email protected]>
for applying this 03/14 patch separately before the 1.7.8 release,
Thanks,
Kazu
Quoted 30 lines
--- erase_info.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)erase_info.c +4
-4@@ -1881,7 +1881,7 @@ get_symbol_addr_all(char *name) {if (!strcmp (get_dwarf_module_name (),"vmlinux" )) { symbol_addr =get_symbol_addr (name);if (symbol_addr) -return symbol_addr; +return symbol_addr + info->kaslr_offset; vmlinux_searched =1 ; }@@ -1942,9 +1942,9 @@ get_symbol_addr_all(char *name) {* this function is called with debuginfo set to a particular * kernel module and we are looking for symbol in vmlinux */ -if (!vmlinux_searched) -return get_symbol_addr (name); -else +if (!vmlinux_searched && !!(symbol_addr =get_symbol_addr (name))) { +return symbol_addr + info->kaslr_offset; + }else return NOT_FOUND_SYMBOL ; }
YAMAZAKI MASAMITSU(山崎 真光)
Acked-byRe: [PATCH v2][makedumpfile 03/14] dwarf_info: Support kernel address randomization
Liu
Thanks your new patch.
I think the patche is good.
I ack this patch.
Thanks,
Masa
Quoted 41 lines · HAGIO KAZUHITO(萩尾 一仁)
Quoted 3 lines · Tao Liu
There is a bug of not supporting randomized kernel address, this patch fix it.
Signed-off-by Tao Liu <[email protected]>for applying this 03/14 patch separately before the 1.7.8 release,
Acked-by Kazuhito Hagio <[email protected]>Thanks,
KazuQuoted 30 lines
--- erase_info.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)erase_info.c +4
-4@@ -1881,7 +1881,7 @@ get_symbol_addr_all(char *name) {if (!strcmp (get_dwarf_module_name (),"vmlinux" )) { symbol_addr =get_symbol_addr (name);if (symbol_addr) -return symbol_addr; +return symbol_addr + info->kaslr_offset; vmlinux_searched =1 ; }@@ -1942,9 +1942,9 @@ get_symbol_addr_all(char *name) {* this function is called with debuginfo set to a particular * kernel module and we are looking for symbol in vmlinux */ -if (!vmlinux_searched) -return get_symbol_addr (name); -else +if (!vmlinux_searched && !!(symbol_addr =get_symbol_addr (name))) { +return symbol_addr + info->kaslr_offset; + }else return NOT_FOUND_SYMBOL ; }
Re: [PATCH v2][makedumpfile 03/14] dwarf_info: Support kernel address randomization
Hi YAMAZAKI & Kazu,
Thank you for reviewing and ack the patch!
Thanks,
Tao Liu
Quoted 54 lines · YAMAZAKI MASAMITSU(山崎 真光)
Liu
Thanks your new patch.
I think the patche is good.
I ack this patch.Thanks,
MasaQuoted 41 lines · HAGIO KAZUHITO(萩尾 一仁)
Quoted 3 lines · Tao Liu
There is a bug of not supporting randomized kernel address, this patch fix it.
Signed-off-by Tao Liu <[email protected]>for applying this 03/14 patch separately before the 1.7.8 release,
Acked-by Kazuhito Hagio <[email protected]>Thanks,
KazuQuoted 30 lines
--- erase_info.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)erase_info.c +4
-4@@ -1881,7 +1881,7 @@ get_symbol_addr_all(char *name) {if (!strcmp (get_dwarf_module_name (),"vmlinux" )) { symbol_addr =get_symbol_addr (name);if (symbol_addr) -return symbol_addr; +return symbol_addr + info->kaslr_offset; vmlinux_searched =1 ; }@@ -1942,9 +1942,9 @@ get_symbol_addr_all(char *name) {* this function is called with debuginfo set to a particular * kernel module and we are looking for symbol in vmlinux */ -if (!vmlinux_searched) -return get_symbol_addr (name); -else +if (!vmlinux_searched && !!(symbol_addr =get_symbol_addr (name))) { +return symbol_addr + info->kaslr_offset; + }else return NOT_FOUND_SYMBOL ; }
PATCH v2 4/14 dwarf_info: Fix a infinite recursion bug for rust symbols
In reply to Tao LiuThere is an infinite recursion bug noticed in rust symbols. The root cause is unclear to me. This patch will avoid the bug by skip the rust CompileUnits, since currently we don't need to deal with rust symbols.
--- dwarf_info.c | 7 +++++++ 1 file changed, 7 insertions(+)
dwarf_info.c +7 -0
@@ -1007,6 +1007,13 @@ get_debug_info(void)ERRMSG ("Can't get CU die.\n" ); goto out; } + +/* NOTE: currently Rust is not supported. */ +if (dwarf_srclang (&cu_die) == DW_LANG_Rust) { + off = next_off; +continue ; + } +search_die_tree (&cu_die, &found);if (found)break ;
Signature · 1 line
2.47.0
Re: [PATCH v2][makedumpfile 04/14] dwarf_info: Fix a infinite recursion bug for rust symbols
Quoted 6 lines · Tao Liu
There is an infinite recursion bug noticed in rust symbols. The root cause is unclear to me. This patch will avoid the bug by skip the rust CompileUnits, since currently we don't need to deal with rust symbols.
Signed-off-by Tao Liu <[email protected]>Suggested-by Kazuhito Hagio <[email protected]>
For applying this 04/14 patch separately before the 1.7.8 release, with adding the following note to the man page:
makedumpfile.8.in +1 -0
@@ -80,6 +80,7 @@ nested loops to traverse and erase kernel data. --eppic requires can be built from makedumpfile source. Refer to \fBhttp://code.google.com/p/eppic/\fR to build eppic library \fBlibeppic.a\fR and for more information on writing eppic macros. +NOTE: currently Rust is not supported in this function. .PP To analyze the first kernel's memory usage, makedumpfile can refer to \fIVMCOREINFO\fR instead of \fIVMLINUX\fR. \fIVMCOREINFO\fR contains the first
Thanks,
Kazu
Quoted 22 lines
--- dwarf_info.c | 7 +++++++ 1 file changed, 7 insertions(+)dwarf_info.c +7
-0@@ -1007,6 +1007,13 @@ get_debug_info(void)ERRMSG ("Can't get CU die.\n" ); goto out; } + +/* NOTE: currently Rust is not supported. */ +if (dwarf_srclang (&cu_die) == DW_LANG_Rust) { + off = next_off; +continue ; + } +search_die_tree (&cu_die, &found);if (found)break ;
YAMAZAKI MASAMITSU(山崎 真光)
Acked-byRe: [PATCH v2][makedumpfile 04/14] dwarf_info: Fix a infinite recursion bug for rust symbols
Liu
Thanks your new patch.
I think the patche is better than before. So I ack this patch.
As you pointed out, if exists Rust modules then processing for dwarf occur infinite loop. This will add the function to the stack and cause an error. I also have resarch this bug, but cannot find the root cause.
Therefore, I think it would be better not to support Rust now. However, the cause should eventually be revealed.
Thanks,
Masa
Quoted 49 lines · HAGIO KAZUHITO(萩尾 一仁)
Quoted 6 lines · Tao Liu
There is an infinite recursion bug noticed in rust symbols. The root cause is unclear to me. This patch will avoid the bug by skip the rust CompileUnits, since currently we don't need to deal with rust symbols.
Signed-off-by Tao Liu <[email protected]>Suggested-by Kazuhito Hagio <[email protected]>For applying this 04/14 patch separately before the 1.7.8 release, with adding the following note to the man page:
makedumpfile.8.in +1
-0@@ -80,6 +80,7 @@ nested loops to traverse and erase kernel data. --eppic requires can be built from makedumpfile source. Refer to \fBhttp://code.google.com/p/eppic/\fR to build eppic library \fBlibeppic.a\fR and for more information on writing eppic macros. +NOTE: currently Rust is not supported in this function. .PP To analyze the first kernel's memory usage, makedumpfile can refer to \fIVMCOREINFO\fR instead of \fIVMLINUX\fR. \fIVMCOREINFO\fR contains the firstAcked-by Kazuhito Hagio <[email protected]>Thanks,
KazuQuoted 22 lines
--- dwarf_info.c | 7 +++++++ 1 file changed, 7 insertions(+)dwarf_info.c +7
-0@@ -1007,6 +1007,13 @@ get_debug_info(void)ERRMSG ("Can't get CU die.\n" ); goto out; } + +/* NOTE: currently Rust is not supported. */ +if (dwarf_srclang (&cu_die) == DW_LANG_Rust) { + off = next_off; +continue ; + } +search_die_tree (&cu_die, &found);if (found)break ;
Re: [PATCH v2][makedumpfile 04/14] dwarf_info: Fix a infinite recursion bug for rust symbols
Hi YAMAZAKI & Kazu,
Thanks for Kazu's suggestion on the v1, and thank you for reviewing and ack the patch!
Thanks,
Tao Liu
Quoted 69 lines · YAMAZAKI MASAMITSU(山崎 真光)
Liu
Thanks your new patch.
I think the patche is better than before. So I ack this patch.As you pointed out, if exists Rust modules then processing for dwarf occur infinite loop. This will add the function to the stack and cause an error. I also have resarch this bug, but cannot find the root cause.
Therefore, I think it would be better not to support Rust now. However, the cause should eventually be revealed.
Thanks,
MasaQuoted 49 lines · HAGIO KAZUHITO(萩尾 一仁)
Quoted 6 lines · Tao Liu
There is an infinite recursion bug noticed in rust symbols. The root cause is unclear to me. This patch will avoid the bug by skip the rust CompileUnits, since currently we don't need to deal with rust symbols.
Signed-off-by Tao Liu <[email protected]>Suggested-by Kazuhito Hagio <[email protected]>For applying this 04/14 patch separately before the 1.7.8 release, with adding the following note to the man page:
makedumpfile.8.in +1
-0@@ -80,6 +80,7 @@ nested loops to traverse and erase kernel data. --eppic requires can be built from makedumpfile source. Refer to \fBhttp://code.google.com/p/eppic/\fR to build eppic library \fBlibeppic.a\fR and for more information on writing eppic macros. +NOTE: currently Rust is not supported in this function. .PP To analyze the first kernel's memory usage, makedumpfile can refer to \fIVMCOREINFO\fR instead of \fIVMLINUX\fR. \fIVMCOREINFO\fR contains the firstAcked-by Kazuhito Hagio <[email protected]>Thanks,
KazuQuoted 22 lines
--- dwarf_info.c | 7 +++++++ 1 file changed, 7 insertions(+)dwarf_info.c +7
-0@@ -1007,6 +1007,13 @@ get_debug_info(void)ERRMSG ("Can't get CU die.\n" ); goto out; } + +/* NOTE: currently Rust is not supported. */ +if (dwarf_srclang (&cu_die) == DW_LANG_Rust) { + off = next_off; +continue ; + } +search_die_tree (&cu_die, &found);if (found)break ;
Re: [PATCH v2][makedumpfile 04/14] dwarf_info: Fix a infinite recursion bug for rust symbols
Hi Tao,
applied the two patches with tweaks.
https://github.com/makedumpfile/makedumpfile/commit/f1f3561cdbdf3f5e626b6e15e3899f80b6d1a975
https://github.com/makedumpfile/makedumpfile/commit/a443eb654bc91858d22ccdf5d68a090df94f9423
Thanks,
Kazu
Quoted 79 lines · Tao Liu
Hi YAMAZAKI & Kazu,
Thanks for Kazu's suggestion on the v1, and thank you for reviewing and ack the patch!
Thanks,
Tao LiuQuoted 69 lines · YAMAZAKI MASAMITSU(山崎 真光)
Liu
Thanks your new patch.
I think the patche is better than before. So I ack this patch.As you pointed out, if exists Rust modules then processing for dwarf occur infinite loop. This will add the function to the stack and cause an error. I also have resarch this bug, but cannot find the root cause.
Therefore, I think it would be better not to support Rust now. However, the cause should eventually be revealed.
Thanks,
MasaQuoted 49 lines · HAGIO KAZUHITO(萩尾 一仁)
Quoted 6 lines · Tao Liu
There is an infinite recursion bug noticed in rust symbols. The root cause is unclear to me. This patch will avoid the bug by skip the rust CompileUnits, since currently we don't need to deal with rust symbols.
Signed-off-by Tao Liu <[email protected]>Suggested-by Kazuhito Hagio <[email protected]>For applying this 04/14 patch separately before the 1.7.8 release, with adding the following note to the man page:
makedumpfile.8.in +1
-0@@ -80,6 +80,7 @@ nested loops to traverse and erase kernel data. --eppic requires can be built from makedumpfile source. Refer to \fBhttp://code.google.com/p/eppic/\fR to build eppic library \fBlibeppic.a\fR and for more information on writing eppic macros. +NOTE: currently Rust is not supported in this function. .PP To analyze the first kernel's memory usage, makedumpfile can refer to \fIVMCOREINFO\fR instead of \fIVMLINUX\fR. \fIVMCOREINFO\fR contains the firstAcked-by Kazuhito Hagio <[email protected]>Thanks,
KazuQuoted 22 lines
--- dwarf_info.c | 7 +++++++ 1 file changed, 7 insertions(+)dwarf_info.c +7
-0@@ -1007,6 +1007,13 @@ get_debug_info(void)ERRMSG ("Can't get CU die.\n" ); goto out; } + +/* NOTE: currently Rust is not supported. */ +if (dwarf_srclang (&cu_die) == DW_LANG_Rust) { + off = next_off; +continue ; + } +search_die_tree (&cu_die, &found);if (found)break ;
PATCH v2 5/14 eppic dwarf: support anonymous structs member resolving
In reply to Tao LiuPreviously, apimember cannot dive into members which belongs to anonymous structs. E.g. mm_struct has structure as follows:
0x000027bc: DW_TAG_structure_type
DW_AT_name ("mm_struct")0x000027cb: DW_TAG_member
DW_AT_type (0x00008dde "structure ")
0x000027d2: DW_TAG_member
DW_AT_name ("cpu_bitmap")
0x00008dde: DW_TAG_structure_type0x00008deb: DW_TAG_member
DW_AT_type (0x00008d8e "structure ")
0x00008df2: DW_TAG_member
DW_AT_name ("mm_mt")
0x00008e00: DW_TAG_member
DW_AT_name ("mmap_base")0x00008e0e: DW_TAG_member
DW_AT_name ("mmap_legacy_base")
GET_DIE_NFIELDS_ALL() for mm_struct will only iterate 2 members, it will not dive into the anonymous structure and iterate members as mm_mt, mmap_base etc. In other words, there is no way to reference members as mm_struct.mm_mt.
This patch will make apimember recursive, when anonymous struct/union found, it will dive deeper to discover further members to make mm_struct.mm_mt reference possible.
E.g for the following eppic program:
int main()
{
struct mm_struct *m = (struct mm_struct *)0xffff96b7d1afa680;
printf("%lx\n", &m->mm_mt);
return 0;
}Before the patch:
File /tmp/test.c, line 4, Error: Unknown member name [mm_mt]
After the patch:
ffff96b7d1afa6c0 --- extension_eppic.c | 73 ++++++++++++++++++++++++++++++----------------- 1 file changed, 47 insertions(+), 26 deletions(-)
extension_eppic.c +47 -26
@@ -223,13 +223,16 @@ out:* Get the type, size and position information for a member of a structure. */ static char * -apimember (char *mname,ull idx,type_t *tm,member_t *m,ull *last_index) +apimember_internal (char *mname,ull idx,type_t *tm,member_t *m, +long offset_base,ull *target_index,ull *global_index) { -int index, nfields =-1 , size; +int index =0 , nfields =-1 , size;int nbits =0 , fbits =0 ;long offset;unsigned long long m_die, die_off = idx; -char *name =NULL ; +int type_flag; +unsigned long long t_die_off; +char *name; nfields =GET_DIE_NFIELDS_ALL (die_off);/*
@@ -239,43 +242,61 @@ apimember(char *mname, ull idx, type_t *tm, member_t *m, ull *last_index)if (nfields <=0 )return NULL ; -/* if we're being asked the next member in a getfirst/getnext -* sequence -*/ -if (mname && !mname[0 ] && last_index && (*last_index)) - index = *last_index; -else - index =0 ; -while (index < nfields) { + name =NULL ; size =GET_DIE_MEMBER_ALL (die_off, index, &offset, &name, &nbits, &fbits, &m_die);if (size <0 )return NULL ; -if (!mname || !mname[0 ] || !strcmp (mname, name)) { -eppic_member_ssize (m, size); -if (name) { -eppic_member_sname (m, name); -/* -* Free the memory allocated by makedumpfile. -*/ -free (name); +if (!name &&GET_DIE_ATTR_TYPE (m_die, &type_flag, &t_die_off)) { +char *ret; + ret =apimember_internal (mname, t_die_off, tm, m, + offset_base + offset, target_index, global_index); +if (ret) +return ret; +else { + index++; +continue ; } -else -eppic_member_sname (m,"" ); -eppic_member_soffset (m, offset); -eppic_member_snbits (m, nbits); -eppic_member_sfbit (m, fbits); - *last_index = index +1 ; -return drilldown (m_die, tm); + } + +if (*target_index == *global_index) { +if (!mname || !mname[0 ] || !strcmp (mname, name)) { +eppic_member_ssize (m, size); +if (name) { +eppic_member_sname (m, name); +/* +* Free the memory allocated by makedumpfile. +*/ +free (name); + } +else +eppic_member_sname (m,"" ); +eppic_member_soffset (m, offset + offset_base); +eppic_member_snbits (m, nbits); +eppic_member_sfbit (m, fbits); +return drilldown (m_die, tm); + }else +return eppic_strdup ("" ); } index++; + (*global_index)++; }return NULL ; } +static char * +apimember (char *mname,ull idx,type_t *tm,member_t *m,ull *target_index) +{ +char *ret; +ull global_index =0 ; + ret =apimember_internal (mname, idx, tm, m,0 , target_index, &global_index); + (*target_index)++; +return ret; +} +static int apigetctype (int ctype,char *name,type_t *tout) {
Signature · 1 line
2.47.0
PATCH v2 6/14 Enable page filtering for dwarf eppic
In reply to Tao LiuSince dwarf and btf have different apiops, so making it as parameter to eppic_init, and initialized respectively.
--- erase_info.c | 8 +++++--- extension_eppic.c | 24 +++++++++++++++++++++--- extension_eppic.h | 6 ++++-- 3 files changed, 30 insertions(+), 8 deletions(-)
erase_info.c +5 -3
@@ -2188,7 +2188,7 @@ get_die_member_all(unsigned long long die_off, int index, long *offset,/* Process the eppic macro using eppic library */ static int -process_eppic_file (char *name_config) +process_eppic_file (char *name_config,bool is_btf) {void *handle;void (*eppic_load)(char *), (*eppic_unload)(char *);
@@ -2207,7 +2207,9 @@ process_eppic_file(char *name_config)* Support specifying eppic macros in makedumpfile.conf file */ - eppic_init =dlsym (handle,"eppic_init" ); +if (!is_btf) { + eppic_init =dlsym (handle,"eppic_dwarf_init" ); + }if (!eppic_init) {ERRMSG ("Could not find eppic_init function\n" );return FALSE;
@@ -2354,7 +2356,7 @@ gather_filter_info(void) ret =process_config_file (info->name_filterconfig);if (info->name_eppic_config) - ret &=process_eppic_file (info->name_eppic_config); + ret &=process_eppic_file (info->name_eppic_config, false);/* * Remove modules symbol information, we dont need now.
extension_eppic.c +21 -3
@@ -25,6 +25,7 @@#include "extension_eppic.h" static int apigetctype (int ,char *,type_t *); +struct call_back *cb;/* * Most of the functions included in this file performs similar
@@ -437,7 +438,7 @@ apifindsym(char *p)return NULL ; } -apiops icops = { +apiops dwarf_icops = { apigetmem, apiputmem, apimember,
@@ -470,17 +471,27 @@ eppic_memset(VALUE_S *vaddr, VALUE_S *vch, VALUE_S *vlen)return eppic_makebtype (1 ); } +VALUE_S * +eppic_filter_pages (VALUE_S *p,VALUE_S *n) +{ +unsigned long pfn =eppic_getval (p); +unsigned long num =eppic_getval (n); + +UPDATE_FILTER_PAGE_INFO (pfn, num); +return eppic_makebtype (1 ); +} +/* Initialize eppic */ int -eppic_init (void *fun_ptr) +eppic_init (void *fun_ptr,apiops *ops,bool is_btf) { cb = (struct call_back *)fun_ptr;if (eppic_open () >=0 ) {/* Register call back functions */ -eppic_apiset (&icops,3 ,sizeof (long ),0 ); +eppic_apiset (ops,3 ,sizeof (long ),0 );/* set the new function callback */ eppic_setcallback (reg_callback);
@@ -489,8 +500,15 @@ eppic_init(void *fun_ptr)eppic_builtin ("int memset(char *, int, int)" , (bf_t *)eppic_memset); +eppic_builtin ("unsigned long filter_pages(unsigned long, unsigned long)" , + (bf_t *)eppic_filter_pages); +return 0 ; }return 1 ; } +int eppic_dwarf_init (void *fun_ptr) +{ +return eppic_init (fun_ptr, &dwarf_icops, false); +}
extension_eppic.h +4 -2
@@ -79,8 +79,7 @@ do { \fprintf (stderr, x); \ }while (0 ) - -struct call_back *cb; +extern struct call_back *cb;#define GET_DOMAIN_ALL cb->get_domain_all#define READMEM cb->readmem
@@ -92,5 +91,8 @@ struct call_back *cb;#define GET_DIE_NFIELDS_ALL cb->get_die_nfields_all#define GET_SYMBOL_ADDR_ALL cb->get_symbol_addr_all#define UPDATE_FILTER_INFO_RAW cb->update_filter_info_raw +#define UPDATE_FILTER_PAGE_INFO cb->update_filter_pages_info + +int eppic_init (void *,apiops *,bool );#endif /* _EXTENSION_EPPIC_H */
Signature · 1 line
2.47.0
PATCH v2 7/14 Implement kernel kallsyms resolving
In reply to Tao LiuThis patch will parse kernel's kallsyms data, and store them into a hash table so they can be referenced later in a fast speed.
--- Makefile | 2 +- kallsyms.c | 266 +++++++++++++++++++++++++++++++++++++++++++++++++ kallsyms.h | 22 ++++ makedumpfile.c | 3 + makedumpfile.h | 11 ++ 5 files changed, 303 insertions(+), 1 deletion(-) create mode 100644 kallsyms.c create mode 100644 kallsyms.h
Makefile +1 -1
@@ -45,7 +45,7 @@ CFLAGS_ARCH += -m32 endif SRC_BASE = makedumpfile.c makedumpfile.h diskdump_mod.h sadump_mod.h sadump_info.h -SRC_PART = print_info.c dwarf_info.c elf_info.c erase_info.c sadump_info.c cache.c tools.c printk.c detect_cycle.c +SRC_PART = print_info.c dwarf_info.c elf_info.c erase_info.c sadump_info.c cache.c tools.c printk.c detect_cycle.c kallsyms.c OBJ_PART=$(patsubst %.c,%.o,$(SRC_PART)) SRC_ARCH = arch/arm.c arch/arm64.c arch/x86.c arch/x86_64.c arch/ia64.c arch/ppc64.c arch/s390x.c arch/ppc.c arch/sparc64.c arch/mips64.c arch/loongarch64.c arch/riscv64.c OBJ_ARCH=$(patsubst %.c,%.o,$(SRC_ARCH))
kallsyms.c new file +266 -0
267 lines not on this page · raw message
kallsyms.h new file +22 -0
@@ -0,0 +1,22 @@ +#ifndef _KALLSYMS_H +#define _KALLSYMS_H + +#include <stdint.h> + +struct syment { +uint64_t value; +char *name; +struct syment *name_hash_next; +char type; +}; + +int init_kernel_kallsyms (void ); +int read_vmcoreinfo_kallsyms (void ); +struct syment *search_kallsyms_by_name (char *); +uint64_t get_kallsyms_value_by_name (char *); + +unsigned int hash_index (const char *,unsigned int ); +void hash_install (void **,unsigned int ,void *,const char *, +void *(*)(void *),void (*)(void *,void *)); + +#endif /* _KALLSYMS_H */
makedumpfile.c +3 -0
@@ -27,6 +27,7 @@#include <limits.h> #include <assert.h> #include <zlib.h> +#include "kallsyms.h" struct symbol_table symbol_table;struct size_table size_table;
@@ -3103,6 +3104,8 @@ read_vmcoreinfo_from_vmcore(off_t offset, unsigned long size, int flag_xen_hv)if (!read_vmcoreinfo ()) goto out; } +read_vmcoreinfo_kallsyms (); +close_vmcoreinfo (); ret = TRUE;
makedumpfile.h +11 -0
@@ -257,6 +257,7 @@ static inline int string_exists(char *s) { return (s ? TRUE : FALSE); }#define UINT (ADDR ) *((unsigned int *)(ADDR))#define ULONG (ADDR ) *((unsigned long *)(ADDR))#define ULONGLONG (ADDR ) *((unsigned long long *)(ADDR)) +#define VOID_PTR (ADDR ) *((void **)(ADDR))/*
@@ -1917,6 +1918,16 @@ struct symbol_table {* symbols on sparc64 arch */ unsigned long long vmemmap_table; + +/* +* kallsyms related +*/ +unsigned long long kallsyms_names; +unsigned long long kallsyms_num_syms; +unsigned long long kallsyms_token_table; +unsigned long long kallsyms_token_index; +unsigned long long kallsyms_offsets; +unsigned long long kallsyms_relative_base; };struct size_table {
Signature · 1 line
2.47.0
PATCH v2 8/14 Implement kernel btf resolving
In reply to Tao LiuThis patch will parse kernel's btf data. The data can be located via __start_BTF and __stop_BTF symbols which have been resolved by kallsyms of the previous patch.
The btf data is organized as follows: each one of kernel modules, including vmlinux itself, will have a btf_file struct describing its btf data, and holding an array of struct name_entry. By given the btf type id, we can resolve its name_entry fast by array index. In addition, name_entry can also be organized in hash table. So given a type name, we can also resolve its name_entry in a fast speed. In other words, both btf type id and btf type name can we get it resolved fast. Once we get the name_entry structure of the btf type, we can resolve its member/size etc easily.
Since all name_entry array starting from index 0, which cannot identify a btf type globally. So a uniq id is used, its value is accumulated by the location of the btf_file within btf_file_array and total quantity of btf types within each btf_file.
--- Makefile | 2 +- btf.c | 798 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ btf.h | 175 ++++++++++++ 3 files changed, 974 insertions(+), 1 deletion(-) create mode 100644 btf.c create mode 100644 btf.h
Makefile +1 -1
@@ -45,7 +45,7 @@ CFLAGS_ARCH += -m32 endif SRC_BASE = makedumpfile.c makedumpfile.h diskdump_mod.h sadump_mod.h sadump_info.h -SRC_PART = print_info.c dwarf_info.c elf_info.c erase_info.c sadump_info.c cache.c tools.c printk.c detect_cycle.c kallsyms.c +SRC_PART = print_info.c dwarf_info.c elf_info.c erase_info.c sadump_info.c cache.c tools.c printk.c detect_cycle.c kallsyms.c btf.c OBJ_PART=$(patsubst %.c,%.o,$(SRC_PART)) SRC_ARCH = arch/arm.c arch/arm64.c arch/x86.c arch/x86_64.c arch/ia64.c arch/ppc64.c arch/s390x.c arch/ppc.c arch/sparc64.c arch/mips64.c arch/loongarch64.c arch/riscv64.c OBJ_ARCH=$(patsubst %.c,%.o,$(SRC_ARCH))
btf.c new file +775 -0
775 lines not on this page · raw message
btf.h new file +175 -0
175 lines not on this page · raw message
Signature · 1 line
2.47.0
Re: [PATCH v2][makedumpfile 08/14] Implement kernel btf resolving
Quoted 13 lines · Tao Liu
This patch will parse kernel's btf data. The data can be located via
__start_BTFand__stop_BTFsymbols which have been resolved by kallsyms of the previous patch.The btf data is organized as follows: each one of kernel modules, including vmlinux itself, will have a
btf_filestruct describing its btf data, and holding an array of structname_entry. By given the btf type id, we can resolve itsname_entryfast by array index. In addition,name_entrycan also be organized in hash table. So given a type name, we can also resolve itsname_entryin a fast speed. In other words, both btf type id and btf type name can we get it resolved fast. Once we get thename_entrystructure of the btf type, we can resolve its member/size etc easily.
Quoted 6 lines
Since all
name_entryarray starting from index 0, which cannot identify a btf type globally. So a uniq id is used, its value is accumulated by the location of thebtf_filewithinbtf_file_arrayand total quantity of btf types within eachbtf_file.Signed-off-by Tao Liu <[email protected]>
Hi Tao,
I haven't read this patch in detail, but I wanted to ask if you had any particular reason to avoid using libbpf for parsing the BTF?
For my implementation of BTF parsing for drgn I was not aware of it, so I did the parsing entirely manually, similar to this. But for my recent attempt[1] at adding BTF support to makedumpfile (unaware of this patch), I used libbpf and found that it did reduce the amount of required code by a lot. These are specific functions I found useful:
- btf__new() -> returns a "struct btf *" for use with the library
- btf__type_by_id() -> returns the "struct btf_type *". Then, the linux/btf.h header provides several inline functions to work with it:
- btf_members(), btf_vlen()
- btf_member_bit_offset()
- btf__pointer_size()
- btf__name_by_offset() -> returns the name for a string offset
- btf__find_by_name() & btf__find_by_name_kind() -> looks up a type by name. Note that this does a linear search, so your hash table approach is better for random lookups.
- btf__resolve_size()
So with libbpf, the only thing you'd really need to implement is resolving member offsets (supporting anonyomus structs, recursively, etc), and a hash table for string lookups, if you feel it's necessary. For module BTF, btf__new_split() is available as well.
Thanks,
Stephen
Quoted 985 lines
--- Makefile | 2 +- btf.c | 798 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ btf.h | 175 ++++++++++++ 3 files changed, 974 insertions(+), 1 deletion(-) create mode 100644 btf.c create mode 100644 btf.hMakefile +1
-1@@ -45,7 +45,7 @@ CFLAGS_ARCH += -m32 endif SRC_BASE = makedumpfile.c makedumpfile.h diskdump_mod.h sadump_mod.h sadump_info.h -SRC_PART = print_info.c dwarf_info.c elf_info.c erase_info.c sadump_info.c cache.c tools.c printk.c detect_cycle.c kallsyms.c +SRC_PART = print_info.c dwarf_info.c elf_info.c erase_info.c sadump_info.c cache.c tools.c printk.c detect_cycle.c kallsyms.c btf.c OBJ_PART=$(patsubst %.c,%.o,$(SRC_PART)) SRC_ARCH = arch/arm.c arch/arm64.c arch/x86.c arch/x86_64.c arch/ia64.c arch/ppc64.c arch/s390x.c arch/ppc.c arch/sparc64.c arch/mips64.c arch/loongarch64.c arch/riscv64.c OBJ_ARCH=$(patsubst %.c,%.o,$(SRC_ARCH))btf.c new file +775
-0@@ -0,0 +1,775 @@ +#include "btf.h" +#include <stdio.h> +#include <assert.h> +#include <stdlib.h> +#include <string.h> +#include <unistd.h> +#include <sys/mman.h> +#include <sys/stat.h> +#include <fcntl.h> +#include <libelf.h> +#include <gelf.h> +#include <dirent.h> +#include "kallsyms.h" +#include "makedumpfile.h" + +// btf_files_array[0] must be kernel itself +// btf_files_array[1..] are kernel modules +static struct btf_file **btf_files_array =NULL ; +static int btf_files_array_index =0 ; + +static inline uint16_t btf_vlen (uint32_t info) +{ +return info &0xFFFF ; +} + +static inline uint32_t btf_kind_flag (uint32_t info) +{ +return info & (1 <<31 ); +} + +static inline uint32_t btf_member_bit_offset (uint32_t value) +{ +return value &0xffffff ; +} + +static inline uint32_t btf_member_bitfield_size (uint32_t value) +{ +return value >>24 ; +} + +static struct btf_type *btf_next (struct btf_type *tp) +{ +uintptr_t next = (uintptr_t )&tp[1 ]; + +switch (btf_kind (tp->info)) { +case BTF_KIND_INT : + next +=sizeof (uint32_t ); +break ; + +case BTF_KIND_ARRAY : + next +=sizeof (struct btf_array ); +break ; + +case BTF_KIND_STRUCT : +case BTF_KIND_UNION : + next +=btf_vlen (tp->info) *sizeof (struct btf_member ); +break ; + +case BTF_KIND_ENUM : + next +=btf_vlen (tp->info) *sizeof (struct btf_enum ); +break ; + +case BTF_KIND_FUNC_PROTO : + next +=btf_vlen (tp->info) *sizeof (struct btf_param ); +break ; + +case BTF_KIND_VAR : + next +=sizeof (struct btf_var ); +break ; + +case BTF_KIND_DATASEC : + next +=btf_vlen (tp->info) *sizeof (struct btf_var_secinfo ); +break ; + +case BTF_KIND_DECL_TAG : + next +=sizeof (struct btf_decl_tag ); +break ; + +case BTF_KIND_ENUM64 : + next +=btf_vlen (tp->info) *sizeof (struct btf_enum64 ); +break ; + +case BTF_KIND_PTR : +case BTF_KIND_FWD : +case BTF_KIND_TYPEDEF : +case BTF_KIND_VOLATILE : +case BTF_KIND_CONST : +case BTF_KIND_RESTRICT : +case BTF_KIND_FUNC : +case BTF_KIND_FLOAT : +case BTF_KIND_TYPE_TAG : +break ;// no extra data + +default : +__builtin_unreachable (); + } +return (struct btf_type *)next; +} + +#define NAME_HASH 512 +static struct name_entry *name_hash_table[NAME_HASH ] = {0 }; + +static void *get_name_entry_next (void *entry) +{ +return ((struct name_entry *)entry)->name_hash_next; +} + +static void set_name_entry_next (void *entry,void *next) +{ + ((struct name_entry *)entry)->name_hash_next = next; +} + +static void name_hash_install (struct name_entry *en) +{ +hash_install ((void **)name_hash_table,NAME_HASH , en, en->name, + get_name_entry_next, set_name_entry_next); +} + +static unsigned int name_hash_index (char *name) +{ +return hash_index (name,NAME_HASH ); +} + +int read_file_at_offset (char *f,int f_off,int r_size,void *buf) +{ +int got =0 ; +int r, fd, ret =-1 ; + + fd =open (f,O_RDONLY ); +if (fd <0 ) { +fprintf (stderr,"%s: Failed to open file %s\n" , __func__, f); + goto out; + } + +while (got < r_size) { + r =pread (fd, buf + got, r_size - got, f_off + got); +if (r <0 ) { +fprintf (stderr,"%s: Failed to read file %s\n" , __func__, f); + goto clean_fd; + } + got += r; + } + ret = got; + +clean_fd: +close (fd); +out: +return ret; +} + +static char *get_str_by_name_off (struct btf_file *bf,uint32_t name_off) +{ +struct btf_file *bf_vmlinux = btf_files_array[0 ]; + +if (bf != bf_vmlinux && name_off >= bf_vmlinux->str_cache_len) +return bf->str_cache + name_off - bf_vmlinux->str_cache_len; +else +return bf_vmlinux->str_cache + name_off; +} + +int get_btf_type_by_type_id (struct btf_file *bf_in,uint32_t id_in, +struct btf_type *bt_out,struct name_entry **en_out) +{ +struct name_entry *en; +struct btf_file *bf_vmlinux = btf_files_array[0 ]; + +if (bf_in != bf_vmlinux && id_in > bf_vmlinux->array_len) { + en = bf_in->array[id_in - bf_vmlinux->array_len -1 ]; + }else { + en = bf_vmlinux->array[id_in -1 ]; + } +if (en_out) + *en_out = en; +return read_file_at_offset (en->bf->file_name, + en->btf_type_offset + en->bf->types_data_offset, +sizeof (*bt_out), bt_out); +} + +/* +* Parse the btf data and install elements into hashtable and array for quick +* reference. +*/ +static int parse_btf_data (char *file_path,char *data_start,uint32_t data_len) +{ +struct btf_header *hdr = (struct btf_header *)data_start; +struct btf_file *bf =NULL ; +void *type_start; +char *str_start; +struct btf_type *tp; +struct name_entry *en; +int i, j, scale; + +/* We do some check first */ +if (hdr->magic !=BTF_MAGIC ) { +fprintf (stderr,"%s: Invalid BTF magic in %s\n" , + __func__, file_path); + goto out; + } +if (hdr->hdr_len !=sizeof (*hdr)) { +fprintf (stderr,"%s: Invalid BTF header length in %s\n" , + __func__, file_path); + goto out; + } +if (hdr->hdr_len + hdr->str_off + hdr->str_len > data_len) { +fprintf (stderr,"%s: String section exceeds data length in %s\n" , + __func__, file_path); + goto out; + } +if (hdr->hdr_len + hdr->type_off + hdr->type_len > data_len) { +fprintf (stderr,"%s: Type section exceeds data length in %s\n" , + __func__, file_path); + goto out; + } + +/* Let's start parsing */ + bf = (struct btf_file *)malloc (sizeof (*bf)); +if (!bf) + goto no_mem; +memset (bf,0 ,sizeof (*bf)); + +/* Enlarge array when reach power of 2 */ + btf_files_array[btf_files_array_index++] = bf; +if ((btf_files_array_index & (btf_files_array_index -1 )) ==0 ) { +struct btf_file **tmp = (struct btf_file **)reallocarray (btf_files_array, + btf_files_array_index <<1 , +sizeof (struct btf_file *)); +if (!tmp) + goto no_mem; + btf_files_array = tmp; + } + + type_start = data_start + hdr->hdr_len + hdr->type_off; + str_start = data_start + hdr->hdr_len + hdr->str_off; + + bf->str_cache =malloc (hdr->str_len); +if (!bf->str_cache) + goto no_mem; +memcpy (bf->str_cache, str_start, hdr->str_len); + bf->str_cache_len = hdr->str_len; + + bf->array_len =64 ; + bf->array = (struct name_entry **)calloc (bf->array_len, +sizeof (struct name_entry *)); +if (!bf->array) + goto no_mem; + + bf->file_name =strdup (file_path); +if (!bf->file_name) + goto no_mem; + bf->types_data_offset = (char *)type_start - data_start; + +for (tp = (struct btf_type *)type_start, i =0 ; + (void *)tp < type_start + hdr->type_len; + tp =btf_next (tp), i++) { + en = (struct name_entry *)malloc (sizeof (struct name_entry )); +if (!en) { + bf->array_len = i; + goto no_mem; + } +memset (en,0 ,sizeof (struct name_entry )); + + en->btf_type_offset = (void *)tp - type_start; + en->id = i +1 ; + en->bf = bf; +if (tp->name_off) { + en->name =get_str_by_name_off (bf, tp->name_off); +if (en->name[0 ]) +name_hash_install (en); + } + bf->array[i] = en; + +/* Now deal with sub elements which also have a name */ +if (btf_kind (tp->info) ==BTF_KIND_ENUM || +btf_kind (tp->info) ==BTF_KIND_ENUM64 ) { + scale =btf_kind (tp->info) ==BTF_KIND_ENUM ? +sizeof (struct btf_enum ) :sizeof (struct btf_enum64 ); +for (j =0 ; j <btf_vlen (tp->info); j++) { + en = (struct name_entry *)malloc (sizeof (struct name_entry )); +if (!en) { + bf->array_len = i +1 ; + goto no_mem; + } +memset (en,0 ,sizeof (struct name_entry )); + en->id =0 ; + en->p_id = i +1 ; + en->p_id += btf_files_array_index ==1 ? +0 : btf_files_array[0 ]->array_len; + en->bf = bf; + en->name =get_str_by_name_off (bf, + *(uint32_t *)((char *)&tp[1 ] + j * scale)); +// printf("%s\n", en->name); +name_hash_install (en); + } + } + +/* Enlarge array when over 3/4 */ +if (i > (bf->array_len >>2 ) *3 ) { +struct name_entry **tmp = (struct name_entry **)reallocarray (bf->array, + bf->array_len <<1 ,sizeof (struct name_entry *)); +if (!tmp) + goto no_mem; + bf->array = tmp; + bf->array_len <<=1 ; + } + } + bf->array_len = i; +return 0 ; +no_mem: +/* All the memory free will be dealed later. */ +fprintf (stderr,"%s: Not enough memory!\n" , __func__); +out: +return -1 ; +} + +/* +* Used by search_name_by_cond() for searching specific type: +* such as struct/union/typedef +*/ +static bool is_type (struct name_entry *sp,void *data_in,void *data_out) +{ +struct btf_type *bt = (struct btf_type *)data_out; +int type = *(int *)data_in; +if (sp->id) { +read_file_at_offset (sp->bf->file_name, + sp->btf_type_offset + sp->bf->types_data_offset, +sizeof (*bt), bt); +return btf_kind (bt->info) == type; + } +return false; +} + +static struct name_entry *search_name_by_cond (char *name, +bool (*fn)(struct name_entry *,void *,void *), +void *data_in,void *data_out) +{ +unsigned int index; +struct name_entry *sp; + + index =name_hash_index (name); +for (sp = name_hash_table[index]; sp; sp = sp->name_hash_next) { +if (!strcmp (name, sp->name) &&fn (sp, data_in, data_out)) { +return sp; + } + } +return sp; +} + +// caller should prepare bt +void resolve_typedef (struct name_entry *en_in,struct name_entry **en_out, +struct btf_type *bt) +{ +uint32_t id; +struct name_entry *en; + +read_file_at_offset (en_in->bf->file_name, + en_in->btf_type_offset + en_in->bf->types_data_offset, +sizeof (*bt), bt); +if (btf_kind (bt->info) ==BTF_KIND_TYPEDEF ) { + id = bt->type; +get_btf_type_by_type_id (en_in->bf, id, bt, &en); +return resolve_typedef (en, en_out, bt); + }else { + *en_out = en_in; + } +} + +struct cond_args { +int index; +char *name; +}; + +static bool cond (struct cond_args *c1,struct cond_args *c2) +{ +if (c1->name && c2->name) +/* Check if the member name is what we want */ +return !strcmp (c1->name, c2->name); +else +/* Check if the member index is what we want */ +return c1->index == c2->index; +} + +static bool get_internal_member_info (struct name_entry *en,int base_position, +int *global_index,struct cond_args *tar_arg, +struct member_info *mi,bool initial_dive) +{ +struct btf_type bt, sub_bt; +int member_num; +char *member_array_buf =NULL ; +int i; +struct btf_member *bm; +struct btf_array *ba; +int bm_position; +struct name_entry *sub_en; +bool res; +struct cond_args cur_arg = {0 }; + +read_file_at_offset (en->bf->file_name, + en->btf_type_offset + en->bf->types_data_offset, +sizeof (bt), &bt);// this struct +if (initial_dive || + ((!en->name || en->name[0 ] =='\0' ) && + ((btf_kind (bt.info) ==BTF_KIND_STRUCT ) || + (btf_kind (bt.info) ==BTF_KIND_UNION )))) { +/* anonymous struct/union, dive into */ + member_num =btf_vlen (bt.info); + member_array_buf =calloc (member_num,sizeof (struct btf_member )); +if (!member_array_buf) { +fprintf (stderr,"%s: Not enough memory!\n" , __func__); +return false; + } +read_file_at_offset (en->bf->file_name, + en->btf_type_offset + en->bf->types_data_offset +sizeof (bt), + member_num *sizeof (struct btf_member ), member_array_buf); +for (i =0 , bm = (struct btf_member *)member_array_buf; + i < member_num; i++) { +if (btf_kind_flag (bt.info)) { + bm_position = base_position + +btf_member_bit_offset (bm[i].offset); + mi->bits =btf_member_bitfield_size (bm[i].offset); + }else { + bm_position = base_position + + bm[i].offset; + } + mi->sname =get_str_by_name_off (en->bf, bm[i].name_off); +get_btf_type_by_type_id (en->bf, bm[i].type, + &sub_bt, &sub_en); +// Dive into this member + res =get_internal_member_info (sub_en, bm_position, global_index, + tar_arg, mi, false); +if (res) { +free (member_array_buf); +return res; + } + } +free (member_array_buf); +return false; + } + + cur_arg.index = *global_index; + cur_arg.name = mi->sname; +if (cond (&cur_arg, tar_arg)) { + mi->bit_pos = base_position; +resolve_typedef (en, &sub_en, &bt); + mi->uniq_id =id_to_uniq_id (sub_en->id, sub_en->bf); +if (btf_kind (bt.info) ==BTF_KIND_PTR ) { +/* +* BUG? No pointer size in btf, +* 32bit btf target on 64bit machine +*/ + mi->size =sizeof (void *); + mi->type ="char *" ; + }else if (btf_kind (bt.info) ==BTF_KIND_ARRAY ) { + en = sub_en; + ba = (struct btf_array *)malloc (sizeof (struct btf_array )); +if (!ba) { +fprintf (stderr,"%s: Not enough memory!\n" , __func__); +return false; + } +read_file_at_offset (en->bf->file_name, + en->btf_type_offset + en->bf->types_data_offset +sizeof (bt), +sizeof (struct btf_array ), ba); + mi->size = ba->nelems;// array elements +get_btf_type_by_type_id (en->bf, ba->type, &sub_bt, &sub_en); +resolve_typedef (sub_en, &en, &bt); +free (ba); + mi->size *= bt.size;// element size + mi->type =NULL ;// "char [64]? Leave it NULL for now" + }else { + mi->size = bt.size; + mi->type =get_str_by_name_off (sub_en->bf, bt.name_off); + } +return true; + }else { + (*global_index)++; +return false; + } +} + +static bool get_member_info_by_index (struct name_entry *en,int target_index, +struct member_info *mi) +{ +int g_index =1 ; +struct cond_args args = {0 }; + args.index = target_index; +memset (mi,0 ,sizeof (*mi)); +return get_internal_member_info (en,0 , &g_index, &args, mi, true); +} + +static bool get_member_info_by_member_name (struct name_entry *en,char *member_name, +struct member_info *mi) +{ +int g_index =1 ; +struct cond_args args = {0 }; + args.name = member_name; +memset (mi,0 ,sizeof (*mi)); +return get_internal_member_info (en,0 , &g_index, &args, mi, true); +} + +/* +* Entry for query struct members +* Return: struct size, mi_out: member details +*/ +uint32_t get_struct_member_by_name (char *struct_name,char *member_name, +struct member_info *mi_out) +{ +struct name_entry *en; +struct btf_type bt = {0 }; +int type =BTF_KIND_STRUCT ; + + en =search_name_by_cond (struct_name, is_type, (void *)&type, (void *)&bt); +if (!en) + goto out; +if (get_member_info_by_member_name (en, member_name, mi_out)) +return bt.size; +out: +return 0 ; +} + +/* +* Entry for query type members, similar as above +* Return: found-the-type, mi_out: member details +*/ +static uint32_t uniq_id_to_id (uint32_t ,struct btf_file **); +bool get_type_member_by_index (uint64_t type_uniq_id,int member_index, +struct member_info *mi_out) +{ +uint32_t id; +struct btf_file *bf; + + id =uniq_id_to_id ((uint32_t )type_uniq_id, &bf); +return get_member_info_by_index (bf->array[id -1 ], member_index, mi_out); +} + +uint32_t id_to_uniq_id (uint32_t id,struct btf_file *bf) +{ +int i; +uint32_t uniq_id =0 ; +for (i =0 ; i < btf_files_array_index; i++) { +if (btf_files_array[i] != bf) + uniq_id += btf_files_array[i]->array_len; +else +return id + uniq_id; + } +__builtin_unreachable (); +assert (false); +} + +static uint32_t uniq_id_to_id (uint32_t uid,struct btf_file **bf) +{ +int i; + +for (i =0 ; i < btf_files_array_index; i++) { +if (uid > btf_files_array[i]->array_len) { + uid -= btf_files_array[i]->array_len; + }else { +if (bf) + *bf = btf_files_array[i]; +return uid; + } + + } +__builtin_unreachable (); +assert (false); +} + +// api for eppic +uint32_t get_type_size_by_name (char *type_name,int type,uint32_t *uniq_id) +{ +struct name_entry *en; +struct btf_type bt = {0 }; + + en =search_name_by_cond (type_name, is_type, (void *)&type, (void *)&bt); +if (!en) + goto out; +if (uniq_id) + *uniq_id =id_to_uniq_id (en->id, en->bf); +return bt.size; +out: +return 0 ; +} + +// Caller should prepare bt_out +struct name_entry *get_en_by_uniq_id (uint32_t uniq_id,struct btf_type *bt_out) +{ +uint32_t id; +struct btf_file *bf; +struct name_entry *en; +struct name_entry *en_sub; + + id =uniq_id_to_id ((uint32_t )uniq_id, &bf); + en = bf->array[id -1 ]; +resolve_typedef (en, &en_sub, bt_out); +return en_sub; +} + +/* Deal with btf file */ +static int btf_file_init (int fd) +{ +struct stat f_stat; +char *buf =NULL ; +char proc_path[32 ]; +char real_path[512 ]; +int ret =-1 ; + +memset (real_path,0 ,sizeof (real_path)); +if (fstat (fd, &f_stat) <0 ) { +fprintf (stderr,"%s: fstat fail!\n" , __func__); + goto out; + } + buf =malloc (f_stat.st_size); +if (!buf) { +fprintf (stderr,"%s: Not enough memory!\n" , __func__); + goto out; + } + +snprintf (proc_path,sizeof (proc_path),"/proc/self/fd/%d" , fd); +readlink (proc_path, real_path,sizeof (real_path) -1 ); +if (read_file_at_offset (real_path,0 , f_stat.st_size, buf) <0 ) + goto out; + + ret =parse_btf_data (real_path, buf, f_stat.st_size); +out: +if (buf) +free (buf); +return ret; +} + +/* Deal with elf file which contains .BTF section */ +static int elf_file_init (int fd) +{ +char proc_path[32 ]; +char real_path[512 ]; +Elf *elf =NULL ; +Elf_Scn *scn =NULL ; +size_t shstrndx; +GElf_Shdr shdr; +Elf_Data *data =NULL ; +char *str, *databuf =NULL ; +int ret =-1 ; + +snprintf (proc_path,sizeof (proc_path),"/proc/self/fd/%d" , fd); +readlink (proc_path, real_path,sizeof (real_path) -1 ); + +if (elf_version (EV_CURRENT ) ==EV_NONE ) + goto elf_err; + elf =elf_begin (fd,ELF_C_READ ,NULL ); +if (!elf ||elf_getshdrstrndx (elf, &shstrndx)) + goto elf_err; +while ((scn =elf_nextscn (elf, scn)) !=NULL ) { +if (!gelf_getshdr (scn, &shdr)) +continue ; + str =elf_strptr (elf, shstrndx, shdr.sh_name); +if (!strcmp (str,".BTF" )) +break ; + } +if (!scn) { +fprintf (stderr,"%s: No .BTF found in %s!\n" , __func__, real_path); + goto out; + } + + data =elf_rawdata (scn, data); + databuf =malloc (data->d_size); +if (!databuf) { +fprintf (stderr,"%s: Not enough memory!\n" , __func__); + goto out; + } +memcpy (databuf, data->d_buf, data->d_size); +elf_end (elf); + elf =NULL ; + ret =parse_btf_data (real_path, databuf, data->d_size); + goto out; + +elf_err: +fprintf (stderr,"%s: elf error in %s!\n" , __func__, real_path); +out: +if (databuf) +free (databuf); +if (elf) +elf_end (elf); +return ret; +} + +/* +* Entry for parse btf file and elf file. +*/ +static int file_init (char *name) +{ +int fd =0 , ret =-1 ; +char buf[4 ] = {0 }; + +/* Will be enlarged automatically */ +if (!btf_files_array) { + btf_files_array =calloc (1 ,sizeof (struct btf_file *)); +if (!btf_files_array) { +fprintf (stderr,"%s: Not enough memory!\n" , __func__); + goto out; + } + } + + fd =open (name,O_RDONLY ); +if (read_file_at_offset (name,0 ,sizeof (buf), buf) <0 ) + goto out; +if (*(u_int16_t *)&buf ==BTF_MAGIC ) { + ret =btf_file_init (fd); + }else if (*(u_int32_t *)&buf == (((u_int32_t )ELFMAG0 <<24 ) | + ((u_int32_t )ELFMAG1 <<16 ) | + ((u_int32_t )ELFMAG2 <<8 ) | + ((u_int32_t )ELFMAG3 <<0 ))) { + ret =elf_file_init (fd); + }else { +__builtin_unreachable (); + } + +out: +if (fd) +close (fd); +return ret; +} + +/* For pure buffer data, wrap it as a file, and handover to file_init() */ +char temp[] ="/tmp/btf_XXXXXX" ; +static int init_btf_from_buf (char *mod_name,char *buf,int size) +{ +int fd; +char name_buf[64 ]; +int w, got =0 ; +static char *temp_dir =NULL ; + +if (!temp_dir) + temp_dir =mkdtemp (temp); +snprintf (name_buf,sizeof (name_buf),"%s/%s" , temp_dir, mod_name); + fd =open (name_buf,O_RDWR |O_CREAT ,S_IRUSR |S_IWUSR ); +if (fd <0 ) { +fprintf (stderr,"%s: open fail in %s!\n" , __func__, name_buf); +return -1 ; + } + +while (got < size) { + w =pwrite (fd, buf + got, size - got, got); +if (w <0 ) { +fprintf (stderr,"%s: pwrite fail in %s!\n" , __func__, name_buf); +return -1 ; + } + got += w; + } +close (fd); +return file_init (name_buf); +} + +int init_kernel_btf (void ) +{ +uint64_t size; +char *buf; +int ret; + +uint64_t start_btf =get_kallsyms_value_by_name ("__start_BTF" ); +uint64_t stop_btf =get_kallsyms_value_by_name ("__stop_BTF" ); +if (!start_btf || !stop_btf) { +fprintf (stderr,"%s: symbol __start/stop_BTF not found!\n" , __func__); +return -1 ; + } + + size = stop_btf - start_btf; + buf = (char *)malloc (size); +if (!buf) { +fprintf (stderr,"%s: Not enough memory!\n" , __func__); +return -1 ; + } +readmem (VADDR , start_btf, buf, size); + ret =init_btf_from_buf ("vmlinux" , buf, size); +free (buf); + +return ret; +} +btf.h new file +175
-0@@ -0,0 +1,175 @@ +#ifndef _BTF_H +#define _BTF_H +#include <stdint.h> +#include <stdbool.h> + +typedef uint8_t __u8 ; +typedef uint16_t __u16 ; +typedef uint32_t __u32 ; +typedef int32_t __s32 ; + +#define BTF_MAGIC 0xeb9f + +enum { +BTF_KIND_UNKN =0 ,/* Unknown */ +BTF_KIND_INT =1 ,/* Integer */ +BTF_KIND_PTR =2 ,/* Pointer */ +BTF_KIND_ARRAY =3 ,/* Array */ +BTF_KIND_STRUCT =4 ,/* Struct */ +BTF_KIND_UNION =5 ,/* Union */ +BTF_KIND_ENUM =6 ,/* Enumeration */ +BTF_KIND_FWD =7 ,/* Forward */ +BTF_KIND_TYPEDEF =8 ,/* Typedef */ +BTF_KIND_VOLATILE =9 ,/* Volatile */ +BTF_KIND_CONST =10 ,/* Const */ +BTF_KIND_RESTRICT =11 ,/* Restrict */ +BTF_KIND_FUNC =12 ,/* Function */ +BTF_KIND_FUNC_PROTO =13 ,/* Function Proto */ +BTF_KIND_VAR =14 ,/* Variable */ +BTF_KIND_DATASEC =15 ,/* Section */ +BTF_KIND_FLOAT =16 ,/* Floating point */ +BTF_KIND_DECL_TAG =17 ,/* Decl Tag */ +BTF_KIND_TYPE_TAG =18 ,/* Type Tag */ +BTF_KIND_ENUM64 =19 ,/* Enumeration up to 64-bit values */ + +NR_BTF_KINDS , +BTF_KIND_MAX =NR_BTF_KINDS -1 , +}; + +struct btf_type { +__u32 name_off; + +/* "info" bits arrangement +* bits 0-15: vlen (e.g. # of struct's members) +* bits 16-23: unused +* bits 24-27: kind (e.g. int, ptr, array...etc) +* bits 28-30: unused +* bit 31: kind_flag, currently used by +* struct, union and fwd +*/ +__u32 info; + +/* "size" is used by INT, ENUM, STRUCT, UNION and DATASEC. +* "size" tells the size of the type it is describing. +* +* "type" is used by PTR, TYPEDEF, VOLATILE, CONST, RESTRICT, +* FUNC, FUNC_PROTO, VAR, DECL_TAG and TYPE_TAG. +* "type" is a type_id referring to another type. +*/ +union { +__u32 size; +__u32 type; + }; +}; + +struct btf_header { +__u16 magic; +__u8 version; +__u8 flags; +__u32 hdr_len; + +/* All offsets are in bytes relative to the end of this header */ +__u32 type_off;/* offset of type section */ +__u32 type_len;/* length of type section */ +__u32 str_off;/* offset of string section */ +__u32 str_len;/* length of string section */ +}; + +struct btf_array { +__u32 type; +__u32 index_type; +__u32 nelems; +}; + +struct btf_member { +__u32 name_off; +__u32 type; +/* If the type info kind_flag is set, the btf_member offset +* contains both member bitfield size and bit offset. The +* bitfield size is set for bitfield members. If the type +* info kind_flag is not set, the offset contains only bit +* offset. +*/ +__u32 offset; +}; + +struct btf_enum { +__u32 name_off; +__s32 val; +}; + +struct btf_enum64 { +__u32 name_off; +__u32 val_lo32; +__u32 val_hi32; +}; + +struct btf_param { +__u32 name_off; +__u32 type; +}; + +struct btf_var { +__u32 linkage; +}; + +struct btf_var_secinfo { +__u32 type; +__u32 offset; +__u32 size; +}; + +struct btf_decl_tag { +__s32 component_idx; +}; + +/*************************************/ +struct btf_file { +char *file_name; +char *str_cache; +uint32_t str_cache_len; +uint32_t types_data_offset; +uint32_t array_len; +struct name_entry **array; +}; + +struct name_entry { +union { +uint32_t btf_type_offset; +uint32_t p_id; + }; + +uint32_t id; +char *name; +struct btf_file *bf; +struct name_entry *name_hash_next; +}; + +struct member_info { +char *sname;// member name +char *type;// member type: int, long etc +uint32_t bit_pos;// member position in bits +uint32_t bits;// member width in bits +uint32_t size;// member size in bytes +int uniq_id;// uniq_id of btf +}; + +/*************************************/ + +int read_file_at_offset (char *,int ,int ,void *); +int init_kernel_btf (void ); +uint32_t get_struct_member_by_name (char *,char *,struct member_info *); +bool get_type_member_by_index (uint64_t ,int ,struct member_info *); +uint32_t get_type_size_by_name (char *,int ,uint32_t *); +struct name_entry *get_en_by_uniq_id (uint32_t ,struct btf_type *); +void resolve_typedef (struct name_entry *,struct name_entry **,struct btf_type *); +int get_btf_type_by_type_id (struct btf_file *,uint32_t , +struct btf_type *,struct name_entry **); +uint32_t id_to_uniq_id (uint32_t ,struct btf_file *); + +static inline uint32_t btf_kind (uint32_t info) +{ +return (info &0x1F000000 ) >>24 ; +} + +#endif /* _BTF_H */ Signature · 1 line
2.47.0
Re: [PATCH v2][makedumpfile 08/14] Implement kernel btf resolving
Hi Stephen,
Thanks for your comments!
Quoted 20 lines · Stephen Brennan
Quoted 18 lines · Tao Liu
This patch will parse kernel's btf data. The data can be located via
__start_BTFand__stop_BTFsymbols which have been resolved by kallsyms of the previous patch.The btf data is organized as follows: each one of kernel modules, including vmlinux itself, will have a
btf_filestruct describing its btf data, and holding an array of structname_entry. By given the btf type id, we can resolve itsname_entryfast by array index. In addition,name_entrycan also be organized in hash table. So given a type name, we can also resolve itsname_entryin a fast speed. In other words, both btf type id and btf type name can we get it resolved fast. Once we get thename_entrystructure of the btf type, we can resolve its member/size etc easily.Since all
name_entryarray starting from index 0, which cannot identify a btf type globally. So a uniq id is used, its value is accumulated by the location of thebtf_filewithinbtf_file_arrayand total quantity of btf types within eachbtf_file.
Quoted 6 lines
Quoted 1 line
Signed-off-by Tao Liu <[email protected]>Hi Tao,
I haven't read this patch in detail, but I wanted to ask if you had any particular reason to avoid using libbpf for parsing the BTF?
Actually I manually parse BTF data due to the following concerns:
1) Learn BTF data structure by manually parsing. I haven't worked on BTF previously, and I prefer the way of doing things from scratch in order to learn as fast and as deeply. I didn't do any project prior to this one relating to BTF. So I started manual parsing and kept it in the patchset.
2) Let everything be under control. Before actually working on this, I know that the BTF/kallsyms parsing will be running in 2nd kernel, so it should have a good balance of 1) as fast, 2) less memory consumption. So I created 2 ways for BTF resolving: 1) Given id of btf type, resolve it by array index(some meta data of BTF types are organized in array, so easy to resolve the corresponding BTF type by the meta data); 2) Given name of btf type, resolve it by hash table. Like I said, only necessary data is stored within memory, others are left as disk files to read when needed. I'm not saying I did a better job than libbpf, my approach is more white-box to me, and easy to improve.
3) I'm happy to improve the patchset, if libbpf is better in 2nd kernel + eppic case, I can replace all those into libbpf APIs.
Quoted 19 lines
For my implementation of BTF parsing for drgn I was not aware of it, so I did the parsing entirely manually, similar to this. But for my recent attempt[1] at adding BTF support to makedumpfile (unaware of this patch), I used libbpf and found that it did reduce the amount of required code by a lot. These are specific functions I found useful:
-
btf__new()-> returns a "struct btf *" for use with the library
-btf__type_by_id()-> returns the "structbtf_type*". Then, thelinux/btf.hheader provides several inline functions to work with it:
-btf_members(),btf_vlen()
-btf_member_bit_offset()
-btf__pointer_size()
-btf__name_by_offset()-> returns the name for a string offset
-btf__find_by_name()&btf__find_by_name_kind()-> looks up a type by name. Note that this does a linear search, so your hash table approach is better for random lookups.
-btf__resolve_size()
Quoted 4 lines
So with libbpf, the only thing you'd really need to implement is resolving member offsets (supporting anonyomus structs, recursively, etc), and a hash table for string lookups, if you feel it's necessary. For module BTF,
btf__new_split()is available as well.
Thanks for providing the info, I can give it a try and do a performance/memory consumption measurement. In the meantime, I'm keen for any suggestions on the whole BTF/kallsyms + eppic approach for makedumpfile customization. I guess there are cases other than GPU mm filtering which are demanding for extending the current page-flag filtering of makedumpfile.
Thanks,
Tao Liu
Quoted 992 lines
Thanks,
StephenQuoted 985 lines
--- Makefile | 2 +- btf.c | 798 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ btf.h | 175 ++++++++++++ 3 files changed, 974 insertions(+), 1 deletion(-) create mode 100644 btf.c create mode 100644 btf.hMakefile +1
-1@@ -45,7 +45,7 @@ CFLAGS_ARCH += -m32 endif SRC_BASE = makedumpfile.c makedumpfile.h diskdump_mod.h sadump_mod.h sadump_info.h -SRC_PART = print_info.c dwarf_info.c elf_info.c erase_info.c sadump_info.c cache.c tools.c printk.c detect_cycle.c kallsyms.c +SRC_PART = print_info.c dwarf_info.c elf_info.c erase_info.c sadump_info.c cache.c tools.c printk.c detect_cycle.c kallsyms.c btf.c OBJ_PART=$(patsubst %.c,%.o,$(SRC_PART)) SRC_ARCH = arch/arm.c arch/arm64.c arch/x86.c arch/x86_64.c arch/ia64.c arch/ppc64.c arch/s390x.c arch/ppc.c arch/sparc64.c arch/mips64.c arch/loongarch64.c arch/riscv64.c OBJ_ARCH=$(patsubst %.c,%.o,$(SRC_ARCH))btf.c new file +775
-0@@ -0,0 +1,775 @@ +#include "btf.h" +#include <stdio.h> +#include <assert.h> +#include <stdlib.h> +#include <string.h> +#include <unistd.h> +#include <sys/mman.h> +#include <sys/stat.h> +#include <fcntl.h> +#include <libelf.h> +#include <gelf.h> +#include <dirent.h> +#include "kallsyms.h" +#include "makedumpfile.h" + +// btf_files_array[0] must be kernel itself +// btf_files_array[1..] are kernel modules +static struct btf_file **btf_files_array =NULL ; +static int btf_files_array_index =0 ; + +static inline uint16_t btf_vlen (uint32_t info) +{ +return info &0xFFFF ; +} + +static inline uint32_t btf_kind_flag (uint32_t info) +{ +return info & (1 <<31 ); +} + +static inline uint32_t btf_member_bit_offset (uint32_t value) +{ +return value &0xffffff ; +} + +static inline uint32_t btf_member_bitfield_size (uint32_t value) +{ +return value >>24 ; +} + +static struct btf_type *btf_next (struct btf_type *tp) +{ +uintptr_t next = (uintptr_t )&tp[1 ]; + +switch (btf_kind (tp->info)) { +case BTF_KIND_INT : + next +=sizeof (uint32_t ); +break ; + +case BTF_KIND_ARRAY : + next +=sizeof (struct btf_array ); +break ; + +case BTF_KIND_STRUCT : +case BTF_KIND_UNION : + next +=btf_vlen (tp->info) *sizeof (struct btf_member ); +break ; + +case BTF_KIND_ENUM : + next +=btf_vlen (tp->info) *sizeof (struct btf_enum ); +break ; + +case BTF_KIND_FUNC_PROTO : + next +=btf_vlen (tp->info) *sizeof (struct btf_param ); +break ; + +case BTF_KIND_VAR : + next +=sizeof (struct btf_var ); +break ; + +case BTF_KIND_DATASEC : + next +=btf_vlen (tp->info) *sizeof (struct btf_var_secinfo ); +break ; + +case BTF_KIND_DECL_TAG : + next +=sizeof (struct btf_decl_tag ); +break ; + +case BTF_KIND_ENUM64 : + next +=btf_vlen (tp->info) *sizeof (struct btf_enum64 ); +break ; + +case BTF_KIND_PTR : +case BTF_KIND_FWD : +case BTF_KIND_TYPEDEF : +case BTF_KIND_VOLATILE : +case BTF_KIND_CONST : +case BTF_KIND_RESTRICT : +case BTF_KIND_FUNC : +case BTF_KIND_FLOAT : +case BTF_KIND_TYPE_TAG : +break ;// no extra data + +default : +__builtin_unreachable (); + } +return (struct btf_type *)next; +} + +#define NAME_HASH 512 +static struct name_entry *name_hash_table[NAME_HASH ] = {0 }; + +static void *get_name_entry_next (void *entry) +{ +return ((struct name_entry *)entry)->name_hash_next; +} + +static void set_name_entry_next (void *entry,void *next) +{ + ((struct name_entry *)entry)->name_hash_next = next; +} + +static void name_hash_install (struct name_entry *en) +{ +hash_install ((void **)name_hash_table,NAME_HASH , en, en->name, + get_name_entry_next, set_name_entry_next); +} + +static unsigned int name_hash_index (char *name) +{ +return hash_index (name,NAME_HASH ); +} + +int read_file_at_offset (char *f,int f_off,int r_size,void *buf) +{ +int got =0 ; +int r, fd, ret =-1 ; + + fd =open (f,O_RDONLY ); +if (fd <0 ) { +fprintf (stderr,"%s: Failed to open file %s\n" , __func__, f); + goto out; + } + +while (got < r_size) { + r =pread (fd, buf + got, r_size - got, f_off + got); +if (r <0 ) { +fprintf (stderr,"%s: Failed to read file %s\n" , __func__, f); + goto clean_fd; + } + got += r; + } + ret = got; + +clean_fd: +close (fd); +out: +return ret; +} + +static char *get_str_by_name_off (struct btf_file *bf,uint32_t name_off) +{ +struct btf_file *bf_vmlinux = btf_files_array[0 ]; + +if (bf != bf_vmlinux && name_off >= bf_vmlinux->str_cache_len) +return bf->str_cache + name_off - bf_vmlinux->str_cache_len; +else +return bf_vmlinux->str_cache + name_off; +} + +int get_btf_type_by_type_id (struct btf_file *bf_in,uint32_t id_in, +struct btf_type *bt_out,struct name_entry **en_out) +{ +struct name_entry *en; +struct btf_file *bf_vmlinux = btf_files_array[0 ]; + +if (bf_in != bf_vmlinux && id_in > bf_vmlinux->array_len) { + en = bf_in->array[id_in - bf_vmlinux->array_len -1 ]; + }else { + en = bf_vmlinux->array[id_in -1 ]; + } +if (en_out) + *en_out = en; +return read_file_at_offset (en->bf->file_name, + en->btf_type_offset + en->bf->types_data_offset, +sizeof (*bt_out), bt_out); +} + +/* +* Parse the btf data and install elements into hashtable and array for quick +* reference. +*/ +static int parse_btf_data (char *file_path,char *data_start,uint32_t data_len) +{ +struct btf_header *hdr = (struct btf_header *)data_start; +struct btf_file *bf =NULL ; +void *type_start; +char *str_start; +struct btf_type *tp; +struct name_entry *en; +int i, j, scale; + +/* We do some check first */ +if (hdr->magic !=BTF_MAGIC ) { +fprintf (stderr,"%s: Invalid BTF magic in %s\n" , + __func__, file_path); + goto out; + } +if (hdr->hdr_len !=sizeof (*hdr)) { +fprintf (stderr,"%s: Invalid BTF header length in %s\n" , + __func__, file_path); + goto out; + } +if (hdr->hdr_len + hdr->str_off + hdr->str_len > data_len) { +fprintf (stderr,"%s: String section exceeds data length in %s\n" , + __func__, file_path); + goto out; + } +if (hdr->hdr_len + hdr->type_off + hdr->type_len > data_len) { +fprintf (stderr,"%s: Type section exceeds data length in %s\n" , + __func__, file_path); + goto out; + } + +/* Let's start parsing */ + bf = (struct btf_file *)malloc (sizeof (*bf)); +if (!bf) + goto no_mem; +memset (bf,0 ,sizeof (*bf)); + +/* Enlarge array when reach power of 2 */ + btf_files_array[btf_files_array_index++] = bf; +if ((btf_files_array_index & (btf_files_array_index -1 )) ==0 ) { +struct btf_file **tmp = (struct btf_file **)reallocarray (btf_files_array, + btf_files_array_index <<1 , +sizeof (struct btf_file *)); +if (!tmp) + goto no_mem; + btf_files_array = tmp; + } + + type_start = data_start + hdr->hdr_len + hdr->type_off; + str_start = data_start + hdr->hdr_len + hdr->str_off; + + bf->str_cache =malloc (hdr->str_len); +if (!bf->str_cache) + goto no_mem; +memcpy (bf->str_cache, str_start, hdr->str_len); + bf->str_cache_len = hdr->str_len; + + bf->array_len =64 ; + bf->array = (struct name_entry **)calloc (bf->array_len, +sizeof (struct name_entry *)); +if (!bf->array) + goto no_mem; + + bf->file_name =strdup (file_path); +if (!bf->file_name) + goto no_mem; + bf->types_data_offset = (char *)type_start - data_start; + +for (tp = (struct btf_type *)type_start, i =0 ; + (void *)tp < type_start + hdr->type_len; + tp =btf_next (tp), i++) { + en = (struct name_entry *)malloc (sizeof (struct name_entry )); +if (!en) { + bf->array_len = i; + goto no_mem; + } +memset (en,0 ,sizeof (struct name_entry )); + + en->btf_type_offset = (void *)tp - type_start; + en->id = i +1 ; + en->bf = bf; +if (tp->name_off) { + en->name =get_str_by_name_off (bf, tp->name_off); +if (en->name[0 ]) +name_hash_install (en); + } + bf->array[i] = en; + +/* Now deal with sub elements which also have a name */ +if (btf_kind (tp->info) ==BTF_KIND_ENUM || +btf_kind (tp->info) ==BTF_KIND_ENUM64 ) { + scale =btf_kind (tp->info) ==BTF_KIND_ENUM ? +sizeof (struct btf_enum ) :sizeof (struct btf_enum64 ); +for (j =0 ; j <btf_vlen (tp->info); j++) { + en = (struct name_entry *)malloc (sizeof (struct name_entry )); +if (!en) { + bf->array_len = i +1 ; + goto no_mem; + } +memset (en,0 ,sizeof (struct name_entry )); + en->id =0 ; + en->p_id = i +1 ; + en->p_id += btf_files_array_index ==1 ? +0 : btf_files_array[0 ]->array_len; + en->bf = bf; + en->name =get_str_by_name_off (bf, + *(uint32_t *)((char *)&tp[1 ] + j * scale)); +// printf("%s\n", en->name); +name_hash_install (en); + } + } + +/* Enlarge array when over 3/4 */ +if (i > (bf->array_len >>2 ) *3 ) { +struct name_entry **tmp = (struct name_entry **)reallocarray (bf->array, + bf->array_len <<1 ,sizeof (struct name_entry *)); +if (!tmp) + goto no_mem; + bf->array = tmp; + bf->array_len <<=1 ; + } + } + bf->array_len = i; +return 0 ; +no_mem: +/* All the memory free will be dealed later. */ +fprintf (stderr,"%s: Not enough memory!\n" , __func__); +out: +return -1 ; +} + +/* +* Used by search_name_by_cond() for searching specific type: +* such as struct/union/typedef +*/ +static bool is_type (struct name_entry *sp,void *data_in,void *data_out) +{ +struct btf_type *bt = (struct btf_type *)data_out; +int type = *(int *)data_in; +if (sp->id) { +read_file_at_offset (sp->bf->file_name, + sp->btf_type_offset + sp->bf->types_data_offset, +sizeof (*bt), bt); +return btf_kind (bt->info) == type; + } +return false; +} + +static struct name_entry *search_name_by_cond (char *name, +bool (*fn)(struct name_entry *,void *,void *), +void *data_in,void *data_out) +{ +unsigned int index; +struct name_entry *sp; + + index =name_hash_index (name); +for (sp = name_hash_table[index]; sp; sp = sp->name_hash_next) { +if (!strcmp (name, sp->name) &&fn (sp, data_in, data_out)) { +return sp; + } + } +return sp; +} + +// caller should prepare bt +void resolve_typedef (struct name_entry *en_in,struct name_entry **en_out, +struct btf_type *bt) +{ +uint32_t id; +struct name_entry *en; + +read_file_at_offset (en_in->bf->file_name, + en_in->btf_type_offset + en_in->bf->types_data_offset, +sizeof (*bt), bt); +if (btf_kind (bt->info) ==BTF_KIND_TYPEDEF ) { + id = bt->type; +get_btf_type_by_type_id (en_in->bf, id, bt, &en); +return resolve_typedef (en, en_out, bt); + }else { + *en_out = en_in; + } +} + +struct cond_args { +int index; +char *name; +}; + +static bool cond (struct cond_args *c1,struct cond_args *c2) +{ +if (c1->name && c2->name) +/* Check if the member name is what we want */ +return !strcmp (c1->name, c2->name); +else +/* Check if the member index is what we want */ +return c1->index == c2->index; +} + +static bool get_internal_member_info (struct name_entry *en,int base_position, +int *global_index,struct cond_args *tar_arg, +struct member_info *mi,bool initial_dive) +{ +struct btf_type bt, sub_bt; +int member_num; +char *member_array_buf =NULL ; +int i; +struct btf_member *bm; +struct btf_array *ba; +int bm_position; +struct name_entry *sub_en; +bool res; +struct cond_args cur_arg = {0 }; + +read_file_at_offset (en->bf->file_name, + en->btf_type_offset + en->bf->types_data_offset, +sizeof (bt), &bt);// this struct +if (initial_dive || + ((!en->name || en->name[0 ] =='\0' ) && + ((btf_kind (bt.info) ==BTF_KIND_STRUCT ) || + (btf_kind (bt.info) ==BTF_KIND_UNION )))) { +/* anonymous struct/union, dive into */ + member_num =btf_vlen (bt.info); + member_array_buf =calloc (member_num,sizeof (struct btf_member )); +if (!member_array_buf) { +fprintf (stderr,"%s: Not enough memory!\n" , __func__); +return false; + } +read_file_at_offset (en->bf->file_name, + en->btf_type_offset + en->bf->types_data_offset +sizeof (bt), + member_num *sizeof (struct btf_member ), member_array_buf); +for (i =0 , bm = (struct btf_member *)member_array_buf; + i < member_num; i++) { +if (btf_kind_flag (bt.info)) { + bm_position = base_position + +btf_member_bit_offset (bm[i].offset); + mi->bits =btf_member_bitfield_size (bm[i].offset); + }else { + bm_position = base_position + + bm[i].offset; + } + mi->sname =get_str_by_name_off (en->bf, bm[i].name_off); +get_btf_type_by_type_id (en->bf, bm[i].type, + &sub_bt, &sub_en); +// Dive into this member + res =get_internal_member_info (sub_en, bm_position, global_index, + tar_arg, mi, false); +if (res) { +free (member_array_buf); +return res; + } + } +free (member_array_buf); +return false; + } + + cur_arg.index = *global_index; + cur_arg.name = mi->sname; +if (cond (&cur_arg, tar_arg)) { + mi->bit_pos = base_position; +resolve_typedef (en, &sub_en, &bt); + mi->uniq_id =id_to_uniq_id (sub_en->id, sub_en->bf); +if (btf_kind (bt.info) ==BTF_KIND_PTR ) { +/* +* BUG? No pointer size in btf, +* 32bit btf target on 64bit machine +*/ + mi->size =sizeof (void *); + mi->type ="char *" ; + }else if (btf_kind (bt.info) ==BTF_KIND_ARRAY ) { + en = sub_en; + ba = (struct btf_array *)malloc (sizeof (struct btf_array )); +if (!ba) { +fprintf (stderr,"%s: Not enough memory!\n" , __func__); +return false; + } +read_file_at_offset (en->bf->file_name, + en->btf_type_offset + en->bf->types_data_offset +sizeof (bt), +sizeof (struct btf_array ), ba); + mi->size = ba->nelems;// array elements +get_btf_type_by_type_id (en->bf, ba->type, &sub_bt, &sub_en); +resolve_typedef (sub_en, &en, &bt); +free (ba); + mi->size *= bt.size;// element size + mi->type =NULL ;// "char [64]? Leave it NULL for now" + }else { + mi->size = bt.size; + mi->type =get_str_by_name_off (sub_en->bf, bt.name_off); + } +return true; + }else { + (*global_index)++; +return false; + } +} + +static bool get_member_info_by_index (struct name_entry *en,int target_index, +struct member_info *mi) +{ +int g_index =1 ; +struct cond_args args = {0 }; + args.index = target_index; +memset (mi,0 ,sizeof (*mi)); +return get_internal_member_info (en,0 , &g_index, &args, mi, true); +} + +static bool get_member_info_by_member_name (struct name_entry *en,char *member_name, +struct member_info *mi) +{ +int g_index =1 ; +struct cond_args args = {0 }; + args.name = member_name; +memset (mi,0 ,sizeof (*mi)); +return get_internal_member_info (en,0 , &g_index, &args, mi, true); +} + +/* +* Entry for query struct members +* Return: struct size, mi_out: member details +*/ +uint32_t get_struct_member_by_name (char *struct_name,char *member_name, +struct member_info *mi_out) +{ +struct name_entry *en; +struct btf_type bt = {0 }; +int type =BTF_KIND_STRUCT ; + + en =search_name_by_cond (struct_name, is_type, (void *)&type, (void *)&bt); +if (!en) + goto out; +if (get_member_info_by_member_name (en, member_name, mi_out)) +return bt.size; +out: +return 0 ; +} + +/* +* Entry for query type members, similar as above +* Return: found-the-type, mi_out: member details +*/ +static uint32_t uniq_id_to_id (uint32_t ,struct btf_file **); +bool get_type_member_by_index (uint64_t type_uniq_id,int member_index, +struct member_info *mi_out) +{ +uint32_t id; +struct btf_file *bf; + + id =uniq_id_to_id ((uint32_t )type_uniq_id, &bf); +return get_member_info_by_index (bf->array[id -1 ], member_index, mi_out); +} + +uint32_t id_to_uniq_id (uint32_t id,struct btf_file *bf) +{ +int i; +uint32_t uniq_id =0 ; +for (i =0 ; i < btf_files_array_index; i++) { +if (btf_files_array[i] != bf) + uniq_id += btf_files_array[i]->array_len; +else +return id + uniq_id; + } +__builtin_unreachable (); +assert (false); +} + +static uint32_t uniq_id_to_id (uint32_t uid,struct btf_file **bf) +{ +int i; + +for (i =0 ; i < btf_files_array_index; i++) { +if (uid > btf_files_array[i]->array_len) { + uid -= btf_files_array[i]->array_len; + }else { +if (bf) + *bf = btf_files_array[i]; +return uid; + } + + } +__builtin_unreachable (); +assert (false); +} + +// api for eppic +uint32_t get_type_size_by_name (char *type_name,int type,uint32_t *uniq_id) +{ +struct name_entry *en; +struct btf_type bt = {0 }; + + en =search_name_by_cond (type_name, is_type, (void *)&type, (void *)&bt); +if (!en) + goto out; +if (uniq_id) + *uniq_id =id_to_uniq_id (en->id, en->bf); +return bt.size; +out: +return 0 ; +} + +// Caller should prepare bt_out +struct name_entry *get_en_by_uniq_id (uint32_t uniq_id,struct btf_type *bt_out) +{ +uint32_t id; +struct btf_file *bf; +struct name_entry *en; +struct name_entry *en_sub; + + id =uniq_id_to_id ((uint32_t )uniq_id, &bf); + en = bf->array[id -1 ]; +resolve_typedef (en, &en_sub, bt_out); +return en_sub; +} + +/* Deal with btf file */ +static int btf_file_init (int fd) +{ +struct stat f_stat; +char *buf =NULL ; +char proc_path[32 ]; +char real_path[512 ]; +int ret =-1 ; + +memset (real_path,0 ,sizeof (real_path)); +if (fstat (fd, &f_stat) <0 ) { +fprintf (stderr,"%s: fstat fail!\n" , __func__); + goto out; + } + buf =malloc (f_stat.st_size); +if (!buf) { +fprintf (stderr,"%s: Not enough memory!\n" , __func__); + goto out; + } + +snprintf (proc_path,sizeof (proc_path),"/proc/self/fd/%d" , fd); +readlink (proc_path, real_path,sizeof (real_path) -1 ); +if (read_file_at_offset (real_path,0 , f_stat.st_size, buf) <0 ) + goto out; + + ret =parse_btf_data (real_path, buf, f_stat.st_size); +out: +if (buf) +free (buf); +return ret; +} + +/* Deal with elf file which contains .BTF section */ +static int elf_file_init (int fd) +{ +char proc_path[32 ]; +char real_path[512 ]; +Elf *elf =NULL ; +Elf_Scn *scn =NULL ; +size_t shstrndx; +GElf_Shdr shdr; +Elf_Data *data =NULL ; +char *str, *databuf =NULL ; +int ret =-1 ; + +snprintf (proc_path,sizeof (proc_path),"/proc/self/fd/%d" , fd); +readlink (proc_path, real_path,sizeof (real_path) -1 ); + +if (elf_version (EV_CURRENT ) ==EV_NONE ) + goto elf_err; + elf =elf_begin (fd,ELF_C_READ ,NULL ); +if (!elf ||elf_getshdrstrndx (elf, &shstrndx)) + goto elf_err; +while ((scn =elf_nextscn (elf, scn)) !=NULL ) { +if (!gelf_getshdr (scn, &shdr)) +continue ; + str =elf_strptr (elf, shstrndx, shdr.sh_name); +if (!strcmp (str,".BTF" )) +break ; + } +if (!scn) { +fprintf (stderr,"%s: No .BTF found in %s!\n" , __func__, real_path); + goto out; + } + + data =elf_rawdata (scn, data); + databuf =malloc (data->d_size); +if (!databuf) { +fprintf (stderr,"%s: Not enough memory!\n" , __func__); + goto out; + } +memcpy (databuf, data->d_buf, data->d_size); +elf_end (elf); + elf =NULL ; + ret =parse_btf_data (real_path, databuf, data->d_size); + goto out; + +elf_err: +fprintf (stderr,"%s: elf error in %s!\n" , __func__, real_path); +out: +if (databuf) +free (databuf); +if (elf) +elf_end (elf); +return ret; +} + +/* +* Entry for parse btf file and elf file. +*/ +static int file_init (char *name) +{ +int fd =0 , ret =-1 ; +char buf[4 ] = {0 }; + +/* Will be enlarged automatically */ +if (!btf_files_array) { + btf_files_array =calloc (1 ,sizeof (struct btf_file *)); +if (!btf_files_array) { +fprintf (stderr,"%s: Not enough memory!\n" , __func__); + goto out; + } + } + + fd =open (name,O_RDONLY ); +if (read_file_at_offset (name,0 ,sizeof (buf), buf) <0 ) + goto out; +if (*(u_int16_t *)&buf ==BTF_MAGIC ) { + ret =btf_file_init (fd); + }else if (*(u_int32_t *)&buf == (((u_int32_t )ELFMAG0 <<24 ) | + ((u_int32_t )ELFMAG1 <<16 ) | + ((u_int32_t )ELFMAG2 <<8 ) | + ((u_int32_t )ELFMAG3 <<0 ))) { + ret =elf_file_init (fd); + }else { +__builtin_unreachable (); + } + +out: +if (fd) +close (fd); +return ret; +} + +/* For pure buffer data, wrap it as a file, and handover to file_init() */ +char temp[] ="/tmp/btf_XXXXXX" ; +static int init_btf_from_buf (char *mod_name,char *buf,int size) +{ +int fd; +char name_buf[64 ]; +int w, got =0 ; +static char *temp_dir =NULL ; + +if (!temp_dir) + temp_dir =mkdtemp (temp); +snprintf (name_buf,sizeof (name_buf),"%s/%s" , temp_dir, mod_name); + fd =open (name_buf,O_RDWR |O_CREAT ,S_IRUSR |S_IWUSR ); +if (fd <0 ) { +fprintf (stderr,"%s: open fail in %s!\n" , __func__, name_buf); +return -1 ; + } + +while (got < size) { + w =pwrite (fd, buf + got, size - got, got); +if (w <0 ) { +fprintf (stderr,"%s: pwrite fail in %s!\n" , __func__, name_buf); +return -1 ; + } + got += w; + } +close (fd); +return file_init (name_buf); +} + +int init_kernel_btf (void ) +{ +uint64_t size; +char *buf; +int ret; + +uint64_t start_btf =get_kallsyms_value_by_name ("__start_BTF" ); +uint64_t stop_btf =get_kallsyms_value_by_name ("__stop_BTF" ); +if (!start_btf || !stop_btf) { +fprintf (stderr,"%s: symbol __start/stop_BTF not found!\n" , __func__); +return -1 ; + } + + size = stop_btf - start_btf; + buf = (char *)malloc (size); +if (!buf) { +fprintf (stderr,"%s: Not enough memory!\n" , __func__); +return -1 ; + } +readmem (VADDR , start_btf, buf, size); + ret =init_btf_from_buf ("vmlinux" , buf, size); +free (buf); + +return ret; +} +btf.h new file +175
-1@@ -0,0 +1,175 @@ +#ifndef _BTF_H +#define _BTF_H +#include <stdint.h> +#include <stdbool.h> + +typedef uint8_t __u8 ; +typedef uint16_t __u16 ; +typedef uint32_t __u32 ; +typedef int32_t __s32 ; + +#define BTF_MAGIC 0xeb9f + +enum { +BTF_KIND_UNKN =0 ,/* Unknown */ +BTF_KIND_INT =1 ,/* Integer */ +BTF_KIND_PTR =2 ,/* Pointer */ +BTF_KIND_ARRAY =3 ,/* Array */ +BTF_KIND_STRUCT =4 ,/* Struct */ +BTF_KIND_UNION =5 ,/* Union */ +BTF_KIND_ENUM =6 ,/* Enumeration */ +BTF_KIND_FWD =7 ,/* Forward */ +BTF_KIND_TYPEDEF =8 ,/* Typedef */ +BTF_KIND_VOLATILE =9 ,/* Volatile */ +BTF_KIND_CONST =10 ,/* Const */ +BTF_KIND_RESTRICT =11 ,/* Restrict */ +BTF_KIND_FUNC =12 ,/* Function */ +BTF_KIND_FUNC_PROTO =13 ,/* Function Proto */ +BTF_KIND_VAR =14 ,/* Variable */ +BTF_KIND_DATASEC =15 ,/* Section */ +BTF_KIND_FLOAT =16 ,/* Floating point */ +BTF_KIND_DECL_TAG =17 ,/* Decl Tag */ +BTF_KIND_TYPE_TAG =18 ,/* Type Tag */ +BTF_KIND_ENUM64 =19 ,/* Enumeration up to 64-bit values */ + +NR_BTF_KINDS , +BTF_KIND_MAX =NR_BTF_KINDS -1 , +}; + +struct btf_type { +__u32 name_off; + +/* "info" bits arrangement +* bits 0-15: vlen (e.g. # of struct's members) +* bits 16-23: unused +* bits 24-27: kind (e.g. int, ptr, array...etc) +* bits 28-30: unused +* bit 31: kind_flag, currently used by +* struct, union and fwd +*/ +__u32 info; + +/* "size" is used by INT, ENUM, STRUCT, UNION and DATASEC. +* "size" tells the size of the type it is describing. +* +* "type" is used by PTR, TYPEDEF, VOLATILE, CONST, RESTRICT, +* FUNC, FUNC_PROTO, VAR, DECL_TAG and TYPE_TAG. +* "type" is a type_id referring to another type. +*/ +union { +__u32 size; +__u32 type; + }; +}; + +struct btf_header { +__u16 magic; +__u8 version; +__u8 flags; +__u32 hdr_len; + +/* All offsets are in bytes relative to the end of this header */ +__u32 type_off;/* offset of type section */ +__u32 type_len;/* length of type section */ +__u32 str_off;/* offset of string section */ +__u32 str_len;/* length of string section */ +}; + +struct btf_array { +__u32 type; +__u32 index_type; +__u32 nelems; +}; + +struct btf_member { +__u32 name_off; +__u32 type; +/* If the type info kind_flag is set, the btf_member offset +* contains both member bitfield size and bit offset. The +* bitfield size is set for bitfield members. If the type +* info kind_flag is not set, the offset contains only bit +* offset. +*/ +__u32 offset; +}; + +struct btf_enum { +__u32 name_off; +__s32 val; +}; + +struct btf_enum64 { +__u32 name_off; +__u32 val_lo32; +__u32 val_hi32; +}; + +struct btf_param { +__u32 name_off; +__u32 type; +}; + +struct btf_var { +__u32 linkage; +}; + +struct btf_var_secinfo { +__u32 type; +__u32 offset; +__u32 size; +}; + +struct btf_decl_tag { +__s32 component_idx; +}; + +/*************************************/ +struct btf_file { +char *file_name; +char *str_cache; +uint32_t str_cache_len; +uint32_t types_data_offset; +uint32_t array_len; +struct name_entry **array; +}; + +struct name_entry { +union { +uint32_t btf_type_offset; +uint32_t p_id; + }; + +uint32_t id; +char *name; +struct btf_file *bf; +struct name_entry *name_hash_next; +}; + +struct member_info { +char *sname;// member name +char *type;// member type: int, long etc +uint32_t bit_pos;// member position in bits +uint32_t bits;// member width in bits +uint32_t size;// member size in bytes +int uniq_id;// uniq_id of btf +}; + +/*************************************/ + +int read_file_at_offset (char *,int ,int ,void *); +int init_kernel_btf (void ); +uint32_t get_struct_member_by_name (char *,char *,struct member_info *); +bool get_type_member_by_index (uint64_t ,int ,struct member_info *); +uint32_t get_type_size_by_name (char *,int ,uint32_t *); +struct name_entry *get_en_by_uniq_id (uint32_t ,struct btf_type *); +void resolve_typedef (struct name_entry *,struct name_entry **,struct btf_type *); +int get_btf_type_by_type_id (struct btf_file *,uint32_t , +struct btf_type *,struct name_entry **); +uint32_t id_to_uniq_id (uint32_t ,struct btf_file *); + +static inline uint32_t btf_kind (uint32_t info) +{ +return (info &0x1F000000 ) >>24 ; +} + +#endif /* _BTF_H */ --2.47.0
Re: [PATCH v2][makedumpfile 08/14] Implement kernel btf resolving
Quoted 75 lines · Tao Liu
Hi Stephen,
Thanks for your comments!
Quoted 20 lines · Stephen Brennan
Quoted 18 lines · Tao Liu
This patch will parse kernel's btf data. The data can be located via
__start_BTFand__stop_BTFsymbols which have been resolved by kallsyms of the previous patch.The btf data is organized as follows: each one of kernel modules, including vmlinux itself, will have a
btf_filestruct describing its btf data, and holding an array of structname_entry. By given the btf type id, we can resolve itsname_entryfast by array index. In addition,name_entrycan also be organized in hash table. So given a type name, we can also resolve itsname_entryin a fast speed. In other words, both btf type id and btf type name can we get it resolved fast. Once we get thename_entrystructure of the btf type, we can resolve its member/size etc easily.Since all
name_entryarray starting from index 0, which cannot identify a btf type globally. So a uniq id is used, its value is accumulated by the location of thebtf_filewithinbtf_file_arrayand total quantity of btf types within eachbtf_file.Quoted 6 lines
Quoted 1 line
Signed-off-by Tao Liu <[email protected]>Hi Tao,
I haven't read this patch in detail, but I wanted to ask if you had any particular reason to avoid using libbpf for parsing the BTF?
Actually I manually parse BTF data due to the following concerns:
1) Learn BTF data structure by manually parsing. I haven't worked on BTF previously, and I prefer the way of doing things from scratch in order to learn as fast and as deeply. I didn't do any project prior to this one relating to BTF. So I started manual parsing and kept it in the patchset.
2) Let everything be under control. Before actually working on this, I know that the BTF/kallsyms parsing will be running in 2nd kernel, so it should have a good balance of 1) as fast, 2) less memory consumption. So I created 2 ways for BTF resolving: 1) Given id of btf type, resolve it by array index(some meta data of BTF types are organized in array, so easy to resolve the corresponding BTF type by the meta data); 2) Given name of btf type, resolve it by hash table. Like I said, only necessary data is stored within memory, others are left as disk files to read when needed. I'm not saying I did a better job than libbpf, my approach is more white-box to me, and easy to improve.
3) I'm happy to improve the patchset, if libbpf is better in 2nd kernel + eppic case, I can replace all those into libbpf APIs.
Quoted 19 lines
For my implementation of BTF parsing for drgn I was not aware of it, so I did the parsing entirely manually, similar to this. But for my recent attempt[1] at adding BTF support to makedumpfile (unaware of this patch), I used libbpf and found that it did reduce the amount of required code by a lot. These are specific functions I found useful:
-
btf__new()-> returns a "struct btf *" for use with the library
-btf__type_by_id()-> returns the "structbtf_type*". Then, thelinux/btf.hheader provides several inline functions to work with it:
-btf_members(),btf_vlen()
-btf_member_bit_offset()
-btf__pointer_size()
-btf__name_by_offset()-> returns the name for a string offset
-btf__find_by_name()&btf__find_by_name_kind()-> looks up a type by name. Note that this does a linear search, so your hash table approach is better for random lookups.
-btf__resolve_size()
Quoted 7 lines
Quoted 4 lines
So with libbpf, the only thing you'd really need to implement is resolving member offsets (supporting anonyomus structs, recursively, etc), and a hash table for string lookups, if you feel it's necessary. For module BTF,
btf__new_split()is available as well.Thanks for providing the info, I can give it a try and do a performance/memory consumption measurement. In the meantime, I'm keen
Thank you for the information and thoughts.
Considering its maintenance, I think it would be better to use the library if possible. It's hard for others than the auther to maintain scratch code and follow the changes of BTF. If you don't want to use the library, please give us performance/memory consumption data or something to convince that the library cannot be used.
Thanks,
Kazu
Quoted 14 lines
for any suggestions on the whole BTF/kallsyms + eppic approach for makedumpfile customization. I guess there are cases other than GPU mm filtering which are demanding for extending the current page-flag filtering of makedumpfile.
Thanks,
Tao Liu
Re: [PATCH v2][makedumpfile 08/14] Implement kernel btf resolving
Hi Kazu,
Quoted 85 lines · HAGIO KAZUHITO(萩尾 一仁)
Quoted 75 lines · Tao Liu
Hi Stephen,
Thanks for your comments!
Quoted 20 lines · Stephen Brennan
Quoted 18 lines · Tao Liu
This patch will parse kernel's btf data. The data can be located via
__start_BTFand__stop_BTFsymbols which have been resolved by kallsyms of the previous patch.The btf data is organized as follows: each one of kernel modules, including vmlinux itself, will have a
btf_filestruct describing its btf data, and holding an array of structname_entry. By given the btf type id, we can resolve itsname_entryfast by array index. In addition,name_entrycan also be organized in hash table. So given a type name, we can also resolve itsname_entryin a fast speed. In other words, both btf type id and btf type name can we get it resolved fast. Once we get thename_entrystructure of the btf type, we can resolve its member/size etc easily.Since all
name_entryarray starting from index 0, which cannot identify a btf type globally. So a uniq id is used, its value is accumulated by the location of thebtf_filewithinbtf_file_arrayand total quantity of btf types within eachbtf_file.Quoted 6 lines
Quoted 1 line
Signed-off-by Tao Liu <[email protected]>Hi Tao,
I haven't read this patch in detail, but I wanted to ask if you had any particular reason to avoid using libbpf for parsing the BTF?
Actually I manually parse BTF data due to the following concerns:
1) Learn BTF data structure by manually parsing. I haven't worked on BTF previously, and I prefer the way of doing things from scratch in order to learn as fast and as deeply. I didn't do any project prior to this one relating to BTF. So I started manual parsing and kept it in the patchset.
2) Let everything be under control. Before actually working on this, I know that the BTF/kallsyms parsing will be running in 2nd kernel, so it should have a good balance of 1) as fast, 2) less memory consumption. So I created 2 ways for BTF resolving: 1) Given id of btf type, resolve it by array index(some meta data of BTF types are organized in array, so easy to resolve the corresponding BTF type by the meta data); 2) Given name of btf type, resolve it by hash table. Like I said, only necessary data is stored within memory, others are left as disk files to read when needed. I'm not saying I did a better job than libbpf, my approach is more white-box to me, and easy to improve.
3) I'm happy to improve the patchset, if libbpf is better in 2nd kernel + eppic case, I can replace all those into libbpf APIs.
Quoted 19 lines
For my implementation of BTF parsing for drgn I was not aware of it, so I did the parsing entirely manually, similar to this. But for my recent attempt[1] at adding BTF support to makedumpfile (unaware of this patch), I used libbpf and found that it did reduce the amount of required code by a lot. These are specific functions I found useful:
-
btf__new()-> returns a "struct btf *" for use with the library
-btf__type_by_id()-> returns the "structbtf_type*". Then, thelinux/btf.hheader provides several inline functions to work with it:
-btf_members(),btf_vlen()
-btf_member_bit_offset()
-btf__pointer_size()
-btf__name_by_offset()-> returns the name for a string offset
-btf__find_by_name()&btf__find_by_name_kind()-> looks up a type by name. Note that this does a linear search, so your hash table approach is better for random lookups.
-btf__resolve_size()Quoted 7 lines
Quoted 4 lines
So with libbpf, the only thing you'd really need to implement is resolving member offsets (supporting anonyomus structs, recursively, etc), and a hash table for string lookups, if you feel it's necessary. For module BTF,
btf__new_split()is available as well.Thanks for providing the info, I can give it a try and do a performance/memory consumption measurement. In the meantime, I'm keen
Quoted 8 lines
Thank you for the information and thoughts.
Considering its maintenance, I think it would be better to use the library if possible. It's hard for others than the auther to maintain scratch code and follow the changes of BTF. If you don't want to use the library, please give us performance/memory consumption data or something to convince that the library cannot be used.
Thanks for your comments. I will give the library a try and see which approach has better performance/memory balance in v3.
Thanks,
Tao Liu
Quoted 17 lines
Thanks,
KazuQuoted 14 lines
for any suggestions on the whole BTF/kallsyms + eppic approach for makedumpfile customization. I guess there are cases other than GPU mm filtering which are demanding for extending the current page-flag filtering of makedumpfile.
Thanks,
Tao Liu
PATCH v2 9/14 Implement kernel module's kallsyms resolving
In reply to Tao LiuWith kernel's kallsyms and btf ready, we can get any kernel types and symbol addresses. So we can iterate kernel modules' linked list, and parse & install each one of kernel module's structure to get its kallsyms data.
--- kallsyms.c | 126 +++++++++++++++++++++++++++++++++++++++++++++++++++++ kallsyms.h | 19 ++++++++ 2 files changed, 145 insertions(+)
kallsyms.c +126 -0
@@ -4,6 +4,7 @@#include <assert.h> #include "makedumpfile.h" #include "kallsyms.h" +#include "btf.h" static uint32_t *kallsyms_offsets =NULL ;static uint16_t *kallsyms_token_index =NULL ;
@@ -231,6 +232,131 @@ out:return ret; } +uint64_t next_list (uint64_t list) +{ +static int list_head_next_offset =0 ; +static int list_head_next_size =0 ; + +struct member_info mi; +uint64_t next =0 ; + +if (!list_head_next_size) { +get_struct_member_by_name ("list_head" ,"next" , &mi); + list_head_next_size = mi.size; + list_head_next_offset = mi.bit_pos /8 ; + } +readmem (VADDR , list + list_head_next_offset, &next, list_head_next_size); +return next; +} + +int init_module_kallsyms (void ) +{ +struct member_info mi; +uint64_t modules, list, value =0 , symtab =0 , + strtab =0 , typetab =0 ; +uint32_t st_name =0 ; +int num_symtab, i, j; +struct syment *mod_syment; +char symname[512 ], ch; +int ret =-1 ; + + modules =get_kallsyms_value_by_name ("modules" ); +if (!modules) { +/* Not a failure if no module enabled */ + ret =0 ; + goto out; + } + +INIT_MEMBER_OFF_SIZE (module, list); +INIT_MEMBER_OFF_SIZE (module, core_kallsyms); +INIT_MEMBER_OFF_SIZE (mod_kallsyms, symtab); +INIT_MEMBER_OFF_SIZE (mod_kallsyms, num_symtab); +INIT_MEMBER_OFF_SIZE (mod_kallsyms, strtab); +INIT_MEMBER_OFF_SIZE (mod_kallsyms, typetab); +INIT_MEMBER_OFF_SIZE (elf64_sym, st_name); +INIT_MEMBER_OFF_SIZE (elf64_sym, st_value); + +for (list =next_list (modules); list != modules; list =next_list (list)) { +readmem (VADDR , list -GET_MEMBER_OFF (module, list) + +GET_MEMBER_OFF (module, core_kallsyms) + +GET_MEMBER_OFF (mod_kallsyms, num_symtab), + &num_symtab,GET_MEMBER_SIZE (mod_kallsyms, num_symtab)); +readmem (VADDR , list -GET_MEMBER_OFF (module, list) + +GET_MEMBER_OFF (module, core_kallsyms) + +GET_MEMBER_OFF (mod_kallsyms, symtab), + &symtab,GET_MEMBER_SIZE (mod_kallsyms, symtab)); +readmem (VADDR , list -GET_MEMBER_OFF (module, list) + +GET_MEMBER_OFF (module, core_kallsyms) + +GET_MEMBER_OFF (mod_kallsyms, strtab), + &strtab,GET_MEMBER_SIZE (mod_kallsyms, strtab)); +readmem (VADDR , list -GET_MEMBER_OFF (module, list) + +GET_MEMBER_OFF (module, core_kallsyms) + +GET_MEMBER_OFF (mod_kallsyms, typetab), + &typetab,GET_MEMBER_SIZE (mod_kallsyms, typetab)); +for (i =0 ; i < num_symtab; i++) { + j =0 ; +readmem (VADDR , symtab + i *GET_STRUCT_SIZE (elf64_sym, st_value) + +GET_MEMBER_OFF (elf64_sym, st_value), + &value,GET_MEMBER_SIZE (elf64_sym, st_value)); +readmem (VADDR , symtab + i *GET_STRUCT_SIZE (elf64_sym, st_name) + +GET_MEMBER_OFF (elf64_sym, st_name), + &st_name,GET_MEMBER_SIZE (elf64_sym, st_name)); +do { +readmem (VADDR , strtab + st_name + j++, &ch,1 ); + }while (ch !='\0' ); +if (j ==1 ) +/* Skip empty string */ +continue ; +assert (j <=sizeof (symname)); + mod_syment = (struct syment *)calloc (1 ,sizeof (struct syment )); +if (!mod_syment) + goto no_mem; +readmem (VADDR , strtab + st_name, symname, j); + mod_syment->name =strdup (symname); +if (!mod_syment->name) { +free (mod_syment); + goto no_mem; + } + mod_syment->value = value; +readmem (VADDR , typetab + i, &mod_syment->type,1 ); +name_hash_install (mod_syment); + } + } + ret =0 ; + goto out; +no_mem: +/* Hashtable will be cleaned later */ +fprintf (stderr,"%s: Not enough memory!\n" , __func__); +out: +return ret; +} + +void cleanup_kallsyms (void ) +{ +struct syment *en, *en_tmp; +int i; + +/* Free the module's kallsyms first */ +for (i =0 ; i <NAME_HASH ; i++) { +for (en = name_hash_table[i]; en;) { + en_tmp = en; + en = en->name_hash_next; +if (en_tmp <= &symtable[kallsyms_num_syms -1 ] && + en_tmp >= &symtable[0 ]) +continue ; +free (en_tmp->name); +free (en_tmp); + } + } + +/* Free the kernel ones */ +for (i =0 ; i < kallsyms_num_syms; i++) { +if (symtable[i].name) +free (symtable[i].name); + } +free (symtable); +} +/* Hash table utils */ unsigned int hash_index (const char *name,unsigned int hash_size) {
kallsyms.h +19 -0
@@ -13,7 +13,26 @@ struct syment {int init_kernel_kallsyms (void );int read_vmcoreinfo_kallsyms (void );struct syment *search_kallsyms_by_name (char *); +uint64_t next_list (uint64_t );uint64_t get_kallsyms_value_by_name (char *); +int init_module_kallsyms (void ); +void cleanup_kallsyms (void ); +int read_vmcoreinfo_kallsyms (void ); + +struct member_off_size { +int m_off; +int m_size; +int s_size; +}; +#define QUATE (x) #x +#define INIT_MEMBER_OFF_SIZE (S ,M ) \ + struct member_off_size S##_##M; \ + S##_##M.s_size = get_struct_member_by_name(QUATE(S), QUATE(M), &mi); \ + S##_##M.m_off = mi.bit_pos / 8; \ + S##_##M.m_size = mi.size; +#define GET_MEMBER_OFF (S ,M ) (S##_##M.m_off) +#define GET_MEMBER_SIZE (S ,M ) (S##_##M.m_size) +#define GET_STRUCT_SIZE (S ,M ) (S##_##M.s_size)unsigned int hash_index (const char *,unsigned int ); void hash_install(void **, unsigned int, void *, const char *,
Signature · 1 line
2.47.0
PATCH v2 10/14 Implement kernel module's btf resolving
In reply to Tao Liu--- btf.c | 121 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ btf.h | 2 + 2 files changed, 123 insertions(+)
btf.c +121 -0
@@ -796,3 +796,124 @@ int init_kernel_btf(void)return ret; } +int init_module_btf (void ) +{ +uint64_t btf_modules, list; +struct member_info mi; +uint64_t btf =0 , data =0 , module =0 ; +int data_size =0 ; +char *btf_buf =NULL ; +char *modname =NULL ; + + btf_modules =get_kallsyms_value_by_name ("btf_modules" ); +if (!btf_modules) +/* Maybe module is not enabled, this is not an error */ +return 0 ; + +INIT_MEMBER_OFF_SIZE (btf_module, list); +INIT_MEMBER_OFF_SIZE (btf_module, btf); +INIT_MEMBER_OFF_SIZE (btf_module, module); +INIT_MEMBER_OFF_SIZE (module, name); +INIT_MEMBER_OFF_SIZE (btf, data); +INIT_MEMBER_OFF_SIZE (btf, data_size); + modname = (char *)malloc (GET_MEMBER_SIZE (module, name)); +if (!modname) { +fprintf (stderr,"%s: Not enough memory!\n" , __func__); + goto out; + } + +for (list =next_list (btf_modules); list != btf_modules; list =next_list (list)) { +readmem (VADDR , list -GET_MEMBER_OFF (btf_module, list) + +GET_MEMBER_OFF (btf_module, btf), + &btf,GET_MEMBER_SIZE (btf_module, btf)); +readmem (VADDR , list -GET_MEMBER_OFF (btf_module, list) + +GET_MEMBER_OFF (btf_module, module), + &module,GET_MEMBER_SIZE (btf_module, module)); +readmem (VADDR , module +GET_MEMBER_OFF (module, name), + modname,GET_MEMBER_SIZE (module, name)); +readmem (VADDR , btf +GET_MEMBER_OFF (btf, data), + &data,GET_MEMBER_SIZE (btf, data)); +readmem (VADDR , btf +GET_MEMBER_OFF (btf, data_size), + &data_size,GET_MEMBER_SIZE (btf, data_size)); + btf_buf = (char *)malloc (data_size); +if (!btf_buf) { +fprintf (stderr,"%s: Not enough memory!\n" , __func__); + goto out; + } +readmem (VADDR , data, btf_buf, data_size); +if (init_btf_from_buf (modname, btf_buf, data_size)) + goto out; +free (btf_buf); + } +return 0 ; +out: +if (modname) +free (modname); +if (btf_buf) +free (btf_buf); +return -1 ; +} + +void cleanup_btf (void ) +{ +int bf_index, i; +struct btf_file *bf; +struct name_entry *en, *en_tmp; +DIR *temp_dir; +struct dirent *entry; +char path[512 ]; + +/* Free no-hashtable-installed elements */ +for (bf_index =0 ; bf_index < btf_files_array_index; bf_index++) { + bf = btf_files_array[bf_index]; +if (bf && bf->array) { +for (i =0 ; i < bf->array_len; i++) { +if (!bf->array[i]->name || !bf->array[i]->name[0 ]) +free (bf->array[i]); + } + } + } + +/* Free the hashtable-installed elements */ +for (i =0 ; i <NAME_HASH ; i++) { +for (en = name_hash_table[i]; en;) { + en_tmp = en; + en = en->name_hash_next; +free (en_tmp); + } + } + +/* Cleanup anything else */ +for (bf_index =0 ; bf_index < btf_files_array_index; bf_index++) { + bf = btf_files_array[bf_index]; +if (bf && bf->array) { +free (bf->array); + } +if (bf && bf->file_name) +free (bf->file_name); +if (bf && bf->str_cache) +free (bf->str_cache); +if (bf) +free (bf); + } + +if (btf_files_array) +free (btf_files_array); + +/* Cleanup the dir /tmp/btf_XXXXXX */ + temp_dir =opendir (temp); +if (!temp_dir) +return ; +while ((entry =readdir (temp_dir)) !=NULL ) { +if (!strcmp (entry->d_name,"." ) || !strcmp (entry->d_name,".." )) +continue ; +snprintf (path,sizeof (path),"%s/%s" , temp, entry->d_name); +if (remove (path)) { +fprintf (stderr,"%s: fail del %s!\n" , __func__, path); +/* In case too many fail messages of files in the dir */ +return ; + } + } +closedir (temp_dir); +rmdir (temp); +}
btf.h +2 -0
@@ -166,6 +166,8 @@ void resolve_typedef(struct name_entry *, struct name_entry **, struct btf_typeint get_btf_type_by_type_id (struct btf_file *,uint32_t ,struct btf_type *,struct name_entry **);uint32_t id_to_uniq_id (uint32_t ,struct btf_file *); +int init_module_btf (void ); +void cleanup_btf (void );static inline uint32_t btf_kind (uint32_t info) {
Signature · 1 line
2.47.0
PATCH v2 11/14 Export necessary btf/kallsyms functions to eppic extension
In reply to Tao Liu--- erase_info.c | 13 ++++++++++++- erase_info.h | 11 +++++++++++ 2 files changed, 23 insertions(+), 1 deletion(-)
erase_info.c +12 -1
@@ -20,6 +20,8 @@#include "print_info.h" #include "dwarf_info.h" #include "erase_info.h" +#include "kallsyms.h" +#include "btf.h" #include <dlfcn.h>
@@ -36,7 +38,16 @@ struct call_back eppic_cb = { &get_die_member_all, &get_die_nfields_all, &get_symbol_addr_all, - &update_filter_info_raw + &update_filter_info_raw, +/**********************/ + &update_filter_pages_info, + &get_kallsyms_value_by_name, + &get_type_member_by_index, + &get_type_size_by_name, + &get_en_by_uniq_id, + &get_btf_type_by_type_id, + &id_to_uniq_id, + &read_file_at_offset, };
erase_info.h +11 -0
@@ -19,6 +19,7 @@#ifndef _ERASE_INFO_H#define _ERASE_INFO_H +#include "btf.h" #define MAX_SIZE_STR_LEN (26)/*
@@ -52,6 +53,16 @@ struct call_back {int (*get_die_nfields_all)(unsigned long long die_off);unsigned long long (*get_symbol_addr_all)(char *symname);int (*update_filter_info_raw)(unsigned long long ,int ,int ); +/********************************/ +int (*update_filter_pages_info)(unsigned long ,unsigned long ); +uint64_t (*get_kallsyms_value_by_name)(char *); +bool (*get_type_member_by_index)(uint64_t ,int ,struct member_info *); +uint32_t (*get_type_size_by_name)(char *,int ,uint32_t *); +struct name_entry *(*get_en_by_uniq_id)(uint32_t ,struct btf_type *); +int (*get_btf_type_by_type_id)(struct btf_file *,uint32_t , +struct btf_type *,struct name_entry **); +uint32_t (*id_to_uniq_id)(uint32_t ,struct btf_file *); +int (*read_file_at_offset)(char *,int ,int ,void *); };extern struct erase_info *erase_info;
Signature · 1 line
2.47.0
PATCH v2 12/14 Enable page filtering for btf/kallsyms eppic
In reply to Tao LiuThis patch mainly replicate extension_eppic.c, implements the eppic interfaces using btf/kallsyms functions rather than dwarf functions.
--- Makefile | 4 +- erase_info.c | 22 ++++ extension_btf.c | 258 ++++++++++++++++++++++++++++++++++++++++++++++ extension_eppic.c | 2 +- 4 files changed, 283 insertions(+), 3 deletions(-) create mode 100644 extension_btf.c
Makefile +2 -2
@@ -121,8 +121,8 @@ makedumpfile: $(SRC_BASE) $(OBJ_PART) $(OBJ_ARCH) -e "s/@VERSION@/$(VERSION)/" \ $(VPATH)makedumpfile.conf.5.in > $(VPATH)makedumpfile.conf.5 -eppic_makedumpfile.so: extension_eppic.c - $(CC) $(CFLAGS) $(LDFLAGS) -shared -rdynamic -o $@ extension_eppic.c -fPIC -leppic -ltinfo +eppic_makedumpfile.so: extension_eppic.c extension_btf.c + $(CC) $(CFLAGS) $(LDFLAGS) -shared -rdynamic -o $@ $^ -fPIC -leppic -ltinfo clean: rm -f $(OBJ) $(OBJ_PART) $(OBJ_ARCH) makedumpfile makedumpfile.8 makedumpfile.conf.5
erase_info.c +22 -0
@@ -2220,6 +2220,8 @@ process_eppic_file(char *name_config, bool is_btf)if (!is_btf) { eppic_init =dlsym (handle,"eppic_dwarf_init" ); + }else { + eppic_init =dlsym (handle,"eppic_btf_init" ); }if (!eppic_init) {ERRMSG ("Could not find eppic_init function\n" );
@@ -2348,6 +2350,26 @@ gather_filter_info(void) {int ret = TRUE; +if (!info->name_vmlinux && info->name_eppic_config) { +/* No vmlinux is present, use btf & kallsyms instead. */ +if (init_kernel_kallsyms ()) + goto fail; +if (init_kernel_btf ()) + goto fail; +if (init_module_kallsyms ()) + goto fail; +if (init_module_btf ()) + goto fail; + ret =process_eppic_file (info->name_eppic_config, true); + goto out; +fail: + ret = FALSE; +out: +cleanup_btf (); +cleanup_kallsyms (); +return ret; + } +/* * Before processing filter config file, load the symbol data of * loaded modules from vmcore.
extension_btf.c new file +258 -0
259 lines not on this page · raw message
extension_eppic.c +1 -1
@@ -83,7 +83,7 @@ reg_callback(char *name, int load)* * set idx value to actual array indexes from specified size */ -static void +void eppic_setupidx (TYPE_S *t,int ref,int nidx,int *idxlst) {/* put the idxlst in index size format */
Signature · 1 line
2.47.0
PATCH v2 13/14 Docs: Update eppic related entries
In reply to Tao Liu--- makedumpfile.8.in | 24 +++++++++++++++--------- print_info.c | 11 +++++------ 2 files changed, 20 insertions(+), 15 deletions(-)
makedumpfile.8.in +15 -9
@@ -6,7 +6,7 @@ makedumpfile \- make a small dumpfile of kdump .br \fBmakedumpfile\fR \-F [\fIOPTION\fR] [\-x \fIVMLINUX\fR|\-i \fIVMCOREINFO\fR] \fIVMCORE\fR .br -\fBmakedumpfile\fR [\fIOPTION\fR] \-x \fIVMLINUX\fR [\-\-config \fIFILTERCONFIGFILE\fR] [\-\-eppic \fIEPPICMACRO\fR] \fIVMCORE\fR \fIDUMPFILE\fR +\fBmakedumpfile\fR [\fIOPTION\fR] [\-x \fIVMLINUX\fR] [\-\-config \fIFILTERCONFIGFILE\fR] [\-\-eppic \fIEPPICMACRO\fR] \fIVMCORE\fR \fIDUMPFILE\fR .br \fBmakedumpfile\fR \-R \fIDUMPFILE\fR .br
@@ -54,6 +54,8 @@ be excluded. .B \- User process data pages .br .B \- Free pages +.br +.B \- Customized pages which specified by eppic macro .PP makedumpfile provides two \fIDUMPFILE\fR formats (the ELF format and the kdump\-compressed format). By default, makedumpfile makes a \fIDUMPFILE\fR in
@@ -72,10 +74,10 @@ filter commands. The memory locations that require to be filtered out are then poisoned with character 'X' (58 in Hex). Refer to \fBmakedumpfile.conf(5)\fR for file format. .PP -Eppic macros can also be used to specify kernel symbols and its members that -need to be filtered. Eppic provides C semantics including language constructs -such as conditional statements, logical and arithmetic operators, functions, -nested loops to traverse and erase kernel data. --eppic requires +Eppic macros can also be used to specify kernel pages or kernel symbols and its +members that need to be filtered. Eppic provides C semantics including language +constructs such as conditional statements, logical and arithmetic operators, +functions, nested loops to traverse and erase kernel data. --eppic requires \fBeppic_makedumpfile.so\fR and eppic library. \fBeppic_makedumpfile.so\fR can be built from makedumpfile source. Refer to \fBhttp://code.google.com/p/eppic/\fR to build eppic library \fBlibeppic.a\fR
@@ -316,10 +318,14 @@ command syntax please refer to \fBmakedumpfile.conf(5)\fR. .TP \fB\-\-eppic\fR \fIEPPICMACRO\fR -Used in conjunction with \-x \fIVMLINUX\fR option, to specify the eppic macro -file that contains filter rules or directory that contains eppic macro -files to filter out desired kernel data from vmcore while creating \fIDUMPFILE\fR. -When directory is specified, all the eppic macros in the directory are processed. +To specify the eppic macro file that contains filter rules or directory that +contains eppic macro files to filter out desired kernel data/pages from vmcore +while creating \fIDUMPFILE\fR. When directory is specified, all the eppic macros +in the directory are processed. Please note, if the option is used in conjunction +with \-x \fIVMLINUX\fR option, makedumpfile will use the dwarf data of +\fIVMLINUX\fR as debuginfo for symbol resolving; otherwise, btf & kallsyms +data(if available) of \fIVMCORE\fR will be used instead. Rust symbols are not +supported in eppic macro file. .TP \fB\-F\fR
print_info.c +5 -6
@@ -77,7 +77,7 @@ print_usage(void)MSG ("\n" );MSG (" Creating DUMPFILE with filtered kernel data specified through filter config\n" );MSG (" file or eppic macro:\n" ); -MSG (" # makedumpfile [-c|-l|-p|-z|-E] [-d DL] -x VMLINUX [--config FILTERCONFIGFILE]\n" ); +MSG (" # makedumpfile [-c|-l|-p|-z|-E] [-d DL] [-x VMLINUX] [--config FILTERCONFIGFILE]\n" );MSG (" [--eppic EPPICMACRO] VMCORE DUMPFILE\n" );MSG ("\n" );MSG (" Outputting the dump data in the flattened format to the standard output:\n" );
@@ -190,11 +190,10 @@ print_usage(void)MSG (" from vmcore while creating DUMPFILE.\n" );MSG ("\n" );MSG (" [--eppic EPPICMACRO]:\n" ); -MSG (" Used in conjunction with -x VMLINUX option, to specify the eppic macro\n" ); -MSG (" file that contains filter rules or directory that contains eppic macro\n" ); -MSG (" files to filter out desired kernel data from vmcore while creating DUMPFILE.\n" ); -MSG (" When directory is specified, all the eppic macros in the directory are\n" ); -MSG (" processed\n" ); +MSG (" To specify the eppic macro file that contains filter rules or directory\n" ); +MSG (" that contains eppic macro files to filter out desired kernel data/pages\n" ); +MSG (" from vmcore while creating DUMPFILE. When directory is specified, all the\n" ); +MSG (" eppic macros in the directory are processed\n" );MSG ("\n" );MSG (" [-F]:\n" );MSG (" Output the dump data in the flattened format to the standard output\n" );
Signature · 1 line
2.47.0
PATCH v2 14/14 Introducing 2 eppic scripts to test the dwarf/btf eppic extension
In reply to Tao LiuThis patch will introduce 2 eppic scripts. One is for filtering out amdgpu mm pages, the other is for printing all tasks VMAs. dwarf & btf eppic extension should produce the same result for every eppic script, mainly for test use.
--- eppic_scripts/filter_amdgpu_mm_pages.c | 255 +++++++++++++++++++++++++ eppic_scripts/print_all_vma.c | 244 +++++++++++++++++++++++ 2 files changed, 499 insertions(+) create mode 100644 eppic_scripts/filter_amdgpu_mm_pages.c create mode 100644 eppic_scripts/print_all_vma.c
eppic_scripts/filter_amdgpu_mm_pages.c new file +255 -0
@@ -0,0 +1,255 @@ +enum { + maple_dense_enum, + maple_leaf_64_enum, + maple_range_64_enum, + maple_arange_64_enum, +}; + +#define MAPLE_NODE_MASK 255UL +#define MAPLE_NODE_TYPE_MASK 0x0F +#define MAPLE_NODE_TYPE_SHIFT 0x03 +#define XA_ZERO_ENTRY xa_mk_internal(257) +#define bool int + +unsigned long mt_max; + +static unsigned long xa_mk_internal (unsigned long v) +{ +return (v <<2 ) |2 ; +} + +static bool xa_is_internal (unsigned long entry) +{ +return (entry &3 ) ==2 ; +} + +static bool xa_is_node (unsigned long entry) +{ +return xa_is_internal (entry) && entry >4096 ; +} + +static bool xa_is_value (unsigned long entry) +{ +return entry &1 ; +} + +static bool xa_is_zero (unsigned long entry) +{ +return entry ==XA_ZERO_ENTRY ; +} + +static unsigned long xa_to_internal (unsigned long entry) +{ +return entry >>2 ; +} + +static unsigned long xa_to_value (unsigned long entry) +{ +return entry >>1 ; +} + +static struct maple_node *mte_to_node (void *entry) +{ +return (struct maple_node *)((unsigned long )entry & ~MAPLE_NODE_MASK ); +} + +static unsigned long mte_node_type (unsigned long maple_enode_entry) +{ +return (maple_enode_entry >>MAPLE_NODE_TYPE_SHIFT ) & +MAPLE_NODE_TYPE_MASK ; +} + +static unsigned long mt_slot (void **slots,unsigned char offset) +{ +return *((unsigned long *)slots + offset); +} + +static bool ma_is_leaf (unsigned long type) +{ +return type < maple_range_64_enum; +} + +static bool mte_is_leaf (unsigned long maple_enode_entry) +{ +return ma_is_leaf (mte_node_type (maple_enode_entry)); +} + +static void mt_dump_entry (unsigned long entry,unsigned long min, +unsigned long max,unsigned int depth,unsigned long array_out) +{ +int i; +if (entry ==0 ) +return ; + for (i in array_out) { + if (array_out[i] == 0) { + array_out[i] = entry; + return; + } + } + array_out[i + 1] = entry; +} + +static void mt_dump_range64(void *entry, unsigned long min, + unsigned long max, unsigned int depth, unsigned long array_out) +{ + struct maple_range_64 *node = &mte_to_node(entry)->mr64; + + bool leaf = mte_is_leaf((unsigned long)entry); + unsigned long first = min; + int i; + + for (i = 0; i < *((char *)(&mt_slots) + maple_range_64_enum); i++) { + unsigned long last = max; + + if (i < (*((char *)(&mt_slots) + maple_range_64_enum) - 1)) + last = *((unsigned long *)&(node->pivot) + i); + else if (!*((unsigned long *)&(node->slot) + i) && + max != mt_max[mte_node_type((unsigned long)entry)]) + break; + if (last == 0 && i > 0) + break; + if (leaf) + mt_dump_entry(mt_slot(&(node->slot), i), + first, last, depth + 1, array_out); + else if (*((unsigned long *)&(node->slot) + i)) + mt_dump_node(mt_slot(&(node->slot), i), + first, last, depth + 1, array_out); + + if (last == max) + break; + if (last > max) { + printf("node %p last (%lu) > max (%lu) at pivot %d!\n", + node, last, max, i); + break; + } + first = last + 1; + } +} + +static void mt_dump_arange64(void *entry, unsigned long min, + unsigned long max, unsigned int depth, unsigned long array_out) +{ + struct maple_arange_64 *node = &mte_to_node(entry)->ma64; + unsigned long first = min; + int i; + + for (i = 0; i < *((char *)(&mt_slots) + maple_arange_64_enum); i++) { + unsigned long last = max; + if (i < (*((char *)(&mt_slots) + maple_arange_64_enum) - 1)) + last = *((unsigned long *)&(node->pivot) + i); + else if (!*((unsigned long *)&(node->slot) + i)) + break; + if (last == 0 && i > 0) + break; + if (*((unsigned long *)&(node->slot) + i)) + mt_dump_node((void *)mt_slot(&(node->slot), i), + first, last, depth + 1, array_out); + + if (last == max) + break; + if (last > max) { + printf("node %p last (%lu) > max (%lu) at pivot %d!\n", + node, last, max, i); + break; + } + first = last + 1; + } +} + +static void mt_dump_node(void *entry, unsigned long min, + unsigned long max, unsigned int depth, unsigned long array_out) +{ + struct maple_node *node = mte_to_node(entry); + unsigned int type = mte_node_type((unsigned long)entry); + unsigned int i; + + switch (type) { + case maple_dense_enum: + for (i = 0; i < *((char *)(&mt_slots) + maple_dense_enum); i++) { + if (min + i > max) + printf("OUT OF RANGE: "); + mt_dump_entry(mt_slot(&(node->slot), i), + min + i, min + i, depth, array_out); + } + break; + case maple_leaf_64_enum: + case maple_range_64_enum: + mt_dump_range64(entry, min, max, depth, array_out); + break; + case maple_arange_64_enum: + mt_dump_arange64(entry, min, max, depth, array_out); + break; + + default: + printf(" UNKNOWN TYPE\n"); + } +} + +void mt_dump(const struct maple_tree *mt, unsigned long array_out) +{ + unsigned long entry = (unsigned long)(mt->ma_root); + +/* init mt_max array */ + mt_max[maple_dense_enum] = *((char *)(&mt_slots) + maple_dense_enum); + mt_max[maple_leaf_64_enum] = (unsigned long)(-1); + mt_max[maple_range_64_enum] = (unsigned long)(-1); + mt_max[maple_arange_64_enum] = (unsigned long)(-1); + /* done */ + + if (xa_is_node(entry)) + mt_dump_node((void *)entry, 0, mt_max[mte_node_type(entry)], 0, array_out); + else if (entry) + mt_dump_entry((void *)entry, 0, 0, 0, array_out); + else + printf("(empty)\n"); +} + +int main() +{ + struct task_struct *p; + unsigned long p_off; + int i, c; + struct vm_area_struct *vma; + struct ttm_buffer_object *tbo; + unsigned long pfn, num; + struct maple_tree *mt; + unsigned long tmp; + + p = (struct task_struct *)&init_task; + p_off = (unsigned long )&(p->tasks) - (unsigned long )p; + +do { +unsigned long array_out; + array_out[0 ] =0 ; + +if (!(p->mm)) { + p = (struct task_struct *) + ((unsigned long )(p->tasks.next) - p_off); +continue ; + } + + mt = &(p->mm->mm_mt); + +mt_dump (mt, array_out); + + for (i in array_out) { +if (((struct vm_area_struct *)array_out[i])->vm_ops == + &amdgpu_gem_vm_ops) { + tbo = (struct ttm_buffer_object *) + (((struct vm_area_struct *)array_out[i])-> + vm_private_data); +if (tbo->ttm) { + num = (unsigned long )(tbo->ttm->num_pages); + pfn = ((unsigned long )(tbo->ttm->pages[0 ]) - + *(unsigned long *)&vmemmap_base) / +sizeof (struct page ); +filter_pages (pfn, num); + } + } + } + p = (struct task_struct *) + ((unsigned long )(p->tasks.next) - p_off); + }while (p != &init_task); + +return 0 ; +}
eppic_scripts/print_all_vma.c new file +244 -0
244 lines not on this page · raw message
Signature · 1 line
2.47.0
Re: [PATCH v2][makedumpfile 14/14] Introducing 2 eppic scripts to test the dwarf/btf eppic extension
Quoted 6 lines · Tao Liu
This patch will introduce 2 eppic scripts. One is for filtering out amdgpu mm pages, the other is for printing all tasks VMAs. dwarf & btf eppic extension should produce the same result for every eppic script, mainly for test use.
Signed-off-by Tao Liu <[email protected]>
Yes, I think it's very good to have moved the maple tree code into this eppic script.
but I may think about not having new eppic scripts in the makedumpfile repository any more, like as the crash extension modules are. But please keep including this patch in the patchset for review and test.
Thanks,
Kazu
Kindly ping... Any comments on this?
Thanks,
Tao Liu
Quoted 196 lines · Tao Liu
A) This patchset will introduce the following features to makedumpfile:
1) Enable eppic script for memory pages filtering.
2) Enable btf and kallsyms for symbol type and address resolving.B) The purpose of the features are:
1) Currently makedumpfile filters mm pages based on page flags, because flags can help to determine one page's usage. But this page-flag-checking method lacks of flexibility in certain cases, e.g. if we want to filter those mm pages occupied by GPU during vmcore dumping due to:
a) GPU may be taking a large memory and contains sensitive data;
b) GPU mm pages have no relations to kernel crash and useless for vmcore analysis.But there is no GPU mm page specific flags, and apparently we don't need to create one just for kdump use. A programmable filtering tool is more suitable for such cases. In addition, different GPU vendors may use different ways for mm pages allocating, programmable filtering is better than hard coding these GPU specific logics into makedumpfile in this case.
2) Currently makedumpfile already contains a programmable filtering tool, aka eppic script, which allows user to write customized code for data erasing. However it has the following drawbacks:
a) cannot do mm page filtering.
b) need to access to debuginfo of both kernel and modules, which is not applicable in the 2nd kernel.
c) Poor performance, making vmcore dumping time unacceptable (See the following performance testing).makedumpfile need to resolve the dwarf data from debuginfo, to get symbols types and addresses. In recent kernel there are dwarf alternatives such as btf/kallsyms which can be used for this purpose. And btf/kallsyms info are already packed within vmcore, so we can use it directly.
With these, this patchset introduces an upgraded eppic, which is based on btf/kallsyms symbol resolving, and is programmable for mm page filtering. The following info shows its usage and performance, please note the tests are performed in 1st kernel:
$ time ./makedumpfile -d 31 -l /var/crash/127.0.0.1-2025-06-10-18\:03\:12/vmcore /tmp/dwarf.out -x /lib/debug/lib/modules/6.11.8-300.fc41.x86_64/vmlinux --eppic eppic_scripts/filter_amdgpu_mm_pages.c real 14m6.894s user 4m16.900s sys 9m44.695s$ time ./makedumpfile -d 31 -l /var/crash/127.0.0.1-2025-06-10-18\:03\:12/vmcore /tmp/btf.out --eppic eppic_scripts/filter_amdgpu_mm_pages.c real 0m10.672s user 0m9.270s sys 0m1.130s-rw------- 1 root root 367475074 Jun 10 18:06 btf.out
-rw------- 1 root root 367475074 Jun 10 21:05 dwarf.out
-rw-rw-rw- 1 root root 387181418 Jun 10 18:03 /var/crash/127.0.0.1-2025-06-10-18:03:12/vmcoreC) Discussion:
1) GPU types: Currently only tested with amdgpu's mm page filtering, others are not tested.
2) OS: The code can work on rhel-10+/rhel9.5+ on x86_64/arm64/s390/ppc64. Others are not tested.D) Testing:
1) If you don't want to create your vmcore, you can find a vmcore which I created with amdgpu mm pages unfiltered [1], the amdgpu mm pages are allocated by program [2]. You can use the vmcore in 1st kernel to filter the amdgpu mm pages by the previous performance testing cmdline. To verify the pages are filtered in crash:
Unfiltered:
crash> search -c "!QAZXSW@#EDC"
ffff96b7fa800000: !QAZXSW@#EDCXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
ffff96b87c800000: !QAZXSW@#EDCXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX crash> rdffff96b7fa800000
ffff96b7fa800000:405753585a415121!QAZXSW@
crash> rdffff96b87c800000
ffff96b87c800000:405753585a415121!QAZXSW@Filtered:
crash> search -c "!QAZXSW@#EDC"
crash> rdffff96b7fa800000
rd: page excluded: kernel virtual address:ffff96b7fa800000type: "64-bit KVADDR" crash> rdffff96b87c800000
rd: page excluded: kernel virtual address:ffff96b87c800000type: "64-bit KVADDR"2) You can use
eppic_scripts/print_all_vma.cagainst an ordinary vmcore to test only btf/kallsyms functions by output all VMAs if no amdgpu vmcores/machine avaliable.[1]: https://people.redhat.com/~ltao/core/
[2]: https://gist.github.com/liutgnu/a8cbce1c666452f1530e1410d1f352dfv2 -> v1:
1) Moved maple tree related code(for VMA iteration) into eppic script, so we don't need to port maple tree code to makedumpfile.
2) Reorganized the patchset as follows:
--- <common modification> ---
1.Add page filtering function 2.Supportingmain()as the entry of eppic script--- <dwarf related modification> --- 3.dwarf_info: Support kernel address randomization 4.dwarf_info: Fix a infinite recursion bug for rust 5.eppic dwarf: support anonymous structs member resolving 6.Enable page filtering for dwarf eppic
--- <btf & kallsyms related modification> --- 7.Implement kernel kallsyms resolving
8.Implement kernel btf resolving
9.Implement kernel module's kallsyms resolving 10.Implement kernel module's btf resolving 11.Export necessary btf/kallsyms functions to eppic extension 12.Enable page filtering for btf/kallsyms eppic 13.Docs: Update eppic related entries--- <only for test purpose, don't merge> --- 14.Introducing 2 eppic scripts to test the dwarf/btf eppic extension
The modification on dwarf is primary for comparision purpose, that for the same eppic program, mm page filtering should get exact same outputs for dwarf & kallsyms/btf based approaches. If outputs unmatch, this indicates bugs. In fact, we will never take dwarf mm pages filtering in real use, due to its poor performance as well as inaccessibility of debuginfo during kdump in 2nd kernel. So patch 3/4/5 won't affect the function of btf/kallsyms eppic mm page filtering, but there are functions shared in patch 6, so it is a must-have one. Patch 14 is only for test purpose, to demonstrate how to write eppic script for mm page filtering, so it isn't a must-have patch.
Please note, in patch 14, I have deliberately converted all array operation into pointer operation, e.g. modified "node->slot[i]" into "*((unsigned long *)&(node->slot) + i)". This is because there are bugs for array operation support in
extension_eppic.c. I didn't have effort to test and fix them all because as I mentioned previously, mm page filtering in dwarf side is only for comparision and will never be used in real use. There is no such issue for kallsyms/btf eppic side.3) Since we ported maple tree code to eppic script, several bugs found both for eppic library & eppic btf support. Please use master branch of eppic library to co-compile with this patchset.
Tao Liu (14):
Add page filtering function
Supportingmain()as the entry of eppic script
dwarf_info: Support kernel address randomization
dwarf_info: Fix a infinite recursion bug for rust eppic dwarf: support anonymous structs member resolving Enable page filtering for dwarf eppic
Implement kernel kallsyms resolving
Implement kernel btf resolving
Implement kernel module's kallsyms resolving
Implement kernel module's btf resolving
Export necessary btf/kallsyms functions to eppic extension Enable page filtering for btf/kallsyms eppic
Docs: Update eppic related entries
Introducing 2 eppic scripts to test the dwarf/btf eppic extensionMakefile | 6 +- btf.c | 919 +++++++++++++++++++++++++ btf.h | 177 +++++ dwarf_info.c | 7 + eppic_scripts/filter_amdgpu_mm_pages.c | 255 +++++++ eppic_scripts/print_all_vma.c | 239 +++++++ erase_info.c | 120 +++- erase_info.h | 19 + extension_btf.c | 258 +++++++ extension_eppic.c | 106 ++- extension_eppic.h | 6 +- kallsyms.c | 392 +++++++++++ kallsyms.h | 41 ++ makedumpfile.8.in | 24 +- makedumpfile.c | 21 +- makedumpfile.h | 11 + print_info.c | 11 +- 17 files changed, 2550 insertions(+), 62 deletions(-) create mode 100644 btf.c create mode 100644 btf.h create mode 100644 eppic_scripts/filter_amdgpu_mm_pages.c create mode 100644 eppic_scripts/print_all_vma.c create mode 100644 extension_btf.c create mode 100644 kallsyms.c create mode 100644 kallsyms.h--
2.47.0
Quoted 1 line · Tao Liu
Kindly ping... Any comments on this?
Hi Tao,
I'm sorry for the delay. I think I can look into this next month.
Thanks,
Kazu
Quoted 201 lines
Thanks,
Tao LiuQuoted 196 lines · Tao Liu
A) This patchset will introduce the following features to makedumpfile:
1) Enable eppic script for memory pages filtering.
2) Enable btf and kallsyms for symbol type and address resolving.B) The purpose of the features are:
1) Currently makedumpfile filters mm pages based on page flags, because flags can help to determine one page's usage. But this page-flag-checking method lacks of flexibility in certain cases, e.g. if we want to filter those mm pages occupied by GPU during vmcore dumping due to:
a) GPU may be taking a large memory and contains sensitive data;
b) GPU mm pages have no relations to kernel crash and useless for vmcore analysis.But there is no GPU mm page specific flags, and apparently we don't need to create one just for kdump use. A programmable filtering tool is more suitable for such cases. In addition, different GPU vendors may use different ways for mm pages allocating, programmable filtering is better than hard coding these GPU specific logics into makedumpfile in this case.
2) Currently makedumpfile already contains a programmable filtering tool, aka eppic script, which allows user to write customized code for data erasing. However it has the following drawbacks:
a) cannot do mm page filtering.
b) need to access to debuginfo of both kernel and modules, which is not applicable in the 2nd kernel.
c) Poor performance, making vmcore dumping time unacceptable (See the following performance testing).makedumpfile need to resolve the dwarf data from debuginfo, to get symbols types and addresses. In recent kernel there are dwarf alternatives such as btf/kallsyms which can be used for this purpose. And btf/kallsyms info are already packed within vmcore, so we can use it directly.
With these, this patchset introduces an upgraded eppic, which is based on btf/kallsyms symbol resolving, and is programmable for mm page filtering. The following info shows its usage and performance, please note the tests are performed in 1st kernel: $ time ./makedumpfile -d 31 -l /var/crash/127.0.0.1-2025-06-10-18\:03\:12/vmcore /tmp/dwarf.out -x /lib/debug/lib/modules/6.11.8-300.fc41.x86_64/vmlinux --eppic eppic_scripts/filter_amdgpu_mm_pages.c real 14m6.894s user 4m16.900s sys 9m44.695s $ time ./makedumpfile -d 31 -l /var/crash/127.0.0.1-2025-06-10-18\:03\:12/vmcore /tmp/btf.out --eppic eppic_scripts/filter_amdgpu_mm_pages.c real 0m10.672s user 0m9.270s sys 0m1.130s -rw------- 1 root root 367475074 Jun 10 18:06 btf.out -rw------- 1 root root 367475074 Jun 10 21:05 dwarf.out -rw-rw-rw- 1 root root 387181418 Jun 10 18:03 /var/crash/127.0.0.1-2025-06-10-18:03:12/vmcoreC) Discussion:
1) GPU types: Currently only tested with amdgpu's mm page filtering, others are not tested.
2) OS: The code can work on rhel-10+/rhel9.5+ on x86_64/arm64/s390/ppc64. Others are not tested.D) Testing:
1) If you don't want to create your vmcore, you can find a vmcore which I created with amdgpu mm pages unfiltered [1], the amdgpu mm pages are allocated by program [2]. You can use the vmcore in 1st kernel to filter the amdgpu mm pages by the previous performance testing cmdline. To verify the pages are filtered in crash:
Unfiltered:
crash> search -c "!QAZXSW@#EDC"
ffff96b7fa800000: !QAZXSW@#EDCXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
ffff96b87c800000: !QAZXSW@#EDCXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX crash> rdffff96b7fa800000
ffff96b7fa800000:405753585a415121!QAZXSW@
crash> rdffff96b87c800000
ffff96b87c800000:405753585a415121!QAZXSW@Filtered:
crash> search -c "!QAZXSW@#EDC"
crash> rdffff96b7fa800000
rd: page excluded: kernel virtual address:ffff96b7fa800000type: "64-bit KVADDR" crash> rdffff96b87c800000
rd: page excluded: kernel virtual address:ffff96b87c800000type: "64-bit KVADDR"2) You can use
eppic_scripts/print_all_vma.cagainst an ordinary vmcore to test only btf/kallsyms functions by output all VMAs if no amdgpu vmcores/machine avaliable.[1]: https://people.redhat.com/~ltao/core/
[2]: https://gist.github.com/liutgnu/a8cbce1c666452f1530e1410d1f352dfv2 -> v1:
1) Moved maple tree related code(for VMA iteration) into eppic script, so we don't need to port maple tree code to makedumpfile.
2) Reorganized the patchset as follows:
--- <common modification> ---
1.Add page filtering function 2.Supportingmain()as the entry of eppic script--- <dwarf related modification> --- 3.dwarf_info: Support kernel address randomization 4.dwarf_info: Fix a infinite recursion bug for rust 5.eppic dwarf: support anonymous structs member resolving 6.Enable page filtering for dwarf eppic
--- <btf & kallsyms related modification> --- 7.Implement kernel kallsyms resolving
8.Implement kernel btf resolving
9.Implement kernel module's kallsyms resolving 10.Implement kernel module's btf resolving 11.Export necessary btf/kallsyms functions to eppic extension 12.Enable page filtering for btf/kallsyms eppic 13.Docs: Update eppic related entries--- <only for test purpose, don't merge> --- 14.Introducing 2 eppic scripts to test the dwarf/btf eppic extension
The modification on dwarf is primary for comparision purpose, that for the same eppic program, mm page filtering should get exact same outputs for dwarf & kallsyms/btf based approaches. If outputs unmatch, this indicates bugs. In fact, we will never take dwarf mm pages filtering in real use, due to its poor performance as well as inaccessibility of debuginfo during kdump in 2nd kernel. So patch 3/4/5 won't affect the function of btf/kallsyms eppic mm page filtering, but there are functions shared in patch 6, so it is a must-have one. Patch 14 is only for test purpose, to demonstrate how to write eppic script for mm page filtering, so it isn't a must-have patch.
Please note, in patch 14, I have deliberately converted all array operation into pointer operation, e.g. modified "node->slot[i]" into "*((unsigned long *)&(node->slot) + i)". This is because there are bugs for array operation support in
extension_eppic.c. I didn't have effort to test and fix them all because as I mentioned previously, mm page filtering in dwarf side is only for comparision and will never be used in real use. There is no such issue for kallsyms/btf eppic side.3) Since we ported maple tree code to eppic script, several bugs found both for eppic library & eppic btf support. Please use master branch of eppic library to co-compile with this patchset.
Tao Liu (14):
Add page filtering function
Supportingmain()as the entry of eppic script
dwarf_info: Support kernel address randomization
dwarf_info: Fix a infinite recursion bug for rust eppic dwarf: support anonymous structs member resolving Enable page filtering for dwarf eppic
Implement kernel kallsyms resolving
Implement kernel btf resolving
Implement kernel module's kallsyms resolving
Implement kernel module's btf resolving
Export necessary btf/kallsyms functions to eppic extension Enable page filtering for btf/kallsyms eppic
Docs: Update eppic related entries
Introducing 2 eppic scripts to test the dwarf/btf eppic extensionMakefile | 6 +- btf.c | 919 +++++++++++++++++++++++++ btf.h | 177 +++++ dwarf_info.c | 7 + eppic_scripts/filter_amdgpu_mm_pages.c | 255 +++++++ eppic_scripts/print_all_vma.c | 239 +++++++ erase_info.c | 120 +++- erase_info.h | 19 + extension_btf.c | 258 +++++++ extension_eppic.c | 106 ++- extension_eppic.h | 6 +- kallsyms.c | 392 +++++++++++ kallsyms.h | 41 ++ makedumpfile.8.in | 24 +- makedumpfile.c | 21 +- makedumpfile.h | 11 + print_info.c | 11 +- 17 files changed, 2550 insertions(+), 62 deletions(-)create mode 100644
btf.c
create mode 100644btf.h
create mode 100644eppic_scripts/filter_amdgpu_mm_pages.ccreate mode 100644eppic_scripts/print_all_vma.ccreate mode 100644extension_btf.c
create mode 100644kallsyms.c
create mode 100644kallsyms.h--
2.47.0
Hi Kazu,
Quoted 7 lines · HAGIO KAZUHITO(萩尾 一仁)
Quoted 1 line · Tao Liu
Kindly ping... Any comments on this?
Hi Tao,
I'm sorry for the delay. I think I can look into this next month.
No worries, please take your time :)
Thanks,
Tao Liu
Quoted 205 lines
Thanks,
KazuQuoted 201 lines
Thanks,
Tao LiuQuoted 196 lines · Tao Liu
A) This patchset will introduce the following features to makedumpfile:
1) Enable eppic script for memory pages filtering.
2) Enable btf and kallsyms for symbol type and address resolving.B) The purpose of the features are:
1) Currently makedumpfile filters mm pages based on page flags, because flags can help to determine one page's usage. But this page-flag-checking method lacks of flexibility in certain cases, e.g. if we want to filter those mm pages occupied by GPU during vmcore dumping due to:
a) GPU may be taking a large memory and contains sensitive data;
b) GPU mm pages have no relations to kernel crash and useless for vmcore analysis.But there is no GPU mm page specific flags, and apparently we don't need to create one just for kdump use. A programmable filtering tool is more suitable for such cases. In addition, different GPU vendors may use different ways for mm pages allocating, programmable filtering is better than hard coding these GPU specific logics into makedumpfile in this case.
2) Currently makedumpfile already contains a programmable filtering tool, aka eppic script, which allows user to write customized code for data erasing. However it has the following drawbacks:
a) cannot do mm page filtering.
b) need to access to debuginfo of both kernel and modules, which is not applicable in the 2nd kernel.
c) Poor performance, making vmcore dumping time unacceptable (See the following performance testing).makedumpfile need to resolve the dwarf data from debuginfo, to get symbols types and addresses. In recent kernel there are dwarf alternatives such as btf/kallsyms which can be used for this purpose. And btf/kallsyms info are already packed within vmcore, so we can use it directly.
With these, this patchset introduces an upgraded eppic, which is based on btf/kallsyms symbol resolving, and is programmable for mm page filtering. The following info shows its usage and performance, please note the tests are performed in 1st kernel: $ time ./makedumpfile -d 31 -l /var/crash/127.0.0.1-2025-06-10-18\:03\:12/vmcore /tmp/dwarf.out -x /lib/debug/lib/modules/6.11.8-300.fc41.x86_64/vmlinux --eppic eppic_scripts/filter_amdgpu_mm_pages.c real 14m6.894s user 4m16.900s sys 9m44.695s $ time ./makedumpfile -d 31 -l /var/crash/127.0.0.1-2025-06-10-18\:03\:12/vmcore /tmp/btf.out --eppic eppic_scripts/filter_amdgpu_mm_pages.c real 0m10.672s user 0m9.270s sys 0m1.130s -rw------- 1 root root 367475074 Jun 10 18:06 btf.out -rw------- 1 root root 367475074 Jun 10 21:05 dwarf.out -rw-rw-rw- 1 root root 387181418 Jun 10 18:03 /var/crash/127.0.0.1-2025-06-10-18:03:12/vmcoreC) Discussion:
1) GPU types: Currently only tested with amdgpu's mm page filtering, others are not tested.
2) OS: The code can work on rhel-10+/rhel9.5+ on x86_64/arm64/s390/ppc64. Others are not tested.D) Testing:
1) If you don't want to create your vmcore, you can find a vmcore which I created with amdgpu mm pages unfiltered [1], the amdgpu mm pages are allocated by program [2]. You can use the vmcore in 1st kernel to filter the amdgpu mm pages by the previous performance testing cmdline. To verify the pages are filtered in crash:
Unfiltered:
crash> search -c "!QAZXSW@#EDC"
ffff96b7fa800000: !QAZXSW@#EDCXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
ffff96b87c800000: !QAZXSW@#EDCXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX crash> rdffff96b7fa800000
ffff96b7fa800000:405753585a415121!QAZXSW@
crash> rdffff96b87c800000
ffff96b87c800000:405753585a415121!QAZXSW@Filtered:
crash> search -c "!QAZXSW@#EDC"
crash> rdffff96b7fa800000
rd: page excluded: kernel virtual address:ffff96b7fa800000type: "64-bit KVADDR" crash> rdffff96b87c800000
rd: page excluded: kernel virtual address:ffff96b87c800000type: "64-bit KVADDR"2) You can use
eppic_scripts/print_all_vma.cagainst an ordinary vmcore to test only btf/kallsyms functions by output all VMAs if no amdgpu vmcores/machine avaliable.[1]: https://people.redhat.com/~ltao/core/
[2]: https://gist.github.com/liutgnu/a8cbce1c666452f1530e1410d1f352dfv2 -> v1:
1) Moved maple tree related code(for VMA iteration) into eppic script, so we don't need to port maple tree code to makedumpfile.
2) Reorganized the patchset as follows:
--- <common modification> ---
1.Add page filtering function 2.Supportingmain()as the entry of eppic script--- <dwarf related modification> --- 3.dwarf_info: Support kernel address randomization 4.dwarf_info: Fix a infinite recursion bug for rust 5.eppic dwarf: support anonymous structs member resolving 6.Enable page filtering for dwarf eppic
--- <btf & kallsyms related modification> --- 7.Implement kernel kallsyms resolving
8.Implement kernel btf resolving
9.Implement kernel module's kallsyms resolving 10.Implement kernel module's btf resolving 11.Export necessary btf/kallsyms functions to eppic extension 12.Enable page filtering for btf/kallsyms eppic 13.Docs: Update eppic related entries--- <only for test purpose, don't merge> --- 14.Introducing 2 eppic scripts to test the dwarf/btf eppic extension
The modification on dwarf is primary for comparision purpose, that for the same eppic program, mm page filtering should get exact same outputs for dwarf & kallsyms/btf based approaches. If outputs unmatch, this indicates bugs. In fact, we will never take dwarf mm pages filtering in real use, due to its poor performance as well as inaccessibility of debuginfo during kdump in 2nd kernel. So patch 3/4/5 won't affect the function of btf/kallsyms eppic mm page filtering, but there are functions shared in patch 6, so it is a must-have one. Patch 14 is only for test purpose, to demonstrate how to write eppic script for mm page filtering, so it isn't a must-have patch.
Please note, in patch 14, I have deliberately converted all array operation into pointer operation, e.g. modified "node->slot[i]" into "*((unsigned long *)&(node->slot) + i)". This is because there are bugs for array operation support in
extension_eppic.c. I didn't have effort to test and fix them all because as I mentioned previously, mm page filtering in dwarf side is only for comparision and will never be used in real use. There is no such issue for kallsyms/btf eppic side.3) Since we ported maple tree code to eppic script, several bugs found both for eppic library & eppic btf support. Please use master branch of eppic library to co-compile with this patchset.
Tao Liu (14):
Add page filtering function
Supportingmain()as the entry of eppic script
dwarf_info: Support kernel address randomization
dwarf_info: Fix a infinite recursion bug for rust eppic dwarf: support anonymous structs member resolving Enable page filtering for dwarf eppic
Implement kernel kallsyms resolving
Implement kernel btf resolving
Implement kernel module's kallsyms resolving
Implement kernel module's btf resolving
Export necessary btf/kallsyms functions to eppic extension Enable page filtering for btf/kallsyms eppic
Docs: Update eppic related entries
Introducing 2 eppic scripts to test the dwarf/btf eppic extensionMakefile | 6 +- btf.c | 919 +++++++++++++++++++++++++ btf.h | 177 +++++ dwarf_info.c | 7 + eppic_scripts/filter_amdgpu_mm_pages.c | 255 +++++++ eppic_scripts/print_all_vma.c | 239 +++++++ erase_info.c | 120 +++- erase_info.h | 19 + extension_btf.c | 258 +++++++ extension_eppic.c | 106 ++- extension_eppic.h | 6 +- kallsyms.c | 392 +++++++++++ kallsyms.h | 41 ++ makedumpfile.8.in | 24 +- makedumpfile.c | 21 +- makedumpfile.h | 11 + print_info.c | 11 +- 17 files changed, 2550 insertions(+), 62 deletions(-)create mode 100644
btf.c
create mode 100644btf.h
create mode 100644eppic_scripts/filter_amdgpu_mm_pages.ccreate mode 100644eppic_scripts/print_all_vma.ccreate mode 100644extension_btf.c
create mode 100644kallsyms.c
create mode 100644kallsyms.h--
2.47.0