/proc/vmcore mmap() failure issue
Subsystems (3)
Hi Hatayama,
We are facing some /proc/vmcore mmap() failure issues and then makdumpfile exits without saving dump and system reboots.
I tried latest makedumpfile (devel branch) with 3.12 kernel.
I think this issue happens only on some machines. And it looks like it happens when end of system RAM chunk in first kernel is not page aligned. For example, I have one machine where I noticed it and this is how system RAM looks like.
00100000-dafa57ff : System RAM
01000000-015892fa : Kernel code
015892fb-0195c9ff : Kernel data
01ae6000-01d31fff : Kernel bss
24000000-33ffffff : Crash kernel
dafa5800-dbffffff : reserved
Notice that dafa57ff does not end at page boundary and next reserved range does not start at page boundary. I think that next reserved range is referenced through some ACPI data. More on this later.
So we put some printk() messages to get more info. In a nut shell, remap_pfn_range() fails when we try to map the last section of system RAM not ending on page boundary.
remap_pfn_range()
track_pfn_remap() {
/*
* For anything smaller than the vma size we set prot based on the
* lookup.
*/
flags = lookup_memtype(paddr);
/* Check memtype for the remaining pages */
while (size > PAGE_SIZE) {
size -= PAGE_SIZE;
paddr += PAGE_SIZE;
if (flags != lookup_memtype(paddr))
return -EINVAL; <---------------- Failure.
}}
So we pass in a range to track_pfn_remap. Say pfn=0xdad62 size=0x244000. Now we call lookup_memtype() on every page in the range and make sure they all are same, otherwise we fail. Guess what, all all same except last page (which does not end at page boundary).
I dived deeper in to lookup_memtype() and noticed that all regular ranges are not registered anywhere and their flags are _PAGE_CACHE_UC_MINUS. But last unaligned page/range, is registered in memtype rb tree and has attribute, _PAGE_CACHE_WB.
Then I hooked into reserve_memtype() to figure out who is registering page 0xdafa5000 and it is acpi_init() which does it.
[ 0.721655] Hardware name: <edited> [ 0.730590] ffff8800340f3830 ffff8800340f37c0 ffffffff81575509 00000000dafa5000 [ 0.738010] ffff8800340f3800 ffffffff810566cc 00000000000dafa5 00000000dafa5000 [ 0.745428] 00000000dafa6000 00000000dafa5000 0000000000000000 0000000000001000 [ 0.752845] Call Trace: [ 0.755288] [<ffffffff81575509>] dump_stack+0x45/0x56 [ 0.760414] [<ffffffff810566cc>] reserve_memtype+0x31c/0x3f0 [ 0.766144] [<ffffffff810537ef>] __ioremap_caller+0x12f/0x360 [ 0.771963] [<ffffffff8130ad56>] ? acpi_os_release_object+0xe/0x12 [ 0.778217] [<ffffffff815686ba>] ? acpi_os_map_memory+0xf6/0x14e [ 0.784295] [<ffffffff81053a54>] ioremap_cache+0x14/0x20 [ 0.789679] [<ffffffff815686ba>] acpi_os_map_memory+0xf6/0x14e [ 0.795582] [<ffffffff81322ac9>] acpi_ex_system_memory_space_handler+0xdd/0x1ca [ 0.802961] [<ffffffff8131ca48>] acpi_ev_address_space_dispatch+0x1b0/0x208 [ 0.809993] [<ffffffff8131fd49>] acpi_ex_access_region+0x20e/0x2a2 [ 0.816244] [<ffffffff81149464>] ? __alloc_pages_nodemask+0x134/0x300 [ 0.822754] [<ffffffff813200e4>] acpi_ex_field_datum_io+0xf6/0x171 [ 0.829004] [<ffffffff81320301>] acpi_ex_extract_from_field+0xd7/0x20a [ 0.835602] [<ffffffff81331d80>] ? acpi_ut_create_internal_object_dbg+0x23/0x8a [ 0.842981] [<ffffffff8131f8e7>] acpi_ex_read_data_from_field+0x10f/0x14b [ 0.849838] [<ffffffff81322e16>] acpi_ex_resolve_node_to_value+0x18e/0x21c [ 0.856780] [<ffffffff813230a6>] acpi_ex_resolve_to_value+0x202/0x209 [ 0.863291] [<ffffffff81319486>] acpi_ds_evaluate_name_path+0x7b/0xf5 [ 0.869803] [<ffffffff81319834>] acpi_ds_exec_end_op+0x98/0x3e8 [ 0.875793] [<ffffffff8132aca4>] acpi_ps_parse_loop+0x514/0x560 [ 0.881784] [<ffffffff8132b738>] acpi_ps_parse_aml+0x98/0x28c [ 0.887601] [<ffffffff8132bf8d>] acpi_ps_execute_method+0x1c1/0x26c [ 0.893939] [<ffffffff813269c5>] acpi_ns_evaluate+0x1c1/0x258 [ 0.899755] [<ffffffff8131cb98>] acpi_ev_execute_reg_method+0xca/0x112 [ 0.906353] [<ffffffff8131cd6e>] acpi_ev_reg_run+0x48/0x52 [ 0.911910] [<ffffffff81328fad>] acpi_ns_walk_namespace+0xc8/0x17f [ 0.918160] [<ffffffff8131cd26>] ? acpi_ev_detach_region+0x146/0x146 [ 0.924585] [<ffffffff8131cdbc>] acpi_ev_execute_reg_methods+0x44/0xf7 [ 0.931184] [<ffffffff819b2324>] ? acpi_sleep_proc_init+0x2a/0x2a [ 0.937349] [<ffffffff8130ac66>] ? acpi_os_wait_semaphore+0x43/0x57 [ 0.943686] [<ffffffff81331a3f>] ? acpi_ut_acquire_mutex+0x48/0x88 [ 0.949938] [<ffffffff8131ceb8>] acpi_ev_initialize_op_regions+0x49/0x71 [ 0.956709] [<ffffffff819b2324>] ? acpi_sleep_proc_init+0x2a/0x2a [ 0.962873] [<ffffffff81333310>] acpi_initialize_objects+0x23/0x4f [ 0.969125] [<ffffffff819b23b4>] acpi_init+0x90/0x268
So basically, this split page seems to be a problem. Some other code thinks that it has access to full page and goes ahead and registers that with PAT rb tree and this causes problems in mmap() code.
I suspect we might have to go back to idea of copying first and last non page aligned ranges in new kernel's memory and read it from there to solve this issue. Do you have other ideas?
Thanks
Vivek
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
[CC hpa ]
And this issue brings me to the question that why do we allow sytem RAM ranges which do not start on page boundary or do not end on page boundary. Can't we truncate the BIOS reported RAM ranges in such a way so that they start and end at PAGE boundary and rest of the kernel will never see unaligned portion of RAM and this will make life so much simpler for other tools.
Thanks
Vivek
Quoted 113 lines · Vivek Goyal
Hi Hatayama,
We are facing some
/proc/vmcoremmap()failure issues and then makdumpfile exits without saving dump and system reboots.I tried latest makedumpfile (devel branch) with 3.12 kernel.
I think this issue happens only on some machines. And it looks like it happens when end of system RAM chunk in first kernel is not page aligned. For example, I have one machine where I noticed it and this is how system RAM looks like.
00100000-dafa57ff : System RAM
01000000-015892fa : Kernel code
015892fb-0195c9ff : Kernel data
01ae6000-01d31fff : Kernel bss
24000000-33ffffff : Crash kernel
dafa5800-dbffffff : reservedNotice that
dafa57ffdoes not end at page boundary and next reserved range does not start at page boundary. I think that next reserved range is referenced through some ACPI data. More on this later.So we put some
printk()messages to get more info. In a nut shell,remap_pfn_range()fails when we try to map the last section of system RAM not ending on page boundary.
remap_pfn_range()track_pfn_remap() { /* * For anything smaller than the vma size we set prot based on the * lookup. */ flags = lookup_memtype(paddr); /* Check memtype for the remaining pages */ while (size > PAGE_SIZE) { size -= PAGE_SIZE; paddr += PAGE_SIZE; if (flags != lookup_memtype(paddr)) return -EINVAL; <---------------- Failure. }}So we pass in a range to
track_pfn_remap. Say pfn=0xdad62 size=0x244000. Now we calllookup_memtype()on every page in the range and make sure they all are same, otherwise we fail. Guess what, all all same except last page (which does not end at page boundary).I dived deeper in to
lookup_memtype()and noticed that all regular ranges are not registered anywhere and their flags are_PAGE_CACHE_UC_MINUS. But last unaligned page/range, is registered in memtype rb tree and has attribute,_PAGE_CACHE_WB.Then I hooked into
reserve_memtype()to figure out who is registering page0xdafa5000and it isacpi_init()which does it.[ 0.721655] Hardware name: <edited> [ 0.730590] ffff8800340f3830 ffff8800340f37c0 ffffffff81575509 00000000dafa5000 [ 0.738010] ffff8800340f3800 ffffffff810566cc 00000000000dafa5 00000000dafa5000 [ 0.745428] 00000000dafa6000 00000000dafa5000 0000000000000000 0000000000001000 [ 0.752845] Call Trace: [ 0.755288] [<ffffffff81575509>] dump_stack+0x45/0x56 [ 0.760414] [<ffffffff810566cc>] reserve_memtype+0x31c/0x3f0 [ 0.766144] [<ffffffff810537ef>] __ioremap_caller+0x12f/0x360 [ 0.771963] [<ffffffff8130ad56>] ? acpi_os_release_object+0xe/0x12 [ 0.778217] [<ffffffff815686ba>] ? acpi_os_map_memory+0xf6/0x14e [ 0.784295] [<ffffffff81053a54>] ioremap_cache+0x14/0x20 [ 0.789679] [<ffffffff815686ba>] acpi_os_map_memory+0xf6/0x14e [ 0.795582] [<ffffffff81322ac9>] acpi_ex_system_memory_space_handler+0xdd/0x1ca [ 0.802961] [<ffffffff8131ca48>] acpi_ev_address_space_dispatch+0x1b0/0x208 [ 0.809993] [<ffffffff8131fd49>] acpi_ex_access_region+0x20e/0x2a2 [ 0.816244] [<ffffffff81149464>] ? __alloc_pages_nodemask+0x134/0x300 [ 0.822754] [<ffffffff813200e4>] acpi_ex_field_datum_io+0xf6/0x171 [ 0.829004] [<ffffffff81320301>] acpi_ex_extract_from_field+0xd7/0x20a [ 0.835602] [<ffffffff81331d80>] ? acpi_ut_create_internal_object_dbg+0x23/0x8a [ 0.842981] [<ffffffff8131f8e7>] acpi_ex_read_data_from_field+0x10f/0x14b [ 0.849838] [<ffffffff81322e16>] acpi_ex_resolve_node_to_value+0x18e/0x21c [ 0.856780] [<ffffffff813230a6>] acpi_ex_resolve_to_value+0x202/0x209 [ 0.863291] [<ffffffff81319486>] acpi_ds_evaluate_name_path+0x7b/0xf5 [ 0.869803] [<ffffffff81319834>] acpi_ds_exec_end_op+0x98/0x3e8 [ 0.875793] [<ffffffff8132aca4>] acpi_ps_parse_loop+0x514/0x560 [ 0.881784] [<ffffffff8132b738>] acpi_ps_parse_aml+0x98/0x28c [ 0.887601] [<ffffffff8132bf8d>] acpi_ps_execute_method+0x1c1/0x26c [ 0.893939] [<ffffffff813269c5>] acpi_ns_evaluate+0x1c1/0x258 [ 0.899755] [<ffffffff8131cb98>] acpi_ev_execute_reg_method+0xca/0x112 [ 0.906353] [<ffffffff8131cd6e>] acpi_ev_reg_run+0x48/0x52 [ 0.911910] [<ffffffff81328fad>] acpi_ns_walk_namespace+0xc8/0x17f [ 0.918160] [<ffffffff8131cd26>] ? acpi_ev_detach_region+0x146/0x146 [ 0.924585] [<ffffffff8131cdbc>] acpi_ev_execute_reg_methods+0x44/0xf7 [ 0.931184] [<ffffffff819b2324>] ? acpi_sleep_proc_init+0x2a/0x2a [ 0.937349] [<ffffffff8130ac66>] ? acpi_os_wait_semaphore+0x43/0x57 [ 0.943686] [<ffffffff81331a3f>] ? acpi_ut_acquire_mutex+0x48/0x88 [ 0.949938] [<ffffffff8131ceb8>] acpi_ev_initialize_op_regions+0x49/0x71 [ 0.956709] [<ffffffff819b2324>] ? acpi_sleep_proc_init+0x2a/0x2a [ 0.962873] [<ffffffff81333310>] acpi_initialize_objects+0x23/0x4f [ 0.969125] [<ffffffff819b23b4>] acpi_init+0x90/0x268So basically, this split page seems to be a problem. Some other code thinks that it has access to full page and goes ahead and registers that with PAT rb tree and this causes problems in
mmap()code.
Quoted 6 lines
I suspect we might have to go back to idea of copying first and last non page aligned ranges in new kernel's memory and read it from there to solve this issue. Do you have other ideas?
Thanks
Vivek
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
H. Peter Anvin
reviewerQuoted 9 lines · Vivek Goyal
[CC hpa ]
And this issue brings me to the question that why do we allow sytem RAM ranges which do not start on page boundary or do not end on page boundary. Can't we truncate the BIOS reported RAM ranges in such a way so that they start and end at PAGE boundary and rest of the kernel will never see unaligned portion of RAM and this will make life so much simpler for other tools.
That is a bit of a headache for doing in the memblock space. We do, in fact, truncate partial pages, but later in the game. It is possible we should push that sooner in the stack. The fact that it makes into the rbtrees is fishy, but it also makes me wonder if we're doing something totally stupid with regards to the memory mappings -- if this means we're mapping ACPI data as noncacheable, that is not just a performance problem but just plain wrong. I don't even think the MTRRs can represent different caching attributes for different parts of a page, so this is something that we are doing.
-hpa
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Quoted 11 lines · H. Peter Anvin
Quoted 9 lines · Vivek Goyal
[CC hpa ]
And this issue brings me to the question that why do we allow sytem RAM ranges which do not start on page boundary or do not end on page boundary. Can't we truncate the BIOS reported RAM ranges in such a way so that they start and end at PAGE boundary and rest of the kernel will never see unaligned portion of RAM and this will make life so much simpler for other tools.
Quoted 3 lines
That is a bit of a headache for doing in the memblock space. We do, in fact, truncate partial pages, but later in the game. It is possible we should push that sooner in the stack.
Hi Peter,
I noticed we seem to be trimming away partial pages in memblock.
memblock_x86_fill() {
/* throw away partial pages */
memblock_trim_memory(PAGE_SIZE);
}But not in e820 hence they show up in /proc/iomem.
How about something along the lines as below patch. This fixes my /proc/vmcore mmap() issue.
Thanks
Vivek
--- arch/x86/kernel/e820.c | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+)
arch/x86/kernel/e820.c +29 -0
@@ -169,6 +169,33 @@ void __init e820_print_map(char *who) } } +static int e820_trim_memory (struct e820entry *map,unsigned int nr_entries, +unsigned int align) +{ +int i; +struct e820entry *ei; +u64 start, end, orig_start, orig_end; + +for (i =0 ; i < nr_entries; i++) { + ei = &map[i]; +if (ei->type !=E820_RAM ) +continue ; + orig_start = ei->addr; + orig_end = ei->addr + ei->size; + + start =round_up (orig_start, align); + end =round_down (orig_end, align); + +if (start == orig_start && end == orig_end) +continue ; + + ei->addr = start; + ei->size = end - start; + } + +return 0 ; +} +/* * Sanitize the BIOS e820 map. *
@@ -267,6 +294,8 @@ int __init sanitize_e820_map(struct e820int old_nr, new_nr, chg_nr;int i; +e820_trim_memory (biosmap, *pnr_map,PAGE_SIZE ); +/* if there's only one memory region, don't bother */ if (*pnr_map <2 )return -1 ;
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
H. Peter Anvin
reviewerQuoted 12 lines · Vivek Goyal
Hi Peter,
I noticed we seem to be trimming away partial pages in memblock.
memblock_x86_fill() { /* throw away partial pages */ memblock_trim_memory(PAGE_SIZE); }But not in e820 hence they show up in
/proc/iomem.
Why does /proc/iomem matter?
Quoted 2 lines
How about something along the lines as below patch. This fixes my
/proc/vmcoremmap()issue.
I'm not sure if what you're seeing is something that is better handled in userspace.
-hpa
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Quoted 11 lines · H. Peter Anvin
Quoted 10 lines · Vivek Goyal
Hi Peter,
I noticed we seem to be trimming away partial pages in memblock.
memblock_x86_fill() { /* throw away partial pages */ memblock_trim_memory(PAGE_SIZE); }
Quoted 4 lines
Quoted 2 lines
But not in e820 hence they show up in
/proc/iomem.Why does
/proc/iomemmatter?
Kexec-tools parse /proc/iomem and prepare PT_LOAD elf headers for all the RAM regions which need to be dumped out, including the partial page ones. Second kernel parses these headers and exports to user space in /proc/vmcore. Makedumpfile tries to mmap() vmcore then tries to map the partial page pfn too and we run into issues.
Quoted 6 lines
Quoted 3 lines
How about something along the lines as below patch. This fixes my
/proc/vmcoremmap()issue.I'm not sure if what you're seeing is something that is better handled in userspace.
I think it should be easy to truncate ELF headers in kexec-tools too when we are preparing elf headers. I am not sure if it is right thing to do or not. If some entry is showing up in /proc/iomem as RAM, then kexec-tools need to believe that it is a possibility that kernel is using that pfn and that pfn needs to be dumped out in vmcore. IMHO, kernel should fix this issue.
Secondly, I am writing in-kernel kexec support too and I prepare ELF headers there too. And I am facing the same problem there too. So if we truncate partial pages in e820 both kexec-tools and in kernel kexec implementation don't have to do anything.
Thanks
Vivek
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
H. Peter Anvin
reviewerQuoted 6 lines · Vivek Goyal
I think it should be easy to truncate ELF headers in kexec-tools too when we are preparing elf headers. I am not sure if it is right thing to do or not. If some entry is showing up in
/proc/iomemas RAM, then kexec-tools need to believe that it is a possibility that kernel is using that pfn and that pfn needs to be dumped out in vmcore. IMHO, kernel should fix this issue.
The kernel will never use a fractional page, so that is not an issue.
Quoted 4 lines
Secondly, I am writing in-kernel kexec support too and I prepare ELF headers there too. And I am facing the same problem there too. So if we truncate partial pages in e820 both kexec-tools and in kernel kexec implementation don't have to do anything.
I'm mostly worried about truncation upon truncation causing problems. As long as the trimming is done once and with proper consideration for abutting regions, I guess I'm okay with it, although it feels wrong to me.
-hpa
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Quoted 109 lines · Vivek Goyal
Hi Hatayama,
We are facing some
/proc/vmcoremmap()failure issues and then makdumpfile exits without saving dump and system reboots.I tried latest makedumpfile (devel branch) with 3.12 kernel.
I think this issue happens only on some machines. And it looks like it happens when end of system RAM chunk in first kernel is not page aligned. For example, I have one machine where I noticed it and this is how system RAM looks like.
00100000-dafa57ff : System RAM
01000000-015892fa : Kernel code
015892fb-0195c9ff : Kernel data
01ae6000-01d31fff : Kernel bss
24000000-33ffffff : Crash kernel
dafa5800-dbffffff : reservedNotice that
dafa57ffdoes not end at page boundary and next reserved range does not start at page boundary. I think that next reserved range is referenced through some ACPI data. More on this later.So we put some
printk()messages to get more info. In a nut shell,remap_pfn_range()fails when we try to map the last section of system RAM not ending on page boundary.
remap_pfn_range()track_pfn_remap() { /* * For anything smaller than the vma size we set prot based on the * lookup. */ flags = lookup_memtype(paddr); /* Check memtype for the remaining pages */ while (size > PAGE_SIZE) { size -= PAGE_SIZE; paddr += PAGE_SIZE; if (flags != lookup_memtype(paddr)) return -EINVAL; <---------------- Failure. } }So we pass in a range to
track_pfn_remap. Say pfn=0xdad62 size=0x244000. Now we calllookup_memtype()on every page in the range and make sure they all are same, otherwise we fail. Guess what, all all same except last page (which does not end at page boundary).I dived deeper in to
lookup_memtype()and noticed that all regular ranges are not registered anywhere and their flags are_PAGE_CACHE_UC_MINUS. But last unaligned page/range, is registered in memtype rb tree and has attribute,_PAGE_CACHE_WB.Then I hooked into
reserve_memtype()to figure out who is registering page0xdafa5000and it isacpi_init()which does it.[ 0.721655] Hardware name: <edited> [ 0.730590] ffff8800340f3830 ffff8800340f37c0 ffffffff81575509 00000000dafa5000 [ 0.738010] ffff8800340f3800 ffffffff810566cc 00000000000dafa5 00000000dafa5000 [ 0.745428] 00000000dafa6000 00000000dafa5000 0000000000000000 0000000000001000 [ 0.752845] Call Trace: [ 0.755288] [<ffffffff81575509>] dump_stack+0x45/0x56 [ 0.760414] [<ffffffff810566cc>] reserve_memtype+0x31c/0x3f0 [ 0.766144] [<ffffffff810537ef>] __ioremap_caller+0x12f/0x360 [ 0.771963] [<ffffffff8130ad56>] ? acpi_os_release_object+0xe/0x12 [ 0.778217] [<ffffffff815686ba>] ? acpi_os_map_memory+0xf6/0x14e [ 0.784295] [<ffffffff81053a54>] ioremap_cache+0x14/0x20 [ 0.789679] [<ffffffff815686ba>] acpi_os_map_memory+0xf6/0x14e [ 0.795582] [<ffffffff81322ac9>] acpi_ex_system_memory_space_handler+0xdd/0x1ca [ 0.802961] [<ffffffff8131ca48>] acpi_ev_address_space_dispatch+0x1b0/0x208 [ 0.809993] [<ffffffff8131fd49>] acpi_ex_access_region+0x20e/0x2a2 [ 0.816244] [<ffffffff81149464>] ? __alloc_pages_nodemask+0x134/0x300 [ 0.822754] [<ffffffff813200e4>] acpi_ex_field_datum_io+0xf6/0x171 [ 0.829004] [<ffffffff81320301>] acpi_ex_extract_from_field+0xd7/0x20a [ 0.835602] [<ffffffff81331d80>] ? acpi_ut_create_internal_object_dbg+0x23/0x8a [ 0.842981] [<ffffffff8131f8e7>] acpi_ex_read_data_from_field+0x10f/0x14b [ 0.849838] [<ffffffff81322e16>] acpi_ex_resolve_node_to_value+0x18e/0x21c [ 0.856780] [<ffffffff813230a6>] acpi_ex_resolve_to_value+0x202/0x209 [ 0.863291] [<ffffffff81319486>] acpi_ds_evaluate_name_path+0x7b/0xf5 [ 0.869803] [<ffffffff81319834>] acpi_ds_exec_end_op+0x98/0x3e8 [ 0.875793] [<ffffffff8132aca4>] acpi_ps_parse_loop+0x514/0x560 [ 0.881784] [<ffffffff8132b738>] acpi_ps_parse_aml+0x98/0x28c [ 0.887601] [<ffffffff8132bf8d>] acpi_ps_execute_method+0x1c1/0x26c [ 0.893939] [<ffffffff813269c5>] acpi_ns_evaluate+0x1c1/0x258 [ 0.899755] [<ffffffff8131cb98>] acpi_ev_execute_reg_method+0xca/0x112 [ 0.906353] [<ffffffff8131cd6e>] acpi_ev_reg_run+0x48/0x52 [ 0.911910] [<ffffffff81328fad>] acpi_ns_walk_namespace+0xc8/0x17f [ 0.918160] [<ffffffff8131cd26>] ? acpi_ev_detach_region+0x146/0x146 [ 0.924585] [<ffffffff8131cdbc>] acpi_ev_execute_reg_methods+0x44/0xf7 [ 0.931184] [<ffffffff819b2324>] ? acpi_sleep_proc_init+0x2a/0x2a [ 0.937349] [<ffffffff8130ac66>] ? acpi_os_wait_semaphore+0x43/0x57 [ 0.943686] [<ffffffff81331a3f>] ? acpi_ut_acquire_mutex+0x48/0x88 [ 0.949938] [<ffffffff8131ceb8>] acpi_ev_initialize_op_regions+0x49/0x71 [ 0.956709] [<ffffffff819b2324>] ? acpi_sleep_proc_init+0x2a/0x2a [ 0.962873] [<ffffffff81333310>] acpi_initialize_objects+0x23/0x4f [ 0.969125] [<ffffffff819b23b4>] acpi_init+0x90/0x268
Quoted 8 lines
So basically, this split page seems to be a problem. Some other code thinks that it has access to full page and goes ahead and registers that with PAT rb tree and this causes problems in
mmap()code.I suspect we might have to go back to idea of copying first and last non page aligned ranges in new kernel's memory and read it from there to solve this issue. Do you have other ideas?
Sorry for delayed response, although it looks like you have already found a way to fix this issue.
BTW, I previously found a part of makedumpfile that truncates the first and last pages if they are not aligned in page size. Discussing with Kumagai-san, the truncation is performed on some ia64 system and he found a valid data in the truncated area, and the latest makedumpfile no longer does such truncation.
The commit is:
commit f854b37adba223d5b4801accbedd17b447266d51
Author: Atsushi Kumagai <[email protected]> Date: Fri Jun 21 15:25:31 2013 +0900
[PATCH 2/2] Fix the handling of the pages correspond to border of PT_LOAD.
The pages correspond to border of PT_LOAD were removed as holes. For example, pfn:N showed below was removed but we know even odd region like [0x40ffda7000 - 0x40ffda8000] can include valid dates, so we shouldn't remove it as holes.
phys_start
= 0x40ffda7000
|<-- frac_head -->|------------- PT_LOAD -------------
----+-----------------------+---------------------+----
| pfn:N | pfn:N+1 | ...
----+-----------------------+---------------------+----
|
pfn_to_paddr(pfn:N) # page size = 16k
= 0x40ffda4000
This patch handles such odd regions correctly. Then read pfn:N and write it to disk, the ranges not covered by any PT_LOAD entries will be filled with 0.
The log on the web is:
http://lists.infradead.org/pipermail/kexec/2013-May/008875.html
So, without this change, you would not have seen this issue. The original reason why the code was implemented so might be the issues similar to here.
Next, I think it necessary to consider whether or not to revert the above commit or not since makedumpfile fails on some kind of system as you reported.
Signature · 2 lines
Thanks. HATAYAMA, Daisuke
On Thu, Nov 14, 2013 at 07:31:37PM +0900, HATAYAMA Daisuke wrote:
[..]
Quoted 5 lines
BTW, I previously found a part of makedumpfile that truncates the first and last pages if they are not aligned in page size. Discussing with Kumagai-san, the truncation is performed on some ia64 system and he found a valid data in the truncated area, and the latest makedumpfile no longer does such truncation.
I went through the mail thread link you posted below. Looks like bootloader had put command line in that area and that would be truncated if we exclude partial pages from vmcore.
I don't know about IA64, but on x86, I see that we are trimming partial pages before they are added to memblock. So any memory allocations after this should not happen in partial page area. Bootloader still might place some things in those partial pages I guess.
So do we care about that little bootloader data if it happens to be there. Dump mechanism works only after kdump service has been loaded. That means first kernel is up and running. That means any relevant data passed to us from bootloader has already been copied in kernel memory and kernel did not crash. So to me, we don't have a strong need to look at exactly how bootloader passed data looked originally. We can just look at kernel copy of associated data structures (bootparams, command line etc).
I think being able to mmap() vmcore is much more important. BTW, is it possible to change makedumpfile so that it uses mmap() for reading page aligned areas and falls back to read() interface for reading partial pages? (Though it is beginning to sound already complicatated to me).
Given the fact that hpa does not like fixing it in kernel. We are left with option of fixing it in following places.
- Drop partial pages in kexec-tools
- Drop partial pages in makeudmpfile.
- Read partial pages using read() interface in makedumpfile
- Modify /proc/vmcore to copy partial pages in second kernel's memory.
It is not clear to me that partial pages are really useful. So I want to avoid modifying /proc/vmcore to deal with partial pages and increase complexity.
So fixing makedumpfile (either option2 or option 3) seems least risky to me. In fact I would say let us keep it simple and truncate partial pages in makedumpfile to keep it simple. And look at option 3 once we have a strong use case for partial pages.
What do you think?
Thanks
Vivek
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Quoted 45 lines · Vivek Goyal
On Thu, Nov 14, 2013 at 07:31:37PM +0900, HATAYAMA Daisuke wrote:
[..]
Quoted 5 lines
BTW, I previously found a part of makedumpfile that truncates the first and last pages if they are not aligned in page size. Discussing with Kumagai-san, the truncation is performed on some ia64 system and he found a valid data in the truncated area, and the latest makedumpfile no longer does such truncation.
I went through the mail thread link you posted below. Looks like bootloader had put command line in that area and that would be truncated if we exclude partial pages from vmcore.
I don't know about IA64, but on x86, I see that we are trimming partial pages before they are added to memblock. So any memory allocations after this should not happen in partial page area. Bootloader still might place some things in those partial pages I guess.
So do we care about that little bootloader data if it happens to be there. Dump mechanism works only after kdump service has been loaded. That means first kernel is up and running. That means any relevant data passed to us from bootloader has already been copied in kernel memory and kernel did not crash. So to me, we don't have a strong need to look at exactly how bootloader passed data looked originally. We can just look at kernel copy of associated data structures (bootparams, command line etc).
I think being able to
mmap()vmcore is much more important. BTW, is it possible to change makedumpfile so that it usesmmap()for reading page aligned areas and falls back toread()interface for reading partial pages? (Though it is beginning to sound already complicatated to me).Given the fact that hpa does not like fixing it in kernel. We are left with option of fixing it in following places.
- Drop partial pages in kexec-tools
- Drop partial pages in makeudmpfile.
- Read partial pages usingread()interface in makedumpfile
- Modify/proc/vmcoreto copy partial pages in second kernel's memory.It is not clear to me that partial pages are really useful. So I want to avoid modifying
/proc/vmcoreto deal with partial pages and increase complexity.
Quoted 7 lines
So fixing makedumpfile (either option2 or option 3) seems least risky to me. In fact I would say let us keep it simple and truncate partial pages in makedumpfile to keep it simple. And look at option 3 once we have a strong use case for partial pages.
What do you think?
As you say, it's not clear that partial pages are really useful, but on the other hand, it seems to me not clear that they are really useless. I think we should get them as long as we have access to them.
It seems best to me the option 3). Switching between read and mmap would be not so complex and also it's by far flexible in makedumpfile than in kernel.
Also, I think it better for /proc/vmcore to disable mmap on partial pages in order to avoid the issue here.
Signature · 2 lines
Thanks. HATAYAMA, Daisuke
On Fri, Nov 15, 2013 at 06:41:52PM +0900, HATAYAMA Daisuke wrote:
[..]
Quoted 20 lines
Quoted 12 lines
Given the fact that hpa does not like fixing it in kernel. We are left with option of fixing it in following places.
- Drop partial pages in kexec-tools
- Drop partial pages in makeudmpfile.
- Read partial pages usingread()interface in makedumpfile
- Modify/proc/vmcoreto copy partial pages in second kernel's memory.It is not clear to me that partial pages are really useful. So I want to avoid modifying
/proc/vmcoreto deal with partial pages and increase complexity.Quoted 7 lines
So fixing makedumpfile (either option2 or option 3) seems least risky to me. In fact I would say let us keep it simple and truncate partial pages in makedumpfile to keep it simple. And look at option 3 once we have a strong use case for partial pages.
What do you think?
Quoted 7 lines
As you say, it's not clear that partial pages are really useful, but on the other hand, it seems to me not clear that they are really useless. I think we should get them as long as we have access to them.
It seems best to me the option 3). Switching between read and mmap would be not so complex and also it's by far flexible in makedumpfile than in kernel.
Ok, I am fine with option 3. It is more complicated option but safe option.
Is there any chance that you could look into fixing this. I have no experience writing code for makedumpfile.
Thanks
Vivek
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Quoted 27 lines · Vivek Goyal
On Fri, Nov 15, 2013 at 06:41:52PM +0900, HATAYAMA Daisuke wrote:
[..]
Quoted 24 lines
Quoted 12 lines
Given the fact that hpa does not like fixing it in kernel. We are left with option of fixing it in following places.
- Drop partial pages in kexec-tools
- Drop partial pages in makeudmpfile.
- Read partial pages usingread()interface in makedumpfile
- Modify/proc/vmcoreto copy partial pages in second kernel's memory.It is not clear to me that partial pages are really useful. So I want to avoid modifying
/proc/vmcoreto deal with partial pages and increase complexity.Quoted 7 lines
So fixing makedumpfile (either option2 or option 3) seems least risky to me. In fact I would say let us keep it simple and truncate partial pages in makedumpfile to keep it simple. And look at option 3 once we have a strong use case for partial pages.
What do you think?
As you say, it's not clear that partial pages are really useful, but on the other hand, it seems to me not clear that they are really useless. I think we should get them as long as we have access to them.
Quoted 6 lines
Quoted 3 lines
It seems best to me the option 3). Switching between read and mmap would be not so complex and also it's by far flexible in makedumpfile than in kernel.
Ok, I am fine with option 3. It is more complicated option but safe option.
It sounds reasonable also to me.
Quoted 2 lines
Is there any chance that you could look into fixing this. I have no experience writing code for makedumpfile.
I'll send a patch to fix this soon.
Thanks
Atsushi Kumagai
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
On Mon, Nov 18, 2013 at 12:51:39AM +0000, Atsushi Kumagai wrote:
[..]
Quoted 4 lines
Quoted 2 lines
Is there any chance that you could look into fixing this. I have no experience writing code for makedumpfile.
I'll send a patch to fix this soon.
Thanks Atsushi.
Vivek
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Quoted 35 lines · Atsushi Kumagai
Quoted 27 lines · Vivek Goyal
On Fri, Nov 15, 2013 at 06:41:52PM +0900, HATAYAMA Daisuke wrote:
[..]
Quoted 24 lines
Quoted 12 lines
Given the fact that hpa does not like fixing it in kernel. We are left with option of fixing it in following places.
- Drop partial pages in kexec-tools
- Drop partial pages in makeudmpfile.
- Read partial pages usingread()interface in makedumpfile
- Modify/proc/vmcoreto copy partial pages in second kernel's memory.It is not clear to me that partial pages are really useful. So I want to avoid modifying
/proc/vmcoreto deal with partial pages and increase complexity.Quoted 7 lines
So fixing makedumpfile (either option2 or option 3) seems least risky to me. In fact I would say let us keep it simple and truncate partial pages in makedumpfile to keep it simple. And look at option 3 once we have a strong use case for partial pages.
What do you think?
As you say, it's not clear that partial pages are really useful, but on the other hand, it seems to me not clear that they are really useless. I think we should get them as long as we have access to them.
Quoted 6 lines
Quoted 3 lines
It seems best to me the option 3). Switching between read and mmap would be not so complex and also it's by far flexible in makedumpfile than in kernel.
Ok, I am fine with option 3. It is more complicated option but safe option.
Quoted 7 lines
It sounds reasonable also to me.
Quoted 2 lines
Is there any chance that you could look into fixing this. I have no experience writing code for makedumpfile.
I'll send a patch to fix this soon.
Thanks.
BTW, now the following patch has been applied on top of makedumpfile in kexec-tools package on fedora in order to avoid the issue.
https://lists.fedoraproject.org/pipermail/kexec/2013-November/000254.html
I remember prototype version of mmap patch implemented a kind of --no-mmap option and we could use it to disable mmap() use and use read() instead, I think which is useful when we face this kind of issue.
Signature · 2 lines
Thanks. HATAYAMA, Daisuke
Quoted 44 lines · kexec
Quoted 35 lines · Atsushi Kumagai
Quoted 27 lines · Vivek Goyal
On Fri, Nov 15, 2013 at 06:41:52PM +0900, HATAYAMA Daisuke wrote:
[..]
Quoted 24 lines
Quoted 12 lines
Given the fact that hpa does not like fixing it in kernel. We are left with option of fixing it in following places.
- Drop partial pages in kexec-tools
- Drop partial pages in makeudmpfile.
- Read partial pages usingread()interface in makedumpfile
- Modify/proc/vmcoreto copy partial pages in second kernel's memory.It is not clear to me that partial pages are really useful. So I want to avoid modifying
/proc/vmcoreto deal with partial pages and increase complexity.Quoted 7 lines
So fixing makedumpfile (either option2 or option 3) seems least risky to me. In fact I would say let us keep it simple and truncate partial pages in makedumpfile to keep it simple. And look at option 3 once we have a strong use case for partial pages.
What do you think?
As you say, it's not clear that partial pages are really useful, but on the other hand, it seems to me not clear that they are really useless. I think we should get them as long as we have access to them.
Quoted 6 lines
Quoted 3 lines
It seems best to me the option 3). Switching between read and mmap would be not so complex and also it's by far flexible in makedumpfile than in kernel.
Ok, I am fine with option 3. It is more complicated option but safe option.
Quoted 7 lines
It sounds reasonable also to me.
Quoted 2 lines
Is there any chance that you could look into fixing this. I have no experience writing code for makedumpfile.
I'll send a patch to fix this soon.
Quoted 7 lines
Thanks.
BTW, now the following patch has been applied on top of makedumpfile in kexec-tools package on fedora in order to avoid the issue.
https://lists.fedoraproject.org/pipermail/kexec/2013-November/000254.html
I remember prototype version of mmap patch implemented a kind of --no-mmap option and we could use it to disable
mmap()use and useread()instead, I think which is useful when we face this kind of issue.
How about this fail back structure instead of such an extra option ?
Thanks
Atsushi Kumagai
From: Atsushi Kumagai <[email protected]>
Date: Wed, 20 Nov 2013 14:10:19 +0900
Subject: [PATCH] Fall back to read() when mmap() fails.
--- makedumpfile.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-)
makedumpfile.c +9 -1
@@ -324,7 +324,15 @@ read_from_vmcore(off_t offset, void *bufptr, unsigned long size)if (!read_with_mmap (offset, bufptr, size)) {ERRMSG ("Can't read the dump memory(%s) with mmap().\n" , info->name_memory); -return FALSE; + +ERRMSG ("This kernel might have some problems about mmap().\n" ); +ERRMSG ("read() will be used instead of mmap() from now.\n" ); + +/* +* Fall back to read(). +*/ + info->flag_usemmap = FALSE; +read_from_vmcore (offset, bufptr, size); } }else {if (lseek (info->fd_memory, offset,SEEK_SET ) == failed) {
Signature · 6 lines
1.8.0.2 _______________________________________________ kexec mailing list [email protected] http://lists.infradead.org/mailman/listinfo/kexec
Quoted 11 lines · kexec
On Mon, Nov 18, 2013 at 12:51:39AM +0000, Atsushi Kumagai wrote:
[..]
Quoted 4 lines
Quoted 2 lines
Is there any chance that you could look into fixing this. I have no experience writing code for makedumpfile.
I'll send a patch to fix this soon.
Thanks Atsushi.
Vivek
Vivek, could you test this patch ?
Thanks
Atsushi Kumagai
From: Atsushi Kumagai <[email protected]>
Date: Wed, 20 Nov 2013 10:05:03 +0900
Subject: [PATCH] Disable mmap() for reading fractional pages.
Since mmap() was introduced on /proc/vmcore, it fails for fractional pages which don't start or end at page boundary due to kernel issue.
This patch disables mmap() temporarily for fractional pages to avoid this issue, so mmap() will be used only for aligned pages.
--- makedumpfile.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-)
makedumpfile.c +10 -2
@@ -368,6 +368,7 @@ readpage_elf(unsigned long long paddr, void *bufptr)off_t offset1, offset2;size_t size1, size2;unsigned long long phys_start, phys_end, frac_head =0 ; +int original_usemmap = info->flag_usemmap; offset1 =paddr_to_offset (paddr); offset2 =paddr_to_offset (paddr + info->page_size);
@@ -392,6 +393,7 @@ readpage_elf(unsigned long long paddr, void *bufptr) offset1 =paddr_to_offset (phys_start); frac_head = phys_start - paddr;memset (bufptr,0 , frac_head); + info->flag_usemmap = FALSE; }/*
@@ -402,6 +404,7 @@ readpage_elf(unsigned long long paddr, void *bufptr) phys_end =page_head_to_phys_end (paddr); offset2 =paddr_to_offset (phys_end);memset (bufptr + (phys_end - paddr),0 , info->page_size - (phys_end - paddr)); + info->flag_usemmap = FALSE; }/*
@@ -420,7 +423,7 @@ readpage_elf(unsigned long long paddr, void *bufptr)if (!read_from_vmcore (offset1, bufptr + frac_head, size1)) {ERRMSG ("Can't read the dump memory(%s).\n" , info->name_memory); -return FALSE; + goto error; }if (size1 + frac_head != info->page_size) {
@@ -429,11 +432,16 @@ readpage_elf(unsigned long long paddr, void *bufptr)if (!read_from_vmcore (offset2, bufptr + frac_head + size1, size2)) {ERRMSG ("Can't read the dump memory(%s).\n" , info->name_memory); -return FALSE; + goto error; } } + info->flag_usemmap = original_usemmap;return TRUE; + +error: + info->flag_usemmap = original_usemmap; +return FALSE; } static int
Signature · 6 lines
1.8.0.2 _______________________________________________ kexec mailing list [email protected] http://lists.infradead.org/mailman/listinfo/kexec
Quoted 47 lines · Atsushi Kumagai
Quoted 46 lines · kexec
Quoted 35 lines · Atsushi Kumagai
Quoted 27 lines · Vivek Goyal
On Fri, Nov 15, 2013 at 06:41:52PM +0900, HATAYAMA Daisuke wrote:
[..]
Quoted 24 lines
Quoted 12 lines
Given the fact that hpa does not like fixing it in kernel. We are left with option of fixing it in following places.
- Drop partial pages in kexec-tools
- Drop partial pages in makeudmpfile.
- Read partial pages usingread()interface in makedumpfile
- Modify/proc/vmcoreto copy partial pages in second kernel's memory.It is not clear to me that partial pages are really useful. So I want to avoid modifying
/proc/vmcoreto deal with partial pages and increase complexity.Quoted 7 lines
So fixing makedumpfile (either option2 or option 3) seems least risky to me. In fact I would say let us keep it simple and truncate partial pages in makedumpfile to keep it simple. And look at option 3 once we have a strong use case for partial pages.
What do you think?
As you say, it's not clear that partial pages are really useful, but on the other hand, it seems to me not clear that they are really useless. I think we should get them as long as we have access to them.
Quoted 6 lines
Quoted 3 lines
It seems best to me the option 3). Switching between read and mmap would be not so complex and also it's by far flexible in makedumpfile than in kernel.
Ok, I am fine with option 3. It is more complicated option but safe option.
Quoted 7 lines
It sounds reasonable also to me.
Quoted 2 lines
Is there any chance that you could look into fixing this. I have no experience writing code for makedumpfile.
I'll send a patch to fix this soon.
Thanks.
Quoted 8 lines
Quoted 5 lines
BTW, now the following patch has been applied on top of makedumpfile in kexec-tools package on fedora in order to avoid the issue.
https://lists.fedoraproject.org/pipermail/kexec/2013-November/000254.html
I remember prototype version of mmap patch implemented a kind of --no-mmap option and we could use it to disable
mmap()use and useread()instead, I think which is useful when we face this kind of issue.How about this fail back structure instead of such an extra option ?
I think this logic is useful and should be merged together in this fix.
However, I still think a kind of --no-mmap option is needed. There could happen worse case due to mmap() in the future on some system, of course, I don't know what the system actually is, but at least it must be behaving differently from typical systems... Then, option is more flexible than patching.
It would also be useful for debugging use. read() is simpler than mmap(), and read() is basic in the sense that initially makedumpfile didn't use mmap(). There might be a situation where we want to avoid using mmap(); for example, when makedumpfile works badly and it looks like caused by mmap() code in kernel code; Then, we would want to see if makedumpfile works well by disabling mmap(),
Signature · 2 lines
Thanks. HATAYAMA, Daisuke
Quoted 23 lines · Atsushi Kumagai
Quoted 11 lines · kexec
On Mon, Nov 18, 2013 at 12:51:39AM +0000, Atsushi Kumagai wrote:
[..]
Quoted 4 lines
Quoted 2 lines
Is there any chance that you could look into fixing this. I have no experience writing code for makedumpfile.
I'll send a patch to fix this soon.
Thanks Atsushi.
Vivek
Vivek, could you test this patch ?
Thanks
Atsushi KumagaiFrom: Atsushi Kumagai <[email protected]>
Date: Wed, 20 Nov 2013 10:05:03 +0900
Subject: [PATCH] Disablemmap()for reading fractional pages.
Quoted 7 lines
Since
mmap()was introduced on/proc/vmcore, it fails for fractional pages which don't start or end at page boundary due to kernel issue.
This patch disablesmmap()temporarily for fractional pages to avoid this issue, sommap()will be used only for aligned pages.Signed-off-by Atsushi Kumagai <[email protected]>
Hi Atsushi,
Even with this patch applied I see mmap() failure.
mem_map (39)
mem_map : ffffea0004e00000 pfn_start : 138000 pfn_end : 140000
read /proc/vmcore with mmap()
Excluding unnecessary pages : [100.0 %] |STEP [Excluding
unnecessary pages] : 0.035925 seconds
Excluding unnecessary pages : [100.0 %] \STEP [Excluding
unnecessary pages] : 0.035774 seconds
Excluding unnecessary pages : [100.0 %] -STEP [Excluding
unnecessary pages] : 0.035229 seconds
Copying data : [ 40.9 %] -Can't map
[b98fd000-b9cfd000] with mmap()read_from_vmcore: Can't read the dump memory(/proc/vmcore) with mmap().readpage_elf: Can't read the dump memory(/proc/vmcore).
readmem: type_addr: 1, addr:bffba000, size:4096read_pfn: Can't get the page data.
Resource temporarily unavailable
makedumpfile Failed.
kdump: saving vmcore failed
Following is part of /proc/iomem on my system.
00100000-bffc283f : System RAM
01000000-018c551d : Kernel code
018c551e-01ef3f3f : Kernel data
0204a000-02984fff : Kernel bss
2e000000-35ffffff : Crash kernel
bffc2840-bfffffff : reserved
This is a different system than what I used last time. So I am not sure if this is same error or something else. But one thing is clear that System RAM last page is partial and we should face mmap() failure.
Thanks
Vivek
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Hello Vivek,
Quoted 53 lines · kexec
Quoted 19 lines
Quoted 8 lines
Quoted 4 lines
Quoted 2 lines
Is there any chance that you could look into fixing this. I have no experience writing code for makedumpfile.
I'll send a patch to fix this soon.
Thanks Atsushi.
Vivek
Vivek, could you test this patch ?
Thanks
Atsushi KumagaiFrom: Atsushi Kumagai <[email protected]>
Date: Wed, 20 Nov 2013 10:05:03 +0900
Subject: [PATCH] Disablemmap()for reading fractional pages.Quoted 6 lines
Since
mmap()was introduced on/proc/vmcore, it fails for fractional pages which don't start or end at page boundary due to kernel issue. This patch disablesmmap()temporarily for fractional pages to avoid this issue, sommap()will be used only for aligned pages.Signed-off-by Atsushi Kumagai <[email protected]>Hi Atsushi,
Even with this patch applied I see
mmap()failure.
mem_map(39)mem_map : ffffea0004e00000 pfn_start : 138000 pfn_end : 140000read
/proc/vmcorewithmmap()
Excluding unnecessary pages : [100.0 %] |STEP [Excluding
unnecessary pages] : 0.035925 seconds
Excluding unnecessary pages : [100.0 %] \STEP [Excluding
unnecessary pages] : 0.035774 seconds
Excluding unnecessary pages : [100.0 %] -STEP [Excluding
unnecessary pages] : 0.035229 seconds
Copying data : [ 40.9 %] -Can't map
[b98fd000-b9cfd000] withmmap()read_from_vmcore: Can't read the dump memory(/proc/vmcore) withmmap().readpage_elf: Can't read the dump memory(/proc/vmcore).
readmem:type_addr: 1, addr:bffba000, size:4096read_pfn: Can't get the page data.
Resource temporarily unavailable
makedumpfile Failed.
kdump: saving vmcore failedFollowing is part of
/proc/iomemon my system.
Quoted 8 lines
00100000-bffc283f : System RAM
01000000-018c551d : Kernel code
018c551e-01ef3f3f : Kernel data
0204a000-02984fff : Kernel bss
2e000000-35ffffff : Crash kernel
bffc2840-bfffffff : reservedThis is a different system than what I used last time. So I am not sure if this is same error or something else. But one thing is clear that System RAM last page is partial and we should face
mmap()failure.
Thanks for your testing, I've found my mistake.
My patch tries to disable mmap() when a partial page is found, but actually mmap() has already been called because update_mmap_range() calls mmap() for every 4MB region in advance.
If we try to keep using mmap() as much as possible, update_mmap_range() has to check whether the target region of mmap() includes the partial pages before calling mmap(), but it's too tough as workaround.
So I think the patch I sent is enough, the policy will be simpler as "Don't use mmap() for buggy kernels".
[PATCH] Fall back to read() when mmap() fails.
http://lists.infradead.org/pipermail/kexec/2013-November/010199.html
Thanks
Atsushi Kumagai
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Quoted 75 lines · Atsushi Kumagai
Quoted 44 lines · kexec
Quoted 35 lines · Atsushi Kumagai
Quoted 27 lines · Vivek Goyal
On Fri, Nov 15, 2013 at 06:41:52PM +0900, HATAYAMA Daisuke wrote:
[..]
Quoted 24 lines
Quoted 12 lines
Given the fact that hpa does not like fixing it in kernel. We are left with option of fixing it in following places.
- Drop partial pages in kexec-tools
- Drop partial pages in makeudmpfile.
- Read partial pages usingread()interface in makedumpfile
- Modify/proc/vmcoreto copy partial pages in second kernel's memory.It is not clear to me that partial pages are really useful. So I want to avoid modifying
/proc/vmcoreto deal with partial pages and increase complexity.Quoted 7 lines
So fixing makedumpfile (either option2 or option 3) seems least risky to me. In fact I would say let us keep it simple and truncate partial pages in makedumpfile to keep it simple. And look at option 3 once we have a strong use case for partial pages.
What do you think?
As you say, it's not clear that partial pages are really useful, but on the other hand, it seems to me not clear that they are really useless. I think we should get them as long as we have access to them.
Quoted 6 lines
Quoted 3 lines
It seems best to me the option 3). Switching between read and mmap would be not so complex and also it's by far flexible in makedumpfile than in kernel.
Ok, I am fine with option 3. It is more complicated option but safe option.
Quoted 7 lines
It sounds reasonable also to me.
Quoted 2 lines
Is there any chance that you could look into fixing this. I have no experience writing code for makedumpfile.
I'll send a patch to fix this soon.
Quoted 7 lines
Thanks.
BTW, now the following patch has been applied on top of makedumpfile in kexec-tools package on fedora in order to avoid the issue.
https://lists.fedoraproject.org/pipermail/kexec/2013-November/000254.html
I remember prototype version of mmap patch implemented a kind of --no-mmap option and we could use it to disable
mmap()use and useread()instead, I think which is useful when we face this kind of issue.How about this fail back structure instead of such an extra option ?
Thanks
Atsushi KumagaiFrom: Atsushi Kumagai <[email protected]>
Date: Wed, 20 Nov 2013 14:10:19 +0900
Subject: [PATCH] Fall back toread()whenmmap()fails.Signed-off-by Atsushi Kumagai <[email protected]>--- makedumpfile.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-)makedumpfile.c +0
-0@@ -324,7 +324,15 @@ read_from_vmcore(off_t offset, void *bufptr, unsigned long size)if (!read_with_mmap (offset, bufptr, size)) {ERRMSG ("Can't read the dump memory(%s) with mmap().\n" , info->name_memory);
makedumpfile.c read_from_vmcore(off_t offset, void *bufptr, unsigned long size) · line 335
Quoted 10 lines
makedumpfile.c +9
-1-return FALSE; + +ERRMSG ("This kernel might have some problems about mmap().\n" ); +ERRMSG ("read() will be used instead of mmap() from now.\n" ); + +/* +* Fall back to read(). +*/ + info->flag_usemmap = FALSE; +read_from_vmcore (offset, bufptr, size);
Hi, Atsushi
I've got such a workstation too. And I confirm this patch works for me.
However, I have a question:
Why not switch to mmap() back after read()?
Thanks
WANG Chao
Quoted 5 lines
makedumpfile.c +0
-0} }else {if (lseek (info->fd_memory, offset,SEEK_SET ) == failed) {Signature · 1 line
1.8.0.2
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Quoted 74 lines · Atsushi Kumagai
Hello Vivek,
Quoted 53 lines · kexec
Quoted 19 lines
Quoted 8 lines
Quoted 4 lines
Quoted 2 lines
Is there any chance that you could look into fixing this. I have no experience writing code for makedumpfile.
I'll send a patch to fix this soon.
Thanks Atsushi.
Vivek
Vivek, could you test this patch ?
Thanks
Atsushi KumagaiFrom: Atsushi Kumagai <[email protected]>
Date: Wed, 20 Nov 2013 10:05:03 +0900
Subject: [PATCH] Disablemmap()for reading fractional pages.Quoted 6 lines
Since
mmap()was introduced on/proc/vmcore, it fails for fractional pages which don't start or end at page boundary due to kernel issue. This patch disablesmmap()temporarily for fractional pages to avoid this issue, sommap()will be used only for aligned pages.Signed-off-by Atsushi Kumagai <[email protected]>Hi Atsushi,
Even with this patch applied I see
mmap()failure.
mem_map(39)mem_map : ffffea0004e00000 pfn_start : 138000 pfn_end : 140000read
/proc/vmcorewithmmap()
Excluding unnecessary pages : [100.0 %] |STEP [Excluding
unnecessary pages] : 0.035925 seconds
Excluding unnecessary pages : [100.0 %] \STEP [Excluding
unnecessary pages] : 0.035774 seconds
Excluding unnecessary pages : [100.0 %] -STEP [Excluding
unnecessary pages] : 0.035229 seconds
Copying data : [ 40.9 %] -Can't map
[b98fd000-b9cfd000] withmmap()read_from_vmcore: Can't read the dump memory(/proc/vmcore) withmmap().readpage_elf: Can't read the dump memory(/proc/vmcore).
readmem:type_addr: 1, addr:bffba000, size:4096read_pfn: Can't get the page data.
Resource temporarily unavailable
makedumpfile Failed.
kdump: saving vmcore failedFollowing is part of
/proc/iomemon my system.Quoted 8 lines
00100000-bffc283f : System RAM
01000000-018c551d : Kernel code
018c551e-01ef3f3f : Kernel data
0204a000-02984fff : Kernel bss
2e000000-35ffffff : Crash kernel
bffc2840-bfffffff : reservedThis is a different system than what I used last time. So I am not sure if this is same error or something else. But one thing is clear that System RAM last page is partial and we should face
mmap()failure.Thanks for your testing, I've found my mistake.
My patch tries to disable
mmap()when a partial page is found, but actuallymmap()has already been called becauseupdate_mmap_range()callsmmap()for every 4MB region in advance.
If we try to keep usingmmap()as much as possible,update_mmap_range()has to check whether the target region ofmmap()includes the partial pages before callingmmap(), but it's too tough as workaround.
Quoted 6 lines
So I think the patch I sent is enough, the policy will be simpler as "Don't use
mmap()for buggy kernels".[PATCH] Fall back to
read()whenmmap()fails.
http://lists.infradead.org/pipermail/kexec/2013-November/010199.html
I think logic becomes not so complex. For example, if input vmcore format is ELF, then:
o in update_mmap_range():
- first calculate a range of the corresponding PT_LOAD entry truncated with PAGE_SIZE.
- Then, truncate range of mmap() by the truncated range of the corresponding PT_LOAD entry, i.e., exlucde partial pages from mmap() target range.
- Then determine offsets of two partial pages; the number of partial pages are always at most two. The offsets can easily be calculated from the original range of the corresponding PT_LOAD entry
o in read_from_vmcore(), if a given offset belongs to either of two partial pages, then go to read() path; if not, go to mmap() path.
Signature · 2 lines
Thanks. HATAYAMA, Daisuke
On Thu, Nov 21, 2013 at 05:31:46PM +0900, HATAYAMA Daisuke wrote:
[..]
Quoted 19 lines
Quoted 6 lines
So I think the patch I sent is enough, the policy will be simpler as "Don't use
mmap()for buggy kernels".[PATCH] Fall back to
read()whenmmap()fails.
http://lists.infradead.org/pipermail/kexec/2013-November/010199.htmlI think logic becomes not so complex. For example, if input vmcore format is ELF, then:
o in
update_mmap_range():
- first calculate a range of the correspondingPT_LOADentry truncated withPAGE_SIZE.
- Then, truncate range ofmmap()by the truncated range of the correspondingPT_LOADentry, i.e., exlucde partial pages frommmap()target range.
- Then determine offsets of two partial pages; the number of partial pages are always at most two. The offsets can easily be calculated from the original range of the correspondingPT_LOADentry
Quoted 2 lines
o in
read_from_vmcore(), if a given offset belongs to either of two partial pages, then go toread()path; if not, go tommap()path.
I agree that we should do mmap() on all non-partial pages and do read() on all partial pages. Otherwise we lose the benefit of faster speed of mmap().
Thanks
Vivek
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Hello WANG,
Quoted 35 lines · kexec
Quoted 22 lines
How about this fail back structure instead of such an extra option ?
Thanks
Atsushi KumagaiFrom: Atsushi Kumagai <[email protected]>
Date: Wed, 20 Nov 2013 14:10:19 +0900
Subject: [PATCH] Fall back toread()whenmmap()fails.Signed-off-by Atsushi Kumagai <[email protected]>--- makedumpfile.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-)makedumpfile.c +0
-0@@ -324,7 +324,15 @@ read_from_vmcore(off_t offset, void *bufptr, unsigned long size)if (!read_with_mmap (offset, bufptr, size)) {ERRMSG ("Can't read the dump memory(%s) with mmap().\n" , info->name_memory);makedumpfile.c read_from_vmcore(off_t offset, void *bufptr, unsigned long size) · line 335
Quoted 10 lines
makedumpfile.c +9
-1-return FALSE; + +ERRMSG ("This kernel might have some problems about mmap().\n" ); +ERRMSG ("read() will be used instead of mmap() from now.\n" ); + +/* +* Fall back to read(). +*/ + info->flag_usemmap = FALSE; +read_from_vmcore (offset, bufptr, size);Hi, Atsushi
Quoted 1 line
I've got such a workstation too. And I confirm this patch works for me.
Thanks for your testing !
Quoted 2 lines
However, I have a question:
Why not switch tommap()back afterread()?
I made this patch as a general safety net, not only for the partial page issue.
When facing unknown issues related mmap(), the kernel may have some bugs and mmap() can fail for every pages. In the worst case, most all mmap() will fail and try read() with error messages after every fail, but this patch will prevent the chattering of the switch and so many error messages.
Thanks
Atsushi Kumagai
Quoted 9 lines
Thanks
WANG ChaoQuoted 5 lines
makedumpfile.c +0
-0} }else {if (lseek (info->fd_memory, offset,SEEK_SET ) == failed) {Signature · 1 line
1.8.0.2
Quoted 5 lines
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Quoted 22 lines · kexec
On Thu, Nov 21, 2013 at 05:31:46PM +0900, HATAYAMA Daisuke wrote:
[..]
Quoted 19 lines
Quoted 6 lines
So I think the patch I sent is enough, the policy will be simpler as "Don't use
mmap()for buggy kernels".[PATCH] Fall back to
read()whenmmap()fails.
http://lists.infradead.org/pipermail/kexec/2013-November/010199.htmlI think logic becomes not so complex. For example, if input vmcore format is ELF, then:
o in
update_mmap_range():
- first calculate a range of the correspondingPT_LOADentry truncated withPAGE_SIZE.
- Then, truncate range ofmmap()by the truncated range of the correspondingPT_LOADentry, i.e., exlucde partial pages frommmap()target range.
- Then determine offsets of two partial pages; the number of partial pages are always at most two. The offsets can easily be calculated from the original range of the correspondingPT_LOADentry
Quoted 6 lines
Quoted 2 lines
o in
read_from_vmcore(), if a given offset belongs to either of two partial pages, then go toread()path; if not, go tommap()path.I agree that we should do
mmap()on all non-partial pages and doread()on all partial pages. Otherwise we lose the benefit of faster speed ofmmap().
I agree to avoid this issue by fixing makedumpfile as workaround while to fix kernel is so tough and risky. However, it sounds strange to me to fix userspace side elaborately for such definite kernel issue whose cause is known, so we should fix the kernel itself.
Otherwise, will you continue to add specific fixes into user tools to address kernel issues like this case ?
Thanks
Atsushi Kumagai
Quoted 8 lines
Thanks
Vivek_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Quoted 30 lines · Atsushi Kumagai
Quoted 22 lines · kexec
On Thu, Nov 21, 2013 at 05:31:46PM +0900, HATAYAMA Daisuke wrote:
[..]
Quoted 19 lines
Quoted 6 lines
So I think the patch I sent is enough, the policy will be simpler as "Don't use
mmap()for buggy kernels".[PATCH] Fall back to
read()whenmmap()fails.
http://lists.infradead.org/pipermail/kexec/2013-November/010199.htmlI think logic becomes not so complex. For example, if input vmcore format is ELF, then:
o in
update_mmap_range():
- first calculate a range of the correspondingPT_LOADentry truncated withPAGE_SIZE.
- Then, truncate range ofmmap()by the truncated range of the correspondingPT_LOADentry, i.e., exlucde partial pages frommmap()target range.
- Then determine offsets of two partial pages; the number of partial pages are always at most two. The offsets can easily be calculated from the original range of the correspondingPT_LOADentryQuoted 6 lines
Quoted 2 lines
o in
read_from_vmcore(), if a given offset belongs to either of two partial pages, then go toread()path; if not, go tommap()path.I agree that we should do
mmap()on all non-partial pages and doread()on all partial pages. Otherwise we lose the benefit of faster speed ofmmap().
Quoted 9 lines
I agree to avoid this issue by fixing makedumpfile as workaround while to fix kernel is so tough and risky. However, it sounds strange to me to fix userspace side elaborately for such definite kernel issue whose cause is known, so we should fix the kernel itself.
Otherwise, will you continue to add specific fixes into user tools to address kernel issues like this case ?
makedumpfile supports a wide range of kernel versions and needs to satisfy backward compatibility. mmap() on /proc/vmcore might be backported to some of the old versions on some distributions if necessary. Then, it's hard to fix each old kernel at each back port. The method that can be applied to all the kernels in general, is necessary.
Also, looking at ia64 case where there's boot loader data on partial pages, there could be other environments where partial pages contain other important data other components have. So, the issue depends not only on kernels but also other components such as boot loader and firmwares that can put data on partial pages. We need to get there as long as there's important data there and we have access to there.
Signature · 2 lines
Thanks. HATAYAMA, Daisuke
On Mon, Nov 25, 2013 at 06:01:37PM +0900, HATAYAMA Daisuke wrote:
[..]
Quoted 16 lines
Quoted 9 lines
I agree to avoid this issue by fixing makedumpfile as workaround while to fix kernel is so tough and risky. However, it sounds strange to me to fix userspace side elaborately for such definite kernel issue whose cause is known, so we should fix the kernel itself.
Otherwise, will you continue to add specific fixes into user tools to address kernel issues like this case ?
makedumpfile supports a wide range of kernel versions and needs to satisfy backward compatibility.
mmap()on/proc/vmcoremight be backported to some of the old versions on some distributions if necessary. Then, it's hard to fix each old kernel at each back port. The method that can be applied to all the kernels in general, is necessary.
Quoted 6 lines
Also, looking at ia64 case where there's boot loader data on partial pages, there could be other environments where partial pages contain other important data other components have. So, the issue depends not only on kernels but also other components such as boot loader and firmwares that can put data on partial pages. We need to get there as long as there's important data there and we have access to there.
Hi Atsushi, Hatayama,
So even if we fix the mmap() issue in kernel, looks like it will be a good idea to ship the fix in makedumpfile as there have been a kernel release where mmap() will cause issues.
Having said that, I think we need to fix it in kernel also. I was not sure that what's the right fix. Should we truncate partial pages or should we just copy partial pages from old memory to new kernel's memory and fill partial page with zeros. And that's why I was hoping that makedumpfile can fill the gap.
Copying partial pages to new memory seems like a safer approach. So may be we can take a fix in makeudmpfile and another in kernel.
Hatayama, I know that in the past your initial mmap() patches were copying partial pages to new kernel's memory. Would you like to resurrect that patch again?
Thanks
Vivek
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Quoted 26 lines · kexec
On Mon, Nov 25, 2013 at 06:01:37PM +0900, HATAYAMA Daisuke wrote:
[..]
Quoted 16 lines
Quoted 9 lines
I agree to avoid this issue by fixing makedumpfile as workaround while to fix kernel is so tough and risky. However, it sounds strange to me to fix userspace side elaborately for such definite kernel issue whose cause is known, so we should fix the kernel itself.
Otherwise, will you continue to add specific fixes into user tools to address kernel issues like this case ?
makedumpfile supports a wide range of kernel versions and needs to satisfy backward compatibility.
mmap()on/proc/vmcoremight be backported to some of the old versions on some distributions if necessary. Then, it's hard to fix each old kernel at each back port. The method that can be applied to all the kernels in general, is necessary.Quoted 6 lines
Also, looking at ia64 case where there's boot loader data on partial pages, there could be other environments where partial pages contain other important data other components have. So, the issue depends not only on kernels but also other components such as boot loader and firmwares that can put data on partial pages. We need to get there as long as there's important data there and we have access to there.
Quoted 5 lines
Hi Atsushi, Hatayama,
So even if we fix the
mmap()issue in kernel, looks like it will be a good idea to ship the fix in makedumpfile as there have been a kernel release wheremmap()will cause issues.
OK, I'll make a patch set for makedumpfile to address issues about mmap():
1. Fix the partial page issue
2. Introduce general fall back structure (I already posted)
http://lists.infradead.org/pipermail/kexec/2013-November/010199.html
3. Add --non-mmap option
Thanks
Atsushi Kumagai
Quoted 13 lines
Having said that, I think we need to fix it in kernel also. I was not sure that what's the right fix. Should we truncate partial pages or should we just copy partial pages from old memory to new kernel's memory and fill partial page with zeros. And that's why I was hoping that makedumpfile can fill the gap.
Copying partial pages to new memory seems like a safer approach. So may be we can take a fix in makeudmpfile and another in kernel.
Hatayama, I know that in the past your initial
mmap()patches were copying partial pages to new kernel's memory. Would you like to resurrect that patch again?
Quoted 8 lines
Thanks
Vivek_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Quoted 64 lines · kexec
Quoted 47 lines · Atsushi Kumagai
Quoted 46 lines · kexec
Quoted 35 lines · Atsushi Kumagai
Quoted 27 lines · Vivek Goyal
On Fri, Nov 15, 2013 at 06:41:52PM +0900, HATAYAMA Daisuke wrote:
[..]
Quoted 24 lines
Quoted 12 lines
Given the fact that hpa does not like fixing it in kernel. We are left with option of fixing it in following places.
- Drop partial pages in kexec-tools
- Drop partial pages in makeudmpfile.
- Read partial pages usingread()interface in makedumpfile
- Modify/proc/vmcoreto copy partial pages in second kernel's memory.It is not clear to me that partial pages are really useful. So I want to avoid modifying
/proc/vmcoreto deal with partial pages and increase complexity.Quoted 7 lines
So fixing makedumpfile (either option2 or option 3) seems least risky to me. In fact I would say let us keep it simple and truncate partial pages in makedumpfile to keep it simple. And look at option 3 once we have a strong use case for partial pages.
What do you think?
As you say, it's not clear that partial pages are really useful, but on the other hand, it seems to me not clear that they are really useless. I think we should get them as long as we have access to them.
Quoted 6 lines
Quoted 3 lines
It seems best to me the option 3). Switching between read and mmap would be not so complex and also it's by far flexible in makedumpfile than in kernel.
Ok, I am fine with option 3. It is more complicated option but safe option.
Quoted 7 lines
It sounds reasonable also to me.
Quoted 2 lines
Is there any chance that you could look into fixing this. I have no experience writing code for makedumpfile.
I'll send a patch to fix this soon.
Thanks.
Quoted 8 lines
Quoted 5 lines
BTW, now the following patch has been applied on top of makedumpfile in kexec-tools package on fedora in order to avoid the issue.
https://lists.fedoraproject.org/pipermail/kexec/2013-November/000254.html
I remember prototype version of mmap patch implemented a kind of --no-mmap option and we could use it to disable
mmap()use and useread()instead, I think which is useful when we face this kind of issue.How about this fail back structure instead of such an extra option ?
I think this logic is useful and should be merged together in this fix.
However, I still think a kind of --no-mmap option is needed. There could happen worse case due to
mmap()in the future on some system, of course, I don't know what the system actually is, but at least it must be behaving differently from typical systems... Then, option is more flexible than patching.
Quoted 5 lines
It would also be useful for debugging use.
read()is simpler thanmmap(), andread()is basic in the sense that initially makedumpfile didn't usemmap(). There might be a situation where we want to avoid usingmmap(); for example, when makedumpfile works badly and it looks like caused bymmap()code in kernel code; Then, we would want to see if makedumpfile works well by disablingmmap(),
Thanks for your explanation.
Additionally, the option to disable mmap() manually will help my test, so I should introduce the option into upstream.
Thanks
Atsushi Kumagai
Quoted 10 lines
Signature · 8 lines
Thanks. HATAYAMA, Daisuke _______________________________________________ kexec mailing list [email protected] http://lists.infradead.org/mailman/listinfo/kexec
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Quoted 45 lines · Atsushi Kumagai
Hello WANG,
Quoted 35 lines · kexec
Quoted 22 lines
How about this fail back structure instead of such an extra option ?
Thanks
Atsushi KumagaiFrom: Atsushi Kumagai <[email protected]>
Date: Wed, 20 Nov 2013 14:10:19 +0900
Subject: [PATCH] Fall back toread()whenmmap()fails.Signed-off-by Atsushi Kumagai <[email protected]>--- makedumpfile.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-)makedumpfile.c +0
-0@@ -324,7 +324,15 @@ read_from_vmcore(off_t offset, void *bufptr, unsigned long size)if (!read_with_mmap (offset, bufptr, size)) {ERRMSG ("Can't read the dump memory(%s) with mmap().\n" , info->name_memory);makedumpfile.c read_from_vmcore(off_t offset, void *bufptr, unsigned long size) · line 335
Quoted 10 lines
makedumpfile.c +9
-1-return FALSE; + +ERRMSG ("This kernel might have some problems about mmap().\n" ); +ERRMSG ("read() will be used instead of mmap() from now.\n" ); + +/* +* Fall back to read(). +*/ + info->flag_usemmap = FALSE; +read_from_vmcore (offset, bufptr, size);Hi, Atsushi
Quoted 1 line
I've got such a workstation too. And I confirm this patch works for me.
Thanks for your testing !
Quoted 2 lines
However, I have a question:
Why not switch tommap()back afterread()?
Quoted 6 lines
I made this patch as a general safety net, not only for the partial page issue.
When facing unknown issues relatedmmap(), the kernel may have some bugs andmmap()can fail for every pages. In the worst case, most allmmap()will fail and tryread()with error messages after every fail, but this patch will prevent the chattering of the switch and so many error messages.
Thanks for you explanation. I agree with you. Since mmap() is error prone after first mmap failure, use read() instead as a fail safe makes much sense to me.
WANG Chao
Quoted 19 lines
Thanks
Atsushi KumagaiQuoted 14 lines
Thanks
WANG ChaoQuoted 5 lines
makedumpfile.c +0
-0} }else {if (lseek (info->fd_memory, offset,SEEK_SET ) == failed) {Signature · 1 line
1.8.0.2_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Quoted 38 lines · Vivek Goyal
On Mon, Nov 25, 2013 at 06:01:37PM +0900, HATAYAMA Daisuke wrote:
[..]
Quoted 16 lines
Quoted 9 lines
I agree to avoid this issue by fixing makedumpfile as workaround while to fix kernel is so tough and risky. However, it sounds strange to me to fix userspace side elaborately for such definite kernel issue whose cause is known, so we should fix the kernel itself.
Otherwise, will you continue to add specific fixes into user tools to address kernel issues like this case ?
makedumpfile supports a wide range of kernel versions and needs to satisfy backward compatibility.
mmap()on/proc/vmcoremight be backported to some of the old versions on some distributions if necessary. Then, it's hard to fix each old kernel at each back port. The method that can be applied to all the kernels in general, is necessary.Quoted 6 lines
Also, looking at ia64 case where there's boot loader data on partial pages, there could be other environments where partial pages contain other important data other components have. So, the issue depends not only on kernels but also other components such as boot loader and firmwares that can put data on partial pages. We need to get there as long as there's important data there and we have access to there.
Hi Atsushi, Hatayama,
So even if we fix the
mmap()issue in kernel, looks like it will be a good idea to ship the fix in makedumpfile as there have been a kernel release wheremmap()will cause issues.Having said that, I think we need to fix it in kernel also. I was not sure that what's the right fix. Should we truncate partial pages or should we just copy partial pages from old memory to new kernel's memory and fill partial page with zeros. And that's why I was hoping that makedumpfile can fill the gap.
Quoted 7 lines
Copying partial pages to new memory seems like a safer approach. So may be we can take a fix in makeudmpfile and another in kernel.
Hatayama, I know that in the past your initial
mmap()patches were copying partial pages to new kernel's memory. Would you like to resurrect that patch again?
(Oh, I yesterday had totally forgotten partial page handling that was dropped from mmap() patch set in the middle of development...)
Yes, but wait until next week.
Signature · 2 lines
Thanks. HATAYAMA, Daisuke