Skip to contentSkip to conversation

min_low_pfn and max_low_pfn calcultion fix

PATCH In review Zou Nan hai 5 participants 14 messages Feb 27, 2007 – Mar 20, 2007

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 min_low_pfn and max_low_pfn calcultion fix

Hi,
We have seen bad_pte_print when testing crashdump on an SN machine in
recent 2.6.20 kernel.
There are tons of bad pte print (pfn < max_low_pfn) reports when the
crash kernel boots up, all those reported bad pages are inside initmem range;
That is because if the crash kernel code and data happens to be at the
beginning of the 1st node. build_node_maps in discontig.c will bypass reserved regions with filter_rsvd_memory. Since min_low_pfn is calculated in build_node_map, so in this case, min_low_pfn will be greater than kernel code and data.

Because pages inside initmem are freed and reused later, we saw
pfn_valid check fail on those pages.

I think this theoretically happen on a normal kernel. When I check
min_low_pfn and max_low_pfn calculation in contig.c and discontig.c. I found more issues than this.

1. min_low_pfn and max_low_pfn calculation is inconsistent between
contig.c and discontig.c,
min_low_pfn is calculated as the first page number of boot memmap in
contig.c (Why? Though this may work at the most of the time, I don't think it is the right logic). It is calculated as the lowest physical memory page number bypass reserved regions in discontig.c.
max_low_pfn is calculated include reserved regions in contig.c. It is
calculated exclude reserved regions in discontig.c.

2. If kernel code and data region is happen to be at the begin or the
end of physical memory, when min_low_pfn and max_low_pfn calculation is bypassed kernel code and data, pages in initmem will report bad.

3. initrd is also in reserved regions, if it is at the begin or at the
end of physical memory, kernel will refuse to reuse the memory. Because the virt_addr_valid check in free_initrd_mem.

So it is better to fix and clean up those issues.
Calculate min_low_pfn and max_low_pfn in a consistent way.

Below is the patch, please review and comments

arch/ia64/mm/contig.c +6 -24
diff -Nraup a/arch/ia64/mm/contig.c b/arch/ia64/mm/contig.c
@@ -75,26 +75,6 @@ show_mem (void)
 unsigned long bootmap_start;
 
 /**
- * find_max_pfn - adjust the maximum page number callback
- * @start: start of range
- * @end: end of range
- * @arg: address of pointer to global max_pfn variable
- *
- * Passed as a callback function to efi_memmap_walk() to determine the highest
- * available page frame number in the system.
- */
-int
-find_max_pfn (unsigned long start, unsigned long end, void *arg)
-{
-	unsigned long *max_pfnp = arg, pfn;
-
-	pfn = (PAGE_ALIGN(end - 1) - PAGE_OFFSET) >> PAGE_SHIFT;
-	if (pfn > *max_pfnp)
-		*max_pfnp = pfn;
-	return 0;
-}
-
-/**
  * find_bootmap_location - callback to find a memory area for the bootmap
  * @start: start of region
  * @end: end of region
@@ -155,9 +135,10 @@ find_memory (void)
 	reserve_memory();
 
 	/* first find highest page frame number */
-	max_pfn = 0;
-	efi_memmap_walk(find_max_pfn, &max_pfn);
-
+	min_low_pfn = -1;
+	max_low_pfn = 0;
+	efi_memmap_walk(find_max_min_low_pfn, NULL);
+	max_pfn = max_low_pfn;
 	/* how many bytes to cover all the pages */
 	bootmap_size = bootmem_bootmap_pages(max_pfn) << PAGE_SHIFT;
 
@@ -167,7 +148,8 @@ find_memory (void)
 	if (bootmap_start = ~0UL)
 		panic("Cannot find %ld bytes for bootmap\n", bootmap_size);
 
-	bootmap_size = init_bootmem(bootmap_start >> PAGE_SHIFT, max_pfn);
+	bootmap_size = init_bootmem_node(NODE_DATA(0), 
+			(bootmap_start >> PAGE_SHIFT), 0, max_pfn);
 
 	/* Free all available memory, then mark bootmem-map as being in use. */
 	efi_memmap_walk(filter_rsvd_memory, free_bootmem);
arch/ia64/mm/discontig.c +1 -3
diff -Nraup a/arch/ia64/mm/discontig.c b/arch/ia64/mm/discontig.c
@@ -86,9 +86,6 @@ static int __init build_node_maps(unsign
 		bdp->node_low_pfn = max(epfn, bdp->node_low_pfn);
 	}
 
-	min_low_pfn = min(min_low_pfn, bdp->node_boot_start>>PAGE_SHIFT);
-	max_low_pfn = max(max_low_pfn, bdp->node_low_pfn);
-
 	return 0;
 }
 
@@ -467,6 +464,7 @@ void __init find_memory(void)
 	/* These actually end up getting called by call_pernode_memory() */
 	efi_memmap_walk(filter_rsvd_memory, build_node_maps);
 	efi_memmap_walk(filter_rsvd_memory, find_pernode_space);
+	efi_memmap_walk(find_max_min_low_pfn, NULL);
 
 	for_each_online_node(node)
 		if (mem_data[node].bootmem_data.node_low_pfn) {
arch/ia64/mm/init.c +16 -0
diff -Nraup a/arch/ia64/mm/init.c b/arch/ia64/mm/init.c
@@ -616,6 +616,22 @@ count_reserved_pages (u64 start, u64 end
 	return 0;
 }
 
+int
+find_max_min_low_pfn (unsigned long start, unsigned long end, void *arg)
+{
+	unsigned long pfn_start, pfn_end;
+#if CONFIG_FLATMEM
+	pfn_start = (PAGE_ALIGN(__pa(start))) >> PAGE_SHIFT;
+	pfn_end = (PAGE_ALIGN(__pa(end - 1))) >> PAGE_SHIFT;
+#else
+	pfn_start = GRANULEROUNDDOWN(__pa(start)) >> PAGE_SHIFT;
+	pfn_end = GRANULEROUNDUP(__pa(end - 1)) >> PAGE_SHIFT;
+#endif
+	min_low_pfn = min(min_low_pfn, pfn_start);
+	max_low_pfn = max(max_low_pfn, pfn_end);
+	return 0;
+}
+
 /*
  * Boot command-line option "nolwsys" can be used to disable the use of any light-weight
  * system call handler.  When this option is in effect, all fsyscalls will end up bubbling
include/asm-ia64/meminit.h +1 -0
diff -Nraup a/include/asm-ia64/meminit.h b/include/asm-ia64/meminit.h
@@ -35,6 +35,7 @@ extern void reserve_memory (void);
 extern void find_initrd (void);
 extern int filter_rsvd_memory (unsigned long start, unsigned long end, void *arg);
 extern void efi_memmap_init(unsigned long *, unsigned long *);
+extern int find_max_min_low_pfn (unsigned long , unsigned long, void *);
 
 /*
  * For rounding an address to the next IA64_GRANULE_SIZE or order
Acked-by on v1 1/1
#1 lore raw

This fixed my problem on SN.

Quoted 166 lines · Zou Nan hai

Hi,
We have seen bad_pte_print when testing crashdump on an SN machine in
recent 2.6.20 kernel.
There are tons of bad pte print (pfn < max_low_pfn) reports when the
crash kernel boots up, all those reported bad pages are inside initmem range;
That is because if the crash kernel code and data happens to be at the
beginning of the 1st node. build_node_maps in discontig.c will bypass reserved regions with filter_rsvd_memory. Since min_low_pfn is calculated in build_node_map, so in this case, min_low_pfn will be greater than kernel code and data.

Because pages inside initmem are freed and reused later, we saw
pfn_valid check fail on those pages.

I think this theoretically happen on a normal kernel. When I check
min_low_pfn and max_low_pfn calculation in contig.c and discontig.c. I found more issues than this.

1. min_low_pfn and max_low_pfn calculation is inconsistent between
contig.c and discontig.c,
min_low_pfn is calculated as the first page number of boot memmap in
contig.c (Why? Though this may work at the most of the time, I don't think it is the right logic). It is calculated as the lowest physical memory page number bypass reserved regions in discontig.c.
max_low_pfn is calculated include reserved regions in contig.c. It is
calculated exclude reserved regions in discontig.c.

2. If kernel code and data region is happen to be at the begin or the
end of physical memory, when min_low_pfn and max_low_pfn calculation is bypassed kernel code and data, pages in initmem will report bad.

3. initrd is also in reserved regions, if it is at the begin or at the
end of physical memory, kernel will refuse to reuse the memory. Because the virt_addr_valid check in free_initrd_mem.

So it is better to fix and clean up those issues.
Calculate min_low_pfn and max_low_pfn in a consistent way.

Below is the patch, please review and comments

diff -Nraup a/arch/ia64/mm/contig.c b/arch/ia64/mm/contig.c

arch/ia64/mm/contig.c +6 -24
@@ -75,26 +75,6 @@ show_mem (void)
 unsigned long bootmap_start;
 
 /**
- * find_max_pfn - adjust the maximum page number callback
- * @start: start of range
- * @end: end of range
- * @arg: address of pointer to global max_pfn variable
- *
- * Passed as a callback function to efi_memmap_walk() to determine the highest
- * available page frame number in the system.
- */
-int
-find_max_pfn (unsigned long start, unsigned long end, void *arg)
-{
-	unsigned long *max_pfnp = arg, pfn;
-
-	pfn = (PAGE_ALIGN(end - 1) - PAGE_OFFSET) >> PAGE_SHIFT;
-	if (pfn > *max_pfnp)
-		*max_pfnp = pfn;
-	return 0;
-}
-
-/**
  * find_bootmap_location - callback to find a memory area for the bootmap
  * @start: start of region
  * @end: end of region
@@ -155,9 +135,10 @@ find_memory (void)
 	reserve_memory();
 
 	/* first find highest page frame number */
-	max_pfn = 0;
-	efi_memmap_walk(find_max_pfn, &max_pfn);
-
+	min_low_pfn = -1;
+	max_low_pfn = 0;
+	efi_memmap_walk(find_max_min_low_pfn, NULL);
+	max_pfn = max_low_pfn;
 	/* how many bytes to cover all the pages */
 	bootmap_size = bootmem_bootmap_pages(max_pfn) << PAGE_SHIFT;
 
@@ -167,7 +148,8 @@ find_memory (void)
 	if (bootmap_start = ~0UL)
 		panic("Cannot find %ld bytes for bootmap\n", bootmap_size);
 
-	bootmap_size = init_bootmem(bootmap_start >> PAGE_SHIFT, max_pfn);
+	bootmap_size = init_bootmem_node(NODE_DATA(0), 
+			(bootmap_start >> PAGE_SHIFT), 0, max_pfn);
 
 	/* Free all available memory, then mark bootmem-map as being in use. */
 	efi_memmap_walk(filter_rsvd_memory, free_bootmem);

diff -Nraup a/arch/ia64/mm/discontig.c b/arch/ia64/mm/discontig.c

arch/ia64/mm/discontig.c +1 -3
@@ -86,9 +86,6 @@ static int __init build_node_maps(unsign
 		bdp->node_low_pfn = max(epfn, bdp->node_low_pfn);
 	}
 
-	min_low_pfn = min(min_low_pfn, bdp->node_boot_start>>PAGE_SHIFT);
-	max_low_pfn = max(max_low_pfn, bdp->node_low_pfn);
-
 	return 0;
 }
 
@@ -467,6 +464,7 @@ void __init find_memory(void)
 	/* These actually end up getting called by call_pernode_memory() */
 	efi_memmap_walk(filter_rsvd_memory, build_node_maps);
 	efi_memmap_walk(filter_rsvd_memory, find_pernode_space);
+	efi_memmap_walk(find_max_min_low_pfn, NULL);
 
 	for_each_online_node(node)
 		if (mem_data[node].bootmem_data.node_low_pfn) {

diff -Nraup a/arch/ia64/mm/init.c b/arch/ia64/mm/init.c

arch/ia64/mm/init.c +16 -0
@@ -616,6 +616,22 @@ count_reserved_pages (u64 start, u64 end
 	return 0;
 }
 
+int
+find_max_min_low_pfn (unsigned long start, unsigned long end, void *arg)
+{
+	unsigned long pfn_start, pfn_end;
+#if CONFIG_FLATMEM
+	pfn_start = (PAGE_ALIGN(__pa(start))) >> PAGE_SHIFT;
+	pfn_end = (PAGE_ALIGN(__pa(end - 1))) >> PAGE_SHIFT;
+#else
+	pfn_start = GRANULEROUNDDOWN(__pa(start)) >> PAGE_SHIFT;
+	pfn_end = GRANULEROUNDUP(__pa(end - 1)) >> PAGE_SHIFT;
+#endif
+	min_low_pfn = min(min_low_pfn, pfn_start);
+	max_low_pfn = max(max_low_pfn, pfn_end);
+	return 0;
+}
+
 /*
  * Boot command-line option "nolwsys" can be used to disable the use of any light-weight
  * system call handler.  When this option is in effect, all fsyscalls will end up bubbling

diff -Nraup a/include/asm-ia64/meminit.h b/include/asm-ia64/meminit.h

include/asm-ia64/meminit.h +1 -1
@@ -35,6 +35,7 @@ extern void reserve_memory (void);
 extern void find_initrd (void);
 extern int filter_rsvd_memory (unsigned long start, unsigned long end, void *arg);
 extern void efi_memmap_init(unsigned long *, unsigned long *);
+extern int find_max_min_low_pfn (unsigned long , unsigned long, void *);
 
 /*
  * For rounding an address to the next IA64_GRANULE_SIZE or order
 
 
 
 
 
 
 
-

To unsubscribe from this list: send the line "unsubscribe linux-ia64" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#2 lore raw
In reply to Zou Nan hai
Quoted 37 lines · Zou Nan hai

Hi,
We have seen bad_pte_print when testing crashdump on an SN machine in
recent 2.6.20 kernel.
There are tons of bad pte print (pfn < max_low_pfn) reports when the
crash kernel boots up, all those reported bad pages are inside initmem range;
That is because if the crash kernel code and data happens to be at the
beginning of the 1st node. build_node_maps in discontig.c will bypass reserved regions with filter_rsvd_memory. Since min_low_pfn is calculated in build_node_map, so in this case, min_low_pfn will be greater than kernel code and data.

Because pages inside initmem are freed and reused later, we saw
pfn_valid check fail on those pages.

I think this theoretically happen on a normal kernel. When I check
min_low_pfn and max_low_pfn calculation in contig.c and discontig.c. I found more issues than this.

1. min_low_pfn and max_low_pfn calculation is inconsistent between
contig.c and discontig.c,
min_low_pfn is calculated as the first page number of boot memmap in
contig.c (Why? Though this may work at the most of the time, I don't think it is the right logic). It is calculated as the lowest physical memory page number bypass reserved regions in discontig.c.
max_low_pfn is calculated include reserved regions in contig.c. It is
calculated exclude reserved regions in discontig.c.

2. If kernel code and data region is happen to be at the begin or the
end of physical memory, when min_low_pfn and max_low_pfn calculation is bypassed kernel code and data, pages in initmem will report bad.

3. initrd is also in reserved regions, if it is at the begin or at the
end of physical memory, kernel will refuse to reuse the memory. Because the virt_addr_valid check in free_initrd_mem.

Quoted 6 lines

So it is better to fix and clean up those issues.
Calculate min_low_pfn and max_low_pfn in a consistent way.

Below is the patch, please review and comments

I've seen similar problems on a HP rx2620 box using 2.6.20. I managed to resolve that problem with a patch similar to this one, but then I realized that the issue I was seeing had been solved already by some mm-related patch by Bob Picco that got included in 2.6.21-rc1.

So, I suspect that 2.6.21-rc1 and rc2 should work just fine without this patch.

/ magnus

PS. Any comments on the EFI_LOADER_DATA for ELF core header would be greatly appreciated, I think it is sort of related to this issue as well.

#3 lore raw
In reply to Zou Nan hai
Quoted 2 lines · Magnus Damm
Quoted 1 line · Zou Nan hai

[excessive text deleted]

Quoted 11 lines
Quoted 6 lines

So it is better to fix and clean up those issues.
Calculate min_low_pfn and max_low_pfn in a consistent way.

Below is the patch, please review and comments

I've seen similar problems on a HP rx2620 box using 2.6.20. I managed to resolve that problem with a patch similar to this one, but then I realized that the issue I was seeing had been solved already by some mm-related patch by Bob Picco that got included in 2.6.21-rc1.

I tested on 2.6.21-rc3 with DEBUG_VM turned on. The vanilla 2.6.21-rc3 without Nan-hai's patch, panicked on bugcheck on free_initmem->free_page as predicted. We still need this patch.

However, the zero-size vmcore problem is back on SN. But that is a dfiffernet problem.

Regards,
- jay

Quoted 13 lines

So, I suspect that 2.6.21-rc1 and rc2 should work just fine without this patch.

/ magnus

PS. Any comments on the EFI_LOADER_DATA for ELF core header would be greatly appreciated, I think it is sort of related to this issue as well.
-
To unsubscribe from this list: send the line "unsubscribe linux-ia64" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#4 lore raw
In reply to Zou Nan hai
Quoted 3 lines

I tested on 2.6.21-rc3 with DEBUG_VM turned on. The vanilla 2.6.21-rc3 without Nan-hai's patch, panicked on bugcheck on free_initmem->free_page as predicted. We still need this patch.

I've applied Nan hai's patch ... will push upstream in a little while.

Quoted 2 lines

However, the zero-size vmcore problem is back on SN. But that is a differnet problem.

I applied a fix that was supposed to fix this after -rc3. It is in Linus' tree now. Let me know if it really is still broken.

-Tony

#5 lore raw
In reply to Zou Nan hai
Quoted 5 lines · Tony
Quoted 3 lines

I tested on 2.6.21-rc3 with DEBUG_VM turned on. The vanilla 2.6.21-rc3 without Nan-hai's patch, panicked on bugcheck on free_initmem->free_page as predicted. We still need this patch.

I've applied Nan hai's patch ... will push upstream in a little while.

Thanks!

Quoted 6 lines
Quoted 3 lines

However, the zero-size vmcore problem is back on SN. But that is a differnet problem.

I applied a fix that was supposed to fix this after -rc3. It is in Linus' tree now. Let me know if it really is still broken.

I git pull from Linus' tree yesterday afternoon, and i still have the zero sized vmcore problem on SN. I will look into this.

Regards,
- jay

Quoted 6 lines
(quoted diff) +0 -2
-Tony
-

To unsubscribe from this list: send the line "unsubscribe linux-ia64" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#6 lore raw
In reply to Zou Nan hai
Quoted 12 lines · Jay Lan
Quoted 2 lines · Magnus Damm
Quoted 1 line · Zou Nan hai

[excessive text deleted]

Quoted 7 lines
Quoted 6 lines

So it is better to fix and clean up those issues.
Calculate min_low_pfn and max_low_pfn in a consistent way.

Below is the patch, please review and comments

Quoted 8 lines
Quoted 4 lines

I've seen similar problems on a HP rx2620 box using 2.6.20. I managed to resolve that problem with a patch similar to this one, but then I realized that the issue I was seeing had been solved already by some mm-related patch by Bob Picco that got included in 2.6.21-rc1.

I tested on 2.6.21-rc3 with DEBUG_VM turned on. The vanilla 2.6.21-rc3 without Nan-hai's patch, panicked on bugcheck on free_initmem->free_page as predicted. We still need this patch.

Ok, thanks for testing. =)

Quoted 2 lines

However, the zero-size vmcore problem is back on SN. But that is a dfiffernet problem.

Argh, more problems...

/ magnus

#7 lore raw
In reply to Zou Nan hai

Magnus Damm wrote:
[snip]

Quoted 11 lines
Quoted 4 lines

I tested on 2.6.21-rc3 with DEBUG_VM turned on. The vanilla 2.6.21-rc3 without Nan-hai's patch, panicked on bugcheck on free_initmem->free_page as predicted. We still need this patch.

Ok, thanks for testing. =)

Quoted 2 lines

However, the zero-size vmcore problem is back on SN. But that is a dfiffernet problem.

Argh, more problems...

I found the problem. It was the "elfcorehdr" introduced in 2.6.21-rc1. Without specifying it, the elfcorehdr_addr is initialized to ELFCORE_ADDR_MAX. Later, a check in reserve_elfcorehdr will fail:
if (elfcorehdr_addr >= ELFCORE_ADDR_MAX)
return -EINVAL;

Is it supposed to be a physical address to store elf core header? If so, it is not possible for SN to provide a physical address at boot time, just like in the case of crashkernel=X@Y where Y is not used.

Thanks,
- jay

Quoted 6 lines

/ magnus
-
To unsubscribe from this list: send the line "unsubscribe linux-ia64" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#8 lore raw
In reply to Zou Nan hai
Quoted 17 lines · Jay Lan

Magnus Damm wrote:
[snip]

Quoted 8 lines
Quoted 3 lines

I tested on 2.6.21-rc3 with DEBUG_VM turned on. The vanilla 2.6.21-rc3 without Nan-hai's patch, panicked on bugcheck on free_initmem->free_page as predicted. We still need this patch.

Ok, thanks for testing. =)

Quoted 2 lines

However, the zero-size vmcore problem is back on SN. But that is a dfiffernet problem.

Argh, more problems...

I found the problem. It was the "elfcorehdr" introduced in 2.6.21-rc1. Without specifying it, the elfcorehdr_addr is initialized to ELFCORE_ADDR_MAX. Later, a check in reserve_elfcorehdr will fail:
if (elfcorehdr_addr >= ELFCORE_ADDR_MAX)
return -EINVAL;

Quoted 3 lines

Is it supposed to be a physical address to store elf core header? If so, it is not possible for SN to provide a physical address at boot time, just like in the case of crashkernel=X@Y where Y is not used.

Sorry, the elfcorehdr parameter is provided to the kdump kernel by kexec. The problem is in the reserve_elfcorehdr logic introduced in 2.6.21-rc1.

When booting up the kdump kernel, i observed a failure in reserve_elfcorehdr. The below are my debugging messages:

elfcorehdr_addr027fe4000, ELFCORE_ADDR_MAXÿffffffffffffff Cannot locate EFI vmcore descriptor
reserve_elfcorehdr: vmcore descriptor size = 0
reserve_memory: FAIL to reserve reserve_elfcorehdr

- jay

Quoted 11 lines

Thanks,
- jay

Quoted 5 lines

/ magnus
-
To unsubscribe from this list: send the line "unsubscribe linux-ia64" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#9 lore raw
In reply to Zou Nan hai
Quoted 25 lines · Jay Lan
Quoted 16 lines · Jay Lan

Magnus Damm wrote:
[snip]

Quoted 8 lines
Quoted 3 lines

I tested on 2.6.21-rc3 with DEBUG_VM turned on. The vanilla 2.6.21-rc3 without Nan-hai's patch, panicked on bugcheck on free_initmem->free_page as predicted. We still need this patch.

Ok, thanks for testing. =)

Quoted 2 lines

However, the zero-size vmcore problem is back on SN. But that is a dfiffernet problem.

Argh, more problems...

I found the problem. It was the "elfcorehdr" introduced in 2.6.21-rc1. Without specifying it, the elfcorehdr_addr is initialized to ELFCORE_ADDR_MAX. Later, a check in reserve_elfcorehdr will fail:
if (elfcorehdr_addr >= ELFCORE_ADDR_MAX)
return -EINVAL;

Quoted 3 lines

Is it supposed to be a physical address to store elf core header? If so, it is not possible for SN to provide a physical address at boot time, just like in the case of crashkernel=X@Y where Y is not used.

Sorry, the elfcorehdr parameter is provided to the kdump kernel by kexec. The problem is in the reserve_elfcorehdr logic introduced in 2.6.21-rc1.

Quoted 7 lines

When booting up the kdump kernel, i observed a failure in reserve_elfcorehdr. The below are my debugging messages:

elfcorehdr_addr027fe4000, ELFCORE_ADDR_MAXÿffffffffffffff Cannot locate EFI vmcore descriptor
reserve_elfcorehdr: vmcore descriptor size = 0
reserve_memory: FAIL to reserve reserve_elfcorehdr

Hi Tony,

The kernel code 2.6.21-rc3+ is fine wrt zero-size-vmcore issue.

I have been testing on a rhel5 environment. The /sbin/kexec worked for me up to 2.6.20.

I then built a new kexec from tot kexec-tools-testing git tree and i no longer see the zero-size-vmcore problem.

Regards,
- jay

#10 lore raw
In reply to Zou Nan hai
Quoted 36 lines · Jay Lan
Quoted 25 lines · Jay Lan
Quoted 16 lines · Jay Lan

Magnus Damm wrote:
[snip]

Quoted 8 lines
Quoted 3 lines

I tested on 2.6.21-rc3 with DEBUG_VM turned on. The vanilla 2.6.21-rc3 without Nan-hai's patch, panicked on bugcheck on free_initmem->free_page as predicted. We still need this patch.

Ok, thanks for testing. =)

Quoted 2 lines

However, the zero-size vmcore problem is back on SN. But that is a dfiffernet problem.

Argh, more problems...

I found the problem. It was the "elfcorehdr" introduced in 2.6.21-rc1. Without specifying it, the elfcorehdr_addr is initialized to ELFCORE_ADDR_MAX. Later, a check in reserve_elfcorehdr will fail:
if (elfcorehdr_addr >= ELFCORE_ADDR_MAX)
return -EINVAL;

Quoted 3 lines

Is it supposed to be a physical address to store elf core header? If so, it is not possible for SN to provide a physical address at boot time, just like in the case of crashkernel=X@Y where Y is not used.

Sorry, the elfcorehdr parameter is provided to the kdump kernel by kexec. The problem is in the reserve_elfcorehdr logic introduced in 2.6.21-rc1.

Quoted 7 lines

When booting up the kdump kernel, i observed a failure in reserve_elfcorehdr. The below are my debugging messages:

elfcorehdr_addr027fe4000, ELFCORE_ADDR_MAXÿffffffffffffff Cannot locate EFI vmcore descriptor
reserve_elfcorehdr: vmcore descriptor size = 0
reserve_memory: FAIL to reserve reserve_elfcorehdr

Hi Tony,

Quoted 7 lines

The kernel code 2.6.21-rc3+ is fine wrt zero-size-vmcore issue.

I have been testing on a rhel5 environment. The /sbin/kexec worked for me up to 2.6.20.

I then built a new kexec from tot kexec-tools-testing git tree and i no longer see the zero-size-vmcore problem.

I take it that elfcorehdr is also correct in this environment?

The latest (ia64) kernels need a fairly recent kexec-tools-testing because of the change "kexec: Use EFI_LOADER_DATA for ELF core header". A recent kexec-tools-testing is probably needed for other reasons, though I can't think of them off-hand.

Signature · 3 lines
Horms
  H: http://www.vergenet.net/~horms/
  W: http://www.valinux.co.jp/en/
#11 lore raw
In reply to Zou Nan hai
Quoted 37 lines · Horms
Quoted 36 lines · Jay Lan
Quoted 24 lines · Jay Lan
Quoted 16 lines · Jay Lan

Magnus Damm wrote:
[snip]

Quoted 8 lines
Quoted 3 lines

I tested on 2.6.21-rc3 with DEBUG_VM turned on. The vanilla 2.6.21-rc3 without Nan-hai's patch, panicked on bugcheck on free_initmem->free_page as predicted. We still need this patch.

Ok, thanks for testing. =)

Quoted 2 lines

However, the zero-size vmcore problem is back on SN. But that is a dfiffernet problem.

Argh, more problems...

I found the problem. It was the "elfcorehdr" introduced in 2.6.21-rc1. Without specifying it, the elfcorehdr_addr is initialized to ELFCORE_ADDR_MAX. Later, a check in reserve_elfcorehdr will fail:
if (elfcorehdr_addr >= ELFCORE_ADDR_MAX)
return -EINVAL;

Quoted 3 lines

Is it supposed to be a physical address to store elf core header? If so, it is not possible for SN to provide a physical address at boot time, just like in the case of crashkernel=X@Y where Y is not used.

Sorry, the elfcorehdr parameter is provided to the kdump kernel by kexec. The problem is in the reserve_elfcorehdr logic introduced in 2.6.21-rc1.

Quoted 7 lines

When booting up the kdump kernel, i observed a failure in reserve_elfcorehdr. The below are my debugging messages:

elfcorehdr_addr027fe4000, ELFCORE_ADDR_MAXÿffffffffffffff Cannot locate EFI vmcore descriptor
reserve_elfcorehdr: vmcore descriptor size = 0
reserve_memory: FAIL to reserve reserve_elfcorehdr

Hi Tony,

The kernel code 2.6.21-rc3+ is fine wrt zero-size-vmcore issue.

Quoted 7 lines
Quoted 5 lines

I have been testing on a rhel5 environment. The /sbin/kexec worked for me up to 2.6.20.

I then built a new kexec from tot kexec-tools-testing git tree and i no longer see the zero-size-vmcore problem.

I take it that elfcorehdr is also correct in this environment?

Yes. I got a good vmcore.

Regards,
- jay

Quoted 6 lines

The latest (ia64) kernels need a fairly recent kexec-tools-testing because of the change "kexec: Use EFI_LOADER_DATA for ELF core header". A recent kexec-tools-testing is probably needed for other reasons, though I can't think of them off-hand.

#12 lore raw
In reply to Zou Nan hai
Quoted 37 lines · Zou Nan hai

Hi,
We have seen bad_pte_print when testing crashdump on an SN machine in
recent 2.6.20 kernel.
There are tons of bad pte print (pfn < max_low_pfn) reports when the
crash kernel boots up, all those reported bad pages are inside initmem range;
That is because if the crash kernel code and data happens to be at the
beginning of the 1st node. build_node_maps in discontig.c will bypass reserved regions with filter_rsvd_memory. Since min_low_pfn is calculated in build_node_map, so in this case, min_low_pfn will be greater than kernel code and data.

Because pages inside initmem are freed and reused later, we saw
pfn_valid check fail on those pages.

I think this theoretically happen on a normal kernel. When I check
min_low_pfn and max_low_pfn calculation in contig.c and discontig.c. I found more issues than this.

1. min_low_pfn and max_low_pfn calculation is inconsistent between
contig.c and discontig.c,
min_low_pfn is calculated as the first page number of boot memmap in
contig.c (Why? Though this may work at the most of the time, I don't think it is the right logic). It is calculated as the lowest physical memory page number bypass reserved regions in discontig.c.
max_low_pfn is calculated include reserved regions in contig.c. It is
calculated exclude reserved regions in discontig.c.

2. If kernel code and data region is happen to be at the begin or the
end of physical memory, when min_low_pfn and max_low_pfn calculation is bypassed kernel code and data, pages in initmem will report bad.

3. initrd is also in reserved regions, if it is at the begin or at the
end of physical memory, kernel will refuse to reuse the memory. Because the virt_addr_valid check in free_initrd_mem.

Quoted 6 lines

So it is better to fix and clean up those issues.
Calculate min_low_pfn and max_low_pfn in a consistent way.

Below is the patch, please review and comments

Hi Nanhai,

this patch seems generaly ok to me. I've put it in my working tree to see if it helps or breaks anything. I've also made a few minor comments inline.

Quoted 32 lines

diff -Nraup a/arch/ia64/mm/contig.c b/arch/ia64/mm/contig.c

arch/ia64/mm/contig.c +0 -20
@@ -75,26 +75,6 @@ show_mem (void)
 unsigned long bootmap_start;
 
 /**
- * find_max_pfn - adjust the maximum page number callback
- * @start: start of range
- * @end: end of range
- * @arg: address of pointer to global max_pfn variable
- *
- * Passed as a callback function to efi_memmap_walk() to determine the highest
- * available page frame number in the system.
- */
-int
-find_max_pfn (unsigned long start, unsigned long end, void *arg)
-{
-	unsigned long *max_pfnp = arg, pfn;
-
-	pfn = (PAGE_ALIGN(end - 1) - PAGE_OFFSET) >> PAGE_SHIFT;
-	if (pfn > *max_pfnp)
-		*max_pfnp = pfn;
-	return 0;
-}
-
-/**
  * find_bootmap_location - callback to find a memory area for the bootmap
  * @start: start of region
  * @end: end of region

arch/ia64/mm/contig.c find_memory (void) · line 138

Quoted 7 lines
arch/ia64/mm/contig.c +1 -3
@@ -155,9 +135,10 @@ find_memory (void)
 	reserve_memory();
 
 	/* first find highest page frame number */
-	max_pfn = 0;
-	efi_memmap_walk(find_max_pfn, &max_pfn);
-
+	min_low_pfn = -1;

I think that this should be ~0UL rather than -1
as min_low_pfn is an unsigned long.

arch/ia64/mm/contig.c find_memory (void) · line 152

Quoted 13 lines
arch/ia64/mm/contig.c +5 -1
+	max_low_pfn = 0;
+	efi_memmap_walk(find_max_min_low_pfn, NULL);
+	max_pfn = max_low_pfn;
 	/* how many bytes to cover all the pages */
 	bootmap_size = bootmem_bootmap_pages(max_pfn) << PAGE_SHIFT;
 
@@ -167,7 +148,8 @@ find_memory (void)
 	if (bootmap_start = ~0UL)
 		panic("Cannot find %ld bytes for bootmap\n", bootmap_size);
 
-	bootmap_size = init_bootmem(bootmap_start >> PAGE_SHIFT, max_pfn);
+	bootmap_size = init_bootmem_node(NODE_DATA(0), 
+			(bootmap_start >> PAGE_SHIFT), 0, max_pfn);

This seems like an akward way to get around the architecture-idependant behaviour of init_bootmem() which sets min_low_pfn and max_low_pfn. Though I guess its ok.

Quoted 27 lines
arch/ia64/mm/contig.c +0 -0
 	/* Free all available memory, then mark bootmem-map as being in use. */
 	efi_memmap_walk(filter_rsvd_memory, free_bootmem);

diff -Nraup a/arch/ia64/mm/discontig.c b/arch/ia64/mm/discontig.c

arch/ia64/mm/discontig.c +1 -3
@@ -86,9 +86,6 @@ static int __init build_node_maps(unsign
 		bdp->node_low_pfn = max(epfn, bdp->node_low_pfn);
 	}
 
-	min_low_pfn = min(min_low_pfn, bdp->node_boot_start>>PAGE_SHIFT);
-	max_low_pfn = max(max_low_pfn, bdp->node_low_pfn);
-
 	return 0;
 }
 
@@ -467,6 +464,7 @@ void __init find_memory(void)
 	/* These actually end up getting called by call_pernode_memory() */
 	efi_memmap_walk(filter_rsvd_memory, build_node_maps);
 	efi_memmap_walk(filter_rsvd_memory, find_pernode_space);
+	efi_memmap_walk(find_max_min_low_pfn, NULL);
 
 	for_each_online_node(node)
 		if (mem_data[node].bootmem_data.node_low_pfn) {

diff -Nraup a/arch/ia64/mm/init.c b/arch/ia64/mm/init.c

arch/ia64/mm/init.c count_reserved_pages (u64 start, u64 end · line 623

Quoted 8 lines
arch/ia64/mm/init.c +5 -0
@@ -616,6 +616,22 @@ count_reserved_pages (u64 start, u64 end
 	return 0;
 }
 
+int
+find_max_min_low_pfn (unsigned long start, unsigned long end, void *arg)
+{
+	unsigned long pfn_start, pfn_end;
+#if CONFIG_FLATMEM

I think this should be #ifdef not #if, I currently see the following compiler warning:

arch/ia64/mm/init.c:655:5: warning: "CONFIG_FLATMEM" is not defined arch/ia64/mm/init.c:655:5: warning: "CONFIG_FLATMEM" is not defined

Quoted 36 lines
arch/ia64/mm/init.c +11 -0
+	pfn_start = (PAGE_ALIGN(__pa(start))) >> PAGE_SHIFT;
+	pfn_end = (PAGE_ALIGN(__pa(end - 1))) >> PAGE_SHIFT;
+#else
+	pfn_start = GRANULEROUNDDOWN(__pa(start)) >> PAGE_SHIFT;
+	pfn_end = GRANULEROUNDUP(__pa(end - 1)) >> PAGE_SHIFT;
+#endif
+	min_low_pfn = min(min_low_pfn, pfn_start);
+	max_low_pfn = max(max_low_pfn, pfn_end);
+	return 0;
+}
+
 /*
  * Boot command-line option "nolwsys" can be used to disable the use of any light-weight
  * system call handler.  When this option is in effect, all fsyscalls will end up bubbling

diff -Nraup a/include/asm-ia64/meminit.h b/include/asm-ia64/meminit.h

include/asm-ia64/meminit.h +1 -1
@@ -35,6 +35,7 @@ extern void reserve_memory (void);
 extern void find_initrd (void);
 extern int filter_rsvd_memory (unsigned long start, unsigned long end, void *arg);
 extern void efi_memmap_init(unsigned long *, unsigned long *);
+extern int find_max_min_low_pfn (unsigned long , unsigned long, void *);
 
 /*
  * For rounding an address to the next IA64_GRANULE_SIZE or order
 
 
 
 
 
 
 
-

To unsubscribe from this list: send the line "unsubscribe linux-ia64" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Signature · 3 lines
Horms
  H: http://www.vergenet.net/~horms/
  W: http://www.valinux.co.jp/en/
Acked-by on v1 1/1
#13 lore raw
In reply to Zou Nan hai

Simon had one comment and two changes to Nan-hai's patches. Both changes make sense to me. I tested on SN with the both changes integrated.

I need this patch for SN to work. Simon, please comment if you need more testing on this patch. Thanks!

Quoted 189 lines · Horms
Quoted 36 lines · Zou Nan hai

Hi,
We have seen bad_pte_print when testing crashdump on an SN machine in
recent 2.6.20 kernel.
There are tons of bad pte print (pfn < max_low_pfn) reports when the
crash kernel boots up, all those reported bad pages are inside initmem range;
That is because if the crash kernel code and data happens to be at the
beginning of the 1st node. build_node_maps in discontig.c will bypass reserved regions with filter_rsvd_memory. Since min_low_pfn is calculated in build_node_map, so in this case, min_low_pfn will be greater than kernel code and data.

Because pages inside initmem are freed and reused later, we saw
pfn_valid check fail on those pages.

I think this theoretically happen on a normal kernel. When I check
min_low_pfn and max_low_pfn calculation in contig.c and discontig.c. I found more issues than this.

1. min_low_pfn and max_low_pfn calculation is inconsistent between
contig.c and discontig.c,
min_low_pfn is calculated as the first page number of boot memmap in
contig.c (Why? Though this may work at the most of the time, I don't think it is the right logic). It is calculated as the lowest physical memory page number bypass reserved regions in discontig.c.
max_low_pfn is calculated include reserved regions in contig.c. It is
calculated exclude reserved regions in discontig.c.

2. If kernel code and data region is happen to be at the begin or the
end of physical memory, when min_low_pfn and max_low_pfn calculation is bypassed kernel code and data, pages in initmem will report bad.

3. initrd is also in reserved regions, if it is at the begin or at the
end of physical memory, kernel will refuse to reuse the memory. Because the virt_addr_valid check in free_initrd_mem.

Quoted 6 lines

So it is better to fix and clean up those issues.
Calculate min_low_pfn and max_low_pfn in a consistent way.

Below is the patch, please review and comments

Hi Nanhai,

this patch seems generaly ok to me. I've put it in my working tree to see if it helps or breaks anything. I've also made a few minor comments inline.

Quoted 31 lines

diff -Nraup a/arch/ia64/mm/contig.c b/arch/ia64/mm/contig.c

arch/ia64/mm/contig.c +0 -20
@@ -75,26 +75,6 @@ show_mem (void)
 unsigned long bootmap_start;
 
 /**
- * find_max_pfn - adjust the maximum page number callback
- * @start: start of range
- * @end: end of range
- * @arg: address of pointer to global max_pfn variable
- *
- * Passed as a callback function to efi_memmap_walk() to determine the highest
- * available page frame number in the system.
- */
-int
-find_max_pfn (unsigned long start, unsigned long end, void *arg)
-{
-	unsigned long *max_pfnp = arg, pfn;
-
-	pfn = (PAGE_ALIGN(end - 1) - PAGE_OFFSET) >> PAGE_SHIFT;
-	if (pfn > *max_pfnp)
-		*max_pfnp = pfn;
-	return 0;
-}
-
-/**
  * find_bootmap_location - callback to find a memory area for the bootmap
  * @start: start of region
  * @end: end of region

arch/ia64/mm/contig.c find_memory (void) · line 138

Quoted 7 lines
arch/ia64/mm/contig.c +1 -3
@@ -155,9 +135,10 @@ find_memory (void)
 	reserve_memory();
 
 	/* first find highest page frame number */
-	max_pfn = 0;
-	efi_memmap_walk(find_max_pfn, &max_pfn);
-
+	min_low_pfn = -1;

I think that this should be ~0UL rather than -1
as min_low_pfn is an unsigned long.

arch/ia64/mm/contig.c find_memory (void) · line 152

Quoted 13 lines
arch/ia64/mm/contig.c +5 -1
+	max_low_pfn = 0;
+	efi_memmap_walk(find_max_min_low_pfn, NULL);
+	max_pfn = max_low_pfn;
 	/* how many bytes to cover all the pages */
 	bootmap_size = bootmem_bootmap_pages(max_pfn) << PAGE_SHIFT;
 
@@ -167,7 +148,8 @@ find_memory (void)
 	if (bootmap_start = ~0UL)
 		panic("Cannot find %ld bytes for bootmap\n", bootmap_size);
 
-	bootmap_size = init_bootmem(bootmap_start >> PAGE_SHIFT, max_pfn);
+	bootmap_size = init_bootmem_node(NODE_DATA(0), 
+			(bootmap_start >> PAGE_SHIFT), 0, max_pfn);

This seems like an akward way to get around the architecture-idependant behaviour of init_bootmem() which sets min_low_pfn and max_low_pfn. Though I guess its ok.

Quoted 27 lines
arch/ia64/mm/contig.c +0 -0
 	/* Free all available memory, then mark bootmem-map as being in use. */
 	efi_memmap_walk(filter_rsvd_memory, free_bootmem);

diff -Nraup a/arch/ia64/mm/discontig.c b/arch/ia64/mm/discontig.c

arch/ia64/mm/discontig.c +1 -3
@@ -86,9 +86,6 @@ static int __init build_node_maps(unsign
 		bdp->node_low_pfn = max(epfn, bdp->node_low_pfn);
 	}
 
-	min_low_pfn = min(min_low_pfn, bdp->node_boot_start>>PAGE_SHIFT);
-	max_low_pfn = max(max_low_pfn, bdp->node_low_pfn);
-
 	return 0;
 }
 
@@ -467,6 +464,7 @@ void __init find_memory(void)
 	/* These actually end up getting called by call_pernode_memory() */
 	efi_memmap_walk(filter_rsvd_memory, build_node_maps);
 	efi_memmap_walk(filter_rsvd_memory, find_pernode_space);
+	efi_memmap_walk(find_max_min_low_pfn, NULL);
 
 	for_each_online_node(node)
 		if (mem_data[node].bootmem_data.node_low_pfn) {

diff -Nraup a/arch/ia64/mm/init.c b/arch/ia64/mm/init.c

arch/ia64/mm/init.c count_reserved_pages (u64 start, u64 end · line 623

Quoted 8 lines
arch/ia64/mm/init.c +5 -0
@@ -616,6 +616,22 @@ count_reserved_pages (u64 start, u64 end
 	return 0;
 }
 
+int
+find_max_min_low_pfn (unsigned long start, unsigned long end, void *arg)
+{
+	unsigned long pfn_start, pfn_end;
+#if CONFIG_FLATMEM

I think this should be #ifdef not #if, I currently see the following compiler warning:

arch/ia64/mm/init.c:655:5: warning: "CONFIG_FLATMEM" is not defined arch/ia64/mm/init.c:655:5: warning: "CONFIG_FLATMEM" is not defined

Quoted 36 lines
arch/ia64/mm/init.c +11 -0
+	pfn_start = (PAGE_ALIGN(__pa(start))) >> PAGE_SHIFT;
+	pfn_end = (PAGE_ALIGN(__pa(end - 1))) >> PAGE_SHIFT;
+#else
+	pfn_start = GRANULEROUNDDOWN(__pa(start)) >> PAGE_SHIFT;
+	pfn_end = GRANULEROUNDUP(__pa(end - 1)) >> PAGE_SHIFT;
+#endif
+	min_low_pfn = min(min_low_pfn, pfn_start);
+	max_low_pfn = max(max_low_pfn, pfn_end);
+	return 0;
+}
+
 /*
  * Boot command-line option "nolwsys" can be used to disable the use of any light-weight
  * system call handler.  When this option is in effect, all fsyscalls will end up bubbling

diff -Nraup a/include/asm-ia64/meminit.h b/include/asm-ia64/meminit.h

include/asm-ia64/meminit.h +1 -1
@@ -35,6 +35,7 @@ extern void reserve_memory (void);
 extern void find_initrd (void);
 extern int filter_rsvd_memory (unsigned long start, unsigned long end, void *arg);
 extern void efi_memmap_init(unsigned long *, unsigned long *);
+extern int find_max_min_low_pfn (unsigned long , unsigned long, void *);
 
 /*
  * For rounding an address to the next IA64_GRANULE_SIZE or order
 
 
 
 
 
 
 
-

To unsubscribe from this list: send the line "unsubscribe linux-ia64" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html