Skip to contentSkip to conversation

Do not reserve crashkernel high memory if crashkernel low memory reserving failed

PATCH In review Baoquan He 2 participants 6 messages Sep 22, 2015 – Sep 23, 2015 x86

lore
Share & tools
lore

Baoquan He

author
#0 lore raw

PATCH v4 1/1 Do not reserve crashkernel high memory if crashkernel low memory reserving failed

People reported that when allocating crashkernel memory using ",high" and ",low" syntax, there were cases where the reservation of the "high" portion succeeds, but the reservation of the "low" portion fails. Then kexec can load kdump kernel successfully, but the boot of kdump kernel fails as there's no low memory. This is because allocation of low memory for kdump kernel can fail on large systems for reasons. E.g it could be manually specified crashkernel low memory is too large to find in memblock region.

In this patch add return value for reserve_crashkernel_low. Then try to reserve crashkernel low memory after crashkernel high memory has been allocated. If crashkernel low memory reservation failed free crashkernel high memory and return. User can take measures when they found kdump kernel cann't be loaded successfully.

---
v1->v2:
Boris commented that error value EINVAL is negative, should use "return -EINVAL".

v2->v3:
Yinghai pointed out that during memblock_reserve, we could double the memblock reserve array. New memblock reserve could be overlapped with range for crashkernel high. So we have to reserve crashkernel high firstly, then free it if crashkernel low memory allocation failed.

v3->v4:
Dave suggested using "return -ENOMEM" when low memory reservation failed and printing failure message anyway.

 arch/x86/kernel/setup.c | 19 ++++++++++---------
 1 file changed, 10 insertions(+), 9 deletions(-)
arch/x86/kernel/setup.c +10 -9
index fdb7f2a..e362f92 100644
@@ -493,7 +493,7 @@ static void __init memblock_x86_reserve_range_setup_data(void)
 # define CRASH_KERNEL_ADDR_HIGH_MAX	MAXMEM
 #endif
 
-static void __init reserve_crashkernel_low(void)
+static int __init reserve_crashkernel_low(void)
 {
 #ifdef CONFIG_X86_64
 	const unsigned long long alignment = 16<<20;	/* 16M */
@@ -522,17 +522,15 @@ static void __init reserve_crashkernel_low(void)
 	} else {
 		/* passed with crashkernel=0,low ? */
 		if (!low_size)
-			return;
+			return 0;
 	}
 
 	low_base = memblock_find_in_range(low_size, (1ULL<<32),
 					low_size, alignment);
 
 	if (!low_base) {
-		if (!auto_set)
-			pr_info("crashkernel low reservation failed - No suitable area found.\n");
-
-		return;
+		pr_info("crashkernel low reservation failed - No suitable area found.\n");
+		return -ENOMEM;
 	}
 
 	memblock_reserve(low_base, low_size);
@@ -544,6 +542,7 @@ static void __init reserve_crashkernel_low(void)
 	crashk_low_res.end   = low_base + low_size - 1;
 	insert_resource(&iomem_resource, &crashk_low_res);
 #endif
+	return 0;
 }
 
 static void __init reserve_crashkernel(void)
@@ -595,6 +594,11 @@ static void __init reserve_crashkernel(void)
 	}
 	memblock_reserve(crash_base, crash_size);
 
+	if (crash_base >= (1ULL<<32) && reserve_crashkernel_low()) {
+		memblock_free(crash_base, crash_size);
+		return;
+	}
+
 	printk(KERN_INFO "Reserving %ldMB of memory at %ldMB "
 			"for crashkernel (System RAM: %ldMB)\n",
 			(unsigned long)(crash_size >> 20),
@@ -604,9 +608,6 @@ static void __init reserve_crashkernel(void)
 	crashk_res.start = crash_base;
 	crashk_res.end   = crash_base + crash_size - 1;
 	insert_resource(&iomem_resource, &crashk_res);
-
-	if (crash_base >= (1ULL<<32))
-		reserve_crashkernel_low();
 }
 #else
 static void __init reserve_crashkernel(void)
Signature · 1 line
2.1.0
author
#1 lore raw

Add Jerry to CC list.

Quoted 102 lines · Baoquan He

People reported that when allocating crashkernel memory using ",high" and ",low" syntax, there were cases where the reservation of the "high" portion succeeds, but the reservation of the "low" portion fails. Then kexec can load kdump kernel successfully, but the boot of kdump kernel fails as there's no low memory. This is because allocation of low memory for kdump kernel can fail on large systems for reasons. E.g it could be manually specified crashkernel low memory is too large to find in memblock region.

In this patch add return value for reserve_crashkernel_low. Then try to reserve crashkernel low memory after crashkernel high memory has been allocated. If crashkernel low memory reservation failed free crashkernel high memory and return. User can take measures when they found kdump kernel cann't be loaded successfully.

---
v1->v2:
Boris commented that error value EINVAL is negative, should use "return -EINVAL".

v2->v3:
Yinghai pointed out that during memblock_reserve, we could double the memblock reserve array. New memblock reserve could be overlapped with range for crashkernel high. So we have to reserve crashkernel high firstly, then free it if crashkernel low memory allocation failed.

v3->v4:
Dave suggested using "return -ENOMEM" when low memory reservation failed and printing failure message anyway.

 arch/x86/kernel/setup.c | 19 ++++++++++---------
 1 file changed, 10 insertions(+), 9 deletions(-)
arch/x86/kernel/setup.c +10 -9
index fdb7f2a..e362f92 100644
@@ -493,7 +493,7 @@ static void __init memblock_x86_reserve_range_setup_data(void)
 # define CRASH_KERNEL_ADDR_HIGH_MAX	MAXMEM
 #endif
 
-static void __init reserve_crashkernel_low(void)
+static int __init reserve_crashkernel_low(void)
 {
 #ifdef CONFIG_X86_64
 	const unsigned long long alignment = 16<<20;	/* 16M */
@@ -522,17 +522,15 @@ static void __init reserve_crashkernel_low(void)
 	} else {
 		/* passed with crashkernel=0,low ? */
 		if (!low_size)
-			return;
+			return 0;
 	}
 
 	low_base = memblock_find_in_range(low_size, (1ULL<<32),
 					low_size, alignment);
 
 	if (!low_base) {
-		if (!auto_set)
-			pr_info("crashkernel low reservation failed - No suitable area found.\n");
-
-		return;
+		pr_info("crashkernel low reservation failed - No suitable area found.\n");
+		return -ENOMEM;
 	}
 
 	memblock_reserve(low_base, low_size);
@@ -544,6 +542,7 @@ static void __init reserve_crashkernel_low(void)
 	crashk_low_res.end   = low_base + low_size - 1;
 	insert_resource(&iomem_resource, &crashk_low_res);
 #endif
+	return 0;
 }
 
 static void __init reserve_crashkernel(void)
@@ -595,6 +594,11 @@ static void __init reserve_crashkernel(void)
 	}
 	memblock_reserve(crash_base, crash_size);
 
+	if (crash_base >= (1ULL<<32) && reserve_crashkernel_low()) {
+		memblock_free(crash_base, crash_size);
+		return;
+	}
+
 	printk(KERN_INFO "Reserving %ldMB of memory at %ldMB "
 			"for crashkernel (System RAM: %ldMB)\n",
 			(unsigned long)(crash_size >> 20),
@@ -604,9 +608,6 @@ static void __init reserve_crashkernel(void)
 	crashk_res.start = crash_base;
 	crashk_res.end   = crash_base + crash_size - 1;
 	insert_resource(&iomem_resource, &crashk_res);
-
-	if (crash_base >= (1ULL<<32))
-		reserve_crashkernel_low();
 }
 #else
 static void __init reserve_crashkernel(void)
Signature · 1 line
2.1.0
#2 lore raw
In reply to Baoquan He
Quoted 29 lines · On Tue, 22 Sep 2015 19:48:14 +0800 Baoquan He <[email protected]> wrote:

People reported that when allocating crashkernel memory using ",high" and ",low" syntax, there were cases where the reservation of the "high" portion succeeds, but the reservation of the "low" portion fails. Then kexec can load kdump kernel successfully, but the boot of kdump kernel fails as there's no low memory. This is because allocation of low memory for kdump kernel can fail on large systems for reasons. E.g it could be manually specified crashkernel low memory is too large to find in memblock region.

In this patch add return value for reserve_crashkernel_low. Then try to reserve crashkernel low memory after crashkernel high memory has been allocated. If crashkernel low memory reservation failed free crashkernel high memory and return. User can take measures when they found kdump kernel cann't be loaded successfully.

...

arch/x86/kernel/setup.c +1 -1
@@ -493,7 +493,7 @@ static void __init memblock_x86_reserve_range_setup_data(void)
 # define CRASH_KERNEL_ADDR_HIGH_MAX	MAXMEM
 #endif
 
-static void __init reserve_crashkernel_low(void)
+static int __init reserve_crashkernel_low(void)
 {
 #ifdef CONFIG_X86_64
 	const unsigned long long alignment = 16<<20;	/* 16M */

arch/x86/kernel/setup.c static void __init reserve_crashkernel_low(void) · line 525

Quoted 5 lines
arch/x86/kernel/setup.c +1 -1
@@ -522,17 +522,15 @@ static void __init reserve_crashkernel_low(void)
 	} else {
 		/* passed with crashkernel=0,low ? */
 		if (!low_size)
-			return;
+			return 0;

What's happening here? It's returning "success" when parse_crashkernel_low() fails?

arch/x86/kernel/setup.c static void __init reserve_crashkernel_low(void) · line 532

Quoted 11 lines
arch/x86/kernel/setup.c +1 -4
 	}
 
 	low_base = memblock_find_in_range(low_size, (1ULL<<32),
 					low_size, alignment);
 
 	if (!low_base) {
-		if (!auto_set)
-			pr_info("crashkernel low reservation failed - No suitable area found.\n");
-
-		return;
+		pr_info("crashkernel low reservation failed - No suitable area found.\n");

That's not a terribly useful message. If kdump is now unavailable and the operator needs to take some remedial action then we should inform them of this.

Also, such a message should have higher severity than KERN_INFO?

author
#3 lore raw
Quoted 6 lines · Andrew Morton
Quoted 6 lines
arch/x86/kernel/setup.c +0 -0
@@ -493,7 +493,7 @@ static void __init memblock_x86_reserve_range_setup_data(void)
 # define CRASH_KERNEL_ADDR_HIGH_MAX	MAXMEM
 #endif
 
Quoted 14 lines

arch/x86/kernel/setup.c static void __init reserve_crashkernel_low(void) · line 525

Quoted 11 lines
arch/x86/kernel/setup.c +2 -2
-static void __init reserve_crashkernel_low(void)
+static int __init reserve_crashkernel_low(void)
 {
 #ifdef CONFIG_X86_64
 	const unsigned long long alignment = 16<<20;	/* 16M */
@@ -522,17 +522,15 @@ static void __init reserve_crashkernel_low(void)
 	} else {
 		/* passed with crashkernel=0,low ? */
 		if (!low_size)
-			return;
+			return 0;

What's happening here? It's returning "success" when parse_crashkernel_low() fails?

It's the case user specify "crashkernel=0,low" to disable crashkernel low memory allocation explicitly. So here we parse the cmdline and get it's in this case, reture 0 directly.

Quoted 17 lines

arch/x86/kernel/setup.c static void __init reserve_crashkernel_low(void) · line 532

Quoted 12 lines
arch/x86/kernel/setup.c +1 -4
 	}
 
 	low_base = memblock_find_in_range(low_size, (1ULL<<32),
 					low_size, alignment);
 
 	if (!low_base) {
-		if (!auto_set)
-			pr_info("crashkernel low reservation failed - No suitable area found.\n");
-
-		return;
+		pr_info("crashkernel low reservation failed - No suitable area found.\n");

That's not a terribly useful message. If kdump is now unavailable and the operator needs to take some remedial action then we should inform them of this.

Quoted 1 line

Also, such a message should have higher severity than KERN_INFO?

Yes, how about KERN_ERR? It's an unexpected result from kdump side though it doesn't harm the normal kernel.

Quoted 1 line
#4 lore raw
Quoted 17 lines · On Wed, 23 Sep 2015 08:02:55 +0800 Baoquan He <[email protected]> wrote:
Quoted 17 lines

arch/x86/kernel/setup.c static void __init reserve_crashkernel_low(void) · line 532

Quoted 12 lines
arch/x86/kernel/setup.c +1 -4
 	}
 
 	low_base = memblock_find_in_range(low_size, (1ULL<<32),
 					low_size, alignment);
 
 	if (!low_base) {
-		if (!auto_set)
-			pr_info("crashkernel low reservation failed - No suitable area found.\n");
-
-		return;
+		pr_info("crashkernel low reservation failed - No suitable area found.\n");

That's not a terribly useful message. If kdump is now unavailable and the operator needs to take some remedial action then we should inform them of this.

Quoted 4 lines
Quoted 1 line

Also, such a message should have higher severity than KERN_INFO?

Yes, how about KERN_ERR? It's an unexpected result from kdump side though it doesn't harm the normal kernel.

Sure, KERN_ERR is good. Along with more useful message text.

author
#5 lore raw
Quoted 19 lines · Andrew Morton
Quoted 17 lines · On Wed, 23 Sep 2015 08:02:55 +0800 Baoquan He <[email protected]> wrote:
Quoted 17 lines

arch/x86/kernel/setup.c static void __init reserve_crashkernel_low(void) · line 532

Quoted 12 lines
arch/x86/kernel/setup.c +1 -4
 	}
 
 	low_base = memblock_find_in_range(low_size, (1ULL<<32),
 					low_size, alignment);
 
 	if (!low_base) {
-		if (!auto_set)
-			pr_info("crashkernel low reservation failed - No suitable area found.\n");
-
-		return;
+		pr_info("crashkernel low reservation failed - No suitable area found.\n");

That's not a terribly useful message. If kdump is now unavailable and the operator needs to take some remedial action then we should inform them of this.

Quoted 6 lines
Quoted 4 lines
Quoted 1 line

Also, such a message should have higher severity than KERN_INFO?

Yes, how about KERN_ERR? It's an unexpected result from kdump side though it doesn't harm the normal kernel.

Sure, KERN_ERR is good. Along with more useful message text.

OK, will do. Thanks for your suggestion.