Skip to contentSkip to conversation

x86/mm: Limit 2/4M size calculation to x86_32

PATCH In review Stefan Bader 5 participants 16 messages Jul 13, 2012 – Sep 7, 2012

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

I was bisecting a problem on 64bit where any attempt to cause a crash kernel to boot would hang. The bisect ended up on commit 722bc6b (x86/mm: Fix the size calculation of mapping tables) and somehow, looking at the calling function and the ranges printed on boot, I think the calculations should only be done in the 32bit case.

On 64bit:

[    0.000000] init_memory_mapping: [mem 0x00000000-0x77e87fff]
[    0.000000]  [mem 0x00000000-0x77dfffff] page 2M
[    0.000000]  [mem 0x77e00000-0x77e87fff] page 4k

Attached patch would fix this if you agree with it. Thanks.

-Stefan

From 6b679d1af20656929c0e829f29eed60b0a86a74f Mon Sep 17 00:00:00 2001
From: Stefan Bader <[email protected]>
Date: Fri, 13 Jul 2012 15:16:33 +0200
Subject: [PATCH] x86/mm: Limit 2/4M size calculation to x86_32

commit 722bc6b (x86/mm: Fix the size calculation of mapping tables) did modify the extra space calculation for mapping tables in order to make up for the first 2/4M memory range using 4K pages. However this setup is only used when compiling for 32bit. On 64bit there is only the trailing area of 4K pages (which is already added).

The code was already adapted once for things went wrong on a 8TB machine (bd2753b x86/mm: Only add extra pages count for the first memory range during pre-allocation early page table space), but it looks a bit like it currently would overdo things for 64bit.
I only noticed while bisecting for the reason I could not make a crash kernel boot (which ended up on this patch).

Cc WANG Cong <[email protected]>
Cc Yinghai Lu <[email protected]>
Cc Tejun Heo <[email protected]>

---

arch/x86/mm/init.c +0 -0
index bc4e9d8..636bbfd 100644
@@ -60,10 +60,11 @@ static void __init find_early_table_space(struct map_range

*mr, unsigned long en

                extra = end - ((end>>PMD_SHIFT) << PMD_SHIFT);
 #ifdef CONFIG_X86_32
                extra += PMD_SIZE;
-#endif
+
                /* The first 2/4M doesn't use large pages. */
                if (mr->start < PMD_SIZE)
                        extra += mr->end - mr->start;

+#endif

                ptes = (extra + PAGE_SIZE - 1) >> PAGE_SHIFT;
        } else
Signature · 1 line
1.7.9.5

From 6b679d1af20656929c0e829f29eed60b0a86a74f Mon Sep 17 00:00:00 2001
From: Stefan Bader <[email protected]>
Date: Fri, 13 Jul 2012 15:16:33 +0200
Subject: [PATCH] x86/mm: Limit 2/4M size calculation to x86_32

commit 722bc6b (x86/mm: Fix the size calculation of mapping tables) did modify the extra space calculation for mapping tables in order to make up for the first 2/4M memory range using 4K pages. However this setup is only used when compiling for 32bit. On 64bit there is only the trailing area of 4K pages (which is already added).

The code was already adapted once for things went wrong on a 8TB machine (bd2753b x86/mm: Only add extra pages count for the first memory range during pre-allocation early page table space), but it looks a bit like it currently would overdo things for 64bit.
I only noticed while bisecting for the reason I could not make a crash kernel boot (which ended up on this patch).

Cc WANG Cong <[email protected]>
Cc Yinghai Lu <[email protected]>
Cc Tejun Heo <[email protected]>
---
 arch/x86/mm/init.c |    3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
arch/x86/mm/init.c +2 -1
index bc4e9d8..636bbfd 100644
@@ -60,10 +60,11 @@ static void __init find_early_table_space(struct map_range *mr, unsigned long en
 		extra = end - ((end>>PMD_SHIFT) << PMD_SHIFT);
 #ifdef CONFIG_X86_32
 		extra += PMD_SIZE;
-#endif
+
 		/* The first 2/4M doesn't use large pages. */
 		if (mr->start < PMD_SIZE)
 			extra += mr->end - mr->start;
+#endif
 
 		ptes = (extra + PAGE_SIZE - 1) >> PAGE_SHIFT;
 	} else
Signature · 1 line
1.7.9.5
#1 lore raw
Quoted 6 lines · Stefan Bader

I was bisecting a problem on 64bit where any attempt to cause a crash kernel to boot would hang. The bisect ended up on commit 722bc6b (x86/mm: Fix the size calculation of mapping tables) and somehow, looking at the calling function and the ranges printed on boot, I think the calculations should only be done in the 32bit case.

Quoted 6 lines

On 64bit:

[    0.000000] init_memory_mapping: [mem 0x00000000-0x77e87fff]
[    0.000000]  [mem 0x00000000-0x77dfffff] page 2M
[    0.000000]  [mem 0x77e00000-0x77e87fff] page 4k

Attached patch would fix this if you agree with it. Thanks.

it does not look like for the hang for your system. maybe just because it change a bit memblock allocation layout.

can you please post whole boot log that is working and not?

Thanks

Yinghai

#2 lore raw
Quoted 13 lines · Yinghai Lu
Quoted 11 lines · Stefan Bader

I was bisecting a problem on 64bit where any attempt to cause a crash kernel to boot would hang. The bisect ended up on commit 722bc6b (x86/mm: Fix the size calculation of mapping tables) and somehow, looking at the calling function and the ranges printed on boot, I think the calculations should only be done in the 32bit case.

On 64bit:

[    0.000000] init_memory_mapping: [mem 0x00000000-0x77e87fff]
[    0.000000]  [mem 0x00000000-0x77dfffff] page 2M
[    0.000000]  [mem 0x77e00000-0x77e87fff] page 4k
Quoted 5 lines
Quoted 1 line

Attached patch would fix this if you agree with it. Thanks.

it does not look like for the hang for your system. maybe just because it change a bit memblock allocation layout.

The hang is merely the effect of limited memory getting even more limited and running out of it while trying to uncompress your initramfs and/or kernel is not helping.

Quoted 2 lines

can you please post whole boot log that is working and not?

I am traveling this week and have no access to the machine. But basically you can see the issue relatively simple. As 64bit does not have the first 2/4M area as 4k pages. So with the current state of the patch this would allocate extra space of about 3MB for the first range (about 1.9GB). Again the problem is not something bad going on beyond the fact that it wastes memory and it just happens to be more than it used to be, so the memory set aside for getting it to boot suddenly failed to be enough.

-Stefan

Quoted 4 lines

Thanks

Yinghai

#3 lore raw
In reply to Stefan Bader
Quoted 6 lines · Stefan Bader

I was bisecting a problem on 64bit where any attempt to cause a crash kernel to boot would hang. The bisect ended up on commit 722bc6b (x86/mm: Fix the size calculation of mapping tables) and somehow, looking at the calling function and the ranges printed on boot, I think the calculations should only be done in the 32bit case.

Quoted 7 lines

On 64bit:

[    0.000000] init_memory_mapping: [mem 0x00000000-0x77e87fff]
[    0.000000]  [mem 0x00000000-0x77dfffff] page 2M
[    0.000000]  [mem 0x77e00000-0x77e87fff] page 4k

Attached patch would fix this if you agree with it. Thanks.

Any news on this one? I thought it would be quite simple to check for sanity and not wasting memory sounds like a good thing to do. Even though there is plenty of it around most of the time. ;)

-Stefan

Quoted 45 lines

-Stefan

From 6b679d1af20656929c0e829f29eed60b0a86a74f Mon Sep 17 00:00:00 2001
From: Stefan Bader <[email protected]>
Date: Fri, 13 Jul 2012 15:16:33 +0200
Subject: [PATCH] x86/mm: Limit 2/4M size calculation to x86_32

commit 722bc6b (x86/mm: Fix the size calculation of mapping tables) did modify the extra space calculation for mapping tables in order to make up for the first 2/4M memory range using 4K pages. However this setup is only used when compiling for 32bit. On 64bit there is only the trailing area of 4K pages (which is already added).

The code was already adapted once for things went wrong on a 8TB machine (bd2753b x86/mm: Only add extra pages count for the first memory range during pre-allocation early page table space), but it looks a bit like it currently would overdo things for 64bit.
I only noticed while bisecting for the reason I could not make a crash kernel boot (which ended up on this patch).

Cc WANG Cong <[email protected]>
Cc Yinghai Lu <[email protected]>
Cc Tejun Heo <[email protected]>

---

arch/x86/mm/init.c +2 -1
index bc4e9d8..636bbfd 100644
@@ -60,10 +60,11 @@ static void __init find_early_table_space(struct map_range
 *mr, unsigned long en
                 extra = end - ((end>>PMD_SHIFT) << PMD_SHIFT);
  #ifdef CONFIG_X86_32
                 extra += PMD_SIZE;
-#endif
+
                 /* The first 2/4M doesn't use large pages. */
                 if (mr->start < PMD_SIZE)
                         extra += mr->end - mr->start;
+#endif
 
                 ptes = (extra + PAGE_SIZE - 1) >> PAGE_SHIFT;
         } else
Acked-by
#4 lore raw
In reply to Stefan Bader
Quoted 34 lines · Stefan Bader

I was bisecting a problem on 64bit where any attempt to cause a crash kernel to boot would hang. The bisect ended up on commit 722bc6b (x86/mm: Fix the size calculation of mapping tables) and somehow, looking at the calling function and the ranges printed on boot, I think the calculations should only be done in the 32bit case.

On 64bit:

[    0.000000] init_memory_mapping: [mem 0x00000000-0x77e87fff]
[    0.000000]  [mem 0x00000000-0x77dfffff] page 2M
[    0.000000]  [mem 0x77e00000-0x77e87fff] page 4k

Attached patch would fix this if you agree with it. Thanks.

-Stefan

From 6b679d1af20656929c0e829f29eed60b0a86a74f Mon Sep 17 00:00:00 2001
From: Stefan Bader <[email protected]>
Date: Fri, 13 Jul 2012 15:16:33 +0200
Subject: [PATCH] x86/mm: Limit 2/4M size calculation to x86_32

commit 722bc6b (x86/mm: Fix the size calculation of mapping tables) did modify the extra space calculation for mapping tables in order to make up for the first 2/4M memory range using 4K pages. However this setup is only used when compiling for 32bit. On 64bit there is only the trailing area of 4K pages (which is already added).

The code was already adapted once for things went wrong on a 8TB machine (bd2753b x86/mm: Only add extra pages count for the first memory range during pre-allocation early page table space), but it looks a bit like it currently would overdo things for 64bit.
I only noticed while bisecting for the reason I could not make a crash kernel boot (which ended up on this patch).

Quoted 4 lines
Cc WANG Cong <[email protected]>
Cc Yinghai Lu <[email protected]>
Cc Tejun Heo <[email protected]>

Sorry for that I was not aware of x86_64 is different with x86 in the first 2/4M.

#5 lore raw
Quoted 23 lines · Cong Wang
Quoted 18 lines

From 6b679d1af20656929c0e829f29eed60b0a86a74f Mon Sep 17 00:00:00 2001
From: Stefan Bader <[email protected]>
Date: Fri, 13 Jul 2012 15:16:33 +0200
Subject: [PATCH] x86/mm: Limit 2/4M size calculation to x86_32

commit 722bc6b (x86/mm: Fix the size calculation of mapping tables) did modify the extra space calculation for mapping tables in order to make up for the first 2/4M memory range using 4K pages. However this setup is only used when compiling for 32bit. On 64bit there is only the trailing area of 4K pages (which is already added).

The code was already adapted once for things went wrong on a 8TB machine (bd2753b x86/mm: Only add extra pages count for the first memory range during pre-allocation early page table space), but it looks a bit like it currently would overdo things for 64bit.
I only noticed while bisecting for the reason I could not make a crash kernel boot (which ended up on this patch).

Quoted 4 lines
Cc WANG Cong <[email protected]>
Cc Yinghai Lu <[email protected]>
Cc Tejun Heo <[email protected]>
Quoted 4 lines

Sorry for that I was not aware of x86_64 is different with x86 in the first 2/4M.

Why would there be a difference?

Shouldn't the IO space at 0xa0000-0x100000 be mapped with uncacheable attributes (or WC for VGA)? If it's done later, it can be done later for both.

Signature · 1 line
error compiling committee.c: too many arguments to function
#6 lore raw
Quoted 27 lines · Avi Kivity
Quoted 25 lines · Cong Wang
Quoted 18 lines

From 6b679d1af20656929c0e829f29eed60b0a86a74f Mon Sep 17 00:00:00 2001
From: Stefan Bader <[email protected]>
Date: Fri, 13 Jul 2012 15:16:33 +0200
Subject: [PATCH] x86/mm: Limit 2/4M size calculation to x86_32

commit 722bc6b (x86/mm: Fix the size calculation of mapping tables) did modify the extra space calculation for mapping tables in order to make up for the first 2/4M memory range using 4K pages. However this setup is only used when compiling for 32bit. On 64bit there is only the trailing area of 4K pages (which is already added).

The code was already adapted once for things went wrong on a 8TB machine (bd2753b x86/mm: Only add extra pages count for the first memory range during pre-allocation early page table space), but it looks a bit like it currently would overdo things for 64bit.
I only noticed while bisecting for the reason I could not make a crash kernel boot (which ended up on this patch).

Quoted 4 lines
Cc WANG Cong <[email protected]>
Cc Yinghai Lu <[email protected]>
Cc Tejun Heo <[email protected]>
Quoted 9 lines
Quoted 2 lines

Sorry for that I was not aware of x86_64 is different with x86 in the first 2/4M.

Why would there be a difference?

Shouldn't the IO space at 0xa0000-0x100000 be mapped with uncacheable attributes (or WC for VGA)? If it's done later, it can be done later for both.

arch/x86/mm/init.c

unsigned long __init_refok init_memory_mapping(...
...
ifdef CONFIG_X86_32
/*
* Don't use a large page for the first 2/4MB of memory
* because there are often fixed size MTRRs in there
* and overlapping MTRRs into large pages can cause
* slowdowns.
*/

#7 lore raw
Quoted 39 lines · Stefan Bader
Quoted 27 lines · Avi Kivity
Quoted 25 lines · Cong Wang
Quoted 18 lines

From 6b679d1af20656929c0e829f29eed60b0a86a74f Mon Sep 17 00:00:00 2001
From: Stefan Bader <[email protected]>
Date: Fri, 13 Jul 2012 15:16:33 +0200
Subject: [PATCH] x86/mm: Limit 2/4M size calculation to x86_32

commit 722bc6b (x86/mm: Fix the size calculation of mapping tables) did modify the extra space calculation for mapping tables in order to make up for the first 2/4M memory range using 4K pages. However this setup is only used when compiling for 32bit. On 64bit there is only the trailing area of 4K pages (which is already added).

The code was already adapted once for things went wrong on a 8TB machine (bd2753b x86/mm: Only add extra pages count for the first memory range during pre-allocation early page table space), but it looks a bit like it currently would overdo things for 64bit.
I only noticed while bisecting for the reason I could not make a crash kernel boot (which ended up on this patch).

Quoted 4 lines
Cc WANG Cong <[email protected]>
Cc Yinghai Lu <[email protected]>
Cc Tejun Heo <[email protected]>
Quoted 9 lines
Quoted 2 lines

Sorry for that I was not aware of x86_64 is different with x86 in the first 2/4M.

Why would there be a difference?

Shouldn't the IO space at 0xa0000-0x100000 be mapped with uncacheable attributes (or WC for VGA)? If it's done later, it can be done later for both.

arch/x86/mm/init.c

Quoted 10 lines

unsigned long __init_refok init_memory_mapping(...
...
ifdef CONFIG_X86_32
/*
* Don't use a large page for the first 2/4MB of memory
* because there are often fixed size MTRRs in there
* and overlapping MTRRs into large pages can cause
* slowdowns.
*/

That's equally true for X86_64.

Best would be to merge the MTRRs into PAT, but that might not work for SMM.

Signature · 1 line
error compiling committee.c: too many arguments to function
#8 lore raw
Quoted 51 lines · Avi Kivity
Quoted 39 lines · Stefan Bader
Quoted 27 lines · Avi Kivity
Quoted 25 lines · Cong Wang
Quoted 18 lines

From 6b679d1af20656929c0e829f29eed60b0a86a74f Mon Sep 17 00:00:00 2001
From: Stefan Bader <[email protected]>
Date: Fri, 13 Jul 2012 15:16:33 +0200
Subject: [PATCH] x86/mm: Limit 2/4M size calculation to x86_32

commit 722bc6b (x86/mm: Fix the size calculation of mapping tables) did modify the extra space calculation for mapping tables in order to make up for the first 2/4M memory range using 4K pages. However this setup is only used when compiling for 32bit. On 64bit there is only the trailing area of 4K pages (which is already added).

The code was already adapted once for things went wrong on a 8TB machine (bd2753b x86/mm: Only add extra pages count for the first memory range during pre-allocation early page table space), but it looks a bit like it currently would overdo things for 64bit.
I only noticed while bisecting for the reason I could not make a crash kernel boot (which ended up on this patch).

Quoted 4 lines
Cc WANG Cong <[email protected]>
Cc Yinghai Lu <[email protected]>
Cc Tejun Heo <[email protected]>
Quoted 9 lines
Quoted 2 lines

Sorry for that I was not aware of x86_64 is different with x86 in the first 2/4M.

Why would there be a difference?

Shouldn't the IO space at 0xa0000-0x100000 be mapped with uncacheable attributes (or WC for VGA)? If it's done later, it can be done later for both.

arch/x86/mm/init.c

Quoted 10 lines

unsigned long __init_refok init_memory_mapping(...
...
ifdef CONFIG_X86_32
/*
* Don't use a large page for the first 2/4MB of memory
* because there are often fixed size MTRRs in there
* and overlapping MTRRs into large pages can cause
* slowdowns.
*/

Quoted 5 lines

That's equally true for X86_64.

Best would be to merge the MTRRs into PAT, but that might not work for SMM.

Ok, true. Not sure why this was restricted to 32bit when reconsidering. Except if in 64bit it was assumed (or asserted) that the regions are aligned to 2M... But maybe this can be answered by someone knowing the details. I would not mind either way (have the first range with 4K pages in all cases or fixing the additional PTE allocation). Just as it is now it is inconsistent.

#9 lore raw
Quoted 12 lines · Stefan Bader
Quoted 12 lines
Quoted 9 lines

...
ifdef CONFIG_X86_32
/*
* Don't use a large page for the first 2/4MB of memory
* because there are often fixed size MTRRs in there
* and overlapping MTRRs into large pages can cause
* slowdowns.
*/

That's equally true for X86_64.

Quoted 8 lines
Quoted 3 lines

Best would be to merge the MTRRs into PAT, but that might not work for SMM.

Ok, true. Not sure why this was restricted to 32bit when reconsidering. Except if in 64bit it was assumed (or asserted) that the regions are aligned to 2M... But maybe this can be answered by someone knowing the details. I would not mind either way (have the first range with 4K pages in all cases or fixing the additional PTE allocation). Just as it is now it is inconsistent.

Sometimes CONFIG_X86_32 is used as an alias for "machines so old they don't support x86_64". As a 32-bit kernel can be run on a machine that does support x86_64, it should be replaced by a runtime test for X86_FEATURE_LM, until a more accurate test can be found.

Signature · 1 line
error compiling committee.c: too many arguments to function
#10 lore raw
Quoted 22 lines · Avi Kivity
Quoted 12 lines · Stefan Bader
Quoted 12 lines
Quoted 9 lines

...
ifdef CONFIG_X86_32
/*
* Don't use a large page for the first 2/4MB of memory
* because there are often fixed size MTRRs in there
* and overlapping MTRRs into large pages can cause
* slowdowns.
*/

That's equally true for X86_64.

Quoted 8 lines
Quoted 3 lines

Best would be to merge the MTRRs into PAT, but that might not work for SMM.

Ok, true. Not sure why this was restricted to 32bit when reconsidering. Except if in 64bit it was assumed (or asserted) that the regions are aligned to 2M... But maybe this can be answered by someone knowing the details. I would not mind either way (have the first range with 4K pages in all cases or fixing the additional PTE allocation). Just as it is now it is inconsistent.

Quoted 5 lines

Sometimes CONFIG_X86_32 is used as an alias for "machines so old they don't support x86_64". As a 32-bit kernel can be run on a machine that does support x86_64, it should be replaced by a runtime test for X86_FEATURE_LM, until a more accurate test can be found.

So basically the first range being 4k exist because MTRRs might define ranges there and those are always aligned to 4k but not necessarily to the bigger pages used. Reading through the Intel and AMD docs indicates various levels of badness when this is not the case. Though afaict MTRRs are not tied to long mode capable CPUs. For example Atom is 32bit only (the earlier ones at least) and uses MTRRs. So testing for LM would miss those.
Would it not be better to unconditionally have the first 2/4M as 4k pages? At least as long as there is no check for the alignment of the MTRR ranges. Or thinking of it, the runtime test should look for X86_FEATURE_MTRR, shouldn't it?

#11 lore raw
Quoted 29 lines · Stefan Bader
Quoted 22 lines · Avi Kivity
Quoted 12 lines · Stefan Bader
Quoted 12 lines
Quoted 9 lines

...
ifdef CONFIG_X86_32
/*
* Don't use a large page for the first 2/4MB of memory
* because there are often fixed size MTRRs in there
* and overlapping MTRRs into large pages can cause
* slowdowns.
*/

That's equally true for X86_64.

Quoted 8 lines
Quoted 3 lines

Best would be to merge the MTRRs into PAT, but that might not work for SMM.

Ok, true. Not sure why this was restricted to 32bit when reconsidering. Except if in 64bit it was assumed (or asserted) that the regions are aligned to 2M... But maybe this can be answered by someone knowing the details. I would not mind either way (have the first range with 4K pages in all cases or fixing the additional PTE allocation). Just as it is now it is inconsistent.

Quoted 5 lines

Sometimes CONFIG_X86_32 is used as an alias for "machines so old they don't support x86_64". As a 32-bit kernel can be run on a machine that does support x86_64, it should be replaced by a runtime test for X86_FEATURE_LM, until a more accurate test can be found.

Quoted 9 lines

So basically the first range being 4k exist because MTRRs might define ranges there and those are always aligned to 4k but not necessarily to the bigger pages used. Reading through the Intel and AMD docs indicates various levels of badness when this is not the case. Though afaict MTRRs are not tied to long mode capable CPUs. For example Atom is 32bit only (the earlier ones at least) and uses MTRRs. So testing for LM would miss those.
Would it not be better to unconditionally have the first 2/4M as 4k pages? At least as long as there is no check for the alignment of the MTRR ranges. Or thinking of it, the runtime test should look for X86_FEATURE_MTRR, shouldn't it?

MTRRs are indeed far older than x86_64; it's almost pointless to test for them, since practically all processors have them.

The fact that the check is only done on i386 and not on x86_64 may come from one of

- an oversight
- by the time x86_64 processors came along, the problem with
conflicting sizes was resolved
- the whole thing is bogus

Copying hpa who may be in a position to find out which.

Signature · 1 line
error compiling committee.c: too many arguments to function
#12 lore raw

PATCH 1/1 x86/mm: Limit 2/4M size calculation to x86_32

Quoted 9 lines · Avi

The fact that the check is only done on i386 and not on x86_64 may come from one of

- an oversight
- by the time x86_64 processors came along, the problem with conflicting sizes was resolved
- the whole thing is bogus

Copying hpa who may be in a position to find out which.

Talking to hpa it is more of the last. For more than just this reason. Since the whole area of initial page tables seems to be rather sensitive and easy to break there have been discussions and plans to come up with a rewrite to improve on all those shortcomings.

The detail I am not agreeing with hpa is the fixup for the immediate breakage at head. IMO right now the code just has regressed and that should be fixed as soon as possible. Plus doing a specific and small fix allows that to be applicable to stable (which again still depends on things being upstream).

Hence the re-send in the hope that on the larger scale the may be agreement on the immediate fix. I am not doubting the usefulness or need of a better solution, but I think that having a remedy of the current situation just until then has enough benefit to be considered.

-Stefan

Quoted 1 line

From 1d5cc3971716a039c91abc18cb6f9bcbe5dde490 Mon Sep 17 00:00:00 2001

From: Stefan Bader <[email protected]>
Date: Fri, 13 Jul 2012 15:16:33 +0200
Subject: [PATCH] x86/mm: Limit 2/4M size calculation to x86_32

commit 722bc6b (x86/mm: Fix the size calculation of mapping tables) did modify the extra space calculation for mapping tables in order to make up for the first 2/4M memory range using 4K pages. However this setup is only used when compiling for 32bit. On 64bit there is only the trailing area of 4K pages (which is already added).

The code was already adapted once for things went wrong on a 8TB machine (bd2753b x86/mm: Only add extra pages count for the first memory range during pre-allocation early page table space), but it looks a bit like it currently would overdo things for 64bit.
I only noticed while bisecting for the reason I could not make a crash kernel boot (which ended up on this patch).

Cc WANG Cong <[email protected]>
Cc Yinghai Lu <[email protected]>
Cc Tejun Heo <[email protected]>
---
 arch/x86/mm/init.c |    3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
arch/x86/mm/init.c +2 -1
index e0e6990..28a1c99 100644
@@ -60,10 +60,11 @@ static void __init find_early_table_space(struct map_range *mr, unsigned long en
 		extra = end - ((end>>PMD_SHIFT) << PMD_SHIFT);
 #ifdef CONFIG_X86_32
 		extra += PMD_SIZE;
-#endif
+
 		/* The first 2/4M doesn't use large pages. */
 		if (mr->start < PMD_SIZE)
 			extra += mr->end - mr->start;
+#endif
 
 		ptes = (extra + PAGE_SIZE - 1) >> PAGE_SHIFT;
 	} else
Signature · 1 line
1.7.10.4
#13 lore raw

I'm not saying we shouldn't patch the regression, but this house of cards *needs* to be replaced with something robust and correct by construction.

Quoted 81 lines · Stefan Bader
Quoted 9 lines · Avi

The fact that the check is only done on i386 and not on x86_64 may come from one of

- an oversight
- by the time x86_64 processors came along, the problem with conflicting sizes was resolved
- the whole thing is bogus

Copying hpa who may be in a position to find out which.

Talking to hpa it is more of the last. For more than just this reason. Since the whole area of initial page tables seems to be rather sensitive and easy to break there have been discussions and plans to come up with a rewrite to improve on all those shortcomings.

The detail I am not agreeing with hpa is the fixup for the immediate breakage at head. IMO right now the code just has regressed and that should be fixed as soon as possible. Plus doing a specific and small fix allows that to be applicable to stable (which again still depends on things being upstream).

Hence the re-send in the hope that on the larger scale the may be agreement on the immediate fix. I am not doubting the usefulness or need of a better solution, but I think that having a remedy of the current situation just until then has enough benefit to be considered.

-Stefan

From 1d5cc3971716a039c91abc18cb6f9bcbe5dde490 Mon Sep 17 00:00:00 2001
From: Stefan Bader <[email protected]>
Date: Fri, 13 Jul 2012 15:16:33 +0200
Subject: [PATCH] x86/mm: Limit 2/4M size calculation to x86_32

commit 722bc6b (x86/mm: Fix the size calculation of mapping tables) did modify the extra space calculation for mapping tables in order to make up for the first 2/4M memory range using 4K pages. However this setup is only used when compiling for 32bit. On 64bit there is only the trailing area of 4K pages (which is already added).

The code was already adapted once for things went wrong on a 8TB machine (bd2753b x86/mm: Only add extra pages count for the first memory
range during pre-allocation early page table space), but it looks a bit like it currently would overdo things for 64bit.
I only noticed while bisecting for the reason I could not make a crash kernel boot (which ended up on this patch).

Cc WANG Cong <[email protected]>
Cc Yinghai Lu <[email protected]>
Cc Tejun Heo <[email protected]>

---
arch/x86/mm/init.c | 3 ++-

1 file changed, 2 insertions(+), 1 deletion(-)
arch/x86/mm/init.c +0 -0
index e0e6990..28a1c99 100644
@@ -60,10 +60,11 @@ static void __init find_early_table_space(struct

map_range *mr, unsigned long en

		extra = end - ((end>>PMD_SHIFT) << PMD_SHIFT);
#ifdef CONFIG_X86_32
		extra += PMD_SIZE;
arch/x86/mm/init.c +1 -1
-#endif
+
		/* The first 2/4M doesn't use large pages. */
		if (mr->start < PMD_SIZE)
			extra += mr->end - mr->start;

+#endif

		ptes = (extra + PAGE_SIZE - 1) >> PAGE_SHIFT;
	} else
Signature · 1 line
1.7.10.4
Signature · 1 line
Sent from my mobile phone. Please excuse brevity and lack of formatting.
#14 lore raw
Quoted 2 lines · H. Peter Anvin

I'm not saying we shouldn't patch the regression, but this house of cards *needs* to be replaced with something robust and correct by construction.

Then I did misunderstand/-interpret you about the former part and we actually are agreeing on the whole topic. Sorry about that. So the re-post just should serve as a reminder as the last comment here was quite a while ago.

Quoted 29 lines
Quoted 26 lines · Stefan Bader
Quoted 7 lines · Avi

The fact that the check is only done on i386 and not on x86_64 may come from one of

- an oversight - by the time x86_64 processors came along, the problem with conflicting sizes was resolved - the whole thing is bogus

Copying hpa who may be in a position to find out which.

Talking to hpa it is more of the last. For more than just this reason. Since the whole area of initial page tables seems to be rather sensitive and easy to break there have been discussions and plans to come up with a rewrite to improve on all those shortcomings.

The detail I am not agreeing with hpa is the fixup for the immediate breakage at head. IMO right now the code just has regressed and that should be fixed as soon as possible. Plus doing a specific and small fix allows that to be applicable to stable (which again still depends on things being upstream).

Hence the re-send in the hope that on the larger scale the may be agreement on the immediate fix. I am not doubting the usefulness or need of a better solution, but I think that having a remedy of the current situation just until then has enough benefit to be considered.

-Stefan

#15 lore raw
In reply to H. Peter Anvin
Quoted 2 lines · H. Peter Anvin

I'm not saying we shouldn't patch the regression, but this house of cards *needs* to be replaced with something robust and correct by construction.

Could that patch then get applied? Though I got some feedback, that the description might be not really well written. So I am attaching a version that tries to do better. The code change itself is the same.

-Stefan

---

From 737a5ebdd7ac1f4106cb0b0c53cc8f73b6ff1aca Mon Sep 17 00:00:00 2001
From: Stefan Bader <[email protected]>
Date: Fri, 13 Jul 2012 15:16:33 +0200
Subject: [PATCH] x86/mm: Limit extra padding calculation to x86_32

Starting with kernel v3.5 kexec based crash dumping was observed to fail (without any apparent message) on x86_64 machines. This was traced to a lack of memory triggered by a substantial increase (several megabyes) in the size of the initial page tables.

After regression (on a VM with 2GB of memory):
kernel direct mapping tables up to 0x7fffcfff @ [mem 0x1fbfd000-0x1fffffff] size = 4206591 bytes

With this patch applied:
kernel direct mapping tables up to 0x7fffcfff @ [mem 0x1fffc000-0x1fffffff] size = 16383 bytes

A bisection lead to the commit below:

commit 722bc6b (x86/mm: Fix the size calculation of mapping tables)

This change modified the extra space calculation to take into account that the first 2/4M range of memory would be mapped as 4K pages as suggested in chapter 11.11.9 of the Intel software developer's manual.

However this is currently only true for x86_32 (the reasons behind that are unclear but apparently the whole page table setup needs to be re- visited as it turns out to be very easy to break and has flaws in its current form).

Until the logic can be revisited and combined, pair up the extra space calculation with the logic which creates the extra mappings.

Cc WANG Cong <[email protected]>
Cc Yinghai Lu <[email protected]>
Cc Tejun Heo <[email protected]>
---
 arch/x86/mm/init.c |    3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
arch/x86/mm/init.c +0 -0
index bc4e9d8..636bbfd 100644
@@ -60,10 +60,11 @@ static void __init find_early_table_space(struct map_range

*mr, unsigned long en
extra = end - ((end>>PMD_SHIFT) << PMD_SHIFT);
#ifdef CONFIG_X86_32
extra += PMD_SIZE;
-#endif
+
/* The first 2/4M doesn't use large pages. */ if (mr->start < PMD_SIZE)
extra += mr->end - mr->start;
+#endif

ptes = (extra + PAGE_SIZE - 1) >> PAGE_SHIFT;
} else

Signature · 1 line
1.7.9.5

From 737a5ebdd7ac1f4106cb0b0c53cc8f73b6ff1aca Mon Sep 17 00:00:00 2001
From: Stefan Bader <[email protected]>
Date: Fri, 13 Jul 2012 15:16:33 +0200
Subject: [PATCH] x86/mm: Limit extra padding calculation to x86_32

Starting with kernel v3.5 kexec based crash dumping was observed to fail (without any apparent message) on x86_64 machines. This was traced to a lack of memory triggered by a substantial increase (several megabyes) in the size of the initial page tables.

After regression (on a VM with 2GB of memory):
kernel direct mapping tables up to 0x7fffcfff @ [mem 0x1fbfd000-0x1fffffff] size = 4206591 bytes

With this patch applied:
kernel direct mapping tables up to 0x7fffcfff @ [mem 0x1fffc000-0x1fffffff] size = 16383 bytes

A bisection lead to the commit below:

commit 722bc6b (x86/mm: Fix the size calculation of mapping tables)

This change modified the extra space calculation to take into account that the first 2/4M range of memory would be mapped as 4K pages as suggested in chapter 11.11.9 of the Intel software developer's manual.

However this is currently only true for x86_32 (the reasons behind that are unclear but apparently the whole page table setup needs to be re- visited as it turns out to be very easy to break and has flaws in its current form).

Until the logic can be revisited and combined, pair up the extra space calculation with the logic which creates the extra mappings.

Cc WANG Cong <[email protected]>
Cc Yinghai Lu <[email protected]>
Cc Tejun Heo <[email protected]>
---
 arch/x86/mm/init.c |    3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
arch/x86/mm/init.c +2 -1
index bc4e9d8..636bbfd 100644
@@ -60,10 +60,11 @@ static void __init find_early_table_space(struct map_range *mr, unsigned long en
 		extra = end - ((end>>PMD_SHIFT) << PMD_SHIFT);
 #ifdef CONFIG_X86_32
 		extra += PMD_SIZE;
-#endif
+
 		/* The first 2/4M doesn't use large pages. */
 		if (mr->start < PMD_SIZE)
 			extra += mr->end - mr->start;
+#endif
 
 		ptes = (extra + PAGE_SIZE - 1) >> PAGE_SHIFT;
 	} else
Signature · 1 line
1.7.9.5