Memory size unaligned to section boundary

Hi,

It seems to me that I hit an issue in low memory mapping (map_lowmem). I'm using a custom memory size, which leads to an freeze on Linux 4.0 and also with Linus master on two tested ARMv7-A SoC's (Freescale Vybrid and NVIDIA Tegra 3):

With mem=259744K

[    0.000000] Booting Linux on physical CPU 0x0
[    0.000000] Linux version 4.0.0-00189-ga4d2a4c3-dirty
(ags at trochilidae) (gcc version 4.8.3 20140401 (prerelease) (Linaro GCC
4.8-2014.04) ) #506 Thu Apr 23 14:13:21 CEST 2015
[    0.000000] CPU: ARMv7 Processor [410fc051] revision 1 (ARMv7),
cr=10c5387d
[    0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing
instruction cache
[    0.000000] Machine model: Toradex Colibri VF61 on Colibri Evaluation
Board
[    0.000000] bootconsole [earlycon0] enabled
[    0.000000] cma: Reserved 16 MiB at 0x8e400000
[    0.000000] Memory policy: Data cache writeback

<freeze>

I dug a bit more into that, and it unveiled that when creating the mapping for the non-kernel_x part (if (kernel_x_end < end) in map_lowmem), the unaligned section at the end leads to the freeze. In alloc_init_pmd, if the memory end is section unaligned, alloc_init_pte gets called which allocates a PTE outside of the initialized region (in early_alloc_aligned). The system freezes at the call of memset in early_alloc_aligned function.

With some debug print, this can be better illustrated:

[    0.000000] pgd 800063f0, addr 8fc00000, end 8fda8000, next 8fda8000
[    0.000000] pud 800063f0, addr 8fc00000, end 8fda8000, next 8fda8000
[    0.000000] pmd 800063f0, addr 8fc00000, next 8fda8000
                          => actual end of memory ^^^^^^^^
[    0.000000] alloc_init_pte
[    0.000000] set_pte_ext, pte 00000000, addr 8fc00000, end 8fda8000
[    0.000000] early_pte_alloc
[    0.000000] early_alloc_aligned, 00001000, ptr 8fcff000, align

00001000
=> PTE allocated outside of initialized area ^^^^^^^^

It seems that memory gets allocation in the last section. When the last section was in the previous PMD, the allocation works, however if the last section is within the same PMD, the allocation ends up in the non-initialized area. So:

In other words, sizes which end in a upper part of the 2MB sized PMD fail, while sizes in the lower part of a PMD work. 0xFF80000 => fails (mem=261632K)
0xFE80000 => works (mem=260608K)
0xFD80000 => fails (mem=261632K)
...

While I understand the reason for the freeze, I don't know to properly fix it. It looks to me that in alloc_init_pmd, we should use __map_init_section first to map the last aligned section, before calling alloc_init_pte on the non aligned section.

Background: I tried to reuse the boot loader part of the simplefb implementation for sunxi. It decreases memory size by the size of the framebuffer. Hence the actually memory size can be unaligned, depending on the display size used. In the case at hand, a framebuffer of the size 800x600 worked while 1024x600 did not work... The implementation uses device tree to report the memory size, but the kernel arguments show the same behavior.

Maybe a regression of e651eab0af ("ARM: 7677/1: LPAE: Fix mapping in alloc_init_section for unaligned addresses"). I currently do not have a platform at hand which works on that Linux version out of the box.

Hi all,

Quoted 68 lines · Stefan Agner

Hi,

It seems to me that I hit an issue in low memory mapping (map_lowmem). I'm using a custom memory size, which leads to an freeze on Linux 4.0 and also with Linus master on two tested ARMv7-A SoC's (Freescale Vybrid and NVIDIA Tegra 3):

With mem=259744K

[    0.000000] Booting Linux on physical CPU 0x0
[    0.000000] Linux version 4.0.0-00189-ga4d2a4c3-dirty
(ags at trochilidae) (gcc version 4.8.3 20140401 (prerelease) (Linaro GCC
4.8-2014.04) ) #506 Thu Apr 23 14:13:21 CEST 2015
[    0.000000] CPU: ARMv7 Processor [410fc051] revision 1 (ARMv7),
cr=10c5387d
[    0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing
instruction cache
[    0.000000] Machine model: Toradex Colibri VF61 on Colibri Evaluation
Board
[    0.000000] bootconsole [earlycon0] enabled
[    0.000000] cma: Reserved 16 MiB at 0x8e400000
[    0.000000] Memory policy: Data cache writeback

<freeze>

I dug a bit more into that, and it unveiled that when creating the mapping for the non-kernel_x part (if (kernel_x_end < end) in map_lowmem), the unaligned section at the end leads to the freeze. In alloc_init_pmd, if the memory end is section unaligned, alloc_init_pte gets called which allocates a PTE outside of the initialized region (in early_alloc_aligned). The system freezes at the call of memset in early_alloc_aligned function.

With some debug print, this can be better illustrated:

[    0.000000] pgd 800063f0, addr 8fc00000, end 8fda8000, next 8fda8000
[    0.000000] pud 800063f0, addr 8fc00000, end 8fda8000, next 8fda8000
[    0.000000] pmd 800063f0, addr 8fc00000, next 8fda8000
                          => actual end of memory ^^^^^^^^
[    0.000000] alloc_init_pte
[    0.000000] set_pte_ext, pte 00000000, addr 8fc00000, end 8fda8000
[    0.000000] early_pte_alloc
[    0.000000] early_alloc_aligned, 00001000, ptr 8fcff000, align

00001000
=> PTE allocated outside of initialized area ^^^^^^^^

It seems that memory gets allocation in the last section. When the last section was in the previous PMD, the allocation works, however if the last section is within the same PMD, the allocation ends up in the non-initialized area. So:

In other words, sizes which end in a upper part of the 2MB sized PMD fail, while sizes in the lower part of a PMD work. 0xFF80000 => fails (mem=261632K)
0xFE80000 => works (mem=260608K)
0xFD80000 => fails (mem=261632K)
...

While I understand the reason for the freeze, I don't know to properly fix it. It looks to me that in alloc_init_pmd, we should use __map_init_section first to map the last aligned section, before calling alloc_init_pte on the non aligned section.

Background: I tried to reuse the boot loader part of the simplefb implementation for sunxi. It decreases memory size by the size of the framebuffer. Hence the actually memory size can be unaligned, depending on the display size used. In the case at hand, a framebuffer of the size 800x600 worked while 1024x600 did not work... The implementation uses device tree to report the memory size, but the kernel arguments show the same behavior.

Quoted 3 lines

Maybe a regression of e651eab0af ("ARM: 7677/1: LPAE: Fix mapping in alloc_init_section for unaligned addresses"). I currently do not have a platform at hand which works on that Linux version out of the box.

I'm seeing this to an Allwinner Cortex A7 based SoCs, specifically on tablets with a 1024x600 lcd screen it seems that shaving exactly the amount of memory needed for a 32bpp 1024x600 framebuffer of from the top of memory triggers this.

[Adding potentially interested parties, those involved in 7677/1]

Quoted 71 lines · Hans de Goede

Hi all,

Quoted 68 lines · Stefan Agner

Hi,

It seems to me that I hit an issue in low memory mapping (map_lowmem). I'm using a custom memory size, which leads to an freeze on Linux 4.0 and also with Linus master on two tested ARMv7-A SoC's (Freescale Vybrid and NVIDIA Tegra 3):

With mem=259744K

[    0.000000] Booting Linux on physical CPU 0x0
[    0.000000] Linux version 4.0.0-00189-ga4d2a4c3-dirty
(ags at trochilidae) (gcc version 4.8.3 20140401 (prerelease) (Linaro GCC
4.8-2014.04) ) #506 Thu Apr 23 14:13:21 CEST 2015
[    0.000000] CPU: ARMv7 Processor [410fc051] revision 1 (ARMv7),
cr=10c5387d
[    0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing
instruction cache
[    0.000000] Machine model: Toradex Colibri VF61 on Colibri Evaluation
Board
[    0.000000] bootconsole [earlycon0] enabled
[    0.000000] cma: Reserved 16 MiB at 0x8e400000
[    0.000000] Memory policy: Data cache writeback

<freeze>

I dug a bit more into that, and it unveiled that when creating the mapping for the non-kernel_x part (if (kernel_x_end < end) in map_lowmem), the unaligned section at the end leads to the freeze. In alloc_init_pmd, if the memory end is section unaligned, alloc_init_pte gets called which allocates a PTE outside of the initialized region (in early_alloc_aligned). The system freezes at the call of memset in early_alloc_aligned function.

With some debug print, this can be better illustrated:

[    0.000000] pgd 800063f0, addr 8fc00000, end 8fda8000, next 8fda8000
[    0.000000] pud 800063f0, addr 8fc00000, end 8fda8000, next 8fda8000
[    0.000000] pmd 800063f0, addr 8fc00000, next 8fda8000
                           => actual end of memory ^^^^^^^^
[    0.000000] alloc_init_pte
[    0.000000] set_pte_ext, pte 00000000, addr 8fc00000, end 8fda8000
[    0.000000] early_pte_alloc
[    0.000000] early_alloc_aligned, 00001000, ptr 8fcff000, align

00001000
=> PTE allocated outside of initialized area ^^^^^^^^

It seems that memory gets allocation in the last section. When the last section was in the previous PMD, the allocation works, however if the last section is within the same PMD, the allocation ends up in the non-initialized area. So:

In other words, sizes which end in a upper part of the 2MB sized PMD fail, while sizes in the lower part of a PMD work. 0xFF80000 => fails (mem=261632K)
0xFE80000 => works (mem=260608K)
0xFD80000 => fails (mem=261632K)
...

While I understand the reason for the freeze, I don't know to properly fix it. It looks to me that in alloc_init_pmd, we should use __map_init_section first to map the last aligned section, before calling alloc_init_pte on the non aligned section.

Background: I tried to reuse the boot loader part of the simplefb implementation for sunxi. It decreases memory size by the size of the framebuffer. Hence the actually memory size can be unaligned, depending on the display size used. In the case at hand, a framebuffer of the size 800x600 worked while 1024x600 did not work... The implementation uses device tree to report the memory size, but the kernel arguments show the same behavior.

Quoted 8 lines

Quoted 3 lines

Maybe a regression of e651eab0af ("ARM: 7677/1: LPAE: Fix mapping in alloc_init_section for unaligned addresses"). I currently do not have a platform at hand which works on that Linux version out of the box.

I'm seeing this to an Allwinner Cortex A7 based SoCs, specifically on tablets with a 1024x600 lcd screen it seems that shaving exactly the amount of memory needed for a 32bpp 1024x600 framebuffer of from the top of memory triggers this.

I'm able to trigger the issue on TC2 by passing mem=259744K. If I hack sanity_check_meminfo to round the memblock limit down to PMD_SIZE I avoid the immediate freeze, but later things blew up seemingly due to an unmapped DTB (panic below) I'm not entirely sure why that's the case.

I wasn't able to come up with a DTB that would trigger this. Do you have an example set of memory nodes + memreserves? Where are your kernel and DTB loaded in memory?

Unable to handle kernel paging request at virtual address 9fee6000
pgd = 80004000
[9fee6000] *pgd=00000000
Internal error: Oops: 5 [#1] SMP ARM
Modules linked in:
CPU: 0 PID: 0 Comm: swapper Not tainted 4.1.0-rc1+ #17
Hardware name: ARM-Versatile Express
task: 8065e7a8 ti: 8065a000 task.ti: 8065a000
PC is at fdt_check_header+0x0/0x74
LR is at __unflatten_device_tree+0x1c/0x128
pc : [<80490350>]    lr : [<803a1554>]    psr: a00001d3

sp : 8065bf28 ip : 806a7d77 fp : 80000200
r10: 8056d84c r9 : 8069fc9c r8 : 80635b0c
r7 : 80683140 r6 : 9fee6000 r5 : 8063eac4 r4 : 80635b0c r3 : 8069fcb4 r2 : 80635b0c r1 : 8069fc9c r0 : 9fee6000
Flags: NzCv IRQs off FIQs off Mode SVC_32 ISA ARM Segment kernel
Control: 10c5387d Table: 8000406a DAC: 00000015
Process swapper (pid: 0, stack limit = 0x8065a210)

Stack: (0x8065bf28 to 0x8065c000)

bf20: ffffffff 00000000 ffffffff 0008fbfd 00000000 00000000
bf40: 00000000 80635b0c 8063eac4 8065f79c 80683140 8068d5e4 806650e0 806366e8
bf60: 8065c3c8 8061b43c ffffffff 10c5387d 80683000 8fbfb340 80008000 8064aa88
bf80: 00000000 00000000 00000000 80058674 8056c3e8 8065bfb4 00000000 00000000
bfa0: 80683000 00000001 8065c3c0 ffffffff 00000000 00000000 00000000 8061895c
bfc0: 00000000 00000000 00000000 00000000 00000000 8064aa88 80683394 8065c440
bfe0: 8064aa84 8065f8bc 8000406a 412fc0f1 00000000 8000807c 00000000 00000000 [<80490350>] (fdt_check_header) from [<803a1554>] (__unflatten_device_tree+0x1c/0x128) [<803a1554>] (__unflatten_device_tree) from [<806366e8>] (unflatten_device_tree+0x28/0x34) [<806366e8>] (unflatten_device_tree) from [<8061b43c>] (setup_arch+0x778/0x984) [<8061b43c>] (setup_arch) from [<8061895c>] (start_kernel+0x9c/0x3ac) [<8061895c>] (start_kernel) from [<8000807c>] (0x8000807c)

Code: e3e0300d eafd2608 e3e0300d eafd260d (e5903000)
---[ end trace cb88537fdc8fa200 ]---
Kernel panic - not syncing: Attempted to kill the idle task!
---[ end Kernel panic - not syncing: Attempted to kill the idle task!

Quoted 3 lines

I wasn't able to come up with a DTB that would trigger this. Do you have an example set of memory nodes + memreserves? Where are your kernel and DTB loaded in memory?

We've a single memory node/bank from 0x40000000 to end of memory, we carve out a framebuffer at the end, and do not report that to Linux, so the end becomes 0x40000000 + memory-size - fb-size, and we use no mem reserves, because if we do use mem-reserves the mmap of the fbmem by the simplefb driver fails as that maps it non cach-able and if it is part of the main membank it already is mapped cachable.

We substract exactly the necessary fb-size, one known fb-size which triggers this is 1024x600 which means we end up substracting 1024x600x4 bytes from the end of memory, so effectively we are doing the same as passing a mem= argument which is not 2MiB aligned.

Quoted 10 lines

Quoted 3 lines

I wasn't able to come up with a DTB that would trigger this. Do you have an example set of memory nodes + memreserves? Where are your kernel and DTB loaded in memory?

We've a single memory node/bank from 0x40000000 to end of memory, we carve out a framebuffer at the end, and do not report that to Linux, so the end becomes 0x40000000 + memory-size - fb-size, and we use no mem reserves, because if we do use mem-reserves the mmap of the fbmem by the simplefb driver fails as that maps it non cach-able and if it is part of the main membank it already is mapped cachable.

Sure. The only reason for caring about any memreserves was in case they inadvertently affected the memblock_limit or anything else generated by iterating over the memblock array. If you have none to begin with then they clearly aren't involved.

Quoted 4 lines

We substract exactly the necessary fb-size, one known fb-size which triggers this is 1024x600 which means we end up substracting 1024x600x4 bytes from the end of memory, so effectively we are doing the same as passing a mem= argument which is not 2MiB aligned.

Thanks for the info.

It turns out my bootloader was silently rewriting the memory nodes, which was why I couldn't reproduce the issue with a DTB alone. With the memory node reg munged to <0 0x80000000 0 0x3FDA8000> without bootloader interference, TC2 dies similarly to what you described.

As far as I can see the issue is not a regression; it looks like we'd previously fail to use a (1M) section unless we had precisely 1M or 2M of the section to map (as those are the only cases when end would be section aligned).

The below hack prevents the issue by rounding the memblock_limit down to a full (2M) pmd boundary, so we don't try to allocate from the first section in a partial pmd. That does mean that if your memory ends on a 1M boundary you lose that last 1M for early memblock allocations.

Balancing the pmd manipulation turned out to be a lot more painful than I'd anticipated, so I gave up on trying to map the first section in a partial pmd. If people are happy with the below diff I can respin as a patch (with comment updates and so on).

Thanks,
Mark.

---->8----

arch/arm/mm/mmu.c +2 -2

index 4e6ef89..2ea13f0 100644
@@ -1125,9 +1125,9 @@ void __init sanity_check_meminfo(void)
                          * occurs before any free memory is mapped.
                          */
                         if (!memblock_limit) {
-                               if (!IS_ALIGNED(block_start, SECTION_SIZE))
+                               if (!IS_ALIGNED(block_start, PMD_SIZE))
                                         memblock_limit = block_start;
-                               else if (!IS_ALIGNED(block_end, SECTION_SIZE))
+                               else if (!IS_ALIGNED(block_end, PMD_SIZE))
                                         memblock_limit = arm_lowmem_limit;
                         }
 

Hi,

It seems to me that I hit an issue in low memory mapping (map_lowmem). I'm using a custom memory size, which leads to an freeze on Linux 4.0 and also with Linus master on two tested ARMv7-A SoC's (Freescale Vybrid and NVIDIA Tegra 3):

With mem=259744K

[    0.000000] Booting Linux on physical CPU 0x0
[    0.000000] Linux version 4.0.0-00189-ga4d2a4c3-dirty
(ags at trochilidae) (gcc version 4.8.3 20140401 (prerelease) (Linaro GCC
4.8-2014.04) ) #506 Thu Apr 23 14:13:21 CEST 2015
[    0.000000] CPU: ARMv7 Processor [410fc051] revision 1 (ARMv7),
cr=10c5387d
[    0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing
instruction cache
[    0.000000] Machine model: Toradex Colibri VF61 on Colibri Evaluation
Board
[    0.000000] bootconsole [earlycon0] enabled
[    0.000000] cma: Reserved 16 MiB at 0x8e400000
[    0.000000] Memory policy: Data cache writeback

<freeze>

I dug a bit more into that, and it unveiled that when creating the mapping for the non-kernel_x part (if (kernel_x_end < end) in map_lowmem), the unaligned section at the end leads to the freeze. In alloc_init_pmd, if the memory end is section unaligned, alloc_init_pte gets called which allocates a PTE outside of the initialized region (in early_alloc_aligned). The system freezes at the call of memset in early_alloc_aligned function.

With some debug print, this can be better illustrated:

[    0.000000] pgd 800063f0, addr 8fc00000, end 8fda8000, next 8fda8000
[    0.000000] pud 800063f0, addr 8fc00000, end 8fda8000, next 8fda8000
[    0.000000] pmd 800063f0, addr 8fc00000, next 8fda8000
                         => actual end of memory ^^^^^^^^
[    0.000000] alloc_init_pte
[    0.000000] set_pte_ext, pte 00000000, addr 8fc00000, end 8fda8000
[    0.000000] early_pte_alloc
[    0.000000] early_alloc_aligned, 00001000, ptr 8fcff000, align

00001000
=> PTE allocated outside of initialized area ^^^^^^^^

Quoted 40 lines · Stefan Agner

Hi,

It seems to me that I hit an issue in low memory mapping (map_lowmem). I'm using a custom memory size, which leads to an freeze on Linux 4.0 and also with Linus master on two tested ARMv7-A SoC's (Freescale Vybrid and NVIDIA Tegra 3):

With mem=259744K

[    0.000000] Booting Linux on physical CPU 0x0
[    0.000000] Linux version 4.0.0-00189-ga4d2a4c3-dirty
(ags at trochilidae) (gcc version 4.8.3 20140401 (prerelease) (Linaro GCC
4.8-2014.04) ) #506 Thu Apr 23 14:13:21 CEST 2015
[    0.000000] CPU: ARMv7 Processor [410fc051] revision 1 (ARMv7),
cr=10c5387d
[    0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing
instruction cache
[    0.000000] Machine model: Toradex Colibri VF61 on Colibri Evaluation
Board
[    0.000000] bootconsole [earlycon0] enabled
[    0.000000] cma: Reserved 16 MiB at 0x8e400000
[    0.000000] Memory policy: Data cache writeback

<freeze>

I dug a bit more into that, and it unveiled that when creating the mapping for the non-kernel_x part (if (kernel_x_end < end) in map_lowmem), the unaligned section at the end leads to the freeze. In alloc_init_pmd, if the memory end is section unaligned, alloc_init_pte gets called which allocates a PTE outside of the initialized region (in early_alloc_aligned). The system freezes at the call of memset in early_alloc_aligned function.

With some debug print, this can be better illustrated:

[    0.000000] pgd 800063f0, addr 8fc00000, end 8fda8000, next 8fda8000
[    0.000000] pud 800063f0, addr 8fc00000, end 8fda8000, next 8fda8000
[    0.000000] pmd 800063f0, addr 8fc00000, next 8fda8000
                         => actual end of memory ^^^^^^^^
[    0.000000] alloc_init_pte
[    0.000000] set_pte_ext, pte 00000000, addr 8fc00000, end 8fda8000
[    0.000000] early_pte_alloc
[    0.000000] early_alloc_aligned, 00001000, ptr 8fcff000, align

Quoted 2 lines

00001000
=> PTE allocated outside of initialized area ^^^^^^^^

Right, and the question is why does that happen - and the answer is this:

/*
* Round the memblock limit down to a section size. This
* helps to ensure that we will allocate memory from the
* last full section, which should be mapped.
*/
if (memblock_limit)
memblock_limit = round_down(memblock_limit, SECTION_SIZE);

That should round down by 2x SECTION_SIZE to ensure that we don't start allocating the L2 page table in a section which isn't mapped. Please try this patch:

 arch/arm/mm/mmu.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

arch/arm/mm/mmu.c +0 -0

index 4e6ef896c619..387becac5c86 100644

Quoted 7 lines · Stefan Agner

I dug a bit more into that, and it unveiled that when creating the mapping for the non-kernel_x part (if (kernel_x_end < end) in map_lowmem), the unaligned section at the end leads to the freeze. In alloc_init_pmd, if the memory end is section unaligned, alloc_init_pte gets called which allocates a PTE outside of the initialized region (in early_alloc_aligned). The system freezes at the call of memset in early_alloc_aligned function.

Right, and the question is why does that happen - and the answer is this:

/*
* Round the memblock limit down to a section size. This
* helps to ensure that we will allocate memory from the
* last full section, which should be mapped.
*/
if (memblock_limit)
memblock_limit = round_down(memblock_limit, SECTION_SIZE);

That should round down by 2x SECTION_SIZE to ensure that we don't start allocating the L2 page table in a section which isn't mapped. Please try this patch:

 arch/arm/mm/mmu.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

arch/arm/mm/mmu.c +0 -0

index 4e6ef896c619..387becac5c86 100644

arch/arm/mm/mmu.c void __init sanity_check_meminfo(void) · line 1145

Quoted 6 lines · Russell King - ARM Linux

arch/arm/mm/mmu.c +1 -1

@@ -1142,7 +1142,7 @@ void __init sanity_check_meminfo(void)
 	 * last full section, which should be mapped.
 	 */
 	if (memblock_limit)
-		memblock_limit = round_down(memblock_limit, SECTION_SIZE);
+		memblock_limit = round_down(memblock_limit, 2 * SECTION_SIZE);

Why not PMD_SIZE? We don't need 4MB round down with LPAE.

Quoted 9 lines · Catalin Marinas

Review void __init sanity_check_meminfo(void) · line 1145

Quoted 6 lines · Russell King - ARM Linux

(quoted diff) +1 -1

@@ -1142,7 +1142,7 @@ void __init sanity_check_meminfo(void)
  	 * last full section, which should be mapped.
  	 */
  	if (memblock_limit)
-		memblock_limit = round_down(memblock_limit, SECTION_SIZE);
+		memblock_limit = round_down(memblock_limit, 2 * SECTION_SIZE);

Why not PMD_SIZE? We don't need 4MB round down with LPAE.

No reason - PMD_SIZE would be good there.

Ok, so does that mean that Mark's original patch:

---->8----

arch/arm/mm/mmu.c +3 -3

index 4e6ef89..2ea13f0 100644
@@ -1125,9 +1125,9 @@ void __init sanity_check_meminfo(void)
                         * occurs before any free memory is mapped.
                         */
                        if (!memblock_limit) {
-                               if (!IS_ALIGNED(block_start, SECTION_SIZE))
+                               if (!IS_ALIGNED(block_start, PMD_SIZE))
                                        memblock_limit = block_start;
-                               else if (!IS_ALIGNED(block_end, SECTION_SIZE))
+                               else if (!IS_ALIGNED(block_end, PMD_SIZE))
                                        memblock_limit = arm_lowmem_limit;
                        }
 

@@ -1142,7 +1142,7 @@ void __init sanity_check_meminfo(void)
         * last full section, which should be mapped.
         */
        if (memblock_limit)
-               memblock_limit = round_down(memblock_limit, SECTION_SIZE);
+               memblock_limit = round_down(memblock_limit, PMD_SIZE);
        if (!memblock_limit)
                memblock_limit = arm_lowmem_limit;
 

Is good, or do we only need to have the last chunk of this patch ?

Quoted 29 lines · Hans de Goede

Ok, so does that mean that Mark's original patch:

---->8----

arch/arm/mm/mmu.c +3 -3

index 4e6ef89..2ea13f0 100644
@@ -1125,9 +1125,9 @@ void __init sanity_check_meminfo(void)
                          * occurs before any free memory is mapped.
                          */
                         if (!memblock_limit) {
-                               if (!IS_ALIGNED(block_start, SECTION_SIZE))
+                               if (!IS_ALIGNED(block_start, PMD_SIZE))
                                         memblock_limit = block_start;
-                               else if (!IS_ALIGNED(block_end, SECTION_SIZE))
+                               else if (!IS_ALIGNED(block_end, PMD_SIZE))
                                         memblock_limit = arm_lowmem_limit;
                         }
 

@@ -1142,7 +1142,7 @@ void __init sanity_check_meminfo(void)
          * last full section, which should be mapped.
          */
         if (memblock_limit)
-               memblock_limit = round_down(memblock_limit, SECTION_SIZE);
+               memblock_limit = round_down(memblock_limit, PMD_SIZE);
         if (!memblock_limit)
                 memblock_limit = arm_lowmem_limit;
 

Quoted 1 line

Is good, or do we only need to have the last chunk of this patch ?

That should do it, thanks.

Hi,

Quoted 30 lines · Russell King - ARM Linux

Quoted 29 lines · Hans de Goede

Ok, so does that mean that Mark's original patch:

---->8----

arch/arm/mm/mmu.c +3 -3

index 4e6ef89..2ea13f0 100644
@@ -1125,9 +1125,9 @@ void __init sanity_check_meminfo(void)
                         * occurs before any free memory is mapped.
                         */
                        if (!memblock_limit) {
-                               if (!IS_ALIGNED(block_start, SECTION_SIZE))
+                               if (!IS_ALIGNED(block_start, PMD_SIZE))
                                        memblock_limit = block_start;
-                               else if (!IS_ALIGNED(block_end, SECTION_SIZE))
+                               else if (!IS_ALIGNED(block_end, PMD_SIZE))
                                        memblock_limit = arm_lowmem_limit;
                        }
 

@@ -1142,7 +1142,7 @@ void __init sanity_check_meminfo(void)
         * last full section, which should be mapped.
         */
        if (memblock_limit)
-               memblock_limit = round_down(memblock_limit, SECTION_SIZE);
+               memblock_limit = round_down(memblock_limit, PMD_SIZE);
        if (!memblock_limit)
                memblock_limit = arm_lowmem_limit;
 

Quoted 3 lines

Quoted 1 line

Is good, or do we only need to have the last chunk of this patch ?

That should do it, thanks.

"that should do it" means the entire patch or only the last chunk?

Quoted 33 lines · Hans de Goede

Hi,

Quoted 30 lines · Russell King - ARM Linux

Quoted 29 lines · Hans de Goede

Ok, so does that mean that Mark's original patch:

---->8----

arch/arm/mm/mmu.c +3 -3

index 4e6ef89..2ea13f0 100644
@@ -1125,9 +1125,9 @@ void __init sanity_check_meminfo(void)
                          * occurs before any free memory is mapped.
                          */
                         if (!memblock_limit) {
-                               if (!IS_ALIGNED(block_start, SECTION_SIZE))
+                               if (!IS_ALIGNED(block_start, PMD_SIZE))
                                         memblock_limit = block_start;
-                               else if (!IS_ALIGNED(block_end, SECTION_SIZE))
+                               else if (!IS_ALIGNED(block_end, PMD_SIZE))
                                         memblock_limit = arm_lowmem_limit;
                         }
 

@@ -1142,7 +1142,7 @@ void __init sanity_check_meminfo(void)
          * last full section, which should be mapped.
          */
         if (memblock_limit)
-               memblock_limit = round_down(memblock_limit, SECTION_SIZE);
+               memblock_limit = round_down(memblock_limit, PMD_SIZE);
         if (!memblock_limit)
                 memblock_limit = arm_lowmem_limit;
 

Quoted 5 lines

Quoted 3 lines

Quoted 1 line

Is good, or do we only need to have the last chunk of this patch ?

That should do it, thanks.

"that should do it" means the entire patch or only the last chunk?

Entire patch.

At boot time we round the memblock limit down to section size in an attempt to ensure that we will have mapped this RAM with section mappings prior to allocating from it. When mapping RAM we iterate over PMD-sized chunks, creating these section mappings.

Section mappings are only created when the end of a chunk is aligned to section size. Unfortunately, with classic page tables (where PMD_SIZE is 2 * SECTION_SIZE) this means that if a chunk is between 1M and 2M in size the first 1M will not be mapped despite having been accounted for in the memblock limit. This has been observed to result in page tables being allocated from unmapped memory, causing boot-time hangs.

This patch modifies the memblock limit rounding to always round down to PMD_SIZE instead of SECTION_SIZE. For classic MMU this means that we will round the memblock limit down to a 2M boundary, matching the limits on section mappings, and preventing allocations from unmapped memory. For LPAE there should be no change as PMD_SIZE == SECTION_SIZE.

Signed-off-by Mark Rutland <[email protected]>

Reported-by Stefan Agner <[email protected]>

Cc Catalin Marinas <[email protected]>

Cc Hans de Goede <[email protected]>

Cc Laura Abbott <[email protected]>

Cc Russell King <[email protected]>

Cc Steve Capper <[email protected]>

---
 arch/arm/mm/mmu.c | 20 ++++++++++----------
 1 file changed, 10 insertions(+), 10 deletions(-)

arch/arm/mm/mmu.c +10 -10

index 4e6ef89..7186382 100644
@@ -1112,22 +1112,22 @@ void __init sanity_check_meminfo(void)
 			}
 
 			/*
-			 * Find the first non-section-aligned page, and point
+			 * Find the first non-pmd-aligned page, and point
 			 * memblock_limit at it. This relies on rounding the
-			 * limit down to be section-aligned, which happens at
-			 * the end of this function.
+			 * limit down to be pmd-aligned, which happens at the
+			 * end of this function.
 			 *
 			 * With this algorithm, the start or end of almost any
-			 * bank can be non-section-aligned. The only exception
-			 * is that the start of the bank 0 must be section-
+			 * bank can be non-pmd-aligned. The only exception is
+			 * that the start of the bank 0 must be section-
 			 * aligned, since otherwise memory would need to be
 			 * allocated when mapping the start of bank 0, which
 			 * occurs before any free memory is mapped.
 			 */
 			if (!memblock_limit) {
-				if (!IS_ALIGNED(block_start, SECTION_SIZE))
+				if (!IS_ALIGNED(block_start, PMD_SIZE))
 					memblock_limit = block_start;
-				else if (!IS_ALIGNED(block_end, SECTION_SIZE))
+				else if (!IS_ALIGNED(block_end, PMD_SIZE))
 					memblock_limit = arm_lowmem_limit;
 			}
 

@@ -1137,12 +1137,12 @@ void __init sanity_check_meminfo(void)
 	high_memory = __va(arm_lowmem_limit - 1) + 1;
 
 	/*
-	 * Round the memblock limit down to a section size.  This
+	 * Round the memblock limit down to a pmd size.  This
 	 * helps to ensure that we will allocate memory from the
-	 * last full section, which should be mapped.
+	 * last full pmd, which should be mapped.
 	 */
 	if (memblock_limit)
-		memblock_limit = round_down(memblock_limit, SECTION_SIZE);
+		memblock_limit = round_down(memblock_limit, PMD_SIZE);
 	if (!memblock_limit)
 		memblock_limit = arm_lowmem_limit;

Quoted 12 lines · Mark Rutland

At boot time we round the memblock limit down to section size in an attempt to ensure that we will have mapped this RAM with section mappings prior to allocating from it. When mapping RAM we iterate over PMD-sized chunks, creating these section mappings.

Section mappings are only created when the end of a chunk is aligned to section size. Unfortunately, with classic page tables (where PMD_SIZE is 2 * SECTION_SIZE) this means that if a chunk is between 1M and 2M in size the first 1M will not be mapped despite having been accounted for in the memblock limit. This has been observed to result in page tables being allocated from unmapped memory, causing boot-time hangs.

Quoted 5 lines

This patch modifies the memblock limit rounding to always round down to PMD_SIZE instead of SECTION_SIZE. For classic MMU this means that we will round the memblock limit down to a 2M boundary, matching the limits on section mappings, and preventing allocations from unmapped memory. For LPAE there should be no change as PMD_SIZE == SECTION_SIZE.

Thanks Mark, just tested that patch on the hardware I had the issue, looks good.

Tested-by Stefan Agner <[email protected]>

Quoted 62 lines

Signed-off-by Mark Rutland <[email protected]>

Reported-by Stefan Agner <[email protected]>

Cc Catalin Marinas <[email protected]>

Cc Hans de Goede <[email protected]>

Cc Laura Abbott <[email protected]>

Cc Russell King <[email protected]>

Cc Steve Capper <[email protected]>

---
  arch/arm/mm/mmu.c | 20 ++++++++++----------
  1 file changed, 10 insertions(+), 10 deletions(-)

arch/arm/mm/mmu.c +10 -10

index 4e6ef89..7186382 100644
@@ -1112,22 +1112,22 @@ void __init sanity_check_meminfo(void)
  			}
 
  			/*
-			 * Find the first non-section-aligned page, and point
+			 * Find the first non-pmd-aligned page, and point
  			 * memblock_limit at it. This relies on rounding the
-			 * limit down to be section-aligned, which happens at
-			 * the end of this function.
+			 * limit down to be pmd-aligned, which happens at the
+			 * end of this function.
  			 *
  			 * With this algorithm, the start or end of almost any
-			 * bank can be non-section-aligned. The only exception
-			 * is that the start of the bank 0 must be section-
+			 * bank can be non-pmd-aligned. The only exception is
+			 * that the start of the bank 0 must be section-
  			 * aligned, since otherwise memory would need to be
  			 * allocated when mapping the start of bank 0, which
  			 * occurs before any free memory is mapped.
  			 */
  			if (!memblock_limit) {
-				if (!IS_ALIGNED(block_start, SECTION_SIZE))
+				if (!IS_ALIGNED(block_start, PMD_SIZE))
  					memblock_limit = block_start;
-				else if (!IS_ALIGNED(block_end, SECTION_SIZE))
+				else if (!IS_ALIGNED(block_end, PMD_SIZE))
  					memblock_limit = arm_lowmem_limit;
  			}
 

@@ -1137,12 +1137,12 @@ void __init sanity_check_meminfo(void)
  	high_memory = __va(arm_lowmem_limit - 1) + 1;
 
  	/*
-	 * Round the memblock limit down to a section size.  This
+	 * Round the memblock limit down to a pmd size.  This
  	 * helps to ensure that we will allocate memory from the
-	 * last full section, which should be mapped.
+	 * last full pmd, which should be mapped.
  	 */
  	if (memblock_limit)
-		memblock_limit = round_down(memblock_limit, SECTION_SIZE);
+		memblock_limit = round_down(memblock_limit, PMD_SIZE);
  	if (!memblock_limit)
  		memblock_limit = arm_lowmem_limit;

Hi,

Quoted 22 lines · Stefan Agner

Quoted 12 lines · Mark Rutland

At boot time we round the memblock limit down to section size in an attempt to ensure that we will have mapped this RAM with section mappings prior to allocating from it. When mapping RAM we iterate over PMD-sized chunks, creating these section mappings.

Section mappings are only created when the end of a chunk is aligned to section size. Unfortunately, with classic page tables (where PMD_SIZE is 2 * SECTION_SIZE) this means that if a chunk is between 1M and 2M in size the first 1M will not be mapped despite having been accounted for in the memblock limit. This has been observed to result in page tables being allocated from unmapped memory, causing boot-time hangs.

Quoted 5 lines

This patch modifies the memblock limit rounding to always round down to PMD_SIZE instead of SECTION_SIZE. For classic MMU this means that we will round the memblock limit down to a 2M boundary, matching the limits on section mappings, and preventing allocations from unmapped memory. For LPAE there should be no change as PMD_SIZE == SECTION_SIZE.

Thanks Mark, just tested that patch on the hardware I had the issue, looks good.

Quoted 1 line

Tested-by Stefan Agner <[email protected]>

Same for me, this also fixes the issue I was seeing no an Allwinner A33 tablet with 1024z600 lcd screen.

Tested-by Hans de Goede <[email protected]>

Can we get this Cc-ed to stable at vger.kernel.org please? At least for 4.0 ?

Regards,

Hans

Quoted 63 lines

Quoted 62 lines

Signed-off-by Mark Rutland <[email protected]>

Reported-by Stefan Agner <[email protected]>

Cc Catalin Marinas <[email protected]>

Cc Hans de Goede <[email protected]>

Cc Laura Abbott <[email protected]>

Cc Russell King <[email protected]>

Cc Steve Capper <[email protected]>

---
  arch/arm/mm/mmu.c | 20 ++++++++++----------
  1 file changed, 10 insertions(+), 10 deletions(-)

arch/arm/mm/mmu.c +10 -10

index 4e6ef89..7186382 100644
@@ -1112,22 +1112,22 @@ void __init sanity_check_meminfo(void)
  			}
 
  			/*
-			 * Find the first non-section-aligned page, and point
+			 * Find the first non-pmd-aligned page, and point
  			 * memblock_limit at it. This relies on rounding the
-			 * limit down to be section-aligned, which happens at
-			 * the end of this function.
+			 * limit down to be pmd-aligned, which happens at the
+			 * end of this function.
  			 *
  			 * With this algorithm, the start or end of almost any
-			 * bank can be non-section-aligned. The only exception
-			 * is that the start of the bank 0 must be section-
+			 * bank can be non-pmd-aligned. The only exception is
+			 * that the start of the bank 0 must be section-
  			 * aligned, since otherwise memory would need to be
  			 * allocated when mapping the start of bank 0, which
  			 * occurs before any free memory is mapped.
  			 */
  			if (!memblock_limit) {
-				if (!IS_ALIGNED(block_start, SECTION_SIZE))
+				if (!IS_ALIGNED(block_start, PMD_SIZE))
  					memblock_limit = block_start;
-				else if (!IS_ALIGNED(block_end, SECTION_SIZE))
+				else if (!IS_ALIGNED(block_end, PMD_SIZE))
  					memblock_limit = arm_lowmem_limit;
  			}
 

@@ -1137,12 +1137,12 @@ void __init sanity_check_meminfo(void)
  	high_memory = __va(arm_lowmem_limit - 1) + 1;
 
  	/*
-	 * Round the memblock limit down to a section size.  This
+	 * Round the memblock limit down to a pmd size.  This
  	 * helps to ensure that we will allocate memory from the
-	 * last full section, which should be mapped.
+	 * last full pmd, which should be mapped.
  	 */
  	if (memblock_limit)
-		memblock_limit = round_down(memblock_limit, SECTION_SIZE);
+		memblock_limit = round_down(memblock_limit, PMD_SIZE);
  	if (!memblock_limit)
  		memblock_limit = arm_lowmem_limit;

Russell, on the assumption you're ok with the patch I've submitted it to the patch system as 8356/1.

At boot time we round the memblock limit down to section size in an attempt to ensure that we will have mapped this RAM with section mappings prior to allocating from it. When mapping RAM we iterate over PMD-sized chunks, creating these section mappings.

Section mappings are only created when the end of a chunk is aligned to section size. Unfortunately, with classic page tables (where PMD_SIZE is 2 * SECTION_SIZE) this means that if a chunk is between 1M and 2M in size the first 1M will not be mapped despite having been accounted for in the memblock limit. This has been observed to result in page tables being allocated from unmapped memory, causing boot-time hangs.

This patch modifies the memblock limit rounding to always round down to PMD_SIZE instead of SECTION_SIZE. For classic MMU this means that we will round the memblock limit down to a 2M boundary, matching the limits on section mappings, and preventing allocations from unmapped memory. For LPAE there should be no change as PMD_SIZE == SECTION_SIZE.

Signed-off-by Mark Rutland <[email protected]>

Reported-by Stefan Agner <[email protected]>

Cc Catalin Marinas <[email protected]>

Cc Hans de Goede <[email protected]>

Cc Laura Abbott <[email protected]>

Cc Russell King <[email protected]>

Cc Steve Capper <[email protected]>

---
  arch/arm/mm/mmu.c | 20 ++++++++++----------
  1 file changed, 10 insertions(+), 10 deletions(-)

arch/arm/mm/mmu.c +10 -10

index 4e6ef89..7186382 100644
@@ -1112,22 +1112,22 @@ void __init sanity_check_meminfo(void)
  			}
 
  			/*
-			 * Find the first non-section-aligned page, and point
+			 * Find the first non-pmd-aligned page, and point
  			 * memblock_limit at it. This relies on rounding the
-			 * limit down to be section-aligned, which happens at
-			 * the end of this function.
+			 * limit down to be pmd-aligned, which happens at the
+			 * end of this function.
  			 *
  			 * With this algorithm, the start or end of almost any
-			 * bank can be non-section-aligned. The only exception
-			 * is that the start of the bank 0 must be section-
+			 * bank can be non-pmd-aligned. The only exception is
+			 * that the start of the bank 0 must be section-
  			 * aligned, since otherwise memory would need to be
  			 * allocated when mapping the start of bank 0, which
  			 * occurs before any free memory is mapped.
  			 */
  			if (!memblock_limit) {
-				if (!IS_ALIGNED(block_start, SECTION_SIZE))
+				if (!IS_ALIGNED(block_start, PMD_SIZE))
  					memblock_limit = block_start;
-				else if (!IS_ALIGNED(block_end, SECTION_SIZE))
+				else if (!IS_ALIGNED(block_end, PMD_SIZE))
  					memblock_limit = arm_lowmem_limit;
  			}
 

@@ -1137,12 +1137,12 @@ void __init sanity_check_meminfo(void)
  	high_memory = __va(arm_lowmem_limit - 1) + 1;
 
  	/*
-	 * Round the memblock limit down to a section size.  This
+	 * Round the memblock limit down to a pmd size.  This
  	 * helps to ensure that we will allocate memory from the
-	 * last full section, which should be mapped.
+	 * last full pmd, which should be mapped.
  	 */
  	if (memblock_limit)
-		memblock_limit = round_down(memblock_limit, SECTION_SIZE);
+		memblock_limit = round_down(memblock_limit, PMD_SIZE);
  	if (!memblock_limit)
  		memblock_limit = arm_lowmem_limit;
 

At boot time we round the memblock limit down to section size in an attempt to ensure that we will have mapped this RAM with section mappings prior to allocating from it. When mapping RAM we iterate over PMD-sized chunks, creating these section mappings.

Section mappings are only created when the end of a chunk is aligned to section size. Unfortunately, with classic page tables (where PMD_SIZE is 2 * SECTION_SIZE) this means that if a chunk is between 1M and 2M in size the first 1M will not be mapped despite having been accounted for in the memblock limit. This has been observed to result in page tables being allocated from unmapped memory, causing boot-time hangs.

This patch modifies the memblock limit rounding to always round down to PMD_SIZE instead of SECTION_SIZE. For classic MMU this means that we will round the memblock limit down to a 2M boundary, matching the limits on section mappings, and preventing allocations from unmapped memory. For LPAE there should be no change as PMD_SIZE == SECTION_SIZE.

Signed-off-by Mark Rutland <[email protected]>

Reported-by Stefan Agner <[email protected]>

Cc Catalin Marinas <[email protected]>

Cc Hans de Goede <[email protected]>

Cc Laura Abbott <[email protected]>

Cc Russell King <[email protected]>

Cc Steve Capper <[email protected]>

---
 arch/arm/mm/mmu.c | 20 ++++++++++----------
 1 file changed, 10 insertions(+), 10 deletions(-)

arch/arm/mm/mmu.c +8 -8

index 4e6ef89..7186382 100644
@@ -1112,22 +1112,22 @@ void __init sanity_check_meminfo(void)
                        }
 
                        /*
-                        * Find the first non-section-aligned page, and point
+                        * Find the first non-pmd-aligned page, and point
                         * memblock_limit at it. This relies on rounding the
-                        * limit down to be section-aligned, which happens at
-                        * the end of this function.
+                        * limit down to be pmd-aligned, which happens at the
+                        * end of this function.
                         *
                         * With this algorithm, the start or end of almost any
-                        * bank can be non-section-aligned. The only exception
-                        * is that the start of the bank 0 must be section-
+                        * bank can be non-pmd-aligned. The only exception is
+                        * that the start of the bank 0 must be section-
                         * aligned, since otherwise memory would need to be
                         * allocated when mapping the start of bank 0, which
                         * occurs before any free memory is mapped.
                         */
                        if (!memblock_limit) {
-                               if (!IS_ALIGNED(block_start, SECTION_SIZE))
+                               if (!IS_ALIGNED(block_start, PMD_SIZE))
                                        memblock_limit = block_start;
-                               else if (!IS_ALIGNED(block_end, SECTION_SIZE))
+                               else if (!IS_ALIGNED(block_end, PMD_SIZE))
                                        memblock_limit = arm_lowmem_limit;
                        }
 

@@ -1137,12 +1137,12 @@ void __init sanity_check_meminfo(void)
        high_memory = __va(arm_lowmem_limit - 1) + 1;
 
        /*
-        * Round the memblock limit down to a section size.  This
+        * Round the memblock limit down to a pmd size.  This
         * helps to ensure that we will allocate memory from the

There were reports of 4.1 not booting on the Exynos Chromebooks. I did a bisect on an Exynos5420 Peach Pit Chromebook and tracked it down to $subject as the cause. Reverting this commit makes it to boot again.