Skip to contentSkip to conversation

n900 in next-20170901

DISCUSSION Pavel Machek 7 participants 59 messages Sep 3, 2017 – Nov 15, 2017

Subsystems (7)
lore
Share & tools
lore
#0 lore raw

Hi!

It compiles ok, but it hangs on boot; black screen, so sometime before display is initialized.
Pavel

Signature · 9 lines
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20170903/0fa9bc6e/attachment.sig>
maintainer
#1 lore raw
Quoted 4 lines · Pavel Machek

Hi!

It compiles ok, but it hangs on boot; black screen, so sometime before display is initialized.

Thanks for reporting it. Based on git bisect, the regression causing commit is 9caf25f996e8 ("mm/cma: manage the memory of the CMA area by using the ZONE_MOVABLE"). With this path applied, booting hangs with an error in omap3_save_secure_ram_context() after a call to _omap_save_secure_sram() that calls the related assembly code save_secure_ram_context.

However, looks like there is also some other commit causing issue.

Just reverting 9caf25f996e8 on Linux next causes the oops below.

Anybody got ideas why this now happens?

Regards,

Tony

8< --------------------

Unable to handle kernel paging request at virtual address ce800000
pgd = c0004000 [ce800000] *pgd=00000000
Internal error: Oops: 805 [#1] SMP ARM
Modules linked in:
CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.13.0-next-20170905+ #662
Hardware name: Nokia RX-51 board
task: ce0a0040 task.stack: ce0a4000
PC is at __memzero+0x24/0x7c
LR is@0x0
pc : [<c084fa84>]    lr : [<00000000>]    psr: 20000013

sp : ce0a5e84 ip : 00000000 fp : c0c005a8
r10: 00040000 r9 : cfc95000 r8 : 00000247
r7 : ce0a5ef4 r6 : 00000000 r5 : 00000001 r4 : ce800000 r3 : 00000000 r2 : 00000000 r1 : 0003ffc0 r0 : ce800000
Flags: nzCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment none
Control: 10c5387d Table: 80004019 DAC: 00000051 Process swapper/0 (pid: 1, stack limit = 0xce0a4218)

Stack: (0xce0a5e84 to 0xce0a6000)

5e80: c0116718 00000040 00000006 c01a1470 00040000 00000001 00000040
5ea0: ce0a5ef4 00000247 cfc95000 00000000 c0c005a8 c0116844 c0dce000 c01a1484 5ec0: 00000247 c0d0e2e0 c0dce29c c0b5cbbc c0dce000 00000003 c0c5389c c0c06c54 5ee0: c0c06bd8 00000001 00000000 014000c0 00000000 c0b5cbbc ffffe000 c0c06bd8 5f00: 00000000 c0101ef8 000000aa 00000000 cfdfcbdb cfdfcbe7 c0b5d904 000000aa 5f20: 000000aa c015cf54 c0b5cbbc 00000000 00000002 00000002 cfdfcbe7 cfdfcbec 5f40: c0c6c59c 00000002 c0dce000 c0c53880 c0c6c66c c0dce000 c0c53884 c0dce000 5f60: 00000003 c0c00ecc 00000002 00000002 00000000 c0c005a8 c0864f3c 000000aa 5f80: 00000000 00000000 c0864f3c 00000000 00000000 00000000 00000000 00000000 5fa0: 00000000 c0864f44 00000000 c0107e98 00000000 00000000 00000000 00000000 5fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 5fe0: 00000000 00000000 00000000 00000000 00000013 00000000 ffffffff ffffffff [<c084fa84>] (__memzero) from [<c0116718>] (__dma_clear_buffer+0x140/0x154) [<c0116718>] (__dma_clear_buffer) from [<c0116844>] (__alloc_from_contiguous+0x50/0xdc) [<c0116844>] (__alloc_from_contiguous) from [<c0c06c54>] (atomic_pool_init+0x7c/0x178) [<c0c06c54>] (atomic_pool_init) from [<c0101ef8>] (do_one_initcall+0x3c/0x170) [<c0101ef8>] (do_one_initcall) from [<c0c00ecc>] (kernel_init_freeable+0x1fc/0x2c4) [<c0c00ecc>] (kernel_init_freeable) from [<c0864f44>] (kernel_init+0x8/0x110) [<c0864f44>] (kernel_init) from [<c0107e98>] (ret_from_fork+0x14/0x3c)

Code: e52de004 e1a0c002 e1a0e002 e2511040 (a8a0500c)
#2 lore raw
Quoted 13 lines · Tony Lindgren
Quoted 4 lines · Pavel Machek

Hi!

It compiles ok, but it hangs on boot; black screen, so sometime before display is initialized.

Thanks for reporting it. Based on git bisect, the regression causing commit is 9caf25f996e8 ("mm/cma: manage the memory of the CMA area by using the ZONE_MOVABLE"). With this path applied, booting hangs with an error in omap3_save_secure_ram_context() after a call to _omap_save_secure_sram() that calls the related assembly code save_secure_ram_context.

Quoted 5 lines

However, looks like there is also some other commit causing issue.

Just reverting 9caf25f996e8 on Linux next causes the oops below.

Anybody got ideas why this now happens?

I know that there are other two patches from the same series depending on the one you reverted:

mm/cma: remove ALLOC_CMA
ARM: CMA: avoid double mapping to the CMA area if CONFIG_HIGHMEM = y

I'm guessing you need to revert all to have something consistent to test.

Vlastimil

Quoted 44 lines

Regards,

Tony

8< --------------------

Unable to handle kernel paging request at virtual address ce800000
pgd = c0004000 [ce800000] *pgd=00000000
Internal error: Oops: 805 [#1] SMP ARM
Modules linked in:
CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.13.0-next-20170905+ #662
Hardware name: Nokia RX-51 board
task: ce0a0040 task.stack: ce0a4000
PC is at __memzero+0x24/0x7c
LR is at 0x0
pc : [<c084fa84>]    lr : [<00000000>]    psr: 20000013

sp : ce0a5e84 ip : 00000000 fp : c0c005a8
r10: 00040000 r9 : cfc95000 r8 : 00000247
r7 : ce0a5ef4 r6 : 00000000 r5 : 00000001 r4 : ce800000 r3 : 00000000 r2 : 00000000 r1 : 0003ffc0 r0 : ce800000
Flags: nzCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment none
Control: 10c5387d Table: 80004019 DAC: 00000051 Process swapper/0 (pid: 1, stack limit = 0xce0a4218)

Stack: (0xce0a5e84 to 0xce0a6000)

5e80: c0116718 00000040 00000006 c01a1470 00040000 00000001 00000040
5ea0: ce0a5ef4 00000247 cfc95000 00000000 c0c005a8 c0116844 c0dce000 c01a1484 5ec0: 00000247 c0d0e2e0 c0dce29c c0b5cbbc c0dce000 00000003 c0c5389c c0c06c54 5ee0: c0c06bd8 00000001 00000000 014000c0 00000000 c0b5cbbc ffffe000 c0c06bd8 5f00: 00000000 c0101ef8 000000aa 00000000 cfdfcbdb cfdfcbe7 c0b5d904 000000aa 5f20: 000000aa c015cf54 c0b5cbbc 00000000 00000002 00000002 cfdfcbe7 cfdfcbec 5f40: c0c6c59c 00000002 c0dce000 c0c53880 c0c6c66c c0dce000 c0c53884 c0dce000 5f60: 00000003 c0c00ecc 00000002 00000002 00000000 c0c005a8 c0864f3c 000000aa 5f80: 00000000 00000000 c0864f3c 00000000 00000000 00000000 00000000 00000000 5fa0: 00000000 c0864f44 00000000 c0107e98 00000000 00000000 00000000 00000000 5fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 5fe0: 00000000 00000000 00000000 00000000 00000013 00000000 ffffffff ffffffff [<c084fa84>] (__memzero) from [<c0116718>] (__dma_clear_buffer+0x140/0x154) [<c0116718>] (__dma_clear_buffer) from [<c0116844>] (__alloc_from_contiguous+0x50/0xdc) [<c0116844>] (__alloc_from_contiguous) from [<c0c06c54>] (atomic_pool_init+0x7c/0x178) [<c0c06c54>] (atomic_pool_init) from [<c0101ef8>] (do_one_initcall+0x3c/0x170) [<c0101ef8>] (do_one_initcall) from [<c0c00ecc>] (kernel_init_freeable+0x1fc/0x2c4) [<c0c00ecc>] (kernel_init_freeable) from [<c0864f44>] (kernel_init+0x8/0x110) [<c0864f44>] (kernel_init) from [<c0107e98>] (ret_from_fork+0x14/0x3c)

Code: e52de004 e1a0c002 e1a0e002 e2511040 (a8a0500c)
maintainer
#3 lore raw
Quoted 20 lines · Vlastimil Babka
Quoted 13 lines · Tony Lindgren
Quoted 4 lines · Pavel Machek

Hi!

It compiles ok, but it hangs on boot; black screen, so sometime before display is initialized.

Thanks for reporting it. Based on git bisect, the regression causing commit is 9caf25f996e8 ("mm/cma: manage the memory of the CMA area by using the ZONE_MOVABLE"). With this path applied, booting hangs with an error in omap3_save_secure_ram_context() after a call to _omap_save_secure_sram() that calls the related assembly code save_secure_ram_context.

Quoted 5 lines

However, looks like there is also some other commit causing issue.

Just reverting 9caf25f996e8 on Linux next causes the oops below.

Anybody got ideas why this now happens?

Quoted 7 lines

I know that there are other two patches from the same series depending on the one you reverted:

mm/cma: remove ALLOC_CMA
ARM: CMA: avoid double mapping to the CMA area if CONFIG_HIGHMEM = y

I'm guessing you need to revert all to have something consistent to test.

Thanks, yes reverting all three in next makes things boot for me again.

Regards,

Tony

#4 lore raw
In reply to Tony Lindgren
Quoted 61 lines · Tony Lindgren
Quoted 4 lines · Pavel Machek

Hi!

It compiles ok, but it hangs on boot; black screen, so sometime before display is initialized.

Thanks for reporting it. Based on git bisect, the regression causing commit is 9caf25f996e8 ("mm/cma: manage the memory of the CMA area by using the ZONE_MOVABLE"). With this path applied, booting hangs with an error in omap3_save_secure_ram_context() after a call to _omap_save_secure_sram() that calls the related assembly code save_secure_ram_context.

However, looks like there is also some other commit causing issue.

Just reverting 9caf25f996e8 on Linux next causes the oops below.

Anybody got ideas why this now happens?

Regards,

Tony

8< --------------------

Unable to handle kernel paging request at virtual address ce800000
pgd = c0004000 [ce800000] *pgd=00000000
Internal error: Oops: 805 [#1] SMP ARM
Modules linked in:
CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.13.0-next-20170905+ #662
Hardware name: Nokia RX-51 board
task: ce0a0040 task.stack: ce0a4000
PC is at __memzero+0x24/0x7c
LR is at 0x0
pc : [<c084fa84>]    lr : [<00000000>]    psr: 20000013

sp : ce0a5e84 ip : 00000000 fp : c0c005a8
r10: 00040000 r9 : cfc95000 r8 : 00000247
r7 : ce0a5ef4 r6 : 00000000 r5 : 00000001 r4 : ce800000 r3 : 00000000 r2 : 00000000 r1 : 0003ffc0 r0 : ce800000
Flags: nzCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment none
Control: 10c5387d Table: 80004019 DAC: 00000051 Process swapper/0 (pid: 1, stack limit = 0xce0a4218)

Stack: (0xce0a5e84 to 0xce0a6000)

5e80: c0116718 00000040 00000006 c01a1470 00040000 00000001 00000040
5ea0: ce0a5ef4 00000247 cfc95000 00000000 c0c005a8 c0116844 c0dce000 c01a1484 5ec0: 00000247 c0d0e2e0 c0dce29c c0b5cbbc c0dce000 00000003 c0c5389c c0c06c54 5ee0: c0c06bd8 00000001 00000000 014000c0 00000000 c0b5cbbc ffffe000 c0c06bd8 5f00: 00000000 c0101ef8 000000aa 00000000 cfdfcbdb cfdfcbe7 c0b5d904 000000aa 5f20: 000000aa c015cf54 c0b5cbbc 00000000 00000002 00000002 cfdfcbe7 cfdfcbec 5f40: c0c6c59c 00000002 c0dce000 c0c53880 c0c6c66c c0dce000 c0c53884 c0dce000 5f60: 00000003 c0c00ecc 00000002 00000002 00000000 c0c005a8 c0864f3c 000000aa 5f80: 00000000 00000000 c0864f3c 00000000 00000000 00000000 00000000 00000000 5fa0: 00000000 c0864f44 00000000 c0107e98 00000000 00000000 00000000 00000000 5fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 5fe0: 00000000 00000000 00000000 00000000 00000013 00000000 ffffffff ffffffff [<c084fa84>] (__memzero) from [<c0116718>] (__dma_clear_buffer+0x140/0x154) [<c0116718>] (__dma_clear_buffer) from [<c0116844>] (__alloc_from_contiguous+0x50/0xdc) [<c0116844>] (__alloc_from_contiguous) from [<c0c06c54>] (atomic_pool_init+0x7c/0x178) [<c0c06c54>] (atomic_pool_init) from [<c0101ef8>] (do_one_initcall+0x3c/0x170) [<c0101ef8>] (do_one_initcall) from [<c0c00ecc>] (kernel_init_freeable+0x1fc/0x2c4) [<c0c00ecc>] (kernel_init_freeable) from [<c0864f44>] (kernel_init+0x8/0x110) [<c0864f44>] (kernel_init) from [<c0107e98>] (ret_from_fork+0x14/0x3c)

Quoted 1 line
Code: e52de004 e1a0c002 e1a0e002 e2511040 (a8a0500c)

Hello,

I think that I made a mistake for configuration CONFIG_HIGHMEM=y and CONFIG_HAVE_MEMBLOCK_NODE_MAP=y. In this case, the MOVABLE_ZONE can be *!highmem*. Could you check that your configuration have above options?

And, could you check that following patch works for you?

Thanks.

------------>8-----------------

arch/arm/mm/dma-mapping.c +1 -1
index 38f0fde..4c39c92 100644
@@ -518,7 +518,7 @@ void __init dma_contiguous_remap(void)
                 * considered as highmem even if it's physical address belong
                 * to lowmem. Therefore, re-mapping isn't required.
                 */
-               if (!IS_ENABLED(CONFIG_HIGHMEM))
+               if (!is_highmem_idx(ZONE_MOVABLE))
                        iotable_init(&map, 1);
        }
 }
maintainer
#5 lore raw

Hi,

Quoted 4 lines · Joonsoo Kim

I think that I made a mistake for configuration CONFIG_HIGHMEM=y and CONFIG_HAVE_MEMBLOCK_NODE_MAP=y. In this case, the MOVABLE_ZONE can be *!highmem*. Could you check that your configuration have above options?

CONFIG_HIGHMEM is set yeah.

Quoted 1 line

And, could you check that following patch works for you?

Does not seem to help, tried against next with just 9caf25f996e8 revert and also with 9caf25f996e8.

Regards,

Tony

Quoted 15 lines

------------>8-----------------

arch/arm/mm/dma-mapping.c +1 -1
index 38f0fde..4c39c92 100644
@@ -518,7 +518,7 @@ void __init dma_contiguous_remap(void)
                 * considered as highmem even if it's physical address belong
                 * to lowmem. Therefore, re-mapping isn't required.
                 */
-               if (!IS_ENABLED(CONFIG_HIGHMEM))
+               if (!is_highmem_idx(ZONE_MOVABLE))
                        iotable_init(&map, 1);
        }
 }
#6 lore raw
Quoted 8 lines · Tony Lindgren

Hi,

Quoted 4 lines · Joonsoo Kim

I think that I made a mistake for configuration CONFIG_HIGHMEM=y and CONFIG_HAVE_MEMBLOCK_NODE_MAP=y. In this case, the MOVABLE_ZONE can be *!highmem*. Could you check that your configuration have above options?

Quoted 6 lines

CONFIG_HIGHMEM is set yeah.

Quoted 1 line

And, could you check that following patch works for you?

Does not seem to help, tried against next with just 9caf25f996e8 revert and also with 9caf25f996e8.

Oops. I misunderstood your problem. Could you test with CONFIG_DEBUG_VIRTUAL?

After commit 9caf25f996e8, user for CMA memory should use to check PageHighmem in order to get proper virtual address of the page. If someone doesn't use it, it is possible to use wrong virtual address and it then causes the use of wrong physical address. CONFIG_DEBUG_VIRTUAL would catch this case.

If it doesn't help, is there a way to test n900 configuration in QEMU?

Thanks.

maintainer
#7 lore raw
Quoted 11 lines · Joonsoo Kim
Quoted 10 lines · Tony Lindgren

Hi,

Quoted 4 lines · Joonsoo Kim

I think that I made a mistake for configuration CONFIG_HIGHMEM=y and CONFIG_HAVE_MEMBLOCK_NODE_MAP=y. In this case, the MOVABLE_ZONE can be *!highmem*. Could you check that your configuration have above options?

CONFIG_HIGHMEM is set yeah.

Quoted 7 lines
Quoted 4 lines
Quoted 1 line

And, could you check that following patch works for you?

Does not seem to help, tried against next with just 9caf25f996e8 revert and also with 9caf25f996e8.

Oops. I misunderstood your problem. Could you test with CONFIG_DEBUG_VIRTUAL?

Sure.

Quoted 5 lines

After commit 9caf25f996e8, user for CMA memory should use to check PageHighmem in order to get proper virtual address of the page. If someone doesn't use it, it is possible to use wrong virtual address and it then causes the use of wrong physical address. CONFIG_DEBUG_VIRTUAL would catch this case.

OK, no extra output of current next with CONFIG_DEBUG_VIRTUAL=y. Booting of n900 hangs with just the same error:

save_secure_sram() returns 0000ff02

Quoted 1 line

If it doesn't help, is there a way to test n900 configuration in QEMU?

I doubt that QEMU n900 boots in secure mode but instead shows the SoC as general purpose SoC. If so, you'd have to patch the omap3_save_secure_ram_context() to attempt to save secure RAM context in all cases. If that works then debugging with any omap3 board like beagleboard in QEMU should work.

Regards,

Tony

#8 lore raw
Quoted 33 lines · Tony Lindgren
Quoted 11 lines · Joonsoo Kim
Quoted 10 lines · Tony Lindgren

Hi,

Quoted 4 lines · Joonsoo Kim

I think that I made a mistake for configuration CONFIG_HIGHMEM=y and CONFIG_HAVE_MEMBLOCK_NODE_MAP=y. In this case, the MOVABLE_ZONE can be *!highmem*. Could you check that your configuration have above options?

CONFIG_HIGHMEM is set yeah.

Quoted 7 lines
Quoted 4 lines
Quoted 1 line

And, could you check that following patch works for you?

Does not seem to help, tried against next with just 9caf25f996e8 revert and also with 9caf25f996e8.

Oops. I misunderstood your problem. Could you test with CONFIG_DEBUG_VIRTUAL?

Sure.

Quoted 5 lines

After commit 9caf25f996e8, user for CMA memory should use to check PageHighmem in order to get proper virtual address of the page. If someone doesn't use it, it is possible to use wrong virtual address and it then causes the use of wrong physical address. CONFIG_DEBUG_VIRTUAL would catch this case.

OK, no extra output of current next with CONFIG_DEBUG_VIRTUAL=y. Booting of n900 hangs with just the same error:

save_secure_sram() returns 0000ff02

Quoted 7 lines
Quoted 1 line

If it doesn't help, is there a way to test n900 configuration in QEMU?

I doubt that QEMU n900 boots in secure mode but instead shows the SoC as general purpose SoC. If so, you'd have to patch the omap3_save_secure_ram_context() to attempt to save secure RAM context in all cases. If that works then debugging with any omap3 board like beagleboard in QEMU should work.

Sorry for late response.

I tried to emulate beagle board by using QEMU and now I find the way and it works. However, it doesn't call omap3_save_secure_ram_context() due to different omap_type(). And, even if I call it forcibly, the system dies with prefetch abort regardless of commit 9caf25f996e8.

Could you let me know the better way to test your situation?

Anyway, could you test linux-next with 'CONFIG_HIGHMEM = n'? I'd like to know if the issue is related to the change that all CMA memory is managed like as highmem.

Thanks.

maintainer
#9 lore raw
Quoted 7 lines · Joonsoo Kim
Quoted 5 lines · Tony Lindgren

I doubt that QEMU n900 boots in secure mode but instead shows the SoC as general purpose SoC. If so, you'd have to patch the omap3_save_secure_ram_context() to attempt to save secure RAM context in all cases. If that works then debugging with any omap3 board like beagleboard in QEMU should work.

Quoted 6 lines

Sorry for late response.

I tried to emulate beagle board by using QEMU and now I find the way and it works. However, it doesn't call omap3_save_secure_ram_context() due to different omap_type(). And, even if I call it forcibly, the system dies with prefetch abort regardless of commit 9caf25f996e8.

OK

Quoted 1 line

Could you let me know the better way to test your situation?

Hmm yeah it seems to always return 0x19 on GP devices at least with the following test hack. But maybe that's enough for you to still see some differences with your patches.

Quoted 3 lines

Anyway, could you test linux-next with 'CONFIG_HIGHMEM = n'? I'd like to know if the issue is related to the change that all CMA memory is managed like as highmem.

Yes I disabled CONFIG_HIGHMEM and n900 boots. To disable it, you need to remove it from arch/arm/mach-omap2/Kconfig that selects it if ARCH_OMAP2PLUS_TYPICAL is selected.

Regards,

Tony

8< ------

arch/arm/mach-omap2/pm34xx.c +3 -3
@@ -114,7 +114,7 @@ static void omap3_save_secure_ram_context(void)
 	u32 ret;
 	int mpu_next_state = pwrdm_read_next_pwrst(mpu_pwrdm);
 
-	if (omap_type() != OMAP2_DEVICE_TYPE_GP) {
+	if (1 || omap_type() != OMAP2_DEVICE_TYPE_GP) {
 		/*
 		 * MPU next state must be set to POWER_ON temporarily,
 		 * otherwise the WFI executed inside the ROM code
@@ -440,7 +440,7 @@ void omap_push_sram_idle(void)
 {
 	omap3_do_wfi_sram = omap_sram_push(omap3_do_wfi, omap3_do_wfi_sz);
 
-	if (omap_type() != OMAP2_DEVICE_TYPE_GP)
+	if (1 || omap_type() != OMAP2_DEVICE_TYPE_GP)
 		_omap_save_secure_sram = omap_sram_push(save_secure_ram_context,
 				save_secure_ram_context_sz);
 }
@@ -551,7 +551,7 @@ int __init omap3_pm_init(void)
 		clkdm_add_wkdep(per_clkdm, wkup_clkdm);
 
 	clkdm_add_wkdep(neon_clkdm, mpu_clkdm);
-	if (omap_type() != OMAP2_DEVICE_TYPE_GP) {
+	if (1 || omap_type() != OMAP2_DEVICE_TYPE_GP) {
 		omap3_secure_ram_storage =
 			kmalloc(0x803F, GFP_KERNEL);
 		if (!omap3_secure_ram_storage)
arch/arm/mach-omap2/sleep34xx.S +1 -1
@@ -114,7 +114,7 @@ ENTRY(save_secure_ram_context)
 	mov	r6, #0xff
 	dsb				@ data write barrier
 	dmb				@ data memory barrier
-	smc	#1			@ call SMI monitor (smi #1)
+	@smc	#1			@ call SMI monitor (smi #1)
 	nop
 	nop
 	nop
Signature · 1 line
2.14.1
#10 lore raw
Quoted 15 lines · Tony Lindgren
Quoted 7 lines · Joonsoo Kim
Quoted 5 lines · Tony Lindgren

I doubt that QEMU n900 boots in secure mode but instead shows the SoC as general purpose SoC. If so, you'd have to patch the omap3_save_secure_ram_context() to attempt to save secure RAM context in all cases. If that works then debugging with any omap3 board like beagleboard in QEMU should work.

Quoted 6 lines

Sorry for late response.

I tried to emulate beagle board by using QEMU and now I find the way and it works. However, it doesn't call omap3_save_secure_ram_context() due to different omap_type(). And, even if I call it forcibly, the system dies with prefetch abort regardless of commit 9caf25f996e8.

Quoted 7 lines

OK

Quoted 1 line

Could you let me know the better way to test your situation?

Hmm yeah it seems to always return 0x19 on GP devices at least with the following test hack. But maybe that's enough for you to still see some differences with your patches.

Hmm... I cannot see any difference with this test hack. Without 'smc' instruction, r0 should always be 0x19 (#25) and it's not possible to see any difference.

Anyway, I did various test in my QEMU and I cannot see any difference with my patches. It seems that I need to ask your help more.

Quoted 8 lines
Quoted 4 lines

Anyway, could you test linux-next with 'CONFIG_HIGHMEM = n'? I'd like to know if the issue is related to the change that all CMA memory is managed like as highmem.

Yes I disabled CONFIG_HIGHMEM and n900 boots. To disable it, you need to remove it from arch/arm/mach-omap2/Kconfig that selects it if ARCH_OMAP2PLUS_TYPICAL is selected.

Okay. Problem would be related to address traslation. I'd like to check address traslation more. Could you apply following patch and test it? And, please send me the dmesg log and your kernel config. Please test this with CONFIG_DEBUG_VIRTUAL = n and CONFIG_CMA_DEBUG=y and CONFIG_HIGHMEM=y and with kernel bootparam 'ignore_loglevel'.

It would be really appreciate if you send me two logs for before/after commit 9caf25f996e8.

Thanks.

-------------------->8------------------------

arch/arm/include/asm/memory.h +8 -2
index 1f54e4e..545993d 100644
@@ -220,6 +220,9 @@ extern const void *__pv_table_begin, *__pv_table_end;
 	: "r" (x), "I" (__PV_BITS_31_24)		\
 	: "cc")
 
+extern void __virt_to_phys_debug(unsigned long x, phys_addr_t t);
+extern void __phys_to_virt_debug(phys_addr_t x, unsigned long t);
+
 static inline phys_addr_t __virt_to_phys_nodebug(unsigned long x)
 {
 	phys_addr_t t;
@@ -230,6 +233,8 @@ static inline phys_addr_t __virt_to_phys_nodebug(unsigned long x)
 		__pv_stub_mov_hi(t);
 		__pv_add_carry_stub(x, t);
 	}
+
+	__virt_to_phys_debug(x, t);
 	return t;
 }
 
@@ -244,6 +249,8 @@ static inline unsigned long __phys_to_virt(phys_addr_t x)
 	 * in place where 'r' 32 bit operand is expected.
 	 */
 	__pv_stub((unsigned long) x, t, "sub", __PV_BITS_31_24);
+
+	__phys_to_virt_debug(x, t);
 	return t;
 }
 
@@ -265,8 +272,7 @@ static inline unsigned long __phys_to_virt(phys_addr_t x)
 #endif
 
 #define virt_to_pfn(kaddr) \
-	((((unsigned long)(kaddr) - PAGE_OFFSET) >> PAGE_SHIFT) + \
-	 PHYS_PFN_OFFSET)
+	PHYS_PFN(__virt_to_phys_nodebug((unsigned long)kaddr))
 
 #define __pa_symbol_nodebug(x)	__virt_to_phys_nodebug((x))
 
arch/arm/mach-omap2/pm34xx.c +16 -0
index 841ba19..7728535 100644
@@ -103,6 +103,21 @@ static void omap3_core_restore_context(void)
 	omap_dma_global_context_restore();
 }
 
+static void omap3_print_secure_ram_context(void)
+{
+	int i;
+	int *sram_ctx_api_params;
+
+	sram_ctx_api_params = (int *)_omap_save_secure_sram;
+	sram_ctx_api_params += (save_secure_ram_context_sz / 4);
+	sram_ctx_api_params -= 5;
+
+	for (i = 0; i < 5; i++) {
+		pr_err("save_secure_sram()'s param: %d: 0x%x\n",
+			i, sram_ctx_api_params[i]);
+	}
+}
+
 /*
  * FIXME: This function should be called before entering off-mode after
  * OMAP3 secure services have been accessed. Currently it is only called
@@ -127,6 +142,7 @@ static void omap3_save_secure_ram_context(void)
 		/* Following is for error tracking, it should not happen */
 		if (ret) {
 			pr_err("save_secure_sram() returns %08x\n", ret);
+			omap3_print_secure_ram_context();
 			while (1)
 				;
 		}
arch/arm/mm/dma-mapping.c +3 -0
index 38f0fde..519c294 100644
@@ -615,6 +615,9 @@ static void *__alloc_from_contiguous(struct device *dev, size_t size,
 	struct page *page;
 	void *ptr = NULL;
 
+	printk("%s\n", __func__);
+	dump_stack();
+
 	page = dma_alloc_from_contiguous(dev, count, order, gfp);
 	if (!page)
 		return NULL;
arch/arm/mm/mmu.c +20 -0
index e46a6a4..1793024 100644
@@ -1651,3 +1651,23 @@ void __init early_mm_init(const struct machine_desc *mdesc)
 	build_mem_type_table();
 	early_paging_init(mdesc);
 }
+
+extern bool cma_check_addr(phys_addr_t x);
+
+void __virt_to_phys_debug(unsigned long x, phys_addr_t t)
+{
+	if (!cma_check_addr(t))
+		return;
+
+	printk("CMA_ADDR: %s: 0x%lx to 0x%lx\n", __func__, x, (unsigned long)t);
+	dump_stack();
+}
+
+void __phys_to_virt_debug(phys_addr_t x, unsigned long t)
+{
+	if (!cma_check_addr(x))
+		return;
+
+	printk("CMA_ADDR: %s: 0x%lx to 0x%lx\n", __func__, (unsigned long)x, t);
+	dump_stack();
+}
arch/arm/plat-omap/sram.c +8 -0
index a5bc92d..1fc7487 100644
@@ -57,6 +57,9 @@ void *omap_sram_push_address(unsigned long size)
 	new_ceil = ROUND_DOWN(new_ceil, FNCPY_ALIGN);
 	omap_sram_ceil = IOMEM(new_ceil);
 
+	printk("SRAM_ADDR: %s: 0x%lx - 0x%lx\n",
+		__func__, omap_sram_ceil, omap_sram_ceil + size);
+
 	return (void *)omap_sram_ceil;
 }
 
@@ -89,6 +92,11 @@ void __init omap_map_sram(unsigned long start, unsigned long size,
 
 	omap_sram_reset();
 
+	printk("SRAM_ADDR: %s: P: 0x%lx - 0x%lx\n",
+		__func__, start, start + size);
+	printk("SRAM_ADDR: %s: V: 0x%lx - 0x%lx\n",
+		__func__, omap_sram_base, omap_sram_base + size);
+
 	/*
 	 * Looks like we need to preserve some bootloader code at the
 	 * beginning of SRAM for jumping to flash for reboot to work...
mm/cma.c +18 -0
index a8ababb..33a0455 100644
@@ -577,3 +577,21 @@ int cma_for_each_area(int (*it)(struct cma *cma, void *data), void *data)
 
 	return 0;
 }
+
+bool cma_check_addr(phys_addr_t x)
+{
+	int i;
+	struct cma *cma;
+	phys_addr_t s, e;
+
+	for (i = 0; i < cma_area_count; i++) {
+		cma = &cma_areas[i];
+		s = cma_get_base(cma);
+		e = s + cma_get_size(cma);
+
+		if (s <= x && x < e)
+			return true;
+	}
+
+	return false;
+}
maintainer
#11 lore raw
Quoted 5 lines · Joonsoo Kim
Quoted 3 lines · Tony Lindgren

Yes I disabled CONFIG_HIGHMEM and n900 boots. To disable it, you need to remove it from arch/arm/mach-omap2/Kconfig that selects it if ARCH_OMAP2PLUS_TYPICAL is selected.

Quoted 8 lines

Okay. Problem would be related to address traslation. I'd like to check address traslation more. Could you apply following patch and test it? And, please send me the dmesg log and your kernel config. Please test this with CONFIG_DEBUG_VIRTUAL = n and CONFIG_CMA_DEBUG=y and CONFIG_HIGHMEM=y and with kernel bootparam 'ignore_loglevel'.

It would be really appreciate if you send me two logs for before/after commit 9caf25f996e8.

Sorry for the delays, I finally got around testing this for you. Compile with your patch failed for modules with __virt_to_phys_debug being undefined so I added EXPORT_SYMBOL there. I also enabled DEBUG_LL and EARLYPRINTK to get output.

Below is dmesg output for 9caf25f996e8 + your patch. I'll send you the full logs separately.

Regards,

Tony

8< -----------------------

[    0.000000] Booting Linux on physical CPU 0x0
[    0.000000] Linux version 4.13.0-rc7-00246-g9caf25f996e8-dirty (tmlind at sampyla) (gcc version 6.2.0 (crosstool-NG crosstool-ng-1.22.0-248-gdf5a341)) #416 SMP Thu Sep 21 10:04:29 PDT 2017
[    0.000000] CPU: ARMv7 Processor [411fc083] revision 3 (ARMv7), cr=10c5387d
[    0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT nonaliasing instruction cache
[    0.000000] OF: fdt: Machine model: Nokia N900
[    0.000000] bootconsole [earlycon0] enabled
[    0.000000] Memory policy: Data cache writeback
[    0.000000] cma: dma_contiguous_reserve(limit ffffffff)
[    0.000000] cma: dma_contiguous_reserve: reserving 16 MiB for global area
[    0.000000] cma: cma_declare_contiguous(size 0x01000000, base 0x00000000, limit 0xffffffff alignment 0x00000000)
[    0.000000] cma: Reserved 16 MiB@0x8e800000
[    0.000000] CMA_ADDR: __phys_to_virt_debug: 0x8e800000 to 0xce800000
[    0.000000] CPU: 0 PID: 0 Comm: swapper Not tainted 4.13.0-rc7-00246-g9caf25f996e8-dirty #416
[    0.000000] Hardware name: Nokia RX-51 board
[    0.000000] [<c0110570>] (unwind_backtrace) from [<c010c510>] (show_stack+0x10/0x14)
[    0.000000] [<c010c510>] (show_stack) from [<c081efe8>] (dump_stack+0xac/0xe0)
[    0.000000] [<c081efe8>] (dump_stack) from [<c0c06f14>] (dma_contiguous_remap+0x68/0x138)
[    0.000000] [<c0c06f14>] (dma_contiguous_remap) from [<c0c08158>] (paging_init+0x324/0x708)
[    0.000000] [<c0c08158>] (paging_init) from [<c0c042a4>] (setup_arch+0x5b4/0xca8)
[    0.000000] [<c0c042a4>] (setup_arch) from [<c0c0093c>] (start_kernel+0x58/0x3ec)
[    0.000000] [<c0c0093c>] (start_kernel) from [<8000807c>] (0x8000807c)
[    0.000000] CMA_ADDR: __phys_to_virt_debug: 0x8e800000 to 0xce800000
[    0.000000] CPU: 0 PID: 0 Comm: swapper Not tainted 4.13.0-rc7-00246-g9caf25f996e8-dirty #416
[    0.000000] Hardware name: Nokia RX-51 board
[    0.000000] [<c0110570>] (unwind_backtrace) from [<c010c510>] (show_stack+0x10/0x14)
[    0.000000] [<c010c510>] (show_stack) from [<c081efe8>] (dump_stack+0xac/0xe0)
[    0.000000] [<c081efe8>] (dump_stack) from [<c0c06f3c>] (dma_contiguous_remap+0x90/0x138)
[    0.000000] [<c0c06f3c>] (dma_contiguous_remap) from [<c0c08158>] (paging_init+0x324/0x708)
[    0.000000] [<c0c08158>] (paging_init) from [<c0c042a4>] (setup_arch+0x5b4/0xca8)
[    0.000000] [<c0c042a4>] (setup_arch) from [<c0c0093c>] (start_kernel+0x58/0x3ec)
[    0.000000] [<c0c0093c>] (start_kernel) from [<8000807c>] (0x8000807c)
[    0.000000] CMA_ADDR: __phys_to_virt_debug: 0x8e800000 to 0xce800000
[    0.000000] CPU: 0 PID: 0 Comm: swapper Not tainted 4.13.0-rc7-00246-g9caf25f996e8-dirty #416
[    0.000000] Hardware name: Nokia RX-51 board
[    0.000000] [<c0110570>] (unwind_backtrace) from [<c010c510>] (show_stack+0x10/0x14)
[    0.000000] [<c010c510>] (show_stack) from [<c081efe8>] (dump_stack+0xac/0xe0)
[    0.000000] [<c081efe8>] (dump_stack) from [<c0c06f9c>] (dma_contiguous_remap+0xf0/0x138)
[    0.000000] [<c0c06f9c>] (dma_contiguous_remap) from [<c0c08158>] (paging_init+0x324/0x708)
[    0.000000] [<c0c08158>] (paging_init) from [<c0c042a4>] (setup_arch+0x5b4/0xca8)
[    0.000000] [<c0c042a4>] (setup_arch) from [<c0c0093c>] (start_kernel+0x58/0x3ec)
[    0.000000] [<c0c0093c>] (start_kernel) from [<8000807c>] (0x8000807c)
[    0.000000] On node 0 totalpages: 65024
[    0.000000] free_area_init_node: node 0, pgdat c0dc22c0, node_mem_map cfa8b000
[    0.000000]   Normal zone: 572 pages used for memmap
[    0.000000]   Normal zone: 0 pages reserved
[    0.000000]   Normal zone: 65024 pages, LIFO batch:15
[    0.000000] CPU: All CPU(s) started in SVC mode.
[    0.000000] OMAP3430/3530 ES3.1 (l2cache iva sgx neon isp)
[    0.000000] percpu: Embedded 18 pages/cpu @cfd94000 s41000 r8192 d24536 u73728
[    0.000000] pcpu-alloc: s41000 r8192 d24536 u73728 alloc=18*4096
[    0.000000] pcpu-alloc: [0] 0 
[    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 64452
[    0.000000] Kernel command line: root=/dev/nfs ip=dhcp console=ttyO2,115200 memmap=2M$0x88000000 ramoops.mem_address=0x88000000 ramoops.mem_size=0x200000 ramoops.record_size=0x40000 debug earlyprintk init=/root/init
[    0.000000] PID hash table entries: 1024 (order: 0, 4096 bytes)
[    0.000000] Dentry cache hash table entries: 32768 (order: 5, 131072 bytes)
[    0.000000] Inode-cache hash table entries: 16384 (order: 4, 65536 bytes)
[    0.000000] Memory: 219536K/260096K available (8192K kernel code, 805K rwdata, 2388K rodata, 1024K init, 8058K bss, 24176K reserved, 16384K cma-reserved, 0K highmem)
[    0.000000] Virtual kernel memory layout:
[    0.000000]     vector  : 0xffff0000 - 0xffff1000   (   4 kB)
[    0.000000]     fixmap  : 0xffc00000 - 0xfff00000   (3072 kB)
[    0.000000]     vmalloc : 0xd0000000 - 0xff800000   ( 760 MB)
[    0.000000]     lowmem  : 0xc0000000 - 0xcfe00000   ( 254 MB)
[    0.000000]     pkmap   : 0xbfe00000 - 0xc0000000   (   2 MB)
[    0.000000]     modules : 0xbf000000 - 0xbfe00000   (  14 MB)
[    0.000000]       .text : 0xc0008000 - 0xc0900000   (9184 kB)
[    0.000000]       .init : 0xc0c00000 - 0xc0d00000   (1024 kB)
[    0.000000]       .data : 0xc0d00000 - 0xc0dc9430   ( 806 kB)
[    0.000000]        .bss : 0xc0dcb000 - 0xc15a9a4c   (8059 kB)
[    0.000000] Running RCU self tests
[    0.000000] Hierarchical RCU implementation.
[    0.000000] 	RCU event tracing is enabled.
[    0.000000] 	RCU lockdep checking is enabled.
[    0.000000] 	RCU restricting CPUs from NR_CPUS=2 to nr_cpu_ids=1.
[    0.000000] RCU: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1
[    0.000000] NR_IRQS: 16, nr_irqs: 16, preallocated irqs: 16
[    0.000000] IRQ: Found an INTC at 0xfa200000 (revision 4.0) with 96 interrupts
[    0.000000] Clocking rate (Crystal/Core/MPU): 19.2/332/500 MHz
[    0.000000] OMAP clockevent source: timer1 at 32768 Hz
[    0.000000] clocksource: 32k_counter: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 58327039986419 ns
[    0.000000] sched_clock: 32 bits at 32kHz, resolution 30517ns, wraps every 65535999984741ns
[    0.008697] OMAP clocksource: 32k_counter at 32768 Hz
[    0.017150] Console: colour dummy device 80x30
[    0.021972] Lock dependency validator: Copyright (c) 2006 Red Hat, Inc., Ingo Molnar
[    0.030120] ... MAX_LOCKDEP_SUBCLASSES:  8
[    0.034454] ... MAX_LOCK_DEPTH:          48
[    0.038848] ... MAX_LOCKDEP_KEYS:        8191
[    0.043518] ... CLASSHASH_SIZE:          4096
[    0.048095] ... MAX_LOCKDEP_ENTRIES:     32768
[    0.052825] ... MAX_LOCKDEP_CHAINS:      65536
[    0.057525] ... CHAINHASH_SIZE:          32768
[    0.062255]  memory used by lock dependency info: 5167 kB
[    0.067932]  per task-struct memory footprint: 1536 bytes
[    0.073669] Calibrating delay loop... 493.97 BogoMIPS (lpj=2469888)
[    0.135528] pid_max: default: 32768 minimum: 301
[    0.141265] Security Framework initialized
[    0.145843] Mount-cache hash table entries: 1024 (order: 0, 4096 bytes)
[    0.152893] Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes)
[    0.166473] CPU: Testing write buffer coherency: ok
[    0.174652] CPU0: thread -1, cpu 0, socket -1, mpidr 0
[    0.183135] Setting up static identity map for 0x80100000 - 0x80100078
[    0.191253] Hierarchical SRCU implementation.
[    0.199310] smp: Bringing up secondary CPUs ...
[    0.204101] smp: Brought up 1 node, 1 CPU
[    0.208343] SMP: Total of 1 processors activated (493.97 BogoMIPS).
[    0.214996] CPU: All CPU(s) started in SVC mode.
[    0.227752] devtmpfs: initialized
[    0.344909] Built 1 zonelists, mobility grouping on.  Total pages: 58980
[    0.353942] VFP support v0.3: implementor 41 architecture 3 part 30 variant c rev 1
[    0.363677] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
[    0.374145] futex hash table entries: 256 (order: 2, 16384 bytes)
[    0.381561] pinctrl core: initialized pinctrl subsystem
[    0.394714] random: get_random_u32 called from bucket_table_alloc+0xdc/0x264 with crng_init=0
[    0.405181] NET: Registered protocol family 16
[    0.410919] __alloc_from_contiguous
[    0.414642] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.13.0-rc7-00246-g9caf25f996e8-dirty #416
[    0.423706] Hardware name: Nokia RX-51 board
[    0.428222] [<c0110570>] (unwind_backtrace) from [<c010c510>] (show_stack+0x10/0x14)
[    0.436309] [<c010c510>] (show_stack) from [<c081efe8>] (dump_stack+0xac/0xe0)
[    0.443878] [<c081efe8>] (dump_stack) from [<c01161c4>] (__alloc_from_contiguous+0x30/0xf8)
[    0.452606] [<c01161c4>] (__alloc_from_contiguous) from [<c0c06d60>] (atomic_pool_init+0x7c/0x178)
[    0.461944] [<c0c06d60>] (atomic_pool_init) from [<c0101918>] (do_one_initcall+0x3c/0x170)
[    0.470550] [<c0101918>] (do_one_initcall) from [<c0c00ecc>] (kernel_init_freeable+0x1fc/0x2c4)
[    0.479644] [<c0c00ecc>] (kernel_init_freeable) from [<c08328e4>] (kernel_init+0x8/0x110)
[    0.488159] [<c08328e4>] (kernel_init) from [<c0107850>] (ret_from_fork+0x14/0x24)
[    0.496185] cma: cma_alloc(cma c15774bc, count 64, align 6)
[    0.511199] cma: cma_alloc(): returned cfc95000
[    0.517578] DMA: preallocated 256 KiB pool for atomic coherent allocations
[    0.651306] omap_hwmod: mcbsp2_sidetone using broken dt data from mcbsp
[    0.662017] omap_hwmod: mcbsp3_sidetone using broken dt data from mcbsp
[    0.802093] cpuidle: using governor menu
[    0.807403] SRAM_ADDR: omap_map_sram: P: 0x40208000 - 0x4020f000
[    0.813873] SRAM_ADDR: omap_map_sram: V: 0xd0050000 - 0xd0057000
[    0.820312] SRAM_ADDR: omap_sram_push_address: 0xd0056f00 - 0xd0056ffc
[    0.827148] SRAM_ADDR: omap_sram_push_address: 0xd0056e90 - 0xd0056efc
[    0.834106] Reprogramming SDRC clock to 332000000 Hz
[    0.857757] gpio gpiochip0: (gpio): added GPIO chardev (254:0)
[    0.864685] gpiochip_setup_dev: registered GPIOs 0 to 31 on device: gpiochip0 (gpio)
[    0.879302] OMAP GPIO hardware version 2.5
[    0.888824] gpio gpiochip1: (gpio): added GPIO chardev (254:1)
[    0.895416] gpiochip_setup_dev: registered GPIOs 32 to 63 on device: gpiochip1 (gpio)
[    0.913238] gpio gpiochip2: (gpio): added GPIO chardev (254:2)
[    0.919952] gpiochip_setup_dev: registered GPIOs 64 to 95 on device: gpiochip2 (gpio)
[    0.937927] gpio gpiochip3: (gpio): added GPIO chardev (254:3)
[    0.944519] gpiochip_setup_dev: registered GPIOs 96 to 127 on device: gpiochip3 (gpio)
[    0.962524] gpio gpiochip4: (gpio): added GPIO chardev (254:4)
[    0.969207] gpiochip_setup_dev: registered GPIOs 128 to 159 on device: gpiochip4 (gpio)
[    0.986816] gpio gpiochip5: (gpio): added GPIO chardev (254:5)
[    0.993530] gpiochip_setup_dev: registered GPIOs 160 to 191 on device: gpiochip5 (gpio)
[    1.045806] omap-gpmc 6e000000.gpmc: could not find pctldev for node /ocp at 68000000/l4 at 48000000/scm at 2000/pinmux at 30/pinmux_gpmc_pins, deferring probe
[    1.075134] hw-breakpoint: debug architecture 0x4 unsupported.
[    1.083435] omap4_sram_init:Unable to allocate sram needed to handle errata I688
[    1.091369] omap4_sram_init:Unable to get sram pool needed to handle errata I688
[    1.101593] Reserving DMA channels 0 and 1 for HS ROM code
[    1.107543] OMAP DMA hardware revision 4.0
[    1.201812] omap-dma-engine 48056000.dma-controller: OMAP DMA engine driver
[    1.221221] omap-iommu 480bd400.mmu: 480bd400.mmu registered
[    1.228912] iommu: Adding device 480bc000.isp to group 0
[    1.236175] SCSI subsystem initialized
[    1.241912] libata version 3.00 loaded.
[    1.247253] omap_i2c 48070000.i2c: could not find pctldev for node /ocp at 68000000/l4 at 48000000/scm at 2000/pinmux at 30/pinmux_i2c1_pins, deferring probe
[    1.261108] omap_i2c 48072000.i2c: could not find pctldev for node /ocp at 68000000/l4 at 48000000/scm at 2000/pinmux at 30/pinmux_i2c2_pins, deferring probe
[    1.274871] omap_i2c 48060000.i2c: could not find pctldev for node /ocp at 68000000/l4 at 48000000/scm at 2000/pinmux at 30/pinmux_i2c3_pins, deferring probe
[    1.289215] pps_core: LinuxPPS API ver. 1 registered
[    1.294433] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <[email protected]>
[    1.304138] PTP clock support registered
[    1.315582] clocksource: Switched to clocksource 32k_counter
[    1.595153] VFS: Disk quotas dquot_6.6.0
[    1.599853] VFS: Dquot-cache hash table entries: 1024 (order 0, 4096 bytes)
[    1.660949] NET: Registered protocol family 2
[    1.669342] TCP established hash table entries: 2048 (order: 1, 8192 bytes)
[    1.676910] TCP bind hash table entries: 2048 (order: 4, 73728 bytes)
[    1.684814] TCP: Hash tables configured (established 2048 bind 2048)
[    1.692321] UDP hash table entries: 256 (order: 2, 20480 bytes)
[    1.699005] UDP-Lite hash table entries: 256 (order: 2, 20480 bytes)
[    1.707244] NET: Registered protocol family 1
[    1.715698] RPC: Registered named UNIX socket transport module.
[    1.721954] RPC: Registered udp transport module.
[    1.727050] RPC: Registered tcp transport module.
[    1.732025] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    1.748840] hw perfevents: no interrupt-affinity property for /pmu at 54000000, guessing.
[    1.759094] hw perfevents: enabled with armv7_cortex_a8 PMU driver, 5 counters available
[    1.777587] audit: initializing netlink subsys (disabled)
[    1.788665] audit: type=2000 audit(1.689:1): state=initialized audit_enabled=0 res=1
[    1.798400] workingset: timestamp_bits=14 max_order=16 bucket_order=2
[    1.810302] NFS: Registering the id_resolver key type
[    1.816619] Key type id_resolver registered
[    1.821075] Key type id_legacy registered
[    1.825775] jffs2: version 2.2. (NAND) (SUMMARY)  ? 2001-2006 Red Hat, Inc.
[    1.838653] random: fast init done
[    1.851684] io scheduler noop registered
[    1.856201] io scheduler deadline registered
[    1.860870] io scheduler cfq registered (default)
[    1.865997] io scheduler mq-deadline registered
[    1.870758] io scheduler kyber registered
[    1.880615] pinctrl-single 48002030.pinmux: 284 pins at pa fa002030 size 568
[    1.889617] pinctrl-single 48002a00.pinmux: 46 pins at pa fa002a00 size 92
[    1.898406] pinctrl-single 480025d8.pinmux: 18 pins at pa fa0025d8 size 36
[    1.916870] Serial: 8250/16550 driver, 6 ports, IRQ sharing enabled
[    1.936401] omap_uart 4806c000.serial: no wakeirq for uart1
[    1.942291] of_get_named_gpiod_flags: can't parse 'rts-gpio' property of node '/ocp at 68000000/serial at 4806c000[0]'
[    1.953826] 4806c000.serial: ttyO1 at MMIO 0x4806c000 (irq = 89, base_baud = 3000000) is a OMAP UART1
[    1.967803] of_get_named_gpiod_flags: can't parse 'rts-gpio' property of node '/ocp at 68000000/serial at 49020000[0]'
[    1.978912] 49020000.serial: ttyO2 at MMIO 0x49020000 (irq = 90, base_baud = 3000000) is a OMAP UART2
[    1.989379] console [ttyO2] enabled
[    1.989379] console [ttyO2] enabled
[    1.997009] bootconsole [earlycon0] disabled
[    1.997009] bootconsole [earlycon0] disabled
[    2.064758] brd: module loaded
[    2.114959] loop: module loaded
[    2.123657] mtdoops: mtd device (mtddev=name/number) must be supplied
[    2.151977] mdio_bus fixed-0: GPIO lookup for consumer reset
[    2.158264] mdio_bus fixed-0: using lookup tables for GPIO lookup
[    2.164916] mdio_bus fixed-0: lookup for GPIO reset failed
[    2.171386] libphy: Fixed MDIO Bus: probed
[    2.183044] i2c /dev entries driver
[    2.191650] omap_hsmmc 4809c000.mmc: GPIO lookup for consumer cd
[    2.198303] omap_hsmmc 4809c000.mmc: using device tree for GPIO lookup
[    2.205322] of_get_named_gpiod_flags: parsed 'cd-gpios' property of node '/ocp at 68000000/mmc at 4809c000[0]' - status (0)
[    2.216827] omap_hsmmc 4809c000.mmc: Got CD GPIO
[    2.221771] omap_hsmmc 4809c000.mmc: GPIO lookup for consumer wp
[    2.228271] omap_hsmmc 4809c000.mmc: using device tree for GPIO lookup
[    2.235198] of_get_named_gpiod_flags: can't parse 'wp-gpios' property of node '/ocp at 68000000/mmc at 4809c000[0]'
[    2.245788] of_get_named_gpiod_flags: can't parse 'wp-gpio' property of node '/ocp at 68000000/mmc at 4809c000[0]'
[    2.256317] omap_hsmmc 4809c000.mmc: using lookup tables for GPIO lookup
[    2.263397] omap_hsmmc 4809c000.mmc: lookup for GPIO wp failed
[    2.277252] omap_hsmmc 480b4000.mmc: GPIO lookup for consumer wp
[    2.283660] omap_hsmmc 480b4000.mmc: using device tree for GPIO lookup
[    2.290771] of_get_named_gpiod_flags: can't parse 'wp-gpios' property of node '/ocp at 68000000/mmc at 480b4000[0]'
[    2.301422] of_get_named_gpiod_flags: can't parse 'wp-gpio' property of node '/ocp at 68000000/mmc at 480b4000[0]'
[    2.311920] omap_hsmmc 480b4000.mmc: using lookup tables for GPIO lookup
[    2.319152] omap_hsmmc 480b4000.mmc: lookup for GPIO wp failed
[    2.328521] ledtrig-cpu: registered to indicate activity on CPUs
[    2.336822] oprofile: using arm/armv7
[    2.342010] Initializing XFRM netlink socket
[    2.347503] NET: Registered protocol family 10
[    2.357482] Segment Routing with IPv6
[    2.361602] sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
[    2.371490] NET: Registered protocol family 17
[    2.376525] NET: Registered protocol family 15
[    2.381835] Key type dns_resolver registered
[    2.387268] omap2_set_init_voltage: unable to find boot up OPP for vdd_mpu_iva
[    2.394927] omap2_set_init_voltage: unable to set vdd_mpu_iva
[    2.401214] omap2_set_init_voltage: unable to find boot up OPP for vdd_core
[    2.408721] omap2_set_init_voltage: unable to set vdd_core
[    2.419464] save_secure_sram() returns 0000ff02
[    2.424285] save_secure_sram()'s param: 0: 0x4
[    2.428985] save_secure_sram()'s param: 1: 0x8e700000
[    2.434356] save_secure_sram()'s param: 2: 0x0
[    2.439056] save_secure_sram()'s param: 3: 0x1
[    2.443756] save_secure_sram()'s param: 4: 0x1
#12 lore raw
Quoted 12 lines · Tony Lindgren
Quoted 11 lines · Joonsoo Kim
Quoted 3 lines · Tony Lindgren

Yes I disabled CONFIG_HIGHMEM and n900 boots. To disable it, you need to remove it from arch/arm/mach-omap2/Kconfig that selects it if ARCH_OMAP2PLUS_TYPICAL is selected.

Okay. Problem would be related to address traslation. I'd like to check address traslation more. Could you apply following patch and test it? And, please send me the dmesg log and your kernel config. Please test this with CONFIG_DEBUG_VIRTUAL = n and CONFIG_CMA_DEBUG=y and CONFIG_HIGHMEM=y and with kernel bootparam 'ignore_loglevel'.

Quoted 4 lines
Quoted 2 lines

It would be really appreciate if you send me two logs for before/after commit 9caf25f996e8.

Sorry for the delays, I finally got around testing this for you.

No problem! I really appreciate your help!

Quoted 6 lines

Compile with your patch failed for modules with __virt_to_phys_debug being undefined so I added EXPORT_SYMBOL there. I also enabled DEBUG_LL and EARLYPRINTK to get output.

Below is dmesg output for 9caf25f996e8 + your patch. I'll send you the full logs separately.

Hmm...there is only one caller for the CMA memory, that is, atomic_pool_init(). Could you test one more time with 9caf25f996e8 + following patch? I'd like to know the actual user for the CMA memory.

Thanks.

-------------------------->8------------------------

arch/arm/mm/dma-mapping.c +2 -0
index 519c294..c68f34a 100644
@@ -587,6 +587,8 @@ static void *__alloc_from_pool(size_t size, struct page **ret_page)
                ptr = (void *)val;
        }
 
+       dump_stack();
+
        return ptr;
 }
 
maintainer
#13 lore raw
Quoted 24 lines · Joonsoo Kim
Quoted 12 lines · Tony Lindgren
Quoted 11 lines · Joonsoo Kim
Quoted 3 lines · Tony Lindgren

Yes I disabled CONFIG_HIGHMEM and n900 boots. To disable it, you need to remove it from arch/arm/mach-omap2/Kconfig that selects it if ARCH_OMAP2PLUS_TYPICAL is selected.

Okay. Problem would be related to address traslation. I'd like to check address traslation more. Could you apply following patch and test it? And, please send me the dmesg log and your kernel config. Please test this with CONFIG_DEBUG_VIRTUAL = n and CONFIG_CMA_DEBUG=y and CONFIG_HIGHMEM=y and with kernel bootparam 'ignore_loglevel'.

Quoted 4 lines
Quoted 2 lines

It would be really appreciate if you send me two logs for before/after commit 9caf25f996e8.

Sorry for the delays, I finally got around testing this for you.

No problem! I really appreciate your help!

Quoted 4 lines

Compile with your patch failed for modules with __virt_to_phys_debug being undefined so I added EXPORT_SYMBOL there. I also enabled DEBUG_LL and EARLYPRINTK to get output.

Quoted 6 lines
Quoted 2 lines

Below is dmesg output for 9caf25f996e8 + your patch. I'll send you the full logs separately.

Hmm...there is only one caller for the CMA memory, that is, atomic_pool_init(). Could you test one more time with 9caf25f996e8 + following patch? I'd like to know the actual user for the CMA memory.

Hmm not getting any stack with that patch after manually applying it because of tabs to spaces mangling.

Regards,

Tony

Quoted 14 lines

-------------------------->8------------------------

arch/arm/mm/dma-mapping.c +2 -0
index 519c294..c68f34a 100644
@@ -587,6 +587,8 @@ static void *__alloc_from_pool(size_t size, struct page **ret_page)
                ptr = (void *)val;
        }
 
+       dump_stack();
+
        return ptr;
 }
 
#14 lore raw
Quoted 28 lines · Tony Lindgren
Quoted 27 lines · Joonsoo Kim
Quoted 12 lines · Tony Lindgren
Quoted 11 lines · Joonsoo Kim
Quoted 3 lines · Tony Lindgren

Yes I disabled CONFIG_HIGHMEM and n900 boots. To disable it, you need to remove it from arch/arm/mach-omap2/Kconfig that selects it if ARCH_OMAP2PLUS_TYPICAL is selected.

Okay. Problem would be related to address traslation. I'd like to check address traslation more. Could you apply following patch and test it? And, please send me the dmesg log and your kernel config. Please test this with CONFIG_DEBUG_VIRTUAL = n and CONFIG_CMA_DEBUG=y and CONFIG_HIGHMEM=y and with kernel bootparam 'ignore_loglevel'.

Quoted 4 lines
Quoted 2 lines

It would be really appreciate if you send me two logs for before/after commit 9caf25f996e8.

Sorry for the delays, I finally got around testing this for you.

No problem! I really appreciate your help!

Quoted 6 lines

Compile with your patch failed for modules with __virt_to_phys_debug being undefined so I added EXPORT_SYMBOL there. I also enabled DEBUG_LL and EARLYPRINTK to get output.

Below is dmesg output for 9caf25f996e8 + your patch. I'll send you the full logs separately.

Quoted 6 lines
Quoted 3 lines

Hmm...there is only one caller for the CMA memory, that is, atomic_pool_init(). Could you test one more time with 9caf25f996e8 + following patch? I'd like to know the actual user for the CMA memory.

Hmm not getting any stack with that patch after manually applying it because of tabs to spaces mangling.

Sorry for long delay.

Seems like your system doesn't use any CMA memory by CMA API.

Could you test one more thing?
This one is to disable CMA memory allocation from the page allocator. With this, we can be sure that CMA memory isn't used at all.

If there is no difference with this patch, that is, the system down, I think that some initialization step is broken. In this case, please test following patch.

I make a branch in github that all these patch is applied. Feel free to use it.

https://github.com/JoonsooKim/linux/tree/cma-debug4-next-20180901

Thanks.

--------------->8----------------------

mm/page_alloc.c +1 -1
index 6dbc49e..1e48e67 100644
@@ -1861,7 +1861,7 @@ static int fallbacks[MIGRATE_TYPES][4] = {
 static struct page *__rmqueue_cma_fallback(struct zone *zone,
                                        unsigned int order)
 {
-       return __rmqueue_smallest(zone, order, MIGRATE_CMA);
+       return NULL;
 }
 #else
 static inline struct page *__rmqueue_cma_fallback(struct zone *zone,

----------------->8----------------------

arch/arm/mm/dma-mapping.c +4 -0
index c68f34a..c72b4c3 100644
@@ -497,6 +497,9 @@ void __init dma_contiguous_remap(void)
                map.length = end - start;
                map.type = MT_MEMORY_DMA_READY;
 
+               dmac_flush_range(map.virtual, map.virtual + map.length);
+               outer_flush_range(start, end);
+
                /*
                 * Clear previous low-memory mapping to ensure that the
                 * TLB does not see any conflicting entries, then flush
@@ -510,6 +513,7 @@ void __init dma_contiguous_remap(void)
                     addr += PMD_SIZE)
                        pmd_clear(pmd_off_k(addr));
 
+               flush_cache_all();
                flush_tlb_kernel_range(__phys_to_virt(start),
                                       __phys_to_virt(end));
 
maintainer
#15 lore raw
Quoted 44 lines · Joonsoo Kim
Quoted 28 lines · Tony Lindgren
Quoted 27 lines · Joonsoo Kim
Quoted 12 lines · Tony Lindgren
Quoted 11 lines · Joonsoo Kim
Quoted 3 lines · Tony Lindgren

Yes I disabled CONFIG_HIGHMEM and n900 boots. To disable it, you need to remove it from arch/arm/mach-omap2/Kconfig that selects it if ARCH_OMAP2PLUS_TYPICAL is selected.

Okay. Problem would be related to address traslation. I'd like to check address traslation more. Could you apply following patch and test it? And, please send me the dmesg log and your kernel config. Please test this with CONFIG_DEBUG_VIRTUAL = n and CONFIG_CMA_DEBUG=y and CONFIG_HIGHMEM=y and with kernel bootparam 'ignore_loglevel'.

Quoted 4 lines
Quoted 2 lines

It would be really appreciate if you send me two logs for before/after commit 9caf25f996e8.

Sorry for the delays, I finally got around testing this for you.

No problem! I really appreciate your help!

Quoted 6 lines

Compile with your patch failed for modules with __virt_to_phys_debug being undefined so I added EXPORT_SYMBOL there. I also enabled DEBUG_LL and EARLYPRINTK to get output.

Below is dmesg output for 9caf25f996e8 + your patch. I'll send you the full logs separately.

Quoted 6 lines
Quoted 3 lines

Hmm...there is only one caller for the CMA memory, that is, atomic_pool_init(). Could you test one more time with 9caf25f996e8 + following patch? I'd like to know the actual user for the CMA memory.

Hmm not getting any stack with that patch after manually applying it because of tabs to spaces mangling.

Sorry for long delay.

Seems like your system doesn't use any CMA memory by CMA API.

Could you test one more thing?
This one is to disable CMA memory allocation from the page allocator. With this, we can be sure that CMA memory isn't used at all.

Quoted 8 lines

If there is no difference with this patch, that is, the system down, I think that some initialization step is broken. In this case, please test following patch.

I make a branch in github that all these patch is applied. Feel free to use it.

https://github.com/JoonsooKim/linux/tree/cma-debug4-next-20180901

Still not booting, dmesg output of your test branch below.

Regards,

Tony

8< -------------------------

[    0.000000] Booting Linux on physical CPU 0x0
[    0.000000] Linux version 4.13.0-rc7-next-20170901-00010-gf93e559a038d (tmlind at sampyla) (gcc version 6.3.1 20170109 (crosstool-NG crosstool-ng-1.23.0)) #475 SMP Thu Oct 19 11:09:34 PDT 2017
[    0.000000] CPU: ARMv7 Processor [411fc083] revision 3 (ARMv7), cr=10c5387d
[    0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT nonaliasing instruction cache
[    0.000000] OF: fdt: Machine model: Nokia N900
[    0.000000] Memory policy: Data cache writeback
[    0.000000] cma: Reserved 16 MiB@0x8e800000
[    0.000000] CMA_ADDR: __phys_to_virt_debug: 0x8e800000 to 0xce800000
[    0.000000] CPU: 0 PID: 0 Comm: swapper Not tainted 4.13.0-rc7-next-20170901-00010-gf93e559a038d #475
[    0.000000] Hardware name: Nokia RX-51 board
[    0.000000] [<c0110b38>] (unwind_backtrace) from [<c010caec>] (show_stack+0x10/0x14)
[    0.000000] [<c010caec>] (show_stack) from [<c082eac4>] (dump_stack+0xac/0xe0)
[    0.000000] [<c082eac4>] (dump_stack) from [<c0c06f10>] (dma_contiguous_remap+0x64/0x168)
[    0.000000] [<c0c06f10>] (dma_contiguous_remap) from [<c0c08114>] (paging_init+0x324/0x700)
[    0.000000] [<c0c08114>] (paging_init) from [<c0c042ac>] (setup_arch+0x5b4/0xc94)
[    0.000000] [<c0c042ac>] (setup_arch) from [<c0c00940>] (start_kernel+0x54/0x3fc)
[    0.000000] [<c0c00940>] (start_kernel) from [<8000807c>] (0x8000807c)
[    0.000000] CMA_ADDR: __phys_to_virt_debug: 0x8e800000 to 0xce800000
[    0.000000] CPU: 0 PID: 0 Comm: swapper Not tainted 4.13.0-rc7-next-20170901-00010-gf93e559a038d #475
[    0.000000] Hardware name: Nokia RX-51 board
[    0.000000] [<c0110b38>] (unwind_backtrace) from [<c010caec>] (show_stack+0x10/0x14)
[    0.000000] [<c010caec>] (show_stack) from [<c082eac4>] (dump_stack+0xac/0xe0)
[    0.000000] [<c082eac4>] (dump_stack) from [<c0c06f50>] (dma_contiguous_remap+0xa4/0x168)
[    0.000000] [<c0c06f50>] (dma_contiguous_remap) from [<c0c08114>] (paging_init+0x324/0x700)
[    0.000000] [<c0c08114>] (paging_init) from [<c0c042ac>] (setup_arch+0x5b4/0xc94)
[    0.000000] [<c0c042ac>] (setup_arch) from [<c0c00940>] (start_kernel+0x54/0x3fc)
[    0.000000] [<c0c00940>] (start_kernel) from [<8000807c>] (0x8000807c)
[    0.000000] CMA_ADDR: __phys_to_virt_debug: 0x8e800000 to 0xce800000
[    0.000000] CPU: 0 PID: 0 Comm: swapper Not tainted 4.13.0-rc7-next-20170901-00010-gf93e559a038d #475
[    0.000000] Hardware name: Nokia RX-51 board
[    0.000000] [<c0110b38>] (unwind_backtrace) from [<c010caec>] (show_stack+0x10/0x14)
[    0.000000] [<c010caec>] (show_stack) from [<c082eac4>] (dump_stack+0xac/0xe0)
[    0.000000] [<c082eac4>] (dump_stack) from [<c0c06fcc>] (dma_contiguous_remap+0x120/0x168)
[    0.000000] [<c0c06fcc>] (dma_contiguous_remap) from [<c0c08114>] (paging_init+0x324/0x700)
[    0.000000] [<c0c08114>] (paging_init) from [<c0c042ac>] (setup_arch+0x5b4/0xc94)
[    0.000000] [<c0c042ac>] (setup_arch) from [<c0c00940>] (start_kernel+0x54/0x3fc)
[    0.000000] [<c0c00940>] (start_kernel) from [<8000807c>] (0x8000807c)
[    0.000000] On node 0 totalpages: 65024
[    0.000000] free_area_init_node: node 0, pgdat c0dc5040, node_mem_map cfa8b000
[    0.000000]   Normal zone: 572 pages used for memmap
[    0.000000]   Normal zone: 0 pages reserved
[    0.000000]   Normal zone: 65024 pages, LIFO batch:15
[    0.000000] CPU: All CPU(s) started in SVC mode.
[    0.000000] OMAP3430/3530 ES3.1 (l2cache iva sgx neon isp)
[    0.000000] random: fast init done
[    0.000000] percpu: Embedded 18 pages/cpu @cfd9c000 s41644 r8192 d23892 u73728
[    0.000000] pcpu-alloc: s41644 r8192 d23892 u73728 alloc=18*4096
[    0.000000] pcpu-alloc: [0] 0 
[    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 64452
[    0.000000] Kernel command line: root=/dev/nfs ip=dhcp console=ttyO2,115200 memmap=2M$0x88000000 ramoops.mem_address=0x88000000 ramoops.mem_size=0x200000 ramoops.record_size=0x40000 debug earlyprintk init=/root/init
[    0.000000] PID hash table entries: 1024 (order: 0, 4096 bytes)
[    0.000000] Dentry cache hash table entries: 32768 (order: 5, 131072 bytes)
[    0.000000] Inode-cache hash table entries: 16384 (order: 4, 65536 bytes)
[    0.000000] Memory: 220052K/260096K available (8192K kernel code, 816K rwdata, 2420K rodata, 1024K init, 7557K bss, 23660K reserved, 16384K cma-reserved, 0K highmem)
[    0.000000] Virtual kernel memory layout:
[    0.000000]     vector  : 0xffff0000 - 0xffff1000   (   4 kB)
[    0.000000]     fixmap  : 0xffc00000 - 0xfff00000   (3072 kB)
[    0.000000]     vmalloc : 0xd0000000 - 0xff800000   ( 760 MB)
[    0.000000]     lowmem  : 0xc0000000 - 0xcfe00000   ( 254 MB)
[    0.000000]     pkmap   : 0xbfe00000 - 0xc0000000   (   2 MB)
[    0.000000]     modules : 0xbf000000 - 0xbfe00000   (  14 MB)
[    0.000000]       .text : 0xc0008000 - 0xc0900000   (9184 kB)
[    0.000000]       .init : 0xc0c00000 - 0xc0d00000   (1024 kB)
[    0.000000]       .data : 0xc0d00000 - 0xc0dcc280   ( 817 kB)
[    0.000000]        .bss : 0xc0dce000 - 0xc152f644   (7558 kB)
[    0.000000] Running RCU self tests
[    0.000000] Hierarchical RCU implementation.
[    0.000000] 	RCU event tracing is enabled.
[    0.000000] 	RCU lockdep checking is enabled.
[    0.000000] 	RCU restricting CPUs from NR_CPUS=2 to nr_cpu_ids=1.
[    0.000000] RCU: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1
[    0.000000] NR_IRQS: 16, nr_irqs: 16, preallocated irqs: 16
[    0.000000] IRQ: Found an INTC at 0xfa200000 (revision 4.0) with 96 interrupts
[    0.000000] Clocking rate (Crystal/Core/MPU): 19.2/332/500 MHz
[    0.000000] OMAP clockevent source: timer1 at 32768 Hz
[    0.000000] clocksource: 32k_counter: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 58327039986419 ns
[    0.000030] sched_clock: 32 bits at 32kHz, resolution 30517ns, wraps every 65535999984741ns
[    0.000061] OMAP clocksource: 32k_counter at 32768 Hz
[    0.002716] Console: colour dummy device 80x30
[    0.002807] WARNING: Your 'console=ttyO2' has been replaced by 'ttyS2'
[    0.002838] This ensures that you still see kernel messages. Please
[    0.002868] update your kernel commandline.
[    0.002899] Lock dependency validator: Copyright (c) 2006 Red Hat, Inc., Ingo Molnar
[    0.002929] ... MAX_LOCKDEP_SUBCLASSES:  8
[    0.002960] ... MAX_LOCK_DEPTH:          48
[    0.002990] ... MAX_LOCKDEP_KEYS:        8191
[    0.003021] ... CLASSHASH_SIZE:          4096
[    0.003051] ... MAX_LOCKDEP_ENTRIES:     32768
[    0.003082] ... MAX_LOCKDEP_CHAINS:      65536
[    0.003112] ... CHAINHASH_SIZE:          32768
[    0.003143]  memory used by lock dependency info: 4655 kB
[    0.003173]  per task-struct memory footprint: 1536 bytes
[    0.003234] Calibrating delay loop... 493.97 BogoMIPS (lpj=2469888)
[    0.104431] pid_max: default: 32768 minimum: 301
[    0.105895] Security Framework initialized
[    0.106658] Mount-cache hash table entries: 1024 (order: 0, 4096 bytes)
[    0.106781] Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes)
[    0.116973] CPU: Testing write buffer coherency: ok
[    0.121459] CPU0: thread -1, cpu 0, socket -1, mpidr 0
[    0.128570] Setting up static identity map for 0x80100000 - 0x80100078
[    0.130950] Hierarchical SRCU implementation.
[    0.140258] smp: Bringing up secondary CPUs ...
[    0.140380] smp: Brought up 1 node, 1 CPU
[    0.140502] SMP: Total of 1 processors activated (493.97 BogoMIPS).
[    0.140594] CPU: All CPU(s) started in SVC mode.
[    0.152923] devtmpfs: initialized
[    0.661376] Built 1 zonelists, mobility grouping on.  Total pages: 59109
[    0.665405] VFP support v0.3: implementor 41 architecture 3 part 30 variant c rev 1
[    0.669372] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
[    0.669586] futex hash table entries: 256 (order: 2, 16384 bytes)
[    0.671386] pinctrl core: initialized pinctrl subsystem
[    0.690399] NET: Registered protocol family 16
[    0.691802] __alloc_from_contiguous
[    0.691925] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.13.0-rc7-next-20170901-00010-gf93e559a038d #475
[    0.692016] Hardware name: Nokia RX-51 board
[    0.692138] [<c0110b38>] (unwind_backtrace) from [<c010caec>] (show_stack+0x10/0x14)
[    0.692260] [<c010caec>] (show_stack) from [<c082eac4>] (dump_stack+0xac/0xe0)
[    0.692382] [<c082eac4>] (dump_stack) from [<c011653c>] (__alloc_from_contiguous+0x30/0xf8)
[    0.692504] [<c011653c>] (__alloc_from_contiguous) from [<c0c06d60>] (atomic_pool_init+0x7c/0x178)
[    0.692626] [<c0c06d60>] (atomic_pool_init) from [<c0101df4>] (do_one_initcall+0x3c/0x170)
[    0.692718] [<c0101df4>] (do_one_initcall) from [<c0c00ee4>] (kernel_init_freeable+0x1fc/0x2c4)
[    0.692840] [<c0c00ee4>] (kernel_init_freeable) from [<c0842a0c>] (kernel_init+0x8/0x110)
[    0.692962] [<c0842a0c>] (kernel_init) from [<c0107d78>] (ret_from_fork+0x14/0x3c)
[    0.726104] DMA: preallocated 256 KiB pool for atomic coherent allocations
[    1.273651] omap_hwmod: mcbsp2_sidetone using broken dt data from mcbsp
[    1.289337] omap_hwmod: mcbsp3_sidetone using broken dt data from mcbsp
[    1.796203] cpuidle: using governor menu
[    1.798645] SRAM_ADDR: omap_map_sram: P: 0x40208000 - 0x4020f000
[    1.798736] SRAM_ADDR: omap_map_sram: V: 0xd0050000 - 0xd0057000
[    1.798889] SRAM_ADDR: omap_sram_push_address: 0xd0056f00 - 0xd0056ffc
[    1.798980] SRAM_ADDR: omap_sram_push_address: 0xd0056e98 - 0xd0056f00
[    1.799163] Reprogramming SDRC clock to 332000000 Hz
[    1.865997] gpio gpiochip0: (gpio): added GPIO chardev (254:0)
[    1.868011] gpiochip_setup_dev: registered GPIOs 0 to 31 on device: gpiochip0 (gpio)
[    1.868835] OMAP GPIO hardware version 2.5
[    1.880462] gpio gpiochip1: (gpio): added GPIO chardev (254:1)
[    1.882446] gpiochip_setup_dev: registered GPIOs 32 to 63 on device: gpiochip1 (gpio)
[    1.894042] gpio gpiochip2: (gpio): added GPIO chardev (254:2)
[    1.895568] gpiochip_setup_dev: registered GPIOs 64 to 95 on device: gpiochip2 (gpio)
[    1.907318] gpio gpiochip3: (gpio): added GPIO chardev (254:3)
[    1.908843] gpiochip_setup_dev: registered GPIOs 96 to 127 on device: gpiochip3 (gpio)
[    1.920501] gpio gpiochip4: (gpio): added GPIO chardev (254:4)
[    1.922424] gpiochip_setup_dev: registered GPIOs 128 to 159 on device: gpiochip4 (gpio)
[    1.933990] gpio gpiochip5: (gpio): added GPIO chardev (254:5)
[    1.935485] gpiochip_setup_dev: registered GPIOs 160 to 191 on device: gpiochip5 (gpio)
[    2.065551] omap-gpmc 6e000000.gpmc: could not find pctldev for node /ocp at 68000000/l4 at 48000000/scm at 2000/pinmux at 30/pinmux_gpmc_pins, deferring probe
[    2.119781] hw-breakpoint: debug architecture 0x4 unsupported.
[    2.123596] omap4_sram_init:Unable to allocate sram needed to handle errata I688
[    2.123748] omap4_sram_init:Unable to get sram pool needed to handle errata I688
[    2.131469] Reserving DMA channels 0 and 1 for HS ROM code
[    2.131591] OMAP DMA hardware revision 4.0
[    2.426635] omap-dma-engine 48056000.dma-controller: OMAP DMA engine driver
[    2.460449] omap-iommu 480bd400.mmu: 480bd400.mmu registered
[    2.465270] iommu: Adding device 480bc000.isp to group 0
[    2.477874] SCSI subsystem initialized
[    2.481719] libata version 3.00 loaded.
[    2.485290] omap_i2c 48070000.i2c: could not find pctldev for node /ocp at 68000000/l4 at 48000000/scm at 2000/pinmux at 30/pinmux_i2c1_pins, deferring probe
[    2.485931] omap_i2c 48072000.i2c: could not find pctldev for node /ocp at 68000000/l4 at 48000000/scm at 2000/pinmux at 30/pinmux_i2c2_pins, deferring probe
[    2.486511] omap_i2c 48060000.i2c: could not find pctldev for node /ocp at 68000000/l4 at 48000000/scm at 2000/pinmux at 30/pinmux_i2c3_pins, deferring probe
[    2.488647] pps_core: LinuxPPS API ver. 1 registered
[    2.488739] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <[email protected]>
[    2.489166] PTP clock support registered
[    2.505584] clocksource: Switched to clocksource 32k_counter
[    3.327270] VFS: Disk quotas dquot_6.6.0
[    3.328155] VFS: Dquot-cache hash table entries: 1024 (order 0, 4096 bytes)
[    3.490722] NET: Registered protocol family 2
[    3.497497] TCP established hash table entries: 2048 (order: 1, 8192 bytes)
[    3.497741] TCP bind hash table entries: 2048 (order: 4, 81920 bytes)
[    3.498992] TCP: Hash tables configured (established 2048 bind 2048)
[    3.500183] UDP hash table entries: 256 (order: 2, 24576 bytes)
[    3.500701] UDP-Lite hash table entries: 256 (order: 2, 24576 bytes)
[    3.503479] NET: Registered protocol family 1
[    3.511474] RPC: Registered named UNIX socket transport module.
[    3.511627] RPC: Registered udp transport module.
[    3.511749] RPC: Registered tcp transport module.
[    3.511871] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    3.538238] hw perfevents: no interrupt-affinity property for /pmu at 54000000, guessing.
[    3.540618] hw perfevents: enabled with armv7_cortex_a8 PMU driver, 5 counters available
[    3.572448] audit: initializing netlink subsys (disabled)
[    3.587646] audit: type=2000 audit(3.571:1): state=initialized audit_enabled=0 res=1
[    3.590087] workingset: timestamp_bits=14 max_order=16 bucket_order=2
[    3.602478] NFS: Registering the id_resolver key type
[    3.603515] Key type id_resolver registered
[    3.603668] Key type id_legacy registered
[    3.604309] jffs2: version 2.2. (NAND) (SUMMARY)  ? 2001-2006 Red Hat, Inc.
[    3.637115] io scheduler noop registered
[    3.637237] io scheduler deadline registered
[    3.637512] io scheduler cfq registered (default)
[    3.637634] io scheduler mq-deadline registered
[    3.637756] io scheduler kyber registered
[    3.658416] pinctrl-single 48002030.pinmux: 284 pins at pa fa002030 size 568
[    3.663146] pinctrl-single 48002a00.pinmux: 46 pins at pa fa002a00 size 92
[    3.668548] pinctrl-single 480025d8.pinmux: 18 pins at pa fa0025d8 size 36
[    3.711212] Serial: 8250/16550 driver, 6 ports, IRQ sharing enabled
[    3.759765] 4806c000.serial: ttyS1 at MMIO 0x4806c000 (irq = 89, base_baud = 3000000) is a 8250
[    3.774414] 49020000.serial: ttyS2 at MMIO 0x49020000 (irq = 90, base_baud = 3000000) is a 8250
[    5.113739] console [ttyS2] enabled
[    5.301635] brd: module loaded
[    5.439788] loop: module loaded
[    5.459564] mtdoops: mtd device (mtddev=name/number) must be supplied
[    5.530944] mdio_bus fixed-0: GPIO lookup for consumer reset
[    5.537200] mdio_bus fixed-0: using lookup tables for GPIO lookup
[    5.543701] mdio_bus fixed-0: lookup for GPIO reset failed
[    5.550903] libphy: Fixed MDIO Bus: probed
[    5.582519] i2c /dev entries driver
[    5.603973] omap_hsmmc 4809c000.mmc: GPIO lookup for consumer cd
[    5.610595] omap_hsmmc 4809c000.mmc: using device tree for GPIO lookup
[    5.617767] of_get_named_gpiod_flags: parsed 'cd-gpios' property of node '/ocp at 68000000/mmc at 4809c000[0]' - status (0)
[    5.629211] omap_hsmmc 4809c000.mmc: Got CD GPIO
[    5.634033] omap_hsmmc 4809c000.mmc: GPIO lookup for consumer wp
[    5.640502] omap_hsmmc 4809c000.mmc: using device tree for GPIO lookup
[    5.647491] of_get_named_gpiod_flags: can't parse 'wp-gpios' property of node '/ocp at 68000000/mmc at 4809c000[0]'
[    5.657928] of_get_named_gpiod_flags: can't parse 'wp-gpio' property of node '/ocp at 68000000/mmc at 4809c000[0]'
[    5.668212] omap_hsmmc 4809c000.mmc: using lookup tables for GPIO lookup
[    5.675109] omap_hsmmc 4809c000.mmc: lookup for GPIO wp failed
[    5.693267] omap_hsmmc 480b4000.mmc: GPIO lookup for consumer wp
[    5.699981] omap_hsmmc 480b4000.mmc: using device tree for GPIO lookup
[    5.707031] of_get_named_gpiod_flags: can't parse 'wp-gpios' property of node '/ocp at 68000000/mmc at 480b4000[0]'
[    5.717437] of_get_named_gpiod_flags: can't parse 'wp-gpio' property of node '/ocp at 68000000/mmc at 480b4000[0]'
[    5.727752] omap_hsmmc 480b4000.mmc: using lookup tables for GPIO lookup
[    5.734619] omap_hsmmc 480b4000.mmc: lookup for GPIO wp failed
[    5.749328] ledtrig-cpu: registered to indicate activity on CPUs
[    5.761383] oprofile: using arm/armv7
[    5.767517] Initializing XFRM netlink socket
[    5.773590] NET: Registered protocol family 10
[    5.789123] Segment Routing with IPv6
[    5.793579] sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
[    5.808959] NET: Registered protocol family 17
[    5.813842] NET: Registered protocol family 15
[    5.819976] Key type dns_resolver registered
[    5.825561] omap2_set_init_voltage: unable to find boot up OPP for vdd_mpu_iva
[    5.832977] omap2_set_init_voltage: unable to set vdd_mpu_iva
[    5.839263] omap2_set_init_voltage: unable to find boot up OPP for vdd_core
[    5.846710] omap2_set_init_voltage: unable to set vdd_core
[    5.864746] save_secure_sram() returns 00000019
[    5.869445] save_secure_sram()'s param: 0: 0x4
[    5.874023] save_secure_sram()'s param: 1: 0x8e700000
[    5.879211] save_secure_sram()'s param: 2: 0x0
[    5.883789] save_secure_sram()'s param: 3: 0x1
[    5.888366] save_secure_sram()'s param: 4: 0x1
#16 lore raw
Quoted 49 lines · Tony Lindgren
Quoted 48 lines · Joonsoo Kim
Quoted 28 lines · Tony Lindgren
Quoted 27 lines · Joonsoo Kim
Quoted 12 lines · Tony Lindgren
Quoted 11 lines · Joonsoo Kim
Quoted 3 lines · Tony Lindgren

Yes I disabled CONFIG_HIGHMEM and n900 boots. To disable it, you need to remove it from arch/arm/mach-omap2/Kconfig that selects it if ARCH_OMAP2PLUS_TYPICAL is selected.

Okay. Problem would be related to address traslation. I'd like to check address traslation more. Could you apply following patch and test it? And, please send me the dmesg log and your kernel config. Please test this with CONFIG_DEBUG_VIRTUAL = n and CONFIG_CMA_DEBUG=y and CONFIG_HIGHMEM=y and with kernel bootparam 'ignore_loglevel'.

Quoted 4 lines
Quoted 2 lines

It would be really appreciate if you send me two logs for before/after commit 9caf25f996e8.

Sorry for the delays, I finally got around testing this for you.

No problem! I really appreciate your help!

Quoted 6 lines

Compile with your patch failed for modules with __virt_to_phys_debug being undefined so I added EXPORT_SYMBOL there. I also enabled DEBUG_LL and EARLYPRINTK to get output.

Below is dmesg output for 9caf25f996e8 + your patch. I'll send you the full logs separately.

Quoted 6 lines
Quoted 3 lines

Hmm...there is only one caller for the CMA memory, that is, atomic_pool_init(). Could you test one more time with 9caf25f996e8 + following patch? I'd like to know the actual user for the CMA memory.

Hmm not getting any stack with that patch after manually applying it because of tabs to spaces mangling.

Sorry for long delay.

Seems like your system doesn't use any CMA memory by CMA API.

Could you test one more thing?
This one is to disable CMA memory allocation from the page allocator. With this, we can be sure that CMA memory isn't used at all.

If there is no difference with this patch, that is, the system down, I think that some initialization step is broken. In this case, please test following patch.

Quoted 6 lines
Quoted 4 lines

I make a branch in github that all these patch is applied. Feel free to use it.

https://github.com/JoonsooKim/linux/tree/cma-debug4-next-20180901

Still not booting, dmesg output of your test branch below.

Oops... I made a mistak. Could you test with reverting commit c977ee2803787363187d6aca9cebdabc793c6531 ("omap: forcibly call save_secure_ram_context() for test") in that branch? Without reverting it, it doesn't call 'smc' so it always cause a hang.

Thanks.

maintainer
#17 lore raw
Quoted 5 lines · Joonsoo Kim

Oops... I made a mistak. Could you test with reverting commit c977ee2803787363187d6aca9cebdabc793c6531 ("omap: forcibly call save_secure_ram_context() for test") in that branch? Without reverting it, it doesn't call 'smc' so it always cause a hang.

Oops I should have noticed that one. Here you go with commit c977ee280378 reverted. Still not booting.

Regards,

Tony

8< ---------------------------
Booting Linux on physical CPU 0x0
Linux version 4.13.0-rc7-next-20170901-00011-geca3cddaf1a8 (tmlind at sampyla) (gcc version 6.3.1 20170109 (crosstool-NG crosstool-ng-1.23.0)) #541 SMP Fri Oct 20 10:15:45 PDT 2017
CPU: ARMv7 Processor [411fc083] revision 3 (ARMv7), cr=10c5387d
CPU: PIPT / VIPT nonaliasing data cache, VIPT nonaliasing instruction cache
OF: fdt: Machine model: Nokia N900
Memory policy: Data cache writeback
cma: Reserved 16 MiB@0x8e800000
CMA_ADDR: __phys_to_virt_debug: 0x8e800000 to 0xce800000

CPU: 0 PID: 0 Comm: swapper Not tainted 4.13.0-rc7-next-20170901-00011-geca3cddaf1a8 #541
Hardware name: Nokia RX-51 board
[<c0110b38>] (unwind_backtrace) from [<c010caec>] (show_stack+0x10/0x14)
[<c010caec>] (show_stack) from [<c082eae4>] (dump_stack+0xac/0xe0)
[<c082eae4>] (dump_stack) from [<c0c06f10>] (dma_contiguous_remap+0x64/0x168)
[<c0c06f10>] (dma_contiguous_remap) from [<c0c08114>] (paging_init+0x324/0x700)
[<c0c08114>] (paging_init) from [<c0c042ac>] (setup_arch+0x5b4/0xc94)
[<c0c042ac>] (setup_arch) from [<c0c00940>] (start_kernel+0x54/0x3fc)
[<c0c00940>] (start_kernel) from [<8000807c>] (0x8000807c)
CMA_ADDR: __phys_to_virt_debug: 0x8e800000 to 0xce800000
CPU: 0 PID: 0 Comm: swapper Not tainted 4.13.0-rc7-next-20170901-00011-geca3cddaf1a8 #541
Hardware name: Nokia RX-51 board
[<c0110b38>] (unwind_backtrace) from [<c010caec>] (show_stack+0x10/0x14)
[<c010caec>] (show_stack) from [<c082eae4>] (dump_stack+0xac/0xe0)
[<c082eae4>] (dump_stack) from [<c0c06f50>] (dma_contiguous_remap+0xa4/0x168)
[<c0c06f50>] (dma_contiguous_remap) from [<c0c08114>] (paging_init+0x324/0x700)
[<c0c08114>] (paging_init) from [<c0c042ac>] (setup_arch+0x5b4/0xc94)
[<c0c042ac>] (setup_arch) from [<c0c00940>] (start_kernel+0x54/0x3fc)
[<c0c00940>] (start_kernel) from [<8000807c>] (0x8000807c)
CMA_ADDR: __phys_to_virt_debug: 0x8e800000 to 0xce800000
CPU: 0 PID: 0 Comm: swapper Not tainted 4.13.0-rc7-next-20170901-00011-geca3cddaf1a8 #541
Hardware name: Nokia RX-51 board
[<c0110b38>] (unwind_backtrace) from [<c010caec>] (show_stack+0x10/0x14)
[<c010caec>] (show_stack) from [<c082eae4>] (dump_stack+0xac/0xe0)
[<c082eae4>] (dump_stack) from [<c0c06fcc>] (dma_contiguous_remap+0x120/0x168)
[<c0c06fcc>] (dma_contiguous_remap) from [<c0c08114>] (paging_init+0x324/0x700)
[<c0c08114>] (paging_init) from [<c0c042ac>] (setup_arch+0x5b4/0xc94)
[<c0c042ac>] (setup_arch) from [<c0c00940>] (start_kernel+0x54/0x3fc)

[<c0c00940>] (start_kernel) from [<8000807c>] (0x8000807c)
On node 0 totalpages: 65024
free_area_init_node: node 0, pgdat c0dc5040, node_mem_map cfa8b000
Normal zone: 572 pages used for memmap
Normal zone: 0 pages reserved
Normal zone: 65024 pages, LIFO batch:15
CPU: All CPU(s) started in SVC mode.
OMAP3430/3530 ES3.1 (l2cache iva sgx neon isp)
random: fast init done
percpu: Embedded 18 pages/cpu @cfd9c000 s41644 r8192 d23892 u73728
pcpu-alloc: s41644 r8192 d23892 u73728 alloc=18*4096
pcpu-alloc: [0] 0
Built 1 zonelists, mobility grouping on. Total pages: 64452
Kernel command line: root=/dev/nfs ip=dhcp console=ttyO2,115200 memmap=2M$0x88000000 ramoops.mem_address=0x88000000 ramoops.mem_size=0x200000 ramoops.record_size=0x40000 debug earlyprintk init=/root/init PID hash table entries: 1024 (order: 0, 4096 bytes)
Dentry cache hash table entries: 32768 (order: 5, 131072 bytes)
Inode-cache hash table entries: 16384 (order: 4, 65536 bytes)
Memory: 220052K/260096K available (8192K kernel code, 816K rwdata, 2420K rodata, 1024K init, 7557K bss, 23660K reserved, 16384K cma-reserved, 0K highmem)
Virtual kernel memory layout:

    vector  : 0xffff0000 - 0xffff1000   (   4 kB)
    fixmap  : 0xffc00000 - 0xfff00000   (3072 kB)
    vmalloc : 0xd0000000 - 0xff800000   ( 760 MB)
    lowmem  : 0xc0000000 - 0xcfe00000   ( 254 MB)
    pkmap   : 0xbfe00000 - 0xc0000000   (   2 MB)
    modules : 0xbf000000 - 0xbfe00000   (  14 MB)
      .text : 0xc0008000 - 0xc0900000   (9184 kB)
      .init : 0xc0c00000 - 0xc0d00000   (1024 kB)
      .data : 0xc0d00000 - 0xc0dcc280   ( 817 kB)
       .bss : 0xc0dce000 - 0xc152f644   (7558 kB)

Running RCU self tests
Hierarchical RCU implementation.
RCU event tracing is enabled.
RCU lockdep checking is enabled.
RCU restricting CPUs from NR_CPUS=2 to nr_cpu_ids=1.
RCU: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1
NR_IRQS: 16, nr_irqs: 16, preallocated irqs: 16
IRQ: Found an INTC at 0xfa200000 (revision 4.0) with 96 interrupts
Clocking rate (Crystal/Core/MPU): 19.2/332/500 MHz
OMAP clockevent source: timer1 at 32768 Hz
clocksource: 32k_counter: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 58327039986419 ns
sched_clock: 32 bits at 32kHz, resolution 30517ns, wraps every 65535999984741ns OMAP clocksource: 32k_counter at 32768 Hz
Console: colour dummy device 80x30
WARNING: Your 'console=ttyO2' has been replaced by 'ttyS2'
This ensures that you still see kernel messages. Please
update your kernel commandline.
Lock dependency validator: Copyright (c) 2006 Red Hat, Inc., Ingo Molnar

... MAX_LOCKDEP_SUBCLASSES:  8
... MAX_LOCK_DEPTH:          48
... MAX_LOCKDEP_KEYS:        8191
... CLASSHASH_SIZE:          4096
... MAX_LOCKDEP_ENTRIES:     32768
... MAX_LOCKDEP_CHAINS:      65536
... CHAINHASH_SIZE:          32768

memory used by lock dependency info: 4655 kB
per task-struct memory footprint: 1536 bytes
Calibrating delay loop... 493.97 BogoMIPS (lpj=2469888)
pid_max: default: 32768 minimum: 301
Security Framework initialized
Mount-cache hash table entries: 1024 (order: 0, 4096 bytes)
Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes)
CPU: Testing write buffer coherency: ok
CPU0: thread -1, cpu 0, socket -1, mpidr 0
Setting up static identity map for 0x80100000 - 0x80100078
Hierarchical SRCU implementation.
smp: Bringing up secondary CPUs ...
smp: Brought up 1 node, 1 CPU
SMP: Total of 1 processors activated (493.97 BogoMIPS).
CPU: All CPU(s) started in SVC mode.
devtmpfs: initialized
Built 1 zonelists, mobility grouping on. Total pages: 59109
VFP support v0.3: implementor 41 architecture 3 part 30 variant c rev 1
clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns futex hash table entries: 256 (order: 2, 16384 bytes)
pinctrl core: initialized pinctrl subsystem
NET: Registered protocol family 16
__alloc_from_contiguous

CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.13.0-rc7-next-20170901-00011-geca3cddaf1a8 #541
Hardware name: Nokia RX-51 board
[<c0110b38>] (unwind_backtrace) from [<c010caec>] (show_stack+0x10/0x14)
[<c010caec>] (show_stack) from [<c082eae4>] (dump_stack+0xac/0xe0)
[<c082eae4>] (dump_stack) from [<c011653c>] (__alloc_from_contiguous+0x30/0xf8)
[<c011653c>] (__alloc_from_contiguous) from [<c0c06d60>] (atomic_pool_init+0x7c/0x178)
[<c0c06d60>] (atomic_pool_init) from [<c0101df4>] (do_one_initcall+0x3c/0x170)
[<c0101df4>] (do_one_initcall) from [<c0c00ee4>] (kernel_init_freeable+0x1fc/0x2c4)
[<c0c00ee4>] (kernel_init_freeable) from [<c0842a2c>] (kernel_init+0x8/0x110)
[<c0842a2c>] (kernel_init) from [<c0107d78>] (ret_from_fork+0x14/0x3c)

DMA: preallocated 256 KiB pool for atomic coherent allocations
omap_hwmod: mcbsp2_sidetone using broken dt data from mcbsp
omap_hwmod: mcbsp3_sidetone using broken dt data from mcbsp
cpuidle: using governor menu
SRAM_ADDR: omap_map_sram: P: 0x40208000 - 0x4020f000
SRAM_ADDR: omap_map_sram: V: 0xd0050000 - 0xd0057000
SRAM_ADDR: omap_sram_push_address: 0xd0056f00 - 0xd0056ffc
SRAM_ADDR: omap_sram_push_address: 0xd0056e90 - 0xd0056efc
Reprogramming SDRC clock to 332000000 Hz
gpio gpiochip0: (gpio): added GPIO chardev (254:0)
gpiochip_setup_dev: registered GPIOs 0 to 31 on device: gpiochip0 (gpio)
OMAP GPIO hardware version 2.5
gpio gpiochip1: (gpio): added GPIO chardev (254:1)
gpiochip_setup_dev: registered GPIOs 32 to 63 on device: gpiochip1 (gpio)
gpio gpiochip2: (gpio): added GPIO chardev (254:2)
gpiochip_setup_dev: registered GPIOs 64 to 95 on device: gpiochip2 (gpio)
gpio gpiochip3: (gpio): added GPIO chardev (254:3)
gpiochip_setup_dev: registered GPIOs 96 to 127 on device: gpiochip3 (gpio)
gpio gpiochip4: (gpio): added GPIO chardev (254:4)
gpiochip_setup_dev: registered GPIOs 128 to 159 on device: gpiochip4 (gpio)
gpio gpiochip5: (gpio): added GPIO chardev (254:5)
gpiochip_setup_dev: registered GPIOs 160 to 191 on device: gpiochip5 (gpio)
omap-gpmc 6e000000.gpmc: could not find pctldev for node /ocp at 68000000/l4 at 48000000/scm at 2000/pinmux at 30/pinmux_gpmc_pins, deferring probe
hw-breakpoint: debug architecture 0x4 unsupported.
omap4_sram_init:Unable to allocate sram needed to handle errata I688
omap4_sram_init:Unable to get sram pool needed to handle errata I688
Reserving DMA channels 0 and 1 for HS ROM code
OMAP DMA hardware revision 4.0
omap-dma-engine 48056000.dma-controller: OMAP DMA engine driver
omap-iommu 480bd400.mmu: 480bd400.mmu registered
iommu: Adding device 480bc000.isp to group 0
SCSI subsystem initialized
libata version 3.00 loaded.
omap_i2c 48070000.i2c: could not find pctldev for node /ocp at 68000000/l4 at 48000000/scm at 2000/pinmux at 30/pinmux_i2c1_pins, deferring probe omap_i2c 48072000.i2c: could not find pctldev for node /ocp at 68000000/l4 at 48000000/scm at 2000/pinmux at 30/pinmux_i2c2_pins, deferring probe omap_i2c 48060000.i2c: could not find pctldev for node /ocp at 68000000/l4 at 48000000/scm at 2000/pinmux at 30/pinmux_i2c3_pins, deferring probe
pps_core: LinuxPPS API ver. 1 registered
pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <[email protected]>
PTP clock support registered
clocksource: Switched to clocksource 32k_counter
VFS: Disk quotas dquot_6.6.0
VFS: Dquot-cache hash table entries: 1024 (order 0, 4096 bytes)
NET: Registered protocol family 2
TCP established hash table entries: 2048 (order: 1, 8192 bytes)
TCP bind hash table entries: 2048 (order: 4, 81920 bytes)
TCP: Hash tables configured (established 2048 bind 2048)
UDP hash table entries: 256 (order: 2, 24576 bytes)
UDP-Lite hash table entries: 256 (order: 2, 24576 bytes)
NET: Registered protocol family 1
RPC: Registered named UNIX socket transport module.
RPC: Registered udp transport module.
RPC: Registered tcp transport module.
RPC: Registered tcp NFSv4.1 backchannel transport module.
hw perfevents: no interrupt-affinity property for /pmu at 54000000, guessing.
hw perfevents: enabled with armv7_cortex_a8 PMU driver, 5 counters available
audit: initializing netlink subsys (disabled)
audit: type=2000 audit(3.541:1): state=initialized audit_enabled=0 res=1
workingset: timestamp_bits=14 max_order=16 bucket_order=2
NFS: Registering the id_resolver key type
Key type id_resolver registered
Key type id_legacy registered
jffs2: version 2.2. (NAND) (SUMMARY) ? 2001-2006 Red Hat, Inc.
io scheduler noop registered
io scheduler deadline registered
io scheduler cfq registered (default)
io scheduler mq-deadline registered
io scheduler kyber registered
pinctrl-single 48002030.pinmux: 284 pins at pa fa002030 size 568
pinctrl-single 48002a00.pinmux: 46 pins at pa fa002a00 size 92
pinctrl-single 480025d8.pinmux: 18 pins at pa fa0025d8 size 36
Serial: 8250/16550 driver, 6 ports, IRQ sharing enabled
4806c000.serial: ttyS1 at MMIO 0x4806c000 (irq = 89, base_baud = 3000000) is a 8250
49020000.serial: ttyS2 at MMIO 0x49020000 (irq = 90, base_baud = 3000000) is a 8250
console [ttyS2] enabled
brd: module loaded
loop: module loaded
mtdoops: mtd device (mtddev=name/number) must be supplied
mdio_bus fixed-0: GPIO lookup for consumer reset
mdio_bus fixed-0: using lookup tables for GPIO lookup
mdio_bus fixed-0: lookup for GPIO reset failed
libphy: Fixed MDIO Bus: probed
i2c /dev entries driver
omap_hsmmc 4809c000.mmc: GPIO lookup for consumer cd
omap_hsmmc 4809c000.mmc: using device tree for GPIO lookup
of_get_named_gpiod_flags: parsed 'cd-gpios' property of node '/ocp at 68000000/mmc at 4809c000[0]' - status (0)
omap_hsmmc 4809c000.mmc: Got CD GPIO
omap_hsmmc 4809c000.mmc: GPIO lookup for consumer wp
omap_hsmmc 4809c000.mmc: using device tree for GPIO lookup
of_get_named_gpiod_flags: can't parse 'wp-gpios' property of node '/ocp at 68000000/mmc at 4809c000[0]'
of_get_named_gpiod_flags: can't parse 'wp-gpio' property of node '/ocp at 68000000/mmc at 4809c000[0]'
omap_hsmmc 4809c000.mmc: using lookup tables for GPIO lookup
omap_hsmmc 4809c000.mmc: lookup for GPIO wp failed
omap_hsmmc 480b4000.mmc: GPIO lookup for consumer wp
omap_hsmmc 480b4000.mmc: using device tree for GPIO lookup
of_get_named_gpiod_flags: can't parse 'wp-gpios' property of node '/ocp at 68000000/mmc at 480b4000[0]'
of_get_named_gpiod_flags: can't parse 'wp-gpio' property of node '/ocp at 68000000/mmc at 480b4000[0]'
omap_hsmmc 480b4000.mmc: using lookup tables for GPIO lookup
omap_hsmmc 480b4000.mmc: lookup for GPIO wp failed
ledtrig-cpu: registered to indicate activity on CPUs
oprofile: using arm/armv7
Initializing XFRM netlink socket
NET: Registered protocol family 10
Segment Routing with IPv6
sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
NET: Registered protocol family 17
NET: Registered protocol family 15
Key type dns_resolver registered
omap2_set_init_voltage: unable to find boot up OPP for vdd_mpu_iva
omap2_set_init_voltage: unable to set vdd_mpu_iva
omap2_set_init_voltage: unable to find boot up OPP for vdd_core
omap2_set_init_voltage: unable to set vdd_core
save_secure_sram() returns 0000ff02
save_secure_sram()'s param: 0: 0x4
save_secure_sram()'s param: 1: 0x8e700000
save_secure_sram()'s param: 2: 0x0
save_secure_sram()'s param: 3: 0x1
save_secure_sram()'s param: 4: 0x1

#18 lore raw
Quoted 9 lines · Tony Lindgren
Quoted 5 lines · Joonsoo Kim

Oops... I made a mistak. Could you test with reverting commit c977ee2803787363187d6aca9cebdabc793c6531 ("omap: forcibly call save_secure_ram_context() for test") in that branch? Without reverting it, it doesn't call 'smc' so it always cause a hang.

Oops I should have noticed that one. Here you go with commit c977ee280378 reverted. Still not booting.

Still very thanks to you. :)

Okay. Could you test my updated branch? In there, I also disable atomic_pool initialization and disable to remap the CMA area in order to completely make any operation on CMA area as no-op.

And, it enables memblock_debug to check how memblock region is allocated.

https://github.com/JoonsooKim/linux/tree/cma-debug4-next-20180901

Thanks.

maintainer
#19 lore raw
Quoted 8 lines · Joonsoo Kim
Quoted 7 lines · Tony Lindgren
Quoted 5 lines · Joonsoo Kim

Oops... I made a mistak. Could you test with reverting commit c977ee2803787363187d6aca9cebdabc793c6531 ("omap: forcibly call save_secure_ram_context() for test") in that branch? Without reverting it, it doesn't call 'smc' so it always cause a hang.

Quoted 4 lines
Quoted 2 lines

Oops I should have noticed that one. Here you go with commit c977ee280378 reverted. Still not booting.

Still very thanks to you. :)

No problem, sorry for the delay in testing this one.

Quoted 8 lines

Okay. Could you test my updated branch? In there, I also disable atomic_pool initialization and disable to remap the CMA area in order to completely make any operation on CMA area as no-op.

And, it enables memblock_debug to check how memblock region is allocated.

https://github.com/JoonsooKim/linux/tree/cma-debug4-next-20180901

Great, this branch boots on n900! Early parts of the dmesg attached below.

Regards,

Tony

8< --------------------------
Booting Linux on physical CPU 0x0
Linux version 4.13.0-rc7-next-20170901-00015-g32cc67b3e8c6 (tmlind at sampyla) (gcc version 6.3.1 20170109 (crosstool-NG crosstool-ng-1.23.0)) #544 SMP Wed Oct 25 10:22:42 PDT 2017
CPU: ARMv7 Processor [411fc083] revision 3 (ARMv7), cr=10c5387d
CPU: PIPT / VIPT nonaliasing data cache, VIPT nonaliasing instruction cache
OF: fdt: Machine model: Nokia N900
memblock_add: [0x80000000-0x8fffffff] early_init_dt_scan_memory+0xec/0x158
Memory policy: Data cache writeback
memblock_reserve: [0x80100000-0x8152f643] arm_memblock_init+0x30/0x1b0
memblock_reserve: [0x80004000-0x80007fff] arm_memblock_init+0x13c/0x1b0
memblock_reserve: [0x8ff00000-0x8fffffff] memblock_alloc_range_nid+0x38/0x50
memblock_free: [0x8ff00000-0x8fffffff] arm_memblock_steal+0x30/0x48
memblock_reserve: [0x8fe00000-0x8fefffff] memblock_alloc_range_nid+0x38/0x50
memblock_free: [0x8fe00000-0x8fefffff] arm_memblock_steal+0x30/0x48
memblock_reserve: [0x8fccb000-0x8fcddfff] arm_memblock_init+0x150/0x1b0
memblock_reserve: [0x8fccb000-0x8fcddfff] early_init_fdt_scan_reserved_mem+0x58/0x7c
memblock_reserve: [0x8e800000-0x8f7fffff] memblock_alloc_range_nid+0x38/0x50
cma: Reserved 16 MiB at 0x8e800000
MEMBLOCK configuration:
memory size = 0x0fe00000 reserved size = 0x02446644
memory.cnt = 0x1
memory[0x0] [0x80000000-0x8fdfffff], 0x0fe00000 bytes flags: 0x0
reserved.cnt = 0x4
reserved[0x0] [0x80004000-0x80007fff], 0x00004000 bytes flags: 0x0
reserved[0x1] [0x80100000-0x8152f643], 0x0142f644 bytes flags: 0x0
reserved[0x2] [0x8e800000-0x8f7fffff], 0x01000000 bytes flags: 0x0
reserved[0x3] [0x8fccb000-0x8fcddfff], 0x00013000 bytes flags: 0x0
memblock_reserve: [0x8fdfe000-0x8fdfffff] memblock_alloc_range_nid+0x38/0x50
memblock_reserve: [0x8fdfd000-0x8fdfdfff] memblock_alloc_range_nid+0x38/0x50
memblock_reserve: [0x8fdfcee8-0x8fdfcfff] memblock_alloc_range_nid+0x38/0x50
memblock_reserve: [0x8fdfcec0-0x8fdfcee7] memblock_alloc_range_nid+0x38/0x50
memblock_reserve: [0x8fdfce98-0x8fdfcebf] memblock_alloc_range_nid+0x38/0x50
memblock_reserve: [0x8fdfce70-0x8fdfce97] memblock_alloc_range_nid+0x38/0x50
memblock_reserve: [0x8fdfce48-0x8fdfce6f] memblock_alloc_range_nid+0x38/0x50
memblock_reserve: [0x8fdfce20-0x8fdfce47] memblock_alloc_range_nid+0x38/0x50
memblock_reserve: [0x8fdfb000-0x8fdfbfff] memblock_alloc_range_nid+0x38/0x50
memblock_reserve: [0x8fdfa000-0x8fdfafff] memblock_alloc_range_nid+0x38/0x50
memblock_reserve: [0x8fdf9000-0x8fdf9fff] memblock_alloc_range_nid+0x38/0x50
On node 0 totalpages: 65024
memblock_virt_alloc_try_nid_nopanic: 2359296 bytes align=0x0 nid=0 from=0x0 max_addr=0x0 alloc_node_mem_map.constprop.10+0x68/0xb4
memblock_reserve: [0x8fa8b000-0x8fccafff] memblock_virt_alloc_internal+0xfc/0x1c0
free_area_init_node: node 0, pgdat c0dc5040, node_mem_map cfa8b000
Normal zone: 572 pages used for memmap
Normal zone: 0 pages reserved
Normal zone: 65024 pages, LIFO batch:15
memblock_virt_alloc_try_nid_nopanic: 16 bytes align=0x0 nid=0 from=0x0 max_addr=0x0 setup_usemap.constprop.12+0x5c/0x6c
memblock_reserve: [0x8fdfce00-0x8fdfce0f] memblock_virt_alloc_internal+0xfc/0x1c0
memblock_virt_alloc_try_nid_nopanic: 16 bytes align=0x0 nid=0 from=0x0 max_addr=0x0 setup_usemap.constprop.12+0x5c/0x6c
memblock_reserve: [0x8fdfcdc0-0x8fdfcdcf] memblock_virt_alloc_internal+0xfc/0x1c0
memblock_virt_alloc_try_nid: 32 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 setup_arch+0x6e8/0xc94
memblock_reserve: [0x8fdfcd80-0x8fdfcd9f] memblock_virt_alloc_internal+0xfc/0x1c0
memblock_reserve: [0x8fdb0548-0x8fdf8fff] memblock_alloc_range_nid+0x38/0x50
memblock_reserve: [0x8fdfcde8-0x8fdfcdff] memblock_alloc_range_nid+0x38/0x50
memblock_reserve: [0x8fdfcdd0-0x8fdfcde7] memblock_alloc_range_nid+0x38/0x50
memblock_reserve: [0x8fdfcda4-0x8fdfcdbe] memblock_alloc_range_nid+0x38/0x50
memblock_reserve: [0x8fdfcd64-0x8fdfcd7e] memblock_alloc_range_nid+0x38/0x50
memblock_reserve: [0x8fdfcd48-0x8fdfcd62] memblock_alloc_range_nid+0x38/0x50
memblock_reserve: [0x8fdfcd30-0x8fdfcd47] memblock_alloc_range_nid+0x38/0x50
CPU: All CPU(s) started in SVC mode.
OMAP3430/3530 ES3.1 (l2cache iva sgx neon isp)
memblock_virt_alloc_try_nid: 8 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 omap2_clk_legacy_provider_init+0x2c/0x44
memblock_reserve: [0x8fdfcd00-0x8fdfcd07] memblock_virt_alloc_internal+0xfc/0x1c0
memblock_virt_alloc_try_nid: 8 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 omap2_clk_legacy_provider_init+0x2c/0x44
memblock_reserve: [0x8fdfccc0-0x8fdfccc7] memblock_virt_alloc_internal+0xfc/0x1c0
random: fast init done
memblock_virt_alloc_try_nid: 183 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 start_kernel+0x9c/0x3fc
memblock_reserve: [0x8fdfcc00-0x8fdfccb6] memblock_virt_alloc_internal+0xfc/0x1c0
memblock_virt_alloc_try_nid: 183 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 start_kernel+0xc0/0x3fc
memblock_reserve: [0x8fdfcb40-0x8fdfcbf6] memblock_virt_alloc_internal+0xfc/0x1c0
memblock_virt_alloc_try_nid: 183 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 start_kernel+0xe4/0x3fc
memblock_reserve: [0x8fdfca80-0x8fdfcb36] memblock_virt_alloc_internal+0xfc/0x1c0
memblock_virt_alloc_try_nid_nopanic: 4096 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 pcpu_alloc_alloc_info+0x4c/0x88
memblock_reserve: [0x8fdaf540-0x8fdb053f] memblock_virt_alloc_internal+0xfc/0x1c0
memblock_virt_alloc_try_nid_nopanic: 4096 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 pcpu_embed_first_chunk+0x464/0x714
memblock_reserve: [0x8fdae540-0x8fdaf53f] memblock_virt_alloc_internal+0xfc/0x1c0
memblock_virt_alloc_try_nid_nopanic: 73728 bytes align=0x1000 nid=-1 from=0xbfffffff max_addr=0x0 pcpu_dfl_fc_alloc+0x3c/0x44
memblock_reserve: [0x8fd9c000-0x8fdadfff] memblock_virt_alloc_internal+0xfc/0x1c0
__memblock_free_early: [0x0000008fdae000-0x0000008fdadfff] pcpu_embed_first_chunk+0x5d8/0x714
percpu: Embedded 18 pages/cpu @cfd9c000 s41644 r8192 d23892 u73728
memblock_virt_alloc_try_nid: 4 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 pcpu_setup_first_chunk+0xf0/0x678
memblock_reserve: [0x8fdfca40-0x8fdfca43] memblock_virt_alloc_internal+0xfc/0x1c0
memblock_virt_alloc_try_nid: 4 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 pcpu_setup_first_chunk+0x110/0x678
memblock_reserve: [0x8fdfca00-0x8fdfca03] memblock_virt_alloc_internal+0xfc/0x1c0
memblock_virt_alloc_try_nid: 4 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 pcpu_setup_first_chunk+0x130/0x678
memblock_reserve: [0x8fdfc9c0-0x8fdfc9c3] memblock_virt_alloc_internal+0xfc/0x1c0
memblock_virt_alloc_try_nid: 4 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 pcpu_setup_first_chunk+0x150/0x678
memblock_reserve: [0x8fdfc980-0x8fdfc983] memblock_virt_alloc_internal+0xfc/0x1c0
pcpu-alloc: s41644 r8192 d23892 u73728 alloc=18*4096
pcpu-alloc: [0] 0
memblock_virt_alloc_try_nid: 128 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 pcpu_setup_first_chunk+0x41c/0x678
memblock_reserve: [0x8fdfc900-0x8fdfc97f] memblock_virt_alloc_internal+0xfc/0x1c0
memblock_virt_alloc_try_nid: 69 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 pcpu_alloc_first_chunk+0x64/0x2b4
memblock_reserve: [0x8fdfc880-0x8fdfc8c4] memblock_virt_alloc_internal+0xfc/0x1c0
memblock_virt_alloc_try_nid: 384 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 pcpu_alloc_first_chunk+0xa8/0x2b4
memblock_reserve: [0x8fdfc700-0x8fdfc87f] memblock_virt_alloc_internal+0xfc/0x1c0
memblock_virt_alloc_try_nid: 388 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 pcpu_alloc_first_chunk+0xc8/0x2b4
memblock_reserve: [0x8fdfc540-0x8fdfc6c3] memblock_virt_alloc_internal+0xfc/0x1c0
memblock_virt_alloc_try_nid: 60 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 pcpu_alloc_first_chunk+0xf4/0x2b4
memblock_reserve: [0x8fdfc500-0x8fdfc53b] memblock_virt_alloc_internal+0xfc/0x1c0
memblock_virt_alloc_try_nid: 69 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 pcpu_alloc_first_chunk+0x64/0x2b4
memblock_reserve: [0x8fdfc480-0x8fdfc4c4] memblock_virt_alloc_internal+0xfc/0x1c0
memblock_virt_alloc_try_nid: 768 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 pcpu_alloc_first_chunk+0xa8/0x2b4
memblock_reserve: [0x8fdfc180-0x8fdfc47f] memblock_virt_alloc_internal+0xfc/0x1c0
memblock_virt_alloc_try_nid: 772 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 pcpu_alloc_first_chunk+0xc8/0x2b4
memblock_reserve: [0x8fdae200-0x8fdae503] memblock_virt_alloc_internal+0xfc/0x1c0
memblock_virt_alloc_try_nid: 120 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 pcpu_alloc_first_chunk+0xf4/0x2b4
memblock_reserve: [0x8fdfc100-0x8fdfc177] memblock_virt_alloc_internal+0xfc/0x1c0
__memblock_free_early: [0x0000008fdaf540-0x0000008fdb053f] pcpu_embed_first_chunk+0x664/0x714
__memblock_free_early: [0x0000008fdae540-0x0000008fdaf53f] pcpu_embed_first_chunk+0x6c4/0x714
Built 1 zonelists, mobility grouping on. Total pages: 64452
Kernel command line: root=/dev/nfs ip=dhcp console=ttyO2,115200 memmap=2M$0x88000000 ramoops.mem_address=0x88000000 ramoops.mem_size=0x200000 ramoops.record_size=0x40000 debug earlyprintk init=/root/init memblock_virt_alloc_try_nid_nopanic: 4096 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 alloc_large_system_hash+0x180/0x268
memblock_reserve: [0x8fdaf540-0x8fdb053f] memblock_virt_alloc_internal+0xfc/0x1c0
PID hash table entries: 1024 (order: 0, 4096 bytes)
memblock_virt_alloc_try_nid_nopanic: 131072 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 alloc_large_system_hash+0x180/0x268
memblock_reserve: [0x8fd7c000-0x8fd9bfff] memblock_virt_alloc_internal+0xfc/0x1c0
Dentry cache hash table entries: 32768 (order: 5, 131072 bytes)
memblock_virt_alloc_try_nid_nopanic: 65536 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 alloc_large_system_hash+0x180/0x268
memblock_reserve: [0x8fd6c000-0x8fd7bfff] memblock_virt_alloc_internal+0xfc/0x1c0
Inode-cache hash table entries: 16384 (order: 4, 65536 bytes)
Memory: 220052K/260096K available (8192K kernel code, 816K rwdata, 2420K rodata, 1024K init, 7557K bss, 23660K reserved, 16384K cma-reserved, 0K highmem)
Virtual kernel memory layout:

    vector  : 0xffff0000 - 0xffff1000   (   4 kB)
    fixmap  : 0xffc00000 - 0xfff00000   (3072 kB)
    vmalloc : 0xd0000000 - 0xff800000   ( 760 MB)
    lowmem  : 0xc0000000 - 0xcfe00000   ( 254 MB)
    pkmap   : 0xbfe00000 - 0xc0000000   (   2 MB)
    modules : 0xbf000000 - 0xbfe00000   (  14 MB)
      .text : 0xc0008000 - 0xc0900000   (9184 kB)
      .init : 0xc0c00000 - 0xc0d00000   (1024 kB)
      .data : 0xc0d00000 - 0xc0dcc280   ( 817 kB)
       .bss : 0xc0dce000 - 0xc152f644   (7558 kB)

Running RCU self tests
Hierarchical RCU implementation.
RCU event tracing is enabled.
RCU lockdep checking is enabled.
RCU restricting CPUs from NR_CPUS=2 to nr_cpu_ids=1.
RCU: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1
NR_IRQS: 16, nr_irqs: 16, preallocated irqs: 16
IRQ: Found an INTC at 0xfa200000 (revision 4.0) with 96 interrupts
Clocking rate (Crystal/Core/MPU): 19.2/332/500 MHz
OMAP clockevent source: timer1 at 32768 Hz
clocksource: 32k_counter: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 58327039986419 ns
sched_clock: 32 bits at 32kHz, resolution 30517ns, wraps every 65535999984741ns OMAP clocksource: 32k_counter@32768 Hz
Console: colour dummy device 80x30
WARNING: Your 'console=ttyO2' has been replaced by 'ttyS2'
This ensures that you still see kernel messages. Please
update your kernel commandline.
Lock dependency validator: Copyright (c) 2006 Red Hat, Inc., Ingo Molnar

... MAX_LOCKDEP_SUBCLASSES:  8
... MAX_LOCK_DEPTH:          48
... MAX_LOCKDEP_KEYS:        8191
... CLASSHASH_SIZE:          4096
... MAX_LOCKDEP_ENTRIES:     32768
... MAX_LOCKDEP_CHAINS:      65536
... CHAINHASH_SIZE:          32768

memory used by lock dependency info: 4655 kB
per task-struct memory footprint: 1536 bytes
Calibrating delay loop... 493.97 BogoMIPS (lpj=2469888)
pid_max: default: 32768 minimum: 301
Security Framework initialized
Mount-cache hash table entries: 1024 (order: 0, 4096 bytes)
Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes)
CPU: Testing write buffer coherency: ok
CPU0: thread -1, cpu 0, socket -1, mpidr 0
Setting up static identity map for 0x80100000 - 0x80100078
Hierarchical SRCU implementation.
smp: Bringing up secondary CPUs ...
smp: Brought up 1 node, 1 CPU
SMP: Total of 1 processors activated (493.97 BogoMIPS).
CPU: All CPU(s) started in SVC mode.
devtmpfs: initialized
Built 1 zonelists, mobility grouping on. Total pages: 59109
VFP support v0.3: implementor 41 architecture 3 part 30 variant c rev 1
clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns futex hash table entries: 256 (order: 2, 16384 bytes)
pinctrl core: initialized pinctrl subsystem
...

#20 lore raw
Quoted 20 lines · Tony Lindgren
Quoted 8 lines · Joonsoo Kim
Quoted 7 lines · Tony Lindgren
Quoted 5 lines · Joonsoo Kim

Oops... I made a mistak. Could you test with reverting commit c977ee2803787363187d6aca9cebdabc793c6531 ("omap: forcibly call save_secure_ram_context() for test") in that branch? Without reverting it, it doesn't call 'smc' so it always cause a hang.

Quoted 4 lines
Quoted 2 lines

Oops I should have noticed that one. Here you go with commit c977ee280378 reverted. Still not booting.

Still very thanks to you. :)

No problem, sorry for the delay in testing this one.

Quoted 4 lines

Okay. Could you test my updated branch? In there, I also disable atomic_pool initialization and disable to remap the CMA area in order to completely make any operation on CMA area as no-op.

Quoted 7 lines
Quoted 4 lines

And, it enables memblock_debug to check how memblock region is allocated.

https://github.com/JoonsooKim/linux/tree/cma-debug4-next-20180901

Great, this branch boots on n900! Early parts of the dmesg attached below.

Sound good. Perhaps, problem is due to the cache management. With my patchset, direct mapping for CMA area is cleared in dma_contiguous_remap() if CONFIG_HIGHMEM. I guess that proper cache operation is required there.

Could you test my newly updated branch again? I re-enable dma_contiguous_remap() to clear direct mapping for CMA area and add proper(?) cache operation although I'm not the expert on that area.

https://github.com/JoonsooKim/linux/tree/cma-debug4-next-20180901

Thanks.

maintainer
#21 lore raw
Quoted 9 lines · Joonsoo Kim
Quoted 2 lines · Tony Lindgren

Great, this branch boots on n900! Early parts of the dmesg attached below.

Sound good. Perhaps, problem is due to the cache management. With my patchset, direct mapping for CMA area is cleared in dma_contiguous_remap() if CONFIG_HIGHMEM. I guess that proper cache operation is required there.

Quoted 5 lines

Could you test my newly updated branch again? I re-enable dma_contiguous_remap() to clear direct mapping for CMA area and add proper(?) cache operation although I'm not the expert on that area.

https://github.com/JoonsooKim/linux/tree/cma-debug4-next-20180901

Yeah that one boots too, dmesg below. I wonder why flushing the range with dmac_flush_range() and outer_flush_range() no longer seems enough though?

Reverting the arch/arm/mm/dma-mapping.c changes in your patch "arm/dma: re-enable to remap the CMA area and flush cache before remapping" also boots, but produces the following build warnings:

arch/arm/mm/dma-mapping.c: In function 'dma_contiguous_remap':
arch/arm/mm/dma-mapping.c:503:20: warning: passing argument 1 of 'cpu_cache.dma_flush_range' makes pointer from integer without a cast [-Wint-conversion]

dmac_flush_range(map.virtual, map.virtual + map.length);
                    ^~~

arch/arm/mm/dma-mapping.c:503:20: note: expected 'const void *' but argument is of type 'long unsigned int' arch/arm/mm/dma-mapping.c:503:33: warning: passing argument 2 of 'cpu_cache.dma_flush_range' makes pointer from integer without a cast [-Wint-conversion]
dmac_flush_range(map.virtual, map.virtual + map.length);

Regards,

Tony

8< -----------------------
Linux version 4.13.0-rc7-next-20170901-00016-g69a5a607f945 (tmlind at sampyla) (gcc version 6.3.1 20170109 (crosstool-NG crosstool-ng-1.23.0)) #546 SMP Thu Oct 26 07:03:19 PDT 2017
CPU: ARMv7 Processor [411fc083] revision 3 (ARMv7), cr=10c5387d
CPU: PIPT / VIPT nonaliasing data cache, VIPT nonaliasing instruction cache
OF: fdt: Machine model: Nokia N900
memblock_add: [0x80000000-0x8fffffff] early_init_dt_scan_memory+0xec/0x158
Memory policy: Data cache writeback
memblock_reserve: [0x80100000-0x8152f643] arm_memblock_init+0x30/0x1b0
memblock_reserve: [0x80004000-0x80007fff] arm_memblock_init+0x13c/0x1b0
memblock_reserve: [0x8ff00000-0x8fffffff] memblock_alloc_range_nid+0x38/0x50
memblock_free: [0x8ff00000-0x8fffffff] arm_memblock_steal+0x30/0x48
memblock_reserve: [0x8fe00000-0x8fefffff] memblock_alloc_range_nid+0x38/0x50
memblock_free: [0x8fe00000-0x8fefffff] arm_memblock_steal+0x30/0x48
memblock_reserve: [0x8fccb000-0x8fcddfff] arm_memblock_init+0x150/0x1b0
memblock_reserve: [0x8fccb000-0x8fcddfff] early_init_fdt_scan_reserved_mem+0x58/0x7c
memblock_reserve: [0x8e800000-0x8f7fffff] memblock_alloc_range_nid+0x38/0x50
cma: Reserved 16 MiB@0x8e800000
MEMBLOCK configuration:
memory size = 0x0fe00000 reserved size = 0x02446644
memory.cnt = 0x1
memory[0x0] [0x80000000-0x8fdfffff], 0x0fe00000 bytes flags: 0x0
reserved.cnt = 0x4
reserved[0x0] [0x80004000-0x80007fff], 0x00004000 bytes flags: 0x0
reserved[0x1] [0x80100000-0x8152f643], 0x0142f644 bytes flags: 0x0
reserved[0x2] [0x8e800000-0x8f7fffff], 0x01000000 bytes flags: 0x0
reserved[0x3] [0x8fccb000-0x8fcddfff], 0x00013000 bytes flags: 0x0
CMA_ADDR: __phys_to_virt_debug: 0x8e800000 to 0xce800000

CPU: 0 PID: 0 Comm: swapper Not tainted 4.13.0-rc7-next-20170901-00016-g69a5a607f945 #546
Hardware name: Nokia RX-51 board
[<c0110b38>] (unwind_backtrace) from [<c010caec>] (show_stack+0x10/0x14)
[<c010caec>] (show_stack) from [<c082e9e4>] (dump_stack+0xac/0xe0)
[<c082e9e4>] (dump_stack) from [<c0c06d9c>] (dma_contiguous_remap+0x60/0x140)
[<c0c06d9c>] (dma_contiguous_remap) from [<c0c07f7c>] (paging_init+0x324/0x700)
[<c0c07f7c>] (paging_init) from [<c0c042ac>] (setup_arch+0x5b4/0xc94)
[<c0c042ac>] (setup_arch) from [<c0c00940>] (start_kernel+0x54/0x3fc)
[<c0c00940>] (start_kernel) from [<8000807c>] (0x8000807c)
CMA_ADDR: __phys_to_virt_debug: 0x8e800000 to 0xce800000
CPU: 0 PID: 0 Comm: swapper Not tainted 4.13.0-rc7-next-20170901-00016-g69a5a607f945 #546
Hardware name: Nokia RX-51 board
[<c0110b38>] (unwind_backtrace) from [<c010caec>] (show_stack+0x10/0x14)
[<c010caec>] (show_stack) from [<c082e9e4>] (dump_stack+0xac/0xe0)
[<c082e9e4>] (dump_stack) from [<c0c06dcc>] (dma_contiguous_remap+0x90/0x140)
[<c0c06dcc>] (dma_contiguous_remap) from [<c0c07f7c>] (paging_init+0x324/0x700)
[<c0c07f7c>] (paging_init) from [<c0c042ac>] (setup_arch+0x5b4/0xc94)
[<c0c042ac>] (setup_arch) from [<c0c00940>] (start_kernel+0x54/0x3fc)
[<c0c00940>] (start_kernel) from [<8000807c>] (0x8000807c)
CMA_ADDR: __phys_to_virt_debug: 0x8e800000 to 0xce800000
CPU: 0 PID: 0 Comm: swapper Not tainted 4.13.0-rc7-next-20170901-00016-g69a5a607f945 #546
Hardware name: Nokia RX-51 board
[<c0110b38>] (unwind_backtrace) from [<c010caec>] (show_stack+0x10/0x14)
[<c010caec>] (show_stack) from [<c082e9e4>] (dump_stack+0xac/0xe0)
[<c082e9e4>] (dump_stack) from [<c0c06e34>] (dma_contiguous_remap+0xf8/0x140)
[<c0c06e34>] (dma_contiguous_remap) from [<c0c07f7c>] (paging_init+0x324/0x700)
[<c0c07f7c>] (paging_init) from [<c0c042ac>] (setup_arch+0x5b4/0xc94)
[<c0c042ac>] (setup_arch) from [<c0c00940>] (start_kernel+0x54/0x3fc)

[<c0c00940>] (start_kernel) from [<8000807c>] (0x8000807c)
memblock_reserve: [0x8fdfe000-0x8fdfffff] memblock_alloc_range_nid+0x38/0x50
memblock_reserve: [0x8fdfd000-0x8fdfdfff] memblock_alloc_range_nid+0x38/0x50
memblock_reserve: [0x8fdfcee8-0x8fdfcfff] memblock_alloc_range_nid+0x38/0x50
memblock_reserve: [0x8fdfcec0-0x8fdfcee7] memblock_alloc_range_nid+0x38/0x50
memblock_reserve: [0x8fdfce98-0x8fdfcebf] memblock_alloc_range_nid+0x38/0x50
memblock_reserve: [0x8fdfce70-0x8fdfce97] memblock_alloc_range_nid+0x38/0x50
memblock_reserve: [0x8fdfce48-0x8fdfce6f] memblock_alloc_range_nid+0x38/0x50
memblock_reserve: [0x8fdfce20-0x8fdfce47] memblock_alloc_range_nid+0x38/0x50
memblock_reserve: [0x8fdfb000-0x8fdfbfff] memblock_alloc_range_nid+0x38/0x50
memblock_reserve: [0x8fdfa000-0x8fdfafff] memblock_alloc_range_nid+0x38/0x50
memblock_reserve: [0x8fdf9000-0x8fdf9fff] memblock_alloc_range_nid+0x38/0x50
On node 0 totalpages: 65024
memblock_virt_alloc_try_nid_nopanic: 2359296 bytes align=0x0 nid=0 from=0x0 max_addr=0x0 alloc_node_mem_map.constprop.10+0x68/0xb4
memblock_reserve: [0x8fa8b000-0x8fccafff] memblock_virt_alloc_internal+0xfc/0x1c0
free_area_init_node: node 0, pgdat c0dc5040, node_mem_map cfa8b000
Normal zone: 572 pages used for memmap
Normal zone: 0 pages reserved
Normal zone: 65024 pages, LIFO batch:15
memblock_virt_alloc_try_nid_nopanic: 16 bytes align=0x0 nid=0 from=0x0 max_addr=0x0 setup_usemap.constprop.12+0x5c/0x6c
memblock_reserve: [0x8fdfce00-0x8fdfce0f] memblock_virt_alloc_internal+0xfc/0x1c0
memblock_virt_alloc_try_nid_nopanic: 16 bytes align=0x0 nid=0 from=0x0 max_addr=0x0 setup_usemap.constprop.12+0x5c/0x6c
memblock_reserve: [0x8fdfcdc0-0x8fdfcdcf] memblock_virt_alloc_internal+0xfc/0x1c0
memblock_virt_alloc_try_nid: 32 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 setup_arch+0x6e8/0xc94
memblock_reserve: [0x8fdfcd80-0x8fdfcd9f] memblock_virt_alloc_internal+0xfc/0x1c0
memblock_reserve: [0x8fdb0548-0x8fdf8fff] memblock_alloc_range_nid+0x38/0x50
memblock_reserve: [0x8fdfcde8-0x8fdfcdff] memblock_alloc_range_nid+0x38/0x50
memblock_reserve: [0x8fdfcdd0-0x8fdfcde7] memblock_alloc_range_nid+0x38/0x50
memblock_reserve: [0x8fdfcda4-0x8fdfcdbe] memblock_alloc_range_nid+0x38/0x50
memblock_reserve: [0x8fdfcd64-0x8fdfcd7e] memblock_alloc_range_nid+0x38/0x50
memblock_reserve: [0x8fdfcd48-0x8fdfcd62] memblock_alloc_range_nid+0x38/0x50
memblock_reserve: [0x8fdfcd30-0x8fdfcd47] memblock_alloc_range_nid+0x38/0x50
CPU: All CPU(s) started in SVC mode.
OMAP3430/3530 ES3.1 (l2cache iva sgx neon isp)
memblock_virt_alloc_try_nid: 8 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 omap2_clk_legacy_provider_init+0x2c/0x44
memblock_reserve: [0x8fdfcd00-0x8fdfcd07] memblock_virt_alloc_internal+0xfc/0x1c0
memblock_virt_alloc_try_nid: 8 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 omap2_clk_legacy_provider_init+0x2c/0x44
memblock_reserve: [0x8fdfccc0-0x8fdfccc7] memblock_virt_alloc_internal+0xfc/0x1c0
random: fast init done
memblock_virt_alloc_try_nid: 183 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 start_kernel+0x9c/0x3fc
memblock_reserve: [0x8fdfcc00-0x8fdfccb6] memblock_virt_alloc_internal+0xfc/0x1c0
memblock_virt_alloc_try_nid: 183 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 start_kernel+0xc0/0x3fc
memblock_reserve: [0x8fdfcb40-0x8fdfcbf6] memblock_virt_alloc_internal+0xfc/0x1c0
memblock_virt_alloc_try_nid: 183 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 start_kernel+0xe4/0x3fc
memblock_reserve: [0x8fdfca80-0x8fdfcb36] memblock_virt_alloc_internal+0xfc/0x1c0
memblock_virt_alloc_try_nid_nopanic: 4096 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 pcpu_alloc_alloc_info+0x4c/0x88
memblock_reserve: [0x8fdaf540-0x8fdb053f] memblock_virt_alloc_internal+0xfc/0x1c0
memblock_virt_alloc_try_nid_nopanic: 4096 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 pcpu_embed_first_chunk+0x464/0x714
memblock_reserve: [0x8fdae540-0x8fdaf53f] memblock_virt_alloc_internal+0xfc/0x1c0
memblock_virt_alloc_try_nid_nopanic: 73728 bytes align=0x1000 nid=-1 from=0xbfffffff max_addr=0x0 pcpu_dfl_fc_alloc+0x3c/0x44
memblock_reserve: [0x8fd9c000-0x8fdadfff] memblock_virt_alloc_internal+0xfc/0x1c0
__memblock_free_early: [0x0000008fdae000-0x0000008fdadfff] pcpu_embed_first_chunk+0x5d8/0x714
percpu: Embedded 18 pages/cpu @cfd9c000 s41644 r8192 d23892 u73728
memblock_virt_alloc_try_nid: 4 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 pcpu_setup_first_chunk+0xf0/0x678
memblock_reserve: [0x8fdfca40-0x8fdfca43] memblock_virt_alloc_internal+0xfc/0x1c0
memblock_virt_alloc_try_nid: 4 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 pcpu_setup_first_chunk+0x110/0x678
memblock_reserve: [0x8fdfca00-0x8fdfca03] memblock_virt_alloc_internal+0xfc/0x1c0
memblock_virt_alloc_try_nid: 4 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 pcpu_setup_first_chunk+0x130/0x678
memblock_reserve: [0x8fdfc9c0-0x8fdfc9c3] memblock_virt_alloc_internal+0xfc/0x1c0
memblock_virt_alloc_try_nid: 4 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 pcpu_setup_first_chunk+0x150/0x678
memblock_reserve: [0x8fdfc980-0x8fdfc983] memblock_virt_alloc_internal+0xfc/0x1c0
pcpu-alloc: s41644 r8192 d23892 u73728 alloc=18*4096
pcpu-alloc: [0] 0
memblock_virt_alloc_try_nid: 128 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 pcpu_setup_first_chunk+0x41c/0x678
memblock_reserve: [0x8fdfc900-0x8fdfc97f] memblock_virt_alloc_internal+0xfc/0x1c0
memblock_virt_alloc_try_nid: 69 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 pcpu_alloc_first_chunk+0x64/0x2b4
memblock_reserve: [0x8fdfc880-0x8fdfc8c4] memblock_virt_alloc_internal+0xfc/0x1c0
memblock_virt_alloc_try_nid: 384 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 pcpu_alloc_first_chunk+0xa8/0x2b4
memblock_reserve: [0x8fdfc700-0x8fdfc87f] memblock_virt_alloc_internal+0xfc/0x1c0
memblock_virt_alloc_try_nid: 388 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 pcpu_alloc_first_chunk+0xc8/0x2b4
memblock_reserve: [0x8fdfc540-0x8fdfc6c3] memblock_virt_alloc_internal+0xfc/0x1c0
memblock_virt_alloc_try_nid: 60 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 pcpu_alloc_first_chunk+0xf4/0x2b4
memblock_reserve: [0x8fdfc500-0x8fdfc53b] memblock_virt_alloc_internal+0xfc/0x1c0
memblock_virt_alloc_try_nid: 69 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 pcpu_alloc_first_chunk+0x64/0x2b4
memblock_reserve: [0x8fdfc480-0x8fdfc4c4] memblock_virt_alloc_internal+0xfc/0x1c0
memblock_virt_alloc_try_nid: 768 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 pcpu_alloc_first_chunk+0xa8/0x2b4
memblock_reserve: [0x8fdfc180-0x8fdfc47f] memblock_virt_alloc_internal+0xfc/0x1c0
memblock_virt_alloc_try_nid: 772 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 pcpu_alloc_first_chunk+0xc8/0x2b4
memblock_reserve: [0x8fdae200-0x8fdae503] memblock_virt_alloc_internal+0xfc/0x1c0
memblock_virt_alloc_try_nid: 120 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 pcpu_alloc_first_chunk+0xf4/0x2b4
memblock_reserve: [0x8fdfc100-0x8fdfc177] memblock_virt_alloc_internal+0xfc/0x1c0
__memblock_free_early: [0x0000008fdaf540-0x0000008fdb053f] pcpu_embed_first_chunk+0x664/0x714
__memblock_free_early: [0x0000008fdae540-0x0000008fdaf53f] pcpu_embed_first_chunk+0x6c4/0x714
Built 1 zonelists, mobility grouping on. Total pages: 64452
Kernel command line: root=/dev/nfs ip=dhcp console=ttyO2,115200 memmap=2M$0x88000000 ramoops.mem_address=0x88000000 ramoops.mem_size=0x200000 ramoops.record_size=0x40000 debug earlyprintk init=/root/init memblock_virt_alloc_try_nid_nopanic: 4096 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 alloc_large_system_hash+0x180/0x268
memblock_reserve: [0x8fdaf540-0x8fdb053f] memblock_virt_alloc_internal+0xfc/0x1c0
PID hash table entries: 1024 (order: 0, 4096 bytes)
memblock_virt_alloc_try_nid_nopanic: 131072 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 alloc_large_system_hash+0x180/0x268
memblock_reserve: [0x8fd7c000-0x8fd9bfff] memblock_virt_alloc_internal+0xfc/0x1c0
Dentry cache hash table entries: 32768 (order: 5, 131072 bytes)
memblock_virt_alloc_try_nid_nopanic: 65536 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 alloc_large_system_hash+0x180/0x268
memblock_reserve: [0x8fd6c000-0x8fd7bfff] memblock_virt_alloc_internal+0xfc/0x1c0
Inode-cache hash table entries: 16384 (order: 4, 65536 bytes)
Memory: 220052K/260096K available (8192K kernel code, 816K rwdata, 2420K rodata, 1024K init, 7557K bss, 23660K reserved, 16384K cma-reserved, 0K highmem)
Virtual kernel memory layout:

    vector  : 0xffff0000 - 0xffff1000   (   4 kB)
    fixmap  : 0xffc00000 - 0xfff00000   (3072 kB)
    vmalloc : 0xd0000000 - 0xff800000   ( 760 MB)
    lowmem  : 0xc0000000 - 0xcfe00000   ( 254 MB)
    pkmap   : 0xbfe00000 - 0xc0000000   (   2 MB)
    modules : 0xbf000000 - 0xbfe00000   (  14 MB)
      .text : 0xc0008000 - 0xc0900000   (9184 kB)
      .init : 0xc0c00000 - 0xc0d00000   (1024 kB)
      .data : 0xc0d00000 - 0xc0dcc280   ( 817 kB)
       .bss : 0xc0dce000 - 0xc152f644   (7558 kB)

Running RCU self tests
Hierarchical RCU implementation.
RCU event tracing is enabled.
RCU lockdep checking is enabled.
RCU restricting CPUs from NR_CPUS=2 to nr_cpu_ids=1.
RCU: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1
NR_IRQS: 16, nr_irqs: 16, preallocated irqs: 16
IRQ: Found an INTC at 0xfa200000 (revision 4.0) with 96 interrupts
Clocking rate (Crystal/Core/MPU): 19.2/332/500 MHz
OMAP clockevent source: timer1 at 32768 Hz
clocksource: 32k_counter: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 58327039986419 ns
sched_clock: 32 bits at 32kHz, resolution 30517ns, wraps every 65535999984741ns OMAP clocksource: 32k_counter at 32768 Hz
Console: colour dummy device 80x30
WARNING: Your 'console=ttyO2' has been replaced by 'ttyS2'
This ensures that you still see kernel messages. Please
update your kernel commandline.
Lock dependency validator: Copyright (c) 2006 Red Hat, Inc., Ingo Molnar

... MAX_LOCKDEP_SUBCLASSES:  8
... MAX_LOCK_DEPTH:          48
... MAX_LOCKDEP_KEYS:        8191
... CLASSHASH_SIZE:          4096
... MAX_LOCKDEP_ENTRIES:     32768
... MAX_LOCKDEP_CHAINS:      65536
... CHAINHASH_SIZE:          32768

memory used by lock dependency info: 4655 kB per task-struct memory footprint: 1536 bytes
Calibrating delay loop... 493.97 BogoMIPS (lpj=2469888)
...

#22 lore raw

Hello,

Sorry for dealy. I was on vacation during last week.

Quoted 14 lines · Tony Lindgren
Quoted 13 lines · Joonsoo Kim
Quoted 2 lines · Tony Lindgren

Great, this branch boots on n900! Early parts of the dmesg attached below.

Sound good. Perhaps, problem is due to the cache management. With my patchset, direct mapping for CMA area is cleared in dma_contiguous_remap() if CONFIG_HIGHMEM. I guess that proper cache operation is required there.

Could you test my newly updated branch again? I re-enable dma_contiguous_remap() to clear direct mapping for CMA area and add proper(?) cache operation although I'm not the expert on that area.

Quoted 5 lines
Quoted 1 line

https://github.com/JoonsooKim/linux/tree/cma-debug4-next-20180901

Yeah that one boots too, dmesg below. I wonder why flushing the range with dmac_flush_range() and outer_flush_range() no longer seems enough though?

I don't know the reason. AFAIK, it should be enough to flush cpu cache before clearing page table entry, however, it doesn't work for n900.

flush_cache_all() has not only d-cache flushing but also i-cache flushing. So, I'd like to test effect of i-cache flushing.

Could you test follwing updated branch?

https://github.com/JoonsooKim/linux/tree/cma-debug4-next-20180901

It has three relevant commits on top and enables CMA memory use.

"arm/dma: remove i-cache flush"
"arm/dma: flush i-cache and d-cache separately"
"arm/dma: call flush_cache_all() and don't do outer cache operation"

I think that flush_cache_all() here looks fine because size of CMA area is usually large enough to use flush_cache_all() but understanding root cause would be important so I ask this test. If u don't mind, please test each commit with reverting one by one and let me know the result of each test.

Quoted 8 lines

Reverting the arch/arm/mm/dma-mapping.c changes in your patch "arm/dma: re-enable to remap the CMA area and flush cache before remapping" also boots, but produces the following build warnings:

arch/arm/mm/dma-mapping.c: In function 'dma_contiguous_remap':
arch/arm/mm/dma-mapping.c:503:20: warning: passing argument 1 of 'cpu_cache.dma_flush_range' makes pointer from integer without a cast [-Wint-conversion]

Quoted 8 lines
dmac_flush_range(map.virtual, map.virtual + map.length);
                    ^~~

arch/arm/mm/dma-mapping.c:503:20: note: expected 'const void *' but argument is of type 'long unsigned int' arch/arm/mm/dma-mapping.c:503:33: warning: passing argument 2 of 'cpu_cache.dma_flush_range' makes pointer from integer without a cast [-Wint-conversion]
dmac_flush_range(map.virtual, map.virtual + map.length);

Thanks for info.
I think that incorrect type would not be related to this problem.

Thanks.

maintainer
#23 lore raw

Hi,

Quoted 5 lines · Joonsoo Kim

Could you test follwing updated branch?

https://github.com/JoonsooKim/linux/tree/cma-debug4-next-20180901

It has three relevant commits on top and enables CMA memory use.

Okie dokie.

Quoted 1 line

"arm/dma: remove i-cache flush"

Your branch at the above commit is not booting on n900.

Quoted 1 line

"arm/dma: flush i-cache and d-cache separately"

Not booting at this commit either.

Quoted 1 line

"arm/dma: call flush_cache_all() and don't do outer cache operation"

Not booting at this commit either.

Earlier commit f14f3479c0d7 ("arm/dma: re-enable to remap the CMA area and flush cache before remapping") in you branch boots.

Then the following commit d6512d6d0171 ("Revert "arm/mm: disable atomic_pool"") won't boot.

Also your tree at commit 6d0525cef962 ("arm/dma: remove i-cache flush") with only commit d6512d6d0171 ("Revert "arm/mm: disable atomic_pool"") reverted boots on n900.

So it seems the issue is currently at the atomic_pool_init() related code?

Regards,

Tony

#24 lore raw
Quoted 26 lines · Tony Lindgren

Hi,

Quoted 5 lines · Joonsoo Kim

Could you test follwing updated branch?

https://github.com/JoonsooKim/linux/tree/cma-debug4-next-20180901

It has three relevant commits on top and enables CMA memory use.

Okie dokie.

Quoted 1 line

"arm/dma: remove i-cache flush"

Your branch at the above commit is not booting on n900.

Quoted 1 line

"arm/dma: flush i-cache and d-cache separately"

Not booting at this commit either.

Quoted 1 line

"arm/dma: call flush_cache_all() and don't do outer cache operation"

Not booting at this commit either.

Earlier commit f14f3479c0d7 ("arm/dma: re-enable to remap the CMA area and flush cache before remapping") in you branch boots.

Quoted 6 lines

Then the following commit d6512d6d0171 ("Revert "arm/mm: disable atomic_pool"") won't boot.

Also your tree at commit 6d0525cef962 ("arm/dma: remove i-cache flush") with only commit d6512d6d0171 ("Revert "arm/mm: disable atomic_pool"") reverted boots on n900.

Thanks a lot!

Quoted 2 lines

So it seems the issue is currently at the atomic_pool_init() related code?

Yes, your test showed it although I can't find any clue in atomic_pool_init().

Could you test updated branch?

https://github.com/JoonsooKim/linux/tree/cma-debug4-next-20180901

There are two relevant commits.

arm/dma: stop dma allocation before __dma_alloc_remap() arm/dma: disable atomic pool after dma allocation

atomic pool initialization will be done partially to check exact point of failure. These are brain-dead commits however I have no idea what's going on here until now. :/

Thanks.

maintainer
#25 lore raw
Quoted 11 lines · Joonsoo Kim
Quoted 2 lines · Tony Lindgren

So it seems the issue is currently at the atomic_pool_init() related code?

Yes, your test showed it although I can't find any clue in atomic_pool_init().

Could you test updated branch?

https://github.com/JoonsooKim/linux/tree/cma-debug4-next-20180901

Quoted 8 lines

There are two relevant commits.

arm/dma: stop dma allocation before __dma_alloc_remap() arm/dma: disable atomic pool after dma allocation

atomic pool initialization will be done partially to check exact point of failure. These are brain-dead commits however I have no idea what's going on here until now. :/

OK that booted, dmesg output below. Hopefully that provides you with some more clues.

Regards,

Tony

8< ---------------------
Linux version 4.13.0-rc7-next-20170901-00024-gcf9a104b2f62 (tmlind at sampyla) (gcc version 6.3.1 20170109 (crosstool-NG crosstool-ng-1.23.0)) #568 SMP Wed Nov 8 08:25:47 PST 2017
CPU: ARMv7 Processor [411fc083] revision 3 (ARMv7), cr=10c5387d
CPU: PIPT / VIPT nonaliasing data cache, VIPT nonaliasing instruction cache
OF: fdt: Machine model: Nokia N900
memblock_add: [0x80000000-0x8fffffff] early_init_dt_scan_memory+0xec/0x158
Memory policy: Data cache writeback
memblock_reserve: [0x80100000-0x8152f643] arm_memblock_init+0x30/0x1b0
memblock_reserve: [0x80004000-0x80007fff] arm_memblock_init+0x13c/0x1b0
memblock_reserve: [0x8ff00000-0x8fffffff] memblock_alloc_range_nid+0x38/0x50
memblock_free: [0x8ff00000-0x8fffffff] arm_memblock_steal+0x30/0x48
memblock_reserve: [0x8fe00000-0x8fefffff] memblock_alloc_range_nid+0x38/0x50
memblock_free: [0x8fe00000-0x8fefffff] arm_memblock_steal+0x30/0x48
memblock_reserve: [0x8fccb000-0x8fcddfff] arm_memblock_init+0x150/0x1b0
memblock_reserve: [0x8fccb000-0x8fcddfff] early_init_fdt_scan_reserved_mem+0x58/0x7c
memblock_reserve: [0x8e800000-0x8f7fffff] memblock_alloc_range_nid+0x38/0x50
cma: Reserved 16 MiB@0x8e800000
MEMBLOCK configuration:
memory size = 0x0fe00000 reserved size = 0x02446644
memory.cnt = 0x1
memory[0x0] [0x80000000-0x8fdfffff], 0x0fe00000 bytes flags: 0x0
reserved.cnt = 0x4
reserved[0x0] [0x80004000-0x80007fff], 0x00004000 bytes flags: 0x0
reserved[0x1] [0x80100000-0x8152f643], 0x0142f644 bytes flags: 0x0
reserved[0x2] [0x8e800000-0x8f7fffff], 0x01000000 bytes flags: 0x0
reserved[0x3] [0x8fccb000-0x8fcddfff], 0x00013000 bytes flags: 0x0
CMA_ADDR: __phys_to_virt_debug: 0x8e800000 to 0xce800000

CPU: 0 PID: 0 Comm: swapper Not tainted 4.13.0-rc7-next-20170901-00024-gcf9a104b2f62 #568
Hardware name: Nokia RX-51 board
[<c0110b38>] (unwind_backtrace) from [<c010caec>] (show_stack+0x10/0x14)
[<c010caec>] (show_stack) from [<c082eb04>] (dump_stack+0xac/0xe0)
[<c082eb04>] (dump_stack) from [<c0c06e94>] (dma_contiguous_remap+0x70/0x144)
[<c0c06e94>] (dma_contiguous_remap) from [<c0c08068>] (paging_init+0x324/0x700)
[<c0c08068>] (paging_init) from [<c0c042ac>] (setup_arch+0x5b4/0xc94)
[<c0c042ac>] (setup_arch) from [<c0c00940>] (start_kernel+0x54/0x3fc)
[<c0c00940>] (start_kernel) from [<8000807c>] (0x8000807c)
CMA_ADDR: __phys_to_virt_debug: 0x8e800000 to 0xce800000
CPU: 0 PID: 0 Comm: swapper Not tainted 4.13.0-rc7-next-20170901-00024-gcf9a104b2f62 #568
Hardware name: Nokia RX-51 board
[<c0110b38>] (unwind_backtrace) from [<c010caec>] (show_stack+0x10/0x14)
[<c010caec>] (show_stack) from [<c082eb04>] (dump_stack+0xac/0xe0)
[<c082eb04>] (dump_stack) from [<c0c06eb8>] (dma_contiguous_remap+0x94/0x144)
[<c0c06eb8>] (dma_contiguous_remap) from [<c0c08068>] (paging_init+0x324/0x700)
[<c0c08068>] (paging_init) from [<c0c042ac>] (setup_arch+0x5b4/0xc94)
[<c0c042ac>] (setup_arch) from [<c0c00940>] (start_kernel+0x54/0x3fc)
[<c0c00940>] (start_kernel) from [<8000807c>] (0x8000807c)
CMA_ADDR: __phys_to_virt_debug: 0x8e800000 to 0xce800000
CPU: 0 PID: 0 Comm: swapper Not tainted 4.13.0-rc7-next-20170901-00024-gcf9a104b2f62 #568
Hardware name: Nokia RX-51 board
[<c0110b38>] (unwind_backtrace) from [<c010caec>] (show_stack+0x10/0x14)
[<c010caec>] (show_stack) from [<c082eb04>] (dump_stack+0xac/0xe0)
[<c082eb04>] (dump_stack) from [<c0c06f24>] (dma_contiguous_remap+0x100/0x144)
[<c0c06f24>] (dma_contiguous_remap) from [<c0c08068>] (paging_init+0x324/0x700)
[<c0c08068>] (paging_init) from [<c0c042ac>] (setup_arch+0x5b4/0xc94)
[<c0c042ac>] (setup_arch) from [<c0c00940>] (start_kernel+0x54/0x3fc)

[<c0c00940>] (start_kernel) from [<8000807c>] (0x8000807c)
memblock_reserve: [0x8fdfe000-0x8fdfffff] memblock_alloc_range_nid+0x38/0x50
memblock_reserve: [0x8fdfd000-0x8fdfdfff] memblock_alloc_range_nid+0x38/0x50
memblock_reserve: [0x8fdfcee8-0x8fdfcfff] memblock_alloc_range_nid+0x38/0x50
memblock_reserve: [0x8fdfcec0-0x8fdfcee7] memblock_alloc_range_nid+0x38/0x50
memblock_reserve: [0x8fdfce98-0x8fdfcebf] memblock_alloc_range_nid+0x38/0x50
memblock_reserve: [0x8fdfce70-0x8fdfce97] memblock_alloc_range_nid+0x38/0x50
memblock_reserve: [0x8fdfce48-0x8fdfce6f] memblock_alloc_range_nid+0x38/0x50
memblock_reserve: [0x8fdfce20-0x8fdfce47] memblock_alloc_range_nid+0x38/0x50
memblock_reserve: [0x8fdfb000-0x8fdfbfff] memblock_alloc_range_nid+0x38/0x50
memblock_reserve: [0x8fdfa000-0x8fdfafff] memblock_alloc_range_nid+0x38/0x50
memblock_reserve: [0x8fdf9000-0x8fdf9fff] memblock_alloc_range_nid+0x38/0x50
On node 0 totalpages: 65024
memblock_virt_alloc_try_nid_nopanic: 2359296 bytes align=0x0 nid=0 from=0x0 max_addr=0x0 alloc_node_mem_map.constprop.10+0x68/0xb4
memblock_reserve: [0x8fa8b000-0x8fccafff] memblock_virt_alloc_internal+0xfc/0x1c0
free_area_init_node: node 0, pgdat c0dc5040, node_mem_map cfa8b000
Normal zone: 572 pages used for memmap
Normal zone: 0 pages reserved
Normal zone: 65024 pages, LIFO batch:15
memblock_virt_alloc_try_nid_nopanic: 16 bytes align=0x0 nid=0 from=0x0 max_addr=0x0 setup_usemap.constprop.12+0x5c/0x6c
memblock_reserve: [0x8fdfce00-0x8fdfce0f] memblock_virt_alloc_internal+0xfc/0x1c0
memblock_virt_alloc_try_nid_nopanic: 16 bytes align=0x0 nid=0 from=0x0 max_addr=0x0 setup_usemap.constprop.12+0x5c/0x6c
memblock_reserve: [0x8fdfcdc0-0x8fdfcdcf] memblock_virt_alloc_internal+0xfc/0x1c0
memblock_virt_alloc_try_nid: 32 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 setup_arch+0x6e8/0xc94
memblock_reserve: [0x8fdfcd80-0x8fdfcd9f] memblock_virt_alloc_internal+0xfc/0x1c0
memblock_reserve: [0x8fdb0548-0x8fdf8fff] memblock_alloc_range_nid+0x38/0x50
memblock_reserve: [0x8fdfcde8-0x8fdfcdff] memblock_alloc_range_nid+0x38/0x50
memblock_reserve: [0x8fdfcdd0-0x8fdfcde7] memblock_alloc_range_nid+0x38/0x50
memblock_reserve: [0x8fdfcda4-0x8fdfcdbe] memblock_alloc_range_nid+0x38/0x50
memblock_reserve: [0x8fdfcd64-0x8fdfcd7e] memblock_alloc_range_nid+0x38/0x50
memblock_reserve: [0x8fdfcd48-0x8fdfcd62] memblock_alloc_range_nid+0x38/0x50
memblock_reserve: [0x8fdfcd30-0x8fdfcd47] memblock_alloc_range_nid+0x38/0x50
CPU: All CPU(s) started in SVC mode.
OMAP3430/3530 ES3.1 (l2cache iva sgx neon isp)
memblock_virt_alloc_try_nid: 8 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 omap2_clk_legacy_provider_init+0x2c/0x44
memblock_reserve: [0x8fdfcd00-0x8fdfcd07] memblock_virt_alloc_internal+0xfc/0x1c0
memblock_virt_alloc_try_nid: 8 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 omap2_clk_legacy_provider_init+0x2c/0x44
memblock_reserve: [0x8fdfccc0-0x8fdfccc7] memblock_virt_alloc_internal+0xfc/0x1c0
random: fast init done
memblock_virt_alloc_try_nid: 183 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 start_kernel+0x9c/0x3fc
memblock_reserve: [0x8fdfcc00-0x8fdfccb6] memblock_virt_alloc_internal+0xfc/0x1c0
memblock_virt_alloc_try_nid: 183 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 start_kernel+0xc0/0x3fc
memblock_reserve: [0x8fdfcb40-0x8fdfcbf6] memblock_virt_alloc_internal+0xfc/0x1c0
memblock_virt_alloc_try_nid: 183 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 start_kernel+0xe4/0x3fc
memblock_reserve: [0x8fdfca80-0x8fdfcb36] memblock_virt_alloc_internal+0xfc/0x1c0
memblock_virt_alloc_try_nid_nopanic: 4096 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 pcpu_alloc_alloc_info+0x4c/0x88
memblock_reserve: [0x8fdaf540-0x8fdb053f] memblock_virt_alloc_internal+0xfc/0x1c0
memblock_virt_alloc_try_nid_nopanic: 4096 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 pcpu_embed_first_chunk+0x464/0x714
memblock_reserve: [0x8fdae540-0x8fdaf53f] memblock_virt_alloc_internal+0xfc/0x1c0
memblock_virt_alloc_try_nid_nopanic: 73728 bytes align=0x1000 nid=-1 from=0xbfffffff max_addr=0x0 pcpu_dfl_fc_alloc+0x3c/0x44
memblock_reserve: [0x8fd9c000-0x8fdadfff] memblock_virt_alloc_internal+0xfc/0x1c0
__memblock_free_early: [0x0000008fdae000-0x0000008fdadfff] pcpu_embed_first_chunk+0x5d8/0x714
percpu: Embedded 18 pages/cpu @cfd9c000 s41644 r8192 d23892 u73728
memblock_virt_alloc_try_nid: 4 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 pcpu_setup_first_chunk+0xf0/0x678
memblock_reserve: [0x8fdfca40-0x8fdfca43] memblock_virt_alloc_internal+0xfc/0x1c0
memblock_virt_alloc_try_nid: 4 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 pcpu_setup_first_chunk+0x110/0x678
memblock_reserve: [0x8fdfca00-0x8fdfca03] memblock_virt_alloc_internal+0xfc/0x1c0
memblock_virt_alloc_try_nid: 4 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 pcpu_setup_first_chunk+0x130/0x678
memblock_reserve: [0x8fdfc9c0-0x8fdfc9c3] memblock_virt_alloc_internal+0xfc/0x1c0
memblock_virt_alloc_try_nid: 4 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 pcpu_setup_first_chunk+0x150/0x678
memblock_reserve: [0x8fdfc980-0x8fdfc983] memblock_virt_alloc_internal+0xfc/0x1c0
pcpu-alloc: s41644 r8192 d23892 u73728 alloc=18*4096
pcpu-alloc: [0] 0
memblock_virt_alloc_try_nid: 128 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 pcpu_setup_first_chunk+0x41c/0x678
memblock_reserve: [0x8fdfc900-0x8fdfc97f] memblock_virt_alloc_internal+0xfc/0x1c0
memblock_virt_alloc_try_nid: 69 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 pcpu_alloc_first_chunk+0x64/0x2b4
memblock_reserve: [0x8fdfc880-0x8fdfc8c4] memblock_virt_alloc_internal+0xfc/0x1c0
memblock_virt_alloc_try_nid: 384 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 pcpu_alloc_first_chunk+0xa8/0x2b4
memblock_reserve: [0x8fdfc700-0x8fdfc87f] memblock_virt_alloc_internal+0xfc/0x1c0
memblock_virt_alloc_try_nid: 388 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 pcpu_alloc_first_chunk+0xc8/0x2b4
memblock_reserve: [0x8fdfc540-0x8fdfc6c3] memblock_virt_alloc_internal+0xfc/0x1c0
memblock_virt_alloc_try_nid: 60 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 pcpu_alloc_first_chunk+0xf4/0x2b4
memblock_reserve: [0x8fdfc500-0x8fdfc53b] memblock_virt_alloc_internal+0xfc/0x1c0
memblock_virt_alloc_try_nid: 69 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 pcpu_alloc_first_chunk+0x64/0x2b4
memblock_reserve: [0x8fdfc480-0x8fdfc4c4] memblock_virt_alloc_internal+0xfc/0x1c0
memblock_virt_alloc_try_nid: 768 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 pcpu_alloc_first_chunk+0xa8/0x2b4
memblock_reserve: [0x8fdfc180-0x8fdfc47f] memblock_virt_alloc_internal+0xfc/0x1c0
memblock_virt_alloc_try_nid: 772 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 pcpu_alloc_first_chunk+0xc8/0x2b4
memblock_reserve: [0x8fdae200-0x8fdae503] memblock_virt_alloc_internal+0xfc/0x1c0
memblock_virt_alloc_try_nid: 120 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 pcpu_alloc_first_chunk+0xf4/0x2b4
memblock_reserve: [0x8fdfc100-0x8fdfc177] memblock_virt_alloc_internal+0xfc/0x1c0
__memblock_free_early: [0x0000008fdaf540-0x0000008fdb053f] pcpu_embed_first_chunk+0x664/0x714
__memblock_free_early: [0x0000008fdae540-0x0000008fdaf53f] pcpu_embed_first_chunk+0x6c4/0x714
Built 1 zonelists, mobility grouping on. Total pages: 64452
Kernel command line: root=/dev/nfs ip=dhcp console=ttyO2,115200 memmap=2M$0x88000000 ramoops.mem_address=0x88000000 ramoops.mem_size=0x200000 ramoops.record_size=0x40000 debug earlyprintk init=/root/init memblock_virt_alloc_try_nid_nopanic: 4096 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 alloc_large_system_hash+0x180/0x268
memblock_reserve: [0x8fdaf540-0x8fdb053f] memblock_virt_alloc_internal+0xfc/0x1c0
PID hash table entries: 1024 (order: 0, 4096 bytes)
memblock_virt_alloc_try_nid_nopanic: 131072 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 alloc_large_system_hash+0x180/0x268
memblock_reserve: [0x8fd7c000-0x8fd9bfff] memblock_virt_alloc_internal+0xfc/0x1c0
Dentry cache hash table entries: 32768 (order: 5, 131072 bytes)
memblock_virt_alloc_try_nid_nopanic: 65536 bytes align=0x0 nid=-1 from=0x0 max_addr=0x0 alloc_large_system_hash+0x180/0x268
memblock_reserve: [0x8fd6c000-0x8fd7bfff] memblock_virt_alloc_internal+0xfc/0x1c0
Inode-cache hash table entries: 16384 (order: 4, 65536 bytes)
Memory: 220052K/260096K available (8192K kernel code, 816K rwdata, 2420K rodata, 1024K init, 7557K bss, 23660K reserved, 16384K cma-reserved, 0K highmem)
Virtual kernel memory layout:

    vector  : 0xffff0000 - 0xffff1000   (   4 kB)
    fixmap  : 0xffc00000 - 0xfff00000   (3072 kB)
    vmalloc : 0xd0000000 - 0xff800000   ( 760 MB)
    lowmem  : 0xc0000000 - 0xcfe00000   ( 254 MB)
    pkmap   : 0xbfe00000 - 0xc0000000   (   2 MB)
    modules : 0xbf000000 - 0xbfe00000   (  14 MB)
      .text : 0xc0008000 - 0xc0900000   (9184 kB)
      .init : 0xc0c00000 - 0xc0d00000   (1024 kB)
      .data : 0xc0d00000 - 0xc0dcc280   ( 817 kB)
       .bss : 0xc0dce000 - 0xc152f644   (7558 kB)

Running RCU self tests
Hierarchical RCU implementation.
RCU event tracing is enabled.
RCU lockdep checking is enabled.
RCU restricting CPUs from NR_CPUS=2 to nr_cpu_ids=1.
RCU: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1
NR_IRQS: 16, nr_irqs: 16, preallocated irqs: 16
IRQ: Found an INTC at 0xfa200000 (revision 4.0) with 96 interrupts
Clocking rate (Crystal/Core/MPU): 19.2/332/500 MHz
OMAP clockevent source: timer1 at 32768 Hz
clocksource: 32k_counter: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 58327039986419 ns
sched_clock: 32 bits at 32kHz, resolution 30517ns, wraps every 65535999984741ns OMAP clocksource: 32k_counter at 32768 Hz
Console: colour dummy device 80x30
WARNING: Your 'console=ttyO2' has been replaced by 'ttyS2'
This ensures that you still see kernel messages. Please
update your kernel commandline.
Lock dependency validator: Copyright (c) 2006 Red Hat, Inc., Ingo Molnar

... MAX_LOCKDEP_SUBCLASSES:  8
... MAX_LOCK_DEPTH:          48
... MAX_LOCKDEP_KEYS:        8191
... CLASSHASH_SIZE:          4096
... MAX_LOCKDEP_ENTRIES:     32768
... MAX_LOCKDEP_CHAINS:      65536
... CHAINHASH_SIZE:          32768

memory used by lock dependency info: 4655 kB
per task-struct memory footprint: 1536 bytes
Calibrating delay loop... 493.97 BogoMIPS (lpj=2469888)
pid_max: default: 32768 minimum: 301
Security Framework initialized
Mount-cache hash table entries: 1024 (order: 0, 4096 bytes)
Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes)
CPU: Testing write buffer coherency: ok
CPU0: thread -1, cpu 0, socket -1, mpidr 0
Setting up static identity map for 0x80100000 - 0x80100078
Hierarchical SRCU implementation.
smp: Bringing up secondary CPUs ...
smp: Brought up 1 node, 1 CPU
SMP: Total of 1 processors activated (493.97 BogoMIPS).
CPU: All CPU(s) started in SVC mode.
devtmpfs: initialized
Built 1 zonelists, mobility grouping on. Total pages: 59109
VFP support v0.3: implementor 41 architecture 3 part 30 variant c rev 1
clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns futex hash table entries: 256 (order: 2, 16384 bytes)
pinctrl core: initialized pinctrl subsystem
NET: Registered protocol family 16
__alloc_from_contiguous

CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.13.0-rc7-next-20170901-00024-gcf9a104b2f62 #568
Hardware name: Nokia RX-51 board
[<c0110b38>] (unwind_backtrace) from [<c010caec>] (show_stack+0x10/0x14)
[<c010caec>] (show_stack) from [<c082eb04>] (dump_stack+0xac/0xe0)
[<c082eb04>] (dump_stack) from [<c0118b94>] (__alloc_from_contiguous.constprop.7+0x28/0x68)
[<c0118b94>] (__alloc_from_contiguous.constprop.7) from [<c0c06d3c>] (atomic_pool_init+0x58/0xf0)
[<c0c06d3c>] (atomic_pool_init) from [<c0101df4>] (do_one_initcall+0x3c/0x170)
[<c0101df4>] (do_one_initcall) from [<c0c00ee4>] (kernel_init_freeable+0x1fc/0x2c4)
[<c0c00ee4>] (kernel_init_freeable) from [<c0842a4c>] (kernel_init+0x8/0x110)
[<c0842a4c>] (kernel_init) from [<c0107d78>] (ret_from_fork+0x14/0x3c)

atomic_pool_init: DMA: disable atomic_pool
omap_hwmod: mcbsp2_sidetone using broken dt data from mcbsp
omap_hwmod: mcbsp3_sidetone using broken dt data from mcbsp
cpuidle: using governor menu
SRAM_ADDR: omap_map_sram: P: 0x40208000 - 0x4020f000
SRAM_ADDR: omap_map_sram: V: 0xd0008000 - 0xd000f000
SRAM_ADDR: omap_sram_push_address: 0xd000ef00 - 0xd000effc
SRAM_ADDR: omap_sram_push_address: 0xd000ee90 - 0xd000eefc Reprogramming SDRC clock to 332000000 Hz
...

#26 lore raw
Quoted 17 lines · Tony Lindgren
Quoted 16 lines · Joonsoo Kim
Quoted 2 lines · Tony Lindgren

So it seems the issue is currently at the atomic_pool_init() related code?

Yes, your test showed it although I can't find any clue in atomic_pool_init().

Could you test updated branch?

https://github.com/JoonsooKim/linux/tree/cma-debug4-next-20180901

There are two relevant commits.

arm/dma: stop dma allocation before __dma_alloc_remap() arm/dma: disable atomic pool after dma allocation

Quoted 6 lines
Quoted 3 lines

atomic pool initialization will be done partially to check exact point of failure. These are brain-dead commits however I have no idea what's going on here until now. :/

OK that booted, dmesg output below. Hopefully that provides you with some more clues.

Thanks!

Could you let me know which one is booted? Both of them? or just top commit ("arm/dma: stop dma allocation before __dma_alloc_remap()")?

Thanks.

maintainer
#27 lore raw
Quoted 22 lines · Joonsoo Kim
Quoted 21 lines · Tony Lindgren
Quoted 19 lines · Joonsoo Kim
Quoted 2 lines · Tony Lindgren

So it seems the issue is currently at the atomic_pool_init() related code?

Yes, your test showed it although I can't find any clue in atomic_pool_init().

Could you test updated branch?

https://github.com/JoonsooKim/linux/tree/cma-debug4-next-20180901

There are two relevant commits.

arm/dma: stop dma allocation before __dma_alloc_remap() arm/dma: disable atomic pool after dma allocation

atomic pool initialization will be done partially to check exact point of failure. These are brain-dead commits however I have no idea what's going on here until now. :/

Quoted 7 lines
Quoted 2 lines

OK that booted, dmesg output below. Hopefully that provides you with some more clues.

Thanks!

Could you let me know which one is booted? Both of them? or just top commit ("arm/dma: stop dma allocation before __dma_alloc_remap()")?

Oh OK. Only the top commit boots.

Regards,

Tony

#28 lore raw
Quoted 26 lines · Tony Lindgren
Quoted 25 lines · Joonsoo Kim
Quoted 23 lines · Tony Lindgren
Quoted 11 lines · Joonsoo Kim
Quoted 2 lines · Tony Lindgren

So it seems the issue is currently at the atomic_pool_init() related code?

Yes, your test showed it although I can't find any clue in atomic_pool_init().

Could you test updated branch?

https://github.com/JoonsooKim/linux/tree/cma-debug4-next-20180901

Quoted 8 lines

There are two relevant commits.

arm/dma: stop dma allocation before __dma_alloc_remap() arm/dma: disable atomic pool after dma allocation

atomic pool initialization will be done partially to check exact point of failure. These are brain-dead commits however I have no idea what's going on here until now. :/

OK that booted, dmesg output below. Hopefully that provides you with some more clues.

Quoted 6 lines
Quoted 4 lines

Thanks!

Could you let me know which one is booted? Both of them? or just top commit ("arm/dma: stop dma allocation before __dma_alloc_remap()")?

Oh OK. Only the top commit boots.

Okay! I will try to analyze!

Thanks.

#29 lore raw
Quoted 27 lines · Joonsoo Kim
Quoted 26 lines · Tony Lindgren
Quoted 25 lines · Joonsoo Kim
Quoted 23 lines · Tony Lindgren
Quoted 11 lines · Joonsoo Kim
Quoted 2 lines · Tony Lindgren

So it seems the issue is currently at the atomic_pool_init() related code?

Yes, your test showed it although I can't find any clue in atomic_pool_init().

Could you test updated branch?

https://github.com/JoonsooKim/linux/tree/cma-debug4-next-20180901

Quoted 8 lines

There are two relevant commits.

arm/dma: stop dma allocation before __dma_alloc_remap() arm/dma: disable atomic pool after dma allocation

atomic pool initialization will be done partially to check exact point of failure. These are brain-dead commits however I have no idea what's going on here until now. :/

OK that booted, dmesg output below. Hopefully that provides you with some more clues.

Quoted 9 lines
Quoted 6 lines
Quoted 4 lines

Thanks!

Could you let me know which one is booted? Both of them? or just top commit ("arm/dma: stop dma allocation before __dma_alloc_remap()")?

Oh OK. Only the top commit boots.

Okay! I will try to analyze!

Could you test following two commits on my updated branch?

"arm/dma: vmalloc area allocation"
"arm/dma: defer atomic pool initialization"

I suspect that changed virtual address of the sram due to early __dma_alloc_remap() call causes the problem and above two commits test this theory.

Thanks.

maintainer
#30 lore raw
Quoted 3 lines · Joonsoo Kim

Could you test following two commits on my updated branch?

"arm/dma: vmalloc area allocation"

Won't boot at this commit:

[    6.747283] save_secure_sram() returns 0000ff02
[    6.751983] save_secure_sram()'s param: 0: 0x4
[    6.756561] save_secure_sram()'s param: 1: 0x8e700000
[    6.761749] save_secure_sram()'s param: 2: 0x0
[    6.766326] save_secure_sram()'s param: 3: 0x1
[    6.770904] save_secure_sram()'s param: 4: 0x1
Quoted 1 line

"arm/dma: defer atomic pool initialization"

Boots at this commit.

Quoted 3 lines

I suspect that changed virtual address of the sram due to early __dma_alloc_remap() call causes the problem and above two commits test this theory.

Hmm OK. Does your first patch above now have the initcall issue too? It boots if I make that also subsys_initcall and then I get:

[    2.078094] vmalloc_pool_init: DMA: get vmalloc area: d0010000

Regards,

Tony

#31 lore raw
Quoted 18 lines · Tony Lindgren
Quoted 3 lines · Joonsoo Kim

Could you test following two commits on my updated branch?

"arm/dma: vmalloc area allocation"

Won't boot at this commit:

[    6.747283] save_secure_sram() returns 0000ff02
[    6.751983] save_secure_sram()'s param: 0: 0x4
[    6.756561] save_secure_sram()'s param: 1: 0x8e700000
[    6.761749] save_secure_sram()'s param: 2: 0x0
[    6.766326] save_secure_sram()'s param: 3: 0x1
[    6.770904] save_secure_sram()'s param: 4: 0x1
Quoted 1 line

"arm/dma: defer atomic pool initialization"

Boots at this commit.

Quoted 8 lines
Quoted 3 lines

I suspect that changed virtual address of the sram due to early __dma_alloc_remap() call causes the problem and above two commits test this theory.

Hmm OK. Does your first patch above now have the initcall issue too? It boots if I make that also subsys_initcall and then I get:

[    2.078094] vmalloc_pool_init: DMA: get vmalloc area: d0010000

Yes, first patch has the initcall issue and it's intentional in order to check the theory. I checked following log for this.

- Boot failure
SRAM_ADDR: omap_map_sram: P: 0x40208000 - 0x4020f000
SRAM_ADDR: omap_map_sram: V: 0xd0050000 - 0xd0057000

- Boot success
SRAM_ADDR: omap_map_sram: P: 0x40208000 - 0x4020f000
SRAM_ADDR: omap_map_sram: V: 0xd0008000 - 0xd000f000

When failure, virtual address for sram is higher than normal one due to vmalloc area allocation in __dma_alloc_remap(). If it is deferred, virtual address is the same with success case and then the system work.

So, my next theory is that there is n900 specific assumption that sram should have that address. Could you check if any working tree for n900 which doesn't have my CMA series work or not with adding "arm/dma: vmalloc area allocation"?

Thanks.

maintainer
#32 lore raw
Quoted 17 lines · Joonsoo Kim
Quoted 4 lines · Tony Lindgren

Hmm OK. Does your first patch above now have the initcall issue too? It boots if I make that also subsys_initcall and then I get:

[    2.078094] vmalloc_pool_init: DMA: get vmalloc area: d0010000

Yes, first patch has the initcall issue and it's intentional in order to check the theory. I checked following log for this.

- Boot failure
SRAM_ADDR: omap_map_sram: P: 0x40208000 - 0x4020f000
SRAM_ADDR: omap_map_sram: V: 0xd0050000 - 0xd0057000

- Boot success
SRAM_ADDR: omap_map_sram: P: 0x40208000 - 0x4020f000
SRAM_ADDR: omap_map_sram: V: 0xd0008000 - 0xd000f000

Quoted 8 lines

When failure, virtual address for sram is higher than normal one due to vmalloc area allocation in __dma_alloc_remap(). If it is deferred, virtual address is the same with success case and then the system work.

So, my next theory is that there is n900 specific assumption that sram should have that address. Could you check if any working tree for n900 which doesn't have my CMA series work or not with adding "arm/dma: vmalloc area allocation"?

Oh I see, sorry I was not following you earlier. So you mean that by adding the vmalloc_pool_init() initcall the va mapping for SRAM changes.

And yes, save_secure_ram_context seems to be doing some sketchy virt to phys calculation with sram_phy_addr_mask. Here's a small patch to fix that for your CMA series, maybe you can merge it with your series to avoid breaking booting for git bisect.

Then I'll follow up on cleaning up save_secure_ram_context later.

Regards,

Tony

8< -------------------------

Quoted 1 line

From tony Mon Sep 17 00:00:00 2001

From: Tony Lindgren <[email protected]>
Date: Thu, 9 Nov 2017 17:05:34 -0800
Subject: [PATCH] ARM: OMAP2+: Add static SRAM mapping for
save_secure_ram_context

With the CMA changes from Joonsoo Kim <[email protected]>, it was noticed that n900 stopped booting. After investigating it turned out that n900 save_secure_ram_context does some whacky virtual to physical address translation for the SRAM data address.

Let's fix this for CMA changes by adding a static mapping for SRAM on omap3. Then we can follow up with a patch to clean up the address translation in save_secure_ram_context later on.

---
 arch/arm/mach-omap2/io.c    | 6 ++++++
 arch/arm/mach-omap2/iomap.h | 4 ++++
 2 files changed, 10 insertions(+)
arch/arm/mach-omap2/io.c +6 -0
@@ -139,6 +139,12 @@ static struct map_desc omap243x_io_desc[] __initdata = {
 
 #ifdef	CONFIG_ARCH_OMAP3
 static struct map_desc omap34xx_io_desc[] __initdata = {
+	{
+		.virtual	= OMAP34XX_SRAM_VIRT,
+		.pfn		= __phys_to_pfn(OMAP34XX_SRAM_PHYS),
+		.length		= OMAP34XX_SRAM_SIZE,
+		.type		= MT_DEVICE
+	},
 	{
 		.virtual	= L3_34XX_VIRT,
 		.pfn		= __phys_to_pfn(L3_34XX_PHYS),
arch/arm/mach-omap2/iomap.h +4 -0
@@ -123,6 +123,10 @@
  * VPOM3430 was not working for Int controller
  */
 
+#define OMAP34XX_SRAM_PHYS	0x40200000
+#define OMAP34XX_SRAM_VIRT	0xd0010000
+#define OMAP34XX_SRAM_SIZE	0x10000
+
 #define L4_PER_34XX_PHYS	L4_PER_34XX_BASE
 						/* 0x49000000 --> 0xfb000000 */
 #define L4_PER_34XX_VIRT	(L4_PER_34XX_PHYS + OMAP2_L4_IO_OFFSET)
Signature · 1 line
2.15.0
maintainer
#33 lore raw
Quoted 31 lines · Tony Lindgren
Quoted 17 lines · Joonsoo Kim
Quoted 4 lines · Tony Lindgren

Hmm OK. Does your first patch above now have the initcall issue too? It boots if I make that also subsys_initcall and then I get:

[    2.078094] vmalloc_pool_init: DMA: get vmalloc area: d0010000

Yes, first patch has the initcall issue and it's intentional in order to check the theory. I checked following log for this.

- Boot failure
SRAM_ADDR: omap_map_sram: P: 0x40208000 - 0x4020f000
SRAM_ADDR: omap_map_sram: V: 0xd0050000 - 0xd0057000

- Boot success
SRAM_ADDR: omap_map_sram: P: 0x40208000 - 0x4020f000
SRAM_ADDR: omap_map_sram: V: 0xd0008000 - 0xd000f000

Quoted 8 lines

When failure, virtual address for sram is higher than normal one due to vmalloc area allocation in __dma_alloc_remap(). If it is deferred, virtual address is the same with success case and then the system work.

So, my next theory is that there is n900 specific assumption that sram should have that address. Could you check if any working tree for n900 which doesn't have my CMA series work or not with adding "arm/dma: vmalloc area allocation"?

Oh I see, sorry I was not following you earlier. So you mean that by adding the vmalloc_pool_init() initcall the va mapping for SRAM changes.

Quoted 6 lines

And yes, save_secure_ram_context seems to be doing some sketchy virt to phys calculation with sram_phy_addr_mask. Here's a small patch to fix that for your CMA series, maybe you can merge it with your series to avoid breaking booting for git bisect.

Then I'll follow up on cleaning up save_secure_ram_context later.

Here's a better version, the static mapping did not get used.. It just moved the area so it happened to work. It needs to be set up as MT_MEMORY_RWX_NONCACHED instead.

Regards,

Tony

8< -------------------

Quoted 1 line

From tony Mon Sep 17 00:00:00 2001

From: Tony Lindgren <[email protected]>
Date: Thu, 9 Nov 2017 17:05:34 -0800
Subject: [PATCH] ARM: OMAP2+: Add static SRAM mapping for
save_secure_ram_context

With the CMA changes from Joonsoo Kim <[email protected]>, it was noticed that n900 stopped booting. After investigating it turned out that n900 save_secure_ram_context does some whacky virtual to physical address translation for the SRAM data address.

Let's fix this for CMA changes by adding a static mapping for SRAM on omap3. Then we can follow up with a patch to clean up the address translation in save_secure_ram_context later on.

---
 arch/arm/mach-omap2/io.c    | 6 ++++++
 arch/arm/mach-omap2/iomap.h | 4 ++++
 2 files changed, 10 insertions(+)
arch/arm/mach-omap2/io.c +6 -0
@@ -139,6 +139,12 @@ static struct map_desc omap243x_io_desc[] __initdata = {
 
 #ifdef	CONFIG_ARCH_OMAP3
 static struct map_desc omap34xx_io_desc[] __initdata = {
+	{
+		.virtual	= OMAP34XX_SRAM_VIRT,
+		.pfn		= __phys_to_pfn(OMAP34XX_SRAM_PHYS),
+		.length		= OMAP34XX_SRAM_SIZE,
+		.type		= MT_MEMORY_RWX_NONCACHED
+	},
 	{
 		.virtual	= L3_34XX_VIRT,
 		.pfn		= __phys_to_pfn(L3_34XX_PHYS),
arch/arm/mach-omap2/iomap.h +4 -0
@@ -123,6 +123,10 @@
  * VPOM3430 was not working for Int controller
  */
 
+#define OMAP34XX_SRAM_PHYS	0x40200000
+#define OMAP34XX_SRAM_VIRT	0xd0010000
+#define OMAP34XX_SRAM_SIZE	0x10000
+
 #define L4_PER_34XX_PHYS	L4_PER_34XX_BASE
 						/* 0x49000000 --> 0xfb000000 */
 #define L4_PER_34XX_VIRT	(L4_PER_34XX_PHYS + OMAP2_L4_IO_OFFSET)
Signature · 1 line
2.15.0
maintainer
#34 lore raw
Quoted 6 lines · Tony Lindgren
Quoted 1 line · Tony Lindgren

Then I'll follow up on cleaning up save_secure_ram_context later.

Here's a better version, the static mapping did not get used.. It just moved the area so it happened to work. It needs to be set up as MT_MEMORY_RWX_NONCACHED instead.

And FYI, here's what I currently have for the follow-up patch, but that can wait a bit.

Regards,

Tony

8< ------------------------

arch/arm/mach-omap2/sleep34xx.S +5 -10
@@ -45,7 +45,6 @@
 #define PM_PWSTCTRL_MPU_P	OMAP3430_PRM_BASE + MPU_MOD + OMAP2_PM_PWSTCTRL
 #define CM_IDLEST1_CORE_V	OMAP34XX_CM_REGADDR(CORE_MOD, CM_IDLEST1)
 #define CM_IDLEST_CKGEN_V	OMAP34XX_CM_REGADDR(PLL_MOD, CM_IDLEST)
-#define SRAM_BASE_P		OMAP3_SRAM_PA
 #define CONTROL_STAT		OMAP343X_CTRL_BASE + OMAP343X_CONTROL_STATUS
 #define CONTROL_MEM_RTA_CTRL	(OMAP343X_CTRL_BASE +\
 					OMAP36XX_CONTROL_MEM_RTA_CTRL)
@@ -103,10 +102,8 @@ ENTRY(save_secure_ram_context)
 	stmfd	sp!, {r4 - r11, lr}	@ save registers on stack
 	adr	r3, api_params		@ r3 points to parameters
 	str	r0, [r3,#0x4]		@ r0 has sdram address
-	ldr	r12, high_mask
-	and	r3, r3, r12
-	ldr	r12, sram_phy_addr_mask
-	orr	r3, r3, r12
+	ldr	r12, sram_phys_offset	@ load sram physical offset
+	sub	r3, r3, r12		@ parameters physical address
 	mov	r0, #25			@ set service ID for PPA
 	mov	r12, r0			@ copy secure service ID in r12
 	mov	r1, #0			@ set task id for ROM code in r1
@@ -121,10 +118,8 @@ ENTRY(save_secure_ram_context)
 	nop
 	ldmfd	sp!, {r4 - r11, pc}
 	.align
-sram_phy_addr_mask:
-	.word	SRAM_BASE_P
-high_mask:
-	.word	0xffff
+sram_phys_offset:
+	.word	OMAP34XX_SRAM_VIRT - OMAP34XX_SRAM_PHYS
 api_params:
 	.word	0x4, 0x0, 0x0, 0x1, 0x1
 ENDPROC(save_secure_ram_context)
@@ -521,7 +516,7 @@ pm_pwstctrl_mpu:
 scratchpad_base:
 	.word	SCRATCHPAD_BASE_P
 sram_base:
-	.word	SRAM_BASE_P + 0x8000
+	.word	OMAP34XX_SRAM_PHYS + 0x8000
 control_stat:
 	.word	CONTROL_STAT
 control_mem_rta:
arch/arm/mach-omap2/sram.c +2 -2
@@ -31,7 +31,7 @@
 #include "sram.h"
 
 #define OMAP2_SRAM_PUB_PA	(OMAP2_SRAM_PA + 0xf800)
-#define OMAP3_SRAM_PUB_PA       (OMAP3_SRAM_PA + 0x8000)
+#define OMAP3_SRAM_PUB_PA       (OMAP34XX_SRAM_PHYS + 0x8000)
 
 #define SRAM_BOOTLOADER_SZ	0x00
 
@@ -105,7 +105,7 @@ static void __init omap_detect_sram(void)
 		}
 	} else {
 		if (cpu_is_omap34xx()) {
-			omap_sram_start = OMAP3_SRAM_PA;
+			omap_sram_start = OMAP34XX_SRAM_PHYS;
 			omap_sram_size = 0x10000; /* 64K */
 		} else {
 			omap_sram_start = OMAP2_SRAM_PA;
arch/arm/mach-omap2/sram.h +0 -1
@@ -59,4 +59,3 @@ static inline void omap_push_sram_idle(void) {}
  * Used by the SRAM management code and the idle sleep code.
  */
 #define OMAP2_SRAM_PA		0x40200000
-#define OMAP3_SRAM_PA           0x40200000
Signature · 1 line
2.15.0
#35 lore raw
Quoted 8 lines · Tony Lindgren
Quoted 6 lines · Tony Lindgren
Quoted 1 line · Tony Lindgren

Then I'll follow up on cleaning up save_secure_ram_context later.

Here's a better version, the static mapping did not get used.. It just moved the area so it happened to work. It needs to be set up as MT_MEMORY_RWX_NONCACHED instead.

I see a better version now. Hmm... I guess that it also has the problem that I mentioned on first version.

Quoted 2 lines

And FYI, here's what I currently have for the follow-up patch, but that can wait a bit.

Okay. So, this patch should be applied on the top of above better version?

Thanks.

maintainer
#36 lore raw
Quoted 10 lines · Joonsoo Kim
Quoted 8 lines · Tony Lindgren
Quoted 6 lines · Tony Lindgren
Quoted 1 line · Tony Lindgren

Then I'll follow up on cleaning up save_secure_ram_context later.

Here's a better version, the static mapping did not get used.. It just moved the area so it happened to work. It needs to be set up as MT_MEMORY_RWX_NONCACHED instead.

Quoted 7 lines

I see a better version now. Hmm... I guess that it also has the problem that I mentioned on first version.

Quoted 2 lines

And FYI, here's what I currently have for the follow-up patch, but that can wait a bit.

Okay. So, this patch should be applied on the top of above better version?

Yeah, but we may actually want to combine both patches into a single patch to avoid the save_secure_ram_context 0xffff translation limit, see my comments in the first version.

Regards,

Tony

#37 lore raw
In reply to Tony Lindgren
Quoted 22 lines · Tony Lindgren
Quoted 21 lines · Joonsoo Kim
Quoted 4 lines · Tony Lindgren

Hmm OK. Does your first patch above now have the initcall issue too? It boots if I make that also subsys_initcall and then I get:

[    2.078094] vmalloc_pool_init: DMA: get vmalloc area: d0010000

Yes, first patch has the initcall issue and it's intentional in order to check the theory. I checked following log for this.

- Boot failure
SRAM_ADDR: omap_map_sram: P: 0x40208000 - 0x4020f000
SRAM_ADDR: omap_map_sram: V: 0xd0050000 - 0xd0057000

- Boot success
SRAM_ADDR: omap_map_sram: P: 0x40208000 - 0x4020f000
SRAM_ADDR: omap_map_sram: V: 0xd0008000 - 0xd000f000

When failure, virtual address for sram is higher than normal one due to vmalloc area allocation in __dma_alloc_remap(). If it is deferred, virtual address is the same with success case and then the system work.

Quoted 8 lines
Quoted 4 lines

So, my next theory is that there is n900 specific assumption that sram should have that address. Could you check if any working tree for n900 which doesn't have my CMA series work or not with adding "arm/dma: vmalloc area allocation"?

Oh I see, sorry I was not following you earlier. So you mean that by adding the vmalloc_pool_init() initcall the va mapping for SRAM changes.

Exactly.

Quoted 7 lines

And yes, save_secure_ram_context seems to be doing some sketchy virt to phys calculation with sram_phy_addr_mask. Here's a small patch to fix that for your CMA series, maybe you can merge it with your series to avoid breaking booting for git bisect.

Then I'll follow up on cleaning up save_secure_ram_context later.

Thanks for the patch. However, the patch should be modified. See below.

Quoted 47 lines

Regards,

Tony

8< -------------------------

Quoted 1 line

From tony Mon Sep 17 00:00:00 2001

From: Tony Lindgren <[email protected]>
Date: Thu, 9 Nov 2017 17:05:34 -0800
Subject: [PATCH] ARM: OMAP2+: Add static SRAM mapping for
save_secure_ram_context

With the CMA changes from Joonsoo Kim <[email protected]>, it was noticed that n900 stopped booting. After investigating it turned out that n900 save_secure_ram_context does some whacky virtual to physical address translation for the SRAM data address.

Let's fix this for CMA changes by adding a static mapping for SRAM on omap3. Then we can follow up with a patch to clean up the address translation in save_secure_ram_context later on.

---
 arch/arm/mach-omap2/io.c    | 6 ++++++
 arch/arm/mach-omap2/iomap.h | 4 ++++
 2 files changed, 10 insertions(+)
arch/arm/mach-omap2/io.c +6 -0
@@ -139,6 +139,12 @@ static struct map_desc omap243x_io_desc[] __initdata = {
 
 #ifdef	CONFIG_ARCH_OMAP3
 static struct map_desc omap34xx_io_desc[] __initdata = {
+	{
+		.virtual	= OMAP34XX_SRAM_VIRT,
+		.pfn		= __phys_to_pfn(OMAP34XX_SRAM_PHYS),
+		.length		= OMAP34XX_SRAM_SIZE,
+		.type		= MT_DEVICE
+	},
 	{
 		.virtual	= L3_34XX_VIRT,
 		.pfn		= __phys_to_pfn(L3_34XX_PHYS),

arch/arm/mach-omap2/iomap.h line 128

Quoted 6 lines
arch/arm/mach-omap2/iomap.h +3 -0
@@ -123,6 +123,10 @@
  * VPOM3430 was not working for Int controller
  */
 
+#define OMAP34XX_SRAM_PHYS	0x40200000
+#define OMAP34XX_SRAM_VIRT	0xd0010000
+#define OMAP34XX_SRAM_SIZE	0x10000

For my testing environment, vmalloc address space is started at roughly 0xe0000000 so 0xd0010000 would not be valid. And, PHYS can be different according to the system type. Maybe either OMAP3_SRAM_PUB_PA or OMAP3_SRAM_PA. It seems that SIZE and TYPE should be considered, too. My understanding is correct?

Thanks.

maintainer
#38 lore raw
Quoted 7 lines · Joonsoo Kim

arch/arm/mach-omap2/iomap.h line 128

Quoted 3 lines · Tony Lindgren
arch/arm/mach-omap2/iomap.h +3 -0
+#define OMAP34XX_SRAM_PHYS	0x40200000
+#define OMAP34XX_SRAM_VIRT	0xd0010000
+#define OMAP34XX_SRAM_SIZE	0x10000

For my testing environment, vmalloc address space is started at roughly 0xe0000000 so 0xd0010000 would not be valid.

Well we can map it anywhere we want, got any preferences?

Just that the current save_secure_ram_context uses "high_mask" of 0xffff to translate the address. To make this more flexible, we need the save_secure_ram_context changes too. So we might want to do the static mapping and save_secure_ram_context changes as a single patch.

Quoted 3 lines

And, PHYS can be different according to the system type. Maybe either OMAP3_SRAM_PUB_PA or OMAP3_SRAM_PA. It seems that SIZE and TYPE should be considered, too. My understanding is correct?

We can have a static map for the whole SRAM area, see function __arm_ioremap_pfn_caller() for the comment "Try to reuse one of the static mapping whenever possible". So the different public SRAM start addresses and sizes don't matter there.

Regards,

Tony

maintainer
#39 lore raw
Quoted 10 lines · Tony Lindgren
Quoted 7 lines · Joonsoo Kim

arch/arm/mach-omap2/iomap.h line 128

Quoted 3 lines · Tony Lindgren
arch/arm/mach-omap2/iomap.h +3 -0
+#define OMAP34XX_SRAM_PHYS	0x40200000
+#define OMAP34XX_SRAM_VIRT	0xd0010000
+#define OMAP34XX_SRAM_SIZE	0x10000

For my testing environment, vmalloc address space is started at roughly 0xe0000000 so 0xd0010000 would not be valid.

Well we can map it anywhere we want, got any preferences?

Hmm and I'm also wondering what you do to make vmalloc space to start at 0xe0000000 instead of 0xd0000000?

The reason I'm asking is because I think we can just move all of save_secure_ram_context to run from DDR instead of SRAM. But I'd rather do a minimal patch first that fixes your series and then we can test the further changes with more time.

After moving save_secure_ram_context to DDR, we can call save_secure_ram_context directly with something like:

	args_pa = __pa(omap3_secure_ram_storage);
	offset = (unsigned long)omap3_secure_ram_storage - args_pa;
	ret = save_secure_ram_context(args_pa, offset);
Quoted 6 lines

Just that the current save_secure_ram_context uses "high_mask" of 0xffff to translate the address. To make this more flexible, we need the save_secure_ram_context changes too. So we might want to do the static mapping and save_secure_ram_context changes as a single patch.

Quoted 8 lines
Quoted 3 lines

And, PHYS can be different according to the system type. Maybe either OMAP3_SRAM_PUB_PA or OMAP3_SRAM_PA. It seems that SIZE and TYPE should be considered, too. My understanding is correct?

We can have a static map for the whole SRAM area, see function __arm_ioremap_pfn_caller() for the comment "Try to reuse one of the static mapping whenever possible". So the different public SRAM start addresses and sizes don't matter there.

And then if save_secure_ram_contet runs in DDR, no static map is needed.

Regards,

Tony

#40 lore raw
Quoted 10 lines · Tony Lindgren
Quoted 9 lines · Tony Lindgren
Quoted 7 lines · Joonsoo Kim

arch/arm/mach-omap2/iomap.h line 128

Quoted 3 lines · Tony Lindgren
arch/arm/mach-omap2/iomap.h +3 -0
+#define OMAP34XX_SRAM_PHYS	0x40200000
+#define OMAP34XX_SRAM_VIRT	0xd0010000
+#define OMAP34XX_SRAM_SIZE	0x10000

For my testing environment, vmalloc address space is started at roughly 0xe0000000 so 0xd0010000 would not be valid.

Quoted 4 lines
Quoted 1 line

Well we can map it anywhere we want, got any preferences?

Hmm and I'm also wondering what you do to make vmalloc space to start at 0xe0000000 instead of 0xd0000000?

Please see the another reply.

Quoted 5 lines

The reason I'm asking is because I think we can just move all of save_secure_ram_context to run from DDR instead of SRAM. But I'd rather do a minimal patch first that fixes your series and then we can test the further changes with more time.

Okay. I agree to make a minimal patch first and then go next step.

Quoted 17 lines

After moving save_secure_ram_context to DDR, we can call save_secure_ram_context directly with something like:

	args_pa = __pa(omap3_secure_ram_storage);
	offset = (unsigned long)omap3_secure_ram_storage - args_pa;
	ret = save_secure_ram_context(args_pa, offset);
Quoted 10 lines

Just that the current save_secure_ram_context uses "high_mask" of 0xffff to translate the address. To make this more flexible, we need the save_secure_ram_context changes too. So we might want to do the static mapping and save_secure_ram_context changes as a single patch.

Quoted 3 lines

And, PHYS can be different according to the system type. Maybe either OMAP3_SRAM_PUB_PA or OMAP3_SRAM_PA. It seems that SIZE and TYPE should be considered, too. My understanding is correct?

Quoted 7 lines
Quoted 4 lines

We can have a static map for the whole SRAM area, see function __arm_ioremap_pfn_caller() for the comment "Try to reuse one of the static mapping whenever possible". So the different public SRAM start addresses and sizes don't matter there.

And then if save_secure_ram_contet runs in DDR, no static map is needed.

Okay.

Thanks.

#41 lore raw
In reply to Tony Lindgren
Quoted 10 lines · Tony Lindgren
Quoted 7 lines · Joonsoo Kim

arch/arm/mach-omap2/iomap.h line 128

Quoted 3 lines · Tony Lindgren
arch/arm/mach-omap2/iomap.h +3 -0
+#define OMAP34XX_SRAM_PHYS	0x40200000
+#define OMAP34XX_SRAM_VIRT	0xd0010000
+#define OMAP34XX_SRAM_SIZE	0x10000

For my testing environment, vmalloc address space is started at roughly 0xe0000000 so 0xd0010000 would not be valid.

Well we can map it anywhere we want, got any preferences?

My testing environment is a beagle-(xm?) for QEMU. It is configured by CONFIG_VMSPLIT_3G=y so kernel address space is started at 0xc0000000. And, it has 512 MB memory so 0xc0000000 ~ 0xdff00000 is used for direct mapping. See below.

[    0.000000] Memory: 429504K/522240K available (11264K kernel code,
1562K rwdata, 4288K rodata, 2048K init, 405K bss, 27200K reserved,
65536K cma-reserved, 0K highmem)
[    0.000000] Virtual kernel memory layout:
[    0.000000]     vector  : 0xffff0000 - 0xffff1000   (   4 kB)
[    0.000000]     fixmap  : 0xffc00000 - 0xfff00000   (3072 kB)
[    0.000000]     vmalloc : 0xe0000000 - 0xff800000   ( 504 MB)
[    0.000000]     lowmem  : 0xc0000000 - 0xdff00000   ( 511 MB)
[    0.000000]     pkmap   : 0xbfe00000 - 0xc0000000   (   2 MB)
[    0.000000]     modules : 0xbf000000 - 0xbfe00000   (  14 MB)
[    0.000000]       .text : 0xc0208000 - 0xc0e00000   (12256 kB)
[    0.000000]       .init : 0xc1300000 - 0xc1500000   (2048 kB)
[    0.000000]       .data : 0xc1500000 - 0xc1686810   (1563 kB)
[    0.000000]        .bss : 0xc168fc68 - 0xc16f512c   ( 406 kB)

Therefore, if OMAP34XX_SRAM_VIRT is 0xd0010000, direct mapping is broken and the system doesn't work. I guess that we should use more stable address like as 0xf0000000.

Quoted 7 lines

Just that the current save_secure_ram_context uses "high_mask" of 0xffff to translate the address. To make this more flexible, we need the save_secure_ram_context changes too. So we might want to do the static mapping and save_secure_ram_context changes as a single patch.

Quoted 8 lines
Quoted 3 lines

And, PHYS can be different according to the system type. Maybe either OMAP3_SRAM_PUB_PA or OMAP3_SRAM_PA. It seems that SIZE and TYPE should be considered, too. My understanding is correct?

We can have a static map for the whole SRAM area, see function __arm_ioremap_pfn_caller() for the comment "Try to reuse one of the static mapping whenever possible". So the different public SRAM start addresses and sizes don't matter there.

Okay. Look fine with SRAM start addresses and sizes. However, we need to consider mtype since __arm_ioremap_pfn_caller() doesn't reuse the mapping if mtype is different. mtype can be either MT_MEMORY_RWX or MT_MEMORY_RWX_NONCACHED.

Thanks.

maintainer
#42 lore raw
Quoted 32 lines · Joonsoo Kim
Quoted 10 lines · Tony Lindgren
Quoted 7 lines · Joonsoo Kim

Review

Quoted 3 lines · Tony Lindgren
(quoted diff) +3 -0
+#define OMAP34XX_SRAM_PHYS	0x40200000
+#define OMAP34XX_SRAM_VIRT	0xd0010000
+#define OMAP34XX_SRAM_SIZE	0x10000

For my testing environment, vmalloc address space is started at roughly 0xe0000000 so 0xd0010000 would not be valid.

Well we can map it anywhere we want, got any preferences?

My testing environment is a beagle-(xm?) for QEMU. It is configured by CONFIG_VMSPLIT_3G=y so kernel address space is started at 0xc0000000. And, it has 512 MB memory so 0xc0000000 ~ 0xdff00000 is used for direct mapping. See below.

[    0.000000] Memory: 429504K/522240K available (11264K kernel code,
1562K rwdata, 4288K rodata, 2048K init, 405K bss, 27200K reserved,
65536K cma-reserved, 0K highmem)
[    0.000000] Virtual kernel memory layout:
[    0.000000]     vector  : 0xffff0000 - 0xffff1000   (   4 kB)
[    0.000000]     fixmap  : 0xffc00000 - 0xfff00000   (3072 kB)
[    0.000000]     vmalloc : 0xe0000000 - 0xff800000   ( 504 MB)
[    0.000000]     lowmem  : 0xc0000000 - 0xdff00000   ( 511 MB)
[    0.000000]     pkmap   : 0xbfe00000 - 0xc0000000   (   2 MB)
[    0.000000]     modules : 0xbf000000 - 0xbfe00000   (  14 MB)
[    0.000000]       .text : 0xc0208000 - 0xc0e00000   (12256 kB)
[    0.000000]       .init : 0xc1300000 - 0xc1500000   (2048 kB)
[    0.000000]       .data : 0xc1500000 - 0xc1686810   (1563 kB)
[    0.000000]        .bss : 0xc168fc68 - 0xc16f512c   ( 406 kB)
Quoted 3 lines

Therefore, if OMAP34XX_SRAM_VIRT is 0xd0010000, direct mapping is broken and the system doesn't work. I guess that we should use more stable address like as 0xf0000000.

OK. Let's forget about adding static mappings and my earlier patches to attempt to fix this. Below is what I now think we should merge as a fix before merging Joonsoo's patches. Please all review and test, adding Tero to Cc also.

Regards,

Tony

8< -----------------------

Quoted 1 line

From tony Mon Sep 17 00:00:00 2001

From: Tony Lindgren <[email protected]>
Date: Mon, 13 Nov 2017 13:23:33 -0800
Subject: [PATCH] ARM: OMAP2+: Fix SRAM virt to phys translation for
save_secure_ram_context

With the CMA changes from Joonsoo Kim <[email protected]>, it was noticed that n900 stopped booting. After investigating it turned out that n900 save_secure_ram_context does some whacky virtual to physical address translation for the SRAM data address.

As we now only have minimal parts of omap3 idle code copied to SRAM, running save_secure_ram_context() in SRAM is not needed. It only gets called on PM init. And it seems there's no need to ever call this from SRAM idle code.

So let's just keep save_secure_ram_context() in DDR, and pass it the physical address of the parameters. We can do everything else in omap-secure.c like we already do for other secure code.

And since we don't have any documentation, I still have no clue what the values for 0, 1 and 1 for the parameters might be. If somebody has figured it out, please do send a patch to add some comments.

---
 arch/arm/mach-omap2/omap-secure.c | 19 +++++++++++++++++++
 arch/arm/mach-omap2/omap-secure.h |  4 ++++
 arch/arm/mach-omap2/pm.h          |  4 ----
 arch/arm/mach-omap2/pm34xx.c      | 13 ++++---------
 arch/arm/mach-omap2/sleep34xx.S   | 26 ++++----------------------
 5 files changed, 31 insertions(+), 35 deletions(-)
arch/arm/mach-omap2/omap-secure.c +19 -0
@@ -73,6 +73,25 @@ phys_addr_t omap_secure_ram_mempool_base(void)
 	return omap_secure_memblock_base;
 }
 
+u32 omap3_save_secure_ram(void __iomem *addr, int size)
+{
+	u32 ret;
+	u32 param[5];
+
+	if (size != OMAP3_SAVE_SECURE_RAM_SZ)
+		return OMAP3_SAVE_SECURE_RAM_SZ;
+
+	param[0] = 4;		/* Number of arguments */
+	param[1] = __pa(addr);	/* Physical address for saving */
+	param[2] = 0;
+	param[3] = 1;
+	param[4] = 1;
+
+	ret = save_secure_ram_context(__pa(param));
+
+	return ret;
+}
+
 /**
  * rx51_secure_dispatcher: Routine to dispatch secure PPA API calls
  * @idx: The PPA API index
arch/arm/mach-omap2/omap-secure.h +4 -0
@@ -31,6 +31,8 @@
 /* Maximum Secure memory storage size */
 #define OMAP_SECURE_RAM_STORAGE	(88 * SZ_1K)
 
+#define OMAP3_SAVE_SECURE_RAM_SZ	0x803F
+
 /* Secure low power HAL API index */
 #define OMAP4_HAL_SAVESECURERAM_INDEX	0x1a
 #define OMAP4_HAL_SAVEHW_INDEX		0x1b
@@ -65,6 +67,8 @@ extern u32 omap_smc2(u32 id, u32 falg, u32 pargs);
 extern u32 omap_smc3(u32 id, u32 process, u32 flag, u32 pargs);
 extern phys_addr_t omap_secure_ram_mempool_base(void);
 extern int omap_secure_ram_reserve_memblock(void);
+extern u32 save_secure_ram_context(u32 args_pa);
+extern u32 omap3_save_secure_ram(void __iomem *save_regs, int size);
 
 extern u32 rx51_secure_dispatcher(u32 idx, u32 process, u32 flag, u32 nargs,
 				  u32 arg1, u32 arg2, u32 arg3, u32 arg4);
arch/arm/mach-omap2/pm.h +0 -4
@@ -81,10 +81,6 @@ extern unsigned int omap3_do_wfi_sz;
 /* ... and its pointer from SRAM after copy */
 extern void (*omap3_do_wfi_sram)(void);
 
-/* save_secure_ram_context function pointer and size, for copy to SRAM */
-extern int save_secure_ram_context(u32 *addr);
-extern unsigned int save_secure_ram_context_sz;
-
 extern void omap3_save_scratchpad_contents(void);
 
 #define PM_RTA_ERRATUM_i608		(1 << 0)
arch/arm/mach-omap2/pm34xx.c +4 -9
@@ -48,6 +48,7 @@
 #include "prm3xxx.h"
 #include "pm.h"
 #include "sdrc.h"
+#include "omap-secure.h"
 #include "sram.h"
 #include "control.h"
 #include "vc.h"
@@ -66,7 +67,6 @@ struct power_state {
 
 static LIST_HEAD(pwrst_list);
 
-static int (*_omap_save_secure_sram)(u32 *addr);
 void (*omap3_do_wfi_sram)(void);
 
 static struct powerdomain *mpu_pwrdm, *neon_pwrdm;
@@ -121,8 +121,8 @@ static void omap3_save_secure_ram_context(void)
 		 * will hang the system.
 		 */
 		pwrdm_set_next_pwrst(mpu_pwrdm, PWRDM_POWER_ON);
-		ret = _omap_save_secure_sram((u32 *)(unsigned long)
-				__pa(omap3_secure_ram_storage));
+		ret = omap3_save_secure_ram(omap3_secure_ram_storage,
+					    OMAP3_SAVE_SECURE_RAM_SZ);
 		pwrdm_set_next_pwrst(mpu_pwrdm, mpu_next_state);
 		/* Following is for error tracking, it should not happen */
 		if (ret) {
@@ -434,15 +434,10 @@ static int __init pwrdms_setup(struct powerdomain *pwrdm, void *unused)
  *
  * The minimum set of functions is pushed to SRAM for execution:
  * - omap3_do_wfi for erratum i581 WA,
- * - save_secure_ram_context for security extensions.
  */
 void omap_push_sram_idle(void)
 {
 	omap3_do_wfi_sram = omap_sram_push(omap3_do_wfi, omap3_do_wfi_sz);
-
-	if (omap_type() != OMAP2_DEVICE_TYPE_GP)
-		_omap_save_secure_sram = omap_sram_push(save_secure_ram_context,
-				save_secure_ram_context_sz);
 }
 
 static void __init pm_errata_configure(void)
@@ -553,7 +548,7 @@ int __init omap3_pm_init(void)
 	clkdm_add_wkdep(neon_clkdm, mpu_clkdm);
 	if (omap_type() != OMAP2_DEVICE_TYPE_GP) {
 		omap3_secure_ram_storage =
-			kmalloc(0x803F, GFP_KERNEL);
+			kmalloc(OMAP3_SAVE_SECURE_RAM_SZ, GFP_KERNEL);
 		if (!omap3_secure_ram_storage)
 			pr_err("Memory allocation failed when allocating for secure sram context\n");
 
arch/arm/mach-omap2/sleep34xx.S +4 -22
@@ -93,20 +93,13 @@ ENTRY(enable_omap3630_toggle_l2_on_restore)
 ENDPROC(enable_omap3630_toggle_l2_on_restore)
 
 /*
- * Function to call rom code to save secure ram context. This gets
- * relocated to SRAM, so it can be all in .data section. Otherwise
- * we need to initialize api_params separately.
+ * Function to call rom code to save secure ram context.
+ *
+ * r0 = physical address of the parameters
  */
-	.data
-	.align	3
 ENTRY(save_secure_ram_context)
 	stmfd	sp!, {r4 - r11, lr}	@ save registers on stack
-	adr	r3, api_params		@ r3 points to parameters
-	str	r0, [r3,#0x4]		@ r0 has sdram address
-	ldr	r12, high_mask
-	and	r3, r3, r12
-	ldr	r12, sram_phy_addr_mask
-	orr	r3, r3, r12
+	mov	r3, r0			@ physical address of parameters
 	mov	r0, #25			@ set service ID for PPA
 	mov	r12, r0			@ copy secure service ID in r12
 	mov	r1, #0			@ set task id for ROM code in r1
@@ -120,18 +113,7 @@ ENTRY(save_secure_ram_context)
 	nop
 	nop
 	ldmfd	sp!, {r4 - r11, pc}
-	.align
-sram_phy_addr_mask:
-	.word	SRAM_BASE_P
-high_mask:
-	.word	0xffff
-api_params:
-	.word	0x4, 0x0, 0x0, 0x1, 0x1
 ENDPROC(save_secure_ram_context)
-ENTRY(save_secure_ram_context_sz)
-	.word	. - save_secure_ram_context
-
-	.text
 
 /*
  * ======================
Signature · 1 line
2.15.0
#43 lore raw
Quoted 73 lines · Tony Lindgren
Quoted 32 lines · Joonsoo Kim
Quoted 10 lines · Tony Lindgren
Quoted 7 lines · Joonsoo Kim

Review

Quoted 3 lines · Tony Lindgren
(quoted diff) +3 -0
+#define OMAP34XX_SRAM_PHYS	0x40200000
+#define OMAP34XX_SRAM_VIRT	0xd0010000
+#define OMAP34XX_SRAM_SIZE	0x10000

For my testing environment, vmalloc address space is started at roughly 0xe0000000 so 0xd0010000 would not be valid.

Well we can map it anywhere we want, got any preferences?

My testing environment is a beagle-(xm?) for QEMU. It is configured by CONFIG_VMSPLIT_3G=y so kernel address space is started at 0xc0000000. And, it has 512 MB memory so 0xc0000000 ~ 0xdff00000 is used for direct mapping. See below.

[    0.000000] Memory: 429504K/522240K available (11264K kernel code,
1562K rwdata, 4288K rodata, 2048K init, 405K bss, 27200K reserved,
65536K cma-reserved, 0K highmem)
[    0.000000] Virtual kernel memory layout:
[    0.000000]     vector  : 0xffff0000 - 0xffff1000   (   4 kB)
[    0.000000]     fixmap  : 0xffc00000 - 0xfff00000   (3072 kB)
[    0.000000]     vmalloc : 0xe0000000 - 0xff800000   ( 504 MB)
[    0.000000]     lowmem  : 0xc0000000 - 0xdff00000   ( 511 MB)
[    0.000000]     pkmap   : 0xbfe00000 - 0xc0000000   (   2 MB)
[    0.000000]     modules : 0xbf000000 - 0xbfe00000   (  14 MB)
[    0.000000]       .text : 0xc0208000 - 0xc0e00000   (12256 kB)
[    0.000000]       .init : 0xc1300000 - 0xc1500000   (2048 kB)
[    0.000000]       .data : 0xc1500000 - 0xc1686810   (1563 kB)
[    0.000000]        .bss : 0xc168fc68 - 0xc16f512c   ( 406 kB)
Quoted 3 lines

Therefore, if OMAP34XX_SRAM_VIRT is 0xd0010000, direct mapping is broken and the system doesn't work. I guess that we should use more stable address like as 0xf0000000.

OK. Let's forget about adding static mappings and my earlier patches to attempt to fix this. Below is what I now think we should merge as a fix before merging Joonsoo's patches. Please all review and test, adding Tero to Cc also.

Regards,

Tony

8< -----------------------
From tony Mon Sep 17 00:00:00 2001
From: Tony Lindgren <[email protected]>
Date: Mon, 13 Nov 2017 13:23:33 -0800
Subject: [PATCH] ARM: OMAP2+: Fix SRAM virt to phys translation for
save_secure_ram_context

With the CMA changes from Joonsoo Kim <[email protected]>, it was noticed that n900 stopped booting. After investigating it turned out that n900 save_secure_ram_context does some whacky virtual to physical address translation for the SRAM data address.

As we now only have minimal parts of omap3 idle code copied to SRAM, running save_secure_ram_context() in SRAM is not needed. It only gets called on PM init. And it seems there's no need to ever call this from SRAM idle code.

So let's just keep save_secure_ram_context() in DDR, and pass it the physical address of the parameters. We can do everything else in omap-secure.c like we already do for other secure code.

And since we don't have any documentation, I still have no clue what the values for 0, 1 and 1 for the parameters might be. If somebody has figured it out, please do send a patch to add some comments.

Quoted 7 lines
---
  arch/arm/mach-omap2/omap-secure.c | 19 +++++++++++++++++++
  arch/arm/mach-omap2/omap-secure.h |  4 ++++
  arch/arm/mach-omap2/pm.h          |  4 ----
  arch/arm/mach-omap2/pm34xx.c      | 13 ++++---------
  arch/arm/mach-omap2/sleep34xx.S   | 26 ++++----------------------
  5 files changed, 31 insertions(+), 35 deletions(-)

I guess you could just use rx51_secure_dispatcher and ditch the save_secure_ram_context call completely (and most of the other related code)? That one would handle the cache also in a clean manner.

Something like:

rx51_secure_dispatcher(25, 0, FLAG_START_CRITICAL, 4, __pa(omap3_secure_ram_storage), 0, 1, 1);

Can't test it myself as I don't have omap3 secure HW.

-Tero

Quoted 158 lines
arch/arm/mach-omap2/omap-secure.c +19 -0
@@ -73,6 +73,25 @@ phys_addr_t omap_secure_ram_mempool_base(void)
  	return omap_secure_memblock_base;
  }
 
+u32 omap3_save_secure_ram(void __iomem *addr, int size)
+{
+	u32 ret;
+	u32 param[5];
+
+	if (size != OMAP3_SAVE_SECURE_RAM_SZ)
+		return OMAP3_SAVE_SECURE_RAM_SZ;
+
+	param[0] = 4;		/* Number of arguments */
+	param[1] = __pa(addr);	/* Physical address for saving */
+	param[2] = 0;
+	param[3] = 1;
+	param[4] = 1;
+
+	ret = save_secure_ram_context(__pa(param));
+
+	return ret;
+}
+
  /**
   * rx51_secure_dispatcher: Routine to dispatch secure PPA API calls
   * @idx: The PPA API index
arch/arm/mach-omap2/omap-secure.h +4 -0
@@ -31,6 +31,8 @@
  /* Maximum Secure memory storage size */
  #define OMAP_SECURE_RAM_STORAGE	(88 * SZ_1K)
 
+#define OMAP3_SAVE_SECURE_RAM_SZ	0x803F
+
  /* Secure low power HAL API index */
  #define OMAP4_HAL_SAVESECURERAM_INDEX	0x1a
  #define OMAP4_HAL_SAVEHW_INDEX		0x1b
@@ -65,6 +67,8 @@ extern u32 omap_smc2(u32 id, u32 falg, u32 pargs);
  extern u32 omap_smc3(u32 id, u32 process, u32 flag, u32 pargs);
  extern phys_addr_t omap_secure_ram_mempool_base(void);
  extern int omap_secure_ram_reserve_memblock(void);
+extern u32 save_secure_ram_context(u32 args_pa);
+extern u32 omap3_save_secure_ram(void __iomem *save_regs, int size);
 
  extern u32 rx51_secure_dispatcher(u32 idx, u32 process, u32 flag, u32 nargs,
  				  u32 arg1, u32 arg2, u32 arg3, u32 arg4);
arch/arm/mach-omap2/pm.h +0 -4
@@ -81,10 +81,6 @@ extern unsigned int omap3_do_wfi_sz;
  /* ... and its pointer from SRAM after copy */
  extern void (*omap3_do_wfi_sram)(void);
 
-/* save_secure_ram_context function pointer and size, for copy to SRAM */
-extern int save_secure_ram_context(u32 *addr);
-extern unsigned int save_secure_ram_context_sz;
-
  extern void omap3_save_scratchpad_contents(void);
 
  #define PM_RTA_ERRATUM_i608		(1 << 0)
arch/arm/mach-omap2/pm34xx.c +4 -9
@@ -48,6 +48,7 @@
  #include "prm3xxx.h"
  #include "pm.h"
  #include "sdrc.h"
+#include "omap-secure.h"
  #include "sram.h"
  #include "control.h"
  #include "vc.h"
@@ -66,7 +67,6 @@ struct power_state {
 
  static LIST_HEAD(pwrst_list);
 
-static int (*_omap_save_secure_sram)(u32 *addr);
  void (*omap3_do_wfi_sram)(void);
 
  static struct powerdomain *mpu_pwrdm, *neon_pwrdm;
@@ -121,8 +121,8 @@ static void omap3_save_secure_ram_context(void)
  		 * will hang the system.
  		 */
  		pwrdm_set_next_pwrst(mpu_pwrdm, PWRDM_POWER_ON);
-		ret = _omap_save_secure_sram((u32 *)(unsigned long)
-				__pa(omap3_secure_ram_storage));
+		ret = omap3_save_secure_ram(omap3_secure_ram_storage,
+					    OMAP3_SAVE_SECURE_RAM_SZ);
  		pwrdm_set_next_pwrst(mpu_pwrdm, mpu_next_state);
  		/* Following is for error tracking, it should not happen */
  		if (ret) {
@@ -434,15 +434,10 @@ static int __init pwrdms_setup(struct powerdomain *pwrdm, void *unused)
   *
   * The minimum set of functions is pushed to SRAM for execution:
   * - omap3_do_wfi for erratum i581 WA,
- * - save_secure_ram_context for security extensions.
   */
  void omap_push_sram_idle(void)
  {
  	omap3_do_wfi_sram = omap_sram_push(omap3_do_wfi, omap3_do_wfi_sz);
-
-	if (omap_type() != OMAP2_DEVICE_TYPE_GP)
-		_omap_save_secure_sram = omap_sram_push(save_secure_ram_context,
-				save_secure_ram_context_sz);
  }
 
  static void __init pm_errata_configure(void)
@@ -553,7 +548,7 @@ int __init omap3_pm_init(void)
  	clkdm_add_wkdep(neon_clkdm, mpu_clkdm);
  	if (omap_type() != OMAP2_DEVICE_TYPE_GP) {
  		omap3_secure_ram_storage =
-			kmalloc(0x803F, GFP_KERNEL);
+			kmalloc(OMAP3_SAVE_SECURE_RAM_SZ, GFP_KERNEL);
  		if (!omap3_secure_ram_storage)
  			pr_err("Memory allocation failed when allocating for secure sram context\n");
 
arch/arm/mach-omap2/sleep34xx.S +4 -16
@@ -93,20 +93,13 @@ ENTRY(enable_omap3630_toggle_l2_on_restore)
  ENDPROC(enable_omap3630_toggle_l2_on_restore)
 
  /*
- * Function to call rom code to save secure ram context. This gets
- * relocated to SRAM, so it can be all in .data section. Otherwise
- * we need to initialize api_params separately.
+ * Function to call rom code to save secure ram context.
+ *
+ * r0 = physical address of the parameters
   */
-	.data
-	.align	3
  ENTRY(save_secure_ram_context)
  	stmfd	sp!, {r4 - r11, lr}	@ save registers on stack
-	adr	r3, api_params		@ r3 points to parameters
-	str	r0, [r3,#0x4]		@ r0 has sdram address
-	ldr	r12, high_mask
-	and	r3, r3, r12
-	ldr	r12, sram_phy_addr_mask
-	orr	r3, r3, r12
+	mov	r3, r0			@ physical address of parameters
  	mov	r0, #25			@ set service ID for PPA
  	mov	r12, r0			@ copy secure service ID in r12
  	mov	r1, #0			@ set task id for ROM code in r1
@@ -120,18 +113,7 @@ ENTRY(save_secure_ram_context)
  	nop
  	nop
  	ldmfd	sp!, {r4 - r11, pc}
-	.align
-sram_phy_addr_mask:
-	.word	SRAM_BASE_P
-high_mask:
-	.word	0xffff

arch/arm/mach-omap2/sleep34xx.S ENTRY(save_secure_ram_context) · line 119

Quoted 10 lines
arch/arm/mach-omap2/sleep34xx.S +0 -6
-api_params:
-	.word	0x4, 0x0, 0x0, 0x1, 0x1
  ENDPROC(save_secure_ram_context)
-ENTRY(save_secure_ram_context_sz)
-	.word	. - save_secure_ram_context
-
-	.text
 
  /*
   * ======================

--
Texas Instruments Finland Oy, Porkkalankatu 22, 00180 Helsinki. Y-tunnus/Business ID: 0615521-4. Kotipaikka/Domicile: Helsinki

maintainer
#44 lore raw
Quoted 8 lines · Tero Kristo

I guess you could just use rx51_secure_dispatcher and ditch the save_secure_ram_context call completely (and most of the other related code)? That one would handle the cache also in a clean manner.

Something like:

rx51_secure_dispatcher(25, 0, FLAG_START_CRITICAL, 4, __pa(omap3_secure_ram_storage), 0, 1, 1);

That's different, as rx51_secure_dispatcher does the following:

- Use arguments + 1 instead of 4, we currently use just 4
- Disables local_irq and fiq, we are not doing that now
- Flushes and invalidates cache range, we are not doing that
- Calls omap_smc3 that only does mov r6, #0xff, and does not do mov r2, #4
- Missing nops after it's done

This just based on a quick look I did earlier. So just because of the extra work it does we don't want to do it even if it worked :)

Regards,

Tony

#45 lore raw
Quoted 19 lines · Tony Lindgren
Quoted 8 lines · Tero Kristo

I guess you could just use rx51_secure_dispatcher and ditch the save_secure_ram_context call completely (and most of the other related code)? That one would handle the cache also in a clean manner.

Something like:

rx51_secure_dispatcher(25, 0, FLAG_START_CRITICAL, 4, __pa(omap3_secure_ram_storage), 0, 1, 1);

That's different, as rx51_secure_dispatcher does the following:

- Use arguments + 1 instead of 4, we currently use just 4
- Disables local_irq and fiq, we are not doing that now
- Flushes and invalidates cache range, we are not doing that
- Calls omap_smc3 that only does mov r6, #0xff, and does not do mov r2, #4
- Missing nops after it's done

Quoted 3 lines

This just based on a quick look I did earlier. So just because of the extra work it does we don't want to do it even if it worked :)

Hmm ok, I was just thinking that all the extra flushes, irq disables etc. might be good to have in place, as a safeguard when entering secure mode. You might get glitches in certain conditions otherwise.

The things it is missing might just be clutter.

Anyway, that said, the changes you did look sane, but I might have cleaned it up a bit further. :)

-Tero

--
Texas Instruments Finland Oy, Porkkalankatu 22, 00180 Helsinki. Y-tunnus/Business ID: 0615521-4. Kotipaikka/Domicile: Helsinki

maintainer
#46 lore raw
Quoted 20 lines · Tero Kristo
Quoted 19 lines · Tony Lindgren
Quoted 8 lines · Tero Kristo

I guess you could just use rx51_secure_dispatcher and ditch the save_secure_ram_context call completely (and most of the other related code)? That one would handle the cache also in a clean manner.

Something like:

rx51_secure_dispatcher(25, 0, FLAG_START_CRITICAL, 4, __pa(omap3_secure_ram_storage), 0, 1, 1);

That's different, as rx51_secure_dispatcher does the following:

- Use arguments + 1 instead of 4, we currently use just 4
- Disables local_irq and fiq, we are not doing that now
- Flushes and invalidates cache range, we are not doing that
- Calls omap_smc3 that only does mov r6, #0xff, and does not do mov r2, #4
- Missing nops after it's done

Quoted 7 lines
Quoted 3 lines

This just based on a quick look I did earlier. So just because of the extra work it does we don't want to do it even if it worked :)

Hmm ok, I was just thinking that all the extra flushes, irq disables etc. might be good to have in place, as a safeguard when entering secure mode. You might get glitches in certain conditions otherwise.

Well it's been close to 10 years already without those flushes. And we only call this once on init.. And further changes should be a lot easier now.

Quoted 4 lines

The things it is missing might just be clutter.

Anyway, that said, the changes you did look sane, but I might have cleaned it up a bit further. :)

Yeah OK, let's consider that as a separate patch. This attempts to not change the functionality, just move it out of SRAM.

Regards,

Tony

#47 lore raw
In reply to Tony Lindgren
Quoted 33 lines · Tony Lindgren
Quoted 32 lines · Joonsoo Kim
Quoted 10 lines · Tony Lindgren
Quoted 7 lines · Joonsoo Kim

Review

Quoted 3 lines · Tony Lindgren
(quoted diff) +3 -0
+#define OMAP34XX_SRAM_PHYS	0x40200000
+#define OMAP34XX_SRAM_VIRT	0xd0010000
+#define OMAP34XX_SRAM_SIZE	0x10000

For my testing environment, vmalloc address space is started at roughly 0xe0000000 so 0xd0010000 would not be valid.

Well we can map it anywhere we want, got any preferences?

My testing environment is a beagle-(xm?) for QEMU. It is configured by CONFIG_VMSPLIT_3G=y so kernel address space is started at 0xc0000000. And, it has 512 MB memory so 0xc0000000 ~ 0xdff00000 is used for direct mapping. See below.

[    0.000000] Memory: 429504K/522240K available (11264K kernel code,
1562K rwdata, 4288K rodata, 2048K init, 405K bss, 27200K reserved,
65536K cma-reserved, 0K highmem)
[    0.000000] Virtual kernel memory layout:
[    0.000000]     vector  : 0xffff0000 - 0xffff1000   (   4 kB)
[    0.000000]     fixmap  : 0xffc00000 - 0xfff00000   (3072 kB)
[    0.000000]     vmalloc : 0xe0000000 - 0xff800000   ( 504 MB)
[    0.000000]     lowmem  : 0xc0000000 - 0xdff00000   ( 511 MB)
[    0.000000]     pkmap   : 0xbfe00000 - 0xc0000000   (   2 MB)
[    0.000000]     modules : 0xbf000000 - 0xbfe00000   (  14 MB)
[    0.000000]       .text : 0xc0208000 - 0xc0e00000   (12256 kB)
[    0.000000]       .init : 0xc1300000 - 0xc1500000   (2048 kB)
[    0.000000]       .data : 0xc1500000 - 0xc1686810   (1563 kB)
[    0.000000]        .bss : 0xc168fc68 - 0xc16f512c   ( 406 kB)
Quoted 8 lines
Quoted 3 lines

Therefore, if OMAP34XX_SRAM_VIRT is 0xd0010000, direct mapping is broken and the system doesn't work. I guess that we should use more stable address like as 0xf0000000.

OK. Let's forget about adding static mappings and my earlier patches to attempt to fix this. Below is what I now think we should merge as a fix before merging Joonsoo's patches. Please all review and test, adding Tero to Cc also.

Okay. Then, this patch will be merged by yourself as a fix? I'm okay with either way. Just let me know. :)

Thanks.

maintainer
#48 lore raw
Quoted 38 lines · Joonsoo Kim
Quoted 37 lines · Tony Lindgren
Quoted 35 lines · Joonsoo Kim
Quoted 10 lines · Tony Lindgren
Quoted 7 lines · Joonsoo Kim

Review

Quoted 3 lines · Tony Lindgren
(quoted diff) +3 -0
+#define OMAP34XX_SRAM_PHYS	0x40200000
+#define OMAP34XX_SRAM_VIRT	0xd0010000
+#define OMAP34XX_SRAM_SIZE	0x10000

For my testing environment, vmalloc address space is started at roughly 0xe0000000 so 0xd0010000 would not be valid.

Well we can map it anywhere we want, got any preferences?

My testing environment is a beagle-(xm?) for QEMU. It is configured by CONFIG_VMSPLIT_3G=y so kernel address space is started at 0xc0000000. And, it has 512 MB memory so 0xc0000000 ~ 0xdff00000 is used for direct mapping. See below.

[    0.000000] Memory: 429504K/522240K available (11264K kernel code,
1562K rwdata, 4288K rodata, 2048K init, 405K bss, 27200K reserved,
65536K cma-reserved, 0K highmem)
[    0.000000] Virtual kernel memory layout:
[    0.000000]     vector  : 0xffff0000 - 0xffff1000   (   4 kB)
[    0.000000]     fixmap  : 0xffc00000 - 0xfff00000   (3072 kB)
[    0.000000]     vmalloc : 0xe0000000 - 0xff800000   ( 504 MB)
[    0.000000]     lowmem  : 0xc0000000 - 0xdff00000   ( 511 MB)
[    0.000000]     pkmap   : 0xbfe00000 - 0xc0000000   (   2 MB)
[    0.000000]     modules : 0xbf000000 - 0xbfe00000   (  14 MB)
[    0.000000]       .text : 0xc0208000 - 0xc0e00000   (12256 kB)
[    0.000000]       .init : 0xc1300000 - 0xc1500000   (2048 kB)
[    0.000000]       .data : 0xc1500000 - 0xc1686810   (1563 kB)
[    0.000000]        .bss : 0xc168fc68 - 0xc16f512c   ( 406 kB)

Therefore, if OMAP34XX_SRAM_VIRT is 0xd0010000, direct mapping is broken and the system doesn't work. I guess that we should use more stable address like as 0xf0000000.

Quoted 7 lines
Quoted 4 lines

OK. Let's forget about adding static mappings and my earlier patches to attempt to fix this. Below is what I now think we should merge as a fix before merging Joonsoo's patches. Please all review and test, adding Tero to Cc also.

Okay. Then, this patch will be merged by yourself as a fix? I'm okay with either way. Just let me know. :)

Well what's the plan with your patches? I'd not have mainline kernel broken so if this was the only blocker for the v4.15, then you should pick it.

However, if your patches are now planned for v4.16, then I'll queue it for early v4.15-rc.

Regards,

Tony

#49 lore raw
Quoted 47 lines · Tony Lindgren
Quoted 38 lines · Joonsoo Kim
Quoted 37 lines · Tony Lindgren
Quoted 35 lines · Joonsoo Kim
Quoted 10 lines · Tony Lindgren
Quoted 7 lines · Joonsoo Kim

Review

Quoted 3 lines · Tony Lindgren
(quoted diff) +3 -0
+#define OMAP34XX_SRAM_PHYS	0x40200000
+#define OMAP34XX_SRAM_VIRT	0xd0010000
+#define OMAP34XX_SRAM_SIZE	0x10000

For my testing environment, vmalloc address space is started at roughly 0xe0000000 so 0xd0010000 would not be valid.

Well we can map it anywhere we want, got any preferences?

My testing environment is a beagle-(xm?) for QEMU. It is configured by CONFIG_VMSPLIT_3G=y so kernel address space is started at 0xc0000000. And, it has 512 MB memory so 0xc0000000 ~ 0xdff00000 is used for direct mapping. See below.

[    0.000000] Memory: 429504K/522240K available (11264K kernel code,
1562K rwdata, 4288K rodata, 2048K init, 405K bss, 27200K reserved,
65536K cma-reserved, 0K highmem)
[    0.000000] Virtual kernel memory layout:
[    0.000000]     vector  : 0xffff0000 - 0xffff1000   (   4 kB)
[    0.000000]     fixmap  : 0xffc00000 - 0xfff00000   (3072 kB)
[    0.000000]     vmalloc : 0xe0000000 - 0xff800000   ( 504 MB)
[    0.000000]     lowmem  : 0xc0000000 - 0xdff00000   ( 511 MB)
[    0.000000]     pkmap   : 0xbfe00000 - 0xc0000000   (   2 MB)
[    0.000000]     modules : 0xbf000000 - 0xbfe00000   (  14 MB)
[    0.000000]       .text : 0xc0208000 - 0xc0e00000   (12256 kB)
[    0.000000]       .init : 0xc1300000 - 0xc1500000   (2048 kB)
[    0.000000]       .data : 0xc1500000 - 0xc1686810   (1563 kB)
[    0.000000]        .bss : 0xc168fc68 - 0xc16f512c   ( 406 kB)

Therefore, if OMAP34XX_SRAM_VIRT is 0xd0010000, direct mapping is broken and the system doesn't work. I guess that we should use more stable address like as 0xf0000000.

Quoted 7 lines
Quoted 4 lines

OK. Let's forget about adding static mappings and my earlier patches to attempt to fix this. Below is what I now think we should merge as a fix before merging Joonsoo's patches. Please all review and test, adding Tero to Cc also.

Okay. Then, this patch will be merged by yourself as a fix? I'm okay with either way. Just let me know. :)

Quoted 6 lines

Well what's the plan with your patches? I'd not have mainline kernel broken so if this was the only blocker for the v4.15, then you should pick it.

However, if your patches are now planned for v4.16, then I'll queue it for early v4.15-rc.

It was the only blocker but I think that it's too late for v4.15. I will try again for v4.16. Please queue your fix for early v4.15-rc.

Thanks.

maintainer
#50 lore raw
Quoted 52 lines · Joonsoo Kim
Quoted 51 lines · Tony Lindgren
Quoted 38 lines · Joonsoo Kim
Quoted 37 lines · Tony Lindgren
Quoted 35 lines · Joonsoo Kim
Quoted 10 lines · Tony Lindgren
Quoted 7 lines · Joonsoo Kim

Review

Quoted 3 lines · Tony Lindgren
(quoted diff) +3 -0
+#define OMAP34XX_SRAM_PHYS	0x40200000
+#define OMAP34XX_SRAM_VIRT	0xd0010000
+#define OMAP34XX_SRAM_SIZE	0x10000

For my testing environment, vmalloc address space is started at roughly 0xe0000000 so 0xd0010000 would not be valid.

Well we can map it anywhere we want, got any preferences?

My testing environment is a beagle-(xm?) for QEMU. It is configured by CONFIG_VMSPLIT_3G=y so kernel address space is started at 0xc0000000. And, it has 512 MB memory so 0xc0000000 ~ 0xdff00000 is used for direct mapping. See below.

[    0.000000] Memory: 429504K/522240K available (11264K kernel code,
1562K rwdata, 4288K rodata, 2048K init, 405K bss, 27200K reserved,
65536K cma-reserved, 0K highmem)
[    0.000000] Virtual kernel memory layout:
[    0.000000]     vector  : 0xffff0000 - 0xffff1000   (   4 kB)
[    0.000000]     fixmap  : 0xffc00000 - 0xfff00000   (3072 kB)
[    0.000000]     vmalloc : 0xe0000000 - 0xff800000   ( 504 MB)
[    0.000000]     lowmem  : 0xc0000000 - 0xdff00000   ( 511 MB)
[    0.000000]     pkmap   : 0xbfe00000 - 0xc0000000   (   2 MB)
[    0.000000]     modules : 0xbf000000 - 0xbfe00000   (  14 MB)
[    0.000000]       .text : 0xc0208000 - 0xc0e00000   (12256 kB)
[    0.000000]       .init : 0xc1300000 - 0xc1500000   (2048 kB)
[    0.000000]       .data : 0xc1500000 - 0xc1686810   (1563 kB)
[    0.000000]        .bss : 0xc168fc68 - 0xc16f512c   ( 406 kB)

Therefore, if OMAP34XX_SRAM_VIRT is 0xd0010000, direct mapping is broken and the system doesn't work. I guess that we should use more stable address like as 0xf0000000.

Quoted 7 lines
Quoted 4 lines

OK. Let's forget about adding static mappings and my earlier patches to attempt to fix this. Below is what I now think we should merge as a fix before merging Joonsoo's patches. Please all review and test, adding Tero to Cc also.

Okay. Then, this patch will be merged by yourself as a fix? I'm okay with either way. Just let me know. :)

Well what's the plan with your patches? I'd not have mainline kernel broken so if this was the only blocker for the v4.15, then you should pick it.

Quoted 5 lines
Quoted 2 lines

However, if your patches are now planned for v4.16, then I'll queue it for early v4.15-rc.

It was the only blocker but I think that it's too late for v4.15. I will try again for v4.16. Please queue your fix for early v4.15-rc.

OK let's do that. I'll wait few days for comments.

Regards,

Tony

#51 lore raw
In reply to Tony Lindgren

Hi!

Quoted 11 lines
Quoted 5 lines

After commit 9caf25f996e8, user for CMA memory should use to check PageHighmem in order to get proper virtual address of the page. If someone doesn't use it, it is possible to use wrong virtual address and it then causes the use of wrong physical address. CONFIG_DEBUG_VIRTUAL would catch this case.

OK, no extra output of current next with CONFIG_DEBUG_VIRTUAL=y. Booting of n900 hangs with just the same error:

save_secure_sram() returns 0000ff02

Quoted 7 lines
Quoted 1 line

If it doesn't help, is there a way to test n900 configuration in QEMU?

I doubt that QEMU n900 boots in secure mode but instead shows the SoC as general purpose SoC. If so, you'd have to patch the omap3_save_secure_ram_context() to attempt to save secure RAM context in all cases. If that works then debugging with any omap3 board like beagleboard in QEMU should work.

Okay, linux-next from today still does not boot on n900. Is it something new, or was this still not fixed in -next?

Pavel

Signature · 9 lines
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20170915/1fdb70e2/attachment.sig>
#52 lore raw
Quoted 15 lines · Pavel Machek

Hi!

Quoted 13 lines
Quoted 5 lines

After commit 9caf25f996e8, user for CMA memory should use to check PageHighmem in order to get proper virtual address of the page. If someone doesn't use it, it is possible to use wrong virtual address and it then causes the use of wrong physical address. CONFIG_DEBUG_VIRTUAL would catch this case.

OK, no extra output of current next with CONFIG_DEBUG_VIRTUAL=y. Booting of n900 hangs with just the same error:

save_secure_sram() returns 0000ff02

Quoted 1 line

If it doesn't help, is there a way to test n900 configuration in QEMU?

Quoted 8 lines
Quoted 5 lines

I doubt that QEMU n900 boots in secure mode but instead shows the SoC as general purpose SoC. If so, you'd have to patch the omap3_save_secure_ram_context() to attempt to save secure RAM context in all cases. If that works then debugging with any omap3 board like beagleboard in QEMU should work.

Okay, linux-next from today still does not boot on n900. Is it something new, or was this still not fixed in -next?

Hello,

Still not fixed in -next since I cannot regenerate the error.

Thanks.

#53 lore raw

Linux-next broken for 2 weeks was Re: n900 in next-20170901

Hi!

Quoted 19 lines
Quoted 19 lines
Quoted 18 lines
Quoted 5 lines

After commit 9caf25f996e8, user for CMA memory should use to check PageHighmem in order to get proper virtual address of the page. If someone doesn't use it, it is possible to use wrong virtual address and it then causes the use of wrong physical address. CONFIG_DEBUG_VIRTUAL would catch this case.

OK, no extra output of current next with CONFIG_DEBUG_VIRTUAL=y. Booting of n900 hangs with just the same error:

save_secure_sram() returns 0000ff02

Quoted 1 line

If it doesn't help, is there a way to test n900 configuration in QEMU?

I doubt that QEMU n900 boots in secure mode but instead shows the SoC as general purpose SoC. If so, you'd have to patch the omap3_save_secure_ram_context() to attempt to save secure RAM context in all cases. If that works then debugging with any omap3 board like beagleboard in QEMU should work.

Quoted 6 lines
Quoted 2 lines

Okay, linux-next from today still does not boot on n900. Is it something new, or was this still not fixed in -next?

Hello,

Still not fixed in -next since I cannot regenerate the error.

Unfortunately, rest of the world can reproduce the error, and it means linux-next is useless for us.

I'd expect you to drop the relevant tree from linux-next when the error was reported. Clearly, those patches are unsuitable for 4.15, as they are broken, so they should not be in linux-next.

Thanks,
Pavel

Signature · 9 lines
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20170918/2e123912/attachment.sig>
#54 lore raw

Re: Linux-next broken for 2 weeks was Re: n900 in next-20170901

Hi Pavel,

Quoted 7 lines · On Mon, 18 Sep 2017 10:11:09 +0200 Pavel Machek <[email protected]> wrote:

Unfortunately, rest of the world can reproduce the error, and it means linux-next is useless for us.

I'd expect you to drop the relevant tree from linux-next when the error was reported. Clearly, those patches are unsuitable for 4.15, as they are broken, so they should not be in linux-next.

Andrew has asked me to drop the patches from linux-next today and I have done so.

Signature · 2 lines
Cheers,
Stephen Rothwell
#55 lore raw

Re: Linux-next broken for 2 weeks was Re: n900 in next-20170901

Quoted 7 lines · Stephen Rothwell

Hi Pavel,

Quoted 4 lines · On Mon, 18 Sep 2017 10:11:09 +0200 Pavel Machek <[email protected]> wrote:

Unfortunately, rest of the world can reproduce the error, and it means linux-next is useless for us.

Quoted 6 lines
Quoted 3 lines

I'd expect you to drop the relevant tree from linux-next when the error was reported. Clearly, those patches are unsuitable for 4.15, as they are broken, so they should not be in linux-next.

Andrew has asked me to drop the patches from linux-next today and I have done so.

Thanks!
Pavel

Signature · 9 lines
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20170919/f81382d2/attachment.sig>
#56 lore raw
In reply to Joonsoo Kim
Quoted 1 line · On Thursday 07 September 2017 16:30:38 Joonsoo Kim wrote:

If it doesn't help, is there a way to test n900 configuration in QEMU?

Hi Joonsoo, linaro version of QEMU has support for n900 machine. For more information how to prepare & run kernel image see this email:
https://lists.denx.de/pipermail/u-boot/2015-January/200171.html
(instead u-boot.bin you would supply kernel's zImage)

But QEMU does not support HS mode, so there is probably no secure ram. IIRC smc instructions should not be used in normal GP mode.

Signature · 2 lines
Pali Roh?r
pali.rohar at gmail.com
#57 lore raw

Hello,

Quoted 7 lines · Pali Rohár
Quoted 1 line · On Thursday 07 September 2017 16:30:38 Joonsoo Kim wrote:

If it doesn't help, is there a way to test n900 configuration in QEMU?

Hi Joonsoo, linaro version of QEMU has support for n900 machine. For more information how to prepare & run kernel image see this email:
https://lists.denx.de/pipermail/u-boot/2015-January/200171.html
(instead u-boot.bin you would supply kernel's zImage)

I tried to search to download required tools but cannot find a qflasher. Looks like you have qflasher and other *.bin for n900 image. Could you share them to me, please?

Quoted 2 lines

But QEMU does not support HS mode, so there is probably no secure ram. IIRC smc instructions should not be used in normal GP mode.

Okay. Thanks for information. I'm not sure that I can regenerate the error with n900 QEMU emulation but that would be my best so I will try.

Thanks.

#58 lore raw
In reply to Tony Lindgren

Hi!

Quoted 10 lines
Quoted 2 lines

It compiles ok, but it hangs on boot; black screen, so sometime before display is initialized.

Thanks for reporting it. Based on git bisect, the regression causing commit is 9caf25f996e8 ("mm/cma: manage the memory of the CMA area by using the ZONE_MOVABLE"). With this path applied, booting hangs with an error in omap3_save_secure_ram_context() after a call to _omap_save_secure_sram() that calls the related assembly code save_secure_ram_context.

Quoted 5 lines

However, looks like there is also some other commit causing issue.

Just reverting 9caf25f996e8 on Linux next causes the oops below.

Anybody got ideas why this now happens?

Thanks for quick bisect.

I take it 9caf25f996e8 is not going to be merged anywhere until this is fixed...?

Best regards,
Pavel

Signature · 9 lines
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20170908/b6c71cac/attachment-0001.sig>