Skip to contentSkip to conversation

AM335x: enabling datacache in SPL slows down system?

U-Boot Bas van den Berg 4 participants 7 messages Jun 20, 2013 – Jun 24, 2013

lore
Share & tools
lore
#0 lore raw

I'm trying to optimize the boottime for an AM335x based board. Currently we're loading the kernel from SPL directly (falcon mode). Loading the kernel from flash to ram takes roughly 3 seconds.
When doing this from U-boot itself with 'nand read .., it only takes 1.8 seconds. It seems that U-boot has data_cache enabled, while SPL did not.

When enabling the data cache with dcache_enable(), loading takes 3.5 seconds, so even slower! The enabling itself does not take a significant amount of time.

Anyone have any idea why enabling the data cache slows SPL down? Or does U-boot do anything else that I'm missing?

Bas

#1 lore raw

Hallo Bas,

Quoted 11 lines · Bas van den Berg

I'm trying to optimize the boottime for an AM335x based board. Currently we're loading the kernel from SPL directly (falcon mode). Loading the kernel from flash to ram takes roughly 3 seconds.
When doing this from U-boot itself with 'nand read .., it only takes 1.8 seconds. It seems that U-boot has data_cache enabled, while SPL did not.

When enabling the data cache with dcache_enable(), loading takes 3.5 seconds, so even slower! The enabling itself does not take a significant amount of time.

Anyone have any idea why enabling the data cache slows SPL down? Or does U-boot do anything else that I'm missing?

For an am3517 I added the code below to spl_board_init to enable caches in SPL. It adds the SRAM region to the mmu, by an obvious hack, but it works. I don't remember by heart why I change / update gd->tlb_addr. The used address is somewhere in the end of DRAM.

No guarantees this is related to your issue (or to work at all).

---
Groet,
Jeroen

	if (!spl_start_uboot()) {
		dram_init_banksize();

		/* reserve TLB table (4k) */
		gd->tlb_addr = 0x8fff0000;
		debug("TLB table at: %08lx\n", gd->tlb_addr);

		/* add sram (64k actually, but the cp15 counts in MiBs) */
		gd->bd->bi_dram[1].start = CONFIG_SPL_TEXT_BASE;
		gd->bd->bi_dram[1].size =  (1 << 20);

		/* Enable caches */
		enable_caches();
	}
#2 lore raw
In reply to Bas van den Berg

Dear "Bas van den Berg",

Quoted 9 lines · In message <[email protected]> you wrote:

I'm trying to optimize the boottime for an AM335x based board. Currently we're loading the kernel from SPL directly (falcon mode). Loading the kernel from flash to ram takes roughly 3 seconds.
When doing this from U-boot itself with 'nand read .., it only takes 1.8 seconds. It seems that U-boot has data_cache enabled, while SPL did not.

When enabling the data cache with dcache_enable(), loading takes 3.5 seconds, so even slower! The enabling itself does not take a significant amount of time.

We observed the very same issue with the verry first prototype implementation of Falcon mode on an AM3517 based board. At this time we did not have time nor resources to figure out what exactly was causing this issue, but it's interesting to see that 1) it still persists and 2) it also happens on AM335x

Quoted 2 lines

Anyone have any idea why enabling the data cache slows SPL down? Or does U-boot do anything else that I'm missing?

@Tom: do you have any additional input what we should check?

Best regards,

Wolfgang Denk

Signature · 4 lines
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
An Elephant is a mouse with an Operating System.              - Knuth
#3 lore raw

Hello Wolfgang,

Quoted 2 lines · Wolfgang Denk

We observed the very same issue with the verry first prototype implementation of Falcon mode on an AM3517 based board. [..]

For the am3517 the only thing needed is to add SRAM to the known memory. See [1], that should change the cache misses (which likely cause the extra delays) to actual caches. U-boot only adds the detected memory and since U-boot is running from DRAM and SPL is not, the difference seems obvious.

Regards,
Jeroen

[1] http://lists.denx.de/pipermail/u-boot/2013-June/156949.html

#4 lore raw
Quoted 11 lines · Jeroen Hofstee

Hello Wolfgang,

Quoted 2 lines · Wolfgang Denk

We observed the very same issue with the verry first prototype implementation of Falcon mode on an AM3517 based board. [..]

For the am3517 the only thing needed is to add SRAM to the known memory. See [1], that should change the cache misses (which likely cause the extra delays) to actual caches. U-boot only adds the detected memory and since U-boot is running from DRAM and SPL is not, the difference seems obvious.

This would be my guess as folks have done functional but not mainlineable changes in this direction, as well as DMA, before.

Signature · 8 lines
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20130621/f31f281f/attachment.pgp>
#5 lore raw

Hi Jeroen,

After debugging a bit more, I found that in our code, we didn't set the RAM size correctly (so no real RAM section was setup for caching). Fixing this fixed our time to 2.4 seconds (U-boot still took 1.8 sec).

Adding your SRAM patch (modified, since AM335x has more SRAM),
The load time dropped from 2.4 -> 1.9 seconds. So adding this small area saved half a second!
I write off the remaing difference (1.8 vs 1.9 seconds) to code differences between our SPL and U-boot nand functions.

Thanks,
Bas

_____

From: Tom Rini [mailto:trini at ti.com]
To: Jeroen Hofstee [mailto:jeroen at myspectrum.nl]
Cc: Wolfgang Denk [mailto:wd at denx.de], bas.van.den.berg at alten.nl, u-boot at lists.denx.de
Sent: Fri, 21 Jun 2013 20:56:53 +0100
Subject: Re: [U-Boot] AM335x: enabling datacache in SPL slows down system?

Quoted 11 lines · Jeroen Hofstee

Hello Wolfgang,

Quoted 2 lines · Wolfgang Denk

We observed the very same issue with the verry first prototype implementation of Falcon mode on an AM3517 based board. [..]

For the am3517 the only thing needed is to add SRAM to the known memory. See [1], that should change the cache misses (which likely cause the extra delays) to actual caches. U-boot only adds the detected memory and since U-boot is running from DRAM and SPL is not, the difference seems obvious.

This would be my guess as folks have done functional but not mainlineable changes in this direction, as well as DMA, before.

--
Tom

#6 lore raw

Hello Bas,

Quoted 9 lines · Bas van den Berg

After debugging a bit more, I found that in our code, we didn't set the RAM size correctly (so no real RAM section was setup for caching). Fixing this fixed our time to 2.4 seconds (U-boot still took 1.8 sec).

Adding your SRAM patch (modified, since AM335x has more SRAM),
The load time dropped from 2.4 -> 1.9 seconds. So adding this small area saved half a second!
I write off the remaing difference (1.8 vs 1.9 seconds) to code differences between our SPL and U-boot nand functions.

Thanks for reporting and checking that adding the SRAM region is needed as well (I suppose you already verified that -O2 cannot get rid of the last difference).

Regards,
Jeroen

ps: the consensus on this mailinglist is not to top post..