Skip to contentSkip to conversation

Memory performance / Cache problem

DISCUSSION [email protected] 3 participants 4 messages Oct 12, 2009 – Oct 13, 2009

lore
Share & tools
lore
#0 lore raw

I found the memory performance of newer kernels are quit poor on an EVM-Omap3 board. It works with 2-6 times performance on the old 2.6.22 kernel from TI's PSP.

Possible reasons:
- problem in config the kernel (did omap3_evm_defconfig)
- problem in kernel
- kernel expects some settings from uboot, which are not done there

I have tried the 2.6.29rc3 (from TI's PSP) and the 2.6.31 from git-tree. Both behave quite simular:

Transport in MByte:
memcpy = 204.073, loop4 = 183.212, loop1 = 81.693, rand =
4.534

while the 22 kernel:
memcpy = 453.932, loop4 = 469.934, loop1 = 125.031, rand =
29.631

Can someone give me help or can at least confirm that? Best regards
Steffen

Signature · 6 lines
GRATIS für alle GMX-Mitglieder: Die maxdome Movie-FLAT!
Jetzt freischalten unter http://portal.gmx.net/de/go/maxdome01
--
To unsubscribe from this list: send the line "unsubscribe linux-omap" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#1 lore raw

Can you dump the first few lines of the kernel log_buf during the kernel boot?

Something like this:

[    0.000000] Linux version 2.6.29-omap1-00005-gf9f407c-dirty (romit@maxwell) (gcc version 4.3.3 (Sourcery G++ Lite 2009q1-203) ) #14 PREEMPT Fri Oct 9 14:04:52 IST 2009
[    0.000000] CPU: ARMv7 Processor [411fc083] revision 3 (ARMv7), cr=10c5387f
[    0.000000] CPU: VIPT nonaliasing data cache, VIPT nonaliasing instruction cache
[    0.000000] Machine: OMAP ZOOM2 board
[    0.000000] Memory policy: ECC disabled, Data cache writeback
[    0.000000] L2 CACHE is enabled in bootloader
[    0.000000] OMAP3430 ES3.1

It should tell you if L2 cache is enabled or not.

Thanks,
-Romit

Quoted 27 lines

-----Original Message-----
From: [email protected] [mailto:linux-omap- [email protected]] On Behalf Of [email protected]
Sent: Monday, October 12, 2009 1:24 PM
To: [email protected]
Subject: Memory performance / Cache problem

I found the memory performance of newer kernels are quit poor on an EVM- Omap3 board. It works with 2-6 times performance on the old 2.6.22 kernel from TI's PSP.

Possible reasons:
- problem in config the kernel (did omap3_evm_defconfig)
- problem in kernel
- kernel expects some settings from uboot, which are not done there

I have tried the 2.6.29rc3 (from TI's PSP) and the 2.6.31 from git-tree. Both behave quite simular:

Transport in MByte:
memcpy = 204.073, loop4 = 183.212, loop1 = 81.693, rand =
4.534

while the 22 kernel:
memcpy = 453.932, loop4 = 469.934, loop1 = 125.031, rand =
29.631

Quoted 10 lines

Can someone give me help or can at least confirm that?
Best regards
Steffen
--
GRATIS für alle GMX-Mitglieder: Die maxdome Movie-FLAT! Jetzt freischalten unter http://portal.gmx.net/de/go/maxdome01
--
To unsubscribe from this list: send the line "unsubscribe linux-omap" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-omap" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#2 lore raw
In reply to [email protected]
Quoted 16 lines · On Monday 12 October 2009 10:54:09 ext [email protected] wrote:

I found the memory performance of newer kernels are quit poor on an EVM-Omap3 board. It works with 2-6 times performance on the old 2.6.22 kernel from TI's PSP.

Possible reasons:
- problem in config the kernel (did omap3_evm_defconfig)
- problem in kernel
- kernel expects some settings from uboot, which are not done there

I have tried the 2.6.29rc3 (from TI's PSP) and the 2.6.31 from git-tree. Both behave quite simular:

Transport in MByte:
memcpy = 204.073, loop4 = 183.212, loop1 = 81.693, rand =
4.534

Quoted 5 lines

while the 22 kernel:
memcpy = 453.932, loop4 = 469.934, loop1 = 125.031, rand =
29.631

Can someone give me help or can at least confirm that?

The numbers from 2.6.22 kernel look much better than anything I have ever seen with OMAP3.

How are you doing benchmarking? Is source buffer properly initialized?

The point is that if you just happen to allocate a large buffer without initializing it, it may end up having all the memory pages referencing to a single zero page in physical memory. In this case reading from this buffer will in fact be perfectly cached in L1 cache and memcpy would look fast.

If it is not the case, investigating how to boost memory performance in the latest kernels is very interesting for sure.

Signature · 2 lines
Best regards,
Siarhei Siamashka
#3 lore raw

The L2 cache is set and running.
I don't know - can it be configured or misconfigured somehow?

I just checked the output of 2.6.22 kernel and get these lines (which I don't have in newer kernels):

CPU0: D VIPT write-through cache
CPU0: cache: 768 bytes, associativity 1, 8 byte lines, 64 sets Built 1 zonelists. Total pages: 32512

I am wondering what is this? First thought was L1 cache, but it's to small.

The benchmark is running on same hardware, same uboot, same rootfs, just the kernel is different.

Quoted 41 lines · On Monday 12 October 2009 10:54:09 ext [email protected] wrote:
Quoted 16 lines

I found the memory performance of newer kernels are quit poor on an EVM-Omap3 board. It works with 2-6 times performance on the old 2.6.22 kernel from TI's PSP.

Possible reasons:
- problem in config the kernel (did omap3_evm_defconfig)
- problem in kernel
- kernel expects some settings from uboot, which are not done there

I have tried the 2.6.29rc3 (from TI's PSP) and the 2.6.31 from git-tree. Both behave quite simular:

Transport in MByte:
memcpy = 204.073, loop4 = 183.212, loop1 = 81.693, rand =
4.534

Quoted 5 lines

while the 22 kernel:
memcpy = 453.932, loop4 = 469.934, loop1 = 125.031, rand =
29.631

Can someone give me help or can at least confirm that?

The numbers from 2.6.22 kernel look much better than anything I have ever seen with OMAP3.

How are you doing benchmarking? Is source buffer properly initialized?

The point is that if you just happen to allocate a large buffer without initializing it, it may end up having all the memory pages referencing to a
single zero page in physical memory. In this case reading from this buffer will in fact be perfectly cached in L1 cache and memcpy would look fast.

If it is not the case, investigating how to boost memory performance in the
latest kernels is very interesting for sure.

Signature · 2 lines
Best regards,
Siarhei Siamashka
Signature · 2 lines
Neu: GMX DSL bis 50.000 kBit/s und 200,- Euro Startguthaben!
http://portal.gmx.net/de/go/dsl02