x86-64: Kernel with large page size
Hi
When we run our application on AMD Opteron processors, we are seeing a
large number of L1_AND_L2_DTLB_MISSES. We used oprofile to measure
these numbers.
We wanted to try with a bigger page size and see if we could bring it down. TLB caches 4k page translations. I don't know if larger page size would even help here.
I changed PAGE_SHIFT to 14 ( 16k page size ) in include/asm/page.h
and recompiled kernel and modules. It crashed ( PANIC: early
exception ) at the very initial stage of loading the image.
I looked at some of the mailing list archives for any information on this. I couldn't find anything on this subject . I appreciate any help on this.
There seem to be two 2-4MB page translations in L1 TLB cache on AMD machines. Will it be used only when the page size is 2MB or can they be used with smaller page sizes too.
Thanks for your help.
Regards
Subramanyam
Quoted 7 lines · Subrahmanyam Ongole
When we run our application on AMD Opteron processors, we are seeing a
large number ofL1_AND_L2_DTLB_MISSES. We used oprofile to measure
these numbers.
We wanted to try with a bigger page size and see if we could bring it down. TLB caches 4k page translations. I don't know if larger page size would even help here.
I changedPAGE_SHIFTto 14 ( 16k page size ) ininclude/asm/page.h
Quoted 8 lines
and recompiled kernel and modules. It crashed ( PANIC: early
exception ) at the very initial stage of loading the image. I looked at some of the mailing list archives for any information on this. I couldn't find anything on this subject . I appreciate any help on this.
There seem to be two 2-4MB page translations in L1 TLB cache on AMD machines. Will it be used only when the page size is 2MB or can they be used with smaller page sizes too.
PAGE_SIZE at the moment is intimately tied to the MMU's notions of address translation, which are determined by hardware.
-- wli
Quoted 5 lines · William Lee Irwin III
Quoted 3 lines · Subrahmanyam Ongole
When we run our application on AMD Opteron processors, we are seeing a
large number ofL1_AND_L2_DTLB_MISSES. We used oprofile to measure
these numbers.
[...]
Quoted 5 lines
PAGE_SIZEat the moment is intimately tied to the MMU's notions of address translation, which are determined by hardware.
And even if we were able to increase the PAGE_SIZE that the kernel uses, this wouldn't really help your TLB misses.
You may be able to use "huge pages" for your workload, which can use the 2/4MB pages. There is some good documentation for it in Documentation/vm/ (and probably on the web).
Signature · 3 lines
SUSE Labs, Novell Inc. Send instant messages to your online friends http://au.messenger.yahoo.com
Quoted 6 lines · Subrahmanyam Ongole
When we run our application on AMD Opteron processors, we are seeing a
large number ofL1_AND_L2_DTLB_MISSES. We used oprofile to measure
these numbers.We wanted to try with a bigger page size and see if we could bring it down. TLB caches 4k page translations. I don't know if larger page
You can use large pages in your application by mmaping from a file in hugetlbfs and configuring large pages using sysctl.
Quoted 1 line
size would even help here.
It probably wouldn't because Opteron has much more 4K DTLB entries than 2M DTLB entries. I have had people trying the opposite from what you tried (using 4K pages for the kernel instead of 2MB), but that also doesn't work right now.
-Andi