x86: vmalloc and THP
Hello,
Does memory allocated using vmalloc() will be mapped using huge pages either directly or later by THP ?
If not, is there any fast way to change this behavior ? Maybe by changing the granularity/alignment of such allocations to allow such mapping ?
Thanks,
Oren Twaig.
This email is free from viruses and malware because avast! Antivirus protection is active.
-- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: [email protected]
Quoted 1 line · Oren Twaig
<html style="direction: ltr;">
plain/text, please.
Quoted 4 lines
Hello,
Does memory allocated using
vmalloc()will be mapped using huge pages either directly or later by THP ?
No. It's neither aligned properly, nor physically contiguous.
Quoted 3 lines
If not, is there any fast way to change this behavior ? Maybe by changing the granularity/alignment of such allocations to allow such mapping ?
What's the point to use vmalloc() in this case?
Signature · 7 lines
Kirill A. Shutemov -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 5 lines · Kirill A. Shutemov
Quoted 2 lines · Oren Twaig
Does memory allocated using
vmalloc()will be mapped using huge pages either directly or later by THP ?
Quoted 7 lines
No. It's neither aligned properly, nor physically contiguous.
Quoted 3 lines
If not, is there any fast way to change this behavior ? Maybe by changing the granularity/alignment of such allocations to allow such mapping ?
What's the point to use
vmalloc()in this case?
Look at various large hashes we have in the system, all using vmalloc() :
[ 0.006856] Dentry cache hash table entries: 16777216 (order: 15, 134217728 bytes) [ 0.033130] Inode-cache hash table entries: 8388608 (order: 14, 67108864 bytes) [ 1.197621] TCP established hash table entries: 524288 (order: 11, 8388608 bytes)
I would imagine a performance difference if we were using hugepages.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 6 lines · Eric Dumazet
Quoted 5 lines · Kirill A. Shutemov
Quoted 3 lines · Oren Twaig
If not, is there any fast way to change this behavior ? Maybe by changing the granularity/alignment of such allocations to allow such mapping ?
Quoted 8 lines
Quoted 1 line
What's the point to use
vmalloc()in this case?Look at various large hashes we have in the system, all using
vmalloc():[ 0.006856] Dentry cache hash table entries: 16777216 (order: 15, 134217728 bytes) [ 0.033130] Inode-cache hash table entries: 8388608 (order: 14, 67108864 bytes) [ 1.197621] TCP established hash table entries: 524288 (order: 11, 8388608 bytes)
I see lower-order allocation in upstream code. Is it some distribution tweak?
Quoted 1 line
I would imagine a performance difference if we were using hugepages.
Okay, it's *probably* a valid point.
The hash tables are only allocated with vmalloc() on NUMA system, if hashdist=1 (default on NUMA). It does it to distribute memory between nodes. vmalloc() in NUMA_NO_NODE case will allocate all memory with 0-order page allocations: no physical contiguous memory for hugepage mappings.
I guess we could teach vmalloc() to interleave between nodes on PMD_SIZE chunks rather then on PAGE_SIZE if caller asks for big memory allocations. Although, I'm not sure it it would fit all vmalloc() users.
We also would need to allocate PMD_SIZE-aligned virtual address range to be able to mapped allocated memory with pmds.
It's *potentially* interesting research project. Any volunteers?
Signature · 7 lines
Kirill A. Shutemov -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Hi Kirill,
I saw the thread has developed nicely :), still - wanted to answer your question below.
Quoted 3 lines · Kirill A. Shutemov
Quoted 1 line · Oren Twaig
<html style="direction: ltr;">
plain/text, please.
Yes - noticed the html, sent again in plain text.
Quoted 4 lines
Quoted 3 lines
If not, is there any fast way to change this behavior ? Maybe by changing the granularity/alignment of such allocations to allow such mapping ?
What's the point to use
vmalloc()in this case?
I've noticed that some lock/s are using linear addresses which are located at 0xffffc901922b4500 and from what I understand from mm.txt (kernel 3.0.101):
*ffffc90000000000 - ffffe8ffffffffff (=45 bits) vmalloc/ioremap space
*So I'm not sure who/how/why this lock got allocated there, but obviously it is using that linear set. No ?
Quoted 1 line
---
This email is free from viruses and malware because avast! Antivirus protection is active.
http://www.avast.com
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 8 lines · Oren Twaig
Quoted 1 line
What's the point to use
vmalloc()in this case?I've noticed that some lock/s are using linear addresses which are located at
0xffffc901922b4500and from what I understand from mm.txt (kernel 3.0.101):
*ffffc90000000000 -ffffe8ffffffffff(=45 bits) vmalloc/ioremap space*So I'm not sure who/how/why this lock got allocated there, but obviously it is using that linear set. No ?
It would be nice to know what lock it was, but nothing is inherently wrong with lock in vmalloc space.
Signature · 7 lines
Kirill A. Shutemov -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>