Skip to contentSkip to conversation

Transparent Hugepage Nit

DISCUSSION John McCorquodale 3 participants 3 messages Jan 15, 2013 – Jan 16, 2013

lore
Share & tools
lore
#0 lore raw

Suppose a hugepage-aligned mmap(MAP_ANONYMOUS) mapping has been madvise()d HUGEPAGE. If a subeqeuent call to mremap() grows the mapping and has to move the mapping, the hugepage-alignment is not preserved in the choice of new address (in 3.7.2).

I can workaround this by doing a 1-hugepage-oversized remap to find a new aligned address and then size it back down MREMAP_FIXED, but that's probably a lot of frags to 4k pages and back that aren't necessary.

Should it not be the case that mremap(MAYMOVE) on something advised hugepage ALWAYS chooses a hugepage-aligned address? This would be handy when doing the initial allocation too: mmap, madvise, mremap (to the same size) to get alignment.

Cheers,

-mcq

#1 lore raw
Quoted 5 lines · John McCorquodale

Suppose a hugepage-aligned mmap(MAP_ANONYMOUS) mapping has been madvise()d HUGEPAGE. If a subeqeuent call to mremap() grows the mapping and has to move the mapping, the hugepage-alignment is not preserved in the choice of new address (in 3.7.2).

Quoted 8 lines

I can workaround this by doing a 1-hugepage-oversized remap to find a new aligned address and then size it back down MREMAP_FIXED, but that's probably a lot of frags to 4k pages and back that aren't necessary.

Should it not be the case that mremap(MAYMOVE) on something advised hugepage ALWAYS chooses a hugepage-aligned address? This would be handy when doing the initial allocation too: mmap, madvise, mremap (to the same size) to get alignment.

The hole searching currently doesn't know anything about transparent huge pages. There were some discussions on fixing it. But it's essentially a trade off between memory fragmentation and huge page
optimization: aggressively aligning to 2MB can lose address space in holes.

Usually if the program uses large enough mappings and enough memory it shouldn't be a problem.

-Andi

Signature · 1 line
[email protected] -- Speaking for myself only
#2 lore raw
Quoted 24 lines · Andi Kleen
Quoted 1 line · John McCorquodale

Suppose a hugepage-aligned mmap(MAP_ANONYMOUS) mapping has been

madvise()d

Quoted 1 line

HUGEPAGE. If a subeqeuent call to mremap() grows the mapping and

has to

Quoted 1 line

move the mapping, the hugepage-alignment is not preserved in the

choice of

Quoted 3 lines

new address (in 3.7.2).

I can workaround this by doing a 1-hugepage-oversized remap to find

a new

Quoted 1 line

aligned address and then size it back down MREMAP_FIXED, but that's

probably

Quoted 3 lines

a lot of frags to 4k pages and back that aren't necessary.

Should it not be the case that mremap(MAYMOVE) on something advised

hugepage

Quoted 1 line

ALWAYS chooses a hugepage-aligned address? This would be handy

when doing the

Quoted 1 line

initial allocation too: mmap, madvise, mremap (to the same size) to

get

Quoted 1 line

alignment.

Quoted 8 lines

The hole searching currently doesn't know anything about transparent huge pages. There were some discussions on fixing it. But it's essentially a trade off between memory fragmentation and huge page
optimization: aggressively aligning to 2MB can lose address space in holes.

Usually if the program uses large enough mappings and enough memory it shouldn't be a problem.

Possibly we should just document that doing mremap() on a hugepage loses the hugepageness, and consider it pilot error to do that?

Rob