Skip to contentSkip to conversation

Request for clarification regarding huge pages and remapping of virtual to physical address

DISCUSSION sunil bhargo 2 participants 4 messages Apr 17, 2024 1 message not in the archive Memory management

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

#1 lore raw

Hi,
We have an application which relies on virtual address to physical address mapping to remain static. The buffer is allocated using malloc. It is in user space and we were using the mlock thinking that as it would prevent the swapping the va->pa address mapping would be static. But it seems to be a faulty assumption. If the memory is allocated from huge pages by reserving space upfront at the kernel boot up and then mmap’ing using MAP_HUGETLB, is it assured that virtual address to physical mapping won’t be remapped and will be static ?
Thanks,Sunil Bhargo

#2 lore raw
Quoted 5 lines · sunil bhargo

Hi,

We have an application which relies on virtual address to physical address mapping to remain static. The buffer is allocated using malloc. It is in user space and we were using the mlock thinking that as it would prevent the swapping the va->pa address mapping would be static. But it seems to be a faulty assumption.

If the memory is allocated from huge pages by reserving space upfront at the kernel boot up and then mmap’ing using MAP_HUGETLB, is it assured that virtual address to physical mapping won’t be remapped and will be static ?

Unless you have a kernel that has page migration disabled, no.

You could use secretmem (which is currently not migratabale/swappable/ ...), but that has very limited capabilities.

A "hack" would be to register that (ordinary) memory using iouring as a fixed buffer. That way, it will be longterm pinned and can neither get migrated nor swapped out.

Signature · 3 lines
Cheers,

David / dhildenb
#3 lore raw

David,Thanks for your response. So if i get it right then reserving the memory at boot time with/without the huge page, the page migration can happen ?

Thanks,Sunil Bhargo

    On Wednesday, April 17, 2024 at 02:47:30 PM GMT+5:30, David Hildenbrand <[email protected]> wrote:  
Quoted 8 lines · sunil bhargo

Hi,

We have an application which relies on virtual address to physical address mapping to remain static. The buffer is allocated using malloc. It is in user space and we were using the mlock thinking that as it would prevent the swapping the va->pa address mapping would be static. But it seems to be a faulty assumption.

Quoted 4 lines

If the memory is allocated from huge pages by reserving space upfront at the kernel boot up and then mmap’ing using MAP_HUGETLB, is it assured that virtual address to physical mapping won’t be remapped and will be static ?

Unless you have a kernel that has page migration disabled, no.

You could use secretmem (which is currently not migratabale/swappable/ ...), but that has very limited capabilities.

A "hack" would be to register that (ordinary) memory using iouring as a fixed buffer. That way, it will be longterm pinned and can neither get migrated nor swapped out.

Signature · 3 lines
Cheers,

David / dhildenb
#4 lore raw
Quoted 2 lines · sunil bhargo

David,
Thanks for your response. So if i get it right then reserving the memory at boot time with/without the huge page, the page migration can happen ?

I *think* even hugetlb folios allocated/reserved during early boot can be migrated in memory later. The chance of that happening is small, though. Or I might be wrong :)

Migration of gigantic huge pages (e.g., 1 GiB) is even more unlikely to happen.

Signature · 3 lines
Cheers,

David / dhildenb