New: Softdirty bit does not work with hugetlb
Quoted 1 line
Quoted 1 line
Start Bug Report <<
Bug ID: 211287
https://bugzilla.kernel.org/show_bug.cgi?id=211287
Summary: Softdirty bit does not work with hugetlb
When a memory region is mapped with huge pages, the softdirty bit is set only right after the huge pages is mapped. After the memory mapping, if the softdirty bit is cleared and the memory is written again, the softdirty bit is not set when reading from the process's pagemap.
Quoted 1 line
Quoted 1 line
End Bug Report <<
I am not surprised with this reported bug. The page fault code diverges pretty quickly for hugetlb pages. I have not looked closely at the details of soft dirty, and do not immediately see where in the fault path for normal pages it gets reset. But, I only took a quick glance.
I can work on adding support for hugetlb.
Can you provide some details about your use case?
Signature · 1 line
Mike Kravetz
Our use case is:
We maps both PMEM and hugetlb DRAM to process’s address space to provide large amount of memory to the application. Periodically, we clear the soft dirty bit by writing ‘4’ to the clear_refs. Then, we check the page’s soft dirty bit from pagemap and use this information to track write activity to memory. We migrate the data between DRAM and PMEM based the write activity for better performance.
Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10
From: Mike Kravetz<mailto:[email protected]>
Sent: 2021年1月21日 16:58
To: [email protected]<mailto:[email protected]>
Cc: Linux Memory Management List<mailto:[email protected]>
Subject: [Bug 211287] New: Softdirty bit does not work with hugetlb
Quoted 1 line
Quoted 1 line
Start Bug Report <<
Bug ID: 211287
https://bugzilla.kernel.org/show_bug.cgi?id=211287
Summary: Softdirty bit does not work with hugetlb
When a memory region is mapped with huge pages, the softdirty bit is set only right after the huge pages is mapped. After the memory mapping, if the softdirty bit is cleared and the memory is written again, the softdirty bit is not set when reading from the process's pagemap.
Quoted 1 line
Quoted 1 line
End Bug Report <<
I am not surprised with this reported bug. The page fault code diverges pretty quickly for hugetlb pages. I have not looked closely at the details of soft dirty, and do not immediately see where in the fault path for normal pages it gets reset. But, I only took a quick glance.
I can work on adding support for hugetlb.
Can you provide some details about your use case?
--
Mike Kravetz
Quoted 3 lines · shu wang
Our use case is:
We maps both PMEM and hugetlb DRAM to process’s address space to provide large amount of memory to the application. Periodically, we clear the soft dirty bit by writing ‘4’ to theclear_refs. Then, we check the page’s soft dirty bit from pagemap and use this information to track write activity to memory. We migrate the data between DRAM and PMEM based the write activity for better performance.
Thank you for the information!
If someone does not beat me to it, I should be able to look at this next week.
Signature · 1 line
Mike Kravetz
Quoted 29 lines
Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10
From: Mike Kravetz<mailto:[email protected]>
Sent: 2021年1月21日 16:58
To: [email protected]<mailto:[email protected]>
Cc: Linux Memory Management List<mailto:[email protected]>
Subject: [Bug 211287] New: Softdirty bit does not work with hugetlbQuoted 1 line
Quoted 1 line
Start Bug Report <<
Bug ID: 211287
https://bugzilla.kernel.org/show_bug.cgi?id=211287
Summary: Softdirty bit does not work with hugetlbWhen a memory region is mapped with huge pages, the softdirty bit is set only right after the huge pages is mapped. After the memory mapping, if the softdirty bit is cleared and the memory is written again, the softdirty bit is not set when reading from the process's pagemap.
Quoted 1 line
Quoted 1 line
End Bug Report <<
I am not surprised with this reported bug. The page fault code diverges pretty quickly for hugetlb pages. I have not looked closely at the details of soft dirty, and do not immediately see where in the fault path for normal pages it gets reset. But, I only took a quick glance.
I can work on adding support for hugetlb.
Can you provide some details about your use case?
--
Mike Kravetz
Quoted 8 lines · Mike Kravetz
Quoted 3 lines · shu wang
Our use case is:
We maps both PMEM and hugetlb DRAM to process’s address space to provide large amount of memory to the application. Periodically, we clear the soft dirty bit by writing ‘4’ to theclear_refs. Then, we check the page’s soft dirty bit from pagemap and use this information to track write activity to memory. We migrate the data between DRAM and PMEM based the write activity for better performance.Thank you for the information!
If someone does not beat me to it, I should be able to look at this next week.
Just a quick update.
The /proc clear_refs code handles THP pages and 'normal' pages. There is no check/processing for hugetlb pages. All the code will do is clear the VM_SOFTDIRTY vma flag and vma_set_page_prot. None of the individual ptes or page flags are touched.
The only soft dirty checking done by the /proc pagemap code is at the vma
level. It checks the VM_SOFTDIRTY flag. It does not look for soft dirty
on the individual ptes.
That explains the behavior described in the bug report.
With a quick and dirty hack I could make the code work in limited testing. Much work remains.
If anyone wants to jump in and help, feel free.
Signature · 1 line
Mike Kravetz