docs: update THP admin guide about non-tmpfs filesystem support
Subsystems (4)
PATCH v3 1/1 docs: update THP admin guide about non-tmpfs filesystem support
From: Pankaj Raghav <[email protected]>
THP support for non-tmpfs filesystem has been around for some time now. Update the admin guide to reflect it.
While we are at it, move FilePmdMapped to previous paragraph for clarity, and clarify ShmemPmdMapped & ShmemHugePage.
---
Changes since v2:
- Address comment from Bagas Sanjaya
- Squash commits and Ack from David
Documentation/admin-guide/mm/transhuge.rst | 22 +++++++++++++++------- 1 file changed, 15 insertions(+), 7 deletions(-)
Documentation/admin-guide/mm/transhuge.rst +15 -7
@@ -12,8 +12,8 @@ using huge pages for the backing of virtual memory with huge pages that supports the automatic promotion and demotion of page sizes and without the shortcomings of hugetlbfs. -Currently THP only works for anonymous memory mappings and tmpfs/shmem. -But in the future it can expand to other filesystems. +Currently, THP only works for anonymous memory mappings, tmpfs/shmem and +filesystems that support large folios. .. note:: in the examples below we presume that the basic page size is 4K and
@@ -463,11 +463,19 @@ fields for each mapping. (Note that AnonHugePages only applies to traditional PMD-sized THP for historical reasons and should have been called AnonHugePmdMapped). -The number of file transparent huge pages mapped to userspace is available -by reading ShmemPmdMapped and ShmemHugePages fields in ``/proc/meminfo``. -To identify what applications are mapping file transparent huge pages, it -is necessary to read ``/proc/PID/smaps`` and count the FilePmdMapped fields -for each mapping. +The number of PMD-sized transparent huge pages currently used by +filesystem data (page cache) is available by reading the FileHugePages field +in ``/proc/meminfo``. The number of these huge pages that are mapped to userspace +is available by reading FilePmdMapped field in ``proc/meminfo``. To identify +what applications are mapping these huge pages, it is necessary to read +``/proc/PID/smaps`` and count the FilePmdMapped fields for each mapping. + +In similar fashion, the number of PMD-sized transparent huge pages currently +used by tmpfs/shmem is available by reading the ShmemHugePages field +in ``/proc/meminfo``. The number of these huge pages that are mapped to userspace +is available by reading ShmemPmdMapped field in ``proc/meminfo``. To identify +what applications are mapping these huge pages, it is necessary to read +``/proc/PID/smaps`` and count the ShmemPmdMapped fields for each mapping. Note that reading the smaps file is expensive and reading it frequently will incur overhead.
base-commit: f0a16f5363325cc8d9382471cdc7b654c53254c9
Signature · 1 line
2.47.2
Quoted 24 lines · Pankaj Raghav (Samsung)
From: Pankaj Raghav <[email protected]>
THP support for non-tmpfs filesystem has been around for some time now. Update the admin guide to reflect it.
While we are at it, move FilePmdMapped to previous paragraph for clarity, and clarify ShmemPmdMapped & ShmemHugePage.
Signed-off-by Pankaj Raghav <[email protected]>---
Changes since v2:
- Address comment from Bagas Sanjaya
- Squash commits and Ack from DavidDocumentation/admin-guide/mm/transhuge.rst | 22 +++++++++++++++------- 1 file changed, 15 insertions(+), 7 deletions(-)Documentation/admin-guide/mm/transhuge.rst +0
-0
Documentation/admin-guide/mm/transhuge.rst using huge pages for the backing of virtual memory with huge pages · line 16
Quoted 7 lines
Documentation/admin-guide/mm/transhuge.rst +2
-2@@ -12,8 +12,8 @@ using huge pages for the backing of virtual memory with huge pages that supports the automatic promotion and demotion of page sizes and without the shortcomings of hugetlbfs. -Currently THP only works for anonymous memory mappings and tmpfs/shmem. -But in the future it can expand to other filesystems. +Currently, THP only works for anonymous memory mappings, tmpfs/shmem and +filesystems that support large folios.
That seems to allude that THP can be supported on filesystems that suppor large folios. I don't think we want to call that THP and that can confuse folks. Leaving "currently" also seems to indicate that there is more work to be done for THP for filesystems but that's not true as well. So how about something like:
THP only works for anonymous memory mappings, and the tmpfs/shmem is the only filesystem to support it. The alternative to THP for other filesystems is to support large folios and with it you can end up using huge pages opportunistically.
Luis
Quoted 39 lines · Luis Chamberlain
Quoted 24 lines · Pankaj Raghav (Samsung)
From: Pankaj Raghav <[email protected]>
THP support for non-tmpfs filesystem has been around for some time now. Update the admin guide to reflect it.
While we are at it, move FilePmdMapped to previous paragraph for clarity, and clarify ShmemPmdMapped & ShmemHugePage.
Signed-off-by Pankaj Raghav <[email protected]>---
Changes since v2:
- Address comment from Bagas Sanjaya
- Squash commits and Ack from DavidDocumentation/admin-guide/mm/transhuge.rst | 22 +++++++++++++++------- 1 file changed, 15 insertions(+), 7 deletions(-)Documentation/admin-guide/mm/transhuge.rst +0
-0Documentation/admin-guide/mm/transhuge.rst using huge pages for the backing of virtual memory with huge pages · line 16
Quoted 7 lines
Documentation/admin-guide/mm/transhuge.rst +2
-2@@ -12,8 +12,8 @@ using huge pages for the backing of virtual memory with huge pages that supports the automatic promotion and demotion of page sizes and without the shortcomings of hugetlbfs. -Currently THP only works for anonymous memory mappings and tmpfs/shmem. -But in the future it can expand to other filesystems. +Currently, THP only works for anonymous memory mappings, tmpfs/shmem and +filesystems that support large folios.That seems to allude that THP can be supported on filesystems that suppor large folios. I don't think we want to call that THP and that can confuse folks. Leaving "currently" also seems to indicate that there is more work to be done for THP for filesystems but that's not true as well. So how about something like:
Quoted 3 lines
THP only works for anonymous memory mappings, and the tmpfs/shmem is the only filesystem to support it. The alternative to THP for other filesystems is to support large folios and with it you can end up using huge pages
That makes things more complicated without a good reason.
See CONFIG_READ_ONLY_THP_FOR_FS as an early usage of the term "THP" for stuff we have in the pagecache. (with large folios we now properly implement this concept, and support more than only PMD size)
Signature · 3 lines
Cheers, David / dhildenb
Quoted 39 lines · David Hildenbrand
Quoted 38 lines · Luis Chamberlain
Quoted 24 lines · Pankaj Raghav (Samsung)
From: Pankaj Raghav <[email protected]>
THP support for non-tmpfs filesystem has been around for some time now. Update the admin guide to reflect it.
While we are at it, move FilePmdMapped to previous paragraph for clarity, and clarify ShmemPmdMapped & ShmemHugePage.
Signed-off-by Pankaj Raghav <[email protected]>---
Changes since v2:
- Address comment from Bagas Sanjaya
- Squash commits and Ack from DavidDocumentation/admin-guide/mm/transhuge.rst | 22 +++++++++++++++------- 1 file changed, 15 insertions(+), 7 deletions(-)Documentation/admin-guide/mm/transhuge.rst +0
-0Documentation/admin-guide/mm/transhuge.rst using huge pages for the backing of virtual memory with huge pages · line 15
Quoted 6 lines
Documentation/admin-guide/mm/transhuge.rst +2
-2@@ -12,8 +12,8 @@ using huge pages for the backing of virtual memory with huge pages that supports the automatic promotion and demotion of page sizes and without the shortcomings of hugetlbfs. -Currently THP only works for anonymous memory mappings and tmpfs/shmem. -But in the future it can expand to other filesystems. +Currently, THP only works for anonymous memory mappings, tmpfs/shmem and +filesystems that support large folios.That seems to allude that THP can be supported on filesystems that suppor large folios. I don't think we want to call that THP and that can confuse folks. Leaving "currently" also seems to indicate that there is more work to be done for THP for filesystems but that's not true as well. So how about something like:
Quoted 8 lines
Quoted 3 lines
THP only works for anonymous memory mappings, and the tmpfs/shmem is the only filesystem to support it. The alternative to THP for other filesystems is to support large folios and with it you can end up using huge pages
That makes things more complicated without a good reason.
See
CONFIG_READ_ONLY_THP_FOR_FSas an early usage of the term "THP" for stuff we have in the pagecache.
OK.
Quoted 2 lines
(with large folios we now properly implement this concept, and support more than only PMD size)
Do we really want to call large folio support on filesystems THP?
Luis
Quoted 49 lines · Luis Chamberlain
Quoted 39 lines · David Hildenbrand
Quoted 38 lines · Luis Chamberlain
Quoted 24 lines · Pankaj Raghav (Samsung)
From: Pankaj Raghav <[email protected]>
THP support for non-tmpfs filesystem has been around for some time now. Update the admin guide to reflect it.
While we are at it, move FilePmdMapped to previous paragraph for clarity, and clarify ShmemPmdMapped & ShmemHugePage.
Signed-off-by Pankaj Raghav <[email protected]>---
Changes since v2:
- Address comment from Bagas Sanjaya
- Squash commits and Ack from DavidDocumentation/admin-guide/mm/transhuge.rst | 22 +++++++++++++++------- 1 file changed, 15 insertions(+), 7 deletions(-)Documentation/admin-guide/mm/transhuge.rst +0
-0Documentation/admin-guide/mm/transhuge.rst using huge pages for the backing of virtual memory with huge pages · line 15
Quoted 6 lines
Documentation/admin-guide/mm/transhuge.rst +2
-2@@ -12,8 +12,8 @@ using huge pages for the backing of virtual memory with huge pages that supports the automatic promotion and demotion of page sizes and without the shortcomings of hugetlbfs. -Currently THP only works for anonymous memory mappings and tmpfs/shmem. -But in the future it can expand to other filesystems. +Currently, THP only works for anonymous memory mappings, tmpfs/shmem and +filesystems that support large folios.That seems to allude that THP can be supported on filesystems that suppor large folios. I don't think we want to call that THP and that can confuse folks. Leaving "currently" also seems to indicate that there is more work to be done for THP for filesystems but that's not true as well. So how about something like:
Quoted 8 lines
Quoted 3 lines
THP only works for anonymous memory mappings, and the tmpfs/shmem is the only filesystem to support it. The alternative to THP for other filesystems is to support large folios and with it you can end up using huge pages
That makes things more complicated without a good reason.
See
CONFIG_READ_ONLY_THP_FOR_FSas an early usage of the term "THP" for stuff we have in the pagecache.
Quoted 6 lines
OK.
Quoted 2 lines
(with large folios we now properly implement this concept, and support more than only PMD size)
Do we really want to call large folio support on filesystems THP?
Good question.
"folio" is just the metadata we currently use to manage a chunk of memory, and a "large folio" is one that spans more than a single page -- huge page, large page, super page, ... in the past the metadata for that used to be a complicated piece of "compound page". In the future, we might call it differently (struct file_mem ?), who knows.
So "large folio" support in a fs allows for the usage of these larger chunks of memory (huge pages).
The filesystem tries to transparently make use of huge pages (of various sizes); in contrast to hugetlb, that is "transparent" in a way that these can be split, partially mapped, partially truncated -- just like for anon or shmem.
(Well, the "transparent" part was never really 100% true, but that's a different story)
I see more similarity with THP than with hugetlb in that sense.
In the end I don't think the terminology matters that much -- note that we are talking about an admin guide that resides in transhuge.rst -- as long as people don't confuse it with "ordinary" huge pages as in hugetlb. "file huge page" is rather misleading.
I'd be curious in which sense these file thingies are different than what we have for anon/shmem/... except that some toggles don't apply to them and the pagecache mostly "more transparently" makes use of them. (which makes them even more transparent? ;) )
Signature · 3 lines
Cheers, David / dhildenb
Quoted 59 lines · David Hildenbrand
Quoted 49 lines · Luis Chamberlain
Quoted 39 lines · David Hildenbrand
Quoted 38 lines · Luis Chamberlain
Quoted 24 lines · Pankaj Raghav (Samsung)
From: Pankaj Raghav <[email protected]>
THP support for non-tmpfs filesystem has been around for some time now. Update the admin guide to reflect it.
While we are at it, move FilePmdMapped to previous paragraph for clarity, and clarify ShmemPmdMapped & ShmemHugePage.
Signed-off-by Pankaj Raghav <[email protected]>---
Changes since v2:
- Address comment from Bagas Sanjaya
- Squash commits and Ack from DavidDocumentation/admin-guide/mm/transhuge.rst | 22 +++++++++++++++------- 1 file changed, 15 insertions(+), 7 deletions(-)Documentation/admin-guide/mm/transhuge.rst +0
-0Documentation/admin-guide/mm/transhuge.rst using huge pages for the backing of virtual memory with huge pages · line 15
Quoted 6 lines
Documentation/admin-guide/mm/transhuge.rst +2
-2@@ -12,8 +12,8 @@ using huge pages for the backing of virtual memory with huge pages that supports the automatic promotion and demotion of page sizes and without the shortcomings of hugetlbfs. -Currently THP only works for anonymous memory mappings and tmpfs/shmem. -But in the future it can expand to other filesystems. +Currently, THP only works for anonymous memory mappings, tmpfs/shmem and +filesystems that support large folios.That seems to allude that THP can be supported on filesystems that suppor large folios. I don't think we want to call that THP and that can confuse folks. Leaving "currently" also seems to indicate that there is more work to be done for THP for filesystems but that's not true as well. So how about something like:
Quoted 8 lines
Quoted 3 lines
THP only works for anonymous memory mappings, and the tmpfs/shmem is the only filesystem to support it. The alternative to THP for other filesystems is to support large folios and with it you can end up using huge pages
That makes things more complicated without a good reason.
See
CONFIG_READ_ONLY_THP_FOR_FSas an early usage of the term "THP" for stuff we have in the pagecache.Quoted 6 lines
OK.
Quoted 2 lines
(with large folios we now properly implement this concept, and support more than only PMD size)
Do we really want to call large folio support on filesystems THP?
Good question.
Quoted 8 lines
"folio" is just the metadata we currently use to manage a chunk of memory, and a "large folio" is one that spans more than a single page -- huge page, large page, super page, ... in the past the metadata for that used to be a complicated piece of "compound page". In the future, we might call it differently (struct
file_mem?), who knows.So "large folio" support in a fs allows for the usage of these larger chunks of memory (huge pages).
I'm a bit confused here. I thought the term 'huge pages' referred to specific page table level like PMD, PUD, or PGD (and not PTE). And "large folio" term can span up to anything (currently up to PMD). Could you clarify if I'm misunderstanding something?
Quoted 66 lines · Daniel Gomez
Quoted 65 lines · David Hildenbrand
Quoted 49 lines · Luis Chamberlain
Quoted 39 lines · David Hildenbrand
Quoted 38 lines · Luis Chamberlain
Quoted 24 lines · Pankaj Raghav (Samsung)
From: Pankaj Raghav <[email protected]>
THP support for non-tmpfs filesystem has been around for some time now. Update the admin guide to reflect it.
While we are at it, move FilePmdMapped to previous paragraph for clarity, and clarify ShmemPmdMapped & ShmemHugePage.
Signed-off-by Pankaj Raghav <[email protected]>---
Changes since v2:
- Address comment from Bagas Sanjaya
- Squash commits and Ack from DavidDocumentation/admin-guide/mm/transhuge.rst | 22 +++++++++++++++------- 1 file changed, 15 insertions(+), 7 deletions(-)Documentation/admin-guide/mm/transhuge.rst +0
-0Documentation/admin-guide/mm/transhuge.rst using huge pages for the backing of virtual memory with huge pages · line 15
Quoted 6 lines
Documentation/admin-guide/mm/transhuge.rst +2
-2@@ -12,8 +12,8 @@ using huge pages for the backing of virtual memory with huge pages that supports the automatic promotion and demotion of page sizes and without the shortcomings of hugetlbfs. -Currently THP only works for anonymous memory mappings and tmpfs/shmem. -But in the future it can expand to other filesystems. +Currently, THP only works for anonymous memory mappings, tmpfs/shmem and +filesystems that support large folios.That seems to allude that THP can be supported on filesystems that suppor large folios. I don't think we want to call that THP and that can confuse folks. Leaving "currently" also seems to indicate that there is more work to be done for THP for filesystems but that's not true as well. So how about something like:
Quoted 8 lines
Quoted 3 lines
THP only works for anonymous memory mappings, and the tmpfs/shmem is the only filesystem to support it. The alternative to THP for other filesystems is to support large folios and with it you can end up using huge pages
That makes things more complicated without a good reason.
See
CONFIG_READ_ONLY_THP_FOR_FSas an early usage of the term "THP" for stuff we have in the pagecache.Quoted 6 lines
OK.
Quoted 2 lines
(with large folios we now properly implement this concept, and support more than only PMD size)
Do we really want to call large folio support on filesystems THP?
Good question.
"folio" is just the metadata we currently use to manage a chunk of memory, and a "large folio" is one that spans more than a single page -- huge page, large page, super page, ... in the past the metadata for that used to be a complicated piece of "compound page". In the future, we might call it differently (struct
file_mem?), who knows.
Quoted 7 lines
Quoted 2 lines
So "large folio" support in a fs allows for the usage of these larger chunks of memory (huge pages).
I'm a bit confused here. I thought the term 'huge pages' referred to specific page table level like PMD, PUD, or PGD (and not PTE). And "large folio" term can span up to anything (currently up to PMD). Could you clarify if I'm misunderstanding something?
The transhuge.rst is still a bit inconsistent and confusing, but we started generalizing the concept to special-case "Traditional THPs" to be PMD-sized THPs. See the description about mTHP in the document.
If a small folio represents a single page, then a large folio represents
multiple pages (large/huge/super page). But again, "folio" is the
metadata we use to manage this chunk of memory. Just like we didn't call a "huge page" a "compound page" back in the days; the "compound page" was a way to manage the metadata assigned to a "larger chunk of memory covering multiple base pages" as one unit.
I once had a longer writeup about all that (including how hugetlb supports different huge page sizes, how freebsd calls these thing super pages etc).
This topic seems to pop every 6 months, likely we really should do a better job at documenting all that.
Signature · 3 lines
Cheers, David / dhildenb
Quoted 28 lines · Luis Chamberlain
Quoted 27 lines · Pankaj Raghav (Samsung)
From: Pankaj Raghav <[email protected]>
THP support for non-tmpfs filesystem has been around for some time now. Update the admin guide to reflect it.
While we are at it, move FilePmdMapped to previous paragraph for clarity, and clarify ShmemPmdMapped & ShmemHugePage.
Signed-off-by Pankaj Raghav <[email protected]>---
Changes since v2:
- Address comment from Bagas Sanjaya
- Squash commits and Ack from DavidDocumentation/admin-guide/mm/transhuge.rst | 22 +++++++++++++++------- 1 file changed, 15 insertions(+), 7 deletions(-)Documentation/admin-guide/mm/transhuge.rst +0
-0@@ -12,8 +12,8 @@ using huge pages for the backing of virtual memory with huge pages that supports the automatic promotion and demotion of page sizes and without the shortcomings of hugetlbfs.
Quoted 7 lines
Documentation/admin-guide/mm/transhuge.rst using huge pages for the backing of virtual memory with huge pages · line 16
Quoted 4 lines
Documentation/admin-guide/mm/transhuge.rst +2
-2-Currently THP only works for anonymous memory mappings and tmpfs/shmem. -But in the future it can expand to other filesystems. +Currently, THP only works for anonymous memory mappings, tmpfs/shmem and +filesystems that support large folios.That seems to allude that THP can be supported on filesystems that suppor large folios. I don't think we want to call that THP
But we do allocate a THP in the page cache if we support large folios.
See [1] where THP was supported through page cache. From what I understand, THP support was added first to the page cache and then large folios (orders in between) support came later.
For example, you can mount XFS and do a simple dd as follows:
$ dd if=/dev/random of=/mnt/hello bs=1G count=2
And if you monitor FileHugePages after dd is done, it would have filled up if THP is enabled.
Quoted 3 lines
and that can confuse folks. Leaving "currently" also seems to indicate that there is more work to be done for THP for filesystems but that's not true as well. So how about something like:
There is more work to be done in the filesystems right? Adding large folio support is the prerequisite here to get the THP support.
Quoted 7 lines
THP only works for anonymous memory mappings, and the tmpfs/shmem is the only filesystem to support it. The alternative to THP for other filesystems is to support large folios and with it you can end up using huge pages opportunistically.
Luis
[1] https://lore.kernel.org/linux-mm/[email protected]/
--
Pankaj
Quoted 37 lines · Pankaj Raghav (Samsung)
Quoted 28 lines · Luis Chamberlain
Quoted 27 lines · Pankaj Raghav (Samsung)
From: Pankaj Raghav <[email protected]>
THP support for non-tmpfs filesystem has been around for some time now. Update the admin guide to reflect it.
While we are at it, move FilePmdMapped to previous paragraph for clarity, and clarify ShmemPmdMapped & ShmemHugePage.
Signed-off-by Pankaj Raghav <[email protected]>---
Changes since v2:
- Address comment from Bagas Sanjaya
- Squash commits and Ack from DavidDocumentation/admin-guide/mm/transhuge.rst | 22 +++++++++++++++------- 1 file changed, 15 insertions(+), 7 deletions(-)Documentation/admin-guide/mm/transhuge.rst +0
-0@@ -12,8 +12,8 @@ using huge pages for the backing of virtual memory with huge pages that supports the automatic promotion and demotion of page sizes and without the shortcomings of hugetlbfs.Quoted 7 lines
Documentation/admin-guide/mm/transhuge.rst using huge pages for the backing of virtual memory with huge pages · line 16
Quoted 4 lines
Documentation/admin-guide/mm/transhuge.rst +2
-2-Currently THP only works for anonymous memory mappings and tmpfs/shmem. -But in the future it can expand to other filesystems. +Currently, THP only works for anonymous memory mappings, tmpfs/shmem and +filesystems that support large folios.That seems to allude that THP can be supported on filesystems that suppor large folios. I don't think we want to call that THP
Quoted 5 lines
But we do allocate a THP in the page cache if we support large folios.
See [1] where THP was supported through page cache. From what I understand, THP support was added first to the page cache and then large folios (orders in between) support came later.
I see, yes Do we want to clarify this further?
Quoted 7 lines
For example, you can mount XFS and do a simple dd as follows:
$ dd if=/dev/random of=/mnt/hello bs=1G count=2And if you monitor
FileHugePagesafter dd is done, it would have filled up if THP is enabled.
Quoted 6 lines
Quoted 3 lines
and that can confuse folks. Leaving "currently" also seems to indicate that there is more work to be done for THP for filesystems but that's not true as well. So how about something like:
There is more work to be done in the filesystems right? Adding large folio support is the prerequisite here to get the THP support.
This would be good to clarify as well.
Luis
Quoted 38 lines · Luis Chamberlain
Quoted 37 lines · Pankaj Raghav (Samsung)
Quoted 35 lines · Luis Chamberlain
Quoted 24 lines · Pankaj Raghav (Samsung)
From: Pankaj Raghav <[email protected]>
THP support for non-tmpfs filesystem has been around for some time now. Update the admin guide to reflect it.
While we are at it, move FilePmdMapped to previous paragraph for clarity, and clarify ShmemPmdMapped & ShmemHugePage.
Signed-off-by Pankaj Raghav <[email protected]>---
Changes since v2:
- Address comment from Bagas Sanjaya
- Squash commits and Ack from DavidDocumentation/admin-guide/mm/transhuge.rst | 22 +++++++++++++++------- 1 file changed, 15 insertions(+), 7 deletions(-)Documentation/admin-guide/mm/transhuge.rst +0
-0Documentation/admin-guide/mm/transhuge.rst using huge pages for the backing of virtual memory with huge pages · line 16
Quoted 7 lines
Documentation/admin-guide/mm/transhuge.rst +2
-2@@ -12,8 +12,8 @@ using huge pages for the backing of virtual memory with huge pages that supports the automatic promotion and demotion of page sizes and without the shortcomings of hugetlbfs. -Currently THP only works for anonymous memory mappings and tmpfs/shmem. -But in the future it can expand to other filesystems. +Currently, THP only works for anonymous memory mappings, tmpfs/shmem and +filesystems that support large folios.That seems to allude that THP can be supported on filesystems that suppor large folios. I don't think we want to call that THP
Quoted 7 lines
Quoted 5 lines
But we do allocate a THP in the page cache if we support large folios.
See [1] where THP was supported through page cache. From what I understand, THP support was added first to the page cache and then large folios (orders in between) support came later.
I see, yes Do we want to clarify this further?
According to this thread [1], large folios currently depends on CONFIG_TRANSPARENT_HUGEPAGE, but that will be removed eventually.
https://lore.kernel.org/all/[email protected]/
I agree this needs to be clarified. THP for anonymous memory and tmpfs/shmem is not the same for filesystems that support large folios.
Quoted 11 lines · Pankaj Raghav (Samsung)
From: Pankaj Raghav <[email protected]>
THP support for non-tmpfs filesystem has been around for some time now. Update the admin guide to reflect it.
While we are at it, move FilePmdMapped to previous paragraph for clarity, and clarify ShmemPmdMapped & ShmemHugePage.
Signed-off-by Pankaj Raghav <[email protected]>---
BTW, we should go over the whole document and bring it up to date. Interestingly, it starts with
"Performance critical computing applications dealing with large memory working sets are already running on top of libhugetlbfs and in turn hugetlbfs. Transparent HugePage Support (THP) is an alternative mean of using huge pages for the backing of virtual memory with huge pages that supports the automatic promotion and demotion of page sizes and without the shortcomings of hugetlbfs.
Currently THP only works for anonymous memory mappings and tmpfs/shmem. But in the future it can expand to other filesystems."
Signature · 3 lines
Cheers, David / dhildenb
Quoted 23 lines · David Hildenbrand
Quoted 11 lines · Pankaj Raghav (Samsung)
From: Pankaj Raghav <[email protected]>
THP support for non-tmpfs filesystem has been around for some time now. Update the admin guide to reflect it.
While we are at it, move FilePmdMapped to previous paragraph for clarity, and clarify ShmemPmdMapped & ShmemHugePage.
Signed-off-by Pankaj Raghav <[email protected]>---
BTW, we should go over the whole document and bring it up to date. Interestingly, it starts with
"Performance critical computing applications dealing with large memory working sets are already running on top of libhugetlbfs and in turn hugetlbfs. Transparent HugePage Support (THP) is an alternative mean of using huge pages for the backing of virtual memory with huge pages that supports the automatic promotion and demotion of page sizes and without the shortcomings of hugetlbfs.
Quoted 2 lines
Currently THP only works for anonymous memory mappings and tmpfs/shmem. But in the future it can expand to other filesystems."
As a part of this patch, I do change 2nd paragraph:
-Currently THP only works for anonymous memory mappings and tmpfs/shmem. -But in the future it can expand to other filesystems. +Currently, THP only works for anonymous memory mappings, tmpfs/shmem and +filesystems that support large folios.
Do we need to add more changes to the first paragraph as well?
--
Pankaj
Quoted 31 lines · Pankaj Raghav (Samsung)
Documentation/admin-guide/mm/transhuge.rst +8
-7@@ -12,8 +12,8 @@ using huge pages for the backing of virtual memory with huge pages that supports the automatic promotion and demotion of page sizes and without the shortcomings of hugetlbfs. -Currently THP only works for anonymous memory mappings and tmpfs/shmem. -But in the future it can expand to other filesystems. +Currently, THP only works for anonymous memory mappings, tmpfs/shmem and +filesystems that support large folios. .. note:: in the examples below we presume that the basic page size is 4K and@@ -463,11 +463,19 @@ fields for each mapping. (Note that AnonHugePages only applies to traditional PMD-sized THP for historical reasons and should have been called AnonHugePmdMapped). -The number of file transparent huge pages mapped to userspace is available -by reading ShmemPmdMapped and ShmemHugePages fields in ``/proc/meminfo``. -To identify what applications are mapping file transparent huge pages, it -is necessary to read ``/proc/PID/smaps`` and count the FilePmdMapped fields -for each mapping. +The number of PMD-sized transparent huge pages currently used by +filesystem data (page cache) is available by reading the FileHugePages field +in ``/proc/meminfo``. The number of these huge pages that are mapped to userspace +is available by reading FilePmdMapped field in ``proc/meminfo``. To identify +what applications are mapping these huge pages, it is necessary to read +``/proc/PID/smaps`` and count the FilePmdMapped fields for each mapping.
Documentation/admin-guide/mm/transhuge.rst fields for each mapping. (Note that AnonHugePages only applies to traditional · line 481
Quoted 10 lines
Documentation/admin-guide/mm/transhuge.rst +7
-0+ +In similar fashion, the number of PMD-sized transparent huge pages currently +used by tmpfs/shmem is available by reading the ShmemHugePages field +in ``/proc/meminfo``. The number of these huge pages that are mapped to userspace +is available by reading ShmemPmdMapped field in ``proc/meminfo``. To identify +what applications are mapping these huge pages, it is necessary to read +``/proc/PID/smaps`` and count the ShmemPmdMapped fields for each mapping. Note that reading the smaps file is expensive and reading it frequently will incur overhead.
Looks good, thanks!
Signature · 1 line
An old man doll... just what I always wanted! - Clara