mm/page_idle: Add per-pid idle page tracking using virtual indexing
Subsystems (12)
PATCH v4 1/5 mm/page_idle: Add per-pid idle page tracking using virtual indexing
The page_idle tracking feature currently requires looking up the pagemap for a process followed by interacting with /sys/kernel/mm/page_idle. Looking up PFN from pagemap in Android devices is not supported by unprivileged process and requires SYS_ADMIN and gives 0 for the PFN.
This patch adds support to directly interact with page_idle tracking at the PID level by introducing a /proc/<pid>/page_idle file. It follows the exact same semantics as the global /sys/kernel/mm/page_idle, but now looking up PFN through pagemap is not needed since the interface uses virtual frame numbers, and at the same time also does not require SYS_ADMIN.
In Android, we are using this for the heap profiler (heapprofd) which profiles and pin points code paths which allocates and leaves memory idle for long periods of time. This method solves the security issue with userspace learning the PFN, and while at it is also shown to yield better results than the pagemap lookup, the theory being that the window where the address space can change is reduced by eliminating the intermediate pagemap look up stage. In virtual address indexing, the process's mmap_sem is held for the duration of the access.
---v3->v4: Minor fixups (Minchan)
Add swap pte handling (Konstantin, Minchan)v2->v3:
Fixed a bug where I was doing a kfree that is not needed due to not needing to do GFP_ATOMIC allocations.
v1->v2:
Mark swap ptes as idle (Minchan)
Avoid need for GFP_ATOMIC (Andrew)
Get rid of idle_page_list lock by moving list to stack
Internal review -> v1:
Fixes from Suren.
Corrections to change log, docs (Florian, Sandeep)
arch/Kconfig | 3 + fs/proc/base.c | 3 + fs/proc/internal.h | 1 + fs/proc/task_mmu.c | 43 ++++ include/asm-generic/pgtable.h | 6 + include/linux/page_idle.h | 4 + mm/page_idle.c | 359 +++++++++++++++++++++++++++++----- mm/rmap.c | 2 + 8 files changed, 376 insertions(+), 45 deletions(-)
arch/Kconfig +3 -0
@@ -575,6 +575,9 @@ config ARCH_WANT_HUGE_PMD_SHARE config HAVE_ARCH_SOFT_DIRTY bool +config HAVE_ARCH_PTE_SWP_PGIDLE + bool + config HAVE_MOD_ARCH_SPECIFIC bool help
fs/proc/base.c +3 -0
@@ -3039,6 +3039,9 @@ static const struct pid_entry tgid_base_stuff[] = {REG ("smaps" ,S_IRUGO , proc_pid_smaps_operations),REG ("smaps_rollup" ,S_IRUGO , proc_pid_smaps_rollup_operations),REG ("pagemap" ,S_IRUSR , proc_pagemap_operations), +#ifdef CONFIG_IDLE_PAGE_TRACKING +REG ("page_idle" ,S_IRUSR |S_IWUSR , proc_page_idle_operations), +#endif #endif #ifdef CONFIG_SECURITY DIR("attr", S_IRUGO|S_IXUGO, proc_attr_dir_inode_operations, proc_attr_dir_operations),
fs/proc/internal.h +1 -0
@@ -293,6 +293,7 @@ extern const struct file_operations proc_pid_smaps_operations;extern const struct file_operations proc_pid_smaps_rollup_operations;extern const struct file_operations proc_clear_refs_operations;extern const struct file_operations proc_pagemap_operations; +extern const struct file_operations proc_page_idle_operations;extern unsigned long task_vsize (struct mm_struct *); extern unsigned long task_statm(struct mm_struct *,
fs/proc/task_mmu.c +43 -0
49 lines not on this page · raw message
include/asm-generic/pgtable.h +6 -0
@@ -712,6 +712,12 @@ static inline void ptep_modify_prot_commit(struct vm_area_struct *vma,#define arch_start_context_switch (prev) do {} while (0)#endif +#ifndef CONFIG_HAVE_ARCH_PTE_SWP_PGIDLE +static inline pte_t pte_swp_mkpage_idle (pte_t pte) {return pte; } +static inline int pte_swp_page_idle (pte_t pte) {return 0 ; } +static inline pte_t pte_swp_clear_mkpage_idle (pte_t pte) {return pte; } +#endif +#ifdef CONFIG_HAVE_ARCH_SOFT_DIRTY #ifndef CONFIG_ARCH_ENABLE_THP_MIGRATION static inline pmd_t pmd_swp_mksoft_dirty (pmd_t pmd)
include/linux/page_idle.h +4 -0
@@ -106,6 +106,10 @@ static inline void clear_page_idle(struct page *page) }#endif /* CONFIG_64BIT */ +ssize_t page_idle_proc_write (struct file *file, +char __user *buf,size_t count,loff_t *ppos,struct task_struct *tsk); +ssize_t page_idle_proc_read (struct file *file, +char __user *buf,size_t count,loff_t *ppos,struct task_struct *tsk);#else /* !CONFIG_IDLE_PAGE_TRACKING */ static inline bool page_is_young (struct page *page)
mm/page_idle.c +314 -45
425 lines not on this page · raw message
mm/rmap.c +2 -0
@@ -1629,6 +1629,8 @@ static bool try_to_unmap_one(struct page *page, struct vm_area_struct *vma, swp_pte =swp_entry_to_pte (entry);if (pte_soft_dirty (pteval)) swp_pte =pte_swp_mksoft_dirty (swp_pte); +if (page_is_idle (page)) + swp_pte =pte_swp_mkpage_idle (swp_pte);set_pte_at (mm, address, pvmw.pte, swp_pte);/* Invalidate as we cleared the pte */ mmu_notifier_invalidate_range(mm, address,
Signature · 1 line
2.22.0.770.g0f2c4a37fd-goog
RFC PATCH v4 2/5 x86: Add support for idle bit in swap PTE
This bit will be used by idle page tracking code to correctly identify if a page that was swapped out was idle before it got swapped out. Without this PTE bit, we lose information about if a page is idle or not since the page frame gets unmapped and the page gets freed.
Bits 2-6 are unused in the swap PTE (see the comment in arch/x86/include/asm/pgtable_64.h). Bit 2 corresponds to _PAGE_USER. Use it for swap PTE purposes.
--- arch/x86/Kconfig | 1 + arch/x86/include/asm/pgtable.h | 15 +++++++++++++++ arch/x86/include/asm/pgtable_types.h | 6 ++++++ 3 files changed, 22 insertions(+)
arch/x86/Kconfig +1 -0
@@ -139,6 +139,7 @@ config X86 select HAVE_ARCH_MMAP_RND_COMPAT_BITS if MMU && COMPAT select HAVE_ARCH_COMPAT_MMAP_BASES if MMU && COMPAT select HAVE_ARCH_PREL32_RELOCATIONS + select HAVE_ARCH_PTE_SWP_PGIDLE select HAVE_ARCH_SECCOMP_FILTER select HAVE_ARCH_THREAD_STRUCT_WHITELIST select HAVE_ARCH_STACKLEAK
arch/x86/include/asm/pgtable.h +15 -0
@@ -1371,6 +1371,21 @@ static inline pmd_t pmd_swp_clear_soft_dirty(pmd_t pmd)#endif #endif +static inline pte_t pte_swp_mkpage_idle (pte_t pte) +{ +return pte_set_flags (pte, _PAGE_SWP_PGIDLE); +} + +static inline int pte_swp_page_idle (pte_t pte) +{ +return pte_flags (pte) & _PAGE_SWP_PGIDLE; +} + +static inline pte_t pte_swp_clear_mkpage_idle (pte_t pte) +{ +return pte_clear_flags (pte, _PAGE_SWP_PGIDLE); +} +#define PKRU_AD_BIT 0x1#define PKRU_WD_BIT 0x2#define PKRU_BITS_PER_PKEY 2
arch/x86/include/asm/pgtable_types.h +6 -0
@@ -100,6 +100,12 @@#define _PAGE_SWP_SOFT_DIRTY (_AT(pteval_t, 0))#endif +#ifdef CONFIG_IDLE_PAGE_TRACKING +#define _PAGE_SWP_PGIDLE _PAGE_USER +#else +#define _PAGE_SWP_PGIDLE (_AT(pteval_t, 0)) +#endif +#if defined(CONFIG_X86_64 ) || defined(CONFIG_X86_PAE )#define _PAGE_NX (_AT(pteval_t, 1) << _PAGE_BIT_NX)#define _PAGE_DEVMAP (_AT(u64, 1) << _PAGE_BIT_DEVMAP)
Signature · 1 line
2.22.0.770.g0f2c4a37fd-goog
RFC PATCH v4 3/5 arm64: Add support for idle bit in swap PTE
In reply to Joel Fernandes (Google)This bit will be used by idle page tracking code to correctly identify if a page that was swapped out was idle before it got swapped out. Without this PTE bit, we lose information about if a page is idle or not since the page frame gets unmapped.
In this patch we reuse PTE_DEVMAP bit since idle page tracking only works on user pages in the LRU. Device pages should not consitute those so it should be unused and safe to use.
--- arch/arm64/Kconfig | 1 + arch/arm64/include/asm/pgtable-prot.h | 1 + arch/arm64/include/asm/pgtable.h | 15 +++++++++++++++ 3 files changed, 17 insertions(+)
arch/arm64/Kconfig +1 -0
@@ -128,6 +128,7 @@ config ARM64 select HAVE_ARCH_MMAP_RND_BITS select HAVE_ARCH_MMAP_RND_COMPAT_BITS if COMPAT select HAVE_ARCH_PREL32_RELOCATIONS + select HAVE_ARCH_PTE_SWP_PGIDLE select HAVE_ARCH_SECCOMP_FILTER select HAVE_ARCH_STACKLEAK select HAVE_ARCH_THREAD_STRUCT_WHITELIST
arch/arm64/include/asm/pgtable-prot.h +1 -0
@@ -18,6 +18,7 @@#define PTE_SPECIAL (_AT(pteval_t, 1) << 56)#define PTE_DEVMAP (_AT(pteval_t, 1) << 57)#define PTE_PROT_NONE (_AT(pteval_t, 1) << 58)/* only when !PTE_VALID */ +#define PTE_SWP_PGIDLE PTE_DEVMAP/* for idle page tracking during swapout */ #ifndef __ASSEMBLY__
arch/arm64/include/asm/pgtable.h +15 -0
@@ -212,6 +212,21 @@ static inline pte_t pte_mkdevmap(pte_t pte)return set_pte_bit (pte,__pgprot (PTE_DEVMAP )); } +static inline int pte_swp_page_idle (pte_t pte) +{ +return 0 ; +} + +static inline pte_t pte_swp_mkpage_idle (pte_t pte) +{ +return set_pte_bit (pte,__pgprot (PTE_SWP_PGIDLE )); +} + +static inline pte_t pte_swp_clear_page_idle (pte_t pte) +{ +return clear_pte_bit (pte,__pgprot (PTE_SWP_PGIDLE )); +} +static inline void set_pte (pte_t *ptep,pte_t pte) {WRITE_ONCE (*ptep, pte);
Signature · 1 line
2.22.0.770.g0f2c4a37fd-goog
Re: [PATCH v4 3/5] [RFC] arm64: Add support for idle bit in swap PTE
Quoted 4 lines · Joel Fernandes (Google)
This bit will be used by idle page tracking code to correctly identify if a page that was swapped out was idle before it got swapped out. Without this PTE bit, we lose information about if a page is idle or not since the page frame gets unmapped.
And why do we need that? Why cannot we simply assume all swapped out pages to be idle? They were certainly idle enough to be reclaimed, right? Or what does idle actualy mean here?
Quoted 64 lines
In this patch we reuse
PTE_DEVMAPbit since idle page tracking only works on user pages in the LRU. Device pages should not consitute those so it should be unused and safe to use.Cc Robin Murphy <[email protected]>Signed-off-by Joel Fernandes (Google) <[email protected]>--- arch/arm64/Kconfig | 1 + arch/arm64/include/asm/pgtable-prot.h | 1 + arch/arm64/include/asm/pgtable.h | 15 +++++++++++++++ 3 files changed, 17 insertions(+)arch/arm64/Kconfig +1
-0@@ -128,6 +128,7 @@ config ARM64 select HAVE_ARCH_MMAP_RND_BITS select HAVE_ARCH_MMAP_RND_COMPAT_BITS if COMPAT select HAVE_ARCH_PREL32_RELOCATIONS + select HAVE_ARCH_PTE_SWP_PGIDLE select HAVE_ARCH_SECCOMP_FILTER select HAVE_ARCH_STACKLEAK select HAVE_ARCH_THREAD_STRUCT_WHITELISTarch/arm64/include/asm/pgtable-prot.h +1
-0@@ -18,6 +18,7 @@#define PTE_SPECIAL (_AT(pteval_t, 1) << 56)#define PTE_DEVMAP (_AT(pteval_t, 1) << 57)#define PTE_PROT_NONE (_AT(pteval_t, 1) << 58)/* only when !PTE_VALID */ +#define PTE_SWP_PGIDLE PTE_DEVMAP/* for idle page tracking during swapout */ #ifndef __ASSEMBLY__arch/arm64/include/asm/pgtable.h +15
-0@@ -212,6 +212,21 @@ static inline pte_t pte_mkdevmap(pte_t pte)return set_pte_bit (pte,__pgprot (PTE_DEVMAP )); } +static inline int pte_swp_page_idle (pte_t pte) +{ +return 0 ; +} + +static inline pte_t pte_swp_mkpage_idle (pte_t pte) +{ +return set_pte_bit (pte,__pgprot (PTE_SWP_PGIDLE )); +} + +static inline pte_t pte_swp_clear_page_idle (pte_t pte) +{ +return clear_pte_bit (pte,__pgprot (PTE_SWP_PGIDLE )); +} +static inline void set_pte (pte_t *ptep,pte_t pte) {WRITE_ONCE (*ptep, pte);Signature · 1 line
2.22.0.770.g0f2c4a37fd-goog
Signature · 2 lines
Michal Hocko SUSE Labs
Re: [PATCH v4 3/5] [RFC] arm64: Add support for idle bit in swap PTE
Quoted 9 lines · Michal Hocko
Quoted 4 lines · Joel Fernandes (Google)
This bit will be used by idle page tracking code to correctly identify if a page that was swapped out was idle before it got swapped out. Without this PTE bit, we lose information about if a page is idle or not since the page frame gets unmapped.
And why do we need that? Why cannot we simply assume all swapped out pages to be idle? They were certainly idle enough to be reclaimed, right? Or what does idle actualy mean here?
Yes, but other than swapping, in Android a page can be forced to be swapped out as well using the new hints that Minchan is adding?
Also, even if they were idle enough to be swapped, there is a chance that they were marked as idle and *accessed* before the swapping. Due to swapping, the "page was accessed since we last marked it as idle" information is lost. I am able to verify this.
Idle in this context means the same thing as in page idle tracking terms, the page was not accessed by userspace since we last marked it as idle (using /proc/<pid>/page_idle).
thanks,
- Joel
Quoted 68 lines
Quoted 64 lines
In this patch we reuse
PTE_DEVMAPbit since idle page tracking only works on user pages in the LRU. Device pages should not consitute those so it should be unused and safe to use.Cc Robin Murphy <[email protected]>Signed-off-by Joel Fernandes (Google) <[email protected]>--- arch/arm64/Kconfig | 1 + arch/arm64/include/asm/pgtable-prot.h | 1 + arch/arm64/include/asm/pgtable.h | 15 +++++++++++++++ 3 files changed, 17 insertions(+)arch/arm64/Kconfig +1
-0@@ -128,6 +128,7 @@ config ARM64 select HAVE_ARCH_MMAP_RND_BITS select HAVE_ARCH_MMAP_RND_COMPAT_BITS if COMPAT select HAVE_ARCH_PREL32_RELOCATIONS + select HAVE_ARCH_PTE_SWP_PGIDLE select HAVE_ARCH_SECCOMP_FILTER select HAVE_ARCH_STACKLEAK select HAVE_ARCH_THREAD_STRUCT_WHITELISTarch/arm64/include/asm/pgtable-prot.h +1
-0@@ -18,6 +18,7 @@#define PTE_SPECIAL (_AT(pteval_t, 1) << 56)#define PTE_DEVMAP (_AT(pteval_t, 1) << 57)#define PTE_PROT_NONE (_AT(pteval_t, 1) << 58)/* only when !PTE_VALID */ +#define PTE_SWP_PGIDLE PTE_DEVMAP/* for idle page tracking during swapout */ #ifndef __ASSEMBLY__arch/arm64/include/asm/pgtable.h +15
-0@@ -212,6 +212,21 @@ static inline pte_t pte_mkdevmap(pte_t pte)return set_pte_bit (pte,__pgprot (PTE_DEVMAP )); } +static inline int pte_swp_page_idle (pte_t pte) +{ +return 0 ; +} + +static inline pte_t pte_swp_mkpage_idle (pte_t pte) +{ +return set_pte_bit (pte,__pgprot (PTE_SWP_PGIDLE )); +} + +static inline pte_t pte_swp_clear_page_idle (pte_t pte) +{ +return clear_pte_bit (pte,__pgprot (PTE_SWP_PGIDLE )); +} +static inline void set_pte (pte_t *ptep,pte_t pte) {WRITE_ONCE (*ptep, pte);Signature · 1 line
2.22.0.770.g0f2c4a37fd-googSignature · 2 lines
Michal Hocko SUSE Labs
Re: [PATCH v4 3/5] [RFC] arm64: Add support for idle bit in swap PTE
Quoted 7 lines · Joel Fernandes
Quoted 6 lines · Michal Hocko
Quoted 4 lines · Joel Fernandes (Google)
This bit will be used by idle page tracking code to correctly identify if a page that was swapped out was idle before it got swapped out. Without this PTE bit, we lose information about if a page is idle or not since the page frame gets unmapped.
Quoted 6 lines
Quoted 3 lines
And why do we need that? Why cannot we simply assume all swapped out pages to be idle? They were certainly idle enough to be reclaimed, right? Or what does idle actualy mean here?
Yes, but other than swapping, in Android a page can be forced to be swapped out as well using the new hints that Minchan is adding?
Yes and that is effectivelly making them idle, no?
Quoted 8 lines
Also, even if they were idle enough to be swapped, there is a chance that they were marked as idle and *accessed* before the swapping. Due to swapping, the "page was accessed since we last marked it as idle" information is lost. I am able to verify this.
Idle in this context means the same thing as in page idle tracking terms, the page was not accessed by userspace since we last marked it as idle (using /proc/<pid>/page_idle).
Please describe a usecase and why that information might be useful.
Signature · 2 lines
Michal Hocko SUSE Labs
Re: [PATCH v4 3/5] [RFC] arm64: Add support for idle bit in swap PTE
Quoted 8 lines · Michal Hocko
Quoted 7 lines · Joel Fernandes
Quoted 6 lines · Michal Hocko
Quoted 4 lines · Joel Fernandes (Google)
This bit will be used by idle page tracking code to correctly identify if a page that was swapped out was idle before it got swapped out. Without this PTE bit, we lose information about if a page is idle or not since the page frame gets unmapped.
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
And why do we need that? Why cannot we simply assume all swapped out pages to be idle? They were certainly idle enough to be reclaimed, right? Or what does idle actualy mean here?
Yes, but other than swapping, in Android a page can be forced to be swapped out as well using the new hints that Minchan is adding?
Yes and that is effectivelly making them idle, no?
1. mark page-A idle which was present at that time.
2. run workload
3. page-A is touched several times
4. *sudden* memory pressure happen so finally page A is finally swapped out
5. now see the page A idle - but it's incorrect.
Re: [PATCH v4 3/5] [RFC] arm64: Add support for idle bit in swap PTE
Quoted 16 lines · Minchan Kim
Quoted 15 lines · Michal Hocko
Quoted 13 lines · Joel Fernandes
Quoted 9 lines · Michal Hocko
Quoted 4 lines · Joel Fernandes (Google)
This bit will be used by idle page tracking code to correctly identify if a page that was swapped out was idle before it got swapped out. Without this PTE bit, we lose information about if a page is idle or not since the page frame gets unmapped.
And why do we need that? Why cannot we simply assume all swapped out pages to be idle? They were certainly idle enough to be reclaimed, right? Or what does idle actualy mean here?
Yes, but other than swapping, in Android a page can be forced to be swapped out as well using the new hints that Minchan is adding?
Quoted 7 lines
Quoted 1 line
Yes and that is effectivelly making them idle, no?
1. mark page-A idle which was present at that time.
2. run workload
3. page-A is touched several times
4. *sudden* memory pressure happen so finally page A is finally swapped out
5. now see the page A idle - but it's incorrect.
Could you expand on what you mean by idle exactly? Why pageout doesn't really qualify as "mark-idle and reclaim"? Also could you describe a usecase where the swapout distinction really matters and it would lead to incorrect behavior?
Signature · 2 lines
Michal Hocko SUSE Labs
Re: [PATCH v4 3/5] [RFC] arm64: Add support for idle bit in swap PTE
Quoted 25 lines · Michal Hocko
Quoted 16 lines · Minchan Kim
Quoted 15 lines · Michal Hocko
Quoted 13 lines · Joel Fernandes
Quoted 9 lines · Michal Hocko
Quoted 4 lines · Joel Fernandes (Google)
This bit will be used by idle page tracking code to correctly identify if a page that was swapped out was idle before it got swapped out. Without this PTE bit, we lose information about if a page is idle or not since the page frame gets unmapped.
And why do we need that? Why cannot we simply assume all swapped out pages to be idle? They were certainly idle enough to be reclaimed, right? Or what does idle actualy mean here?
Yes, but other than swapping, in Android a page can be forced to be swapped out as well using the new hints that Minchan is adding?
Quoted 7 lines
Quoted 1 line
Yes and that is effectivelly making them idle, no?
1. mark page-A idle which was present at that time.
2. run workload
3. page-A is touched several times
4. *sudden* memory pressure happen so finally page A is finally swapped out
5. now see the page A idle - but it's incorrect.
Quoted 4 lines
Could you expand on what you mean by idle exactly? Why pageout doesn't really qualify as "mark-idle and reclaim"? Also could you describe a usecase where the swapout distinction really matters and it would lead to incorrect behavior?
Michal,
Did you read this post ? :
https://lore.kernel.org/lkml/[email protected]/T/#m4ece68ceaf6e54d4d29e974f5f4c1080e733f6c1
Just wanted to be sure you did not miss it.
thanks,
- Joel
Re: [PATCH v4 3/5] [RFC] arm64: Add support for idle bit in swap PTE
In reply to Michal HockoQuoted 8 lines · Michal Hocko
Quoted 7 lines · Joel Fernandes
Quoted 6 lines · Michal Hocko
Quoted 4 lines · Joel Fernandes (Google)
This bit will be used by idle page tracking code to correctly identify if a page that was swapped out was idle before it got swapped out. Without this PTE bit, we lose information about if a page is idle or not since the page frame gets unmapped.
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
And why do we need that? Why cannot we simply assume all swapped out pages to be idle? They were certainly idle enough to be reclaimed, right? Or what does idle actualy mean here?
Yes, but other than swapping, in Android a page can be forced to be swapped out as well using the new hints that Minchan is adding?
Yes and that is effectivelly making them idle, no?
That depends on how you think of it. If you are thinking of a monitoring process like a heap profiler, then from the heap profiler's (that only cares about the process it is monitoring) perspective it will look extremely odd if pages that are recently accessed by the process appear to be idle which would falsely look like those processes are leaking memory. The reality being, Android forced those pages into swap because of other reasons. I would like for the swapping mechanism, whether forced swapping or memory reclaim, not to interfere with the idle detection.
This is just an effort to make the idle tracking a little bit better. We would like to not lose the 'accessed' information of the pages.
Initially, I had proposed what you are suggesting as well however the above reasons made me to do it like this. Also Minchan and Konstantin suggested this, so there are more people interested in the swap idle bit. Minchan, can you provide more thoughts here? (He is on 2-week vacation from today so hopefully replies before he vanishes ;-)).
Also assuming all swap pages as idle has other "semantic" issues. It is quite odd if a swapped page is automatically marked as idle without userspace telling it to. Consider the following set of events: 1. Userspace marks only a certain memory region as idle. 2. Userspace reads back the bits corresponding to a bigger region. Part of this bigger region is swapped. Userspace expects all of the pages it did not mark, to have idle bit set to '0' because it never marked them as idle. However if it is now surprised by what it read back (not all '0' read back). Since a page is swapped, it will be now marked "automatically" as idle as per your proposal, even if userspace never marked it explicity before. This would be quite confusing/ambiguous.
I will include this and other information in future commit messages.
thanks,
- Joel
Re: [PATCH v4 3/5] [RFC] arm64: Add support for idle bit in swap PTE
Quoted 13 lines · Joel Fernandes
Quoted 12 lines · Michal Hocko
Quoted 11 lines · Joel Fernandes
Quoted 9 lines · Michal Hocko
Quoted 4 lines · Joel Fernandes (Google)
This bit will be used by idle page tracking code to correctly identify if a page that was swapped out was idle before it got swapped out. Without this PTE bit, we lose information about if a page is idle or not since the page frame gets unmapped.
And why do we need that? Why cannot we simply assume all swapped out pages to be idle? They were certainly idle enough to be reclaimed, right? Or what does idle actualy mean here?
Quoted 6 lines
Quoted 4 lines
Quoted 2 lines
Yes, but other than swapping, in Android a page can be forced to be swapped out as well using the new hints that Minchan is adding?
Yes and that is effectivelly making them idle, no?
That depends on how you think of it.
I would much prefer to have it documented so that I do not have to guess ;)
Quoted 8 lines
If you are thinking of a monitoring
process like a heap profiler, then from the heap profiler's (that only cares about the process it is monitoring) perspective it will look extremely odd if pages that are recently accessed by the process appear to be idle which would falsely look like those processes are leaking memory. The reality being, Android forced those pages into swap because of other reasons. I would like for the swapping mechanism, whether forced swapping or memory reclaim, not to interfere with the idle detection.
Hmm, but how are you going to handle situation when the page is unmapped and refaulted again (e.g. a normal reclaim of a pagecache)? You are losing that information same was as in the swapout case, no? Or am I missing something?
Quoted 8 lines
This is just an effort to make the idle tracking a little bit better. We would like to not lose the 'accessed' information of the pages.
Initially, I had proposed what you are suggesting as well however the above reasons made me to do it like this. Also Minchan and Konstantin suggested this, so there are more people interested in the swap idle bit. Minchan, can you provide more thoughts here? (He is on 2-week vacation from today so hopefully replies before he vanishes ;-)).
We can move on with the rest of the series in the mean time but I would like to see a proper justification for the swap entries and why they should be handled special.
Quoted 10 lines
Also assuming all swap pages as idle has other "semantic" issues. It is quite odd if a swapped page is automatically marked as idle without userspace telling it to. Consider the following set of events: 1. Userspace marks only a certain memory region as idle. 2. Userspace reads back the bits corresponding to a bigger region. Part of this bigger region is swapped. Userspace expects all of the pages it did not mark, to have idle bit set to '0' because it never marked them as idle. However if it is now surprised by what it read back (not all '0' read back). Since a page is swapped, it will be now marked "automatically" as idle as per your proposal, even if userspace never marked it explicity before. This would be quite confusing/ambiguous.
OK, I see. I guess the primary question I have is how do you distinguish Idle page which got unmapped and faulted in again from swapped out page and refaulted - including the time the pte is not present.
Signature · 2 lines
Michal Hocko SUSE Labs
Re: [PATCH v4 3/5] [RFC] arm64: Add support for idle bit in swap PTE
Quoted 14 lines · Michal Hocko
Quoted 13 lines · Joel Fernandes
Quoted 12 lines · Michal Hocko
Quoted 11 lines · Joel Fernandes
Quoted 9 lines · Michal Hocko
Quoted 4 lines · Joel Fernandes (Google)
This bit will be used by idle page tracking code to correctly identify if a page that was swapped out was idle before it got swapped out. Without this PTE bit, we lose information about if a page is idle or not since the page frame gets unmapped.
And why do we need that? Why cannot we simply assume all swapped out pages to be idle? They were certainly idle enough to be reclaimed, right? Or what does idle actualy mean here?
Quoted 8 lines
Quoted 6 lines
Quoted 4 lines
Quoted 2 lines
Yes, but other than swapping, in Android a page can be forced to be swapped out as well using the new hints that Minchan is adding?
Yes and that is effectivelly making them idle, no?
That depends on how you think of it.
I would much prefer to have it documented so that I do not have to guess ;)
Sure :)
Quoted 9 lines
Quoted 8 lines
If you are thinking of a monitoring
process like a heap profiler, then from the heap profiler's (that only cares about the process it is monitoring) perspective it will look extremely odd if pages that are recently accessed by the process appear to be idle which would falsely look like those processes are leaking memory. The reality being, Android forced those pages into swap because of other reasons. I would like for the swapping mechanism, whether forced swapping or memory reclaim, not to interfere with the idle detection.
Quoted 4 lines
Hmm, but how are you going to handle situation when the page is unmapped and refaulted again (e.g. a normal reclaim of a pagecache)? You are losing that information same was as in the swapout case, no? Or am I missing something?
Yes you are right, it would have the same issue, thanks for bringing it up. Should we rename this bit to PTE_IDLE and do the same thing that we are doing for swap?
i.e. if (page_idle(page)) and page is a file page, then we write state into the PTE of the page. Later on refault, the PTE bit would automatically get cleared (just like it does on swap-in). But before refault, the idle tracking code sees the page as still marked idle. Do you see any issue with that?
Quoted 9 lines
Quoted 8 lines
This is just an effort to make the idle tracking a little bit better. We would like to not lose the 'accessed' information of the pages.
Initially, I had proposed what you are suggesting as well however the above reasons made me to do it like this. Also Minchan and Konstantin suggested this, so there are more people interested in the swap idle bit. Minchan, can you provide more thoughts here? (He is on 2-week vacation from today so hopefully replies before he vanishes ;-)).
Quoted 3 lines
We can move on with the rest of the series in the mean time but I would like to see a proper justification for the swap entries and why they should be handled special.
Ok, I will improve the changelog.
Quoted 11 lines
Quoted 10 lines
Also assuming all swap pages as idle has other "semantic" issues. It is quite odd if a swapped page is automatically marked as idle without userspace telling it to. Consider the following set of events: 1. Userspace marks only a certain memory region as idle. 2. Userspace reads back the bits corresponding to a bigger region. Part of this bigger region is swapped. Userspace expects all of the pages it did not mark, to have idle bit set to '0' because it never marked them as idle. However if it is now surprised by what it read back (not all '0' read back). Since a page is swapped, it will be now marked "automatically" as idle as per your proposal, even if userspace never marked it explicity before. This would be quite confusing/ambiguous.
Quoted 3 lines
OK, I see. I guess the primary question I have is how do you distinguish Idle page which got unmapped and faulted in again from swapped out page and refaulted - including the time the pte is not present.
Ok, lets discuss more.
thanks Michal!
- Joel
Re: [PATCH v4 3/5] [RFC] arm64: Add support for idle bit in swap PTE
Quoted 35 lines · Joel Fernandes
Quoted 14 lines · Michal Hocko
Quoted 13 lines · Joel Fernandes
Quoted 12 lines · Michal Hocko
Quoted 11 lines · Joel Fernandes
Quoted 9 lines · Michal Hocko
Quoted 4 lines · Joel Fernandes (Google)
This bit will be used by idle page tracking code to correctly identify if a page that was swapped out was idle before it got swapped out. Without this PTE bit, we lose information about if a page is idle or not since the page frame gets unmapped.
And why do we need that? Why cannot we simply assume all swapped out pages to be idle? They were certainly idle enough to be reclaimed, right? Or what does idle actualy mean here?
Quoted 8 lines
Quoted 6 lines
Quoted 4 lines
Quoted 2 lines
Yes, but other than swapping, in Android a page can be forced to be swapped out as well using the new hints that Minchan is adding?
Yes and that is effectivelly making them idle, no?
That depends on how you think of it.
I would much prefer to have it documented so that I do not have to guess ;)
Sure :)
Quoted 9 lines
Quoted 8 lines
If you are thinking of a monitoring
process like a heap profiler, then from the heap profiler's (that only cares about the process it is monitoring) perspective it will look extremely odd if pages that are recently accessed by the process appear to be idle which would falsely look like those processes are leaking memory. The reality being, Android forced those pages into swap because of other reasons. I would like for the swapping mechanism, whether forced swapping or memory reclaim, not to interfere with the idle detection.
Quoted 8 lines
Quoted 4 lines
Hmm, but how are you going to handle situation when the page is unmapped and refaulted again (e.g. a normal reclaim of a pagecache)? You are losing that information same was as in the swapout case, no? Or am I missing something?
Yes you are right, it would have the same issue, thanks for bringing it up. Should we rename this bit to
PTE_IDLEand do the same thing that we are doing for swap?
What if we decide to tear the page table down as well? E.g. because we can reclaim file backed mappings and free some memory used for page tables. We do not do that right now but I can see that really large mappings might push us that direction. Sure this is mostly a theoretical concern but I am wondering whether promissing to keep the idle bit over unmapping is not too much.
I am not sure how to deal with this myself, TBH. In any case the current semantic - via pfn - will lose the idle bit already so can we mimic it as well? We only have 1 bit for each address which makes it challenging. The easiest way would be to declare that the idle bit might disappear on activating or reclaiming the page. How well that suits different usecases is a different question. I would be interested in hearing from other people about this of course.
Signature · 2 lines
Michal Hocko SUSE Labs
Re: [PATCH v4 3/5] [RFC] arm64: Add support for idle bit in swap PTE
In reply to Michal HockoQuoted 32 lines · Michal Hocko
Quoted 13 lines · Joel Fernandes
Quoted 12 lines · Michal Hocko
Quoted 11 lines · Joel Fernandes
Quoted 9 lines · Michal Hocko
Quoted 4 lines · Joel Fernandes (Google)
This bit will be used by idle page tracking code to correctly identify if a page that was swapped out was idle before it got swapped out. Without this PTE bit, we lose information about if a page is idle or not since the page frame gets unmapped.
And why do we need that? Why cannot we simply assume all swapped out pages to be idle? They were certainly idle enough to be reclaimed, right? Or what does idle actualy mean here?
Quoted 6 lines
Quoted 4 lines
Quoted 2 lines
Yes, but other than swapping, in Android a page can be forced to be swapped out as well using the new hints that Minchan is adding?
Yes and that is effectivelly making them idle, no?
That depends on how you think of it.
I would much prefer to have it documented so that I do not have to guess ;)
Quoted 8 lines
If you are thinking of a monitoring
process like a heap profiler, then from the heap profiler's (that only cares about the process it is monitoring) perspective it will look extremely odd if pages that are recently accessed by the process appear to be idle which would falsely look like those processes are leaking memory. The reality being, Android forced those pages into swap because of other reasons. I would like for the swapping mechanism, whether forced swapping or memory reclaim, not to interfere with the idle detection.
Quoted 4 lines
Hmm, but how are you going to handle situation when the page is unmapped and refaulted again (e.g. a normal reclaim of a pagecache)? You are losing that information same was as in the swapout case, no? Or am I missing something?
If page is unmapped, it's not a idle memory any longer because it's free memory. We could detect the pte is not present.
If page is refaulted, it's not a idle memory any longer because it's accessed again. We could detect it because the newly allocated page doesn't have a PG_idle page flag.
Both case, idle page tracking couldn't report them as IDLE so it's okay.
Re: [PATCH v4 3/5] [RFC] arm64: Add support for idle bit in swap PTE
Quoted 33 lines · Minchan Kim
Quoted 32 lines · Michal Hocko
Quoted 13 lines · Joel Fernandes
Quoted 12 lines · Michal Hocko
Quoted 11 lines · Joel Fernandes
Quoted 9 lines · Michal Hocko
Quoted 4 lines · Joel Fernandes (Google)
This bit will be used by idle page tracking code to correctly identify if a page that was swapped out was idle before it got swapped out. Without this PTE bit, we lose information about if a page is idle or not since the page frame gets unmapped.
And why do we need that? Why cannot we simply assume all swapped out pages to be idle? They were certainly idle enough to be reclaimed, right? Or what does idle actualy mean here?
Quoted 6 lines
Quoted 4 lines
Quoted 2 lines
Yes, but other than swapping, in Android a page can be forced to be swapped out as well using the new hints that Minchan is adding?
Yes and that is effectivelly making them idle, no?
That depends on how you think of it.
I would much prefer to have it documented so that I do not have to guess ;)
Quoted 8 lines
If you are thinking of a monitoring
process like a heap profiler, then from the heap profiler's (that only cares about the process it is monitoring) perspective it will look extremely odd if pages that are recently accessed by the process appear to be idle which would falsely look like those processes are leaking memory. The reality being, Android forced those pages into swap because of other reasons. I would like for the swapping mechanism, whether forced swapping or memory reclaim, not to interfere with the idle detection.
Quoted 7 lines
Quoted 4 lines
Hmm, but how are you going to handle situation when the page is unmapped and refaulted again (e.g. a normal reclaim of a pagecache)? You are losing that information same was as in the swapout case, no? Or am I missing something?
If page is unmapped, it's not a idle memory any longer because it's free memory. We could detect the pte is not present.
I think Michal is not talking of explictly being unmapped, but about the case where a file-backed mapped page is unmapped due to memory pressure ? This is similar to the swap situation.
Basically... file page is marked idle, then it is accessed by userspace. Then memory pressure drops it off the page cache so the idle information is lost. Next time we check the page_idle, we miss that it was accessed indeed.
It is not an issue for the heap profiler or anonymous memory per-se. But is similar to the swap situation.
Quoted 3 lines
If page is refaulted, it's not a idle memory any longer because it's accessed again. We could detect it because the newly allocated page doesn't have a
PG_idlepage flag.
In the refault case, yes it should not be a problem.
thanks,
- Joel
PATCH v4 4/5 page_idle: Drain all LRU pagevec before idle tracking
In reply to Joel Fernandes (Google)During idle tracking, we see that sometimes faulted anon pages are in pagevec but are not drained to LRU. Idle tracking considers pages only on LRU. Drain all CPU's LRU before starting idle tracking.
--- mm/page_idle.c | 6 ++++++ 1 file changed, 6 insertions(+)
mm/page_idle.c +6 -0
@@ -180,6 +180,8 @@ static ssize_t page_idle_bitmap_read(struct file *file, struct kobject *kobj,unsigned long pfn, end_pfn;int bit, ret; +lru_add_drain_all (); + ret =page_idle_get_frames (pos, count,NULL , &pfn, &end_pfn);if (ret == -ENXIO )return 0 ;/* Reads beyond max_pfn do nothing */
@@ -211,6 +213,8 @@ static ssize_t page_idle_bitmap_write(struct file *file, struct kobject *kobj,unsigned long pfn, end_pfn;int bit, ret; +lru_add_drain_all (); + ret =page_idle_get_frames (pos, count,NULL , &pfn, &end_pfn);if (ret)return ret;
@@ -428,6 +432,8 @@ ssize_t page_idle_proc_generic(struct file *file, char __user *ubuff, walk.private = &priv; walk.mm = mm; +lru_add_drain_all (); +down_read (&mm->mmap_sem);/*
Signature · 1 line
2.22.0.770.g0f2c4a37fd-goog
Re: [PATCH v4 4/5] page_idle: Drain all LRU pagevec before idle tracking
Quoted 3 lines · Joel Fernandes (Google)
During idle tracking, we see that sometimes faulted anon pages are in pagevec but are not drained to LRU. Idle tracking considers pages only on LRU. Drain all CPU's LRU before starting idle tracking.
Please expand on why does this matter enough to introduce a potentially expensinve draining which has to schedule a work on each CPU and wait for them to finish.
Quoted 38 lines
Signed-off-by Joel Fernandes (Google) <[email protected]>--- mm/page_idle.c | 6 ++++++ 1 file changed, 6 insertions(+)mm/page_idle.c +6
-0@@ -180,6 +180,8 @@ static ssize_t page_idle_bitmap_read(struct file *file, struct kobject *kobj,unsigned long pfn, end_pfn;int bit, ret; +lru_add_drain_all (); + ret =page_idle_get_frames (pos, count,NULL , &pfn, &end_pfn);if (ret == -ENXIO )return 0 ;/* Reads beyond max_pfn do nothing */ @@ -211,6 +213,8 @@ static ssize_t page_idle_bitmap_write(struct file *file, struct kobject *kobj,unsigned long pfn, end_pfn;int bit, ret; +lru_add_drain_all (); + ret =page_idle_get_frames (pos, count,NULL , &pfn, &end_pfn);if (ret)return ret;@@ -428,6 +432,8 @@ ssize_t page_idle_proc_generic(struct file *file, char __user *ubuff, walk.private = &priv; walk.mm = mm; +lru_add_drain_all (); +down_read (&mm->mmap_sem);/* Signature · 1 line
2.22.0.770.g0f2c4a37fd-goog
Signature · 2 lines
Michal Hocko SUSE Labs
Re: [PATCH v4 4/5] page_idle: Drain all LRU pagevec before idle tracking
Quoted 8 lines · Michal Hocko
Quoted 3 lines · Joel Fernandes (Google)
During idle tracking, we see that sometimes faulted anon pages are in pagevec but are not drained to LRU. Idle tracking considers pages only on LRU. Drain all CPU's LRU before starting idle tracking.
Please expand on why does this matter enough to introduce a potentially expensinve draining which has to schedule a work on each CPU and wait for them to finish.
Sure, I can expand. I am able to find multiple issues involving this. One issue looks like idle tracking is completely broken. It shows up in my testing as if a page that is marked as idle is always "accessed" -- because it was never marked as idle (due to not draining of pagevec).
The other issue shows up as a failure in my "swap test", with the following sequence:
1. Allocate some pages
2. Write to them
3. Mark them as idle <--- fails
4. Introduce some memory pressure to induce swapping.
5. Check the swap bit I introduced in this series. <--- fails to set idle
bit in swap PTE.
Draining the pagevec in advance fixes both of these issues.
This operation even if expensive is only done once during the access of the page_idle file. Did you have a better fix in mind?
thanks,
- Joel
Quoted 42 lines
Quoted 38 lines
Signed-off-by Joel Fernandes (Google) <[email protected]>--- mm/page_idle.c | 6 ++++++ 1 file changed, 6 insertions(+)mm/page_idle.c +6
-0@@ -180,6 +180,8 @@ static ssize_t page_idle_bitmap_read(struct file *file, struct kobject *kobj,unsigned long pfn, end_pfn;int bit, ret; +lru_add_drain_all (); + ret =page_idle_get_frames (pos, count,NULL , &pfn, &end_pfn);if (ret == -ENXIO )return 0 ;/* Reads beyond max_pfn do nothing */ @@ -211,6 +213,8 @@ static ssize_t page_idle_bitmap_write(struct file *file, struct kobject *kobj,unsigned long pfn, end_pfn;int bit, ret; +lru_add_drain_all (); + ret =page_idle_get_frames (pos, count,NULL , &pfn, &end_pfn);if (ret)return ret;@@ -428,6 +432,8 @@ ssize_t page_idle_proc_generic(struct file *file, char __user *ubuff, walk.private = &priv; walk.mm = mm; +lru_add_drain_all (); +down_read (&mm->mmap_sem);/* Signature · 1 line
2.22.0.770.g0f2c4a37fd-googSignature · 2 lines
Michal Hocko SUSE Labs
Re: [PATCH v4 4/5] page_idle: Drain all LRU pagevec before idle tracking
Quoted 24 lines · Joel Fernandes
Quoted 8 lines · Michal Hocko
Quoted 3 lines · Joel Fernandes (Google)
During idle tracking, we see that sometimes faulted anon pages are in pagevec but are not drained to LRU. Idle tracking considers pages only on LRU. Drain all CPU's LRU before starting idle tracking.
Please expand on why does this matter enough to introduce a potentially expensinve draining which has to schedule a work on each CPU and wait for them to finish.
Sure, I can expand. I am able to find multiple issues involving this. One issue looks like idle tracking is completely broken. It shows up in my testing as if a page that is marked as idle is always "accessed" -- because it was never marked as idle (due to not draining of pagevec).
The other issue shows up as a failure in my "swap test", with the following sequence:
1. Allocate some pages
2. Write to them
3. Mark them as idle <--- fails
4. Introduce some memory pressure to induce swapping.
5. Check the swap bit I introduced in this series. <--- fails to set idle
bit in swap PTE.
Quoted 1 line
Draining the pagevec in advance fixes both of these issues.
This belongs to the changelog.
Quoted 2 lines
This operation even if expensive is only done once during the access of the
page_idlefile. Did you have a better fix in mind?
Can we set the idle bit also for non-lru pages as long as they are reachable via pte?
Signature · 2 lines
Michal Hocko SUSE Labs
Re: [PATCH v4 4/5] page_idle: Drain all LRU pagevec before idle tracking
Quoted 25 lines · Michal Hocko
Quoted 24 lines · Joel Fernandes
Quoted 8 lines · Michal Hocko
Quoted 3 lines · Joel Fernandes (Google)
During idle tracking, we see that sometimes faulted anon pages are in pagevec but are not drained to LRU. Idle tracking considers pages only on LRU. Drain all CPU's LRU before starting idle tracking.
Please expand on why does this matter enough to introduce a potentially expensinve draining which has to schedule a work on each CPU and wait for them to finish.
Sure, I can expand. I am able to find multiple issues involving this. One issue looks like idle tracking is completely broken. It shows up in my testing as if a page that is marked as idle is always "accessed" -- because it was never marked as idle (due to not draining of pagevec).
The other issue shows up as a failure in my "swap test", with the following sequence:
1. Allocate some pages
2. Write to them
3. Mark them as idle <--- fails
4. Introduce some memory pressure to induce swapping.
5. Check the swap bit I introduced in this series. <--- fails to set idle
bit in swap PTE.
Quoted 3 lines
Quoted 1 line
Draining the pagevec in advance fixes both of these issues.
This belongs to the changelog.
Sure, will add.
Quoted 5 lines
Quoted 2 lines
This operation even if expensive is only done once during the access of the
page_idlefile. Did you have a better fix in mind?Can we set the idle bit also for non-lru pages as long as they are reachable via pte?
Not at the moment with the current page idle tracking code. PageLRU(page) flag is checked in page_idle_get_page().
Even if we could set it for non-LRU, the idle bit (page flag) would not be cleared if page is not on LRU because page-reclaim code (page_referenced() I believe) would not clear it. This whole mechanism depends on page-reclaim. Or did I miss your point?
thanks,
- Joel
Re: [PATCH v4 4/5] page_idle: Drain all LRU pagevec before idle tracking
Quoted 33 lines · Joel Fernandes
Quoted 25 lines · Michal Hocko
Quoted 24 lines · Joel Fernandes
Quoted 8 lines · Michal Hocko
Quoted 3 lines · Joel Fernandes (Google)
During idle tracking, we see that sometimes faulted anon pages are in pagevec but are not drained to LRU. Idle tracking considers pages only on LRU. Drain all CPU's LRU before starting idle tracking.
Please expand on why does this matter enough to introduce a potentially expensinve draining which has to schedule a work on each CPU and wait for them to finish.
Sure, I can expand. I am able to find multiple issues involving this. One issue looks like idle tracking is completely broken. It shows up in my testing as if a page that is marked as idle is always "accessed" -- because it was never marked as idle (due to not draining of pagevec).
The other issue shows up as a failure in my "swap test", with the following sequence:
1. Allocate some pages
2. Write to them
3. Mark them as idle <--- fails
4. Introduce some memory pressure to induce swapping.
5. Check the swap bit I introduced in this series. <--- fails to set idle
bit in swap PTE.Quoted 3 lines
Quoted 1 line
Draining the pagevec in advance fixes both of these issues.
This belongs to the changelog.
Sure, will add.
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
This operation even if expensive is only done once during the access of the
page_idlefile. Did you have a better fix in mind?Can we set the idle bit also for non-lru pages as long as they are reachable via pte?
Not at the moment with the current page idle tracking code. PageLRU(page) flag is checked in
page_idle_get_page().
yes, I am aware of the current code. I strongly suspect that the PageLRU check was there to not mark arbitrary page looked up by pfn with the idle bit because that would be unexpected. But I might be easily wrong here.
Quoted 3 lines
Even if we could set it for non-LRU, the idle bit (page flag) would not be cleared if page is not on LRU because page-reclaim code (
page_referenced()I believe) would not clear it.
Yes, it is either reclaim when checking references as you say but also mark_page_accessed. I believe the later might still have the page on the pcp LRU add cache. Maybe I am missing something something but it seems that there is nothing fundamentally requiring the user mapped page to be on the LRU list when seting the idle bit.
That being said, your big hammer approach will work more reliable but if you do not feel like changing the underlying PageLRU assumption then document that draining should be removed longterm.
Signature · 2 lines
Michal Hocko SUSE Labs
Re: [PATCH v4 4/5] page_idle: Drain all LRU pagevec before idle tracking
On Tue, Aug 06, 2019 at 01:44:02PM +0200, Michal Hocko wrote:
[snip]
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
Quoted 2 lines
This operation even if expensive is only done once during the access of the
page_idlefile. Did you have a better fix in mind?Can we set the idle bit also for non-lru pages as long as they are reachable via pte?
Quoted 7 lines
Quoted 2 lines
Not at the moment with the current page idle tracking code. PageLRU(page) flag is checked in
page_idle_get_page().yes, I am aware of the current code. I strongly suspect that the PageLRU check was there to not mark arbitrary page looked up by pfn with the idle bit because that would be unexpected. But I might be easily wrong here.
Yes, quite possible.
Quoted 10 lines
Quoted 3 lines
Even if we could set it for non-LRU, the idle bit (page flag) would not be cleared if page is not on LRU because page-reclaim code (
page_referenced()I believe) would not clear it.Yes, it is either reclaim when checking references as you say but also
mark_page_accessed. I believe the later might still have the page on the pcp LRU add cache. Maybe I am missing something something but it seems that there is nothing fundamentally requiring the user mapped page to be on the LRU list when seting the idle bit.
Quoted 3 lines
That being said, your big hammer approach will work more reliable but if you do not feel like changing the underlying PageLRU assumption then document that draining should be removed longterm.
Yes, at the moment I am in preference of keeping the underlying assumption same. I am Ok with adding of a comment on the drain call that it is to be removed longterm.
thanks,
- Joel
PATCH v4 5/5 doc: Update documentation for page_idle virtual address indexing
In reply to Joel Fernandes (Google)This patch updates the documentation with the new page_idle tracking feature which uses virtual address indexing.
--- .../admin-guide/mm/idle_page_tracking.rst | 43 ++++++++++++++++--- 1 file changed, 36 insertions(+), 7 deletions(-)
Documentation/admin-guide/mm/idle_page_tracking.rst +36 -7
@@ -19,10 +19,14 @@ It is enabled by CONFIG_IDLE_PAGE_TRACKING=y. User API ======== +There are 2 ways to access the idle page tracking API. One uses physical +address indexing, another uses a simpler virtual address indexing scheme. -The idle page tracking API is located at ``/sys/kernel/mm/page_idle``. -Currently, it consists of the only read-write file, -``/sys/kernel/mm/page_idle/bitmap``. +Physical address indexing +------------------------- +The idle page tracking API for physical address indexing using page frame +numbers (PFN) is located at ``/sys/kernel/mm/page_idle``. Currently, it +consists of the only read-write file, ``/sys/kernel/mm/page_idle/bitmap``. The file implements a bitmap where each bit corresponds to a memory page. The bitmap is represented by an array of 8-byte integers, and the page at PFN #i is
@@ -74,6 +78,31 @@ See :ref:`Documentation/admin-guide/mm/pagemap.rst <pagemap>` for more information about ``/proc/pid/pagemap``, ``/proc/kpageflags``, and ``/proc/kpagecgroup``. +Virtual address indexing +------------------------ +The idle page tracking API for virtual address indexing using virtual frame +numbers (VFN) for a process ``<pid>`` is located at ``/proc/<pid>/page_idle``. +It is a bitmap that follows the same semantics as +``/sys/kernel/mm/page_idle/bitmap`` except that it uses virtual instead of +physical frame numbers. + +This idle page tracking API does not deal with PFN so it does not require prior +lookups of ``pagemap``. This is an advantage on some systems where looking up +PFN is considered a security issue. Also in some cases, this interface could +be slightly more reliable to use than physical address indexing, since in +physical address indexing, address space changes can occur between reading the +``pagemap`` and reading the ``bitmap``, while in virtual address indexing, the +process's ``mmap_sem`` is held for the duration of the access. + +To estimate the amount of pages that are not used by a workload one should: + + 1. Mark all the workload's pages as idle by setting corresponding bits in + ``/proc/<pid>/page_idle``. + + 2. Wait until the workload accesses its working set. + + 3. Read ``/proc/<pid>/page_idle`` and count the number of bits set. + .. _impl_details: Implementation Details
@@ -99,10 +128,10 @@ When a dirty page is written to swap or disk as a result of memory reclaim or exceeding the dirty memory limit, it is not marked referenced. The idle memory tracking feature adds a new page flag, the Idle flag. This flag -is set manually, by writing to ``/sys/kernel/mm/page_idle/bitmap`` (see the -:ref:`User API <user_api>` -section), and cleared automatically whenever a page is referenced as defined -above. +is set manually, by writing to ``/sys/kernel/mm/page_idle/bitmap`` for physical +addressing or by writing to ``/proc/<pid>/page_idle`` for virtual +addressing (see the :ref:`User API <user_api>` section), and cleared +automatically whenever a page is referenced as defined above. When a page is marked idle, the Accessed bit must be cleared in all PTEs it is mapped to, otherwise we will not be able to detect accesses to the page coming
Signature · 1 line
2.22.0.770.g0f2c4a37fd-goog
Quoted 12 lines · Joel Fernandes (Google)
The
page_idletracking feature currently requires looking up the pagemap for a process followed by interacting with/sys/kernel/mm/page_idle. Looking up PFN from pagemap in Android devices is not supported by unprivileged process and requiresSYS_ADMINand gives 0 for the PFN.This patch adds support to directly interact with
page_idletracking at the PID level by introducing a /proc/<pid>/page_idle file. It follows the exact same semantics as the global/sys/kernel/mm/page_idle, but now looking up PFN through pagemap is not needed since the interface uses virtual frame numbers, and at the same time also does not requireSYS_ADMIN.
Quoted 8 lines
In Android, we are using this for the heap profiler (heapprofd) which profiles and pin points code paths which allocates and leaves memory idle for long periods of time. This method solves the security issue with userspace learning the PFN, and while at it is also shown to yield better results than the pagemap lookup, the theory being that the window where the address space can change is reduced by eliminating the intermediate pagemap look up stage. In virtual address indexing, the process's
mmap_semis held for the duration of the access.
As already mentioned in one of the previous versions. The interface seems sane and the usecase as well. So I do not really have high level objections.
Quoted 1 line
From a quick look at the patch I would just object to pulling swap idle
tracking into this patch because it makes the review harder and it is essentially a dead code until a later patch. I am also not sure whether that is really necessary and it really begs for an explicit justification.
I will try to go through the patch more carefully later as time allows.
Quoted 1 line
Signed-off-by Joel Fernandes (Google) <[email protected]>
Signature · 2 lines
Michal Hocko SUSE Labs
Quoted 22 lines · Michal Hocko
Quoted 12 lines · Joel Fernandes (Google)
The
page_idletracking feature currently requires looking up the pagemap for a process followed by interacting with/sys/kernel/mm/page_idle. Looking up PFN from pagemap in Android devices is not supported by unprivileged process and requiresSYS_ADMINand gives 0 for the PFN.This patch adds support to directly interact with
page_idletracking at the PID level by introducing a /proc/<pid>/page_idle file. It follows the exact same semantics as the global/sys/kernel/mm/page_idle, but now looking up PFN through pagemap is not needed since the interface uses virtual frame numbers, and at the same time also does not requireSYS_ADMIN.Quoted 8 lines
In Android, we are using this for the heap profiler (heapprofd) which profiles and pin points code paths which allocates and leaves memory idle for long periods of time. This method solves the security issue with userspace learning the PFN, and while at it is also shown to yield better results than the pagemap lookup, the theory being that the window where the address space can change is reduced by eliminating the intermediate pagemap look up stage. In virtual address indexing, the process's
mmap_semis held for the duration of the access.
Quoted 3 lines
As already mentioned in one of the previous versions. The interface seems sane and the usecase as well. So I do not really have high level objections.
That is great to know.
Quoted 5 lines
From a quick look at the patch I would just object to pulling swap idle tracking into this patch because it makes the review harder and it is essentially a dead code until a later patch. I am also not sure whether that is really necessary and it really begs for an explicit justification.
Ok I will split it out, and also expand on the need for it a bit more.
Quoted 2 lines
I will try to go through the patch more carefully later as time allows.
Thanks a lot.
Quoted 4 lines
Quoted 1 line
Signed-off-by Joel Fernandes (Google) <[email protected]>Signature · 2 lines
Michal Hocko SUSE Labs
- Joel
Andrew Morton
maintainer(cc Brendan's other email address, hoping for review input ;))
Quoted 12 lines · On Mon, 5 Aug 2019 13:04:47 -0400 "Joel Fernandes (Google)"…
The
page_idletracking feature currently requires looking up the pagemap for a process followed by interacting with/sys/kernel/mm/page_idle. Looking up PFN from pagemap in Android devices is not supported by unprivileged process and requiresSYS_ADMINand gives 0 for the PFN.This patch adds support to directly interact with
page_idletracking at the PID level by introducing a /proc/<pid>/page_idle file. It follows the exact same semantics as the global/sys/kernel/mm/page_idle, but now looking up PFN through pagemap is not needed since the interface uses virtual frame numbers, and at the same time also does not requireSYS_ADMIN.
Quoted 8 lines
In Android, we are using this for the heap profiler (heapprofd) which profiles and pin points code paths which allocates and leaves memory idle for long periods of time. This method solves the security issue with userspace learning the PFN, and while at it is also shown to yield better results than the pagemap lookup, the theory being that the window where the address space can change is reduced by eliminating the intermediate pagemap look up stage. In virtual address indexing, the process's
mmap_semis held for the duration of the access.
Quite a lot of changes to the page_idle code. Has this all been runtime tested on architectures where CONFIG_HAVE_ARCH_PTE_SWP_PGIDLE=n? That could be x86 with a little Kconfig fiddle-for-testing-purposes.
Quoted 1 line
8 files changed, 376 insertions(+), 45 deletions(-)
Quite a lot of new code unconditionally added to major architectures. Are we confident that everyone will want this feature?
Quoted 9 lines
...
fs/proc/task_mmu.c +6
-0+static int proc_page_idle_open (struct inode *inode,struct file *file) +{ +struct mm_struct *mm; + + mm =proc_mem_open (inode,PTRACE_MODE_READ ); +if (IS_ERR (mm))
fs/proc/task_mmu.c const struct file_operations proc_pagemap_operations = { · line 1682
Quoted 10 lines
fs/proc/task_mmu.c +10
-0+return PTR_ERR (mm); + file->private_data = mm; +return 0 ; +} + +static int proc_page_idle_release (struct inode *inode,struct file *file) +{ +struct mm_struct *mm = file->private_data; + +if (mm)
I suspect the test isn't needed? proc_page_idle_release) won't be called if proc_page_idle_open() failed?
Quoted 6 lines
fs/proc/task_mmu.c +3
-0+mmdrop (mm); +return 0 ; +}
Quoted 1 line · Andrew Morton
(cc Brendan's other email address, hoping for review input ;))
;)
Quoted 23 lines · On Mon, 5 Aug 2019 13:04:47 -0400 "Joel Fernandes (Google)"…
Quoted 13 lines
The
page_idletracking feature currently requires looking up the pagemap for a process followed by interacting with/sys/kernel/mm/page_idle. Looking up PFN from pagemap in Android devices is not supported by unprivileged process and requiresSYS_ADMINand gives 0 for the PFN.This patch adds support to directly interact with
page_idletracking at the PID level by introducing a /proc/<pid>/page_idle file. It follows the exact same semantics as the global/sys/kernel/mm/page_idle, but now looking up PFN through pagemap is not needed since the interface uses virtual frame numbers, and at the same time also does not requireSYS_ADMIN.Quoted 8 lines
In Android, we are using this for the heap profiler (heapprofd) which profiles and pin points code paths which allocates and leaves memory idle for long periods of time. This method solves the security issue with userspace learning the PFN, and while at it is also shown to yield better results than the pagemap lookup, the theory being that the window where the address space can change is reduced by eliminating the intermediate pagemap look up stage. In virtual address indexing, the process's
mmap_semis held for the duration of the access.
Quoted 4 lines
Quite a lot of changes to the
page_idlecode. Has this all been runtime tested on architectures whereCONFIG_HAVE_ARCH_PTE_SWP_PGIDLE=n? That could be x86 with a littleKconfigfiddle-for-testing-purposes.
I will do this Kconfig fiddle test with CONFIG_HAVE_ARCH_PTE_SWP_PGIDLE=n and test the patch as well.
In previous series, this flag was not there (which should have been equivalent to the above test), and things are working fine.
Quoted 4 lines
Quoted 1 line
8 files changed, 376 insertions(+), 45 deletions(-)Quite a lot of new code unconditionally added to major architectures. Are we confident that everyone will want this feature?
I did not follow, could you clarify more? All of this diff stat is not to architecture code:
arch/Kconfig | 3 ++ fs/proc/base.c | 3 ++ fs/proc/internal.h | 1 + fs/proc/task_mmu.c | 43 +++++++++++++++++++++ include/asm-generic/pgtable.h | 6 +++ include/linux/page_idle.h | 4 ++ mm/page_idle.c | 359 +++++++++++++++++++++++++++++.. mm/rmap.c | 2 + 8 files changed, 376 insertions(+), 45 deletions(-)
The arcitecture change is in a later patch, and is not that many lines.
Also, I am planning to split the swap functionality of the patch into a separate one for easier review.
Quoted 11 lines
Quoted 11 lines
fs/proc/task_mmu.c +11
-0+static int proc_page_idle_open (struct inode *inode,struct file *file) +{ +struct mm_struct *mm; + + mm =proc_mem_open (inode,PTRACE_MODE_READ ); +if (IS_ERR (mm)) +return PTR_ERR (mm); + file->private_data = mm; +return 0 ; +} +
Quoted 8 lines
fs/proc/task_mmu.c const struct file_operations proc_pagemap_operations = { · line 1682
Quoted 5 lines
fs/proc/task_mmu.c +5
-0+static int proc_page_idle_release (struct inode *inode,struct file *file) +{ +struct mm_struct *mm = file->private_data; + +if (mm)I suspect the test isn't needed?
proc_page_idle_release) won't be called ifproc_page_idle_open()failed?
Yes you are right, will remove the test.
thanks,
- Joel
Andrew Morton
maintainerQuoted 7 lines · On Wed, 7 Aug 2019 06:00:13 -0400 Joel Fernandes <[email protected]> wrote:
Quoted 4 lines
Quoted 1 line
8 files changed, 376 insertions(+), 45 deletions(-)Quite a lot of new code unconditionally added to major architectures. Are we confident that everyone will want this feature?
I did not follow, could you clarify more? All of this diff stat is not to architecture code:
My point is that the patchset adds a lot of new code with no way in which users can opt out. Almost everyone gets a fatter kernel - how many of those users will actually benefit from it?
If "not many" then shouldn't we be making it Kconfigurable?
Are there userspace tools which present this info to users or which provide monitoring of some form? Do major distros ship those tools? Do people use them? etcetera.
Quoted 11 lines · Andrew Morton
Quoted 7 lines · On Wed, 7 Aug 2019 06:00:13 -0400 Joel Fernandes <[email protected]> wrote:
Quoted 4 lines
Quoted 1 line
8 files changed, 376 insertions(+), 45 deletions(-)Quite a lot of new code unconditionally added to major architectures. Are we confident that everyone will want this feature?
I did not follow, could you clarify more? All of this diff stat is not to architecture code:
Quoted 5 lines
My point is that the patchset adds a lot of new code with no way in which users can opt out. Almost everyone gets a fatter kernel - how many of those users will actually benefit from it?
If "not many" then shouldn't we be making it Kconfigurable?
Almost all of this code is already configurable with CONFIG_IDLE_PAGE_TRACKING. If you disable it, then all of this code gets disabled.
Or are you referring to something else that needs to be made configurable?
Quoted 4 lines
Are there userspace tools which present this info to users or which provide monitoring of some form? Do major distros ship those tools? Do people use them? etcetera.
Android's heapprofd is what I was working on which is already using it (patch is not yet upstreamed). There is working set tracking which Sandeep (also from Android) said he wants to use. Minchan plans to use this in combination with ZRAM-based idle tracking. Mike Rappoport also showed some interest, but I am not sure where/how he is using it. These are just some of the usecases I am aware off. I am pretty sure more will come as well.
thanks,
- Joel