Make anon_vma operations testable
Subsystems (3)
cover Make anon_vma operations testable
There are several anon_vma manipulation functions implemented in mm/rmap.c, those concerning anon_vma preparing, cloning and forking, which logically could be stand-alone.
This patch series isolates anon_vma manipulation functionality into its own file, mm/anon_vma.c, and provides an API to the rest of the kernel in include/linux/anon_vma.h.
It also introduce mm/anon_vma_internal.h, which specifies which headers need to be imported by anon_vma.c, leading to the very useful property that anon_vma.c depends only on include/linux/anon_vma.h and mm/anon_vma_internal.h.
This means we can then re-implement anon_vma_internal.h in userland, adding shims for kernel mechanisms as required, allowing us to unit test internal anon_vma functionality.
This patch series takes advantage of existing shim logic and full userland interval tree support contained in tools/testing/rbtree/ and tools/include/linux/.
Kernel functionality is stubbed and shimmed as needed in tools/testing/anon_vma/ which contains a fully functional userland anon_vma_internal.h file and which imports mm/anon_vma.c to be directly tested from userland.
Patch 1 split anon_vma related logic to mm/anon_vma.c Patch 2 add a simple skeleton testing on simple fault and fork Patch 3/4 add tests for mergeable and reusable anon_vma Patch 5 assert the anon_vma double-reuse is fixed
Wei Yang (5):
mm: move anon_vma manipulation functions to own file
anon_vma: add skeleton code for userland testing of anon_vma logic
anon_vma: add test for mergeable anon_vma
anon_vma: add test for reusable anon_vma
anon_vma: add test to assert no double-reuse
MAINTAINERS | 3 + include/linux/anon_vma.h | 163 +++++ include/linux/rmap.h | 147 +--- mm/Makefile | 2 +- mm/anon_vma.c | 396 +++++++++++ mm/anon_vma_internal.h | 14 + mm/rmap.c | 391 ----------- tools/include/linux/rwsem.h | 10 + tools/include/linux/slab.h | 4 + tools/testing/anon_vma/.gitignore | 3 + tools/testing/anon_vma/Makefile | 25 + tools/testing/anon_vma/anon_vma.c | 773 +++++++++++++++++++++ tools/testing/anon_vma/anon_vma_internal.h | 88 +++ tools/testing/anon_vma/interval_tree.c | 53 ++ tools/testing/anon_vma/linux/atomic.h | 18 + tools/testing/anon_vma/linux/fs.h | 6 + tools/testing/anon_vma/linux/mm.h | 44 ++ tools/testing/anon_vma/linux/mm_types.h | 57 ++ tools/testing/anon_vma/linux/mmzone.h | 6 + tools/testing/anon_vma/linux/rmap.h | 8 + tools/testing/shared/linux/anon_vma.h | 7 + 21 files changed, 1680 insertions(+), 538 deletions(-) create mode 100644 include/linux/anon_vma.h create mode 100644 mm/anon_vma.c create mode 100644 mm/anon_vma_internal.h create mode 100644 tools/testing/anon_vma/.gitignore create mode 100644 tools/testing/anon_vma/Makefile create mode 100644 tools/testing/anon_vma/anon_vma.c create mode 100644 tools/testing/anon_vma/anon_vma_internal.h create mode 100644 tools/testing/anon_vma/interval_tree.c create mode 100644 tools/testing/anon_vma/linux/atomic.h create mode 100644 tools/testing/anon_vma/linux/fs.h create mode 100644 tools/testing/anon_vma/linux/mm.h create mode 100644 tools/testing/anon_vma/linux/mm_types.h create mode 100644 tools/testing/anon_vma/linux/mmzone.h create mode 100644 tools/testing/anon_vma/linux/rmap.h create mode 100644 tools/testing/shared/linux/anon_vma.h
Signature · 1 line
2.34.1
RFC PATCH 1/5 mm: move anon_vma manipulation functions to own file
This patch introduce anon_vma.c and move anon_vma manipulation function to this file from rmap.c.
This allows us to create userland testing code and verify the functionality for further change.
--- MAINTAINERS | 3 + include/linux/anon_vma.h | 163 ++++++++++++++++ include/linux/rmap.h | 147 +-------------- mm/Makefile | 2 +- mm/anon_vma.c | 396 +++++++++++++++++++++++++++++++++++++++ mm/anon_vma_internal.h | 14 ++ mm/rmap.c | 391 -------------------------------------- 7 files changed, 578 insertions(+), 538 deletions(-) create mode 100644 include/linux/anon_vma.h create mode 100644 mm/anon_vma.c create mode 100644 mm/anon_vma_internal.h
MAINTAINERS +3 -0
@@ -15575,7 +15575,10 @@ R: Harry Yoo <[email protected]> L: [email protected] S: Maintained F: include/linux/rmap.h +F: include/linux/anon_vma.h F: mm/rmap.c +F: mm/anon_vma.c +F: mm/anon_vma_internal.c MEMORY MANAGEMENT - SECRETMEM M: Andrew Morton <[email protected]>
include/linux/anon_vma.h new file +163 -0
163 lines not on this page · raw message
include/linux/rmap.h +1 -146
165 lines not on this page · raw message
mm/Makefile +1 -1
@@ -37,7 +37,7 @@ mmu-y := nommu.o mmu-$(CONFIG_MMU) := highmem.o memory.o mincore.o \ mlock.o mmap.o mmu_gather.o mprotect.o mremap.o \ msync.o page_vma_mapped.o pagewalk.o \ - pgtable-generic.o rmap.o vmalloc.o vma.o + pgtable-generic.o rmap.o vmalloc.o vma.o anon_vma.o ifdef CONFIG_CROSS_MEMORY_ATTACH
mm/anon_vma.c new file +396 -0
396 lines not on this page · raw message
mm/anon_vma_internal.h new file +14 -0
@@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* +* anon_vma_internal.h +* +* Headers required by anon_vma.c, which can be substituted accordingly when +* testing anon_vma functionality. +*/ + +#ifndef __MM_ANON_VMA_INTERNAL_H +#define __MM_ANON_VMA_INTERNAL_H + +#include "internal.h" + +#endif /* __MM_ANON_VMA_INTERNAL_H */
mm/rmap.c +0 -391
409 lines not on this page · raw message
Signature · 1 line
2.34.1
RFC PATCH 2/5 anon_vma: add skeleton code for userland testing of anon_vma logic
In reply to Wei YangEstablish a new userland anon_vma unit testing implementation under tools/testing.
Current is a skeleton with simple cases and provides possibility for new tests.
--- tools/include/linux/rwsem.h | 10 + tools/include/linux/slab.h | 4 + tools/testing/anon_vma/.gitignore | 3 + tools/testing/anon_vma/Makefile | 25 ++ tools/testing/anon_vma/anon_vma.c | 438 +++++++++++++++++++++ tools/testing/anon_vma/anon_vma_internal.h | 55 +++ tools/testing/anon_vma/interval_tree.c | 53 +++ tools/testing/anon_vma/linux/atomic.h | 18 + tools/testing/anon_vma/linux/fs.h | 6 + tools/testing/anon_vma/linux/mm.h | 44 +++ tools/testing/anon_vma/linux/mm_types.h | 57 +++ tools/testing/anon_vma/linux/mmzone.h | 6 + tools/testing/anon_vma/linux/rmap.h | 8 + tools/testing/shared/linux/anon_vma.h | 7 + 14 files changed, 734 insertions(+) create mode 100644 tools/testing/anon_vma/.gitignore create mode 100644 tools/testing/anon_vma/Makefile create mode 100644 tools/testing/anon_vma/anon_vma.c create mode 100644 tools/testing/anon_vma/anon_vma_internal.h create mode 100644 tools/testing/anon_vma/interval_tree.c create mode 100644 tools/testing/anon_vma/linux/atomic.h create mode 100644 tools/testing/anon_vma/linux/fs.h create mode 100644 tools/testing/anon_vma/linux/mm.h create mode 100644 tools/testing/anon_vma/linux/mm_types.h create mode 100644 tools/testing/anon_vma/linux/mmzone.h create mode 100644 tools/testing/anon_vma/linux/rmap.h create mode 100644 tools/testing/shared/linux/anon_vma.h
tools/include/linux/rwsem.h +10 -0
@@ -38,6 +38,16 @@ static inline int up_write(struct rw_semaphore *sem)return pthread_rwlock_unlock (&sem->lock); } +static inline int down_read_trylock (struct rw_semaphore *sem) +{ +return pthread_rwlock_tryrdlock (&sem->lock); +} + +static inline int down_write_trylock (struct rw_semaphore *sem) +{ +return pthread_rwlock_trywrlock (&sem->lock); +} +#define down_read_nested (sem, subclass) down_read(sem)#define down_write_nested (sem, subclass) down_write(sem)
tools/include/linux/slab.h +4 -0
@@ -41,6 +41,10 @@ struct kmem_cache *kmem_cache_create(const char *name, unsigned int size, unsigned int align, unsigned int flags, void (*ctor)(void *)); +#define KMEM_CACHE (__struct, __flags) \ + kmem_cache_create(#__struct, sizeof(struct __struct), \ + __alignof__(struct __struct), __flags, NULL) +void kmem_cache_free_bulk (struct kmem_cache *cachep,size_t size,void **list);int kmem_cache_alloc_bulk (struct kmem_cache *cachep,gfp_t gfp,size_t size,void **list);
tools/testing/anon_vma/.gitignore new file +3 -0
@@ -0,0 +1,3 @@ +# SPDX-License-Identifier: GPL-2.0-or-later +generated/ +anon_vma
tools/testing/anon_vma/Makefile new file +25 -0
@@ -0,0 +1,25 @@ +# SPDX-License-Identifier: GPL-2.0-or-later + +.PHONY: default clean + +default: anon_vma + +include ../shared/shared.mk + +CFLAGS += -D CONFIG_PHYS_ADDR_T_64BIT +ifeq ($(DEBUG), 1) + CFLAGS += -D ANON_VMA_INTERVAL_TREE_DEBUG -g +endif + +OFILES = $(LIBS) linux.o anon_vma.o rbtree-shim.o \ + interval_tree-shim.o interval_tree.o +TARGETS = anon_vma + +interval_tree.o: ../../../mm/interval_tree.c +anon_vma.o: anon_vma.c anon_vma_internal.h ../../../mm/anon_vma.c \ + ../../../include/linux/anon_vma.h + +anon_vma: $(OFILES) + +clean: + $(RM) $(TARGETS) *.o generated/*
tools/testing/anon_vma/anon_vma.c new file +438 -0
438 lines not on this page · raw message
tools/testing/anon_vma/anon_vma_internal.h new file +55 -0
55 lines not on this page · raw message
tools/testing/anon_vma/interval_tree.c new file +53 -0
53 lines not on this page · raw message
tools/testing/anon_vma/linux/atomic.h new file +18 -0
@@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ + +#ifndef __TOOLS_LINUX_ATOMIC_H +#define __TOOLS_LINUX_ATOMIC_H + +#include <urcu/uatomic.h> + +#define atomic_t int32_t +#define atomic_inc (x) uatomic_inc(x) +#define atomic_read (x) uatomic_read(x) +#define atomic_set (x, y) uatomic_set(x, y) + +static inline bool atomic_dec_and_test (atomic_t *v) +{ +return uatomic_sub_return (v,1 ) ==0 ; +} + +#endif /* __TOOLS_LINUX_ATOMIC_H */
tools/testing/anon_vma/linux/fs.h new file +6 -0
@@ -0,0 +1,6 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ + +#ifndef __TOOLS_LINUX_FS_H +#define __TOOLS_LINUX_FS_H + +#endif /* __TOOLS_LINUX_FS_H */
tools/testing/anon_vma/linux/mm.h new file +44 -0
44 lines not on this page · raw message
tools/testing/anon_vma/linux/mm_types.h new file +57 -0
57 lines not on this page · raw message
tools/testing/anon_vma/linux/mmzone.h new file +6 -0
@@ -0,0 +1,6 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ + +#ifndef __TOOLS_LINUX_MMZONE_H +#define __TOOLS_LINUX_MMZONE_H + +#endif /* __TOOLS_LINUX_MMZONE_H */
tools/testing/anon_vma/linux/rmap.h new file +8 -0
@@ -0,0 +1,8 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ + +#ifndef __TOOLS_LINUX_RMAP_H +#define __TOOLS_LINUX_RMAP_H + +#include <linux/anon_vma.h> + +#endif /* __TOOLS_LINUX_RMAP_H */
tools/testing/shared/linux/anon_vma.h new file +7 -0
@@ -0,0 +1,7 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +#ifndef _TEST_ANON_VMA_H +#define _TEST_ANON_VMA_H + +#include "../../../../include/linux/anon_vma.h" + +#endif /* _TEST_ANON_VMA_H */
Signature · 1 line
2.34.1
Re: [RFC Patch 2/5] anon_vma: add skeleton code for userland testing of anon_vma logic
On Tue, Apr 29, 2025 at 09:06:36AM +0000, Wei Yang wrote:
[...]
Quoted 84 lines
tools/testing/anon_vma/anon_vma.c new file +84
-0+ +static bool test_fork_grand_child (void ) +{ +struct vm_area_struct *root_vma, *grand_vma, *vma1, *vma2; +struct anon_vma_chain *avc; +struct anon_vma *root_anon_vma; +DECLARE_BITMAP (expected,10 ); +DECLARE_BITMAP (found,10 ); + +bitmap_zero (expected,10 ); +bitmap_zero (found,10 ); + +/* +* root_anon_vma root_vma +* +-----------+ +-----------+ +* | | ---> | | +* +-----------+ +-----------+ +*/ + + root_vma =alloc_vma (0x3000 ,0x5000 ,3 ); +/* First fault on parent anonymous vma. */ +__anon_vma_prepare (root_vma); + root_anon_vma = root_vma->anon_vma; +ASSERT_NE (NULL , root_anon_vma); +bitmap_set (expected, root_vma->index,1 ); + +/* First fork */ +/* +* root_anon_vma root_vma +* +-----------+ +-----------+ +* | | ---> | | +* +-----------+ +-----------+ +* \ +* \ vma1 +* \ +-----------+ +* > | | +* +-----------+ +*/ + vma1 =alloc_vma (0x3000 ,0x5000 ,3 ); +anon_vma_fork (vma1, root_vma); +ASSERT_NE (NULL , vma1->anon_vma); +bitmap_set (expected, vma1->index,1 ); +/* Parent/Root is root_vma->anon_vma */ +ASSERT_EQ (vma1->anon_vma->parent, root_vma->anon_vma); +ASSERT_EQ (vma1->anon_vma->root, root_vma->anon_vma); + +/* Second fork */ +/* +* root_anon_vma root_vma +* +-----------+ +-----------+ +* | | ---> | | +* +-----------+ +-----------+ +* \ +* \------------------+ +* \ vma1 \ vma2 +* \ +-----------+ \ +-----------+ +* > | | > | | +* +-----------+ +-----------+ +*/ + vma2 =alloc_vma (0x3000 ,0x5000 ,3 ); +anon_vma_fork (vma2, root_vma); +ASSERT_NE (NULL , vma2->anon_vma); +bitmap_set (expected, vma2->index,1 ); +/* Parent/Root is root_vma->anon_vma */ +ASSERT_EQ (vma2->anon_vma->parent, root_vma->anon_vma); +ASSERT_EQ (vma2->anon_vma->root, root_vma->anon_vma); +dump_anon_vma_interval_tree (root_vma->anon_vma); + +/* Fork grand child from second child */ +/* +* root_anon_vma root_vma +* +-----------+ +-----------+ +* | | ---> | | +* +-----------+ +-----------+ +* \ +* \------------------+ +* |\ vma1 \ vma2 +* | \ +-----------+ \ +-----------+ +* | > | | > | | +* | +-----------+ +-----------+ +* \ +* \ grand_vma +* \ +-----------+ +* > | |
tools/testing/anon_vma/anon_vma.c line 363
Quoted 10 lines
tools/testing/anon_vma/anon_vma.c new file +10
-0+* +-----------+ +*/ + grand_vma =alloc_vma (0x3000 ,0x5000 ,3 ); +anon_vma_fork (grand_vma, vma2); +ASSERT_NE (NULL , grand_vma->anon_vma); +bitmap_set (expected, grand_vma->index,1 ); +/* Root is root_vma->anon_vma */ +ASSERT_EQ (grand_vma->anon_vma->root, root_vma->anon_vma); +/* Parent is vma2->anon_vma */ +ASSERT_EQ (grand_vma->anon_vma->parent, vma2->anon_vma);
Hi, Lorenzo
Here is the case I am talking about in another thread[1].
The naming is a little different from that.
* root_vma is VMA A
* vma2 is VMA B
* grand_vma is VMA C
If you add following debug code here.
```
printf("root num_children %d\n", root_vma->anon_vma->num_children);
printf("vma2 num_children %d\n", vma2->anon_vma->num_children);
printf("grand_vma num_children %d\n", grand_vma->anon_vma->num_children);
```You would see vma2 num_children is 1, but it has a child.
If I missed something, feel free to correct me.
[1]: https://lkml.kernel.org/r/20250501011845.ktbfgymor4oz5sok@master
Quoted 47 lines
tools/testing/anon_vma/anon_vma.c new file +47
-0+ +/* Expect to find only vmas from second fork */ + anon_vma_interval_tree_foreach(avc, &vma2->anon_vma->rb_root, 3, 4) { +ASSERT_TRUE (avc->vma == vma2 || avc->vma == grand_vma); + } + + anon_vma_interval_tree_foreach(avc, &root_vma->anon_vma->rb_root, 3, 4) { +bitmap_set (found, avc->vma->index,1 ); + } +/* Expect to find all vma including child and grand child. */ +ASSERT_TRUE (bitmap_equal (expected, found,10 )); + +/* Root process exit or unmap root_vma. */ +/* +* root_anon_vma +* +-----------+ +* | | +* +-----------+ +* \ +* \------------------+ +* |\ vma1 \ vma2 +* | \ +-----------+ \ +-----------+ +* | > | | > | | +* | +-----------+ +-----------+ +* \ +* \ grand_vma +* \ +-----------+ +* > | | +* +-----------+ +*/ +bitmap_clear (expected, root_vma->index,1 ); +unlink_anon_vmas (root_vma); +ASSERT_EQ (0 , root_anon_vma->num_active_vmas); + +bitmap_zero (found,10 ); + anon_vma_interval_tree_foreach(avc, &root_anon_vma->rb_root, 3, 4) { +bitmap_set (found, avc->vma->index,1 ); + } +/* Expect to find all vmas even root_vma released. */ +ASSERT_TRUE (bitmap_equal (expected, found,10 )); + +cleanup (); + +ASSERT_EQ (0 , nr_allocated); +return true; +} +
Signature · 2 lines
Wei Yang Help you, Help me
Re: [RFC Patch 2/5] anon_vma: add skeleton code for userland testing of anon_vma logic
Quoted 115 lines · Wei Yang
On Tue, Apr 29, 2025 at 09:06:36AM +0000, Wei Yang wrote:
[...]Quoted 84 lines
tools/testing/anon_vma/anon_vma.c new file +84
-0+ +static bool test_fork_grand_child (void ) +{ +struct vm_area_struct *root_vma, *grand_vma, *vma1, *vma2; +struct anon_vma_chain *avc; +struct anon_vma *root_anon_vma; +DECLARE_BITMAP (expected,10 ); +DECLARE_BITMAP (found,10 ); + +bitmap_zero (expected,10 ); +bitmap_zero (found,10 ); + +/* +* root_anon_vma root_vma +* +-----------+ +-----------+ +* | | ---> | | +* +-----------+ +-----------+ +*/ + + root_vma =alloc_vma (0x3000 ,0x5000 ,3 ); +/* First fault on parent anonymous vma. */ +__anon_vma_prepare (root_vma); + root_anon_vma = root_vma->anon_vma; +ASSERT_NE (NULL , root_anon_vma); +bitmap_set (expected, root_vma->index,1 ); + +/* First fork */ +/* +* root_anon_vma root_vma +* +-----------+ +-----------+ +* | | ---> | | +* +-----------+ +-----------+ +* \ +* \ vma1 +* \ +-----------+ +* > | | +* +-----------+ +*/ + vma1 =alloc_vma (0x3000 ,0x5000 ,3 ); +anon_vma_fork (vma1, root_vma); +ASSERT_NE (NULL , vma1->anon_vma); +bitmap_set (expected, vma1->index,1 ); +/* Parent/Root is root_vma->anon_vma */ +ASSERT_EQ (vma1->anon_vma->parent, root_vma->anon_vma); +ASSERT_EQ (vma1->anon_vma->root, root_vma->anon_vma); + +/* Second fork */ +/* +* root_anon_vma root_vma +* +-----------+ +-----------+ +* | | ---> | | +* +-----------+ +-----------+ +* \ +* \------------------+ +* \ vma1 \ vma2 +* \ +-----------+ \ +-----------+ +* > | | > | | +* +-----------+ +-----------+ +*/ + vma2 =alloc_vma (0x3000 ,0x5000 ,3 ); +anon_vma_fork (vma2, root_vma); +ASSERT_NE (NULL , vma2->anon_vma); +bitmap_set (expected, vma2->index,1 ); +/* Parent/Root is root_vma->anon_vma */ +ASSERT_EQ (vma2->anon_vma->parent, root_vma->anon_vma); +ASSERT_EQ (vma2->anon_vma->root, root_vma->anon_vma); +dump_anon_vma_interval_tree (root_vma->anon_vma); + +/* Fork grand child from second child */ +/* +* root_anon_vma root_vma +* +-----------+ +-----------+ +* | | ---> | | +* +-----------+ +-----------+ +* \ +* \------------------+ +* |\ vma1 \ vma2 +* | \ +-----------+ \ +-----------+ +* | > | | > | | +* | +-----------+ +-----------+ +* \ +* \ grand_vma +* \ +-----------+ +* > | | tools/testing/anon_vma/anon_vma.c line 363
Quoted 10 lines
tools/testing/anon_vma/anon_vma.c new file +10
-0+* +-----------+ +*/ + grand_vma =alloc_vma (0x3000 ,0x5000 ,3 ); +anon_vma_fork (grand_vma, vma2); +ASSERT_NE (NULL , grand_vma->anon_vma); +bitmap_set (expected, grand_vma->index,1 ); +/* Root is root_vma->anon_vma */ +ASSERT_EQ (grand_vma->anon_vma->root, root_vma->anon_vma); +/* Parent is vma2->anon_vma */ +ASSERT_EQ (grand_vma->anon_vma->parent, vma2->anon_vma);Hi, Lorenzo
Here is the case I am talking about in another thread[1].
The naming is a little different from that.
*
root_vmais VMA A
* vma2 is VMA B
*grand_vmais VMA CIf you add following debug code here.
``` printf("root num_children %d\n", root_vma->anon_vma->num_children); printf("vma2 num_children %d\n", vma2->anon_vma->num_children); printf("grand_vma num_children %d\n", grand_vma->anon_vma->num_children); ```
Quoted 6 lines
You would see vma2
num_childrenis 1, but it has a child.If I missed something, feel free to correct me.
[1]: https://lkml.kernel.org/r/20250501011845.ktbfgymor4oz5sok@master
See the thread over there. This explanation isn't quite right, and the diagram is wrong, but indeed anon_vma reuse is a thing that needs addressing.
Having looked at this more I'm not a huge fan of these tests, you're essentially open-coding fork over and over again, any change in overlying fork logic will break.
The VMA and maple tree, etc. unit tests work much better as the whole thing essentially can be separated out, but in this case you really can't. This is obviously in addition to the aforementioned issues with causing separation in an area where I want to try to unify.
So I definitely think that, again, you'd be better off looking at tests as suggested by David, but moreover I think looking at bugs etc. is more helpful.
This report was in effect reporting a bug before it happened so all good for this kind of thing :>) and that is _always_ welcome. But do try to be more direct and to the point if you can be.
Thanks!
Quoted 53 lines
Quoted 37 lines
tools/testing/anon_vma/anon_vma.c new file +37
-0+ +/* Expect to find only vmas from second fork */ + anon_vma_interval_tree_foreach(avc, &vma2->anon_vma->rb_root, 3, 4) { +ASSERT_TRUE (avc->vma == vma2 || avc->vma == grand_vma); + } + + anon_vma_interval_tree_foreach(avc, &root_vma->anon_vma->rb_root, 3, 4) { +bitmap_set (found, avc->vma->index,1 ); + } +/* Expect to find all vma including child and grand child. */ +ASSERT_TRUE (bitmap_equal (expected, found,10 )); + +/* Root process exit or unmap root_vma. */ +/* +* root_anon_vma +* +-----------+ +* | | +* +-----------+ +* \ +* \------------------+ +* |\ vma1 \ vma2 +* | \ +-----------+ \ +-----------+ +* | > | | > | | +* | +-----------+ +-----------+ +* \ +* \ grand_vma +* \ +-----------+ +* > | | +* +-----------+ +*/ +bitmap_clear (expected, root_vma->index,1 ); +unlink_anon_vmas (root_vma); +ASSERT_EQ (0 , root_anon_vma->num_active_vmas); + +bitmap_zero (found,10 ); + anon_vma_interval_tree_foreach(avc, &root_anon_vma->rb_root, 3, 4) { +bitmap_set (found, avc->vma->index,1 );tools/testing/anon_vma/anon_vma.c line 410
Quoted 10 lines
tools/testing/anon_vma/anon_vma.c new file +10
-0+ } +/* Expect to find all vmas even root_vma released. */ +ASSERT_TRUE (bitmap_equal (expected, found,10 )); + +cleanup (); + +ASSERT_EQ (0 , nr_allocated); +return true; +} +--
Wei Yang
Help you, Help me
Re: [RFC Patch 2/5] anon_vma: add skeleton code for userland testing of anon_vma logic
Quoted 131 lines · Lorenzo Stoakes
Quoted 115 lines · Wei Yang
On Tue, Apr 29, 2025 at 09:06:36AM +0000, Wei Yang wrote:
[...]Quoted 84 lines
tools/testing/anon_vma/anon_vma.c new file +84
-0+ +static bool test_fork_grand_child (void ) +{ +struct vm_area_struct *root_vma, *grand_vma, *vma1, *vma2; +struct anon_vma_chain *avc; +struct anon_vma *root_anon_vma; +DECLARE_BITMAP (expected,10 ); +DECLARE_BITMAP (found,10 ); + +bitmap_zero (expected,10 ); +bitmap_zero (found,10 ); + +/* +* root_anon_vma root_vma +* +-----------+ +-----------+ +* | | ---> | | +* +-----------+ +-----------+ +*/ + + root_vma =alloc_vma (0x3000 ,0x5000 ,3 ); +/* First fault on parent anonymous vma. */ +__anon_vma_prepare (root_vma); + root_anon_vma = root_vma->anon_vma; +ASSERT_NE (NULL , root_anon_vma); +bitmap_set (expected, root_vma->index,1 ); + +/* First fork */ +/* +* root_anon_vma root_vma +* +-----------+ +-----------+ +* | | ---> | | +* +-----------+ +-----------+ +* \ +* \ vma1 +* \ +-----------+ +* > | | +* +-----------+ +*/ + vma1 =alloc_vma (0x3000 ,0x5000 ,3 ); +anon_vma_fork (vma1, root_vma); +ASSERT_NE (NULL , vma1->anon_vma); +bitmap_set (expected, vma1->index,1 ); +/* Parent/Root is root_vma->anon_vma */ +ASSERT_EQ (vma1->anon_vma->parent, root_vma->anon_vma); +ASSERT_EQ (vma1->anon_vma->root, root_vma->anon_vma); + +/* Second fork */ +/* +* root_anon_vma root_vma +* +-----------+ +-----------+ +* | | ---> | | +* +-----------+ +-----------+ +* \ +* \------------------+ +* \ vma1 \ vma2 +* \ +-----------+ \ +-----------+ +* > | | > | | +* +-----------+ +-----------+ +*/ + vma2 =alloc_vma (0x3000 ,0x5000 ,3 ); +anon_vma_fork (vma2, root_vma); +ASSERT_NE (NULL , vma2->anon_vma); +bitmap_set (expected, vma2->index,1 ); +/* Parent/Root is root_vma->anon_vma */ +ASSERT_EQ (vma2->anon_vma->parent, root_vma->anon_vma); +ASSERT_EQ (vma2->anon_vma->root, root_vma->anon_vma); +dump_anon_vma_interval_tree (root_vma->anon_vma); + +/* Fork grand child from second child */ +/* +* root_anon_vma root_vma +* +-----------+ +-----------+ +* | | ---> | | +* +-----------+ +-----------+ +* \ +* \------------------+ +* |\ vma1 \ vma2 +* | \ +-----------+ \ +-----------+ +* | > | | > | | +* | +-----------+ +-----------+ +* \ +* \ grand_vma +* \ +-----------+ +* > | | tools/testing/anon_vma/anon_vma.c line 363
Quoted 10 lines
tools/testing/anon_vma/anon_vma.c new file +10
-0+* +-----------+ +*/ + grand_vma =alloc_vma (0x3000 ,0x5000 ,3 ); +anon_vma_fork (grand_vma, vma2); +ASSERT_NE (NULL , grand_vma->anon_vma); +bitmap_set (expected, grand_vma->index,1 ); +/* Root is root_vma->anon_vma */ +ASSERT_EQ (grand_vma->anon_vma->root, root_vma->anon_vma); +/* Parent is vma2->anon_vma */ +ASSERT_EQ (grand_vma->anon_vma->parent, vma2->anon_vma);Hi, Lorenzo
Here is the case I am talking about in another thread[1].
The naming is a little different from that.
*
root_vmais VMA A
* vma2 is VMA B
*grand_vmais VMA CIf you add following debug code here.
``` printf("root num_children %d\n", root_vma->anon_vma->num_children); printf("vma2 num_children %d\n", vma2->anon_vma->num_children); printf("grand_vma num_children %d\n", grand_vma->anon_vma->num_children); ```Quoted 6 lines
You would see vma2
num_childrenis 1, but it has a child.If I missed something, feel free to correct me.
[1]: https://lkml.kernel.org/r/20250501011845.ktbfgymor4oz5sok@master
See the thread over there. This explanation isn't quite right, and the diagram is wrong, but indeed
anon_vmareuse is a thing that needs addressing.Having looked at this more I'm not a huge fan of these tests, you're essentially open-coding fork over and over again, any change in overlying fork logic will break.
Quoted 8 lines
The VMA and maple tree, etc. unit tests work much better as the whole thing essentially can be separated out, but in this case you really can't. This is obviously in addition to the aforementioned issues with causing separation in an area where I want to try to unify.
So I definitely think that, again, you'd be better off looking at tests as suggested by David, but moreover I think looking at bugs etc. is more helpful.
Sure. Thank you and David, pointing me a correct direction. And yes, bug has higher priority.
Quoted 4 lines
This report was in effect reporting a bug before it happened so all good for this kind of thing :>) and that is _always_ welcome. But do try to be more direct and to the point if you can be.
Ah, I am glad you like it. Would try to be more clear on expressing my point next time. And thanks for your patience on reading my "confusing" mail.
Quoted 57 lines
Thanks!
Quoted 53 lines
Quoted 37 lines
tools/testing/anon_vma/anon_vma.c new file +37
-0+ +/* Expect to find only vmas from second fork */ + anon_vma_interval_tree_foreach(avc, &vma2->anon_vma->rb_root, 3, 4) { +ASSERT_TRUE (avc->vma == vma2 || avc->vma == grand_vma); + } + + anon_vma_interval_tree_foreach(avc, &root_vma->anon_vma->rb_root, 3, 4) { +bitmap_set (found, avc->vma->index,1 ); + } +/* Expect to find all vma including child and grand child. */ +ASSERT_TRUE (bitmap_equal (expected, found,10 )); + +/* Root process exit or unmap root_vma. */ +/* +* root_anon_vma +* +-----------+ +* | | +* +-----------+ +* \ +* \------------------+ +* |\ vma1 \ vma2 +* | \ +-----------+ \ +-----------+ +* | > | | > | | +* | +-----------+ +-----------+ +* \ +* \ grand_vma +* \ +-----------+ +* > | | +* +-----------+ +*/ +bitmap_clear (expected, root_vma->index,1 ); +unlink_anon_vmas (root_vma); +ASSERT_EQ (0 , root_anon_vma->num_active_vmas); + +bitmap_zero (found,10 ); + anon_vma_interval_tree_foreach(avc, &root_anon_vma->rb_root, 3, 4) { +bitmap_set (found, avc->vma->index,1 );tools/testing/anon_vma/anon_vma.c line 410
Quoted 10 lines
tools/testing/anon_vma/anon_vma.c new file +10
-0+ } +/* Expect to find all vmas even root_vma released. */ +ASSERT_TRUE (bitmap_equal (expected, found,10 )); + +cleanup (); + +ASSERT_EQ (0 , nr_allocated); +return true; +} +--
Wei Yang
Help you, Help me
Signature · 2 lines
Wei Yang Help you, Help me
RFC PATCH 3/5 anon_vma: add test for mergeable anon_vma
In reply to Wei YangAdd test to assert anon_vma is mergeable at the first level.
--- tools/testing/anon_vma/anon_vma.c | 149 +++++++++++++++++++++ tools/testing/anon_vma/anon_vma_internal.h | 35 ++++- 2 files changed, 183 insertions(+), 1 deletion(-)
tools/testing/anon_vma/anon_vma.c +149 -0
@@ -29,6 +29,7 @@ static struct mm_struct dummy_mm = {static int vmas_idx;static struct vm_area_struct *vmas[NUM_VMAS ];static struct kmem_cache *vm_area_cachep; +struct vm_area_struct *mergeable_vma =NULL ;static void vma_ctor (void *data) {
@@ -79,6 +80,7 @@ void cleanup(void) } vmas_idx =0 ; + mergeable_vma =NULL ; }static bool test_simple_fault (void )
@@ -408,6 +410,152 @@ static bool test_fork_grand_child(void)return true; } +static bool test_mergeable_vma (void ) +{ +struct vm_area_struct *root_vma, *child_vma, *vma1, *vma2; +struct anon_vma_chain *avc; +struct anon_vma *root_anon_vma; +DECLARE_BITMAP (expected,10 ); +DECLARE_BITMAP (found,10 ); + +bitmap_zero (expected,10 ); +bitmap_zero (found,10 ); + +/* +* root_anon_vma +* +-----------+ +* | | +* +-----------+ +* \ +* \ +* \ root_vma +* \ +-----------+ +* > | | +* +-----------+ +*/ + root_vma =alloc_vma (0x3000 ,0x5000 ,3 ); +/* First fault on anonymous vma. */ +__anon_vma_prepare (root_vma); +bitmap_set (expected, root_vma->index,1 ); + root_anon_vma = root_vma->anon_vma; +ASSERT_NE (NULL , root_anon_vma); +ASSERT_EQ (1 , root_anon_vma->num_active_vmas); + + mergeable_vma = root_vma; + + vma1 =alloc_vma (0x5000 ,0x7000 ,5 ); +/* First fault on next adjacent anonymous vma. */ +/* +* root_anon_vma +* +-----------+ +* | | +* +-----------+ +* \ +* \------------------+ +* \ root_vma \ vma1 +* \ +-----------+ \ +-----------+ +* > | | > | | +* +-----------+ +-----------+ +*/ +__anon_vma_prepare (vma1); +bitmap_set (expected, vma1->index,1 ); +ASSERT_EQ (vma1->anon_vma, root_anon_vma); +ASSERT_EQ (2 , root_anon_vma->num_active_vmas); + + vma2 =alloc_vma (0x2000 ,0x3000 ,2 ); +/* First fault on previous adjacent anonymous vma. */ +/* +* root_anon_vma +* +-----------+ +* | | +* +-----------+ +* \ +* \------------------+------------------+ +* \ vma2 \ root_vma \ vma1 +* \ +-----------+ \ +-----------+ \ +-----------+ +* > | | > | | > | | +* +-----------+ +-----------+ +-----------+ +*/ +__anon_vma_prepare (vma2); +bitmap_set (expected, vma2->index,1 ); +ASSERT_EQ (vma2->anon_vma, root_anon_vma); +ASSERT_EQ (3 , root_anon_vma->num_active_vmas); +dump_anon_vma_interval_tree (root_anon_vma); + + anon_vma_interval_tree_foreach(avc, &root_anon_vma->rb_root, 2, 7) { +bitmap_set (found, avc->vma->index,1 ); + } +/* Expect to find all vmas in range [2, 7] */ +ASSERT_TRUE (bitmap_equal (expected, found,10 )); + +/* Expect to find only root_vma in range [3, 4] */ + anon_vma_interval_tree_foreach(avc, &root_anon_vma->rb_root, 3, 4) { +ASSERT_EQ (avc->vma, root_vma); + } + +/* unmap adjacent vmas before fork */ +unlink_anon_vmas (vma1); +unlink_anon_vmas (vma2); +ASSERT_EQ (1 , root_anon_vma->num_active_vmas); + +/* Fork a child from root_vma */ +/* +* root_anon_vma root_vma +* +-----------+ +-----------+ +* | | ---> | | +* +-----------+ +-----------+ +* \ +* \ child_vma +* \ +-----------+ +* > | | +* +-----------+ +*/ + child_vma =alloc_vma (0x3000 ,0x5000 ,3 ); +anon_vma_fork (child_vma, root_vma); +ASSERT_NE (NULL , child_vma->anon_vma); +/* Parent/Root is root_vma->anon_vma */ +ASSERT_EQ (child_vma->anon_vma->parent, root_vma->anon_vma); +ASSERT_EQ (child_vma->anon_vma->root, root_vma->anon_vma); + + mergeable_vma = child_vma; + + vma1 =alloc_vma (0x5000 ,0x7000 ,5 ); +/* First fault on next adjacent anonymous vma in child. */ +/* +* vma1->anon_vma vma1 +* +-----------+ +-----------+ +* | | ---> | | +* +-----------+ +-----------+ +*/ +__anon_vma_prepare (vma1); +ASSERT_NE (vma1->anon_vma, child_vma->anon_vma); +ASSERT_EQ (1 , child_vma->anon_vma->num_active_vmas); +ASSERT_EQ (1 , root_anon_vma->num_active_vmas); + + vma2 =alloc_vma (0x2000 ,0x3000 ,2 ); +/* First fault on previous adjacent anonymous vma in child. */ +/* +* vma2->anon_vma vma2 +* +-----------+ +-----------+ +* | | ---> | | +* +-----------+ +-----------+ +*/ +__anon_vma_prepare (vma2); +ASSERT_NE (vma2->anon_vma, child_vma->anon_vma); +ASSERT_EQ (1 , child_vma->anon_vma->num_active_vmas); +ASSERT_EQ (1 , root_anon_vma->num_active_vmas); + +/* Expect to find only 'child_vma' in range [2, 7] */ + anon_vma_interval_tree_foreach(avc, &child_vma->anon_vma->rb_root, 2, 7) { +ASSERT_EQ (avc->vma, child_vma); + } + +cleanup (); + +ASSERT_EQ (0 , nr_allocated); +return true; +} +int main (void ) {int num_tests =0 , num_fail =0 ;
@@ -428,6 +576,7 @@ int main(void)TEST (simple_fork);TEST (fork_two);TEST (fork_grand_child); +TEST (mergeable_vma);#undef TEST
tools/testing/anon_vma/anon_vma_internal.h +34 -1
@@ -33,11 +33,44 @@ static inline int rwsem_is_locked(struct rw_semaphore *sem)return 0 ; } -static inline struct anon_vma *find_mergeable_anon_vma (struct vm_area_struct *vma) +extern struct vm_area_struct *mergeable_vma; + +static int anon_vma_compatible (struct vm_area_struct *a,struct vm_area_struct *b) { +return a->vm_end == b->vm_start && + b->vm_pgoff == a->vm_pgoff + ((b->vm_start - a->vm_start) >>PAGE_SHIFT ); +} + +static struct anon_vma *reusable_anon_vma (struct vm_area_struct *old, +struct vm_area_struct *a, +struct vm_area_struct *b) +{ +if (anon_vma_compatible (a, b)) { +struct anon_vma *anon_vma =READ_ONCE (old->anon_vma); + +if (anon_vma &&list_is_singular (&old->anon_vma_chain)) +return anon_vma; + }return NULL ; } +static inline struct anon_vma *find_mergeable_anon_vma (struct vm_area_struct *vma) +{ +struct anon_vma *anon_vma =NULL ; + +if (!mergeable_vma) +return NULL ; + +/* Try next first. */ +if (mergeable_vma->vm_start >= vma->vm_end) { + anon_vma =reusable_anon_vma (mergeable_vma, vma, mergeable_vma); +if (anon_vma) +return anon_vma; + } + +return reusable_anon_vma (mergeable_vma, mergeable_vma, vma); +} +#ifndef pgoff_t#define pgoff_t unsigned long#endif
Signature · 1 line
2.34.1
RFC PATCH 4/5 anon_vma: add test for reusable anon_vma
In reply to Wei YangAdd test to assert anon_vma is reusable when there is no active vma, except root anon_vma.
--- tools/testing/anon_vma/anon_vma.c | 141 ++++++++++++++++++++++++++++++ 1 file changed, 141 insertions(+)
tools/testing/anon_vma/anon_vma.c +141 -0
@@ -556,6 +556,146 @@ static bool test_mergeable_vma(void)return true; } +static bool test_reuse_anon_vma (void ) +{ +struct vm_area_struct *root_vma, *vma, *vma1, *vma2; +struct anon_vma *root_anon_vma, *reused_anon_vma; +struct anon_vma_chain *avc; + +/* +* root_anon_vma root_vma +* +-----------+ +-----------+ +* | | ---> | av| = root_anon_vma +* +-----------+ +-----------+ +*/ + root_vma =alloc_vma (0x3000 ,0x5000 ,3 ); +__anon_vma_prepare (root_vma); + root_anon_vma = root_vma->anon_vma; +ASSERT_NE (NULL , root_anon_vma); +ASSERT_EQ (1 , root_anon_vma->num_active_vmas); + +/* First fork */ +/* +* root_anon_vma root_vma +* +-----------+ +-----------+ +* | | ---> | av| = root_anon_vma +* +-----------+ +-----------+ +* \ +* \ vma +* \ +-----------+ +* > | av| != root_anon_vma +* +-----------+ +*/ + vma =alloc_vma (0x3000 ,0x5000 ,3 ); +anon_vma_fork (vma, root_vma); +ASSERT_NE (NULL , vma->anon_vma); +/* Parent/Root is root_vma->anon_vma */ +ASSERT_EQ (vma->anon_vma->parent, root_vma->anon_vma); +ASSERT_EQ (vma->anon_vma->root, root_vma->anon_vma); + +/* unlink the root */ +/* +* root_anon_vma +* +-----------+ +* | | +* +-----------+ +* \ +* \ vma +* \ +-----------+ +* > | av| != root_anon_vma +* +-----------+ +*/ +unlink_anon_vmas (root_vma); +ASSERT_EQ (0 , root_anon_vma->num_active_vmas); + +/* Fork grand child from vma */ +/* +* root_anon_vma +* +-----------+ +* | | +* +-----------+ +* \ +* |\ vma +* | \ +-----------+ +* | > | av| != root_anon_vma +* | +-----------+ +* \ +* \ vma1 +* \ +-----------+ +* > | av| != root_anon_vma +* +-----------+ +*/ + vma1 =alloc_vma (0x3000 ,0x5000 ,3 ); +anon_vma_fork (vma1, vma); +ASSERT_NE (NULL , vma1->anon_vma); +/* Root is root_anon_vma */ +ASSERT_EQ (vma1->anon_vma->root, root_anon_vma); +/* Parent is vma1->anon_vma */ +ASSERT_EQ (vma1->anon_vma->parent, vma->anon_vma); +/* vma1->anon_vma != root_anon_vma, since we don't reuse root */ +ASSERT_NE (vma1->anon_vma, root_anon_vma); + +/* unlink vma */ +/* +* root_anon_vma +* +-----------+ +* | | +* +-----------+ +* \ +* | +* \ +* \ vma1 +* \ +-----------+ +* > | av| != root_anon_vma +* +-----------+ +*/ + reused_anon_vma = vma->anon_vma; +unlink_anon_vmas (vma); +ASSERT_EQ (0 , reused_anon_vma->num_active_vmas); + +/* Fork from vma1 */ +/* +* root_anon_vma +* +-----------+ +* | | +* +-----------+ +* \ +* | +* \ +* |\ vma1 +* | \ +-----------+ +* | > | av| != root_anon_vma +* | +-----------+ +* \ +* \ vma2 +* \ +-----------+ +* > | av| == reused_anon_vma +* +-----------+ +*/ + vma2 =alloc_vma (0x3000 ,0x5000 ,3 ); +anon_vma_fork (vma2, vma1); +ASSERT_NE (NULL , vma2->anon_vma); +/* Root is root_vma->anon_vma */ +ASSERT_EQ (vma2->anon_vma->root, root_anon_vma); +/* vma->anon_vma (reused_anon_vma) is reused here */ +ASSERT_EQ (vma2->anon_vma, reused_anon_vma); + +/* Expect to find vma1 and vma2 in reused_anon_vma */ + anon_vma_interval_tree_foreach(avc, &reused_anon_vma->rb_root, 3, 4) { +ASSERT_TRUE (avc->vma == vma1 || avc->vma == vma2); + } + +/* Expect to find vma1 and vma2 in root_anon_vma */ + anon_vma_interval_tree_foreach(avc, &root_anon_vma->rb_root, 3, 4) { +ASSERT_TRUE (avc->vma == vma1 || avc->vma == vma2); + } + +cleanup (); + +ASSERT_EQ (0 , nr_allocated); +return true; +} +int main (void ) {int num_tests =0 , num_fail =0 ;
@@ -577,6 +717,7 @@ int main(void)TEST (fork_two);TEST (fork_grand_child);TEST (mergeable_vma); +TEST (reuse_anon_vma);#undef TEST
Signature · 1 line
2.34.1
RFC PATCH 5/5 anon_vma: add test to assert no double-reuse
In reply to Wei Yangcommit 2555283eb40d ("mm/rmap: Fix anon_vma->degree ambiguity leading to double-reuse") fixed anon_vma double-reuse issue introduced by commit 7a3ef208e662 ("mm: prevent endless growth of anon_vma hierarchy").
Add a test case to assert no double-reuse.
--- tools/testing/anon_vma/anon_vma.c | 47 ++++++++++++++++++++++++++++++- 1 file changed, 46 insertions(+), 1 deletion(-)
tools/testing/anon_vma/anon_vma.c +46 -1
@@ -558,7 +558,7 @@ static bool test_mergeable_vma(void)static bool test_reuse_anon_vma (void ) { -struct vm_area_struct *root_vma, *vma, *vma1, *vma2; +struct vm_area_struct *root_vma, *vma, *vma1, *vma2, *vma3;struct anon_vma *root_anon_vma, *reused_anon_vma;struct anon_vma_chain *avc;
@@ -690,6 +690,51 @@ static bool test_reuse_anon_vma(void)ASSERT_TRUE (avc->vma == vma1 || avc->vma == vma2); } +/* Fork from vma2 */ +/* +* When commit 7a3ef208e662 ("mm: prevent endless growth of anon_vma +* hierarchy") introduce anon_vma reuse, it embedded an issue of +* double-reuse. +* +* It happens when vma2 reuse an existing anon_vma and we fork from +* vma2 later. Before commit 2555283eb40d ("mm/rmap: Fix +* anon_vma->degree ambiguity leading to double-reuse"), the forked +* vma3 would reuse vma1->anon_vma, which is already in use. +* +* The following case assert vma1->anon_vma will not double-reuse. +* +* root_anon_vma +* +-----------+ +* | | +* +-----------+ +* \ +* | +* \ +* |\ vma1 +* | \ +-----------+ +* | > | av| != root_anon_vma +* | +-----------+ +* \ +* |\ vma2 +* | \ +-----------+ +* | > | av| == reused_anon_vma +* | +-----------+ +* \ +* \ vma3 +* \ +-----------+ +* > | av| != vma1->anon_vma +* +-----------+ +*/ +/* vma1->anon_vma already has active vma */ +ASSERT_NE (NULL , vma1->anon_vma); + vma3 =alloc_vma (0x3000 ,0x5000 ,3 ); +anon_vma_fork (vma3, vma2); +ASSERT_NE (NULL , vma3->anon_vma); +/* Root is root_vma->anon_vma */ +ASSERT_EQ (vma3->anon_vma->root, root_anon_vma); +/* vma1->anon_vma is NOT reused here */ +ASSERT_NE (vma3->anon_vma, vma1->anon_vma); +cleanup ();ASSERT_EQ (0 , nr_allocated);
Signature · 1 line
2.34.1
Wei,
NACK the whole series.
I'm really not sure how to get through to you. You were _explicitly_ advised not to send this series. And yet you've sent it anyway.
I mean, I appreciate your enthusiasm and the fact you've made tests here etc. obviously. And you've clearly put a TON of work in. But I just don't know why you would when explicitly told not to without at least discussing it first?
This just isn't a great way of interacting with the community. We're all human, please try to have some empathy for others here, as I really do try to have with you as best I can.
This adds a ton of churn and LOCKS IN assumptions about how anon_vma works, clashes with other series (most notably series I've been working on), takes away from efforts I want to make to start to join file-backed and anon reverse mapping logic, separates the two in such a way as to encourage this to nonly grow and generally isn't conducive to where I want to go with rmap.
This is part of why I explicitly told you please don't go down this road, because you're likely to end up doing work that doesn't get used. It's not a great use of your time either.
Since there's something useful here in tests, I may at a later date come back to those.
So in order for it not to be an _entirely_ wasted effort, I will come back to this later when the time is right and progress is made with rmap, and see if we can extract some value from the testing.
Lorenzo
Quoted 80 lines · Wei Yang
There are several
anon_vmamanipulation functions implemented inmm/rmap.c, those concerninganon_vmapreparing, cloning and forking, which logically could be stand-alone.This patch series isolates
anon_vmamanipulation functionality into its own file,mm/anon_vma.c, and provides an API to the rest of the kernel ininclude/linux/anon_vma.h.It also introduce
mm/anon_vma_internal.h, which specifies which headers need to be imported byanon_vma.c, leading to the very useful property thatanon_vma.cdepends only oninclude/linux/anon_vma.handmm/anon_vma_internal.h.This means we can then re-implement
anon_vma_internal.hin userland, adding shims for kernel mechanisms as required, allowing us to unit test internalanon_vmafunctionality.This patch series takes advantage of existing shim logic and full userland interval tree support contained in
tools/testing/rbtree/andtools/include/linux/.Kernel functionality is stubbed and shimmed as needed in
tools/testing/anon_vma/which contains a fully functional userlandanon_vma_internal.hfile and which importsmm/anon_vma.cto be directly tested from userland.Patch 1 split
anon_vmarelated logic tomm/anon_vma.cPatch 2 add a simple skeleton testing on simple fault and fork Patch 3/4 add tests for mergeable and reusableanon_vmaPatch 5 assert theanon_vmadouble-reuse is fixedWei Yang (5):
mm: moveanon_vmamanipulation functions to own file
anon_vma: add skeleton code for userland testing ofanon_vmalogic
anon_vma: add test for mergeableanon_vma
anon_vma: add test for reusableanon_vma
anon_vma: add test to assert no double-reuseMAINTAINERS | 3 + include/linux/anon_vma.h | 163 +++++ include/linux/rmap.h | 147 +--- mm/Makefile | 2 +- mm/anon_vma.c | 396 +++++++++++ mm/anon_vma_internal.h | 14 + mm/rmap.c | 391 ----------- tools/include/linux/rwsem.h | 10 + tools/include/linux/slab.h | 4 + tools/testing/anon_vma/.gitignore | 3 + tools/testing/anon_vma/Makefile | 25 + tools/testing/anon_vma/anon_vma.c | 773 +++++++++++++++++++++ tools/testing/anon_vma/anon_vma_internal.h | 88 +++ tools/testing/anon_vma/interval_tree.c | 53 ++ tools/testing/anon_vma/linux/atomic.h | 18 + tools/testing/anon_vma/linux/fs.h | 6 + tools/testing/anon_vma/linux/mm.h | 44 ++ tools/testing/anon_vma/linux/mm_types.h | 57 ++ tools/testing/anon_vma/linux/mmzone.h | 6 + tools/testing/anon_vma/linux/rmap.h | 8 + tools/testing/shared/linux/anon_vma.h | 7 + 21 files changed, 1680 insertions(+), 538 deletions(-) create mode 100644 include/linux/anon_vma.h create mode 100644 mm/anon_vma.c create mode 100644 mm/anon_vma_internal.h create mode 100644 tools/testing/anon_vma/.gitignore create mode 100644 tools/testing/anon_vma/Makefile create mode 100644 tools/testing/anon_vma/anon_vma.c create mode 100644 tools/testing/anon_vma/anon_vma_internal.h create mode 100644 tools/testing/anon_vma/interval_tree.c create mode 100644 tools/testing/anon_vma/linux/atomic.h create mode 100644 tools/testing/anon_vma/linux/fs.h create mode 100644 tools/testing/anon_vma/linux/mm.h create mode 100644 tools/testing/anon_vma/linux/mm_types.h create mode 100644 tools/testing/anon_vma/linux/mmzone.h create mode 100644 tools/testing/anon_vma/linux/rmap.h create mode 100644 tools/testing/shared/linux/anon_vma.h--
2.34.1
Quoted 16 lines · Lorenzo Stoakes
Wei,
NACK the whole series.
I'm really not sure how to get through to you. You were _explicitly_ advised not to send this series. And yet you've sent it anyway.
I mean, I appreciate your enthusiasm and the fact you've made tests here etc. obviously. And you've clearly put a TON of work in. But I just don't know why you would when explicitly told not to without at least discussing it first?
This just isn't a great way of interacting with the community. We're all human, please try to have some empathy for others here, as I really do try to have with you as best I can.
Quoted 6 lines
This adds a ton of churn and LOCKS IN assumptions about how
anon_vmaworks, clashes with other series (most notably series I've been working on), takes away from efforts I want to make to start to join file-backed and anon reverse mapping logic, separates the two in such a way as to encourage this to nonly grow and generally isn't conducive to where I want to go with rmap.
anon_vma, the unloved child. :)
I would love to see a simplification that makes it less special, and I can understand how adding tests for the ways it is special can be counter-productive.
Quoted 7 lines
This is part of why I explicitly told you please don't go down this road, because you're likely to end up doing work that doesn't get used. It's not a great use of your time either.
Since there's something useful here in tests, I may at a later date come back to those.
Agreed, skimming over the tests there are some nice diagrams and cases.
But I would hope that for most of these cases we could test on a higher level: test our expectations when running real programs that we want to check, especially when performing internal changes on how we handle anon memory + rmap.
E.g., do fork(), then test if we can successfully perform rmap lookups/updates (e.g., migrate folio to a different numa node etc).
Signature · 3 lines
Cheers, David / dhildenb
Quoted 40 lines · David Hildenbrand
Quoted 16 lines · Lorenzo Stoakes
Wei,
NACK the whole series.
I'm really not sure how to get through to you. You were _explicitly_ advised not to send this series. And yet you've sent it anyway.
I mean, I appreciate your enthusiasm and the fact you've made tests here etc. obviously. And you've clearly put a TON of work in. But I just don't know why you would when explicitly told not to without at least discussing it first?
This just isn't a great way of interacting with the community. We're all human, please try to have some empathy for others here, as I really do try to have with you as best I can.
Quoted 6 lines
This adds a ton of churn and LOCKS IN assumptions about how
anon_vmaworks, clashes with other series (most notably series I've been working on), takes away from efforts I want to make to start to join file-backed and anon reverse mapping logic, separates the two in such a way as to encourage this to nonly grow and generally isn't conducive to where I want to go with rmap.
anon_vma, the unloved child. :)I would love to see a simplification that makes it less special, and I can understand how adding tests for the ways it is special can be counter-productive.
Quoted 7 lines
This is part of why I explicitly told you please don't go down this road, because you're likely to end up doing work that doesn't get used. It's not a great use of your time either.
Since there's something useful here in tests, I may at a later date come back to those.
Agreed, skimming over the tests there are some nice diagrams and cases.
Quoted 8 lines
But I would hope that for most of these cases we could test on a higher
level: test our expectations when running real programs that we want to check, especially when performing internal changes on how we handle anon memory + rmap.E.g., do
fork(), then test if we can successfully perform rmap lookups/updates (e.g., migrate folio to a different numa node etc).
That's a great point! Wei - if you could look at making some self-tests (i.e. that live in tools/testing/selftests/mm) that try to recreate _real_ scenarios that use the rmap like this and assert correct behaviour there, that could be a positive way of moving forward with this.
We'd be absolutely happy to take patches like that!
Quoted 6 lines
--
Cheers,David / dhildenb
Cheers, Lorenzo
Quoted 46 lines · Lorenzo Stoakes
Quoted 45 lines · David Hildenbrand
Quoted 16 lines · Lorenzo Stoakes
Wei,
NACK the whole series.
I'm really not sure how to get through to you. You were _explicitly_ advised not to send this series. And yet you've sent it anyway.
I mean, I appreciate your enthusiasm and the fact you've made tests here etc. obviously. And you've clearly put a TON of work in. But I just don't know why you would when explicitly told not to without at least discussing it first?
This just isn't a great way of interacting with the community. We're all human, please try to have some empathy for others here, as I really do try to have with you as best I can.
Quoted 6 lines
This adds a ton of churn and LOCKS IN assumptions about how
anon_vmaworks, clashes with other series (most notably series I've been working on), takes away from efforts I want to make to start to join file-backed and anon reverse mapping logic, separates the two in such a way as to encourage this to nonly grow and generally isn't conducive to where I want to go with rmap.
anon_vma, the unloved child. :)I would love to see a simplification that makes it less special, and I can understand how adding tests for the ways it is special can be counter-productive.
Quoted 7 lines
This is part of why I explicitly told you please don't go down this road, because you're likely to end up doing work that doesn't get used. It's not a great use of your time either.
Since there's something useful here in tests, I may at a later date come back to those.
Agreed, skimming over the tests there are some nice diagrams and cases.
But I would hope that for most of these cases we could test on a higher
level: test our expectations when running real programs that we want to check, especially when performing internal changes on how we handle anon memory + rmap.
Quoted 9 lines
Quoted 3 lines
E.g., do
fork(), then test if we can successfully perform rmap lookups/updates (e.g., migrate folio to a different numa node etc).That's a great point! Wei - if you could look at making some self-tests (i.e. that live in
tools/testing/selftests/mm) that try to recreate _real_ scenarios that use the rmap like this and assert correct behaviour there, that could be a positive way of moving forward with this.
I am trying to understand what scenarios you want.
Something like below?
* fork and migrate a range in child
* fork/unmap in parent and migrate a range in child
If the operation is successful, then we are good, right?
Quoted 10 lines
We'd be absolutely happy to take patches like that!
Quoted 6 lines
--
Cheers,David / dhildenb
Cheers, Lorenzo
Signature · 2 lines
Wei Yang Help you, Help me
Quoted 51 lines · Wei Yang
Quoted 50 lines · Lorenzo Stoakes
Quoted 48 lines · David Hildenbrand
Quoted 16 lines · Lorenzo Stoakes
Wei,
NACK the whole series.
I'm really not sure how to get through to you. You were _explicitly_ advised not to send this series. And yet you've sent it anyway.
I mean, I appreciate your enthusiasm and the fact you've made tests here etc. obviously. And you've clearly put a TON of work in. But I just don't know why you would when explicitly told not to without at least discussing it first?
This just isn't a great way of interacting with the community. We're all human, please try to have some empathy for others here, as I really do try to have with you as best I can.
Quoted 6 lines
This adds a ton of churn and LOCKS IN assumptions about how
anon_vmaworks, clashes with other series (most notably series I've been working on), takes away from efforts I want to make to start to join file-backed and anon reverse mapping logic, separates the two in such a way as to encourage this to nonly grow and generally isn't conducive to where I want to go with rmap.
anon_vma, the unloved child. :)I would love to see a simplification that makes it less special, and I can understand how adding tests for the ways it is special can be counter-productive.
Quoted 7 lines
This is part of why I explicitly told you please don't go down this road, because you're likely to end up doing work that doesn't get used. It's not a great use of your time either.
Since there's something useful here in tests, I may at a later date come back to those.
Agreed, skimming over the tests there are some nice diagrams and cases.
But I would hope that for most of these cases we could test on a higher
level: test our expectations when running real programs that we want to check, especially when performing internal changes on how we handle anon memory + rmap.E.g., do
fork(), then test if we can successfully perform rmap lookups/updates (e.g., migrate folio to a different numa node etc).
Quoted 7 lines
Quoted 5 lines
That's a great point! Wei - if you could look at making some self-tests (i.e. that live in
tools/testing/selftests/mm) that try to recreate _real_ scenarios that use the rmap like this and assert correct behaviour there, that could be a positive way of moving forward with this.I am trying to understand what scenarios you want.
That is exactly the task to figure out: how can we actually test our rmap implementation from a higher level. The example regarding fork and migration is possibly a low-hanging fruit.
We might already have the functionality to achieve it, *maybe* we'd even want some extensions to make it all even easier to test.
For example, MADV_PAGEOUT is refused on folios that are mapped into multiple processes. Maybe we'd want the option to *still* page it out, just like MPOL_MF_MOVE_ALL allows with CAP_SYS_NICE to *still* migrate a folio that is mapped into multiple processes.
Some rmap tests could make sense for both, anon and pagecache folios.
Quoted 7 lines
Something like below?
* fork and migrate a range in child
* fork/unmap in parent and migrate a range in childIf the operation is successful, then we are good, right?
Yes. And one can come up with a bunch of similar rmap test cases, like doing a partial mremap() of a THP, then testing if the rmap walk still works as expected, pairing the whole thing with for etc.
One "problem" here is that even with MPOL_MF_MOVE_ALL,move_pages() will not move a folio if it already resides on the target node. So one always needs two NUMA nodes, which is a bit suboptimal for testing purposes.
For testing purposes, it could have been helpful a couple of times already to just have a way of migrating a folio even if it already resides on the expected node.
Signature · 3 lines
Cheers, David / dhildenb
Quoted 86 lines · David Hildenbrand
Quoted 51 lines · Wei Yang
Quoted 50 lines · Lorenzo Stoakes
Quoted 48 lines · David Hildenbrand
Quoted 16 lines · Lorenzo Stoakes
Wei,
NACK the whole series.
I'm really not sure how to get through to you. You were _explicitly_ advised not to send this series. And yet you've sent it anyway.
I mean, I appreciate your enthusiasm and the fact you've made tests here etc. obviously. And you've clearly put a TON of work in. But I just don't know why you would when explicitly told not to without at least discussing it first?
This just isn't a great way of interacting with the community. We're all human, please try to have some empathy for others here, as I really do try to have with you as best I can.
Quoted 6 lines
This adds a ton of churn and LOCKS IN assumptions about how
anon_vmaworks, clashes with other series (most notably series I've been working on), takes away from efforts I want to make to start to join file-backed and anon reverse mapping logic, separates the two in such a way as to encourage this to nonly grow and generally isn't conducive to where I want to go with rmap.
anon_vma, the unloved child. :)I would love to see a simplification that makes it less special, and I can understand how adding tests for the ways it is special can be counter-productive.
Quoted 7 lines
This is part of why I explicitly told you please don't go down this road, because you're likely to end up doing work that doesn't get used. It's not a great use of your time either.
Since there's something useful here in tests, I may at a later date come back to those.
Agreed, skimming over the tests there are some nice diagrams and cases.
But I would hope that for most of these cases we could test on a higher
level: test our expectations when running real programs that we want to check, especially when performing internal changes on how we handle anon memory + rmap.E.g., do
fork(), then test if we can successfully perform rmap lookups/updates (e.g., migrate folio to a different numa node etc).Quoted 7 lines
Quoted 5 lines
That's a great point! Wei - if you could look at making some self-tests (i.e. that live in
tools/testing/selftests/mm) that try to recreate _real_ scenarios that use the rmap like this and assert correct behaviour there, that could be a positive way of moving forward with this.I am trying to understand what scenarios you want.
That is exactly the task to figure out: how can we actually test our rmap implementation from a higher level. The example regarding fork and migration is possibly a low-hanging fruit.
We might already have the functionality to achieve it, *maybe* we'd even want some extensions to make it all even easier to test.
For example,
MADV_PAGEOUTis refused on folios that are mapped into multiple processes. Maybe we'd want the option to *still* page it out, just likeMPOL_MF_MOVE_ALLallows withCAP_SYS_NICEto *still* migrate a folio that is mapped into multiple processes.Some rmap tests could make sense for both, anon and pagecache folios.
Quoted 7 lines
Something like below?
* fork and migrate a range in child
* fork/unmap in parent and migrate a range in childIf the operation is successful, then we are good, right?
Yes. And one can come up with a bunch of similar rmap test cases, like doing a partial
mremap()of a THP, then testing if the rmap walk still works as expected, pairing the whole thing with for etc.
Quoted 9 lines
One "problem" here is that even with
MPOL_MF_MOVE_ALL,move_pages()will not move a folio if it already resides on the target node. So one always needs two NUMA nodes, which is a bit suboptimal for testing purposes.For testing purposes, it could have been helpful a couple of times already to just have a way of migrating a folio even if it already resides on the expected node.
Thanks for all those detail explanation. I need some time to digest it.
Since lack of some background knowledge, I may have further questions on this. Hope won't bother you too much.
Quoted 4 lines
Signature · 3 lines
Cheers, David / dhildenb
Signature · 2 lines
Wei Yang Help you, Help me
Quoted 97 lines · Wei Yang
Quoted 86 lines · David Hildenbrand
Quoted 51 lines · Wei Yang
Quoted 50 lines · Lorenzo Stoakes
Quoted 48 lines · David Hildenbrand
Quoted 16 lines · Lorenzo Stoakes
Wei,
NACK the whole series.
I'm really not sure how to get through to you. You were _explicitly_ advised not to send this series. And yet you've sent it anyway.
I mean, I appreciate your enthusiasm and the fact you've made tests here etc. obviously. And you've clearly put a TON of work in. But I just don't know why you would when explicitly told not to without at least discussing it first?
This just isn't a great way of interacting with the community. We're all human, please try to have some empathy for others here, as I really do try to have with you as best I can.
Quoted 6 lines
This adds a ton of churn and LOCKS IN assumptions about how
anon_vmaworks, clashes with other series (most notably series I've been working on), takes away from efforts I want to make to start to join file-backed and anon reverse mapping logic, separates the two in such a way as to encourage this to nonly grow and generally isn't conducive to where I want to go with rmap.
anon_vma, the unloved child. :)I would love to see a simplification that makes it less special, and I can understand how adding tests for the ways it is special can be counter-productive.
Quoted 7 lines
This is part of why I explicitly told you please don't go down this road, because you're likely to end up doing work that doesn't get used. It's not a great use of your time either.
Since there's something useful here in tests, I may at a later date come back to those.
Agreed, skimming over the tests there are some nice diagrams and cases.
But I would hope that for most of these cases we could test on a higher
level: test our expectations when running real programs that we want to check, especially when performing internal changes on how we handle anon memory + rmap.E.g., do
fork(), then test if we can successfully perform rmap lookups/updates (e.g., migrate folio to a different numa node etc).Quoted 7 lines
Quoted 5 lines
That's a great point! Wei - if you could look at making some self-tests (i.e. that live in
tools/testing/selftests/mm) that try to recreate _real_ scenarios that use the rmap like this and assert correct behaviour there, that could be a positive way of moving forward with this.I am trying to understand what scenarios you want.
That is exactly the task to figure out: how can we actually test our rmap implementation from a higher level. The example regarding fork and migration is possibly a low-hanging fruit.
We might already have the functionality to achieve it, *maybe* we'd even want some extensions to make it all even easier to test.
For example,
MADV_PAGEOUTis refused on folios that are mapped into multiple processes. Maybe we'd want the option to *still* page it out, just likeMPOL_MF_MOVE_ALLallows withCAP_SYS_NICEto *still* migrate a folio that is mapped into multiple processes.Some rmap tests could make sense for both, anon and pagecache folios.
Quoted 7 lines
Something like below?
* fork and migrate a range in child
* fork/unmap in parent and migrate a range in childIf the operation is successful, then we are good, right?
Yes. And one can come up with a bunch of similar rmap test cases, like doing a partial
mremap()of a THP, then testing if the rmap walk still works as expected, pairing the whole thing with for etc.Quoted 9 lines
One "problem" here is that even with
MPOL_MF_MOVE_ALL,move_pages()will not move a folio if it already resides on the target node. So one always needs two NUMA nodes, which is a bit suboptimal for testing purposes.For testing purposes, it could have been helpful a couple of times already to just have a way of migrating a folio even if it already resides on the expected node.
Quoted 4 lines
Thanks for all those detail explanation. I need some time to digest it.
Since lack of some background knowledge, I may have further questions on this. Hope won't bother you too much.
Sure, feel free to reach out. Having more selftests that test rmap behavior could be really helpful.
Signature · 3 lines
Cheers, David / dhildenb
On Wed, Apr 30, 2025 at 09:47:16AM +0200, David Hildenbrand wrote:
[...]
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 10 lines
Agreed, skimming over the tests there are some nice diagrams and cases.
But I would hope that for most of these cases we could test on a higher
level: test our expectations when running real programs that we want to check, especially when performing internal changes on how we handle anon memory + rmap.E.g., do
fork(), then test if we can successfully perform rmap lookups/updates (e.g., migrate folio to a different numa node etc).
Quoted 8 lines
Quoted 7 lines
Quoted 5 lines
That's a great point! Wei - if you could look at making some self-tests (i.e. that live in
tools/testing/selftests/mm) that try to recreate _real_ scenarios that use the rmap like this and assert correct behaviour there, that could be a positive way of moving forward with this.I am trying to understand what scenarios you want.
Sorry for the late reply, I handled other things a while.
Quoted 3 lines
That is exactly the task to figure out: how can we actually test our rmap implementation from a higher level. The example regarding fork and migration is possibly a low-hanging fruit.
If my understanding is correct, you suggested two high level way:
1. fork + migrate (move_pages)
Quoted 9 lines
We might already have the functionality to achieve it, *maybe* we'd even want some extensions to make it all even easier to test.
For example,
MADV_PAGEOUTis refused on folios that are mapped into multiple processes. Maybe we'd want the option to *still* page it out, just likeMPOL_MF_MOVE_ALLallows withCAP_SYS_NICEto *still* migrate a folio that is mapped into multiple processes.
2. madvise(MADV_PAGEOUT)
Not fully get it here. You mean fork + madvise(MADV_PAGEOUT) + migrate ?
But we need to enable pageout in this way first.
I am not sure why this one is easier way to test. Would you mind sharing more idea on this?
Quoted 5 lines
Some rmap tests could make sense for both, anon and pagecache folios.
Quoted 3 lines
Something like below?
Quoted 9 lines
Quoted 4 lines
* fork and migrate a range in child
* fork/unmap in parent and migrate a range in childIf the operation is successful, then we are good, right?
Yes. And one can come up with a bunch of similar rmap test cases, like doing a partial
mremap()of a THP, then testing if the rmap walk still works as expected, pairing the whole thing with for etc.
For both way, we could arrange all those scenarios and also do partial mremap() during it.
Quoted 9 lines
One "problem" here is that even with
MPOL_MF_MOVE_ALL,move_pages()will not move a folio if it already resides on the target node. So one always needs two NUMA nodes, which is a bit suboptimal for testing purposes.For testing purposes, it could have been helpful a couple of times already to just have a way of migrating a folio even if it already resides on the expected node.
This looks we need a new flag for it?
Here is my plan if my understanding is correct.
1. Add test cases for fork + migrate. We may limit it only works on machine with 2 NUMA nodes.
2. Enable move_pages() on local node, then remove the test limitation
3. Enable madvise(MADV_PAGEOUT) with multiple mapping, then add related cases
4. Add mremap() or other cases
In general, to verify rmap dose the work correctly, my idea is to
* mmap(MAP_SHARED)
* write some initial data before fork
* after fork and migrate, we write some different data to it
* if each process do see the new data, rmap is good.
Does it sound good to you?
Quoted 4 lines
Signature · 3 lines
Cheers, David / dhildenb
Signature · 2 lines
Wei Yang Help you, Help me
Quoted 19 lines · Wei Yang
On Wed, Apr 30, 2025 at 09:47:16AM +0200, David Hildenbrand wrote:
[...]Quoted 17 lines
Quoted 17 lines
Quoted 16 lines
Quoted 10 lines
Agreed, skimming over the tests there are some nice diagrams and cases.
But I would hope that for most of these cases we could test on a higher
level: test our expectations when running real programs that we want to check, especially when performing internal changes on how we handle anon memory + rmap.E.g., do
fork(), then test if we can successfully perform rmap lookups/updates (e.g., migrate folio to a different numa node etc).That's a great point! Wei - if you could look at making some self-tests (i.e. that live in
tools/testing/selftests/mm) that try to recreate _real_ scenarios that use the rmap like this and assert correct behaviour there, that could be a positive way of moving forward with this.
Ping
Quoted 87 lines
Quoted 2 lines
Quoted 1 line
I am trying to understand what scenarios you want.
Sorry for the late reply, I handled other things a while.
Quoted 3 lines
That is exactly the task to figure out: how can we actually test our rmap implementation from a higher level. The example regarding fork and migration is possibly a low-hanging fruit.
If my understanding is correct, you suggested two high level way:
1. fork + migrate (
move_pages)Quoted 9 lines
We might already have the functionality to achieve it, *maybe* we'd even want some extensions to make it all even easier to test.
For example,
MADV_PAGEOUTis refused on folios that are mapped into multiple processes. Maybe we'd want the option to *still* page it out, just likeMPOL_MF_MOVE_ALLallows withCAP_SYS_NICEto *still* migrate a folio that is mapped into multiple processes.2.
madvise(MADV_PAGEOUT)Not fully get it here. You mean fork +
madvise(MADV_PAGEOUT)+ migrate ?But we need to enable pageout in this way first.
I am not sure why this one is easier way to test. Would you mind sharing more idea on this?
Quoted 5 lines
Some rmap tests could make sense for both, anon and pagecache folios.
Quoted 3 lines
Something like below?
Quoted 9 lines
Quoted 4 lines
* fork and migrate a range in child
* fork/unmap in parent and migrate a range in childIf the operation is successful, then we are good, right?
Yes. And one can come up with a bunch of similar rmap test cases, like doing a partial
mremap()of a THP, then testing if the rmap walk still works as expected, pairing the whole thing with for etc.For both way, we could arrange all those scenarios and also do partial
mremap()during it.Quoted 9 lines
One "problem" here is that even with
MPOL_MF_MOVE_ALL,move_pages()will not move a folio if it already resides on the target node. So one always needs two NUMA nodes, which is a bit suboptimal for testing purposes.For testing purposes, it could have been helpful a couple of times already to just have a way of migrating a folio even if it already resides on the expected node.
This looks we need a new flag for it?
Here is my plan if my understanding is correct.
1. Add test cases for fork + migrate. We may limit it only works on machine with 2 NUMA nodes.
2. Enablemove_pages()on local node, then remove the test limitation
3. Enablemadvise(MADV_PAGEOUT)with multiple mapping, then add related cases
4. Addmremap()or other casesIn general, to verify rmap dose the work correctly, my idea is to
*
mmap(MAP_SHARED)
* write some initial data before fork
* after fork and migrate, we write some different data to it
* if each process do see the new data, rmap is good.Does it sound good to you?
Quoted 4 lines
Signature · 3 lines
Cheers, David / dhildenbSignature · 2 lines
Wei Yang Help you, Help me
Signature · 2 lines
Wei Yang Help you, Help me
Quoted 14 lines · Wei Yang
Quoted 13 lines · Wei Yang
On Wed, Apr 30, 2025 at 09:47:16AM +0200, David Hildenbrand wrote:
[...]Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 10 lines
Agreed, skimming over the tests there are some nice diagrams and cases.
But I would hope that for most of these cases we could test on a higher
level: test our expectations when running real programs that we want to check, especially when performing internal changes on how we handle anon memory + rmap.E.g., do
fork(), then test if we can successfully perform rmap lookups/updates (e.g., migrate folio to a different numa node etc).
Quoted 8 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
That's a great point! Wei - if you could look at making some self-tests (i.e. that live in
tools/testing/selftests/mm) that try to recreate _real_ scenarios that use the rmap like this and assert correct behaviour there, that could be a positive way of moving forward with this.Ping
Thanks for reminding me and sorry for the late reply.
Quoted 6 lines
Quoted 6 lines
Quoted 3 lines
Quoted 2 lines
I am trying to understand what scenarios you want.
Sorry for the late reply, I handled other things a while.
Quoted 7 lines
Quoted 7 lines
Quoted 3 lines
That is exactly the task to figure out: how can we actually test our rmap implementation from a higher level. The example regarding fork and migration is possibly a low-hanging fruit.
If my understanding is correct, you suggested two high level way:
1. fork + migrate (
move_pages)
Yes, that should be one way of testing it. We could even get multiple child processes into place.
Quoted 11 lines
Quoted 11 lines
Quoted 10 lines
We might already have the functionality to achieve it, *maybe* we'd even want some extensions to make it all even easier to test.
For example,
MADV_PAGEOUTis refused on folios that are mapped into multiple processes. Maybe we'd want the option to *still* page it out, just likeMPOL_MF_MOVE_ALLallows withCAP_SYS_NICEto *still* migrate a folio that is mapped into multiple processes.
Quoted 3 lines
Quoted 3 lines
2.
madvise(MADV_PAGEOUT)Not fully get it here. You mean fork +
madvise(MADV_PAGEOUT)+ migrate ?
fork + madvise(MADV_PAGEOUT) only.
Then verify, that it was actually paged out.
For example, we could have 10 processes mapping the page, then call madvise(MADV_PAGEOUT) from one process. We can verify in all processes if the page is actually no longer mapped using /proc/self/pagemap
We could likely do it with anon pages (swap), shmem pages (swap) and pagecache pages (file backed).
... and possibly even with KSM pages!
Quoted 2 lines
Quoted 2 lines
But we need to enable pageout in this way first.
Yes, madvise(MADV_PAGEOUT) needs to be extended to allow forcing a pageout (e.g., a new MADVISE_PAGEOUT_SHARED that only works with with CAP_SYY_ADMIN).
Quoted 3 lines
Quoted 3 lines
I am not sure why this one is easier way to test. Would you mind sharing more idea on this?
It might be easier than the migration case, because for migration we currently need 2 NUMA nodes ...
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Some rmap tests could make sense for both, anon and pagecache folios.
Quoted 7 lines
Something like below?
* fork and migrate a range in child
* fork/unmap in parent and migrate a range in childIf the operation is successful, then we are good, right?
Quoted 7 lines
Quoted 7 lines
Quoted 4 lines
Yes. And one can come up with a bunch of similar rmap test cases, like doing a partial
mremap()of a THP, then testing if the rmap walk still works as expected, pairing the whole thing with for etc.For both way, we could arrange all those scenarios and also do partial
mremap()during it.
Exactly.
Quoted 21 lines
Quoted 21 lines
Quoted 10 lines
One "problem" here is that even with
MPOL_MF_MOVE_ALL,move_pages()will not move a folio if it already resides on the target node. So one always needs two NUMA nodes, which is a bit suboptimal for testing purposes.For testing purposes, it could have been helpful a couple of times already to just have a way of migrating a folio even if it already resides on the expected node.
This looks we need a new flag for it?
Here is my plan if my understanding is correct.
1. Add test cases for fork + migrate. We may limit it only works on machine with 2 NUMA nodes.
2. Enablemove_pages()on local node, then remove the test limitation
3. Enablemadvise(MADV_PAGEOUT)with multiple mapping, then add related cases
4. Addmremap()or other cases
Quoted 5 lines
Quoted 5 lines
In general, to verify rmap dose the work correctly, my idea is to
*
mmap(MAP_SHARED)
* write some initial data before fork>> * after fork and migrate, we write some different data to it
* if each process do see the new data, rmap is good.
With pageout (see above) test, we can just verify using /proc/self/pagemap in all processes whether the page was paged out (IOW: the rmap was able to identify all page locations). And that should work with anon/shmem/file/ksm pages IIUC.
Quoted 2 lines
Quoted 2 lines
Does it sound good to you?
Yes, that absolutely goes into the right direction. Having also rmap tests for KSM as raised above might be a real benefit.
Signature · 3 lines
Cheers, David / dhildenb
Quoted 58 lines · David Hildenbrand
Quoted 14 lines · Wei Yang
Quoted 13 lines · Wei Yang
On Wed, Apr 30, 2025 at 09:47:16AM +0200, David Hildenbrand wrote:
[...]Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 10 lines
Agreed, skimming over the tests there are some nice diagrams and cases.
But I would hope that for most of these cases we could test on a higher
level: test our expectations when running real programs that we want to check, especially when performing internal changes on how we handle anon memory + rmap.E.g., do
fork(), then test if we can successfully perform rmap lookups/updates (e.g., migrate folio to a different numa node etc).Quoted 8 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
That's a great point! Wei - if you could look at making some self-tests (i.e. that live in
tools/testing/selftests/mm) that try to recreate _real_ scenarios that use the rmap like this and assert correct behaviour there, that could be a positive way of moving forward with this.Ping
Thanks for reminding me and sorry for the late reply.
Quoted 6 lines
Quoted 6 lines
Quoted 3 lines
Quoted 2 lines
I am trying to understand what scenarios you want.
Sorry for the late reply, I handled other things a while.
Quoted 7 lines
Quoted 7 lines
Quoted 3 lines
That is exactly the task to figure out: how can we actually test our rmap implementation from a higher level. The example regarding fork and migration is possibly a low-hanging fruit.
If my understanding is correct, you suggested two high level way:
1. fork + migrate (
move_pages)Yes, that should be one way of testing it. We could even get multiple child processes into place.
Quoted 11 lines
Quoted 11 lines
Quoted 10 lines
We might already have the functionality to achieve it, *maybe* we'd even want some extensions to make it all even easier to test.
For example,
MADV_PAGEOUTis refused on folios that are mapped into multiple processes. Maybe we'd want the option to *still* page it out, just likeMPOL_MF_MOVE_ALLallows withCAP_SYS_NICEto *still* migrate a folio that is mapped into multiple processes.Quoted 3 lines
Quoted 3 lines
2.
madvise(MADV_PAGEOUT)Not fully get it here. You mean fork +
madvise(MADV_PAGEOUT)+ migrate ?
Quoted 7 lines
fork +
madvise(MADV_PAGEOUT)only.Then verify, that it was actually paged out.
For example, we could have 10 processes mapping the page, then call
madvise(MADV_PAGEOUT)from one process. We can verify in all processes if the page is actually no longer mapped using/proc/self/pagemap
Got it, thanks.
Will prepare the test cases.
Quoted 84 lines
We could likely do it with anon pages (swap), shmem pages (swap) and pagecache pages (file backed).
... and possibly even with KSM pages!
Quoted 2 lines
Quoted 2 lines
But we need to enable pageout in this way first.
Yes,
madvise(MADV_PAGEOUT)needs to be extended to allow forcing a pageout (e.g., a newMADVISE_PAGEOUT_SHAREDthat only works with withCAP_SYY_ADMIN).Quoted 3 lines
Quoted 3 lines
I am not sure why this one is easier way to test. Would you mind sharing more idea on this?
It might be easier than the migration case, because for migration we currently need 2 NUMA nodes ...
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Some rmap tests could make sense for both, anon and pagecache folios.
Quoted 7 lines
Something like below?
* fork and migrate a range in child
* fork/unmap in parent and migrate a range in childIf the operation is successful, then we are good, right?
Quoted 7 lines
Quoted 7 lines
Quoted 4 lines
Yes. And one can come up with a bunch of similar rmap test cases, like doing a partial
mremap()of a THP, then testing if the rmap walk still works as expected, pairing the whole thing with for etc.For both way, we could arrange all those scenarios and also do partial
mremap()during it.Exactly.
Quoted 21 lines
Quoted 21 lines
Quoted 10 lines
One "problem" here is that even with
MPOL_MF_MOVE_ALL,move_pages()will not move a folio if it already resides on the target node. So one always needs two NUMA nodes, which is a bit suboptimal for testing purposes.For testing purposes, it could have been helpful a couple of times already to just have a way of migrating a folio even if it already resides on the expected node.
This looks we need a new flag for it?
Here is my plan if my understanding is correct.
1. Add test cases for fork + migrate. We may limit it only works on machine with 2 NUMA nodes.
2. Enablemove_pages()on local node, then remove the test limitation
3. Enablemadvise(MADV_PAGEOUT)with multiple mapping, then add related cases
4. Addmremap()or other casesQuoted 4 lines
Quoted 4 lines
In general, to verify rmap dose the work correctly, my idea is to
*
mmap(MAP_SHARED)
* write some initial data before fork>> * after fork and migrate, wewrite some different data to it
Quoted 1 line
Quoted 1 line
* if each process do see the new data, rmap is good.
With pageout (see above) test, we can just verify using
/proc/self/pagemapin all processes whether the page was paged out (IOW: the rmap was able to identify all page locations). And that should work with anon/shmem/file/ksm pages IIUC.Quoted 2 lines
Quoted 2 lines
Does it sound good to you?
Yes, that absolutely goes into the right direction. Having also rmap tests for KSM as raised above might be a real benefit.
Signature · 3 lines
Cheers, David / dhildenb
Signature · 2 lines
Wei Yang Help you, Help me
Quoted 58 lines · David Hildenbrand
Quoted 14 lines · Wei Yang
Quoted 13 lines · Wei Yang
On Wed, Apr 30, 2025 at 09:47:16AM +0200, David Hildenbrand wrote:
[...]Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 10 lines
Agreed, skimming over the tests there are some nice diagrams and cases.
But I would hope that for most of these cases we could test on a higher
level: test our expectations when running real programs that we want to check, especially when performing internal changes on how we handle anon memory + rmap.E.g., do
fork(), then test if we can successfully perform rmap lookups/updates (e.g., migrate folio to a different numa node etc).Quoted 8 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
That's a great point! Wei - if you could look at making some self-tests (i.e. that live in
tools/testing/selftests/mm) that try to recreate _real_ scenarios that use the rmap like this and assert correct behaviour there, that could be a positive way of moving forward with this.Ping
Thanks for reminding me and sorry for the late reply.
Quoted 6 lines
Quoted 6 lines
Quoted 3 lines
Quoted 2 lines
I am trying to understand what scenarios you want.
Sorry for the late reply, I handled other things a while.
Quoted 7 lines
Quoted 7 lines
Quoted 3 lines
That is exactly the task to figure out: how can we actually test our rmap implementation from a higher level. The example regarding fork and migration is possibly a low-hanging fruit.
If my understanding is correct, you suggested two high level way:
1. fork + migrate (
move_pages)Yes, that should be one way of testing it. We could even get multiple child processes into place.
Quoted 11 lines
Quoted 11 lines
Quoted 10 lines
We might already have the functionality to achieve it, *maybe* we'd even want some extensions to make it all even easier to test.
For example,
MADV_PAGEOUTis refused on folios that are mapped into multiple processes. Maybe we'd want the option to *still* page it out, just likeMPOL_MF_MOVE_ALLallows withCAP_SYS_NICEto *still* migrate a folio that is mapped into multiple processes.Quoted 3 lines
Quoted 3 lines
2.
madvise(MADV_PAGEOUT)Not fully get it here. You mean fork +
madvise(MADV_PAGEOUT)+ migrate ?
Quoted 8 lines
fork +
madvise(MADV_PAGEOUT)only.Then verify, that it was actually paged out.
For example, we could have 10 processes mapping the page, then call
madvise(MADV_PAGEOUT)from one process. We can verify in all processes if the page is actually no longer mapped using/proc/self/pagemap
Hi, David
I'd like to clarify some detail here.
Quoted 3 lines
We could likely do it with anon pages (swap), shmem pages (swap) and pagecache pages (file backed).
I could understand the three cases here, but for the (swap) case, it is not clear to me. You mean we force the page to be swapped out before migration? I may not have an idea to ensure that.
Quoted 2 lines
... and possibly even with KSM pages!
For KSM, it looks like an general background routine. I guess you want something like:
* fork process tree
* do KSM and wait for it to finish?
* migrate
I guess the KSM duration would related to the total memory on the system. The bigger, the longer. While the selftest generally have timeout setting. Not sure it would be suitable for all situation.
Would you mind sharing more on KSM case?
Signature · 2 lines
Wei Yang Help you, Help me
Quoted 61 lines · Wei Yang
Quoted 60 lines · David Hildenbrand
Quoted 14 lines · Wei Yang
Quoted 13 lines · Wei Yang
On Wed, Apr 30, 2025 at 09:47:16AM +0200, David Hildenbrand wrote:
[...]Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 10 lines
Agreed, skimming over the tests there are some nice diagrams and cases.
But I would hope that for most of these cases we could test on a higher
level: test our expectations when running real programs that we want to check, especially when performing internal changes on how we handle anon memory + rmap.E.g., do
fork(), then test if we can successfully perform rmap lookups/updates (e.g., migrate folio to a different numa node etc).Quoted 8 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
That's a great point! Wei - if you could look at making some self-tests (i.e. that live in
tools/testing/selftests/mm) that try to recreate _real_ scenarios that use the rmap like this and assert correct behaviour there, that could be a positive way of moving forward with this.Ping
Thanks for reminding me and sorry for the late reply.
Quoted 6 lines
Quoted 6 lines
Quoted 3 lines
Quoted 2 lines
I am trying to understand what scenarios you want.
Sorry for the late reply, I handled other things a while.
Quoted 7 lines
Quoted 7 lines
Quoted 3 lines
That is exactly the task to figure out: how can we actually test our rmap implementation from a higher level. The example regarding fork and migration is possibly a low-hanging fruit.
If my understanding is correct, you suggested two high level way:
1. fork + migrate (
move_pages)Yes, that should be one way of testing it. We could even get multiple child processes into place.
Quoted 11 lines
Quoted 11 lines
Quoted 10 lines
We might already have the functionality to achieve it, *maybe* we'd even want some extensions to make it all even easier to test.
For example,
MADV_PAGEOUTis refused on folios that are mapped into multiple processes. Maybe we'd want the option to *still* page it out, just likeMPOL_MF_MOVE_ALLallows withCAP_SYS_NICEto *still* migrate a folio that is mapped into multiple processes.Quoted 3 lines
Quoted 3 lines
2.
madvise(MADV_PAGEOUT)Not fully get it here. You mean fork +
madvise(MADV_PAGEOUT)+ migrate ?fork +
madvise(MADV_PAGEOUT)only.
Quoted 8 lines
Quoted 6 lines
Then verify, that it was actually paged out.
For example, we could have 10 processes mapping the page, then call
madvise(MADV_PAGEOUT)from one process. We can verify in all processes if the page is actually no longer mapped using/proc/self/pagemapHi, David
Hi!
Quoted 10 lines
I'd like to clarify some detail here.
Quoted 3 lines
We could likely do it with anon pages (swap), shmem pages (swap) and pagecache pages (file backed).
I could understand the three cases here, but for the (swap) case, it is not clear to me. You mean we force the page to be swapped out before migration? I may not have an idea to ensure that.
No need to involve migration for that test. It's sufficient to trigger pageout to then check if the page was unmapped in all other processes.
Quoted 9 lines
Quoted 3 lines
... and possibly even with KSM pages!
For KSM, it looks like an general background routine. I guess you want something like:
* fork process tree
* do KSM and wait for it to finish?
Probably allocate memory in each process and fill it with the same value.
Then wait for KSM to deduplciate.
Quoted 1 line
* migrate
Yes, or instead of migrate, trigger pageout from one process, then check if pageout in all other processes (/proc/self/pagemap)
Quoted 5 lines
Quoted 1 line
I guess the KSM duration would related to the total memory on the system. The
bigger, the longer. While the selftest generally have timeout setting. Not sure it would be suitable for all situation.
Would you mind sharing more on KSM case?
Worth looking at tools/testing/selftests/mm/ksm_functional_tests.c (and the other KSM selftests)
In particular, in ksm_merge() we can detect whether KSM completed the scan when two full scans passed.
A lot of that functionality could probably be factored out into vm_utils.c to be reused in other tests.
Signature · 3 lines
Cheers, David / dhildenb
On Fri, May 30, 2025 at 10:00:18AM +0200, David Hildenbrand wrote:
[...]
Quoted 55 lines
Quoted 37 lines
Quoted 37 lines
Thanks for reminding me and sorry for the late reply.
Quoted 6 lines
Quoted 6 lines
Quoted 3 lines
Quoted 2 lines
I am trying to understand what scenarios you want.
Sorry for the late reply, I handled other things a while.
Quoted 7 lines
Quoted 7 lines
Quoted 3 lines
That is exactly the task to figure out: how can we actually test our rmap implementation from a higher level. The example regarding fork and migration is possibly a low-hanging fruit.
If my understanding is correct, you suggested two high level way:
1. fork + migrate (
move_pages)Yes, that should be one way of testing it. We could even get multiple child processes into place.
Quoted 11 lines
Quoted 11 lines
Quoted 10 lines
We might already have the functionality to achieve it, *maybe* we'd even want some extensions to make it all even easier to test.
For example,
MADV_PAGEOUTis refused on folios that are mapped into multiple processes. Maybe we'd want the option to *still* page it out, just likeMPOL_MF_MOVE_ALLallows withCAP_SYS_NICEto *still* migrate a folio that is mapped into multiple processes.Quoted 3 lines
Quoted 3 lines
2.
madvise(MADV_PAGEOUT)Not fully get it here. You mean fork +
madvise(MADV_PAGEOUT)+ migrate ?fork +
madvise(MADV_PAGEOUT)only.Quoted 8 lines
Quoted 6 lines
Then verify, that it was actually paged out.
For example, we could have 10 processes mapping the page, then call
madvise(MADV_PAGEOUT)from one process. We can verify in all processes if the page is actually no longer mapped using/proc/self/pagemapHi, David
Hi!
Quoted 7 lines
I'd like to clarify some detail here.
Quoted 3 lines
We could likely do it with anon pages (swap), shmem pages (swap) and pagecache pages (file backed).
Quoted 7 lines
Quoted 3 lines
I could understand the three cases here, but for the (swap) case, it is not clear to me. You mean we force the page to be swapped out before migration? I may not have an idea to ensure that.
No need to involve migration for that test. It's sufficient to trigger pageout to then check if the page was unmapped in all other processes.
Here you mean for all anon/shmem/pagecache pages, we could have two category tests:
* migrate and write different data from on process, verify each process see new data
* trigger pageout from one process, verify each process has it pageout (/proc/self/pagemap)
While one of these category is enough, right? Just like KSM below, migrate or pageout.
Quoted 20 lines
Quoted 9 lines
Quoted 3 lines
... and possibly even with KSM pages!
For KSM, it looks like an general background routine. I guess you want something like:
* fork process tree
* do KSM and wait for it to finish?Probably allocate memory in each process and fill it with the same value.
Then wait for KSM to deduplciate.
Quoted 1 line
* migrate
Yes, or instead of migrate, trigger pageout from one process, then check if pageout in all other processes (
/proc/self/pagemap)Quoted 1 line
Quoted 1 line
I guess the KSM duration would related to the total memory on the system.
Quoted 8 lines
The
Quoted 4 lines
bigger, the longer. While the selftest generally have timeout setting. Not sure it would be suitable for all situation.
Would you mind sharing more on KSM case?
Worth looking at
tools/testing/selftests/mm/ksm_functional_tests.c(and the other KSM selftests)
Thanks, I would take a look into this first.
Quoted 11 lines
In particular, in
ksm_merge()we can detect whether KSM completed the scan when two full scans passed.A lot of that functionality could probably be factored out into
vm_utils.cto be reused in other tests.Signature · 3 lines
Cheers, David / dhildenb
Signature · 2 lines
Wei Yang Help you, Help me
Quoted 68 lines · Wei Yang
On Fri, May 30, 2025 at 10:00:18AM +0200, David Hildenbrand wrote:
[...]Quoted 55 lines
Quoted 37 lines
Quoted 37 lines
Thanks for reminding me and sorry for the late reply.
Quoted 6 lines
Quoted 6 lines
Quoted 3 lines
Quoted 2 lines
I am trying to understand what scenarios you want.
Sorry for the late reply, I handled other things a while.
Quoted 7 lines
Quoted 7 lines
Quoted 3 lines
That is exactly the task to figure out: how can we actually test our rmap implementation from a higher level. The example regarding fork and migration is possibly a low-hanging fruit.
If my understanding is correct, you suggested two high level way:
1. fork + migrate (
move_pages)Yes, that should be one way of testing it. We could even get multiple child processes into place.
Quoted 11 lines
Quoted 11 lines
Quoted 10 lines
We might already have the functionality to achieve it, *maybe* we'd even want some extensions to make it all even easier to test.
For example,
MADV_PAGEOUTis refused on folios that are mapped into multiple processes. Maybe we'd want the option to *still* page it out, just likeMPOL_MF_MOVE_ALLallows withCAP_SYS_NICEto *still* migrate a folio that is mapped into multiple processes.Quoted 3 lines
Quoted 3 lines
2.
madvise(MADV_PAGEOUT)Not fully get it here. You mean fork +
madvise(MADV_PAGEOUT)+ migrate ?fork +
madvise(MADV_PAGEOUT)only.Quoted 8 lines
Quoted 6 lines
Then verify, that it was actually paged out.
For example, we could have 10 processes mapping the page, then call
madvise(MADV_PAGEOUT)from one process. We can verify in all processes if the page is actually no longer mapped using/proc/self/pagemapHi, David
Hi!
Quoted 7 lines
I'd like to clarify some detail here.
Quoted 3 lines
We could likely do it with anon pages (swap), shmem pages (swap) and pagecache pages (file backed).
Quoted 7 lines
Quoted 3 lines
I could understand the three cases here, but for the (swap) case, it is not clear to me. You mean we force the page to be swapped out before migration? I may not have an idea to ensure that.
No need to involve migration for that test. It's sufficient to trigger pageout to then check if the page was unmapped in all other processes.
Here you mean for all anon/shmem/pagecache pages, we could have two category tests:
Quoted 7 lines
* migrate and write different data from on process, verify each process see new data
* trigger pageout from one process, verify each process has it pageout (/proc/self/pagemap)While one of these category is enough, right? Just like KSM below, migrate or pageout.
Both cases will trigger different code paths: for example, migration will trigger restoring of migration entries, which is a different rmap operation not triggered by pageout/pagein :)
Pageout is probably easier to implement: but we couldn't test hugetlb. With migration we probably could also test hugetlb rmap code (yet another case we should probably cover).
Signature · 3 lines
Cheers, David / dhildenb
On Fri, May 30, 2025 at 04:39:21PM +0200, David Hildenbrand wrote:
[...]
Quoted 12 lines
Quoted 11 lines
Here you mean for all anon/shmem/pagecache pages, we could have two category tests:
* migrate and write different data from on process, verify each process see new data
* trigger pageout from one process, verify each process has it pageout (/proc/self/pagemap)While one of these category is enough, right? Just like KSM below, migrate or pageout.
Quoted 8 lines
Both cases will trigger different code paths: for example, migration will trigger restoring of migration entries, which is a different rmap operation not triggered by pageout/pagein :)
Pageout is probably easier to implement: but we couldn't test hugetlb. With migration we probably could also test hugetlb rmap code (yet another case we should probably cover).
Oh, the pageout/pagein here is the madvise(MADV_PAGEOUT) one you mention in your previous reply?
I thought you propose another scenario. Sorry for not following your message.
Quoted 4 lines
Signature · 3 lines
Cheers, David / dhildenb
Signature · 2 lines
Wei Yang Help you, Help me
Quoted 18 lines · Wei Yang
On Fri, May 30, 2025 at 04:39:21PM +0200, David Hildenbrand wrote:
[...]Quoted 16 lines
Quoted 11 lines
Here you mean for all anon/shmem/pagecache pages, we could have two category tests:
* migrate and write different data from on process, verify each process see new data
* trigger pageout from one process, verify each process has it pageout (/proc/self/pagemap)While one of these category is enough, right? Just like KSM below, migrate or pageout.
Both cases will trigger different code paths: for example, migration will trigger restoring of migration entries, which is a different rmap operation not triggered by pageout/pagein :)
Quoted 7 lines
Quoted 4 lines
Pageout is probably easier to implement: but we couldn't test hugetlb. With migration we probably could also test hugetlb rmap code (yet another case we should probably cover).
Oh, the pageout/pagein here is the
madvise(MADV_PAGEOUT)one you mention in your previous reply?
Yes, migration (via move_pages(), but requiring two NUMA nodes) vs. pageout (via madvise(MADV_PAGEOUT) which would require a similar "force" way of succeeding on shared pages).
Signature · 3 lines
Cheers, David / dhildenb
Quoted 4 lines · Lorenzo Stoakes
Wei,
NACK the whole series.
Quoted 8 lines
I'm really not sure how to get through to you. You were _explicitly_ advised not to send this series. And yet you've sent it anyway.
I mean, I appreciate your enthusiasm and the fact you've made tests here etc. obviously. And you've clearly put a TON of work in. But I just don't know why you would when explicitly told not to without at least discussing it first?
Would you minding letting me know what is a preferred way of discussion?
Send a question to mail list?
---
Wei Yang
Help you, Help me
Quoted 14 lines · Wei Yang
Quoted 4 lines · Lorenzo Stoakes
Wei,
NACK the whole series.
Quoted 8 lines
I'm really not sure how to get through to you. You were _explicitly_ advised not to send this series. And yet you've sent it anyway.
I mean, I appreciate your enthusiasm and the fact you've made tests here etc. obviously. And you've clearly put a TON of work in. But I just don't know why you would when explicitly told not to without at least discussing it first?
Quoted 3 lines
Would you minding letting me know what is a preferred way of discussion?
Send a question to mail list?
Yeah, you can use [DISCUSSION] tags for such things.
Quoted 5 lines
---
Wei Yang
Help you, Help me
Quoted 15 lines · Lorenzo Stoakes
Quoted 14 lines · Wei Yang
Quoted 12 lines · Lorenzo Stoakes
Wei,
NACK the whole series.
I'm really not sure how to get through to you. You were _explicitly_ advised not to send this series. And yet you've sent it anyway.
I mean, I appreciate your enthusiasm and the fact you've made tests here etc. obviously. And you've clearly put a TON of work in. But I just don't know why you would when explicitly told not to without at least discussing it first?
Quoted 6 lines
Quoted 3 lines
Would you minding letting me know what is a preferred way of discussion?
Send a question to mail list?
Yeah, you can use [DISCUSSION] tags for such things.
Thanks for your information.
Quoted 5 lines
Quoted 5 lines
---
Wei Yang
Help you, Help me
Signature · 2 lines
Wei Yang Help you, Help me