initial work on making VMA flags a bitmap
Subsystems (24)
cover initial work on making VMA flags a bitmap
We are in the rather silly situation that we are running out of VMA flags as they are currently limited to a system word in size.
This leads to absurd situations where we limit features to 64-bit architectures only because we simply do not have the ability to add a flag for 32-bit ones.
This is very constraining and leads to hacks or, in the worst case, simply an inability to implement features we want for entirely arbitrary reasons.
This also of course gives us something of a Y2K type situation in mm where we might eventually exhaust all of the VMA flags even on 64-bit systems.
This series lays the groundwork for getting away from this limitation by establishing VMA flags as a bitmap whose size we can increase in future beyond 64 bits if required.
This is necessarily a highly iterative process given the extensive use of VMA flags throughout the kernel, so we start by performing basic steps.
Firstly, we declare VMA flags by bit number rather than by value, retaining the VM_xxx fields but in terms of these newly introduced VMA_xxx_BIT fields.
While we are here, we use sparse annotations to ensure that, when dealing with VMA bit number parameters, we cannot be passed values which are not declared as such - providing some useful type safety.
We then introduce an opaque VMA flag type, much like the opaque mm_struct flag type introduced in commit bb6525f2f8c4 ("mm: add bitmap mm->flags field"), which we establish in union with vma->vm_flags (but still set at system word size meaning there is no functional or data type size change).
We update the vm_flags_xxx() helpers to use this new bitmap, introducing sensible helpers to do so.
We then provide vma_flags_test() and vma_test() to allow for testing of VMA flag bits, and utilise these across the memory management subsystem.
Since it would be entirely inefficient to do away with the bitwise operations used throughout the kernel with respect to VMA flags, we permit these to exist, providing helpers for these operations against the new bitmap.
These operate on the assumption that such operations will only be required for flags which can exist within a system word. This allows the fundamental flags to be used efficiently as before.
This series lays the foundation for further work to expand the use of bitmap VMA flags and eventually eliminate these arbitrary restrictions.
ANDREW/REVIEWS NOTES:
Apologies, but the nature of this series is that it's going to be a little painful, I've based it on [0] to make life a bit easier. Let me know if you need rebases etc.
[0]: https://lore.kernel.org/linux-mm/[email protected]/
Lorenzo Stoakes (4):
mm: declare VMA flags by bit
mm: simplify and rename mm flags function for clarity
mm: introduce VMA flags bitmap type
mm: introduce and use VMA flag test helpers
fs/proc/task_mmu.c | 4 +- include/linux/hugetlb.h | 2 +- include/linux/mm.h | 341 +++++++++++++------- include/linux/mm_inline.h | 2 +- include/linux/mm_types.h | 120 ++++++- include/linux/userfaultfd_k.h | 12 +- kernel/fork.c | 4 +- mm/filemap.c | 4 +- mm/gup.c | 16 +- mm/hmm.c | 6 +- mm/huge_memory.c | 34 +- mm/hugetlb.c | 48 +-- mm/internal.h | 8 +- mm/khugepaged.c | 2 +- mm/ksm.c | 12 +- mm/madvise.c | 8 +- mm/memory.c | 77 +++-- mm/mempolicy.c | 4 +- mm/migrate.c | 4 +- mm/migrate_device.c | 10 +- mm/mlock.c | 8 +- mm/mmap.c | 16 +- mm/mmap_lock.c | 4 +- mm/mprotect.c | 12 +- mm/mremap.c | 18 +- mm/mseal.c | 2 +- mm/msync.c | 4 +- mm/nommu.c | 16 +- mm/oom_kill.c | 4 +- mm/pagewalk.c | 2 +- mm/rmap.c | 16 +- mm/swap.c | 3 +- mm/userfaultfd.c | 33 +- mm/vma.c | 37 ++- mm/vma.h | 6 +- mm/vmscan.c | 4 +- tools/testing/vma/vma.c | 20 +- tools/testing/vma/vma_internal.h | 536 +++++++++++++++++++++++++++---- 38 files changed, 1037 insertions(+), 422 deletions(-)
--
2.51.0
PATCH 1/4 mm: declare VMA flags by bit
In order to lay the groundwork for VMA flags being a bitmap rather than a system word in size, we need to be able to consistently refer to VMA flags by bit number rather than value.
Take this opportunity to do so in an enum which we which is additionally useful for tooling to extract metadata from.
This additionally makes it very clear which bits are being used for what at a glance.
We use the VMA_ prefix for the bit values as it is logical to do so since these reference VMAs. We consistently suffix with _BIT to make it clear what the values refer to.
We place all bits 32+ in an #ifdef CONFIG_64BIT block as indeed these all require a 64-bit system and it's neater and self-documenting to do so.
We declare a sparse-bitwise type vma_flag_t which ensures that users can't pass around invalid VMA flags by accident and prepares for future work towards VMA flags being a bitmap where we want to ensure bit values are type safe.
Finally, we have to update some rather silly if-deffery found in mm/task_mmu.c which would otherwise break.
Additionally, update the VMA userland testing vma_internal.h header to include these changes.
--- fs/proc/task_mmu.c | 4 +- include/linux/mm.h | 286 +++++++++++++++++--------- tools/testing/vma/vma_internal.h | 341 +++++++++++++++++++++++++++---- 3 files changed, 488 insertions(+), 143 deletions(-)
fs/proc/task_mmu.c +2 -2
@@ -1182,10 +1182,10 @@ static void show_smap_vma_flags(struct seq_file *m, struct vm_area_struct *vma) [ilog2(VM_PKEY_BIT0)] = "", [ilog2(VM_PKEY_BIT1)] = "", [ilog2 (VM_PKEY_BIT2 )] ="" , -#if VM_PKEY_BIT3 +#if CONFIG_ARCH_PKEY_BITS > 3 [ilog2 (VM_PKEY_BIT3 )] ="" , #endif -#if VM_PKEY_BIT4 +#if CONFIG_ARCH_PKEY_BITS > 4 [ilog2(VM_PKEY_BIT4)] = "", #endif #endif /* CONFIG_ARCH_HAS_PKEYS */
include/linux/mm.h +187 -99
382 lines not on this page · raw message
tools/testing/vma/vma_internal.h +299 -42
368 lines not on this page · raw message
Signature · 1 line
2.51.0
Re: [PATCH 1/4] mm: declare VMA flags by bit
Quoted 4 lines · Lorenzo Stoakes
We declare a sparse-bitwise type
vma_flag_twhich ensures that users can't pass around invalid VMA flags by accident and prepares for future work towards VMA flags being a bitmap where we want to ensure bit values are type safe.
Does sparse attach the type to the enum item? Normal C says the enum item's type is always 'int' if the value fits in int..
And I'm not sure bitwise rules work quite the way you'd like for this enum, it was ment for things that are |'d..
I have seen an agressively abuse-resistent technique before, I don't really recommend it, but FYI:
struct vma_bits {
u8 VMA_READ_BIT;
u8 VMA_WRITE_BIT;
..
};
#define VMA_BIT(bit_name) BIT(offsetof(struct vma_bits, bit_name))
Quoted 11 lines
Finally, we have to update some rather silly if-deffery found in
mm/task_mmu.cwhich would otherwise break.Additionally, update the VMA userland testing
vma_internal.hheader to include these changes.Signed-off-by Lorenzo Stoakes <[email protected]>--- fs/proc/task_mmu.c | 4 +- include/linux/mm.h | 286 +++++++++++++++++--------- tools/testing/vma/vma_internal.h | 341 +++++++++++++++++++++++++++----
Maybe take the moment to put them in some vma_flags.h and then can that be included from tools/testing to avoid this copying??
Quoted 86 lines
include/linux/mm.h +86
-0+/** +* vma_flag_t - specifies an individual VMA flag by bit number. +* +* This value is made type safe by sparse to avoid passing invalid flag values +* around. +*/ +typedef int __bitwise vma_flag_t ; + +enum { +/* currently active flags */ +VMA_READ_BIT = (__force vma_flag_t )0 , +VMA_WRITE_BIT = (__force vma_flag_t )1 , +VMA_EXEC_BIT = (__force vma_flag_t )2 , +VMA_SHARED_BIT = (__force vma_flag_t )3 , + +/* mprotect() hardcodes VM_MAYREAD >> 4 == VM_READ, and so for r/w/x bits. */ +VMA_MAYREAD_BIT = (__force vma_flag_t )4 ,/* limits for mprotect() etc */ +VMA_MAYWRITE_BIT = (__force vma_flag_t )5 , +VMA_MAYEXEC_BIT = (__force vma_flag_t )6 , +VMA_MAYSHARE_BIT = (__force vma_flag_t )7 , + +VMA_GROWSDOWN_BIT = (__force vma_flag_t )8 ,/* general info on the segment */ +#ifdef CONFIG_MMU +VMA_UFFD_MISSING_BIT = (__force vma_flag_t )9 ,/* missing pages tracking */ +#else +/* nommu: R/O MAP_PRIVATE mapping that might overlay a file mapping */ +VMA_MAYOVERLAY_BIT = (__force vma_flag_t )9 , +#endif +/* Page-ranges managed without "struct page", just pure PFN */ +VMA_PFNMAP_BIT = (__force vma_flag_t )10 , + +VMA_MAYBE_GUARD_BIT = (__force vma_flag_t )11 , + +VMA_UFFD_WP_BIT = (__force vma_flag_t )12 ,/* wrprotect pages tracking */ + +VMA_LOCKED_BIT = (__force vma_flag_t )13 , +VMA_IO_BIT = (__force vma_flag_t )14 ,/* Memory mapped I/O or similar */ + +/* Used by madvise() */ +VMA_SEQ_READ_BIT = (__force vma_flag_t )15 ,/* App will access data sequentially */ +VMA_RAND_READ_BIT = (__force vma_flag_t )16 ,/* App will not benefit from clustered reads */ + +VMA_DONTCOPY_BIT = (__force vma_flag_t )17 ,/* Do not copy this vma on fork */ +VMA_DONTEXPAND_BIT = (__force vma_flag_t )18 ,/* Cannot expand with mremap() */ +VMA_LOCKONFAULT_BIT = (__force vma_flag_t )19 ,/* Lock pages covered when faulted in */ +VMA_ACCOUNT_BIT = (__force vma_flag_t )20 ,/* Is a VM accounted object */ +VMA_NORESERVE_BIT = (__force vma_flag_t )21 ,/* should the VM suppress accounting */ +VMA_HUGETLB_BIT = (__force vma_flag_t )22 ,/* Huge TLB Page VM */ +VMA_SYNC_BIT = (__force vma_flag_t )23 ,/* Synchronous page faults */ +VMA_ARCH_1_BIT = (__force vma_flag_t )24 ,/* Architecture-specific flag */ +VMA_WIPEONFORK_BIT = (__force vma_flag_t )25 ,/* Wipe VMA contents in child. */ +VMA_DONTDUMP_BIT = (__force vma_flag_t )26 ,/* Do not include in the core dump */ + +#ifdef CONFIG_MEM_SOFT_DIRTY +VMA_SOFTDIRTY_BIT = (__force vma_flag_t )27 ,/* Not soft dirty clean area */ +#endif + +VMA_MIXEDMAP_BIT = (__force vma_flag_t )28 ,/* Can contain struct page and pure PFN pages */ +VMA_HUGEPAGE_BIT = (__force vma_flag_t )29 ,/* MADV_HUGEPAGE marked this vma */ +VMA_NOHUGEPAGE_BIT = (__force vma_flag_t )30 ,/* MADV_NOHUGEPAGE marked this vma */ +VMA_MERGEABLE_BIT = (__force vma_flag_t )31 ,/* KSM may merge identical pages */ + +#ifdef CONFIG_64BIT + /* These bits are reused, we define specific uses below. */ +#ifdef CONFIG_ARCH_USES_HIGH_VMA_FLAGS +VMA_HIGH_ARCH_0_BIT = (__force vma_flag_t )32 , +VMA_HIGH_ARCH_1_BIT = (__force vma_flag_t )33 , +VMA_HIGH_ARCH_2_BIT = (__force vma_flag_t )34 , +VMA_HIGH_ARCH_3_BIT = (__force vma_flag_t )35 , +VMA_HIGH_ARCH_4_BIT = (__force vma_flag_t )36 , +VMA_HIGH_ARCH_5_BIT = (__force vma_flag_t )37 , +VMA_HIGH_ARCH_6_BIT = (__force vma_flag_t )38 , +#endif + +VMA_ALLOW_ANY_UNCACHED_BIT = (__force vma_flag_t )39 , +VMA_DROPPABLE_BIT = (__force vma_flag_t )40 , + +#ifdef CONFIG_HAVE_ARCH_USERFAULTFD_MINOR +VMA_UFFD_MINOR_BIT = (__force vma_flag_t )41 , +#endif + +VMA_SEALED_BIT = (__force vma_flag_t )42 , +#endif /* CONFIG_64BIT */ +}; + +#define VMA_BIT (bit) BIT((__force int)bit)
include/linux/mm.h extern struct rw_semaphore nommu_region_sem; · line 375
Quoted 10 lines
include/linux/mm.h +4
-5-/* mprotect() hardcodes VM_MAYREAD >> 4 == VM_READ, and so for r/w/x bits. */ -#define VM_MAYREAD 0x00000010/* limits for mprotect() etc */ -#define VM_MAYWRITE 0x00000020 -#define VM_MAYEXEC 0x00000040 -#define VM_MAYSHARE 0x00000080 +#define VM_MAYREAD VMA_BIT(VMA_MAYREAD_BIT) +#define VM_MAYWRITE VMA_BIT(VMA_MAYWRITE_BIT) +#define VM_MAYEXEC VMA_BIT(VMA_MAYEXEC_BIT) +#define VM_MAYSHARE VMA_BIT(VMA_MAYSHARE_BIT)
I suggest removing some of this duplication..
#define DECLARE_VMA_BIT(name, bitno) \
NAME ## _BIT = (__force vma_flag_t)bitno,
NAME = BIT(bitno),
enum {
DECLARE_VMA_BIT(VMA_READ, 0),
}
Especially since the #defines and enum need to have matching #ifdefs.
It is OK to abuse the enum like the above, C won't get mad and works better in gdb/clangd.
Later you can have a variation of the macro for your first sytem word/second system word idea.
Otherwise I think this is a great thing to do, thanks!
Jason
Re: [PATCH 1/4] mm: declare VMA flags by bit
Quoted 8 lines · Jason Gunthorpe
Quoted 4 lines · Lorenzo Stoakes
We declare a sparse-bitwise type
vma_flag_twhich ensures that users can't pass around invalid VMA flags by accident and prepares for future work towards VMA flags being a bitmap where we want to ensure bit values are type safe.Does sparse attach the type to the enum item? Normal C says the enum item's type is always 'int' if the value fits in int..
It does, have tested this, not sure if due to sparse doing extra work to make that happen or GNU C doing more there.
You can see an anon enum being used for this in the examples in the sparse docs for instance (see [0]) so it's kind of a 'thing' it seems.
I also tested this to make sure, when intentionally passing some non-flag value to the functions which accept vma_flag_t and it got picked up right away, checked via:
make C=2 -j $(nproc) 2>&1 | grep vma_flag_t
[0]:https://docs.kernel.org/dev-tools/sparse.html
Quoted 7 lines
And I'm not sure bitwise rules work quite the way you'd like for this enum, it was ment for things that are |'d..
I have seen an agressively abuse-resistent technique before, I don't really recommend it, but FYI:
Quoted 6 lines
struct
vma_bits{
u8VMA_READ_BIT;
u8VMA_WRITE_BIT;
..
};
#defineVMA_BIT(bit_name)BIT(offsetof(struct vma_bits, bit_name))
Oh my eyes! :P I mean kinda clever but also lord above :)
I don't think we need this afaict. The idea is to catch accidental instances of e.g.:
vma_test(vma, VM_WRITE);
Rather than abuse. Doing the above is _very easy_ and so I wanted to explicitly have the bots moan if people make this mistake.
If only C had a stronger type system...
Quoted 7 lines
Quoted 7 lines
Finally, we have to update some rather silly if-deffery found in
mm/task_mmu.cwhich would otherwise break.Additionally, update the VMA userland testing
vma_internal.hheader to include these changes.
Quoted 8 lines
Quoted 5 lines
Signed-off-by Lorenzo Stoakes <[email protected]>--- fs/proc/task_mmu.c | 4 +- include/linux/mm.h | 286 +++++++++++++++++--------- tools/testing/vma/vma_internal.h | 341 +++++++++++++++++++++++++++----Maybe take the moment to put them in some
vma_flags.hand then can that be included fromtools/testingto avoid this copying??
It sucks to have this copy/paste yeah. The problem is to make the VMA userland testing work, we intentionally isolate vma.h/vma.c dependencies into vma_internal.h in mm/ and also do the same in the userland component, so we can #include vma.c/h in the userland code.
So we'd have to have a strict requirement that vma_flags.h doesn't import any other headers or at least none which aren't substituted somehow in the tools/include directory.
The issue is people might quite reasonably update include/linux/vma_flags.h to do more later and then break all of the VMA userland testing...
It's a bit of a delicate thing to keep it all
Quoted 104 lines
Quoted 87 lines
include/linux/mm.h +86
-0+/** +* vma_flag_t - specifies an individual VMA flag by bit number. +* +* This value is made type safe by sparse to avoid passing invalid flag values +* around. +*/ +typedef int __bitwise vma_flag_t ; + +enum { +/* currently active flags */ +VMA_READ_BIT = (__force vma_flag_t )0 , +VMA_WRITE_BIT = (__force vma_flag_t )1 , +VMA_EXEC_BIT = (__force vma_flag_t )2 , +VMA_SHARED_BIT = (__force vma_flag_t )3 , + +/* mprotect() hardcodes VM_MAYREAD >> 4 == VM_READ, and so for r/w/x bits. */ +VMA_MAYREAD_BIT = (__force vma_flag_t )4 ,/* limits for mprotect() etc */ +VMA_MAYWRITE_BIT = (__force vma_flag_t )5 , +VMA_MAYEXEC_BIT = (__force vma_flag_t )6 , +VMA_MAYSHARE_BIT = (__force vma_flag_t )7 , + +VMA_GROWSDOWN_BIT = (__force vma_flag_t )8 ,/* general info on the segment */ +#ifdef CONFIG_MMU +VMA_UFFD_MISSING_BIT = (__force vma_flag_t )9 ,/* missing pages tracking */ +#else +/* nommu: R/O MAP_PRIVATE mapping that might overlay a file mapping */ +VMA_MAYOVERLAY_BIT = (__force vma_flag_t )9 , +#endif +/* Page-ranges managed without "struct page", just pure PFN */ +VMA_PFNMAP_BIT = (__force vma_flag_t )10 , + +VMA_MAYBE_GUARD_BIT = (__force vma_flag_t )11 , + +VMA_UFFD_WP_BIT = (__force vma_flag_t )12 ,/* wrprotect pages tracking */ + +VMA_LOCKED_BIT = (__force vma_flag_t )13 , +VMA_IO_BIT = (__force vma_flag_t )14 ,/* Memory mapped I/O or similar */ + +/* Used by madvise() */ +VMA_SEQ_READ_BIT = (__force vma_flag_t )15 ,/* App will access data sequentially */ +VMA_RAND_READ_BIT = (__force vma_flag_t )16 ,/* App will not benefit from clustered reads */ + +VMA_DONTCOPY_BIT = (__force vma_flag_t )17 ,/* Do not copy this vma on fork */ +VMA_DONTEXPAND_BIT = (__force vma_flag_t )18 ,/* Cannot expand with mremap() */ +VMA_LOCKONFAULT_BIT = (__force vma_flag_t )19 ,/* Lock pages covered when faulted in */ +VMA_ACCOUNT_BIT = (__force vma_flag_t )20 ,/* Is a VM accounted object */ +VMA_NORESERVE_BIT = (__force vma_flag_t )21 ,/* should the VM suppress accounting */ +VMA_HUGETLB_BIT = (__force vma_flag_t )22 ,/* Huge TLB Page VM */ +VMA_SYNC_BIT = (__force vma_flag_t )23 ,/* Synchronous page faults */ +VMA_ARCH_1_BIT = (__force vma_flag_t )24 ,/* Architecture-specific flag */ +VMA_WIPEONFORK_BIT = (__force vma_flag_t )25 ,/* Wipe VMA contents in child. */ +VMA_DONTDUMP_BIT = (__force vma_flag_t )26 ,/* Do not include in the core dump */ + +#ifdef CONFIG_MEM_SOFT_DIRTY +VMA_SOFTDIRTY_BIT = (__force vma_flag_t )27 ,/* Not soft dirty clean area */ +#endif + +VMA_MIXEDMAP_BIT = (__force vma_flag_t )28 ,/* Can contain struct page and pure PFN pages */ +VMA_HUGEPAGE_BIT = (__force vma_flag_t )29 ,/* MADV_HUGEPAGE marked this vma */ +VMA_NOHUGEPAGE_BIT = (__force vma_flag_t )30 ,/* MADV_NOHUGEPAGE marked this vma */ +VMA_MERGEABLE_BIT = (__force vma_flag_t )31 ,/* KSM may merge identical pages */ + +#ifdef CONFIG_64BIT + /* These bits are reused, we define specific uses below. */ +#ifdef CONFIG_ARCH_USES_HIGH_VMA_FLAGS +VMA_HIGH_ARCH_0_BIT = (__force vma_flag_t )32 , +VMA_HIGH_ARCH_1_BIT = (__force vma_flag_t )33 , +VMA_HIGH_ARCH_2_BIT = (__force vma_flag_t )34 , +VMA_HIGH_ARCH_3_BIT = (__force vma_flag_t )35 , +VMA_HIGH_ARCH_4_BIT = (__force vma_flag_t )36 , +VMA_HIGH_ARCH_5_BIT = (__force vma_flag_t )37 , +VMA_HIGH_ARCH_6_BIT = (__force vma_flag_t )38 , +#endif + +VMA_ALLOW_ANY_UNCACHED_BIT = (__force vma_flag_t )39 , +VMA_DROPPABLE_BIT = (__force vma_flag_t )40 , + +#ifdef CONFIG_HAVE_ARCH_USERFAULTFD_MINOR +VMA_UFFD_MINOR_BIT = (__force vma_flag_t )41 , +#endif + +VMA_SEALED_BIT = (__force vma_flag_t )42 , +#endif /* CONFIG_64BIT */ +}; + +#define VMA_BIT (bit) BIT((__force int)bit)include/linux/mm.h extern struct rw_semaphore nommu_region_sem; · line 375
Quoted 10 lines
include/linux/mm.h +4
-5-/* mprotect() hardcodes VM_MAYREAD >> 4 == VM_READ, and so for r/w/x bits. */ -#define VM_MAYREAD 0x00000010/* limits for mprotect() etc */ -#define VM_MAYWRITE 0x00000020 -#define VM_MAYEXEC 0x00000040 -#define VM_MAYSHARE 0x00000080 +#define VM_MAYREAD VMA_BIT(VMA_MAYREAD_BIT) +#define VM_MAYWRITE VMA_BIT(VMA_MAYWRITE_BIT) +#define VM_MAYEXEC VMA_BIT(VMA_MAYEXEC_BIT) +#define VM_MAYSHARE VMA_BIT(VMA_MAYSHARE_BIT)I suggest removing some of this duplication..
#define
DECLARE_VMA_BIT(name, bitno)\
NAME ##_BIT= (__forcevma_flag_t)bitno,
NAME = BIT(bitno),
Quoted 8 lines
enum { DECLARE_VMA_BIT(VMA_READ, 0), }Especially since the #defines and enum need to have matching #ifdefs.
It is OK to abuse the enum like the above, C won't get mad and works better in gdb/clangd.
I think having the enum anon avoids issues I've been concerned about with named enum's containing flags when used as parameters yes.
Quoted 3 lines
Later you can have a variation of the macro for your first sytem word/second system word idea.
Well I think we'd probably want to name the macro accordingly.
DECLARE_VMA_BIT_AND_FLAG() maybe? And mention in the comment that it's for system word siz
Quoted 2 lines
Otherwise I think this is a great thing to do, thanks!
Thanks :)
To give due credit - Matthew suggested this a while ago, I've been working towards it with the mm flags first as an easier case to tackle.
It came out of my assuming that the VM_MAYBE_GUARD stuff didn't have a flag free to do this in the 32-bit space. As part of this work it became apparent I was wrong, so I implemented + sent that series yesterday (doh!) but this change is still useful as it's beyond silly that we're constrained like this.
I should actually probably put a Suggested-by for this, didn't even think to, sorry Matthew! :)
Quoted 2 lines
Jason
Cheers, Lorenzo
Re: [PATCH 1/4] mm: declare VMA flags by bit
Quoted 7 lines · Lorenzo Stoakes
Quoted 6 lines
Quoted 3 lines
fs/proc/task_mmu.c | 4 +- include/linux/mm.h | 286 +++++++++++++++++--------- tools/testing/vma/vma_internal.h | 341 +++++++++++++++++++++++++++----Maybe take the moment to put them in some
vma_flags.hand then can that be included fromtools/testingto avoid this copying??
Quoted 8 lines
It sucks to have this copy/paste yeah. The problem is to make the VMA userland testing work, we intentionally isolate
vma.h/vma.cdependencies intovma_internal.hinmm/and also do the same in the userland component, so we can #include vma.c/h in the userland code.So we'd have to have a strict requirement that
vma_flags.hdoesn't import any other headers or at least none which aren't substituted somehow in thetools/includedirectory.
I think that's fine, much better than copying it like this..
Quoted 2 lines
The issue is people might quite reasonably update
include/linux/vma_flags.hto do more later and then break all of the VMA userland testing...
If only the selftest build system wasn't such a PITA maybe more people would run it :(
Jason
Re: [PATCH 1/4] mm: declare VMA flags by bit
Quoted 13 lines · Jason Gunthorpe
Quoted 12 lines · Lorenzo Stoakes
Quoted 6 lines
Quoted 3 lines
fs/proc/task_mmu.c | 4 +- include/linux/mm.h | 286 +++++++++++++++++--------- tools/testing/vma/vma_internal.h | 341 +++++++++++++++++++++++++++----Maybe take the moment to put them in some
vma_flags.hand then can that be included fromtools/testingto avoid this copying??It sucks to have this copy/paste yeah. The problem is to make the VMA userland testing work, we intentionally isolate
vma.h/vma.cdependencies intovma_internal.hinmm/and also do the same in the userland component, so we can #include vma.c/h in the userland code.
Quoted 5 lines
Quoted 3 lines
So we'd have to have a strict requirement that
vma_flags.hdoesn't import any other headers or at least none which aren't substituted somehow in thetools/includedirectory.I think that's fine, much better than copying it like this..
Ack will give it a go!
Quoted 6 lines
Quoted 3 lines
The issue is people might quite reasonably update
include/linux/vma_flags.hto do more later and then break all of the VMA userland testing...If only the selftest build system wasn't such a PITA maybe more people would run it :(
This isn't a selftest thing :)
So it's literally:
$ cd tools/testing/vma $ make && ./vma
Quoted 2 lines
Jason
Cheers, Lorenzo
Gregory Price
maintainerRe: [PATCH 1/4] mm: declare VMA flags by bit
In reply to Lorenzo StoakesQuoted 8 lines · Lorenzo Stoakes
fs/proc/task_mmu.c +0
-0@@ -1182,10 +1182,10 @@ static void show_smap_vma_flags(struct seq_file *m, struct vm_area_struct *vma) [ilog2(VM_PKEY_BIT0)] = "", [ilog2(VM_PKEY_BIT1)] = "",
fs/proc/task_mmu.c static void show_smap_vma_flags(struct seq_file *m, struct vm_area_struct *vma) · line 1191
Quoted 10 lines
fs/proc/task_mmu.c +2
-2[ilog2(VM_PKEY_BIT2)] = "", -#if VM_PKEY_BIT3 +#if CONFIG_ARCH_PKEY_BITS > 3 [ilog2 (VM_PKEY_BIT3 )] ="" , #endif -#if VM_PKEY_BIT4 +#if CONFIG_ARCH_PKEY_BITS > 4 [ilog2(VM_PKEY_BIT4)] = "", #endif #endif /* CONFIG_ARCH_HAS_PKEYS */
I realize this causes some annoying churn, but is it possible/reasonable to break the no-op ifdefsphagetti fixes into a separate diff?
it makes it easier to see this change:
fs/proc/task_mmu.c
Quoted 2 lines
fs/proc/task_mmu.c +1
-1-# define VM_PKEY_BIT0 VM_HIGH_ARCH_0 +#define VM_PKEY_BIT0 VMA_BIT(VMA_PKEY_BIT0_BIT)
~Gregory
PATCH 2/4 mm: simplify and rename mm flags function for clarity
In reply to Lorenzo StoakesThe __mm_flags_set_word() function is slightly ambiguous - we use 'set' to refer to setting individual bits (such as in mm_flags_set()) but here we use it to refer to overwriting the value altogether.
Rename it to __mm_flags_overwrite_word() to eliniate this ambiguity.
We additionally simplify the functions, eliminating unnecessary bitmap_xxx() operations (the compiler would have optimised these out but it's worth being as clear as we can be here).
--- include/linux/mm_types.h | 14 +++++--------- kernel/fork.c | 4 ++-- 2 files changed, 7 insertions(+), 11 deletions(-)
include/linux/mm_types.h +5 -9
@@ -1289,15 +1289,13 @@ struct mm_struct {unsigned long cpu_bitmap[]; }; -/* Set the first system word of mm flags, non-atomically. */ -static inline void __mm_flags_set_word (struct mm_struct *mm,unsigned long value) +/* Copy value to the first system word of mm flags, non-atomically. */ +static inline void __mm_flags_overwrite_word (struct mm_struct *mm,unsigned long value) { -unsigned long *bitmap =ACCESS_PRIVATE (&mm->flags, __mm_flags); - -bitmap_copy (bitmap, &value,BITS_PER_LONG ); + *ACCESS_PRIVATE (&mm->flags, __mm_flags) = value; } -/* Obtain a read-only view of the bitmap. */ +/* Obtain a read-only view of the mm flags bitmap. */ static inline const unsigned long *__mm_flags_get_bitmap (const struct mm_struct *mm) {return (const unsigned long *)ACCESS_PRIVATE (&mm->flags, __mm_flags);
@@ -1306,9 +1304,7 @@ static inline const unsigned long *__mm_flags_get_bitmap(const struct mm_struct/* Read the first system word of mm flags, non-atomically. */ static inline unsigned long __mm_flags_get_word (const struct mm_struct *mm) { -const unsigned long *bitmap =__mm_flags_get_bitmap (mm); - -return bitmap_read (bitmap,0 ,BITS_PER_LONG ); +return *__mm_flags_get_bitmap (mm); }/*
kernel/fork.c +2 -2
@@ -1061,10 +1061,10 @@ static struct mm_struct *mm_init(struct mm_struct *mm, struct task_struct *p,if (current->mm) {unsigned long flags =__mm_flags_get_word (current->mm); -__mm_flags_set_word (mm,mmf_init_legacy_flags (flags)); +__mm_flags_overwrite_word (mm,mmf_init_legacy_flags (flags)); mm->def_flags = current->mm->def_flags &VM_INIT_DEF_MASK ; }else { -__mm_flags_set_word (mm, default_dump_filter); +__mm_flags_overwrite_word (mm, default_dump_filter); mm->def_flags =0 ; }
Signature · 1 line
2.51.0
PATCH 3/4 mm: introduce VMA flags bitmap type
In reply to Lorenzo StoakesIt is useful to transition to using a bitmap for VMA flags so we can avoid running out of flags, especially for 32-bit kernels which are constrained to 32 flags, necessitating some features to be limited to 64-bit kernels only.
By doing so, we remove any constraint on the number of VMA flags moving forwards no matter the platform and can decide in future to extend beyond 64 if required.
We start by declaring an opaque types, vma_flags_t (which resembles mm_struct flags of type mm_flags_t), setting it to precisely the same size as vm_flags_t, and place it in union with vm_flags in the VMA declaration.
We additionally update struct vm_area_desc equivalently placing the new opaque type in union with vm_flags.
This change therefore does not impact the size of struct vm_area_struct or struct vm_area_desc.
In order for the change to be iterative and to avoid impacting performance, we designate VM_xxx declared bitmap flag values as those which must exist in the first system word of the VMA flags bitmap.
We therefore declare vma_flags_clear_all(), vma_flags_overwrite_word(), vma_flags_overwrite_word(), vma_flags_overwrite_word_once(), vma_flags_set_word() and vma_flags_clear_word() in order to allow us to update the existing vm_flags_*() functions to utilise these helpers.
This is a stepping stone towards converting users to the VMA flags bitmap and behaves precisely as before.
By doing this, we can eliminate the existing private vma->__vm_flags field in the vma->vm_flags union and replace it with the newly introduced opaque type vma_flags, which we call flags so we refer to the new bitmap field as vma->flags.
We additionally update the VMA userland test declarations to implement the same changes there.
No functional change intended.
--- include/linux/mm.h | 14 ++- include/linux/mm_types.h | 64 +++++++++++++- tools/testing/vma/vma.c | 20 ++--- tools/testing/vma/vma_internal.h | 143 ++++++++++++++++++++++++++----- 4 files changed, 202 insertions(+), 39 deletions(-)
include/linux/mm.h +10 -4
@@ -921,7 +921,8 @@ static inline void vma_init(struct vm_area_struct *vma, struct mm_struct *mm)static inline void vm_flags_init (struct vm_area_struct *vma,vm_flags_t flags) { -ACCESS_PRIVATE (vma, __vm_flags) = flags; +vma_flags_clear_all (&vma->flags); +vma_flags_overwrite_word (&vma->flags, flags); }/*
@@ -940,21 +941,26 @@ static inline void vm_flags_reset_once(struct vm_area_struct *vma, vm_flags_t flags) {vma_assert_write_locked (vma); -WRITE_ONCE (ACCESS_PRIVATE (vma, __vm_flags), flags); +/* +* The user should only be interested in avoiding reordering of +* assignment to the first word. +*/ +vma_flags_clear_all (&vma->flags); +vma_flags_overwrite_word_once (&vma->flags, flags); }static inline void vm_flags_set (struct vm_area_struct *vma,vm_flags_t flags) {vma_start_write (vma); -ACCESS_PRIVATE (vma, __vm_flags) |= flags; +vma_flags_set_word (&vma->flags, flags); }static inline void vm_flags_clear (struct vm_area_struct *vma,vm_flags_t flags) {vma_start_write (vma); -ACCESS_PRIVATE (vma, __vm_flags) &= ~flags; +vma_flags_clear_word (&vma->flags, flags); }/*
include/linux/mm_types.h +62 -2
91 lines not on this page · raw message
tools/testing/vma/vma.c +10 -10
@@ -69,18 +69,18 @@ static struct vm_area_struct *alloc_vma(struct mm_struct *mm, pgoff_t pgoff, vm_flags_t vm_flags) { -struct vm_area_struct *ret =vm_area_alloc (mm); +struct vm_area_struct *vma =vm_area_alloc (mm); -if (ret ==NULL ) +if (vma ==NULL )return NULL ; - ret->vm_start = start; - ret->vm_end = end; - ret->vm_pgoff = pgoff; - ret->__vm_flags = vm_flags; -vma_assert_detached (ret); + vma->vm_start = start; + vma->vm_end = end; + vma->vm_pgoff = pgoff; +vm_flags_reset (vma, vm_flags); +vma_assert_detached (vma); -return ret; +return vma; }/* Helper function to allocate a VMA and link it to the tree. */
@@ -713,7 +713,7 @@ static bool test_vma_merge_special_flags(void)for (i =0 ; i <ARRAY_SIZE (special_flags); i++) {vm_flags_t special_flag = special_flags[i]; - vma_left->__vm_flags = vm_flags | special_flag; +vm_flags_reset (vma_left, vm_flags | special_flag); vmg.vm_flags = vm_flags | special_flag; vma =merge_new (&vmg);ASSERT_EQ (vma,NULL );
@@ -735,7 +735,7 @@ static bool test_vma_merge_special_flags(void)for (i =0 ; i <ARRAY_SIZE (special_flags); i++) {vm_flags_t special_flag = special_flags[i]; - vma_left->__vm_flags = vm_flags | special_flag; +vm_flags_reset (vma_left, vm_flags | special_flag); vmg.vm_flags = vm_flags | special_flag; vma =merge_existing (&vmg);ASSERT_EQ (vma,NULL );
tools/testing/vma/vma_internal.h +120 -23
179 lines not on this page · raw message
Signature · 1 line
2.51.0
PATCH 4/4 mm: introduce and use VMA flag test helpers
In reply to Lorenzo StoakesWe introduce vma_flags_test() and vma_test() (the latter operating on a VMA, the former on a pointer to a vma_flags_t value).
It's useful to have both, as many functions modify a local VMA flags variable before setting the VMA flags to this value.
Since it would be inefficient to force every single VMA flag users to reference flags by bit number, we must have some operations that continue to work against bitmap values.
Therefore, all flags which are specified as VM_xxx flags are designated to be ones which fit within a system word.
In future, when we remove the limitation on some flags being 64-bit only, we will remove all VM_xxx flags at bit 32 or higher and use these bitwise only.
To work with these flags, we provide vma_flags_get_word() and vma_flags_word_[and, any, all]() which behave identically to the existing bitwise logic.
We then utilise all the new helpers throughout the memory management subsystem as a starting point for the refactoring required to move to use of the new VMA flags across the kernel code base.
For cases where we either define VM_xxx to a certain value if certain config settings are enabled (or other conditions met) or 0 otherwise, we must use vma_flags_word_any() as there is no efficient way to specify this with a bit value.
Once all VMA flags are converted to a bitmap we no longer have to worry about this as flags will be plentiful and we can simply assign one bit per setting and eliminate this.
Additionally update the VMA userland test code to accommodate these changes.
No functional change intended.
--- include/linux/hugetlb.h | 2 +- include/linux/mm.h | 41 ++++++++++------- include/linux/mm_inline.h | 2 +- include/linux/mm_types.h | 42 +++++++++++++++++ include/linux/userfaultfd_k.h | 12 ++--- mm/filemap.c | 4 +- mm/gup.c | 16 +++---- mm/hmm.c | 6 +-- mm/huge_memory.c | 34 +++++++------- mm/hugetlb.c | 48 ++++++++++---------- mm/internal.h | 8 ++-- mm/khugepaged.c | 2 +- mm/ksm.c | 12 ++--- mm/madvise.c | 8 ++-- mm/memory.c | 77 ++++++++++++++++---------------- mm/mempolicy.c | 4 +- mm/migrate.c | 4 +- mm/migrate_device.c | 10 ++--- mm/mlock.c | 8 ++-- mm/mmap.c | 16 +++---- mm/mmap_lock.c | 4 +- mm/mprotect.c | 12 ++--- mm/mremap.c | 18 ++++---- mm/mseal.c | 2 +- mm/msync.c | 4 +- mm/nommu.c | 16 +++---- mm/oom_kill.c | 4 +- mm/pagewalk.c | 2 +- mm/rmap.c | 16 ++++--- mm/swap.c | 3 +- mm/userfaultfd.c | 33 +++++++------- mm/vma.c | 37 ++++++++------- mm/vma.h | 6 +-- mm/vmscan.c | 4 +- tools/testing/vma/vma_internal.h | 52 +++++++++++++++++++++ 35 files changed, 340 insertions(+), 229 deletions(-)
include/linux/hugetlb.h +1 -1
@@ -1349,7 +1349,7 @@ bool want_pmd_share(struct vm_area_struct *vma, unsigned long addr);static inline bool __vma_shareable_lock (struct vm_area_struct *vma) { -return (vma->vm_flags &VM_MAYSHARE ) && vma->vm_private_data; +return vma_test (vma,VMA_MAYSHARE_BIT ) && vma->vm_private_data; }bool __vma_private_lock (struct vm_area_struct *vma);
include/linux/mm.h +26 -15
88 lines not on this page · raw message
include/linux/mm_inline.h +1 -1
@@ -608,7 +608,7 @@ pte_install_uffd_wp_if_needed(struct vm_area_struct *vma, unsigned long addr,static inline bool vma_has_recency (const struct vm_area_struct *vma) { -if (vma->vm_flags & (VM_SEQ_READ |VM_RAND_READ )) +if (vma_flags_word_any (&vma->flags,VM_SEQ_READ |VM_RAND_READ ))return false;if (vma->vm_file && (vma->vm_file->f_mode &FMODE_NOREUSE ))
include/linux/mm_types.h +42 -0
48 lines not on this page · raw message
include/linux/userfaultfd_k.h +6 -6
@@ -161,7 +161,7 @@ static inline bool is_mergeable_vm_userfaultfd_ctx(struct vm_area_struct *vma,*/ static inline bool uffd_disable_huge_pmd_share (struct vm_area_struct *vma) { -return vma->vm_flags & (VM_UFFD_WP |VM_UFFD_MINOR ); +return vma_flags_word_any (&vma->flags,VM_UFFD_WP |VM_UFFD_MINOR ); }/*
@@ -173,22 +173,22 @@ static inline bool uffd_disable_huge_pmd_share(struct vm_area_struct *vma)*/ static inline bool uffd_disable_fault_around (struct vm_area_struct *vma) { -return vma->vm_flags & (VM_UFFD_WP |VM_UFFD_MINOR ); +return vma_flags_word_any (&vma->flags,VM_UFFD_WP |VM_UFFD_MINOR ); }static inline bool userfaultfd_missing (struct vm_area_struct *vma) { -return vma->vm_flags &VM_UFFD_MISSING ; +return vma_flags_word_any (&vma->flags,VM_UFFD_MISSING ); }static inline bool userfaultfd_wp (struct vm_area_struct *vma) { -return vma->vm_flags &VM_UFFD_WP ; +return vma_test (vma,VMA_UFFD_WP_BIT ); }static inline bool userfaultfd_minor (struct vm_area_struct *vma) { -return vma->vm_flags &VM_UFFD_MINOR ; +return vma_flags_word_any (&vma->flags,VM_UFFD_MINOR ); } static inline bool userfaultfd_pte_wp(struct vm_area_struct *vma,
@@ -214,7 +214,7 @@ static inline bool vma_can_userfault(struct vm_area_struct *vma, { vm_flags &= __VM_UFFD_FLAGS; -if (vma->vm_flags &VM_DROPPABLE ) +if (vma_flags_word_any (&vma->flags,VM_DROPPABLE ))return false; if ((vm_flags & VM_UFFD_MINOR) &&
mm/filemap.c +2 -2
@@ -3365,7 +3365,7 @@ static struct file *do_async_mmap_readahead(struct vm_fault *vmf,unsigned short mmap_miss;/* If we don't want any read-ahead, don't bother */ -if (vmf->vma->vm_flags &VM_RAND_READ || !ra->ra_pages) +if (vma_test (vmf->vma,VMA_RAND_READ_BIT ) || !ra->ra_pages)return fpin;/*
@@ -3407,7 +3407,7 @@ static vm_fault_t filemap_fault_recheck_pte_none(struct vm_fault *vmf)* scenarios. Recheck the PTE without PT lock firstly, thereby reducing * the number of times we hold PT lock. */ -if (!(vma->vm_flags &VM_LOCKED )) +if (!vma_test (vma,VMA_LOCKED_BIT ))return 0 ;if (!(vmf->flags &FAULT_FLAG_ORIG_PTE_VALID ))
mm/gup.c +8 -8
@@ -590,15 +590,15 @@ static inline bool can_follow_write_common(struct page *page,return false;/* But FOLL_FORCE has no effect on shared mappings */ -if (vma->vm_flags & (VM_MAYSHARE |VM_SHARED )) +if (vma_flags_word_any (&vma->flags,VM_MAYSHARE |VM_SHARED ))return false;/* ... or read-only private ones */ -if (!(vma->vm_flags &VM_MAYWRITE )) +if (!vma_test (vma,VMA_MAYWRITE_BIT ))return false;/* ... or already writable ones that just need to take a write fault */ -if (vma->vm_flags &VM_WRITE ) +if (vma_test (vma,VMA_WRITE_BIT ))return false;/*
@@ -1277,7 +1277,7 @@ static struct vm_area_struct *gup_vma_lookup(struct mm_struct *mm,return vma;/* Only warn for half-way relevant accesses */ -if (!(vma->vm_flags &VM_GROWSDOWN )) +if (!vma_test (vma,VMA_GROWSDOWN_BIT ))return NULL ;if (vma->vm_start - addr >65536 )return NULL ;
@@ -1829,7 +1829,7 @@ long populate_vma_page_range(struct vm_area_struct *vma,* Rightly or wrongly, the VM_LOCKONFAULT case has never used * faultin_page() to break COW, so it has no work to do here. */ -if (vma->vm_flags &VM_LOCKONFAULT ) +if (vma_test (vma,VMA_LOCKONFAULT_BIT ))return nr_pages;/* ... similarly, we've never faulted in PROT_NONE pages */
@@ -1845,7 +1845,7 @@ long populate_vma_page_range(struct vm_area_struct *vma,* Otherwise, do a read fault, and use FOLL_FORCE in case it's not * readable (ie write-only or executable). */ -if ((vma->vm_flags & (VM_WRITE |VM_SHARED )) ==VM_WRITE ) +if (vma_flags_word_and (&vma->flags,VM_WRITE |VM_SHARED ) ==VM_WRITE ) gup_flags |=FOLL_WRITE ;else gup_flags |=FOLL_FORCE ;
@@ -1951,7 +1951,7 @@ int __mm_populate(unsigned long start, unsigned long len, int ignore_errors)* range with the first VMA. Also, skip undesirable VMA types. */ nend =min (end, vma->vm_end); -if (vma->vm_flags & (VM_IO |VM_PFNMAP )) +if (vma_flags_word_any (&vma->flags,VM_IO |VM_PFNMAP ))continue ;if (nstart < vma->vm_start) nstart = vma->vm_start;
@@ -2013,7 +2013,7 @@ static long __get_user_pages_locked(struct mm_struct *mm, unsigned long start,break ;/* protect what we can, including chardevs */ -if ((vma->vm_flags & (VM_IO |VM_PFNMAP )) || +if (vma_flags_word_any (&vma->flags,VM_IO |VM_PFNMAP ) || !(vm_flags & vma->vm_flags))break ;
mm/hmm.c +3 -3
@@ -81,7 +81,7 @@ static int hmm_vma_fault(unsigned long addr, unsigned long end, hmm_vma_walk->last = addr;if (required_fault &HMM_NEED_WRITE_FAULT ) { -if (!(vma->vm_flags &VM_WRITE )) +if (!vma_test (vma,VMA_WRITE_BIT ))return -EPERM ; fault_flags |=FAULT_FLAG_WRITE ; }
@@ -596,8 +596,8 @@ static int hmm_vma_walk_test(unsigned long start, unsigned long end,struct hmm_range *range = hmm_vma_walk->range;struct vm_area_struct *vma = walk->vma; -if (!(vma->vm_flags & (VM_IO |VM_PFNMAP )) && - vma->vm_flags &VM_READ ) +if (!vma_flags_word_any (&vma->flags,VM_IO |VM_PFNMAP ) && +vma_test (vma,VMA_READ_BIT ))return 0 ;/*
mm/huge_memory.c +17 -17
@@ -1071,7 +1071,7 @@ __setup("thp_anon=", setup_thp_anon);pmd_t maybe_pmd_mkwrite (pmd_t pmd,struct vm_area_struct *vma) { -if (likely (vma->vm_flags &VM_WRITE )) +if (likely (vma_test (vma,VMA_WRITE_BIT ))) pmd =pmd_mkwrite (pmd, vma);return pmd; }
@@ -1417,7 +1417,7 @@ vm_fault_t do_huge_pmd_device_private(struct vm_fault *vmf)*/ gfp_t vma_thp_gfp_mask (struct vm_area_struct *vma) { -const bool vma_madvised = vma && (vma->vm_flags &VM_HUGEPAGE ); +const bool vma_madvised = vma &&vma_test (vma,VMA_HUGEPAGE_BIT );/* Always do synchronous compaction */ if (test_bit (TRANSPARENT_HUGEPAGE_DEFRAG_DIRECT_FLAG , &transparent_hugepage_flags))
@@ -1615,10 +1615,9 @@ vm_fault_t vmf_insert_pfn_pmd(struct vm_fault *vmf, unsigned long pfn,* but we need to be consistent with PTEs and architectures that * can't support a 'special' bit. */ -BUG_ON (!(vma->vm_flags & (VM_PFNMAP |VM_MIXEDMAP ))); -BUG_ON ((vma->vm_flags & (VM_PFNMAP |VM_MIXEDMAP )) == - (VM_PFNMAP |VM_MIXEDMAP )); -BUG_ON ((vma->vm_flags &VM_PFNMAP ) &&is_cow_mapping (vma->vm_flags)); +BUG_ON (!vma_flags_word_any (&vma->flags,VM_PFNMAP |VM_MIXEDMAP )); +BUG_ON (vma_flags_word_all (&vma->flags,VM_PFNMAP |VM_MIXEDMAP )); +BUG_ON (vma_test (vma,VMA_PFNMAP_BIT ) &&is_cow_mapping (vma->vm_flags));pfnmap_setup_cachemode_pfn (pfn, &pgprot);
@@ -1646,7 +1645,7 @@ EXPORT_SYMBOL_GPL(vmf_insert_folio_pmd);#ifdef CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD static pud_t maybe_pud_mkwrite (pud_t pud,struct vm_area_struct *vma) { -if (likely (vma->vm_flags &VM_WRITE )) +if (likely (vma_test (vma,VMA_WRITE_BIT ))) pud =pud_mkwrite (pud);return pud; }
@@ -1723,10 +1722,9 @@ vm_fault_t vmf_insert_pfn_pud(struct vm_fault *vmf, unsigned long pfn,* but we need to be consistent with PTEs and architectures that * can't support a 'special' bit. */ -BUG_ON (!(vma->vm_flags & (VM_PFNMAP |VM_MIXEDMAP ))); -BUG_ON ((vma->vm_flags & (VM_PFNMAP |VM_MIXEDMAP )) == - (VM_PFNMAP |VM_MIXEDMAP )); -BUG_ON ((vma->vm_flags &VM_PFNMAP ) &&is_cow_mapping (vma->vm_flags)); +BUG_ON (!vma_flags_word_any (&vma->flags,VM_PFNMAP |VM_MIXEDMAP )); +BUG_ON (vma_flags_word_all (&vma->flags,VM_PFNMAP |VM_MIXEDMAP )); +BUG_ON (vma_test (vma,VMA_PFNMAP_BIT ) &&is_cow_mapping (vma->vm_flags));pfnmap_setup_cachemode_pfn (pfn, &pgprot);
@@ -2133,7 +2131,7 @@ static inline bool can_change_pmd_writable(struct vm_area_struct *vma, {struct page *page; -if (WARN_ON_ONCE (!(vma->vm_flags &VM_WRITE ))) +if (WARN_ON_ONCE (!vma_test (vma,VMA_WRITE_BIT )))return false;/* Don't touch entries that are not even readable (NUMA hinting). */
@@ -2148,7 +2146,7 @@ static inline bool can_change_pmd_writable(struct vm_area_struct *vma,if (userfaultfd_huge_pmd_wp (vma, pmd))return false; -if (!(vma->vm_flags &VM_SHARED )) { +if (!vma_test (vma,VMA_SHARED_BIT )) {/* See can_change_pte_writable(). */ page =vm_normal_page_pmd (vma, addr, pmd);return page &&PageAnon (page) &&PageAnonExclusive (page);
@@ -3328,7 +3326,8 @@ static bool __discard_anon_folio_pmd_locked(struct vm_area_struct *vma,if (pmd_dirty (orig_pmd))folio_set_dirty (folio); -if (folio_test_dirty (folio) && !(vma->vm_flags &VM_DROPPABLE )) { +if (folio_test_dirty (folio) && + !vma_flags_word_any (&vma->flags,VM_DROPPABLE )) {folio_set_swapbacked (folio);return false; }
@@ -3360,7 +3359,8 @@ static bool __discard_anon_folio_pmd_locked(struct vm_area_struct *vma,*/ if (pmd_dirty (orig_pmd))folio_set_dirty (folio); -if (folio_test_dirty (folio) && !(vma->vm_flags &VM_DROPPABLE )) { +if (folio_test_dirty (folio) && + !vma_flags_word_any (&vma->flags,VM_DROPPABLE )) {folio_set_swapbacked (folio);set_pmd_at (mm, addr, pmdp, orig_pmd);return false;
@@ -3374,7 +3374,7 @@ static bool __discard_anon_folio_pmd_locked(struct vm_area_struct *vma,folio_remove_rmap_pmd (folio,pmd_page (orig_pmd), vma);zap_deposited_table (mm, pmdp);add_mm_counter (mm,MM_ANONPAGES , -HPAGE_PMD_NR ); -if (vma->vm_flags &VM_LOCKED ) +if (vma_test (vma,VMA_LOCKED_BIT ))mlock_drain_local ();folio_put (folio);
@@ -4481,7 +4481,7 @@ static void split_huge_pages_all(void)static inline bool vma_not_suitable_for_thp_split (struct vm_area_struct *vma) { -return vma_is_special_huge (vma) || (vma->vm_flags &VM_IO ) || +return vma_is_special_huge (vma) ||vma_test (vma,VMA_IO_BIT ) ||is_vm_hugetlb_page (vma); }
mm/hugetlb.c +24 -24
192 lines not on this page · raw message
mm/internal.h +4 -4
@@ -1042,7 +1042,7 @@ static inline void mlock_vma_folio(struct folio *folio,* file->f_op->mmap() is using vm_insert_page(s), when VM_LOCKED may * still be set while VM_SPECIAL bits are added: so ignore it then. */ -if (unlikely ((vma->vm_flags & (VM_LOCKED |VM_SPECIAL )) ==VM_LOCKED )) +if (unlikely (vma_flags_word_and (&vma->flags,VM_LOCKED |VM_SPECIAL ) ==VM_LOCKED ))mlock_folio (folio); }
@@ -1059,7 +1059,7 @@ static inline void munlock_vma_folio(struct folio *folio,* always munlock the folio and page reclaim will correct it * if it's wrong. */ -if (unlikely (vma->vm_flags &VM_LOCKED )) +if (unlikely (vma_test (vma,VMA_LOCKED_BIT )))munlock_folio (folio); }
@@ -1383,7 +1383,7 @@ void __vunmap_range_noflush(unsigned long start, unsigned long end);static inline bool vma_is_single_threaded_private (struct vm_area_struct *vma) { -if (vma->vm_flags &VM_SHARED ) +if (vma_test (vma,VMA_SHARED_BIT ))return false;return atomic_read (&vma->vm_mm->mm_users) ==1 ;
@@ -1564,7 +1564,7 @@ static inline bool vma_soft_dirty_enabled(struct vm_area_struct *vma)* Soft-dirty is kind of special: its tracking is enabled when the * vma flags not set. */ -return !(vma->vm_flags &VM_SOFTDIRTY ); +return !vma_flags_word_any (&vma->flags,VM_SOFTDIRTY ); }static inline bool pmd_needs_soft_dirty_wp (struct vm_area_struct *vma,pmd_t pmd)
mm/khugepaged.c +1 -1
@@ -1600,7 +1600,7 @@ int collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr,* So page lock of folio does not protect from it, so we must not drop * ptl before pgt_pmd is removed, so uffd private needs pml taken now. */ -if (userfaultfd_armed (vma) && !(vma->vm_flags &VM_SHARED )) +if (userfaultfd_armed (vma) && !vma_test (vma,VMA_SHARED_BIT )) pml =pmd_lock (mm, pmd); start_pte =pte_offset_map_rw_nolock (mm, pmd, haddr, &pgt_pmd, &ptl);
mm/ksm.c +6 -6
@@ -774,7 +774,7 @@ static struct vm_area_struct *find_mergeable_vma(struct mm_struct *mm,if (ksm_test_exit (mm))return NULL ; vma =vma_lookup (mm, addr); -if (!vma || !(vma->vm_flags &VM_MERGEABLE ) || !vma->anon_vma) +if (!vma || !vma_test (vma,VMA_MERGEABLE_BIT ) || !vma->anon_vma)return NULL ;return vma; }
@@ -1224,7 +1224,7 @@ static int unmerge_and_remove_all_rmap_items(void) goto mm_exiting;for_each_vma (vmi, vma) { -if (!(vma->vm_flags &VM_MERGEABLE ) || !vma->anon_vma) +if (!vma_test (vma,VMA_MERGEABLE_BIT ) || !vma->anon_vma)continue ; err =break_ksm (vma, vma->vm_start, vma->vm_end, false);
@@ -2657,7 +2657,7 @@ static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page) goto no_vmas;for_each_vma (vmi, vma) { -if (!(vma->vm_flags &VM_MERGEABLE )) +if (!vma_test (vma,VMA_MERGEABLE_BIT ))continue ;if (ksm_scan.address < vma->vm_start) ksm_scan.address = vma->vm_start;
@@ -2850,7 +2850,7 @@ static int __ksm_del_vma(struct vm_area_struct *vma) {int err; -if (!(vma->vm_flags &VM_MERGEABLE )) +if (!vma_test (vma,VMA_MERGEABLE_BIT ))return 0 ;if (vma->anon_vma) {
@@ -2987,7 +2987,7 @@ int ksm_madvise(struct vm_area_struct *vma, unsigned long start,switch (advice) {case MADV_MERGEABLE : -if (vma->vm_flags &VM_MERGEABLE ) +if (vma_test (vma,VMA_MERGEABLE_BIT ))return 0 ;if (!vma_ksm_compatible (vma))return 0 ;
@@ -3438,7 +3438,7 @@ bool ksm_process_mergeable(struct mm_struct *mm)mmap_assert_locked (mm);VMA_ITERATOR (vmi, mm,0 );for_each_vma (vmi, vma) -if (vma->vm_flags &VM_MERGEABLE ) +if (vma_test (vma,VMA_MERGEABLE_BIT ))return true;return false;
mm/madvise.c +4 -4
@@ -592,7 +592,7 @@ static void madvise_cold_page_range(struct mmu_gather *tlb,static inline bool can_madv_lru_vma (struct vm_area_struct *vma) { -return !(vma->vm_flags & (VM_LOCKED |VM_PFNMAP |VM_HUGETLB )); +return !vma_flags_word_any (&vma->flags,VM_LOCKED |VM_PFNMAP |VM_HUGETLB ); }static long madvise_cold (struct madvise_behavior *madv_behavior)
@@ -641,7 +641,7 @@ static long madvise_pageout(struct madvise_behavior *madv_behavior)* further to pageout dirty anon pages. */ if (!vma_is_anonymous (vma) && (!can_do_file_pageout (vma) && - (vma->vm_flags &VM_MAYSHARE ))) +vma_test (vma,VMA_MAYSHARE_BIT )))return 0 ;lru_add_drain ();
@@ -1020,7 +1020,7 @@ static long madvise_remove(struct madvise_behavior *madv_behavior)mark_mmap_lock_dropped (madv_behavior); -if (vma->vm_flags &VM_LOCKED ) +if (vma_test (vma,VMA_LOCKED_BIT ))return -EINVAL ; f = vma->vm_file;
@@ -1317,7 +1317,7 @@ static bool can_madvise_modify(struct madvise_behavior *madv_behavior)return true;/* If the user could write to the mapping anyway, then this is fine. */ -if ((vma->vm_flags &VM_WRITE ) && +if (vma_test (vma,VMA_WRITE_BIT ) &&arch_vma_access_permitted (vma,/* write= */ true,/* execute= */ false,/* foreign= */ false))return true;
mm/memory.c +38 -39
267 lines not on this page · raw message
mm/mempolicy.c +2 -2
@@ -1976,7 +1976,7 @@ SYSCALL_DEFINE5(get_mempolicy, int __user *, policy,bool vma_migratable (struct vm_area_struct *vma) { -if (vma->vm_flags & (VM_IO |VM_PFNMAP )) +if (vma_flags_word_any (&vma->flags,VM_IO |VM_PFNMAP ))return false;/*
@@ -2524,7 +2524,7 @@ struct folio *vma_alloc_folio_noprof(gfp_t gfp, int order, struct vm_area_structpgoff_t ilx;struct folio *folio; -if (vma->vm_flags &VM_DROPPABLE ) +if (vma_flags_word_any (&vma->flags,VM_DROPPABLE )) gfp |= __GFP_NOWARN; pol =get_vma_policy (vma, addr, order, &ilx);
mm/migrate.c +2 -2
@@ -309,7 +309,7 @@ static bool try_to_map_unused_to_zeropage(struct page_vma_mapped_walk *pvmw,VM_BUG_ON_PAGE (pte_present (old_pte), page);VM_WARN_ON_ONCE_FOLIO (folio_is_device_private (folio), folio); -if (folio_test_mlocked (folio) || (pvmw->vma->vm_flags &VM_LOCKED ) || +if (folio_test_mlocked (folio) ||vma_test (pvmw->vma,VMA_LOCKED_BIT ) ||mm_forbids_zeropage (pvmw->vma->vm_mm))return false;
@@ -2662,7 +2662,7 @@ int migrate_misplaced_folio_prepare(struct folio *folio,* See folio_maybe_mapped_shared() on possible imprecision * when we cannot easily detect if a folio is shared. */ -if ((vma->vm_flags &VM_EXEC ) &&folio_maybe_mapped_shared (folio)) +if (vma_test (vma,VMA_EXEC_BIT ) &&folio_maybe_mapped_shared (folio))return -EACCES ;/*
mm/migrate_device.c +5 -5
@@ -739,7 +739,7 @@ int migrate_vma_setup(struct migrate_vma *args) args->start &=PAGE_MASK ; args->end &=PAGE_MASK ;if (!args->vma ||is_vm_hugetlb_page (args->vma) || - (args->vma->vm_flags &VM_SPECIAL ) ||vma_is_dax (args->vma)) +vma_flags_word_any (&args->vma->flags,VM_SPECIAL ) ||vma_is_dax (args->vma))return -EINVAL ;if (nr_pages <=0 )return -EINVAL ;
@@ -838,7 +838,7 @@ static int migrate_vma_insert_huge_pmd_page(struct migrate_vma *migrate,if (folio_is_device_private (folio)) {swp_entry_t swp_entry; -if (vma->vm_flags &VM_WRITE ) +if (vma_test (vma,VMA_WRITE_BIT )) swp_entry =make_writable_device_private_entry (page_to_pfn (page));else
@@ -851,7 +851,7 @@ static int migrate_vma_insert_huge_pmd_page(struct migrate_vma *migrate, goto abort; } entry =folio_mk_pmd (folio, vma->vm_page_prot); -if (vma->vm_flags &VM_WRITE ) +if (vma_test (vma,VMA_WRITE_BIT )) entry =pmd_mkwrite (pmd_mkdirty (entry), vma); }
@@ -1036,7 +1036,7 @@ static void migrate_vma_insert_page(struct migrate_vma *migrate,if (folio_is_device_private (folio)) {swp_entry_t swp_entry; -if (vma->vm_flags &VM_WRITE ) +if (vma_test (vma,VMA_WRITE_BIT )) swp_entry =make_writable_device_private_entry (page_to_pfn (page));else
@@ -1050,7 +1050,7 @@ static void migrate_vma_insert_page(struct migrate_vma *migrate, goto abort; } entry =mk_pte (page, vma->vm_page_prot); -if (vma->vm_flags &VM_WRITE ) +if (vma_test (vma,VMA_WRITE_BIT )) entry =pte_mkwrite (pte_mkdirty (entry), vma); }
mm/mlock.c +4 -4
@@ -329,7 +329,7 @@ static inline bool allow_mlock_munlock(struct folio *folio,* be split. And the pages are not in VM_LOCKed VMA * can be reclaimed. */ -if (!(vma->vm_flags &VM_LOCKED )) +if (!vma_test (vma,VMA_LOCKED_BIT ))return true;/* folio_within_range() cannot take KSM, but any small folio is OK */
@@ -368,7 +368,7 @@ static int mlock_pte_range(pmd_t *pmd, unsigned long addr, folio =pmd_folio (*pmd);if (folio_is_zone_device (folio)) goto out; -if (vma->vm_flags &VM_LOCKED ) +if (vma_test (vma,VMA_LOCKED_BIT ))mlock_folio (folio);else munlock_folio (folio);
@@ -393,7 +393,7 @@ static int mlock_pte_range(pmd_t *pmd, unsigned long addr,if (!allow_mlock_munlock (folio, vma, start, end, step)) goto next_entry; -if (vma->vm_flags &VM_LOCKED ) +if (vma_test (vma,VMA_LOCKED_BIT ))mlock_folio (folio);else munlock_folio (folio);
@@ -583,7 +583,7 @@ static unsigned long count_mm_mlocked_page_nr(struct mm_struct *mm, end = start + len;for_each_vma_range (vmi, vma, end) { -if (vma->vm_flags &VM_LOCKED ) { +if (vma_test (vma,VMA_LOCKED_BIT )) {if (start > vma->vm_start) count -= (start - vma->vm_start);if (end < vma->vm_end) {
mm/mmap.c +8 -8
@@ -992,7 +992,7 @@ struct vm_area_struct *find_extend_vma_locked(struct mm_struct *mm, unsigned lon start = vma->vm_start;if (expand_stack_locked (vma, addr))return NULL ; -if (vma->vm_flags &VM_LOCKED ) +if (vma_test (vma,VMA_LOCKED_BIT ))populate_vma_page_range (vma, addr, start,NULL );return vma; }
@@ -1117,18 +1117,18 @@ SYSCALL_DEFINE5(remap_file_pages, unsigned long, start, unsigned long, size,*/ vma =vma_lookup (mm, start); -if (!vma || !(vma->vm_flags &VM_SHARED )) { +if (!vma || !vma_test (vma,VMA_SHARED_BIT )) {mmap_read_unlock (mm);return -EINVAL ; } - prot |= vma->vm_flags &VM_READ ?PROT_READ :0 ; - prot |= vma->vm_flags &VM_WRITE ?PROT_WRITE :0 ; - prot |= vma->vm_flags &VM_EXEC ?PROT_EXEC :0 ; + prot |=vma_test (vma,VMA_READ_BIT ) ?PROT_READ :0 ; + prot |=vma_test (vma,VMA_WRITE_BIT ) ?PROT_WRITE :0 ; + prot |=vma_test (vma,VMA_EXEC_BIT ) ?PROT_EXEC :0 ; flags &=MAP_NONBLOCK ; flags |=MAP_SHARED |MAP_FIXED |MAP_POPULATE ; -if (vma->vm_flags &VM_LOCKED ) +if (vma_test (vma,VMA_LOCKED_BIT )) flags |=MAP_LOCKED ;/* Save vm_flags used to calculate prot and flags, and recheck later. */
@@ -1296,7 +1296,7 @@ void exit_mmap(struct mm_struct *mm)*/ vma_iter_set (&vmi, vma->vm_end); do { - if (vma->vm_flags & VM_ACCOUNT) + if (vma_test(vma, VMA_ACCOUNT_BIT)) nr_accounted += vma_pages(vma); vma_mark_detached(vma); remove_vma(vma);
@@ -1700,7 +1700,7 @@ bool mmap_read_lock_maybe_expand(struct mm_struct *mm,return true; } -if (!(new_vma->vm_flags &VM_GROWSDOWN )) +if (!vma_test (new_vma,VMA_GROWSDOWN_BIT ))return false;mmap_write_lock (mm);
mm/mmap_lock.c +2 -2
@@ -436,7 +436,7 @@ struct vm_area_struct *lock_mm_and_find_vma(struct mm_struct *mm,* Well, dang. We might still be successful, but only * if we can extend a vma to do so. */ -if (!vma || !(vma->vm_flags &VM_GROWSDOWN )) { +if (!vma || !vma_test (vma,VMA_GROWSDOWN_BIT )) {mmap_read_unlock (mm);return NULL ; }
@@ -459,7 +459,7 @@ struct vm_area_struct *lock_mm_and_find_vma(struct mm_struct *mm, goto fail;if (vma->vm_start <= addr) goto success; -if (!(vma->vm_flags &VM_GROWSDOWN )) +if (!vma_test (vma,VMA_GROWSDOWN_BIT )) goto fail; }
mm/mprotect.c +6 -6
@@ -40,7 +40,7 @@static bool maybe_change_pte_writable (struct vm_area_struct *vma,pte_t pte) { -if (WARN_ON_ONCE (!(vma->vm_flags &VM_WRITE ))) +if (WARN_ON_ONCE (!vma_test (vma,VMA_WRITE_BIT )))return false;/* Don't touch entries that are not even readable. */
@@ -97,7 +97,7 @@ static bool can_change_shared_pte_writable(struct vm_area_struct *vma,bool can_change_pte_writable (struct vm_area_struct *vma,unsigned long addr,pte_t pte) { -if (!(vma->vm_flags &VM_SHARED )) +if (!vma_test (vma,VMA_SHARED_BIT ))return can_change_private_pte_writable (vma, addr, pte);return can_change_shared_pte_writable (vma, pte);
@@ -194,7 +194,7 @@ static void set_write_prot_commit_flush_ptes(struct vm_area_struct *vma, {bool set_write; -if (vma->vm_flags &VM_SHARED ) { +if (vma_test (vma,VMA_SHARED_BIT )) { set_write =can_change_shared_pte_writable (vma, ptent);prot_commit_flush_ptes (vma, addr, ptep, oldpte, ptent, nr_ptes,/* idx = */ 0 , set_write, tlb);
@@ -854,7 +854,7 @@ static int do_mprotect_pkey(unsigned long start, size_t len, goto out; start = vma->vm_start; error = -EINVAL ; -if (!(vma->vm_flags &VM_GROWSDOWN )) +if (!vma_test (vma,VMA_GROWSDOWN_BIT )) goto out; }else {if (vma->vm_start > start)
@@ -862,7 +862,7 @@ static int do_mprotect_pkey(unsigned long start, size_t len,if (unlikely (grows &PROT_GROWSUP )) { end = vma->vm_end; error = -EINVAL ; -if (!(vma->vm_flags &VM_GROWSUP )) +if (!vma_flags_word_any (&vma->flags,VM_GROWSUP )) goto out; } }
@@ -885,7 +885,7 @@ static int do_mprotect_pkey(unsigned long start, size_t len, }/* Does the application expect PROT_READ to imply PROT_EXEC */ -if (rier && (vma->vm_flags &VM_MAYEXEC )) +if (rier &&vma_test (vma,VMA_MAYEXEC_BIT )) prot |=PROT_EXEC ;/*
mm/mremap.c +9 -9
@@ -951,7 +951,7 @@ static unsigned long vrm_set_new_addr(struct vma_remap_struct *vrm)if (vrm->flags &MREMAP_FIXED ) map_flags |=MAP_FIXED ; -if (vma->vm_flags &VM_MAYSHARE ) +if (vma_test (vma,VMA_MAYSHARE_BIT )) map_flags |=MAP_SHARED ; res = get_unmapped_area(vma->vm_file, new_addr, vrm->new_len, pgoff,
@@ -973,7 +973,7 @@ static bool vrm_calc_charge(struct vma_remap_struct *vrm) {unsigned long charged; -if (!(vrm->vma->vm_flags &VM_ACCOUNT )) +if (!vma_test (vrm->vma,VMA_ACCOUNT_BIT ))return true;/*
@@ -1000,7 +1000,7 @@ static bool vrm_calc_charge(struct vma_remap_struct *vrm)*/ static void vrm_uncharge (struct vma_remap_struct *vrm) { -if (!(vrm->vma->vm_flags &VM_ACCOUNT )) +if (!vma_test (vrm->vma,VMA_ACCOUNT_BIT ))return ;vm_unacct_memory (vrm->charged);
@@ -1020,7 +1020,7 @@ static void vrm_stat_account(struct vma_remap_struct *vrm,struct vm_area_struct *vma = vrm->vma;vm_stat_account (mm, vma->vm_flags, pages); -if (vma->vm_flags &VM_LOCKED ) +if (vma_test (vma,VMA_LOCKED_BIT )) mm->locked_vm += pages; }
@@ -1094,7 +1094,7 @@ static void unmap_source_vma(struct vma_remap_struct *vrm)* arose, in which case we _do_ wish to unmap the _new_ VMA, which means * we actually _do_ want it be unaccounted. */ -bool accountable_move = (vma->vm_flags &VM_ACCOUNT ) && +bool accountable_move =vma_test (vma,VMA_ACCOUNT_BIT ) && !(vrm->flags &MREMAP_DONTUNMAP );/*
@@ -1687,14 +1687,14 @@ static int check_prep_vma(struct vma_remap_struct *vrm)* based on the original. There are no known use cases for this * behavior. As a result, fail such attempts. */ -if (!old_len && !(vma->vm_flags & (VM_SHARED |VM_MAYSHARE ))) { +if (!old_len && !vma_flags_word_any (&vma->flags,VM_SHARED |VM_MAYSHARE )) {pr_warn_once ("%s (%d): attempted to duplicate a private mapping with mremap. This is not supported.\n" , current->comm, current->pid);return -EINVAL ; }if ((vrm->flags &MREMAP_DONTUNMAP ) && - (vma->vm_flags & (VM_DONTEXPAND |VM_PFNMAP ))) +vma_flags_word_any (&vma->flags,VM_DONTEXPAND |VM_PFNMAP ))return -EINVAL ;/*
@@ -1724,7 +1724,7 @@ static int check_prep_vma(struct vma_remap_struct *vrm)return 0 ;/* We are expanding and the VMA is mlock()'d so we need to populate. */ -if (vma->vm_flags &VM_LOCKED ) +if (vma_test (vma,VMA_LOCKED_BIT )) vrm->populate_expand = true;/* Need to be careful about a growing mapping */
@@ -1733,7 +1733,7 @@ static int check_prep_vma(struct vma_remap_struct *vrm)if (pgoff + (new_len >>PAGE_SHIFT ) < pgoff)return -EINVAL ; -if (vma->vm_flags & (VM_DONTEXPAND |VM_PFNMAP )) +if (vma_flags_word_any (&vma->flags,VM_DONTEXPAND |VM_PFNMAP ))return -EFAULT ;if (!mlock_future_ok (mm, vma->vm_flags, vrm->delta))
mm/mseal.c +1 -1
@@ -68,7 +68,7 @@ static int mseal_apply(struct mm_struct *mm,for_each_vma_range (vmi, vma, end) {unsigned long curr_end =MIN (vma->vm_end, end); -if (!(vma->vm_flags &VM_SEALED )) { +if (!vma_flags_word_any (&vma->flags,VM_SEALED )) { vma =vma_modify_flags (&vmi, prev, vma, curr_start, curr_end, vma->vm_flags |VM_SEALED );
mm/msync.c +2 -2
@@ -80,7 +80,7 @@ SYSCALL_DEFINE3(msync, unsigned long, start, size_t, len, int, flags) }/* Here vma->vm_start <= start < vma->vm_end. */ if ((flags &MS_INVALIDATE ) && - (vma->vm_flags &VM_LOCKED )) { +vma_test (vma,VMA_LOCKED_BIT )) { error = -EBUSY ; goto out_unlock; }
@@ -90,7 +90,7 @@ SYSCALL_DEFINE3(msync, unsigned long, start, size_t, len, int, flags) fend = fstart + (min (end, vma->vm_end) - start) -1 ; start = vma->vm_end;if ((flags &MS_SYNC ) && file && - (vma->vm_flags &VM_SHARED )) { +vma_test (vma,VMA_SHARED_BIT )) {get_file (file);mmap_read_unlock (mm); error =vfs_fsync_range (file, fstart, fend,1 );
mm/nommu.c +8 -8
@@ -1172,7 +1172,7 @@ unsigned long do_mmap(struct file *file,/* set up the mapping * - the region is filled in if NOMMU_MAP_DIRECT is still set */ -if (file && vma->vm_flags &VM_SHARED ) +if (file &&vma_test (vma,VMA_SHARED_BIT )) ret =do_mmap_shared_file (vma);else ret =do_mmap_private (vma, region, len, capabilities);
@@ -1205,7 +1205,7 @@ unsigned long do_mmap(struct file *file,/* we flush the region from the icache only when the first executable * mapping of it is made */ -if (vma->vm_flags &VM_EXEC && !region->vm_icache_flushed) { +if (vma_test (vma,VMA_EXEC_BIT ) && !region->vm_icache_flushed) {flush_icache_user_range (region->vm_start, region->vm_end); region->vm_icache_flushed = true; }
@@ -1613,7 +1613,7 @@ int remap_vmalloc_range(struct vm_area_struct *vma, void *addr, {unsigned int size = vma->vm_end - vma->vm_start; -if (!(vma->vm_flags &VM_USERMAP )) +if (!vma_test (vma,VMA_USERMAP_BIT ))return -EINVAL ; vma->vm_start = (unsigned long )(addr + (pgoff <<PAGE_SHIFT ));
@@ -1655,10 +1655,10 @@ static int __access_remote_vm(struct mm_struct *mm, unsigned long addr, len = vma->vm_end - addr;/* only read or write mappings where it is permitted */ -if (write && vma->vm_flags &VM_MAYWRITE ) +if (write &&vma_test (vma,VMA_MAYWRITE_BIT ))copy_to_user_page (vma,NULL , addr, (void *) addr, buf, len); -else if (!write && vma->vm_flags &VM_MAYREAD ) +else if (!write &&vma_test (vma,VMA_MAYREAD_BIT ))copy_from_user_page (vma,NULL , addr, buf, (void *) addr, len);else
@@ -1741,7 +1741,7 @@ static int __copy_remote_vm_str(struct mm_struct *mm, unsigned long addr, len = vma->vm_end - addr;/* only read mappings where it is permitted */ -if (vma->vm_flags &VM_MAYREAD ) { +if (vma_test (vma,VMA_MAYREAD_BIT )) { ret =strscpy (buf, (char *)addr, len);if (ret <0 ) ret = len -1 ;
@@ -1819,7 +1819,7 @@ int nommu_shrink_inode_mappings(struct inode *inode, size_t size, vma_interval_tree_foreach(vma, &inode->i_mapping->i_mmap, low, high) {/* found one - only interested if it's shared out of the page * cache */ -if (vma->vm_flags &VM_SHARED ) { +if (vma_test (vma,VMA_SHARED_BIT )) {i_mmap_unlock_read (inode->i_mapping);up_write (&nommu_region_sem);return -ETXTBSY ;/* not quite true, but near enough */
@@ -1833,7 +1833,7 @@ int nommu_shrink_inode_mappings(struct inode *inode, size_t size,* shouldn't be any */ vma_interval_tree_foreach(vma, &inode->i_mapping->i_mmap, 0, ULONG_MAX) { -if (!(vma->vm_flags &VM_SHARED )) +if (!vma_test (vma,VMA_SHARED_BIT ))continue ; region = vma->vm_region;
mm/oom_kill.c +2 -2
@@ -533,7 +533,7 @@ static bool __oom_reap_task_mm(struct mm_struct *mm)* of the address space. */ mas_for_each_rev (&mas, vma,0 ) { -if (vma->vm_flags & (VM_HUGETLB |VM_PFNMAP )) +if (vma_flags_word_any (&vma->flags,VM_HUGETLB |VM_PFNMAP ))continue ;/*
@@ -546,7 +546,7 @@ static bool __oom_reap_task_mm(struct mm_struct *mm)* we do not want to block exit_mmap by keeping mm ref * count elevated without a good reason. */ -if (vma_is_anonymous (vma) || !(vma->vm_flags &VM_SHARED )) { +if (vma_is_anonymous (vma) || !vma_test (vma,VMA_SHARED_BIT )) {struct mmu_notifier_range range;struct mmu_gather tlb;
mm/pagewalk.c +1 -1
@@ -376,7 +376,7 @@ static int walk_page_test(unsigned long start, unsigned long end,* define their ->pte_hole() callbacks, so let's delegate them to handle * vma(VM_PFNMAP). */ -if (vma->vm_flags &VM_PFNMAP ) { +if (vma_test (vma,VMA_PFNMAP_BIT )) {int err =1 ;if (ops->pte_hole) err = ops->pte_hole (start, end,-1 , walk);
mm/rmap.c +9 -7
@@ -831,7 +831,7 @@ static bool folio_referenced_one(struct folio *folio,while (page_vma_mapped_walk (&pvmw)) { address = pvmw.address; -if (vma->vm_flags &VM_LOCKED ) { +if (vma_test (vma,VMA_LOCKED_BIT )) { ptes++; pra->mapcount--;
@@ -1069,7 +1069,7 @@ static bool page_mkclean_one(struct folio *folio, struct vm_area_struct *vma,static bool invalid_mkclean_vma (struct vm_area_struct *vma,void *arg) { -if (vma->vm_flags &VM_SHARED ) +if (vma_test (vma,VMA_SHARED_BIT ))return false;return true;
@@ -1531,7 +1531,8 @@ void folio_add_new_anon_rmap(struct folio *folio, struct vm_area_struct *vma,* VM_DROPPABLE mappings don't swap; instead they're just dropped when * under memory pressure. */ -if (!folio_test_swapbacked (folio) && !(vma->vm_flags &VM_DROPPABLE )) +if (!folio_test_swapbacked (folio) && + !vma_flags_word_any (&vma->flags,VM_DROPPABLE ))__folio_set_swapbacked (folio);__folio_set_anon (folio, vma, address, exclusive);
@@ -1902,7 +1903,7 @@ static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma,* If the folio is in an mlock()d vma, we must not swap it out. */ if (!(flags &TTU_IGNORE_MLOCK ) && - (vma->vm_flags &VM_LOCKED )) { +vma_test (vma,VMA_LOCKED_BIT )) { ptes++;/*
@@ -2121,7 +2122,8 @@ static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma,*/ smp_rmb (); -if (folio_test_dirty (folio) && !(vma->vm_flags &VM_DROPPABLE )) { +if (folio_test_dirty (folio) && + !vma_flags_word_any (&vma->flags,VM_DROPPABLE )) {/* * redirtied either using the page table or a previously * obtained GUP reference.
@@ -2212,7 +2214,7 @@ static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma, }else {folio_remove_rmap_ptes (folio, subpage, nr_pages, vma); } -if (vma->vm_flags &VM_LOCKED ) +if (vma_test (vma,VMA_LOCKED_BIT ))mlock_drain_local ();folio_put_refs (folio, nr_pages);
@@ -2574,7 +2576,7 @@ static bool try_to_migrate_one(struct folio *folio, struct vm_area_struct *vma,hugetlb_remove_rmap (folio); elsefolio_remove_rmap_pte (folio ,subpage ,vma ); -if (vma->vm_flags &VM_LOCKED ) +if (vma_test (vma,VMA_LOCKED_BIT ))mlock_drain_local ();folio_put (folio); }
mm/swap.c +2 -1
@@ -524,7 +524,8 @@ void folio_add_lru_vma(struct folio *folio, struct vm_area_struct *vma) {VM_BUG_ON_FOLIO (folio_test_lru (folio), folio); -if (unlikely ((vma->vm_flags & (VM_LOCKED |VM_SPECIAL )) ==VM_LOCKED )) +if (unlikely (vma_flags_word_and (&vma->flags,VM_LOCKED |VM_SPECIAL ) == +VM_LOCKED ))mlock_new_folio (folio);else folio_add_lru (folio);
mm/userfaultfd.c +17 -16
@@ -48,7 +48,7 @@ struct vm_area_struct *find_vma_and_prepare_anon(struct mm_struct *mm, vma =vma_lookup (mm, addr);if (!vma) vma =ERR_PTR (-ENOENT ); -else if (!(vma->vm_flags &VM_SHARED ) && +else if (!vma_test (vma,VMA_SHARED_BIT ) &&unlikely (anon_vma_prepare (vma))) vma =ERR_PTR (-ENOMEM );
@@ -77,7 +77,7 @@ static struct vm_area_struct *uffd_lock_vma(struct mm_struct *mm,* We know we're going to need to use anon_vma, so check * that early. */ -if (!(vma->vm_flags &VM_SHARED ) &&unlikely (!vma->anon_vma)) +if (!vma_test (vma,VMA_SHARED_BIT ) &&unlikely (!vma->anon_vma))vma_end_read (vma);else return vma;
@@ -173,8 +173,8 @@ int mfill_atomic_install_pte(pmd_t *dst_pmd,int ret;struct mm_struct *dst_mm = dst_vma->vm_mm;pte_t _dst_pte, *dst_pte; -bool writable = dst_vma->vm_flags &VM_WRITE ; -bool vm_shared = dst_vma->vm_flags &VM_SHARED ; +bool writable =vma_test (dst_vma,VMA_WRITE_BIT ); +bool vm_shared =vma_test (dst_vma,VMA_SHARED_BIT );spinlock_t *ptl;struct folio *folio =page_folio (page);bool page_in_cache =folio_mapping (folio);
@@ -677,7 +677,7 @@ static __always_inline ssize_t mfill_atomic_pte(pmd_t *dst_pmd,* only happens in the pagetable (to verify it's still none) * and not in the radix tree. */ -if (!(dst_vma->vm_flags &VM_SHARED )) { +if (!vma_test (dst_vma,VMA_SHARED_BIT )) {if (uffd_flags_mode_is (flags,MFILL_ATOMIC_COPY )) err = mfill_atomic_pte_copy(dst_pmd, dst_vma, dst_addr, src_addr,
@@ -749,14 +749,14 @@ static __always_inline ssize_t mfill_atomic(struct userfaultfd_ctx *ctx,* it will overwrite vm_ops, so vma_is_anonymous must return false. */ if (WARN_ON_ONCE (vma_is_anonymous (dst_vma) && - dst_vma->vm_flags &VM_SHARED )) +vma_test (dst_vma,VMA_SHARED_BIT ))) goto out_unlock;/* * validate 'mode' now that we know the dst_vma: don't allow * a wrprotect copy if the userfaultfd didn't register as WP. */ -if ((flags &MFILL_ATOMIC_WP ) && !(dst_vma->vm_flags &VM_UFFD_WP )) +if ((flags &MFILL_ATOMIC_WP ) && !vma_test (dst_vma,VMA_UFFD_WP_BIT )) goto out_unlock;/*
@@ -1528,8 +1528,8 @@ static inline bool move_splits_huge_pmd(unsigned long dst_addr,static inline bool vma_move_compatible (struct vm_area_struct *vma) { -return !(vma->vm_flags & (VM_PFNMAP |VM_IO |VM_HUGETLB | -VM_MIXEDMAP |VM_SHADOW_STACK )); +return !vma_flags_word_any (&vma->flags,VM_PFNMAP |VM_IO |VM_HUGETLB | +VM_MIXEDMAP |VM_SHADOW_STACK ); } static int validate_move_areas(struct userfaultfd_ctx *ctx,
@@ -1537,19 +1537,20 @@ static int validate_move_areas(struct userfaultfd_ctx *ctx, struct vm_area_struct *dst_vma) {/* Only allow moving if both have the same access and protection */ -if ((src_vma->vm_flags &VM_ACCESS_FLAGS ) != (dst_vma->vm_flags &VM_ACCESS_FLAGS ) || +if (vma_flags_word_and (&src_vma->flags,VM_ACCESS_FLAGS ) != +vma_flags_word_and (&dst_vma->flags,VM_ACCESS_FLAGS ) ||pgprot_val (src_vma->vm_page_prot) !=pgprot_val (dst_vma->vm_page_prot))return -EINVAL ;/* Only allow moving if both are mlocked or both aren't */ -if ((src_vma->vm_flags &VM_LOCKED ) != (dst_vma->vm_flags &VM_LOCKED )) +if (vma_test (src_vma,VMA_LOCKED_BIT ) !=vma_test (dst_vma,VMA_LOCKED_BIT ))return -EINVAL ;/* * For now, we keep it simple and only move between writable VMAs. * Access flags are equal, therefore checking only the source is enough. */ -if (!(src_vma->vm_flags &VM_WRITE )) +if (!vma_test (src_vma,VMA_WRITE_BIT ))return -EINVAL ;/* Check if vma flags indicate content which can be moved */
@@ -1796,12 +1797,12 @@ ssize_t move_pages(struct userfaultfd_ctx *ctx, unsigned long dst_start,* vma. */ err = -EINVAL ; -if (src_vma->vm_flags &VM_SHARED ) +if (vma_test (src_vma,VMA_SHARED_BIT )) goto out_unlock;if (src_start + len > src_vma->vm_end) goto out_unlock; -if (dst_vma->vm_flags &VM_SHARED ) +if (vma_test (dst_vma,VMA_SHARED_BIT )) goto out_unlock;if (dst_start + len > dst_vma->vm_end) goto out_unlock;
@@ -1948,7 +1949,7 @@ static void userfaultfd_set_vm_flags(struct vm_area_struct *vma,* userfaultfd-wp is enabled (see vma_wants_writenotify()). We'll simply * recalculate vma->vm_page_prot whenever userfaultfd-wp changes. */ -if ((vma->vm_flags &VM_SHARED ) && uffd_wp_changed) +if (vma_test (vma,VMA_SHARED_BIT ) && uffd_wp_changed)vma_set_page_prot (vma); }
@@ -2023,7 +2024,7 @@ int userfaultfd_register_range(struct userfaultfd_ctx *ctx,VM_WARN_ON_ONCE (!vma_can_userfault (vma, vm_flags, wp_async));VM_WARN_ON_ONCE (vma->vm_userfaultfd_ctx.ctx && vma->vm_userfaultfd_ctx.ctx != ctx); -VM_WARN_ON_ONCE (!(vma->vm_flags &VM_MAYWRITE )); +VM_WARN_ON_ONCE (!vma_test (vma,VMA_MAYWRITE_BIT ));/* * Nothing to do: this vma is already registered into this
mm/vma.c +20 -17
@@ -89,7 +89,7 @@ static inline bool is_mergeable_vma(struct vma_merge_struct *vmg, bool merge_nexif (!mpol_equal (vmg->policy,vma_policy (vma)))return false; -if ((vma->vm_flags ^ vmg->vm_flags) & ~VM_IGNORE_MERGE ) +if ((vma_flags_get_word (&vma->flags) ^ vmg->vm_flags) & ~VM_IGNORE_MERGE )return false;if (vma->vm_file != vmg->file)return false;
@@ -894,13 +894,13 @@ static __must_check struct vm_area_struct *vma_merge_existing_range(if (merge_right) {vma_start_write (next); vmg->target = next; - sticky_flags |= (next->vm_flags &VM_STICKY ); + sticky_flags |=vma_flags_word_and (&next->flags,VM_STICKY ); }if (merge_left) {vma_start_write (prev); vmg->target = prev; - sticky_flags |= (prev->vm_flags &VM_STICKY ); + sticky_flags |=vma_flags_word_and (&prev->flags,VM_STICKY ); }if (merge_both) {
@@ -1124,7 +1124,7 @@ int vma_expand(struct vma_merge_struct *vmg)vm_flags_t sticky_flags; sticky_flags = vmg->vm_flags &VM_STICKY ; - sticky_flags |= target->vm_flags &VM_STICKY ; + sticky_flags |=vma_flags_word_and (&target->flags,VM_STICKY );VM_WARN_ON_VMG (!target, vmg);
@@ -1134,7 +1134,7 @@ int vma_expand(struct vma_merge_struct *vmg)if (next && (target != next) && (vmg->end == next->vm_end)) {int ret; - sticky_flags |= next->vm_flags &VM_STICKY ; + sticky_flags |=vma_flags_word_and (&next->flags,VM_STICKY ); remove_next = true;/* This should already have been checked by this point. */ VM_WARN_ON_VMG (!can_merge_remove_vma (next), vmg);
@@ -1993,14 +1993,13 @@ static bool vm_ops_needs_writenotify(const struct vm_operations_struct *vm_ops)static bool vma_is_shared_writable (struct vm_area_struct *vma) { -return (vma->vm_flags & (VM_WRITE |VM_SHARED )) == - (VM_WRITE |VM_SHARED ); +return vma_flags_word_all (&vma->flags,VM_WRITE |VM_SHARED ); }static bool vma_fs_can_writeback (struct vm_area_struct *vma) {/* No managed pages to writeback. */ -if (vma->vm_flags &VM_PFNMAP ) +if (vma_test (vma,VMA_PFNMAP_BIT ))return false; return vma->vm_file && vma->vm_file->f_mapping &&
@@ -2435,7 +2434,7 @@ static int __mmap_new_file_vma(struct mmap_state *map,*/ VM_WARN_ON_ONCE (map->vm_flags != vma->vm_flags && !(map->vm_flags &VM_MAYWRITE ) && - (vma->vm_flags &VM_MAYWRITE )); +vma_test (vma,VMA_MAYWRITE_BIT )); map->file = vma->vm_file; map->vm_flags = vma->vm_flags;
@@ -3004,8 +3003,12 @@ static int acct_stack_growth(struct vm_area_struct *vma,return -ENOMEM ;/* Check to ensure the stack will not grow into a hugetlb-only region */ - new_start = (vma->vm_flags &VM_GROWSUP ) ? vma->vm_start : - vma->vm_end - size; + +if (vma_flags_word_any (&vma->flags,VM_GROWSUP )) + new_start = vma->vm_start; +else + new_start = vma->vm_end - size; +if (is_hugepage_only_range (vma->vm_mm, new_start, size))return -EFAULT ;
@@ -3032,7 +3035,7 @@ int expand_upwards(struct vm_area_struct *vma, unsigned long address)int error =0 ;VMA_ITERATOR (vmi, mm, vma->vm_start); -if (!(vma->vm_flags &VM_GROWSUP )) +if (!vma_test (vma,VMA_GROWSUP_BIT ))return -EFAULT ;mmap_assert_write_locked (mm);
@@ -3086,7 +3089,7 @@ int expand_upwards(struct vm_area_struct *vma, unsigned long address)if (vma->vm_pgoff + (size >>PAGE_SHIFT ) >= vma->vm_pgoff) { error =acct_stack_growth (vma, size, grow);if (!error) { -if (vma->vm_flags &VM_LOCKED ) +if (vma_test (vma,VMA_LOCKED_BIT )) mm->locked_vm += grow;vm_stat_account (mm, vma->vm_flags, grow);anon_vma_interval_tree_pre_update_vma (vma);
@@ -3117,7 +3120,7 @@ int expand_downwards(struct vm_area_struct *vma, unsigned long address)int error =0 ;VMA_ITERATOR (vmi, mm, vma->vm_start); -if (!(vma->vm_flags &VM_GROWSDOWN )) +if (!vma_test (vma,VMA_GROWSDOWN_BIT ))return -EFAULT ;mmap_assert_write_locked (mm);
@@ -3165,7 +3168,7 @@ int expand_downwards(struct vm_area_struct *vma, unsigned long address)if (grow <= vma->vm_pgoff) { error =acct_stack_growth (vma, size, grow);if (!error) { -if (vma->vm_flags &VM_LOCKED ) +if (vma_test (vma,VMA_LOCKED_BIT )) mm->locked_vm += grow;vm_stat_account (mm, vma->vm_flags, grow);anon_vma_interval_tree_pre_update_vma (vma);
@@ -3215,7 +3218,7 @@ int insert_vm_struct(struct mm_struct *mm, struct vm_area_struct *vma)if (find_vma_intersection (mm, vma->vm_start, vma->vm_end))return -ENOMEM ; -if ((vma->vm_flags &VM_ACCOUNT ) && +if (vma_test (vma,VMA_ACCOUNT_BIT ) &&security_vm_enough_memory_mm (mm, charged))return -ENOMEM ;
@@ -3237,7 +3240,7 @@ int insert_vm_struct(struct mm_struct *mm, struct vm_area_struct *vma) }if (vma_link (mm, vma)) { -if (vma->vm_flags &VM_ACCOUNT ) +if (vma_test (vma,VMA_ACCOUNT_BIT ))vm_unacct_memory (charged);return -ENOMEM ; }
mm/vma.h +3 -3
@@ -342,9 +342,9 @@ static inline bool vma_wants_manual_pte_write_upgrade(struct vm_area_struct *vma* private mappings, that's always the case when we have write * permissions as we properly have to handle COW. */ -if (vma->vm_flags &VM_SHARED ) +if (vma_test (vma,VMA_SHARED_BIT ))return vma_wants_writenotify (vma, vma->vm_page_prot); -return !!(vma->vm_flags &VM_WRITE ); +return vma_test (vma,VMA_WRITE_BIT ); }#ifdef CONFIG_MMU
@@ -535,7 +535,7 @@ struct vm_area_struct *vma_iter_next_rewind(struct vma_iterator *vmi,#ifdef CONFIG_64BIT static inline bool vma_is_sealed (struct vm_area_struct *vma) { -return (vma->vm_flags &VM_SEALED ); +return vma_test (vma,VMA_SEALED_BIT ); }#else static inline bool vma_is_sealed (struct vm_area_struct *vma)
mm/vmscan.c +2 -2
@@ -3331,7 +3331,7 @@ static int should_skip_vma(unsigned long start, unsigned long end, struct mm_walif (!vma_has_recency (vma))return true; -if (vma->vm_flags & (VM_LOCKED |VM_SPECIAL )) +if (vma_flags_word_any (&vma->flags,VM_LOCKED |VM_SPECIAL ))return true;if (vma ==get_gate_vma (vma->vm_mm))
@@ -4221,7 +4221,7 @@ bool lru_gen_look_around(struct page_vma_mapped_walk *pvmw)return true;/* exclude special VMAs containing anon pages from COW */ -if (vma->vm_flags &VM_SPECIAL ) +if (vma_flags_word_any (&vma->flags,VM_SPECIAL ))return true;/* avoid taking the LRU lock under the PTL when possible */
tools/testing/vma/vma_internal.h +52 -0
58 lines not on this page · raw message
Signature · 1 line
2.51.0
Re: [PATCH 4/4] mm: introduce and use VMA flag test helpers
Quoted 5 lines · Lorenzo Stoakes
We introduce
vma_flags_test()andvma_test()(the latter operating on a VMA, the former on a pointer to avma_flags_tvalue).It's useful to have both, as many functions modify a local VMA flags variable before setting the VMA flags to this value.
Hmm, sure would be nice to not have this inconsistency though.
It is a bit wordy but with the C preprocessor we can make this work:
struct vm_flags_t {DECLARE_BITMAP(..)};
void func(..)
{
struct vm_flags_t flags = OR_VMA_FLAGS(VMA_READ_BIT, VMA_WRITE_BIT);
flags = vm_flags_or(flags, OR_VMA_FLAGS(VMA_MAYREAD_BIT, VMA_MAYWRITE_BIT);
}
Where OR_VMA_FLAGS's OR's together its __VA_ARGS__ and returns a struct vm_flags_t.
Would that be interesting? Eliminate the inconsistency?
eg
Or other similar solutions.
The compiler is pretty smart so this would all fold away to very few instructions.
Then everything only works with _BIT and we don't have the special first word situation.
Jason
Re: [PATCH 4/4] mm: introduce and use VMA flag test helpers
Quoted 8 lines · Jason Gunthorpe
Quoted 5 lines · Lorenzo Stoakes
We introduce
vma_flags_test()andvma_test()(the latter operating on a VMA, the former on a pointer to avma_flags_tvalue).It's useful to have both, as many functions modify a local VMA flags variable before setting the VMA flags to this value.
Hmm, sure would be nice to not have this inconsistency though.
Yes!
Quoted 14 lines
It is a bit wordy but with the C preprocessor we can make this work:
struct
vm_flags_t{DECLARE_BITMAP(..)};void func(..)
{ struct vm_flags_t flags = OR_VMA_FLAGS(VMA_READ_BIT, VMA_WRITE_BIT); flags = vm_flags_or(flags, OR_VMA_FLAGS(VMA_MAYREAD_BIT, VMA_MAYWRITE_BIT); }Where OR_VMA_FLAGS's OR's together its
__VA_ARGS__and returns a structvm_flags_t.
Quoted 7 lines
Would that be interesting? Eliminate the inconsistency?
eg
Or other similar solutions.
Well this would help things be more succinct rather than doing, e.g.:
vma_flags_set(&flags, VMA_READ_BIT);vma_flags_set(&flags, VMA_WRITE_BIT);
But the reason for this separation is more so needing to also do other operations like testing for bits against local flags.
It may also just be sensible to drop the vma_test() since I've named VMA flags vma->flags which is kinda neat and not so painful to do:
if (vma_flags_test(&vma->flags, VMA_READ_BIT)) {
}Another note - I do hope to drop the _BIT at some point. But it felt egregious to do so _now_ since VM_READ, VMA_READ are so close it'd be _super_ easy to mistake the two.
The sparse stuff will flag that up (no pun intended), but I didn't want to make that kind of error _too_ easy to achieve.
Buuut I'm guessing actually you're thinking more of getting rid of vm_flags_word_[and, any, all]() all of which take VM_xxx parameters.
Quoted 4 lines
The compiler is pretty smart so this would all fold away to very few instructions.
Well I'm not sure, hopefully. Maybe I need to test this and see exactly what the it comes up with.
I mean you could in theory have:
vma_flags_any(&vma->flags, OR_VMA_FLAGS(VMA_PFNMAP_BIT, VMA_SEALED_BIT))
Where OR_VMA_FLAGS() generates a vma_flags_t which can then be bitmap_or()'d.
It'd then have to be something like (say for 64 bit flags on a 32-bit system):
unsigned long val[2] = {};
__set_bit(VMA_PFNMAP_BIT, &val);
__set_bit(VMA_SEALED_BIT, &val);
/* ...assuming dst can be src also... */
bitmap_or(&val, ACCESS_PRIVATE(&vma->flags, __vma_flags), &val);
return !bitmap_empty(&val);And... I can't really see the compiler finding a way to make that efficient, esp. given e.g. bitmap_or() -> __bitmap_or() for the non-small number-of-bits case.
I feel like we're going to need the 'special first word' stuff permanently for performance reasons.
At the same time, can definitely look into what the compiler actually generates/even look at improving the bitmap stuff if it's inefficient. But I think that should be a future project :)
Quoted 2 lines
Then everything only works with
_BITand we don't have the special first word situation.
In any case we still need to maintain the word stuff for legacy purposes at least to handle the existing vm_flags_*() interfaces until the work is complete.
I think it's reasonable either way to treat this as iterative - if we can find efficient ways to do this stuff with _BIT only then let's do that, but for now I think it's reasonable to have the various compromises to make the initial conversion easier.
Of course we need to try to get things as right as we can early on, but we also don't want to get stuck in analysis paralysis either.
Quoted 2 lines
Jason
Cheers, Lorenzo
Re: [PATCH 4/4] mm: introduce and use VMA flag test helpers
Quoted 9 lines · Lorenzo Stoakes
It may also just be sensible to drop the
vma_test()since I've named VMA flagsvma->flagswhich is kinda neat and not so painful to do:if (vma_flags_test(&vma->flags, VMA_READ_BIT)) { }Another note - I do hope to drop the
_BITat some point. But it felt egregious to do so _now_ sinceVM_READ,VMA_READare so close it'd be _super_ easy to mistake the two.
Yes, you should have the bit until the non-bit versions are removed entirely.
Quoted 2 lines
Buuut I'm guessing actually you're thinking more of getting rid of vm_flags_word_[and, any, all]() all of which take
VM_xxxparameters.
Yes
Quoted 9 lines
Quoted 2 lines
few instructions.
Well I'm not sure, hopefully. Maybe I need to test this and see exactly what the it comes up with.
I mean you could in theory have:
vma_flags_any(&vma->flags, OR_VMA_FLAGS(VMA_PFNMAP_BIT, VMA_SEALED_BIT))
'any' here means any of the given bits set, yes? So the operation is
(flags & to_test) != 0
Which is bitmap_and, not or. In this case the compiler goes word by word:
flags[0] & to_test[0] != 0
flags[1] & to_test[1] != 0
And constant propagation turns it into
flags[1] & 0 != 0 ----> 0
So the extra word just disappears.
Similarly if you want to do a set bit using or
flags[0] = flags[0] | to_set[0]
flags[1] = flags[1] | to_set[1]
And again constant propagation
flags[1] = flags[1] | 0 ------> NOP
Quoted 2 lines
I feel like we're going to need the 'special first word' stuff permanently for performance reasons.
I think not, look above..
Quoted 5 lines
Quoted 2 lines
Then everything only works with
_BITand we don't have the special first word situation.In any case we still need to maintain the word stuff for legacy purposes at least to handle the existing vm_flags_*() interfaces until the work is complete.
I think it will be hard to sustain this idea that some operations do not work on the bits in the second word, it is just not very natural going forward..
So I'd try to structure things to remove the non-BIT users before adding multi-word..
Jason
Re: [PATCH 4/4] mm: introduce and use VMA flag test helpers
Quoted 25 lines · Jason Gunthorpe
Quoted 9 lines · Lorenzo Stoakes
It may also just be sensible to drop the
vma_test()since I've named VMA flagsvma->flagswhich is kinda neat and not so painful to do:if (vma_flags_test(&vma->flags, VMA_READ_BIT)) { }Another note - I do hope to drop the
_BITat some point. But it felt egregious to do so _now_ sinceVM_READ,VMA_READare so close it'd be _super_ easy to mistake the two.Yes, you should have the bit until the non-bit versions are removed entirely.
Quoted 2 lines
Buuut I'm guessing actually you're thinking more of getting rid of vm_flags_word_[and, any, all]() all of which take
VM_xxxparameters.Yes
Quoted 6 lines
Quoted 2 lines
few instructions.
Well I'm not sure, hopefully. Maybe I need to test this and see exactly what the it comes up with.
Quoted 7 lines
Quoted 3 lines
I mean you could in theory have:
vma_flags_any(&vma->flags, OR_VMA_FLAGS(VMA_PFNMAP_BIT, VMA_SEALED_BIT))'any' here means any of the given bits set, yes? So the operation is
(flags &
to_test) != 0
Yeah sorry, you're right. nd __bitmap_and() for BITS_PER_LONG aligned bitmap size (which it always will be) amoutns to:
for (k = 0; k < lim; k++) result |= (dst[k] = bitmap1[k] & bitmap2[k]);
So yeah... ok interesting.
Quoted 3 lines
Which is
bitmap_and, not or. In this case the compiler goes word by word:
Yeah indeed.
Quoted 8 lines
flags[0] & to_test[0] != 0
flags[1] & to_test[1] != 0And constant propagation turns it into
flags[1] & 0 != 0 ----> 0So the extra word just disappears.
Hmm on the assumption that we can construct a vma_flags_t that the compiler knows to be constant... I wonder if:
int to_test[2];
to_test[0] |= 1 << SOME_CONST;
to_test[1] |= 1 << ANOTHER_CONST;
flags[0] & to_test[0]
flags[1] & to_test[1]
Will propagate like this.
But enough theory... I godbolt'd it and https://godbolt.org/z/8PW6PzK1f
Well knock me down with a feather modern compilers _are_ clever enough :)
I have underestimated this...
Quoted 6 lines
Similarly if you want to do a set bit using or
flags[0] = flags[0] | to_set[0]
flags[1] = flags[1] | to_set[1]
Quoted 7 lines
And again constant propagation
flags[1] = flags[1] | 0 ------> NOPQuoted 2 lines
I feel like we're going to need the 'special first word' stuff permanently for performance reasons.
I think not, look above..
No, this is quite interesting then, I was concerned about this!
Quoted 10 lines
Quoted 6 lines
Quoted 3 lines
Then everything only works with
_BITand we don't have the special first word situation.In any case we still need to maintain the word stuff for legacy purposes at least to handle the existing vm_flags_*() interfaces until the work is complete.
I think it will be hard to sustain this idea that some operations do not work on the bits in the second word, it is just not very natural going forward..
Yeah of course it's far from ideal.
Quoted 3 lines
So I'd try to structure things to remove the non-BIT users before adding multi-word..
Yeah, OK well your point has been made here, the compiler _is_ smart enough to save us here :)
Let's avoid this first word stuff altogether then.
Quoted 2 lines
Jason
Cheers, Lorenzo
Re: [PATCH 4/4] mm: introduce and use VMA flag test helpers
Quoted 4 lines · Lorenzo Stoakes
Yeah, OK well your point has been made here, the compiler _is_ smart enough to save us here :)
Let's avoid this first word stuff altogether then.
I suggest to add some helpers to the general include/linux/bitmap.h "subsystem" that lets it help do this:
#define BITMAP_OR_BITS(type, member, bit1, bit2, bit3)
returns a type with the bitmap array member initialized to those bits
Then some other bitmap helpers that are doing the specific maths you want..
* bitmap_and_eq(src1, src2, src3, nbits) true if *src1 & *src2 == *src3
* bitmap_and_eq_zero(src1, src2, nbits) true if *src1 & *src1 == 0
etc
Jason
Re: [PATCH 4/4] mm: introduce and use VMA flag test helpers
Quoted 14 lines · Jason Gunthorpe
Quoted 4 lines · Lorenzo Stoakes
Yeah, OK well your point has been made here, the compiler _is_ smart enough to save us here :)
Let's avoid this first word stuff altogether then.
I suggest to add some helpers to the general
include/linux/bitmap.h"subsystem" that lets it help do this:#define
BITMAP_OR_BITS(type, member, bit1, bit2, bit3)returns a type with the bitmap array member initialized to those bits
Quoted 8 lines
Then some other bitmap helpers that are doing the specific maths you want..
*
bitmap_and_eq(src1, src2, src3, nbits)true if *src1 & *src2 == *src3
*bitmap_and_eq_zero(src1, src2, nbits)true if *src1 & *src1 == 0
etcJason
Yeah makes sense to put it there.
Thanks, Lorenzo
Quoted 53 lines · Lorenzo Stoakes
We are in the rather silly situation that we are running out of VMA flags as they are currently limited to a system word in size.
This leads to absurd situations where we limit features to 64-bit architectures only because we simply do not have the ability to add a flag for 32-bit ones.
This is very constraining and leads to hacks or, in the worst case, simply an inability to implement features we want for entirely arbitrary reasons.
This also of course gives us something of a Y2K type situation in mm where we might eventually exhaust all of the VMA flags even on 64-bit systems.
This series lays the groundwork for getting away from this limitation by establishing VMA flags as a bitmap whose size we can increase in future beyond 64 bits if required.
This is necessarily a highly iterative process given the extensive use of VMA flags throughout the kernel, so we start by performing basic steps.
Firstly, we declare VMA flags by bit number rather than by value, retaining the
VM_xxxfields but in terms of these newly introducedVMA_xxx_BITfields.While we are here, we use sparse annotations to ensure that, when dealing with VMA bit number parameters, we cannot be passed values which are not declared as such - providing some useful type safety.
We then introduce an opaque VMA flag type, much like the opaque
mm_structflag type introduced in commitbb6525f2f8c4("mm: add bitmapmm->flagsfield"), which we establish in union withvma->vm_flags(but still set at system word size meaning there is no functional or data type size change).We update the
vm_flags_xxx()helpers to use this new bitmap, introducing sensible helpers to do so.We then provide
vma_flags_test()andvma_test()to allow for testing of VMA flag bits, and utilise these across the memory management subsystem.Since it would be entirely inefficient to do away with the bitwise operations used throughout the kernel with respect to VMA flags, we permit these to exist, providing helpers for these operations against the new bitmap.
These operate on the assumption that such operations will only be required for flags which can exist within a system word. This allows the fundamental flags to be used efficiently as before.
This series lays the foundation for further work to expand the use of bitmap VMA flags and eventually eliminate these arbitrary restrictions.
Quoted 5 lines
ANDREW/REVIEWS NOTES:
Apologies, but the nature of this series is that it's going to be a little painful, I've based it on [0] to make life a bit easier. Let me know if you need rebases etc.
Hey Lorenzo,
I put your patchset into the Fedora Koji system to run some CI on it for you.
It failed to build due to what looks like some Rust bindings.
Heres the build: https://koji.fedoraproject.org/koji/taskinfo?taskID=138547842
And x86 build logs:
https://kojipkgs.fedoraproject.org//work/tasks/7966/138547966/build.log
The error is pretty large but here's a snippet if you want an idea
error[E0425]: cannot find value VM_READ in crate bindings
--> rust/kernel/mm/virt.rs:399:44
|
399 | pub const READ: vm_flags_t = bindings::VM_READ as vm_flags_t;
| ^^^^^^^ not found in `bindings`
error[E0425]: cannot find value `VM_WRITE` in crate `bindings`
--> rust/kernel/mm/virt.rs:402:45
|
402 | pub const WRITE: vm_flags_t = bindings::VM_WRITE as vm_flags_t;
| ^^^^^^^^ not found
in `bindings`
error[E0425]: cannot find value `VM_EXEC` in crate `bindings`
--> rust/kernel/mm/virt.rs:405:44
|
405 | pub const EXEC: vm_flags_t = bindings::VM_EXEC as vm_flags_t;
| ^^^^^^^ help: a
constant with a similar name exists: `ET_EXEC`
|
::: /builddir/build/BUILD/kernel-6.18.0-build/kernel-6.18-rc3-16-ge53642b87a4f/linux-6.18.0-0.rc3.e53642b87a4f.31.bitvma.fc44.x86_64/rust/bindings/bindings_generated.rs:13881:1
|
13881 | pub const ET_EXEC: u32 = 2;
| ---------------------- similarly named constant `ET_EXEC` defined here
error[E0425]: cannot find value `VM_SHARED` in crate `bindings`
--> rust/kernel/mm/virt.rs:408:46
|
408 | pub const SHARED: vm_flags_t = bindings::VM_SHARED as vm_flags_t;
| ^^^^^^^^^ not foundin bindings
In the next version Ill do the same and continue with the CI testing for you!
Cheers,
-- Nico
Quoted 52 lines
[0]: https://lore.kernel.org/linux-mm/[email protected]/
Lorenzo Stoakes (4):
mm: declare VMA flags by bit
mm: simplify and rename mm flags function for clarity
mm: introduce VMA flags bitmap type
mm: introduce and use VMA flag test helpersfs/proc/task_mmu.c | 4 +- include/linux/hugetlb.h | 2 +- include/linux/mm.h | 341 +++++++++++++------- include/linux/mm_inline.h | 2 +- include/linux/mm_types.h | 120 ++++++- include/linux/userfaultfd_k.h | 12 +- kernel/fork.c | 4 +- mm/filemap.c | 4 +- mm/gup.c | 16 +- mm/hmm.c | 6 +- mm/huge_memory.c | 34 +- mm/hugetlb.c | 48 +-- mm/internal.h | 8 +- mm/khugepaged.c | 2 +- mm/ksm.c | 12 +- mm/madvise.c | 8 +- mm/memory.c | 77 +++-- mm/mempolicy.c | 4 +- mm/migrate.c | 4 +- mm/migrate_device.c | 10 +- mm/mlock.c | 8 +- mm/mmap.c | 16 +- mm/mmap_lock.c | 4 +- mm/mprotect.c | 12 +- mm/mremap.c | 18 +- mm/mseal.c | 2 +- mm/msync.c | 4 +- mm/nommu.c | 16 +- mm/oom_kill.c | 4 +- mm/pagewalk.c | 2 +- mm/rmap.c | 16 +- mm/swap.c | 3 +- mm/userfaultfd.c | 33 +- mm/vma.c | 37 ++- mm/vma.h | 6 +- mm/vmscan.c | 4 +- tools/testing/vma/vma.c | 20 +- tools/testing/vma/vma_internal.h | 536 +++++++++++++++++++++++++++---- 38 files changed, 1037 insertions(+), 422 deletions(-)--
2.51.0
+cc Alice - could you help look at this? It seems I have broken the rust bindings here :)
Thanks!
Quoted 40 lines · Nico Pache
Hey Lorenzo,
I put your patchset into the Fedora Koji system to run some CI on it for you.
It failed to build due to what looks like some Rust bindings.
Heres the build: https://koji.fedoraproject.org/koji/taskinfo?taskID=138547842
And x86 build logs:
https://kojipkgs.fedoraproject.org//work/tasks/7966/138547966/build.logThe error is pretty large but here's a snippet if you want an idea
error[E0425]: cannot find value
VM_READin cratebindings
-->rust/kernel/mm/virt.rs:399:44| 399 | pub const READ: vm_flags_t = bindings::VM_READ as vm_flags_t; | ^^^^^^^ not found in `bindings` error[E0425]: cannot find value `VM_WRITE` in crate `bindings` --> rust/kernel/mm/virt.rs:402:45 | 402 | pub const WRITE: vm_flags_t = bindings::VM_WRITE as vm_flags_t; | ^^^^^^^^ not found in `bindings` error[E0425]: cannot find value `VM_EXEC` in crate `bindings` --> rust/kernel/mm/virt.rs:405:44 | 405 | pub const EXEC: vm_flags_t = bindings::VM_EXEC as vm_flags_t; | ^^^^^^^ help: a constant with a similar name exists: `ET_EXEC` | ::: /builddir/build/BUILD/kernel-6.18.0-build/kernel-6.18-rc3-16-ge53642b87a4f/linux-6.18.0-0.rc3.e53642b87a4f.31.bitvma.fc44.x86_64/rust/bindings/bindings_generated.rs:13881:1 | 13881 | pub const ET_EXEC: u32 = 2; | ---------------------- similarly named constant `ET_EXEC` defined here error[E0425]: cannot find value `VM_SHARED` in crate `bindings` --> rust/kernel/mm/virt.rs:408:46 | 408 | pub const SHARED: vm_flags_t = bindings::VM_SHARED as vm_flags_t; | ^^^^^^^^^ not found
Quoted 3 lines
in
bindingsIn the next version Ill do the same and continue with the CI testing for you!
Thanks much appreciated :)
It seems I broke the rust bindings (clearly), have pinged Alice to have a look!
May try and repro my side to see if there's something trivial that I could take a look at.
I ran this through mm self tests, allmodconfig + a bunch of other checks but ofc enabling rust was not one, I should probably update my scripts [0] to do that too :)
Cheers, Lorenzo
Quoted 3 lines · Lorenzo Stoakes
+cc Alice - could you help look at this? It seems I have broken the rust bindings here :)
From a first glance it looks trivial to fix, there are a bunch of bindings of the VM_* flags.
for example
kernel/mm/virt.rs: pub const MIXEDMAP: vm_flags_t =bindings::VM_MIXEDMAP as vm_flags_t;
I believe this just needs to be converted to 'bindings::VM_MIXEDMAP_BIT' if I understand your series correctly (havent fully looked at the details).
Quoted 53 lines
Thanks!
Quoted 40 lines · Nico Pache
Hey Lorenzo,
I put your patchset into the Fedora Koji system to run some CI on it for you.
It failed to build due to what looks like some Rust bindings.
Heres the build: https://koji.fedoraproject.org/koji/taskinfo?taskID=138547842
And x86 build logs:
https://kojipkgs.fedoraproject.org//work/tasks/7966/138547966/build.logThe error is pretty large but here's a snippet if you want an idea
error[E0425]: cannot find value
VM_READin cratebindings
-->rust/kernel/mm/virt.rs:399:44| 399 | pub const READ: vm_flags_t = bindings::VM_READ as vm_flags_t; | ^^^^^^^ not found in `bindings` error[E0425]: cannot find value `VM_WRITE` in crate `bindings` --> rust/kernel/mm/virt.rs:402:45 | 402 | pub const WRITE: vm_flags_t = bindings::VM_WRITE as vm_flags_t; | ^^^^^^^^ not found in `bindings` error[E0425]: cannot find value `VM_EXEC` in crate `bindings` --> rust/kernel/mm/virt.rs:405:44 | 405 | pub const EXEC: vm_flags_t = bindings::VM_EXEC as vm_flags_t; | ^^^^^^^ help: a constant with a similar name exists: `ET_EXEC` | ::: /builddir/build/BUILD/kernel-6.18.0-build/kernel-6.18-rc3-16-ge53642b87a4f/linux-6.18.0-0.rc3.e53642b87a4f.31.bitvma.fc44.x86_64/rust/bindings/bindings_generated.rs:13881:1 | 13881 | pub const ET_EXEC: u32 = 2; | ---------------------- similarly named constant `ET_EXEC` defined here error[E0425]: cannot find value `VM_SHARED` in crate `bindings` --> rust/kernel/mm/virt.rs:408:46 | 408 | pub const SHARED: vm_flags_t = bindings::VM_SHARED as vm_flags_t; | ^^^^^^^^^ not foundQuoted 3 lines
in
bindingsIn the next version Ill do the same and continue with the CI testing for you!
Thanks much appreciated :)
It seems I broke the rust bindings (clearly), have pinged Alice to have a look!
Quoted 6 lines
May try and repro my side to see if there's something trivial that I could take a look at.
I ran this through mm self tests, allmodconfig + a bunch of other checks but ofc enabling rust was not one, I should probably update my scripts [0] to do that too :)
Ah cool, thanks for sharing your scripts, Ill take a look into those!
Cheers,
-- Nico
Quoted 5 lines
Cheers, Lorenzo
Quoted 10 lines · Nico Pache
Quoted 3 lines · Lorenzo Stoakes
+cc Alice - could you help look at this? It seems I have broken the rust bindings here :)
From a first glance it looks trivial to fix, there are a bunch of bindings of the VM_* flags.
Quoted 8 lines
for example
kernel/mm/virt.rs: pub const MIXEDMAP:vm_flags_t=bindings::VM_MIXEDMAPasvm_flags_t;I believe this just needs to be converted to '
bindings::VM_MIXEDMAP_BIT' if I understand your series correctly (havent fully looked at the details).
On second thought, I think i'm wrong here.
Quoted 70 lines
Quoted 54 lines
Thanks!
Quoted 40 lines · Nico Pache
Hey Lorenzo,
I put your patchset into the Fedora Koji system to run some CI on it for you.
It failed to build due to what looks like some Rust bindings.
Heres the build: https://koji.fedoraproject.org/koji/taskinfo?taskID=138547842
And x86 build logs:
https://kojipkgs.fedoraproject.org//work/tasks/7966/138547966/build.logThe error is pretty large but here's a snippet if you want an idea
error[E0425]: cannot find value
VM_READin cratebindings
-->rust/kernel/mm/virt.rs:399:44| 399 | pub const READ: vm_flags_t = bindings::VM_READ as vm_flags_t; | ^^^^^^^ not found in `bindings` error[E0425]: cannot find value `VM_WRITE` in crate `bindings` --> rust/kernel/mm/virt.rs:402:45 | 402 | pub const WRITE: vm_flags_t = bindings::VM_WRITE as vm_flags_t; | ^^^^^^^^ not found in `bindings` error[E0425]: cannot find value `VM_EXEC` in crate `bindings` --> rust/kernel/mm/virt.rs:405:44 | 405 | pub const EXEC: vm_flags_t = bindings::VM_EXEC as vm_flags_t; | ^^^^^^^ help: a constant with a similar name exists: `ET_EXEC` | ::: /builddir/build/BUILD/kernel-6.18.0-build/kernel-6.18-rc3-16-ge53642b87a4f/linux-6.18.0-0.rc3.e53642b87a4f.31.bitvma.fc44.x86_64/rust/bindings/bindings_generated.rs:13881:1 | 13881 | pub const ET_EXEC: u32 = 2; | ---------------------- similarly named constant `ET_EXEC` defined here error[E0425]: cannot find value `VM_SHARED` in crate `bindings` --> rust/kernel/mm/virt.rs:408:46 | 408 | pub const SHARED: vm_flags_t = bindings::VM_SHARED as vm_flags_t; | ^^^^^^^^^ not foundQuoted 3 lines
in
bindingsIn the next version Ill do the same and continue with the CI testing for you!
Thanks much appreciated :)
It seems I broke the rust bindings (clearly), have pinged Alice to have a look!
Quoted 6 lines
May try and repro my side to see if there's something trivial that I could take a look at.
I ran this through mm self tests, allmodconfig + a bunch of other checks but ofc enabling rust was not one, I should probably update my scripts [0] to do that too :)
Ah cool, thanks for sharing your scripts, Ill take a look into those!
Cheers,
-- NicoQuoted 5 lines
Cheers, Lorenzo
Alice Ryhl
maintainerQuoted 58 lines · Lorenzo Stoakes
+cc Alice - could you help look at this? It seems I have broken the rust bindings here :)
Thanks!
Quoted 40 lines · Nico Pache
Hey Lorenzo,
I put your patchset into the Fedora Koji system to run some CI on it for you.
It failed to build due to what looks like some Rust bindings.
Heres the build: https://koji.fedoraproject.org/koji/taskinfo?taskID=138547842
And x86 build logs:
https://kojipkgs.fedoraproject.org//work/tasks/7966/138547966/build.logThe error is pretty large but here's a snippet if you want an idea
error[E0425]: cannot find value
VM_READin cratebindings
-->rust/kernel/mm/virt.rs:399:44| 399 | pub const READ: vm_flags_t = bindings::VM_READ as vm_flags_t; | ^^^^^^^ not found in `bindings` error[E0425]: cannot find value `VM_WRITE` in crate `bindings` --> rust/kernel/mm/virt.rs:402:45 | 402 | pub const WRITE: vm_flags_t = bindings::VM_WRITE as vm_flags_t; | ^^^^^^^^ not found in `bindings` error[E0425]: cannot find value `VM_EXEC` in crate `bindings` --> rust/kernel/mm/virt.rs:405:44 | 405 | pub const EXEC: vm_flags_t = bindings::VM_EXEC as vm_flags_t; | ^^^^^^^ help: a constant with a similar name exists: `ET_EXEC` | ::: /builddir/build/BUILD/kernel-6.18.0-build/kernel-6.18-rc3-16-ge53642b87a4f/linux-6.18.0-0.rc3.e53642b87a4f.31.bitvma.fc44.x86_64/rust/bindings/bindings_generated.rs:13881:1 | 13881 | pub const ET_EXEC: u32 = 2; | ---------------------- similarly named constant `ET_EXEC` defined here error[E0425]: cannot find value `VM_SHARED` in crate `bindings` --> rust/kernel/mm/virt.rs:408:46 | 408 | pub const SHARED: vm_flags_t = bindings::VM_SHARED as vm_flags_t; | ^^^^^^^^^ not foundQuoted 3 lines
in
bindingsIn the next version Ill do the same and continue with the CI testing for you!
Thanks much appreciated :)
It seems I broke the rust bindings (clearly), have pinged Alice to have a look!
May try and repro my side to see if there's something trivial that I could take a look at.
Quoted 7 lines
I ran this through mm self tests, allmodconfig + a bunch of other checks but ofc enabling rust was not one, I should probably update my scripts [0] to do that too :)
Cheers, Lorenzo
I can help convert the Rust bindings to work with this approach. I see there is a nice and simple vma_test() method for checking a flag, but I don't see any equivalent method for setting or unsetting a given bit. What would be the best function for Rust to call to set or unset a given bit in the vma flags?
Alice
Quoted 65 lines · Alice Ryhl
Quoted 64 lines · Lorenzo Stoakes
+cc Alice - could you help look at this? It seems I have broken the rust bindings here :)
Thanks!
Quoted 40 lines · Nico Pache
Hey Lorenzo,
I put your patchset into the Fedora Koji system to run some CI on it for you.
It failed to build due to what looks like some Rust bindings.
Heres the build: https://koji.fedoraproject.org/koji/taskinfo?taskID=138547842
And x86 build logs:
https://kojipkgs.fedoraproject.org//work/tasks/7966/138547966/build.logThe error is pretty large but here's a snippet if you want an idea
error[E0425]: cannot find value
VM_READin cratebindings
-->rust/kernel/mm/virt.rs:399:44| 399 | pub const READ: vm_flags_t = bindings::VM_READ as vm_flags_t; | ^^^^^^^ not found in `bindings` error[E0425]: cannot find value `VM_WRITE` in crate `bindings` --> rust/kernel/mm/virt.rs:402:45 | 402 | pub const WRITE: vm_flags_t = bindings::VM_WRITE as vm_flags_t; | ^^^^^^^^ not found in `bindings` error[E0425]: cannot find value `VM_EXEC` in crate `bindings` --> rust/kernel/mm/virt.rs:405:44 | 405 | pub const EXEC: vm_flags_t = bindings::VM_EXEC as vm_flags_t; | ^^^^^^^ help: a constant with a similar name exists: `ET_EXEC` | ::: /builddir/build/BUILD/kernel-6.18.0-build/kernel-6.18-rc3-16-ge53642b87a4f/linux-6.18.0-0.rc3.e53642b87a4f.31.bitvma.fc44.x86_64/rust/bindings/bindings_generated.rs:13881:1 | 13881 | pub const ET_EXEC: u32 = 2; | ---------------------- similarly named constant `ET_EXEC` defined here error[E0425]: cannot find value `VM_SHARED` in crate `bindings` --> rust/kernel/mm/virt.rs:408:46 | 408 | pub const SHARED: vm_flags_t = bindings::VM_SHARED as vm_flags_t; | ^^^^^^^^^ not foundQuoted 3 lines
in
bindingsIn the next version Ill do the same and continue with the CI testing for you!
Thanks much appreciated :)
It seems I broke the rust bindings (clearly), have pinged Alice to have a look!
May try and repro my side to see if there's something trivial that I could take a look at.
I ran this through mm self tests, allmodconfig + a bunch of other checks but ofc enabling rust was not one, I should probably update my scripts [0] to do that too :)
Cheers, Lorenzo
Quoted 7 lines
Quoted 1 line
I can help convert the Rust bindings to work with this approach. I see there is a nice and simple
vma_test()method for checking a flag, but I don't see any equivalent method for setting or unsetting a given bit. What would be the best function for Rust to call to set or unset a given bit in the vma flags?
Thanks much appreciated!
I was thinking of adding that in but have no C users currently, on respin can add.
If you give me a rough idea of what needs to be changed I can always try and wrap that into my respin?
Quoted 2 lines
Alice
Thanks, Lorenzo
Alice Ryhl
maintainerQuoted 74 lines · Lorenzo Stoakes
Quoted 65 lines · Alice Ryhl
Quoted 64 lines · Lorenzo Stoakes
+cc Alice - could you help look at this? It seems I have broken the rust bindings here :)
Thanks!
Quoted 40 lines · Nico Pache
Hey Lorenzo,
I put your patchset into the Fedora Koji system to run some CI on it for you.
It failed to build due to what looks like some Rust bindings.
Heres the build: https://koji.fedoraproject.org/koji/taskinfo?taskID=138547842
And x86 build logs:
https://kojipkgs.fedoraproject.org//work/tasks/7966/138547966/build.logThe error is pretty large but here's a snippet if you want an idea
error[E0425]: cannot find value
VM_READin cratebindings
-->rust/kernel/mm/virt.rs:399:44| 399 | pub const READ: vm_flags_t = bindings::VM_READ as vm_flags_t; | ^^^^^^^ not found in `bindings` error[E0425]: cannot find value `VM_WRITE` in crate `bindings` --> rust/kernel/mm/virt.rs:402:45 | 402 | pub const WRITE: vm_flags_t = bindings::VM_WRITE as vm_flags_t; | ^^^^^^^^ not found in `bindings` error[E0425]: cannot find value `VM_EXEC` in crate `bindings` --> rust/kernel/mm/virt.rs:405:44 | 405 | pub const EXEC: vm_flags_t = bindings::VM_EXEC as vm_flags_t; | ^^^^^^^ help: a constant with a similar name exists: `ET_EXEC` | ::: /builddir/build/BUILD/kernel-6.18.0-build/kernel-6.18-rc3-16-ge53642b87a4f/linux-6.18.0-0.rc3.e53642b87a4f.31.bitvma.fc44.x86_64/rust/bindings/bindings_generated.rs:13881:1 | 13881 | pub const ET_EXEC: u32 = 2; | ---------------------- similarly named constant `ET_EXEC` defined here error[E0425]: cannot find value `VM_SHARED` in crate `bindings` --> rust/kernel/mm/virt.rs:408:46 | 408 | pub const SHARED: vm_flags_t = bindings::VM_SHARED as vm_flags_t; | ^^^^^^^^^ not foundQuoted 3 lines
in
bindingsIn the next version Ill do the same and continue with the CI testing for you!
Thanks much appreciated :)
It seems I broke the rust bindings (clearly), have pinged Alice to have a look!
May try and repro my side to see if there's something trivial that I could take a look at.
I ran this through mm self tests, allmodconfig + a bunch of other checks but ofc enabling rust was not one, I should probably update my scripts [0] to do that too :)
Cheers, Lorenzo
Quoted 7 lines
Quoted 1 line
I can help convert the Rust bindings to work with this approach. I see there is a nice and simple
vma_test()method for checking a flag, but I don't see any equivalent method for setting or unsetting a given bit. What would be the best function for Rust to call to set or unset a given bit in the vma flags?
Quoted 7 lines
Thanks much appreciated!
I was thinking of adding that in but have no C users currently, on respin can add.
If you give me a rough idea of what needs to be changed I can always try and wrap that into my respin?
You'll want something along these lines:
rust/bindings/bindings_helper.h +0 -1
@@ -108,7 +108,6 @@ const xa_mark_t RUST_CONST_HELPER_XA_PRESENT = XA_PRESENT;const gfp_t RUST_CONST_HELPER_XA_FLAGS_ALLOC =XA_FLAGS_ALLOC ;const gfp_t RUST_CONST_HELPER_XA_FLAGS_ALLOC1 =XA_FLAGS_ALLOC1 ; -const vm_flags_t RUST_CONST_HELPER_VM_MERGEABLE =VM_MERGEABLE ;#if IS_ENABLED (CONFIG_ANDROID_BINDER_IPC_RUST )#include "../../drivers/android/binder/rust_binder.h"
rust/helpers/mm.c +15 -0
@@ -48,3 +48,18 @@ void rust_helper_vma_end_read(struct vm_area_struct *vma) {vma_end_read (vma); } + +bool rust_helper_vma_test (const struct vm_area_struct *vma,vma_flag_t flag) +{ +return vma_test (vma, flag); +} + +void rust_helper_vma_set (const struct vm_area_struct *vma,vma_flag_t flag) +{ +// TODO +} + +void rust_helper_vma_unset (const struct vm_area_struct *vma,vma_flag_t flag) +{ +// TODO +}
rust/kernel/mm/virt.rs +93 -96
393 lines not on this page · raw message