Skip to contentSkip to conversation

mm: convert VM flags from macros to enum

RFC PATCH In review Ye Liu 5 participants 13 messages Oct 11, 2025 – Oct 14, 2025

Subsystems (2)
lore
Share & tools
lore

Ye Liu

author
#0 lore raw

RFC PATCH 1/1 mm: convert VM flags from macros to enum

From: Ye Liu <[email protected]>

Hello MM maintainers and drgn community,

This RFC proposes to convert VM_* flags from #define macros to enum vm_flags. The motivation comes from recent drgn development where we encountered difficulties in implementing VM flag parsing due to the current macro-based approach.

Problem:
- Debugging tools like drgn must hard-code VM flag values
- No type information available for external tools
- Maintenance burden when flags change in kernel
- Error-prone flag interpretation

Solution:
Convert to enum while preserving all existing values and semantics.

Benefits:
1. Better type safety for kernel code
2. Enabled dynamic VM flag parsing in debugging tools
3. Improved debugger experience (meaningful symbol names)
4. Easier maintenance and extensibility
5. No functional changes or ABI impact

This change would particularly help with implementing the 'vm -f' option in drgn's crash mode, allowing proper VM flag filtering and display without hard-coded values.

I'm seeking feedback on:
- Overall approach acceptability
- Any concerns about enum usage in this context
- Potential impacts on other subsystems
- Suggestions for alternative solutions
- Whether this should include trace events update

Looking forward to your comments.

---
 include/linux/mm.h   | 90 +++++++++++++++++++++++---------------------
 include/linux/mman.h |  7 ++--
 2 files changed, 50 insertions(+), 47 deletions(-)
include/linux/mm.h +47 -43
index d16b33bacc32..7030108c3759 100644
@@ -268,62 +268,66 @@ extern struct rw_semaphore nommu_region_sem;
 
 extern unsigned int kobjsize(const void *objp);
 #endif
-
 /*
  * vm_flags in vm_area_struct, see mm_types.h.
  * When changing, update also include/trace/events/mmflags.h
  */
-#define VM_NONE		0x00000000
+enum vm_flags {
+	VM_NONE         = 0x00000000,
+
+	VM_READ         = BIT(0), /* currently active flags */
+	VM_WRITE        = BIT(1),
+	VM_EXEC         = BIT(2),
+	VM_SHARED       = BIT(3),
 
-#define VM_READ		0x00000001	/* currently active flags */
-#define VM_WRITE	0x00000002
-#define VM_EXEC		0x00000004
-#define VM_SHARED	0x00000008
+	/* mprotect() hardcodes VM_MAYREAD >> 4 == VM_READ, and so for r/w/x bits. */
+	VM_MAYREAD      = BIT(4), /* limits for mprotect() etc */
+	VM_MAYWRITE     = BIT(5),
+	VM_MAYEXEC      = BIT(6),
+	VM_MAYSHARE     = BIT(7),
 
-/* mprotect() hardcodes VM_MAYREAD >> 4 == VM_READ, and so for r/w/x bits. */
-#define VM_MAYREAD	0x00000010	/* limits for mprotect() etc */
-#define VM_MAYWRITE	0x00000020
-#define VM_MAYEXEC	0x00000040
-#define VM_MAYSHARE	0x00000080
+	VM_GROWSDOWN    = BIT(8), /* general info on the segment */
 
-#define VM_GROWSDOWN	0x00000100	/* general info on the segment */
 #ifdef CONFIG_MMU
-#define VM_UFFD_MISSING	0x00000200	/* missing pages tracking */
-#else /* CONFIG_MMU */
-#define VM_MAYOVERLAY	0x00000200	/* nommu: R/O MAP_PRIVATE mapping that might overlay a file mapping */
-#define VM_UFFD_MISSING	0
-#endif /* CONFIG_MMU */
-#define VM_PFNMAP	0x00000400	/* Page-ranges managed without "struct page", just pure PFN */
-#define VM_UFFD_WP	0x00001000	/* wrprotect pages tracking */
-
-#define VM_LOCKED	0x00002000
-#define VM_IO           0x00004000	/* Memory mapped I/O or similar */
-
-					/* Used by sys_madvise() */
-#define VM_SEQ_READ	0x00008000	/* App will access data sequentially */
-#define VM_RAND_READ	0x00010000	/* App will not benefit from clustered reads */
-
-#define VM_DONTCOPY	0x00020000      /* Do not copy this vma on fork */
-#define VM_DONTEXPAND	0x00040000	/* Cannot expand with mremap() */
-#define VM_LOCKONFAULT	0x00080000	/* Lock the pages covered when they are faulted in */
-#define VM_ACCOUNT	0x00100000	/* Is a VM accounted object */
-#define VM_NORESERVE	0x00200000	/* should the VM suppress accounting */
-#define VM_HUGETLB	0x00400000	/* Huge TLB Page VM */
-#define VM_SYNC		0x00800000	/* Synchronous page faults */
-#define VM_ARCH_1	0x01000000	/* Architecture-specific flag */
-#define VM_WIPEONFORK	0x02000000	/* Wipe VMA contents in child. */
-#define VM_DONTDUMP	0x04000000	/* Do not include in the core dump */
+	VM_UFFD_MISSING = BIT(9),  /* missing pages tracking */
+#else
+	/* nommu: R/O MAP_PRIVATE mapping that might overlay a file mapping */
+	VM_MAYOVERLAY   = BIT(9),
+	VM_UFFD_MISSING = 0,
+#endif
+
+	VM_PFNMAP       = BIT(10), /* Page-ranges managed without "struct page", just pure PFN */
+	VM_UFFD_WP      = BIT(12), /* wrprotect pages tracking */
+
+	VM_LOCKED       = BIT(13),
+	VM_IO           = BIT(14), /* Memory mapped I/O or similar */
+
+	/* Used by sys_madvise() */
+	VM_SEQ_READ     = BIT(15), /* App will access data sequentially */
+	VM_RAND_READ    = BIT(16), /* App will not benefit from clustered reads */
+
+	VM_DONTCOPY     = BIT(17), /* Do not copy this vma on fork */
+	VM_DONTEXPAND   = BIT(18), /* Cannot expand with mremap() */
+	VM_LOCKONFAULT  = BIT(19), /* Lock the pages covered when they are faulted in */
+	VM_ACCOUNT      = BIT(20), /* Is a VM accounted object */
+	VM_NORESERVE    = BIT(21), /* should the VM suppress accounting */
+	VM_HUGETLB      = BIT(22), /* Huge TLB Page VM */
+	VM_SYNC         = BIT(23), /* Synchronous page faults */
+	VM_ARCH_1       = BIT(24), /* Architecture-specific flag */
+	VM_WIPEONFORK   = BIT(25), /* Wipe VMA contents in child. */
+	VM_DONTDUMP     = BIT(26), /* Do not include in the core dump */
 
 #ifdef CONFIG_MEM_SOFT_DIRTY
-# define VM_SOFTDIRTY	0x08000000	/* Not soft dirty clean area */
+	VM_SOFTDIRTY    = BIT(27), /* Not soft dirty clean area */
 #else
-# define VM_SOFTDIRTY	0
+	VM_SOFTDIRTY    = 0,
 #endif
 
-#define VM_MIXEDMAP	0x10000000	/* Can contain "struct page" and pure PFN pages */
-#define VM_HUGEPAGE	0x20000000	/* MADV_HUGEPAGE marked this vma */
-#define VM_NOHUGEPAGE	0x40000000	/* MADV_NOHUGEPAGE marked this vma */
-#define VM_MERGEABLE	BIT(31)		/* KSM may merge identical pages */
+	VM_MIXEDMAP     = BIT(28), /* Can contain "struct page" and pure PFN pages */
+	VM_HUGEPAGE     = BIT(29), /* MADV_HUGEPAGE marked this vma */
+	VM_NOHUGEPAGE   = BIT(30), /* MADV_NOHUGEPAGE marked this vma */
+	VM_MERGEABLE    = BIT(31), /* KSM may merge identical pages */
+};
 
 #ifdef CONFIG_ARCH_USES_HIGH_VMA_FLAGS
 #define VM_HIGH_ARCH_BIT_0	32	/* bit only usable on 64-bit architectures */
include/linux/mman.h +3 -4
index 0ba8a7e8b90a..4a0adbdeccdd 100644
@@ -130,10 +130,9 @@ static inline bool arch_validate_flags(unsigned long flags)
  * ("bit1" and "bit2" must be single bits)
  */
 #define _calc_vm_trans(x, bit1, bit2) \
-  ((!(bit1) || !(bit2)) ? 0 : \
-  ((bit1) <= (bit2) ? ((x) & (bit1)) * ((bit2) / (bit1)) \
-   : ((x) & (bit1)) / ((bit1) / (bit2))))
-
+	(((bit1) == 0 || (bit2) == 0) ? 0 : \
+	 ((bit1) <= (bit2) ? ((x) & (bit1)) * ((bit2) / (bit1)) \
+	  : ((x) & (bit1)) / ((bit1) / (bit2))))
 /*
  * Combine the mmap "prot" argument into "vm_flags" used internally.
  */
Signature · 1 line
2.43.0
#1 lore raw
kernel test robot noticed the following build errors:

Hi Ye,

[This is a private test report for your RFC patch.] kernel test robot noticed the following build errors:

[auto build test ERROR on akpm-mm/mm-everything]

url:    https://github.com/intel-lab-lkp/linux/commits/Ye-Liu/mm-convert-VM-flags-from-macros-to-enum/20251011-173307
base:   https://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm.git mm-everything
patch link:    https://lore.kernel.org/r/20251011093054.886460-1-ye.liu%40linux.dev
patch subject: [RFC RFC PATCH] mm: convert VM flags from macros to enum
config: x86_64-rhel-9.4-rust (https://download.01.org/0day-ci/archive/20251013/[email protected]/config)
compiler: clang version 20.1.8 (https://github.com/llvm/llvm-project 87f0227cb60147a26a1eeb4fb06e3b505e9c7261)
rustc: rustc 1.88.0 (6b00bc388 2025-06-23)
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20251013/[email protected]/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

All errors (new ones prefixed by >>):

In file included from kernel/sched/rq-offsets.c:5:
kernel/sched/sched.h:3743:18: warning: variable 'cpumask' set but not used [-Wunused-but-set-variable]
3743 | struct cpumask *cpumask;
| ^
1 warning generated.

>> error[E0425]: cannot find value `VM_NONE` in crate `bindings`

--> rust/kernel/mm/virt.rs:396:44
|
396 | pub const NONE: vm_flags_t = bindings::VM_NONE as vm_flags_t;
| ^^^^^^^ help: a constant with a similar name exists: EM_NONE
|
::: rust/bindings/bindings_generated.rs:8324:1
|
8324 | pub const EM_NONE: u32 = 0;
| ---------------------- similarly named constant EM_NONE defined here

Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
#2 lore raw
In reply to Ye Liu
Quoted 8 lines · Ye Liu

From: Ye Liu <[email protected]>

Hello MM maintainers and drgn community,

This RFC proposes to convert VM_* flags from #define macros to enum vm_flags. The motivation comes from recent drgn development where we encountered difficulties in implementing VM flag parsing due to the current macro-based approach.

This isn't going to work sorry, it's not valid to have flag values as an enum (they're distinct) and also - importantly - I'm going to be making significant changes to VMA flags soon (to allow us to have arbitrary number of VMA flags on _all_ architectures as recently done with my series doing something similar with mm flags).

HOWEVER.

My upcoming (likely, multiple) series will result in code using VMA flag _bit numbers_ in preference to raw VMA flag values.

Therefore, these _could_ be enums.

I will note this down as something to look at when I implement the series (it's going to take a while, given the very extensive usage of VMA flags throughout the kernel).

Cheers, Lorenzo

Quoted 171 lines

Problem:
- Debugging tools like drgn must hard-code VM flag values
- No type information available for external tools
- Maintenance burden when flags change in kernel
- Error-prone flag interpretation

Solution:
Convert to enum while preserving all existing values and semantics.

Benefits:
1. Better type safety for kernel code
2. Enabled dynamic VM flag parsing in debugging tools
3. Improved debugger experience (meaningful symbol names)
4. Easier maintenance and extensibility
5. No functional changes or ABI impact

This change would particularly help with implementing the 'vm -f' option in drgn's crash mode, allowing proper VM flag filtering and display without hard-coded values.

I'm seeking feedback on:
- Overall approach acceptability
- Any concerns about enum usage in this context
- Potential impacts on other subsystems
- Suggestions for alternative solutions
- Whether this should include trace events update

Looking forward to your comments.

---
 include/linux/mm.h   | 90 +++++++++++++++++++++++---------------------
 include/linux/mman.h |  7 ++--
 2 files changed, 50 insertions(+), 47 deletions(-)
include/linux/mm.h +47 -43
index d16b33bacc32..7030108c3759 100644
@@ -268,62 +268,66 @@ extern struct rw_semaphore nommu_region_sem;
 
 extern unsigned int kobjsize(const void *objp);
 #endif
-
 /*
  * vm_flags in vm_area_struct, see mm_types.h.
  * When changing, update also include/trace/events/mmflags.h
  */
-#define VM_NONE		0x00000000
+enum vm_flags {
+	VM_NONE         = 0x00000000,
+
+	VM_READ         = BIT(0), /* currently active flags */
+	VM_WRITE        = BIT(1),
+	VM_EXEC         = BIT(2),
+	VM_SHARED       = BIT(3),
 
-#define VM_READ		0x00000001	/* currently active flags */
-#define VM_WRITE	0x00000002
-#define VM_EXEC		0x00000004
-#define VM_SHARED	0x00000008
+	/* mprotect() hardcodes VM_MAYREAD >> 4 == VM_READ, and so for r/w/x bits. */
+	VM_MAYREAD      = BIT(4), /* limits for mprotect() etc */
+	VM_MAYWRITE     = BIT(5),
+	VM_MAYEXEC      = BIT(6),
+	VM_MAYSHARE     = BIT(7),
 
-/* mprotect() hardcodes VM_MAYREAD >> 4 == VM_READ, and so for r/w/x bits. */
-#define VM_MAYREAD	0x00000010	/* limits for mprotect() etc */
-#define VM_MAYWRITE	0x00000020
-#define VM_MAYEXEC	0x00000040
-#define VM_MAYSHARE	0x00000080
+	VM_GROWSDOWN    = BIT(8), /* general info on the segment */
 
-#define VM_GROWSDOWN	0x00000100	/* general info on the segment */
 #ifdef CONFIG_MMU
-#define VM_UFFD_MISSING	0x00000200	/* missing pages tracking */
-#else /* CONFIG_MMU */
-#define VM_MAYOVERLAY	0x00000200	/* nommu: R/O MAP_PRIVATE mapping that might overlay a file mapping */
-#define VM_UFFD_MISSING	0
-#endif /* CONFIG_MMU */
-#define VM_PFNMAP	0x00000400	/* Page-ranges managed without "struct page", just pure PFN */
-#define VM_UFFD_WP	0x00001000	/* wrprotect pages tracking */
-
-#define VM_LOCKED	0x00002000
-#define VM_IO           0x00004000	/* Memory mapped I/O or similar */
-
-					/* Used by sys_madvise() */
-#define VM_SEQ_READ	0x00008000	/* App will access data sequentially */
-#define VM_RAND_READ	0x00010000	/* App will not benefit from clustered reads */
-
-#define VM_DONTCOPY	0x00020000      /* Do not copy this vma on fork */
-#define VM_DONTEXPAND	0x00040000	/* Cannot expand with mremap() */
-#define VM_LOCKONFAULT	0x00080000	/* Lock the pages covered when they are faulted in */
-#define VM_ACCOUNT	0x00100000	/* Is a VM accounted object */
-#define VM_NORESERVE	0x00200000	/* should the VM suppress accounting */
-#define VM_HUGETLB	0x00400000	/* Huge TLB Page VM */
-#define VM_SYNC		0x00800000	/* Synchronous page faults */
-#define VM_ARCH_1	0x01000000	/* Architecture-specific flag */
-#define VM_WIPEONFORK	0x02000000	/* Wipe VMA contents in child. */
-#define VM_DONTDUMP	0x04000000	/* Do not include in the core dump */
+	VM_UFFD_MISSING = BIT(9),  /* missing pages tracking */
+#else
+	/* nommu: R/O MAP_PRIVATE mapping that might overlay a file mapping */
+	VM_MAYOVERLAY   = BIT(9),
+	VM_UFFD_MISSING = 0,
+#endif
+
+	VM_PFNMAP       = BIT(10), /* Page-ranges managed without "struct page", just pure PFN */
+	VM_UFFD_WP      = BIT(12), /* wrprotect pages tracking */
+
+	VM_LOCKED       = BIT(13),
+	VM_IO           = BIT(14), /* Memory mapped I/O or similar */
+
+	/* Used by sys_madvise() */
+	VM_SEQ_READ     = BIT(15), /* App will access data sequentially */
+	VM_RAND_READ    = BIT(16), /* App will not benefit from clustered reads */
+
+	VM_DONTCOPY     = BIT(17), /* Do not copy this vma on fork */
+	VM_DONTEXPAND   = BIT(18), /* Cannot expand with mremap() */
+	VM_LOCKONFAULT  = BIT(19), /* Lock the pages covered when they are faulted in */
+	VM_ACCOUNT      = BIT(20), /* Is a VM accounted object */
+	VM_NORESERVE    = BIT(21), /* should the VM suppress accounting */
+	VM_HUGETLB      = BIT(22), /* Huge TLB Page VM */
+	VM_SYNC         = BIT(23), /* Synchronous page faults */
+	VM_ARCH_1       = BIT(24), /* Architecture-specific flag */
+	VM_WIPEONFORK   = BIT(25), /* Wipe VMA contents in child. */
+	VM_DONTDUMP     = BIT(26), /* Do not include in the core dump */
 
 #ifdef CONFIG_MEM_SOFT_DIRTY
-# define VM_SOFTDIRTY	0x08000000	/* Not soft dirty clean area */
+	VM_SOFTDIRTY    = BIT(27), /* Not soft dirty clean area */
 #else
-# define VM_SOFTDIRTY	0
+	VM_SOFTDIRTY    = 0,
 #endif
 
-#define VM_MIXEDMAP	0x10000000	/* Can contain "struct page" and pure PFN pages */
-#define VM_HUGEPAGE	0x20000000	/* MADV_HUGEPAGE marked this vma */
-#define VM_NOHUGEPAGE	0x40000000	/* MADV_NOHUGEPAGE marked this vma */
-#define VM_MERGEABLE	BIT(31)		/* KSM may merge identical pages */
+	VM_MIXEDMAP     = BIT(28), /* Can contain "struct page" and pure PFN pages */
+	VM_HUGEPAGE     = BIT(29), /* MADV_HUGEPAGE marked this vma */
+	VM_NOHUGEPAGE   = BIT(30), /* MADV_NOHUGEPAGE marked this vma */
+	VM_MERGEABLE    = BIT(31), /* KSM may merge identical pages */
+};
 
 #ifdef CONFIG_ARCH_USES_HIGH_VMA_FLAGS
 #define VM_HIGH_ARCH_BIT_0	32	/* bit only usable on 64-bit architectures */
include/linux/mman.h +3 -5
index 0ba8a7e8b90a..4a0adbdeccdd 100644
@@ -130,10 +130,9 @@ static inline bool arch_validate_flags(unsigned long flags)
  * ("bit1" and "bit2" must be single bits)
  */
 #define _calc_vm_trans(x, bit1, bit2) \
-  ((!(bit1) || !(bit2)) ? 0 : \
-  ((bit1) <= (bit2) ? ((x) & (bit1)) * ((bit2) / (bit1)) \
-   : ((x) & (bit1)) / ((bit1) / (bit2))))
-
+	(((bit1) == 0 || (bit2) == 0) ? 0 : \
+	 ((bit1) <= (bit2) ? ((x) & (bit1)) * ((bit2) / (bit1)) \
+	  : ((x) & (bit1)) / ((bit1) / (bit2))))
 /*
  * Combine the mmap "prot" argument into "vm_flags" used internally.
  */
--

2.43.0

#3 lore raw
Quoted 11 lines · Lorenzo Stoakes
Quoted 8 lines · Ye Liu

From: Ye Liu <[email protected]>

Hello MM maintainers and drgn community,

This RFC proposes to convert VM_* flags from #define macros to enum vm_flags. The motivation comes from recent drgn development where we encountered difficulties in implementing VM flag parsing due to the current macro-based approach.

This isn't going to work sorry, it's not valid to have flag values as an enum

I don't follow, can you elaborate? IIRC, the compiler will use an integer type to back the enum that will fit all values.

Quoted 4 lines

(they're distinct) and also - importantly - I'm going to be making significant changes to VMA flags soon (to allow us to have arbitrary number of VMA flags on _all_ architectures as recently done with my series doing something similar with mm flags).

I guess this patch should not really make a big difference regarding your upcoming plans?

I do hate the enum stuff to make these tools happy, though :)

Signature · 3 lines
Cheers

David / dhildenb
#4 lore raw
Quoted 11 lines · David Hildenbrand
Quoted 10 lines · Lorenzo Stoakes
Quoted 8 lines · Ye Liu

From: Ye Liu <[email protected]>

Hello MM maintainers and drgn community,

This RFC proposes to convert VM_* flags from #define macros to enum vm_flags. The motivation comes from recent drgn development where we encountered difficulties in implementing VM flag parsing due to the current macro-based approach.

Quoted 4 lines
Quoted 1 line

This isn't going to work sorry, it's not valid to have flag values as an enum

I don't follow, can you elaborate? IIRC, the compiler will use an integer type to back the enum that will fit all values.

switch (flags) {
case VAL1:
case VAL2:
etc.
}

Is broken (compiler will say you cover all cases when you don't...)

An enum implies independent values that exhaustively describe all state, however these flag values are not that - they're intended to be bit fields.

Also we specifically have the vm_flags_t whose type VMA flags should be, right now it's an integer type - unsigned long - after my changes it'll be an opaque type that'll blow up expectations of what this change does altogether.

Quoted 8 lines
Quoted 5 lines

(they're distinct) and also - importantly - I'm going to be making significant changes to VMA flags soon (to allow us to have arbitrary number of VMA flags on _all_ architectures as recently done with my series doing something similar with mm flags).

I guess this patch should not really make a big difference regarding your upcoming plans?

I don't think it's really useful to do this in any way, and as I said, my change will allow these tools to have a much better solution, which is to reference the
- actually valid as an enum - bit values.

Quoted 2 lines

I do hate the enum stuff to make these tools happy, though :)

Yes, if the tools can't handle standard kernel conventions, the problem is not in the kernel :)

Quoted 6 lines

--
Cheers

David / dhildenb

If it makes life easier - I can prioritise doing the 'make VMA flag bit numbers an enum' patch as an early part of the upcoming series (and in fact likely to be multiple series given the scope).

Cheers, Lorenzo

#5 lore raw
Quoted 17 lines · Lorenzo Stoakes
Quoted 11 lines · David Hildenbrand
Quoted 10 lines · Lorenzo Stoakes
Quoted 8 lines · Ye Liu

From: Ye Liu <[email protected]>

Hello MM maintainers and drgn community,

This RFC proposes to convert VM_* flags from #define macros to enum vm_flags. The motivation comes from recent drgn development where we encountered difficulties in implementing VM flag parsing due to the current macro-based approach.

Quoted 4 lines
Quoted 1 line

This isn't going to work sorry, it's not valid to have flag values as an enum

I don't follow, can you elaborate? IIRC, the compiler will use an integer type to back the enum that will fit all values.

Quoted 7 lines

switch (flags) {
case VAL1:
case VAL2:
etc.
}

Is broken (compiler will say you cover all cases when you don't...)

I assume you mean theoretically, because there is no such code, right?

Quoted 4 lines

An enum implies independent values that exhaustively describe all state, however these flag values are not that - they're intended to be bit fields.

Observe how we use an enum for FOLL_* flags, vm_fault_reason, fault_flag and probably other things.

But more importantly,

enum pageflags { ... :)

Signature · 3 lines
Cheers

David / dhildenb
#6 lore raw
Quoted 33 lines · David Hildenbrand
Quoted 17 lines · Lorenzo Stoakes
Quoted 11 lines · David Hildenbrand
Quoted 10 lines · Lorenzo Stoakes
Quoted 8 lines · Ye Liu

From: Ye Liu <[email protected]>

Hello MM maintainers and drgn community,

This RFC proposes to convert VM_* flags from #define macros to enum vm_flags. The motivation comes from recent drgn development where we encountered difficulties in implementing VM flag parsing due to the current macro-based approach.

Quoted 4 lines
Quoted 1 line

This isn't going to work sorry, it's not valid to have flag values as an enum

I don't follow, can you elaborate? IIRC, the compiler will use an integer type to back the enum that will fit all values.

Quoted 7 lines

switch (flags) {
case VAL1:
case VAL2:
etc.
}

Is broken (compiler will say you cover all cases when you don't...)

I assume you mean theoretically, because there is no such code, right?

Quoted 4 lines

An enum implies independent values that exhaustively describe all state, however these flag values are not that - they're intended to be bit fields.

Quoted 6 lines

Observe how we use an enum for FOLL_* flags, vm_fault_reason, fault_flag and probably other things.

But more importantly,

enum pageflags { ... :)

Okay, that one is different, because it's the actual bit numbers today.

Signature · 3 lines
Cheers

David / dhildenb
#7 lore raw
In reply to David Hildenbrand
Quoted 24 lines · David Hildenbrand
Quoted 23 lines · Lorenzo Stoakes
Quoted 11 lines · David Hildenbrand
Quoted 10 lines · Lorenzo Stoakes
Quoted 8 lines · Ye Liu

From: Ye Liu <[email protected]>

Hello MM maintainers and drgn community,

This RFC proposes to convert VM_* flags from #define macros to enum vm_flags. The motivation comes from recent drgn development where we encountered difficulties in implementing VM flag parsing due to the current macro-based approach.

Quoted 4 lines
Quoted 1 line

This isn't going to work sorry, it's not valid to have flag values as an enum

I don't follow, can you elaborate? IIRC, the compiler will use an integer type to back the enum that will fit all values.

switch (flags) {
case VAL1:
case VAL2:
etc.
}

Quoted 3 lines
Quoted 1 line

Is broken (compiler will say you cover all cases when you don't...)

I assume you mean theoretically, because there is no such code, right?

Right, it's a general point about why enum's are not such a great idea for this.

Quoted 8 lines
Quoted 5 lines

An enum implies independent values that exhaustively describe all state, however these flag values are not that - they're intended to be bit fields.

Observe how we use an enum for FOLL_* flags, vm_fault_reason, fault_flag and probably other things.

FOLL_* flags are an anonymous enum, enum fault_flag is not used as a type anywhere, nor is vm_fault_reason. So those are both kinda weird as to why we even name the type (they're in effect anonymous).

But also 'we do X in the kernel' doesn't mean doing X is right :)

Quoted 4 lines

But more importantly,

enum pageflags { ... :)

Your reply answers this :)

Quoted 6 lines

--
Cheers

David / dhildenb

#8 lore raw
Quoted 37 lines · Lorenzo Stoakes
Quoted 24 lines · David Hildenbrand
Quoted 23 lines · Lorenzo Stoakes
Quoted 11 lines · David Hildenbrand
Quoted 10 lines · Lorenzo Stoakes
Quoted 8 lines · Ye Liu

From: Ye Liu <[email protected]>

Hello MM maintainers and drgn community,

This RFC proposes to convert VM_* flags from #define macros to enum vm_flags. The motivation comes from recent drgn development where we encountered difficulties in implementing VM flag parsing due to the current macro-based approach.

Quoted 4 lines
Quoted 1 line

This isn't going to work sorry, it's not valid to have flag values as an enum

I don't follow, can you elaborate? IIRC, the compiler will use an integer type to back the enum that will fit all values.

switch (flags) {
case VAL1:
case VAL2:
etc.
}

Quoted 3 lines
Quoted 1 line

Is broken (compiler will say you cover all cases when you don't...)

I assume you mean theoretically, because there is no such code, right?

Right, it's a general point about why enum's are not such a great idea for this.

Quoted 6 lines
Quoted 5 lines

An enum implies independent values that exhaustively describe all state, however these flag values are not that - they're intended to be bit fields.

Quoted 8 lines
Quoted 2 lines

Observe how we use an enum for FOLL_* flags, vm_fault_reason, fault_flag and probably other things.

FOLL_* flags are an anonymous enum, enum fault_flag is not used as a type anywhere, nor is vm_fault_reason. So those are both kinda weird as to why we even name the type (they're in effect anonymous).

But also 'we do X in the kernel' doesn't mean doing X is right :)

I think the example to follow could be GFP flags. Nowadays there's an enum below it, and a layer that adds (__force gfp_t), so you could do similar thing with vm_flags_t.

However I'm not sure how compatible is that with Lorenzo's plans.

#12 lore raw
In reply to Lorenzo Stoakes
Quoted 37 lines · Lorenzo Stoakes
Quoted 24 lines · David Hildenbrand
Quoted 23 lines · Lorenzo Stoakes
Quoted 11 lines · David Hildenbrand
Quoted 10 lines · Lorenzo Stoakes
Quoted 8 lines · Ye Liu

From: Ye Liu <[email protected]>

Hello MM maintainers and drgn community,

This RFC proposes to convert VM_* flags from #define macros to enum vm_flags. The motivation comes from recent drgn development where we encountered difficulties in implementing VM flag parsing due to the current macro-based approach.

Quoted 4 lines
Quoted 1 line

This isn't going to work sorry, it's not valid to have flag values as an enum

I don't follow, can you elaborate? IIRC, the compiler will use an integer type to back the enum that will fit all values.

switch (flags) {
case VAL1:
case VAL2:
etc.
}

Quoted 3 lines
Quoted 1 line

Is broken (compiler will say you cover all cases when you don't...)

I assume you mean theoretically, because there is no such code, right?

Right, it's a general point about why enum's are not such a great idea for this.

Quoted 6 lines
Quoted 5 lines

An enum implies independent values that exhaustively describe all state, however these flag values are not that - they're intended to be bit fields.

Quoted 6 lines
Quoted 2 lines

Observe how we use an enum for FOLL_* flags, vm_fault_reason, fault_flag and probably other things.

FOLL_* flags are an anonymous enum, enum fault_flag is not used as a type anywhere, nor is vm_fault_reason. So those are both kinda weird as to why we even name the type (they're in effect anonymous).

Well, there are of course more, even in MM.

For these cases, we indeed don't declare a type, but I am not really sure if it makes a big difference here today.

I agree that once the flags are actually an opaque object (what you are planning on to work on), it will be a different story.

Quoted 2 lines

But also 'we do X in the kernel' doesn't mean doing X is right :)

Right, but it in general works. It's just nasty here in particular.

Signature · 3 lines
Cheers

David / dhildenb
#9 lore raw
In reply to Vlastimil Babka
Quoted 5 lines · Vlastimil Babka
Quoted 4 lines · Lorenzo Stoakes

FOLL_* flags are an anonymous enum, enum fault_flag is not used as a type anywhere, nor is vm_fault_reason. So those are both kinda weird as to why we even name the type (they're in effect anonymous).

Quoted 7 lines
Quoted 1 line

But also 'we do X in the kernel' doesn't mean doing X is right :)

I think the example to follow could be GFP flags. Nowadays there's an enum below it, and a layer that adds (__force gfp_t), so you could do similar thing with vm_flags_t.

However I'm not sure how compatible is that with Lorenzo's plans.

That's defining bit values in an anonymous enum so isn't really comparable.

But what it's doing, ultimately, in broad terms (other than the opaque bitmap type I'll be using for VMA flags) is what my changes will do.

And yeah, trying to do duplicate that is not really a good use of time and will conflict with my work.

Overall I think this change is generally unnecessary given that I'm about to radically alter how VMA flags are implemented, and actually will cause me problems.

But as I said before, I'm happy to prioritise the change that specifies the flags based on the bit numbers, I actually have it ready more-or-less.

Cheers, Lorenzo

#10 lore raw
Quoted 22 lines · Lorenzo Stoakes
Quoted 5 lines · Vlastimil Babka
Quoted 4 lines · Lorenzo Stoakes

FOLL_* flags are an anonymous enum, enum fault_flag is not used as a type anywhere, nor is vm_fault_reason. So those are both kinda weird as to why we even name the type (they're in effect anonymous).

Quoted 7 lines
Quoted 1 line

But also 'we do X in the kernel' doesn't mean doing X is right :)

I think the example to follow could be GFP flags. Nowadays there's an enum below it, and a layer that adds (__force gfp_t), so you could do similar thing with vm_flags_t.

However I'm not sure how compatible is that with Lorenzo's plans.

That's defining bit values in an anonymous enum so isn't really comparable.

But what it's doing, ultimately, in broad terms (other than the opaque bitmap type I'll be using for VMA flags) is what my changes will do.

And yeah, trying to do duplicate that is not really a good use of time and will conflict with my work.

Quoted 6 lines

Overall I think this change is generally unnecessary given that I'm about to radically alter how VMA flags are implemented, and actually will cause me problems.

But as I said before, I'm happy to prioritise the change that specifies the flags based on the bit numbers, I actually have it ready more-or-less.

Yeah, let's do that if you expect to have it ready soonish.

Signature · 3 lines
Cheers

David / dhildenb
author
#11 lore raw
In reply to Lorenzo Stoakes
Quoted 22 lines · Lorenzo Stoakes
Quoted 5 lines · Vlastimil Babka
Quoted 4 lines · Lorenzo Stoakes

FOLL_* flags are an anonymous enum, enum fault_flag is not used as a type anywhere, nor is vm_fault_reason. So those are both kinda weird as to why we even name the type (they're in effect anonymous).

Quoted 7 lines
Quoted 1 line

But also 'we do X in the kernel' doesn't mean doing X is right :)

I think the example to follow could be GFP flags. Nowadays there's an enum below it, and a layer that adds (__force gfp_t), so you could do similar thing with vm_flags_t.

However I'm not sure how compatible is that with Lorenzo's plans.

That's defining bit values in an anonymous enum so isn't really comparable.

But what it's doing, ultimately, in broad terms (other than the opaque bitmap type I'll be using for VMA flags) is what my changes will do.

And yeah, trying to do duplicate that is not really a good use of time and will conflict with my work.

Quoted 7 lines

Overall I think this change is generally unnecessary given that I'm about to radically alter how VMA flags are implemented, and actually will cause me problems.

But as I said before, I'm happy to prioritise the change that specifies the flags based on the bit numbers, I actually have it ready more-or-less.

Thanks for the input, everyone.

Lorenzo, I agree with your assessment.
I'll pause on this from my side and wait for your patch. I'm excited to see the new implementation.

Quoted 1 line

Cheers, Lorenzo

Signature · 2 lines
Thanks,
Ye Liu