Skip to contentSkip to conversation

reading proc/pid/maps under RCU

RFC In review Suren Baghdasaryan 3 participants 10 messages Jan 15, 2024 – Jan 22, 2024

Subsystems (7)
lore
Share & tools
lore

Suren Baghdasaryan

author reviewer
#0 lore raw

cover reading proc/pid/maps under RCU

The issue this patchset is trying to address is mmap_lock contention when a low priority task (monitoring, data collecting, etc.) blocks a higher priority task from making updated to the address space. The contention is due to the mmap_lock being held for read when reading proc/pid/maps. With maple_tree introduction, VMA tree traversals are RCU-safe and per-vma locks make VMA access RCU-safe. this provides an opportunity for lock-less reading of proc/pid/maps. We still need to overcome a couple obstacles:
1. Make all VMA pointer fields used for proc/pid/maps content generation RCU-safe;
2. Ensure that proc/pid/maps data tearing, which is currently possible at page boundaries only, does not get worse.

The patchset deals with these issues but there is a downside which I would like to get input on:
This change introduces unfairness towards the reader of proc/pid/maps, which can be blocked by an overly active/malicious address space modifyer. A couple of ways I though we can address this issue are:
1. After several lock-less retries (or some time limit) to fall back to taking mmap_lock.
2. Employ lock-less reading only if the reader has low priority, indicating that blocking it is not critical.
3. Introducing a separate procfs file which publishes the same data in lock-less manner.

I imagine a combination of these approaches can also be employed. I would like to get feedback on this from the Linux community.

Note: mmap_read_lock/mmap_read_unlock sequence inside validate_map() can be replaced with more efficiend rwsem_wait() proposed by Matthew in [1].

[1] https://lore.kernel.org/all/[email protected]/

Suren Baghdasaryan (3):
mm: make vm_area_struct anon_name field RCU-safe
seq_file: add validate() operation to seq_operations mm/maps: read proc/pid/maps under RCU

 fs/proc/internal.h        |   3 +
 fs/proc/task_mmu.c        | 130 ++++++++++++++++++++++++++++++++++----
 fs/seq_file.c             |  24 ++++++-
 include/linux/mm_inline.h |  10 ++-
 include/linux/mm_types.h  |   3 +-
 include/linux/seq_file.h  |   1 +
 mm/madvise.c              |  30 +++++++--
 7 files changed, 181 insertions(+), 20 deletions(-)
Signature · 1 line
2.43.0.381.gb435a96ce8-goog
author reviewer
#1 lore raw

RFC 1/3 mm: make vm_area_struct anon_name field RCU-safe

For lockless /proc/pid/maps reading we have to ensure all the fields used when generating the output are RCU-safe. The only pointer fields in vm_area_struct which are used to generate that file's output are vm_file and anon_name. vm_file is RCU-safe but anon_name is not. Make anon_name RCU-safe as well.

---
 include/linux/mm_inline.h | 10 +++++++++-
 include/linux/mm_types.h  |  3 ++-
 mm/madvise.c              | 30 ++++++++++++++++++++++++++----
 3 files changed, 37 insertions(+), 6 deletions(-)
include/linux/mm_inline.h +9 -1
index f4fe593c1400..bbdb0ca857f1 100644
@@ -389,7 +389,7 @@ static inline void dup_anon_vma_name(struct vm_area_struct *orig_vma,
 	struct anon_vma_name *anon_name = anon_vma_name(orig_vma);
 
 	if (anon_name)
-		new_vma->anon_name = anon_vma_name_reuse(anon_name);
+		rcu_assign_pointer(new_vma->anon_name, anon_vma_name_reuse(anon_name));
 }
 
 static inline void free_anon_vma_name(struct vm_area_struct *vma)
@@ -411,6 +411,8 @@ static inline bool anon_vma_name_eq(struct anon_vma_name *anon_name1,
 		!strcmp(anon_name1->name, anon_name2->name);
 }
 
+struct anon_vma_name *anon_vma_name_get_rcu(struct vm_area_struct *vma);
+
 #else /* CONFIG_ANON_VMA_NAME */
 static inline void anon_vma_name_get(struct anon_vma_name *anon_name) {}
 static inline void anon_vma_name_put(struct anon_vma_name *anon_name) {}
@@ -424,6 +426,12 @@ static inline bool anon_vma_name_eq(struct anon_vma_name *anon_name1,
 	return true;
 }
 
+static inline
+struct anon_vma_name *anon_vma_name_get_rcu(struct vm_area_struct *vma)
+{
+	return NULL;
+}
+
 #endif  /* CONFIG_ANON_VMA_NAME */
 
 static inline void init_tlb_flush_pending(struct mm_struct *mm)
include/linux/mm_types.h +2 -1
index b2d3a88a34d1..1f0a30c00795 100644
@@ -545,6 +545,7 @@ struct vm_userfaultfd_ctx {};
 
 struct anon_vma_name {
 	struct kref kref;
+	struct rcu_head rcu;
 	/* The name needs to be at the end because it is dynamically sized. */
 	char name[];
 };
@@ -699,7 +700,7 @@ struct vm_area_struct {
 	 * terminated string containing the name given to the vma, or NULL if
 	 * unnamed. Serialized by mmap_lock. Use anon_vma_name to access.
 	 */
-	struct anon_vma_name *anon_name;
+	struct anon_vma_name __rcu *anon_name;
 #endif
 #ifdef CONFIG_SWAP
 	atomic_long_t swap_readahead_info;
mm/madvise.c +26 -4
index 912155a94ed5..0f222d464254 100644
@@ -88,14 +88,15 @@ void anon_vma_name_free(struct kref *kref)
 {
 	struct anon_vma_name *anon_name =
 			container_of(kref, struct anon_vma_name, kref);
-	kfree(anon_name);
+	kfree_rcu(anon_name, rcu);
 }
 
 struct anon_vma_name *anon_vma_name(struct vm_area_struct *vma)
 {
 	mmap_assert_locked(vma->vm_mm);
 
-	return vma->anon_name;
+	return rcu_dereference_protected(vma->anon_name,
+		rwsem_is_locked(&vma->vm_mm->mmap_lock));
 }
 
 /* mmap_lock should be write-locked */
@@ -105,7 +106,7 @@ static int replace_anon_vma_name(struct vm_area_struct *vma,
 	struct anon_vma_name *orig_name = anon_vma_name(vma);
 
 	if (!anon_name) {
-		vma->anon_name = NULL;
+		rcu_assign_pointer(vma->anon_name, NULL);
 		anon_vma_name_put(orig_name);
 		return 0;
 	}
@@ -113,11 +114,32 @@ static int replace_anon_vma_name(struct vm_area_struct *vma,
 	if (anon_vma_name_eq(orig_name, anon_name))
 		return 0;
 
-	vma->anon_name = anon_vma_name_reuse(anon_name);
+	rcu_assign_pointer(vma->anon_name, anon_vma_name_reuse(anon_name));
 	anon_vma_name_put(orig_name);
 
 	return 0;
 }
+
+/*
+ * Returned anon_vma_name is stable due to elevated refcount but not guaranteed
+ * to be assigned to the original VMA after the call.
+ */
+struct anon_vma_name *anon_vma_name_get_rcu(struct vm_area_struct *vma)
+{
+	struct anon_vma_name __rcu *anon_name;
+
+	WARN_ON_ONCE(!rcu_read_lock_held());
+
+	anon_name = rcu_dereference(vma->anon_name);
+	if (!anon_name)
+		return NULL;
+
+	if (unlikely(!kref_get_unless_zero(&anon_name->kref)))
+		return NULL;
+
+	return anon_name;
+}
+
 #else /* CONFIG_ANON_VMA_NAME */
 static int replace_anon_vma_name(struct vm_area_struct *vma,
 				 struct anon_vma_name *anon_name)
Signature · 1 line
2.43.0.381.gb435a96ce8-goog
author reviewer
#2 lore raw

RFC 2/3 seq_file: add validate() operation to seq_operations

In reply to Suren Baghdasaryan

seq_file outputs data in chunks using seq_file.buf as the intermediate storage before outputting the generated data for the current chunk. It is possible for already buffered data to become stale before it gets reported. In certain situations it is desirable to regenerate that data instead of reporting the stale one. Provide a validate() operation called before outputting the buffered data to allow users to validate buffered data. To indicate valid data, user's validate callback should return 0, to request regeneration of the stale data it should return -EAGAIN, any other error will be considered fatal and read operation will be aborted.

---
 fs/seq_file.c            | 24 +++++++++++++++++++++++-
 include/linux/seq_file.h |  1 +
 2 files changed, 24 insertions(+), 1 deletion(-)
fs/seq_file.c +23 -1
index f5fdaf3b1572..77833bbe5909 100644
@@ -172,6 +172,8 @@ ssize_t seq_read_iter(struct kiocb *iocb, struct iov_iter *iter)
 {
 	struct seq_file *m = iocb->ki_filp->private_data;
 	size_t copied = 0;
+	loff_t orig_index;
+	size_t orig_count;
 	size_t n;
 	void *p;
 	int err = 0;
@@ -220,6 +222,10 @@ ssize_t seq_read_iter(struct kiocb *iocb, struct iov_iter *iter)
 		if (m->count)	// hadn't managed to copy everything
 			goto Done;
 	}
+
+	orig_index = m->index;
+	orig_count = m->count;
+Again:
 	// get a non-empty record in the buffer
 	m->from = 0;
 	p = m->op->start(m, &m->index);
@@ -278,6 +284,22 @@ ssize_t seq_read_iter(struct kiocb *iocb, struct iov_iter *iter)
 		}
 	}
 	m->op->stop(m, p);
+	/* Note: we validate even if err<0 to prevent publishing copied data */
+	if (m->op->validate) {
+		int val_err = m->op->validate(m, p);
+
+		if (val_err) {
+			if (val_err == -EAGAIN) {
+				m->index = orig_index;
+				m->count = orig_count;
+				// data is stale, retry
+				goto Again;
+			}
+			// data is invalid, return the last error
+			err = val_err;
+			goto Done;
+		}
+	}
 	n = copy_to_iter(m->buf, m->count, iter);
 	copied += n;
 	m->count -= n;
@@ -572,7 +594,7 @@ static void single_stop(struct seq_file *p, void *v)
 int single_open(struct file *file, int (*show)(struct seq_file *, void *),
 		void *data)
 {
-	struct seq_operations *op = kmalloc(sizeof(*op), GFP_KERNEL_ACCOUNT);
+	struct seq_operations *op = kzalloc(sizeof(*op), GFP_KERNEL_ACCOUNT);
 	int res = -ENOMEM;
 
 	if (op) {
include/linux/seq_file.h +1 -0
index 234bcdb1fba4..d0fefac2990f 100644
@@ -34,6 +34,7 @@ struct seq_operations {
 	void (*stop) (struct seq_file *m, void *v);
 	void * (*next) (struct seq_file *m, void *v, loff_t *pos);
 	int (*show) (struct seq_file *m, void *v);
+	int (*validate)(struct seq_file *m, void *v);
 };
 
 #define SEQ_SKIP 1
Signature · 1 line
2.43.0.381.gb435a96ce8-goog
author reviewer
#3 lore raw

RFC 3/3 mm/maps: read proc/pid/maps under RCU

In reply to Suren Baghdasaryan

With maple_tree supporting vma tree traversal under RCU and per-vma locks making vma access RCU-safe, /proc/pid/maps can be read under RCU and without the need to read-lock mmap_lock. However vma content can change from under us, therefore we need to pin pointer fields used when generating the output (currently only vm_file and anon_name). In addition, we validate data before publishing it to the user using new seq_file validate interface. This way we keep this mechanism consistent with the previous behavior where data tearing is possible only at page boundaries.
This change is designed to reduce mmap_lock contention and prevent a process reading /proc/pid/maps files (often a low priority task, such as monitoring/data collection services) from blocking address space updates.

---
 fs/proc/internal.h |   3 ++
 fs/proc/task_mmu.c | 130 ++++++++++++++++++++++++++++++++++++++++-----
 2 files changed, 120 insertions(+), 13 deletions(-)
fs/proc/internal.h +3 -0
index a71ac5379584..47233408550b 100644
@@ -290,6 +290,9 @@ struct proc_maps_private {
 	struct task_struct *task;
 	struct mm_struct *mm;
 	struct vma_iterator iter;
+	int mm_lock_seq;
+	struct anon_vma_name *anon_name;
+	struct file *vm_file;
 #ifdef CONFIG_NUMA
 	struct mempolicy *task_mempolicy;
 #endif
fs/proc/task_mmu.c +117 -13

168 lines not on this page · raw message

Signature · 1 line
2.43.0.381.gb435a96ce8-goog
#4 lore raw

Re: [RFC 3/3] mm/maps: read proc/pid/maps under RCU

kernel test robot noticed the following build errors:

Hi Suren,

[This is a private test report for your RFC patch.] kernel test robot noticed the following build errors:

[auto build test ERROR on akpm-mm/mm-everything]
[also build test ERROR on linus/master v6.7 next-20240118] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Suren-Baghdasaryan/mm-make-vm_area_struct-anon_name-field-RCU-safe/20240116-024359
base:   https://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm.git mm-everything
patch link:    https://lore.kernel.org/r/20240115183837.205694-4-surenb%40google.com
patch subject: [RFC 3/3] mm/maps: read proc/pid/maps under RCU
config: i386-randconfig-011-20240118 (https://download.01.org/0day-ci/archive/20240118/[email protected]/config)
compiler: gcc-11 (Debian 11.3.0-12) 11.3.0
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20240118/[email protected]/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

All errors (new ones prefixed by >>):

In file included from include/linux/list.h:11,

                    from include/linux/preempt.h:12,
                    from include/linux/spinlock.h:56,
                    from include/linux/mmzone.h:8,
                    from include/linux/gfp.h:7,
                    from include/linux/mm.h:7,
                    from include/linux/pagewalk.h:5,
                    from fs/proc/task_mmu.c:2:
   fs/proc/task_mmu.c: In function 'm_start':
>> fs/proc/task_mmu.c:190:63: error: 'struct mm_struct' has no member named 'mm_lock_seq'
     190 |                 priv->mm_lock_seq = smp_load_acquire(&priv->mm->mm_lock_seq);
         |                                                               ^~
   arch/x86/include/asm/barrier.h:72:17: note: in definition of macro '__smp_load_acquire'
      72 |         typeof(*p) ___p1 = READ_ONCE(*p);                               \
         |                 ^
   fs/proc/task_mmu.c:190:37: note: in expansion of macro 'smp_load_acquire'
     190 |                 priv->mm_lock_seq = smp_load_acquire(&priv->mm->mm_lock_seq);
         |                                     ^~~~~~~~~~~~~~~~
   In file included from <command-line>:
>> fs/proc/task_mmu.c:190:63: error: 'struct mm_struct' has no member named 'mm_lock_seq'
     190 |                 priv->mm_lock_seq = smp_load_acquire(&priv->mm->mm_lock_seq);
         |                                                               ^~
   include/linux/compiler_types.h:415:23: note: in definition of macro '__compiletime_assert'
     415 |                 if (!(condition))                                       \
         |                       ^~~~~~~~~
   include/linux/compiler_types.h:435:9: note: in expansion of macro '_compiletime_assert'
     435 |         _compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__)
         |         ^~~~~~~~~~~~~~~~~~~
   include/asm-generic/rwonce.h:36:9: note: in expansion of macro 'compiletime_assert'
      36 |         compiletime_assert(__native_word(t) || sizeof(t) == sizeof(long long),  \
         |         ^~~~~~~~~~~~~~~~~~
   include/asm-generic/rwonce.h:36:28: note: in expansion of macro '__native_word'
      36 |         compiletime_assert(__native_word(t) || sizeof(t) == sizeof(long long),  \
         |                            ^~~~~~~~~~~~~
   include/asm-generic/rwonce.h:49:9: note: in expansion of macro 'compiletime_assert_rwonce_type'
      49 |         compiletime_assert_rwonce_type(x);                              \
         |         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   arch/x86/include/asm/barrier.h:72:28: note: in expansion of macro 'READ_ONCE'
      72 |         typeof(*p) ___p1 = READ_ONCE(*p);                               \
         |                            ^~~~~~~~~
   include/asm-generic/barrier.h:176:29: note: in expansion of macro '__smp_load_acquire'
     176 | #define smp_load_acquire(p) __smp_load_acquire(p)
         |                             ^~~~~~~~~~~~~~~~~~
   fs/proc/task_mmu.c:190:37: note: in expansion of macro 'smp_load_acquire'
     190 |                 priv->mm_lock_seq = smp_load_acquire(&priv->mm->mm_lock_seq);
         |                                     ^~~~~~~~~~~~~~~~
>> fs/proc/task_mmu.c:190:63: error: 'struct mm_struct' has no member named 'mm_lock_seq'
     190 |                 priv->mm_lock_seq = smp_load_acquire(&priv->mm->mm_lock_seq);
         |                                                               ^~
   include/linux/compiler_types.h:415:23: note: in definition of macro '__compiletime_assert'
     415 |                 if (!(condition))                                       \
         |                       ^~~~~~~~~
   include/linux/compiler_types.h:435:9: note: in expansion of macro '_compiletime_assert'
     435 |         _compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__)
         |         ^~~~~~~~~~~~~~~~~~~
   include/asm-generic/rwonce.h:36:9: note: in expansion of macro 'compiletime_assert'
      36 |         compiletime_assert(__native_word(t) || sizeof(t) == sizeof(long long),  \
         |         ^~~~~~~~~~~~~~~~~~
   include/asm-generic/rwonce.h:36:28: note: in expansion of macro '__native_word'
      36 |         compiletime_assert(__native_word(t) || sizeof(t) == sizeof(long long),  \
         |                            ^~~~~~~~~~~~~
   include/asm-generic/rwonce.h:49:9: note: in expansion of macro 'compiletime_assert_rwonce_type'
      49 |         compiletime_assert_rwonce_type(x);                              \
         |         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   arch/x86/include/asm/barrier.h:72:28: note: in expansion of macro 'READ_ONCE'
      72 |         typeof(*p) ___p1 = READ_ONCE(*p);                               \
         |                            ^~~~~~~~~
   include/asm-generic/barrier.h:176:29: note: in expansion of macro '__smp_load_acquire'
     176 | #define smp_load_acquire(p) __smp_load_acquire(p)
         |                             ^~~~~~~~~~~~~~~~~~
   fs/proc/task_mmu.c:190:37: note: in expansion of macro 'smp_load_acquire'
     190 |                 priv->mm_lock_seq = smp_load_acquire(&priv->mm->mm_lock_seq);
         |                                     ^~~~~~~~~~~~~~~~
>> fs/proc/task_mmu.c:190:63: error: 'struct mm_struct' has no member named 'mm_lock_seq'
     190 |                 priv->mm_lock_seq = smp_load_acquire(&priv->mm->mm_lock_seq);
         |                                                               ^~
   include/linux/compiler_types.h:415:23: note: in definition of macro '__compiletime_assert'
     415 |                 if (!(condition))                                       \
         |                       ^~~~~~~~~
   include/linux/compiler_types.h:435:9: note: in expansion of macro '_compiletime_assert'
     435 |         _compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__)
         |         ^~~~~~~~~~~~~~~~~~~
   include/asm-generic/rwonce.h:36:9: note: in expansion of macro 'compiletime_assert'
      36 |         compiletime_assert(__native_word(t) || sizeof(t) == sizeof(long long),  \
         |         ^~~~~~~~~~~~~~~~~~
   include/asm-generic/rwonce.h:36:28: note: in expansion of macro '__native_word'
      36 |         compiletime_assert(__native_word(t) || sizeof(t) == sizeof(long long),  \
         |                            ^~~~~~~~~~~~~
   include/asm-generic/rwonce.h:49:9: note: in expansion of macro 'compiletime_assert_rwonce_type'
      49 |         compiletime_assert_rwonce_type(x);                              \
         |         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   arch/x86/include/asm/barrier.h:72:28: note: in expansion of macro 'READ_ONCE'
      72 |         typeof(*p) ___p1 = READ_ONCE(*p);                               \
         |                            ^~~~~~~~~
   include/asm-generic/barrier.h:176:29: note: in expansion of macro '__smp_load_acquire'
     176 | #define smp_load_acquire(p) __smp_load_acquire(p)
         |                             ^~~~~~~~~~~~~~~~~~
   fs/proc/task_mmu.c:190:37: note: in expansion of macro 'smp_load_acquire'
     190 |                 priv->mm_lock_seq = smp_load_acquire(&priv->mm->mm_lock_seq);
         |                                     ^~~~~~~~~~~~~~~~
>> fs/proc/task_mmu.c:190:63: error: 'struct mm_struct' has no member named 'mm_lock_seq'
     190 |                 priv->mm_lock_seq = smp_load_acquire(&priv->mm->mm_lock_seq);
         |                                                               ^~
   include/linux/compiler_types.h:415:23: note: in definition of macro '__compiletime_assert'
     415 |                 if (!(condition))                                       \
         |                       ^~~~~~~~~
   include/linux/compiler_types.h:435:9: note: in expansion of macro '_compiletime_assert'
     435 |         _compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__)
         |         ^~~~~~~~~~~~~~~~~~~
   include/asm-generic/rwonce.h:36:9: note: in expansion of macro 'compiletime_assert'
      36 |         compiletime_assert(__native_word(t) || sizeof(t) == sizeof(long long),  \
         |         ^~~~~~~~~~~~~~~~~~
   include/asm-generic/rwonce.h:36:28: note: in expansion of macro '__native_word'
      36 |         compiletime_assert(__native_word(t) || sizeof(t) == sizeof(long long),  \
         |                            ^~~~~~~~~~~~~
   include/asm-generic/rwonce.h:49:9: note: in expansion of macro 'compiletime_assert_rwonce_type'
      49 |         compiletime_assert_rwonce_type(x);                              \
         |         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   arch/x86/include/asm/barrier.h:72:28: note: in expansion of macro 'READ_ONCE'
      72 |         typeof(*p) ___p1 = READ_ONCE(*p);                               \
         |                            ^~~~~~~~~
   include/asm-generic/barrier.h:176:29: note: in expansion of macro '__smp_load_acquire'
     176 | #define smp_load_acquire(p) __smp_load_acquire(p)
         |                             ^~~~~~~~~~~~~~~~~~
   fs/proc/task_mmu.c:190:37: note: in expansion of macro 'smp_load_acquire'
     190 |                 priv->mm_lock_seq = smp_load_acquire(&priv->mm->mm_lock_seq);
         |                                     ^~~~~~~~~~~~~~~~
>> fs/proc/task_mmu.c:190:63: error: 'struct mm_struct' has no member named 'mm_lock_seq'
     190 |                 priv->mm_lock_seq = smp_load_acquire(&priv->mm->mm_lock_seq);
         |                                                               ^~
   include/linux/compiler_types.h:415:23: note: in definition of macro '__compiletime_assert'
     415 |                 if (!(condition))                                       \
         |                       ^~~~~~~~~
   include/linux/compiler_types.h:435:9: note: in expansion of macro '_compiletime_assert'
     435 |         _compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__)
         |         ^~~~~~~~~~~~~~~~~~~
   include/asm-generic/rwonce.h:36:9: note: in expansion of macro 'compiletime_assert'
      36 |         compiletime_assert(__native_word(t) || sizeof(t) == sizeof(long long),  \
         |         ^~~~~~~~~~~~~~~~~~
   include/asm-generic/rwonce.h:49:9: note: in expansion of macro 'compiletime_assert_rwonce_type'
      49 |         compiletime_assert_rwonce_type(x);                              \
         |         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   arch/x86/include/asm/barrier.h:72:28: note: in expansion of macro 'READ_ONCE'
      72 |         typeof(*p) ___p1 = READ_ONCE(*p);                               \
         |                            ^~~~~~~~~
   include/asm-generic/barrier.h:176:29: note: in expansion of macro '__smp_load_acquire'
     176 | #define smp_load_acquire(p) __smp_load_acquire(p)
         |                             ^~~~~~~~~~~~~~~~~~
   fs/proc/task_mmu.c:190:37: note: in expansion of macro 'smp_load_acquire'
     190 |                 priv->mm_lock_seq = smp_load_acquire(&priv->mm->mm_lock_seq);
         |                                     ^~~~~~~~~~~~~~~~
>> fs/proc/task_mmu.c:190:63: error: 'struct mm_struct' has no member named 'mm_lock_seq'
     190 |                 priv->mm_lock_seq = smp_load_acquire(&priv->mm->mm_lock_seq);
         |                                                               ^~
   include/linux/compiler_types.h:391:27: note: in definition of macro '__unqual_scalar_typeof'
     391 |                 _Generic((x),                                           \
         |                           ^
   include/asm-generic/rwonce.h:50:9: note: in expansion of macro '__READ_ONCE'
      50 |         __READ_ONCE(x);                                                 \
         |         ^~~~~~~~~~~
   arch/x86/include/asm/barrier.h:72:28: note: in expansion of macro 'READ_ONCE'
      72 |         typeof(*p) ___p1 = READ_ONCE(*p);                               \
         |                            ^~~~~~~~~
   include/asm-generic/barrier.h:176:29: note: in expansion of macro '__smp_load_acquire'
     176 | #define smp_load_acquire(p) __smp_load_acquire(p)
         |                             ^~~~~~~~~~~~~~~~~~
   fs/proc/task_mmu.c:190:37: note: in expansion of macro 'smp_load_acquire'
     190 |                 priv->mm_lock_seq = smp_load_acquire(&priv->mm->mm_lock_seq);
         |                                     ^~~~~~~~~~~~~~~~
   In file included from ./arch/x86/include/generated/asm/rwonce.h:1,
                    from include/linux/compiler.h:251,
                    from include/asm-generic/bug.h:5,
                    from arch/x86/include/asm/bug.h:87,
                    from include/linux/bug.h:5,
                    from include/linux/mmdebug.h:5,
                    from include/linux/mm.h:6,
                    from include/linux/pagewalk.h:5,
                    from fs/proc/task_mmu.c:2:
>> fs/proc/task_mmu.c:190:63: error: 'struct mm_struct' has no member named 'mm_lock_seq'
     190 |                 priv->mm_lock_seq = smp_load_acquire(&priv->mm->mm_lock_seq);
         |                                                               ^~
   include/asm-generic/rwonce.h:44:73: note: in definition of macro '__READ_ONCE'
      44 | #define __READ_ONCE(x)  (*(const volatile __unqual_scalar_typeof(x) *)&(x))
         |                                                                         ^
   arch/x86/include/asm/barrier.h:72:28: note: in expansion of macro 'READ_ONCE'
      72 |         typeof(*p) ___p1 = READ_ONCE(*p);                               \
         |                            ^~~~~~~~~
   include/asm-generic/barrier.h:176:29: note: in expansion of macro '__smp_load_acquire'
     176 | #define smp_load_acquire(p) __smp_load_acquire(p)
         |                             ^~~~~~~~~~~~~~~~~~
   fs/proc/task_mmu.c:190:37: note: in expansion of macro 'smp_load_acquire'
     190 |                 priv->mm_lock_seq = smp_load_acquire(&priv->mm->mm_lock_seq);
         |                                     ^~~~~~~~~~~~~~~~
   In file included from <command-line>:
>> fs/proc/task_mmu.c:190:63: error: 'struct mm_struct' has no member named 'mm_lock_seq'
     190 |                 priv->mm_lock_seq = smp_load_acquire(&priv->mm->mm_lock_seq);
         |                                                               ^~
   include/linux/compiler_types.h:415:23: note: in definition of macro '__compiletime_assert'
     415 |                 if (!(condition))                                       \
         |                       ^~~~~~~~~
   include/linux/compiler_types.h:435:9: note: in expansion of macro '_compiletime_assert'
     435 |         _compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__)
         |         ^~~~~~~~~~~~~~~~~~~
   include/linux/compiler_types.h:438:9: note: in expansion of macro 'compiletime_assert'
     438 |         compiletime_assert(__native_word(t),                            \
         |         ^~~~~~~~~~~~~~~~~~
   include/linux/compiler_types.h:438:28: note: in expansion of macro '__native_word'
     438 |         compiletime_assert(__native_word(t),                            \
         |                            ^~~~~~~~~~~~~
   arch/x86/include/asm/barrier.h:73:9: note: in expansion of macro 'compiletime_assert_atomic_type'
      73 |         compiletime_assert_atomic_type(*p);                             \
         |         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   include/asm-generic/barrier.h:176:29: note: in expansion of macro '__smp_load_acquire'
     176 | #define smp_load_acquire(p) __smp_load_acquire(p)
         |                             ^~~~~~~~~~~~~~~~~~
   fs/proc/task_mmu.c:190:37: note: in expansion of macro 'smp_load_acquire'
     190 |                 priv->mm_lock_seq = smp_load_acquire(&priv->mm->mm_lock_seq);
         |                                     ^~~~~~~~~~~~~~~~
>> fs/proc/task_mmu.c:190:63: error: 'struct mm_struct' has no member named 'mm_lock_seq'
     190 |                 priv->mm_lock_seq = smp_load_acquire(&priv->mm->mm_lock_seq);
         |                                                               ^~
   include/linux/compiler_types.h:415:23: note: in definition of macro '__compiletime_assert'
     415 |                 if (!(condition))                                       \
         |                       ^~~~~~~~~
   include/linux/compiler_types.h:435:9: note: in expansion of macro '_compiletime_assert'
     435 |         _compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__)
         |         ^~~~~~~~~~~~~~~~~~~
   include/linux/compiler_types.h:438:9: note: in expansion of macro 'compiletime_assert'
     438 |         compiletime_assert(__native_word(t),                            \
         |         ^~~~~~~~~~~~~~~~~~
   include/linux/compiler_types.h:438:28: note: in expansion of macro '__native_word'
     438 |         compiletime_assert(__native_word(t),                            \
         |                            ^~~~~~~~~~~~~
   arch/x86/include/asm/barrier.h:73:9: note: in expansion of macro 'compiletime_assert_atomic_type'
      73 |         compiletime_assert_atomic_type(*p);                             \
         |         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   include/asm-generic/barrier.h:176:29: note: in expansion of macro '__smp_load_acquire'
     176 | #define smp_load_acquire(p) __smp_load_acquire(p)
         |                             ^~~~~~~~~~~~~~~~~~
   fs/proc/task_mmu.c:190:37: note: in expansion of macro 'smp_load_acquire'
     190 |                 priv->mm_lock_seq = smp_load_acquire(&priv->mm->mm_lock_seq);
         |                                     ^~~~~~~~~~~~~~~~
>> fs/proc/task_mmu.c:190:63: error: 'struct mm_struct' has no member named 'mm_lock_seq'
     190 |                 priv->mm_lock_seq = smp_load_acquire(&priv->mm->mm_lock_seq);
         |                                                               ^~
   include/linux/compiler_types.h:415:23: note: in definition of macro '__compiletime_assert'
     415 |                 if (!(condition))                                       \
         |                       ^~~~~~~~~
   include/linux/compiler_types.h:435:9: note: in expansion of macro '_compiletime_assert'
     435 |         _compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__)
         |         ^~~~~~~~~~~~~~~~~~~
   include/linux/compiler_types.h:438:9: note: in expansion of macro 'compiletime_assert'
     438 |         compiletime_assert(__native_word(t),                            \
         |         ^~~~~~~~~~~~~~~~~~
   include/linux/compiler_types.h:438:28: note: in expansion of macro '__native_word'
     438 |         compiletime_assert(__native_word(t),                            \
         |                            ^~~~~~~~~~~~~
   arch/x86/include/asm/barrier.h:73:9: note: in expansion of macro 'compiletime_assert_atomic_type'
      73 |         compiletime_assert_atomic_type(*p);                             \
         |         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   include/asm-generic/barrier.h:176:29: note: in expansion of macro '__smp_load_acquire'
     176 | #define smp_load_acquire(p) __smp_load_acquire(p)
         |                             ^~~~~~~~~~~~~~~~~~
   fs/proc/task_mmu.c:190:37: note: in expansion of macro 'smp_load_acquire'
     190 |                 priv->mm_lock_seq = smp_load_acquire(&priv->mm->mm_lock_seq);
         |                                     ^~~~~~~~~~~~~~~~
>> fs/proc/task_mmu.c:190:63: error: 'struct mm_struct' has no member named 'mm_lock_seq'
     190 |                 priv->mm_lock_seq = smp_load_acquire(&priv->mm->mm_lock_seq);
         |                                                               ^~
   include/linux/compiler_types.h:415:23: note: in definition of macro '__compiletime_assert'
     415 |                 if (!(condition))                                       \
         |                       ^~~~~~~~~
   include/linux/compiler_types.h:435:9: note: in expansion of macro '_compiletime_assert'
     435 |         _compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__)
         |         ^~~~~~~~~~~~~~~~~~~
   include/linux/compiler_types.h:438:9: note: in expansion of macro 'compiletime_assert'
     438 |         compiletime_assert(__native_word(t),                            \
         |         ^~~~~~~~~~~~~~~~~~
   include/linux/compiler_types.h:438:28: note: in expansion of macro '__native_word'
     438 |         compiletime_assert(__native_word(t),                            \
         |                            ^~~~~~~~~~~~~
   arch/x86/include/asm/barrier.h:73:9: note: in expansion of macro 'compiletime_assert_atomic_type'
      73 |         compiletime_assert_atomic_type(*p);                             \
         |         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   include/asm-generic/barrier.h:176:29: note: in expansion of macro '__smp_load_acquire'
     176 | #define smp_load_acquire(p) __smp_load_acquire(p)
         |                             ^~~~~~~~~~~~~~~~~~
   fs/proc/task_mmu.c:190:37: note: in expansion of macro 'smp_load_acquire'
     190 |                 priv->mm_lock_seq = smp_load_acquire(&priv->mm->mm_lock_seq);
         |                                     ^~~~~~~~~~~~~~~~
   fs/proc/task_mmu.c: In function 'get_vma_fields':
>> fs/proc/task_mmu.c:387:18: error: 'struct vm_area_struct' has no member named 'anon_name'; did you mean 'anon_vma'?
     387 |         if (vma->anon_name) {
         |                  ^~~~~~~~~
         |                  anon_vma
   In file included from include/linux/list.h:11,
                    from include/linux/preempt.h:12,
                    from include/linux/spinlock.h:56,
                    from include/linux/mmzone.h:8,
                    from include/linux/gfp.h:7,
                    from include/linux/mm.h:7,
                    from include/linux/pagewalk.h:5,
                    from fs/proc/task_mmu.c:2:
   fs/proc/task_mmu.c: In function 'validate_map':
   fs/proc/task_mmu.c:437:57: error: 'struct mm_struct' has no member named 'mm_lock_seq'
     437 |                 mm_lock_seq = smp_load_acquire(&priv->mm->mm_lock_seq);
         |                                                         ^~
   arch/x86/include/asm/barrier.h:72:17: note: in definition of macro '__smp_load_acquire'
      72 |         typeof(*p) ___p1 = READ_ONCE(*p);                               \
         |                 ^
   fs/proc/task_mmu.c:437:31: note: in expansion of macro 'smp_load_acquire'
     437 |                 mm_lock_seq = smp_load_acquire(&priv->mm->mm_lock_seq);
         |                               ^~~~~~~~~~~~~~~~
   In file included from <command-line>:
   fs/proc/task_mmu.c:437:57: error: 'struct mm_struct' has no member named 'mm_lock_seq'
     437 |                 mm_lock_seq = smp_load_acquire(&priv->mm->mm_lock_seq);
         |                                                         ^~
   include/linux/compiler_types.h:415:23: note: in definition of macro '__compiletime_assert'
     415 |                 if (!(condition))                                       \
         |                       ^~~~~~~~~
   include/linux/compiler_types.h:435:9: note: in expansion of macro '_compiletime_assert'
     435 |         _compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__)
         |         ^~~~~~~~~~~~~~~~~~~
   include/asm-generic/rwonce.h:36:9: note: in expansion of macro 'compiletime_assert'
      36 |         compiletime_assert(__native_word(t) || sizeof(t) == sizeof(long long),  \
         |         ^~~~~~~~~~~~~~~~~~
   include/asm-generic/rwonce.h:36:28: note: in expansion of macro '__native_word'
      36 |         compiletime_assert(__native_word(t) || sizeof(t) == sizeof(long long),  \
         |                            ^~~~~~~~~~~~~
   include/asm-generic/rwonce.h:49:9: note: in expansion of macro 'compiletime_assert_rwonce_type'
      49 |         compiletime_assert_rwonce_type(x);                              \
         |         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   arch/x86/include/asm/barrier.h:72:28: note: in expansion of macro 'READ_ONCE'
      72 |         typeof(*p) ___p1 = READ_ONCE(*p);                               \
         |                            ^~~~~~~~~
   include/asm-generic/barrier.h:176:29: note: in expansion of macro '__smp_load_acquire'
     176 | #define smp_load_acquire(p) __smp_load_acquire(p)
         |                             ^~~~~~~~~~~~~~~~~~
   fs/proc/task_mmu.c:437:31: note: in expansion of macro 'smp_load_acquire'
     437 |                 mm_lock_seq = smp_load_acquire(&priv->mm->mm_lock_seq);
         |                               ^~~~~~~~~~~~~~~~
   fs/proc/task_mmu.c:437:57: error: 'struct mm_struct' has no member named 'mm_lock_seq'
     437 |                 mm_lock_seq = smp_load_acquire(&priv->mm->mm_lock_seq);
         |                                                         ^~
   include/linux/compiler_types.h:415:23: note: in definition of macro '__compiletime_assert'
     415 |                 if (!(condition))                                       \
         |                       ^~~~~~~~~
   include/linux/compiler_types.h:435:9: note: in expansion of macro '_compiletime_assert'
     435 |         _compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__)
         |         ^~~~~~~~~~~~~~~~~~~
   include/asm-generic/rwonce.h:36:9: note: in expansion of macro 'compiletime_assert'
      36 |         compiletime_assert(__native_word(t) || sizeof(t) == sizeof(long long),  \
         |         ^~~~~~~~~~~~~~~~~~
   include/asm-generic/rwonce.h:36:28: note: in expansion of macro '__native_word'
      36 |         compiletime_assert(__native_word(t) || sizeof(t) == sizeof(long long),  \
         |                            ^~~~~~~~~~~~~
   include/asm-generic/rwonce.h:49:9: note: in expansion of macro 'compiletime_assert_rwonce_type'
      49 |         compiletime_assert_rwonce_type(x);                              \
         |         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   arch/x86/include/asm/barrier.h:72:28: note: in expansion of macro 'READ_ONCE'
      72 |         typeof(*p) ___p1 = READ_ONCE(*p);                               \
         |                            ^~~~~~~~~
   include/asm-generic/barrier.h:176:29: note: in expansion of macro '__smp_load_acquire'
     176 | #define smp_load_acquire(p) __smp_load_acquire(p)
         |                             ^~~~~~~~~~~~~~~~~~
   fs/proc/task_mmu.c:437:31: note: in expansion of macro 'smp_load_acquire'
     437 |                 mm_lock_seq = smp_load_acquire(&priv->mm->mm_lock_seq);
         |                               ^~~~~~~~~~~~~~~~
   fs/proc/task_mmu.c:437:57: error: 'struct mm_struct' has no member named 'mm_lock_seq'
     437 |                 mm_lock_seq = smp_load_acquire(&priv->mm->mm_lock_seq);
         |                                                         ^~
   include/linux/compiler_types.h:415:23: note: in definition of macro '__compiletime_assert'
     415 |                 if (!(condition))                                       \
         |                       ^~~~~~~~~
   include/linux/compiler_types.h:435:9: note: in expansion of macro '_compiletime_assert'
     435 |         _compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__)
         |         ^~~~~~~~~~~~~~~~~~~
   include/asm-generic/rwonce.h:36:9: note: in expansion of macro 'compiletime_assert'
      36 |         compiletime_assert(__native_word(t) || sizeof(t) == sizeof(long long),  \
         |         ^~~~~~~~~~~~~~~~~~
   include/asm-generic/rwonce.h:36:28: note: in expansion of macro '__native_word'
      36 |         compiletime_assert(__native_word(t) || sizeof(t) == sizeof(long long),  \
         |                            ^~~~~~~~~~~~~
   include/asm-generic/rwonce.h:49:9: note: in expansion of macro 'compiletime_assert_rwonce_type'
      49 |         compiletime_assert_rwonce_type(x);                              \
         |         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   arch/x86/include/asm/barrier.h:72:28: note: in expansion of macro 'READ_ONCE'
      72 |         typeof(*p) ___p1 = READ_ONCE(*p);                               \
         |                            ^~~~~~~~~
   include/asm-generic/barrier.h:176:29: note: in expansion of macro '__smp_load_acquire'
     176 | #define smp_load_acquire(p) __smp_load_acquire(p)
         |                             ^~~~~~~~~~~~~~~~~~

vim +190 fs/proc/task_mmu.c

   > 2	#include <linux/pagewalk.h>
     3	#include <linux/mm_inline.h>
     4	#include <linux/hugetlb.h>
     5	#include <linux/huge_mm.h>
     6	#include <linux/mount.h>
     7	#include <linux/ksm.h>
     8	#include <linux/seq_file.h>
     9	#include <linux/highmem.h>
    10	#include <linux/ptrace.h>
    11	#include <linux/slab.h>
    12	#include <linux/pagemap.h>
    13	#include <linux/mempolicy.h>
    14	#include <linux/rmap.h>
    15	#include <linux/swap.h>
    16	#include <linux/sched/mm.h>
    17	#include <linux/swapops.h>
    18	#include <linux/mmu_notifier.h>
    19	#include <linux/page_idle.h>
    20	#include <linux/shmem_fs.h>
    21	#include <linux/uaccess.h>
    22	#include <linux/pkeys.h>
    23	#include <linux/minmax.h>
    24	#include <linux/overflow.h>
    25	
    26	#include <asm/elf.h>
    27	#include <asm/tlb.h>
    28	#include <asm/tlbflush.h>
    29	#include "internal.h"
    30	
    31	#define SEQ_PUT_DEC(str, val) \
    32			seq_put_decimal_ull_width(m, str, (val) << (PAGE_SHIFT-10), 8)
    33	void task_mem(struct seq_file *m, struct mm_struct *mm)
    34	{
    35		unsigned long text, lib, swap, anon, file, shmem;
    36		unsigned long hiwater_vm, total_vm, hiwater_rss, total_rss;
    37	
    38		anon = get_mm_counter(mm, MM_ANONPAGES);
    39		file = get_mm_counter(mm, MM_FILEPAGES);
    40		shmem = get_mm_counter(mm, MM_SHMEMPAGES);
    41	
    42		/*
    43		 * Note: to minimize their overhead, mm maintains hiwater_vm and
    44		 * hiwater_rss only when about to *lower* total_vm or rss.  Any
    45		 * collector of these hiwater stats must therefore get total_vm
    46		 * and rss too, which will usually be the higher.  Barriers? not
    47		 * worth the effort, such snapshots can always be inconsistent.
    48		 */
    49		hiwater_vm = total_vm = mm->total_vm;
    50		if (hiwater_vm < mm->hiwater_vm)
    51			hiwater_vm = mm->hiwater_vm;
    52		hiwater_rss = total_rss = anon + file + shmem;
    53		if (hiwater_rss < mm->hiwater_rss)
    54			hiwater_rss = mm->hiwater_rss;
    55	
    56		/* split executable areas between text and lib */
    57		text = PAGE_ALIGN(mm->end_code) - (mm->start_code & PAGE_MASK);
    58		text = min(text, mm->exec_vm << PAGE_SHIFT);
    59		lib = (mm->exec_vm << PAGE_SHIFT) - text;
    60	
    61		swap = get_mm_counter(mm, MM_SWAPENTS);
    62		SEQ_PUT_DEC("VmPeak:\t", hiwater_vm);
    63		SEQ_PUT_DEC(" kB\nVmSize:\t", total_vm);
    64		SEQ_PUT_DEC(" kB\nVmLck:\t", mm->locked_vm);
    65		SEQ_PUT_DEC(" kB\nVmPin:\t", atomic64_read(&mm->pinned_vm));
    66		SEQ_PUT_DEC(" kB\nVmHWM:\t", hiwater_rss);
    67		SEQ_PUT_DEC(" kB\nVmRSS:\t", total_rss);
    68		SEQ_PUT_DEC(" kB\nRssAnon:\t", anon);
    69		SEQ_PUT_DEC(" kB\nRssFile:\t", file);
    70		SEQ_PUT_DEC(" kB\nRssShmem:\t", shmem);
    71		SEQ_PUT_DEC(" kB\nVmData:\t", mm->data_vm);
    72		SEQ_PUT_DEC(" kB\nVmStk:\t", mm->stack_vm);
    73		seq_put_decimal_ull_width(m,
    74			    " kB\nVmExe:\t", text >> 10, 8);
    75		seq_put_decimal_ull_width(m,
    76			    " kB\nVmLib:\t", lib >> 10, 8);
    77		seq_put_decimal_ull_width(m,
    78			    " kB\nVmPTE:\t", mm_pgtables_bytes(mm) >> 10, 8);
    79		SEQ_PUT_DEC(" kB\nVmSwap:\t", swap);
    80		seq_puts(m, " kB\n");
    81		hugetlb_report_usage(m, mm);
    82	}
    83	#undef SEQ_PUT_DEC
    84	
    85	unsigned long task_vsize(struct mm_struct *mm)
    86	{
    87		return PAGE_SIZE * mm->total_vm;
    88	}
    89	
    90	unsigned long task_statm(struct mm_struct *mm,
    91				 unsigned long *shared, unsigned long *text,
    92				 unsigned long *data, unsigned long *resident)
    93	{
    94		*shared = get_mm_counter(mm, MM_FILEPAGES) +
    95				get_mm_counter(mm, MM_SHMEMPAGES);
    96		*text = (PAGE_ALIGN(mm->end_code) - (mm->start_code & PAGE_MASK))
    97									>> PAGE_SHIFT;
    98		*data = mm->data_vm + mm->stack_vm;
    99		*resident = *shared + get_mm_counter(mm, MM_ANONPAGES);
   100		return mm->total_vm;
   101	}
   102	
   103	#ifdef CONFIG_NUMA
   104	/*
   105	 * Save get_task_policy() for show_numa_map().
   106	 */
   107	static void hold_task_mempolicy(struct proc_maps_private *priv)
   108	{
   109		struct task_struct *task = priv->task;
   110	
   111		task_lock(task);
   112		priv->task_mempolicy = get_task_policy(task);
   113		mpol_get(priv->task_mempolicy);
   114		task_unlock(task);
   115	}
   116	static void release_task_mempolicy(struct proc_maps_private *priv)
   117	{
   118		mpol_put(priv->task_mempolicy);
   119	}
   120	#else
   121	static void hold_task_mempolicy(struct proc_maps_private *priv)
   122	{
   123	}
   124	static void release_task_mempolicy(struct proc_maps_private *priv)
   125	{
   126	}
   127	#endif
   128	
   129	static struct vm_area_struct *proc_get_vma(struct proc_maps_private *priv,
   130							loff_t *ppos)
   131	{
   132		struct vm_area_struct *vma = vma_next(&priv->iter);
   133	
   134		if (vma) {
   135			*ppos = vma->vm_start;
   136		} else {
   137			*ppos = -2UL;
   138			vma = get_gate_vma(priv->mm);
   139		}
   140	
   141		return vma;
   142	}
   143	
   144	static const struct seq_operations proc_pid_maps_op;
   145	
   146	static inline bool needs_mmap_lock(struct seq_file *m)
   147	{
   148	#ifdef CONFIG_PER_VMA_LOCK
   149		/*
   150		 * smaps and numa_maps perform page table walk, therefore require
   151		 * mmap_lock but maps can be read under RCU.
   152		 */
   153		return m->op != &proc_pid_maps_op;
   154	#else
   155		/* Without per-vma locks VMA access is not RCU-safe */
   156		return true;
   157	#endif
   158	}
   159	
   160	static void *m_start(struct seq_file *m, loff_t *ppos)
   161	{
   162		struct proc_maps_private *priv = m->private;
   163		unsigned long last_addr = *ppos;
   164		struct mm_struct *mm;
   165	
   166		/* See m_next(). Zero at the start or after lseek. */
   167		if (last_addr == -1UL)
   168			return NULL;
   169	
   170		priv->task = get_proc_task(priv->inode);
   171		if (!priv->task)
   172			return ERR_PTR(-ESRCH);
   173	
   174		mm = priv->mm;
   175		if (!mm || !mmget_not_zero(mm)) {
   176			put_task_struct(priv->task);
   177			priv->task = NULL;
   178			return NULL;
   179		}
   180	
   181		if (needs_mmap_lock(m)) {
   182			if (mmap_read_lock_killable(mm)) {
   183				mmput(mm);
   184				put_task_struct(priv->task);
   185				priv->task = NULL;
   186				return ERR_PTR(-EINTR);
   187			}
   188		} else {
   189			/* For memory barrier see the comment for mm_lock_seq in mm_struct */
 > 190			priv->mm_lock_seq = smp_load_acquire(&priv->mm->mm_lock_seq);
   191			rcu_read_lock();
   192		}
   193	
   194		vma_iter_init(&priv->iter, mm, last_addr);
   195		hold_task_mempolicy(priv);
   196		if (last_addr == -2UL)
   197			return get_gate_vma(mm);
   198	
   199		return proc_get_vma(priv, ppos);
   200	}
   201	
   202	static void *m_next(struct seq_file *m, void *v, loff_t *ppos)
   203	{
   204		if (*ppos == -2UL) {
   205			*ppos = -1UL;
   206			return NULL;
   207		}
   208		return proc_get_vma(m->private, ppos);
   209	}
   210	
   211	static void m_stop(struct seq_file *m, void *v)
   212	{
   213		struct proc_maps_private *priv = m->private;
   214		struct mm_struct *mm = priv->mm;
   215	
   216		if (!priv->task)
   217			return;
   218	
   219		release_task_mempolicy(priv);
   220		if (needs_mmap_lock(m))
   221			mmap_read_unlock(mm);
   222		else
   223			rcu_read_unlock();
   224		mmput(mm);
   225		put_task_struct(priv->task);
   226		priv->task = NULL;
   227	}
   228	
   229	static int proc_maps_open(struct inode *inode, struct file *file,
   230				const struct seq_operations *ops, int psize)
   231	{
   232		struct proc_maps_private *priv = __seq_open_private(file, ops, psize);
   233	
   234		if (!priv)
   235			return -ENOMEM;
   236	
   237		priv->inode = inode;
   238		priv->mm = proc_mem_open(inode, PTRACE_MODE_READ);
   239		if (IS_ERR(priv->mm)) {
   240			int err = PTR_ERR(priv->mm);
   241	
   242			seq_release_private(inode, file);
   243			return err;
   244		}
   245	
   246		return 0;
   247	}
   248	
   249	static int proc_map_release(struct inode *inode, struct file *file)
   250	{
   251		struct seq_file *seq = file->private_data;
   252		struct proc_maps_private *priv = seq->private;
   253	
   254		if (priv->mm)
   255			mmdrop(priv->mm);
   256	
   257		return seq_release_private(inode, file);
   258	}
   259	
   260	static int do_maps_open(struct inode *inode, struct file *file,
   261				const struct seq_operations *ops)
   262	{
   263		return proc_maps_open(inode, file, ops,
   264					sizeof(struct proc_maps_private));
   265	}
   266	
   267	static void show_vma_header_prefix(struct seq_file *m,
   268					   unsigned long start, unsigned long end,
   269					   vm_flags_t flags, unsigned long long pgoff,
   270					   dev_t dev, unsigned long ino)
   271	{
   272		seq_setwidth(m, 25 + sizeof(void *) * 6 - 1);
   273		seq_put_hex_ll(m, NULL, start, 8);
   274		seq_put_hex_ll(m, "-", end, 8);
   275		seq_putc(m, ' ');
   276		seq_putc(m, flags & VM_READ ? 'r' : '-');
   277		seq_putc(m, flags & VM_WRITE ? 'w' : '-');
   278		seq_putc(m, flags & VM_EXEC ? 'x' : '-');
   279		seq_putc(m, flags & VM_MAYSHARE ? 's' : 'p');
   280		seq_put_hex_ll(m, " ", pgoff, 8);
   281		seq_put_hex_ll(m, " ", MAJOR(dev), 2);
   282		seq_put_hex_ll(m, ":", MINOR(dev), 2);
   283		seq_put_decimal_ull(m, " ", ino);
   284		seq_putc(m, ' ');
   285	}
   286	
   287	static void
   288	show_map_vma(struct seq_file *m, struct vm_area_struct *vma)
   289	{
   290		struct anon_vma_name *anon_name = NULL;
   291		struct mm_struct *mm = vma->vm_mm;
   292		struct file *file = vma->vm_file;
   293		vm_flags_t flags = vma->vm_flags;
   294		unsigned long ino = 0;
   295		unsigned long long pgoff = 0;
   296		unsigned long start, end;
   297		dev_t dev = 0;
   298		const char *name = NULL;
   299	
   300		if (file) {
   301			const struct inode *inode = file_user_inode(vma->vm_file);
   302	
   303			dev = inode->i_sb->s_dev;
   304			ino = inode->i_ino;
   305			pgoff = ((loff_t)vma->vm_pgoff) << PAGE_SHIFT;
   306		}
   307	
   308		start = vma->vm_start;
   309		end = vma->vm_end;
   310		show_vma_header_prefix(m, start, end, flags, pgoff, dev, ino);
   311		if (mm) {
   312			anon_name = needs_mmap_lock(m) ? anon_vma_name(vma) :
   313					anon_vma_name_get_rcu(vma);
   314		}
   315	
   316		/*
   317		 * Print the dentry name for named mappings, and a
   318		 * special [heap] marker for the heap:
   319		 */
   320		if (file) {
   321			seq_pad(m, ' ');
   322			/*
   323			 * If user named this anon shared memory via
   324			 * prctl(PR_SET_VMA ..., use the provided name.
   325			 */
   326			if (anon_name)
   327				seq_printf(m, "[anon_shmem:%s]", anon_name->name);
   328			else
   329				seq_path(m, file_user_path(file), "\n");
   330			goto done;
   331		}
   332	
   333		if (vma->vm_ops && vma->vm_ops->name) {
   334			name = vma->vm_ops->name(vma);
   335			if (name)
   336				goto done;
   337		}
   338	
   339		name = arch_vma_name(vma);
   340		if (!name) {
   341			if (!mm) {
   342				name = "[vdso]";
   343				goto done;
   344			}
   345	
   346			if (vma_is_initial_heap(vma)) {
   347				name = "[heap]";
   348				goto done;
   349			}
   350	
   351			if (vma_is_initial_stack(vma)) {
   352				name = "[stack]";
   353				goto done;
   354			}
   355	
   356			if (anon_name) {
   357				seq_pad(m, ' ');
   358				seq_printf(m, "[anon:%s]", anon_name->name);
   359			}
   360		}
   361	
   362	done:
   363		if (name) {
   364			seq_pad(m, ' ');
   365			seq_puts(m, name);
   366		}
   367		seq_putc(m, '\n');
   368		if (anon_name && !needs_mmap_lock(m))
   369			anon_vma_name_put(anon_name);
   370	}
   371	
   372	/*
   373	 * Pin vm_area_struct fields used by show_map_vma. We also copy pinned fields
   374	 * into proc_maps_private because by the time put_vma_fields() is called, VMA
   375	 * might have changed and these fields might be pointing to different objects.
   376	 */
   377	static bool get_vma_fields(struct vm_area_struct *vma, struct proc_maps_private *priv)
   378	{
   379		if (vma->vm_file) {
   380			priv->vm_file =  get_file_rcu(&vma->vm_file);
   381			if (!priv->vm_file)
   382				return false;
   383	
   384		} else
   385			priv->vm_file = NULL;
   386	
 > 387		if (vma->anon_name) {

388 priv->anon_name = anon_vma_name_get_rcu(vma);
389 if (!priv->anon_name) {
390 if (priv->vm_file) {
391 fput(priv->vm_file);
392 return false;
393 }
394 }
395 } else
396 priv->anon_name = NULL;
397
398 return true;
399 }
400

Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
#5 lore raw
In reply to Suren Baghdasaryan

On 1/15/24 19:38, Suren Baghdasaryan wrote:

Hi,

Quoted 11 lines

The issue this patchset is trying to address is mmap_lock contention when a low priority task (monitoring, data collecting, etc.) blocks a higher priority task from making updated to the address space. The contention is due to the mmap_lock being held for read when reading proc/pid/maps. With maple_tree introduction, VMA tree traversals are RCU-safe and per-vma locks make VMA access RCU-safe. this provides an opportunity for lock-less reading of proc/pid/maps. We still need to overcome a couple obstacles:
1. Make all VMA pointer fields used for proc/pid/maps content generation RCU-safe;
2. Ensure that proc/pid/maps data tearing, which is currently possible at page boundaries only, does not get worse.

Hm I thought we were to only choose this more complicated in case additional tearing becomes a problem, and at first assume that if software can deal with page boundary tearing, it can deal with sub-page tearing too?

Quoted 4 lines

The patchset deals with these issues but there is a downside which I would like to get input on:
This change introduces unfairness towards the reader of proc/pid/maps, which can be blocked by an overly active/malicious address space modifyer.

So this is a consequence of the validate() operation, right? We could avoid this if we allowed sub-page tearing.

Quoted 31 lines

A couple of ways I though we can address this issue are:
1. After several lock-less retries (or some time limit) to fall back to taking mmap_lock.
2. Employ lock-less reading only if the reader has low priority, indicating that blocking it is not critical.
3. Introducing a separate procfs file which publishes the same data in lock-less manner.

I imagine a combination of these approaches can also be employed. I would like to get feedback on this from the Linux community.

Note: mmap_read_lock/mmap_read_unlock sequence inside validate_map() can be replaced with more efficiend rwsem_wait() proposed by Matthew in [1].

[1] https://lore.kernel.org/all/[email protected]/

Suren Baghdasaryan (3):
mm: make vm_area_struct anon_name field RCU-safe
seq_file: add validate() operation to seq_operations mm/maps: read proc/pid/maps under RCU

 fs/proc/internal.h        |   3 +
 fs/proc/task_mmu.c        | 130 ++++++++++++++++++++++++++++++++++----
 fs/seq_file.c             |  24 ++++++-
 include/linux/mm_inline.h |  10 ++-
 include/linux/mm_types.h  |   3 +-
 include/linux/seq_file.h  |   1 +
 mm/madvise.c              |  30 +++++++--
 7 files changed, 181 insertions(+), 20 deletions(-)
#6 lore raw
Quoted 20 lines · Vlastimil Babka

On 1/15/24 19:38, Suren Baghdasaryan wrote:

Hi,

Quoted 11 lines

The issue this patchset is trying to address is mmap_lock contention when a low priority task (monitoring, data collecting, etc.) blocks a higher priority task from making updated to the address space. The contention is due to the mmap_lock being held for read when reading proc/pid/maps. With maple_tree introduction, VMA tree traversals are RCU-safe and per-vma locks make VMA access RCU-safe. this provides an opportunity for lock-less reading of proc/pid/maps. We still need to overcome a couple obstacles:
1. Make all VMA pointer fields used for proc/pid/maps content generation RCU-safe;
2. Ensure that proc/pid/maps data tearing, which is currently possible at page boundaries only, does not get worse.

Hm I thought we were to only choose this more complicated in case additional tearing becomes a problem, and at first assume that if software can deal with page boundary tearing, it can deal with sub-page tearing too?

Quoted 15 lines
Quoted 4 lines

The patchset deals with these issues but there is a downside which I would like to get input on:
This change introduces unfairness towards the reader of proc/pid/maps, which can be blocked by an overly active/malicious address space modifyer.

So this is a consequence of the validate() operation, right? We could avoid this if we allowed sub-page tearing.

Quoted 7 lines

A couple of ways I though we can address this issue are:
1. After several lock-less retries (or some time limit) to fall back to taking mmap_lock.
2. Employ lock-less reading only if the reader has low priority, indicating that blocking it is not critical.
3. Introducing a separate procfs file which publishes the same data in lock-less manner.

Oh and if this option 3 becomes necessary, then such new file shouldn't validate() either, and whoever wants to avoid the reader contention and converts their monitoring to the new file will have to account for this possible extra tearing from the start. So I would suggest trying to change the existing file with no validate() first, and if existing userspace gets broken, employ option 3. This would mean no validate() in either case?

Quoted 24 lines
Quoted 23 lines

I imagine a combination of these approaches can also be employed. I would like to get feedback on this from the Linux community.

Note: mmap_read_lock/mmap_read_unlock sequence inside validate_map() can be replaced with more efficiend rwsem_wait() proposed by Matthew in [1].

[1] https://lore.kernel.org/all/[email protected]/

Suren Baghdasaryan (3):
mm: make vm_area_struct anon_name field RCU-safe
seq_file: add validate() operation to seq_operations mm/maps: read proc/pid/maps under RCU

 fs/proc/internal.h        |   3 +
 fs/proc/task_mmu.c        | 130 ++++++++++++++++++++++++++++++++++----
 fs/seq_file.c             |  24 ++++++-
 include/linux/mm_inline.h |  10 ++-
 include/linux/mm_types.h  |   3 +-
 include/linux/seq_file.h  |   1 +
 mm/madvise.c              |  30 +++++++--
 7 files changed, 181 insertions(+), 20 deletions(-)
author reviewer
#7 lore raw
Quoted 18 lines · Vlastimil Babka
Quoted 16 lines · Vlastimil Babka

On 1/15/24 19:38, Suren Baghdasaryan wrote:

Hi,

Quoted 11 lines

The issue this patchset is trying to address is mmap_lock contention when a low priority task (monitoring, data collecting, etc.) blocks a higher priority task from making updated to the address space. The contention is due to the mmap_lock being held for read when reading proc/pid/maps. With maple_tree introduction, VMA tree traversals are RCU-safe and per-vma locks make VMA access RCU-safe. this provides an opportunity for lock-less reading of proc/pid/maps. We still need to overcome a couple obstacles:
1. Make all VMA pointer fields used for proc/pid/maps content generation RCU-safe;
2. Ensure that proc/pid/maps data tearing, which is currently possible at page boundaries only, does not get worse.

Quoted 3 lines
Quoted 3 lines

Hm I thought we were to only choose this more complicated in case additional tearing becomes a problem, and at first assume that if software can deal with page boundary tearing, it can deal with sub-page tearing too?

Hi Vlastimil,
Thanks for the feedback!
Yes, originally I thought we wouldn't be able to avoid additional tearing without a big change but then realized it's not that hard, so I tried to keep the change in behavior transparent to the userspace.

Quoted 8 lines
Quoted 8 lines
Quoted 5 lines

The patchset deals with these issues but there is a downside which I would like to get input on:
This change introduces unfairness towards the reader of proc/pid/maps, which can be blocked by an overly active/malicious address space modifyer.

So this is a consequence of the validate() operation, right? We could avoid this if we allowed sub-page tearing.

Yes, if we don't care about sub-page tearing then we could get rid of validate step and this issue with updaters blocking the reader would go away. If we choose that direction there will be one more issue to fix, namely the maple_tree temporary inconsistent state when a VMA is replaced with another one and we might observe NULL there. We might be able to use Matthew's rwsem_wait() to deal with that issue.

Quoted 9 lines
Quoted 8 lines
Quoted 8 lines

A couple of ways I though we can address this issue are:
1. After several lock-less retries (or some time limit) to fall back to taking mmap_lock.
2. Employ lock-less reading only if the reader has low priority, indicating that blocking it is not critical.
3. Introducing a separate procfs file which publishes the same data in lock-less manner.

Quoted 6 lines

Oh and if this option 3 becomes necessary, then such new file shouldn't validate() either, and whoever wants to avoid the reader contention and converts their monitoring to the new file will have to account for this possible extra tearing from the start. So I would suggest trying to change the existing file with no validate() first, and if existing userspace gets broken, employ option 3. This would mean no validate() in either case?

Yes but I was trying to avoid introducing additional file which publishes the same content in a slightly different way. We will have to explain when userspace should use one vs the other and that would require going into low level implementation details, I think. Don't know if that's acceptable/preferable.
Thanks,
Suren.

Quoted 26 lines
Quoted 25 lines
Quoted 24 lines

I imagine a combination of these approaches can also be employed. I would like to get feedback on this from the Linux community.

Note: mmap_read_lock/mmap_read_unlock sequence inside validate_map() can be replaced with more efficiend rwsem_wait() proposed by Matthew in [1].

[1] https://lore.kernel.org/all/[email protected]/

Suren Baghdasaryan (3):
mm: make vm_area_struct anon_name field RCU-safe
seq_file: add validate() operation to seq_operations mm/maps: read proc/pid/maps under RCU

 fs/proc/internal.h        |   3 +
 fs/proc/task_mmu.c        | 130 ++++++++++++++++++++++++++++++++++----
 fs/seq_file.c             |  24 ++++++-
 include/linux/mm_inline.h |  10 ++-
 include/linux/mm_types.h  |   3 +-
 include/linux/seq_file.h  |   1 +
 mm/madvise.c              |  30 +++++++--
 7 files changed, 181 insertions(+), 20 deletions(-)
author reviewer
#8 lore raw
Quoted 24 lines · Suren Baghdasaryan
Quoted 18 lines · Vlastimil Babka
Quoted 16 lines · Vlastimil Babka

On 1/15/24 19:38, Suren Baghdasaryan wrote:

Hi,

Quoted 11 lines

The issue this patchset is trying to address is mmap_lock contention when a low priority task (monitoring, data collecting, etc.) blocks a higher priority task from making updated to the address space. The contention is due to the mmap_lock being held for read when reading proc/pid/maps. With maple_tree introduction, VMA tree traversals are RCU-safe and per-vma locks make VMA access RCU-safe. this provides an opportunity for lock-less reading of proc/pid/maps. We still need to overcome a couple obstacles:
1. Make all VMA pointer fields used for proc/pid/maps content generation RCU-safe;
2. Ensure that proc/pid/maps data tearing, which is currently possible at page boundaries only, does not get worse.

Quoted 3 lines
Quoted 3 lines

Hm I thought we were to only choose this more complicated in case additional tearing becomes a problem, and at first assume that if software can deal with page boundary tearing, it can deal with sub-page tearing too?

Quoted 5 lines

Hi Vlastimil,
Thanks for the feedback!
Yes, originally I thought we wouldn't be able to avoid additional tearing without a big change but then realized it's not that hard, so I tried to keep the change in behavior transparent to the userspace.

In the absence of other feedback I'm going to implement and post the originally envisioned approach: remove validation step and avoid any possibility of blocking but allowing for sub-page tearing. Will use Matthew's rwsem_wait() to deal with possible inconsistent maple_tree state.
Thanks,
Suren.

Quoted 67 lines
Quoted 9 lines
Quoted 9 lines
Quoted 6 lines

The patchset deals with these issues but there is a downside which I would like to get input on:
This change introduces unfairness towards the reader of proc/pid/maps, which can be blocked by an overly active/malicious address space modifyer.

So this is a consequence of the validate() operation, right? We could avoid this if we allowed sub-page tearing.

Yes, if we don't care about sub-page tearing then we could get rid of validate step and this issue with updaters blocking the reader would go away. If we choose that direction there will be one more issue to fix, namely the maple_tree temporary inconsistent state when a VMA is replaced with another one and we might observe NULL there. We might be able to use Matthew's rwsem_wait() to deal with that issue.

Quoted 9 lines
Quoted 8 lines
Quoted 8 lines

A couple of ways I though we can address this issue are:
1. After several lock-less retries (or some time limit) to fall back to taking mmap_lock.
2. Employ lock-less reading only if the reader has low priority, indicating that blocking it is not critical.
3. Introducing a separate procfs file which publishes the same data in lock-less manner.

Quoted 6 lines

Oh and if this option 3 becomes necessary, then such new file shouldn't validate() either, and whoever wants to avoid the reader contention and converts their monitoring to the new file will have to account for this possible extra tearing from the start. So I would suggest trying to change the existing file with no validate() first, and if existing userspace gets broken, employ option 3. This would mean no validate() in either case?

Yes but I was trying to avoid introducing additional file which publishes the same content in a slightly different way. We will have to explain when userspace should use one vs the other and that would require going into low level implementation details, I think. Don't know if that's acceptable/preferable.
Thanks,
Suren.

Quoted 26 lines
Quoted 25 lines
Quoted 24 lines

I imagine a combination of these approaches can also be employed. I would like to get feedback on this from the Linux community.

Note: mmap_read_lock/mmap_read_unlock sequence inside validate_map() can be replaced with more efficiend rwsem_wait() proposed by Matthew in [1].

[1] https://lore.kernel.org/all/[email protected]/

Suren Baghdasaryan (3):
mm: make vm_area_struct anon_name field RCU-safe
seq_file: add validate() operation to seq_operations mm/maps: read proc/pid/maps under RCU

 fs/proc/internal.h        |   3 +
 fs/proc/task_mmu.c        | 130 ++++++++++++++++++++++++++++++++++----
 fs/seq_file.c             |  24 ++++++-
 include/linux/mm_inline.h |  10 ++-
 include/linux/mm_types.h  |   3 +-
 include/linux/seq_file.h  |   1 +
 mm/madvise.c              |  30 +++++++--
 7 files changed, 181 insertions(+), 20 deletions(-)
author reviewer
#9 lore raw
Quoted 32 lines · Suren Baghdasaryan
Quoted 24 lines · Suren Baghdasaryan
Quoted 18 lines · Vlastimil Babka
Quoted 16 lines · Vlastimil Babka

On 1/15/24 19:38, Suren Baghdasaryan wrote:

Hi,

Quoted 11 lines

The issue this patchset is trying to address is mmap_lock contention when a low priority task (monitoring, data collecting, etc.) blocks a higher priority task from making updated to the address space. The contention is due to the mmap_lock being held for read when reading proc/pid/maps. With maple_tree introduction, VMA tree traversals are RCU-safe and per-vma locks make VMA access RCU-safe. this provides an opportunity for lock-less reading of proc/pid/maps. We still need to overcome a couple obstacles:
1. Make all VMA pointer fields used for proc/pid/maps content generation RCU-safe;
2. Ensure that proc/pid/maps data tearing, which is currently possible at page boundaries only, does not get worse.

Quoted 3 lines
Quoted 3 lines

Hm I thought we were to only choose this more complicated in case additional tearing becomes a problem, and at first assume that if software can deal with page boundary tearing, it can deal with sub-page tearing too?

Quoted 5 lines

Hi Vlastimil,
Thanks for the feedback!
Yes, originally I thought we wouldn't be able to avoid additional tearing without a big change but then realized it's not that hard, so I tried to keep the change in behavior transparent to the userspace.

Quoted 5 lines

In the absence of other feedback I'm going to implement and post the originally envisioned approach: remove validation step and avoid any possibility of blocking but allowing for sub-page tearing. Will use Matthew's rwsem_wait() to deal with possible inconsistent maple_tree state.

I posted v1 at
https://lore.kernel.org/all/[email protected]/
In the RFC I used mm_struct.mm_lock_seq to detect if mm is being changed but then I realized that won't work. mm_struct.mm_lock_seq is incremented after mm is changed and right before mmap_lock is write-unlocked. Instead I need a counter that changes once we write-lock mmap_lock and before any mm changes. So the new patchset introduces a separate counter to detect possible mm changes. In addition, I could not use rwsem_wait() and instead had to take mmap_lock for read to wait for the writer to finish and then record the new counter while holding mmap_lock for read. That prevents concurrent mm changes while we are recording the new counter value.

Quoted 70 lines

Thanks,
Suren.

Quoted 67 lines
Quoted 9 lines
Quoted 9 lines
Quoted 6 lines

The patchset deals with these issues but there is a downside which I would like to get input on:
This change introduces unfairness towards the reader of proc/pid/maps, which can be blocked by an overly active/malicious address space modifyer.

So this is a consequence of the validate() operation, right? We could avoid this if we allowed sub-page tearing.

Yes, if we don't care about sub-page tearing then we could get rid of validate step and this issue with updaters blocking the reader would go away. If we choose that direction there will be one more issue to fix, namely the maple_tree temporary inconsistent state when a VMA is replaced with another one and we might observe NULL there. We might be able to use Matthew's rwsem_wait() to deal with that issue.

Quoted 9 lines
Quoted 8 lines
Quoted 8 lines

A couple of ways I though we can address this issue are:
1. After several lock-less retries (or some time limit) to fall back to taking mmap_lock.
2. Employ lock-less reading only if the reader has low priority, indicating that blocking it is not critical.
3. Introducing a separate procfs file which publishes the same data in lock-less manner.

Quoted 6 lines

Oh and if this option 3 becomes necessary, then such new file shouldn't validate() either, and whoever wants to avoid the reader contention and converts their monitoring to the new file will have to account for this possible extra tearing from the start. So I would suggest trying to change the existing file with no validate() first, and if existing userspace gets broken, employ option 3. This would mean no validate() in either case?

Yes but I was trying to avoid introducing additional file which publishes the same content in a slightly different way. We will have to explain when userspace should use one vs the other and that would require going into low level implementation details, I think. Don't know if that's acceptable/preferable.
Thanks,
Suren.

Quoted 26 lines
Quoted 25 lines
Quoted 24 lines

I imagine a combination of these approaches can also be employed. I would like to get feedback on this from the Linux community.

Note: mmap_read_lock/mmap_read_unlock sequence inside validate_map() can be replaced with more efficiend rwsem_wait() proposed by Matthew in [1].

[1] https://lore.kernel.org/all/[email protected]/

Suren Baghdasaryan (3):
mm: make vm_area_struct anon_name field RCU-safe
seq_file: add validate() operation to seq_operations mm/maps: read proc/pid/maps under RCU

 fs/proc/internal.h        |   3 +
 fs/proc/task_mmu.c        | 130 ++++++++++++++++++++++++++++++++++----
 fs/seq_file.c             |  24 ++++++-
 include/linux/mm_inline.h |  10 ++-
 include/linux/mm_types.h  |   3 +-
 include/linux/seq_file.h  |   1 +
 mm/madvise.c              |  30 +++++++--
 7 files changed, 181 insertions(+), 20 deletions(-)