Skip to contentSkip to conversation

proc-report-open-files-as-size-in-stat-for-proc-pid-fd.patch removed from -mm tree

merged mm-nonmm-stable Andrew Morton 1 participant 1 message Nov 15, 2022

lore
Share & tools
lore

Andrew Morton

author tree notice Applied
#0 lore raw

The quilt patch titled
Subject: proc: report open files as size in stat() for /proc/pid/fd
has been removed from the -mm tree. Its filename was
proc-report-open-files-as-size-in-stat-for-proc-pid-fd.patch

This patch was dropped because it was merged into the mm-nonmm-stable branch of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm

------------------------------------------------------
From: Ivan Babrou <[email protected]>
Subject: proc: report open files as size in stat() for /proc/pid/fd
Date: Thu, 22 Sep 2022 15:40:26 -0700

Many monitoring tools include open file count as a metric. Currently the only way to get this number is to enumerate the files in /proc/pid/fd.

The problem with the current approach is that it does many things people generally don't care about when they need one number for a metric. In our tests for cadvisor, which reports open file counts per cgroup, we observed that reading the number of open files is slow. Out of 35.23% of CPU time spent in proc_readfd_common, we see 29.43% spent in proc_fill_cache, which is responsible for filling dentry info. Some of this extra time is spinlock contention, but it's a contention for the lock we don't want to take to begin with.

We considered putting the number of open files in /proc/pid/status. Unfortunately, counting the number of fds involves iterating the open_files bitmap, which has a linear complexity in proportion with the number of open files (bitmap slots really, but it's close). We don't want to make /proc/pid/status any slower, so instead we put this info in /proc/pid/fd as a size member of the stat syscall result. Previously the reported number was zero, so there's very little risk of breaking anything, while still providing a somewhat logical way to count the open files with a fallback if it's zero.

RFC for this patch included iterating open fds under RCU. Thanks to Frank Hofmann for the suggestion to use the bitmap instead.

Previously:

```
$ sudo stat /proc/1/fd | head -n2
  File: /proc/1/fd
  Size: 0         	Blocks: 0          IO Block: 1024   directory
```

With this patch:

```
$ sudo stat /proc/1/fd | head -n2
  File: /proc/1/fd
  Size: 65        	Blocks: 0          IO Block: 1024   directory
```

Correctness check:

```
$ sudo ls /proc/1/fd | wc -l
65
```

I added the docs for /proc/<pid>/fd while I'm at it.

[[email protected]: use bitmap_weight() to count the bits]
[[email protected]: include linux/bitmap.h for bitmap_weight()]
[[email protected]: return errno from proc_fd_getattr() instead of setting negative size]
Cc Alexey Dobriyan <[email protected]>
Cc Al Viro <[email protected]>
Cc Christoph Anton Mitterer <[email protected]>
Cc David Hildenbrand <[email protected]>
Cc David Laight <[email protected]>
Cc Ivan Babrou <[email protected]>
Cc Johannes Weiner <[email protected]>
Cc Jonathan Corbet <[email protected]>
Cc Kalesh Singh <[email protected]>
Cc Mike Rapoport <[email protected]>
Cc Paul Gortmaker <[email protected]>
Cc Theodore Ts'o <[email protected]>

---

 Documentation/filesystems/proc.rst |   17 ++++++++++
 fs/proc/fd.c                       |   45 +++++++++++++++++++++++++++
 2 files changed, 62 insertions(+)
Documentation/filesystems/proc.rst renamed from Documentation/filesystems/proc.rst~proc-report-open-files-as-size-in-stat-for-proc-pid-fd +17 -0
@@ -47,6 +47,7 @@ fixes/update part 1.1  Stefani Seibold <
   3.10  /proc/<pid>/timerslack_ns - Task timerslack value
   3.11	/proc/<pid>/patch_state - Livepatch patch operation state
   3.12	/proc/<pid>/arch_status - Task architecture specific information
+  3.13  /proc/<pid>/fd - List of symlinks to open files
 
   4	Configuring procfs
   4.1	Mount options
@@ -2149,6 +2150,22 @@ AVX512_elapsed_ms
   the task is unlikely an AVX512 user, but depends on the workload and the
   scheduling scenario, it also could be a false negative mentioned above.
 
+3.13 /proc/<pid>/fd - List of symlinks to open files
+-------------------------------------------------------
+This directory contains symbolic links which represent open files
+the process is maintaining.  Example output::
+
+  lr-x------ 1 root root 64 Sep 20 17:53 0 -> /dev/null
+  l-wx------ 1 root root 64 Sep 20 17:53 1 -> /dev/null
+  lrwx------ 1 root root 64 Sep 20 17:53 10 -> 'socket:[12539]'
+  lrwx------ 1 root root 64 Sep 20 17:53 11 -> 'socket:[12540]'
+  lrwx------ 1 root root 64 Sep 20 17:53 12 -> 'socket:[12542]'
+
+The number of open files for the process is stored in 'size' member
+of stat() output for /proc/<pid>/fd for fast access.
+-------------------------------------------------------
+
+
 Chapter 4: Configuring procfs
 =============================
 
fs/proc/fd.c renamed from fs/proc/fd.c~proc-report-open-files-as-size-in-stat-for-proc-pid-fd +45 -0

66 lines not on this page · raw message

_

Patches currently in -mm which might be from [email protected] are