Skip to contentSkip to conversation

inotify: Increase default inotify.max_user_watches limit to 1048576

PATCH Superseded Waiman Long 2 participants 3 messages Oct 29, 2020 – Nov 9, 2020

Subsystems (3)

v3 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

PATCH v3 1/1 inotify: Increase default inotify.max_user_watches limit to 1048576

The default value of inotify.max_user_watches sysctl parameter was set to 8192 since the introduction of the inotify feature in 2005 by commit 0eeca28300df ("[PATCH] inotify"). Today this value is just too small for many modern usage. As a result, users have to explicitly set it to a larger value to make it work.

After some searching around the web, these are the inotify.max_user_watches values used by some projects:
- vscode: 524288
- dropbox support: 100000
- users on stackexchange: 12228
- lsyncd user: 2000000
- code42 support: 1048576
- monodevelop: 16384
- tectonic: 524288
- openshift origin: 65536

Each watch point adds an inotify_inode_mark structure to an inode to be watched. It also pins the watched inode.

Modeled after the epoll.max_user_watches behavior to adjust the default value according to the amount of addressable memory available, make inotify.max_user_watches behave in a similar way to make it use no more than 1% of addressable memory within the range [8192, 1048576].

For 64-bit archs, inotify_inode_mark plus 2 vfs inode have a size that is a bit over 1 kbytes (1284 bytes with my x86-64 config). That means a system with 128GB or more memory will likely have the maximum value of 1048576 for inotify.max_user_watches. This default should be big enough for most use cases.

[v3: increase inotify watch cost as suggested by Amir and Honza]

---
 fs/notify/inotify/inotify_user.c | 23 ++++++++++++++++++++++-
 1 file changed, 22 insertions(+), 1 deletion(-)
fs/notify/inotify/inotify_user.c +22 -1
index 186722ba3894..f8065eda3a02 100644
@@ -37,6 +37,15 @@
 
 #include <asm/ioctls.h>
 
+/*
+ * An inotify watch requires allocating an inotify_inode_mark structure as
+ * well as pinning the watched inode. Doubling the size of a VFS inode
+ * should be more than enough to cover the additional filesystem inode
+ * size increase.
+ */
+#define INOTIFY_WATCH_COST	(sizeof(struct inotify_inode_mark) + \
+				 2 * sizeof(struct inode))
+
 /* configurable via /proc/sys/fs/inotify/ */
 static int inotify_max_queued_events __read_mostly;
 
@@ -801,6 +810,18 @@ SYSCALL_DEFINE2(inotify_rm_watch, int, fd, __s32, wd)
  */
 static int __init inotify_user_setup(void)
 {
+	unsigned int watches_max;
+	struct sysinfo si;
+
+	si_meminfo(&si);
+	/*
+	 * Allow up to 1% of addressible memory to be allocated for inotify
+	 * watches (per user) limited to the range [8192, 1048576].
+	 */
+	watches_max = (((si.totalram - si.totalhigh) / 100) << PAGE_SHIFT) /
+			INOTIFY_WATCH_COST;
+	watches_max = min(1048576U, max(watches_max, 8192U));
+
 	BUILD_BUG_ON(IN_ACCESS != FS_ACCESS);
 	BUILD_BUG_ON(IN_MODIFY != FS_MODIFY);
 	BUILD_BUG_ON(IN_ATTRIB != FS_ATTRIB);
@@ -827,7 +848,7 @@ static int __init inotify_user_setup(void)
 
 	inotify_max_queued_events = 16384;
 	init_user_ns.ucount_max[UCOUNT_INOTIFY_INSTANCES] = 128;
-	init_user_ns.ucount_max[UCOUNT_INOTIFY_WATCHES] = 8192;
+	init_user_ns.ucount_max[UCOUNT_INOTIFY_WATCHES] = watches_max;
 
 	return 0;
 }
Signature · 1 line
2.18.1
maintainer
#1 lore raw
Quoted 31 lines · Waiman Long

The default value of inotify.max_user_watches sysctl parameter was set to 8192 since the introduction of the inotify feature in 2005 by commit 0eeca28300df ("[PATCH] inotify"). Today this value is just too small for many modern usage. As a result, users have to explicitly set it to a larger value to make it work.

After some searching around the web, these are the inotify.max_user_watches values used by some projects:
- vscode: 524288
- dropbox support: 100000
- users on stackexchange: 12228
- lsyncd user: 2000000
- code42 support: 1048576
- monodevelop: 16384
- tectonic: 524288
- openshift origin: 65536

Each watch point adds an inotify_inode_mark structure to an inode to be watched. It also pins the watched inode.

Modeled after the epoll.max_user_watches behavior to adjust the default value according to the amount of addressable memory available, make inotify.max_user_watches behave in a similar way to make it use no more than 1% of addressable memory within the range [8192, 1048576].

For 64-bit archs, inotify_inode_mark plus 2 vfs inode have a size that is a bit over 1 kbytes (1284 bytes with my x86-64 config). That means a system with 128GB or more memory will likely have the maximum value of 1048576 for inotify.max_user_watches. This default should be big enough for most use cases.

Quoted 3 lines

[v3: increase inotify watch cost as suggested by Amir and Honza]

Overall this looks fine. Some remaining nits below.

Quoted 21 lines
fs/notify/inotify/inotify_user.c +9 -0
index 186722ba3894..f8065eda3a02 100644
@@ -37,6 +37,15 @@
 
 #include <asm/ioctls.h>
 
+/*
+ * An inotify watch requires allocating an inotify_inode_mark structure as
+ * well as pinning the watched inode. Doubling the size of a VFS inode
+ * should be more than enough to cover the additional filesystem inode
+ * size increase.
+ */
+#define INOTIFY_WATCH_COST	(sizeof(struct inotify_inode_mark) + \
+				 2 * sizeof(struct inode))
+
 /* configurable via /proc/sys/fs/inotify/ */
 static int inotify_max_queued_events __read_mostly;
 

fs/notify/inotify/inotify_user.c SYSCALL_DEFINE2(inotify_rm_watch, int, fd, __s32, wd) · line 818

Quoted 9 lines
fs/notify/inotify/inotify_user.c +6 -0
@@ -801,6 +810,18 @@ SYSCALL_DEFINE2(inotify_rm_watch, int, fd, __s32, wd)
  */
 static int __init inotify_user_setup(void)
 {
+	unsigned int watches_max;
+	struct sysinfo si;
+
+	si_meminfo(&si);
+	/*
+	 * Allow up to 1% of addressible memory to be allocated for inotify

^^^^ addressable

fs/notify/inotify/inotify_user.c SYSCALL_DEFINE2(inotify_rm_watch, int, fd, __s32, wd) · line 822

Quoted 4 lines
fs/notify/inotify/inotify_user.c +4 -0
+	 * watches (per user) limited to the range [8192, 1048576].
+	 */
+	watches_max = (((si.totalram - si.totalhigh) / 100) << PAGE_SHIFT) /
+			INOTIFY_WATCH_COST;

^^^ So for machines with > 1TB of memory
watches_max would overflow. So you probably need to use ulong for that.

fs/notify/inotify/inotify_user.c SYSCALL_DEFINE2(inotify_rm_watch, int, fd, __s32, wd) · line 823

Quoted 1 line
fs/notify/inotify/inotify_user.c +1 -0
+	watches_max = min(1048576U, max(watches_max, 8192U));
			^^^ use clamp() here?

								Honza
Signature · 2 lines
Jan Kara <[email protected]>
SUSE Labs, CR
#2 lore raw
Quoted 77 lines · Jan Kara
Quoted 31 lines · Waiman Long

The default value of inotify.max_user_watches sysctl parameter was set to 8192 since the introduction of the inotify feature in 2005 by commit 0eeca28300df ("[PATCH] inotify"). Today this value is just too small for many modern usage. As a result, users have to explicitly set it to a larger value to make it work.

After some searching around the web, these are the inotify.max_user_watches values used by some projects:
- vscode: 524288
- dropbox support: 100000
- users on stackexchange: 12228
- lsyncd user: 2000000
- code42 support: 1048576
- monodevelop: 16384
- tectonic: 524288
- openshift origin: 65536

Each watch point adds an inotify_inode_mark structure to an inode to be watched. It also pins the watched inode.

Modeled after the epoll.max_user_watches behavior to adjust the default value according to the amount of addressable memory available, make inotify.max_user_watches behave in a similar way to make it use no more than 1% of addressable memory within the range [8192, 1048576].

For 64-bit archs, inotify_inode_mark plus 2 vfs inode have a size that is a bit over 1 kbytes (1284 bytes with my x86-64 config). That means a system with 128GB or more memory will likely have the maximum value of 1048576 for inotify.max_user_watches. This default should be big enough for most use cases.

Quoted 3 lines

[v3: increase inotify watch cost as suggested by Amir and Honza]

Overall this looks fine. Some remaining nits below.

Quoted 21 lines
fs/notify/inotify/inotify_user.c +9 -0
index 186722ba3894..f8065eda3a02 100644
@@ -37,6 +37,15 @@
 
  #include <asm/ioctls.h>
 
+/*
+ * An inotify watch requires allocating an inotify_inode_mark structure as
+ * well as pinning the watched inode. Doubling the size of a VFS inode
+ * should be more than enough to cover the additional filesystem inode
+ * size increase.
+ */
+#define INOTIFY_WATCH_COST	(sizeof(struct inotify_inode_mark) + \
+				 2 * sizeof(struct inode))
+
  /* configurable via /proc/sys/fs/inotify/ */
  static int inotify_max_queued_events __read_mostly;
 

fs/notify/inotify/inotify_user.c SYSCALL_DEFINE2(inotify_rm_watch, int, fd, __s32, wd) · line 818

Quoted 9 lines
fs/notify/inotify/inotify_user.c +6 -0
@@ -801,6 +810,18 @@ SYSCALL_DEFINE2(inotify_rm_watch, int, fd, __s32, wd)
   */
  static int __init inotify_user_setup(void)
  {
+	unsigned int watches_max;
+	struct sysinfo si;
+
+	si_meminfo(&si);
+	/*
+	 * Allow up to 1% of addressible memory to be allocated for inotify

^^^^ addressable

fs/notify/inotify/inotify_user.c SYSCALL_DEFINE2(inotify_rm_watch, int, fd, __s32, wd) · line 822

Quoted 4 lines
fs/notify/inotify/inotify_user.c +4 -0
+	 * watches (per user) limited to the range [8192, 1048576].
+	 */
+	watches_max = (((si.totalram - si.totalhigh) / 100) << PAGE_SHIFT) /
+			INOTIFY_WATCH_COST;

^^^ So for machines with > 1TB of memory
watches_max would overflow. So you probably need to use ulong for that.

Quoted 2 lines

fs/notify/inotify/inotify_user.c SYSCALL_DEFINE2(inotify_rm_watch, int, fd, __s32, wd) · line 823

Quoted 1 line
fs/notify/inotify/inotify_user.c +1 -0
+	watches_max = min(1048576U, max(watches_max, 8192U));
			^^^ use clamp() here?

Yes, it will be easier to read to use clamp() here. Will send out v4 withat those changes.

Thanks,
Longman