add some drop_caches documentation and info messsge

RFC PATCH In review Dave Hansen 3 participants 5 messages Sep 16, 2010 – Sep 21, 2010

Subsystems (3)

Memory management Documentation VFS

lore

Share & tools

lore

Dave Hansen

author

2010-09-16 16:50 UTC #0 lore raw

RFC PATCH v2 1/1 add some drop_caches documentation and info messsge

This version tones down the BUG_ON(). I also noticed that the documentation fails to mention that more than just the inode and dentry slabs are shrunk.

There is plenty of anecdotal evidence and a load of blog posts suggesting that using "drop_caches" periodically keeps your system running in "tip top shape". Perhaps adding some kernel documentation will increase the amount of accurate data on its use.

If we are not shrinking caches effectively, then we have real bugs. Using drop_caches will simply mask the bugs and make them harder to find, but certainly does not fix them, nor is it an appropriate "workaround" to limit the size of the caches.

It's a great debugging tool, and is really handy for doing things like repeatable benchmark runs. So, add a bit more documentation about it, and add a little KERN_NOTICE. It should help developers who are chasing down reclaim-related bugs.

Signed-off-by Dave Hansen <[email protected]>

---

 linux-2.6.git-dave/Documentation/sysctl/vm.txt |   33 ++++++++++++++++++++-----
 linux-2.6.git-dave/fs/drop_caches.c            |    2 +
 2 files changed, 29 insertions(+), 6 deletions(-)

Documentation/sysctl/vm.txt renamed from Documentation/sysctl/vm.txt~update-drop_caches-documentation +27 -6

diff -puN Documentation/sysctl/vm.txt~update-drop_caches-documentation Documentation/sysctl/vm.txt
@@ -135,18 +135,39 @@ Setting this to zero disables periodic w
 
 drop_caches
 
-Writing to this will cause the kernel to drop clean caches, dentries and
-inodes from memory, causing that memory to become free.
+Writing to this will cause the kernel to drop clean caches, as well as
+reclaimable slab objects like dentries and inodes.  Once dropped, their
+memory becomes free.
 
 To free pagecache:
 	echo 1 > /proc/sys/vm/drop_caches
-To free dentries and inodes:
+To free reclaimable slab objects (includes dentries and inodes):
 	echo 2 > /proc/sys/vm/drop_caches
-To free pagecache, dentries and inodes:
+To free slab objects and pagecache:
 	echo 3 > /proc/sys/vm/drop_caches
 
-As this is a non-destructive operation and dirty objects are not freeable, the
-user should run `sync' first.
+This is a non-destructive operation and will not free any dirty objects.
+To increase the number of objects freed by this operation, the user may run
+`sync' prior to writing to /proc/sys/vm/drop_caches.  This will minimize the
+number of dirty objects on the system and create more candidates to be
+dropped.
+
+This file is not a means to control the growth of the various kernel caches
+(inodes, dentries, pagecache, etc...)  These objects are automatically
+reclaimed by the kernel when memory is needed elsewhere on the system.
+
+Use of this file can cause performance problems.  Since it discards cached
+objects, it may cost a significant amount of I/O and CPU to recreate the
+dropped objects, especially if they were under heavy use.  Because of this,
+use outside of a testing or debugging environment is not recommended.
+
+You may see informational messages in your kernel log when this file is
+used:
+
+	cat (1234): dropped kernel caches: 3
+
+These are informational only.  They do not mean that anything is wrong
+with your system.
 
 ==============================================================

fs/drop_caches.c renamed from fs/drop_caches.c~update-drop_caches-documentation +2 -0

diff -puN fs/drop_caches.c~update-drop_caches-documentation fs/drop_caches.c
@@ -47,6 +47,8 @@ int drop_caches_sysctl_handler(ctl_table
 {
 	proc_dointvec_minmax(table, write, buffer, length, ppos);
 	if (write) {
+		printk(KERN_NOTICE "%s (%d): dropped kernel caches: %d\n",
+			current->comm, task_pid_nr(current), sysctl_drop_caches);
 		if (sysctl_drop_caches & 1)
 			iterate_supers(drop_pagecache_sb, NULL);
 		if (sysctl_drop_caches & 2)

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

KOSAKI Motohiro

2010-09-17 00:26 UTC #1 lore raw

fs/drop_caches.c int drop_caches_sysctl_handler(ctl_table · line 54

Quoted 12 lines

diff -puN fs/drop_caches.c~update-drop_caches-documentation fs/drop_caches.c

fs/drop_caches.c renamed from fs/drop_caches.c~update-drop_caches-documentation +2 -0

@@ -47,6 +47,8 @@ int drop_caches_sysctl_handler(ctl_table
 {
 	proc_dointvec_minmax(table, write, buffer, length, ppos);
 	if (write) {
+		printk(KERN_NOTICE "%s (%d): dropped kernel caches: %d\n",
+			current->comm, task_pid_nr(current), sysctl_drop_caches);
 		if (sysctl_drop_caches & 1)
 			iterate_supers(drop_pagecache_sb, NULL);
 		if (sysctl_drop_caches & 2)

Can't you print it only once?

Dave Hansen

author

2010-09-17 15:54 UTC #2 lore raw

Quoted 6 lines · KOSAKI Motohiro

Quoted 6 lines
diff -puN fs/drop_caches.c~update-drop_caches-documentation fs/drop_caches.c
fs/drop_caches.c renamed from fs/drop_caches.c~update-drop_caches-documentation +0 -0
@@ -47,6 +47,8 @@ int drop_caches_sysctl_handler(ctl_table
 {
 	proc_dointvec_minmax(table, write, buffer, length, ppos);

Quoted 8 lines

fs/drop_caches.c int drop_caches_sysctl_handler(ctl_table · line 54
Quoted 6 lines
fs/drop_caches.c renamed from fs/drop_caches.c~update-drop_caches-documentation +2 -0
 	if (write) {
+		printk(KERN_NOTICE "%s (%d): dropped kernel caches: %d\n",
+			current->comm, task_pid_nr(current), sysctl_drop_caches);
 		if (sysctl_drop_caches & 1)
 			iterate_supers(drop_pagecache_sb, NULL);
 		if (sysctl_drop_caches & 2)
Can't you print it only once?

Sure. But, I also figured that somebody calling it every minute is going to be much more interesting than something just on startup. Should we printk_ratelimit() it, perhaps?

-- Dave

KOSAKI Motohiro

2010-09-21 00:57 UTC #3 lore raw

Quoted 16 lines · KOSAKI Motohiro

Quoted 6 lines
Quoted 6 lines
diff -puN fs/drop_caches.c~update-drop_caches-documentation fs/drop_caches.c
fs/drop_caches.c renamed from fs/drop_caches.c~update-drop_caches-documentation +0 -0
@@ -47,6 +47,8 @@ int drop_caches_sysctl_handler(ctl_table
 {
 	proc_dointvec_minmax(table, write, buffer, length, ppos);
Quoted 8 lines
fs/drop_caches.c int drop_caches_sysctl_handler(ctl_table · line 54
Quoted 6 lines
fs/drop_caches.c renamed from fs/drop_caches.c~update-drop_caches-documentation +2 -0
 	if (write) {
+		printk(KERN_NOTICE "%s (%d): dropped kernel caches: %d\n",
+			current->comm, task_pid_nr(current), sysctl_drop_caches);
 		if (sysctl_drop_caches & 1)
 			iterate_supers(drop_pagecache_sb, NULL);
 		if (sysctl_drop_caches & 2)
Can't you print it only once?

Quoted 3 lines

Sure. But, I also figured that somebody calling it every minute is going to be much more interesting than something just on startup. Should we printk_ratelimit() it, perhaps?

Umm...

every minute drop_caches + printk_ratelimit() mean every drop_caches output printk(). It seems annoying. I'm worry about that I'll see drop_caches's printk fill my syslog.

But, It is not strong opinion. Because I don't use every minute drop_caches, then I have no experience such usecase. It's up to you.

KAMEZAWA Hiroyuki

Reviewed-by on v2 1/1

2010-09-21 01:05 UTC #4 lore raw

In reply to Dave Hansen

Quoted 17 lines · On Thu, 16 Sep 2010 09:50:47 -0700 Dave Hansen <[email protected]> wrote:

This version tones down the BUG_ON(). I also noticed that the documentation fails to mention that more than just the inode and dentry slabs are shrunk.
--
There is plenty of anecdotal evidence and a load of blog posts suggesting that using "drop_caches" periodically keeps your system running in "tip top shape". Perhaps adding some kernel documentation will increase the amount of accurate data on its use.
If we are not shrinking caches effectively, then we have real bugs. Using drop_caches will simply mask the bugs and make them harder to find, but certainly does not fix them, nor is it an appropriate "workaround" to limit the size of the caches.

Quoted 6 lines

It's a great debugging tool, and is really handy for doing things like repeatable benchmark runs. So, add a bit more documentation about it, and add a little KERN_NOTICE. It should help developers who are chasing down reclaim-related bugs.
Signed-off-by Dave Hansen <[email protected]>

Reviewed-by KAMEZAWA Hiroyuki <[email protected]>

thanks.