linux-2.6-block: deactivating pagecache for benchmarks

Hi Jens!

For doing benchmarks on the I/O-Schedulers, I thought it would be very useful to disable the pagecache.

Quoted 4 lines

Hi Jens!

For doing benchmarks on the I/O-Schedulers, I thought it would be very useful to disable the pagecache.

for benchmarks this is not enough though, you also need to clean the inode and dentry caches, as well as any filesystem specific caches (might be buffer cache).....
at which point it's probably nicer to just fake a limited umount since that has to do all of that anyway

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Hi Jens!

For doing benchmarks on the I/O-Schedulers, I thought it would be very useful to disable the pagecache.

I didn't want to make it so complicated so I just mark pages as not-uptodate, so they have to be read again. Another reason was, that I wanted to keep the conditions as near to reality as possible.

Further I thought it would be useful, if you could turn the pagecache on and off without rebooting the system.

I implemented a proc-fs entry "/proc/benchmark/pagecache" for this.

Probably this patch can be useful for anyone else, who wants to do some benchmarks on block-layer stuff.
And if not, I would appreciate if you could have a look on it.

For doing benchmarks on the I/O-Schedulers, I thought it would be very useful to disable the pagecache.

Quoted 3 lines · Dirk Henning Gerdes

For doing benchmarks on the I/O-Schedulers, I thought it would be very useful to disable the pagecache.

That's an FAQ. Something like this?

From: Andrew Morton <[email protected]>

Add /proc/sys/vm/drop-pagecache. When written to, this will cause the kernel to discard as much pagecache and reclaimable slab objects as it can.

It won't drop dirty data, so the user should run `sync' first.

Caveats:

a) Holds inode_lock for exorbitant amounts of time.

b) Needs to be taught about NUMA nodes: propagate these all the way through so the discarding can be controlled on a per-node basis.

c) The pagecache shrinking and slab shrinking should probably have separate controls.

Signed-off-by Andrew Morton <[email protected]>

Quoted 26 lines · Andrew Morton

Quoted 3 lines · Dirk Henning Gerdes

For doing benchmarks on the I/O-Schedulers, I thought it would be very useful to disable the pagecache.

That's an FAQ. Something like this?

From: Andrew Morton <[email protected]>

Add /proc/sys/vm/drop-pagecache. When written to, this will cause the kernel to discard as much pagecache and reclaimable slab objects as it can.

It won't drop dirty data, so the user should run `sync' first.

Caveats:

a) Holds inode_lock for exorbitant amounts of time.

b) Needs to be taught about NUMA nodes: propagate these all the way through so the discarding can be controlled on a per-node basis.

c) The pagecache shrinking and slab shrinking should probably have separate controls.

Quoted 1 line

Signed-off-by Andrew Morton <[email protected]>

ACK, I've wanted something like this for a while.

I really think it should be a config option, though, to discourage people from building with it :)

Quoted 3 lines · Dirk Henning Gerdes

For doing benchmarks on the I/O-Schedulers, I thought it would be very useful to disable the pagecache.

That's an FAQ. Something like this?

From: Andrew Morton <[email protected]>

Add /proc/sys/vm/drop-pagecache. When written to, this will cause the kernel to discard as much pagecache and reclaimable slab objects as it can.

It won't drop dirty data, so the user should run `sync' first.

Caveats:

a) Holds inode_lock for exorbitant amounts of time.

b) Needs to be taught about NUMA nodes: propagate these all the way through so the discarding can be controlled on a per-node basis.

c) The pagecache shrinking and slab shrinking should probably have separate controls.

Signed-off-by Andrew Morton <[email protected]>

Quoted 3 lines · Dirk Henning Gerdes

For doing benchmarks on the I/O-Schedulers, I thought it would be very useful to disable the pagecache.

That's an FAQ. Something like this?

From: Andrew Morton <[email protected]>

Add /proc/sys/vm/drop-pagecache. When written to, this will cause the kernel to discard as much pagecache and reclaimable slab objects as it can.

It won't drop dirty data, so the user should run `sync' first.

Caveats:

a) Holds inode_lock for exorbitant amounts of time.

b) Needs to be taught about NUMA nodes: propagate these all the way through so the discarding can be controlled on a per-node basis.

c) The pagecache shrinking and slab shrinking should probably have separate controls.

Signed-off-by Andrew Morton <[email protected]>

Wondering, if this shrinks shared memory pages (since they are backed by tmpfs) ? (which is not what I want).

Quoted 3 lines · Badari Pulavarty

Wondering, if this shrinks shared memory pages (since they are backed by tmpfs) ? (which is not what I want).

It'll reclaim unused pagecache pages. What effect that has on idioticfs^Wtmpfs pages depends on the state of the pages. If they're attached to tmpfs inodes then they won't be reclaimed because they have no backing store. If they're attached to swapcache then they won't be reclaimed because they have no superblock.

So I guess you got lucky.

Quoted 2 lines · Dirk Henning Gerdes

For doing benchmarks on the I/O-Schedulers, I thought it would be very useful to disable the pagecache.

That's an FAQ. Something like this?

From: Andrew Morton <[email protected]>

Add /proc/sys/vm/drop-pagecache. When written to, this will cause the kernel to discard as much pagecache and reclaimable slab objects as it can.

It won't drop dirty data, so the user should run `sync' first.

Caveats:

a) Holds inode_lock for exorbitant amounts of time.

b) Needs to be taught about NUMA nodes: propagate these all the way through so the discarding can be controlled on a per-node basis.

c) The pagecache shrinking and slab shrinking should probably have separate controls.

Signed-off-by Andrew Morton <[email protected]>

---

 fs/Makefile            |    2 -
 fs/drop-pagecache.c    |   62
+++++++++++++++++++++++++++++++++++++++++++++++++ include/linux/mm.h     | 
  5 +++
 include/linux/sysctl.h |    1
 kernel/sysctl.c        |    9 +++++++
 mm/truncate.c          |    1
 mm/vmscan.c            |    3 --
 7 files changed, 79 insertions(+), 4 deletions(-)

diff -puN /dev/null fs/drop-pagecache.c

fs/drop-pagecache.c 2005-12-01 17:20:55.000000000 -0800 renamed from dev/null 2003-09-15 06:40:47.000000000 -0700 +62 -0

@@ -0,0 +1,62 @@
+/*
+ * Implement the manual drop-all-pagecache function
+ */
+
+#include <linux/kernel.h>
+#include <linux/mm.h>
+#include <linux/fs.h>
+#include <linux/writeback.h>
+#include <linux/sysctl.h>
+#include <linux/gfp.h>
+
+static void drop_pagecache_sb(struct super_block *sb)
+{
+ struct inode *inode;
+
+ spin_lock(&inode_lock);
+ list_for_each_entry(inode, &sb->s_inodes, i_sb_list) {
+  if (inode->i_state & (I_FREEING|I_WILL_FREE))
+   continue;
+  invalidate_inode_pages(inode->i_mapping);
+ }
+ spin_unlock(&inode_lock);
+}
+
+static void drop_pagecache(void)
+{
+ struct super_block *sb;
+
+ spin_lock(&sb_lock);
+restart:
+ list_for_each_entry(sb, &super_blocks, s_list) {
+  sb->s_count++;
+  spin_unlock(&sb_lock);
+  down_read(&sb->s_umount);
+  if (sb->s_root)
+   drop_pagecache_sb(sb);
+  up_read(&sb->s_umount);
+  spin_lock(&sb_lock);
+  if (__put_super_and_need_restart(sb))
+   goto restart;
+ }
+ spin_unlock(&sb_lock);
+ printk("shrunk pagecache\n");
+}
+
+static void drop_slab(void)
+{
+ int nr_objects;
+
+ do {
+  nr_objects = shrink_slab(1000, GFP_KERNEL, 1000);
+  printk("shrunk %d cache objects\n", nr_objects);
+ } while (nr_objects > 10);
+}
+
+int drop_pagecache_sysctl_handler(ctl_table *table, int write,
+ struct file *file, void __user *buffer, size_t *length, loff_t *ppos)
+{
+ drop_pagecache();
+ drop_slab();
+ return 0;
+}

diff -puN fs/Makefile~drop-pagecache fs/Makefile

fs/Makefile 2005-12-01 16:41:22.000000000 -0800 renamed from fs/Makefile~drop-pagecache 2005-12-01 16:41:22.000000000 -0800 +1 -1

@@ -10,7 +10,7 @@ obj-y := open.o read_write.o file_table.
   ioctl.o readdir.o select.o fifo.o locks.o dcache.o inode.o \
   attr.o bad_inode.o file.o filesystems.o namespace.o aio.o \
   seq_file.o xattr.o libfs.o fs-writeback.o mpage.o direct-io.o \
-  ioprio.o pnode.o
+  ioprio.o pnode.o drop-pagecache.o
 
 obj-$(CONFIG_INOTIFY)  += inotify.o
 obj-$(CONFIG_EPOLL)  += eventpoll.o

diff -puN include/linux/mm.h~drop-pagecache include/linux/mm.h --- devel/include/linux/mm.h~drop-pagecache 2005-12-01 16:41:22.000000000 -0800 +++ devel-akpm/include/linux/mm.h 2005-12-01 17:01:57.000000000 -0800

fs/Makefile 2005-12-01 16:41:22.000000000 -0800 renamed from fs/Makefile~drop-pagecache 2005-12-01 16:41:22.000000000 -0800 +5 -0

@@ -1078,5 +1078,10 @@ int in_gate_area_no_task(unsigned long a
 /* /proc/<pid>/oom_adj set to -17 protects from the oom-killer */
 #define OOM_DISABLE -17
 
+int drop_pagecache_sysctl_handler(struct ctl_table *, int, struct file *,
+     void __user *, size_t *, loff_t *);
+int shrink_slab(unsigned long scanned, gfp_t gfp_mask,
+   unsigned long lru_pages);
+
 #endif /* __KERNEL__ */
 #endif /* _LINUX_MM_H */

diff -puN include/linux/sysctl.h~drop-pagecache include/linux/sysctl.h --- devel/include/linux/sysctl.h~drop-pagecache 2005-12-01 16:41:22.000000000 -0800 +++ devel-akpm/include/linux/sysctl.h 2005-12-01 16:41:22.000000000 -0800 @@ -182,6 +182,7 @@ enum
VM_LEGACY_VA_LAYOUT=27, /* legacy/compatibility virtual address space
layout */ VM_SWAP_TOKEN_TIMEOUT=28, /* default time for token time out */
VM_SWAP_PREFETCH=29, /* int: amount to swap prefetch */
+ VM_DROP_PAGECACHE=30, /* int: nuke lots of pagecache */
};

diff -puN kernel/sysctl.c~drop-pagecache kernel/sysctl.c
--- devel/kernel/sysctl.c~drop-pagecache 2005-12-01 16:41:22.000000000 -0800 +++ devel-akpm/kernel/sysctl.c 2005-12-01 16:41:22.000000000 -0800 @@ -783,6 +783,15 @@ static ctl_table vm_table[] = {
.strategy = &sysctl_intvec,
},
{

fs/Makefile 2005-12-01 16:41:22.000000000 -0800 renamed from fs/Makefile~drop-pagecache 2005-12-01 16:41:22.000000000 -0800 +4 -0

+  .proc_handler = drop_pagecache_sysctl_handler,
+  .strategy = &sysctl_intvec,
+ },
+ {
   .ctl_name = VM_MIN_FREE_KBYTES,
   .procname = "min_free_kbytes",
   .data  = &min_free_kbytes,

diff -puN mm/truncate.c~drop-pagecache mm/truncate.c

Quoted 4 lines

Add /proc/sys/vm/drop-pagecache. When written to, this will cause the kernel to discard as much pagecache and reclaimable slab objects as it can.

It won't drop dirty data, so the user should run `sync' first.

This is deeply, deeply cool.

Quoted 3 lines

Caveats:

a) Holds inode_lock for exorbitant amounts of time.

Voluntary preemption point, maybe?

Quoted 5 lines

Quoted 3 lines

Caveats:

a) Holds inode_lock for exorbitant amounts of time.

Voluntary preemption point, maybe?

I thin it's a bad idea, that would just encourage people to use this for anything other than debugging. If you care about latency don't discard the page cache.

The GNOME people have been asking for this for a while, in order to improve startup times, they would like a way to simulate a cold start without rebooting.

Lee

Quoted 3 lines · Dirk Henning Gerdes

For doing benchmarks on the I/O-Schedulers, I thought it would be very useful to disable the pagecache.

That's an FAQ. Something like this?

From: Andrew Morton <[email protected]>

Add /proc/sys/vm/drop-pagecache. When written to, this will cause the kernel to discard as much pagecache and reclaimable slab objects as it can.

It won't drop dirty data, so the user should run `sync' first.

For doing benchmarks on the I/O-Schedulers, I thought it would be very useful to disable the pagecache.

Hey Jeff, you wanted the ability to shrink all the caches and free up memory for the hotpluggy thing?

Quoted 2 lines · Rob Landley

Hey Jeff, you wanted the ability to shrink all the caches and free up memory for the hotpluggy thing?

Close, but what I really want is to free up a specified amount of page cache. So, if I want to remove 16M from a UML, and get_free_pages can only give me 10 before returning NULL, then I want to say "drop 6M of cache, and make it the oldest 6M".

Conceptually, I want to put memory pressure on the thing until it drops enough cache to free the memory I want.