drop page cache of a single file
Subsystems (4)
PATCH 1/1 drop page cache of a single file
Currently, by /proc/sys/vm/drop_caches, applications could drop pagecache, slab(dentries and inodes), or both, but applications couldn't choose to just drop the page cache of one file. An user of VOD (Video-On-Demand) needs this capability to have more detailed control on page cache release.
Below patch against 2.6.19 implements it.
---
Documentation/filesystems/proc.txt +2 -0
@@ -1320,6 +1320,8 @@ To free dentries and inodes: echo 2 > /proc/sys/vm/drop_caches To free pagecache, dentries and inodes: echo 3 > /proc/sys/vm/drop_caches +To free the pagecache of one file: + echo "4 /path/to/filename" > /proc/sys/vm/drop_caches As this is a non-destructive operation and dirty objects are not freeable, the user should run `sync' first.
fs/drop_caches.c +61 -6
85 lines not on this page · raw message
include/linux/mm.h +1 -0
@@ -1121,6 +1121,7 @@ unsigned long shrink_slab(unsigned long unsigned long lru_pages);void drop_pagecache (void );void drop_slab (void ); +void drop_file_pagecache (char *path);#ifndef CONFIG_MMU #define randomize_va_space 0
kernel/sysctl.c +3 -3
@@ -73,7 +73,7 @@ extern int min_free_kbytes;extern int printk_ratelimit_jiffies;extern int printk_ratelimit_burst;extern int pid_max_min, pid_max_max; -extern int sysctl_drop_caches; +extern char sysctl_drop_caches[PATH_MAX +2 ];extern int percpu_pagelist_fraction;extern int compat_log;
@@ -901,10 +901,10 @@ static ctl_table vm_table[] = { .ctl_name = VM_DROP_PAGECACHE, .procname = "drop_caches", .data = &sysctl_drop_caches, - .maxlen = sizeof(int), + .maxlen = sizeof(sysctl_drop_caches), .mode = 0644, .proc_handler = drop_caches_sysctl_handler, - .strategy = &sysctl_intvec, + .strategy = &sysctl_string, }, { .ctl_name = VM_MIN_FREE_KBYTES,
Quoted 4 lines · Yanmin
Currently, by
/proc/sys/vm/drop_caches, applications could drop pagecache, slab(dentries and inodes), or both, but applications couldn't choose to just drop the page cache of one file. An user of VOD (Video-On-Demand) needs this capability to have more detailed control on page cache release.
The posix_fadvise() system call should be used for this. Probably in combination with sys_sync_file_range().
Quoted 10 lines · Andrew Morton
Quoted 4 lines · Yanmin
Currently, by
/proc/sys/vm/drop_caches, applications could drop pagecache, slab(dentries and inodes), or both, but applications couldn't choose to just drop the page cache of one file. An user of VOD (Video-On-Demand) needs this capability to have more detailed control on page cache release.The
posix_fadvise()system call should be used for this. Probably in combination withsys_sync_file_range().
Yanmin: I've been using the fadvise tool from
http://www.zip.com.au/~akpm/linux/patches/stuff/ext3-tools.tar.gz
It's a nice tool:
% fadvise
Usage: fadvise filename offset length advice [loops]
advice: normal sequential willneed noreuse dontneed asyncwrite writewait
% fadvise /var/sparse 0 0x7fffffff dontneed
Regards,
Wu
Quoted 15 lines · On Thu, 28 Dec 2006 15:19:04 +0800 Fengguang Wu <[email protected]> wrote:
Quoted 10 lines · Andrew Morton
Quoted 4 lines · Yanmin
Currently, by
/proc/sys/vm/drop_caches, applications could drop pagecache, slab(dentries and inodes), or both, but applications couldn't choose to just drop the page cache of one file. An user of VOD (Video-On-Demand) needs this capability to have more detailed control on page cache release.The
posix_fadvise()system call should be used for this. Probably in combination withsys_sync_file_range().Yanmin: I've been using the fadvise tool from
http://www.zip.com.au/~akpm/linux/patches/stuff/ext3-tools.tar.gz
Quoted 7 lines
It's a nice tool:
% fadvise
Usage: fadvise filename offset length advice [loops]
advice: normal sequential willneed noreuse dontneed asyncwrite writewait
% fadvise/var/sparse00x7fffffffdontneed
I was a bit reluctant to point at that because it has nasty hacks to make it mostly-work on old glibc's which don't implement posix_fadvise().
Hopefully if you're running a recent distro, you have glibc support for fadvise() and it's possible to write a portable version of that app which doesn't need to know about per-arch syscall numbers.
Quoted 13 lines · Andrew Morton
Quoted 10 lines · On Thu, 28 Dec 2006 15:19:04 +0800 Fengguang Wu <[email protected]> wrote:
Yanmin: I've been using the fadvise tool from
http://www.zip.com.au/~akpm/linux/patches/stuff/ext3-tools.tar.gzIt's a nice tool:
% fadvise
Usage: fadvise filename offset length advice [loops]
advice: normal sequential willneed noreuse dontneed asyncwrite writewait
% fadvise/var/sparse00x7fffffffdontneed
Quoted 6 lines
I was a bit reluctant to point at that because it has nasty hacks to make it mostly-work on old glibc's which don't implement
posix_fadvise().Hopefully if you're running a recent distro, you have glibc support for
fadvise()and it's possible to write a portable version of that app which doesn't need to know about per-arch syscall numbers.
And note that if it gets implemented on ARM on pre-fadvise() glibc, the syscall argument order is rather non-standard: fd, action, start, size rather than fd, start, size, action - since otherwise we run out of registers with EABI.
The kernel community needs to get a grip with the implementation of new syscalls - we need a process where architecture maintainers get to review the arguments _prior_ to them being accepted into the kernel. That way we can avoid silly architecture specific syscall changes like this.
Signature · 3 lines
Russell King Linux kernel 2.6 ARM Linux - http://www.arm.linux.org.uk/ maintainer of:
Quoted 5 lines · On Thu, 28 Dec 2006 10:45:08 +0000 Russell King <[email protected]>…
The kernel community needs to get a grip with the implementation of new syscalls - we need a process where architecture maintainers get to review the arguments _prior_ to them being accepted into the kernel. That way we can avoid silly architecture specific syscall changes like this.
hm, well, actually, sys_fadvise64_64 was discussed on linux-arch when it went in.
(Gad, it was over three years ago! - glibc support should be pretty widespread now)
That's about as much discussion as these things will get. And this assumes that people remember to mention it. Mental note made.
Quoted 14 lines · Andrew Morton
Quoted 10 lines · On Thu, 28 Dec 2006 15:19:04 +0800 Fengguang Wu <[email protected]> wrote:
Yanmin: I've been using the fadvise tool from
http://www.zip.com.au/~akpm/linux/patches/stuff/ext3-tools.tar.gzIt's a nice tool:
% fadvise
Usage: fadvise filename offset length advice [loops]
advice: normal sequential willneed noreuse dontneed asyncwrite writewait
% fadvise/var/sparse00x7fffffffdontneed
Quoted 6 lines
I was a bit reluctant to point at that because it has nasty hacks to make it mostly-work on old glibc's which don't implement
posix_fadvise().Hopefully if you're running a recent distro, you have glibc support for
fadvise()and it's possible to write a portable version of that app which doesn't need to know about per-arch syscall numbers.
Bad news: it's still broken. posix_fadvise() here just failed silently. I'm running Debian Etch with libc6=2.3.6.ds1-7.
Quoted 15 lines · Yanmin
Currently, by
/proc/sys/vm/drop_caches, applications could drop pagecache, slab(dentries and inodes), or both, but applications couldn't choose to just drop the page cache of one file. An user of VOD (Video-On-Demand) needs this capability to have more detailed control on page cache release.Below patch against 2.6.19 implements it.
Signed-off-by Zhang Yanmin <[email protected]>---
diff -Nraup
linux-2.6.19/Documentation/filesystems/proc.txtlinux-2.6.19_dropcache/Documentation/filesystems/proc.txt
Documentation/filesystems/proc.txt To free dentries and inodes: · line 1327
Quoted 8 lines
Documentation/filesystems/proc.txt +2
-0@@ -1320,6 +1320,8 @@ To free dentries and inodes: echo 2 > /proc/sys/vm/drop_caches To free pagecache, dentries and inodes: echo 3 > /proc/sys/vm/drop_caches +To free the pagecache of one file: + echo "4 /path/to/filename" > /proc/sys/vm/drop_caches As this is a non-destructive operation and dirty objects are not freeable, the user should run `sync' first.
"sync" is the most time consuming operation. Clean pagecache pages are immediately reclaimable... they are actually free pages. Writing out dirty pages consumes time.
Hence this approach may not provide the required performance benefits since only clean pagecache pages are marked free. fadvise approach would provide similar behavior.
--Vaidy
[snip]