Skip to contentSkip to conversation

drop page cache of a single file

PATCH In review Zhang, Yanmin 5 participants 8 messages Dec 28, 2006 – Jan 28, 2007

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 drop page cache of a single file

Currently, by /proc/sys/vm/drop_caches, applications could drop pagecache, slab(dentries and inodes), or both, but applications couldn't choose to just drop the page cache of one file. An user of VOD (Video-On-Demand) needs this capability to have more detailed control on page cache release.

Below patch against 2.6.19 implements it.

---

Documentation/filesystems/proc.txt +2 -0
diff -Nraup linux-2.6.19/Documentation/filesystems/proc.txt linux-2.6.19_dropcache/Documentation/filesystems/proc.txt
@@ -1320,6 +1320,8 @@ To free dentries and inodes:
 	echo 2 > /proc/sys/vm/drop_caches
 To free pagecache, dentries and inodes:
 	echo 3 > /proc/sys/vm/drop_caches
+To free the pagecache of one file:
+	echo "4 /path/to/filename" > /proc/sys/vm/drop_caches
 
 As this is a non-destructive operation and dirty objects are not freeable, the
 user should run `sync' first.
fs/drop_caches.c +61 -6

85 lines not on this page · raw message

include/linux/mm.h +1 -0
diff -Nraup linux-2.6.19/include/linux/mm.h linux-2.6.19_dropcache/include/linux/mm.h
@@ -1121,6 +1121,7 @@ unsigned long shrink_slab(unsigned long 
 			unsigned long lru_pages);
 void drop_pagecache(void);
 void drop_slab(void);
+void drop_file_pagecache(char *path);
 
 #ifndef CONFIG_MMU
 #define randomize_va_space 0
kernel/sysctl.c +3 -3
diff -Nraup linux-2.6.19/kernel/sysctl.c linux-2.6.19_dropcache/kernel/sysctl.c
@@ -73,7 +73,7 @@ extern int min_free_kbytes;
 extern int printk_ratelimit_jiffies;
 extern int printk_ratelimit_burst;
 extern int pid_max_min, pid_max_max;
-extern int sysctl_drop_caches;
+extern char sysctl_drop_caches[PATH_MAX+2];
 extern int percpu_pagelist_fraction;
 extern int compat_log;
 
@@ -901,10 +901,10 @@ static ctl_table vm_table[] = {
 		.ctl_name	= VM_DROP_PAGECACHE,
 		.procname	= "drop_caches",
 		.data		= &sysctl_drop_caches,
-		.maxlen		= sizeof(int),
+		.maxlen		= sizeof(sysctl_drop_caches),
 		.mode		= 0644,
 		.proc_handler	= drop_caches_sysctl_handler,
-		.strategy	= &sysctl_intvec,
+		.strategy	= &sysctl_string,
 	},
 	{
 		.ctl_name	= VM_MIN_FREE_KBYTES,
#1 lore raw
Quoted 4 lines · Yanmin

Currently, by /proc/sys/vm/drop_caches, applications could drop pagecache, slab(dentries and inodes), or both, but applications couldn't choose to just drop the page cache of one file. An user of VOD (Video-On-Demand) needs this capability to have more detailed control on page cache release.

The posix_fadvise() system call should be used for this. Probably in combination with sys_sync_file_range().

#2 lore raw
Quoted 10 lines · Andrew Morton
Quoted 4 lines · Yanmin

Currently, by /proc/sys/vm/drop_caches, applications could drop pagecache, slab(dentries and inodes), or both, but applications couldn't choose to just drop the page cache of one file. An user of VOD (Video-On-Demand) needs this capability to have more detailed control on page cache release.

The posix_fadvise() system call should be used for this. Probably in combination with sys_sync_file_range().

Yanmin: I've been using the fadvise tool from
http://www.zip.com.au/~akpm/linux/patches/stuff/ext3-tools.tar.gz

It's a nice tool:

% fadvise
Usage: fadvise filename offset length advice [loops]
advice: normal sequential willneed noreuse dontneed asyncwrite writewait
% fadvise /var/sparse 0 0x7fffffff dontneed

Regards,
Wu

#3 lore raw
Quoted 15 lines · On Thu, 28 Dec 2006 15:19:04 +0800 Fengguang Wu <[email protected]> wrote:
Quoted 10 lines · Andrew Morton
Quoted 4 lines · Yanmin

Currently, by /proc/sys/vm/drop_caches, applications could drop pagecache, slab(dentries and inodes), or both, but applications couldn't choose to just drop the page cache of one file. An user of VOD (Video-On-Demand) needs this capability to have more detailed control on page cache release.

The posix_fadvise() system call should be used for this. Probably in combination with sys_sync_file_range().

Yanmin: I've been using the fadvise tool from
http://www.zip.com.au/~akpm/linux/patches/stuff/ext3-tools.tar.gz

Quoted 7 lines

It's a nice tool:

% fadvise
Usage: fadvise filename offset length advice [loops]
advice: normal sequential willneed noreuse dontneed asyncwrite writewait
% fadvise /var/sparse 0 0x7fffffff dontneed

I was a bit reluctant to point at that because it has nasty hacks to make it mostly-work on old glibc's which don't implement posix_fadvise().

Hopefully if you're running a recent distro, you have glibc support for fadvise() and it's possible to write a portable version of that app which doesn't need to know about per-arch syscall numbers.

#4 lore raw
Quoted 13 lines · Andrew Morton
Quoted 10 lines · On Thu, 28 Dec 2006 15:19:04 +0800 Fengguang Wu <[email protected]> wrote:

Yanmin: I've been using the fadvise tool from
http://www.zip.com.au/~akpm/linux/patches/stuff/ext3-tools.tar.gz

It's a nice tool:

% fadvise
Usage: fadvise filename offset length advice [loops]
advice: normal sequential willneed noreuse dontneed asyncwrite writewait
% fadvise /var/sparse 0 0x7fffffff dontneed

Quoted 6 lines

I was a bit reluctant to point at that because it has nasty hacks to make it mostly-work on old glibc's which don't implement posix_fadvise().

Hopefully if you're running a recent distro, you have glibc support for fadvise() and it's possible to write a portable version of that app which doesn't need to know about per-arch syscall numbers.

And note that if it gets implemented on ARM on pre-fadvise() glibc, the syscall argument order is rather non-standard: fd, action, start, size rather than fd, start, size, action - since otherwise we run out of registers with EABI.

The kernel community needs to get a grip with the implementation of new syscalls - we need a process where architecture maintainers get to review the arguments _prior_ to them being accepted into the kernel. That way we can avoid silly architecture specific syscall changes like this.

Signature · 3 lines
Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:
#5 lore raw
Quoted 5 lines · On Thu, 28 Dec 2006 10:45:08 +0000 Russell King <[email protected]>…

The kernel community needs to get a grip with the implementation of new syscalls - we need a process where architecture maintainers get to review the arguments _prior_ to them being accepted into the kernel. That way we can avoid silly architecture specific syscall changes like this.

hm, well, actually, sys_fadvise64_64 was discussed on linux-arch when it went in.

(Gad, it was over three years ago! - glibc support should be pretty widespread now)

That's about as much discussion as these things will get. And this assumes that people remember to mention it. Mental note made.

#6 lore raw
In reply to Andrew Morton
Quoted 14 lines · Andrew Morton
Quoted 10 lines · On Thu, 28 Dec 2006 15:19:04 +0800 Fengguang Wu <[email protected]> wrote:

Yanmin: I've been using the fadvise tool from
http://www.zip.com.au/~akpm/linux/patches/stuff/ext3-tools.tar.gz

It's a nice tool:

% fadvise
Usage: fadvise filename offset length advice [loops]
advice: normal sequential willneed noreuse dontneed asyncwrite writewait
% fadvise /var/sparse 0 0x7fffffff dontneed

Quoted 6 lines

I was a bit reluctant to point at that because it has nasty hacks to make it mostly-work on old glibc's which don't implement posix_fadvise().

Hopefully if you're running a recent distro, you have glibc support for fadvise() and it's possible to write a portable version of that app which doesn't need to know about per-arch syscall numbers.

Bad news: it's still broken. posix_fadvise() here just failed silently. I'm running Debian Etch with libc6=2.3.6.ds1-7.

#7 lore raw
In reply to Zhang, Yanmin
Quoted 15 lines · Yanmin

Currently, by /proc/sys/vm/drop_caches, applications could drop pagecache, slab(dentries and inodes), or both, but applications couldn't choose to just drop the page cache of one file. An user of VOD (Video-On-Demand) needs this capability to have more detailed control on page cache release.

Below patch against 2.6.19 implements it.

---

diff -Nraup linux-2.6.19/Documentation/filesystems/proc.txt linux-2.6.19_dropcache/Documentation/filesystems/proc.txt

Documentation/filesystems/proc.txt To free dentries and inodes: · line 1327

Quoted 8 lines
Documentation/filesystems/proc.txt +2 -0
@@ -1320,6 +1320,8 @@ To free dentries and inodes:
 	echo 2 > /proc/sys/vm/drop_caches
 To free pagecache, dentries and inodes:
 	echo 3 > /proc/sys/vm/drop_caches
+To free the pagecache of one file:
+	echo "4 /path/to/filename" > /proc/sys/vm/drop_caches
 
 As this is a non-destructive operation and dirty objects are not freeable, the
 user should run `sync' first.

"sync" is the most time consuming operation. Clean pagecache pages are immediately reclaimable... they are actually free pages. Writing out dirty pages consumes time.

Hence this approach may not provide the required performance benefits since only clean pagecache pages are marked free. fadvise approach would provide similar behavior.

--Vaidy

[snip]