CacheFS - general filesystem cache

Hi Linus, Andrew,

I've packaged my generic filesystem cache filesystem into patches and also produced patches for my AFS filesystem to use it. Work is also in progress to alter the NFS client use this interface too, and I think the ISO9660 filesystem could also benefit.

I've put a tarball of them on my Red Hat webpage because one of them is rather large. If you grab:

http://people.redhat.com/~dhowells/cachefs/cachefs-patches.tar.bz2

You'll find the following patches contained therein:

(1) cachefs-nullfileptr-2681mm4.diff

Fix a bug introduced into do_generic_mapping_read() by which a file pointer becomes required. I had arranged things so that it was optional so that I could call it directly on an inode.

(2) cachefs-rastateinit-2681mm4.diff

Revert the change made to file_ra_state_init() that removed the symbol export that I'd put in, and also make it fully initialise the state again.

This was also done so that do_generic_mapping_read() could be called directly on an inode.

(3) cachefs-fs-2681mm4.diff

The cachefs filesystem source and interface header file.

(4) cachefs-doc-2681mm4.diff

Documentation about using cachefs.

(5) cachefs-wppageop-2681mm4.diff

Add a notification operation to the address space that gets invoked when a read-only protected page is about to be made writable. I've shown this to Rik van Riel who seems happy with it.

Using a combination of (5) and (6), I can have cachefs write a netfs page into the cache in the background whilst the netfs readpage completes and passes the now unlocked page back to userspace.

Previously, the netfs readpage function had to be synchronous with respect to writing the page to disc. Now it can use this callback and prepare_write() to synchronise upon the page being modified in the page cache before it gets DMA'd to cachefs's disc block.

David Woodhouse is also interested in using this to do lazy reservations in JFFS2 shared writable mapping support.

(6) cachefs-fspagesync-2681mm4.diff

This adds another page flag for intra-filesystem synchronisation, to operate in a manner similar to PG_writeback.

I've made my AFS client use this to indicate that a page has now been written to the backing cache and can now be safely modified.

(7) cachefs-afs-2681mm4.diff

This changes my AFS client so that it can make use of cachefs.

I've also put a utility on my webpage for dumping the cachefs update journal contents:

http://people.redhat.com/~dhowells/cachefs/dump-journal.c

It just needs the fs/cachefs/cachefs-int.h header file placing in the same directory to compile.

David

Heh:

Three levels of indirection are currently supported:

- single indirection
- double indirection

somebody has trouble counting.

More seriously, I'd _really_ love to see something like a "swapfs", ie tmpfs with cachefs as a backing store. It would be a lot more useful for testing that AFS+cachefs, and would hopefully also act as a example of how to use it _without_ having to worry about AFS.

Is that possible?

Linus

<and the audience goes wild> whee :)

IMHO cachefs is a big deal particularly now that the NFSv4 delegation code has been merged.

For those unfamiliar with the changes from NFSv3 -> NFSv4, one of the many nice features of v4 is that it has sane caching. A file can be "delegated" to the client, such that, the client has complete control over the file including caching. Sorta like a lease. Great for cachefs, IOW. :)

Yay, a user :)

Jeff

Not really. I struck triple indirection off the list because I decided not to go that far yet. I haven't yet persuaded emacs to automatically update counting words in this situation:-)

Ramfs with on-disc caching, perhaps? :-)

I'd have to provide cache file pinning to prevent them being automatically recycled, but I want to do that anyway for other reasons.

It'd also be nice to be able to use it as emergency swap, but I think that might be a little tricky, though if you pin a file, give it a maximum size, and swap it on you could probably do that.

David

Umm, a tarball is the worst possible distribution method for patches. Either inline them in the mail or if they are too large provide a direct url to the patch.

Nah, they have rpms as well ... those are even worse ;-)

M.

Well, in cachefs's current state:

(1) That's mostly netfs policy. CacheFS is mainly for serving pages to the netfs upon demand. The netfs has to create a cachefs index tree suitable for it to be able to find its files. I'm currently arranging things in my AFS client and in the NFS client such that these correspond to server concepts such as cell address, volume ID, file ID; server IP address, file handle. I'm not arranging things as directories and files.

(2) The user can just add or remove caches with mount/umount, even when live; so you could find that your inodes that were being backed in the cache are suddenly unavailable. Caches could be locked in the mount tree as long as a netfs is using them for disconnected mode operation.

(3) CacheFS does not enforce any complete-file reading which you'd require to have disconnected operation. It only cares about pages. This is netfs policy.

(4) CacheFS does not provide pinning facilities, other than active cache inodes don't get eaten because there's too little space and we need to recycle something.

I'm planning on adding pinning facilities for use with kAFS. I'd like to add simple disconnected operation facilities too.

The file, and potentially the nest of directories in which it resides.

David