cachefiles,netfs: sunset ondemand mode
Subsystems (6)
David Howells
maintainerPATCH 1/1 cachefiles,netfs: sunset ondemand mode
From: Gao Xiang <[email protected]>
It was an effort to enhance fscache as a kernel cache for lazy pulling (at least according to previous Incremental FS discussion [1]) and EROFS over fscache was the in-tree user of this mode.
fscache has since evolved to be netfslib-oriented, serving network filesystem inodes via the netfs library, but EROFS never acts as a network filesystem and we need to cache golden filesystem images rather than individual EROFS inodes.
Since EROFS over fscache is now removed, clean up netfs/fscache/ cachefiles upstream too.
[1] https://lore.kernel.org/r/CAOQ4uxi4dzxArY24YO=+kBCK2gGoq3Ptb8WkzCqSogPgU_R3dQ@mail.gmail.com
[dh] Fixed up comments on:
https://sashiko.dev/#/patchset/20260716103030.3065561-1-dhowells%40redhat.com
https://sashiko.dev/#/patchset/20260722130218.78958-1-dhowells%40redhat.com
--- Documentation/filesystems/caching/cachefiles.rst | 179 ----- fs/cachefiles/Kconfig | 12 fs/cachefiles/Makefile | 1 fs/cachefiles/daemon.c | 96 -- fs/cachefiles/interface.c | 8 fs/cachefiles/internal.h | 134 ---- fs/cachefiles/io.c | 26 fs/cachefiles/namei.c | 8 fs/cachefiles/ondemand.c | 761 ----------------------- include/linux/netfs.h | 9 include/trace/events/cachefiles.h | 184 ----- include/uapi/linux/cachefiles.h | 68 -- kernel/bpf/btf.c | 7 13 files changed, 9 insertions(+), 1484 deletions(-)
Documentation/filesystems/caching/cachefiles.rst +0 -179
@@ -28,8 +28,6 @@ Cache on Already Mounted Filesystem (*) Debugging. - (*) On-demand Read. - Overview ========
@@ -483,180 +481,3 @@ the control file. For example:: echo $((1|4|8)) >/sys/module/cachefiles/parameters/debug will turn on all function entry debugging. - - -On-demand Read -============== - -When working in its original mode, CacheFiles serves as a local cache for a -remote networking fs - while in on-demand read mode, CacheFiles can boost the -scenario where on-demand read semantics are needed, e.g. container image -distribution. - -The essential difference between these two modes is seen when a cache miss -occurs: In the original mode, the netfs will fetch the data from the remote -server and then write it to the cache file; in on-demand read mode, fetching -the data and writing it into the cache is delegated to a user daemon. - -``CONFIG_CACHEFILES_ONDEMAND`` should be enabled to support on-demand read mode. - - -Protocol Communication ----------------------- - -The on-demand read mode uses a simple protocol for communication between kernel -and user daemon. The protocol can be modeled as:: - - kernel --[request]--> user daemon --[reply]--> kernel - -CacheFiles will send requests to the user daemon when needed. The user daemon -should poll the devnode ('/dev/cachefiles') to check if there's a pending -request to be processed. A POLLIN event will be returned when there's a pending -request. - -The user daemon then reads the devnode to fetch a request to process. It should -be noted that each read only gets one request. When it has finished processing -the request, the user daemon should write the reply to the devnode. - -Each request starts with a message header of the form:: - - struct cachefiles_msg { - __u32 msg_id; - __u32 opcode; - __u32 len; - __u32 object_id; - __u8 data[]; - }; - -where: - - * ``msg_id`` is a unique ID identifying this request among all pending - requests. - - * ``opcode`` indicates the type of this request. - - * ``object_id`` is a unique ID identifying the cache file operated on. - - * ``data`` indicates the payload of this request. - - * ``len`` indicates the whole length of this request, including the - header and following type-specific payload. - - -Turning on On-demand Mode -------------------------- - -An optional parameter becomes available to the "bind" command:: - - bind [ondemand] - -When the "bind" command is given no argument, it defaults to the original mode. -When it is given the "ondemand" argument, i.e. "bind ondemand", on-demand read -mode will be enabled. - - -The OPEN Request ----------------- - -When the netfs opens a cache file for the first time, a request with the -CACHEFILES_OP_OPEN opcode, a.k.a an OPEN request will be sent to the user -daemon. The payload format is of the form:: - - struct cachefiles_open { - __u32 volume_key_size; - __u32 cookie_key_size; - __u32 fd; - __u32 flags; - __u8 data[]; - }; - -where: - - * ``data`` contains the volume_key followed directly by the cookie_key. - The volume key is a NUL-terminated string; the cookie key is binary - data. - - * ``volume_key_size`` indicates the size of the volume key in bytes. - - * ``cookie_key_size`` indicates the size of the cookie key in bytes. - - * ``fd`` indicates an anonymous fd referring to the cache file, through - which the user daemon can perform write/llseek file operations on the - cache file. - - -The user daemon can use the given (volume_key, cookie_key) pair to distinguish -the requested cache file. With the given anonymous fd, the user daemon can -fetch the data and write it to the cache file in the background, even when -kernel has not triggered a cache miss yet. - -Be noted that each cache file has a unique object_id, while it may have multiple -anonymous fds. The user daemon may duplicate anonymous fds from the initial -anonymous fd indicated by the @fd field through dup(). Thus each object_id can -be mapped to multiple anonymous fds, while the usr daemon itself needs to -maintain the mapping. - -When implementing a user daemon, please be careful of RLIMIT_NOFILE, -``/proc/sys/fs/nr_open`` and ``/proc/sys/fs/file-max``. Typically these needn't -be huge since they're related to the number of open device blobs rather than -open files of each individual filesystem. - -The user daemon should reply the OPEN request by issuing a "copen" (complete -open) command on the devnode:: - - copen <msg_id>,<cache_size> - -where: - - * ``msg_id`` must match the msg_id field of the OPEN request. - - * When >= 0, ``cache_size`` indicates the size of the cache file; - when < 0, ``cache_size`` indicates any error code encountered by the - user daemon. - - -The CLOSE Request ------------------ - -When a cookie withdrawn, a CLOSE request (opcode CACHEFILES_OP_CLOSE) will be -sent to the user daemon. This tells the user daemon to close all anonymous fds -associated with the given object_id. The CLOSE request has no extra payload, -and shouldn't be replied. - - -The READ Request ----------------- - -When a cache miss is encountered in on-demand read mode, CacheFiles will send a -READ request (opcode CACHEFILES_OP_READ) to the user daemon. This tells the user -daemon to fetch the contents of the requested file range. The payload is of the -form:: - - struct cachefiles_read { - __u64 off; - __u64 len; - }; - -where: - - * ``off`` indicates the starting offset of the requested file range. - - * ``len`` indicates the length of the requested file range. - - -When it receives a READ request, the user daemon should fetch the requested data -and write it to the cache file identified by object_id. - -When it has finished processing the READ request, the user daemon should reply -by using the CACHEFILES_IOC_READ_COMPLETE ioctl on one of the anonymous fds -associated with the object_id given in the READ request. The ioctl is of the -form:: - - ioctl(fd, CACHEFILES_IOC_READ_COMPLETE, msg_id); - -where: - - * ``fd`` is one of the anonymous fds associated with the object_id - given. - - * ``msg_id`` must match the msg_id field of the READ request.
fs/cachefiles/Kconfig +0 -12
@@ -26,15 +26,3 @@ config CACHEFILES_ERROR_INJECTION help This permits error injection to be enabled in cachefiles whilst a cache is in service. - -config CACHEFILES_ONDEMAND - bool "Support for on-demand read" - depends on CACHEFILES - default n - help - This permits userspace to enable the cachefiles on-demand read mode. - In this mode, when a cache miss occurs, responsibility for fetching - the data lies with the cachefiles backend instead of with the netfs - and is delegated to userspace. - - If unsure, say N.
fs/cachefiles/Makefile +0 -1
@@ -16,6 +16,5 @@ cachefiles-y := \ xattr.o cachefiles-$(CONFIG_CACHEFILES_ERROR_INJECTION) += error_inject.o -cachefiles-$(CONFIG_CACHEFILES_ONDEMAND) += ondemand.o obj-$(CONFIG_CACHEFILES) := cachefiles.o
fs/cachefiles/daemon.c +7 -89
148 lines not on this page · raw message
fs/cachefiles/interface.c +0 -8
@@ -32,11 +32,6 @@ struct cachefiles_object *cachefiles_alloc_object(struct fscache_cookie *cookie)if (!object)return NULL ; -if (cachefiles_ondemand_init_obj_info (object, volume)) { -kmem_cache_free (cachefiles_object_jar, object); -return NULL ; - } -refcount_set (&object->ref,1 );spin_lock_init (&object->lock);
@@ -94,7 +89,6 @@ void cachefiles_put_object(struct cachefiles_object *object, ASSERTCMP(object->file, ==, NULL);kfree (object->d_name); -cachefiles_ondemand_deinit_obj_info (object); cache = object->volume->cache->cache;fscache_put_cookie (object->cookie, fscache_cookie_put_object); object->cookie =NULL ;
@@ -374,8 +368,6 @@ static void cachefiles_withdraw_cookie(struct fscache_cookie *cookie)spin_unlock (&cache->object_list_lock); } -cachefiles_ondemand_clean_object (object); -if (object->file) {cachefiles_begin_secure (cache, &saved_cred);cachefiles_clean_up_object (object, cache);
fs/cachefiles/internal.h +0 -134
184 lines not on this page · raw message
fs/cachefiles/io.c +1 -25
@@ -390,7 +390,6 @@ cachefiles_do_prepare_read(struct netfs_cache_resources *cres,size_t len = *_len;loff_t off, to;ino_t ino = file ?file_inode (file)->i_ino :0 ; -int rc;_enter ("%zx @%llx/%llx" , len, start, i_size);
@@ -403,8 +402,7 @@ cachefiles_do_prepare_read(struct netfs_cache_resources *cres,if (test_bit (FSCACHE_COOKIE_NO_DATA_TO_READ , &cookie->flags)) {__set_bit (NETFS_SREQ_COPY_TO_CACHE , _flags); why = cachefiles_trace_read_no_data; -if (!test_bit (NETFS_SREQ_ONDEMAND , _flags)) - goto out_no_object; + goto out_no_object; }/* The object and the file may be being created in the background. */
@@ -421,7 +419,6 @@ cachefiles_do_prepare_read(struct netfs_cache_resources *cres, object =cachefiles_cres_object (cres); cache = object->volume->cache;cachefiles_begin_secure (cache, &saved_cred); -retry: off =cachefiles_inject_read_error ();if (off ==0 ) off =vfs_llseek (file, start,SEEK_DATA );
@@ -474,14 +471,6 @@ cachefiles_do_prepare_read(struct netfs_cache_resources *cres, download_and_store:__set_bit (NETFS_SREQ_COPY_TO_CACHE , _flags); -if (test_bit (NETFS_SREQ_ONDEMAND , _flags)) { - rc =cachefiles_ondemand_read (object, start, len); -if (!rc) { -__clear_bit (NETFS_SREQ_ONDEMAND , _flags); - goto retry; - } - ret =NETFS_INVALID_READ ; - } out:cachefiles_end_secure (cache, saved_cred); out_no_object:
@@ -501,18 +490,6 @@ static enum netfs_io_source cachefiles_prepare_read(struct netfs_io_subrequest * &subreq->flags, subreq->rreq->inode->i_ino); } -/* -* Prepare an on-demand read operation, shortening it to a cached/uncached -* boundary as appropriate. -*/ -static enum netfs_io_source -cachefiles_prepare_ondemand_read (struct netfs_cache_resources *cres, -loff_t start,size_t *_len,loff_t i_size, -unsigned long *_flags,ino_t ino) -{ -return cachefiles_do_prepare_read (cres, start, _len, i_size, _flags, ino); -} -/* * Prepare for a write to occur. */
@@ -731,7 +708,6 @@ static const struct netfs_cache_ops cachefiles_netfs_cache_ops = { .prepare_read = cachefiles_prepare_read, .prepare_write = cachefiles_prepare_write, .prepare_write_subreq = cachefiles_prepare_write_subreq, - .prepare_ondemand_read = cachefiles_prepare_ondemand_read, .query_occupancy = cachefiles_query_occupancy, };
fs/cachefiles/namei.c +0 -8
@@ -442,10 +442,6 @@ struct file *cachefiles_create_tmpfile(struct cachefiles_object *object)if (!cachefiles_mark_inode_in_use (object,file_inode (file)))WARN_ON (1 ); - ret =cachefiles_ondemand_init_object (object); -if (ret <0 ) - goto err_unuse; - ni_size = object->cookie->object_size; ni_size =round_up (ni_size,CACHEFILES_DIO_BLOCK_SIZE );
@@ -546,10 +542,6 @@ static bool cachefiles_open_file(struct cachefiles_object *object, }_debug ("file -> %pd positive" , dentry); - ret =cachefiles_ondemand_init_object (object); -if (ret <0 ) - goto error_fput; - ret =cachefiles_check_auxdata (object, file);if (ret <0 ) goto check_failed;
fs/cachefiles/ondemand.c deleted +0 -761
761 lines not on this page · raw message
include/linux/netfs.h +0 -9
@@ -191,7 +191,6 @@ struct netfs_io_subrequest {#define NETFS_SREQ_COPY_TO_CACHE 0/* Set if should copy the data to the cache */ #define NETFS_SREQ_CLEAR_TAIL 1/* Set if the rest of the read should be cleared */ #define NETFS_SREQ_MADE_PROGRESS 4/* Set if we transferred at least some data */ -#define NETFS_SREQ_ONDEMAND 5/* Set if it's from on-demand read mode */ #define NETFS_SREQ_BOUNDARY 6/* Set if ends on hard boundary (eg. ceph object) */ #define NETFS_SREQ_HIT_EOF 7/* Set if short due to EOF */ #define NETFS_SREQ_IN_PROGRESS 8/* Unlocked when the subrequest completes */
@@ -374,14 +373,6 @@ struct netfs_cache_ops { loff_t *_start, size_t *_len, size_t upper_len, loff_t i_size, bool no_space_allocated_yet); -/* Prepare an on-demand read operation, shortening it to a cached/uncached -* boundary as appropriate. -*/ -enum netfs_io_source (*prepare_ondemand_read)(struct netfs_cache_resources *cres, -loff_t start,size_t *_len, -loff_t i_size, -unsigned long *_flags,ino_t ino); -/* Query the occupancy of the cache in a region, returning where the * next chunk of data starts and how long it is. */
include/trace/events/cachefiles.h +1 -183
202 lines not on this page · raw message
include/uapi/linux/cachefiles.h deleted +0 -68
68 lines not on this page · raw message
kernel/bpf/btf.c +0 -7
@@ -6676,13 +6676,6 @@ static const struct bpf_raw_tp_null_args raw_tp_null_args[] = { { "cachefiles_mark_inactive", 0x1 }, { "cachefiles_vfs_error", 0x1 }, { "cachefiles_io_error", 0x1 }, - { "cachefiles_ondemand_open", 0x1 }, - { "cachefiles_ondemand_copen", 0x1 }, - { "cachefiles_ondemand_close", 0x1 }, - { "cachefiles_ondemand_read", 0x1 }, - { "cachefiles_ondemand_cread", 0x1 }, - { "cachefiles_ondemand_fd_write", 0x1 }, - { "cachefiles_ondemand_fd_release", 0x1 },/* ext4, from ext4__mballoc event class */ { "ext4_mballoc_discard", 0x10 }, { "ext4_mballoc_free", 0x10 },
Quoted 10 lines · David Howells
It was an effort to enhance fscache as a kernel cache for lazy pulling (at least according to previous Incremental FS discussion [1]) and EROFS over fscache was the in-tree user of this mode.
fscache has since evolved to be netfslib-oriented, serving network filesystem inodes via the netfs library, but EROFS never acts as a network filesystem and we need to cache golden filesystem images rather than individual EROFS inodes.
[...]
Applied to the vfs-7.3.netfs branch of the vfs/vfs.git tree. Patches in the vfs-7.3.netfs branch should appear in linux-next soon.
Please report any outstanding bugs that were missed during review in a new review to the original patch series allowing us to drop it.
It's encouraged to provide Acked-bys and Reviewed-bys even though the patch has now been applied. If possible patch trailers will be updated.
Note that commit hashes shown below are subject to change due to rebase, trailer updates or similar. If in doubt, please check the listed branch.
tree: https://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs.git branch: vfs-7.3.netfs
[1/1] cachefiles,netfs: sunset ondemand mode
https://git.kernel.org/vfs/vfs/c/949eb95d5bcc
[email protected]
AI reviewThank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] Complete removal of the UAPI header include/uapi/linux/cachefiles.h breaks userspace compilation. --
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] Complete removal of the UAPI header include/uapi/linux/cachefiles.h breaks userspace compilation. --
--- Patch [1]: [PATCH] cachefiles,netfs: sunset ondemand mode --- commit e5845e932fdde67ddaf43301939e4efa8d4c2a07
Author: Gao Xiang <[email protected]>
cachefiles,netfs: sunset ondemand mode
This commit removes the ondemand read mode from cachefiles. Since EROFS over fscache is now removed and netfs has evolved, the ondemand mode is no longer needed and is being cleaned up.
[ ... ]
include/uapi/linux/cachefiles.h line 3
Quoted 9 lines
include/uapi/linux/cachefiles.h deleted +0
-3@@ -1,68 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */ -#ifndef _LINUX_CACHEFILES_H -#define _LINUX_CACHEFILES_H
[Severity: High]
Does this complete removal of the UAPI header break userspace compilation?
Even when a kernel feature is sunset and its runtime support is removed, standard Linux kernel policy is to retain the UAPI headers (or leave them as stubs) to avoid breaking userspace source compilation. Any existing userspace applications (such as ondemand cachefiles daemons) that include linux/cachefiles.h will fail to build against the updated kernel headers.
[ ... ]
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
[email protected]
AI reviewThis isn't a bug, but would it be worth turning those two review tool URLs into Link: tags with lore message-ids instead, or dropping the note before it lands? The rest of the message follows the…
Quoted 14 lines
commit
3d2befc6d8d7e85b20d92f4baab590a997635d81
Author: Gao Xiang <[email protected]>cachefiles,netfs: sunset ondemand mode
It was an effort to enhance fscache as a kernel cache for lazy pulling (at least according to previous Incremental FS discussion [1]) and EROFS over fscache was the in-tree user of this mode.
fscache has since evolved to be netfslib-oriented, serving network filesystem inodes via the netfs library, but EROFS never acts as a network filesystem and we need to cache golden filesystem images rather than individual EROFS inodes.
Quoted 8 lines
Since EROFS over fscache is now removed, clean up netfs/fscache/ cachefiles upstream too.
[1] https://lore.kernel.org/r/CAOQ4uxi4dzxArY24YO=+kBCK2gGoq3Ptb8WkzCqSogPgU_R3dQ@mail.gmail.com
[dh] Fixed up comments on:
https://sashiko.dev/#/patchset/20260716103030.3065561-1-dhowells%40redhat.com
https://sashiko.dev/#/patchset/20260722130218.78958-1-dhowells%40redhat.com
This isn't a bug, but would it be worth turning those two review tool URLs into Link: tags with lore message-ids instead, or dropping the note before it lands? The rest of the message follows the usual format, so this block reads as leftover process metadata.
[ ... ]
Quoted 4 lines
fs/cachefiles/internal.h +0
-0
[ ... ]
Quoted 13 lines
fs/cachefiles/internal.h +0
-9@@ -298,90 +250,6 @@ extern struct file *cachefiles_create_tmpfile(struct cachefiles_object *object);extern bool cachefiles_commit_tmpfile (struct cachefiles_cache *cache,struct cachefiles_object *object); -/* -* ondemand.c -*/ -#ifdef CONFIG_CACHEFILES_ONDEMAND -extern ssize_t cachefiles_ondemand_daemon_read (struct cachefiles_cache *cache, -char __user *_buffer,size_t buflen); - -extern int cachefiles_ondemand_copen (struct cachefiles_cache *cache, -char *args);
[ ... ]
fs/cachefiles/internal.h extern struct file *cachefiles_create_tmpfile(struct cachefiles_object *object); · line 318
Quoted 2 lines
fs/cachefiles/internal.h +0
-2-extern int cachefiles_ondemand_read (struct cachefiles_object *object, -loff_t pos,size_t len);
[ ... ondemand section removed ... ]
This isn't a bug, but now that ondemand.c is gone, io.c is the only user of __cachefiles_prepare_write() and __cachefiles_write(). Would it be worth making them static and dropping their declarations from internal.h?
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/31607781274