Skip to content

fscache,netfs: decouple raw fscache APIs from libnetfs

Jingbo Xu linux-cifs linux-erofs linux-fsdevel linux-nfs lkml oe-kbuild-all 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 Latest9 patches 8 repliesIn review

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/9fscache: decouple prepare_read() from netfs_io_subrequestfs/cachefiles/io.c fs/erofs/fscache.c fs/netfs/io.c +2 more+40 -400 replies
2/9fscache,netfs: rename netfs_io_source as fscache_io_sourcefs/cachefiles/io.c fs/erofs/fscache.c fs/netfs/io.c +3 more+45 -451 reply
3/9fscache,netfs: rename netfs_io_terminated_t as fscache_io_terminated_tfs/cachefiles/internal.h fs/cachefiles/io.c fs/fscache/io.c +2 more+14 -141 reply
4/9fscache,netfs: rename netfs_read_from_hole as fscache_read_from_holefs/cachefiles/io.c fs/cifs/fscache.c fs/erofs/fscache.c +4 more+18 -181 reply
5/9fscache,netfs: rename netfs_cache_ops as fscache_opsfs/cachefiles/io.c include/linux/fscache.h include/linux/netfs.h+8 -81 reply
6/9fscache,netfs: rename netfs_cache_resources as fscache_resourcesfs/cachefiles/interface.c fs/cachefiles/internal.h fs/cachefiles/io.c +9 more+49 -491 reply
7/9fscache,netfs: define flags for prepare_read()fs/cachefiles/io.c fs/erofs/fscache.c fs/netfs/io.c +2 more+21 -121 reply
8/9fscache,netfs: move PageFsCache() family helpers to fscache.hinclude/linux/fscache.h include/linux/netfs.h+90 -891 reply
9/9fscache,netfs: move "fscache_" prefixed structures to fscache.hfs/afs/internal.h fs/erofs/fscache.c fs/nfs/fscache.h +2 more+83 -821 reply

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.

Read the v1 thread

Cover letter [PATCH 0/9] fscache,netfs: decouple raw fscache APIs from libnetfs

Git tree:

https://github.com/lostjeffle/linux.git jingbo/clean-fscache-v1-clean-netfs

Gitweb:

https://github.com/lostjeffle/linux/commits/jingbo/clean-fscache-v1-clean-netfs

[Rationale]
===========
Fscache has been landed as a generic caching management framework in the Linux kernel for decades. It aims to manage cache data availability or fetch data if needed. Currently it's mainly used for network fses, but in principle the main caching subsystem can be used more widely.

We do really like fscache framework and we believe it'd be better to reuse such framework if possible instead of duplicating other alternatives for better maintenance and testing. Therefore for our container image use cases, we applied the existing fscache to implement on-demand read for erofs in the past months. For more details, also see [1].

In short, here each erofs filesystem is composed of multiple blobs (or devices). Each blob corresponds to one fscache cookie to strictly follow on-disk format and implement the image downloading in a deterministic manner, which means it has a unique checksum and is signed by vendors.

Data of each erofs inode can be scattered among multiple blobs (cookie) since erofs supports chunk-level deduplication. In this case, each erofs inode can correspond to multiple cookies, and there's a logical to physical offset mapping between the logical offset in erofs inode and the physical offset in the backing file.

As described above, per-cookie netfs model can not be used here directly. Instead, we'd like to propose/decouple a simple set of raw fscache APIs, e.g. fscache_read(), to access cache for all fses to use. We believe it's useful since it's like the relationship between raw bio and iomap, both of which are useful for local fses. However, after fscache/netfs rework, libnetfs is preferred to access fscache, making fscache closely coupled with libnetfs.

Therefore, a more simple neutral raw fscache APIs is shown here which is independent to libnetfs for those who are not using libnetfs.

[Patchset Organization]
=======================
patch 1: decouple prepare_read() from netfs_io_subrequest

patch 2-9:
All structures related to cache accessing will be transformed with "fscache_" prefix, and defined in fscache namespace. The whole transition is divided into separate patches to facilitate code review, with each patch transforming one structure.

It is worth noting that the structure declaration is temporarily placed in netfs.h, and will be moved to fscache.h when all related structures are transformed to "fscache_" prefixed finally. The reason is that, in the intermediate state during the transition, the declarations of related structures are scattered among fscache.h and netfs.h. This will cause a bidirectional reference of these two headers, and compilation error then. As a work around, keep the declaration in netfs.h temporarily, until all structures are transformed and then moved to fscache.h (in patch 9).

[Following cleanup for erofs]
=============================
We will also cleanup erofs based on fscache_read() so that it won't rely on netfs internals anymore. For example, erofs can implement its own request completion routine, so that it can get rid of reliance on netfs_io_request and netfs_io_subrequest.

[1] https://lore.kernel.org/all/Yoj1AcHoBPqir++H@debian/

Jingbo Xu (9):
fscache: decouple prepare_read() from netfs_io_subrequest fscache,netfs: rename netfs_io_source as fscache_io_source fscache,netfs: rename netfs_io_terminated_t as fscache_io_terminated_t fscache,netfs: rename netfs_read_from_hole as fscache_read_from_hole fscache,netfs: rename netfs_cache_ops as fscache_ops fscache,netfs: rename netfs_cache_resources as fscache_resources fscache,netfs: define flags for prepare_read() fscache,netfs: move PageFsCache() family helpers to fscache.h fscache,netfs: move "fscache_" prefixed structures to fscache.h

 fs/afs/internal.h                 |   2 +-
 fs/cachefiles/interface.c         |   2 +-
 fs/cachefiles/internal.h          |   8 +-
 fs/cachefiles/io.c                |  84 ++++++------
 fs/cifs/fscache.c                 |   8 +-
 fs/erofs/fscache.c                |  17 ++-
 fs/fscache/io.c                   |  18 +--
 fs/netfs/buffered_read.c          |   2 +-
 fs/netfs/io.c                     |  67 +++++-----
 fs/nfs/fscache.c                  |   6 +-
 fs/nfs/fscache.h                  |   2 +-
 include/linux/fscache-cache.h     |   8 +-
 include/linux/fscache.h           | 211 +++++++++++++++++++++++++++---
 include/linux/netfs.h             | 173 +-----------------------
 include/trace/events/cachefiles.h |  27 ++--
 include/trace/events/netfs.h      |  14 +-
 16 files changed, 330 insertions(+), 319 deletions(-)
Signature · 1 line
2.19.1.6.gb485710b