fscache,erofs: fscache-based demand-read semantics
Subsystems (5)
cover fscache,erofs: fscache-based demand-read semantics
changes since RFC:
- patch set organization:
patch 1-16 implement the data plane over fscache without demand reading support, while the demand reading support is left to patch 17-23.
- implement a new devnode ("/dev/cachefiles_demand") for the new mode (patch 1,20-22)
- use flag bit inside cache->flags to distinguish modes (patch 2) (David Howells)
- user daemon is responsible for placing blob files under corresponding fan sub directory, and setting "CacheFiles.cache" xattr in advance, so that cachefiles backend doesn't need modification to adapt to the new mode. (David Howells)
- erofs will allocate an anonymous inode for each backed file. This anonymous inode is responsible for managing page cache of backed file, so that netfs API doesn't need modification to adapt to the new mode. (Gao Xiang)
RFC: https://lore.kernel.org/all/[email protected]/t/
[Background]
============
erofs (Enhanced Read-Only File System) is a filesystem specially optimised for read-only scenarios. (Documentation/filesystem/erofs.rst)
Recently we are focusing on erofs in container images distribution scenario (https://sched.co/pcdL). In this case, erofs can be mounted from one bootstrap file (metadata) with (optional) multiple data blob files (data) stored on another local filesystem. (All these files are actually image files in erofs disk format.)
To accelerate the container startup (fetching container image from remote and then start the container), we do hope that the bootstrap blob file could support demand read. That is, erofs can be mounted and accessed even when the bootstrap/data blob files have not been fully downloaded.
That means we have to manage the cache state of the bootstrap/data blob files (if cache hit, read directly from the local cache; if cache miss, fetch the data somehow). It would be painful and may be dumb for erofs to implement the cache management itself. Thus we prefer fscache/cachefiles to do the cache management. Besides, the demand-read feature shall be general and it can benefit other using scenarios if it can be implemented in fscache level.
[Overall Design]
================
The upper fs uses a backing file on the local fs as the local cache (exactly the "cachefiles" way), and relies on fscache to detect if data is ready or not (cache hit/miss). Since currently fscache detects cache hit/miss by detecting the hole of the backing files, our demand-read mechanism also relies on the hole detecting.
1. initial phase
On the first beginning, the user daemon will touch the backing files (bootstrap/data blob files) under corresponding directory (under <root>/cache/<volume>/<fan>/) in advance. These backing files are completely sparse files (with zero disk usage). Since these backing files are all read-only and the file size is known prior mounting, user daemon will set corresponding file size and thus create all these sparse backing files in advance.
2. cache miss
When a file range (of bootstrap/data blob file) is accessed for the first time, a cache miss will be triggered and then .issue_op() will be called to fetch the data somehow.
In the demand-read case, we relies on a user daemon to fetch the data from local/remote. In this case, .issue_op() just packages the file range into a message and informs the user daemon. User daemon needs to poll and wait on the devnode (/dev/cachefiles_demand). Once awaken, the user daemon will read the devnode to get the file range information, and then fetch the data corresponding to the file range somehow, e.g. download from remote through network. Once data ready, the user daemon will write the fetched data into the backing file and then inform cachefiles backend by writing to the devnode. Cachefiles backend getting blocked on the previous .issue_op() calling will be awaken then. By then the data has been ready in the backing file, and the netfs API will re-initiate a read request from the backing file.
3. cache hit
Once data is already ready in the backing file, netfs API will read from the backing file directly.
[Advantage of fscache-based demand-read]
========================================
1. Asynchronous Prefetch
In current mechanism, fscache is responsible for cache state management, while the data plane (fetch data from local/remote on cache miss) is done on the user daemon side.
If data has already been ready in the backing file, netfs API will read from the backing file directly and won't be trapped to user space anymore. Thus the user daemon could fetch data (from remote) asynchronously on the background, and thus accelerate the backing file accessing in some degree.
2. Support massive blob files
Besides this mechanism supports a large amount of backing files, and thus can benefit the densely employed scenario.
In our using scenario, one container image can correspond to one bootstrap file (required) and multiple data blob files (optional). For example, one container image for node.js will corresponds to ~20 files in total. In densely employed environment, there could be as many as hundreds of containers and thus thousands of backing files on one machine.
[Test]
======
1. create erofs image (bootstrap)
mkfs.erofs --chunksize=1048576 --blobdev=Dblob1.img -Eforce-chunk-indexes Dtest.img tmp/
2. create sparse blob files under fscache root directory ("/root") truncate -s 28672 cache/Ierofs/@9c/Dtest.img
truncate -s 8040448 cache/Ierofs/@b5/Dblob1.img
3. set "CacheFiles.cache" xattr for blob files in advance (refer to
https://github.com/lostjeffle/demand-read-cachefilesd/blob/main/setxattr.c)
4. run user daemon
(https://github.com/lostjeffle/demand-read-cachefilesd/blob/main/cachefilesd2.c)
./cachefilesd2
5. mount erofs from bootstrap
mount -t erofs none -o uuid=test.img /mnt/
Jeffle Xu (23):
cachefiles: add cachefiles_demand devnode
cachefiles: add mode command to distinguish modes
cachefiles: detect backing file size in demand-read mode
netfs: make ops->init_rreq() optional
netfs: add inode parameter to netfs_alloc_read_request()
erofs: export erofs_map_blocks()
erofs: add nodev mode
erofs: register global fscache volume
erofs: add cookie context helper functions
erofs: add anonymous inode managing page cache of blob file
erofs: register cookie context for bootstrap
erofs: implement fscache-based metadata read
erofs: implement fscache-based data read
erofs: register cookie context for data blobs
erofs: implement fscache-based data read for data blobs
erofs: add 'uuid' mount option
netfs: support on demand read
cachefiles: use idr tree managing pending demand read
cachefiles: implement .demand_read() for demand read
cachefiles: implement .poll() for demand read
cachefiles: implement .read() for demand read
cachefiles: add done command for demand read
erofs: support on demand read
fs/cachefiles/daemon.c | 124 +++++++++++++++++ fs/cachefiles/internal.h | 18 +++ fs/cachefiles/io.c | 56 ++++++++ fs/cachefiles/main.c | 12 ++ fs/cachefiles/namei.c | 40 ++++++ fs/ceph/addr.c | 5 - fs/erofs/Makefile | 2 +- fs/erofs/data.c | 20 ++- fs/erofs/fscache.c | 292 +++++++++++++++++++++++++++++++++++++++ fs/erofs/inode.c | 6 +- fs/erofs/internal.h | 25 ++++ fs/erofs/super.c | 98 ++++++++++--- fs/netfs/read_helper.c | 41 +++++- include/linux/netfs.h | 4 + 14 files changed, 708 insertions(+), 35 deletions(-) create mode 100644 fs/erofs/fscache.c
Signature · 1 line
2.27.0
PATCH 1/23 cachefiles: add cachefiles_demand devnode
fscache/cachefiles used to serve as a local cache for remote fs. The following patches will introduce a new use case, in which local read-only fs could implement demand reading with fscache. By then the user daemon needs to read and poll on the devnode, and thus the original cachefiles devnode can't be reused in this case.
Thus create a new devnode specifically for the new mode. The following patches will add more file_operations.
--- fs/cachefiles/daemon.c | 8 ++++++++ fs/cachefiles/internal.h | 1 + fs/cachefiles/main.c | 12 ++++++++++++ 3 files changed, 21 insertions(+)
fs/cachefiles/daemon.c +8 -0
@@ -56,6 +56,14 @@ const struct file_operations cachefiles_daemon_fops = { .llseek = noop_llseek, }; +const struct file_operations cachefiles_demand_fops = { + .owner =THIS_MODULE , + .open = cachefiles_daemon_open, + .release = cachefiles_daemon_release, + .write = cachefiles_daemon_write, + .llseek = noop_llseek, +}; +struct cachefiles_daemon_cmd {char name[8 ];int (*handler)(struct cachefiles_cache *cache,char *args);
fs/cachefiles/internal.h +1 -0
@@ -145,6 +145,7 @@ extern int cachefiles_has_space(struct cachefiles_cache *cache,* daemon.c */ extern const struct file_operations cachefiles_daemon_fops; +extern const struct file_operations cachefiles_demand_fops;/* * error_inject.c
fs/cachefiles/main.c +12 -0
@@ -39,6 +39,12 @@ static struct miscdevice cachefiles_dev = { .fops = &cachefiles_daemon_fops, }; +static struct miscdevice cachefiles_demand_dev = { + .minor =MISC_DYNAMIC_MINOR , + .name ="cachefiles_demand" , + .fops = &cachefiles_demand_fops, +}; +/* * initialise the fs caching module */
@@ -52,6 +58,9 @@ static int __init cachefiles_init(void) ret =misc_register (&cachefiles_dev);if (ret <0 ) goto error_dev; + ret =misc_register (&cachefiles_demand_dev); +if (ret <0 ) + goto error_demand_dev;/* create an object jar */ ret = -ENOMEM ;
@@ -68,6 +77,8 @@ static int __init cachefiles_init(void)return 0 ; error_object_jar: +misc_deregister (&cachefiles_demand_dev); +error_demand_dev:misc_deregister (&cachefiles_dev); error_dev:cachefiles_unregister_error_injection ();
@@ -86,6 +97,7 @@ static void __exit cachefiles_exit(void)pr_info ("Unloading\n" );kmem_cache_destroy (cachefiles_object_jar); +misc_deregister (&cachefiles_demand_dev);misc_deregister (&cachefiles_dev);cachefiles_unregister_error_injection (); }
Signature · 1 line
2.27.0
Re: [PATCH v1 01/23] cachefiles: add cachefiles_demand devnode
Quoted 22 lines · Jeffle Xu
fscache/cachefiles used to serve as a local cache for remote fs. The following patches will introduce a new use case, in which local read-only fs could implement demand reading with fscache. By then the user daemon needs to read and poll on the devnode, and thus the original cachefiles devnode can't be reused in this case.
Thus create a new devnode specifically for the new mode. The following patches will add more
file_operations.Signed-off-by Jeffle Xu <[email protected]>--- fs/cachefiles/daemon.c | 8 ++++++++ fs/cachefiles/internal.h | 1 + fs/cachefiles/main.c | 12 ++++++++++++ 3 files changed, 21 insertions(+)fs/cachefiles/daemon.c +0
-0@@ -56,6 +56,14 @@ const struct file_operations cachefiles_daemon_fops = { .llseek = noop_llseek,
fs/cachefiles/daemon.c const struct file_operations cachefiles_daemon_fops = { · line 66
Quoted 10 lines
fs/cachefiles/daemon.c +8
-0}; +const struct file_operations cachefiles_demand_fops = { + .owner =THIS_MODULE , + .open = cachefiles_daemon_open, + .release = cachefiles_daemon_release, + .write = cachefiles_daemon_write, + .llseek = noop_llseek, +}; +
Better to prepare the on-demand read() and poll() first, and then add the on-demand cachefiles dev.
Thanks,
Joseph
Quoted 59 lines
fs/cachefiles/daemon.c +0
-0struct cachefiles_daemon_cmd {char name[8 ];int (*handler)(struct cachefiles_cache *cache,char *args);fs/cachefiles/internal.h +1
-0@@ -145,6 +145,7 @@ extern int cachefiles_has_space(struct cachefiles_cache *cache,* daemon.c */ extern const struct file_operations cachefiles_daemon_fops; +extern const struct file_operations cachefiles_demand_fops;/* * error_inject.c fs/cachefiles/main.c +12
-0@@ -39,6 +39,12 @@ static struct miscdevice cachefiles_dev = { .fops = &cachefiles_daemon_fops, }; +static struct miscdevice cachefiles_demand_dev = { + .minor =MISC_DYNAMIC_MINOR , + .name ="cachefiles_demand" , + .fops = &cachefiles_demand_fops, +}; +/* * initialise the fs caching module */ @@ -52,6 +58,9 @@ static int __init cachefiles_init(void) ret =misc_register (&cachefiles_dev);if (ret <0 ) goto error_dev; + ret =misc_register (&cachefiles_demand_dev); +if (ret <0 ) + goto error_demand_dev;/* create an object jar */ ret = -ENOMEM ;@@ -68,6 +77,8 @@ static int __init cachefiles_init(void)return 0 ; error_object_jar: +misc_deregister (&cachefiles_demand_dev); +error_demand_dev:misc_deregister (&cachefiles_dev); error_dev:cachefiles_unregister_error_injection ();@@ -86,6 +97,7 @@ static void __exit cachefiles_exit(void)pr_info ("Unloading\n" );kmem_cache_destroy (cachefiles_object_jar); +misc_deregister (&cachefiles_demand_dev);misc_deregister (&cachefiles_dev);cachefiles_unregister_error_injection (); }
Re: [PATCH v1 01/23] cachefiles: add cachefiles_demand devnode
Quoted 4 lines · Joseph Qi
Better to prepare the on-demand
read()andpoll()first, and then add the on-demand cachefiles dev.
Regards. Thanks.
Signature · 2 lines
Thanks, Jeffle
PATCH 2/23 cachefiles: add mode command to distinguish modes
In reply to Jeffle XuAdd one flag bit to distinguish the new introduced demand-read mode from the original mode. User daemon could set the specified mode with 'mode' command. If user daemon doesn't ever explicitly set the mode, then the behaviour is the same with that prior this patch, i.e. cachefiles serves as the local cache for remote fs by default.
--- fs/cachefiles/daemon.c | 32 ++++++++++++++++++++++++++++++++ fs/cachefiles/internal.h | 1 + 2 files changed, 33 insertions(+)
fs/cachefiles/daemon.c +32 -0
@@ -41,6 +41,7 @@ static int cachefiles_daemon_dir(struct cachefiles_cache *, char *);static int cachefiles_daemon_inuse (struct cachefiles_cache *,char *);static int cachefiles_daemon_secctx (struct cachefiles_cache *,char *);static int cachefiles_daemon_tag (struct cachefiles_cache *,char *); +static int cachefiles_daemon_mode (struct cachefiles_cache *,char *);static int cachefiles_daemon_bind (struct cachefiles_cache *,char *);static void cachefiles_daemon_unbind (struct cachefiles_cache *);
@@ -83,6 +84,7 @@ static const struct cachefiles_daemon_cmd cachefiles_daemon_cmds[] = { { "inuse", cachefiles_daemon_inuse }, { "secctx", cachefiles_daemon_secctx }, { "tag", cachefiles_daemon_tag }, + { "mode", cachefiles_daemon_mode }, { "", NULL } };
@@ -671,6 +673,36 @@ static int cachefiles_daemon_inuse(struct cachefiles_cache *cache, char *args)return -EINVAL ; } +/* +* Set the cache mode +* - command: "mode cache|demand" +*/ +static int cachefiles_daemon_mode (struct cachefiles_cache *cache,char *args) +{ +_enter (",%s" , args); + +if (test_bit (CACHEFILES_READY , &cache->flags)) { +pr_err ("Cache already started\n" ); +return -EINVAL ; + } + +if (!*args) { +pr_err ("Empty mode specified\n" ); +return -EINVAL ; + } + +if (!strncmp (args,"cache" ,strlen ("cache" ))) { +clear_bit (CACHEFILES_DEMAND_MODE , &cache->flags); + }else if (!strncmp (args,"demand" ,strlen ("demand" ))) { +set_bit (CACHEFILES_DEMAND_MODE , &cache->flags); + }else { +pr_err ("Invalid mode specified\n" ); +return -EINVAL ; + } + +return 0 ; +} +/* * Bind a directory as a cache */
fs/cachefiles/internal.h +1 -0
@@ -98,6 +98,7 @@ struct cachefiles_cache {#define CACHEFILES_DEAD 1/* T if cache dead */ #define CACHEFILES_CULLING 2/* T if cull engaged */ #define CACHEFILES_STATE_CHANGED 3/* T if state changed (poll trigger) */ +#define CACHEFILES_DEMAND_MODE 4/* T if works in demand read mode for read-only fs */ char *rootdirname;/* name of cache root directory */ char *secctx;/* LSM security context */ char *tag;/* cache binding tag */
Signature · 1 line
2.27.0
PATCH 3/23 cachefiles: detect backing file size in demand-read mode
In reply to Jeffle XuWhen upper read-only fs uses fscache for demand reading, it has no idea on the size of the backed file. (You need to input the file size as the @object_size parameter when calling fscache_acquire_cookie().)
In this using scenario, user daemon is responsible for preparing all backing files with correct file size (though they can be all sparse files), and the upper fs shall guarantee that past EOF access will never happen.
Then with this precondition, cachefiles can detect the actual size of backing file, and set it as the size of backed file.
--- fs/cachefiles/namei.c | 40 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+)
fs/cachefiles/namei.c +40 -0
@@ -511,9 +511,19 @@ struct file *cachefiles_create_tmpfile(struct cachefiles_object *object)*/ static bool cachefiles_create_file (struct cachefiles_object *object) { +struct cachefiles_cache *cache = object->volume->cache;struct file *file;int ret; +/* +* Demand read mode requires that backing files have been prepared with +* correct file size under corresponding directory. We can get here when +* the backing file doesn't exist under corresponding directory, or the +* file size is unexpected 0. +*/ +if (test_bit (CACHEFILES_DEMAND_MODE , &cache->flags)) +return false; + ret =cachefiles_has_space (object->volume->cache,1 ,0 , cachefiles_has_space_for_create);if (ret <0 )
@@ -530,6 +540,32 @@ static bool cachefiles_create_file(struct cachefiles_object *object)return true; } +/* +* Fs using fscache for demand reading may have no idea of the file size of +* backing files. Thus the demand read mode requires that backing files have +* been prepared with correct file size under corresponding directory. Then +* fscache backend is responsible for taking the file size of the backing file +* as the object size. +*/ +static int cachefiles_recheck_size (struct cachefiles_object *object, +struct file *file) +{ +loff_t size; +struct cachefiles_cache *cache = object->volume->cache; + +if (!test_bit (CACHEFILES_DEMAND_MODE , &cache->flags)) +return 0 ; + + size =i_size_read (file_inode (file)); +if (size) { + object->cookie->object_size = size; +return 0 ; + }else { +return -EINVAL ; + } + +} +/* * Open an existing file, checking its attributes and replacing it if it is * stale.
@@ -569,6 +605,10 @@ static bool cachefiles_open_file(struct cachefiles_object *object, }_debug ("file -> %pd positive" , dentry); + ret =cachefiles_recheck_size (object, file); +if (ret <0 ) + goto check_failed; + ret =cachefiles_check_auxdata (object, file);if (ret <0 ) goto check_failed;
Signature · 1 line
2.27.0
PATCH 4/23 netfs: make ops->init_rreq() optional
In reply to Jeffle XuThere's already upper fs implementing empty .init_rreq() callback, and thus make it optional.
--- fs/ceph/addr.c | 5 ----- fs/netfs/read_helper.c | 3 ++- 2 files changed, 2 insertions(+), 6 deletions(-)
fs/ceph/addr.c +0 -5
@@ -297,10 +297,6 @@ static void ceph_netfs_issue_op(struct netfs_read_subrequest *subreq)dout ("%s: result %d\n" , __func__, err); } -static void ceph_init_rreq (struct netfs_read_request *rreq,struct file *file) -{ -} -static void ceph_readahead_cleanup (struct address_space *mapping,void *priv) {struct inode *inode = mapping->host;
@@ -312,7 +308,6 @@ static void ceph_readahead_cleanup(struct address_space *mapping, void *priv) }static const struct netfs_read_request_ops ceph_netfs_read_ops = { - .init_rreq = ceph_init_rreq, .is_cache_enabled = ceph_is_cache_enabled, .begin_cache_operation = ceph_begin_cache_operation, .issue_op = ceph_netfs_issue_op,
fs/netfs/read_helper.c +2 -1
@@ -55,7 +55,8 @@ static struct netfs_read_request *netfs_alloc_read_request(INIT_WORK (&rreq->work, netfs_rreq_work);refcount_set (&rreq->usage,1 );__set_bit (NETFS_RREQ_IN_PROGRESS , &rreq->flags); - ops->init_rreq (rreq, file); +if (ops->init_rreq) + ops->init_rreq (rreq, file);netfs_stat (&netfs_n_rh_rreq); }
Signature · 1 line
2.27.0
PATCH 5/23 netfs: add inode parameter to netfs_alloc_read_request()
In reply to Jeffle XuWhen working as the local cache, the @file parameter of netfs_alloc_read_request() represents the backed file inside netfs. It is for two use: 1) we can derive the corresponding inode from file,
2) works as the argument for ops->init_rreq().
In the new introduced demand-read mode, netfs_readpage() will be called by the upper fs to read from backing files. However in this new mode, the backed file may not be opened, and thus the @file argument is NULL in this case.
For netfs_readpage(), @file parameter represents the backed file inside netfs, while @folio parameter represents one page cache inside the address space of this backed file. We can still derive the inode from the @folio parameter, even when @file parameter is NULL.
Thus refactor netfs_alloc_read_request() somewhat for this change.
--- fs/netfs/read_helper.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-)
fs/netfs/read_helper.c +7 -5
@@ -39,7 +39,7 @@ static void netfs_put_subrequest(struct netfs_read_subrequest *subreq,static struct netfs_read_request *netfs_alloc_read_request (const struct netfs_read_request_ops *ops,void *netfs_priv, -struct file *file) +struct inode *inode,struct file *file) {static atomic_t debug_ids;struct netfs_read_request *rreq;
@@ -48,7 +48,7 @@ static struct netfs_read_request *netfs_alloc_read_request(if (rreq) { rreq->netfs_ops = ops; rreq->netfs_priv = netfs_priv; - rreq->inode =file_inode (file); + rreq->inode = inode; rreq->i_size =i_size_read (rreq->inode); rreq->debug_id =atomic_inc_return (&debug_ids);INIT_LIST_HEAD (&rreq->subrequests);
@@ -870,6 +870,7 @@ void netfs_readahead(struct readahead_control *ractl, void *netfs_priv) {struct netfs_read_request *rreq; +struct inode *inode =file_inode (ractl->file);unsigned int debug_index =0 ;int ret;
@@ -878,7 +879,7 @@ void netfs_readahead(struct readahead_control *ractl,if (readahead_count (ractl) ==0 ) goto cleanup; - rreq =netfs_alloc_read_request (ops, netfs_priv, ractl->file); + rreq =netfs_alloc_read_request (ops, netfs_priv, inode, ractl->file);if (!rreq) goto cleanup; rreq->mapping = ractl->mapping;
@@ -948,12 +949,13 @@ int netfs_readpage(struct file *file, void *netfs_priv) {struct netfs_read_request *rreq; +struct inode *inode =folio_file_mapping (folio)->host;unsigned int debug_index =0 ;int ret;_enter ("%lx" ,folio_index (folio)); - rreq =netfs_alloc_read_request (ops, netfs_priv, file); + rreq =netfs_alloc_read_request (ops, netfs_priv, inode, file);if (!rreq) {if (netfs_priv) ops->cleanup (folio_file_mapping (folio), netfs_priv);
@@ -1122,7 +1124,7 @@ int netfs_write_begin(struct file *file, struct address_space *mapping, } ret = -ENOMEM ; - rreq =netfs_alloc_read_request (ops, netfs_priv, file); + rreq =netfs_alloc_read_request (ops, netfs_priv, inode, file);if (!rreq) goto error; rreq->mapping =folio_file_mapping (folio);
Signature · 1 line
2.27.0
Re: [PATCH v1 05/23] netfs: add inode parameter to netfs_alloc_read_request()
Quoted 10 lines · Jeffle Xu
When working as the local cache, the @file parameter of
netfs_alloc_read_request()represents the backed file inside netfs. It is for two use: 1) we can derive the corresponding inode from file,
2) works as the argument forops->init_rreq().In the new introduced demand-read mode,
netfs_readpage()will be called by the upper fs to read from backing files. However in this new mode, the backed file may not be opened, and thus the @file argument is NULL in this case.
Quoted 8 lines
For
netfs_readpage(), @file parameter represents the backed file inside netfs, while @folio parameter represents one page cache inside the address space of this backed file. We can still derive the inode from the @folio parameter, even when @file parameter is NULL.Thus refactor
netfs_alloc_read_request()somewhat for this change.Signed-off-by Jeffle Xu <[email protected]>
I'm not sure how other folks think.
Yet in principle, personally I think it's reasonable that something like read_cache_page_gfp() could be used directly with fscache backend as well.
So for such internal read requests, @file argument is actually optional as a common practice.
Apart from the commit message itself (which I think it could be simplified a bit), generally it looks good to me.
Thanks,
Gao Xiang
Quoted 69 lines
--- fs/netfs/read_helper.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-)fs/netfs/read_helper.c +7
-5@@ -39,7 +39,7 @@ static void netfs_put_subrequest(struct netfs_read_subrequest *subreq,static struct netfs_read_request *netfs_alloc_read_request (const struct netfs_read_request_ops *ops,void *netfs_priv, -struct file *file) +struct inode *inode,struct file *file) {static atomic_t debug_ids;struct netfs_read_request *rreq;@@ -48,7 +48,7 @@ static struct netfs_read_request *netfs_alloc_read_request(if (rreq) { rreq->netfs_ops = ops; rreq->netfs_priv = netfs_priv; - rreq->inode =file_inode (file); + rreq->inode = inode; rreq->i_size =i_size_read (rreq->inode); rreq->debug_id =atomic_inc_return (&debug_ids);INIT_LIST_HEAD (&rreq->subrequests);@@ -870,6 +870,7 @@ void netfs_readahead(struct readahead_control *ractl, void *netfs_priv) {struct netfs_read_request *rreq; +struct inode *inode =file_inode (ractl->file);unsigned int debug_index =0 ;int ret;@@ -878,7 +879,7 @@ void netfs_readahead(struct readahead_control *ractl,if (readahead_count (ractl) ==0 ) goto cleanup; - rreq =netfs_alloc_read_request (ops, netfs_priv, ractl->file); + rreq =netfs_alloc_read_request (ops, netfs_priv, inode, ractl->file);if (!rreq) goto cleanup; rreq->mapping = ractl->mapping;@@ -948,12 +949,13 @@ int netfs_readpage(struct file *file, void *netfs_priv) {struct netfs_read_request *rreq; +struct inode *inode =folio_file_mapping (folio)->host;unsigned int debug_index =0 ;int ret;_enter ("%lx" ,folio_index (folio)); - rreq =netfs_alloc_read_request (ops, netfs_priv, file); + rreq =netfs_alloc_read_request (ops, netfs_priv, inode, file);if (!rreq) {if (netfs_priv) ops->cleanup (folio_file_mapping (folio), netfs_priv);@@ -1122,7 +1124,7 @@ int netfs_write_begin(struct file *file, struct address_space *mapping, } ret = -ENOMEM ; - rreq =netfs_alloc_read_request (ops, netfs_priv, file); + rreq =netfs_alloc_read_request (ops, netfs_priv, inode, file);if (!rreq) goto error; rreq->mapping =folio_file_mapping (folio);Signature · 1 line
2.27.0
Re: [Linux-cachefs] [PATCH v1 05/23] netfs: add inode parameter to netfs_alloc_read_request()
In reply to Jeffle XuHi David,
What would you think about this cleanup? We need this in prep for the following fscache-based on-demand reading feature. It would be great if it could be cherry picked in advance.
I also simplify the commit message as suggested by Gao Xiang. I could resend a v2 patch with the updated commit message if you'd like.
netfs: add inode parameter to netfs_alloc_read_request()
Make the @file parameter optional, and derive inode from the @folio parameter instead.
@file parameter can't be removed completely, since it also works as the private data of ops->init_rreq().
Quoted 86 lines · Jeffle Xu
When working as the local cache, the @file parameter of
netfs_alloc_read_request()represents the backed file inside netfs. It is for two use: 1) we can derive the corresponding inode from file,
2) works as the argument forops->init_rreq().In the new introduced demand-read mode,
netfs_readpage()will be called by the upper fs to read from backing files. However in this new mode, the backed file may not be opened, and thus the @file argument is NULL in this case.For
netfs_readpage(), @file parameter represents the backed file inside netfs, while @folio parameter represents one page cache inside the address space of this backed file. We can still derive the inode from the @folio parameter, even when @file parameter is NULL.Thus refactor
netfs_alloc_read_request()somewhat for this change.Signed-off-by Jeffle Xu <[email protected]>--- fs/netfs/read_helper.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-)fs/netfs/read_helper.c +7
-5@@ -39,7 +39,7 @@ static void netfs_put_subrequest(struct netfs_read_subrequest *subreq,static struct netfs_read_request *netfs_alloc_read_request (const struct netfs_read_request_ops *ops,void *netfs_priv, -struct file *file) +struct inode *inode,struct file *file) {static atomic_t debug_ids;struct netfs_read_request *rreq;@@ -48,7 +48,7 @@ static struct netfs_read_request *netfs_alloc_read_request(if (rreq) { rreq->netfs_ops = ops; rreq->netfs_priv = netfs_priv; - rreq->inode =file_inode (file); + rreq->inode = inode; rreq->i_size =i_size_read (rreq->inode); rreq->debug_id =atomic_inc_return (&debug_ids);INIT_LIST_HEAD (&rreq->subrequests);@@ -870,6 +870,7 @@ void netfs_readahead(struct readahead_control *ractl, void *netfs_priv) {struct netfs_read_request *rreq; +struct inode *inode =file_inode (ractl->file);unsigned int debug_index =0 ;int ret;@@ -878,7 +879,7 @@ void netfs_readahead(struct readahead_control *ractl,if (readahead_count (ractl) ==0 ) goto cleanup; - rreq =netfs_alloc_read_request (ops, netfs_priv, ractl->file); + rreq =netfs_alloc_read_request (ops, netfs_priv, inode, ractl->file);if (!rreq) goto cleanup; rreq->mapping = ractl->mapping;@@ -948,12 +949,13 @@ int netfs_readpage(struct file *file, void *netfs_priv) {struct netfs_read_request *rreq; +struct inode *inode =folio_file_mapping (folio)->host;unsigned int debug_index =0 ;int ret;_enter ("%lx" ,folio_index (folio)); - rreq =netfs_alloc_read_request (ops, netfs_priv, file); + rreq =netfs_alloc_read_request (ops, netfs_priv, inode, file);if (!rreq) {if (netfs_priv) ops->cleanup (folio_file_mapping (folio), netfs_priv);@@ -1122,7 +1124,7 @@ int netfs_write_begin(struct file *file, struct address_space *mapping, } ret = -ENOMEM ; - rreq =netfs_alloc_read_request (ops, netfs_priv, file); + rreq =netfs_alloc_read_request (ops, netfs_priv, inode, file);if (!rreq) goto error; rreq->mapping =folio_file_mapping (folio);
Signature · 2 lines
Thanks, Jeffle
Re: [Linux-cachefs] [PATCH v1 05/23] netfs: add inode parameter to netfs_alloc_read_request()
Quoted 11 lines · JeffleXu
Hi David,
What would you think about this cleanup? We need this in prep for the following fscache-based on-demand reading feature. It would be great if it could be cherry picked in advance.
I also simplify the commit message as suggested by Gao Xiang. I could resend a v2 patch with the updated commit message if you'd like.
netfs: add inode parameter to
netfs_alloc_read_request()
Quoted 7 lines
Make the @file parameter optional, and derive inode from the @folio parameter instead.
@file parameter can't be removed completely, since it also works as the private data of
ops->init_rreq().Signed-off-by Jeffle Xu <[email protected]>
Hi,
IMHO, How about the following message:
netfs: make @file optional in netfs_alloc_read_request()
Make the @file parameter optional, and derive inode from the @folio parameter instead in order to support file system internal requests.
@file parameter can't be removed completely, since it also works as the private data of ops->init_rreq().
Thanks,
Gao Xiang
PATCH 6/23 erofs: export erofs_map_blocks()
In reply to Jeffle Xu... so that it can be used in the following introduced fs/erofs/fscache.c.
--- fs/erofs/data.c | 4 ++-- fs/erofs/internal.h | 2 ++ 2 files changed, 4 insertions(+), 2 deletions(-)
fs/erofs/data.c +2 -2
@@ -77,8 +77,8 @@ static int erofs_map_blocks_flatmode(struct inode *inode,return err; } -static int erofs_map_blocks (struct inode *inode, -struct erofs_map_blocks *map,int flags) +int erofs_map_blocks (struct inode *inode, +struct erofs_map_blocks *map,int flags) {struct super_block *sb = inode->i_sb;struct erofs_inode *vi =EROFS_I (inode);
fs/erofs/internal.h +2 -0
@@ -447,6 +447,8 @@ struct page *erofs_get_meta_page(struct super_block *sb, erofs_blk_t blkaddr);int erofs_map_dev (struct super_block *sb,struct erofs_map_dev *dev);int erofs_fiemap (struct inode *inode,struct fiemap_extent_info *fieinfo,u64 start,u64 len); +int erofs_map_blocks (struct inode *inode, +struct erofs_map_blocks *map,int flags);/* inode.c */ static inline unsigned long erofs_inode_hash (erofs_nid_t nid)
Signature · 1 line
2.27.0
PATCH 7/23 erofs: add nodev mode
In reply to Jeffle XuUntil then erofs is exactly blockdev based filesystem. In other using scenarios (e.g. container image), erofs needs to run upon files.
This patch introduces a new nodev mode, in which erofs could be mounted from a bootstrap blob file containing the complete erofs image.
The following patch will introduce a new mount option "uuid", by which users could specify the bootstrap blob file.
--- fs/erofs/data.c | 13 ++++++++--- fs/erofs/internal.h | 1 + fs/erofs/super.c | 56 +++++++++++++++++++++++++++++++++------------ 3 files changed, 53 insertions(+), 17 deletions(-)
fs/erofs/data.c +10 -3
@@ -11,11 +11,18 @@struct page *erofs_get_meta_page (struct super_block *sb,erofs_blk_t blkaddr) { -struct address_space *const mapping = sb->s_bdev->bd_inode->i_mapping; +struct address_space *mapping;struct page *page; - page =read_cache_page_gfp (mapping, blkaddr, -mapping_gfp_constraint (mapping, ~__GFP_FS)); +if (sb->s_bdev) { + mapping = sb->s_bdev->bd_inode->i_mapping; + page =read_cache_page_gfp (mapping, blkaddr, +mapping_gfp_constraint (mapping, ~__GFP_FS)); + }else { +/* TODO: data path in nodev mode */ + page =ERR_PTR (-EINVAL ); + } +/* should already be PageUptodate */ if (!IS_ERR (page))lock_page (page);
fs/erofs/internal.h +1 -0
@@ -67,6 +67,7 @@ struct erofs_mount_opts {unsigned int max_sync_decompress_pages;#endif unsigned int mount_opt; +char *uuid; };struct erofs_dev_context {
fs/erofs/super.c +42 -14
109 lines not on this page · raw message
Signature · 1 line
2.27.0
Re: [PATCH v1 07/23] erofs: add nodev mode
Quoted 10 lines · Jeffle Xu
Until then erofs is exactly blockdev based filesystem. In other using scenarios (e.g. container image), erofs needs to run upon files.
This patch introduces a new nodev mode, in which erofs could be mounted from a bootstrap blob file containing the complete erofs image.
The following patch will introduce a new mount option "uuid", by which users could specify the bootstrap blob file.
Signed-off-by Jeffle Xu <[email protected]>
I think the order of some patches in this patchset can be improved.
Take this patch as an example. This patch introduces a new mount option called "uuid", so the kernel will just accept it (which generates a user-visible impact) after this patch but it doesn't actually work.
Therefore, we actually have three different behaviors here:
- kernel doesn't support "uuid" mount option completely;
- kernel support "uuid" but it doesn't work;
- kernel support "uuid" correctly (maybe after some random patch);
Actually that is bad for bisecting since there are some commits having temporary behaviors. And we don't know which commit actually fully implements this "uuid" mount option.
So personally I think the proper order is just like the bottom-up approach, and make sure each patch can be tested / bisected independently.
Quoted 11 lines
--- fs/erofs/data.c | 13 ++++++++--- fs/erofs/internal.h | 1 + fs/erofs/super.c | 56 +++++++++++++++++++++++++++++++++------------ 3 files changed, 53 insertions(+), 17 deletions(-)fs/erofs/data.c +0
-0
fs/erofs/data.c line 18
Quoted 9 lines
fs/erofs/data.c +1
-3@@ -11,11 +11,18 @@struct page *erofs_get_meta_page (struct super_block *sb,erofs_blk_t blkaddr) { -struct address_space *const mapping = sb->s_bdev->bd_inode->i_mapping; +struct address_space *mapping;struct page *page; - page =read_cache_page_gfp (mapping, blkaddr, -mapping_gfp_constraint (mapping, ~__GFP_FS));
Apart from the recommendation above, if my understanding is correct, I think after we implement fscache_aops, read_cache_page_gfp() can work with proper fscache mapping.
So no need to implement something like erofs_readpage_from_fscache() later (at least for the case here.)
Thanks,
Gao Xiang
Quoted 9 lines
fs/erofs/data.c +9
-0+if (sb->s_bdev) { + mapping = sb->s_bdev->bd_inode->i_mapping; + page =read_cache_page_gfp (mapping, blkaddr, +mapping_gfp_constraint (mapping, ~__GFP_FS)); + }else { +/* TODO: data path in nodev mode */ + page =ERR_PTR (-EINVAL ); + } +
Re: [PATCH v1 07/23] erofs: add nodev mode
Quoted 24 lines · Gao Xiang
Quoted 10 lines · Jeffle Xu
Until then erofs is exactly blockdev based filesystem. In other using scenarios (e.g. container image), erofs needs to run upon files.
This patch introduces a new nodev mode, in which erofs could be mounted from a bootstrap blob file containing the complete erofs image.
The following patch will introduce a new mount option "uuid", by which users could specify the bootstrap blob file.
Signed-off-by Jeffle Xu <[email protected]>I think the order of some patches in this patchset can be improved.
Take this patch as an example. This patch introduces a new mount option called "uuid", so the kernel will just accept it (which generates a user-visible impact) after this patch but it doesn't actually work.
Therefore, we actually have three different behaviors here:
- kernel doesn't support "uuid" mount option completely;
- kernel support "uuid" but it doesn't work;
- kernel support "uuid" correctly (maybe after some random patch);
Quoted 7 lines
Actually that is bad for bisecting since there are some commits having temporary behaviors. And we don't know which commit actually fully implements this "uuid" mount option.
So personally I think the proper order is just like the bottom-up approach, and make sure each patch can be tested / bisected independently.
Oh, I may misread this patch, but I still think we'd better to avoid dead paths "TODO" like this as much as possible.
Just do in the bottom-up way.
Thanks,
Gao Xiang
Re: [PATCH v1 07/23] erofs: add nodev mode
Quoted 29 lines · Gao Xiang
Quoted 28 lines · Gao Xiang
Quoted 10 lines · Jeffle Xu
Until then erofs is exactly blockdev based filesystem. In other using scenarios (e.g. container image), erofs needs to run upon files.
This patch introduces a new nodev mode, in which erofs could be mounted from a bootstrap blob file containing the complete erofs image.
The following patch will introduce a new mount option "uuid", by which users could specify the bootstrap blob file.
Signed-off-by Jeffle Xu <[email protected]>I think the order of some patches in this patchset can be improved.
Take this patch as an example. This patch introduces a new mount option called "uuid", so the kernel will just accept it (which generates a user-visible impact) after this patch but it doesn't actually work.
Therefore, we actually have three different behaviors here:
- kernel doesn't support "uuid" mount option completely;
- kernel support "uuid" but it doesn't work;
- kernel support "uuid" correctly (maybe after some random patch);Actually that is bad for bisecting since there are some commits having temporary behaviors. And we don't know which commit actually fully implements this "uuid" mount option.
Quoted 9 lines
Quoted 3 lines
So personally I think the proper order is just like the bottom-up approach, and make sure each patch can be tested / bisected independently.
Oh, I may misread this patch, but I still think we'd better to avoid dead paths "TODO" like this as much as possible.
Just do in the bottom-up way.
OK, it is better to be put at the latter part of the whole patch set. Would be fixed in the next version. Thanks.
Signature · 2 lines
Thanks, Jeffle
PATCH 8/23 erofs: register global fscache volume
In reply to Jeffle XuAll erofs instances will share one global fscache volume.
In this using scenario, one erofs instance could be mounted from one (or multiple) blob files instead of blkdev. The number of blob files that each erofs instance could correspond to is limited, since these blob files are quite large in size. For example, when used for container image distribution, one erofs instance used for container image for node.js will correspond to ~20 blob files in total. Thus in densely employed environment, there could be as many as hundreds of containers and thus thousands of fscache cookies under one fscache volume.
Then as for cachefiles backend, the hash table managing all cookies under one volume contains 32K slots. Thus the hashing functionality shall scale well in this case. Besides, cachefiles backend will scatter backing files under 256 fan sub-directoris, and thus the scalability of looking up backing files shall also not be an issue.
--- fs/erofs/Makefile | 2 +- fs/erofs/fscache.c | 21 +++++++++++++++++++++ fs/erofs/internal.h | 5 +++++ fs/erofs/super.c | 7 +++++++ 4 files changed, 34 insertions(+), 1 deletion(-) create mode 100644 fs/erofs/fscache.c
fs/erofs/Makefile +1 -1
@@ -1,7 +1,7 @@ # SPDX-License-Identifier: GPL-2.0-only obj-$(CONFIG_EROFS_FS) += erofs.o -erofs-objs := super.o inode.o data.o namei.o dir.o utils.o pcpubuf.o +erofs-objs := super.o inode.o data.o namei.o dir.o utils.o pcpubuf.o fscache.o erofs-$(CONFIG_EROFS_FS_XATTR) += xattr.o erofs-$(CONFIG_EROFS_FS_ZIP) += decompressor.o zmap.o zdata.o erofs-$(CONFIG_EROFS_FS_ZIP_LZMA) += decompressor_lzma.o
fs/erofs/fscache.c new file +21 -0
@@ -0,0 +1,21 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* +* Copyright (C) 2021, Alibaba Cloud +*/ +#include "internal.h" + +static struct fscache_volume *volume; + +int __init erofs_fscache_init (void ) +{ + volume =fscache_acquire_volume ("erofs" ,NULL ,NULL ,0 ); +if (!volume) +return -EINVAL ; + +return 0 ; +} + +void erofs_fscache_cleanup (void ) +{ +fscache_relinquish_volume (volume,NULL , false); +}
fs/erofs/internal.h +5 -0
@@ -17,6 +17,7 @@#include <linux/slab.h> #include <linux/vmalloc.h> #include <linux/iomap.h> +#include <linux/fscache.h> #include "erofs_fs.h" /* redefine pr_fmt "erofs: " */
@@ -569,6 +570,10 @@ static inline int z_erofs_load_lzma_config(struct super_block *sb, }#endif /* !CONFIG_EROFS_FS_ZIP */ +/* fscache.c */ +int erofs_fscache_init (void ); +void erofs_fscache_cleanup (void ); +#define EFSCORRUPTED EUCLEAN/* Filesystem is corrupted */ #endif /* __EROFS_INTERNAL_H */
fs/erofs/super.c +7 -0
@@ -876,6 +876,10 @@ static int __init erofs_module_init(void)if (err) goto zip_err; + err =erofs_fscache_init (); +if (err) + goto fscache_err; + err =register_filesystem (&erofs_fs_type);if (err) goto fs_err;
@@ -883,6 +887,8 @@ static int __init erofs_module_init(void)return 0 ; fs_err: +erofs_fscache_cleanup (); +fscache_err:z_erofs_exit_zip_subsystem (); zip_err:z_erofs_lzma_exit ();
@@ -901,6 +907,7 @@ static void __exit erofs_module_exit(void)/* Ensure all RCU free inodes / pclusters are safe to be destroyed. */ rcu_barrier (); +erofs_fscache_cleanup ();z_erofs_exit_zip_subsystem ();z_erofs_lzma_exit ();erofs_exit_shrinker ();
Signature · 1 line
2.27.0
PATCH 9/23 erofs: add cookie context helper functions
In reply to Jeffle XuIntroduce 'struct erofs_cookie_ctx' for managing cookie for backing file, and the following introduced API for reading from backing file.
Besides, introduce two helper functions for initializing and cleaning up erofs_cookie_ctx.
struct erofs_cookie_ctx * erofs_fscache_get_ctx(struct super_block *sb, char *path);
void erofs_fscache_put_ctx(struct erofs_cookie_ctx *ctx);
--- fs/erofs/fscache.c | 75 +++++++++++++++++++++++++++++++++++++++++++++ fs/erofs/internal.h | 8 +++++ 2 files changed, 83 insertions(+)
fs/erofs/fscache.c +75 -0
@@ -6,6 +6,81 @@static struct fscache_volume *volume; +static int erofs_fscache_init_cookie (struct erofs_cookie_ctx *ctx,char *path) +{ +struct fscache_cookie *cookie; + +/* +* @object_size shall be non-zero to avoid +* FSCACHE_COOKIE_NO_DATA_TO_READ. +*/ + cookie =fscache_acquire_cookie (volume,0 , + path,strlen (path), +NULL ,0 ,-1 ); +if (!cookie) +return -EINVAL ; + +fscache_use_cookie (cookie, false); + ctx->cookie = cookie; +return 0 ; +} + +static void erofs_fscache_cleanup_cookie (struct erofs_cookie_ctx *ctx) +{ +struct fscache_cookie *cookie = ctx->cookie; + +fscache_unuse_cookie (cookie,NULL ,NULL ); +fscache_relinquish_cookie (cookie, false); + ctx->cookie =NULL ; +} + +static int erofs_fscahce_init_ctx (struct erofs_cookie_ctx *ctx, +struct super_block *sb,char *path) +{ +int ret; + + ret =erofs_fscache_init_cookie (ctx, path); +if (ret) { +erofs_err (sb,"failed to init cookie\n" ); +return ret; + } + +return 0 ; +} + +static void erofs_fscache_cleanup_ctx (struct erofs_cookie_ctx *ctx) +{ +erofs_fscache_cleanup_cookie (ctx); +} + +struct erofs_cookie_ctx *erofs_fscache_get_ctx (struct super_block *sb, +char *path) +{ +struct erofs_cookie_ctx *ctx; +int ret; + + ctx =kzalloc (sizeof (*ctx),GFP_KERNEL ); +if (!ctx) +return ERR_PTR (-ENOMEM ); + + ret =erofs_fscahce_init_ctx (ctx, sb, path); +if (ret) { +kfree (ctx); +return ERR_PTR (ret); + } + +return ctx; +} + +void erofs_fscache_put_ctx (struct erofs_cookie_ctx *ctx) +{ +if (!ctx) +return ; + +erofs_fscache_cleanup_ctx (ctx); +kfree (ctx); +} +int __init erofs_fscache_init (void ) { volume =fscache_acquire_volume ("erofs" ,NULL ,NULL ,0 );
fs/erofs/internal.h +8 -0
@@ -91,6 +91,10 @@ struct erofs_sb_lz4_info {u16 max_pclusterblks; }; +struct erofs_cookie_ctx { +struct fscache_cookie *cookie; +}; +struct erofs_sb_info {struct erofs_mount_opts opt;/* options */ #ifdef CONFIG_EROFS_FS_ZIP
@@ -574,6 +578,10 @@ static inline int z_erofs_load_lzma_config(struct super_block *sb,int erofs_fscache_init (void );void erofs_fscache_cleanup (void ); +struct erofs_cookie_ctx *erofs_fscache_get_ctx (struct super_block *sb, +char *path); +void erofs_fscache_put_ctx (struct erofs_cookie_ctx *ctx); +#define EFSCORRUPTED EUCLEAN/* Filesystem is corrupted */ #endif /* __EROFS_INTERNAL_H */
Signature · 1 line
2.27.0
PATCH 10/23 erofs: add anonymous inode managing page cache of blob file
In reply to Jeffle XuIntroduce one anonymous inode for managing page cache of corresponding blob file. Then erofs could read directly from the address space of the anonymous inode when cache hit.
--- fs/erofs/fscache.c | 35 +++++++++++++++++++++++++++++++++++ fs/erofs/internal.h | 1 + 2 files changed, 36 insertions(+)
fs/erofs/fscache.c +35 -0
@@ -34,6 +34,33 @@ static void erofs_fscache_cleanup_cookie(struct erofs_cookie_ctx *ctx) ctx->cookie =NULL ; } +static const struct address_space_operations erofs_fscache_aops = { +}; + +static int erofs_fscache_get_inode (struct erofs_cookie_ctx *ctx, +struct super_block *sb) +{ +struct inode *const inode =new_inode (sb); + +if (!inode) +return -ENOMEM ; + +set_nlink (inode,1 ); + inode->i_size =OFFSET_MAX ; + + inode->i_mapping->a_ops = &erofs_fscache_aops; +mapping_set_gfp_mask (inode->i_mapping, +GFP_NOFS | __GFP_HIGHMEM | __GFP_MOVABLE); + ctx->inode = inode; +return 0 ; +} + +static void erofs_fscache_put_inode (struct erofs_cookie_ctx *ctx) +{ +iput (ctx->inode); + ctx->inode =NULL ; +} +static int erofs_fscahce_init_ctx (struct erofs_cookie_ctx *ctx,struct super_block *sb,char *path) {
@@ -45,12 +72,20 @@ static int erofs_fscahce_init_ctx(struct erofs_cookie_ctx *ctx,return ret; } + ret =erofs_fscache_get_inode (ctx, sb); +if (ret) { +erofs_err (sb,"failed to get anonymous inode\n" ); +erofs_fscache_cleanup_cookie (ctx); +return ret; + } +return 0 ; }static void erofs_fscache_cleanup_ctx (struct erofs_cookie_ctx *ctx) {erofs_fscache_cleanup_cookie (ctx); +erofs_fscache_put_inode (ctx); } struct erofs_cookie_ctx *erofs_fscache_get_ctx(struct super_block *sb,
fs/erofs/internal.h +1 -0
@@ -93,6 +93,7 @@ struct erofs_sb_lz4_info {struct erofs_cookie_ctx {struct fscache_cookie *cookie; +struct inode *inode; };struct erofs_sb_info {
Signature · 1 line
2.27.0
PATCH 11/23 erofs: register cookie context for bootstrap
In reply to Jeffle XuSomething worth mentioning about the cleanup routine.
1. The init routine is prior to when the root inode gets initialized, and thus the corresponding cleanup routine shall be placed under .kill_sb() callback.
2. The init routine will instantiate anonymous inodes under the super_block, and thus .put_super() callback shall also contain the cleanup routine. Or we'll get "VFS: Busy inodes after unmount." warning.
--- fs/erofs/internal.h | 2 ++ fs/erofs/super.c | 13 +++++++++++++ 2 files changed, 15 insertions(+)
fs/erofs/internal.h +2 -0
@@ -141,6 +141,8 @@ struct erofs_sb_info {u8 volume_name[16 ];/* volume name */ u32 feature_compat;u32 feature_incompat; + +struct erofs_cookie_ctx *bootstrap; };#define EROFS_SB (sb) ((struct erofs_sb_info *)(sb)->s_fs_info)
fs/erofs/super.c +13 -0
@@ -663,6 +663,16 @@ static int erofs_fc_fill_super(struct super_block *sb, struct fs_context *fc)else sbi->dax_dev =NULL ; +if (!sb->s_bdev) { +struct erofs_cookie_ctx *bootstrap; + + bootstrap =erofs_fscache_get_ctx (sb, ctx->opt.uuid); +if (IS_ERR (bootstrap)) +return PTR_ERR (bootstrap); + + sbi->bootstrap = bootstrap; + } + err =erofs_read_superblock (sb);if (err)return err;
@@ -820,6 +830,7 @@ static void erofs_kill_sb(struct super_block *sb)return ;erofs_free_dev_context (sbi->devs); +erofs_fscache_put_ctx (sbi->bootstrap);fs_put_dax (sbi->dax_dev);kfree (sbi); sb->s_fs_info =NULL ;
@@ -837,6 +848,8 @@ static void erofs_put_super(struct super_block *sb)iput (sbi->managed_cache); sbi->managed_cache =NULL ;#endif +erofs_fscache_put_ctx (sbi->bootstrap); + sbi->bootstrap =NULL ; }static struct file_system_type erofs_fs_type = {
Signature · 1 line
2.27.0
PATCH 12/23 erofs: implement fscache-based metadata read
In reply to Jeffle XuThis patch implements the data plane of reading metadata from bootstrap blob file over fscache.
Be noted that currently it only supports the scenario where the backing file has no hole. Once it hits a hole of the backing file, erofs will fail the IO with -EOPNOTSUPP for now. The following patch will fix this issue, i.e. implementing the demand reading mode.
--- fs/erofs/data.c | 4 +-- fs/erofs/fscache.c | 59 +++++++++++++++++++++++++++++++++++++++++++++ fs/erofs/internal.h | 3 +++ fs/erofs/super.c | 7 ++---- 4 files changed, 66 insertions(+), 7 deletions(-)
fs/erofs/data.c +2 -2
@@ -19,8 +19,8 @@ struct page *erofs_get_meta_page(struct super_block *sb, erofs_blk_t blkaddr) page =read_cache_page_gfp (mapping, blkaddr,mapping_gfp_constraint (mapping, ~__GFP_FS)); }else { -/* TODO: data path in nodev mode */ - page =ERR_PTR (-EINVAL ); + page =erofs_readpage_from_fscache (EROFS_SB (sb)->bootstrap, + blkaddr); }/* should already be PageUptodate */
fs/erofs/fscache.c +59 -0
65 lines not on this page · raw message
fs/erofs/internal.h +3 -0
@@ -585,6 +585,9 @@ struct erofs_cookie_ctx *erofs_fscache_get_ctx(struct super_block *sb, char *path);void erofs_fscache_put_ctx (struct erofs_cookie_ctx *ctx); +struct page *erofs_readpage_from_fscache (struct erofs_cookie_ctx *ctx, +pgoff_t index); +#define EFSCORRUPTED EUCLEAN/* Filesystem is corrupted */ #endif /* __EROFS_INTERNAL_H */
fs/erofs/super.c +2 -5
@@ -334,25 +334,22 @@ static int erofs_init_devices(struct super_block *sb,static int erofs_read_superblock (struct super_block *sb) { -struct erofs_sb_info *sbi; +struct erofs_sb_info *sbi =EROFS_SB (sb);struct page *page;struct erofs_super_block *dsb;unsigned int blkszbits;void *data;int ret; -/* TODO: metadata path in nodev mode */ if (sb->s_bdev) page =read_mapping_page (sb->s_bdev->bd_inode->i_mapping,0 ,NULL );else - page =ERR_PTR (-EOPNOTSUPP ); + page =erofs_readpage_from_fscache (sbi->bootstrap,0 );if (IS_ERR (page)) {erofs_err (sb,"cannot read erofs superblock" );return PTR_ERR (page); } - sbi =EROFS_SB (sb); - data =kmap (page); dsb = (struct erofs_super_block *)(data +EROFS_SUPER_OFFSET );
Signature · 1 line
2.27.0
kernel test robot
build botRe: [PATCH v1 12/23] erofs: implement fscache-based metadata read
Thank you for the patch! Yet something to improve:
Hi Jeffle,
Thank you for the patch! Yet something to improve:
[auto build test ERROR on dhowells-fs/fscache-next]
[cannot apply to xiang-erofs/dev-test ceph-client/for-linus linus/master v5.16-rc7 next-20211224] [If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/Jeffle-Xu/fscache-erofs-fscache-based-demand-read-semantics/20211227-205742 base: https://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs.git fscache-next config: m68k-defconfig (https://download.01.org/0day-ci/archive/20211228/202112280115.O0H8Ow1Q-lkp(a)intel.com/config) compiler: m68k-linux-gcc (GCC) 11.2.0
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# https://github.com/0day-ci/linux/commit/c3453b91df3b4e89c3336453437f761d6cb6bca3
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Jeffle-Xu/fscache-erofs-fscache-based-demand-read-semantics/20211227-205742 git checkout c3453b91df3b4e89c3336453437f761d6cb6bca3
# save the config file to linux build tree
mkdir build_dir
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-11.2.0 make.cross O=build_dir ARCH=m68k SHELL=/bin/bash
If you fix the issue, kindly add following tag as appropriate
All errors (new ones prefixed by >>, old ones prefixed by <<):
ERROR: modpost: "netfs_subreq_terminated" [fs/erofs/erofs.ko] undefined!
>> ERROR: modpost: "netfs_readpage" [fs/erofs/erofs.ko] undefined!
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org
kernel test robot
build botRe: [PATCH v1 12/23] erofs: implement fscache-based metadata read
In reply to Jeffle XuThank you for the patch! Yet something to improve:
Hi Jeffle,
Thank you for the patch! Yet something to improve:
[auto build test ERROR on dhowells-fs/fscache-next]
[cannot apply to xiang-erofs/dev-test ceph-client/for-linus linus/master v5.16-rc7 next-20211224] [If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/Jeffle-Xu/fscache-erofs-fscache-based-demand-read-semantics/20211227-205742 base: https://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs.git fscache-next config: x86_64-randconfig-a014-20211227 (https://download.01.org/0day-ci/archive/20211228/202112280132.0kJ8Vjql-lkp(a)intel.com/config) compiler: gcc-9 (Debian 9.3.0-22) 9.3.0 reproduce (this is a W=1 build): # https://github.com/0day-ci/linux/commit/c3453b91df3b4e89c3336453437f761d6cb6bca3 git remote add linux-review https://github.com/0day-ci/linux git fetch --no-tags linux-review Jeffle-Xu/fscache-erofs-fscache-based-demand-read-semantics/20211227-205742 git checkout c3453b91df3b4e89c3336453437f761d6cb6bca3 # save the config file to linux build tree mkdir build_dir make W=1 O=build_dir ARCH=x86_64 SHELL=/bin/bash
If you fix the issue, kindly add following tag as appropriate
All errors (new ones prefixed by >>):
vmlinux.o: warning: objtool: do_machine_check()+0x89f: call to queue_task_work() leaves .noinstr.text section vmlinux.o: warning: objtool: enter_from_user_mode()+0x4e: call to on_thread_stack() leaves .noinstr.text section vmlinux.o: warning: objtool: syscall_enter_from_user_mode()+0x59: call to on_thread_stack() leaves .noinstr.text section vmlinux.o: warning: objtool: syscall_enter_from_user_mode_prepare()+0x4e: call to on_thread_stack() leaves .noinstr.text section vmlinux.o: warning: objtool: irqentry_enter_from_user_mode()+0x4e: call to on_thread_stack() leaves .noinstr.text section
ld: fs/erofs/fscache.o: in function `erofs_issue_op':
fs/erofs/fscache.c:27: undefined reference to `netfs_subreq_terminated'
ld: fs/erofs/fscache.o: in function `erofs_readpage_from_fscache':
>> fs/erofs/fscache.c:59: undefined reference to `netfs_readpage'
vim +59 fs/erofs/fscache.c
35
36 struct page *erofs_readpage_from_fscache(struct erofs_cookie_ctx *ctx,
37 pgoff_t index)
38 {
39 struct folio *folio;
40 struct page *page;
41 struct super_block *sb = ctx->inode->i_sb;
42 int ret;
43
44 page = find_or_create_page(ctx->inode->i_mapping, index, GFP_KERNEL);
45 if (unlikely(!page)) {
46 erofs_err(sb, "failed to allocate page");
47 return ERR_PTR(-ENOMEM);
48 }
49
50 /* The content is already buffered in the address space */
51 if (PageUptodate(page)) {
52 unlock_page(page);
53 return page;
54 }
55
56 /* Or a new page cache is created, then read the content from fscache */
57 folio = page_folio(page);
58
> 59 ret = netfs_readpage(NULL, folio, &erofs_req_ops, ctx->cookie);
60 if (unlikely(ret || !PageUptodate(page))) {
61 erofs_err(sb, "failed to read from fscache");
62 return ERR_PTR(-EINVAL);
63 }
64
65 return page;
66 }
67 ---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org
PATCH 13/23 erofs: implement fscache-based data read
In reply to Jeffle XuThis patch implements the data plane of reading data from bootstrap blob file over fscache.
Be noted that currently compressed layout is not supported yet.
--- fs/erofs/fscache.c | 91 +++++++++++++++++++++++++++++++++++++++++++++ fs/erofs/inode.c | 6 ++- fs/erofs/internal.h | 1 + 3 files changed, 97 insertions(+), 1 deletion(-)
fs/erofs/fscache.c +91 -0
@@ -65,6 +65,97 @@ struct page *erofs_readpage_from_fscache(struct erofs_cookie_ctx *ctx,return page; } +static inline void do_copy_page (struct page *from,struct page *to, +size_t offset,size_t len) +{ +char *vfrom, *vto; + + vfrom =kmap_atomic (from); + vto =kmap_atomic (to); +memcpy (vto, vfrom + offset, len); +kunmap_atomic (vto); +kunmap_atomic (vfrom); +} + +static int erofs_fscache_do_readpage (struct file *file,struct page *page) +{ +struct inode *inode = page->mapping->host; +struct erofs_inode *vi =EROFS_I (inode); +struct super_block *sb = inode->i_sb; +struct erofs_map_blocks map; +erofs_off_t o_la, pa; +size_t offset, len; +struct page *ipage; +int ret; + +if (erofs_inode_is_data_compressed (vi->datalayout)) { +erofs_info (sb,"compressed layout not supported yet" ); +return -EOPNOTSUPP ; + } + + o_la =page_offset (page); + map.m_la = o_la; + + ret =erofs_map_blocks (inode, &map,EROFS_GET_BLOCKS_RAW ); +if (ret) +return ret; + +if (!(map.m_flags &EROFS_MAP_MAPPED )) { +zero_user (page,0 ,PAGE_SIZE ); +return 0 ; + } + +/* +* 1) For FLAT_PLAIN/FLAT_INLINE layout, the output map.m_la shall be +* equal to o_la, and the output map.m_pa is exactly the physical +* address of o_la. +* 2) For CHUNK_BASED layout, the output map.m_la is rounded down to the +* nearest chunk boundary, and the output map.m_pa is actually the +* physical address of this chunk boundary. So we need to recalculate +* the actual physical address of o_la. +*/ + pa = map.m_pa + o_la - map.m_la; + + ipage =erofs_get_meta_page (sb,erofs_blknr (pa)); +if (IS_ERR (ipage)) +return PTR_ERR (ipage); + +/* +* @offset refers to the page offset inside @ipage. +* 1) Except for the inline layout, the offset shall all be 0, and @pa +* shall be aligned with EROFS_BLKSIZ in this case. Thus we can +* conveniently get the offset from @pa. +* 2) While for the inline layout, the offset may be non-zero. Since +* currently only flat layout supports inline, we can calculate the +* offset from the corresponding physical address. +*/ + offset =erofs_blkoff (pa); + len =min_t (u64, map.m_llen,PAGE_SIZE ); + +do_copy_page (ipage, page, offset, len); + +unlock_page (ipage); +return 0 ; +} + +static int erofs_fscache_readpage (struct file *file,struct page *page) +{ +int ret; + + ret =erofs_fscache_do_readpage (file, page); +if (ret) +SetPageError (page); +else +SetPageUptodate (page); + +unlock_page (page); +return ret; +} + +const struct address_space_operations erofs_fscache_access_aops = { + .readpage = erofs_fscache_readpage, +}; +static int erofs_fscache_init_cookie (struct erofs_cookie_ctx *ctx,char *path) {struct fscache_cookie *cookie;
fs/erofs/inode.c +5 -1
@@ -299,7 +299,11 @@ static int erofs_fill_inode(struct inode *inode, int isdir) err =z_erofs_fill_inode (inode); goto out_unlock; } - inode->i_mapping->a_ops = &erofs_raw_access_aops; + +if (inode->i_sb->s_bdev) + inode->i_mapping->a_ops = &erofs_raw_access_aops; +else + inode->i_mapping->a_ops = &erofs_fscache_access_aops; out_unlock:unlock_page (page);
fs/erofs/internal.h +1 -0
@@ -357,6 +357,7 @@ struct page *erofs_grab_cache_page_nowait(struct address_space *mapping,extern const struct super_operations erofs_sops;extern const struct address_space_operations erofs_raw_access_aops; +extern const struct address_space_operations erofs_fscache_access_aops;extern const struct address_space_operations z_erofs_aops;/*
Signature · 1 line
2.27.0
Re: [PATCH v1 13/23] erofs: implement fscache-based data read
fs/erofs/fscache.c struct page *erofs_readpage_from_fscache(struct erofs_cookie_ctx *ctx, · line 79
Quoted 13 lines · Jeffle Xu
fs/erofs/fscache.c +12
-0+static inline void do_copy_page (struct page *from,struct page *to, +size_t offset,size_t len) +{ +char *vfrom, *vto; + + vfrom =kmap_atomic (from); + vto =kmap_atomic (to); +memcpy (vto, vfrom + offset, len); +kunmap_atomic (vto); +kunmap_atomic (vfrom); +} +
It seems that this private function can be replaced by memcpy_page(). Will be done in the next version.
Signature · 2 lines
Thanks, Jeffle
Re: [PATCH v1 13/23] erofs: implement fscache-based data read
In reply to Jeffle XuQuoted 78 lines · Jeffle Xu
This patch implements the data plane of reading data from bootstrap blob file over fscache.
Be noted that currently compressed layout is not supported yet.
Signed-off-by Jeffle Xu <[email protected]>--- fs/erofs/fscache.c | 91 +++++++++++++++++++++++++++++++++++++++++++++ fs/erofs/inode.c | 6 ++- fs/erofs/internal.h | 1 + 3 files changed, 97 insertions(+), 1 deletion(-)fs/erofs/fscache.c +58
-0@@ -65,6 +65,97 @@ struct page *erofs_readpage_from_fscache(struct erofs_cookie_ctx *ctx,return page; } +static inline void do_copy_page (struct page *from,struct page *to, +size_t offset,size_t len) +{ +char *vfrom, *vto; + + vfrom =kmap_atomic (from); + vto =kmap_atomic (to); +memcpy (vto, vfrom + offset, len); +kunmap_atomic (vto); +kunmap_atomic (vfrom); +} + +static int erofs_fscache_do_readpage (struct file *file,struct page *page) +{ +struct inode *inode = page->mapping->host; +struct erofs_inode *vi =EROFS_I (inode); +struct super_block *sb = inode->i_sb; +struct erofs_map_blocks map; +erofs_off_t o_la, pa; +size_t offset, len; +struct page *ipage; +int ret; + +if (erofs_inode_is_data_compressed (vi->datalayout)) { +erofs_info (sb,"compressed layout not supported yet" ); +return -EOPNOTSUPP ; + } + + o_la =page_offset (page); + map.m_la = o_la; + + ret =erofs_map_blocks (inode, &map,EROFS_GET_BLOCKS_RAW ); +if (ret) +return ret; + +if (!(map.m_flags &EROFS_MAP_MAPPED )) { +zero_user (page,0 ,PAGE_SIZE ); +return 0 ; + } + +/* +* 1) For FLAT_PLAIN/FLAT_INLINE layout, the output map.m_la shall be +* equal to o_la, and the output map.m_pa is exactly the physical +* address of o_la. +* 2) For CHUNK_BASED layout, the output map.m_la is rounded down to the +* nearest chunk boundary, and the output map.m_pa is actually the +* physical address of this chunk boundary. So we need to recalculate +* the actual physical address of o_la. +*/ + pa = map.m_pa + o_la - map.m_la; + + ipage =erofs_get_meta_page (sb,erofs_blknr (pa)); +if (IS_ERR (ipage)) +return PTR_ERR (ipage); + +/* +* @offset refers to the page offset inside @ipage. +* 1) Except for the inline layout, the offset shall all be 0, and @pa
fs/erofs/fscache.c struct page *erofs_readpage_from_fscache(struct erofs_cookie_ctx *ctx, · line 135
Quoted 10 lines
fs/erofs/fscache.c +10
-0+* shall be aligned with EROFS_BLKSIZ in this case. Thus we can +* conveniently get the offset from @pa. +* 2) While for the inline layout, the offset may be non-zero. Since +* currently only flat layout supports inline, we can calculate the +* offset from the corresponding physical address. +*/ + offset =erofs_blkoff (pa); + len =min_t (u64, map.m_llen,PAGE_SIZE ); + +do_copy_page (ipage, page, offset, len);
If my understanding is correct, I still have no idea why we need to copy data here even if fscache can do direct I/O for us without extra efforts.
I think the only case would be tail-packing inline (which should go through metadata path), otherwise, all data is block-aligned. So fscache can handle it directly.
Thanks,
Gao Xiang
Re: [PATCH v1 13/23] erofs: implement fscache-based data read
Quoted 94 lines · Gao Xiang
Quoted 78 lines · Jeffle Xu
This patch implements the data plane of reading data from bootstrap blob file over fscache.
Be noted that currently compressed layout is not supported yet.
Signed-off-by Jeffle Xu <[email protected]>--- fs/erofs/fscache.c | 91 +++++++++++++++++++++++++++++++++++++++++++++ fs/erofs/inode.c | 6 ++- fs/erofs/internal.h | 1 + 3 files changed, 97 insertions(+), 1 deletion(-)fs/erofs/fscache.c +58
-0@@ -65,6 +65,97 @@ struct page *erofs_readpage_from_fscache(struct erofs_cookie_ctx *ctx,return page; } +static inline void do_copy_page (struct page *from,struct page *to, +size_t offset,size_t len) +{ +char *vfrom, *vto; + + vfrom =kmap_atomic (from); + vto =kmap_atomic (to); +memcpy (vto, vfrom + offset, len); +kunmap_atomic (vto); +kunmap_atomic (vfrom); +} + +static int erofs_fscache_do_readpage (struct file *file,struct page *page) +{ +struct inode *inode = page->mapping->host; +struct erofs_inode *vi =EROFS_I (inode); +struct super_block *sb = inode->i_sb; +struct erofs_map_blocks map; +erofs_off_t o_la, pa; +size_t offset, len; +struct page *ipage; +int ret; + +if (erofs_inode_is_data_compressed (vi->datalayout)) { +erofs_info (sb,"compressed layout not supported yet" ); +return -EOPNOTSUPP ; + } + + o_la =page_offset (page); + map.m_la = o_la; + + ret =erofs_map_blocks (inode, &map,EROFS_GET_BLOCKS_RAW ); +if (ret) +return ret; + +if (!(map.m_flags &EROFS_MAP_MAPPED )) { +zero_user (page,0 ,PAGE_SIZE ); +return 0 ; + } + +/* +* 1) For FLAT_PLAIN/FLAT_INLINE layout, the output map.m_la shall be +* equal to o_la, and the output map.m_pa is exactly the physical +* address of o_la. +* 2) For CHUNK_BASED layout, the output map.m_la is rounded down to the +* nearest chunk boundary, and the output map.m_pa is actually the +* physical address of this chunk boundary. So we need to recalculate +* the actual physical address of o_la. +*/ + pa = map.m_pa + o_la - map.m_la; + + ipage =erofs_get_meta_page (sb,erofs_blknr (pa)); +if (IS_ERR (ipage)) +return PTR_ERR (ipage); + +/* +* @offset refers to the page offset inside @ipage. +* 1) Except for the inline layout, the offset shall all be 0, and @pa fs/erofs/fscache.c struct page *erofs_readpage_from_fscache(struct erofs_cookie_ctx *ctx, · line 135
Quoted 10 lines
fs/erofs/fscache.c +10
-0+* shall be aligned with EROFS_BLKSIZ in this case. Thus we can +* conveniently get the offset from @pa. +* 2) While for the inline layout, the offset may be non-zero. Since +* currently only flat layout supports inline, we can calculate the +* offset from the corresponding physical address. +*/ + offset =erofs_blkoff (pa); + len =min_t (u64, map.m_llen,PAGE_SIZE ); + +do_copy_page (ipage, page, offset, len);If my understanding is correct, I still have no idea why we need to copy data here even if fscache can do direct I/O for us without extra efforts.
Quoted 4 lines
I think the only case would be tail-packing inline (which should go through metadata path), otherwise, all data is block-aligned. So fscache can handle it directly.
Right, only tail packing need special handling here. Would be fixed in the next version. Thanks.
Signature · 2 lines
Thanks, Jeffle
PATCH 14/23 erofs: register cookie context for data blobs
In reply to Jeffle XuSimilar to the multi device mode, erofs could be mounted from multiple blob files (one bootstrap blob file and optional multiple data blob files). In this case, each device slot contains the path of corresponding data blob file.
This patch registers corresponding cookie context for each data blob file. Similarly, the cleanup routine shall be contained in both .kill_sb() and .put_super() callback for the same reason.
--- fs/erofs/internal.h | 1 + fs/erofs/super.c | 12 +++++++++++- 2 files changed, 12 insertions(+), 1 deletion(-)
fs/erofs/internal.h +1 -0
@@ -52,6 +52,7 @@ struct erofs_device_info {char *path;struct block_device *bdev;struct dax_device *dax_dev; +struct erofs_cookie_ctx *ctx;u32 blocks;u32 mapped_blkaddr;
fs/erofs/super.c +11 -1
@@ -287,6 +287,7 @@ static int erofs_init_devices(struct super_block *sb,idr_for_each_entry (&sbi->devs->tree, dif, id) {erofs_blk_t blk =erofs_blknr (pos);struct block_device *bdev; +struct erofs_cookie_ctx *ctx;if (!page || page->index != blk) {if (page) {
@@ -315,7 +316,12 @@ static int erofs_init_devices(struct super_block *sb, dif->bdev = bdev; dif->dax_dev =fs_dax_get_by_bdev (bdev); }else { -/* TODO: multi devs in nodev mode */ + ctx =erofs_fscache_get_ctx (sb, dif->path); +if (IS_ERR (ctx)) { + err =PTR_ERR (ctx); + goto err_out; + } + dif->ctx = ctx; } dif->blocks =le32_to_cpu (dis->blocks); dif->mapped_blkaddr =le32_to_cpu (dis->mapped_blkaddr);
@@ -755,6 +761,7 @@ static int erofs_release_device_info(int id, void *ptr, void *data) {struct erofs_device_info *dif = ptr; +erofs_fscache_put_ctx (dif->ctx);fs_put_dax (dif->dax_dev);if (dif->bdev)blkdev_put (dif->bdev,FMODE_READ |FMODE_EXCL );
@@ -845,6 +852,9 @@ static void erofs_put_super(struct super_block *sb)iput (sbi->managed_cache); sbi->managed_cache =NULL ;#endif +erofs_free_dev_context (sbi->devs); + sbi->devs =NULL ; +erofs_fscache_put_ctx (sbi->bootstrap); sbi->bootstrap =NULL ; }
Signature · 1 line
2.27.0
PATCH 15/23 erofs: implement fscache-based data read for data blobs
In reply to Jeffle XuThis patch implements the data plane of reading data from data blob file over fscache.
--- fs/erofs/data.c | 3 +++ fs/erofs/fscache.c | 25 +++++++++++++++++++++++-- fs/erofs/internal.h | 1 + 3 files changed, 27 insertions(+), 2 deletions(-)
fs/erofs/data.c +3 -0
@@ -166,6 +166,7 @@ int erofs_map_dev(struct super_block *sb, struct erofs_map_dev *map)/* primary device by default */ map->m_bdev = sb->s_bdev; map->m_daxdev =EROFS_SB (sb)->dax_dev; + map->m_ctx =EROFS_SB (sb)->bootstrap;if (map->m_deviceid) {down_read (&devs->rwsem);
@@ -176,6 +177,7 @@ int erofs_map_dev(struct super_block *sb, struct erofs_map_dev *map) } map->m_bdev = dif->bdev; map->m_daxdev = dif->dax_dev; + map->m_ctx = dif->ctx;up_read (&devs->rwsem); }else if (devs->extra_devices) {down_read (&devs->rwsem);
@@ -192,6 +194,7 @@ int erofs_map_dev(struct super_block *sb, struct erofs_map_dev *map) map->m_pa -= startoff; map->m_bdev = dif->bdev; map->m_daxdev = dif->dax_dev; + map->m_ctx = dif->ctx;break ; } }
fs/erofs/fscache.c +23 -2
@@ -77,12 +77,24 @@ static inline void do_copy_page(struct page *from, struct page *to,kunmap_atomic (vfrom); } +static struct page *erofs_fscache_get_page (struct erofs_cookie_ctx *ctx, +erofs_blk_t blkaddr) +{ +struct page *page; + + page =erofs_readpage_from_fscache (ctx, blkaddr); +if (!IS_ERR (page)) +lock_page (page); +return page; +} +static int erofs_fscache_do_readpage (struct file *file,struct page *page) {struct inode *inode = page->mapping->host;struct erofs_inode *vi =EROFS_I (inode);struct super_block *sb = inode->i_sb;struct erofs_map_blocks map; +struct erofs_map_dev mdev;erofs_off_t o_la, pa;size_t offset, len;struct page *ipage;
@@ -105,6 +117,15 @@ static int erofs_fscache_do_readpage(struct file *file, struct page *page)return 0 ; } + mdev = (struct erofs_map_dev ) { + .m_deviceid = map.m_deviceid, + .m_pa = map.m_pa, + }; + + ret =erofs_map_dev (inode->i_sb, &mdev); +if (ret) +return ret; +/* * 1) For FLAT_PLAIN/FLAT_INLINE layout, the output map.m_la shall be * equal to o_la, and the output map.m_pa is exactly the physical
@@ -114,9 +135,9 @@ static int erofs_fscache_do_readpage(struct file *file, struct page *page)* physical address of this chunk boundary. So we need to recalculate * the actual physical address of o_la. */ - pa = map.m_pa + o_la - map.m_la; + pa = mdev.m_pa + o_la - map.m_la; - ipage =erofs_get_meta_page (sb,erofs_blknr (pa)); + ipage =erofs_fscache_get_page (mdev.m_ctx,erofs_blknr (pa));if (IS_ERR (ipage))return PTR_ERR (ipage);
fs/erofs/internal.h +1 -0
@@ -447,6 +447,7 @@ static inline int z_erofs_map_blocks_iter(struct inode *inode,struct erofs_map_dev {struct block_device *m_bdev;struct dax_device *m_daxdev; +struct erofs_cookie_ctx *m_ctx;erofs_off_t m_pa;unsigned int m_deviceid;
Signature · 1 line
2.27.0
PATCH 16/23 erofs: add 'uuid' mount option
In reply to Jeffle XuIntroduce 'uuid' mount option to enable the nodev mode, in which erofs could be mounted from blob files instead of blkdev. By then users could specify the path of bootstrap blob file containing the complete erofs image.
--- fs/erofs/super.c | 9 +++++++++ 1 file changed, 9 insertions(+)
fs/erofs/super.c +9 -0
@@ -451,6 +451,7 @@ enum { Opt_dax, Opt_dax_enum, Opt_device, + Opt_uuid, Opt_err };
@@ -475,6 +476,7 @@ static const struct fs_parameter_spec erofs_fs_parameters[] = {fsparam_flag ("dax" , Opt_dax),fsparam_enum ("dax" , Opt_dax_enum, erofs_dax_param_enums),fsparam_string ("device" , Opt_device), + fsparam_string("uuid", Opt_uuid), {} };
@@ -570,6 +572,12 @@ static int erofs_fc_parse_param(struct fs_context *fc, } ++ctx->devs->extra_devices;break ; +case Opt_uuid: +kfree (ctx->opt.uuid); + ctx->opt.uuid =kstrdup (param->string,GFP_KERNEL ); +if (!ctx->opt.uuid) +return -ENOMEM ; +break ;default :return -ENOPARAM ; }
@@ -784,6 +792,7 @@ static void erofs_fc_free(struct fs_context *fc)struct erofs_fs_context *ctx = fc->fs_private;erofs_free_dev_context (ctx->devs); +kfree (ctx->opt.uuid);kfree (ctx); }
Signature · 1 line
2.27.0
PATCH 17/23 netfs: support on demand read
In reply to Jeffle XuAdd demand_read() callback to netfs_cache_ops to implement demand reading.
The precondition for implementing demand reading semantic is that, all blob files are sparse and have been placed under corresponding directory on the first beginning. When upper fs starts to access the blob file, it will "cache miss" (hit the hole) and then .issue_op() callback will be called to prepare the data.
Then the working flow is as described below. The .issue_op() callback could be implemented by netfs_demand_read() helper, which will in turn call .demand_read() callback of corresponding fscache backend to prepare the data.
The implementation of .demand_read() callback can be backend specific. The following patch will introduce an implementation of .demand_read() callback for cachefiles, which will notify user daemon the requested file range to read. The .demand_read() callback will get blocked until the user daemon has prepared the corresponding data.
Then once .demand_read() callback returns with 0, it is guaranteed that the requested data has been ready. In this case, transform this IO request to NETFS_READ_FROM_CACHE state, initiate an incomplete completion and then retry to read from backing file.
--- fs/netfs/read_helper.c | 26 ++++++++++++++++++++++++++ include/linux/netfs.h | 4 ++++ 2 files changed, 30 insertions(+)
fs/netfs/read_helper.c +26 -0
@@ -1006,6 +1006,32 @@ int netfs_readpage(struct file *file, }EXPORT_SYMBOL (netfs_readpage); +void netfs_demand_read (struct netfs_read_subrequest *subreq) +{ +struct netfs_read_request *rreq = subreq->rreq; +struct netfs_cache_resources *cres = &rreq->cache_resources; +loff_t start_pos; +size_t len; +int ret; + + start_pos = subreq->start + subreq->transferred; + len = subreq->len - subreq->transferred; + +/* +* In success case (ret == 0), user daemon has downloaded data for us, +* thus transform to NETFS_READ_FROM_CACHE state and advertise that 0 +* byte readed, so that the request will enter into INCOMPLETE state and +* retry to read from backing file. +*/ + ret = cres->ops->demand_read (cres, start_pos, len); +if (!ret) { + subreq->source =NETFS_READ_FROM_CACHE ; +__clear_bit (NETFS_SREQ_WRITE_TO_CACHE , &subreq->flags); + } + +netfs_subreq_terminated (subreq, ret, false); +} +/* * Prepare a folio for writing without reading first * @folio: The folio being prepared
include/linux/netfs.h +4 -0
@@ -244,6 +244,9 @@ struct netfs_cache_ops {int (*prepare_write)(struct netfs_cache_resources *cres,loff_t *_start,size_t *_len,loff_t i_size,bool no_space_allocated_yet); + +int (*demand_read)(struct netfs_cache_resources *cres, +loff_t start_pos,size_t len); };struct readahead_control ;
@@ -259,6 +262,7 @@ extern int netfs_write_begin(struct file *, struct address_space *, void **, const struct netfs_read_request_ops *, void *); +extern void netfs_demand_read (struct netfs_read_subrequest *);extern void netfs_subreq_terminated (struct netfs_read_subrequest *,ssize_t ,bool );extern void netfs_stats_show (struct seq_file *);
Signature · 1 line
2.27.0
PATCH 18/23 cachefiles: use idr tree managing pending demand read
In reply to Jeffle XuThis is in prep for implementing .demand_read() callback for cachefiles.
In the following patch, .demand_read() callback is responsible for notifying user daemon the requested file range (hole) to read. Then .demand_read() callback will get blocked until user daemon has prepared data and filled the hole.
This patch introduces an idr tree to manage all these pending demand-read IO.
--- fs/cachefiles/daemon.c | 3 +++ fs/cachefiles/internal.h | 4 ++++ 2 files changed, 7 insertions(+)
fs/cachefiles/daemon.c +3 -0
@@ -118,6 +118,8 @@ static int cachefiles_daemon_open(struct inode *inode, struct file *file)INIT_LIST_HEAD (&cache->volumes);INIT_LIST_HEAD (&cache->object_list);spin_lock_init (&cache->object_list_lock); +idr_init (&cache->reqs); +spin_lock_init (&cache->reqs_lock);/* set default caching limits * - limit at 1% free space and/or free files
@@ -152,6 +154,7 @@ static int cachefiles_daemon_release(struct inode *inode, struct file *file)cachefiles_daemon_unbind (cache);/* clean up the control file interface */ +idr_destroy (&cache->reqs); cache->cachefilesd =NULL ; file->private_data =NULL ; cachefiles_open =0 ;
fs/cachefiles/internal.h +4 -0
@@ -15,6 +15,7 @@#include <linux/fscache-cache.h> #include <linux/cred.h> #include <linux/security.h> +#include <linux/idr.h> #define CACHEFILES_DIO_BLOCK_SIZE 4096
@@ -102,6 +103,9 @@ struct cachefiles_cache {char *rootdirname;/* name of cache root directory */ char *secctx;/* LSM security context */ char *tag;/* cache binding tag */ + +struct idr reqs; +spinlock_t reqs_lock; };#include <trace/events/cachefiles.h>
Signature · 1 line
2.27.0
PATCH 19/23 cachefiles: implement .demand_read() for demand read
In reply to Jeffle XuImplement .demand_read() callback for cachefiels backend.
.demand_read() callback is responsible for notifying user daemon the pending request to process, and will get blocked until user daemon has prepared data and filled the hole.
--- fs/cachefiles/internal.h | 12 +++++++++ fs/cachefiles/io.c | 56 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 68 insertions(+)
fs/cachefiles/internal.h +12 -0
@@ -61,6 +61,18 @@ struct cachefiles_object {#define CACHEFILES_OBJECT_USING_TMPFILE 0/* Have an unlinked tmpfile */ }; +struct cachefiles_req_in { +uint64_t id; +uint64_t off; +uint64_t len; +char path[NAME_MAX ]; +}; + +struct cachefiles_req { +struct completion done; +struct cachefiles_req_in req_in; +}; +/* * Cache files cache definition */
fs/cachefiles/io.c +56 -0
68 lines not on this page · raw message
Signature · 1 line
2.27.0
kernel test robot
build botRe: [PATCH v1 19/23] cachefiles: implement .demand_read() for demand read
Thank you for the patch! Perhaps something to improve:
Hi Jeffle,
Thank you for the patch! Perhaps something to improve:
[auto build test WARNING on dhowells-fs/fscache-next]
[cannot apply to xiang-erofs/dev-test ceph-client/for-linus linus/master v5.16-rc7 next-20211224] [If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/Jeffle-Xu/fscache-erofs-fscache-based-demand-read-semantics/20211227-205742 base: https://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs.git fscache-next config: csky-defconfig (https://download.01.org/0day-ci/archive/20211227/202112272340.1ho7sEjG-lkp(a)intel.com/config) compiler: csky-linux-gcc (GCC) 11.2.0
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# https://github.com/0day-ci/linux/commit/54c300b4598e3f2836d8233681da387fe388cfda
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Jeffle-Xu/fscache-erofs-fscache-based-demand-read-semantics/20211227-205742 git checkout 54c300b4598e3f2836d8233681da387fe388cfda
# save the config file to linux build tree
mkdir build_dir
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-11.2.0 make.cross O=build_dir ARCH=csky SHELL=/bin/bash fs/cachefiles/
If you fix the issue, kindly add following tag as appropriate
All warnings (new ones prefixed by >>):
fs/cachefiles/io.c:564:5: warning: no previous prototype for 'cachefiles_demand_read' [-Wmissing-prototypes]
564 | int cachefiles_demand_read(struct netfs_cache_resources *cres,
| ^~~~~~~~~~~~~~~~~~~~~~
In function 'cachefiles_alloc_req',
inlined from 'cachefiles_demand_read' at fs/cachefiles/io.c:575:8:
>> fs/cachefiles/io.c:557:9: warning: 'strncpy' specified bound 255 equals destination size [-Wstringop-truncation]
557 | strncpy(req_in->path, object->d_name, sizeof(req_in->path));
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~vim +/strncpy +557 fs/cachefiles/io.c
541
542 static struct cachefiles_req *cachefiles_alloc_req(struct cachefiles_object *object,
543 loff_t start_pos,
544 size_t len)
545 {
546 struct cachefiles_req *req;
547 struct cachefiles_req_in *req_in;
548
549 req = kzalloc(sizeof(*req), GFP_KERNEL);
550 if (!req)
551 return NULL;
552
553 req_in = &req->req_in;
554
555 req_in->off = start_pos;
556 req_in->len = len;
> 557 strncpy(req_in->path, object->d_name, sizeof(req_in->path));
558
559 init_completion(&req->done);
560
561 return req;
562 }
563
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org
Re: [PATCH v1 19/23] cachefiles: implement .demand_read() for demand read
In reply to Jeffle Xufs/cachefiles/io.c static void cachefiles_end_operation(struct netfs_cache_resources *cres) · line 580
Quoted 2 lines · Jeffle Xu
fs/cachefiles/io.c +2
-0+spin_lock (&cache->reqs_lock); + ret =idr_alloc (&cache->reqs, req,0 ,0 ,GFP_KERNEL );
GFP_KERNEL while holding a spinlock?
You should be using an XArray instead of an IDR in new code anyway.
Re: [PATCH v1 19/23] cachefiles: implement .demand_read() for demand read
Quoted 5 lines · Matthew Wilcox
fs/cachefiles/io.c static void cachefiles_end_operation(struct netfs_cache_resources *cres) · line 580
Quoted 2 lines · Jeffle Xu
fs/cachefiles/io.c +2
-0+spin_lock (&cache->reqs_lock); + ret =idr_alloc (&cache->reqs, req,0 ,0 ,GFP_KERNEL );
GFP_KERNELwhile holding a spinlock?
Right. Thanks for pointing it out.
Quoted 3 lines
You should be using an XArray instead of an IDR in new code anyway.
Regards.
Signature · 2 lines
Thanks, Jeffle
Re: [PATCH v1 19/23] cachefiles: implement .demand_read() for demand read
Quoted 9 lines · JeffleXu
Quoted 5 lines · Matthew Wilcox
fs/cachefiles/io.c static void cachefiles_end_operation(struct netfs_cache_resources *cres) · line 580
Quoted 2 lines · Jeffle Xu
fs/cachefiles/io.c +2
-0+spin_lock (&cache->reqs_lock); + ret =idr_alloc (&cache->reqs, req,0 ,0 ,GFP_KERNEL );
GFP_KERNELwhile holding a spinlock?
Quoted 8 lines
Right. Thanks for pointing it out.
Quoted 3 lines
You should be using an XArray instead of an IDR in new code anyway.
Regards.
Hi Matthew,
I'm afraid IDR can't be replaced by xarray here. Because we need an 'ID' for each pending read request, so that after fetching data from remote, user daemon could notify kernel which read request has finished by this 'ID'.
Currently this 'ID' is get from idr_alloc(), and actually identifies the position of corresponding read request inside the IDR tree. I can't find similar API of xarray implementing similar function, i.e., returning an 'ID'.
As for the issue of GFP_KERNEL while holding a spinlock, I'm going to fix this with idr_preload(), somehing like
+ idr_preload(GFP_KERNEL);
+ idr_lock(&cache->reqs);
+
+ ret = idr_alloc(&cache->reqs, req, 0, 0, GFP_ATOMIC);
+ if (ret >= 0)
+ req->req_in.id = ret;
+
+ idr_unlock(&cache->reqs);
+ idr_preload_end();
Please correct me if I'm wrong.
Signature · 2 lines
Thanks, Jeffle
Re: [PATCH v1 19/23] cachefiles: implement .demand_read() for demand read
Quoted 9 lines · JeffleXu
I'm afraid IDR can't be replaced by xarray here. Because we need an 'ID' for each pending read request, so that after fetching data from remote, user daemon could notify kernel which read request has finished by this 'ID'.
Currently this 'ID' is get from
idr_alloc(), and actually identifies the position of corresponding read request inside the IDR tree. I can't find similar API of xarray implementing similar function, i.e., returning an 'ID'.
xa_alloc().
Re: [PATCH v1 19/23] cachefiles: implement .demand_read() for demand read
Quoted 6 lines · Matthew Wilcox
Quoted 5 lines · JeffleXu
I'm afraid IDR can't be replaced by xarray here. Because we need an 'ID' for each pending read request, so that after fetching data from remote, user daemon could notify kernel which read request has finished by this 'ID'.
Quoted 7 lines
Quoted 4 lines
Currently this 'ID' is get from
idr_alloc(), and actually identifies the position of corresponding read request inside the IDR tree. I can't find similar API of xarray implementing similar function, i.e., returning an 'ID'.
xa_alloc().
Oh yes. Thanks. I will try to convert to xarray API...
Signature · 2 lines
Thanks, Jeffle
kernel test robot
build botRe: [PATCH v1 19/23] cachefiles: implement .demand_read() for demand read
In reply to Jeffle XuThank you for the patch! Perhaps something to improve:
Hi Jeffle,
Thank you for the patch! Perhaps something to improve:
[auto build test WARNING on dhowells-fs/fscache-next]
[cannot apply to xiang-erofs/dev-test ceph-client/for-linus linus/master v5.16-rc7 next-20211224] [If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/Jeffle-Xu/fscache-erofs-fscache-based-demand-read-semantics/20211227-205742 base: https://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs.git fscache-next config: i386-randconfig-r005-20211227 (https://download.01.org/0day-ci/archive/20211227/202112272353.ekKESQX6-lkp(a)intel.com/config) compiler: clang version 14.0.0 (https://github.com/llvm/llvm-project 511726c64d3b6cca66f7c54d457d586aa3129f67)
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# https://github.com/0day-ci/linux/commit/54c300b4598e3f2836d8233681da387fe388cfda
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Jeffle-Xu/fscache-erofs-fscache-based-demand-read-semantics/20211227-205742 git checkout 54c300b4598e3f2836d8233681da387fe388cfda
# save the config file to linux build tree
mkdir build_dir
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross W=1 O=build_dir ARCH=i386 SHELL=/bin/bash fs/cachefiles/
If you fix the issue, kindly add following tag as appropriate
All warnings (new ones prefixed by >>):
>> fs/cachefiles/io.c:564:5: warning: no previous prototype for function 'cachefiles_demand_read' [-Wmissing-prototypes]
int cachefiles_demand_read(struct netfs_cache_resources *cres,
^ fs/cachefiles/io.c:564:1: note: declare 'static' if the function is not intended to be used outside of this translation unit int cachefiles_demand_read(struct netfs_cache_resources *cres,
^
static
1 warning generated.
vim +/cachefiles_demand_read +564 fs/cachefiles/io.c
563
> 564 int cachefiles_demand_read(struct netfs_cache_resources *cres,
565 loff_t start_pos, size_t len)
566 {
567 struct cachefiles_object *object;
568 struct cachefiles_cache *cache;
569 struct cachefiles_req *req;
570 int ret;
571
572 object = cachefiles_cres_object(cres);
573 cache = object->volume->cache;
574
575 req = cachefiles_alloc_req(object, start_pos, len);
576 if (!req)
577 return -ENOMEM;
578
579 spin_lock(&cache->reqs_lock);
580 ret = idr_alloc(&cache->reqs, req, 0, 0, GFP_KERNEL);
581 if (ret >= 0)
582 req->req_in.id = ret;
583 spin_unlock(&cache->reqs_lock);
584 if (ret < 0) {
585 kfree(req);
586 return -ENOMEM;
587 }
588
589 wake_up_all(&cache->daemon_pollwq);
590
591 wait_for_completion(&req->done);
592 kfree(req);
593
594 return 0;
595 }
596
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org
PATCH 20/23 cachefiles: implement .poll() for demand read
In reply to Jeffle XuUser daemon needs to poll on the devnode, and will be notified once there's pending request to process.
--- fs/cachefiles/daemon.c | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+)
fs/cachefiles/daemon.c +18 -0
@@ -29,6 +29,8 @@ static ssize_t cachefiles_daemon_write(struct file *, const char __user *, size_t, loff_t *);static __poll_t cachefiles_daemon_poll (struct file *,struct poll_table_struct *); +static __poll_t cachefiles_demand_poll (struct file *, +struct poll_table_struct *);static int cachefiles_daemon_frun (struct cachefiles_cache *,char *);static int cachefiles_daemon_fcull (struct cachefiles_cache *,char *);static int cachefiles_daemon_fstop (struct cachefiles_cache *,char *);
@@ -62,6 +64,7 @@ const struct file_operations cachefiles_demand_fops = { .open = cachefiles_daemon_open, .release = cachefiles_daemon_release, .write = cachefiles_daemon_write, + .poll = cachefiles_demand_poll, .llseek = noop_llseek, };
@@ -319,6 +322,21 @@ static __poll_t cachefiles_daemon_poll(struct file *file,return mask; } +static __poll_t cachefiles_demand_poll (struct file *file, +struct poll_table_struct *poll) +{ +struct cachefiles_cache *cache = file->private_data; +__poll_t mask; + +poll_wait (file, &cache->daemon_pollwq, poll); + mask =0 ; + +if (!idr_is_empty (&cache->reqs)) + mask |=EPOLLIN ; + +return mask; +} +/* * Give a range error for cache space constraints * - can be tail-called
Signature · 1 line
2.27.0
PATCH 21/23 cachefiles: implement .read() for demand read
In reply to Jeffle XuOnce notified, user daemon need to get more details of the request by reading the devnode. The readed information includes the file range of the request, with which user daemon could somehow prepare corresponding data (e.g. download from remote through network) and fill the hole.
--- fs/cachefiles/daemon.c | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+)
fs/cachefiles/daemon.c +29 -0
@@ -29,6 +29,8 @@ static ssize_t cachefiles_daemon_write(struct file *, const char __user *, size_t, loff_t *);static __poll_t cachefiles_daemon_poll (struct file *,struct poll_table_struct *); +static ssize_t cachefiles_demand_read (struct file *,char __user *,size_t , +loff_t *);static __poll_t cachefiles_demand_poll (struct file *,struct poll_table_struct *);static int cachefiles_daemon_frun (struct cachefiles_cache *,char *);
@@ -63,6 +65,7 @@ const struct file_operations cachefiles_demand_fops = { .owner = THIS_MODULE, .open = cachefiles_daemon_open, .release = cachefiles_daemon_release, + .read = cachefiles_demand_read, .write = cachefiles_daemon_write, .poll = cachefiles_demand_poll, .llseek = noop_llseek,
@@ -322,6 +325,32 @@ static __poll_t cachefiles_daemon_poll(struct file *file,return mask; } +static ssize_t cachefiles_demand_read (struct file *file,char __user *_buffer, +size_t buflen,loff_t *pos) +{ +struct cachefiles_cache *cache = file->private_data; +struct cachefiles_req *req; +int n, id =0 ; + +if (!test_bit (CACHEFILES_READY , &cache->flags)) +return 0 ; + +spin_lock (&cache->reqs_lock); + req =idr_get_next (&cache->reqs, &id); +spin_unlock (&cache->reqs_lock); +if (!req) +return 0 ; + + n =sizeof (req->req_in); +if (n > buflen) +return -EMSGSIZE ; + +if (copy_to_user (_buffer, &req->req_in, n) !=0 ) +return -EFAULT ; + +return n; +} +static __poll_t cachefiles_demand_poll (struct file *file,struct poll_table_struct *poll) {
Signature · 1 line
2.27.0
PATCH 22/23 cachefiles: add done command for demand read
In reply to Jeffle XuOnce has prepared data and filled the hole, user daemon could notify cachefiles backend by writing 'done' command to devnode.
With done command, the id originally readed from devnode is transferred back to cachefiles backend. This id identifies the position of the completed request inside the idr tree, and thus cachefiles backend could find the corresponding completed request once gets notified.
--- fs/cachefiles/daemon.c | 34 ++++++++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+)
fs/cachefiles/daemon.c +34 -0
@@ -48,6 +48,7 @@ static int cachefiles_daemon_tag(struct cachefiles_cache *, char *);static int cachefiles_daemon_mode (struct cachefiles_cache *,char *);static int cachefiles_daemon_bind (struct cachefiles_cache *,char *);static void cachefiles_daemon_unbind (struct cachefiles_cache *); +static int cachefiles_daemon_done (struct cachefiles_cache *,char *);static unsigned long cachefiles_open;
@@ -91,6 +92,7 @@ static const struct cachefiles_daemon_cmd cachefiles_daemon_cmds[] = { { "secctx", cachefiles_daemon_secctx }, { "tag", cachefiles_daemon_tag }, { "mode", cachefiles_daemon_mode }, + { "done", cachefiles_daemon_done }, { "", NULL } };
@@ -604,6 +606,38 @@ static int cachefiles_daemon_tag(struct cachefiles_cache *cache, char *args)return 0 ; } +/* +* Request completion +* - command: "done <id>" +*/ +static int cachefiles_daemon_done (struct cachefiles_cache *cache,char *args) +{ +unsigned long id; +int ret; +struct cachefiles_req *req; + +_enter (",%s" , args); + +if (!*args) { +pr_err ("Empty id specified\n" ); +return -EINVAL ; + } + + ret =kstrtoul (args,0 , &id); +if (ret) +return ret; + +spin_lock (&cache->reqs_lock); + req =idr_remove (&cache->reqs, id); +spin_unlock (&cache->reqs_lock); +if (!req) +return -EINVAL ; + +complete (&req->done); + +return 0 ; +} +/* * Request a node in the cache be culled from the current working directory * - command: "cull <name>"
Signature · 1 line
2.27.0
PATCH 23/23 erofs: support on demand read
In reply to Jeffle XuImplement .issue_op() callback to support demand reading.
--- fs/erofs/fscache.c | 8 +------- 1 file changed, 1 insertion(+), 7 deletions(-)
fs/erofs/fscache.c +1 -7
@@ -18,13 +18,7 @@ static void erofs_priv_cleanup(struct address_space *mapping, void *netfs_priv)static void erofs_issue_op (struct netfs_read_subrequest *subreq) { -/* -* TODO: implement demand-read logic later. -* We rely on user daemon to prepare blob files under corresponding -* directory, and we can reach here if blob files don't exist. -*/ - -netfs_subreq_terminated (subreq, -EOPNOTSUPP , false); +netfs_demand_read (subreq); }const struct netfs_read_request_ops erofs_req_ops = {
Signature · 1 line
2.27.0