io_uring getdents
Subsystems (33)
v5 cover io_uring getdents
From: Hao Xu <[email protected]>
This series introduce getdents64 to io_uring, the code logic is similar with the snychronized version's. It first try nowait issue, and offload it to io-wq threads if the first try fails.
Patch1 and Patch2 are some preparation
Patch3 supports nowait for xfs getdents code
Patch4-11 are vfs change, include adding helpers and trylock for locks Patch12-29 supports nowait for involved xfs journal stuff note, Patch24 and 27 are actually two questions, might be removed later. an xfs test may come later.
Tests I've done:
a liburing test case for functional test:
https://github.com/HowHsu/liburing/commit/39dc9a8e19c06a8cebf8c2301b85320eb45c061e?diff=unified
xfstests:
test/generic: 1 fails and 171 not run
test/xfs: 72 fails and 156 not run
run the code before without this patchset, same result. I'll try to make the environment more right to run more tests here.
Tested it with a liburing performance test:
https://github.com/HowHsu/liburing/blob/getdents/test/getdents2.c
The test is controlled by the below script[2] which runs getdents2.t 100 times and calulate the avg.
The result show that io_uring version is about 2.6% faster:
note:
[1] the number of getdents call/request in io_uring and normal sync version are made sure to be same beforehand.
[2] run_getdents.py
```python3
import subprocess
N = 100
sum = 0.0
args = ["/data/home/howeyxu/tmpdir", "sync"]
for i in range(N):
output = subprocess.check_output(["./liburing/test/getdents2.t"] + args)
sum += float(output)
average = sum / N
print("Average of sync:", average)
sum = 0.0
args = ["/data/home/howeyxu/tmpdir", "iouring"]
for i in range(N):
output = subprocess.check_output(["./liburing/test/getdents2.t"] + args)
sum += float(output)
average = sum / N
print("Average of iouring:", average)
```v4->v5:
- move atime update to the beginning of getdents operation
- trylock for i_rwsem
- nowait semantics for involved xfs journal stuff
v3->v4:
- add Dave's xfs nowait code and fix a deadlock problem, with some code style tweak.
- disable fixed file to avoid a race problem for now
- add a test program.
v2->v3:
- removed the kernfs patches
- add f_pos_lock logic
- remove the "reduce last EOF getdents try" optimization since Dominique reports that doesn't make difference
- remove the rewind logic, I think the right way is to introduce lseek to io_uring not to patch this logic to getdents.
- add Singed-off-by of Stefan Roesch for patch 1 since checkpatch complained that Co-developed-by someone should be accompanied with Signed-off-by same person, I can remove them if Stefan thinks that's not proper.
Dominique Martinet (1):
fs: split off vfs_getdents function of getdents64 syscall
Hao Xu (28):
xfs: rename XBF_TRYLOCK to XBF_NOWAIT
xfs: add NOWAIT semantics for readdir
vfs: add nowait flag for struct dir_context
vfs: add a vfs helper for io_uring file pos lock
vfs: add file_pos_unlock() for io_uring usage
vfs: add a nowait parameter for touch_atime()
vfs: add nowait parameter for file_accessed()
vfs: move file_accessed() to the beginning of iterate_dir()
vfs: add S_NOWAIT for nowait time update
vfs: trylock inode->i_rwsem in iterate_dir() to support nowait
xfs: enforce GFP_NOIO implicitly during nowait time update
xfs: make xfs_trans_alloc() support nowait semantics
xfs: support nowait for xfs_log_reserve()
xfs: don't wait for free space in xlog_grant_head_check() in nowait
case
xfs: add nowait parameter for xfs_inode_item_init()
xfs: make xfs_trans_ijoin() error out -EAGAIN
xfs: set XBF_NOWAIT for xfs_buf_read_map if necessary
xfs: support nowait memory allocation in _xfs_buf_alloc()
xfs: distinguish error type of memory allocation failure for nowait
case
xfs: return -EAGAIN when bulk memory allocation fails in nowait case
xfs: comment page allocation for nowait case in xfs_buf_find_insert()
xfs: don't print warn info for -EAGAIN error in xfs_buf_get_map()
xfs: support nowait for xfs_buf_read_map()
xfs: support nowait for xfs_buf_item_init()
xfs: return -EAGAIN when nowait meets sync in transaction commit
xfs: add a comment for xlog_kvmalloc()
xfs: support nowait semantics for xc_ctx_lock in xlog_cil_commit()
io_uring: add support for getdents
arch/s390/hypfs/inode.c | 2 +- block/fops.c | 2 +- fs/btrfs/file.c | 2 +- fs/btrfs/inode.c | 2 +- fs/cachefiles/namei.c | 2 +- fs/coda/dir.c | 4 +-- fs/ecryptfs/file.c | 4 +-- fs/ext2/file.c | 4 +-- fs/ext4/file.c | 6 ++-- fs/f2fs/file.c | 4 +-- fs/file.c | 13 +++++++ fs/fuse/dax.c | 2 +- fs/fuse/file.c | 4 +-- fs/gfs2/file.c | 2 +- fs/hugetlbfs/inode.c | 2 +- fs/inode.c | 10 +++--- fs/internal.h | 8 +++++ fs/namei.c | 4 +-- fs/nfsd/vfs.c | 2 +- fs/nilfs2/file.c | 2 +- fs/orangefs/file.c | 2 +- fs/orangefs/inode.c | 2 +- fs/overlayfs/file.c | 2 +- fs/overlayfs/inode.c | 2 +- fs/pipe.c | 2 +- fs/ramfs/file-nommu.c | 2 +- fs/readdir.c | 61 +++++++++++++++++++++++++-------- fs/smb/client/cifsfs.c | 2 +- fs/splice.c | 2 +- fs/stat.c | 2 +- fs/ubifs/file.c | 2 +- fs/udf/file.c | 2 +- fs/xfs/libxfs/xfs_alloc.c | 2 +- fs/xfs/libxfs/xfs_attr_remote.c | 2 +- fs/xfs/libxfs/xfs_btree.c | 2 +- fs/xfs/libxfs/xfs_da_btree.c | 16 +++++++++ fs/xfs/libxfs/xfs_da_btree.h | 1 + fs/xfs/libxfs/xfs_dir2_block.c | 7 ++-- fs/xfs/libxfs/xfs_dir2_priv.h | 2 +- fs/xfs/libxfs/xfs_shared.h | 2 ++ fs/xfs/libxfs/xfs_trans_inode.c | 12 +++++-- fs/xfs/scrub/dir.c | 2 +- fs/xfs/scrub/readdir.c | 2 +- fs/xfs/scrub/repair.c | 2 +- fs/xfs/xfs_buf.c | 43 +++++++++++++++++------ fs/xfs/xfs_buf.h | 4 +-- fs/xfs/xfs_buf_item.c | 9 +++-- fs/xfs/xfs_buf_item.h | 2 +- fs/xfs/xfs_buf_item_recover.c | 2 +- fs/xfs/xfs_dir2_readdir.c | 49 ++++++++++++++++++++------ fs/xfs/xfs_dquot.c | 2 +- fs/xfs/xfs_file.c | 6 ++-- fs/xfs/xfs_inode.c | 27 +++++++++++++++ fs/xfs/xfs_inode.h | 17 +++++---- fs/xfs/xfs_inode_item.c | 12 ++++--- fs/xfs/xfs_inode_item.h | 3 +- fs/xfs/xfs_iops.c | 31 ++++++++++++++--- fs/xfs/xfs_log.c | 33 ++++++++++++------ fs/xfs/xfs_log.h | 5 +-- fs/xfs/xfs_log_cil.c | 17 +++++++-- fs/xfs/xfs_log_priv.h | 4 +-- fs/xfs/xfs_trans.c | 44 ++++++++++++++++++++---- fs/xfs/xfs_trans.h | 2 +- fs/xfs/xfs_trans_buf.c | 18 ++++++++-- fs/zonefs/file.c | 4 +-- include/linux/file.h | 7 ++++ include/linux/fs.h | 16 +++++++-- include/uapi/linux/io_uring.h | 1 + io_uring/fs.c | 53 ++++++++++++++++++++++++++++ io_uring/fs.h | 3 ++ io_uring/opdef.c | 8 +++++ kernel/bpf/inode.c | 4 +-- mm/filemap.c | 8 ++--- mm/shmem.c | 6 ++-- net/unix/af_unix.c | 4 +-- 75 files changed, 499 insertions(+), 161 deletions(-)
Signature · 1 line
2.25.1
PATCH v5 1/29 fs: split off vfs_getdents function of getdents64 syscall
From: Dominique Martinet <[email protected]>
This splits off the vfs_getdents function from the getdents64 system call.
This will allow io_uring to call the vfs_getdents function.
--- fs/internal.h | 8 ++++++++ fs/readdir.c | 34 ++++++++++++++++++++++++++-------- 2 files changed, 34 insertions(+), 8 deletions(-)
fs/internal.h +8 -0
@@ -304,3 +304,11 @@ ssize_t __kernel_write_iter(struct file *file, struct iov_iter *from, loff_t *postruct mnt_idmap *alloc_mnt_idmap (struct user_namespace *mnt_userns);struct mnt_idmap *mnt_idmap_get (struct mnt_idmap *idmap);void mnt_idmap_put (struct mnt_idmap *idmap); + +/* +* fs/readdir.c +*/ +struct linux_dirent64 ; + +int vfs_getdents (struct file *file,struct linux_dirent64 __user *dirent, +unsigned int count);
fs/readdir.c +26 -8
@@ -21,6 +21,7 @@#include <linux/unistd.h> #include <linux/compat.h> #include <linux/uaccess.h> +#include "internal.h" #include <asm/unaligned.h>
@@ -351,10 +352,16 @@ static bool filldir64(struct dir_context *ctx, const char *name, int namlen,return false; } -SYSCALL_DEFINE3(getdents64, unsigned int, fd, - struct linux_dirent64 __user *, dirent, unsigned int, count) + +/** +* vfs_getdents - getdents without fdget +* @file : pointer to file struct of directory +* @dirent : pointer to user directory structure +* @count : size of buffer +*/ +int vfs_getdents (struct file *file,struct linux_dirent64 __user *dirent, +unsigned int count) { -struct fd f;struct getdents_callback64 buf = { .ctx.actor = filldir64, .count = count,
@@ -362,11 +369,7 @@ SYSCALL_DEFINE3(getdents64, unsigned int, fd, };int error; - f =fdget_pos (fd); -if (!f.file) -return -EBADF ; - - error =iterate_dir (f.file, &buf.ctx); + error =iterate_dir (file, &buf.ctx);if (error >=0 ) error = buf.error;if (buf.prev_reclen) {
@@ -379,6 +382,21 @@ SYSCALL_DEFINE3(getdents64, unsigned int, fd,else error = count - buf.count; } +return error; +} + +SYSCALL_DEFINE3(getdents64, unsigned int, fd, + struct linux_dirent64 __user *, dirent, unsigned int, count) +{ +struct fd f; +int error; + + f =fdget_pos (fd); +if (!f.file) +return -EBADF ; + + error =vfs_getdents (f.file, dirent, count); +fdput_pos (f);return error; }
Signature · 1 line
2.25.1
PATCH v5 2/29 xfs: rename XBF_TRYLOCK to XBF_NOWAIT
In reply to Hao XuFrom: Hao Xu <[email protected]>
XBF_TRYLOCK means we need lock but don't block on it, we can use it to stand for not waiting for memory allcation. Rename XBF_TRYLOCK to XBF_NOWAIT, which is more generic.
--- fs/xfs/libxfs/xfs_alloc.c | 2 +- fs/xfs/libxfs/xfs_attr_remote.c | 2 +- fs/xfs/libxfs/xfs_btree.c | 2 +- fs/xfs/scrub/repair.c | 2 +- fs/xfs/xfs_buf.c | 6 +++--- fs/xfs/xfs_buf.h | 4 ++-- fs/xfs/xfs_dquot.c | 2 +- 7 files changed, 10 insertions(+), 10 deletions(-)
fs/xfs/libxfs/xfs_alloc.c +1 -1
@@ -3183,7 +3183,7 @@ xfs_alloc_read_agf(ASSERT ((flags & (XFS_ALLOC_FLAG_FREEING |XFS_ALLOC_FLAG_TRYLOCK )) != (XFS_ALLOC_FLAG_FREEING |XFS_ALLOC_FLAG_TRYLOCK )); error =xfs_read_agf (pag, tp, - (flags &XFS_ALLOC_FLAG_TRYLOCK ) ?XBF_TRYLOCK :0 , + (flags &XFS_ALLOC_FLAG_TRYLOCK ) ?XBF_NOWAIT :0 , &agfbp);if (error)return error;
fs/xfs/libxfs/xfs_attr_remote.c +1 -1
@@ -661,7 +661,7 @@ xfs_attr_rmtval_invalidate(return error;if (XFS_IS_CORRUPT (args->dp->i_mount, nmap !=1 ))return -EFSCORRUPTED ; - error =xfs_attr_rmtval_stale (args->dp, &map,XBF_TRYLOCK ); + error =xfs_attr_rmtval_stale (args->dp, &map,XBF_NOWAIT );if (error)return error;
fs/xfs/libxfs/xfs_btree.c +1 -1
@@ -1343,7 +1343,7 @@ xfs_btree_read_buf_block(int error;/* need to sort out how callers deal with failures first */ -ASSERT (!(flags &XBF_TRYLOCK )); +ASSERT (!(flags &XBF_NOWAIT )); error =xfs_btree_ptr_to_daddr (cur, ptr, &d);if (error)
fs/xfs/scrub/repair.c +1 -1
@@ -460,7 +460,7 @@ xrep_invalidate_block( error =xfs_buf_incore (sc->mp->m_ddev_targp,XFS_FSB_TO_DADDR (sc->mp, fsbno), -XFS_FSB_TO_BB (sc->mp,1 ),XBF_TRYLOCK , &bp); +XFS_FSB_TO_BB (sc->mp,1 ),XBF_NOWAIT , &bp);if (error)return 0 ;
fs/xfs/xfs_buf.c +3 -3
@@ -228,7 +228,7 @@ _xfs_buf_alloc(* We don't want certain flags to appear in b_flags unless they are * specifically set by later operations on the buffer. */ - flags &= ~(XBF_UNMAPPED |XBF_TRYLOCK |XBF_ASYNC |XBF_READ_AHEAD ); + flags &= ~(XBF_UNMAPPED |XBF_NOWAIT |XBF_ASYNC |XBF_READ_AHEAD );atomic_set (&bp->b_hold,1 );atomic_set (&bp->b_lru_ref,1 );
@@ -543,7 +543,7 @@ xfs_buf_find_lock( struct xfs_buf *bp, xfs_buf_flags_t flags) { -if (flags &XBF_TRYLOCK ) { +if (flags &XBF_NOWAIT ) {if (!xfs_buf_trylock (bp)) {XFS_STATS_INC (bp->b_mount, xb_busy_locked);return -EAGAIN ;
@@ -886,7 +886,7 @@ xfs_buf_readahead_map(struct xfs_buf *bp;xfs_buf_read_map (target, map, nmaps, -XBF_TRYLOCK |XBF_ASYNC |XBF_READ_AHEAD , &bp, ops, +XBF_NOWAIT |XBF_ASYNC |XBF_READ_AHEAD , &bp, ops, __this_address); }
fs/xfs/xfs_buf.h +2 -2
@@ -45,7 +45,7 @@ struct xfs_buf;/* flags used only as arguments to access routines */ #define XBF_INCORE (1u << 29)/* lookup only, return if found in cache */ -#define XBF_TRYLOCK (1u << 30)/* lock requested, but do not wait */ +#define XBF_NOWAIT (1u << 30)/* mem/lock requested, but do not wait */ #define XBF_UNMAPPED (1u << 31)/* do not map the buffer */
@@ -68,7 +68,7 @@ typedef unsigned int xfs_buf_flags_t; { _XBF_DELWRI_Q, "DELWRI_Q" }, \/* The following interface flags should never be set */ \ { XBF_INCORE, "INCORE" }, \ - { XBF_TRYLOCK, "TRYLOCK" }, \ + { XBF_NOWAIT, "NOWAIT" }, \ { XBF_UNMAPPED, "UNMAPPED" }/*
fs/xfs/xfs_dquot.c +1 -1
@@ -1233,7 +1233,7 @@ xfs_qm_dqflush(* Get the buffer containing the on-disk dquot */ error =xfs_trans_read_buf (mp,NULL , mp->m_ddev_targp, dqp->q_blkno, - mp->m_quotainfo->qi_dqchunklen,XBF_TRYLOCK , + mp->m_quotainfo->qi_dqchunklen,XBF_NOWAIT , &bp, &xfs_dquot_buf_ops);if (error == -EAGAIN ) goto out_unlock;
Signature · 1 line
2.25.1
Re: [PATCH 02/29] xfs: rename XBF_TRYLOCK to XBF_NOWAIT
Quoted 3 lines · Hao Xu
From: Hao Xu <[email protected]>
XBF_TRYLOCKmeans we need lock but don't block on it,
Yes.
Quoted 3 lines
we can use it to
stand for not waiting for memory allcation. RenameXBF_TRYLOCKtoXBF_NOWAIT, which is more generic.
No.
Not only can XBF_TRYLOCK require memory allocation, it can require IO to be issued. We use TRYLOCK for -readahead- and so we *must* be able to allocate memory and issue IO under TRYLOCK caller conditions.
[...]
fs/xfs/libxfs/xfs_attr_remote.c xfs_attr_rmtval_invalidate( · line 666
Quoted 12 lines
fs/xfs/libxfs/xfs_attr_remote.c +1
-1@@ -661,7 +661,7 @@ xfs_attr_rmtval_invalidate(return error;if (XFS_IS_CORRUPT (args->dp->i_mount, nmap !=1 ))return -EFSCORRUPTED ; - error =xfs_attr_rmtval_stale (args->dp, &map,XBF_TRYLOCK ); + error =xfs_attr_rmtval_stale (args->dp, &map,XBF_NOWAIT );if (error)return error;
XBF_INCORE | XBF_NOWAIT makes no real sense. I mean, XBF_INCORE is exactly "find a cached buffer or fail" - it's not going to do any memory allocation or IO so NOWAIT smeantics don't make any sense here. It's the buffer lock that this lookup is explicitly avoiding, and so TRYLOCK describes exactly the semantics we want from this incore lookup.
Indeed, this is a deadlock avoidance mechanism as the transaction may already have the buffer locked and so we don't want the xfs_buf_incore() lookup to try to lock the buffer again. TRYLOCK documents this pretty clearly - NOWAIT loses that context....
Quoted 18 lines
fs/xfs/libxfs/xfs_btree.c +1
-1@@ -1343,7 +1343,7 @@ xfs_btree_read_buf_block(int error;/* need to sort out how callers deal with failures first */ -ASSERT (!(flags &XBF_TRYLOCK )); +ASSERT (!(flags &XBF_NOWAIT )); error =xfs_btree_ptr_to_daddr (cur, ptr, &d);if (error)fs/xfs/scrub/repair.c +0
-0
fs/xfs/scrub/repair.c xrep_invalidate_block( · line 463
Quoted 5 lines
fs/xfs/scrub/repair.c +1
-1@@ -460,7 +460,7 @@ xrep_invalidate_block( error =xfs_buf_incore (sc->mp->m_ddev_targp,XFS_FSB_TO_DADDR (sc->mp, fsbno), -XFS_FSB_TO_BB (sc->mp,1 ),XBF_TRYLOCK , &bp); +XFS_FSB_TO_BB (sc->mp,1 ),XBF_NOWAIT , &bp);
My point exactly.
xfs_buf_incore() is simply a lookup with XBF_INCORE set. (XBF_INCORE | XBF_TRYLOCK) has the exactly semantics of "return the buffer only if it is cached and we can lock it without blocking.
It will not instantiate a new buffer (i.e. do memory allocation) or do IO because the if it is under IO the buffer lock will be held.
So, essentially, this "NOWAIT" semantic you want is already supplied by (XBF_INCORE | XBF_TRYLOCK) buffer lookups.
Quoted 26 lines
fs/xfs/scrub/repair.c +0
-0if (error)return 0 ;fs/xfs/xfs_buf.c +2
-2@@ -228,7 +228,7 @@ _xfs_buf_alloc(* We don't want certain flags to appear in b_flags unless they are * specifically set by later operations on the buffer. */ - flags &= ~(XBF_UNMAPPED |XBF_TRYLOCK |XBF_ASYNC |XBF_READ_AHEAD ); + flags &= ~(XBF_UNMAPPED |XBF_NOWAIT |XBF_ASYNC |XBF_READ_AHEAD );atomic_set (&bp->b_hold,1 );atomic_set (&bp->b_lru_ref,1 );@@ -543,7 +543,7 @@ xfs_buf_find_lock( struct xfs_buf *bp, xfs_buf_flags_t flags) { -if (flags &XBF_TRYLOCK ) { +if (flags &XBF_NOWAIT ) {if (!xfs_buf_trylock (bp)) {XFS_STATS_INC (bp->b_mount, xb_busy_locked);return -EAGAIN ;
fs/xfs/xfs_buf.c xfs_buf_readahead_map( · line 890
Quoted 6 lines
fs/xfs/xfs_buf.c +1
-1@@ -886,7 +886,7 @@ xfs_buf_readahead_map(struct xfs_buf *bp;xfs_buf_read_map (target, map, nmaps, -XBF_TRYLOCK |XBF_ASYNC |XBF_READ_AHEAD , &bp, ops, +XBF_NOWAIT |XBF_ASYNC |XBF_READ_AHEAD , &bp, ops, __this_address);
That will break readahead (which we use extensively in getdents operations) if we can't allocate buffers and issue IO under NOWAIT conditions.
fs/xfs/xfs_buf.h struct xfs_buf; · line 48
Quoted 12 lines
fs/xfs/xfs_buf.c +0
-0}fs/xfs/xfs_buf.h +1
-1@@ -45,7 +45,7 @@ struct xfs_buf;/* flags used only as arguments to access routines */ #define XBF_INCORE (1u << 29)/* lookup only, return if found in cache */ -#define XBF_TRYLOCK (1u << 30)/* lock requested, but do not wait */ +#define XBF_NOWAIT (1u << 30)/* mem/lock requested, but do not wait */
That's now a really poor comment. It doesn't describe the semantics or constraints that NOWAIT might imply.
-Dave.
Signature · 2 lines
Dave Chinner [email protected]
PATCH v5 3/29 xfs: add NOWAIT semantics for readdir
In reply to Hao XuFrom: Hao Xu <[email protected]>
Implement NOWAIT semantics for readdir. Return EAGAIN error to the caller if it would block, like failing to get locks, or going to do IO.
--- fs/xfs/libxfs/xfs_da_btree.c | 16 +++++++++++ fs/xfs/libxfs/xfs_da_btree.h | 1 + fs/xfs/libxfs/xfs_dir2_block.c | 7 ++--- fs/xfs/libxfs/xfs_dir2_priv.h | 2 +- fs/xfs/scrub/dir.c | 2 +- fs/xfs/scrub/readdir.c | 2 +- fs/xfs/xfs_dir2_readdir.c | 49 ++++++++++++++++++++++++++-------- fs/xfs/xfs_inode.c | 27 +++++++++++++++++++ fs/xfs/xfs_inode.h | 17 +++++++----- 9 files changed, 99 insertions(+), 24 deletions(-)
fs/xfs/libxfs/xfs_da_btree.c +16 -0
@@ -2643,16 +2643,32 @@ xfs_da_read_buf(struct xfs_buf_map map, *mapp = ↦int nmap =1 ;int error; +int buf_flags =0 ; *bpp =NULL ; error =xfs_dabuf_map (dp, bno, flags, whichfork, &mapp, &nmap);if (error || !nmap) goto out_free; +/* +* NOWAIT semantics mean we don't wait on the buffer lock nor do we +* issue IO for this buffer if it is not already in memory. Caller will +* retry. This will return -EAGAIN if the buffer is in memory and cannot +* be locked, and no buffer and no error if it isn't in memory. We +* translate both of those into a return state of -EAGAIN and *bpp = +* NULL. +*/ +if (flags &XFS_DABUF_NOWAIT ) + buf_flags |=XBF_NOWAIT |XBF_INCORE ; error =xfs_trans_read_buf_map (mp, tp, mp->m_ddev_targp, mapp, nmap,0 , &bp, ops);if (error) goto out_free; +if (!bp) { +ASSERT (flags &XFS_DABUF_NOWAIT ); + error = -EAGAIN ; + goto out_free; + }if (whichfork ==XFS_ATTR_FORK )xfs_buf_set_ref (bp,XFS_ATTR_BTREE_REF );
fs/xfs/libxfs/xfs_da_btree.h +1 -0
@@ -205,6 +205,7 @@ int xfs_da3_node_read_mapped(struct xfs_trans *tp, struct xfs_inode *dp,*/ #define XFS_DABUF_MAP_HOLE_OK (1u << 0) +#define XFS_DABUF_NOWAIT (1u << 1)int xfs_da_grow_inode (xfs_da_args_t *args,xfs_dablk_t *new_blkno); int xfs_da_grow_inode_int(struct xfs_da_args *args, xfs_fileoff_t *bno,
fs/xfs/libxfs/xfs_dir2_block.c +4 -3
@@ -135,13 +135,14 @@ intxfs_dir3_block_read ( struct xfs_trans *tp, struct xfs_inode *dp, + unsigned int flags, struct xfs_buf **bpp) { struct xfs_mount *mp = dp->i_mount;xfs_failaddr_t fa;int err; - err =xfs_da_read_buf (tp, dp, mp->m_dir_geo->datablk,0 , bpp, + err =xfs_da_read_buf (tp, dp, mp->m_dir_geo->datablk, flags, bpp,XFS_DATA_FORK , &xfs_dir3_block_buf_ops);if (err || !*bpp)return err;
@@ -380,7 +381,7 @@ xfs_dir2_block_addname( tp = args->trans;/* Read the (one and only) directory block into bp. */ - error =xfs_dir3_block_read (tp, dp, &bp); + error =xfs_dir3_block_read (tp, dp,0 , &bp);if (error)return error;
@@ -695,7 +696,7 @@ xfs_dir2_block_lookup_int( dp = args->dp; tp = args->trans; - error =xfs_dir3_block_read (tp, dp, &bp); + error =xfs_dir3_block_read (tp, dp,0 , &bp);if (error)return error;
fs/xfs/libxfs/xfs_dir2_priv.h +1 -1
@@ -51,7 +51,7 @@ extern int xfs_dir_cilookup_result(struct xfs_da_args *args,/* xfs_dir2_block.c */ extern int xfs_dir3_block_read (struct xfs_trans *tp,struct xfs_inode *dp, -struct xfs_buf **bpp); +unsigned int flags,struct xfs_buf **bpp);extern int xfs_dir2_block_addname (struct xfs_da_args *args);extern int xfs_dir2_block_lookup (struct xfs_da_args *args);extern int xfs_dir2_block_removename (struct xfs_da_args *args);
fs/xfs/scrub/dir.c +1 -1
@@ -313,7 +313,7 @@ xchk_directory_data_bestfree(/* dir block format */ if (lblk !=XFS_B_TO_FSBT (mp,XFS_DIR2_DATA_OFFSET ))xchk_fblock_set_corrupt (sc,XFS_DATA_FORK , lblk); - error =xfs_dir3_block_read (sc->tp, sc->ip, &bp); + error =xfs_dir3_block_read (sc->tp, sc->ip,0 , &bp); }else {/* dir data format */ error =xfs_dir3_data_read (sc->tp, sc->ip, lblk,0 , &bp);
fs/xfs/scrub/readdir.c +1 -1
@@ -101,7 +101,7 @@ xchk_dir_walk_block(unsigned int off, next_off, end;int error; - error =xfs_dir3_block_read (sc->tp, dp, &bp); + error =xfs_dir3_block_read (sc->tp, dp,0 , &bp);if (error)return error;
fs/xfs/xfs_dir2_readdir.c +38 -11
120 lines not on this page · raw message
fs/xfs/xfs_inode.c +27 -0
@@ -120,6 +120,33 @@ xfs_ilock_data_map_shared(return lock_mode; } +/* +* Similar to xfs_ilock_data_map_shared(), except that it will only try to lock +* the inode in shared mode if the extents are already in memory. If it fails to +* get the lock or has to do IO to read the extent list, fail the operation by +* returning 0 as the lock mode. +*/ +uint +xfs_ilock_data_map_shared_nowait ( +struct xfs_inode *ip) +{ +if (xfs_need_iread_extents (&ip->i_df)) +return 0 ; +if (!xfs_ilock_nowait (ip,XFS_ILOCK_SHARED )) +return 0 ; +return XFS_ILOCK_SHARED ; +} + +int +xfs_ilock_data_map_shared_generic ( +struct xfs_inode *dp, +bool nowait) +{ +if (nowait) +return xfs_ilock_data_map_shared_nowait (dp); +return xfs_ilock_data_map_shared (dp); +} +uint xfs_ilock_attr_map_shared (struct xfs_inode *ip)
fs/xfs/xfs_inode.h +10 -7
@@ -490,13 +490,16 @@ int xfs_rename(struct mnt_idmap *idmap,struct xfs_name *target_name, struct xfs_inode *target_ip, unsigned int flags); -void xfs_ilock (xfs_inode_t *,uint ); -int xfs_ilock_nowait (xfs_inode_t *,uint ); -void xfs_iunlock (xfs_inode_t *,uint ); -void xfs_ilock_demote (xfs_inode_t *,uint ); -bool xfs_isilocked (struct xfs_inode *,uint ); -uint xfs_ilock_data_map_shared (struct xfs_inode *); -uint xfs_ilock_attr_map_shared (struct xfs_inode *); +void xfs_ilock (struct xfs_inode *ip,uint lockmode); +int xfs_ilock_nowait (struct xfs_inode *ip,uint lockmode); +void xfs_iunlock (struct xfs_inode *ip,uint lockmode); +void xfs_ilock_demote (struct xfs_inode *ip,uint lockmode); +bool xfs_isilocked (struct xfs_inode *ip,uint lockmode); +uint xfs_ilock_data_map_shared (struct xfs_inode *ip); +uint xfs_ilock_data_map_shared_nowait (struct xfs_inode *ip); +int xfs_ilock_data_map_shared_generic (struct xfs_inode *ip, +bool nowait); +uint xfs_ilock_attr_map_shared (struct xfs_inode *ip);uint xfs_ip2xflags (struct xfs_inode *);int xfs_ifree (struct xfs_trans *,struct xfs_inode *);
Signature · 1 line
2.25.1
PATCH v5 4/29 vfs: add nowait flag for struct dir_context
In reply to Hao XuFrom: Hao Xu <[email protected]>
The flags will allow passing DIR_CONTEXT_F_NOWAIT to iterate() implementations that support it (as signaled through FMODE_NWAIT in file->f_mode)
Notes:
- considered using IOCB_NOWAIT but if we add more flags later it would be confusing to keep track of which values are valid, use dedicated flags
- might want to check ctx.flags & DIR_CONTEXT_F_NOWAIT is only set when file->f_mode & FMODE_NOWAIT in iterate_dir() as e.g. WARN_ONCE?
--- fs/internal.h | 2 +- fs/readdir.c | 6 ++++-- include/linux/fs.h | 8 ++++++++ 3 files changed, 13 insertions(+), 3 deletions(-)
fs/internal.h +1 -1
@@ -311,4 +311,4 @@ void mnt_idmap_put(struct mnt_idmap *idmap);struct linux_dirent64 ;int vfs_getdents (struct file *file,struct linux_dirent64 __user *dirent, -unsigned int count); +unsigned int count,unsigned long flags);
fs/readdir.c +4 -2
@@ -358,12 +358,14 @@ static bool filldir64(struct dir_context *ctx, const char *name, int namlen,* @file : pointer to file struct of directory * @dirent : pointer to user directory structure * @count : size of buffer +* @flags : additional dir_context flags */ int vfs_getdents (struct file *file,struct linux_dirent64 __user *dirent, -unsigned int count) +unsigned int count,unsigned long flags) {struct getdents_callback64 buf = { .ctx.actor = filldir64, + .ctx.flags = flags, .count = count, .current_dir = dirent };
@@ -395,7 +397,7 @@ SYSCALL_DEFINE3(getdents64, unsigned int, fd,if (!f.file)return -EBADF ; - error =vfs_getdents (f.file, dirent, count); + error =vfs_getdents (f.file, dirent, count,0 );fdput_pos (f);return error;
include/linux/fs.h +8 -0
@@ -1719,8 +1719,16 @@ typedef bool (*filldir_t)(struct dir_context *, const char *, int, loff_t, u64,struct dir_context {filldir_t actor;loff_t pos; +unsigned long flags; }; +/* +* flags for dir_context flags +* DIR_CONTEXT_F_NOWAIT: Request non-blocking iterate +* (requires file->f_mode & FMODE_NOWAIT) +*/ +#define DIR_CONTEXT_F_NOWAIT (1 << 0) +/* * These flags let !MMU mmap() govern direct device mapping vs immediate * copying more easily for MAP_PRIVATE, especially for ROM filesystems.
Signature · 1 line
2.25.1
PATCH v5 5/29 vfs: add a vfs helper for io_uring file pos lock
In reply to Hao XuFrom: Hao Xu <[email protected]>
Add a vfs helper file_pos_lock_nowait() for io_uring usage. The function have conditional nowait logic, i.e. if nowait is needed, return -EAGAIN when trylock fails.
--- fs/file.c | 13 +++++++++++++ include/linux/file.h | 2 ++ 2 files changed, 15 insertions(+)
fs/file.c +13 -0
@@ -1053,6 +1053,19 @@ void __f_unlock_pos(struct file *f)mutex_unlock (&f->f_pos_lock); } +int file_pos_lock_nowait (struct file *file,bool nowait) +{ +if (!(file->f_mode &FMODE_ATOMIC_POS )) +return 0 ; + +if (!nowait) +mutex_lock (&file->f_pos_lock); +else if (!mutex_trylock (&file->f_pos_lock)) +return -EAGAIN ; + +return 1 ; +} +/* * We only lock f_pos if we have threads or if the file might be * shared with another process. In both cases we'll have an elevated
include/linux/file.h +2 -0
@@ -81,6 +81,8 @@ static inline void fdput_pos(struct fd f)fdput (f); } +extern int file_pos_lock_nowait (struct file *file,bool nowait); + DEFINE_CLASS(fd, struct fd, fdput(_T), fdget(fd), int fd)extern int f_dupfd (unsigned int from,struct file *file,unsigned flags);
Signature · 1 line
2.25.1
PATCH v5 6/29 vfs: add file_pos_unlock() for io_uring usage
In reply to Hao XuFrom: Hao Xu <[email protected]>
Add a helper to unlock f_pos_lock without any condition. Introduce this since io_uring handles f_pos_lock not with a fd struct, thus FDPUT_POS_UNLOCK isn't used.
--- include/linux/file.h | 5 +++++ 1 file changed, 5 insertions(+)
include/linux/file.h +5 -0
@@ -81,6 +81,11 @@ static inline void fdput_pos(struct fd f)fdput (f); } +static inline void file_pos_unlock (struct file *file) +{ +__f_unlock_pos (file); +} +extern int file_pos_lock_nowait (struct file *file,bool nowait); DEFINE_CLASS(fd, struct fd, fdput(_T), fdget(fd), int fd)
Signature · 1 line
2.25.1
PATCH v5 7/29 vfs: add a nowait parameter for touch_atime()
In reply to Hao XuFrom: Hao Xu <[email protected]>
Add a nowait boolean parameter for touch_atime() to support nowait semantics. It is true only when io_uring is the initial caller.
--- fs/cachefiles/namei.c | 2 +- fs/ecryptfs/file.c | 4 ++-- fs/inode.c | 7 ++++--- fs/namei.c | 4 ++-- fs/nfsd/vfs.c | 2 +- fs/overlayfs/file.c | 2 +- fs/overlayfs/inode.c | 2 +- fs/stat.c | 2 +- include/linux/fs.h | 4 ++-- kernel/bpf/inode.c | 4 ++-- net/unix/af_unix.c | 4 ++-- 11 files changed, 19 insertions(+), 18 deletions(-)
fs/cachefiles/namei.c +1 -1
@@ -591,7 +591,7 @@ static bool cachefiles_open_file(struct cachefiles_object *object,* used to keep track of culling, and atimes are only updated by read, * write and readdir but not lookup or open). */ -touch_atime (&file->f_path); +touch_atime (&file->f_path, false);dput (dentry);return true;
fs/ecryptfs/file.c +2 -2
@@ -39,7 +39,7 @@ static ssize_t ecryptfs_read_update_atime(struct kiocb *iocb, rc =generic_file_read_iter (iocb, to);if (rc >=0 ) { path =ecryptfs_dentry_to_lower_path (file->f_path.dentry); -touch_atime (path); +touch_atime (path, false); }return rc; }
@@ -64,7 +64,7 @@ static ssize_t ecryptfs_splice_read_update_atime(struct file *in, loff_t *ppos, rc =filemap_splice_read (in, ppos, pipe, len, flags);if (rc >=0 ) { path =ecryptfs_dentry_to_lower_path (in->f_path.dentry); -touch_atime (path); +touch_atime (path, false); }return rc; }
fs/inode.c +4 -3
@@ -1961,17 +1961,17 @@ bool atime_needs_update(const struct path *path, struct inode *inode)return true; } -void touch_atime (const struct path *path) +int touch_atime (const struct path *path,bool nowait) {struct vfsmount *mnt = path->mnt;struct inode *inode =d_inode (path->dentry);struct timespec64 now;if (!atime_needs_update (path, inode)) -return ; +return 0 ;if (!sb_start_write_trylock (inode->i_sb)) -return ; +return 0 ;if (__mnt_want_write (mnt) !=0 ) goto skip_update;
@@ -1989,6 +1989,7 @@ void touch_atime(const struct path *path)__mnt_drop_write (mnt); skip_update:sb_end_write (inode->i_sb); +return 0 ; }EXPORT_SYMBOL (touch_atime);
fs/namei.c +2 -2
@@ -1776,12 +1776,12 @@ static const char *pick_link(struct nameidata *nd, struct path *link,return ERR_PTR (-ELOOP );if (!(nd->flags &LOOKUP_RCU )) { -touch_atime (&last->link); +touch_atime (&last->link, false);cond_resched (); }else if (atime_needs_update (&last->link, inode)) {if (!try_to_unlazy (nd))return ERR_PTR (-ECHILD ); -touch_atime (&last->link); +touch_atime (&last->link, false); } error = security_inode_follow_link(link->dentry, inode,
fs/nfsd/vfs.c +1 -1
@@ -1569,7 +1569,7 @@ nfsd_readlink(struct svc_rqst *rqstp, struct svc_fh *fhp, char *buf, int *lenp)if (unlikely (!d_is_symlink (path.dentry)))return nfserr_inval; -touch_atime (&path); +touch_atime (&path, false); link =vfs_get_link (path.dentry, &done);if (IS_ERR (link))
fs/overlayfs/file.c +1 -1
@@ -255,7 +255,7 @@ static void ovl_file_accessed(struct file *file) inode->i_ctime = upperinode->i_ctime; } -touch_atime (&file->f_path); +touch_atime (&file->f_path, false); }static rwf_t ovl_iocb_to_rwf (int ifl)
fs/overlayfs/inode.c +1 -1
@@ -703,7 +703,7 @@ int ovl_update_time(struct inode *inode, struct timespec64 *ts, int flags) };if (upperpath.dentry) { -touch_atime (&upperpath); +touch_atime (&upperpath, false); inode->i_atime =d_inode (upperpath.dentry)->i_atime; } }
fs/stat.c +1 -1
@@ -485,7 +485,7 @@ static int do_readlinkat(int dfd, const char __user *pathname,if (d_is_symlink (path.dentry) || inode->i_op->readlink) { error =security_inode_readlink (path.dentry);if (!error) { -touch_atime (&path); +touch_atime (&path, false); error =vfs_readlink (path.dentry, buf, bufsiz); } }
include/linux/fs.h +2 -2
@@ -2201,13 +2201,13 @@ enum file_time_flags { };extern bool atime_needs_update (const struct path *,struct inode *); -extern void touch_atime (const struct path *); +extern int touch_atime (const struct path *path,bool nowait);int inode_update_time (struct inode *inode,struct timespec64 *time,int flags);static inline void file_accessed (struct file *file) {if (!(file->f_flags &O_NOATIME )) -touch_atime (&file->f_path); +touch_atime (&file->f_path, false); }extern int file_modified (struct file *file);
kernel/bpf/inode.c +2 -2
@@ -517,7 +517,7 @@ static void *bpf_obj_do_get(int path_fd, const char __user *pathname, raw =bpf_any_get (inode->i_private, *type);if (!IS_ERR (raw)) -touch_atime (&path); +touch_atime (&path, false);path_put (&path);return raw;
@@ -591,7 +591,7 @@ struct bpf_prog *bpf_prog_get_type_path(const char *name, enum bpf_prog_type typreturn ERR_PTR (ret); prog =__get_prog_inode (d_backing_inode (path.dentry), type);if (!IS_ERR (prog)) -touch_atime (&path); +touch_atime (&path, false);path_put (&path);return prog; }
net/unix/af_unix.c +2 -2
@@ -1084,7 +1084,7 @@ static struct sock *unix_find_bsd(struct sockaddr_un *sunaddr, int addr_len, err = -EPROTOTYPE ;if (sk->sk_type == type) -touch_atime (&path); +touch_atime (&path, false);else goto sock_put;
@@ -1114,7 +1114,7 @@ static struct sock *unix_find_abstract(struct net *net, dentry =unix_sk (sk)->path.dentry;if (dentry) -touch_atime (&unix_sk (sk)->path); +touch_atime (&unix_sk (sk)->path, false);return sk; }
Signature · 1 line
2.25.1
PATCH v5 8/29 vfs: add nowait parameter for file_accessed()
In reply to Hao XuFrom: Hao Xu <[email protected]>
Add a boolean parameter for file_accessed() to support nowait semantics. Currently it is true only with io_uring as its initial caller.
--- arch/s390/hypfs/inode.c | 2 +- block/fops.c | 2 +- fs/btrfs/file.c | 2 +- fs/btrfs/inode.c | 2 +- fs/coda/dir.c | 4 ++-- fs/ext2/file.c | 4 ++-- fs/ext4/file.c | 6 +++--- fs/f2fs/file.c | 4 ++-- fs/fuse/dax.c | 2 +- fs/fuse/file.c | 4 ++-- fs/gfs2/file.c | 2 +- fs/hugetlbfs/inode.c | 2 +- fs/nilfs2/file.c | 2 +- fs/orangefs/file.c | 2 +- fs/orangefs/inode.c | 2 +- fs/pipe.c | 2 +- fs/ramfs/file-nommu.c | 2 +- fs/readdir.c | 2 +- fs/smb/client/cifsfs.c | 2 +- fs/splice.c | 2 +- fs/ubifs/file.c | 2 +- fs/udf/file.c | 2 +- fs/xfs/xfs_file.c | 6 +++--- fs/zonefs/file.c | 4 ++-- include/linux/fs.h | 5 +++-- mm/filemap.c | 8 ++++---- mm/shmem.c | 6 +++--- 27 files changed, 43 insertions(+), 42 deletions(-)
arch/s390/hypfs/inode.c +1 -1
@@ -157,7 +157,7 @@ static ssize_t hypfs_read_iter(struct kiocb *iocb, struct iov_iter *to)if (!count)return -EFAULT ; iocb->ki_pos = pos + count; -file_accessed (file); +file_accessed (file, false);return count; }
block/fops.c +1 -1
@@ -601,7 +601,7 @@ static ssize_t blkdev_read_iter(struct kiocb *iocb, struct iov_iter *to) ret =kiocb_write_and_wait (iocb, count);if (ret <0 ) goto reexpand; -file_accessed (iocb->ki_filp); +file_accessed (iocb->ki_filp, false); ret =blkdev_direct_IO (iocb, to);if (ret >=0 ) {
fs/btrfs/file.c +1 -1
@@ -2013,7 +2013,7 @@ static int btrfs_file_mmap(struct file *filp, struct vm_area_struct *vma)if (!mapping->a_ops->read_folio)return -ENOEXEC ; -file_accessed (filp); +file_accessed (filp, false); vma->vm_ops = &btrfs_file_vm_ops;return 0 ;
fs/btrfs/inode.c +1 -1
@@ -10153,7 +10153,7 @@ ssize_t btrfs_encoded_read(struct kiocb *iocb, struct iov_iter *iter,struct extent_map *em;bool unlocked = false; -file_accessed (iocb->ki_filp); +file_accessed (iocb->ki_filp, false);btrfs_inode_lock (inode,BTRFS_ILOCK_SHARED );
fs/coda/dir.c +2 -2
@@ -436,12 +436,12 @@ static int coda_readdir(struct file *coda_file, struct dir_context *ctx)if (host_file->f_op->iterate_shared) {inode_lock_shared (host_inode); ret = host_file->f_op->iterate_shared (host_file, ctx); -file_accessed (host_file); +file_accessed (host_file, false);inode_unlock_shared (host_inode); }else {inode_lock (host_inode); ret = host_file->f_op->iterate (host_file, ctx); -file_accessed (host_file); +file_accessed (host_file, false);inode_unlock (host_inode); } }
fs/ext2/file.c +2 -2
@@ -44,7 +44,7 @@ static ssize_t ext2_dax_read_iter(struct kiocb *iocb, struct iov_iter *to) ret =dax_iomap_rw (iocb, to, &ext2_iomap_ops);inode_unlock_shared (inode); -file_accessed (iocb->ki_filp); +file_accessed (iocb->ki_filp, false);return ret; }
@@ -127,7 +127,7 @@ static int ext2_file_mmap(struct file *file, struct vm_area_struct *vma)if (!IS_DAX (file_inode (file)))return generic_file_mmap (file, vma); -file_accessed (file); +file_accessed (file, false); vma->vm_ops = &ext2_dax_vm_ops;return 0 ; }
fs/ext4/file.c +3 -3
@@ -94,7 +94,7 @@ static ssize_t ext4_dio_read_iter(struct kiocb *iocb, struct iov_iter *to) ret =iomap_dio_rw (iocb, to, &ext4_iomap_ops,NULL ,0 ,NULL ,0 );inode_unlock_shared (inode); -file_accessed (iocb->ki_filp); +file_accessed (iocb->ki_filp, false);return ret; }
@@ -122,7 +122,7 @@ static ssize_t ext4_dax_read_iter(struct kiocb *iocb, struct iov_iter *to) ret =dax_iomap_rw (iocb, to, &ext4_iomap_ops);inode_unlock_shared (inode); -file_accessed (iocb->ki_filp); +file_accessed (iocb->ki_filp, false); return ret; }#endif
@@ -820,7 +820,7 @@ static int ext4_file_mmap(struct file *file, struct vm_area_struct *vma)if (!daxdev_mapping_supported (vma, dax_dev))return -EOPNOTSUPP ; -file_accessed (file); +file_accessed (file, false);if (IS_DAX (file_inode (file))) { vma->vm_ops = &ext4_dax_vm_ops;vm_flags_set (vma,VM_HUGEPAGE );
fs/f2fs/file.c +2 -2
@@ -524,7 +524,7 @@ static int f2fs_file_mmap(struct file *file, struct vm_area_struct *vma)if (!f2fs_is_compress_backend_ready (inode))return -EOPNOTSUPP ; -file_accessed (file); +file_accessed (file, false); vma->vm_ops = &f2fs_file_vm_ops;set_inode_flag (inode,FI_MMAP_FILE );return 0 ;
@@ -4380,7 +4380,7 @@ static ssize_t f2fs_dio_read_iter(struct kiocb *iocb, struct iov_iter *to)f2fs_up_read (&fi->i_gc_rwsem[READ ]); -file_accessed (file); +file_accessed (file, false); out:trace_f2fs_direct_IO_exit (inode, pos, count,READ , ret);return ret;
fs/fuse/dax.c +1 -1
@@ -858,7 +858,7 @@ static const struct vm_operations_struct fuse_dax_vm_ops = {int fuse_dax_mmap (struct file *file,struct vm_area_struct *vma) { -file_accessed (file); +file_accessed (file, false); vma->vm_ops = &fuse_dax_vm_ops;vm_flags_set (vma,VM_MIXEDMAP |VM_HUGEPAGE );return 0 ;
fs/fuse/file.c +2 -2
@@ -2496,7 +2496,7 @@ static int fuse_file_mmap(struct file *file, struct vm_area_struct *vma)if ((vma->vm_flags &VM_SHARED ) && (vma->vm_flags &VM_MAYWRITE ))fuse_link_write_file (file); -file_accessed (file); +file_accessed (file, false); vma->vm_ops = &fuse_file_vm_ops;return 0 ; }
@@ -3193,7 +3193,7 @@ static ssize_t __fuse_copy_file_range(struct file *file_in, loff_t pos_in,clear_bit (FUSE_I_SIZE_UNSTABLE , &fi_out->state);inode_unlock (inode_out); -file_accessed (file_in); +file_accessed (file_in, false);fuse_flush_time_update (inode_out);
fs/gfs2/file.c +1 -1
@@ -601,7 +601,7 @@ static int gfs2_mmap(struct file *file, struct vm_area_struct *vma)return error;/* grab lock to update inode */ gfs2_glock_dq_uninit (&i_gh); -file_accessed (file); +file_accessed (file, false); } vma->vm_ops = &gfs2_vm_ops;
fs/hugetlbfs/inode.c +1 -1
@@ -161,7 +161,7 @@ static int hugetlbfs_file_mmap(struct file *file, struct vm_area_struct *vma)return -EINVAL ;inode_lock (inode); -file_accessed (file); +file_accessed (file, false); ret = -ENOMEM ; if (!hugetlb_reserve_pages(inode,
fs/nilfs2/file.c +1 -1
@@ -119,7 +119,7 @@ static const struct vm_operations_struct nilfs_file_vm_ops = {static int nilfs_file_mmap (struct file *file,struct vm_area_struct *vma) { -file_accessed (file); +file_accessed (file, false); vma->vm_ops = &nilfs_file_vm_ops;return 0 ; }
fs/orangefs/file.c +1 -1
@@ -412,7 +412,7 @@ static int orangefs_file_mmap(struct file *file, struct vm_area_struct *vma)/* set the sequential readahead hint */ vm_flags_mod (vma,VM_SEQ_READ ,VM_RAND_READ ); -file_accessed (file); +file_accessed (file, false); vma->vm_ops = &orangefs_file_vm_ops;return 0 ; }
fs/orangefs/inode.c +1 -1
@@ -597,7 +597,7 @@ static ssize_t orangefs_direct_IO(struct kiocb *iocb, ret = total_count;if (ret >0 ) {if (type ==ORANGEFS_IO_READ ) { -file_accessed (file); +file_accessed (file, false); }else {file_update_time (file);if (*offset >i_size_read (inode))
fs/pipe.c +1 -1
@@ -393,7 +393,7 @@ pipe_read(struct kiocb *iocb, struct iov_iter *to)wake_up_interruptible_sync_poll (&pipe->rd_wait,EPOLLIN |EPOLLRDNORM );kill_fasync (&pipe->fasync_writers,SIGIO ,POLL_OUT );if (ret >0 ) -file_accessed (filp); +file_accessed (filp, false);return ret; }
fs/ramfs/file-nommu.c +1 -1
@@ -267,7 +267,7 @@ static int ramfs_nommu_mmap(struct file *file, struct vm_area_struct *vma)if (!is_nommu_shared_mapping (vma->vm_flags))return -ENOSYS ; -file_accessed (file); +file_accessed (file, false); vma->vm_ops = &generic_file_vm_ops;return 0 ; }
fs/readdir.c +1 -1
@@ -68,7 +68,7 @@ int iterate_dir(struct file *file, struct dir_context *ctx) res = file->f_op->iterate (file, ctx); file->f_pos = ctx->pos;fsnotify_access (file); -file_accessed (file); +file_accessed (file, ctx->flags &DIR_CONTEXT_F_NOWAIT ); }if (shared)inode_unlock_shared (inode);
fs/smb/client/cifsfs.c +1 -1
@@ -1307,7 +1307,7 @@ ssize_t cifs_file_copychunk_range(unsigned int xid, rc = target_tcon->ses->server->ops->copychunk_range (xid, smb_file_src, smb_file_target, off, len, destoff); -file_accessed (src_file); +file_accessed (src_file, false);/* force revalidate of size and timestamps of target file now * that target is updated on the server
fs/splice.c +1 -1
@@ -1104,7 +1104,7 @@ ssize_t splice_direct_to_actor(struct file *in, struct splice_desc *sd, done: pipe->tail = pipe->head =0 ; -file_accessed (in); +file_accessed (in, false);return bytes; read_failure:
fs/ubifs/file.c +1 -1
@@ -1603,7 +1603,7 @@ static int ubifs_file_mmap(struct file *file, struct vm_area_struct *vma) vma->vm_ops = &ubifs_file_vm_ops;if (IS_ENABLED (CONFIG_UBIFS_ATIME_SUPPORT )) -file_accessed (file); +file_accessed (file, false);return 0 ; }
fs/udf/file.c +1 -1
@@ -191,7 +191,7 @@ static int udf_release_file(struct inode *inode, struct file *filp)static int udf_file_mmap (struct file *file,struct vm_area_struct *vma) { -file_accessed (file); +file_accessed (file, false); vma->vm_ops = &udf_file_vm_ops;return 0 ;
fs/xfs/xfs_file.c +3 -3
@@ -227,7 +227,7 @@ xfs_file_dio_read(if (!iov_iter_count (to))return 0 ;/* skip atime */ -file_accessed (iocb->ki_filp); +file_accessed (iocb->ki_filp, false); ret =xfs_ilock_iocb (iocb,XFS_IOLOCK_SHARED );if (ret)
@@ -257,7 +257,7 @@ xfs_file_dax_read( ret =dax_iomap_rw (iocb, to, &xfs_read_iomap_ops);xfs_iunlock (ip,XFS_IOLOCK_SHARED ); -file_accessed (iocb->ki_filp); +file_accessed (iocb->ki_filp, false);return ret; }
@@ -1434,7 +1434,7 @@ xfs_file_mmap(if (!daxdev_mapping_supported (vma, target->bt_daxdev))return -EOPNOTSUPP ; -file_accessed (file); +file_accessed (file, false); vma->vm_ops = &xfs_file_vm_ops;if (IS_DAX (inode))vm_flags_set (vma,VM_HUGEPAGE );
fs/zonefs/file.c +2 -2
@@ -323,7 +323,7 @@ static int zonefs_file_mmap(struct file *file, struct vm_area_struct *vma) (vma->vm_flags & VM_SHARED) && (vma->vm_flags & VM_MAYWRITE))return -EINVAL ; -file_accessed (file); +file_accessed (file, false); vma->vm_ops = &zonefs_file_vm_ops;return 0 ;
@@ -736,7 +736,7 @@ static ssize_t zonefs_file_read_iter(struct kiocb *iocb, struct iov_iter *to) ret = -EINVAL ; goto inode_unlock; } -file_accessed (iocb->ki_filp); +file_accessed (iocb->ki_filp, false); ret =iomap_dio_rw (iocb, to, &zonefs_read_iomap_ops, &zonefs_read_dio_ops,0 ,NULL ,0 ); }else {
include/linux/fs.h +3 -2
@@ -2204,10 +2204,11 @@ extern bool atime_needs_update(const struct path *, struct inode *);extern int touch_atime (const struct path *path,bool nowait);int inode_update_time (struct inode *inode,struct timespec64 *time,int flags); -static inline void file_accessed (struct file *file) +static inline int file_accessed (struct file *file,bool nowait) {if (!(file->f_flags &O_NOATIME )) -touch_atime (&file->f_path, false); +return touch_atime (&file->f_path, nowait); +return 0 ; }extern int file_modified (struct file *file);
mm/filemap.c +4 -4
@@ -2723,7 +2723,7 @@ ssize_t filemap_read(struct kiocb *iocb, struct iov_iter *iter,folio_batch_init (&fbatch); }while (iov_iter_count (iter) && iocb->ki_pos < isize && !error); -file_accessed (filp); +file_accessed (filp, false);return already_read ? already_read : error; }
@@ -2809,7 +2809,7 @@ generic_file_read_iter(struct kiocb *iocb, struct iov_iter *iter) retval =kiocb_write_and_wait (iocb, count);if (retval <0 )return retval; -file_accessed (file); +file_accessed (file, false); retval = mapping->a_ops->direct_IO (iocb, iter);if (retval >=0 ) {
@@ -2978,7 +2978,7 @@ ssize_t filemap_splice_read(struct file *in, loff_t *ppos, out:folio_batch_release (&fbatch); -file_accessed (in); +file_accessed (in, false);return total_spliced ? total_spliced : error; }
@@ -3613,7 +3613,7 @@ int generic_file_mmap(struct file *file, struct vm_area_struct *vma)if (!mapping->a_ops->read_folio)return -ENOEXEC ; -file_accessed (file); +file_accessed (file, false); vma->vm_ops = &generic_file_vm_ops;return 0 ; }
mm/shmem.c +3 -3
@@ -2317,7 +2317,7 @@ static int shmem_mmap(struct file *file, struct vm_area_struct *vma)/* arm64 - allow memory tagging on RAM-based files */ vm_flags_set (vma,VM_MTE_ALLOWED ); -file_accessed (file); +file_accessed (file, false);/* This is anonymous shared memory if it is unlinked at the time of mmap */ if (inode->i_nlink) vma->vm_ops = &shmem_vm_ops;
@@ -2727,7 +2727,7 @@ static ssize_t shmem_file_read_iter(struct kiocb *iocb, struct iov_iter *to) } *ppos = ((loff_t ) index <<PAGE_SHIFT ) + offset; -file_accessed (file); +file_accessed (file, false);return retval ? retval : error; }
@@ -2859,7 +2859,7 @@ static ssize_t shmem_file_splice_read(struct file *in, loff_t *ppos,if (folio)folio_put (folio); -file_accessed (in); +file_accessed (in, false);return total_spliced ? total_spliced : error; }
Signature · 1 line
2.25.1
PATCH v5 9/29 vfs: move file_accessed() to the beginning of iterate_dir()
In reply to Hao XuFrom: Hao Xu <[email protected]>
Move file_accessed() to the beginning of iterate_dir() so that we don't need to rollback all the work done when file_accessed() returns -EAGAIN at the end of getdents.
--- fs/readdir.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)
fs/readdir.c +6 -1
@@ -61,6 +61,10 @@ int iterate_dir(struct file *file, struct dir_context *ctx) res = -ENOENT ;if (!IS_DEADDIR (inode)) { + res =file_accessed (file, ctx->flags &DIR_CONTEXT_F_NOWAIT ); +if (res == -EAGAIN ) + goto out_unlock; + ctx->pos = file->f_pos;if (shared) res = file->f_op->iterate_shared (file, ctx);
@@ -68,8 +72,9 @@ int iterate_dir(struct file *file, struct dir_context *ctx) res = file->f_op->iterate (file, ctx); file->f_pos = ctx->pos;fsnotify_access (file); -file_accessed (file, ctx->flags &DIR_CONTEXT_F_NOWAIT ); } + +out_unlock:if (shared)inode_unlock_shared (inode);else
Signature · 1 line
2.25.1
PATCH v5 10/29 vfs: add S_NOWAIT for nowait time update
In reply to Hao XuFrom: Hao Xu <[email protected]>
Add a new time flag S_NOWAIT to support nowait time update. Deliver it to specific filesystem and error out -EAGAIN when it would block.
--- fs/inode.c | 9 +++++---- fs/xfs/xfs_iops.c | 8 +++++++- include/linux/fs.h | 1 + 3 files changed, 13 insertions(+), 5 deletions(-)
fs/inode.c +5 -4
@@ -1966,12 +1966,13 @@ int touch_atime(const struct path *path, bool nowait)struct vfsmount *mnt = path->mnt;struct inode *inode =d_inode (path->dentry);struct timespec64 now; +int ret =0 ;if (!atime_needs_update (path, inode)) -return 0 ; +return ret;if (!sb_start_write_trylock (inode->i_sb)) -return 0 ; +return ret;if (__mnt_want_write (mnt) !=0 ) goto skip_update;
@@ -1985,11 +1986,11 @@ int touch_atime(const struct path *path, bool nowait)* of the fs read only, e.g. subvolumes in Btrfs. */ now =current_time (inode); -inode_update_time (inode, &now,S_ATIME ); + ret =inode_update_time (inode, &now,S_ATIME | (nowait ?S_NOWAIT :0 ));__mnt_drop_write (mnt); skip_update:sb_end_write (inode->i_sb); -return 0 ; +return ret; }EXPORT_SYMBOL (touch_atime);
fs/xfs/xfs_iops.c +7 -1
@@ -1053,7 +1053,13 @@ xfs_vn_update_time(if (error)return error; -xfs_ilock (ip,XFS_ILOCK_EXCL ); +if (flags &S_NOWAIT ) { +if (!xfs_ilock_nowait (ip,XFS_ILOCK_EXCL )) +return -EAGAIN ; + }else { +xfs_ilock (ip,XFS_ILOCK_EXCL ); + } +if (flags &S_CTIME ) inode->i_ctime = *now;if (flags &S_MTIME )
include/linux/fs.h +1 -0
@@ -2198,6 +2198,7 @@ enum file_time_flags {S_MTIME =2 ,S_CTIME =4 ,S_VERSION =8 , + S_NOWAIT = 16, };extern bool atime_needs_update (const struct path *,struct inode *);
Signature · 1 line
2.25.1
PATCH v5 11/29 vfs: trylock inode->i_rwsem in iterate_dir() to support nowait
In reply to Hao XuFrom: Hao Xu <[email protected]>
Trylock inode->i_rwsem in iterate_dir() to support nowait semantics and error out -EAGAIN when there is contention.
--- fs/readdir.c | 20 ++++++++++++++------ 1 file changed, 14 insertions(+), 6 deletions(-)
fs/readdir.c +14 -6
@@ -43,6 +43,8 @@ int iterate_dir(struct file *file, struct dir_context *ctx)struct inode *inode =file_inode (file);bool shared = false;int res = -ENOTDIR ; +bool nowait; +if (file->f_op->iterate_shared) shared = true;else if (!file->f_op->iterate)
@@ -52,16 +54,22 @@ int iterate_dir(struct file *file, struct dir_context *ctx)if (res) goto out; -if (shared) - res =down_read_killable (&inode->i_rwsem); -else - res =down_write_killable (&inode->i_rwsem); -if (res) + nowait = ctx->flags &DIR_CONTEXT_F_NOWAIT ; +if (nowait) { + res = shared ?down_read_trylock (&inode->i_rwsem) : +down_write_trylock (&inode->i_rwsem); +if (!res) + res = -EAGAIN ; + }else { + res = shared ?down_read_killable (&inode->i_rwsem) : +down_write_killable (&inode->i_rwsem); + } +if (res <0 ) goto out; res = -ENOENT ;if (!IS_DEADDIR (inode)) { - res =file_accessed (file, ctx->flags &DIR_CONTEXT_F_NOWAIT ); + res =file_accessed (file, nowait);if (res == -EAGAIN ) goto out_unlock;
Signature · 1 line
2.25.1
PATCH v5 12/29 xfs: enforce GFP_NOIO implicitly during nowait time update
In reply to Hao XuFrom: Hao Xu <[email protected]>
Enforce GFP_NOIO logic implicitly by set pflags if we are in nowait time update process. Nowait semantics means no waiting for IO, therefore GFP_NOIO is needed.
--- fs/xfs/xfs_iops.c | 22 +++++++++++++++++----- 1 file changed, 17 insertions(+), 5 deletions(-)
fs/xfs/xfs_iops.c +17 -5
@@ -1037,6 +1037,8 @@ xfs_vn_update_time(int log_flags =XFS_ILOG_TIMESTAMP ;struct xfs_trans *tp;int error; +int old_pflags; +bool nowait = flags &S_NOWAIT ;trace_xfs_update_time (ip);
@@ -1049,13 +1051,18 @@ xfs_vn_update_time( log_flags |=XFS_ILOG_CORE ; } +if (nowait) + old_pflags =memalloc_noio_save (); + error =xfs_trans_alloc (mp, &M_RES (mp)->tr_fsyncts,0 ,0 ,0 , &tp);if (error) -return error; + goto out; -if (flags &S_NOWAIT ) { -if (!xfs_ilock_nowait (ip,XFS_ILOCK_EXCL )) -return -EAGAIN ; +if (nowait) { +if (!xfs_ilock_nowait (ip,XFS_ILOCK_EXCL )) { + error = -EAGAIN ; + goto out; + } }else {xfs_ilock (ip,XFS_ILOCK_EXCL ); }
@@ -1069,7 +1076,12 @@ xfs_vn_update_time(xfs_trans_ijoin (tp, ip,XFS_ILOCK_EXCL );xfs_trans_log_inode (tp, ip, log_flags); -return xfs_trans_commit (tp); + error =xfs_trans_commit (tp); + +out: +if (nowait) +memalloc_noio_restore (old_pflags); +return error; }STATIC int
Signature · 1 line
2.25.1
Matthew Wilcox
maintainerRe: [PATCH 12/29] xfs: enforce GFP_NOIO implicitly during nowait time update
Quoted 11 lines · Hao Xu
+++
b/fs/xfs/xfs_iops.cfs/xfs/xfs_iops.c +2
-0@@ -1037,6 +1037,8 @@ xfs_vn_update_time(int log_flags =XFS_ILOG_TIMESTAMP ;struct xfs_trans *tp;int error; +int old_pflags; +bool nowait = flags &S_NOWAIT ;trace_xfs_update_time (ip);
fs/xfs/xfs_iops.c xfs_vn_update_time( · line 1057
Quoted 7 lines
fs/xfs/xfs_iops.c +3
-0@@ -1049,13 +1051,18 @@ xfs_vn_update_time( log_flags |=XFS_ILOG_CORE ; } +if (nowait) + old_pflags =memalloc_noio_save (); + error =xfs_trans_alloc (mp, &M_RES (mp)->tr_fsyncts,0 ,0 ,0 , &tp);
This is an abuse of the memalloc_noio_save() interface. You shouldn't be setting it around individual allocations; it's the part of the kernel which decides "I can't afford to do I/O" that should be setting it. In this case, it should probably be set by io_uring, way way way up at the top.
But Jens didn't actually answer my question about that:
PATCH v5 13/29 xfs: make xfs_trans_alloc() support nowait semantics
In reply to Hao XuFrom: Hao Xu <[email protected]>
There are locks in xfs_trans_alloc(), spot them and apply trylock logic. Make them return -EAGAIN when it would block. To achieve this, add nowait parameter for those functions in the path. Besides, add a generic transaction flag XFS_TRANS_NOWAIT to deliver nowait info.
--- fs/xfs/libxfs/xfs_shared.h | 2 ++ fs/xfs/xfs_iops.c | 3 ++- fs/xfs/xfs_trans.c | 21 ++++++++++++++++++--- 3 files changed, 22 insertions(+), 4 deletions(-)
fs/xfs/libxfs/xfs_shared.h +2 -0
@@ -83,6 +83,8 @@ void xfs_log_get_max_trans_res(struct xfs_mount *mp,* made then this algorithm will eventually find all the space it needs. */ #define XFS_TRANS_LOWMODE 0x100/* allocate in low space mode */ +/* Transaction should follow nowait semantics */ +#define XFS_TRANS_NOWAIT (1u << 9)/* * Field values for xfs_trans_mod_sb.
fs/xfs/xfs_iops.c +2 -1
@@ -1054,7 +1054,8 @@ xfs_vn_update_time(if (nowait) old_pflags =memalloc_noio_save (); - error =xfs_trans_alloc (mp, &M_RES (mp)->tr_fsyncts,0 ,0 ,0 , &tp); + error =xfs_trans_alloc (mp, &M_RES (mp)->tr_fsyncts,0 ,0 , + nowait ?XFS_TRANS_NOWAIT :0 , &tp);if (error) goto out;
fs/xfs/xfs_trans.c +18 -3
@@ -251,6 +251,9 @@ xfs_trans_alloc(struct xfs_trans *tp;bool want_retry = true;int error; +bool nowait = flags &XFS_TRANS_NOWAIT ; +gfp_t gfp_flags =GFP_KERNEL | + (nowait ?0 : __GFP_NOFAIL);/* * Allocate the handle before we do our freeze accounting and setting up
@@ -258,9 +261,21 @@ xfs_trans_alloc(* by doing GFP_KERNEL allocations inside sb_start_intwrite(). */ retry: - tp =kmem_cache_zalloc (xfs_trans_cache,GFP_KERNEL | __GFP_NOFAIL); -if (!(flags &XFS_TRANS_NO_WRITECOUNT )) -sb_start_intwrite (mp->m_super); + tp =kmem_cache_zalloc (xfs_trans_cache, gfp_flags); +if (!tp) +return -EAGAIN ; +if (!(flags &XFS_TRANS_NO_WRITECOUNT )) { +if (nowait) { +bool locked =sb_start_intwrite_trylock (mp->m_super); + +if (!locked) { +xfs_trans_cancel (tp); +return -EAGAIN ; + } + }else { +sb_start_intwrite (mp->m_super); + } + }xfs_trans_set_context (tp);/*
Signature · 1 line
2.25.1
PATCH v5 14/29 xfs: support nowait for xfs_log_reserve()
In reply to Hao XuFrom: Hao Xu <[email protected]>
Support nowait logic for xfs_log_reserve(), including add a nowait boolean parameter and error out -EAGAIN for ticket allocation.
--- fs/xfs/xfs_log.c | 18 +++++++++++++----- fs/xfs/xfs_log.h | 5 +++-- fs/xfs/xfs_log_cil.c | 2 +- fs/xfs/xfs_log_priv.h | 2 +- fs/xfs/xfs_trans.c | 5 +++-- 5 files changed, 21 insertions(+), 11 deletions(-)
fs/xfs/xfs_log.c +13 -5
@@ -462,7 +462,8 @@ xfs_log_reserve(int unit_bytes, int cnt, struct xlog_ticket **ticp, - bool permanent) + bool permanent, + bool nowait) { struct xlog *log = mp->m_log;struct xlog_ticket *tic;
@@ -475,7 +476,9 @@ xfs_log_reserve(XFS_STATS_INC (mp, xs_try_logspace);ASSERT (*ticp ==NULL ); - tic =xlog_ticket_alloc (log, unit_bytes, cnt, permanent); + tic =xlog_ticket_alloc (log, unit_bytes, cnt, permanent, nowait); +if (!tic) +return -EAGAIN ; *ticp = tic; xlog_grant_push_ail(log, tic->t_cnt ? tic->t_unit_res * tic->t_cnt
@@ -974,7 +977,7 @@ xlog_unmount_write(struct xlog_ticket *tic =NULL ;int error; - error =xfs_log_reserve (mp,600 ,1 , &tic,0 ); + error =xfs_log_reserve (mp,600 ,1 , &tic,0 , false);if (error) goto out_err;
@@ -3527,12 +3530,17 @@ xlog_ticket_alloc(struct xlog *log, int unit_bytes, int cnt, - bool permanent) + bool permanent, + bool nowait) { struct xlog_ticket *tic;int unit_res; - tic =kmem_cache_zalloc (xfs_log_ticket_cache,GFP_NOFS | __GFP_NOFAIL); +gfp_t gfp_flags =GFP_NOFS | + (nowait ?0 : __GFP_NOFAIL); + tic =kmem_cache_zalloc (xfs_log_ticket_cache, gfp_flags); +if (!tic) +return NULL ; unit_res =xlog_calc_unit_res (log, unit_bytes, &tic->t_iclog_hdrs);
fs/xfs/xfs_log.h +3 -2
@@ -139,8 +139,9 @@ void xfs_log_mount_cancel(struct xfs_mount *);xfs_lsn_t xlog_assign_tail_lsn (struct xfs_mount *mp);xfs_lsn_t xlog_assign_tail_lsn_locked (struct xfs_mount *mp);void xfs_log_space_wake (struct xfs_mount *mp); -int xfs_log_reserve (struct xfs_mount *mp,int length,int count, -struct xlog_ticket **ticket,bool permanent); +int xfs_log_reserve (struct xfs_mount *mp,int length, +int count,struct xlog_ticket **ticket, +bool permanent,bool nowait);int xfs_log_regrant (struct xfs_mount *mp,struct xlog_ticket *tic);void xfs_log_unmount (struct xfs_mount *mp);bool xfs_log_writable (struct xfs_mount *mp);
fs/xfs/xfs_log_cil.c +1 -1
@@ -37,7 +37,7 @@ xlog_cil_ticket_alloc( {struct xlog_ticket *tic; - tic =xlog_ticket_alloc (log,0 ,1 ,0 ); + tic =xlog_ticket_alloc (log,0 ,1 ,0 , false);/* * set the current reservation to zero so we know to steal the basic
fs/xfs/xfs_log_priv.h +1 -1
@@ -503,7 +503,7 @@ extern __le32 xlog_cksum(struct xlog *log, struct xlog_rec_header *rhead,extern struct kmem_cache *xfs_log_ticket_cache;struct xlog_ticket *xlog_ticket_alloc (struct xlog *log,int unit_bytes, -int count,bool permanent); +int count,bool permanent,bool nowait);void xlog_print_tic_res (struct xfs_mount *mp,struct xlog_ticket *ticket);void xlog_print_trans (struct xfs_trans *);
fs/xfs/xfs_trans.c +3 -2
@@ -155,6 +155,7 @@ xfs_trans_reserve(struct xfs_mount *mp = tp->t_mountp;int error =0 ;bool rsvd = (tp->t_flags &XFS_TRANS_RESERVE ) !=0 ; +bool nowait = tp->t_flags &XFS_TRANS_NOWAIT ;/* * Attempt to reserve the needed disk blocks by decrementing
@@ -192,8 +193,8 @@ xfs_trans_reserve( error =xfs_log_regrant (mp, tp->t_ticket); }else { error =xfs_log_reserve (mp, resp->tr_logres, - resp->tr_logcount, - &tp->t_ticket, permanent); + resp->tr_logcount, &tp->t_ticket, + permanent, nowait); }if (error)
Signature · 1 line
2.25.1
PATCH v5 15/29 xfs: don't wait for free space in xlog_grant_head_check() in nowait case
In reply to Hao XuFrom: Hao Xu <[email protected]>
Don't sleep and wait for more space for a log ticket in xlog_grant_head_check() when it is in nowait case.
--- fs/xfs/xfs_log.c | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-)
fs/xfs/xfs_log.c +9 -6
@@ -341,7 +341,8 @@ xlog_grant_head_check(struct xlog *log, struct xlog_grant_head *head, struct xlog_ticket *tic, - int *need_bytes) + int *need_bytes, + bool nowait) { int free_bytes;int error =0 ;
@@ -360,13 +361,15 @@ xlog_grant_head_check(spin_lock (&head->lock);if (!xlog_grant_head_wake (log, head, &free_bytes) || free_bytes < *need_bytes) { - error =xlog_grant_head_wait (log, head, tic, - *need_bytes); + error = nowait ? + -EAGAIN :xlog_grant_head_wait (log, head, tic, + *need_bytes); }spin_unlock (&head->lock); }else if (free_bytes < *need_bytes) {spin_lock (&head->lock); - error =xlog_grant_head_wait (log, head, tic, *need_bytes); + error = nowait ? -EAGAIN :xlog_grant_head_wait (log, head, tic, + *need_bytes);spin_unlock (&head->lock); }
@@ -428,7 +431,7 @@ xfs_log_regrant(trace_xfs_log_regrant (log, tic); error =xlog_grant_head_check (log, &log->l_write_head, tic, - &need_bytes); + &need_bytes, false);if (error) goto out_error;
@@ -487,7 +490,7 @@ xfs_log_reserve(trace_xfs_log_reserve (log, tic); error =xlog_grant_head_check (log, &log->l_reserve_head, tic, - &need_bytes); + &need_bytes, nowait);if (error) goto out_error;
Signature · 1 line
2.25.1
PATCH v5 16/29 xfs: add nowait parameter for xfs_inode_item_init()
In reply to Hao XuFrom: Hao Xu <[email protected]>
Add nowait parameter for xfs_inode_item_init() to support nowait semantics.
--- fs/xfs/libxfs/xfs_trans_inode.c | 3 ++- fs/xfs/xfs_inode_item.c | 12 ++++++++---- fs/xfs/xfs_inode_item.h | 3 ++- 3 files changed, 12 insertions(+), 6 deletions(-)
fs/xfs/libxfs/xfs_trans_inode.c +2 -1
@@ -33,7 +33,8 @@ xfs_trans_ijoin(ASSERT (xfs_isilocked (ip,XFS_ILOCK_EXCL ));if (ip->i_itemp ==NULL ) -xfs_inode_item_init (ip, ip->i_mount); +xfs_inode_item_init (ip, ip->i_mount, + tp->t_flags &XFS_TRANS_NOWAIT ); iip = ip->i_itemp;ASSERT (iip->ili_lock_flags ==0 );
fs/xfs/xfs_inode_item.c +8 -4
@@ -825,21 +825,25 @@ static const struct xfs_item_ops xfs_inode_item_ops = {/* * Initialize the inode log item for a newly allocated (in-core) inode. */ -void +int xfs_inode_item_init (struct xfs_inode *ip, -struct xfs_mount *mp) +struct xfs_mount *mp, +bool nowait) {struct xfs_inode_log_item *iip; +gfp_t gfp_flags =GFP_KERNEL | (nowait ?0 : __GFP_NOFAIL);ASSERT (ip->i_itemp ==NULL ); - iip = ip->i_itemp =kmem_cache_zalloc (xfs_ili_cache, -GFP_KERNEL | __GFP_NOFAIL); + iip = ip->i_itemp =kmem_cache_zalloc (xfs_ili_cache, gfp_flags); +if (!iip) +return -EAGAIN ; iip->ili_inode = ip;spin_lock_init (&iip->ili_lock);xfs_log_item_init (mp, &iip->ili_item,XFS_LI_INODE , &xfs_inode_item_ops); +return 0 ; }/*
fs/xfs/xfs_inode_item.h +2 -1
@@ -42,7 +42,8 @@ static inline int xfs_inode_clean(struct xfs_inode *ip)return !ip->i_itemp || !(ip->i_itemp->ili_fields &XFS_ILOG_ALL ); } -extern void xfs_inode_item_init (struct xfs_inode *,struct xfs_mount *); +extern int xfs_inode_item_init (struct xfs_inode *ip,struct xfs_mount *mp, +bool nowait);extern void xfs_inode_item_destroy (struct xfs_inode *);extern void xfs_iflush_abort (struct xfs_inode *);extern void xfs_iflush_shutdown_abort (struct xfs_inode *);
Signature · 1 line
2.25.1
PATCH v5 17/29 xfs: make xfs_trans_ijoin() error out -EAGAIN
In reply to Hao XuFrom: Hao Xu <[email protected]>
Change return value of xfs_trans_ijoin() to error out -EAGAIN.
--- fs/xfs/libxfs/xfs_trans_inode.c | 13 +++++++++---- fs/xfs/xfs_iops.c | 4 +++- fs/xfs/xfs_trans.h | 2 +- 3 files changed, 13 insertions(+), 6 deletions(-)
fs/xfs/libxfs/xfs_trans_inode.c +9 -4
@@ -23,7 +23,7 @@* The inode must be locked, and it cannot be associated with any transaction. * If lock_flags is non-zero the inode will be unlocked on transaction commit. */ -void +int xfs_trans_ijoin( struct xfs_trans *tp, struct xfs_inode *ip,
@@ -32,9 +32,12 @@ xfs_trans_ijoin(struct xfs_inode_log_item *iip;ASSERT (xfs_isilocked (ip,XFS_ILOCK_EXCL )); -if (ip->i_itemp ==NULL ) -xfs_inode_item_init (ip, ip->i_mount, - tp->t_flags &XFS_TRANS_NOWAIT ); +if (ip->i_itemp ==NULL ) { +int ret =xfs_inode_item_init (ip, ip->i_mount, + tp->t_flags &XFS_TRANS_NOWAIT ); +if (ret == -EAGAIN ) +return ret; + } iip = ip->i_itemp;ASSERT (iip->ili_lock_flags ==0 );
@@ -44,6 +47,8 @@ xfs_trans_ijoin(/* Reset the per-tx dirty context and add the item to the tx. */ iip->ili_dirty_flags =0 ;xfs_trans_add_item (tp, &iip->ili_item); + +return 0 ; }/*
fs/xfs/xfs_iops.c +3 -1
@@ -1075,7 +1075,9 @@ xfs_vn_update_time(if (flags &S_ATIME ) inode->i_atime = *now; -xfs_trans_ijoin (tp, ip,XFS_ILOCK_EXCL ); + error =xfs_trans_ijoin (tp, ip,XFS_ILOCK_EXCL ); +if (error) + goto out;xfs_trans_log_inode (tp, ip, log_flags); error =xfs_trans_commit (tp);
fs/xfs/xfs_trans.h +1 -1
@@ -226,7 +226,7 @@ bool xfs_trans_ordered_buf(xfs_trans_t *, struct xfs_buf *);void xfs_trans_dquot_buf (xfs_trans_t *,struct xfs_buf *,uint );void xfs_trans_inode_alloc_buf (xfs_trans_t *,struct xfs_buf *);void xfs_trans_ichgtime (struct xfs_trans *,struct xfs_inode *,int ); -void xfs_trans_ijoin (struct xfs_trans *,struct xfs_inode *,uint ); +int xfs_trans_ijoin (struct xfs_trans *,struct xfs_inode *,uint );void xfs_trans_log_buf (struct xfs_trans *,struct xfs_buf *,uint ,uint );void xfs_trans_dirty_buf (struct xfs_trans *,struct xfs_buf *);
Signature · 1 line
2.25.1
PATCH v5 18/29 xfs: set XBF_NOWAIT for xfs_buf_read_map if necessary
In reply to Hao XuFrom: Hao Xu <[email protected]>
Set XBF_NOWAIT for xfs_buf_read_map() if necessary.
--- fs/xfs/xfs_trans_buf.c | 2 ++ 1 file changed, 2 insertions(+)
fs/xfs/xfs_trans_buf.c +2 -0
@@ -286,6 +286,8 @@ xfs_trans_read_buf_map(return 0 ; } +if (tp && (tp->t_flags &XFS_TRANS_NOWAIT )) + flags |=XBF_NOWAIT ; error =xfs_buf_read_map (target, map, nmaps, flags, &bp, ops, __return_address);switch (error) {
Signature · 1 line
2.25.1
PATCH v5 19/29 xfs: support nowait memory allocation in _xfs_buf_alloc()
In reply to Hao XuFrom: Hao Xu <[email protected]>
Choose different gfp flags to support nowait memory allocation in _xfs_buf_alloc().
--- fs/xfs/xfs_buf.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)
fs/xfs/xfs_buf.c +6 -1
@@ -220,9 +220,14 @@ _xfs_buf_alloc(struct xfs_buf *bp;int error;int i; +bool nowait = flags &XBF_NOWAIT ; +gfp_t gfp_flags =GFP_NOFS | + (nowait ?0 : __GFP_NOFAIL); *bpp =NULL ; - bp =kmem_cache_zalloc (xfs_buf_cache,GFP_NOFS | __GFP_NOFAIL); + bp =kmem_cache_zalloc (xfs_buf_cache, gfp_flags); +if (!bp) +return -EAGAIN ;/* * We don't want certain flags to appear in b_flags unless they are
Signature · 1 line
2.25.1
PATCH v5 20/29 xfs: distinguish error type of memory allocation failure for nowait case
In reply to Hao XuFrom: Hao Xu <[email protected]>
Previously, if we fail to get the memory we need, -ENOMEM is returned. It can be -EAGAIN now since we support nowait now. Return the latter when it is the case. Involved functions are: _xfs_buf_map_pages(), xfs_buf_get_maps(), xfs_buf_alloc_kmem() and xfs_buf_alloc_pages().
--- fs/xfs/xfs_buf.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-)
fs/xfs/xfs_buf.c +5 -4
@@ -192,7 +192,7 @@ xfs_buf_get_maps( bp->b_maps =kmem_zalloc (map_count *sizeof (struct xfs_buf_map ),KM_NOFS );if (!bp->b_maps) -return -ENOMEM ; +return bp->b_flags &XBF_NOWAIT ? -EAGAIN : -ENOMEM ;return 0 ; }
@@ -339,7 +339,7 @@ xfs_buf_alloc_kmem( bp->b_addr =kmem_alloc (size, kmflag_mask);if (!bp->b_addr) -return -ENOMEM ; +return flags &XBF_NOWAIT ? -EAGAIN : -ENOMEM ;if (((unsigned long )(bp->b_addr + size -1 ) &PAGE_MASK ) != ((unsigned long )bp->b_addr &PAGE_MASK )) {
@@ -363,6 +363,7 @@ xfs_buf_alloc_pages( {gfp_t gfp_mask = __GFP_NOWARN;long filled =0 ; +bool nowait = flags &XBF_NOWAIT ;if (flags &XBF_READ_AHEAD ) gfp_mask |= __GFP_NORETRY;
@@ -377,7 +378,7 @@ xfs_buf_alloc_pages( bp->b_pages =kzalloc (sizeof (struct page *) * bp->b_page_count, gfp_mask);if (!bp->b_pages) -return -ENOMEM ; +return nowait ? -EAGAIN : -ENOMEM ; } bp->b_flags |= _XBF_PAGES;
@@ -451,7 +452,7 @@ _xfs_buf_map_pages(memalloc_nofs_restore (nofs_flag);if (!bp->b_addr) -return -ENOMEM ; +return flags &XBF_NOWAIT ? -EAGAIN : -ENOMEM ; }return 0 ;
Signature · 1 line
2.25.1
PATCH v5 21/29 xfs: return -EAGAIN when bulk memory allocation fails in nowait case
In reply to Hao XuFrom: Hao Xu <[email protected]>
Rather than wait for a moment and retry, we return -EAGAIN when we fail to allocate bulk memory in xfs_buf_alloc_pages() in nowait case.
--- fs/xfs/xfs_buf.c | 5 +++++ 1 file changed, 5 insertions(+)
fs/xfs/xfs_buf.c +5 -0
@@ -404,6 +404,11 @@ xfs_buf_alloc_pages(if (filled != last)continue ; +if (nowait) { +xfs_buf_free_pages (bp); +return -EAGAIN ; + } +if (flags &XBF_READ_AHEAD ) {xfs_buf_free_pages (bp);return -ENOMEM ;
Signature · 1 line
2.25.1
PATCH v5 22/29 xfs: comment page allocation for nowait case in xfs_buf_find_insert()
In reply to Hao XuFrom: Hao Xu <[email protected]>
Add comments for page allocation in nowait case in xfs_buf_find_insert()
--- fs/xfs/xfs_buf.c | 2 ++ 1 file changed, 2 insertions(+)
fs/xfs/xfs_buf.c +2 -0
@@ -633,6 +633,8 @@ xfs_buf_find_insert(* allocate the memory from the heap to minimise memory usage. If we * can't get heap memory for these small buffers, we fall back to using * the page allocator. +* xfs_buf_alloc_kmem may return -EAGAIN, let's not return it but turn +* to page allocator as well. */ if (BBTOB (new_bp->b_length) >=PAGE_SIZE ||xfs_buf_alloc_kmem (new_bp, flags) <0 ) {
Signature · 1 line
2.25.1
Matthew Wilcox
maintainerRe: [PATCH 22/29] xfs: comment page allocation for nowait case in xfs_buf_find_insert()
fs/xfs/xfs_buf.c xfs_buf_find_insert( · line 637
Quoted 6 lines · Hao Xu
fs/xfs/xfs_buf.c +2
-0@@ -633,6 +633,8 @@ xfs_buf_find_insert(* allocate the memory from the heap to minimise memory usage. If we * can't get heap memory for these small buffers, we fall back to using * the page allocator. +* xfs_buf_alloc_kmem may return -EAGAIN, let's not return it but turn +* to page allocator as well.
This new sentence seems like it says exactly the same thing as the previous sentence. What am I missing?
PATCH v5 23/29 xfs: don't print warn info for -EAGAIN error in xfs_buf_get_map()
In reply to Hao XuFrom: Hao Xu <[email protected]>
-EAGAIN is internal error to indicate a retry, no needs to print a warn.
--- fs/xfs/xfs_buf.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)
fs/xfs/xfs_buf.c +4 -3
@@ -730,9 +730,10 @@ xfs_buf_get_map(if (!bp->b_addr) { error =_xfs_buf_map_pages (bp, flags);if (unlikely (error)) { -xfs_warn_ratelimited (btp->bt_mount, -"%s: failed to map %u pages" , __func__, - bp->b_page_count); +if (error != -EAGAIN ) +xfs_warn_ratelimited (btp->bt_mount, +"%s: failed to map %u pages" , __func__, + bp->b_page_count);xfs_buf_relse (bp);return error; }
Signature · 1 line
2.25.1
PATCH v5 24/29 xfs: support nowait for xfs_buf_read_map()
In reply to Hao XuFrom: Hao Xu <[email protected]>
This causes xfstests generic/232 hung in umount process, waiting for ail push, so I comment it for now, need some hints from xfs folks. Not a real patch.
--- fs/xfs/xfs_buf.c | 7 +++++++ 1 file changed, 7 insertions(+)
fs/xfs/xfs_buf.c +7 -0
@@ -828,6 +828,13 @@ xfs_buf_read_map(trace_xfs_buf_read (bp, flags, _RET_IP_);if (!(bp->b_flags &XBF_DONE )) { +// /* +// * Let's bypass the _xfs_buf_read() for now +// */ +// if (flags & XBF_NOWAIT) { +// xfs_buf_relse(bp); +// return -EAGAIN; +// } /* Initiate the buffer read and wait. */ XFS_STATS_INC (target->bt_mount, xb_get_read); bp->b_ops = ops;
Signature · 1 line
2.25.1
Re: [PATCH 24/29] xfs: support nowait for xfs_buf_read_map()
Quoted 16 lines · Hao Xu
From: Hao Xu <[email protected]>
This causes xfstests generic/232 hung in umount process, waiting for ail push, so I comment it for now, need some hints from xfs folks. Not a real patch.
Signed-off-by Hao Xu <[email protected]>--- fs/xfs/xfs_buf.c | 7 +++++++ 1 file changed, 7 insertions(+)fs/xfs/xfs_buf.c +0
-0
fs/xfs/xfs_buf.c xfs_buf_read_map( · line 837
Quoted 10 lines
fs/xfs/xfs_buf.c +7
-0@@ -828,6 +828,13 @@ xfs_buf_read_map(trace_xfs_buf_read (bp, flags, _RET_IP_);if (!(bp->b_flags &XBF_DONE )) { +// /* +// * Let's bypass the _xfs_buf_read() for now +// */ +// if (flags & XBF_NOWAIT) { +// xfs_buf_relse(bp); +// return -EAGAIN; +// }
This is *fundamentally broken*, and apart from anything else breaks readahead.
IF we asked for a read, we cannot instantiate the buffer and then *not issue any IO on it* and release it. That leaves an uninitialised buffer in memory, and there's every chance that something then trips over it and bad things happen.
A buffer like this *must* be errored out and marked stale so that the next access to it will then re-initialise the buffer state and trigger any preparatory work that needs to be done for the new operation.
This comes back to my first comments that XBF_TRYLOCK cannot simpy be replaced with XBF_NOWAIT semantics...
-Dave.
Signature · 2 lines
Dave Chinner [email protected]
PATCH v5 25/29 xfs: support nowait for xfs_buf_item_init()
In reply to Hao XuFrom: Hao Xu <[email protected]>
support nowait for xfs_buf_item_init() and error out -EAGAIN to _xfs_trans_bjoin() when it would block.
--- fs/xfs/xfs_buf_item.c | 9 +++++++-- fs/xfs/xfs_buf_item.h | 2 +- fs/xfs/xfs_buf_item_recover.c | 2 +- fs/xfs/xfs_trans_buf.c | 16 +++++++++++++--- 4 files changed, 22 insertions(+), 7 deletions(-)
fs/xfs/xfs_buf_item.c +7 -2
@@ -827,7 +827,8 @@ xfs_buf_item_free_format(int xfs_buf_item_init (struct xfs_buf *bp, -struct xfs_mount *mp) +struct xfs_mount *mp, +bool nowait) {struct xfs_buf_log_item *bip = bp->b_log_item;int chunks;
@@ -847,7 +848,11 @@ xfs_buf_item_init(return 0 ; } - bip =kmem_cache_zalloc (xfs_buf_item_cache,GFP_KERNEL | __GFP_NOFAIL); + bip =kmem_cache_zalloc (xfs_buf_item_cache, +GFP_KERNEL | (nowait ?0 : __GFP_NOFAIL)); +if (!bip) +return -EAGAIN ; +xfs_log_item_init (mp, &bip->bli_item,XFS_LI_BUF , &xfs_buf_item_ops); bip->bli_buf = bp;
fs/xfs/xfs_buf_item.h +1 -1
@@ -47,7 +47,7 @@ struct xfs_buf_log_item {struct xfs_buf_log_format __bli_format;/* embedded in-log header */ }; -int xfs_buf_item_init (struct xfs_buf *,struct xfs_mount *); +int xfs_buf_item_init (struct xfs_buf *bp,struct xfs_mount *mp,bool nowait);void xfs_buf_item_done (struct xfs_buf *bp);void xfs_buf_item_relse (struct xfs_buf *);bool xfs_buf_item_put (struct xfs_buf_log_item *);
fs/xfs/xfs_buf_item_recover.c +1 -1
@@ -429,7 +429,7 @@ xlog_recover_validate_buf_type(struct xfs_buf_log_item *bip; bp->b_flags |= _XBF_LOGRECOVERY; -xfs_buf_item_init (bp, mp); +xfs_buf_item_init (bp, mp, false); bip = bp->b_log_item; bip->bli_item.li_lsn = current_lsn; }
fs/xfs/xfs_trans_buf.c +13 -3
@@ -57,13 +57,14 @@ xfs_trans_buf_item_match(* If the buffer does not yet have a buf log item associated with it, * then allocate one for it. Then add the buf item to the transaction. */ -STATIC void +STATIC int_xfs_trans_bjoin (struct xfs_trans *tp,struct xfs_buf *bp,int reset_recur) {struct xfs_buf_log_item *bip; +int ret;ASSERT (bp->b_transp ==NULL );
@@ -72,7 +73,11 @@ _xfs_trans_bjoin(* it doesn't have one yet, then allocate one and initialize it. * The checks to see if one is there are in xfs_buf_item_init(). */ -xfs_buf_item_init (bp, tp->t_mountp); + ret =xfs_buf_item_init (bp, tp->t_mountp, + tp->t_flags &XFS_TRANS_NOWAIT ); +if (ret <0 ) +return ret; + bip = bp->b_log_item;ASSERT (!(bip->bli_flags &XFS_BLI_STALE ));ASSERT (!(bip->__bli_format.blf_flags &XFS_BLF_CANCEL ));
@@ -92,6 +97,7 @@ _xfs_trans_bjoin(xfs_trans_add_item (tp, &bip->bli_item); bp->b_transp = tp; +return 0 ; }void
@@ -309,7 +315,11 @@ xfs_trans_read_buf_map( }if (tp) { -_xfs_trans_bjoin (tp, bp,1 ); + error =_xfs_trans_bjoin (tp, bp,1 ); +if (error) { +xfs_buf_relse (bp); +return error; + }trace_xfs_trans_read_buf (bp->b_log_item); }ASSERT (bp->b_ops !=NULL || ops ==NULL );
Signature · 1 line
2.25.1
Re: [PATCH 25/29] xfs: support nowait for xfs_buf_item_init()
Quoted 29 lines · Hao Xu
From: Hao Xu <[email protected]>
support nowait for
xfs_buf_item_init()and error out-EAGAINto_xfs_trans_bjoin()when it would block.Signed-off-by Hao Xu <[email protected]>--- fs/xfs/xfs_buf_item.c | 9 +++++++-- fs/xfs/xfs_buf_item.h | 2 +- fs/xfs/xfs_buf_item_recover.c | 2 +- fs/xfs/xfs_trans_buf.c | 16 +++++++++++++--- 4 files changed, 22 insertions(+), 7 deletions(-)fs/xfs/xfs_buf_item.c +2
-1@@ -827,7 +827,8 @@ xfs_buf_item_free_format(int xfs_buf_item_init (struct xfs_buf *bp, -struct xfs_mount *mp) +struct xfs_mount *mp, +bool nowait) {struct xfs_buf_log_item *bip = bp->b_log_item;int chunks;@@ -847,7 +848,11 @@ xfs_buf_item_init(return 0 ;
fs/xfs/xfs_buf_item.c xfs_buf_item_init( · line 857
Quoted 10 lines
fs/xfs/xfs_buf_item.c +5
-1} - bip =kmem_cache_zalloc (xfs_buf_item_cache,GFP_KERNEL | __GFP_NOFAIL); + bip =kmem_cache_zalloc (xfs_buf_item_cache, +GFP_KERNEL | (nowait ?0 : __GFP_NOFAIL)); +if (!bip) +return -EAGAIN ; +xfs_log_item_init (mp, &bip->bli_item,XFS_LI_BUF , &xfs_buf_item_ops); bip->bli_buf = bp;
I see filesystem shutdowns....
Quoted 42 lines
fs/xfs/xfs_trans_buf.c +8
-2@@ -57,13 +57,14 @@ xfs_trans_buf_item_match(* If the buffer does not yet have a buf log item associated with it, * then allocate one for it. Then add the buf item to the transaction. */ -STATIC void +STATIC int_xfs_trans_bjoin (struct xfs_trans *tp,struct xfs_buf *bp,int reset_recur) {struct xfs_buf_log_item *bip; +int ret;ASSERT (bp->b_transp ==NULL );@@ -72,7 +73,11 @@ _xfs_trans_bjoin(* it doesn't have one yet, then allocate one and initialize it. * The checks to see if one is there are in xfs_buf_item_init(). */ -xfs_buf_item_init (bp, tp->t_mountp); + ret =xfs_buf_item_init (bp, tp->t_mountp, + tp->t_flags &XFS_TRANS_NOWAIT ); +if (ret <0 ) +return ret; + bip = bp->b_log_item;ASSERT (!(bip->bli_flags &XFS_BLI_STALE ));ASSERT (!(bip->__bli_format.blf_flags &XFS_BLF_CANCEL ));@@ -92,6 +97,7 @@ _xfs_trans_bjoin(xfs_trans_add_item (tp, &bip->bli_item); bp->b_transp = tp; +return 0 ; }void
fs/xfs/xfs_trans_buf.c xfs_trans_read_buf_map( · line 323
Quoted 10 lines
fs/xfs/xfs_trans_buf.c +5
-1@@ -309,7 +315,11 @@ xfs_trans_read_buf_map( }if (tp) { -_xfs_trans_bjoin (tp, bp,1 ); + error =_xfs_trans_bjoin (tp, bp,1 ); +if (error) { +xfs_buf_relse (bp); +return error; + }trace_xfs_trans_read_buf (bp->b_log_item);
So what happens at the callers when we have a dirty transaction and joining a buffer fails with -EAGAIN?
Apart from the fact this may well propagate -EAGAIN up to userspace, cancelling a dirty transaction at this point will result in a filesystem shutdown....
Indeed, this can happen in the "simple" timestamp update case that this "nowait" semantic is being aimed at. We log the inode in the timestamp update, which dirties the log item and registers a precommit operation to be run. We commit the transaction, which then runs xfs_inode_item_precommit() and that may need to attach the inode to the inode cluster buffer. This results in:
xfs_inode_item_precommit
xfs_imap_to_bp
xfs_trans_read_buf_map
_xfs_trans_bjoin
xfs_buf_item_init(XFS_TRANS_NOWAIT)
kmem_cache_zalloc(GFP_NOFS)
<memory allocation fails>
gets -EAGAIN error
propagates -EAGAIN
fails due to -EAGAIN
And now xfs_trans_commit() fails with a dirty transaction and the filesystem shuts down.
IOWs, XFS_TRANS_NOWAIT as it stands is fundamentally broken. Once we dirty an item in a transaction, we *cannot* back out of the transaction. We *must block* in every place that could fail - locking, memory allocation and/or IO - until the transaction completes because we cannot undo the changes we've already made to the dirty items in the transaction....
It's even worse than that - once we have committed intents, the whole chain of intent processing must be run to completionr. Hence we can't tolerate backing out of that defered processing chain half way through because we might have to block.
Until we can roll back partial dirty transactions and partially completed defered intent chains at any random point of completion, XFS_TRANS_NOWAIT will not work.
-Dave.
Signature · 2 lines
Dave Chinner [email protected]
PATCH v5 26/29 xfs: return -EAGAIN when nowait meets sync in transaction commit
In reply to Hao XuFrom: Hao Xu <[email protected]>
if the log transaction is a sync one, let's fail the nowait try and return -EAGAIN directly since sync transaction means blocked by IO.
--- fs/xfs/xfs_trans.c | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-)
fs/xfs/xfs_trans.c +13 -1
@@ -968,12 +968,24 @@ __xfs_trans_commit(xfs_csn_t commit_seq =0 ;int error =0 ;int sync = tp->t_flags &XFS_TRANS_SYNC ; +bool nowait = tp->t_flags &XFS_TRANS_NOWAIT ; +bool perm_log = tp->t_flags &XFS_TRANS_PERM_LOG_RES ;trace_xfs_trans_commit (tp, _RET_IP_); +if (nowait && sync) { +/* +* Currently nowait is only from xfs_vn_update_time() +* so perm_log is always false here, but let's make +* code general. +*/ +if (perm_log) +xfs_defer_cancel (tp); + goto out_unreserve; + } error =xfs_trans_run_precommits (tp);if (error) { -if (tp->t_flags &XFS_TRANS_PERM_LOG_RES ) +if (perm_log)xfs_defer_cancel (tp); goto out_unreserve; }
Signature · 1 line
2.25.1
Re: [PATCH 26/29] xfs: return -EAGAIN when nowait meets sync in transaction commit
Quoted 23 lines · Hao Xu
From: Hao Xu <[email protected]>
if the log transaction is a sync one, let's fail the nowait try and return
-EAGAINdirectly since sync transaction means blocked by IO.Signed-off-by Hao Xu <[email protected]>--- fs/xfs/xfs_trans.c | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-)fs/xfs/xfs_trans.c +2
-0@@ -968,12 +968,24 @@ __xfs_trans_commit(xfs_csn_t commit_seq =0 ;int error =0 ;int sync = tp->t_flags &XFS_TRANS_SYNC ; +bool nowait = tp->t_flags &XFS_TRANS_NOWAIT ; +bool perm_log = tp->t_flags &XFS_TRANS_PERM_LOG_RES ;trace_xfs_trans_commit (tp, _RET_IP_);
fs/xfs/xfs_trans.c __xfs_trans_commit( · line 985
Quoted 10 lines
fs/xfs/xfs_trans.c +10
-0+if (nowait && sync) { +/* +* Currently nowait is only from xfs_vn_update_time() +* so perm_log is always false here, but let's make +* code general. +*/ +if (perm_log) +xfs_defer_cancel (tp); + goto out_unreserve; + }
This is fundamentally broken. We cannot about a transaction commit with dirty items at this point with shutting down the filesystem.
This points to XFS_TRANS_NOWAIT being completely broken, too, because we don't call xfs_trans_set_sync() until just before we commit the transaction. At this point, it is -too late- for nowait+sync to be handled gracefully, and it will *always* go bad.
IOWs, the whole transaction "nowait" semantics as designed and implemented is not a workable solution....
-Dave.
Signature · 2 lines
Dave Chinner [email protected]
PATCH v5 27/29 xfs: add a comment for xlog_kvmalloc()
In reply to Hao XuFrom: Hao Xu <[email protected]>
vmalloc() always succeed in 64 bit system?
Not a real patch.
--- fs/xfs/xfs_log_cil.c | 3 +++ 1 file changed, 3 insertions(+)
fs/xfs/xfs_log_cil.c +3 -0
@@ -335,6 +335,9 @@ xlog_cil_alloc_shadow_bufs(* storage. */ kmem_free (lip->li_lv_shadow); +/* +* May this be indefinite loop in nowait case? +*/ lv =xlog_kvmalloc (buf_size);memset (lv,0 ,xlog_cil_iovec_space (niovecs));
Signature · 1 line
2.25.1
PATCH v5 28/29 xfs: support nowait semantics for xc_ctx_lock in xlog_cil_commit()
In reply to Hao XuFrom: Hao Xu <[email protected]>
Apply trylock logic for xc_ctx_lock in xlog_cil_commit() in nowait case and error out -EAGAIN for xlog_cil_commit().
--- fs/xfs/xfs_log_cil.c | 12 ++++++++++-- fs/xfs/xfs_log_priv.h | 2 +- fs/xfs/xfs_trans.c | 4 +++- 3 files changed, 14 insertions(+), 4 deletions(-)
fs/xfs/xfs_log_cil.c +10 -2
@@ -1613,7 +1613,7 @@ xlog_cil_process_intents(* background commit, returns without it held once background commits are * allowed again. */ -void +int xlog_cil_commit( struct xlog *log, struct xfs_trans *tp,
@@ -1623,6 +1623,7 @@ xlog_cil_commit(struct xfs_cil *cil = log->l_cilp;struct xfs_log_item *lip, *next;uint32_t released_space =0 ; +bool nowait = tp->t_flags &XFS_TRANS_NOWAIT ;/* * Do all necessary memory allocation before we lock the CIL.
@@ -1632,7 +1633,12 @@ xlog_cil_commit(xlog_cil_alloc_shadow_bufs (log, tp);/* lock out background commit */ -down_read (&cil->xc_ctx_lock); +if (nowait) { +if (!down_read_trylock (&cil->xc_ctx_lock)) +return -EAGAIN ; + }else { +down_read (&cil->xc_ctx_lock); + }if (tp->t_flags &XFS_TRANS_HAS_INTENT_DONE ) released_space =xlog_cil_process_intents (cil, tp);
@@ -1668,6 +1674,8 @@ xlog_cil_commit(/* xlog_cil_push_background() releases cil->xc_ctx_lock */ xlog_cil_push_background (log); + +return 0 ; }/*
fs/xfs/xfs_log_priv.h +1 -1
@@ -580,7 +580,7 @@ int xlog_cil_init(struct xlog *log);void xlog_cil_init_post_recovery (struct xlog *log);void xlog_cil_destroy (struct xlog *log);bool xlog_cil_empty (struct xlog *log); -void xlog_cil_commit (struct xlog *log,struct xfs_trans *tp, +int xlog_cil_commit (struct xlog *log,struct xfs_trans *tp,xfs_csn_t *commit_seq,bool regrant);void xlog_cil_set_ctx_write_state (struct xfs_cil_ctx *ctx,struct xlog_in_core *iclog);
fs/xfs/xfs_trans.c +3 -1
@@ -1037,7 +1037,9 @@ __xfs_trans_commit(xfs_trans_apply_sb_deltas (tp);xfs_trans_apply_dquot_deltas (tp); -xlog_cil_commit (log, tp, &commit_seq, regrant); + error =xlog_cil_commit (log, tp, &commit_seq, regrant); +if (error) + goto out_unreserve;xfs_trans_free (tp);
Signature · 1 line
2.25.1
Re: [PATCH 28/29] xfs: support nowait semantics for xc_ctx_lock in xlog_cil_commit()
Quoted 4 lines · Hao Xu
From: Hao Xu <[email protected]>
Apply trylock logic for
xc_ctx_lockinxlog_cil_commit()in nowait case and error out-EAGAINforxlog_cil_commit().
Again, fundamentally broken. Any error from xlog_cil_commit() will result in a filesystem shutdown as we cannot back out from failure with dirty log items gracefully at this point.
-Dave.
Signature · 2 lines
Dave Chinner [email protected]
PATCH v5 29/29 io_uring: add support for getdents
In reply to Hao XuFrom: Hao Xu <[email protected]>
This add support for getdents64 to io_uring, acting exactly like the
syscall: the directory is iterated from it's current's position as stored in the file struct, and the file's position is updated exactly as if getdents64 had been called.
For filesystems that support NOWAIT in iterate_shared(), try to use it first; if a user already knows the filesystem they use do not support nowait they can force async through IOSQE_ASYNC in the sqe flags, avoiding the need to bounce back through a useless EAGAIN return.
--- include/uapi/linux/io_uring.h | 1 + io_uring/fs.c | 53 +++++++++++++++++++++++++++++++++++ io_uring/fs.h | 3 ++ io_uring/opdef.c | 8 ++++++ 4 files changed, 65 insertions(+)
include/uapi/linux/io_uring.h +1 -0
@@ -240,6 +240,7 @@ enum io_uring_op {IORING_OP_URING_CMD ,IORING_OP_SEND_ZC ,IORING_OP_SENDMSG_ZC , +IORING_OP_GETDENTS ,/* this goes last, obviously */ IORING_OP_LAST,
io_uring/fs.c +53 -0
62 lines not on this page · raw message
io_uring/fs.h +3 -0
@@ -18,3 +18,6 @@ int io_symlinkat(struct io_kiocb *req, unsigned int issue_flags);int io_linkat_prep (struct io_kiocb *req,const struct io_uring_sqe *sqe);int io_linkat (struct io_kiocb *req,unsigned int issue_flags);void io_link_cleanup (struct io_kiocb *req); + +int io_getdents_prep (struct io_kiocb *req,const struct io_uring_sqe *sqe); +int io_getdents (struct io_kiocb *req,unsigned int issue_flags);
io_uring/opdef.c +8 -0
@@ -428,6 +428,11 @@ const struct io_issue_def io_issue_defs[] = { .prep = io_eopnotsupp_prep, #endif }, + [IORING_OP_GETDENTS ] = { + .needs_file =1 , + .prep = io_getdents_prep, + .issue = io_getdents, + }, };
@@ -648,6 +653,9 @@ const struct io_cold_def io_cold_defs[] = { .fail = io_sendrecv_fail,#endif }, + [IORING_OP_GETDENTS] = { + .name = "GETDENTS", + }, };const char *io_uring_get_opcode (u8 opcode)
Signature · 1 line
2.25.1
Quoted 5 lines · Hao Xu
From: Hao Xu <[email protected]>
This series introduce getdents64 to
io_uring, the code logic is similar with the snychronized version's. It first try nowait issue, and offload it to io-wq threads if the first try fails.
NAK on the entire series until Jens actually writes down what NOWAIT does, so that we can check that the *existing* nowait code branches actually behave how he says it should.
https://lore.kernel.org/all/[email protected]/
--D
Quoted 198 lines
Patch1 and Patch2 are some preparation
Patch3 supports nowait for xfs getdents code
Patch4-11 are vfs change, include adding helpers and trylock for locks Patch12-29 supports nowait for involved xfs journal stuff note, Patch24 and 27 are actually two questions, might be removed later. an xfs test may come later.Tests I've done:
a liburing test case for functional test:
https://github.com/HowHsu/liburing/commit/39dc9a8e19c06a8cebf8c2301b85320eb45c061e?diff=unifiedxfstests:
test/generic: 1 fails and 171 not run
test/xfs: 72 fails and 156 not run
run the code before without this patchset, same result. I'll try to make the environment more right to run more tests here.Tested it with a liburing performance test:
https://github.com/HowHsu/liburing/blob/getdents/test/getdents2.cThe test is controlled by the below script[2] which runs getdents2.t 100 times and calulate the avg.
The result show thatio_uringversion is about 2.6% faster:note:
[1] the number of getdents call/request inio_uringand normal sync version are made sure to be same beforehand.[2]
run_getdents.py```python3 import subprocess N = 100 sum = 0.0 args = ["/data/home/howeyxu/tmpdir", "sync"] for i in range(N): output = subprocess.check_output(["./liburing/test/getdents2.t"] + args) sum += float(output) average = sum / N print("Average of sync:", average) sum = 0.0 args = ["/data/home/howeyxu/tmpdir", "iouring"] for i in range(N): output = subprocess.check_output(["./liburing/test/getdents2.t"] + args) sum += float(output) average = sum / N print("Average of iouring:", average) ```
v4->v5:
- move atime update to the beginning of getdents operation
- trylock fori_rwsem
- nowait semantics for involved xfs journal stuff
v3->v4:
- add Dave's xfs nowait code and fix a deadlock problem, with some code style tweak.
- disable fixed file to avoid a race problem for now
- add a test program.
v2->v3:
- removed the kernfs patches
- addf_pos_locklogic
- remove the "reduce last EOF getdents try" optimization since Dominique reports that doesn't make difference
- remove the rewind logic, I think the right way is to introduce lseek toio_uringnot to patch this logic to getdents.
- add Singed-off-by of Stefan Roesch for patch 1 since checkpatch complained that Co-developed-by someone should be accompanied with Signed-off-by same person, I can remove them if Stefan thinks that's not proper.Dominique Martinet (1):
fs: split offvfs_getdentsfunction of getdents64 syscallHao Xu (28):
xfs: renameXBF_TRYLOCKtoXBF_NOWAIT
xfs: add NOWAIT semantics for readdir
vfs: add nowait flag for structdir_context
vfs: add a vfs helper forio_uringfile pos lock
vfs: addfile_pos_unlock()forio_uringusage
vfs: add a nowait parameter fortouch_atime()
vfs: add nowait parameter forfile_accessed()
vfs: movefile_accessed()to the beginning ofiterate_dir()
vfs: addS_NOWAITfor nowait time update
vfs: trylockinode->i_rwseminiterate_dir()to support nowait
xfs: enforceGFP_NOIOimplicitly during nowait time update
xfs: makexfs_trans_alloc()support nowait semantics
xfs: support nowait forxfs_log_reserve()
xfs: don't wait for free space inxlog_grant_head_check()in nowait
case
xfs: add nowait parameter forxfs_inode_item_init()
xfs: makexfs_trans_ijoin()error out-EAGAIN
xfs: setXBF_NOWAITforxfs_buf_read_mapif necessary
xfs: support nowait memory allocation in_xfs_buf_alloc()
xfs: distinguish error type of memory allocation failure for nowait
case
xfs: return-EAGAINwhen bulk memory allocation fails in nowait case
xfs: comment page allocation for nowait case inxfs_buf_find_insert()
xfs: don't print warn info for-EAGAINerror inxfs_buf_get_map()
xfs: support nowait forxfs_buf_read_map()
xfs: support nowait forxfs_buf_item_init()
xfs: return-EAGAINwhen nowait meets sync in transaction commit
xfs: add a comment forxlog_kvmalloc()
xfs: support nowait semantics forxc_ctx_lockinxlog_cil_commit()
io_uring: add support for getdentsarch/s390/hypfs/inode.c | 2 +- block/fops.c | 2 +- fs/btrfs/file.c | 2 +- fs/btrfs/inode.c | 2 +- fs/cachefiles/namei.c | 2 +- fs/coda/dir.c | 4 +-- fs/ecryptfs/file.c | 4 +-- fs/ext2/file.c | 4 +-- fs/ext4/file.c | 6 ++-- fs/f2fs/file.c | 4 +-- fs/file.c | 13 +++++++ fs/fuse/dax.c | 2 +- fs/fuse/file.c | 4 +-- fs/gfs2/file.c | 2 +- fs/hugetlbfs/inode.c | 2 +- fs/inode.c | 10 +++--- fs/internal.h | 8 +++++ fs/namei.c | 4 +-- fs/nfsd/vfs.c | 2 +- fs/nilfs2/file.c | 2 +- fs/orangefs/file.c | 2 +- fs/orangefs/inode.c | 2 +- fs/overlayfs/file.c | 2 +- fs/overlayfs/inode.c | 2 +- fs/pipe.c | 2 +- fs/ramfs/file-nommu.c | 2 +- fs/readdir.c | 61 +++++++++++++++++++++++++-------- fs/smb/client/cifsfs.c | 2 +- fs/splice.c | 2 +- fs/stat.c | 2 +- fs/ubifs/file.c | 2 +- fs/udf/file.c | 2 +- fs/xfs/libxfs/xfs_alloc.c | 2 +- fs/xfs/libxfs/xfs_attr_remote.c | 2 +- fs/xfs/libxfs/xfs_btree.c | 2 +- fs/xfs/libxfs/xfs_da_btree.c | 16 +++++++++ fs/xfs/libxfs/xfs_da_btree.h | 1 + fs/xfs/libxfs/xfs_dir2_block.c | 7 ++-- fs/xfs/libxfs/xfs_dir2_priv.h | 2 +- fs/xfs/libxfs/xfs_shared.h | 2 ++ fs/xfs/libxfs/xfs_trans_inode.c | 12 +++++-- fs/xfs/scrub/dir.c | 2 +- fs/xfs/scrub/readdir.c | 2 +- fs/xfs/scrub/repair.c | 2 +- fs/xfs/xfs_buf.c | 43 +++++++++++++++++------ fs/xfs/xfs_buf.h | 4 +-- fs/xfs/xfs_buf_item.c | 9 +++-- fs/xfs/xfs_buf_item.h | 2 +- fs/xfs/xfs_buf_item_recover.c | 2 +- fs/xfs/xfs_dir2_readdir.c | 49 ++++++++++++++++++++------ fs/xfs/xfs_dquot.c | 2 +- fs/xfs/xfs_file.c | 6 ++-- fs/xfs/xfs_inode.c | 27 +++++++++++++++ fs/xfs/xfs_inode.h | 17 +++++---- fs/xfs/xfs_inode_item.c | 12 ++++--- fs/xfs/xfs_inode_item.h | 3 +- fs/xfs/xfs_iops.c | 31 ++++++++++++++--- fs/xfs/xfs_log.c | 33 ++++++++++++------ fs/xfs/xfs_log.h | 5 +-- fs/xfs/xfs_log_cil.c | 17 +++++++-- fs/xfs/xfs_log_priv.h | 4 +-- fs/xfs/xfs_trans.c | 44 ++++++++++++++++++++---- fs/xfs/xfs_trans.h | 2 +- fs/xfs/xfs_trans_buf.c | 18 ++++++++-- fs/zonefs/file.c | 4 +-- include/linux/file.h | 7 ++++ include/linux/fs.h | 16 +++++++-- include/uapi/linux/io_uring.h | 1 + io_uring/fs.c | 53 ++++++++++++++++++++++++++++ io_uring/fs.h | 3 ++ io_uring/opdef.c | 8 +++++ kernel/bpf/inode.c | 4 +-- mm/filemap.c | 8 ++--- mm/shmem.c | 6 ++-- net/unix/af_unix.c | 4 +-- 75 files changed, 499 insertions(+), 161 deletions(-)Signature · 1 line
2.25.1
Quoted 6 lines · Hao Xu
From: Hao Xu <[email protected]>
This series introduce getdents64 to
io_uring, the code logic is similar with the snychronized version's. It first try nowait issue, and offload it to io-wq threads if the first try fails.
Quoted 6 lines
Patch1 and Patch2 are some preparation
Patch3 supports nowait for xfs getdents code
Patch4-11 are vfs change, include adding helpers and trylock for locks Patch12-29 supports nowait for involved xfs journal stuff note, Patch24 and 27 are actually two questions, might be removed later. an xfs test may come later.
You need to drop all the XFS journal stuff. It's fundamentally broken as it stands, and we cannot support non-blocking transactional changes without first putting a massive investment in transaction and intent chain rollback to allow correctly undoing partially complete modifications.
Regardless, non-blocking transactions are completely unnecessary for a non-blocking readdir implementation. readdir should only be touching atime, and with relatime it should only occur once every 24 hours per inode. If that's a problem, then we have noatime mount options. Hence I just don't see any point in worrying about having a timestamp update block occasionally...
I also don't really don't see why you need to fiddle with xfs buffer cache semantics - it already has the functionality "nowait" buffer reads require (i.e. XBF_INCORE|XBF_TRYLOCK).
However, the readahead IO that the xfs readdir code issues cannot use your defined NOWAIT semantics - it must be able to allocate memory and issue IO. Readahead already avoids blocking on memory allocation and blocking on IO via the XBF_READ_AHEAD flag. This sets __GFP_NORETRY for buffer allocation and REQ_RAHEAD for IO. Hence readahead only needs the existing XBF_TRYLOCK flag to be set to be compatible with the required NOWAIT semantics....
As for the NOIO memory allocation restrictions io_uring requires, that should be enforced at the io_uring layer before calling into the VFS using memalloc_noio_save/restore. At that point no memory allocation will trigger IO and none of the code running under NOWAIT conditions even needs to be aware that io_uring has a GFP_NOIO restriction on memory allocation....
Please go back to the simple "do non-blocking buffer IO" implementation we started with and don't try to solve every little blocking problem that might exist in the VFS and filesystems...
-Dave
Signature · 2 lines
Dave Chinner [email protected]