Skip to content

RFC Orangefs Through the Pagecache

[email protected] linux-fsdevel 1 version New

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 RFC Latest22 patches 0 repliesNew

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/22orangefs: implement xattr cachefs/orangefs/inode.c fs/orangefs/orangefs-kernel.h fs/orangefs/super.c +1 more+127 -10 replies
2/22orangefs: do not invalidate attributes on inode createfs/orangefs/inode.c fs/orangefs/namei.c+0 -120 replies
3/22orangefs: simplify orangefs_inode_getattr interfacefs/orangefs/file.c fs/orangefs/inode.c fs/orangefs/orangefs-kernel.h +1 more+28 -380 replies
4/22orangefs: update attributes rather than relying on serverfs/orangefs/file.c fs/orangefs/namei.c+8 -90 replies
5/22orangefs: hold i_lock during inode_getattrfs/orangefs/inode.c fs/orangefs/orangefs-utils.c+25 -120 replies
6/22orangefs: set up and use backing_dev_infofs/orangefs/super.c+14 -70 replies
7/22orangefs: let setattr write to cached inodefs/orangefs/inode.c fs/orangefs/super.c+20 -80 replies
8/22orangefs: reorganize setattr functions to track attribute changesfs/orangefs/acl.c fs/orangefs/inode.c fs/orangefs/namei.c +3 more+129 -1190 replies
9/22orangefs: remove orangefs_readpagesfs/orangefs/inode.c+1 -380 replies
10/22orangefs: service ops done for writeback are not killablefs/orangefs/orangefs-kernel.h fs/orangefs/orangefs-utils.c fs/orangefs/waitqueue.c+12 -90 replies
11/22orangefs: migrate to generic_file_read_iterfs/orangefs/file.c fs/orangefs/inode.c fs/orangefs/orangefs-kernel.h+38 -1040 replies
12/22orangefs: implement writepagefs/orangefs/file.c fs/orangefs/inode.c fs/orangefs/orangefs-utils.c+80 -680 replies
13/22orangefs: do not return successful read when the client-core disappearedfs/orangefs/file.c+4 -10 replies
14/22orangefs: move do_readv_writev to direct_IOfs/orangefs/file.c fs/orangefs/inode.c+111 -1110 replies
15/22orangefs: skip inode writeout if nothing to writefs/orangefs/orangefs-utils.c+5 -00 replies
16/22orangefs: avoid fsync service operation on flushfs/orangefs/file.c+23 -10 replies
17/22orangefs: write range trackingfs/orangefs/file.c fs/orangefs/inode.c fs/orangefs/orangefs-kernel.h+274 -350 replies
18/22orangefs: implement writepagesfs/orangefs/file.c+1 -00 replies
19/22orangefs: add orangefs_revalidate_mappingfs/orangefs/file.c fs/orangefs/inode.c fs/orangefs/orangefs-kernel.h +2 more+328 -190 replies
20/22orangefs: remember count when reading.fs/orangefs/file.c fs/orangefs/orangefs-kernel.h+29 -10 replies
21/22orangefs: pass slot index back to readpage.fs/orangefs/file.c fs/orangefs/inode.c fs/orangefs/orangefs-kernel.h+6 -60 replies
22/22orangefs: copy Orangefs-sized blocks into the pagecache if possible.fs/orangefs/file.c fs/orangefs/inode.c fs/orangefs/orangefs-bufmap.c +2 more+156 -150 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.

Read the v1 thread

Cover letter [RFC PATCH 00/22] Orangefs Through the Pagecache

From: Mike Marshall <[email protected]>

Until now the Orangefs kernel module has not gone through the pagecache and has been a de facto O_DIRECT style module.

This patch series adds pagecache support and greatly improves small IO. For example, xfstest generic/069 used to take several hours on our test VMs, and now finishes in seconds.

Orangefs' kernel module shares a memory buffer with Orangefs' userspace parts for IO. Orangefs can best exploit parallelism when the buffer is significantly filled, and struggles when the buffer is nearly empty. Buffering reads and writes through the pagecache helps us to communicate with userspace less often with buffers that are more full.

Reads can be tuned with "count" from read(2) to simulate page sizes more in line with Orangefs's buffer size (4 meg by default). Userspace will deposit "count" bytes in the shared buffer on a read and when the VFS calls orangefs_readpage, as many pagecache pages as possible will be filled at once.

Writes are coalesced even across pages when possible up to the size of the buffer between the OrangeFS client core and the kernel. This is 4 MB by default. This means that an application writing with a block size less than 4 MB will have its writes upgraded. For sequential writes, this is a big improvement.

Martin Brandenburg (19):
orangefs: implement xattr cache
orangefs: do not invalidate attributes on inode create
orangefs: simplify orangefs_inode_getattr interface
orangefs: update attributes rather than relying on server
orangefs: hold i_lock during inode_getattr
orangefs: set up and use backing_dev_info
orangefs: let setattr write to cached inode
orangefs: reorganize setattr functions to track attribute changes
orangefs: remove orangefs_readpages
orangefs: service ops done for writeback are not killable
orangefs: migrate to generic_file_read_iter
orangefs: implement writepage
orangefs: do not return successful read when the client-core
disappeared
orangefs: move do_readv_writev to direct_IO
orangefs: skip inode writeout if nothing to write
orangefs: avoid fsync service operation on flush
orangefs: write range tracking
orangefs: implement writepages
orangefs: add orangefs_revalidate_mapping

Mike Marshall (3):
orangefs: remember count when reading.
orangefs: pass slot index back to readpage.
orangefs: copy Orangefs-sized blocks into the pagecache if possible.

 fs/orangefs/acl.c              |   4 +-
 fs/orangefs/file.c             | 389 ++++++--------
 fs/orangefs/inode.c            | 908 +++++++++++++++++++++++++++++----
 fs/orangefs/namei.c            |  40 +-
 fs/orangefs/orangefs-bufmap.c  |  13 +
 fs/orangefs/orangefs-bufmap.h  |   2 +
 fs/orangefs/orangefs-debugfs.c |   4 +-
 fs/orangefs/orangefs-kernel.h  |  56 +-
 fs/orangefs/orangefs-mod.c     |   1 +
 fs/orangefs/orangefs-sysfs.c   |  22 +
 fs/orangefs/orangefs-utils.c   | 179 +++----
 fs/orangefs/super.c            |  39 +-
 fs/orangefs/waitqueue.c        |  18 +-
 fs/orangefs/xattr.c            | 106 +++-
 14 files changed, 1293 insertions(+), 488 deletions(-)
Signature · 1 line
2.20.1