Skip to content

Change readahead API

Matthew Wilcox cluster-devel linux-btrfs linux-erofs linux-ext4 linux-f2fs-devel linux-fsdevel linux-mm linux-xfs lkml ocfs2-devel oe-kbuild-all 5 versions – In review

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v11 Latest25 patches 21 repliesIn review 24 of 25 applied
  2. v1025 patches 8 repliesSuperseded
  3. v925 patches 20 repliesSuperseded
Earlier versions · 2
VersionPostedPatchesRepliesStatus
v825 patches13 repliesSuperseded
v724 patches48 repliesSuperseded

v8 from the cover letter · 17 lines

- btrfs, ext4 and xfs all survive an xfstests run (thanks to Kent Overstreet for providing the ktest framework)
- iomap restructuring dropped due to Christoph's opposition and the redesign of readahead_page() meaning it wasn't needed any more.
- f2fs_mpage_readpages() made static again
- Made iomap_readahead() comment more useful
- Added kernel-doc for the entire readahead_control API
- Conditionally zero batch_count in readahead_page() (requested by John)
- Hold RCU read lock while iterating over the xarray in readahead_page_batch()
- Iterate over the correct pages in readahead_page_batch()
- Correct the return type of readahead_index() (spotted by Zi Yan)
- Added a 'skip_page' parameter to read_pages for better documentation purposes and so we can reuse the readahead_control higher in the call chain in future.
- Removed the use_list bool (requested by Christoph)
- Removed the explicit initialisation of _nr_pages to 0 (requested by Christoph & John)
- Add comments explaining why nr_to_read is being capped (requested by John)
- Reshuffled some of the patches:
- Split out adding the readahead_control API from the three patches which added it piecemeal
- Shift the final two mm patches to be with the other mm patches
- Split the f2fs "pass the inode" patch from the "convert to readahead" patch, like ext4

Patches in v11 24 of 25 applied

#Subject and filesSizeRepliesApplied
1/25mm: Move readahead prototypes from mm.hblock/blk-core.c include/linux/mm.h include/linux/pagemap.h +2 more+13 -191 reply
2/25mm: Return void from various readahead functionsmm/fadvise.c mm/internal.h mm/readahead.c+19 -280 repliesApplied
3/25mm: Ignore return value of ->readpagesmm/readahead.c+2 -61 replyApplied
4/25mm: Move readahead nr_pages check into read_pagesmm/readahead.c+7 -51 replyApplied
5/25mm: Add new readahead_control APIinclude/linux/pagemap.h+140 -04 repliesApplied
6/25mm: Use readahead_control to pass argumentsmm/readahead.c+19 -141 replyApplied
7/25mm: Rename various 'offset' parameters to 'index'mm/readahead.c+42 -440 repliesApplied
8/25mm: rename readahead loop variable to 'i'mm/readahead.c+4 -41 replyApplied
9/25mm: Remove 'page_offset' from readahead loopmm/readahead.c+3 -50 repliesApplied
10/25mm: Put readahead pages in cache earliermm/readahead.c+28 -180 repliesApplied
11/25mm: Add readahead address space operationDocumentation/filesystems/locking.rst Documentation/filesystems/vfs.rst include/linux/fs.h +1 more+32 -30 repliesApplied
12/25mm: Move end_index check out of readahead loopmm/readahead.c+8 -60 repliesApplied
13/25mm: Add page_cache_readahead_unboundedfs/ext4/verity.c fs/f2fs/data.c fs/f2fs/f2fs.h +3 more+55 -910 repliesApplied
14/25mm: Document why we don't set PageReadaheadmm/readahead.c+6 -30 repliesApplied
15/25mm: Use memalloc_nofs_save in readahead pathmm/readahead.c+14 -00 repliesApplied
16/25fs: Convert mpage_readpages to mpage_readaheadfs/block_dev.c fs/exfat/inode.c fs/ext2/inode.c +15 more+73 -1267 repliesApplied
17/25btrfs: Convert from readpages to readaheadfs/btrfs/extent_io.c fs/btrfs/extent_io.h fs/btrfs/inode.c+20 -420 repliesApplied
18/25erofs: Convert uncompressed files from readpages to readaheadfs/erofs/data.c fs/erofs/zdata.c include/trace/events/erofs.h+18 -290 repliesApplied
19/25erofs: Convert compressed files from readpages to readaheadfs/erofs/zdata.c+9 -202 repliesApplied
20/25ext4: Convert from readpages to readaheadfs/ext4/ext4.h fs/ext4/inode.c fs/ext4/readpage.c+18 -280 repliesApplied
21/25ext4: Pass the inode to ext4_mpage_readpagesfs/ext4/ext4.h fs/ext4/inode.c fs/ext4/readpage.c+4 -50 repliesApplied
22/25f2fs: Convert from readpages to readaheadfs/f2fs/data.c include/trace/events/f2fs.h+22 -310 repliesApplied
23/25f2fs: Pass the inode to f2fs_mpage_readpagesfs/f2fs/data.c+3 -40 repliesApplied
24/25fuse: Convert from readpages to readaheadfs/fuse/file.c+27 -723 repliesApplied
25/25iomap: Convert from readpages to readaheadfs/iomap/buffered-io.c fs/iomap/trace.h fs/xfs/xfs_aops.c +2 more+41 -740 repliesApplied

Reviews trailers per patch

All 25 patches have a Reviewed-by or Acked-by.

PatchJohn HubbardChristoph HellwigWilliam KucharskiDave ChinnerZi YanEric BiggersJunxiao BiJoseph QiGao XiangChao YuJaegeuk KimDarrick J. WongJohannes ThumshirnMiklos Szeredi
1/25R-b R-b R-b R-b
2/25R-b R-b R-b R-b
3/25R-b R-b R-b R-b R-b
4/25R-b R-b R-b R-b R-b
5/25R-b R-b
6/25R-b R-b R-b R-b
7/25R-b R-b
8/25R-b R-b R-b
9/25R-b R-b R-b
10/25R-b R-b
11/25R-b R-b R-b
12/25R-b R-b
13/25R-b R-b R-b T-b
14/25R-b R-b
15/25R-b
16/25R-b R-b R-b R-b R-b R-b
17/25R-b
18/25R-b A-b R-b
19/25R-b R-b A-b R-b
20/25R-b R-b
21/25R-b R-b
22/25R-b R-b R-b A-b
23/25R-b R-b R-b A-b
24/25R-b R-b A-b
25/25R-b R-b R-b

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes. Scroll the table sideways for all 14 reviewers.

Read the v11 thread

Cover letter [PATCH v11 00/25] Change readahead API

From: "Matthew Wilcox (Oracle)" <[email protected]>

This series adds a readahead address_space operation to replace the readpages operation. The key difference is that pages are added to the page cache as they are allocated (and then looked up by the filesystem) instead of passing them on a list to the readpages operation and having the filesystem add them to the page cache. It's a net reduction in code for each implementation, more efficient than walking a list, and solves the direct-write vs buffered-read problem reported by yu kuai at
https://lore.kernel.org/linux-fsdevel/20200116063601.39201-1-yukuai3 at huawei.com/

The only unconverted filesystems are those which use fscache. Their conversion is pending Dave Howells' rewrite which will make the conversion substantially easier. This should be completed by the end of the year.

I want to thank the reviewers/testers; Dave Chinner, John Hubbard, Eric Biggers, Johannes Thumshirn, Dave Sterba, Zi Yan, Christoph Hellwig and Miklos Szeredi have done a marvellous job of providing constructive criticism.

These patches pass an xfstests run on ext4, xfs & btrfs with no regressions that I can tell (some of the tests seem a little flaky before and remain flaky afterwards).

This series can also be found at
http://git.infradead.org/users/willy/linux-dax.git/shortlog/refs/tags/readahead_v11

v11: Rebased on v5.7-rc1
- Rewrote the fuse conversion to use __readahead_batch() and fix some bugs.

v10: Rebased on linux-next 20200323
- Collected some more reviewed-by tags
- Simplify nr_to_read limits (Eric Biggers)
- Convert fs/exfat instead of drivers/staging/exfat (Namjae Jeon)
- Explicitly convert a pointer to a boolean in f2fs (Eric Biggers)

v9: No code changes. Fixed a changelog and added some reviewed-by tags.

v8:
- btrfs, ext4 and xfs all survive an xfstests run (thanks to Kent Overstreet for providing the ktest framework)
- iomap restructuring dropped due to Christoph's opposition and the redesign of readahead_page() meaning it wasn't needed any more.
- f2fs_mpage_readpages() made static again
- Made iomap_readahead() comment more useful
- Added kernel-doc for the entire readahead_control API
- Conditionally zero batch_count in readahead_page() (requested by John)
- Hold RCU read lock while iterating over the xarray in readahead_page_batch()
- Iterate over the correct pages in readahead_page_batch()
- Correct the return type of readahead_index() (spotted by Zi Yan)
- Added a 'skip_page' parameter to read_pages for better documentation purposes and so we can reuse the readahead_control higher in the call chain in future.
- Removed the use_list bool (requested by Christoph)
- Removed the explicit initialisation of _nr_pages to 0 (requested by Christoph & John)
- Add comments explaining why nr_to_read is being capped (requested by John)
- Reshuffled some of the patches:
- Split out adding the readahead_control API from the three patches which added it piecemeal
- Shift the final two mm patches to be with the other mm patches
- Split the f2fs "pass the inode" patch from the "convert to readahead" patch, like ext4

v7:
- Now passes an xfstests run on ext4!
- Documentation improvements
- Move the readahead prototypes out of mm.h (new patch)
- readahead_for_each* iterators are gone; replaced with readahead_page() and readahead_page_batch()
- page_cache_readahead_limit() renamed to page_cache_readahead_unbounded() and arguments changed
- iomap_readahead_actor() restructured differently
- The readahead code no longer uses the word 'offset' to reduce ambiguity
- read_pages() now maintains the rac so we can just call it and continue instead of mucking around with branches
- More assertions
- More readahead functions return void

v6:
- Name the private members of readahead_control with a leading underscore (suggested by Christoph Hellwig)
- Fix whitespace in rst file
- Remove misleading comment in btrfs patch
- Add readahead_next() API and use it in iomap
- Add iomap_readahead kerneldoc.
- Fix the mpage_readahead kerneldoc
- Make various readahead functions return void
- Keep readahead_index() and readahead_offset() pointing to the start of this batch through the body. No current user requires this, but it's less surprising.
- Add kerneldoc for page_cache_readahead_limit
- Make page_idx an unsigned long, and rename it to just 'i'
- Get rid of page_offset local variable
- Add patch to call memalloc_nofs_save() before allocating pages (suggested by Michal Hocko)
- Resplit a lot of patches for more logical progression and easier review (suggested by John Hubbard)
- Added sign-offs where received, and I deemed still relevant

v5 switched to passing a readahead_control struct (mirroring the writepages_control struct passed to writepages). This has a number of advantages:
- It fixes a number of bugs in various implementations, eg forgetting to increment 'start', an off-by-one error in 'nr_pages' or treating 'start' as a byte offset instead of a page offset.
- It allows us to change the arguments without changing all the implementations of ->readahead which just call mpage_readahead() or iomap_readahead()
- Figuring out which pages haven't been attempted by the implementation is more natural this way.
- There's less code in each implementation.

Matthew Wilcox (Oracle) (25):
mm: Move readahead prototypes from mm.h
mm: Return void from various readahead functions
mm: Ignore return value of ->readpages
mm: Move readahead nr_pages check into read_pages
mm: Add new readahead_control API
mm: Use readahead_control to pass arguments
mm: Rename various 'offset' parameters to 'index'
mm: rename readahead loop variable to 'i'
mm: Remove 'page_offset' from readahead loop
mm: Put readahead pages in cache earlier
mm: Add readahead address space operation
mm: Move end_index check out of readahead loop
mm: Add page_cache_readahead_unbounded
mm: Document why we don't set PageReadahead
mm: Use memalloc_nofs_save in readahead path
fs: Convert mpage_readpages to mpage_readahead
btrfs: Convert from readpages to readahead
erofs: Convert uncompressed files from readpages to readahead
erofs: Convert compressed files from readpages to readahead
ext4: Convert from readpages to readahead
ext4: Pass the inode to ext4_mpage_readpages
f2fs: Convert from readpages to readahead
f2fs: Pass the inode to f2fs_mpage_readpages
fuse: Convert from readpages to readahead
iomap: Convert from readpages to readahead

 Documentation/filesystems/locking.rst |   6 +-
 Documentation/filesystems/vfs.rst     |  15 ++
 block/blk-core.c                      |   1 +
 fs/block_dev.c                        |   7 +-
 fs/btrfs/extent_io.c                  |  43 ++--
 fs/btrfs/extent_io.h                  |   3 +-
 fs/btrfs/inode.c                      |  16 +-
 fs/erofs/data.c                       |  39 ++--
 fs/erofs/zdata.c                      |  29 +--
 fs/exfat/inode.c                      |   7 +-
 fs/ext2/inode.c                       |  10 +-
 fs/ext4/ext4.h                        |   5 +-
 fs/ext4/inode.c                       |  21 +-
 fs/ext4/readpage.c                    |  25 +--
 fs/ext4/verity.c                      |  35 +---
 fs/f2fs/data.c                        |  50 ++---
 fs/f2fs/f2fs.h                        |   3 -
 fs/f2fs/verity.c                      |  35 +---
 fs/fat/inode.c                        |   7 +-
 fs/fuse/file.c                        |  99 +++-------
 fs/gfs2/aops.c                        |  23 +--
 fs/hpfs/file.c                        |   7 +-
 fs/iomap/buffered-io.c                |  92 +++------
 fs/iomap/trace.h                      |   2 +-
 fs/isofs/inode.c                      |   7 +-
 fs/jfs/inode.c                        |   7 +-
 fs/mpage.c                            |  38 ++--
 fs/nilfs2/inode.c                     |  15 +-
 fs/ocfs2/aops.c                       |  34 ++--
 fs/omfs/file.c                        |   7 +-
 fs/qnx6/inode.c                       |   7 +-
 fs/reiserfs/inode.c                   |   8 +-
 fs/udf/inode.c                        |   7 +-
 fs/xfs/xfs_aops.c                     |  13 +-
 fs/zonefs/super.c                     |   7 +-
 include/linux/fs.h                    |   2 +
 include/linux/iomap.h                 |   3 +-
 include/linux/mm.h                    |  19 --
 include/linux/mpage.h                 |   4 +-
 include/linux/pagemap.h               | 151 ++++++++++++++
 include/trace/events/erofs.h          |   6 +-
 include/trace/events/f2fs.h           |   6 +-
 mm/fadvise.c                          |   6 +-
 mm/internal.h                         |  12 +-
 mm/migrate.c                          |   2 +-
 mm/readahead.c                        | 275 ++++++++++++++++----------
 46 files changed, 583 insertions(+), 633 deletions(-)

base-commit: 8f3d9f354286745c751374f5f1fcafee6b3f3136

Signature · 1 line
2.25.1