Skip to content

Adaptive read-ahead V11

Wu Fengguang lkml 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 Latest23 patches 13 repliesIn review

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/23readahead: kconfig optionsmm/Kconfig+55 -00 replies
2/23radixtree: look-aside cacheinclude/linux/radix-tree.h lib/radix-tree.c+156 -322 replies
3/23radixtree: hole scanning functionsinclude/linux/radix-tree.h lib/radix-tree.c+108 -00 replies
4/23readahead: page flag PG_readaheadinclude/linux/page-flags.h mm/page_alloc.c+6 -10 replies
5/23readahead: refactor do_generic_mapping_read()mm/filemap.c+12 -60 replies
6/23readahead: refactor __do_page_cache_readahead()mm/readahead.c+9 -60 replies
7/23readahead: insert cond_resched() callsfs/mpage.c mm/readahead.c+10 -33 replies
8/23readahead: common macrosmm/readahead.c+13 -20 replies
9/23readahead: events accountingmm/readahead.c+269 -10 replies
10/23readahead: support functionsinclude/linux/mm.h mm/readahead.c+127 -00 replies
11/23readahead: sysctl parametersDocumentation/sysctl/vm.txt include/linux/sysctl.h kernel/sysctl.c +1 more+84 -00 replies
12/23readahead: min/max sizesinclude/linux/mm.h mm/readahead.c+21 -00 replies
13/23readahead: page cache aging accountinginclude/linux/mm.h mm/memory.c mm/readahead.c +2 more+66 -00 replies
14/23readahead: state based methodinclude/linux/fs.h mm/readahead.c+317 -100 replies
15/23readahead: context based methodmm/readahead.c+343 -00 replies
16/23readahead: other methodsmm/readahead.c+153 -00 replies
17/23readahead: call schemeinclude/linux/mm.h mm/filemap.c mm/readahead.c+205 -40 replies
18/23readahead: laptop modeinclude/linux/writeback.h mm/page-writeback.c mm/readahead.c+37 -10 replies
19/23readahead: loop casedrivers/block/loop.c+6 -00 replies
20/23readahead: nfsd casefs/nfsd/vfs.c mm/readahead.c+12 -30 replies
21/23readahead: debug radix tree new functionsmm/readahead.c+24 -00 replies
22/23readahead: debug traces showing accessed file namesmm/readahead.c+14 -00 replies
23/23readahead: debug traces showing read patternsmm/filemap.c+22 -10 replies

Reviews trailers per patch

1 of 23 patches has a Reviewed-by or Acked-by.

PatchNick Piggin
1/23
2/23A-b
3/23
4/23
5/23
6/23
7/23
8/23
9/23
10/23
11/23
12/23
13/23
14/23
15/23
16/23
17/23
18/23
19/23
20/23
21/23
22/23
23/23

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v1 thread

Cover letter [PATCH 00/23] Adaptive read-ahead V11

Mornings,

A fresh patch for a fresh new day, and wish you a good appetite ;)

Highlights in this release:
- The patch series are heavily reworked.
- The code is re-audited and made cleaner.
- The old stock read-ahead logic is untouched and will always be available; the new adaptive read-ahead logic will be presented as a compile time selectable feature.

Why do we need this?
In short, the stock read-ahead logic does not cover many important I/O applications. This patch series present linux users a new option. Please refer to the first patch in this series for the new features.

Patches in the series:

[PATCH 01/23] readahead: kconfig options
[PATCH 02/23] radixtree: look-aside cache
[PATCH 03/23] radixtree: hole scanning functions
[PATCH 04/23] readahead: page flag PG_readahead
[PATCH 05/23] readahead: refactor do_generic_mapping_read() [PATCH 06/23] readahead: refactor __do_page_cache_readahead() [PATCH 07/23] readahead: insert cond_resched() calls [PATCH 08/23] readahead: common macros
[PATCH 09/23] readahead: events accounting
[PATCH 10/23] readahead: support functions
[PATCH 11/23] readahead: sysctl parameters
[PATCH 12/23] readahead: min/max sizes
[PATCH 13/23] readahead: page cache aging accounting [PATCH 14/23] readahead: state based method
[PATCH 15/23] readahead: context based method
[PATCH 16/23] readahead: other methods
[PATCH 17/23] readahead: call scheme
[PATCH 18/23] readahead: laptop mode
[PATCH 19/23] readahead: loop case
[PATCH 20/23] readahead: nfsd case
[PATCH 21/23] readahead: debug radix tree new functions [PATCH 22/23] readahead: debug traces showing accessed file names [PATCH 23/23] readahead: debug traces showing read patterns

Note that the last three patches are optional and only to help early stage development.

Patches for stable kernels will soon be available at:
http://www.vanheusden.com/ara/
Thanks to Folkert van Heusden for providing the free hosting!

Changelog
=========

V11 2006-03-19
- patchset rework
- add kconfig option to make the feature compile-time selectable
- improve radix tree scan functions
- fix bug of using smp_processor_id() in preemptible code
- avoid overflow in compute_thrashing_threshold()
- disable sparse read prefetching if (readahead_hit_rate == 1)
- make thrashing recovery a standalone function
- random cleanups

V10 2005-12-16
- remove delayed page activation
- remove live page protection
- revert mmap readaround to old behavior
- default to original readahead logic
- default to original readahead size
- merge comment fixes from Andreas Mohr
- merge radixtree cleanups from Christoph Lameter
- reduce sizeof(struct file_ra_state) by unnamed union
- stateful method cleanups
- account other read-ahead paths

V9 2005-12-3
- standalone mmap read-around code, a little more smart and tunable
- make stateful method sensible of request size
- decouple readahead_ratio from live pages protection
- let readahead_ratio contribute to ra_size grow speed in stateful method
- account variance of ra_size

V8 2005-11-25

- balance zone aging only in page relaim paths and do it right
- do the aging of slabs in the same way as zones
- add debug code to dump the detailed page reclaim steps
- undo exposing of struct radix_tree_node and uninline related functions
- work better with nfsd
- generalize accelerated context based read-ahead
- account smooth read-ahead aging based on page referenced/activate bits
- avoid divide error in compute_thrashing_threshold()
- more low lantency efforts
- update some comments
- rebase debug actions on debugfs entries instead of magic readahead_ratio values

V7 2005-11-09

- new tunable parameters: readahead_hit_rate/readahead_live_chunk
- support sparse sequential accesses
- delay look-ahead if drive is spinned down in laptop mode
- disable look-ahead for loopback file
- make mandatory thrashing protection more simple and robust
- attempt to improve responsiveness on large read-ahead size

V6 2005-11-01

- cancel look-ahead in laptop mode
- increase read-ahead limit to 0xFFFF pages

V5 2005-10-28

- rewrite context based method to make it clean and robust
- improved accuracy of stateful thrashing threshold estimation
- make page aging equal to the number of code pages scanned
- sort out the thrashing protection logic
- enhanced debug/accounting facilities

V4 2005-10-15

- detect and save live chunks on page reclaim
- support database workload
- support reading backward
- radix tree lookup look-aside cache

V3 2005-10-06

- major code reorganization and documention
- stateful estimation of thrashing-threshold
- context method with accelerated grow up phase
- adaptive look-ahead
- early detection and rescue of pages in danger
- statitics data collection
- synchronized page aging between zones

V2 2005-09-15

- delayed page activation
- look-ahead: towards pipelined read-ahead

V1 2005-09-13

Initial release which features:
o stateless (for now)
o adapts to available memory / read speed
o free of thrashing (in theory)

And handles:
o large number of slow streams (FTP server)
o open/read/close access patterns (NFS server)
o multiple interleaved, sequential streams in one file (multithread / multimedia / database)

Cheers,
Wu Fengguang
--
Dept. Automation University of Science and Technology of China