RFC Neural Storage Driver - learning page cache prefetcher
Get series with b4
b4 am [email protected]Versions newest first
- v1 RFC Latest3 patches 13 repliesIn review
Patches in v1 No applies recorded yet
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/3 | mm/filemap: Add NSD prefetch hook pointinclude/ mm/ | +16 | 0 replies | |
| 2/3 | nsd: Core prediction enginefs/ fs/ fs/ | +477 | 0 replies | |
| 3/3 | Documentation: Add NSD filesystem documentationDocumentation/ | +77 | 0 replies |
Reviews trailers per patch
No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.
Cover letter [RFC PATCH 0/3] Neural Storage Driver - learning page cache prefetcher
Hello,
This RFC introduces NSD (Neural Storage Driver), a learning prefetcher for the Linux kernel page cache. NSD monitors I/O patterns via a hook in filemap_read() and prefetches pages using page_cache_sync_readahead().
Background
==========
The kernel existing readahead uses a fixed-window approach that works well for purely sequential access. NSD builds on this by learning access patterns at 4KB region granularity using a synaptic Markov chain:
- Records transitions between file regions
- Detects sequential strides of arbitrary length
- Issues page_cache_sync_readahead() for predicted pages
- Achieves 98% real hit rate on prefetched pages (SSD)
Performance (x86_64, SATA SSD, kernel 7.0.0)
=============================================
SQLite FTS (4GB table): -18.8% query time
Sequential 64K buffered: +22.6% throughput
Random 4K buffered: +1.1% (noise)
Read latency: 130us (no regression)
Key Question
============
Is this approach suitable for the Linux readahead infrastructure? I would like to hear your opinions on the architecture.
The module currently hooks filemap_read() and issues prefetch via page_cache_sync_readahead(). Would a deeper integration with the existing ra_state / file_ra_struct be preferred? Should we extend file_ra_struct to accommodate pattern history, or keep the predictor as a separate subsystem?
Patch Summary
=============
1/3: mm/filemap: Add NSD prefetch hook point (+5 lines) 2/3: nsd: Core prediction engine (fs/nsd/, ~480 lines) 3/3: Documentation: Add NSD documentation and MAINTAINERS entry
---