Skip to content

RFC Neural Storage Driver - learning page cache prefetcher

Ayhan Aydin linux-fsdevel linux-mm lkml 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 RFC Latest3 patches 13 repliesIn review

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/3mm/filemap: Add NSD prefetch hook pointinclude/linux/nsd.h mm/filemap.c+16 -00 replies
2/3nsd: Core prediction enginefs/nsd/Kconfig fs/nsd/Makefile fs/nsd/core.c+477 -00 replies
3/3Documentation: Add NSD filesystem documentationDocumentation/filesystems/nsd.rst+77 -00 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.

Read the v1 thread

Cover letter [RFC PATCH 0/3] Neural Storage Driver - learning page cache prefetcher

Hello,

This RFC introduces NSD (Neural Storage Driver), a learning prefetcher for the Linux kernel page cache. NSD monitors I/O patterns via a hook in filemap_read() and prefetches pages using page_cache_sync_readahead().

Background
==========
The kernel existing readahead uses a fixed-window approach that works well for purely sequential access. NSD builds on this by learning access patterns at 4KB region granularity using a synaptic Markov chain:

- Records transitions between file regions
- Detects sequential strides of arbitrary length
- Issues page_cache_sync_readahead() for predicted pages
- Achieves 98% real hit rate on prefetched pages (SSD)

Performance (x86_64, SATA SSD, kernel 7.0.0)
=============================================
SQLite FTS (4GB table): -18.8% query time
Sequential 64K buffered: +22.6% throughput
Random 4K buffered: +1.1% (noise)

Read latency: 130us (no regression)

Key Question
============
Is this approach suitable for the Linux readahead infrastructure? I would like to hear your opinions on the architecture.

The module currently hooks filemap_read() and issues prefetch via page_cache_sync_readahead(). Would a deeper integration with the existing ra_state / file_ra_struct be preferred? Should we extend file_ra_struct to accommodate pattern history, or keep the predictor as a separate subsystem?

Patch Summary
=============
1/3: mm/filemap: Add NSD prefetch hook point (+5 lines) 2/3: nsd: Core prediction engine (fs/nsd/, ~480 lines) 3/3: Documentation: Add NSD documentation and MAINTAINERS entry

---