Skip to content

dm-pcache – persistent-memory cache for block devices

Dongsheng Yang dm-devel 6 versions – Applied

Status from this message: I merged it to the device mapper git.

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v5 Latest0 of 11 patches 2 repliesApplied
  2. v411 patches 6 repliesSuperseded
  3. v311 patches 0 repliesSuperseded
Earlier versions · 3
VersionPostedPatchesRepliesStatus
v2 posting 2/211 patches6 repliesSuperseded
v2 RFC posting 1/211 patches6 repliesSuperseded
v1 RFC11 patches7 repliesSuperseded

V5 from V4 from the cover letter

- Fix get_n_vecs() bug when the buffer isn’t page-aligned; use
- bio_add_max_vecs() instead.
- Use kvcalloc() for allocations larger than 32,768 bytes.
- For mempool allocations, try GFP_NOWAIT first; if that fails, fall back to GFP_NOIO.

Patches in v5 Applied

None of the 11 patches the cover letter announces are in this archive.

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v5 yet.

Read the v5 thread

Cover letter [PATCH v5 00/11] dm-pcache – persistent-memory cache for block devices

Hi Mikulas,
This is V5 for dm-pcache, please consider merging.

As you requested, I've squashed these patches into a single patch, based
on v6.16. If you find anything that needs tweaking while merging, feel
free to modify it directly. I can also send a rebased version on top of the linux-dm branch if you prefer.

This patch has gone through quite a bit of testing on my side. I can share the highlights. Once you merge this patch into the linux-dm branch, I’ll keep running ongoing tests against dm-6.18.

1. dtg-tests for pcache[1]:
- Includes dmsetup management operations, including failure/exception scenarios.
- Failslab tests (covering the NOWAIT → NOIO fallback path you mentioned).
- fail_make_request fault-injection tests.
- Backing device delay tests.
- xfstests (group generic/rw) under different parameters.

2. Striped pmem as the cache_dev[2]: ran the same dtg-tests suite with a striped pmem configuration.

3. KASAN and memleak modes[3]: all of the above tests were also executed with KASAN and memory leak detection enabled.

4. Coverage with gcov: I used gcov to gather code coverage during the
test runs[4].
- The results show 100% function coverage and 90%+ line coverage. The remaining uncovered code is mostly BUG() lines and a few hard-to-simulate error paths.
- In the coverage report you can see the code path for mempool_alloc with GFP_NOIO is exercised and covered.

[1]: https://datatravelguide.github.io/dtg-blog/pcache/pcache_v5_result_virtio/results.html
[2]: https://datatravelguide.github.io/dtg-blog/pcache/pcache_v5_result_ram/results.html
[3]: https://datatravelguide.github.io/dtg-blog/pcache/pcache_v5_result_kasan/results.html
[4]: https://datatravelguide.github.io/dtg-blog/pcache/pcache_cov_v5/dm-pcache/index.html

Changelogs

V5 from V4:
- Fix get_n_vecs() bug when the buffer isn’t page-aligned; use
- bio_add_max_vecs() instead.
- Use kvcalloc() for allocations larger than 32,768 bytes.
- For mempool allocations, try GFP_NOWAIT first; if that fails, fall back to GFP_NOIO.

V4 from V3:
- go back to use mempool for allocation of cache_key and backing_dev_req.

- introduce backing_bvec_cache and backing_dev->bvec_pool to
allocate bvecs for backing_dev_req. Currently, the obj_size of bvec_cache is max size of bvecs. The bvecs are allocated only for write‑back backing_dev_req requests, and only a limited number of writeback requests are allocated at the same time, so the resulting memory overhead should not be too severe.

If in the futer we found that this implementation results in unacceptable memory waste, we can optimize it by switching to a multi‑level slab design.

- remove ret-value checking for bio_init_clone() as there should not be REQ_INTEGRITY in bio bi_opf, so bio_init_clone() never fail.

- remove ret-value checking for backing_dev_req_alloc() and cache_key_alloc().

V3 from V2:
- rebased against linux-dm dm-6.17
- add missing include file bitfiled.h (Mikulas)
- move kmem_cache from per-device to per-module (Mikulas)
- fix a memleak in error path found by failslab testing
- retry pcache_request in defer_req() when memory allocation fail.

V2 from V1:
- introduce req_alloc() and req_init() in backing_dev.c, then we can do req_alloc() before holding spinlock and do req_init() in subtree_walk().
- introduce pre_alloc_key and pre_alloc_req in walk_ctx, that means we can pre-allocate cache_key or backing_dev_request before subtree walking.
- use mempool_alloc() with NOIO for the allocation of cache_key and backing_dev_req.
- some coding style changes from comments of Jonathan.

V1 from RFC-V2:
- use crc32c to replace crc32
- only retry pcache_req when cache full, add pcache_req into defer_list, and wait cache invalidation happen.
- new format for pcache table, it is more easily extended with new parameters later.
- remove __packed.
- use spin_lock_irq in req_complete_fn to replace spin_lock_irqsave.
- fix bug in backing_dev_bio_end with spin_lock_irqsave.
- queue_work() inside spinlock.
- introduce inline_bvecs in backing_dev_req.
- use kmalloc_array for bvecs allocation.
- calculate ->off with dm_target_offset() before use it.

Dongsheng Yang (1):
dm-pcache: add persistent cache target in device-mapper

 .../admin-guide/device-mapper/dm-pcache.rst   | 201 ++++
 MAINTAINERS                                   |   8 +
 drivers/md/Kconfig                            |   2 +
 drivers/md/Makefile                           |   1 +
 drivers/md/dm-pcache/Kconfig                  |  17 +
 drivers/md/dm-pcache/Makefile                 |   3 +
 drivers/md/dm-pcache/backing_dev.c            | 374 ++++++++
 drivers/md/dm-pcache/backing_dev.h            | 100 ++
 drivers/md/dm-pcache/cache.c                  | 445 +++++++++
 drivers/md/dm-pcache/cache.h                  | 632 +++++++++++++
 drivers/md/dm-pcache/cache_dev.c              | 299 ++++++
 drivers/md/dm-pcache/cache_dev.h              |  70 ++
 drivers/md/dm-pcache/cache_gc.c               | 170 ++++
 drivers/md/dm-pcache/cache_key.c              | 888 ++++++++++++++++++
 drivers/md/dm-pcache/cache_req.c              | 835 ++++++++++++++++
 drivers/md/dm-pcache/cache_segment.c          | 293 ++++++
 drivers/md/dm-pcache/cache_writeback.c        | 258 +++++
 drivers/md/dm-pcache/dm_pcache.c              | 497 ++++++++++
 drivers/md/dm-pcache/dm_pcache.h              |  67 ++
 drivers/md/dm-pcache/pcache_internal.h        | 117 +++
 drivers/md/dm-pcache/segment.c                |  61 ++
 drivers/md/dm-pcache/segment.h                |  74 ++
 22 files changed, 5412 insertions(+)
 create mode 100644 Documentation/admin-guide/device-mapper/dm-pcache.rst
 create mode 100644 drivers/md/dm-pcache/Kconfig
 create mode 100644 drivers/md/dm-pcache/Makefile
 create mode 100644 drivers/md/dm-pcache/backing_dev.c
 create mode 100644 drivers/md/dm-pcache/backing_dev.h
 create mode 100644 drivers/md/dm-pcache/cache.c
 create mode 100644 drivers/md/dm-pcache/cache.h
 create mode 100644 drivers/md/dm-pcache/cache_dev.c
 create mode 100644 drivers/md/dm-pcache/cache_dev.h
 create mode 100644 drivers/md/dm-pcache/cache_gc.c
 create mode 100644 drivers/md/dm-pcache/cache_key.c
 create mode 100644 drivers/md/dm-pcache/cache_req.c
 create mode 100644 drivers/md/dm-pcache/cache_segment.c
 create mode 100644 drivers/md/dm-pcache/cache_writeback.c
 create mode 100644 drivers/md/dm-pcache/dm_pcache.c
 create mode 100644 drivers/md/dm-pcache/dm_pcache.h
 create mode 100644 drivers/md/dm-pcache/pcache_internal.h
 create mode 100644 drivers/md/dm-pcache/segment.c
 create mode 100644 drivers/md/dm-pcache/segment.h
Signature · 1 line
2.43.0