Skip to content

fuse,virtiofs: support per-file DAX

Jeffle Xu linux-fsdevel virtio-fs 8 versions – In review

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v8 Latest7 patches 10 repliesIn review
  2. v77 patches 8 repliesSuperseded
  3. v67 patches 29 repliesSuperseded
Earlier versions · 5
VersionPostedPatchesRepliesStatus
v55 patches17 repliesSuperseded
v48 patches28 repliesSuperseded
v38 patches1 replySuperseded
v24 patches20 repliesSuperseded
v1 RFC3 patches6 repliesSuperseded

Changes in v8 from the cover letter

- rebase to v5.16
- patch 2: rename FUSE_DAX_NONE|FUSE_DAX_INODE to FUSE_DAX_INODE_DEFAULT|FUSE_DAX_INODE_USER
- patch 5: remove redundant call for fuse_is_inode_dax_mode() in process_init_reply()
- patch 5: if server's map alignment is non-compliant (fail fuse_dax_check_alignment()), the mounted fs won't work and users are required to remount it explicitly, instead of silently falling back to 'never' mode.

Patches in v8 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/7fuse: add fuse_should_enable_dax() helperfs/fuse/dax.c+9 -11 reply
2/7fuse: make DAX mount option a tri-statefs/fuse/dax.c fs/fuse/fuse_i.h fs/fuse/inode.c +1 more+52 -91 reply
3/7fuse: support per inode DAX in fuse protocolinclude/uapi/linux/fuse.h+5 -01 reply
4/7fuse: enable per inode DAXfs/fuse/dax.c fs/fuse/file.c fs/fuse/fuse_i.h +1 more+13 -91 reply
5/7fuse: negotiate per inode DAX in FUSE_INITfs/fuse/dax.c fs/fuse/fuse_i.h fs/fuse/inode.c+13 -51 reply
6/7fuse: mark inode DONT_CACHE when per inode DAX hint changesfs/fuse/dax.c fs/fuse/fuse_i.h fs/fuse/inode.c+13 -03 replies
7/7Documentation/filesystem/dax: DAX on virtiofsDocumentation/filesystems/dax.rst+18 -21 reply

Reviews trailers per patch

6 of 7 patches have a Reviewed-by or Acked-by.

PatchVivek Goyal
1/7R-b
2/7R-b
3/7R-b
4/7Reviwed-by
5/7R-b
6/7R-b
7/7R-b

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v8 thread

Cover letter [PATCH v8 0/7] fuse,virtiofs: support per-file DAX

changes since v7:
- rebase to v5.16
- patch 2: rename FUSE_DAX_NONE|FUSE_DAX_INODE to FUSE_DAX_INODE_DEFAULT|FUSE_DAX_INODE_USER
- patch 5: remove redundant call for fuse_is_inode_dax_mode() in process_init_reply()
- patch 5: if server's map alignment is non-compliant (fail fuse_dax_check_alignment()), the mounted fs won't work and users are required to remount it explicitly, instead of silently falling back to 'never' mode.

Corresponding changes to virtiofsd:
https://www.mail-archive.com/[email protected]/msg04349.html

v7: https://lore.kernel.org/all/[email protected]/T/
v6: https://lore.kernel.org/all/[email protected]/
v5: https://lore.kernel.org/all/[email protected]/
v4: https://lore.kernel.org/linux-fsdevel/[email protected]/
v3: https://www.spinics.net/lists/linux-fsdevel/msg200852.html
v2: https://www.spinics.net/lists/linux-fsdevel/msg199584.html
v1: https://www.spinics.net/lists/linux-virtualization/msg51008.html

Original Rationale for this Patchset
====================================

This patchset adds support of per-file DAX for virtiofs, which is inspired by Ira Weiny's work on ext4[1] and xfs[2].

Any comment is welcome.

[1] commit 9cb20f94afcd ("fs/ext4: Make DAX mount option a tri-state")
[2] commit 02beb2686ff9 ("fs/xfs: Make DAX mount option a tri-state")

[Purpose]
DAX may be limited in some specific situation. When the number of usable DAX windows is under watermark, the recalim routine will be triggered to reclaim some DAX windows. It may have a negative impact on the performance, since some processes may need to wait for DAX windows to be recalimed and reused then. To mitigate the performance degradation, the overall DAX window need to be expanded larger.

However, simply expanding the DAX window may not be a good deal in some scenario. To maintain one DAX window chunk (i.e., 2MB in size), 32KB (512 * 64 bytes) memory footprint will be consumed for page descriptors inside guest, which is greater than the memory footprint if it uses guest page cache when DAX disabled. Thus it'd better disable DAX for those files smaller than 32KB, to reduce the demand for DAX window and thus avoid the unworthy memory overhead.

Per-file DAX feature is introduced to address this issue, by offering a finer grained control for dax to users, trying to achieve a balance between performance and memory overhead.

[Note]
When the per-file DAX hint changes while the file is still *opened*, it is quite complicated and maybe fragile to dynamically change the DAX state, since dynamic switching needs to switch a_ops atomiclly. Ira Weiny had ever implemented a so called i_aops_sem lock [3] but eventually gave up since the complexity of the implementation [4][5][6][7].

Hence mark the inode and corresponding dentries as DONE_CACHE once the per-file DAX hint changes, so that the inode instance will be evicted and freed as soon as possible once the file is closed and the last reference to the inode is put. And then when the file gets reopened next time, the new instantiated inode will reflect the new DAX state.

In summary, when the per-file DAX hint changes for an *opened* file, the DAX state of the file won't be updated until this file is closed and reopened later. This is also how ext4/xfs per-file DAX works.

[3] https://lore.kernel.org/lkml/[email protected]/
[4] https://patchwork.kernel.org/project/xfs/cover/[email protected]/
[5] https://lore.kernel.org/lkml/[email protected]/
[6] https://lore.kernel.org/lkml/20200401040021.GC56958@magnolia/
[7] https://lore.kernel.org/lkml/20200403182904.GP80283@magnolia/

Jeffle Xu (7):
fuse: add fuse_should_enable_dax() helper
fuse: make DAX mount option a tri-state
fuse: support per inode DAX in fuse protocol
fuse: enable per inode DAX
fuse: negotiate per inode DAX in FUSE_INIT
fuse: mark inode DONT_CACHE when per inode DAX hint changes Documentation/filesystem/dax: DAX on virtiofs

 Documentation/filesystems/dax.rst | 20 +++++++++++++++--
 fs/fuse/dax.c                     | 36 +++++++++++++++++++++++++++++--
 fs/fuse/file.c                    |  4 ++--
 fs/fuse/fuse_i.h                  | 28 ++++++++++++++++++++----
 fs/fuse/inode.c                   | 28 +++++++++++++++++-------
 fs/fuse/virtio_fs.c               | 18 +++++++++++++---
 include/uapi/linux/fuse.h         |  5 +++++
 7 files changed, 118 insertions(+), 21 deletions(-)
Signature · 1 line
2.27.0