Skip to contentSkip to conversation

tmpfs: add the option to disable swap

RFC Superseded Luis Chamberlain 3 participants 10 messages Feb 23, 2023

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

v2 cover tmpfs: add the option to disable swap

This adds noswap support to tmpfs. This follows up the first RFC [0], you can look at that link for details of the testing done. On this v2 I've addressed the feedback provided by Matthew Wilcox and Yosry Ahmed. I've also made some other changes. Changes on this v2:

o Matthew suggested BUG_ON(!folio_test_locked(folio)) is not needed on writepage() callback for shmem so just remove that.
o Based on Matthew's feedback the inode is set up early as it is not reset in case we split the folio. So now we move all the variables we can set up really early.
o shmem writepage() should only be issued on reclaim, so just move the WARN_ON_ONCE(!wbc->for_reclaim) early so that the code and expectations are easier to read. This also avoid the folio splitting in case of that odd case.
o There are a few cases where the shmem writepage() could possibly hit, but in the total_swap_pages we just bail out. We shouldn't be splitting the folio then. Likewise for VM_LOCKED case. But for a writepage() on a VM_LOCKED case is not expected so we want to learn about it so add a WARN_ON_ONCE() on that condition.
o Based on Yosry Ahmed's feedback the patch which allows tmpfs to disable swap now just uses mapping_set_unevictable() on inode creation. In that case writepage() should not be called so we augment the WARN_ON_ONCE() for writepage() for that case to ensure that never happens.

If this all seems peachy I can move this to PATCH form next. I've tested and indeed just using mapping_set_unevictable() suffices to disable swap upon inode creation.

[0] https://lkml.kernel.org/r/[email protected]

Luis Chamberlain (5):
shmem: remove check for folio lock on writepage()
shmem: set shmem_writepage() variables early
shmem: move reclaim check early on writepages()
shmem: skip page split if we're not reclaiming
shmem: add support to ignore swap

 Documentation/mm/unevictable-lru.rst |  2 +
 include/linux/shmem_fs.h             |  1 +
 mm/shmem.c                           | 68 ++++++++++++++++++----------
 3 files changed, 48 insertions(+), 23 deletions(-)
Signature · 1 line
2.39.1
#1 lore raw

RFC v2 1/5 shmem: remove check for folio lock on writepage()

Matthew notes we should not need to check the folio lock on the writepage() callback so remove it. This sanity check has been lingering since linux-history days. We remove this as we tidy up the writepage() callback to make things a bit clearer.

---
 mm/shmem.c | 1 -
 1 file changed, 1 deletion(-)
mm/shmem.c +0 -1
index 28f3c699c8ce..b3ad619328bf 100644
@@ -1351,7 +1351,6 @@ static int shmem_writepage(struct page *page, struct writeback_control *wbc)
 		folio_clear_dirty(folio);
 	}
 
-	BUG_ON(!folio_test_locked(folio));
 	mapping = folio->mapping;
 	index = folio->index;
 	inode = mapping->host;
Signature · 1 line
2.39.1
#2 lore raw

RFC v2 2/5 shmem: set shmem_writepage() variables early

In reply to Luis Chamberlain

shmem_writepage() sets up variables typically used *after* a possible huge page split. However even if that does happen the address space mapping should not change, and the inode does not change either. So it should be safe to set that from the very beginning.

This commit makes no functional changes.

---
 mm/shmem.c | 9 +++------
 1 file changed, 3 insertions(+), 6 deletions(-)
mm/shmem.c +3 -6
index b3ad619328bf..1269482d0a5c 100644
@@ -1331,9 +1331,9 @@ int shmem_unuse(unsigned int type)
 static int shmem_writepage(struct page *page, struct writeback_control *wbc)
 {
 	struct folio *folio = page_folio(page);
-	struct shmem_inode_info *info;
-	struct address_space *mapping;
-	struct inode *inode;
+	struct address_space *mapping = folio->mapping;
+	struct inode *inode = mapping->host;
+	struct shmem_inode_info *info = SHMEM_I(inode);
 	swp_entry_t swap;
 	pgoff_t index;
 
@@ -1351,10 +1351,7 @@ static int shmem_writepage(struct page *page, struct writeback_control *wbc)
 		folio_clear_dirty(folio);
 	}
 
-	mapping = folio->mapping;
 	index = folio->index;
-	inode = mapping->host;
-	info = SHMEM_I(inode);
 	if (info->flags & VM_LOCKED)
 		goto redirty;
 	if (!total_swap_pages)
Signature · 1 line
2.39.1
#3 lore raw

RFC v2 3/5 shmem: move reclaim check early on writepages()

In reply to Luis Chamberlain

i915_gem requires huge folios to be split when swapping. However we have check for usage of writepages() to ensure it used only for swap purposes later. Avoid the splits if we're not being called for reclaim, even if they should in theory not happen.

This makes the conditions easier to follow on shem_writepage().

---
 mm/shmem.c | 24 ++++++++++++------------
 1 file changed, 12 insertions(+), 12 deletions(-)
mm/shmem.c +12 -12
index 1269482d0a5c..626eb1a0856c 100644
@@ -1337,6 +1337,18 @@ static int shmem_writepage(struct page *page, struct writeback_control *wbc)
 	swp_entry_t swap;
 	pgoff_t index;
 
+	/*
+	 * Our capabilities prevent regular writeback or sync from ever calling
+	 * shmem_writepage; but a stacking filesystem might use ->writepage of
+	 * its underlying filesystem, in which case tmpfs should write out to
+	 * swap only in response to memory pressure, and not for the writeback
+	 * threads or sync.
+	 */
+	if (!wbc->for_reclaim) {
+		WARN_ON_ONCE(1);	/* Still happens? Tell us about it! */
+		goto redirty;
+	}
+
 	/*
 	 * If /sys/kernel/mm/transparent_hugepage/shmem_enabled is "always" or
 	 * "force", drivers/gpu/drm/i915/gem/i915_gem_shmem.c gets huge pages,
@@ -1357,18 +1369,6 @@ static int shmem_writepage(struct page *page, struct writeback_control *wbc)
 	if (!total_swap_pages)
 		goto redirty;
 
-	/*
-	 * Our capabilities prevent regular writeback or sync from ever calling
-	 * shmem_writepage; but a stacking filesystem might use ->writepage of
-	 * its underlying filesystem, in which case tmpfs should write out to
-	 * swap only in response to memory pressure, and not for the writeback
-	 * threads or sync.
-	 */
-	if (!wbc->for_reclaim) {
-		WARN_ON_ONCE(1);	/* Still happens? Tell us about it! */
-		goto redirty;
-	}
-
 	/*
 	 * This is somewhat ridiculous, but without plumbing a SWAP_MAP_FALLOC
 	 * value into swapfile.c, the only way we can correctly account for a
Signature · 1 line
2.39.1
#4 lore raw

RFC v2 4/5 shmem: skip page split if we're not reclaiming

In reply to Luis Chamberlain

In theory when info->flags & VM_LOCKED we should not be getting shem_writepage() called so we should be verifying this with a WARN_ON_ONCE(). Since we should not be swapping then best to ensure we also don't do the folio split earlier too. So just move the check early to avoid folio splits in case its a dubious call.

We also have a similar early bail when !total_swap_pages so just move that earlier to avoid the possible folio split in the same situation.

---
 mm/shmem.c | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)
mm/shmem.c +6 -4
index 626eb1a0856c..a49b31d38627 100644
@@ -1349,6 +1349,12 @@ static int shmem_writepage(struct page *page, struct writeback_control *wbc)
 		goto redirty;
 	}
 
+	if (WARN_ON_ONCE(info->flags & VM_LOCKED))
+		goto redirty;
+
+	if (!total_swap_pages)
+		goto redirty;
+
 	/*
 	 * If /sys/kernel/mm/transparent_hugepage/shmem_enabled is "always" or
 	 * "force", drivers/gpu/drm/i915/gem/i915_gem_shmem.c gets huge pages,
@@ -1364,10 +1370,6 @@ static int shmem_writepage(struct page *page, struct writeback_control *wbc)
 	}
 
 	index = folio->index;
-	if (info->flags & VM_LOCKED)
-		goto redirty;
-	if (!total_swap_pages)
-		goto redirty;
 
 	/*
 	 * This is somewhat ridiculous, but without plumbing a SWAP_MAP_FALLOC
Signature · 1 line
2.39.1
#5 lore raw

RFC v2 5/5 shmem: add support to ignore swap

In reply to Luis Chamberlain

In doing experimentations with shmem having the option to avoid swap becomes a useful mechanism. One of the *raves* about brd over shmem is you can avoid swap, but that's not really a good reason to use brd if we can instead use shmem. Using brd has its own good reasons to exist, but just because "tmpfs" doesn't let you do that is not a great reason to avoid it if we can easily add support for it.

I don't add support for reconfiguring incompatible options, but if we really wanted to we can add support for that.

To avoid swap we use mapping_set_unevictable() upon inode creation, and put a WARN_ON_ONCE() stop-gap on writepages() for reclaim.

---
 Documentation/mm/unevictable-lru.rst |  2 ++
 include/linux/shmem_fs.h             |  1 +
 mm/shmem.c                           | 26 +++++++++++++++++++++++++-
 3 files changed, 28 insertions(+), 1 deletion(-)
Documentation/mm/unevictable-lru.rst +2 -0
index 53e59433497a..d7e11f492289 100644
@@ -44,6 +44,8 @@ The unevictable list addresses the following classes of unevictable pages:
 
  * Those owned by ramfs.
 
+ * Those owned by tmpfs with the noswap option.
+
  * Those mapped into SHM_LOCK'd shared memory regions.
 
  * Those mapped into VM_LOCKED [mlock()ed] VMAs.
include/linux/shmem_fs.h +1 -0
index d09d54be4ffd..98a7d53f6cc5 100644
@@ -45,6 +45,7 @@ struct shmem_sb_info {
 	kuid_t uid;		    /* Mount uid for root directory */
 	kgid_t gid;		    /* Mount gid for root directory */
 	bool full_inums;	    /* If i_ino should be uint or ino_t */
+	bool noswap;	    	    /* ingores VM relcaim / swap requests */
 	ino_t next_ino;		    /* The next per-sb inode number to use */
 	ino_t __percpu *ino_batch;  /* The next per-cpu inode number to use */
 	struct mempolicy *mpol;     /* default memory policy for mappings */
mm/shmem.c +25 -1
index a49b31d38627..d2f34147fc66 100644
@@ -116,10 +116,12 @@ struct shmem_options {
 	bool full_inums;
 	int huge;
 	int seen;
+	bool noswap;
 #define SHMEM_SEEN_BLOCKS 1
 #define SHMEM_SEEN_INODES 2
 #define SHMEM_SEEN_HUGE 4
 #define SHMEM_SEEN_INUMS 8
+#define SHMEM_SEEN_NOSWAP 16
 };
 
 #ifdef CONFIG_TMPFS
@@ -1334,6 +1336,7 @@ static int shmem_writepage(struct page *page, struct writeback_control *wbc)
 	struct address_space *mapping = folio->mapping;
 	struct inode *inode = mapping->host;
 	struct shmem_inode_info *info = SHMEM_I(inode);
+	struct shmem_sb_info *sbinfo = SHMEM_SB(inode->i_sb);
 	swp_entry_t swap;
 	pgoff_t index;
 
@@ -1349,7 +1352,7 @@ static int shmem_writepage(struct page *page, struct writeback_control *wbc)
 		goto redirty;
 	}
 
-	if (WARN_ON_ONCE(info->flags & VM_LOCKED))
+	if (WARN_ON_ONCE((info->flags & VM_LOCKED) || sbinfo->noswap))
 		goto redirty;
 
 	if (!total_swap_pages)
@@ -2374,6 +2377,8 @@ static struct inode *shmem_get_inode(struct mnt_idmap *idmap, struct super_block
 			shmem_set_inode_flags(inode, info->fsflags);
 		INIT_LIST_HEAD(&info->shrinklist);
 		INIT_LIST_HEAD(&info->swaplist);
+		if (sbinfo->noswap)
+			mapping_set_unevictable(inode->i_mapping);
 		simple_xattrs_init(&info->xattrs);
 		cache_no_acl(inode);
 		mapping_set_large_folios(inode->i_mapping);
@@ -3461,6 +3466,7 @@ enum shmem_param {
 	Opt_uid,
 	Opt_inode32,
 	Opt_inode64,
+	Opt_noswap,
 };
 
 static const struct constant_table shmem_param_enums_huge[] = {
@@ -3482,6 +3488,7 @@ const struct fs_parameter_spec shmem_fs_parameters[] = {
 	fsparam_u32   ("uid",		Opt_uid),
 	fsparam_flag  ("inode32",	Opt_inode32),
 	fsparam_flag  ("inode64",	Opt_inode64),
+	fsparam_flag  ("noswap",	Opt_noswap),
 	{}
 };
 
@@ -3565,6 +3572,10 @@ static int shmem_parse_one(struct fs_context *fc, struct fs_parameter *param)
 		ctx->full_inums = true;
 		ctx->seen |= SHMEM_SEEN_INUMS;
 		break;
+	case Opt_noswap:
+		ctx->noswap = true;
+		ctx->seen |= SHMEM_SEEN_NOSWAP;
+		break;
 	}
 	return 0;
 
@@ -3663,6 +3674,14 @@ static int shmem_reconfigure(struct fs_context *fc)
 		err = "Current inum too high to switch to 32-bit inums";
 		goto out;
 	}
+	if ((ctx->seen & SHMEM_SEEN_NOSWAP) && ctx->noswap && !sbinfo->noswap) {
+		err = "Cannot disable swap on remount";
+		goto out;
+	}
+	if (!(ctx->seen & SHMEM_SEEN_NOSWAP) && !ctx->noswap && sbinfo->noswap) {
+		err = "Cannot enable swap on remount if it was disabled on first mount";
+		goto out;
+	}
 
 	if (ctx->seen & SHMEM_SEEN_HUGE)
 		sbinfo->huge = ctx->huge;
@@ -3683,6 +3702,10 @@ static int shmem_reconfigure(struct fs_context *fc)
 		sbinfo->mpol = ctx->mpol;	/* transfers initial ref */
 		ctx->mpol = NULL;
 	}
+
+	if (ctx->noswap)
+		sbinfo->noswap = true;
+
 	raw_spin_unlock(&sbinfo->stat_lock);
 	mpol_put(mpol);
 	return 0;
@@ -3780,6 +3803,7 @@ static int shmem_fill_super(struct super_block *sb, struct fs_context *fc)
 			ctx->inodes = shmem_default_max_inodes();
 		if (!(ctx->seen & SHMEM_SEEN_INUMS))
 			ctx->full_inums = IS_ENABLED(CONFIG_TMPFS_INODE64);
+		sbinfo->noswap = ctx->noswap;
 	} else {
 		sb->s_flags |= SB_NOUSER;
 	}
Signature · 1 line
2.39.1
Acked-by on v2 5/5
#6 lore raw

Re: [RFC v2 5/5] shmem: add support to ignore swap

Quoted 7 lines · Luis Chamberlain

In doing experimentations with shmem having the option to avoid swap becomes a useful mechanism. One of the *raves* about brd over shmem is you can avoid swap, but that's not really a good reason to use brd if we can instead use shmem. Using brd has its own good reasons to exist, but just because "tmpfs" doesn't let you do that is not a great reason to avoid it if we can easily add support for it.

Quoted 8 lines

I don't add support for reconfiguring incompatible options, but if we really wanted to we can add support for that.

To avoid swap we use mapping_set_unevictable() upon inode creation, and put a WARN_ON_ONCE() stop-gap on writepages() for reclaim.

---

We would have use-cases for this in systemd. We currently use ramfs for systemd's credential logic since ramfs is unswappable. It'd be very neat if we could use tmpfs instead,

#7 lore raw

Re: [RFC v2 5/5] shmem: add support to ignore swap

Quoted 3 lines · Christian Brauner

We would have use-cases for this in systemd. We currently use ramfs for systemd's credential logic since ramfs is unswappable. It'd be very neat if we could use tmpfs instead,

What is the advantage of using a swapless tmpfs over ramf?

#8 lore raw

Re: [RFC v2 5/5] shmem: add support to ignore swap

Quoted 6 lines · Christoph Hellwig
Quoted 3 lines · Christian Brauner

We would have use-cases for this in systemd. We currently use ramfs for systemd's credential logic since ramfs is unswappable. It'd be very neat if we could use tmpfs instead,

What is the advantage of using a swapless tmpfs over ramf?

There are a few reasons we usually prefer tmpfs over ramfs. Iirc, ramfs doesn't have limits and grows dynamically. So we currently only use it from the most privileged process where we do our own accounting and immediately remount the superblock read-only. Tmpfs on the other hand offers various ways to restrict memory consumption.

Other reasons are that ramfs doesn't support selinux labels, xattrs, and acls in general which come in quite handy. Starting with kernel v6.3 tmpfs does also support idmapped mounts. So we usually always prefer ramfs over tmpfs unless we have a very specific need such as the memory not being swapped out.

#9 lore raw

Re: [RFC v2 5/5] shmem: add support to ignore swap

Quoted 6 lines · Christian Brauner
Quoted 5 lines · Christoph Hellwig
Quoted 3 lines · Christian Brauner

We would have use-cases for this in systemd. We currently use ramfs for systemd's credential logic since ramfs is unswappable. It'd be very neat if we could use tmpfs instead,

Quoted 7 lines
Quoted 1 line

What is the advantage of using a swapless tmpfs over ramf?

There are a few reasons we usually prefer tmpfs over ramfs. Iirc, ramfs doesn't have limits and grows dynamically. So we currently only use it from the most privileged process where we do our own accounting and immediately remount the superblock read-only. Tmpfs on the other hand offers various ways to restrict memory consumption.

Size limits is just one bell, in fact ramfs has no configurable options. So in fact *all* options parsed on shmem_parse_options() are only available with tmpfs, some of the options are:

* size
* number of blocks
* number of inodes
* NUMA memory allocation policy
* huge pages

Quoted 5 lines

Other reasons are that ramfs doesn't support selinux labels, xattrs, and acls in general which come in quite handy. Starting with kernel v6.3 tmpfs does also support idmapped mounts. So we usually always prefer ramfs over tmpfs unless we have a very specific need such as the memory not being swapped out.

I guess its time to update Documentation/filesystems/tmpfs.rst.

Luis