mm/shmem: various (urgent?) fixes
Subsystems (5)
David Hildenbrand (Arm)
maintainerv4 cover mm/shmem: various (urgent?) fixes
Hi,
Thanks everyone for the fast review and tests!
I'm planning on picking these myself and routing them through mm-core.
This is a handful of fixes for shmem/tmpfs. I sneaked in a related hugetlb fix that stalled upstream.
Some of the stuff is more urgent (patch #1, F_SEAL_EXEC, truncation hangs), others are less urgent but still rather unpleasant.
The majority are fixes from Anthropic that I cleaned up or reworked. While doing that, I found a bunch of other issues and tried to fix them somewhat cleanly.
Mote details, including other issues not fixed by this series can be found in the cover letter of v1.
---
Changes in v4:
- "mm/shmem: honour F_SEAL_EXEC when an ACL is set"
- #define shmem_set_acl simple_set_acl
- Document remaining race in "mm/shmem: hold off eviction in shmem_writeout()"
- Link to v3: https://patch.msgid.link/[email protected]
Changes in v3:
- Rebased on mm-stable
- Fixup for shmem_set_acl() no longer required
- Re-include the commit I dropped by accident
- Apply tags
- Link to v2: https://patch.msgid.link/[email protected]
Changes in v2:
- Rebased on mm-stable
- Fixup shmem_set_acl() compilation issue after rebase
- Dropped: "mm/shmem: avoid moving the truncation start backwards"
- Fixed up: "mm/shmem: stop retrying folios that cannot be split"
- In "hugetlbfs: enforce F_SEAL_EXEC in setattr", check for S_ISREG
- Smaller patch description fixups
- Picked up tags
- Link to v1: https://patch.msgid.link/[email protected]
---
Chris Mason (4):
mm/mempolicy: charge shared policy nodes mm/shmem: honour F_SEAL_EXEC when an ACL is set mm/shmem: stop retrying folios that cannot be split mm/shmem: preserve data in clean folios when splitting fails
David Hildenbrand (Arm) (7):
mm/shmem: discard whole new large folios on fallocate rollback mm/shmem: rescan after nonuniform folio splitting mm/shmem: fix truncate hang in large swap entry
mm/shmem: preserve allocation errors when swapping in large entries mm/shmem: fix truncate loop after boundary swapin failure mm/shmem: fix accounting of swapin-error markers
mm/shmem: fix truncation of files containing only swapin-error markers
Eva Kurchatova (1):
hugetlbfs: enforce F_SEAL_EXEC in setattr
Josef Bacik (3):
tmpfs: keep the quota soft limits of an unused id
tmpfs: preserve explicitly disabled quota limits mm/shmem: hold off eviction in shmem_writeout()
fs/hugetlbfs/inode.c | 8 +++ include/linux/memfd.h | 9 +++ mm/mempolicy.c | 4 +- mm/shmem.c | 168 ++++++++++++++++++++++++++++++++++---------------- mm/shmem_quota.c | 28 +++++---- 5 files changed, 149 insertions(+), 68 deletions(-)
---
base-commit: 65ba8f34ec886ec72b2c07ddc8cf3d815c32869d
change-id: 20261025-fixes_shmem-2782719b130e
--
Cheers,
David
David Hildenbrand (Arm)
maintainerPATCH v4 1/15 mm/mempolicy: charge shared policy nodes
From: Chris Mason <[email protected]>
Any local user can bypass memory.max with mbind() on a tmpfs file, and allocate unbounded kernel memory.
mbind() sets a NUMA memory policy for a range of a mapping. If the mapping is of a shmem and friends, the kernel also stores the policy on the inode, in a tree with one entry per range of the file.
Each mbind() on a single page of a mapping can add one shared policy node (struct sp_node) and one struct mempolicy to that tree. They outlive the mapping and stay around until the inode is destroyed. The tree is not bounded by the file size, so the file can be empty.
Memory that user space can allocate must be charged to a memory cgroup. A slab cache does that for all of its objects if it is created with SLAB_ACCOUNT. numa_policy_init() creates the slab caches for both structs without SLAB_ACCOUNT, so they are not charged to any cgroup.
Reproduced with mbind() on single pages of an empty tmpfs file, from a cgroup with memory.max=8M: the global OOM killer killed tasks owned by root, and the machine panicked.
Fix this by creating both caches with SLAB_ACCOUNT. This change implies that mbind() can now fail when running into memcg limits. Further, the policy_cache also serves task and VMA policies, so they are similarly accounted now.
Note that per-file limits are not really helpful, as the user could simply use many files; having limits per user is tricky as well.
Fix generated by an LLM and submitted by Chris Mason in private.
--- mm/mempolicy.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
mm/mempolicy.c +2 -2
@@ -3372,11 +3372,11 @@ void __init numa_policy_init(void) policy_cache =kmem_cache_create ("numa_policy" ,sizeof (struct mempolicy ), -0 ,SLAB_PANIC ,NULL ); +0 ,SLAB_PANIC |SLAB_ACCOUNT ,NULL ); sn_cache =kmem_cache_create ("shared_policy_node" ,sizeof (struct sp_node ), -0 ,SLAB_PANIC ,NULL ); +0 ,SLAB_PANIC |SLAB_ACCOUNT ,NULL );for_each_node (nid) { preferred_node_policy[nid] = (struct mempolicy ) {
Signature · 1 line
2.43.0
David Hildenbrand (Arm)
maintainerPATCH v4 2/15 tmpfs: keep the quota soft limits of an unused id
In reply to David Hildenbrand (Arm)From: Josef Bacik <[email protected]>
When the last reference to a dquot goes away, shmem_release_dquot() stores its limits in the rb tree of the mount, unless shmem_is_empty_dquot() says that there is nothing worth storing. That function looks at the usage and at the two hard limits only.
So if the administrator sets nothing but soft limits for an id that owns no file yet, the entry is thrown away as soon as the dquot that Q_SETQUOTA used is released, and the id never gets to see a limit.
To fix it, compare the soft limits as well: an entry can be empty only if both soft limits are zero as well, which is exactly what shmem_acquire_dquot() sets for a new entry.
Fix generated by an LLM and sent by Josef Bacik in private.
--- mm/shmem_quota.c | 2 ++ 1 file changed, 2 insertions(+)
mm/shmem_quota.c +2 -0
@@ -256,6 +256,8 @@ static bool shmem_is_empty_dquot(struct dquot *dquot)if (test_bit (DQ_FAKE_B , &dquot->dq_flags) || (dquot->dq_dqb.dqb_curspace ==0 && dquot->dq_dqb.dqb_curinodes ==0 && + dquot->dq_dqb.dqb_bsoftlimit ==0 && + dquot->dq_dqb.dqb_isoftlimit ==0 && dquot->dq_dqb.dqb_bhardlimit == bhardlimit && dquot->dq_dqb.dqb_ihardlimit == ihardlimit))return true;
Signature · 1 line
2.43.0
David Hildenbrand (Arm)
maintainerPATCH v4 3/15 tmpfs: preserve explicitly disabled quota limits
In reply to David Hildenbrand (Arm)From: Josef Bacik <[email protected]>
An administrator’s decision to disable a user’s or group’s quota limits can be silently lost when tmpfs restores the mount defaults after releasing and reacquiring the quota record. This can cause later file creation or writes to fail with EDQUOT, even though the administrator has explicitly disabled quota limits for the user or group.
do_set_dqblk() sets DQ_FAKE_B when all block and inode limits are zero. This means quota enforcement is disabled for that user or group. shmem_is_empty_dquot() treats the flag as a reason to discard the entry from the mount's quota tree. The next shmem_acquire_dquot() finds no saved entry and recreates it with the mount's default hard limits.
If the cached dquot is reused, DQ_FAKE_B remains set: Q_GETQUOTA reports the restored defaults, but enforcement is still disabled. Once that dquot has been reclaimed, a new dquot enforces the restored defaults.
Remove the DQ_FAKE_B test. Only discard entries with zero usage, zero soft limits and hard limits equal to the mount defaults. Explicitly disabled limits are then preserved whenever the mount defaults are nonzero.
Explicitly disabled limits now retain a quota-tree entry, just like other limits that differ from the mount defaults.
Rename shmem_is_empty_dquot() to shmem_is_default_dquot() to describe the check for default values.
Fix generated by an LLM and sent by Josef Bacik in private.
--- mm/shmem_quota.c | 30 +++++++++++++++--------------- 1 file changed, 15 insertions(+), 15 deletions(-)
mm/shmem_quota.c +15 -15
@@ -239,7 +239,7 @@ static int shmem_acquire_dquot(struct dquot *dquot)return ret; } -static bool shmem_is_empty_dquot (struct dquot *dquot) +static bool shmem_is_default_dquot (struct dquot *dquot) {struct shmem_sb_info *sbinfo = dquot->dq_sb->s_fs_info;qsize_t bhardlimit;
@@ -253,21 +253,21 @@ static bool shmem_is_empty_dquot(struct dquot *dquot) ihardlimit = sbinfo->qlimits.grpquota_ihardlimit; } -if (test_bit (DQ_FAKE_B , &dquot->dq_flags) || - (dquot->dq_dqb.dqb_curspace ==0 && - dquot->dq_dqb.dqb_curinodes ==0 && - dquot->dq_dqb.dqb_bsoftlimit ==0 && - dquot->dq_dqb.dqb_isoftlimit ==0 && - dquot->dq_dqb.dqb_bhardlimit == bhardlimit && - dquot->dq_dqb.dqb_ihardlimit == ihardlimit)) -return true; - -return false; +/* +* Defaults are zero usage and soft limits, and the mount's hard limits. +* DQ_FAKE_B only means that *all* limits are zero, which need not match +* the mount defaults. +*/ +return dquot->dq_dqb.dqb_curspace ==0 && + dquot->dq_dqb.dqb_curinodes ==0 && + dquot->dq_dqb.dqb_bsoftlimit ==0 && + dquot->dq_dqb.dqb_isoftlimit ==0 && + dquot->dq_dqb.dqb_bhardlimit == bhardlimit && + dquot->dq_dqb.dqb_ihardlimit == ihardlimit; }/* -* Store limits from dquot in the tree unless it's fake. If it is fake -* remove the id from the tree since there is no useful information in -* there. +* Remove the ID from the tree if it has no usage and its limits match +* the mount defaults. Otherwise, save its limits for the next acquire. */ static int shmem_release_dquot (struct dquot *dquot) {
@@ -302,7 +302,7 @@ static int shmem_release_dquot(struct dquot *dquot)return -ENOENT ; found: -if (shmem_is_empty_dquot (dquot)) { +if (shmem_is_default_dquot (dquot)) {/* Remove entry from the tree */ rb_erase (&entry->node, info->dqi_priv);kfree (entry);
Signature · 1 line
2.43.0
David Hildenbrand (Arm)
maintainerPATCH v4 4/15 mm/shmem: honour F_SEAL_EXEC when an ACL is set
In reply to David Hildenbrand (Arm)From: Chris Mason <[email protected]>
The owner of a memfd sealed with F_SEAL_EXEC can still change its execute bits by setting a POSIX access ACL. With CONFIG_TMPFS_POSIX_ACL, this lets an unprivileged owner make a non-executable memfd executable, bypassing vm.memfd_noexec=2.
simple_set_acl() updates i_mode directly, bypassing the seal check in shmem_setattr(). Add shmem_set_acl() to check the resulting mode. Share the F_SEAL_EXEC check with shmem_setattr() using memfd_check_seals_mode() in include/linux/memfd.h.
Fix generated by an LLM and sent by Chris Mason in private.
--- include/linux/memfd.h | 9 +++++++++ mm/shmem.c | 35 ++++++++++++++++++++++++++++++----- 2 files changed, 39 insertions(+), 5 deletions(-)
include/linux/memfd.h +9 -0
@@ -2,10 +2,19 @@#ifndef __LINUX_MEMFD_H#define __LINUX_MEMFD_H +#include <linux/fcntl.h> #include <linux/file.h> #define MEMFD_ANON_NAME "[memfd]" +static inline int memfd_check_seals_mode (unsigned int seals,umode_t old_mode, +umode_t new_mode) +{ +if ((seals &F_SEAL_EXEC ) && ((old_mode ^ new_mode) &0111 )) +return -EPERM ; +return 0 ; +} +#ifdef CONFIG_MEMFD_CREATE extern long memfd_fcntl (struct file *file,unsigned int cmd,unsigned int arg);struct folio *memfd_alloc_folio (struct file *memfd,pgoff_t idx);
mm/shmem.c +30 -5
@@ -77,6 +77,7 @@ static struct vfsmount *shm_mnt __ro_after_init;#include <linux/magic.h> #include <linux/syscalls.h> #include <linux/fcntl.h> +#include <linux/memfd.h> #include <uapi/linux/memfd.h> #include <linux/rmap.h> #include <linux/uuid.h>
@@ -1546,10 +1547,11 @@ static int shmem_setattr(const struct mnt_idmap *idmap,if (error)return error; -if ((info->seals &F_SEAL_EXEC ) && (attr->ia_valid &ATTR_MODE )) { -if ((inode->i_mode ^ attr->ia_mode) &0111 ) { -return -EPERM ; - } +if (attr->ia_valid &ATTR_MODE ) { + error =memfd_check_seals_mode (info->seals, inode->i_mode, + attr->ia_mode); +if (error) +return error; }if (S_ISREG (inode->i_mode) && (attr->ia_valid &ATTR_SIZE )) {
@@ -5404,12 +5406,35 @@ static const struct file_operations shmem_file_operations = {#endif }; +#ifdef CONFIG_TMPFS_POSIX_ACL +static int shmem_set_acl (const struct mnt_idmap *idmap,struct dentry *dentry, +struct posix_acl *acl,int type) +{ +struct inode *inode =d_inode (dentry); +umode_t mode = inode->i_mode; +int error; + +if (type ==ACL_TYPE_ACCESS ) { + error =posix_acl_equiv_mode (acl, &mode); +if (error <0 ) +return error; + error =memfd_check_seals_mode (SHMEM_I (inode)->seals, + inode->i_mode, mode); +if (error) +return error; + } +return simple_set_acl (idmap, dentry, acl, type); +} +#else +#define shmem_set_acl simple_set_acl +#endif +static const struct inode_operations shmem_inode_operations = { .getattr = shmem_getattr, .setattr = shmem_setattr, #ifdef CONFIG_TMPFS_XATTR .listxattr = shmem_listxattr, - .set_acl = simple_set_acl, + .set_acl = shmem_set_acl, .fileattr_get = shmem_fileattr_get, .fileattr_set = shmem_fileattr_set, #endif
Signature · 1 line
2.43.0
David Hildenbrand (Arm)
maintainerPATCH v4 5/15 hugetlbfs: enforce F_SEAL_EXEC in setattr
In reply to David Hildenbrand (Arm)From: Eva Kurchatova <[email protected]>
F_SEAL_EXEC is enforced by shmem_setattr(), but hugetlbfs_setattr() does not check it. As a result, fchmod() can change the execute bits of a sealed hugetlbfs memfd.
Use memfd_check_seals_mode() in hugetlbfs_setattr() to reject changes to execute bits when F_SEAL_EXEC is set.
This also fixes the memfd_test hugetlbfs SEAL-EXEC subtest, where fchmod() incorrectly succeeded after applying F_SEAL_EXEC.
--- fs/hugetlbfs/inode.c | 8 ++++++++ 1 file changed, 8 insertions(+)
fs/hugetlbfs/inode.c +8 -0
@@ -28,6 +28,7 @@#include <linux/folio_batch.h> #include <linux/fs_parser.h> #include <linux/mman.h> +#include <linux/memfd.h> #include <linux/slab.h> #include <linux/dnotify.h> #include <linux/statfs.h>
@@ -840,6 +841,13 @@ static int hugetlbfs_setattr(const struct mnt_idmap *idmap,if (error)return error; +if (S_ISREG (inode->i_mode) && (ia_valid &ATTR_MODE )) { + error =memfd_check_seals_mode (info->seals, inode->i_mode, + attr->ia_mode); +if (error) +return error; + } +trace_hugetlbfs_setattr (inode, dentry, attr);if (ia_valid &ATTR_SIZE ) {
Signature · 1 line
2.43.0
David Hildenbrand (Arm)
maintainerPATCH v4 6/15 mm/shmem: hold off eviction in shmem_writeout()
In reply to David Hildenbrand (Arm)From: Josef Bacik <[email protected]>
Once shmem_writeout() has moved the folio from the page cache to the swap cache, the folio lock no longer keeps the inode alive: truncation and eviction find a swap entry, free it and do not wait for anybody. That did not matter as long as swap_writeout() was the last thing the function did. But since it puts the folio back when swap_writeout() returns AOP_WRITEPAGE_ACTIVATE, it uses the mapping and the inode afterwards:
reclaim last close of an unlinked file
shmem_writeout()
shmem_delete_from_page_cache()
swap_writeout()
zswap_store() fails shmem_evict_inode()
writeback is disabled shmem_truncate_range()
return AOP_WRITEPAGE_ACTIVATE shmem_free_swap()
destroy_inode() ... RCU grace period, freed
shmem_add_to_page_cache(mapping)
xas_lock_irq() <- use after free
swap_writeout() behaves that way if zswap is enabled, cannot take the folio and the memcg or one of its ancestors has memory.zswap.writeback set to 0.
shmem_unuse() already added infrastructure to deal with that. Reuse info->stop_eviction to protect the inode during writeout. Increment it before removing the folio from the page cache, and decrement it after swap_writeout() and any attempted reinsertion and inode accounting. In shmem_evict_inode(), wait for the counter to reach zero even if swapoff has already removed the inode from the swap list. Add some helpers to make the code easier to digest.
There is a known remaining issue with two swapoffs racing, and the empty list check not being able to detect that race. That will be addressed separately later.
Fix generated by an LLM and sent by Josef Bacik in private.
--- mm/shmem.c | 36 ++++++++++++++++++++++++++++++------ 1 file changed, 30 insertions(+), 6 deletions(-)
mm/shmem.c +30 -6
@@ -1616,6 +1616,17 @@ static int shmem_setattr(const struct mnt_idmap *idmap,return error; } +static void shmem_disallow_eviction (struct shmem_inode_info *info) +{ +atomic_inc (&info->stop_eviction); +} + +static void shmem_allow_eviction (struct shmem_inode_info *info) +{ +if (atomic_dec_and_test (&info->stop_eviction)) +wake_up_var (&info->stop_eviction); +} +static void shmem_evict_inode (struct inode *inode) {struct shmem_inode_info *info =SHMEM_I (inode);
@@ -1628,16 +1639,21 @@ static void shmem_evict_inode(struct inode *inode)mapping_set_exiting (inode->i_mapping);shmem_truncate_range (inode,0 , (loff_t)-1 );shmem_unused_huge_del (inode); -while (!list_empty (&info->swaplist)) { -/* Wait while shmem_unuse() is scanning this inode... */ +/* +* Writeout may still use the inode after swapoff has removed it +* from the swaplist, so always wait at least once. +*/ +do {wait_var_event (&info->stop_eviction, !atomic_read (&info->stop_eviction)); +if (list_empty (&info->swaplist)) +break ;spin_lock (&shmem_swaplist_lock);/* ...but beware of the race if we peeked too early */ if (!atomic_read (&info->stop_eviction))list_del_init (&info->swaplist);spin_unlock (&shmem_swaplist_lock); - } + }while (!list_empty (&info->swaplist)); }simple_xattrs_free (&sbinfo->xa_cache, &info->xattrs, sbinfo->max_inodes ? &freed :NULL );
@@ -1776,15 +1792,14 @@ int shmem_unuse(unsigned int type)* remove placeholder inode from swaplist, nor let it be freed * (igrab() would protect from unlink, but not from unmount). */ -atomic_inc (&info->stop_eviction); +shmem_disallow_eviction (info);spin_unlock (&shmem_swaplist_lock); error =shmem_unuse_inode (&info->vfs_inode, type);cond_resched ();spin_lock (&shmem_swaplist_lock); -if (atomic_dec_and_test (&info->stop_eviction)) -wake_up_var (&info->stop_eviction); +shmem_allow_eviction (info);if (error)break ;if (list_empty (&info->swaplist))
@@ -1915,6 +1930,13 @@ int shmem_writeout(struct swap_io_ctx *ctx, struct folio *folio,spin_unlock (&shmem_swaplist_lock); } +/* +* Once the folio has left the page cache, its lock no longer +* keeps the inode from being evicted and freed: hold eviction +* off, in case the folio has to be put back below. +*/ +shmem_disallow_eviction (info); +folio_dup_swap (folio,NULL );shmem_delete_from_page_cache (folio,swp_to_radix_entry (folio->swap));
@@ -1922,6 +1944,7 @@ int shmem_writeout(struct swap_io_ctx *ctx, struct folio *folio, error =swap_writeout (ctx, folio);if (error !=AOP_WRITEPAGE_ACTIVATE ) {/* folio has been unlocked */ +shmem_allow_eviction (info);return error; }
@@ -1938,6 +1961,7 @@ int shmem_writeout(struct swap_io_ctx *ctx, struct folio *folio,shmem_recalc_inode (inode,0 , -nr_pages);folio_put_swap (folio,NULL ); } +shmem_allow_eviction (info);/* * The swap_cache_del_folio() below could be left for
Signature · 1 line
2.43.0
David Hildenbrand (Arm)
maintainerPATCH v4 7/15 mm/shmem: stop retrying folios that cannot be split
In reply to David Hildenbrand (Arm)From: Chris Mason <[email protected]>
A fault or collapse can insert a large folio crossing EOF after shmem_undo_range() has handled the boundary folios. If splitting that folio fails, truncate_inode_partial_folio() retains it when it is dirty. The final scan then repeatedly finds it because it restarts from an unchanged start.
Truncation keeps the inode lock and ignores fatal signals while looping. A concurrent unlink() can then block other directory operations while holding the directory lock and waiting for the file's lock.
Advance start past the retained folio, whose truncated part has already been zeroed. Do not move start backwards if the failed split has shrunk the folio.
Fix generated by an LLM and sent by Chris Mason in private.
--- mm/shmem.c | 9 +++++++++ 1 file changed, 9 insertions(+)
mm/shmem.c +9 -0
@@ -1481,6 +1481,15 @@ static void shmem_undo_range(struct inode *inode, loff_t lstart, uoff_t lend, index = start;break ; } + }else if (folio->mapping) { +/* +* Splitting failed, but the part in the +* range is already zeroed. The folio +* was not dropped, so skip it when +* restarting the scan to avoid retrying +* it forever. +*/ + start =max (start,folio_next_index (folio)); } }folio_unlock (folio);
Signature · 1 line
2.43.0
David Hildenbrand (Arm)
maintainerPATCH v4 8/15 mm/shmem: preserve data in clean folios when splitting fails
In reply to David Hildenbrand (Arm)From: Chris Mason <[email protected]>
A shared writable mapping can have dirty PTEs while its shmem folio is still clean. A read fault can install a writable PTE, and a later store sets the PTE's dirty bit without marking the folio dirty.
After shmem_undo_range() has handled the boundary folios, a concurrent read fault into a hole below the new EOF can allocate a large folio. For example, on a tmpfs mount with huge=always, the fault's offset must be within the file, but the allocated folio can extend beyond i_size. The final scan then finds this folio and calls truncate_inode_partial_folio() without marking it dirty.
Splitting normally collects the PTE dirty bits when unmapping the folio. An extra reference, for example from vmsplice(), can make it fail before that step. The fallback unmapping on split failure excludes shmem, so the helper still sees a clean folio and drops it, losing data outside the range.
CPU0: ftruncate(fd, 3M) CPU1: MAP_SHARED, tmpfs huge=always
───────────────────────────────────── ────────────────────────────────── i_size = 3Mshmem_undo_range(3M, -1)
first pass: hole, nothing to do
boundary folio at 3M: hole, none
read fault at 2M+x (below EOF)
PMD folio [2M,4M), crosses EOF writable PTE, folio clean
store to 2M+x
PTE dirty, folio still clean
vmsplice(2M+x)
extra folio reference
whole_folios: finds [2M,4M) folio truncate_inode_partial_folio()
zero [3M,4M)
folio_split()
unmap_folio(): extra ref
→ -EAGAIN, PTE dirty bit
never reaches the folio
folio clean
→ truncate_inode_folio(whole)
[2M,3M) dropped
read 2M+x → refault → zeroes
Mark large folios dirty before partial truncation in the final scan, as the boundary handling already does. Exclude failed-fallocate rollback, which must discard the folios it added.
Fix generated by an LLM and sent by Chris Mason in private.
--- mm/shmem.c | 9 +++++++++ 1 file changed, 9 insertions(+)
mm/shmem.c +9 -0
@@ -1464,6 +1464,15 @@ static void shmem_undo_range(struct inode *inode, loff_t lstart, uoff_t lend,VM_BUG_ON_FOLIO (folio_test_writeback (folio), folio); +/* +* A shared mapping can have dirty PTEs while +* the folio is still clean. Mark the folio +* dirty so a failed split cannot discard data +* outside the range. +*/ +if (!unfalloc &&folio_test_large (folio)) +folio_mark_dirty (folio); +if (!folio_test_large (folio)) {truncate_inode_folio (mapping, folio); } else if (truncate_inode_partial_folio(folio,
Signature · 1 line
2.43.0
David Hildenbrand (Arm)
maintainerPATCH v4 9/15 mm/shmem: discard whole new large folios on fallocate rollback
In reply to David Hildenbrand (Arm)When undoing a failed fallocate(), shmem_undo_range() must discard each newly allocated folio. Its second scan instead passes large folios to truncate_inode_partial_folio(), which does not necessarily discard the entire large folio, but instead tries splitting it to discard only the requested range.
For example, with huge=always, a failed allocation starting at offset 4 KiB can leave page 0 allocated in an otherwise empty file.
Discard the whole folio when unfalloc is set. Note that we enter this path only for folios that are not uptodate, and can consequently be dropped without losing any data.
--- mm/shmem.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
mm/shmem.c +1 -1
@@ -1473,7 +1473,7 @@ static void shmem_undo_range(struct inode *inode, loff_t lstart, uoff_t lend,if (!unfalloc &&folio_test_large (folio))folio_mark_dirty (folio); -if (!folio_test_large (folio)) { +if (unfalloc || !folio_test_large (folio)) {truncate_inode_folio (mapping, folio); }else if (truncate_inode_partial_folio (folio, lstart, lend,NULL ,NULL )) {
Signature · 1 line
2.43.0
David Hildenbrand (Arm)
maintainerPATCH v4 10/15 mm/shmem: rescan after nonuniform folio splitting
In reply to David Hildenbrand (Arm)The final scan in shmem_undo_range() restarts after a successful split only if the original folio is no longer large. A nonuniform split can leave that folio large, so the scan skips the newly created folios. Hole punching then leaves pages within the hole allocated even though they could be discarded.
For example, a fault below the hole can allocate a 2 MiB folio after boundary handling. Punching a 1 MiB hole at offset 64 KiB splits the folio successfully but leaves the original folio at order 4. The scan continues past all 512 pages, leaving 2 MiB allocated instead of 1 MiB. The hole reads as zero, but the pages and their block charges remain.
--- mm/shmem.c | 16 +++++----------- 1 file changed, 5 insertions(+), 11 deletions(-)
mm/shmem.c +5 -11
@@ -1478,18 +1478,12 @@ static void shmem_undo_range(struct inode *inode, loff_t lstart, uoff_t lend, }else if (truncate_inode_partial_folio (folio, lstart, lend,NULL ,NULL )) {/* -* If we split a page, reset the loop so -* that we pick up the new sub pages. -* Otherwise the THP was entirely -* dropped or the target range was -* zeroed, so just continue the loop as -* is. +* The folio was split. Retry the scan +* to find the split folios. */ -if (!folio_test_large (folio)) { -folio_unlock (folio); - index = start; -break ; - } +folio_unlock (folio); + index = start; +break ; }else if (folio->mapping) {/* * Splitting failed, but the part in the
Signature · 1 line
2.43.0
David Hildenbrand (Arm)
maintainerPATCH v4 11/15 mm/shmem: fix truncate hang in large swap entry
In reply to David Hildenbrand (Arm)Truncating a tmpfs file or memfd to a page-aligned size inside a large swap entry can loop indefinitely in shmem_undo_range(). The task holds the inode lock and cannot be killed.
A large swap entry represents a whole swapped-out folio. For example, truncate a file whose pages 0-7 share one swap entry to four pages:
[ 0 1 2 3 | 4 5 6 7 ]
keep | discard
^
new EOF, start = 4
i_size has already been updated when boundary handling asks shmem_get_partial_folio() for page 4. Since it is swapped out, the helper calls shmem_get_folio() with SGP_READ. That rejects page 4 with -EINVAL because its starting offset equals i_size. Nothing is read back.
The final scan cannot free the entry without losing pages 0-3. It skips it, restarts from start = 4, and finds it again indefinitely. An unaligned truncate avoids this particular failure because the requested page starts below i_size. Folios already in the page-cache cannot trigger it as the direct lookup will return them beyond EOF.
Fix it by letting shmem_get_partial_folio() call shmem_swapin_folio() directly for swap entries. This bypasses the read path's i_size check, allowing the existing boundary handling to zero and split the folio. If swap-in returns a smaller folio, it also splits the swap entry so the final scan can free the entries beyond EOF. Retry the lookup if the entry changes during swap-in.
Handling swap-in allocation failures remains broken and is unchanged by this commit. Even when shmem_swapin_folio() correctly returns -ENOMEM instead of -EEXIST, shmem_get_partial_folio() returns NULL and shmem_undo_range() skips boundary handling. If the large swap entry remains intact, truncation can still loop in the final scan, while hole punching can return without zeroing the boundary.
This may read an entire large folio back from swap while holding i_rwsem, only to discard part of it. Avoiding that I/O would require splitting both the swap entry and any large folio still in the swap cache. That, however, is much bigger work.
--- mm/shmem.c | 15 +++++++-------- 1 file changed, 7 insertions(+), 8 deletions(-)
mm/shmem.c +7 -8
@@ -1298,12 +1298,10 @@ void shmem_unlock_mapping(struct address_space *mapping)static struct folio *shmem_get_partial_folio (struct inode *inode,pgoff_t index) { +const gfp_t gfp =mapping_gfp_mask (inode->i_mapping);struct folio *folio; -/* -* At first avoid shmem_get_folio(,,,SGP_READ): that fails -* beyond i_size, and reports fallocated folios as holes. -*/ +retry: folio =filemap_get_entry (inode->i_mapping, index);if (!folio)return folio;
@@ -1314,13 +1312,14 @@ static struct folio *shmem_get_partial_folio(struct inode *inode, pgoff_t index)/* The folio has been swapped out */ folio_unlock (folio);folio_put (folio); + goto retry; }/* -* But read a folio back from swap if any of it is within i_size -* (although in some cases this is just a waste of time). +* Truncation can look up a large swap entry at or beyond i_size. +* Swap it in directly to bypass the size check in shmem_get_folio(). */ - folio =NULL ; -shmem_get_folio (inode, index,0 , &folio,SGP_READ ); +if (shmem_swapin_folio (inode, index, &folio,SGP_READ , gfp,NULL ,NULL ) == -EEXIST ) + goto retry;return folio; }
Signature · 1 line
2.43.0
David Hildenbrand (Arm)
maintainerPATCH v4 12/15 mm/shmem: preserve allocation errors when swapping in large entries
In reply to David Hildenbrand (Arm)shmem_swapin_folio() adjusts swap to the requested slot within a large swap entry before allocating a folio. If allocation or charging fails, the error path compares this adjusted value with the original entry still stored in the XArray in shmem_confirm_swap().
Unless the index is at the start of the entry, this turns -ENOMEM into -EEXIST even though nothing changed.
The caller retries -EEXIST, so persistent allocation failures can leave the task spinning instead of returning the error.
Restore swap to index_entry on allocation failure so the existing error path checks the original entry. This preserves the allocation error if the entry is unchanged and still retries if it changed concurrently.
--- mm/shmem.c | 2 ++ 1 file changed, 2 insertions(+)
mm/shmem.c +2 -0
@@ -2552,6 +2552,8 @@ static int shmem_swapin_folio(struct inode *inode, pgoff_t index,else error = -ENOMEM ; folio =NULL ; +/* Make sure to validate the original value. */ + swap = index_entry; goto failed; }if (fault_type) {
Signature · 1 line
2.43.0
David Hildenbrand (Arm)
maintainerPATCH v4 13/15 mm/shmem: fix truncate loop after boundary swapin failure
In reply to David Hildenbrand (Arm)If swapping in a large entry at the new EOF fails, shmem_undo_range() leaves the entry in place. Its final scan skips the entry, then restarts from the same start index and finds it again. Truncation can loop forever while holding i_rwsem.
Advance start past the skipped entry so that a restarted scan can finish, as we already do for folios that cannot be split.
Note that this does not fix the missing zeroing when boundary swapin fails, it merely prevents the endless loop.
--- mm/shmem.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-)
mm/shmem.c +6 -4
@@ -1434,14 +1434,16 @@ static void shmem_undo_range(struct inode *inode, loff_t lstart, uoff_t lend, order =shmem_confirm_swap (mapping, indices[i],radix_to_swp_entry (folio));/* -* If found a large swap entry cross the end or start -* border, skip it as the truncate_inode_partial_folio -* above should have at least zerod its content once. +* Skip boundary entries when restarting +* the scan to avoid endless loops after +* swapin failure. */ if (order >0 ) { base =round_down (base,1 << order); -if (base < start || base + (1 << order) > end) +if (base < start || base + (1 << order) > end) { + start =max (start, base + (1 << order));continue ; + } }/* Swap was replaced by page or extended, retry */ index = base;
Signature · 1 line
2.43.0
David Hildenbrand (Arm)
maintainerPATCH v4 14/15 mm/shmem: fix accounting of swapin-error markers
In reply to David Hildenbrand (Arm)A failed swap read replaces the swap entry with a swapin-error marker in the page-cache and subtracts its pages from "alloced" and "swapped"; however, it leaves their block charge in place.
Removing the marker during truncate then subtracts from "swapped" again. This can underflow the counters or make swapped reach zero while other pages remain swapped out, causing swapoff to loop without finding them.
When installing the marker, subtract only from "swapped" and let shmem_recalc_inode() reduce "alloced" and release the block charge.
Make shmem_free_swap() report successful removal separately from the number of swap pages freed, so removing a swapin-error marker does not subtract from "swapped" again.
It's a bit confusing, but replacing a swapped out page by a swapin-error marker in shmem_set_folio_swapin_error() behaves as follows:
Initially:
alloced=1, swapped=1, nrpages=0 Calling shmem_recalc_inode(inode, 0, -1) -> swapped = 0 -> freed = 1 - 0 - 0 = 1 -> alloced -= freed -> alloced=0, swapped=0, block charge released
--- mm/shmem.c | 31 ++++++++++++++----------------- 1 file changed, 14 insertions(+), 17 deletions(-)
mm/shmem.c +14 -17
@@ -1177,12 +1177,11 @@ static void shmem_delete_from_page_cache(struct folio *folio, void *radswap) }/* -* Remove swap entry from page cache, free the swap and its page cache. Returns -* the number of pages being freed. 0 means entry not found in XArray (0 pages -* being freed). +* Remove a swap entry or swapin-error marker wholly within [index, end]. +* Return whether it was removed, counting freed swap pages in nr_swaps_freed. */ -static long shmem_free_swap (struct address_space *mapping, -pgoff_t index,pgoff_t end,void *radswap) +static bool shmem_free_swap (struct address_space *mapping, +pgoff_t index,pgoff_t end,void *radswap,long *nr_swaps_freed) {XA_STATE (xas, &mapping->i_pages, index);const softleaf_t swp =radix_to_swp_entry (radswap);
@@ -1203,8 +1202,10 @@ static long shmem_free_swap(struct address_space *mapping,xas_unlock_irq (&xas);/* A swapin-error marker holds no swap slot, so just drop it. */ -if (nr_pages &&softleaf_is_swap (swp)) +if (nr_pages &&softleaf_is_swap (swp)) {swap_put_entries_direct (swp, nr_pages); + *nr_swaps_freed += nr_pages; + }return nr_pages; }
@@ -1358,8 +1359,8 @@ static void shmem_undo_range(struct inode *inode, loff_t lstart, uoff_t lend,if (xa_is_value (folio)) {if (unfalloc)continue ; - nr_swaps_freed +=shmem_free_swap (mapping, indices[i], - end -1 , folio); +shmem_free_swap (mapping, indices[i], end -1 , + folio, &nr_swaps_freed);continue ; }
@@ -1422,13 +1423,11 @@ static void shmem_undo_range(struct inode *inode, loff_t lstart, uoff_t lend,if (xa_is_value (folio)) {int order; -long swaps_freed;if (unfalloc)continue ; - swaps_freed =shmem_free_swap (mapping, indices[i], - end -1 , folio); -if (!swaps_freed) { +if (!shmem_free_swap (mapping, indices[i], end -1 , + folio, &nr_swaps_freed)) {pgoff_t base = indices[i]; order = shmem_confirm_swap(mapping, indices[i],
@@ -1449,7 +1448,6 @@ static void shmem_undo_range(struct inode *inode, loff_t lstart, uoff_t lend, index = base;break ; } - nr_swaps_freed += swaps_freed;continue ; }
@@ -2409,11 +2407,10 @@ static void shmem_set_folio_swapin_error(struct inode *inode, pgoff_t index,folio_put_swap (folio,NULL );swap_cache_del_folio (folio);/* -* Don't treat swapin error folio as alloced. Otherwise inode->i_blocks -* won't be 0 when inode is released and thus trigger WARN_ON(i_blocks) -* in shmem_evict_inode(). +* Let shmem_recalc_inode() reduce alloced and release the block charge +* for the freed swap pages. */ -shmem_recalc_inode (inode, -nr_pages, -nr_pages); +shmem_recalc_inode (inode,0 , -nr_pages); } static int shmem_split_large_entry(struct inode *inode, pgoff_t index,
Signature · 1 line
2.43.0
David Hildenbrand (Arm)
maintainerPATCH v4 15/15 mm/shmem: fix truncation of files containing only swapin-error markers
In reply to David Hildenbrand (Arm)It might not be very common, but we can end up with shmem files that are composed only of swapin-error/poison markers.
Such markers are not counted in info->alloced. Consequently, shmem_setattr() can skip shmem_truncate_range(), leaving the markers behind.
Truncating the file to zero and growing it again will result in reads failing with -EIO instead of returning zeroes.
Check whether the mapping contains entries, in addition to info->alloced, before skipping the truncation. Keep the info->alloced check so that shmem_recalc_inode(), called at the end of shmem_undo_range(), can fix up accounting after reclaim has emptied the mapping.
--- mm/shmem.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
mm/shmem.c +2 -1
@@ -1591,7 +1591,8 @@ static int shmem_setattr(const struct mnt_idmap *idmap,if (oldsize > holebegin)unmap_mapping_range (inode->i_mapping, holebegin,0 ,1 ); -if (info->alloced) +/* Swapin-error markers are not counted in alloced. */ +if (info->alloced || !mapping_empty (inode->i_mapping))shmem_truncate_range (inode, newsize, (loff_t)-1 );/* unmap again to remove racily COWed private pages */
Signature · 1 line
2.43.0
Quoted 8 lines · David Hildenbrand (Arm)
Thanks everyone for the fast review and tests!
I'm planning on picking these myself and routing them through mm-core.
This is a handful of fixes for shmem/tmpfs. I sneaked in a related hugetlb fix that stalled upstream.
[...]
Applied, thanks!
tree: mm/core.git
branch: for-next-fixes
Applied patches will appear soon in mm-next, followed by linux-next.
Please report any outstanding bugs that were missed during review, so we can either drop the patch series or decide how to fix it up.
It is encouraged to provide Acked-bys, Reviewed-bys, and Tested-bys even though the patches have already been applied. If possible, patch trailers will be updated.
Note that commit hashes shown below are subject to change due to rebase, trailer updates or similar. Further, this is not a guarantee that the patches will go upstream. If in doubt, please check the listed branch.
[01/15] mm/mempolicy: charge shared policy nodes
https://git.kernel.org/mm/core/c/9e3b76ba [02/15] tmpfs: keep the quota soft limits of an unused id https://git.kernel.org/mm/core/c/85a7d29e [03/15] tmpfs: preserve explicitly disabled quota limits https://git.kernel.org/mm/core/c/3283c3cd [04/15] mm/shmem: honour F_SEAL_EXEC when an ACL is set https://git.kernel.org/mm/core/c/460346e9 [05/15] hugetlbfs: enforce F_SEAL_EXEC in setattr https://git.kernel.org/mm/core/c/73fe78f9 [06/15] mm/shmem: hold off eviction in shmem_writeout() https://git.kernel.org/mm/core/c/69634c49 [07/15] mm/shmem: stop retrying folios that cannot be split https://git.kernel.org/mm/core/c/52f47e06 [08/15] mm/shmem: preserve data in clean folios when splitting fails https://git.kernel.org/mm/core/c/06162197 [09/15] mm/shmem: discard whole new large folios on fallocate rollback https://git.kernel.org/mm/core/c/beba3c5a [10/15] mm/shmem: rescan after nonuniform folio splitting https://git.kernel.org/mm/core/c/920600c0 [11/15] mm/shmem: fix truncate hang in large swap entry https://git.kernel.org/mm/core/c/29c3fae7 [12/15] mm/shmem: preserve allocation errors when swapping in large entries https://git.kernel.org/mm/core/c/f8bb4422 [13/15] mm/shmem: fix truncate loop after boundary swapin failure https://git.kernel.org/mm/core/c/9df76e1e [14/15] mm/shmem: fix accounting of swapin-error markers https://git.kernel.org/mm/core/c/5dca4cd7 [15/15] mm/shmem: fix truncation of files containing only swapin-error markers https://git.kernel.org/mm/core/c/52c5619a
Signature · 3 lines
Cheers, David