FAILED: patch "[PATCH] mm: thp: deny THP for files on anonymous inodes" failed to apply to 6.12-stable tree

The patch below does not apply to the 6.12-stable tree.
If someone wants it applied there, or to any other stable or longterm tree, then please email the backport, including the original git commit id to <[email protected]>.

To reproduce the conflict and resubmit, you may use the following commands:

git fetch https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/ linux-6.12.y
git checkout FETCH_HEAD
git cherry-pick -x dd085fe9a8ebfc5d10314c60452db38d2b75e609
# <resolve conflicts, build, test, etc.>
git commit -s
git send-email --to '<[email protected]>' --in-reply-to '2026030900-shore-output-ef28@gregkh' --subject-prefix 'PATCH 6.12.y' HEAD^..

Possible dependencies:

thanks,

greg k-h

------------------ original commit in Linus's tree ------------------

From dd085fe9a8ebfc5d10314c60452db38d2b75e609 Mon Sep 17 00:00:00 2001
From: Deepanshu Kartikey <[email protected]>
Date: Sat, 14 Feb 2026 05:45:35 +0530
Subject: [PATCH] mm: thp: deny THP for files on anonymous inodes

file_thp_enabled() incorrectly allows THP for files on anonymous inodes (e.g. guest_memfd and secretmem). These files are created via alloc_file_pseudo(), which does not call get_write_access() and leaves inode->i_writecount at 0. Combined with S_ISREG(inode->i_mode) being true, they appear as read-only regular files when CONFIG_READ_ONLY_THP_FOR_FS is enabled, making them eligible for THP collapse.

Anonymous inodes can never pass the inode_is_open_for_write() check since their i_writecount is never incremented through the normal VFS open path. The right thing to do is to exclude them from THP eligibility altogether, since CONFIG_READ_ONLY_THP_FOR_FS was designed for real filesystem files (e.g. shared libraries), not for pseudo-filesystem inodes.

For guest_memfd, this allows khugepaged and MADV_COLLAPSE to create large folios in the page cache via the collapse path, but the guest_memfd fault handler does not support large folios. This triggers WARN_ON_ONCE(folio_test_large(folio)) in kvm_gmem_fault_user_mapping().

For secretmem, collapse_file() tries to copy page contents through the direct map, but secretmem pages are removed from the direct map. This can result in a kernel crash:

    BUG: unable to handle page fault for address: ffff88810284d000
    RIP: 0010:memcpy_orig+0x16/0x130
    Call Trace:
     collapse_file
     hpage_collapse_scan_file
     madvise_collapse

Secretmem is not affected by the crash on upstream as the memory failure recovery handles the failed copy gracefully, but it still triggers confusing false memory failure reports:

Memory failure: 0x106d96f: recovery action for clean unevictable LRU page: Recovered

Check IS_ANON_FILE(inode) in file_thp_enabled() to deny THP for all anonymous inode files.

index d4ca8cfd7f9d..8e2746ea74ad 100644
@@ -94,6 +94,9 @@ static inline bool file_thp_enabled(struct vm_area_struct *vma)
 
 	inode = file_inode(vma->vm_file);
 
+	if (IS_ANON_FILE(inode))
+		return false;
+
 	return !inode_is_open_for_write(inode) && S_ISREG(inode->i_mode);
 }
 

From: Deepanshu Kartikey <[email protected]>

[ Upstream commit dd085fe9a8ebfc5d10314c60452db38d2b75e609 ]

file_thp_enabled() incorrectly allows THP for files on anonymous inodes (e.g. guest_memfd and secretmem). These files are created via alloc_file_pseudo(), which does not call get_write_access() and leaves inode->i_writecount at 0. Combined with S_ISREG(inode->i_mode) being true, they appear as read-only regular files when CONFIG_READ_ONLY_THP_FOR_FS is enabled, making them eligible for THP collapse.

Anonymous inodes can never pass the inode_is_open_for_write() check since their i_writecount is never incremented through the normal VFS open path. The right thing to do is to exclude them from THP eligibility altogether, since CONFIG_READ_ONLY_THP_FOR_FS was designed for real filesystem files (e.g. shared libraries), not for pseudo-filesystem inodes.

For guest_memfd, this allows khugepaged and MADV_COLLAPSE to create large folios in the page cache via the collapse path, but the guest_memfd fault handler does not support large folios. This triggers WARN_ON_ONCE(folio_test_large(folio)) in kvm_gmem_fault_user_mapping().

For secretmem, collapse_file() tries to copy page contents through the direct map, but secretmem pages are removed from the direct map. This can result in a kernel crash:

    BUG: unable to handle page fault for address: ffff88810284d000
    RIP: 0010:memcpy_orig+0x16/0x130
    Call Trace:
     collapse_file
     hpage_collapse_scan_file
     madvise_collapse

Secretmem is not affected by the crash on upstream as the memory failure recovery handles the failed copy gracefully, but it still triggers confusing false memory failure reports:

Memory failure: 0x106d96f: recovery action for clean unevictable LRU page: Recovered

Check IS_ANON_FILE(inode) in file_thp_enabled() to deny THP for all anonymous inode files.

IS_ANON_FILE() is not available in 6.12, hence this backported version checks if the mapping is a secretmem_mapping() instead. This is sufficient for 6.12 since guest_memfd, which was also excluded from THP with the check IS_ANON_FILE(), is not available in 6.12.

---
 include/linux/huge_mm.h | 4 ++++
 1 file changed, 4 insertions(+)

index f70b048596b53..bd34b70dd0cf9 100644
@@ -7,6 +7,7 @@
 
 #include <linux/fs.h> /* only for vma_is_dax() */
 #include <linux/kobject.h>
+#include <linux/secretmem.h>
 
 vm_fault_t do_huge_pmd_anonymous_page(struct vm_fault *vmf);
 int copy_huge_pmd(struct mm_struct *dst_mm, struct mm_struct *src_mm,

@@ -262,6 +263,9 @@ static inline bool file_thp_enabled(struct vm_area_struct *vma)
 
 	inode = vma->vm_file->f_inode;
 
+	if (secretmem_mapping(inode->i_mapping))
+		return false;
+
 	return (IS_ENABLED(CONFIG_READ_ONLY_THP_FOR_FS)) &&
 	       !inode_is_open_for_write(inode) && S_ISREG(inode->i_mode);
 }

2.53.0.851.ga537e3e6e9-goog

You could have added a backport note here, like

[ Ackerley: we don't have IS_ANON_FILE() yet. As guest_memfd does
not apply yet, simply check for secretmem explicitly. ]

LGTM,

Thanks!

Cheers,

David