Skip to contentSkip to conversation

mm: memfd_luo hotfixes

PATCH Applied Pratyush Yadav 4 participants 11 messages Jan 22

Subsystems (3)
lore
Share & tools
lore

Pratyush Yadav

author maintainer
#0 lore raw

cover mm: memfd_luo hotfixes

From: "Pratyush Yadav (Google)" <[email protected]>

Hi,

This series contains a couple of fixes for memfd preservation using LUO. Patch 1 is a preparatory patch for patch 2. Patch 2 sets up the right flags on the file and makes sure security hooks are called. Patch 3 fixes a small memory leak in the error path.

Andrew, it would be great if we can land these in v6.19, especially patches 1 and 2 since they fix a usability and a security problem. Patch 3 can wait, but then it is pretty simple too. Anyway, since it is pretty late in the cycle, I'll leave it to your judgement.

Regards,
Pratyush Yadav

Pratyush Yadav (Google) (3):
memfd: export alloc_file()
mm: memfd_luo: use memfd_alloc_file() instead of shmem_file_setup()
mm: memfd_luo: restore and free memfd_luo_ser on failure

 include/linux/memfd.h |  6 ++++++
 mm/memfd.c            |  4 ++--
 mm/memfd_luo.c        | 10 ++++++----
 3 files changed, 14 insertions(+), 6 deletions(-)

base-commit: 1ac8a55c76e78fc4fc406fd7bd872d0127d04feb

Signature · 1 line
2.52.0.457.g6b5491de43-goog
author maintainer
#1 lore raw

PATCH 1/3 memfd: export alloc_file()

From: "Pratyush Yadav (Google)" <[email protected]>

The Live Update Orchestrator's (LUO) memfd preservation works by preserving all the folios of a memfd, re-creating an empty memfd on the next boot, and then inserting back the preserved folios.

Currently it creates the file by directly calling shmem_file_setup(). This leaves out other work done by alloc_file() like setting up the file mode, flags, or calling the security hooks.

Export alloc_file() to let memfd_luo use it. Rename it to memfd_alloc_file() since it is no longer private and thus needs a subsystem prefix.

---
 include/linux/memfd.h | 6 ++++++
 mm/memfd.c            | 4 ++--
 2 files changed, 8 insertions(+), 2 deletions(-)
include/linux/memfd.h +6 -0
index cc74de3dbcfe..c328a7b356d0 100644
@@ -17,6 +17,7 @@ struct folio *memfd_alloc_folio(struct file *memfd, pgoff_t idx);
  * to by vm_flags_ptr.
  */
 int memfd_check_seals_mmap(struct file *file, vm_flags_t *vm_flags_ptr);
+struct file *memfd_alloc_file(const char *name, unsigned int flags);
 #else
 static inline long memfd_fcntl(struct file *f, unsigned int c, unsigned int a)
 {
@@ -31,6 +32,11 @@ static inline int memfd_check_seals_mmap(struct file *file,
 {
 	return 0;
 }
+
+static inline struct file *memfd_alloc_file(const char *name, unsigned int flags)
+{
+	return ERR_PTR(-EINVAL);
+}
 #endif
 
 #endif /* __LINUX_MEMFD_H */
mm/memfd.c +2 -2
index ab5312aff14b..f032c6052926 100644
@@ -456,7 +456,7 @@ static char *alloc_name(const char __user *uname)
 	return ERR_PTR(error);
 }
 
-static struct file *alloc_file(const char *name, unsigned int flags)
+struct file *memfd_alloc_file(const char *name, unsigned int flags)
 {
 	unsigned int *file_seals;
 	struct file *file;
@@ -520,5 +520,5 @@ SYSCALL_DEFINE2(memfd_create,
 		return PTR_ERR(name);
 
 	fd_flags = (flags & MFD_CLOEXEC) ? O_CLOEXEC : 0;
-	return FD_ADD(fd_flags, alloc_file(name, flags));
+	return FD_ADD(fd_flags, memfd_alloc_file(name, flags));
 }
Signature · 1 line
2.52.0.457.g6b5491de43-goog
maintainer Reviewed-by on v1 1/3
#2 lore raw

Re: [PATCH 1/3] memfd: export alloc_file()

Quoted 10 lines · Pratyush Yadav

From: "Pratyush Yadav (Google)" <[email protected]>

The Live Update Orchestrator's (LUO) memfd preservation works by preserving all the folios of a memfd, re-creating an empty memfd on the next boot, and then inserting back the preserved folios.

Currently it creates the file by directly calling shmem_file_setup(). This leaves out other work done by alloc_file() like setting up the file mode, flags, or calling the security hooks.

Quoted 5 lines

Export alloc_file() to let memfd_luo use it. Rename it to memfd_alloc_file() since it is no longer private and thus needs a subsystem prefix.

Signature · 2 lines
Sincerely yours,
Mike.
maintainer Reviewed-by on v1 1/3
#3 lore raw

Re: [PATCH 1/3] memfd: export alloc_file()

In reply to Pratyush Yadav
Quoted 43 lines
---
 include/linux/memfd.h | 6 ++++++
 mm/memfd.c            | 4 ++--
 2 files changed, 8 insertions(+), 2 deletions(-)
include/linux/memfd.h +6 -0
index cc74de3dbcfe..c328a7b356d0 100644
@@ -17,6 +17,7 @@ struct folio *memfd_alloc_folio(struct file *memfd, pgoff_t idx);
  * to by vm_flags_ptr.
  */
 int memfd_check_seals_mmap(struct file *file, vm_flags_t *vm_flags_ptr);
+struct file *memfd_alloc_file(const char *name, unsigned int flags);
 #else
 static inline long memfd_fcntl(struct file *f, unsigned int c, unsigned int a)
 {
@@ -31,6 +32,11 @@ static inline int memfd_check_seals_mmap(struct file *file,
 {
        return 0;
 }
+
+static inline struct file *memfd_alloc_file(const char *name, unsigned int flags)
+{
+       return ERR_PTR(-EINVAL);
+}
 #endif
 
 #endif /* __LINUX_MEMFD_H */
mm/memfd.c +1 -1
index ab5312aff14b..f032c6052926 100644
@@ -456,7 +456,7 @@ static char *alloc_name(const char __user *uname)
        return ERR_PTR(error);
 }
 
-static struct file *alloc_file(const char *name, unsigned int flags)
+struct file *memfd_alloc_file(const char *name, unsigned int flags)
 {
        unsigned int *file_seals;
        struct file *file;

mm/memfd.c SYSCALL_DEFINE2(memfd_create, · line 524

Quoted 6 lines
mm/memfd.c +1 -1
@@ -520,5 +520,5 @@ SYSCALL_DEFINE2(memfd_create,
                return PTR_ERR(name);
 
        fd_flags = (flags & MFD_CLOEXEC) ? O_CLOEXEC : 0;
-       return FD_ADD(fd_flags, alloc_file(name, flags));
+       return FD_ADD(fd_flags, memfd_alloc_file(name, flags));
 }
author maintainer
#4 lore raw

PATCH 2/3 mm: memfd_luo: use memfd_alloc_file() instead of shmem_file_setup()

In reply to Pratyush Yadav

From: "Pratyush Yadav (Google)" <[email protected]>

When restoring a memfd, the file is created using shmem_file_setup(). While memfd creation also calls this function to get the file, it also does other things:

1. The O_LARGEFILE flag is set on the file. If this is not done, writes on the memfd exceeding 2 GiB fail.

2. FMODE_LSEEK, FMODE_PREAD, and FMODE_PWRITE are set on the file. This makes sure the file is seekable and can be used with pread() and pwrite().

3. Initializes the security field for the inode and makes sure that inode creation is permitted by the security module.

Currently, none of those things are done. This means writes above 2 GiB fail, pread(), and pwrite() fail, and so on. lseek() happens to work because file_init_path() sets it because shmem defines fop->llseek.

Fix this by using memfd_alloc_file() to get the file to make sure the initialization sequence for normal and preserved memfd is the same.

Fixes b3749f174d68 ("mm: memfd_luo: allow preserving memfd")
---
 mm/memfd_luo.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
mm/memfd_luo.c +2 -2
index 4f6ba63b4310..01a72e4d3ef6 100644
@@ -78,6 +78,7 @@
 #include <linux/liveupdate.h>
 #include <linux/shmem_fs.h>
 #include <linux/vmalloc.h>
+#include <linux/memfd.h>
 #include "internal.h"
 
 static int memfd_luo_preserve_folios(struct file *file,
@@ -443,8 +444,7 @@ static int memfd_luo_retrieve(struct liveupdate_file_op_args *args)
 	if (!ser)
 		return -EINVAL;
 
-	file = shmem_file_setup("", 0, VM_NORESERVE);
-
+	file = memfd_alloc_file("", 0);
 	if (IS_ERR(file)) {
 		pr_err("failed to setup file: %pe\n", file);
 		return PTR_ERR(file);
Signature · 1 line
2.52.0.457.g6b5491de43-goog
maintainer Reviewed-by on v1 2/3
#5 lore raw

Re: [PATCH 2/3] mm: memfd_luo: use memfd_alloc_file() instead of shmem_file_setup()

Quoted 20 lines · Pratyush Yadav

From: "Pratyush Yadav (Google)" <[email protected]>

When restoring a memfd, the file is created using shmem_file_setup(). While memfd creation also calls this function to get the file, it also does other things:

1. The O_LARGEFILE flag is set on the file. If this is not done, writes on the memfd exceeding 2 GiB fail.

2. FMODE_LSEEK, FMODE_PREAD, and FMODE_PWRITE are set on the file. This makes sure the file is seekable and can be used with pread() and pwrite().

3. Initializes the security field for the inode and makes sure that inode creation is permitted by the security module.

Currently, none of those things are done. This means writes above 2 GiB fail, pread(), and pwrite() fail, and so on. lseek() happens to work because file_init_path() sets it because shmem defines fop->llseek.

Quoted 5 lines

Fix this by using memfd_alloc_file() to get the file to make sure the initialization sequence for normal and preserved memfd is the same.

Fixes b3749f174d68 ("mm: memfd_luo: allow preserving memfd")
Quoted 29 lines
---
 mm/memfd_luo.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
mm/memfd_luo.c +2 -2
index 4f6ba63b4310..01a72e4d3ef6 100644
@@ -78,6 +78,7 @@
 #include <linux/liveupdate.h>
 #include <linux/shmem_fs.h>
 #include <linux/vmalloc.h>
+#include <linux/memfd.h>
 #include "internal.h"
 
 static int memfd_luo_preserve_folios(struct file *file,
@@ -443,8 +444,7 @@ static int memfd_luo_retrieve(struct liveupdate_file_op_args *args)
 	if (!ser)
 		return -EINVAL;
 
-	file = shmem_file_setup("", 0, VM_NORESERVE);
-
+	file = memfd_alloc_file("", 0);
 	if (IS_ERR(file)) {
 		pr_err("failed to setup file: %pe\n", file);
 		return PTR_ERR(file);
Signature · 1 line
2.52.0.457.g6b5491de43-goog
Signature · 2 lines
Sincerely yours,
Mike.
maintainer Reviewed-by on v1 2/3
#6 lore raw

Re: [PATCH 2/3] mm: memfd_luo: use memfd_alloc_file() instead of shmem_file_setup()

In reply to Pratyush Yadav
Quoted 43 lines · Pratyush Yadav

From: "Pratyush Yadav (Google)" <[email protected]>

When restoring a memfd, the file is created using shmem_file_setup(). While memfd creation also calls this function to get the file, it also does other things:

1. The O_LARGEFILE flag is set on the file. If this is not done, writes on the memfd exceeding 2 GiB fail.

2. FMODE_LSEEK, FMODE_PREAD, and FMODE_PWRITE are set on the file. This makes sure the file is seekable and can be used with pread() and pwrite().

3. Initializes the security field for the inode and makes sure that inode creation is permitted by the security module.

Currently, none of those things are done. This means writes above 2 GiB fail, pread(), and pwrite() fail, and so on. lseek() happens to work because file_init_path() sets it because shmem defines fop->llseek.

Fix this by using memfd_alloc_file() to get the file to make sure the initialization sequence for normal and preserved memfd is the same.

Fixes b3749f174d68 ("mm: memfd_luo: allow preserving memfd")
---
 mm/memfd_luo.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
mm/memfd_luo.c +1 -0
index 4f6ba63b4310..01a72e4d3ef6 100644
@@ -78,6 +78,7 @@
 #include <linux/liveupdate.h>
 #include <linux/shmem_fs.h>
 #include <linux/vmalloc.h>
+#include <linux/memfd.h>
 #include "internal.h"
 
 static int memfd_luo_preserve_folios(struct file *file,

mm/memfd_luo.c static int memfd_luo_retrieve(struct liveupdate_file_op_args *args) · line 447

Quoted 6 lines
mm/memfd_luo.c +1 -2
@@ -443,8 +444,7 @@ static int memfd_luo_retrieve(struct liveupdate_file_op_args *args)
        if (!ser)
                return -EINVAL;
 
-       file = shmem_file_setup("", 0, VM_NORESERVE);
-
+       file = memfd_alloc_file("", 0);
author maintainer
#7 lore raw

PATCH 3/3 mm: memfd_luo: restore and free memfd_luo_ser on failure

In reply to Pratyush Yadav

From: "Pratyush Yadav (Google)" <[email protected]>

memfd_luo_ser has the serialization metadata. It is of no use once restoration fails. Free it on failure.

Fixes b3749f174d68 ("mm: memfd_luo: allow preserving memfd")
---
 mm/memfd_luo.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
mm/memfd_luo.c +4 -2
index 01a72e4d3ef6..a34fccc23b6a 100644
@@ -447,7 +447,8 @@ static int memfd_luo_retrieve(struct liveupdate_file_op_args *args)
 	file = memfd_alloc_file("", 0);
 	if (IS_ERR(file)) {
 		pr_err("failed to setup file: %pe\n", file);
-		return PTR_ERR(file);
+		err = PTR_ERR(file);
+		goto free_ser;
 	}
 
 	vfs_setpos(file, ser->pos, MAX_LFS_FILESIZE);
@@ -473,7 +474,8 @@ static int memfd_luo_retrieve(struct liveupdate_file_op_args *args)
 
 put_file:
 	fput(file);
-
+free_ser:
+	kho_restore_free(ser);
 	return err;
 }
 
Signature · 1 line
2.52.0.457.g6b5491de43-goog
maintainer Reviewed-by on v1 3/3
#8 lore raw

Re: [PATCH 3/3] mm: memfd_luo: restore and free memfd_luo_ser on failure

Quoted 7 lines · Pratyush Yadav

From: "Pratyush Yadav (Google)" <[email protected]>

memfd_luo_ser has the serialization metadata. It is of no use once restoration fails. Free it on failure.

Fixes b3749f174d68 ("mm: memfd_luo: allow preserving memfd")
Signature · 2 lines
Sincerely yours,
Mike.
maintainer Reviewed-by on v1 3/3
#9 lore raw

Re: [PATCH 3/3] mm: memfd_luo: restore and free memfd_luo_ser on failure

In reply to Pratyush Yadav
Quoted 19 lines
---
 mm/memfd_luo.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
mm/memfd_luo.c +2 -1
index 01a72e4d3ef6..a34fccc23b6a 100644
@@ -447,7 +447,8 @@ static int memfd_luo_retrieve(struct liveupdate_file_op_args *args)
        file = memfd_alloc_file("", 0);
        if (IS_ERR(file)) {
                pr_err("failed to setup file: %pe\n", file);
-               return PTR_ERR(file);
+               err = PTR_ERR(file);
+               goto free_ser;
        }
 
        vfs_setpos(file, ser->pos, MAX_LFS_FILESIZE);

mm/memfd_luo.c static int memfd_luo_retrieve(struct liveupdate_file_op_args *args) · line 478

Quoted 6 lines
mm/memfd_luo.c +2 -1
@@ -473,7 +474,8 @@ static int memfd_luo_retrieve(struct liveupdate_file_op_args *args)
 
 put_file:
        fput(file);
-
+free_ser:
+       kho_restore_free(ser);
maintainer
#10 lore raw
In reply to Pratyush Yadav
Quoted 5 lines · On Thu, 22 Jan 2026 16:18:38 +0100 Pratyush Yadav <[email protected]> wrote:

From: "Pratyush Yadav (Google)" <[email protected]>

Hi,

This series contains a couple of fixes for memfd preservation using LUO.

Good stuff.

Quoted 3 lines

Andrew, it would be great if we can land these in v6.19, especially patches 1 and 2 since they fix a usability and a security problem. Patch 3 can wait, but then it is pretty simple too.

Absolutely. Patches which fix material which was added in the current cycle get the red carpet treatment.