fs: add immutable rootfs

and the rootfs get mount id one as it always has. Before we actually mount the rootfs we create an internal tmpfs mount which has mount id zero but is never exposed anywhere. Continue that "tradition".

Fixes 7f9bfafc5f49 ("fs: use xarray for old mount id")

Cc <[email protected]>

Signed-off-by Christian Brauner <[email protected]>

---
 fs/namespace.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

fs/namespace.c +0 -0

index c58674a20cad..8b082b1de7f3 100644

Currently pivot_root() doesn't work on the real rootfs because it cannot be unmounted. Userspace has to do a recursive removal of the initramfs contents manually before continuing the boot.

Really all we want from the real rootfs is to serve as the parent mount for anything that is actually useful such as the tmpfs or ramfs for initramfs unpacking or the rootfs itself. There's no need for the real rootfs to actually be anything meaningful or useful. Add a immutable rootfs called "nullfs" that can be selected via the "nullfs_rootfs" kernel command line option.

The kernel will mount a tmpfs/ramfs on top of it, unpack the initramfs and fire up userspace which mounts the rootfs and can then just do:

  chdir(rootfs);
  pivot_root(".", ".");
  umount2(".", MNT_DETACH);

and be done with it. (Ofc, userspace can also choose to retain the initramfs contents by using something like pivot_root(".", "/initramfs") without unmounting it.)

Technically this also means that the rootfs mount in unprivileged namespaces doesn't need to become MNT_LOCKED anymore as it's guaranteed that the immutable rootfs remains permanently empty so there cannot be anything revealed by unmounting the covering mount.

In the future this will also allow us to create completely empty mount namespaces without risking to leak anything.

systemd already handles this all correctly as it tries to pivot_root() first and falls back to MS_MOVE only when that fails.

This goes back to various discussion in previous years and a LPC 2024 presentation about this very topic.

Now in vfs-7.0.nullfs.

Signed-off-by Christian Brauner <[email protected]>

---
Changes in v2:
- Rename to "nullfs".
- Update documentation.
- Link to v1: https://patch.msgid.link/[email protected]

---
Christian Brauner (4):
fs: ensure that internal tmpfs mount gets mount id zero
fs: add init_pivot_root()
fs: add immutable rootfs
docs: mention nullfs

 .../filesystems/ramfs-rootfs-initramfs.rst         |  32 +++-
 fs/Makefile                                        |   2 +-
 fs/init.c                                          |  17 ++
 fs/internal.h                                      |   1 +
 fs/mount.h                                         |   1 +
 fs/namespace.c                                     | 181 ++++++++++++++-------
 fs/nullfs.c                                        |  70 ++++++++
 include/linux/init_syscalls.h                      |   1 +
 include/uapi/linux/magic.h                         |   1 +
 init/do_mounts.c                                   |  14 ++
 init/do_mounts.h                                   |   1 +
 11 files changed, 254 insertions(+), 67 deletions(-)

---
base-commit: 8f0b4cce4481fb22653697cced8d0d04027cb1e8
change-id: 20260102-work-immutable-rootfs-b5f23e0f5a27

Quoted 64 lines · Christian Brauner

Currently pivot_root() doesn't work on the real rootfs because it cannot be unmounted. Userspace has to do a recursive removal of the initramfs contents manually before continuing the boot.

Really all we want from the real rootfs is to serve as the parent mount for anything that is actually useful such as the tmpfs or ramfs for initramfs unpacking or the rootfs itself. There's no need for the real rootfs to actually be anything meaningful or useful. Add a immutable rootfs called "nullfs" that can be selected via the "nullfs_rootfs" kernel command line option.

The kernel will mount a tmpfs/ramfs on top of it, unpack the initramfs and fire up userspace which mounts the rootfs and can then just do:

  chdir(rootfs);
  pivot_root(".", ".");
  umount2(".", MNT_DETACH);

and be done with it. (Ofc, userspace can also choose to retain the initramfs contents by using something like pivot_root(".", "/initramfs") without unmounting it.)

Technically this also means that the rootfs mount in unprivileged namespaces doesn't need to become MNT_LOCKED anymore as it's guaranteed that the immutable rootfs remains permanently empty so there cannot be anything revealed by unmounting the covering mount.

In the future this will also allow us to create completely empty mount namespaces without risking to leak anything.

systemd already handles this all correctly as it tries to pivot_root() first and falls back to MS_MOVE only when that fails.

This goes back to various discussion in previous years and a LPC 2024 presentation about this very topic.

Now in vfs-7.0.nullfs.

Signed-off-by Christian Brauner <[email protected]>

---
Changes in v2:
- Rename to "nullfs".
- Update documentation.
- Link to v1: https://patch.msgid.link/[email protected]

---
Christian Brauner (4):
fs: ensure that internal tmpfs mount gets mount id zero
fs: add init_pivot_root()
fs: add immutable rootfs
docs: mention nullfs

 .../filesystems/ramfs-rootfs-initramfs.rst         |  32 +++-
 fs/Makefile                                        |   2 +-
 fs/init.c                                          |  17 ++
 fs/internal.h                                      |   1 +
 fs/mount.h                                         |   1 +
 fs/namespace.c                                     | 181 ++++++++++++++-------
 fs/nullfs.c                                        |  70 ++++++++
 include/linux/init_syscalls.h                      |   1 +
 include/uapi/linux/magic.h                         |   1 +
 init/do_mounts.c                                   |  14 ++
 init/do_mounts.h                                   |   1 +
 11 files changed, 254 insertions(+), 67 deletions(-)

Quoted 3 lines

---
base-commit: 8f0b4cce4481fb22653697cced8d0d04027cb1e8
change-id: 20260102-work-immutable-rootfs-b5f23e0f5a27

I like the set overall. My only real gripe is the new command-line option. Won't that make distro adoption hard?

IIUC, with this new method, you're just adding a new nullfs at the bottom of the stack and then mounting the traditional ramfs as the mutable rootfs on top of it.

Would there be any harm in just always doing this (and dropping the command-line option)?

You would end up "leaking" both the nullfs and ramfs in the case of traditional userspace that was unaware of the extra mount, but that seems like it should be something we could live with.

Remove the "nullfs_rootfs" boot parameter and try to simply always use nullfs. The mutable rootfs will be mounted on top of it. Systems that don't use pivot_root() to pivot away from the real rootfs will have an additional mount stick around but that shouldn't be a problem at all. If it is we'll rever this commit.

This also simplifies the boot process and removes the need for the traditional switch_root workarounds.

Suggested-by Jeff Layton <[email protected]>

Signed-off-by Christian Brauner <[email protected]>

---
 .../filesystems/ramfs-rootfs-initramfs.rst    | 24 ++-----
 fs/namespace.c                                | 64 ++++++-------------
 init/do_mounts.c                              | 20 ++----
 init/do_mounts.h                              |  1 -
 4 files changed, 32 insertions(+), 77 deletions(-)

Documentation/filesystems/ramfs-rootfs-initramfs.rst +4 -20

index a8899f849e90..165117a721ce 100644
@@ -76,13 +76,8 @@ What is rootfs?
 ---------------
 
 Rootfs is a special instance of ramfs (or tmpfs, if that's enabled), which is
-always present in 2.6 systems.  Traditionally, you can't unmount rootfs for
-approximately the same reason you can't kill the init process; rather than
-having special code to check for and handle an empty list, it's smaller and
-simpler for the kernel to just make sure certain lists can't become empty.
-
-However, if the kernel is booted with "nullfs_rootfs", an immutable empty
-filesystem called nullfs is used as the true root, with the mutable rootfs
+always present in Linux systems.  The kernel uses an immutable empty filesystem
+called nullfs as the true root of the VFS hierarchy, with the mutable rootfs
 (tmpfs/ramfs) mounted on top of it.  This allows pivot_root() and unmounting
 of the initramfs to work normally.
 

@@ -126,25 +121,14 @@ All this differs from the old initrd in several ways:
     program.  See the switch_root utility, below.)
 
   - When switching another root device, initrd would pivot_root and then
-    umount the ramdisk.  Traditionally, initramfs is rootfs: you can neither
-    pivot_root rootfs, nor unmount it.  Instead delete everything out of
-    rootfs to free up the space (find -xdev / -exec rm '{}' ';'), overmount
-    rootfs with the new root (cd /newmount; mount --move . /; chroot .),
-    attach stdin/stdout/stderr to the new /dev/console, and exec the new init.
-
-    Since this is a remarkably persnickety process (and involves deleting
-    commands before you can run them), the klibc package introduced a helper
-    program (utils/run_init.c) to do all this for you.  Most other packages
-    (such as busybox) have named this command "switch_root".
-
-    However, if the kernel is booted with "nullfs_rootfs", pivot_root() works
+    umount the ramdisk.  With nullfs as the true root, pivot_root() works
     normally from the initramfs.  Userspace can simply do::
 
       chdir(new_root);
       pivot_root(".", ".");
       umount2(".", MNT_DETACH);
 
-    This is the preferred method when nullfs_rootfs is enabled.
+    This is the preferred method for switching root filesystems.
 
 Populating initramfs:
 ---------------------

fs/namespace.c +21 -43

index a44ebb2f1161..53d1055c1825 100644
@@ -75,17 +75,6 @@ static int __init initramfs_options_setup(char *str)
 
 __setup("initramfs_options=", initramfs_options_setup);
 
-bool nullfs_rootfs = false;
-
-static int __init nullfs_rootfs_setup(char *str)
-{
-	if (*str)
-		return 0;
-	nullfs_rootfs = true;
-	return 1;
-}
-__setup("nullfs_rootfs", nullfs_rootfs_setup);
-
 static u64 event;
 static DEFINE_XARRAY_FLAGS(mnt_id_xa, XA_FLAGS_ALLOC);
 static DEFINE_IDA(mnt_group_ida);

@@ -4593,10 +4582,9 @@ int path_pivot_root(struct path *new, struct path *old)
  * pointed to by put_old must yield the same directory as new_root. No other
  * file system may be mounted on put_old. After all, new_root is a mountpoint.
  *
- * Also, the current root cannot be on the 'rootfs' (initial ramfs) filesystem
- * unless the kernel was booted with "nullfs_rootfs". See
- * Documentation/filesystems/ramfs-rootfs-initramfs.rst for alternatives
- * in this situation.
+ * The immutable nullfs filesystem is mounted as the true root of the VFS
+ * hierarchy. The mutable rootfs (tmpfs/ramfs) is layered on top of this,
+ * allowing pivot_root() to work normally from initramfs.
  *
  * Notes:
  *  - we don't move root/cwd if they are not at the root (reason: if something

@@ -5993,49 +5981,39 @@ static void __init init_mount_tree(void)
 	struct path root;
 
 	/*
-	 * When nullfs is used, we create two mounts:
+	 * We create two mounts:
 	 *
 	 * (1) nullfs with mount id 1
 	 * (2) mutable rootfs with mount id 2
 	 *
 	 * with (2) mounted on top of (1).
 	 */
-	if (nullfs_rootfs) {
-		nullfs_mnt = vfs_kern_mount(&nullfs_fs_type, 0, "nullfs", NULL);
-		if (IS_ERR(nullfs_mnt))
-			panic("VFS: Failed to create nullfs");
-	}
+	nullfs_mnt = vfs_kern_mount(&nullfs_fs_type, 0, "nullfs", NULL);
+	if (IS_ERR(nullfs_mnt))
+		panic("VFS: Failed to create nullfs");
 
 	mnt = vfs_kern_mount(&rootfs_fs_type, 0, "rootfs", initramfs_options);
 	if (IS_ERR(mnt))
 		panic("Can't create rootfs");
 
-	if (nullfs_rootfs) {
-		VFS_WARN_ON_ONCE(real_mount(nullfs_mnt)->mnt_id != 1);
-		VFS_WARN_ON_ONCE(real_mount(mnt)->mnt_id != 2);
+	VFS_WARN_ON_ONCE(real_mount(nullfs_mnt)->mnt_id != 1);
+	VFS_WARN_ON_ONCE(real_mount(mnt)->mnt_id != 2);
 
-		/* The namespace root is the nullfs mnt. */
-		mnt_root		= real_mount(nullfs_mnt);
-		init_mnt_ns.root	= mnt_root;
+	/* The namespace root is the nullfs mnt. */
+	mnt_root		= real_mount(nullfs_mnt);
+	init_mnt_ns.root	= mnt_root;
 
-		/* Mount mutable rootfs on top of nullfs. */
-		root.mnt		= nullfs_mnt;
-		root.dentry		= nullfs_mnt->mnt_root;
+	/* Mount mutable rootfs on top of nullfs. */
+	root.mnt		= nullfs_mnt;
+	root.dentry		= nullfs_mnt->mnt_root;
 
-		LOCK_MOUNT_EXACT(mp, &root);
-		if (unlikely(IS_ERR(mp.parent)))
-			panic("VFS: Failed to mount rootfs on nullfs");
-		scoped_guard(mount_writer)
-			attach_mnt(real_mount(mnt), mp.parent, mp.mp);
+	LOCK_MOUNT_EXACT(mp, &root);
+	if (unlikely(IS_ERR(mp.parent)))
+		panic("VFS: Failed to mount rootfs on nullfs");
+	scoped_guard(mount_writer)
+		attach_mnt(real_mount(mnt), mp.parent, mp.mp);
 
-		pr_info("VFS: Finished mounting rootfs on nullfs\n");
-	} else {
-		VFS_WARN_ON_ONCE(real_mount(mnt)->mnt_id != 1);
-
-		/* The namespace root is the mutable rootfs. */
-		mnt_root		= real_mount(mnt);
-		init_mnt_ns.root	= mnt_root;
-	}
+	pr_info("VFS: Finished mounting rootfs on nullfs\n");
 
 	/*
 	 * We've dropped all locks here but that's fine. Not just are we

init/do_mounts.c +7 -13

index 675397c8a7a4..df6847bcf1f2 100644
@@ -493,21 +493,15 @@ void __init prepare_namespace(void)
 out:
 	devtmpfs_mount();
 
-	if (nullfs_rootfs) {
-		if (init_pivot_root(".", ".")) {
-			pr_err("VFS: Failed to pivot into new rootfs\n");
-			return;
-		}
-		if (init_umount(".", MNT_DETACH)) {
-			pr_err("VFS: Failed to unmount old rootfs\n");
-			return;
-		}
-		pr_info("VFS: Pivoted into new rootfs\n");
+	if (init_pivot_root(".", ".")) {
+		pr_err("VFS: Failed to pivot into new rootfs\n");
 		return;
 	}
-
-	init_mount(".", "/", NULL, MS_MOVE, NULL);
-	init_chroot(".");
+	if (init_umount(".", MNT_DETACH)) {
+		pr_err("VFS: Failed to unmount old rootfs\n");
+		return;
+	}
+	pr_info("VFS: Pivoted into new rootfs\n");
 }
 
 static bool is_tmpfs;

init/do_mounts.h +0 -0

index fbfee810aa89..6069ea3eb80d 100644

Currently pivot_root() doesn't work on the real rootfs because it cannot be unmounted. Userspace has to do a recursive removal of the initramfs contents manually before continuing the boot.

Really all we want from the real rootfs is to serve as the parent mount for anything that is actually useful such as the tmpfs or ramfs for initramfs unpacking or the rootfs itself. There's no need for the real rootfs to actually be anything meaningful or useful. Add a immutable rootfs called "nullfs" that can be selected via the "nullfs_rootfs" kernel command line option.

The kernel will mount a tmpfs/ramfs on top of it, unpack the initramfs and fire up userspace which mounts the rootfs and can then just do:

  chdir(rootfs);
  pivot_root(".", ".");
  umount2(".", MNT_DETACH);

and be done with it. (Ofc, userspace can also choose to retain the initramfs contents by using something like pivot_root(".", "/initramfs") without unmounting it.)

Technically this also means that the rootfs mount in unprivileged namespaces doesn't need to become MNT_LOCKED anymore as it's guaranteed that the immutable rootfs remains permanently empty so there cannot be anything revealed by unmounting the covering mount.

In the future this will also allow us to create completely empty mount namespaces without risking to leak anything.

systemd already handles this all correctly as it tries to pivot_root() first and falls back to MS_MOVE only when that fails.

This goes back to various discussion in previous years and a LPC 2024 presentation about this very topic.

Now in vfs-7.0.nullfs.

Signed-off-by Christian Brauner <[email protected]>

---
Changes in v2:
- Rename to "nullfs".
- Update documentation.
- Link to v1: https://patch.msgid.link/[email protected]

---
Christian Brauner (4):
fs: ensure that internal tmpfs mount gets mount id zero
fs: add init_pivot_root()
fs: add immutable rootfs
docs: mention nullfs

 .../filesystems/ramfs-rootfs-initramfs.rst         |  32 +++-
 fs/Makefile                                        |   2 +-
 fs/init.c                                          |  17 ++
 fs/internal.h                                      |   1 +
 fs/mount.h                                         |   1 +
 fs/namespace.c                                     | 181 ++++++++++++++-------
 fs/nullfs.c                                        |  70 ++++++++
 include/linux/init_syscalls.h                      |   1 +
 include/uapi/linux/magic.h                         |   1 +
 init/do_mounts.c                                   |  14 ++
 init/do_mounts.h                                   |   1 +
 11 files changed, 254 insertions(+), 67 deletions(-)

---
base-commit: 8f0b4cce4481fb22653697cced8d0d04027cb1e8
change-id: 20260102-work-immutable-rootfs-b5f23e0f5a27

Currently pivot_root() doesn't work on the real rootfs because it cannot be unmounted. Userspace has to do a recursive removal of the initramfs contents manually before continuing the boot.

Really all we want from the real rootfs is to serve as the parent mount

Currently pivot_root() doesn't work on the real rootfs because it cannot be unmounted. Userspace has to do a recursive removal of the initramfs contents manually before continuing the boot.

Christian, important! Your patchset breaks userspace! (And I tested this.)

I tested listmount behavior I'm talking about. On both vfs.all (i. e. with nullfs patches applied) and on some older vfs.git commit (without nullfs).

Quoted 1 line · Askar Safin

Christian, important! Your patchset breaks userspace! (And I tested this.)

If a bug is found in a piece of code we _calmly_ point it out and fix it.

Quoted 2 lines

I tested listmount behavior I'm talking about. On both vfs.all (i. e. with nullfs patches applied) and on some older vfs.git commit (without nullfs).

Looking at a foreign mount namespace over which the caller is privileged intentionally lists all mounts on top of the namespace root. In contrast to mountinfo which always looks at another mount namespace from the perspective of the process that is located within that mount namespace listmount() on a foreing mount namespace looks at the namespace itself and aims to list all mounts in that namespace. Since it is a new api there can be no regressions.

Currently pivot_root() doesn't work on the real rootfs because it cannot be unmounted. Userspace has to do a recursive removal of the initramfs contents manually before continuing the boot.

Add a immutable
rootfs called "nullfs"