Skip to content

RFC fs,kthread: start all kthreads in nullfs

Christian Brauner linux-fsdevel lkml 4 versions – In review

Get series with b4
b4 am [email protected] change-id 20260303-work-kthread-nullfs-875a837f4198

Versions newest first

  1. v4 RFC Latest25 patches 5 repliesIn review
  2. v3 RFC26 patches 2 repliesSuperseded
  3. v2 RFC23 patches 16 repliesSuperseded
Earlier versions · 1
VersionPostedPatchesRepliesStatus
v1 RFC11 patches2 repliesSuperseded

Changes in v4 from the cover letter

- scsi: target: only override current->fs in kthread context for the
ALUA metadata write. core_alua_write_tpg_metadata() is also reached
from the configfs alua_access_state and alua_tg_pt_offline store
handlers in process context, which must resolve against the caller's
root.
- devtmpfs: drop the kthread_mntns() helper and call ksys_unshare(UNSHARE_EMPTY_MNTNS) directly now that empty mount
namespaces have landed; the unshare path already points the kthread's
root and pwd at the new namespace.
- Reword the copy_fs() usermodehelper comment ("may copy" rather than
"may use").
- Rebase onto v7.1-rc1.
- Link to v3: https://patch.msgid.link/[email protected]

Patches in v4 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/25fs: add switch_fs_struct()fs/fs_struct.c include/linux/fs_struct.h kernel/fork.c+26 -160 replies
2/25fs: notice when init abandons fs sharingfs/fs_struct.c+25 -00 replies
3/25fs: add scoped_with_init_fs()include/linux/fs_struct.h+31 -00 replies
4/25fs: add real_fs to track task's actual fs_structfs/fs_struct.c fs/proc/array.c fs/proc/base.c +5 more+26 -130 replies
5/25fs: make userspace_init_fs a dynamically-initialized pointerfs/fs_struct.c include/linux/fs_struct.h include/linux/init_task.h +1 more+59 -80 replies
6/25rnbd: use scoped_with_init_fs() for block device opendrivers/block/rnbd/rnbd-srv.c+3 -10 replies
7/25crypto: ccp: use scoped_with_init_fs() for SEV file accessdrivers/crypto/ccp/sev-dev.c+4 -80 replies
8/25scsi: target: use scoped_with_init_fs() for ALUA metadatadrivers/target/target_core_alua.c+10 -10 replies
9/25scsi: target: use scoped_with_init_fs() for APTPL metadatadrivers/target/target_core_pr.c+3 -10 replies
10/25btrfs: use scoped_with_init_fs() for update_dev_time()fs/btrfs/volumes.c+10 -10 replies
11/25coredump: use scoped_with_init_fs() for coredump path resolutionfs/coredump.c+3 -80 replies
12/25fs: use scoped_with_init_fs() for kernel_read_file_from_path_initns()fs/kernel_read_file.c+2 -70 replies
13/25ksmbd: use scoped_with_init_fs() for share path resolutionfs/smb/server/mgmt/share_config.c+3 -10 replies
14/25ksmbd: use scoped_with_init_fs() for filesystem info path lookupfs/smb/server/smb2pdu.c+3 -10 replies
15/25ksmbd: use scoped_with_init_fs() for VFS path operationsfs/smb/server/vfs.c+9 -50 replies
16/25pnfs/blocklayout: use scoped_with_init_fs() for SCSI device lookupfs/nfs/blocklayout/dev.c+11 -40 replies
17/25initramfs: use scoped_with_init_fs() for rootfs unpackinginit/initramfs.c+10 -40 replies
18/25af_unix: use scoped_with_init_fs() for coredump socket lookupnet/unix/af_unix.c+6 -110 replies
19/25fs: stop sharing fs_struct between init_task and pid 1init/main.c+6 -10 replies
20/25fs: add umh argument to struct kernel_clone_argsinclude/linux/sched/task.h kernel/fork.c kernel/umh.c+24 -80 replies
21/25devtmpfs: create private mount namespacedrivers/base/devtmpfs.c+1 -10 replies
22/25nullfs: make nullfs multi-instancefs/nullfs.c+3 -90 replies
23/25fs: start all kthreads in nullfsfs/namespace.c+10 -63 replies
24/25fs: stop rewriting kthread fs structsfs/fs_struct.c+4 -00 replies
25/25fs: stop rewriting paths for PF_EXITING | PF_DUMPCOREfs/fs_struct.c+1 -20 replies

Reviews trailers per patch

None of the 25 patches has a Reviewed-by or Acked-by.

PatchZenghui Yu
1/25
2/25
3/25
4/25
5/25
6/25
7/25
8/25
9/25
10/25
11/25
12/25
13/25
14/25
15/25
16/25
17/25
18/25
19/25
20/25
21/25
22/25
23/25T-b
24/25
25/25

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v4 thread

Cover letter [PATCH RFC v4 00/25] fs,kthread: start all kthreads in nullfs

I have not heard any big complaints. So I would like to move this into -next. But I _do not_ intend to merge anything this cycle. Given that this is a fundamental change I want this to be in -next for the rest of this cycle and the whole 7.3 cycle.

Summary:

* all kthreads are isolated in a separate SB_KERNMOUNT of nullfs. -> no lookup of anything else, no mounting on top of it, completely isolated.
* init has a separate fs_struct from all kthreads
* scoped_with_init_fs() allows a kthread to temporarily assume init's fs_struct for filesystem operations.

So this is a bit of a crazy series. When the kernel is started it roughly goes like this:

init_task
==> create pid 1 (systemd etc.)
==> pid 2 (kthreadd)

After this point all kthreads and PID 1 share the same filesystem state. That obviously already came up when we discussed pivot_root() as this allows pivot_root() to rewrite the fs_struct of all kthreads.

This rewriting is really weird and mostly done so kthread can use init's filesystem state when they would like to. But this really should be discouraged. The rewriting should also stop completely. I worked a bit to get rid of it in a more fundamental way. Is it crazy? Yes. Is it likely broken? Yes. Does it at least boot? Yes.

Instead of sharing fs_struct between kernel threads and pid 1, pid 1 get's a completely separate fs_struct. All kthreads continue sharing init_fs as before and pid 1's fs_struct is isolated from kthread's filesystem state. IOW, userspace init cannot affect kthreads filesystem state anymore and kthreads cannot affect userspace's filesystem state anymore - without explicit opt-in.

All kthreads are anchored in a kernel internal mount of nullfs that cannot be mounted on and that cannot be used to follow other mounts. It's a completely private mount that insulates kthreads.

This series makes performing mountains of filesystem work such as path lookup and file opening and so on from kthreads hard - painfully so. I think this is a benefit because it takes the idea of just offloading _security sensitive_ operations in init's filesystem state and running random binaries or opening and creating files to kthreads difficult behind the shed... And imho it should.

The only remaining kernel tasks that actually share init's filesystem state are usermodhelpers - as they execute random binaries in the root filesystem. Another concept we should really show the back of the shed.

This gives a lot stronger guarantees than what we have now. This also makes path lookup from kthreads fail by default. IOW, it won't be possible anymore to just lookup random stuff in init's filesytem state without explicitly opting in to that.

The places that need to perform lookup in init's filesystem state may use scoped_with_init_fs() which will temporarily override the caller's fs_struct with init's fs_struct.

We now also warn and notice when pid 1 simply stops sharing filesystem state with us, i.e., abandons it's userspace_init_fs.

On older kernels if PID 1 unshared its filesystem state with us the kernel simply used the stale fs_struct state implicitly pinning anything that PID 1 had last used. Even if PID 1 might've moved on to some completely different fs_struct state and might've even unmounted the old root.

This has hilarious consequences: Think continuing to dump coredump state into an implicitly pinned directory somewhere. Calling random binaries in the old rootfs via usermodehelpers.

Be aggressive about this: We simply reject operating on stale fs_struct state by reverting userspace_init_fs to nullfs. Every kworker that does lookups after this point will fail. Every usermodehelper call will fail. This is a lot stronger but I wouldn't know what it means for pid 1 to simply stop sharing its fs state with the kernel. Clearly it wanted to separate so cut all ties.

I've went through the kernel and looked at hopefully everything that does path lookup from kthreads (workqueues, ...).

TL;DR:

==== PID 1 (systemd) ====

  root@localhost:~# stat --file-system /proc/1/root
    File: "/proc/1/root"
      ID: e3cb00dd533cd3d7 Namelen: 255     Type: ext2/ext3
  root@localhost:~# cat /proc/1/mountinfo | wc -l
  30

==== PID 2 (kthreadd) ====

  root@localhost:~# stat --file-system /proc/2/root
    File: "/proc/2/root"
      ID: 200000000 Namelen: 255     Type: nullfs
  root@localhost:~# cat /proc/2/mountinfo | wc -l
  0

---
Changes in v4:
- scsi: target: only override current->fs in kthread context for the
ALUA metadata write. core_alua_write_tpg_metadata() is also reached
from the configfs alua_access_state and alua_tg_pt_offline store
handlers in process context, which must resolve against the caller's
root.
- devtmpfs: drop the kthread_mntns() helper and call ksys_unshare(UNSHARE_EMPTY_MNTNS) directly now that empty mount
namespaces have landed; the unshare path already points the kthread's
root and pwd at the new namespace.
- Reword the copy_fs() usermodehelper comment ("may copy" rather than
"may use").
- Rebase onto v7.1-rc1.
- Link to v3: https://patch.msgid.link/[email protected]

Changes in v3:
- Fix __override_init_fs() to save and return the original fs instead of
the override, so __revert_init_fs() actually restores the caller's fs.
- Replace smp_store_release() with WRITE_ONCE() in fs override/revert.
- Move userspace_init_fs wiring commit before conversion patches to fix bisectability for user-process-context callers.
- Switch all remote procfs accesses (mounts_open_common, get_task_root, proc_cwd_link, task_state umask, kcmp) to use task_struct::real_fs
instead of task_struct::fs.
- Move VFS_WARN_ON_ONCE in copy_fs() into else block so it doesn't fire
for UMH threads.
- Fix sleeping under task_lock: validate_fs_switch() now runs outside task_lock() with might_sleep() annotation.
- Add pnfs/blocklayout scoped_with_init_fs() conversion.
- Wrap security_initramfs_populated() inside scoped_with_init_fs() in
initramfs unpacking since IPE accesses current->fs->root.
- Fix stale comments: "two mounts" -> "three mounts", UMH comment, kthread_mntns() nullfs ambiguity.
- Fix commit message mismatches.
- Link to v2: https://patch.msgid.link/[email protected]

Changes in v2:
- Remove LOOKUP_IN_INIT in favor of scoped_with_init_fs().
- Link to v1: https://patch.msgid.link/[email protected]

---
Christian Brauner (25):
fs: add switch_fs_struct()
fs: notice when init abandons fs sharing
fs: add scoped_with_init_fs()
fs: add real_fs to track task's actual fs_struct
fs: make userspace_init_fs a dynamically-initialized pointer
rnbd: use scoped_with_init_fs() for block device open
crypto: ccp: use scoped_with_init_fs() for SEV file access
scsi: target: use scoped_with_init_fs() for ALUA metadata
scsi: target: use scoped_with_init_fs() for APTPL metadata
btrfs: use scoped_with_init_fs() for update_dev_time()
coredump: use scoped_with_init_fs() for coredump path resolution
fs: use scoped_with_init_fs() for kernel_read_file_from_path_initns()
ksmbd: use scoped_with_init_fs() for share path resolution
ksmbd: use scoped_with_init_fs() for filesystem info path lookup
ksmbd: use scoped_with_init_fs() for VFS path operations pnfs/blocklayout: use scoped_with_init_fs() for SCSI device lookup
initramfs: use scoped_with_init_fs() for rootfs unpacking
af_unix: use scoped_with_init_fs() for coredump socket lookup
fs: stop sharing fs_struct between init_task and pid 1
fs: add umh argument to struct kernel_clone_args
devtmpfs: create private mount namespace
nullfs: make nullfs multi-instance
fs: start all kthreads in nullfs
fs: stop rewriting kthread fs structs
fs: stop rewriting paths for PF_EXITING | PF_DUMPCORE

 drivers/base/devtmpfs.c           |   2 +-
 drivers/block/rnbd/rnbd-srv.c     |   4 +-
 drivers/crypto/ccp/sev-dev.c      |  12 ++---
 drivers/target/target_core_alua.c |  11 +++-
 drivers/target/target_core_pr.c   |   4 +-
 fs/btrfs/volumes.c                |  11 +++-
 fs/coredump.c                     |  11 ++--
 fs/fs_struct.c                    | 103 ++++++++++++++++++++++++++++++++++++--
 fs/kernel_read_file.c             |   9 +---
 fs/namespace.c                    |  16 +++---
 fs/nfs/blocklayout/dev.c          |  15 ++++--
 fs/nullfs.c                       |  12 ++---
 fs/proc/array.c                   |   4 +-
 fs/proc/base.c                    |   8 +--
 fs/proc_namespace.c               |   4 +-
 fs/smb/server/mgmt/share_config.c |   4 +-
 fs/smb/server/smb2pdu.c           |   4 +-
 fs/smb/server/vfs.c               |  14 ++++--
 include/linux/fs_struct.h         |  34 +++++++++++++
 include/linux/init_task.h         |   1 +
 include/linux/sched.h             |   1 +
 include/linux/sched/task.h        |   1 +
 init/init_task.c                  |   1 +
 init/initramfs.c                  |  14 ++++--
 init/main.c                       |  10 +++-
 kernel/fork.c                     |  53 ++++++++++++--------
 kernel/kcmp.c                     |   2 +-
 kernel/umh.c                      |   6 +--
 net/unix/af_unix.c                |  17 +++----
 29 files changed, 282 insertions(+), 106 deletions(-)

---
base-commit: 254f49634ee16a731174d2ae34bc50bd5f45e731
change-id: 20260303-work-kthread-nullfs-875a837f4198