Setting owner and permissions of subvolumes in newly-created BTRFS filesystem
I need to create a BTRFS filesystem where /home and /tmp are BTRFS subvolumes owned by root. It's easy to create the subvolumes with --subvol and --rootdir, but they wind up being owned by the user that ran mkfs.btrfs, not by root. I tried using fakeroot and it doesn't work, regardless of whether fakeroot and btrfs-progs come from Arch or Nixpkgs.
What is the best way to do this without needing root privileges? Nix builders don't have root access, and I don't know if they have access to user namespaces either.
Signature · 2 lines
Sincerely, Demi Marie Obenour (she/her/hers)
Quoted 10 lines · Demi Marie Obenour
I need to create a BTRFS filesystem where /home and /tmp are BTRFS subvolumes owned by root. It's easy to create the subvolumes with --subvol and --rootdir, but they wind up being owned by the user that ran mkfs.btrfs, not by root. I tried using fakeroot and it doesn't work, regardless of whether fakeroot and btrfs-progs come from Arch or Nixpkgs.
What is the best way to do this without needing root privileges? Nix builders don't have root access, and I don't know if they have access to user namespaces either.
Not familiar with namespace but I believe we can address it with some extra options like --pid-map and --gid-map options, so that we can map the user pid/gid to 0:0 in that case.
Thanks,
Qu
Quoted 12 lines · Qu Wenruo
Quoted 10 lines · Demi Marie Obenour
I need to create a BTRFS filesystem where /home and /tmp are BTRFS subvolumes owned by root. It's easy to create the subvolumes with --subvol and --rootdir, but they wind up being owned by the user that ran mkfs.btrfs, not by root. I tried using fakeroot and it doesn't work, regardless of whether fakeroot and btrfs-progs come from Arch or Nixpkgs.
What is the best way to do this without needing root privileges? Nix builders don't have root access, and I don't know if they have access to user namespaces either.
Quoted 6 lines
Not familiar with namespace but I believe we can address it with some extra options like --pid-map and --gid-map options, so that we can map the user pid/gid to 0:0 in that case.
Thanks,
Qu
Thank you! This would be awesome. In the meantime I worked around the issue by having systemd-tmpfiles fix up the permissions.
Signature · 2 lines
Sincerely, Demi Marie Obenour (she/her/hers)
Quoted 17 lines · Demi Marie Obenour
Quoted 16 lines · Qu Wenruo
Quoted 10 lines · Demi Marie Obenour
I need to create a BTRFS filesystem where /home and /tmp are BTRFS subvolumes owned by root. It's easy to create the subvolumes with --subvol and --rootdir, but they wind up being owned by the user that ran mkfs.btrfs, not by root. I tried using fakeroot and it doesn't work, regardless of whether fakeroot and btrfs-progs come from Arch or Nixpkgs.
What is the best way to do this without needing root privileges? Nix builders don't have root access, and I don't know if they have access to user namespaces either.
Not familiar with namespace but I believe we can address it with some extra options like --pid-map and --gid-map options, so that we can map the user pid/gid to 0:0 in that case.
Quoted 5 lines
Quoted 2 lines
Thanks,
QuThank you! This would be awesome. In the meantime I worked around the issue by having systemd-tmpfiles fix up the permissions.
Mind to share some details? I believe this will help other users, and I can add a short note into the docs.
Thanks,
Qu
mkfs.btrfs reproducibility
Quoted 20 lines · Qu Wenruo
Quoted 17 lines · Demi Marie Obenour
Quoted 16 lines · Qu Wenruo
Quoted 10 lines · Demi Marie Obenour
I need to create a BTRFS filesystem where /home and /tmp are BTRFS subvolumes owned by root. It's easy to create the subvolumes with --subvol and --rootdir, but they wind up being owned by the user that ran mkfs.btrfs, not by root. I tried using fakeroot and it doesn't work, regardless of whether fakeroot and btrfs-progs come from Arch or Nixpkgs.
What is the best way to do this without needing root privileges? Nix builders don't have root access, and I don't know if they have access to user namespaces either.
Not familiar with namespace but I believe we can address it with some extra options like --pid-map and --gid-map options, so that we can map the user pid/gid to 0:0 in that case.
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
Thanks,
QuThank you! This would be awesome. In the meantime I worked around the issue by having systemd-tmpfiles fix up the permissions.
Mind to share some details? I believe this will help other users, and I can add a short note into the docs.
I fixed the owner and permissions at startup. This is not good because it means that the image is not reproducible.
Is it possible for mkfs.btrfs to be reproducible, or should one run mkfs.btrfs at install-time instead of shipping the BTRFS image?
Signature · 2 lines
Sincerely, Demi Marie Obenour (she/her/hers)
Re: mkfs.btrfs reproducibility
Quoted 24 lines · Demi Marie Obenour
Quoted 23 lines · Qu Wenruo
Quoted 20 lines · Demi Marie Obenour
Quoted 18 lines · Qu Wenruo
Quoted 10 lines · Demi Marie Obenour
I need to create a BTRFS filesystem where /home and /tmp are BTRFS subvolumes owned by root. It's easy to create the subvolumes with --subvol and --rootdir, but they wind up being owned by the user that ran mkfs.btrfs, not by root. I tried using fakeroot and it doesn't work, regardless of whether fakeroot and btrfs-progs come from Arch or Nixpkgs.
What is the best way to do this without needing root privileges? Nix builders don't have root access, and I don't know if they have access to user namespaces either.
Not familiar with namespace but I believe we can address it with some extra options like --pid-map and --gid-map options, so that we can map the user pid/gid to 0:0 in that case.
Thanks,
Qu
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
Thank you! This would be awesome. In the meantime I worked around the issue by having systemd-tmpfiles fix up the permissions.
Mind to share some details? I believe this will help other users, and I can add a short note into the docs.
I fixed the owner and permissions at startup. This is not good because it means that the image is not reproducible.
OK, so it's not the proper fix.
I'll continue working on the new --pid-map/--gid-map solution so that the files will have the proper gid/pid set.
Thanks,
Qu
Quoted 3 lines
Is it possible for mkfs.btrfs to be reproducible, or should one run mkfs.btrfs at install-time instead of shipping the BTRFS image?
Re: mkfs.btrfs reproducibility
Quoted 31 lines · Qu Wenruo
Quoted 30 lines · Demi Marie Obenour
Quoted 28 lines · Qu Wenruo
Quoted 17 lines · Demi Marie Obenour
Quoted 16 lines · Qu Wenruo
Quoted 10 lines · Demi Marie Obenour
I need to create a BTRFS filesystem where /home and /tmp are BTRFS subvolumes owned by root. It's easy to create the subvolumes with --subvol and --rootdir, but they wind up being owned by the user that ran mkfs.btrfs, not by root. I tried using fakeroot and it doesn't work, regardless of whether fakeroot and btrfs-progs come from Arch or Nixpkgs.
What is the best way to do this without needing root privileges? Nix builders don't have root access, and I don't know if they have access to user namespaces either.
Not familiar with namespace but I believe we can address it with some extra options like --pid-map and --gid-map options, so that we can map the user pid/gid to 0:0 in that case.
Quoted 5 lines
Quoted 2 lines
Thanks,
QuThank you! This would be awesome. In the meantime I worked around the issue by having systemd-tmpfiles fix up the permissions.
Mind to share some details? I believe this will help other users, and I can add a short note into the docs.
Quoted 7 lines
Quoted 2 lines
I fixed the owner and permissions at startup. This is not good because it means that the image is not reproducible.
OK, so it's not the proper fix.
I'll continue working on the new --pid-map/--gid-map solution so that the files will have the proper gid/pid set.
Hi Qu,
I bumped into this: the issue is that mkfs.btrfs uses ntfw, which isn't instrumented by the LD_PRELOADs from fakeroot.
The kludge patch below in btrfs-progs gives me the behavior I want, and what I think Demi wanted as well.
Credit: https://github.com/NixOS/nixpkgs/issues/455331
Really though, IMHO it should be fixed in fakeroot: the Yocto clone of it called pseudo does have ntfw support, as noted in the above issue. I haven't had time to follow up on that yet but I will soon.
Thanks,
Calvin
-----8<-----
From: Calvin Owens <[email protected]>
Subject: [KLUDGE][btrfs-progs] Make the --rootdir argument work with fakeroot
--- mkfs/rootdir.c | 17 +++++++++++++++-- 1 file changed, 15 insertions(+), 2 deletions(-)
mkfs/rootdir.c +15 -2
@@ -1631,7 +1631,7 @@ out:return ret; } -static int ftw_add_inode (const char *full_path,const struct stat *st, +static int ftw_add_inode (const char *full_path,const struct stat *unused,int typeflag,struct FTW *ftwbuf) {struct btrfs_fs_info *fs_info = g_trans->fs_info;
@@ -1639,10 +1639,23 @@ static int ftw_add_inode(const char *full_path, const struct stat *st,struct btrfs_inode_item inode_item = {0 };struct inode_entry *parent;struct rootdir_subvol *rds; -const bool have_hard_links = (!S_ISDIR (st->st_mode) && st->st_nlink >1 ); +struct stat kludgebuf; +const struct stat *st; +bool have_hard_links;u64 ino;int ret; +/* +* KLUDGE: Explicitly call lstat(), so our view of the filesystem is +* through the LD_PRELOAD installed by the fakeroot command. +*/ +if (lstat (full_path, &kludgebuf)) { +error ("Kludge with second stat() call failed: %m" ); +return ret; + } + st = &kludgebuf; + have_hard_links = (!S_ISDIR (st->st_mode) && st->st_nlink >1 ); +/* The rootdir itself. */ if (unlikely (ftwbuf->level ==0 )) {u64 root_ino;
Signature · 1 line
2.47.3
Re: mkfs.btrfs reproducibility
Quoted 45 lines · Calvin Owens
Quoted 31 lines · Qu Wenruo
Quoted 30 lines · Demi Marie Obenour
Quoted 28 lines · Qu Wenruo
Quoted 17 lines · Demi Marie Obenour
Quoted 16 lines · Qu Wenruo
Quoted 10 lines · Demi Marie Obenour
I need to create a BTRFS filesystem where /home and /tmp are BTRFS subvolumes owned by root. It's easy to create the subvolumes with --subvol and --rootdir, but they wind up being owned by the user that ran mkfs.btrfs, not by root. I tried using fakeroot and it doesn't work, regardless of whether fakeroot and btrfs-progs come from Arch or Nixpkgs.
What is the best way to do this without needing root privileges? Nix builders don't have root access, and I don't know if they have access to user namespaces either.
Not familiar with namespace but I believe we can address it with some extra options like --pid-map and --gid-map options, so that we can map the user pid/gid to 0:0 in that case.
Quoted 5 lines
Quoted 2 lines
Thanks,
QuThank you! This would be awesome. In the meantime I worked around the issue by having systemd-tmpfiles fix up the permissions.
Mind to share some details? I believe this will help other users, and I can add a short note into the docs.
Quoted 7 lines
Quoted 2 lines
I fixed the owner and permissions at startup. This is not good because it means that the image is not reproducible.
OK, so it's not the proper fix.
I'll continue working on the new --pid-map/--gid-map solution so that the files will have the proper gid/pid set.
Hi Qu,
I bumped into this: the issue is that mkfs.btrfs uses ntfw, which isn't instrumented by the
LD_PRELOADsfrom fakeroot.
Quoted 8 lines
The kludge patch below in btrfs-progs gives me the behavior I want, and what I think Demi wanted as well.
Credit: https://github.com/NixOS/nixpkgs/issues/455331
Really though, IMHO it should be fixed in fakeroot: the Yocto clone of it called pseudo does have ntfw support, as noted in the above issue. I haven't had time to follow up on that yet but I will soon.
To me the LD_PRELOAD looks like just another hack.
I'd like to know how many gid/uid maps normally there are for such usages.
If it's less than a few hundred, I think I can add some gid/uid mapping options to mkfs.btrfs, so that no extra hacks are needed.
Thanks,
Qu
Quoted 51 lines
Thanks,
Calvin-----8<-----
From: Calvin Owens <[email protected]>
Subject: [KLUDGE][btrfs-progs] Make the --rootdir argument work with fakerootSigned-off-by Calvin Owens <[email protected]>--- mkfs/rootdir.c | 17 +++++++++++++++-- 1 file changed, 15 insertions(+), 2 deletions(-)mkfs/rootdir.c +15
-2@@ -1631,7 +1631,7 @@ out:return ret; } -static int ftw_add_inode (const char *full_path,const struct stat *st, +static int ftw_add_inode (const char *full_path,const struct stat *unused,int typeflag,struct FTW *ftwbuf) {struct btrfs_fs_info *fs_info = g_trans->fs_info;@@ -1639,10 +1639,23 @@ static int ftw_add_inode(const char *full_path, const struct stat *st,struct btrfs_inode_item inode_item = {0 };struct inode_entry *parent;struct rootdir_subvol *rds; -const bool have_hard_links = (!S_ISDIR (st->st_mode) && st->st_nlink >1 ); +struct stat kludgebuf; +const struct stat *st; +bool have_hard_links;u64 ino;int ret; +/* +* KLUDGE: Explicitly call lstat(), so our view of the filesystem is +* through the LD_PRELOAD installed by the fakeroot command. +*/ +if (lstat (full_path, &kludgebuf)) { +error ("Kludge with second stat() call failed: %m" ); +return ret; + } + st = &kludgebuf; + have_hard_links = (!S_ISDIR (st->st_mode) && st->st_nlink >1 ); +/* The rootdir itself. */ if (unlikely (ftwbuf->level ==0 )) {u64 root_ino;
Re: mkfs.btrfs reproducibility
Quoted 51 lines · Qu Wenruo
Quoted 50 lines · Calvin Owens
Quoted 31 lines · Qu Wenruo
Quoted 30 lines · Demi Marie Obenour
Quoted 28 lines · Qu Wenruo
Quoted 17 lines · Demi Marie Obenour
Quoted 16 lines · Qu Wenruo
Quoted 10 lines · Demi Marie Obenour
I need to create a BTRFS filesystem where /home and /tmp are BTRFS subvolumes owned by root. It's easy to create the subvolumes with --subvol and --rootdir, but they wind up being owned by the user that ran mkfs.btrfs, not by root. I tried using fakeroot and it doesn't work, regardless of whether fakeroot and btrfs-progs come from Arch or Nixpkgs.
What is the best way to do this without needing root privileges? Nix builders don't have root access, and I don't know if they have access to user namespaces either.
Not familiar with namespace but I believe we can address it with some extra options like --pid-map and --gid-map options, so that we can map the user pid/gid to 0:0 in that case.
Quoted 5 lines
Quoted 2 lines
Thanks,
QuThank you! This would be awesome. In the meantime I worked around the issue by having systemd-tmpfiles fix up the permissions.
Mind to share some details? I believe this will help other users, and I can add a short note into the docs.
Quoted 7 lines
Quoted 2 lines
I fixed the owner and permissions at startup. This is not good because it means that the image is not reproducible.
OK, so it's not the proper fix.
I'll continue working on the new --pid-map/--gid-map solution so that the files will have the proper gid/pid set.
Hi Qu,
I bumped into this: the issue is that mkfs.btrfs uses ntfw, which isn't instrumented by the
LD_PRELOADsfrom fakeroot.The kludge patch below in btrfs-progs gives me the behavior I want, and what I think Demi wanted as well.
Quoted 7 lines
Quoted 3 lines
Really though, IMHO it should be fixed in fakeroot: the Yocto clone of it called pseudo does have ntfw support, as noted in the above issue. I haven't had time to follow up on that yet but I will soon.
To me the
LD_PRELOADlooks like just another hack.I'd like to know how many gid/uid maps normally there are for such usages.
It's an entire Linux system, and the mapping isn't static. On some examples I just looked at it's 30-50 uids and 60K-80K files.
I don't really require a generalized mapping: I only need to create a btrfs filesystem with file uid/gid/mode identical to files created through fakeroot.
This is my crappy script in case it makes more sense (note the shebang):
https://github.com/jcalvinowens/kernel-testing/blob/master/scripts/make-image.sh
That works as-is with my kludgey example patch in btrfs-progs. To be clear, I'm *not* proposing that patch as a real solution.
Quoted 2 lines
If it's less than a few hundred, I think I can add some gid/uid mapping options to mkfs.btrfs, so that no extra hacks are needed.
Mapping uid/gid isn't sufficient for generic images, unfortunately: one counterexample, image creation needs to be able to arbitarily mknod and include the special files in the filesystem without running as root. I've never seen that done any other way than fakeroot/pseudo.
In Yocto, btrfs-progs is the reason pseudo had nftw support added in the first place:
https://lore.kernel.org/yocto-patches/[email protected]/T/#mfd5a2338ea6fefc5ed734c0beb85ac26806b8684
But now that I've had more coffee, I think this is arguably a bug in
glibc: shouldn't nftw respect the LD_PRELOAD interposition of lstat()? I don't understand why it can't be made to, but I need to do more homework on this...
In any case: I welcome uid/gid map support in btrfs-progs, but IMO that's a bit orthogonal to this problem.
Thanks,
Calvin