Skip to contentSkip to conversation

fstests: add idmapped mounts tests

PATCH Superseded Christian Brauner 3 participants 22 messages Mar 22, 2021 – Mar 24, 2021

v10 of 13 Superseded Latest is v13 →

lore
Share & tools
lore
#0 lore raw

v10 cover fstests: add idmapped mounts tests

From: Christian Brauner <[email protected]>

Hey everyone,

This series is available from:
https://git.kernel.org/brauner/xfstests-dev/h/idmapped_mounts
https://gitlab.com/brauner/xfstests/-/tree/idmapped_mounts
https://github.com/brauner/xfstests/tree/idmapped_mounts

/* v10 */
Reworked according to Eryu's comments.

/* v9 */
Rebased onto current master.

ubuntu@f1-vm:~/src/git/xfstests$ sudo ./check generic/631
FSTYP         -- xfs (non-debug)
PLATFORM      -- Linux/x86_64 f1-vm 5.12.0-rc3-idmapped-mounts-inode-helpers #351 SMP Sat Mar 20 10:32:48 UTC 2021
MKFS_OPTIONS  -- -f -bsize=4096 /dev/loop1
MOUNT_OPTIONS -- /dev/loop1 /mnt/scratch

generic/631 files ... 10s
Ran: generic/631
Passed all 1 tests

ubuntu@f1-vm:~/src/git/xfstests$ sudo ./check generic/632
FSTYP         -- xfs (non-debug)
PLATFORM      -- Linux/x86_64 f1-vm 5.12.0-rc3-idmapped-mounts-inode-helpers #351 SMP Sat Mar 20 10:32:48 UTC 2021
MKFS_OPTIONS  -- -f -bsize=4096 /dev/loop1
MOUNT_OPTIONS -- /dev/loop1 /mnt/scratch

generic/632 14s
Ran: generic/632
Passed all 1 tests

ubuntu@f1-vm:~/src/git/xfstests$ sudo ./check xfs/529
FSTYP         -- xfs (non-debug)
PLATFORM      -- Linux/x86_64 f1-vm 5.12.0-rc3-idmapped-mounts-inode-helpers #351 SMP Sat Mar 20 10:32:48 UTC 2021
MKFS_OPTIONS  -- -f -bsize=4096 /dev/loop1
MOUNT_OPTIONS -- /dev/loop1 /mnt/scratch

xfs/529 files ... 43s
Ran: xfs/529
Passed all 1 tests

ubuntu@f1-vm:~/src/git/xfstests$ sudo ./check xfs/530
FSTYP         -- xfs (non-debug)
PLATFORM      -- Linux/x86_64 f1-vm 5.12.0-rc3-idmapped-mounts-inode-helpers #351 SMP Sat Mar 20 10:32:48 UTC 2021
MKFS_OPTIONS  -- -f -bsize=4096 /dev/loop1
MOUNT_OPTIONS -- /dev/loop1 /mnt/scratch

xfs/530 files ... 25s
Ran: xfs/530
Passed all 1 tests

Thanks!
Christian

Christian Brauner (6):
generic/631: add test for detached mount propagation generic/632: add fstests for idmapped mounts common/rc: add _scratch_{u}mount_idmapped() helpers common/quota: move _qsetup() helper to common code xfs/529: quotas and idmapped mounts xfs/530: quotas on idmapped mounts

 .gitignore                            |    3 +
 common/quota                          |   20 +
 common/rc                             |   60 +
 configure.ac                          |    2 +
 include/builddefs.in                  |    1 +
 m4/Makefile                           |    1 +
 m4/package_libcap.m4                  |    4 +
 src/Makefile                          |    8 +-
 src/detached_mounts_propagation.c     |  189 +
 src/feature.c                         |   40 +-
 src/idmapped-mounts/Makefile          |   41 +
 src/idmapped-mounts/idmapped-mounts.c | 8761 +++++++++++++++++++++++++
 src/idmapped-mounts/missing.h         |  151 +
 src/idmapped-mounts/mount-idmapped.c  |  428 ++
 src/idmapped-mounts/utils.c           |  134 +
 src/idmapped-mounts/utils.h           |   30 +
 tests/generic/631                     |   43 +
 tests/generic/631.out                 |    2 +
 tests/generic/632                     |   42 +
 tests/generic/632.out                 |    2 +
 tests/generic/group                   |    2 +
 tests/xfs/050                         |   19 -
 tests/xfs/299                         |   19 -
 tests/xfs/529                         |  377 ++
 tests/xfs/529.out                     |  657 ++
 tests/xfs/530                         |  211 +
 tests/xfs/530.out                     |  129 +
 tests/xfs/group                       |    2 +
 28 files changed, 11335 insertions(+), 43 deletions(-)
 create mode 100644 m4/package_libcap.m4
 create mode 100644 src/detached_mounts_propagation.c
 create mode 100644 src/idmapped-mounts/Makefile
 create mode 100644 src/idmapped-mounts/idmapped-mounts.c
 create mode 100644 src/idmapped-mounts/missing.h
 create mode 100644 src/idmapped-mounts/mount-idmapped.c
 create mode 100644 src/idmapped-mounts/utils.c
 create mode 100644 src/idmapped-mounts/utils.h
 create mode 100644 tests/generic/631
 create mode 100644 tests/generic/631.out
 create mode 100644 tests/generic/632
 create mode 100644 tests/generic/632.out
 create mode 100644 tests/xfs/529
 create mode 100644 tests/xfs/529.out
 create mode 100644 tests/xfs/530
 create mode 100644 tests/xfs/530.out

base-commit: f6ddaf130d5b0817278afe441fdde52f464f321b

Signature · 1 line
2.27.0
#1 lore raw

PATCH v10 1/6 generic/631: add test for detached mount propagation

From: Christian Brauner <[email protected]>

Regression test to verify that creating a series of detached mounts, attaching them to the filesystem, and unmounting them does not trigger an integer overflow in ns->mounts causing the kernel to block any new mounts in count_mounts() and returning ENOSPC because it falsely assumes that the maximum number of mounts in the mount namespace has been reached, i.e. it thinks it can't fit the new mounts into the mount namespace anymore.

The test is written in a way that it will leave the host's mount namespace intact so we are sure to never make the host's mount namespace unuseable!

---
/* v1 - v8 */
patch not present

/* v9 */
- Christian Brauner <[email protected]>:
- Rebased on current master.

/* v10 */
- Eryu Guan <[email protected]>:
- Add missing checks whether test is supported.
- Move status=$? assignment up.

---
 .gitignore                        |   1 +
 src/Makefile                      |   3 +-
 src/detached_mounts_propagation.c | 252 ++++++++++++++++++++++++++++++
 tests/generic/631                 |  43 +++++
 tests/generic/631.out             |   2 +
 tests/generic/group               |   1 +
 6 files changed, 301 insertions(+), 1 deletion(-)
 create mode 100644 src/detached_mounts_propagation.c
 create mode 100644 tests/generic/631
 create mode 100644 tests/generic/631.out
.gitignore +1 -0
index f3bc0a4f..72bd40a0 100644
@@ -62,6 +62,7 @@
 /src/cloner
 /src/dbtest
 /src/deduperace
+/src/detached_mounts_propagation
 /src/devzero
 /src/dio-interleaved
 /src/dio-invalidate-cache
src/Makefile +2 -1
index 3d729a34..99019e6e 100644
@@ -29,7 +29,8 @@ LINUX_TARGETS = xfsctl bstat t_mtab getdevicesize preallo_rw_pattern_reader \
 	attr-list-by-handle-cursor-test listxattr dio-interleaved t_dir_type \
 	dio-invalidate-cache stat_test t_encrypted_d_revalidate \
 	attr_replace_test swapon mkswap t_attr_corruption t_open_tmpfiles \
-	fscrypt-crypt-util bulkstat_null_ocount splice-test chprojid_fail
+	fscrypt-crypt-util bulkstat_null_ocount splice-test chprojid_fail \
+	detached_mounts_propagation
 
 SUBDIRS = log-writes perf
 
src/detached_mounts_propagation.c new file +252 -0

252 lines not on this page · raw message

tests/generic/631 new file +43 -0

43 lines not on this page · raw message

tests/generic/631.out new file +2 -0
new file mode 100644
index 00000000..ce88f447
@@ -0,0 +1,2 @@
+QA output created by 631
+silence is golden
tests/generic/group +1 -0
index 7c7531d1..151f8af2 100644
@@ -633,3 +633,4 @@
 628 auto quick rw clone
 629 auto quick rw copy_range
 630 auto quick rw dedupe clone
+631 auto quick mount
Signature · 1 line
2.27.0
#2 lore raw

Re: [PATCH v10 1/6] generic/631: add test for detached mount propagation

Quoted 27 lines · Christian Brauner

From: Christian Brauner <[email protected]>

Regression test to verify that creating a series of detached mounts, attaching them to the filesystem, and unmounting them does not trigger an integer overflow in ns->mounts causing the kernel to block any new mounts in count_mounts() and returning ENOSPC because it falsely assumes that the maximum number of mounts in the mount namespace has been reached, i.e. it thinks it can't fit the new mounts into the mount namespace anymore.

The test is written in a way that it will leave the host's mount namespace intact so we are sure to never make the host's mount namespace unuseable!

---
/* v1 - v8 */
patch not present

/* v9 */
- Christian Brauner <[email protected]>:
- Rebased on current master.

Quoted 5 lines

/* v10 */
- Eryu Guan <[email protected]>:
- Add missing checks whether test is supported.
- Move status=$? assignment up.
---

Technical nit: why did you add this extra --- line?
It causes all the patch changelog to appear in the commit message. I don't think that was your intention? and I don't think it adds valuable into to git log.

Thanks,
Amir.

#3 lore raw

Re: [PATCH v10 1/6] generic/631: add test for detached mount propagation

Quoted 34 lines · Amir Goldstein
Quoted 27 lines · Christian Brauner

From: Christian Brauner <[email protected]>

Regression test to verify that creating a series of detached mounts, attaching them to the filesystem, and unmounting them does not trigger an integer overflow in ns->mounts causing the kernel to block any new mounts in count_mounts() and returning ENOSPC because it falsely assumes that the maximum number of mounts in the mount namespace has been reached, i.e. it thinks it can't fit the new mounts into the mount namespace anymore.

The test is written in a way that it will leave the host's mount namespace intact so we are sure to never make the host's mount namespace unuseable!

---
/* v1 - v8 */
patch not present

/* v9 */
- Christian Brauner <[email protected]>:
- Rebased on current master.

Quoted 5 lines

/* v10 */
- Eryu Guan <[email protected]>:
- Add missing checks whether test is supported.
- Move status=$? assignment up.
---

Quoted 4 lines

Technical nit: why did you add this extra --- line?
It causes all the patch changelog to appear in the commit message. I don't think that was your intention? and I don't think it adds valuable into to git log.

I've not done that in any of the other patches or on any other patches I ever wrote so I think this was just a copy-paste error when I updated the changelog.

Thanks for pointing this out.
Christian

#4 lore raw

Re: [PATCH v10 1/6] generic/631: add test for detached mount propagation

Quoted 36 lines · Christian Brauner
Quoted 34 lines · Amir Goldstein
Quoted 32 lines · Christian Brauner

From: Christian Brauner <[email protected]>

Regression test to verify that creating a series of detached mounts, attaching them to the filesystem, and unmounting them does not trigger an integer overflow in ns->mounts causing the kernel to block any new mounts in count_mounts() and returning ENOSPC because it falsely assumes that the maximum number of mounts in the mount namespace has been reached, i.e. it thinks it can't fit the new mounts into the mount namespace anymore.

The test is written in a way that it will leave the host's mount namespace intact so we are sure to never make the host's mount namespace unuseable!

---
/* v1 - v8 */
patch not present

/* v9 */
- Christian Brauner <[email protected]>:
- Rebased on current master.

/* v10 */
- Eryu Guan <[email protected]>:
- Add missing checks whether test is supported.
- Move status=$? assignment up.
---

Quoted 9 lines
Quoted 4 lines

Technical nit: why did you add this extra --- line?
It causes all the patch changelog to appear in the commit message. I don't think that was your intention? and I don't think it adds valuable into to git log.

I've not done that in any of the other patches or on any other patches I ever wrote so I think this was just a copy-paste error when I updated the changelog.

Please note this glitch happened in all the patches in the series...

Thanks,
Amir.

#5 lore raw

Re: [PATCH v10 1/6] generic/631: add test for detached mount propagation

Quoted 43 lines · Amir Goldstein
Quoted 41 lines · Christian Brauner
Quoted 38 lines · Amir Goldstein
Quoted 27 lines · Christian Brauner

From: Christian Brauner <[email protected]>

Regression test to verify that creating a series of detached mounts, attaching them to the filesystem, and unmounting them does not trigger an integer overflow in ns->mounts causing the kernel to block any new mounts in count_mounts() and returning ENOSPC because it falsely assumes that the maximum number of mounts in the mount namespace has been reached, i.e. it thinks it can't fit the new mounts into the mount namespace anymore.

The test is written in a way that it will leave the host's mount namespace intact so we are sure to never make the host's mount namespace unuseable!

---
/* v1 - v8 */
patch not present

/* v9 */
- Christian Brauner <[email protected]>:
- Rebased on current master.

Quoted 5 lines

/* v10 */
- Eryu Guan <[email protected]>:
- Add missing checks whether test is supported.
- Move status=$? assignment up.
---

Technical nit: why did you add this extra --- line?
It causes all the patch changelog to appear in the commit message. I don't think that was your intention? and I don't think it adds valuable into to git log.

Quoted 6 lines
Quoted 4 lines

I've not done that in any of the other patches or on any other patches I ever wrote so I think this was just a copy-paste error when I updated the changelog.

Please note this glitch happened in all the patches in the series...

Oh wait, actually looking at this again I think don't understand just yet. This is pretty common and when you apply the series git am will cut after the first
---

And thinking about it so far I've never heard anyone complain about this. But likely I just misunderstand you and I really borked something.

Fwiw, this format seems documented at
https://www.kernel.org/doc/html/latest/process/submitting-patches.html

"The --- marker line serves the essential purpose of marking for patch
handling tools where the changelog message ends.

One good use for the additional comments after the --- marker is for a diffstat, to show what files have changed, and the number of inserted and deleted lines per file. A diffstat is especially useful on bigger patches. Other comments relevant only to the moment or the maintainer, not suitable for the permanent changelog, should also go here. A good example of such comments might be patch changelogs which describe what has changed between the v1 and v2 version of the patch.

If you are going to include a diffstat after the --- marker, please use diffstat options -p 1 -w 70 so that filenames are listed from the top of the kernel source tree and don’t use too much horizontal space (easily fit in 80 columns, maybe with some indentation). (git generates appropriate diffstats by default.)"

And also at
https://www.ozlabs.org/~akpm/stuff/tpp.txt

"g) Non-changelog text:

Sometimes you want to include text in the email which isn't designed to go into the changelog in the git repository. Things like "this is already in Fred's tree" or "this is an updated version of last Friday's patch" or whatever.

You should place such text below the "^---" separator so that it is auto-removed when being placed into the revision control system."

And there are quite a few examples that do it this way just one random pick from my inbox:
https://lore.kernel.org/lkml/[email protected]

Christian

#6 lore raw

Re: [PATCH v10 1/6] generic/631: add test for detached mount propagation

Quoted 52 lines · Christian Brauner
Quoted 43 lines · Amir Goldstein
Quoted 41 lines · Christian Brauner
Quoted 38 lines · Amir Goldstein
Quoted 27 lines · Christian Brauner

From: Christian Brauner <[email protected]>

Regression test to verify that creating a series of detached mounts, attaching them to the filesystem, and unmounting them does not trigger an integer overflow in ns->mounts causing the kernel to block any new mounts in count_mounts() and returning ENOSPC because it falsely assumes that the maximum number of mounts in the mount namespace has been reached, i.e. it thinks it can't fit the new mounts into the mount namespace anymore.

The test is written in a way that it will leave the host's mount namespace intact so we are sure to never make the host's mount namespace unuseable!

---
/* v1 - v8 */
patch not present

/* v9 */
- Christian Brauner <[email protected]>:
- Rebased on current master.

Quoted 5 lines

/* v10 */
- Eryu Guan <[email protected]>:
- Add missing checks whether test is supported.
- Move status=$? assignment up.
---

Technical nit: why did you add this extra --- line?
It causes all the patch changelog to appear in the commit message. I don't think that was your intention? and I don't think it adds valuable into to git log.

Quoted 6 lines
Quoted 4 lines

I've not done that in any of the other patches or on any other patches I ever wrote so I think this was just a copy-paste error when I updated the changelog.

Please note this glitch happened in all the patches in the series...

Quoted 8 lines

Oh wait, actually looking at this again I think don't understand just yet. This is pretty common and when you apply the series git am will cut after the first
---

And thinking about it so far I've never heard anyone complain about this. But likely I just misunderstand you and I really borked something.

I see. The reason I brought this up is because I fetched your branch from
https://github.com/brauner/xfstests/ idmapped_mounts_v2
and it has this changelog visible in git log.

So I am not sure how that came to be.

Thanks,
Amir.

#7 lore raw

Re: [PATCH v10 1/6] generic/631: add test for detached mount propagation

Quoted 63 lines · Amir Goldstein
Quoted 52 lines · Christian Brauner
Quoted 43 lines · Amir Goldstein
Quoted 41 lines · Christian Brauner
Quoted 38 lines · Amir Goldstein
Quoted 27 lines · Christian Brauner

From: Christian Brauner <[email protected]>

Regression test to verify that creating a series of detached mounts, attaching them to the filesystem, and unmounting them does not trigger an integer overflow in ns->mounts causing the kernel to block any new mounts in count_mounts() and returning ENOSPC because it falsely assumes that the maximum number of mounts in the mount namespace has been reached, i.e. it thinks it can't fit the new mounts into the mount namespace anymore.

The test is written in a way that it will leave the host's mount namespace intact so we are sure to never make the host's mount namespace unuseable!

---
/* v1 - v8 */
patch not present

/* v9 */
- Christian Brauner <[email protected]>:
- Rebased on current master.

Quoted 5 lines

/* v10 */
- Eryu Guan <[email protected]>:
- Add missing checks whether test is supported.
- Move status=$? assignment up.
---

Technical nit: why did you add this extra --- line?
It causes all the patch changelog to appear in the commit message. I don't think that was your intention? and I don't think it adds valuable into to git log.

Quoted 6 lines
Quoted 4 lines

I've not done that in any of the other patches or on any other patches I ever wrote so I think this was just a copy-paste error when I updated the changelog.

Please note this glitch happened in all the patches in the series...

Quoted 8 lines

Oh wait, actually looking at this again I think don't understand just yet. This is pretty common and when you apply the series git am will cut after the first
---

And thinking about it so far I've never heard anyone complain about this. But likely I just misunderstand you and I really borked something.

Quoted 5 lines

I see. The reason I brought this up is because I fetched your branch from
https://github.com/brauner/xfstests/ idmapped_mounts_v2
and it has this changelog visible in git log.

So I am not sure how that came to be.

Oh now I understand you. That's intentional, non-final branches, i.e. versioned branches of mine or branches where I'm not the one applying the patches after pulling them via b4 still container the changelogs because I need to assume I'll cut new version from it and will amend the changelogs. This also ensures git format-patch will give retain the changelog.

But the point is will taken. If I provide a branch to pull from I should strip the changelogs. This is what I done now. You can find this series (unchanged apart from the stripped changelog) here:
always have the changelog recorded to

https://github.com/brauner/xfstests/ idmapped_mounts

similar for gitlab and kernel.org.

Thanks for pointing that out!
Christian

#8 lore raw

PATCH v10 3/6 common/rc: add _scratch_{u}mount_idmapped() helpers

In reply to Christian Brauner

From: Christian Brauner <[email protected]>

They will be used in follow-up patches for xfs quota tests but might be useful for other tests in the future.

Cc Eryu Guan <[email protected]>
Cc Christoph Hellwig <[email protected]>
Cc David Howells <[email protected]>
---
 common/rc | 35 +++++++++++++++++++++++++++++++++++
 1 file changed, 35 insertions(+)
common/rc +35 -0
index a8b375a2..351996fc 100644
@@ -342,6 +342,36 @@ _scratch_mount()
 	_try_scratch_mount $* || _fail "mount failed"
 }
 
+_scratch_mount_idmapped()
+{
+	local type="$1"
+	local id="$2"
+
+	if [ "$type" = "u" ]; then
+		# This means root will be able to create files as uid %id in
+		# the underlying filesystem by going through the idmapped mount.
+		$here/src/idmapped-mounts/mount-idmapped --map-mount u:0:$id:1 \
+							 --map-mount u:$id:0:1 \
+							 --map-mount g:0:0:1 \
+							 "$SCRATCH_MNT" "$SCRATCH_MNT" || _fail "mount-idmapped failed"
+	elif [ "$type" = "g" ]; then
+		# This means root will be able to create files as gid %id in
+		# the underlying filesystem by going through the idmapped mount.
+		$here/src/idmapped-mounts/mount-idmapped --map-mount g:0:$id:1 \
+							 --map-mount g:$id:0:1 \
+							 --map-mount u:0:0:1 \
+							 "$SCRATCH_MNT" "$SCRATCH_MNT" || _fail "mount-idmapped failed"
+	elif [ "$type" = "b" ]; then
+		# This means root will be able to create files as uid and gid
+		# %id in the underlying filesystem by going through the idmapped mount.
+		$here/src/idmapped-mounts/mount-idmapped --map-mount b:0:$id:1 \
+							 --map-mount b:$id:0:1 \
+							 "$SCRATCH_MNT" "$SCRATCH_MNT" || _fail "mount-idmapped failed"
+	else
+		_fail "usage: either \"u\" (uid), \"g\" (gid), or \"b\" (uid and gid) must be specified "
+	fi
+}
+
 _scratch_unmount()
 {
 	case "$FSTYP" in
@@ -357,6 +387,11 @@ _scratch_unmount()
 	esac
 }
 
+_scratch_umount_idmapped()
+{
+	$UMOUNT_PROG $SCRATCH_MNT
+}
+
 _scratch_remount()
 {
     local opts="$1"
Signature · 1 line
2.27.0
#9 lore raw

PATCH v10 4/6 common/quota: move _qsetup() helper to common code

In reply to Christian Brauner

From: Christian Brauner <[email protected]>

It's already used in two tests and will be used in a third.

---
 common/quota  | 20 ++++++++++++++++++++
 tests/xfs/050 | 19 -------------------
 tests/xfs/299 | 19 -------------------
 3 files changed, 20 insertions(+), 38 deletions(-)
common/quota +20 -0
index 1437d5f7..32a9a555 100644
@@ -329,5 +329,25 @@ _report_quota_inodes() {
 	repquota $1 | egrep "^($qa_user|root|nobody)" | awk '{print $1, $6, $7, $8}' | sort -r
 }
 
+# Determine which type of quota we're using
+_qsetup()
+{
+	opt=$1
+	enforce=0
+	if [ $opt = "u" -o $opt = "uno" ]; then
+		type=u
+		eval `_choose_uid`
+	elif [ $opt = "g" -o $opt = "gno" ]; then
+		type=g
+		eval `_choose_gid`
+	elif [ $opt = "p" -o $opt = "pno" ]; then
+		type=p
+		eval `_choose_prid`
+	fi
+	[ $opt = "u" -o $opt = "g" -o $opt = "p" ] && enforce=1
+
+	echo "Using type=$type id=$id" >> $seqres.full
+}
+
 # make sure this script returns success
 /bin/true
tests/xfs/050 +0 -19
index 1df97537..e7c81d0a 100755
@@ -69,25 +69,6 @@ _filter_and_check_blks()
 	' | _filter_quota_report
 }
 
-_qsetup()
-{
-	opt=$1
-	enforce=0
-	if [ $opt = "u" -o $opt = "uno" ]; then
-		type=u
-		eval `_choose_uid`
-	elif [ $opt = "g" -o $opt = "gno" ]; then
-		type=g
-		eval `_choose_gid`
-	elif [ $opt = "p" -o $opt = "pno" ]; then
-		type=p
-		eval `_choose_prid`
-	fi
-	[ $opt = "u" -o $opt = "g" -o $opt = "p" ] && enforce=1
-
-	echo "Using type=$type id=$id" >> $seqres.full
-}
-
 _exercise()
 {
 	_scratch_mkfs_xfs | _filter_mkfs 2>$tmp.mkfs
tests/xfs/299 +0 -19
index b862e67e..26b7fcfd 100755
@@ -62,25 +62,6 @@ _filter_and_check_blks()
 	' | _filter_quota_report
 }
 
-_qsetup()
-{
-	opt=$1
-	enforce=0
-	if [ $opt = "u" -o $opt = "uno" ]; then
-		type=u
-		eval `_choose_uid`
-	elif [ $opt = "g" -o $opt = "gno" ]; then
-		type=g
-		eval `_choose_gid`
-	elif [ $opt = "p" -o $opt = "pno" ]; then
-		type=p
-		eval `_choose_prid`
-	fi
-	[ $opt = "u" -o $opt = "g" -o $opt = "p" ] && enforce=1
-
-	echo "Using type=$type id=$id" >> $seqres.full
-}
-
 _exercise()
 {
 
Signature · 1 line
2.27.0
#10 lore raw

PATCH v10 5/6 xfs/529: quotas and idmapped mounts

In reply to Christian Brauner

From: Christian Brauner <[email protected]>

Test that xfs quota behave correctly on idmapped mounts. Mount a scratch device with user and group quota support enabled. Create directories "unmapped" and "idmapped". Create files in the unampped mount and verify quota behavior. Create files through the idmapped mount and verify identical behavior.

---
/* v1 - v8 */
patch not present

/* v9 */
- Christian Brauner <[email protected]>:
- Rebased on current master.

/* v10 */
- Eryu Guan <[email protected]>:
- Remove leading "_" from local helpers.
- Remove redundant "-q" option from umount -l calls.
- Use wipe_mount() in a few more places.

---
 .gitignore                           |   1 +
 src/idmapped-mounts/Makefile         |  14 +-
 src/idmapped-mounts/mount-idmapped.c | 428 +++++++++++++++++
 src/idmapped-mounts/utils.c          |   2 +-
 src/idmapped-mounts/utils.h          |   1 +
 tests/xfs/529                        | 377 +++++++++++++++
 tests/xfs/529.out                    | 657 +++++++++++++++++++++++++++
 tests/xfs/group                      |   1 +
 8 files changed, 1476 insertions(+), 5 deletions(-)
 create mode 100644 src/idmapped-mounts/mount-idmapped.c
 create mode 100644 tests/xfs/529
 create mode 100644 tests/xfs/529.out
.gitignore +1 -0
index 3229bb26..4cc9c807 100644
@@ -179,6 +179,7 @@
 /src/aio-dio-regress/aiocp
 /src/aio-dio-regress/aiodio_sparse2
 /src/idmapped-mounts/idmapped-mounts
+/src/idmapped-mounts/mount-idmapped
 /src/log-writes/replay-log
 /src/perf/*.pyc
 
src/idmapped-mounts/Makefile +10 -4
index 6a934146..40bf9164 100644
@@ -3,9 +3,10 @@
 TOPDIR = ../..
 include $(TOPDIR)/include/builddefs
 
-TARGETS = idmapped-mounts
+TARGETS = idmapped-mounts mount-idmapped
+CFILES_IDMAPPED_MOUNTS = idmapped-mounts.c utils.c
+CFILES_MOUNT_IDMAPPED = mount-idmapped.c utils.c
 
-CFILES = idmapped-mounts.c utils.c
 HFILES = missing.h utils.h
 LLDLIBS += -pthread
 LDIRT = $(TARGETS)
@@ -24,12 +25,17 @@ depend: .dep
 
 include $(BUILDRULES)
 
-$(TARGETS): $(CFILES)
+idmapped-mounts:
 	@echo "    [CC]    $@"
-	$(Q)$(LTLINK) $(CFILES) -o $@ $(CFLAGS) $(LDFLAGS) $(LDLIBS)
+	$(Q)$(LTLINK) $(CFILES_IDMAPPED_MOUNTS) -o $@ $(CFLAGS) $(LDFLAGS) $(LDLIBS)
+
+mount-idmapped:
+	@echo "    [CC]    $@"
+	$(Q)$(LTLINK) $(CFILES_MOUNT_IDMAPPED) -o $@ $(CFLAGS) $(LDFLAGS) $(LDLIBS)
 
 install:
 	$(INSTALL) -m 755 -d $(PKG_LIB_DIR)/src/idmapped-mounts
 	$(INSTALL) -m 755 $(TARGETS) $(PKG_LIB_DIR)/src/idmapped-mounts
+	$(INSTALL) -m 755 $(TARGETS) $(PKG_LIB_DIR)/src/mount-idmapped
 
 -include .dep
src/idmapped-mounts/mount-idmapped.c new file +428 -0

428 lines not on this page · raw message

src/idmapped-mounts/utils.c +1 -1
index b27ba445..977443f1 100644
@@ -88,7 +88,7 @@ pid_t do_clone(int (*fn)(void *), void *arg, int flags)
 #endif
 }
 
-static int get_userns_fd_cb(void *data)
+int get_userns_fd_cb(void *data)
 {
 	return kill(getpid(), SIGSTOP);
 }
src/idmapped-mounts/utils.h +1 -0
index 93425731..efbf3bc3 100644
@@ -20,6 +20,7 @@
 #include "missing.h"
 
 extern pid_t do_clone(int (*fn)(void *), void *arg, int flags);
+extern int get_userns_fd_cb(void *data);
 extern int get_userns_fd(unsigned long nsid, unsigned long hostid,
 			 unsigned long range);
 extern ssize_t read_nointr(int fd, void *buf, size_t count);
tests/xfs/529 new file +377 -0

377 lines not on this page · raw message

tests/xfs/529.out new file +657 -0

657 lines not on this page · raw message

tests/xfs/group +1 -0
index 288b916d..fb4599ca 100644
@@ -505,3 +505,4 @@
 526 auto quick mkfs
 527 auto quick quota
 528 auto quick rw realtime
+529 auto quick quota
Signature · 1 line
2.27.0
#11 lore raw

PATCH v10 6/6 xfs/530: quotas on idmapped mounts

In reply to Christian Brauner

From: Christian Brauner <[email protected]>

This is basically a re-implementation of xfs/050 but each file creation call is done through an idmapped mount which verifies that the semantics are identical even when the mount is idmapped.

---
/* v1 - v8 */
patch not present

/* v9 */
- Christian Brauner <[email protected]>:
- Rebased onto current master.

/* v10 */
- Eryu Guan <[email protected]>:
- Remove leading "_" from local helpers.
- Use newly introduced _scratch_{u}mount_idmapped() helpers

---
 tests/xfs/530     | 211 ++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/530.out | 129 ++++++++++++++++++++++++++++
 tests/xfs/group   |   1 +
 3 files changed, 341 insertions(+)
 create mode 100644 tests/xfs/530
 create mode 100644 tests/xfs/530.out
tests/xfs/530 new file +211 -0
new file mode 100644
index 00000000..7f2d51e8
@@ -0,0 +1,211 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0+
+#
+# Copyright (c) 2021 Christian Brauner <[email protected]>
+# All Rights Reserved.
+#
+# FS QA Test No. 530
+#
+# Exercises basic XFS quota functionality
+#       uquota, gquota, uqnoenforce, gqnoenforce
+#
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+. ./common/quota
+
+_cleanup()
+{
+	cd /
+	_scratch_unmount 2>/dev/null
+	rm -f $tmp.*
+}
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+# real QA test starts here
+_supported_fs xfs
+
+cp /dev/null $seqres.full
+chmod a+rwx $seqres.full	# arbitrary users will write here
+
+_require_scratch
+_require_xfs_quota
+_require_user fsgqa
+
+_scratch_mkfs >/dev/null 2>&1
+_scratch_mount
+bsize=$(_get_file_block_size $SCRATCH_MNT)
+_scratch_unmount
+
+bsoft=$(( 200 * $bsize ))
+bhard=$(( 1000 * $bsize ))
+isoft=4
+ihard=10
+
+# The actual point at which limit enforcement takes place for the
+# hard block limit is variable depending on filesystem blocksize,
+# and iosize.  What we want to test is that the limit is enforced
+# (ie. blksize less than limit but not unduly less - ~85% is kind)
+# nowadays we actually get much closer to the limit before EDQUOT.
+#
+_filter_and_check_blks()
+{
+	perl -npe '
+		if (/^\#'$id'\s+(\d+)/ && '$enforce') {
+			$maximum = '$bhard';
+			$minimum = '$bhard' * 85/100;
+			$used = $1 * 1024;
+			if (($used < $minimum || $used > $maximum) && '$noextsz') {
+				printf(" URK %d: %d is out of range! [%d,%d]\n",
+					'$id', $used, $minimum, $maximum);
+			}
+			s/^(\#'$id'\s+)(\d+)/\1 =OK=/g;
+		}
+	' | _filter_quota_report
+}
+
+run_tests()
+{
+	_scratch_mkfs_xfs | _filter_mkfs 2>$tmp.mkfs
+	cat $tmp.mkfs >>$seqres.full
+
+	# keep the blocksize and data size for dd later
+	. $tmp.mkfs
+
+	_qmount
+
+	# Figure out whether we're doing large allocations
+	# (bail out if they're so large they stuff the test up)
+	_test_inode_flag extsz-inherit $SCRATCH_MNT
+	noextsz=$?
+	extsize=`_test_inode_extsz $SCRATCH_MNT`
+	[ $extsize -ge 512000 ] && \
+		_notrun "Extent size hint is too large ($extsize bytes)"
+
+	_qsetup $1
+
+	echo "Using type=$type id=$id" >>$seqres.full
+
+	$XFS_QUOTA_PROG -x -c "warn -$type 65535 -d" $SCRATCH_DEV
+
+	echo
+	echo "*** report no quota settings" | tee -a $seqres.full
+	$XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \
+		-c "repquota -birnN -$type" $SCRATCH_DEV |
+		_filter_quota_report | LC_COLLATE=POSIX sort -ru
+
+	echo
+	echo "*** report initial settings" | tee -a $seqres.full
+	_scratch_mount_idmapped $type $id
+	_file_as_id $SCRATCH_MNT/initme 0 $type 1024 0
+	_scratch_umount_idmapped
+	echo "ls -l $SCRATCH_MNT" >>$seqres.full
+	ls -l $SCRATCH_MNT >>$seqres.full
+	$XFS_QUOTA_PROG -D $tmp.projects -P $temp.projid -x \
+		-c "limit -$type bsoft=${bsoft} bhard=${bhard} $id" \
+		-c "limit -$type isoft=$isoft ihard=$ihard $id" \
+		$SCRATCH_DEV
+	$XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \
+		-c "repquota -birnN -$type" $SCRATCH_DEV |
+		_filter_quota_report | LC_COLLATE=POSIX sort -ru
+
+	echo
+	echo "*** push past the soft inode limit" | tee -a $seqres.full
+	_scratch_mount_idmapped $type $id
+	_file_as_id $SCRATCH_MNT/softie1 0 $type 1024 0
+	_file_as_id $SCRATCH_MNT/softie2 0 $type 1024 0
+	_file_as_id $SCRATCH_MNT/softie3 0 $type 1024 0
+	_file_as_id $SCRATCH_MNT/softie4 0 $type 1024 0
+	_scratch_umount_idmapped
+	_qmount
+	$XFS_QUOTA_PROG -x -c "warn -i -$type 0 $id" $SCRATCH_DEV
+	$XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \
+		-c "repquota -birnN -$type" $SCRATCH_DEV |
+		_filter_quota_report | LC_COLLATE=POSIX sort -ru
+
+	echo
+	echo "*** push past the soft block limit" | tee -a $seqres.full
+	_scratch_mount_idmapped $type $id
+	_file_as_id $SCRATCH_MNT/softie 0 $type $bsize 300
+	_scratch_umount_idmapped
+	_qmount
+	$XFS_QUOTA_PROG -x -c "warn -i -$type 0 $id" \
+		-c "warn -b -$type 0 $id" $SCRATCH_DEV
+	$XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \
+		-c "repquota -birnN -$type" $SCRATCH_DEV |
+		_filter_quota_report | LC_COLLATE=POSIX sort -ru
+
+	echo
+	# Note: for quota accounting (not enforcement), EDQUOT is not expected
+	echo "*** push past the hard inode limit (expect EDQUOT)" | tee -a $seqres.full
+	for i in 1 2 3 4 5 6 7 8 9 10 11 12
+	do
+		_scratch_mount_idmapped $type $id
+		_file_as_id $SCRATCH_MNT/hard$i 0 $type 1024 0
+		_scratch_umount_idmapped
+	done
+	_qmount
+	$XFS_QUOTA_PROG -x  -c "warn -b -$type 0 $id" \
+		-c "warn -i -$type 0 $id" $SCRATCH_DEV
+	$XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \
+		-c "repquota -birnN -$type" $SCRATCH_DEV |
+		_filter_quota_report | LC_COLLATE=POSIX sort -ru
+
+	echo
+	# Note: for quota accounting (not enforcement), EDQUOT is not expected
+	echo "*** push past the hard block limit (expect EDQUOT)" | tee -a $seqres.full
+	_scratch_mount_idmapped $type $id
+	_file_as_id $SCRATCH_MNT/softie 0 $type $bsize 1200
+	_scratch_umount_idmapped
+	echo "ls -l $SCRATCH_MNT" >>$seqres.full
+	ls -l $SCRATCH_MNT >>$seqres.full
+	_qmount
+	$XFS_QUOTA_PROG -x -c "warn -b -$type 0 $id" $SCRATCH_DEV
+	$XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \
+		-c "repquota -birnN -$type" $SCRATCH_DEV |
+		_filter_and_check_blks | LC_COLLATE=POSIX sort -ru
+
+	echo
+	echo "*** unmount"
+	_scratch_unmount
+
+}
+
+cat >$tmp.projects <<EOF
+1:$SCRATCH_MNT
+EOF
+
+cat >$tmp.projid <<EOF
+root:0
+scrach:1
+EOF
+
+projid_file="$tmp.projid"
+
+echo "*** user"
+_qmount_option "uquota"
+run_tests u
+
+echo "*** group"
+_qmount_option "gquota"
+run_tests g
+
+echo "*** uqnoenforce"
+_qmount_option "uqnoenforce"
+run_tests uno
+
+echo "*** gqnoenforce"
+_qmount_option "gqnoenforce"
+run_tests gno
+
+# success, all done
+status=0
+exit
tests/xfs/530.out new file +129 -0

129 lines not on this page · raw message

tests/xfs/group +1 -0
index fb4599ca..a536d01a 100644
@@ -506,3 +506,4 @@
 527 auto quick quota
 528 auto quick rw realtime
 529 auto quick quota
+530 auto quick quota
Signature · 1 line
2.27.0
#12 lore raw
In reply to Christian Brauner
Quoted 8 lines · Christian Brauner

From: Christian Brauner <[email protected]>

Hey everyone,

This series is available from:
https://git.kernel.org/brauner/xfstests-dev/h/idmapped_mounts
https://gitlab.com/brauner/xfstests/-/tree/idmapped_mounts
https://github.com/brauner/xfstests/tree/idmapped_mounts

I sent this series from my kernel.org mail address and patch 2/6 hasn't made it through this time too. So it seems that vger is rejecting it due to its size is my guess. I'll go poing the #korg folks to ask what's going on and whether that can be handled.

Thanks!
Christian

#13 lore raw
Quoted 10 lines · Christian Brauner
Quoted 8 lines · Christian Brauner

From: Christian Brauner <[email protected]>

Hey everyone,

This series is available from:
https://git.kernel.org/brauner/xfstests-dev/h/idmapped_mounts
https://gitlab.com/brauner/xfstests/-/tree/idmapped_mounts
https://github.com/brauner/xfstests/tree/idmapped_mounts

Quoted 4 lines

I sent this series from my kernel.org mail address and patch 2/6 hasn't made it through this time too. So it seems that vger is rejecting it due to its size is my guess. I'll go poing the #korg folks to ask what's going on and whether that can be handled.

Ok, Konstantin confirmed that patch 2/6 got dropped because of it's size. Nothing to do about this now but just as an fyi.

Thanks!
Christian

#14 lore raw
Quoted 12 lines · Christian Brauner
Quoted 10 lines · Christian Brauner
Quoted 8 lines · Christian Brauner

From: Christian Brauner <[email protected]>

Hey everyone,

This series is available from:
https://git.kernel.org/brauner/xfstests-dev/h/idmapped_mounts
https://gitlab.com/brauner/xfstests/-/tree/idmapped_mounts
https://github.com/brauner/xfstests/tree/idmapped_mounts

Quoted 8 lines
Quoted 4 lines

I sent this series from my kernel.org mail address and patch 2/6 hasn't made it through this time too. So it seems that vger is rejecting it due to its size is my guess. I'll go poing the #korg folks to ask what's going on and whether that can be handled.

Ok, Konstantin confirmed that patch 2/6 got dropped because of it's size. Nothing to do about this now but just as an fyi.

Since 2/6 got dropped, I'll write a small nit here which is also relevant to the rest of the series:

tests/generic/group +1 -0
@@ -634,3 +634,4 @@
 629 auto quick rw copy_range
 630 auto quick rw dedupe clone
 631 auto quick mount
+632 auto atime attr cap io_uring mount perms quick rw unlink

This is a mouthful of test tags, but that doesn't hurt.
I would personally not bother with obscure tags like 'unlink' but whatever.

Two things I would request are:
1. Keep 'auto quick' before all other tags. There is no strong rule about this format, but that's the common practice and it makes sense IMO because -g auto and -g quick are the far more commonly used groups of tests, so it's convenient to be able to 'eye grep' those tests in
the group file.
2. Please tags all the idmapped tests with a tag 'idmapped' (or whatever) This would be used for running tests with -g idmapped for quick sanity when modifiying idmapped mounts related code

Thanks,
Amir.

#15 lore raw
Quoted 19 lines · Amir Goldstein
Quoted 17 lines · Christian Brauner
Quoted 14 lines · Christian Brauner
Quoted 8 lines · Christian Brauner

From: Christian Brauner <[email protected]>

Hey everyone,

This series is available from:
https://git.kernel.org/brauner/xfstests-dev/h/idmapped_mounts
https://gitlab.com/brauner/xfstests/-/tree/idmapped_mounts
https://github.com/brauner/xfstests/tree/idmapped_mounts

I sent this series from my kernel.org mail address and patch 2/6 hasn't made it through this time too. So it seems that vger is rejecting it due to its size is my guess. I'll go poing the #korg folks to ask what's going on and whether that can be handled.

Quoted 5 lines
Quoted 3 lines

Ok, Konstantin confirmed that patch 2/6 got dropped because of it's size. Nothing to do about this now but just as an fyi.

Since 2/6 got dropped, I'll write a small nit here which is also

You could still pull it from above. I don't think resending would retain the patch afaict until vger has been ported by Konstantin.

Quoted 13 lines

relevant to the rest of the series:

tests/generic/group +1 -0
@@ -634,3 +634,4 @@
 629 auto quick rw copy_range
 630 auto quick rw dedupe clone
 631 auto quick mount
+632 auto atime attr cap io_uring mount perms quick rw unlink

This is a mouthful of test tags, but that doesn't hurt.
I would personally not bother with obscure tags like 'unlink' but whatever.

Quoted 9 lines

Two things I would request are:
1. Keep 'auto quick' before all other tags. There is no strong rule about this format, but that's the common practice and it makes sense IMO because -g auto and -g quick are the far more commonly used groups of tests, so it's convenient to be able to 'eye grep' those tests in
the group file.
2. Please tags all the idmapped tests with a tag 'idmapped' (or whatever) This would be used for running tests with -g idmapped for quick sanity when modifiying idmapped mounts related code

Ok, I'll wait for Eryu to respond (Since I assume he'll be the one in charge of applying it?) whether he can fix this up when applying or whether he wants a resend.

Christian

#16 lore raw
Quoted 43 lines · Christian Brauner
Quoted 19 lines · Amir Goldstein
Quoted 17 lines · Christian Brauner
Quoted 14 lines · Christian Brauner
Quoted 8 lines · Christian Brauner

From: Christian Brauner <[email protected]>

Hey everyone,

This series is available from:
https://git.kernel.org/brauner/xfstests-dev/h/idmapped_mounts
https://gitlab.com/brauner/xfstests/-/tree/idmapped_mounts
https://github.com/brauner/xfstests/tree/idmapped_mounts

I sent this series from my kernel.org mail address and patch 2/6 hasn't made it through this time too. So it seems that vger is rejecting it due to its size is my guess. I'll go poing the #korg folks to ask what's going on and whether that can be handled.

Quoted 5 lines
Quoted 3 lines

Ok, Konstantin confirmed that patch 2/6 got dropped because of it's size. Nothing to do about this now but just as an fyi.

Since 2/6 got dropped, I'll write a small nit here which is also

You could still pull it from above. I don't think resending would retain the patch afaict until vger has been ported by Konstantin.

Quoted 13 lines

relevant to the rest of the series:

tests/generic/group +1 -0
@@ -634,3 +634,4 @@
 629 auto quick rw copy_range
 630 auto quick rw dedupe clone
 631 auto quick mount
+632 auto atime attr cap io_uring mount perms quick rw unlink

This is a mouthful of test tags, but that doesn't hurt.
I would personally not bother with obscure tags like 'unlink' but whatever.

Quoted 12 lines
Quoted 9 lines

Two things I would request are:
1. Keep 'auto quick' before all other tags. There is no strong rule about this format, but that's the common practice and it makes sense IMO because -g auto and -g quick are the far more commonly used groups of tests, so it's convenient to be able to 'eye grep' those tests in
the group file.
2. Please tags all the idmapped tests with a tag 'idmapped' (or whatever) This would be used for running tests with -g idmapped for quick sanity when modifiying idmapped mounts related code

Ok, I'll wait for Eryu to respond (Since I assume he'll be the one in charge of applying it?)

He will.

Meanwhile, found a bug in Makefile:

src/idmapped-mounts/Makefile +1 -1
@@ -25,11 +25,11 @@ depend: .dep
 
 include $(BUILDRULES)
 
-idmapped-mounts:
+idmapped-mounts: $(CFILES_IDMAPPED_MOUNTS)
        @echo "    [CC]    $@"
        $(Q)$(LTLINK) $(CFILES_IDMAPPED_MOUNTS) -o $@ $(CFLAGS)

$(LDFLAGS) $(LDLIBS)

-mount-idmapped:
+mount-idmapped: $(CFILES_MOUNT_IDMAPPED)

        @echo "    [CC]    $@"
        $(Q)$(LTLINK) $(CFILES_MOUNT_IDMAPPED) -o $@ $(CFLAGS)

$(LDFLAGS) $(LDLIBS)
---

And in parsing of /proc/<pid>/ns/user:

src/idmapped-mounts/mount-idmapped.c +0 -0
@@ -175,7 +175,7 @@ static int write_id_mapping(idmap_type_t map_type,

pid_t pid, const char *buf, s

        int fd = -EBADF, setgroups_fd = -EBADF;
        int fret = -1;
        int ret;
-       char path[STRLITERALLEN("/proc") + INTTYPE_TO_STRLEN(pid_t) +
+       char path[STRLITERALLEN("/proc/") + INTTYPE_TO_STRLEN(pid_t) +
                  STRLITERALLEN("/setgroups") + 1];

        if (geteuid() != 0 && map_type == ID_TYPE_GID) {
@@ -273,7 +273,7 @@ static int get_userns_fd_from_idmap(struct list *idmap)
 {
        int ret;
        pid_t pid;
-       char path_ns[STRLITERALLEN("/proc") + INTTYPE_TO_STRLEN(pid_t) +
+       char path_ns[STRLITERALLEN("/proc/") + INTTYPE_TO_STRLEN(pid_t) +
                  STRLITERALLEN("/ns/user") + 1];

        pid = do_clone(get_userns_fd_cb, NULL, CLONE_NEWUSER | CLONE_NEWNS);
@@ -364,7 +364,7 @@ int main(int argc, char *argv[])
        while ((ret = getopt_long_only(argc, argv, "", longopts,
&index)) != -1) {
                switch (ret) {
                case 'a':
-                       if (strnequal(optarg, "/proc",
STRLITERALLEN("/proc/"))) {
+                       if (strnequal(optarg, "/proc/",
STRLITERALLEN("/proc/"))) {
                                fd_userns = open(optarg, O_RDONLY | O_CLOEXEC);

---

Thanks,
Amir.

#17 lore raw
Quoted 115 lines · Amir Goldstein
Quoted 43 lines · Christian Brauner
Quoted 19 lines · Amir Goldstein
Quoted 17 lines · Christian Brauner
Quoted 14 lines · Christian Brauner
Quoted 8 lines · Christian Brauner

From: Christian Brauner <[email protected]>

Hey everyone,

This series is available from:
https://git.kernel.org/brauner/xfstests-dev/h/idmapped_mounts
https://gitlab.com/brauner/xfstests/-/tree/idmapped_mounts
https://github.com/brauner/xfstests/tree/idmapped_mounts

I sent this series from my kernel.org mail address and patch 2/6 hasn't made it through this time too. So it seems that vger is rejecting it due to its size is my guess. I'll go poing the #korg folks to ask what's going on and whether that can be handled.

Quoted 5 lines
Quoted 3 lines

Ok, Konstantin confirmed that patch 2/6 got dropped because of it's size. Nothing to do about this now but just as an fyi.

Since 2/6 got dropped, I'll write a small nit here which is also

You could still pull it from above. I don't think resending would retain the patch afaict until vger has been ported by Konstantin.

Quoted 13 lines

relevant to the rest of the series:

tests/generic/group +1 -0
@@ -634,3 +634,4 @@
 629 auto quick rw copy_range
 630 auto quick rw dedupe clone
 631 auto quick mount
+632 auto atime attr cap io_uring mount perms quick rw unlink

This is a mouthful of test tags, but that doesn't hurt.
I would personally not bother with obscure tags like 'unlink' but whatever.

Quoted 12 lines
Quoted 9 lines

Two things I would request are:
1. Keep 'auto quick' before all other tags. There is no strong rule about this format, but that's the common practice and it makes sense IMO because -g auto and -g quick are the far more commonly used groups of tests, so it's convenient to be able to 'eye grep' those tests in
the group file.
2. Please tags all the idmapped tests with a tag 'idmapped' (or whatever) This would be used for running tests with -g idmapped for quick sanity when modifiying idmapped mounts related code

Ok, I'll wait for Eryu to respond (Since I assume he'll be the one in charge of applying it?)

He will.

Meanwhile, found a bug in Makefile:

src/idmapped-mounts/Makefile +2 -2
@@ -25,11 +25,11 @@ depend: .dep
 
 include $(BUILDRULES)
 
-idmapped-mounts:
+idmapped-mounts: $(CFILES_IDMAPPED_MOUNTS)
        @echo "    [CC]    $@"
        $(Q)$(LTLINK) $(CFILES_IDMAPPED_MOUNTS) -o $@ $(CFLAGS)
 $(LDFLAGS) $(LDLIBS)
 
-mount-idmapped:
+mount-idmapped: $(CFILES_MOUNT_IDMAPPED)
        @echo "    [CC]    $@"
        $(Q)$(LTLINK) $(CFILES_MOUNT_IDMAPPED) -o $@ $(CFLAGS)

$(LDFLAGS) $(LDLIBS)
---

And in parsing of /proc/<pid>/ns/user:

src/idmapped-mounts/mount-idmapped.c +0 -0
@@ -175,7 +175,7 @@ static int write_id_mapping(idmap_type_t map_type,

pid_t pid, const char *buf, s

        int fd = -EBADF, setgroups_fd = -EBADF;
        int fret = -1;
        int ret;
src/idmapped-mounts/mount-idmapped.c +3 -3
-       char path[STRLITERALLEN("/proc") + INTTYPE_TO_STRLEN(pid_t) +
+       char path[STRLITERALLEN("/proc/") + INTTYPE_TO_STRLEN(pid_t) +
                  STRLITERALLEN("/setgroups") + 1];
 
        if (geteuid() != 0 && map_type == ID_TYPE_GID) {
@@ -273,7 +273,7 @@ static int get_userns_fd_from_idmap(struct list *idmap)
 {
        int ret;
        pid_t pid;
-       char path_ns[STRLITERALLEN("/proc") + INTTYPE_TO_STRLEN(pid_t) +
+       char path_ns[STRLITERALLEN("/proc/") + INTTYPE_TO_STRLEN(pid_t) +
                  STRLITERALLEN("/ns/user") + 1];
 
        pid = do_clone(get_userns_fd_cb, NULL, CLONE_NEWUSER | CLONE_NEWNS);
@@ -364,7 +364,7 @@ int main(int argc, char *argv[])
        while ((ret = getopt_long_only(argc, argv, "", longopts,
 &index)) != -1) {
                switch (ret) {
                case 'a':
-                       if (strnequal(optarg, "/proc",
 STRLITERALLEN("/proc/"))) {
+                       if (strnequal(optarg, "/proc/",
 STRLITERALLEN("/proc/"))) {
                                fd_userns = open(optarg, O_RDONLY | O_CLOEXEC);
Quoted 2 lines

---

And also:

@@ -402,12 +402,15 @@ int main(int argc, char *argv[])

                exit(EXIT_FAILURE);
        }

- if (!list_empty(&active_map)) {

+       if (!list_empty(&active_map) || fd_userns > 0) {
                struct mount_attr attr = {
                        .attr_set = MOUNT_ATTR_IDMAP,
                };

- attr.userns_fd = get_userns_fd_from_idmap(&active_map);
+ if (fd_userns > 0)
+ attr.userns_fd = fd_userns;
+ else

+                       attr.userns_fd = get_userns_fd_from_idmap(&active_map);
                if (attr.userns_fd < 0)
                        exit_log("%m - Failed to create user namespace\n");

---

It's a bug in the test helper program, not a bug in the test per-se because the test does not use the
--map-mount=/proc/<pid>/ns/user option.

Thanks,
Amir.

#18 lore raw
Quoted 140 lines · Amir Goldstein
Quoted 115 lines · Amir Goldstein
Quoted 43 lines · Christian Brauner
Quoted 19 lines · Amir Goldstein
Quoted 17 lines · Christian Brauner
Quoted 14 lines · Christian Brauner
Quoted 8 lines · Christian Brauner

From: Christian Brauner <[email protected]>

Hey everyone,

This series is available from:
https://git.kernel.org/brauner/xfstests-dev/h/idmapped_mounts
https://gitlab.com/brauner/xfstests/-/tree/idmapped_mounts
https://github.com/brauner/xfstests/tree/idmapped_mounts

I sent this series from my kernel.org mail address and patch 2/6 hasn't made it through this time too. So it seems that vger is rejecting it due to its size is my guess. I'll go poing the #korg folks to ask what's going on and whether that can be handled.

Quoted 5 lines
Quoted 3 lines

Ok, Konstantin confirmed that patch 2/6 got dropped because of it's size. Nothing to do about this now but just as an fyi.

Since 2/6 got dropped, I'll write a small nit here which is also

You could still pull it from above. I don't think resending would retain the patch afaict until vger has been ported by Konstantin.

Quoted 13 lines

relevant to the rest of the series:

tests/generic/group +1 -0
@@ -634,3 +634,4 @@
 629 auto quick rw copy_range
 630 auto quick rw dedupe clone
 631 auto quick mount
+632 auto atime attr cap io_uring mount perms quick rw unlink

This is a mouthful of test tags, but that doesn't hurt.
I would personally not bother with obscure tags like 'unlink' but whatever.

Quoted 12 lines
Quoted 9 lines

Two things I would request are:
1. Keep 'auto quick' before all other tags. There is no strong rule about this format, but that's the common practice and it makes sense IMO because -g auto and -g quick are the far more commonly used groups of tests, so it's convenient to be able to 'eye grep' those tests in
the group file.
2. Please tags all the idmapped tests with a tag 'idmapped' (or whatever) This would be used for running tests with -g idmapped for quick sanity when modifiying idmapped mounts related code

Ok, I'll wait for Eryu to respond (Since I assume he'll be the one in charge of applying it?)

He will.

Meanwhile, found a bug in Makefile:

src/idmapped-mounts/Makefile +2 -2
@@ -25,11 +25,11 @@ depend: .dep
 
 include $(BUILDRULES)
 
-idmapped-mounts:
+idmapped-mounts: $(CFILES_IDMAPPED_MOUNTS)
        @echo "    [CC]    $@"
        $(Q)$(LTLINK) $(CFILES_IDMAPPED_MOUNTS) -o $@ $(CFLAGS)
 $(LDFLAGS) $(LDLIBS)
 
-mount-idmapped:
+mount-idmapped: $(CFILES_MOUNT_IDMAPPED)
        @echo "    [CC]    $@"
        $(Q)$(LTLINK) $(CFILES_MOUNT_IDMAPPED) -o $@ $(CFLAGS)

$(LDFLAGS) $(LDLIBS)
---

And in parsing of /proc/<pid>/ns/user:

src/idmapped-mounts/mount-idmapped.c +0 -0
@@ -175,7 +175,7 @@ static int write_id_mapping(idmap_type_t map_type,

pid_t pid, const char *buf, s

        int fd = -EBADF, setgroups_fd = -EBADF;
        int fret = -1;
        int ret;
src/idmapped-mounts/mount-idmapped.c +3 -3
-       char path[STRLITERALLEN("/proc") + INTTYPE_TO_STRLEN(pid_t) +
+       char path[STRLITERALLEN("/proc/") + INTTYPE_TO_STRLEN(pid_t) +
                  STRLITERALLEN("/setgroups") + 1];
 
        if (geteuid() != 0 && map_type == ID_TYPE_GID) {
@@ -273,7 +273,7 @@ static int get_userns_fd_from_idmap(struct list *idmap)
 {
        int ret;
        pid_t pid;
-       char path_ns[STRLITERALLEN("/proc") + INTTYPE_TO_STRLEN(pid_t) +
+       char path_ns[STRLITERALLEN("/proc/") + INTTYPE_TO_STRLEN(pid_t) +
                  STRLITERALLEN("/ns/user") + 1];
 
        pid = do_clone(get_userns_fd_cb, NULL, CLONE_NEWUSER | CLONE_NEWNS);
@@ -364,7 +364,7 @@ int main(int argc, char *argv[])
        while ((ret = getopt_long_only(argc, argv, "", longopts,
 &index)) != -1) {
                switch (ret) {
                case 'a':
-                       if (strnequal(optarg, "/proc",
 STRLITERALLEN("/proc/"))) {
+                       if (strnequal(optarg, "/proc/",
 STRLITERALLEN("/proc/"))) {
                                fd_userns = open(optarg, O_RDONLY | O_CLOEXEC);
Quoted 2 lines

---

And also:

(quoted diff) +5 -3
@@ -402,12 +402,15 @@ int main(int argc, char *argv[])
                exit(EXIT_FAILURE);
        }
 
-       if (!list_empty(&active_map)) {
+       if (!list_empty(&active_map) || fd_userns > 0) {
                struct mount_attr attr = {
                        .attr_set = MOUNT_ATTR_IDMAP,
                };
 
-               attr.userns_fd = get_userns_fd_from_idmap(&active_map);
+               if (fd_userns > 0)
+                       attr.userns_fd = fd_userns;
+               else
+                       attr.userns_fd = get_userns_fd_from_idmap(&active_map);
                if (attr.userns_fd < 0)
                        exit_log("%m - Failed to create user namespace\n");
---
Quoted 3 lines

It's a bug in the test helper program, not a bug in the test per-se because the test does not use the
--map-mount=/proc/<pid>/ns/user option.

Thanks! Will fix and also re-order the patches.
Christian

#19 lore raw
In reply to Amir Goldstein
Quoted 78 lines · Amir Goldstein
Quoted 43 lines · Christian Brauner
Quoted 19 lines · Amir Goldstein
Quoted 17 lines · Christian Brauner
Quoted 14 lines · Christian Brauner
Quoted 8 lines · Christian Brauner

From: Christian Brauner <[email protected]>

Hey everyone,

This series is available from:
https://git.kernel.org/brauner/xfstests-dev/h/idmapped_mounts
https://gitlab.com/brauner/xfstests/-/tree/idmapped_mounts
https://github.com/brauner/xfstests/tree/idmapped_mounts

I sent this series from my kernel.org mail address and patch 2/6 hasn't made it through this time too. So it seems that vger is rejecting it due to its size is my guess. I'll go poing the #korg folks to ask what's going on and whether that can be handled.

Quoted 5 lines
Quoted 3 lines

Ok, Konstantin confirmed that patch 2/6 got dropped because of it's size. Nothing to do about this now but just as an fyi.

Since 2/6 got dropped, I'll write a small nit here which is also

You could still pull it from above. I don't think resending would retain the patch afaict until vger has been ported by Konstantin.

Quoted 13 lines

relevant to the rest of the series:

tests/generic/group +1 -0
@@ -634,3 +634,4 @@
 629 auto quick rw copy_range
 630 auto quick rw dedupe clone
 631 auto quick mount
+632 auto atime attr cap io_uring mount perms quick rw unlink

This is a mouthful of test tags, but that doesn't hurt.
I would personally not bother with obscure tags like 'unlink' but whatever.

Quoted 12 lines
Quoted 9 lines

Two things I would request are:
1. Keep 'auto quick' before all other tags. There is no strong rule about this format, but that's the common practice and it makes sense IMO because -g auto and -g quick are the far more commonly used groups of tests, so it's convenient to be able to 'eye grep' those tests in
the group file.
2. Please tags all the idmapped tests with a tag 'idmapped' (or whatever) This would be used for running tests with -g idmapped for quick sanity when modifiying idmapped mounts related code

Ok, I'll wait for Eryu to respond (Since I assume he'll be the one in charge of applying it?)

He will.

Meanwhile, found a bug in Makefile:

src/idmapped-mounts/Makefile +2 -2
@@ -25,11 +25,11 @@ depend: .dep
 
 include $(BUILDRULES)
 
-idmapped-mounts:
+idmapped-mounts: $(CFILES_IDMAPPED_MOUNTS)
        @echo "    [CC]    $@"
        $(Q)$(LTLINK) $(CFILES_IDMAPPED_MOUNTS) -o $@ $(CFLAGS)
 $(LDFLAGS) $(LDLIBS)
 
-mount-idmapped:
+mount-idmapped: $(CFILES_MOUNT_IDMAPPED)
        @echo "    [CC]    $@"
        $(Q)$(LTLINK) $(CFILES_MOUNT_IDMAPPED) -o $@ $(CFLAGS)
Quoted 1 line

$(LDFLAGS) $(LDLIBS)

Stupid question maybe but what is the reason for this change? It builds fine without those lines.

Building idmapped-mounts
[CC] idmapped-mounts
[CC] mount-idmapped

Quoted 11 lines

---

And in parsing of /proc/<pid>/ns/user:

src/idmapped-mounts/mount-idmapped.c +0 -0
@@ -175,7 +175,7 @@ static int write_id_mapping(idmap_type_t map_type,

pid_t pid, const char *buf, s

        int fd = -EBADF, setgroups_fd = -EBADF;
        int fret = -1;
        int ret;

src/idmapped-mounts/mount-idmapped.c

Quoted 3 lines
src/idmapped-mounts/mount-idmapped.c +1 -1
-       char path[STRLITERALLEN("/proc") + INTTYPE_TO_STRLEN(pid_t) +
+       char path[STRLITERALLEN("/proc/") + INTTYPE_TO_STRLEN(pid_t) +
                  STRLITERALLEN("/setgroups") + 1];

Thanks. Though it's very unlikely for the kernel to assign a pid that is at least 100 billion. Even newest systems only bump the maximum pid number to 4m and we would've thankfully caught this in the snprintf()s below.

Quoted 20 lines
        if (geteuid() != 0 && map_type == ID_TYPE_GID) {
src/idmapped-mounts/mount-idmapped.c +2 -2
@@ -273,7 +273,7 @@ static int get_userns_fd_from_idmap(struct list *idmap)
 {
        int ret;
        pid_t pid;
-       char path_ns[STRLITERALLEN("/proc") + INTTYPE_TO_STRLEN(pid_t) +
+       char path_ns[STRLITERALLEN("/proc/") + INTTYPE_TO_STRLEN(pid_t) +
                  STRLITERALLEN("/ns/user") + 1];
 
        pid = do_clone(get_userns_fd_cb, NULL, CLONE_NEWUSER | CLONE_NEWNS);
@@ -364,7 +364,7 @@ int main(int argc, char *argv[])
        while ((ret = getopt_long_only(argc, argv, "", longopts,
 &index)) != -1) {
                switch (ret) {
                case 'a':
-                       if (strnequal(optarg, "/proc",
 STRLITERALLEN("/proc/"))) {
+                       if (strnequal(optarg, "/proc/",
 STRLITERALLEN("/proc/"))) {
Quoted 1 line
                                fd_userns = open(optarg, O_RDONLY | O_CLOEXEC);

That's currently not used by the code and would've simply meant we failed.

Will fix this up and push it to the tree I linked to earlier. Thank you!
Christian

#20 lore raw
Quoted 23 lines
Quoted 18 lines

Meanwhile, found a bug in Makefile:

src/idmapped-mounts/Makefile +2 -2
@@ -25,11 +25,11 @@ depend: .dep
 
 include $(BUILDRULES)
 
-idmapped-mounts:
+idmapped-mounts: $(CFILES_IDMAPPED_MOUNTS)
        @echo "    [CC]    $@"
        $(Q)$(LTLINK) $(CFILES_IDMAPPED_MOUNTS) -o $@ $(CFLAGS)
 $(LDFLAGS) $(LDLIBS)
 
-mount-idmapped:
+mount-idmapped: $(CFILES_MOUNT_IDMAPPED)
        @echo "    [CC]    $@"
        $(Q)$(LTLINK) $(CFILES_MOUNT_IDMAPPED) -o $@ $(CFLAGS)
Quoted 1 line

$(LDFLAGS) $(LDLIBS)

Stupid question maybe but what is the reason for this change? It builds fine without those lines.

Quoted 4 lines

Building idmapped-mounts
[CC] idmapped-mounts
[CC] mount-idmapped

My mistake. You are right. It should be fine as is. I may have messed up something else while building.

You may take the fix or leave it - as you wish.

Thanks,
Amir.

#21 lore raw
Quoted 28 lines · Amir Goldstein
Quoted 23 lines
Quoted 18 lines

Meanwhile, found a bug in Makefile:

src/idmapped-mounts/Makefile +2 -2
@@ -25,11 +25,11 @@ depend: .dep
 
 include $(BUILDRULES)
 
-idmapped-mounts:
+idmapped-mounts: $(CFILES_IDMAPPED_MOUNTS)
        @echo "    [CC]    $@"
        $(Q)$(LTLINK) $(CFILES_IDMAPPED_MOUNTS) -o $@ $(CFLAGS)
 $(LDFLAGS) $(LDLIBS)
 
-mount-idmapped:
+mount-idmapped: $(CFILES_MOUNT_IDMAPPED)
        @echo "    [CC]    $@"
        $(Q)$(LTLINK) $(CFILES_MOUNT_IDMAPPED) -o $@ $(CFLAGS)
Quoted 1 line

$(LDFLAGS) $(LDLIBS)

Stupid question maybe but what is the reason for this change? It builds fine without those lines.

Quoted 4 lines

Building idmapped-mounts
[CC] idmapped-mounts
[CC] mount-idmapped

Quoted 4 lines

My mistake. You are right. It should be fine as is. I may have messed up something else while building.

You may take the fix or leave it - as you wish.

Eh, it kinda makes it more obvious to have the lines there and they don't hurt so I've added your change.

Christian