ceph/005: verify correct statfs behaviour with quotas
PATCH v2 1/1 ceph/005: verify correct statfs behaviour with quotas
When using a directory with 'max_bytes' quota as a base for a mount, statfs shall use that 'max_bytes' value as the total disk size. That value shall be used even when using subdirectory as base for the mount.
A bug was found where, when this subdirectory also had a 'max_files' quota, the real filesystem size would be returned instead of the parent 'max_bytes' quota value. This test case verifies this bug is fixed.
---
Finally, I've managed to come back to this test. The major changes since v1 are:
- creation of an helper for getting total mount space using 'df'
- now the test sends quota size to stdout
common/rc | 13 +++++++++++++ tests/ceph/005 | 38 ++++++++++++++++++++++++++++++++++++++ tests/ceph/005.out | 4 ++++ 3 files changed, 55 insertions(+) create mode 100755 tests/ceph/005 create mode 100644 tests/ceph/005.out
common/rc +13 -0
@@ -4254,6 +4254,19 @@ _get_available_space() echo $((avail_kb * 1024)) } +# get the total space in bytes +# +_get_total_space() +{ + if [ -z "$1" ]; then + echo "Usage: _get_total_space <mnt>" + exit 1 + fi + local total_kb; + total_kb=`$DF_PROG $1 | tail -n1 | awk '{ print $3 }'` + echo $(($total_kb * 1024)) +} + # return device size in kb _get_device_size() {
tests/ceph/005 new file +38 -0
@@ -0,0 +1,38 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) 2022 SUSE Linux Products GmbH. All Rights Reserved. +# +# FS QA Test 005 +# +# Make sure statfs reports correct total size when: +# 1. using a directory with 'max_byte' quota as base for a mount +# 2. using a subdirectory of the above directory with 'max_files' quota +# +. ./common/preamble +_begin_fstest auto quick quota + +_supported_fs ceph +_require_scratch + +_scratch_mount +mkdir -p "$SCRATCH_MNT/quota-dir/subdir" + +# set quota +quota=$((2 ** 30)) # 1G +$SETFATTR_PROG -n ceph.quota.max_bytes -v "$quota" "$SCRATCH_MNT/quota-dir" +$SETFATTR_PROG -n ceph.quota.max_files -v "$quota" "$SCRATCH_MNT/quota-dir/subdir" +_scratch_unmount + +SCRATCH_DEV="$SCRATCH_DEV/quota-dir" _scratch_mount +echo ceph quota size is $(_get_total_space "$SCRATCH_MNT") bytes +SCRATCH_DEV="$SCRATCH_DEV/quota-dir" _scratch_unmount + +SCRATCH_DEV="$SCRATCH_DEV/quota-dir/subdir" _scratch_mount +echo subdir ceph quota size is $(_get_total_space "$SCRATCH_MNT") bytes +SCRATCH_DEV="$SCRATCH_DEV/quota-dir/subdir" _scratch_unmount + +echo "Silence is golden" + +# success, all done +status=0 +exit
tests/ceph/005.out new file +4 -0
@@ -0,0 +1,4 @@ +QA output created by 005 +ceph quota size is 1073741824 bytes +subdir ceph quota size is 1073741824 bytes +Silence is golden
Quoted 70 lines · Luís Henriques
When using a directory with '
max_bytes' quota as a base for a mount, statfs shall use that 'max_bytes' value as the total disk size. That value shall be used even when using subdirectory as base for the mount.A bug was found where, when this subdirectory also had a '
max_files' quota, the real filesystem size would be returned instead of the parent 'max_bytes' quota value. This test case verifies this bug is fixed.Signed-off-by Luís Henriques <[email protected]>---
Finally, I've managed to come back to this test. The major changes since v1 are:
- creation of an helper for getting total mount space using 'df'
- now the test sends quota size to stdoutcommon/rc | 13 +++++++++++++ tests/ceph/005 | 38 ++++++++++++++++++++++++++++++++++++++ tests/ceph/005.out | 4 ++++ 3 files changed, 55 insertions(+) create mode 100755 tests/ceph/005 create mode 100644 tests/ceph/005.outcommon/rc +13
-0@@ -4254,6 +4254,19 @@ _get_available_space() echo $((avail_kb * 1024)) } +# get the total space in bytes +# +_get_total_space() +{ + if [ -z "$1" ]; then + echo "Usage: _get_total_space <mnt>" + exit 1 + fi + local total_kb; + total_kb=`$DF_PROG $1 | tail -n1 | awk '{ print $3 }'` + echo $(($total_kb * 1024)) +} + # return device size in kb _get_device_size() {tests/ceph/005 new file +18
-0@@ -0,0 +1,38 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) 2022 SUSE Linux Products GmbH. All Rights Reserved. +# +# FS QA Test 005 +# +# Make sure statfs reports correct total size when: +# 1. using a directory with 'max_byte' quota as base for a mount +# 2. using a subdirectory of the above directory with 'max_files' quota +# +. ./common/preamble +_begin_fstest auto quick quota + +_supported_fs ceph +_require_scratch + +_scratch_mount +mkdir -p "$SCRATCH_MNT/quota-dir/subdir"
tests/ceph/005 line 28
Quoted 10 lines
tests/ceph/005 new file +10
-0+ +# set quota +quota=$((2 ** 30)) # 1G +$SETFATTR_PROG -n ceph.quota.max_bytes -v "$quota" "$SCRATCH_MNT/quota-dir" +$SETFATTR_PROG -n ceph.quota.max_files -v "$quota" "$SCRATCH_MNT/quota-dir/subdir" +_scratch_unmount + +SCRATCH_DEV="$SCRATCH_DEV/quota-dir" _scratch_mount +echo ceph quota size is $(_get_total_space "$SCRATCH_MNT") bytes +SCRATCH_DEV="$SCRATCH_DEV/quota-dir" _scratch_unmount
One week past, I hope to get some reviewing points from ceph mail list, due to I am really not familiar with cephfs, especially about the SCRATCH_DEV change as above.
I tried to ask if we can use a directory under TEST_DIR to do that, but Luís said he need the "SCRATCH_DEV=$SCRATCH_DEV/xxxx" things. Is it always a good usage for cephfs?
Hope to get review from ceph forks, Thanks!
Thanks,
Zorro
Quoted 21 lines
tests/ceph/005 new file +10
-0+ +SCRATCH_DEV="$SCRATCH_DEV/quota-dir/subdir" _scratch_mount +echo subdir ceph quota size is $(_get_total_space "$SCRATCH_MNT") bytes +SCRATCH_DEV="$SCRATCH_DEV/quota-dir/subdir" _scratch_unmount + +echo "Silence is golden" + +# success, all done +status=0 +exittests/ceph/005.out new file +4
-0@@ -0,0 +1,4 @@ +QA output created by 005 +ceph quota size is 1073741824 bytes +subdir ceph quota size is 1073741824 bytes +Silence is golden
Hi Luis,
Sorry for late.
Quoted 71 lines · Luís Henriques
When using a directory with '
max_bytes' quota as a base for a mount, statfs shall use that 'max_bytes' value as the total disk size. That value shall be used even when using subdirectory as base for the mount.A bug was found where, when this subdirectory also had a '
max_files' quota, the real filesystem size would be returned instead of the parent 'max_bytes' quota value. This test case verifies this bug is fixed.Signed-off-by Luís Henriques <[email protected]>---
Finally, I've managed to come back to this test. The major changes since v1 are:
- creation of an helper for getting total mount space using 'df'
- now the test sends quota size to stdoutcommon/rc | 13 +++++++++++++ tests/ceph/005 | 38 ++++++++++++++++++++++++++++++++++++++ tests/ceph/005.out | 4 ++++ 3 files changed, 55 insertions(+)create mode 100755 tests/ceph/005
create mode 100644 tests/ceph/005.outcommon/rc +13
-0@@ -4254,6 +4254,19 @@ _get_available_space() echo $((avail_kb * 1024)) } +# get the total space in bytes +# +_get_total_space() +{ + if [ -z "$1" ]; then + echo "Usage: _get_total_space <mnt>" + exit 1 + fi + local total_kb; + total_kb=`$DF_PROG $1 | tail -n1 | awk '{ print $3 }'` + echo $(($total_kb * 1024)) +} + # return device size in kb _get_device_size() {tests/ceph/005 new file +19
-0@@ -0,0 +1,38 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) 2022 SUSE Linux Products GmbH. All Rights Reserved. +# +# FS QA Test 005 +# +# Make sure statfs reports correct total size when: +# 1. using a directory with 'max_byte' quota as base for a mount +# 2. using a subdirectory of the above directory with 'max_files' quota +# +. ./common/preamble +_begin_fstest auto quick quota + +_supported_fs ceph +_require_scratch + +_scratch_mount +mkdir -p "$SCRATCH_MNT/quota-dir/subdir" +
tests/ceph/005 line 29
Quoted 10 lines
tests/ceph/005 new file +10
-0+# set quota +quota=$((2 ** 30)) # 1G +$SETFATTR_PROG -n ceph.quota.max_bytes -v "$quota" "$SCRATCH_MNT/quota-dir" +$SETFATTR_PROG -n ceph.quota.max_files -v "$quota" "$SCRATCH_MNT/quota-dir/subdir" +_scratch_unmount + +SCRATCH_DEV="$SCRATCH_DEV/quota-dir" _scratch_mount +echo ceph quota size is $(_get_total_space "$SCRATCH_MNT") bytes +SCRATCH_DEV="$SCRATCH_DEV/quota-dir" _scratch_unmount +
For the 'SCRATCH_DEV' here, if you do:
SCRATCH_DEV="$SCRATCH_DEV/quota-dir"
twice, won't it be "${ceph_scratch_dev}/quota-dir/quota-dir" at last ?
Shouldn't it be:
SCRATCH_DEV="$TEST_DIR/quota-dir" ?
-- Xiubo
Quoted 20 lines
tests/ceph/005 new file +9
-0+SCRATCH_DEV="$SCRATCH_DEV/quota-dir/subdir" _scratch_mount +echo subdir ceph quota size is $(_get_total_space "$SCRATCH_MNT") bytes +SCRATCH_DEV="$SCRATCH_DEV/quota-dir/subdir" _scratch_unmount + +echo "Silence is golden" + +# success, all done +status=0 +exittests/ceph/005.out new file +4
-0@@ -0,0 +1,4 @@ +QA output created by 005 +ceph quota size is 1073741824 bytes +subdir ceph quota size is 1073741824 bytes +Silence is golden
Quoted 88 lines · Xiubo Li
Hi Luis,
Sorry for late.
Quoted 70 lines · Luís Henriques
When using a directory with '
max_bytes' quota as a base for a mount, statfs shall use that 'max_bytes' value as the total disk size. That value shall be used even when using subdirectory as base for the mount.A bug was found where, when this subdirectory also had a '
max_files' quota, the real filesystem size would be returned instead of the parent 'max_bytes' quota value. This test case verifies this bug is fixed.Signed-off-by Luís Henriques <[email protected]>---
Finally, I've managed to come back to this test. The major changes since v1 are:
- creation of an helper for getting total mount space using 'df'
- now the test sends quota size to stdoutcommon/rc | 13 +++++++++++++ tests/ceph/005 | 38 ++++++++++++++++++++++++++++++++++++++ tests/ceph/005.out | 4 ++++ 3 files changed, 55 insertions(+)create mode 100755 tests/ceph/005
create mode 100644 tests/ceph/005.outcommon/rc +13
-0@@ -4254,6 +4254,19 @@ _get_available_space() echo $((avail_kb * 1024)) } +# get the total space in bytes +# +_get_total_space() +{ + if [ -z "$1" ]; then + echo "Usage: _get_total_space <mnt>" + exit 1 + fi + local total_kb; + total_kb=`$DF_PROG $1 | tail -n1 | awk '{ print $3 }'` + echo $(($total_kb * 1024)) +} + # return device size in kb _get_device_size() {tests/ceph/005 new file +19
-0@@ -0,0 +1,38 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) 2022 SUSE Linux Products GmbH. All Rights Reserved. +# +# FS QA Test 005 +# +# Make sure statfs reports correct total size when: +# 1. using a directory with 'max_byte' quota as base for a mount +# 2. using a subdirectory of the above directory with 'max_files' quota +# +. ./common/preamble +_begin_fstest auto quick quota + +_supported_fs ceph +_require_scratch + +_scratch_mount +mkdir -p "$SCRATCH_MNT/quota-dir/subdir" +tests/ceph/005 line 29
Quoted 10 lines
tests/ceph/005 new file +10
-0+# set quota +quota=$((2 ** 30)) # 1G +$SETFATTR_PROG -n ceph.quota.max_bytes -v "$quota" "$SCRATCH_MNT/quota-dir" +$SETFATTR_PROG -n ceph.quota.max_files -v "$quota" "$SCRATCH_MNT/quota-dir/subdir" +_scratch_unmount + +SCRATCH_DEV="$SCRATCH_DEV/quota-dir" _scratch_mount +echo ceph quota size is $(_get_total_space "$SCRATCH_MNT") bytes +SCRATCH_DEV="$SCRATCH_DEV/quota-dir" _scratch_unmount +For the '
SCRATCH_DEV' here, if you do:
Quoted 7 lines
SCRATCH_DEV="$SCRATCH_DEV/quota-dir"
twice, won't it be "${ceph_scratch_dev}/quota-dir/quota-dir" at last ?
Shouldn't it be:
SCRATCH_DEV="$TEST_DIR/quota-dir" ?
No, actually we do really need to have $SCRATCH_DEV and not $TEST_DIR here, because we want to have SCRATCH_DEV set to something like:
<mon-ip-addr>:<port>:/quota-dir
so that we will use that directory (with quotas) as base for the mount.
Regarding the second attribution using the already modifed $SCRATCH_DEV variable, that's not really what is happening (and I had to go double-check myself, as you got me confused too :-).
So, if you do:
SCRATCH_DEV="$SCRATCH_DEV/quota-dir" _scratch_mount SCRATCH_DEV="$SCRATCH_DEV/quota-dir" _scratch_unmount
the SCRATCH_DEV value is changed *only* for the _scratch_[un]mount functions, but SCRATCH_DEV value isn't really changed after these attributions. This is probably a bashism (I'm not really sure), but you can see a similar pattern in other places (see, for example, test xfs/234).
Anyway, if you prefer, I'm fine sending v3 of this test doing something like:
SCRATCH_DEV_ORIG="$SCRATCH_DEV" SCRATCH_DEV="$SCRATCH_DEV_ORIG/quota-dir" _scratch_mount
...
Cheers,
--
Luís
Quoted 24 lines
-- Xiubo
Quoted 20 lines
tests/ceph/005 new file +9
-0+SCRATCH_DEV="$SCRATCH_DEV/quota-dir/subdir" _scratch_mount +echo subdir ceph quota size is $(_get_total_space "$SCRATCH_MNT") bytes +SCRATCH_DEV="$SCRATCH_DEV/quota-dir/subdir" _scratch_unmount + +echo "Silence is golden" + +# success, all done +status=0 +exittests/ceph/005.out new file +4
-0@@ -0,0 +1,4 @@ +QA output created by 005 +ceph quota size is 1073741824 bytes +subdir ceph quota size is 1073741824 bytes +Silence is golden
Quoted 108 lines · Luís Henriques
Quoted 87 lines · Xiubo Li
Hi Luis,
Sorry for late.
Quoted 70 lines · Luís Henriques
When using a directory with '
max_bytes' quota as a base for a mount, statfs shall use that 'max_bytes' value as the total disk size. That value shall be used even when using subdirectory as base for the mount.A bug was found where, when this subdirectory also had a '
max_files' quota, the real filesystem size would be returned instead of the parent 'max_bytes' quota value. This test case verifies this bug is fixed.Signed-off-by Luís Henriques <[email protected]>---
Finally, I've managed to come back to this test. The major changes since v1 are:- creation of an helper for getting total mount space using 'df' - now the test sends quota size to stdout common/rc | 13 +++++++++++++ tests/ceph/005 | 38 ++++++++++++++++++++++++++++++++++++++ tests/ceph/005.out | 4 ++++ 3 files changed, 55 insertions(+) create mode 100755 tests/ceph/005 create mode 100644 tests/ceph/005.outcommon/rc +13
-0@@ -4254,6 +4254,19 @@ _get_available_space() echo $((avail_kb * 1024)) } +# get the total space in bytes +# +_get_total_space() +{ + if [ -z "$1" ]; then + echo "Usage: _get_total_space <mnt>" + exit 1 + fi + local total_kb; + total_kb=`$DF_PROG $1 | tail -n1 | awk '{ print $3 }'` + echo $(($total_kb * 1024)) +} + # return device size in kb _get_device_size() {tests/ceph/005 new file +19
-0@@ -0,0 +1,38 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) 2022 SUSE Linux Products GmbH. All Rights Reserved. +# +# FS QA Test 005 +# +# Make sure statfs reports correct total size when: +# 1. using a directory with 'max_byte' quota as base for a mount +# 2. using a subdirectory of the above directory with 'max_files' quota +# +. ./common/preamble +_begin_fstest auto quick quota + +_supported_fs ceph +_require_scratch + +_scratch_mount +mkdir -p "$SCRATCH_MNT/quota-dir/subdir" +tests/ceph/005 line 29
Quoted 10 lines
tests/ceph/005 new file +10
-0+# set quota +quota=$((2 ** 30)) # 1G +$SETFATTR_PROG -n ceph.quota.max_bytes -v "$quota" "$SCRATCH_MNT/quota-dir" +$SETFATTR_PROG -n ceph.quota.max_files -v "$quota" "$SCRATCH_MNT/quota-dir/subdir" +_scratch_unmount + +SCRATCH_DEV="$SCRATCH_DEV/quota-dir" _scratch_mount +echo ceph quota size is $(_get_total_space "$SCRATCH_MNT") bytes +SCRATCH_DEV="$SCRATCH_DEV/quota-dir" _scratch_unmount +For the '
SCRATCH_DEV' here, if you do:Quoted 7 lines
SCRATCH_DEV="$SCRATCH_DEV/quota-dir"
twice, won't it be "${ceph_scratch_dev}/quota-dir/quota-dir" at last ?
Shouldn't it be:
SCRATCH_DEV="$TEST_DIR/quota-dir" ?
No, actually we do really need to have $SCRATCH_DEV and not $TEST_DIR here, because we want to have
SCRATCH_DEVset to something like:<mon-ip-addr>:<port>:/quota-dirso that we will use that directory (with quotas) as base for the mount.
Regarding the second attribution using the already modifed $SCRATCH_DEV variable, that's not really what is happening (and I had to go double-check myself, as you got me confused too :-).
So, if you do:
Quoted 8 lines
SCRATCH_DEV="$SCRATCH_DEV/quota-dir"
_scratch_mountSCRATCH_DEV="$SCRATCH_DEV/quota-dir"_scratch_unmountthe
SCRATCH_DEVvalue is changed *only* for the _scratch_[un]mount functions, butSCRATCH_DEVvalue isn't really changed after these attributions. This is probably a bashism (I'm not really sure), but you can see a similar pattern in other places (see, for example, test xfs/234).
Sorry, could you give the link about this ?
Checked the xfs/234, I didn't find any place is using the similar pattern.
This is what I see:
53 # Now restore the obfuscated one back and take a look around 54 echo "Restore metadump"
55 xfs_mdrestore $metadump_file $TEST_DIR/image
56 SCRATCH_DEV=$TEST_DIR/image _scratch_mount
57 SCRATCH_DEV=$TEST_DIR/image _scratch_unmount
58
59 echo "Check restored fs"
60 _check_generic_filesystem $metadump_file
Quoted 2 lines
Anyway, if you prefer, I'm fine sending v3 of this test doing something like:
Locally I just test this use case, it seems working as my guess.
-- Xiubo
Quoted 31 lines
SCRATCH_DEV_ORIG="$SCRATCH_DEV" SCRATCH_DEV="$SCRATCH_DEV_ORIG/quota-dir"
_scratch_mount
...Cheers,
--
LuísQuoted 22 lines
-- Xiubo
Quoted 20 lines
tests/ceph/005 new file +9
-0+SCRATCH_DEV="$SCRATCH_DEV/quota-dir/subdir" _scratch_mount +echo subdir ceph quota size is $(_get_total_space "$SCRATCH_MNT") bytes +SCRATCH_DEV="$SCRATCH_DEV/quota-dir/subdir" _scratch_unmount + +echo "Silence is golden" + +# success, all done +status=0 +exittests/ceph/005.out new file +4
-0@@ -0,0 +1,4 @@ +QA output created by 005 +ceph quota size is 1073741824 bytes +subdir ceph quota size is 1073741824 bytes +Silence is golden
On Mon, Jun 27, 2022 at 05:35:32PM +0800, Xiubo Li wrote:
<...>
Quoted 6 lines
Sorry, could you give the link about this ?
Checked the xfs/234, I didn't find any place is using the similar pattern.
This is what I see:
Quoted 9 lines
53 # Now restore the obfuscated one back and take a look around 54 echo "Restore metadump"
55xfs_mdrestore$metadump_file $TEST_DIR/image
56 SCRATCH_DEV=$TEST_DIR/image_scratch_mount
57 SCRATCH_DEV=$TEST_DIR/image_scratch_unmount
58
59 echo "Check restored fs"
60_check_generic_filesystem$metadump_file
Doh! OK, you're right, looks like I'll need to change my glasses graduation again.
Quoted 5 lines
Quoted 3 lines
Anyway, if you prefer, I'm fine sending v3 of this test doing something like:
Locally I just test this use case, it seems working as my guess.
That's odd. Here's the test I've used locally to confirm it works:
8<---------------------------------------------------------------- #!/bin/bash
MYVAR="HELLO"
myfunc()
{
echo "in function: $MYVAR"
}
MYVAR="$MYVAR WORLD" myfunc
MYVAR="$MYVAR WORLD" myfunc
myfunc
echo "$MYVAR" 8<----------------------------------------------------------------
When I run this in my laptop I see:
in function: HELLO WORLD
in function: HELLO WORLD
in function: HELLO
HELLO
Anyway, I think it's better to make this behaviour explicit in the test. I'll send out v3 shortly. Thanks Xiubo.
Cheers,
--
Luís