more lbs test fixes
v3 cover more lbs test fixes
From: Pankaj Raghav <[email protected]>
These are some of the tests that failed when we were working on LBS patches [1].
generic/436 and xfs/008 are adapted to work with bs > ps systems.
xfs/161 is an upstream failure, and it happens even on bs <= ps systems.
Changes since v2:
- Addes RVB tags
- Removed pgsize fallback in xfs/008 (Ritesh)
Changes since v1:
- Corrected copy pasta error in generic/436
- xfs/161 removed hardcoded and incorporated ritesh's comments
[1] https://lore.kernel.org/linux-xfs/[email protected]/
Pankaj Raghav (3):
xfs/161: adapt the test case for 64k FS blocksize generic/436: round up bufsz to nearest filesystem blksz xfs/008: use block size instead of the pagesize
src/seek_sanity_test.c | 8 ++++---- tests/xfs/008 | 20 +++++++++----------- tests/xfs/008.out | 8 ++++---- tests/xfs/161 | 16 ++++++++++++---- 4 files changed, 29 insertions(+), 23 deletions(-)
base-commit: 9837971383733ad59ead121dcc4b13f89829d389
Signature · 1 line
2.34.1
PATCH v3 1/3 xfs/161: adapt the test case for 64k FS blocksize
From: Pankaj Raghav <[email protected]>
This test fails when xfs is formatted with 64k filesystem block size*. It fails because the soft quota is not exceeded with the hardcoded 64k pwrite, thereby, the grace time is not set. Even though soft quota is set to 12k for uid1, it is rounded up to the nearest blocksize.
*** Report for user quotas on device /dev/sdb3
Block grace time: 7days; Inode grace time: 7days
Block limits File limits
User used soft hard grace used soft hard grace
----------------------------------------------------------------------
0 -- 0 0 0 0 3 0 0 0 1 -- 64 64 1024 0 1 0 0 0 2 -- 64 0 0 0 1 0 0 0
Adapt the pwrite to do twice the FS block size and set the soft limit to be 1 FS block and hard limit to be 100 FS blocks. This also gets rid of harcoded quota limit values.
* This happens even on a 64k pagesize system and it is not related to LBS effort.
--- tests/xfs/161 | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-)
tests/xfs/161 +12 -4
@@ -38,15 +38,23 @@ _qmount_option "usrquota" _scratch_xfs_db -c 'version' -c 'sb 0' -c 'p' >> $seqres.full _scratch_mount >> $seqres.full + +blksz=$(_get_file_block_size "$SCRATCH_MNT") +# Write more than one block to exceed the soft block quota limit via +# xfs_quota. +filesz=$(( 2 * $blksz)) +lim_bsoft=$blksz +lim_bhard=$(( 100 * blksz )) + # Force the block counters for uid 1 and 2 above zero -_pwrite_byte 0x61 0 64k $SCRATCH_MNT/a >> $seqres.full -_pwrite_byte 0x61 0 64k $SCRATCH_MNT/b >> $seqres.full +_pwrite_byte 0x61 0 $filesz $SCRATCH_MNT/a >> $seqres.full +_pwrite_byte 0x61 0 $filesz $SCRATCH_MNT/b >> $seqres.full sync chown 1 $SCRATCH_MNT/a chown 2 $SCRATCH_MNT/b # Set quota limits on uid 1 before upgrading -$XFS_QUOTA_PROG -x -c 'limit -u bsoft=12k bhard=1m 1' $SCRATCH_MNT +$XFS_QUOTA_PROG -x -c 'limit -u bsoft='"$lim_bsoft"' bhard='"$lim_bhard"' 1' $SCRATCH_MNT # Make sure the grace period is at /some/ point in the future. We have to # use bc because not all bashes can handle integer comparisons with 64-bit
@@ -71,7 +79,7 @@ _scratch_mount # Set a very generous grace period and quota limits on uid 2 after upgrading $XFS_QUOTA_PROG -x -c 'timer -u -b -d 2147483647' $SCRATCH_MNT -$XFS_QUOTA_PROG -x -c 'limit -u bsoft=10000 bhard=150000 2' $SCRATCH_MNT +$XFS_QUOTA_PROG -x -c 'limit -u bsoft='"$lim_bsoft"' bhard='"$lim_bhard"' 2' $SCRATCH_MNT # Query the grace periods to see if they got set properly after the upgrade. repquota -upn $SCRATCH_MNT > $tmp.repquota
Signature · 1 line
2.34.1
PATCH v3 2/3 generic/436: round up bufsz to nearest filesystem blksz
In reply to Pankaj Raghav (Samsung)From: Pankaj Raghav <[email protected]>
SEEK_HOLE and SEEK_DATA work in filesystem block size granularity. So while filling up the buffer for test 13 - 16, round up the bufsz to the closest filesystem blksz.
As we only allowed blocksizes lower than the pagesize, this was never an issue and it always aligned. Once we have blocksize > pagesize, this assumption will break.
Fixes the test for LBS configuration.
--- src/seek_sanity_test.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
src/seek_sanity_test.c +4 -4
@@ -541,7 +541,7 @@ static int test16(int fd, int testnum) {int ret =0 ;char *buf =NULL ; -int bufsz =sysconf (_SC_PAGE_SIZE); +int bufsz =roundup (sysconf (_SC_PAGE_SIZE), alloc_size);int filsz =4 <<20 ;if (!unwritten_extents) {
@@ -591,7 +591,7 @@ static int test15(int fd, int testnum) {int ret =0 ;char *buf =NULL ; -int bufsz =sysconf (_SC_PAGE_SIZE); +int bufsz =roundup (sysconf (_SC_PAGE_SIZE), alloc_size);int filsz =4 <<20 ;if (!unwritten_extents) {
@@ -643,7 +643,7 @@ static int test14(int fd, int testnum) {int ret =0 ;char *buf =NULL ; -int bufsz =sysconf (_SC_PAGE_SIZE) *14 ; +int bufsz =roundup (sysconf (_SC_PAGE_SIZE) *14 , alloc_size);int filsz =4 <<20 ;if (!unwritten_extents) {
@@ -692,7 +692,7 @@ static int test13(int fd, int testnum) {int ret =0 ;char *buf =NULL ; -int bufsz =sysconf (_SC_PAGE_SIZE) *14 ; +int bufsz =roundup (sysconf (_SC_PAGE_SIZE) *14 , alloc_size);int filsz =4 <<20 ;if (!unwritten_extents) {
Signature · 1 line
2.34.1
Re: [PATCH v3 2/3] generic/436: round up bufsz to nearest filesystem blksz
Quoted 6 lines · Pankaj Raghav (Samsung)
From: Pankaj Raghav <[email protected]>
SEEK_HOLEandSEEK_DATAwork in filesystem block size granularity. So while filling up the buffer for test 13 - 16, round up the bufsz to the closest filesystem blksz.
Quoted 8 lines
As we only allowed blocksizes lower than the pagesize, this was never an issue and it always aligned. Once we have blocksize > pagesize, this assumption will break.
Fixes the test for LBS configuration.
Signed-off-by Pankaj Raghav <[email protected]>Reviewed-by Ritesh Harjani (IBM) <[email protected]>
Looks like I missed the RVB from Darrick [1].
@Zorro could you fix it up before you commit?
[1] https://lore.kernel.org/fstests/20240517155437.GK360908@frogsfrogsfrogs/
Quoted 47 lines
--- src/seek_sanity_test.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)src/seek_sanity_test.c +4
-4@@ -541,7 +541,7 @@ static int test16(int fd, int testnum) {int ret =0 ;char *buf =NULL ; -int bufsz =sysconf (_SC_PAGE_SIZE); +int bufsz =roundup (sysconf (_SC_PAGE_SIZE), alloc_size);int filsz =4 <<20 ;if (!unwritten_extents) {@@ -591,7 +591,7 @@ static int test15(int fd, int testnum) {int ret =0 ;char *buf =NULL ; -int bufsz =sysconf (_SC_PAGE_SIZE); +int bufsz =roundup (sysconf (_SC_PAGE_SIZE), alloc_size);int filsz =4 <<20 ;if (!unwritten_extents) {@@ -643,7 +643,7 @@ static int test14(int fd, int testnum) {int ret =0 ;char *buf =NULL ; -int bufsz =sysconf (_SC_PAGE_SIZE) *14 ; +int bufsz =roundup (sysconf (_SC_PAGE_SIZE) *14 , alloc_size);int filsz =4 <<20 ;if (!unwritten_extents) {@@ -692,7 +692,7 @@ static int test13(int fd, int testnum) {int ret =0 ;char *buf =NULL ; -int bufsz =sysconf (_SC_PAGE_SIZE) *14 ; +int bufsz =roundup (sysconf (_SC_PAGE_SIZE) *14 , alloc_size);int filsz =4 <<20 ;if (!unwritten_extents) {Signature · 1 line
2.34.1
Signature · 1 line
Pankaj Raghav
Re: [PATCH v3 2/3] generic/436: round up bufsz to nearest filesystem blksz
Quoted 11 lines · Pankaj Raghav (Samsung)
Quoted 10 lines · Pankaj Raghav (Samsung)
From: Pankaj Raghav <[email protected]>
SEEK_HOLEandSEEK_DATAwork in filesystem block size granularity. So while filling up the buffer for test 13 - 16, round up the bufsz to the closest filesystem blksz.As we only allowed blocksizes lower than the pagesize, this was never an issue and it always aligned. Once we have blocksize > pagesize, this assumption will break.
Quoted 7 lines
Quoted 4 lines
Fixes the test for LBS configuration.
Signed-off-by Pankaj Raghav <[email protected]>Reviewed-by Ritesh Harjani (IBM) <[email protected]>Looks like I missed the RVB from Darrick [1].
@Zorro could you fix it up before you commit?
Sure, if there's not more review points, I'll merge it with this change.
Thanks,
Zorro
Quoted 53 lines
[1] https://lore.kernel.org/fstests/20240517155437.GK360908@frogsfrogsfrogs/
Quoted 47 lines
--- src/seek_sanity_test.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)src/seek_sanity_test.c +4
-4@@ -541,7 +541,7 @@ static int test16(int fd, int testnum) {int ret =0 ;char *buf =NULL ; -int bufsz =sysconf (_SC_PAGE_SIZE); +int bufsz =roundup (sysconf (_SC_PAGE_SIZE), alloc_size);int filsz =4 <<20 ;if (!unwritten_extents) {@@ -591,7 +591,7 @@ static int test15(int fd, int testnum) {int ret =0 ;char *buf =NULL ; -int bufsz =sysconf (_SC_PAGE_SIZE); +int bufsz =roundup (sysconf (_SC_PAGE_SIZE), alloc_size);int filsz =4 <<20 ;if (!unwritten_extents) {@@ -643,7 +643,7 @@ static int test14(int fd, int testnum) {int ret =0 ;char *buf =NULL ; -int bufsz =sysconf (_SC_PAGE_SIZE) *14 ; +int bufsz =roundup (sysconf (_SC_PAGE_SIZE) *14 , alloc_size);int filsz =4 <<20 ;if (!unwritten_extents) {@@ -692,7 +692,7 @@ static int test13(int fd, int testnum) {int ret =0 ;char *buf =NULL ; -int bufsz =sysconf (_SC_PAGE_SIZE) *14 ; +int bufsz =roundup (sysconf (_SC_PAGE_SIZE) *14 , alloc_size);int filsz =4 <<20 ;if (!unwritten_extents) {Signature · 1 line
2.34.1Signature · 1 line
Pankaj Raghav
Re: [PATCH v3 2/3] generic/436: round up bufsz to nearest filesystem blksz
Quoted 14 lines · Zorro Lang
Quoted 13 lines · Pankaj Raghav (Samsung)
Quoted 12 lines · Pankaj Raghav (Samsung)
From: Pankaj Raghav <[email protected]>
SEEK_HOLEandSEEK_DATAwork in filesystem block size granularity. So while filling up the buffer for test 13 - 16, round up the bufsz to the closest filesystem blksz.As we only allowed blocksizes lower than the pagesize, this was never an issue and it always aligned. Once we have blocksize > pagesize, this assumption will break.
Fixes the test for LBS configuration.
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
Signed-off-by Pankaj Raghav <[email protected]>Reviewed-by Ritesh Harjani (IBM) <[email protected]>Looks like I missed the RVB from Darrick [1].
@Zorro could you fix it up before you commit?
Sure, if there's not more review points, I'll merge it with this change.
Sounds good. Thanks!
PATCH v3 3/3 xfs/008: use block size instead of the pagesize
In reply to Pankaj Raghav (Samsung)From: Pankaj Raghav <[email protected]>
The testcase estimates to have ratio of 1:3/4 for holes:filesize. This holds true where the blocksize is always less than or equal to pagesize and the total size of the file is calculated based on the pagesize. There is an implicit assumption that blocksize will always be less than the pagesize.
LBS support will enable bs > ps where a minimum IO size is one block, which can be greater than a page. Adjust the size calculation to be based on the blocksize and not the pagesize.
--- tests/xfs/008 | 20 +++++++++----------- tests/xfs/008.out | 8 ++++---- 2 files changed, 13 insertions(+), 15 deletions(-)
tests/xfs/008 +9 -11
@@ -10,8 +10,7 @@ _begin_fstest rw ioctl auto quick status=0 # success is the default! -pgsize=`$here/src/feature -s` - +blksize=$(_get_file_block_size "$TEST_DIR") # Override the default cleanup function. _cleanup() {
@@ -21,7 +20,7 @@ _cleanup() _filter() { - sed -e "s/-b $pgsize/-b PGSIZE/g" \ + sed -e "s/-b $blksize/-b BLKSIZE/g" \ -e "s/-l .* -c/-l FSIZE -c/g" }
@@ -73,18 +72,17 @@ _require_test # We are trying to create roughly 50 or 100 holes in a file # using random writes. Assuming a good distribution of 50 writes # in a file, the file only needs to be 3-4x the size of the write -# size muliplied by the number of writes. Hence we use 200 * pgsize -# for files we want 50 holes in and 400 * pgsize for files we want +# size muliplied by the number of writes. Hence we use 200 * blksize +# for files we want 50 holes in and 400 * blksize for files we want # 100 holes in. This keeps the runtime down as low as possible. # -_do_test 1 50 "-l `expr 200 \* $pgsize` -c 50 -b $pgsize" -_do_test 2 100 "-l `expr 400 \* $pgsize` -c 100 -b $pgsize" -_do_test 3 100 "-l `expr 400 \* $pgsize` -c 100 -b 512" # test partial pages +_do_test 1 50 "-l `expr 200 \* $blksize` -c 50 -b $blksize" +_do_test 2 100 "-l `expr 400 \* $blksize` -c 100 -b $blksize" +_do_test 3 100 "-l `expr 400 \* $blksize` -c 100 -b 512" # test partial blocks # rinse, lather, repeat for direct IO -_do_test 4 50 "-d -l `expr 200 \* $pgsize` -c 50 -b $pgsize" -_do_test 5 100 "-d -l `expr 400 \* $pgsize` -c 100 -b $pgsize" -# note: direct IO requires page aligned IO +_do_test 4 50 "-d -l `expr 200 \* $blksize` -c 50 -b $blksize" +_do_test 5 100 "-d -l `expr 400 \* $blksize` -c 100 -b $blksize" # todo: realtime.
tests/xfs/008.out +4 -4
@@ -1,10 +1,10 @@ QA output created by 008 -randholes.1 : -l FSIZE -c 50 -b PGSIZE +randholes.1 : -l FSIZE -c 50 -b BLKSIZE ------------------------------------------ holes is in range -randholes.2 : -l FSIZE -c 100 -b PGSIZE +randholes.2 : -l FSIZE -c 100 -b BLKSIZE ------------------------------------------ holes is in range
@@ -12,10 +12,10 @@ randholes.3 : -l FSIZE -c 100 -b 512 ------------------------------------------ holes is in range -randholes.4 : -d -l FSIZE -c 50 -b PGSIZE +randholes.4 : -d -l FSIZE -c 50 -b BLKSIZE ------------------------------------------ holes is in range -randholes.5 : -d -l FSIZE -c 100 -b PGSIZE +randholes.5 : -d -l FSIZE -c 100 -b BLKSIZE ------------------------------------------ holes is in range
Signature · 1 line
2.34.1
Re: [PATCH v3 3/3] xfs/008: use block size instead of the pagesize
Quoted 11 lines · Pankaj Raghav (Samsung)
From: Pankaj Raghav <[email protected]>
The testcase estimates to have ratio of 1:3/4 for holes:filesize. This holds true where the blocksize is always less than or equal to pagesize and the total size of the file is calculated based on the pagesize. There is an implicit assumption that blocksize will always be less than the pagesize.
LBS support will enable bs > ps where a minimum IO size is one block, which can be greater than a page. Adjust the size calculation to be based on the blocksize and not the pagesize.
Thanks for addressing the comments. I agree it should be just blocksize itself. Even the randholes.c only takes -b blocksize.
(Minor nit below)
Looks good to me. Please feel free to add -
Quoted 12 lines
Signed-off-by Pankaj Raghav <[email protected]>--- tests/xfs/008 | 20 +++++++++----------- tests/xfs/008.out | 8 ++++---- 2 files changed, 13 insertions(+), 15 deletions(-)tests/xfs/008 +0
-0
tests/xfs/008 line 13
Quoted 6 lines
tests/xfs/008 +1
-2@@ -10,8 +10,7 @@ _begin_fstest rw ioctl auto quick status=0 # success is the default! -pgsize=`$here/src/feature -s` - +blksize=$(_get_file_block_size "$TEST_DIR")
nit: maybe we should move blksize calculation to after _require_test ??
Quoted 70 lines
tests/xfs/008 +8
-9# Override the default cleanup function. _cleanup() {@@ -21,7 +20,7 @@ _cleanup() _filter() { - sed -e "s/-b $pgsize/-b PGSIZE/g" \ + sed -e "s/-b $blksize/-b BLKSIZE/g" \ -e "s/-l .* -c/-l FSIZE -c/g" }@@ -73,18 +72,17 @@ _require_test # We are trying to create roughly 50 or 100 holes in a file # using random writes. Assuming a good distribution of 50 writes # in a file, the file only needs to be 3-4x the size of the write -# size muliplied by the number of writes. Hence we use 200 * pgsize -# for files we want 50 holes in and 400 * pgsize for files we want +# size muliplied by the number of writes. Hence we use 200 * blksize +# for files we want 50 holes in and 400 * blksize for files we want # 100 holes in. This keeps the runtime down as low as possible. # -_do_test 1 50 "-l `expr 200 \* $pgsize` -c 50 -b $pgsize" -_do_test 2 100 "-l `expr 400 \* $pgsize` -c 100 -b $pgsize" -_do_test 3 100 "-l `expr 400 \* $pgsize` -c 100 -b 512" # test partial pages +_do_test 1 50 "-l `expr 200 \* $blksize` -c 50 -b $blksize" +_do_test 2 100 "-l `expr 400 \* $blksize` -c 100 -b $blksize" +_do_test 3 100 "-l `expr 400 \* $blksize` -c 100 -b 512" # test partial blocks # rinse, lather, repeat for direct IO -_do_test 4 50 "-d -l `expr 200 \* $pgsize` -c 50 -b $pgsize" -_do_test 5 100 "-d -l `expr 400 \* $pgsize` -c 100 -b $pgsize" -# note: direct IO requires page aligned IO +_do_test 4 50 "-d -l `expr 200 \* $blksize` -c 50 -b $blksize" +_do_test 5 100 "-d -l `expr 400 \* $blksize` -c 100 -b $blksize" # todo: realtime.tests/xfs/008.out +4
-4@@ -1,10 +1,10 @@ QA output created by 008 -randholes.1 : -l FSIZE -c 50 -b PGSIZE +randholes.1 : -l FSIZE -c 50 -b BLKSIZE ------------------------------------------ holes is in range -randholes.2 : -l FSIZE -c 100 -b PGSIZE +randholes.2 : -l FSIZE -c 100 -b BLKSIZE ------------------------------------------ holes is in range@@ -12,10 +12,10 @@ randholes.3 : -l FSIZE -c 100 -b 512 ------------------------------------------ holes is in range -randholes.4 : -d -l FSIZE -c 50 -b PGSIZE +randholes.4 : -d -l FSIZE -c 50 -b BLKSIZE ------------------------------------------ holes is in range -randholes.5 : -d -l FSIZE -c 100 -b PGSIZE +randholes.5 : -d -l FSIZE -c 100 -b BLKSIZE ------------------------------------------ holes is in rangeSignature · 1 line
2.34.1