Skip to contentSkip to conversation

more lbs test fixes

PATCH In review Pankaj Raghav (Samsung) 3 participants 8 messages May 27, 2024 – May 28, 2024

lore
Share & tools
lore
#0 lore raw

v3 cover more lbs test fixes

From: Pankaj Raghav <[email protected]>

These are some of the tests that failed when we were working on LBS patches [1].

generic/436 and xfs/008 are adapted to work with bs > ps systems.

xfs/161 is an upstream failure, and it happens even on bs <= ps systems.

Changes since v2:
- Addes RVB tags
- Removed pgsize fallback in xfs/008 (Ritesh)

Changes since v1:
- Corrected copy pasta error in generic/436
- xfs/161 removed hardcoded and incorporated ritesh's comments

[1] https://lore.kernel.org/linux-xfs/[email protected]/

Pankaj Raghav (3):
xfs/161: adapt the test case for 64k FS blocksize generic/436: round up bufsz to nearest filesystem blksz xfs/008: use block size instead of the pagesize

 src/seek_sanity_test.c |  8 ++++----
 tests/xfs/008          | 20 +++++++++-----------
 tests/xfs/008.out      |  8 ++++----
 tests/xfs/161          | 16 ++++++++++++----
 4 files changed, 29 insertions(+), 23 deletions(-)

base-commit: 9837971383733ad59ead121dcc4b13f89829d389

Signature · 1 line
2.34.1
#1 lore raw

PATCH v3 1/3 xfs/161: adapt the test case for 64k FS blocksize

From: Pankaj Raghav <[email protected]>

This test fails when xfs is formatted with 64k filesystem block size*. It fails because the soft quota is not exceeded with the hardcoded 64k pwrite, thereby, the grace time is not set. Even though soft quota is set to 12k for uid1, it is rounded up to the nearest blocksize.

*** Report for user quotas on device /dev/sdb3
Block grace time: 7days; Inode grace time: 7days
Block limits File limits
User used soft hard grace used soft hard grace
----------------------------------------------------------------------

0        --       0       0       0      0       3     0     0      0
1        --      64      64    1024      0       1     0     0      0
2        --      64       0       0      0       1     0     0      0

Adapt the pwrite to do twice the FS block size and set the soft limit to be 1 FS block and hard limit to be 100 FS blocks. This also gets rid of harcoded quota limit values.

* This happens even on a 64k pagesize system and it is not related to LBS effort.

---
 tests/xfs/161 | 16 ++++++++++++----
 1 file changed, 12 insertions(+), 4 deletions(-)
tests/xfs/161 +12 -4
index 486fa6ca..5bda7019 100755
@@ -38,15 +38,23 @@ _qmount_option "usrquota"
 _scratch_xfs_db -c 'version' -c 'sb 0' -c 'p' >> $seqres.full
 _scratch_mount >> $seqres.full
 
+
+blksz=$(_get_file_block_size "$SCRATCH_MNT")
+# Write more than one block to exceed the soft block quota limit via
+# xfs_quota.
+filesz=$(( 2 * $blksz))
+lim_bsoft=$blksz
+lim_bhard=$(( 100 * blksz ))
+
 # Force the block counters for uid 1 and 2 above zero
-_pwrite_byte 0x61 0 64k $SCRATCH_MNT/a >> $seqres.full
-_pwrite_byte 0x61 0 64k $SCRATCH_MNT/b >> $seqres.full
+_pwrite_byte 0x61 0 $filesz $SCRATCH_MNT/a >> $seqres.full
+_pwrite_byte 0x61 0 $filesz $SCRATCH_MNT/b >> $seqres.full
 sync
 chown 1 $SCRATCH_MNT/a
 chown 2 $SCRATCH_MNT/b
 
 # Set quota limits on uid 1 before upgrading
-$XFS_QUOTA_PROG -x -c 'limit -u bsoft=12k bhard=1m 1' $SCRATCH_MNT
+$XFS_QUOTA_PROG -x -c 'limit -u bsoft='"$lim_bsoft"' bhard='"$lim_bhard"' 1' $SCRATCH_MNT
 
 # Make sure the grace period is at /some/ point in the future.  We have to
 # use bc because not all bashes can handle integer comparisons with 64-bit
@@ -71,7 +79,7 @@ _scratch_mount
 
 # Set a very generous grace period and quota limits on uid 2 after upgrading
 $XFS_QUOTA_PROG -x -c 'timer -u -b -d 2147483647' $SCRATCH_MNT
-$XFS_QUOTA_PROG -x -c 'limit -u bsoft=10000 bhard=150000 2' $SCRATCH_MNT
+$XFS_QUOTA_PROG -x -c 'limit -u bsoft='"$lim_bsoft"' bhard='"$lim_bhard"' 2' $SCRATCH_MNT
 
 # Query the grace periods to see if they got set properly after the upgrade.
 repquota -upn $SCRATCH_MNT > $tmp.repquota
Signature · 1 line
2.34.1
#2 lore raw

PATCH v3 2/3 generic/436: round up bufsz to nearest filesystem blksz

In reply to Pankaj Raghav (Samsung)

From: Pankaj Raghav <[email protected]>

SEEK_HOLE and SEEK_DATA work in filesystem block size granularity. So while filling up the buffer for test 13 - 16, round up the bufsz to the closest filesystem blksz.

As we only allowed blocksizes lower than the pagesize, this was never an issue and it always aligned. Once we have blocksize > pagesize, this assumption will break.

Fixes the test for LBS configuration.

---
 src/seek_sanity_test.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
src/seek_sanity_test.c +4 -4
index 48b3ccc0..a61ed3da 100644
@@ -541,7 +541,7 @@ static int test16(int fd, int testnum)
 {
 	int ret = 0;
 	char *buf = NULL;
-	int bufsz = sysconf(_SC_PAGE_SIZE);
+	int bufsz = roundup(sysconf(_SC_PAGE_SIZE), alloc_size);
 	int filsz = 4 << 20;
 
 	if (!unwritten_extents) {
@@ -591,7 +591,7 @@ static int test15(int fd, int testnum)
 {
 	int ret = 0;
 	char *buf = NULL;
-	int bufsz = sysconf(_SC_PAGE_SIZE);
+	int bufsz = roundup(sysconf(_SC_PAGE_SIZE), alloc_size);
 	int filsz = 4 << 20;
 
 	if (!unwritten_extents) {
@@ -643,7 +643,7 @@ static int test14(int fd, int testnum)
 {
 	int ret = 0;
 	char *buf = NULL;
-	int bufsz = sysconf(_SC_PAGE_SIZE) * 14;
+	int bufsz = roundup(sysconf(_SC_PAGE_SIZE) * 14, alloc_size);
 	int filsz = 4 << 20;
 
 	if (!unwritten_extents) {
@@ -692,7 +692,7 @@ static int test13(int fd, int testnum)
 {
 	int ret = 0;
 	char *buf = NULL;
-	int bufsz = sysconf(_SC_PAGE_SIZE) * 14;
+	int bufsz = roundup(sysconf(_SC_PAGE_SIZE) * 14, alloc_size);
 	int filsz = 4 << 20;
 
 	if (!unwritten_extents) {
Signature · 1 line
2.34.1
#3 lore raw

Re: [PATCH v3 2/3] generic/436: round up bufsz to nearest filesystem blksz

Quoted 6 lines · Pankaj Raghav (Samsung)

From: Pankaj Raghav <[email protected]>

SEEK_HOLE and SEEK_DATA work in filesystem block size granularity. So while filling up the buffer for test 13 - 16, round up the bufsz to the closest filesystem blksz.

Quoted 8 lines

As we only allowed blocksizes lower than the pagesize, this was never an issue and it always aligned. Once we have blocksize > pagesize, this assumption will break.

Fixes the test for LBS configuration.

Looks like I missed the RVB from Darrick [1].

@Zorro could you fix it up before you commit?

[1] https://lore.kernel.org/fstests/20240517155437.GK360908@frogsfrogsfrogs/

Quoted 47 lines
---
 src/seek_sanity_test.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
src/seek_sanity_test.c +4 -4
index 48b3ccc0..a61ed3da 100644
@@ -541,7 +541,7 @@ static int test16(int fd, int testnum)
 {
 	int ret = 0;
 	char *buf = NULL;
-	int bufsz = sysconf(_SC_PAGE_SIZE);
+	int bufsz = roundup(sysconf(_SC_PAGE_SIZE), alloc_size);
 	int filsz = 4 << 20;
 
 	if (!unwritten_extents) {
@@ -591,7 +591,7 @@ static int test15(int fd, int testnum)
 {
 	int ret = 0;
 	char *buf = NULL;
-	int bufsz = sysconf(_SC_PAGE_SIZE);
+	int bufsz = roundup(sysconf(_SC_PAGE_SIZE), alloc_size);
 	int filsz = 4 << 20;
 
 	if (!unwritten_extents) {
@@ -643,7 +643,7 @@ static int test14(int fd, int testnum)
 {
 	int ret = 0;
 	char *buf = NULL;
-	int bufsz = sysconf(_SC_PAGE_SIZE) * 14;
+	int bufsz = roundup(sysconf(_SC_PAGE_SIZE) * 14, alloc_size);
 	int filsz = 4 << 20;
 
 	if (!unwritten_extents) {
@@ -692,7 +692,7 @@ static int test13(int fd, int testnum)
 {
 	int ret = 0;
 	char *buf = NULL;
-	int bufsz = sysconf(_SC_PAGE_SIZE) * 14;
+	int bufsz = roundup(sysconf(_SC_PAGE_SIZE) * 14, alloc_size);
 	int filsz = 4 << 20;
 
 	if (!unwritten_extents) {
Signature · 1 line
2.34.1
Signature · 1 line
Pankaj Raghav
#4 lore raw

Re: [PATCH v3 2/3] generic/436: round up bufsz to nearest filesystem blksz

Quoted 11 lines · Pankaj Raghav (Samsung)
Quoted 10 lines · Pankaj Raghav (Samsung)

From: Pankaj Raghav <[email protected]>

SEEK_HOLE and SEEK_DATA work in filesystem block size granularity. So while filling up the buffer for test 13 - 16, round up the bufsz to the closest filesystem blksz.

As we only allowed blocksizes lower than the pagesize, this was never an issue and it always aligned. Once we have blocksize > pagesize, this assumption will break.

Quoted 7 lines
Quoted 4 lines

Fixes the test for LBS configuration.

Looks like I missed the RVB from Darrick [1].

@Zorro could you fix it up before you commit?

Sure, if there's not more review points, I'll merge it with this change.

Thanks,
Zorro

Quoted 53 lines

[1] https://lore.kernel.org/fstests/20240517155437.GK360908@frogsfrogsfrogs/

Quoted 47 lines
---
 src/seek_sanity_test.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
src/seek_sanity_test.c +4 -4
index 48b3ccc0..a61ed3da 100644
@@ -541,7 +541,7 @@ static int test16(int fd, int testnum)
 {
 	int ret = 0;
 	char *buf = NULL;
-	int bufsz = sysconf(_SC_PAGE_SIZE);
+	int bufsz = roundup(sysconf(_SC_PAGE_SIZE), alloc_size);
 	int filsz = 4 << 20;
 
 	if (!unwritten_extents) {
@@ -591,7 +591,7 @@ static int test15(int fd, int testnum)
 {
 	int ret = 0;
 	char *buf = NULL;
-	int bufsz = sysconf(_SC_PAGE_SIZE);
+	int bufsz = roundup(sysconf(_SC_PAGE_SIZE), alloc_size);
 	int filsz = 4 << 20;
 
 	if (!unwritten_extents) {
@@ -643,7 +643,7 @@ static int test14(int fd, int testnum)
 {
 	int ret = 0;
 	char *buf = NULL;
-	int bufsz = sysconf(_SC_PAGE_SIZE) * 14;
+	int bufsz = roundup(sysconf(_SC_PAGE_SIZE) * 14, alloc_size);
 	int filsz = 4 << 20;
 
 	if (!unwritten_extents) {
@@ -692,7 +692,7 @@ static int test13(int fd, int testnum)
 {
 	int ret = 0;
 	char *buf = NULL;
-	int bufsz = sysconf(_SC_PAGE_SIZE) * 14;
+	int bufsz = roundup(sysconf(_SC_PAGE_SIZE) * 14, alloc_size);
 	int filsz = 4 << 20;
 
 	if (!unwritten_extents) {
Signature · 1 line
2.34.1
Signature · 1 line
Pankaj Raghav
#5 lore raw

Re: [PATCH v3 2/3] generic/436: round up bufsz to nearest filesystem blksz

Quoted 14 lines · Zorro Lang
Quoted 13 lines · Pankaj Raghav (Samsung)
Quoted 12 lines · Pankaj Raghav (Samsung)

From: Pankaj Raghav <[email protected]>

SEEK_HOLE and SEEK_DATA work in filesystem block size granularity. So while filling up the buffer for test 13 - 16, round up the bufsz to the closest filesystem blksz.

As we only allowed blocksizes lower than the pagesize, this was never an issue and it always aligned. Once we have blocksize > pagesize, this assumption will break.

Fixes the test for LBS configuration.

Quoted 7 lines
Quoted 5 lines
Quoted 2 lines

Looks like I missed the RVB from Darrick [1].

@Zorro could you fix it up before you commit?

Sure, if there's not more review points, I'll merge it with this change.

Sounds good. Thanks!

#6 lore raw

PATCH v3 3/3 xfs/008: use block size instead of the pagesize

In reply to Pankaj Raghav (Samsung)

From: Pankaj Raghav <[email protected]>

The testcase estimates to have ratio of 1:3/4 for holes:filesize. This holds true where the blocksize is always less than or equal to pagesize and the total size of the file is calculated based on the pagesize. There is an implicit assumption that blocksize will always be less than the pagesize.

LBS support will enable bs > ps where a minimum IO size is one block, which can be greater than a page. Adjust the size calculation to be based on the blocksize and not the pagesize.

---
 tests/xfs/008     | 20 +++++++++-----------
 tests/xfs/008.out |  8 ++++----
 2 files changed, 13 insertions(+), 15 deletions(-)
tests/xfs/008 +9 -11
index e7d6153b..f42b3ac8 100755
@@ -10,8 +10,7 @@
 _begin_fstest rw ioctl auto quick
 
 status=0	# success is the default!
-pgsize=`$here/src/feature -s`
-
+blksize=$(_get_file_block_size "$TEST_DIR")
 # Override the default cleanup function.
 _cleanup()
 {
@@ -21,7 +20,7 @@ _cleanup()
 
 _filter()
 {
-    sed -e "s/-b $pgsize/-b PGSIZE/g" \
+    sed -e "s/-b $blksize/-b BLKSIZE/g" \
 	-e "s/-l .* -c/-l FSIZE -c/g"
 }
 
@@ -73,18 +72,17 @@ _require_test
 # We are trying to create roughly 50 or 100 holes in a file
 # using random writes. Assuming a good distribution of 50 writes
 # in a file, the file only needs to be 3-4x the size of the write
-# size muliplied by the number of writes. Hence we use 200 * pgsize
-# for files we want 50 holes in and 400 * pgsize for files we want
+# size muliplied by the number of writes. Hence we use 200 * blksize
+# for files we want 50 holes in and 400 * blksize for files we want
 # 100 holes in. This keeps the runtime down as low as possible.
 #
-_do_test 1 50 "-l `expr 200 \* $pgsize` -c 50 -b $pgsize"
-_do_test 2 100 "-l `expr 400 \* $pgsize` -c 100 -b $pgsize"
-_do_test 3 100 "-l `expr 400 \* $pgsize` -c 100 -b 512"   # test partial pages
+_do_test 1 50 "-l `expr 200 \* $blksize` -c 50 -b $blksize"
+_do_test 2 100 "-l `expr 400 \* $blksize` -c 100 -b $blksize"
+_do_test 3 100 "-l `expr 400 \* $blksize` -c 100 -b 512"   # test partial blocks
 
 # rinse, lather, repeat for direct IO
-_do_test 4 50 "-d -l `expr 200 \* $pgsize` -c 50 -b $pgsize"
-_do_test 5 100 "-d -l `expr 400 \* $pgsize` -c 100 -b $pgsize"
-# note: direct IO requires page aligned IO
+_do_test 4 50 "-d -l `expr 200 \* $blksize` -c 50 -b $blksize"
+_do_test 5 100 "-d -l `expr 400 \* $blksize` -c 100 -b $blksize"
 
 # todo: realtime.
 
tests/xfs/008.out +4 -4
index 5e3ae8e3..0941e218 100644
@@ -1,10 +1,10 @@
 QA output created by 008
 
-randholes.1 : -l FSIZE -c 50 -b PGSIZE
+randholes.1 : -l FSIZE -c 50 -b BLKSIZE
 ------------------------------------------
 holes is in range
 
-randholes.2 : -l FSIZE -c 100 -b PGSIZE
+randholes.2 : -l FSIZE -c 100 -b BLKSIZE
 ------------------------------------------
 holes is in range
 
@@ -12,10 +12,10 @@ randholes.3 : -l FSIZE -c 100 -b 512
 ------------------------------------------
 holes is in range
 
-randholes.4 : -d -l FSIZE -c 50 -b PGSIZE
+randholes.4 : -d -l FSIZE -c 50 -b BLKSIZE
 ------------------------------------------
 holes is in range
 
-randholes.5 : -d -l FSIZE -c 100 -b PGSIZE
+randholes.5 : -d -l FSIZE -c 100 -b BLKSIZE
 ------------------------------------------
 holes is in range
Signature · 1 line
2.34.1
Reviewed-by on v3 3/3
#7 lore raw

Re: [PATCH v3 3/3] xfs/008: use block size instead of the pagesize

Quoted 11 lines · Pankaj Raghav (Samsung)

From: Pankaj Raghav <[email protected]>

The testcase estimates to have ratio of 1:3/4 for holes:filesize. This holds true where the blocksize is always less than or equal to pagesize and the total size of the file is calculated based on the pagesize. There is an implicit assumption that blocksize will always be less than the pagesize.

LBS support will enable bs > ps where a minimum IO size is one block, which can be greater than a page. Adjust the size calculation to be based on the blocksize and not the pagesize.

Thanks for addressing the comments. I agree it should be just blocksize itself. Even the randholes.c only takes -b blocksize.
(Minor nit below)

Looks good to me. Please feel free to add -

Quoted 12 lines
---
 tests/xfs/008     | 20 +++++++++-----------
 tests/xfs/008.out |  8 ++++----
 2 files changed, 13 insertions(+), 15 deletions(-)
tests/xfs/008 +0 -0
index e7d6153b..f42b3ac8 100755

tests/xfs/008 line 13

Quoted 6 lines
tests/xfs/008 +1 -2
@@ -10,8 +10,7 @@
 _begin_fstest rw ioctl auto quick
 
 status=0	# success is the default!
-pgsize=`$here/src/feature -s`
-
+blksize=$(_get_file_block_size "$TEST_DIR")

nit: maybe we should move blksize calculation to after _require_test ??

Quoted 70 lines
tests/xfs/008 +8 -9
 # Override the default cleanup function.
 _cleanup()
 {
@@ -21,7 +20,7 @@ _cleanup()
 
 _filter()
 {
-    sed -e "s/-b $pgsize/-b PGSIZE/g" \
+    sed -e "s/-b $blksize/-b BLKSIZE/g" \
 	-e "s/-l .* -c/-l FSIZE -c/g"
 }
 
@@ -73,18 +72,17 @@ _require_test
 # We are trying to create roughly 50 or 100 holes in a file
 # using random writes. Assuming a good distribution of 50 writes
 # in a file, the file only needs to be 3-4x the size of the write
-# size muliplied by the number of writes. Hence we use 200 * pgsize
-# for files we want 50 holes in and 400 * pgsize for files we want
+# size muliplied by the number of writes. Hence we use 200 * blksize
+# for files we want 50 holes in and 400 * blksize for files we want
 # 100 holes in. This keeps the runtime down as low as possible.
 #
-_do_test 1 50 "-l `expr 200 \* $pgsize` -c 50 -b $pgsize"
-_do_test 2 100 "-l `expr 400 \* $pgsize` -c 100 -b $pgsize"
-_do_test 3 100 "-l `expr 400 \* $pgsize` -c 100 -b 512"   # test partial pages
+_do_test 1 50 "-l `expr 200 \* $blksize` -c 50 -b $blksize"
+_do_test 2 100 "-l `expr 400 \* $blksize` -c 100 -b $blksize"
+_do_test 3 100 "-l `expr 400 \* $blksize` -c 100 -b 512"   # test partial blocks
 
 # rinse, lather, repeat for direct IO
-_do_test 4 50 "-d -l `expr 200 \* $pgsize` -c 50 -b $pgsize"
-_do_test 5 100 "-d -l `expr 400 \* $pgsize` -c 100 -b $pgsize"
-# note: direct IO requires page aligned IO
+_do_test 4 50 "-d -l `expr 200 \* $blksize` -c 50 -b $blksize"
+_do_test 5 100 "-d -l `expr 400 \* $blksize` -c 100 -b $blksize"
 
 # todo: realtime.
 
tests/xfs/008.out +4 -4
index 5e3ae8e3..0941e218 100644
@@ -1,10 +1,10 @@
 QA output created by 008
 
-randholes.1 : -l FSIZE -c 50 -b PGSIZE
+randholes.1 : -l FSIZE -c 50 -b BLKSIZE
 ------------------------------------------
 holes is in range
 
-randholes.2 : -l FSIZE -c 100 -b PGSIZE
+randholes.2 : -l FSIZE -c 100 -b BLKSIZE
 ------------------------------------------
 holes is in range
 
@@ -12,10 +12,10 @@ randholes.3 : -l FSIZE -c 100 -b 512
 ------------------------------------------
 holes is in range
 
-randholes.4 : -d -l FSIZE -c 50 -b PGSIZE
+randholes.4 : -d -l FSIZE -c 50 -b BLKSIZE
 ------------------------------------------
 holes is in range
 
-randholes.5 : -d -l FSIZE -c 100 -b PGSIZE
+randholes.5 : -d -l FSIZE -c 100 -b BLKSIZE
 ------------------------------------------
 holes is in range
Signature · 1 line
2.34.1