du01 with btrfs on systems with > 4k page size

LTP New Stanislav Kholmanskikh 2 participants 8 messages Nov 9, 2016 – Nov 21, 2016

lore

Share & tools

lore

Stanislav Kholmanskikh

author

2016-11-09 15:29 UTC #0 lore raw

Hi!

On SPARC the default page size is 8k.

And du01 fails with btrfs at check3:

du01 3 TFAIL : 'du -a' failed
du01 4 TINFO : Looking for '[0-4][[:space:]]\.\/testdir\/testsymlink' in:

10240  ./testfile
8      ./testdir/testsymlink
8      ./testdir
10248  .
du01 4 TFAIL : 'du --all' failed
du01 5 TINFO : Looking for '[0-4][[:space:]]\.\/testdir\/testsymlink' in:
10240  ./testfile
8      ./testdir/testsymlink
8      ./testdir
10248  .

i.e. the testsymlink is 8k whereas at most 4k is expected by the test case.

In commit bdd09b1c6f2c8ad ("du01.sh: Fix failures on Btrfs on ppc64le") a similar situation was fixed, but for check5 and check6. I'm curious why check3 doesn't fail on ppc64. It seems it should fail with the current code.

Could, please, anybody with access to a ppc64 box run this test case with btrfs and/or provide the output from commands:

[root@skholman-m7 du]# cd /mnt
[root@skholman-m7 mnt]# mkdir basedir
[root@skholman-m7 mnt]# cd basedir/
[root@skholman-m7 basedir]# dd if=/dev/zero of=testfile bs=1M count=10 10+0 records in
10+0 records out
10485760 bytes (10 MB) copied, 0.00825522 s, 1.3 GB/s [root@skholman-m7 basedir]# mkdir -p testdir [root@skholman-m7 basedir]# ln -s ../testfile testdir/testsymlink [root@skholman-m7 basedir]# du -a
10240 ./testfile
8 ./testdir/testsymlink
8 ./testdir
10248 .
[root@skholman-m7 basedir]#

Thanks.

Cyril Hrubis

2016-11-09 16:18 UTC #1 lore raw

Hi!

Quoted 27 lines

On SPARC the default page size is 8k.
And du01 fails with btrfs at check3:
du01 3 TFAIL : 'du -a' failed
du01 4 TINFO : Looking for '[0-4][[:space:]]\.\/testdir\/testsymlink' in:
10240  ./testfile
8      ./testdir/testsymlink
8      ./testdir
10248  .
du01 4 TFAIL : 'du --all' failed
du01 5 TINFO : Looking for '[0-4][[:space:]]\.\/testdir\/testsymlink' in:
10240  ./testfile
8      ./testdir/testsymlink
8      ./testdir
10248  .
i.e. the testsymlink is 8k whereas at most 4k is expected by the test case.
In commit bdd09b1c6f2c8ad ("du01.sh: Fix failures on Btrfs on ppc64le") a similar situation was fixed, but for check5 and check6. I'm curious why check3 doesn't fail on ppc64. It seems it should fail with the current code.
Could, please, anybody with access to a ppc64 box run this test case with btrfs and/or provide the output from commands:

Quoted 15 lines

[root@skholman-m7 du]# cd /mnt
[root@skholman-m7 mnt]# mkdir basedir
[root@skholman-m7 mnt]# cd basedir/
[root@skholman-m7 basedir]# dd if=/dev/zero of=testfile bs=1M count=10 10+0 records in
10+0 records out
10485760 bytes (10 MB) copied, 0.00825522 s, 1.3 GB/s [root@skholman-m7 basedir]# mkdir -p testdir [root@skholman-m7 basedir]# ln -s ../testfile testdir/testsymlink [root@skholman-m7 basedir]# du -a
10240 ./testfile
8 ./testdir/testsymlink
8 ./testdir
10248 .
[root@skholman-m7 basedir]#

It's really 64k for a symlink:

du -a

10240   ./testfile
64      ./testdir/testsymlink
64      ./testdir
10304   .

And the reason that it works is:

commit 9712f3122a46c43fccd694ee7204ec8c19cfacdc

Author: Cyril Hrubis <[email protected]>
Date:   Wed Jan 13 17:33:51 2016 +0100

commands: du01.sh; Btrfs fix.

Btrfs reports symlinks to be 4 blocks in size.

Also rename the test symlink so that it's clear it's symlink.

Since the grep ignores the 6 at the start of the pattern and succeeds. It's strange that logs I have in SUSE bugzilla says that it was 4k in size, but that may have been true for some other kernel, since the pagesize is configurable in kernel .config.

So I guess that we should fix the pattern to match anyhing that could be pagesize. For PPC64 that should be one of 4K/16K/64K/256K, it's 8K for sparc apparently so I guess that we should go for 2^n between 4 and 256.

And we may also specify that the pattern should start matching at the start of the line, otherwise the test will still pass if there was some garbage before the correct string.

Signature · 2 lines

Cyril Hrubis
[email protected]

Stanislav Kholmanskikh

author

2016-11-10 14:39 UTC #2 lore raw

Quoted 44 lines · Cyril Hrubis

Hi!
Quoted 27 lines
On SPARC the default page size is 8k.
And du01 fails with btrfs at check3:
du01 3 TFAIL : 'du -a' failed
du01 4 TINFO : Looking for '[0-4][[:space:]]\.\/testdir\/testsymlink' in:
10240  ./testfile
8      ./testdir/testsymlink
8      ./testdir
10248  .
du01 4 TFAIL : 'du --all' failed
du01 5 TINFO : Looking for '[0-4][[:space:]]\.\/testdir\/testsymlink' in:
10240  ./testfile
8      ./testdir/testsymlink
8      ./testdir
10248  .
i.e. the testsymlink is 8k whereas at most 4k is expected by the test case.
In commit bdd09b1c6f2c8ad ("du01.sh: Fix failures on Btrfs on ppc64le") a similar situation was fixed, but for check5 and check6. I'm curious why check3 doesn't fail on ppc64. It seems it should fail with the current code.
Could, please, anybody with access to a ppc64 box run this test case with btrfs and/or provide the output from commands:
Quoted 15 lines
[root@skholman-m7 du]# cd /mnt
[root@skholman-m7 mnt]# mkdir basedir
[root@skholman-m7 mnt]# cd basedir/
[root@skholman-m7 basedir]# dd if=/dev/zero of=testfile bs=1M count=10 10+0 records in
10+0 records out
10485760 bytes (10 MB) copied, 0.00825522 s, 1.3 GB/s [root@skholman-m7 basedir]# mkdir -p testdir [root@skholman-m7 basedir]# ln -s ../testfile testdir/testsymlink [root@skholman-m7 basedir]# du -a
10240 ./testfile
8 ./testdir/testsymlink
8 ./testdir
10248 .
[root@skholman-m7 basedir]#

Quoted 8 lines

It's really 64k for a symlink:
du -a
10240   ./testfile
64      ./testdir/testsymlink
64      ./testdir
10304   .

Thank you.

Quoted 19 lines

And the reason that it works is:
commit 9712f3122a46c43fccd694ee7204ec8c19cfacdc
Author: Cyril Hrubis <[email protected]>
Date:   Wed Jan 13 17:33:51 2016 +0100
commands: du01.sh; Btrfs fix.
Btrfs reports symlinks to be 4 blocks in size.
Also rename the test symlink so that it's clear it's symlink.
Since the grep ignores the 6 at the start of the pattern and succeeds. It's strange that logs I have in SUSE bugzilla says that it was 4k in size, but that may have been true for some other kernel, since the pagesize is configurable in kernel .config.

Quoted 4 lines

So I guess that we should fix the pattern to match anyhing that could be pagesize. For PPC64 that should be one of 4K/16K/64K/256K, it's 8K for sparc apparently so I guess that we should go for 2^n between 4 and 256.

Maybe determine the default page size at runtime (getconf PAGESIZE) and check if testsymlink is either 0 or the page size?

Quoted 5 lines

And we may also specify that the pattern should start matching at the start of the line, otherwise the test will still pass if there was some garbage before the correct string.

Agreed. For all the checks.

Cyril Hrubis

2016-11-10 15:12 UTC #3 lore raw

Hi!

Quoted 5 lines

Quoted 5 lines
Since the grep ignores the 6 at the start of the pattern and succeeds. It's strange that logs I have in SUSE bugzilla says that it was 4k in size, but that may have been true for some other kernel, since the pagesize is configurable in kernel .config.

Quoted 7 lines

Quoted 4 lines
So I guess that we should fix the pattern to match anyhing that could be pagesize. For PPC64 that should be one of 4K/16K/64K/256K, it's 8K for sparc apparently so I guess that we should go for 2^n between 4 and 256.
Maybe determine the default page size at runtime (getconf PAGESIZE) and check if testsymlink is either 0 or the page size?

Sounds good.

Signature · 2 lines

Cyril Hrubis
[email protected]

Stanislav Kholmanskikh

author

2016-11-16 15:39 UTC #4 lore raw

PATCH 1/2 du01.sh: start matching at the beginning of a line

Otherwise the test may pass if there is some garbage before the correct string.

Signed-off-by Stanislav Kholmanskikh <[email protected]>

---
 testcases/commands/du/du01.sh |   22 +++++++++++-----------
 1 files changed, 11 insertions(+), 11 deletions(-)

testcases/commands/du/du01.sh +11 ~~-11~~

index a0e3017..823c50a 100755
@@ -86,17 +86,17 @@ block_size=512
 # The output could be different in some systems, if we use du to
 # estimate file space usage with the same filesystem and the same size.
 # So we use the approximate value to check.
-check1="10[2-3][0-9][0-9][[:space:]]\."
-check2="10[2-3][0-9][0-9][[:space:]]testfile"
-check3="[0-4][[:space:]]\.\/testdir\/testsymlink"
-check5="20[4-6][0-9][0-9][[:space:]]\."
-check7="10[4-5][0-9][0-9]\{4\}[[:space:]]\."
-check9="10[2-3][0-9][0-9][[:space:]]total"
-check11="10[2-3][0-9][0-9][[:space:]]testdir\/testsymlink"
-check14="1[0,1]M[[:space:]]\."
-check16="10[2-3][0-9][0-9][[:space:]]testdir\/"
-check20="11M[[:space:]]\."
-check23="[0-9]\{1,2\}[[:space:]]\."
+check1="^10[2-3][0-9][0-9][[:space:]]\."
+check2="^10[2-3][0-9][0-9][[:space:]]testfile"
+check3="^[0-4][[:space:]]\.\/testdir\/testsymlink"
+check5="^20[4-6][0-9][0-9][[:space:]]\."
+check7="^10[4-5][0-9][0-9]\{4\}[[:space:]]\."
+check9="^10[2-3][0-9][0-9][[:space:]]total"
+check11="^10[2-3][0-9][0-9][[:space:]]testdir\/testsymlink"
+check14="^1[0,1]M[[:space:]]\."
+check16="^10[2-3][0-9][0-9][[:space:]]testdir\/"
+check20="^11M[[:space:]]\."
+check23="^[0-9]\{1,2\}[[:space:]]\."
 
 du_test "du" ${check1}
 du_test "du testfile" ${check2}

Signature · 1 line

1.7.1

Stanislav Kholmanskikh

author

2016-11-16 15:39 UTC #5 lore raw

PATCH 2/2 du01.sh: BTRFS fix for the symlink

By default, BTRFS reports symlinks to be of page size.

Modify the corresponding check to verify if the symlink is either 0 or page size.

Signed-off-by Stanislav Kholmanskikh <[email protected]>

---
 testcases/commands/du/du01.sh |    8 +++++++-
 1 files changed, 7 insertions(+), 1 deletions(-)

testcases/commands/du/du01.sh +7 -1

index 823c50a..7b17f6f 100755
@@ -83,12 +83,18 @@ setup
 
 block_size=512
 
+page_size=$(getconf PAGESIZE)
+if [ "$page_size" -lt 1024 ]; then
+	tst_brkm TBROK "Page size < 1024"
+fi
+page_size=$(( page_size / 1024 ))
+
 # The output could be different in some systems, if we use du to
 # estimate file space usage with the same filesystem and the same size.
 # So we use the approximate value to check.
 check1="^10[2-3][0-9][0-9][[:space:]]\."
 check2="^10[2-3][0-9][0-9][[:space:]]testfile"
-check3="^[0-4][[:space:]]\.\/testdir\/testsymlink"
+check3="^\(0\|${page_size}\)[[:space:]]\.\/testdir\/testsymlink"
 check5="^20[4-6][0-9][0-9][[:space:]]\."
 check7="^10[4-5][0-9][0-9]\{4\}[[:space:]]\."
 check9="^10[2-3][0-9][0-9][[:space:]]total"

Signature · 1 line

1.7.1

Cyril Hrubis

2016-11-21 12:40 UTC #6 lore raw

PATCH 2/2 du01.sh: BTRFS fix for the symlink

Hi!

Quoted 15 lines

By default, BTRFS reports symlinks to be of page size.
Modify the corresponding check to verify if the symlink is either 0 or page size.
Signed-off-by Stanislav Kholmanskikh <[email protected]>
---
 testcases/commands/du/du01.sh |    8 +++++++-
 1 files changed, 7 insertions(+), 1 deletions(-)
testcases/commands/du/du01.sh +0 -0
index 823c50a..7b17f6f 100755

testcases/commands/du/du01.sh setup · line 89

Quoted 7 lines

testcases/commands/du/du01.sh +4 -0

@@ -83,12 +83,18 @@ setup
 
 block_size=512
 
+page_size=$(getconf PAGESIZE)
+if [ "$page_size" -lt 1024 ]; then
+	tst_brkm TBROK "Page size < 1024"
+fi

I doubt that this will ever happen, but it does not do any harm anyway.

Both patches acked, thanks for fixing this.

Quoted 18 lines

testcases/commands/du/du01.sh +3 -1

+page_size=$(( page_size / 1024 ))
+
 # The output could be different in some systems, if we use du to
 # estimate file space usage with the same filesystem and the same size.
 # So we use the approximate value to check.
 check1="^10[2-3][0-9][0-9][[:space:]]\."
 check2="^10[2-3][0-9][0-9][[:space:]]testfile"
-check3="^[0-4][[:space:]]\.\/testdir\/testsymlink"
+check3="^\(0\|${page_size}\)[[:space:]]\.\/testdir\/testsymlink"
 check5="^20[4-6][0-9][0-9][[:space:]]\."
 check7="^10[4-5][0-9][0-9]\{4\}[[:space:]]\."
 check9="^10[2-3][0-9][0-9][[:space:]]total"

Signature · 5 lines

1.7.1


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Signature · 2 lines

Cyril Hrubis
[email protected]

Stanislav Kholmanskikh

author

2016-11-21 14:07 UTC #7 lore raw

PATCH 2/2 du01.sh: BTRFS fix for the symlink

Quoted 19 lines · Cyril Hrubis

Hi!
Quoted 18 lines
By default, BTRFS reports symlinks to be of page size.
Modify the corresponding check to verify if the symlink is either 0 or page size.
Signed-off-by Stanislav Kholmanskikh <[email protected]>
---
 testcases/commands/du/du01.sh |    8 +++++++-
 1 files changed, 7 insertions(+), 1 deletions(-)
testcases/commands/du/du01.sh +0 -0
index 823c50a..7b17f6f 100755
@@ -83,12 +83,18 @@ setup
 
 block_size=512
 

Quoted 8 lines

testcases/commands/du/du01.sh setup · line 89
Quoted 4 lines
testcases/commands/du/du01.sh +4 -0
+page_size=$(getconf PAGESIZE)
+if [ "$page_size" -lt 1024 ]; then
+	tst_brkm TBROK "Page size < 1024"
+fi
I doubt that this will ever happen, but it does not do any harm anyway.
Both patches acked, thanks for fixing this.

Thank you. Pushed.

Quoted 20 lines

Quoted 19 lines

testcases/commands/du/du01.sh +3 -1

+page_size=$(( page_size / 1024 ))
+
 # The output could be different in some systems, if we use du to
 # estimate file space usage with the same filesystem and the same size.
 # So we use the approximate value to check.
 check1="^10[2-3][0-9][0-9][[:space:]]\."
 check2="^10[2-3][0-9][0-9][[:space:]]testfile"
-check3="^[0-4][[:space:]]\.\/testdir\/testsymlink"
+check3="^\(0\|${page_size}\)[[:space:]]\.\/testdir\/testsymlink"
 check5="^20[4-6][0-9][0-9][[:space:]]\."
 check7="^10[4-5][0-9][0-9]\{4\}[[:space:]]\."
 check9="^10[2-3][0-9][0-9][[:space:]]total"

Signature · 5 lines

1.7.1


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp