Skip to contentSkip to conversation

qa 166 failure on f8 kernel

DISCUSSION Eric Sandeen 6 participants 16 messages Aug 8, 2007 1 message not in the archive

lore
Share & tools
lore

Eric Sandeen

author
#0 lore raw

Posting this just in case it rings any bells, though I plan to investigate...

[root@inode xfstests]# ./check 166

FSTYP         -- xfs (non-debug)
PLATFORM      -- Linux/x86_64 inode 2.6.23-0.71.rc2.fc8
MKFS_OPTIONS  -- -f -bsize=4096 /dev/sdb6

MOUNT_OPTIONS -- /dev/sdb6 /mnt/sdb6

166 - output mismatch (see 166.out.bad)
2,6c2,7
< 0: [0..31]: XX..YY AG (AA..BB) 32
< 1: [32..127]: XX..YY AG (AA..BB) 96 10000
< 2: [128..159]: XX..YY AG (AA..BB) 32
< 3: [160..223]: XX..YY AG (AA..BB) 64 10000
< 4: [224..255]: XX..YY AG (AA..BB) 32
---

Quoted 6 lines

/mnt/sdb6/test_file: XX..YY AG (AA..BB) 0: [0..7]: XX..YY AG (AA..BB) 8
1: [8..127]: XX..YY AG (AA..BB) 120 10000 2: [128..135]: XX..YY AG (AA..BB) 8 3: [136..247]: XX..YY AG (AA..BB) 112 10000 4: [248..255]: XX..YY AG (AA..BB) 8

Failures: 166
Failed 1 of 1 tests
[root@inode xfstests]# uname -a
Linux inode.lab.msp.redhat.com 2.6.23-0.71.rc2.fc8 #1 SMP Sat Aug 4 01:21:06 EDT 2007 x86_64 x86_64 x86_64 GNU/Linux

-Eric

#1 lore raw
Quoted 9 lines · Eric Sandeen

Posting this just in case it rings any bells, though I plan to investigate...

[root@inode xfstests]# ./check 166

FSTYP         -- xfs (non-debug)
PLATFORM      -- Linux/x86_64 inode 2.6.23-0.71.rc2.fc8
MKFS_OPTIONS  -- -f -bsize=4096 /dev/sdb6

MOUNT_OPTIONS -- /dev/sdb6 /mnt/sdb6

Quoted 14 lines

166 - output mismatch (see 166.out.bad)
2,6c2,7
< 0: [0..31]: XX..YY AG (AA..BB) 32
< 1: [32..127]: XX..YY AG (AA..BB) 96 10000
< 2: [128..159]: XX..YY AG (AA..BB) 32
< 3: [160..223]: XX..YY AG (AA..BB) 64 10000
< 4: [224..255]: XX..YY AG (AA..BB) 32
---

Quoted 6 lines

/mnt/sdb6/test_file: XX..YY AG (AA..BB) 0: [0..7]: XX..YY AG (AA..BB) 8
1: [8..127]: XX..YY AG (AA..BB) 120 10000 2: [128..135]: XX..YY AG (AA..BB) 8 3: [136..247]: XX..YY AG (AA..BB) 112 10000 4: [248..255]: XX..YY AG (AA..BB) 8

When you post the failure, someone will say, "Duh, that's obvious".

Well: Duh! That's obvious. ;)

You've got 3x4k written blocks in the file which is *correct*. There's nothing wrong with the kernel code. It's just that the test is expecting 3x16k extents to be marked written.

See the problem yet?

mmap dirties entire pages. page size differs between platforms - ia64 = 16k, x86 = 4k - so the size of the extent allocated is different. Guess what platform I wrote the test on and use as my primary platform?

The output needs better filtering, methinks.

Cheers,

Dave.

Signature · 3 lines
Dave Chinner
Principal Engineer
SGI Australian Software Group
author
#2 lore raw
Quoted 18 lines · David Chinner
Quoted 17 lines · Eric Sandeen

Posting this just in case it rings any bells, though I plan to investigate...

[root@inode xfstests]# ./check 166

FSTYP         -- xfs (non-debug)
PLATFORM      -- Linux/x86_64 inode 2.6.23-0.71.rc2.fc8
MKFS_OPTIONS  -- -f -bsize=4096 /dev/sdb6

MOUNT_OPTIONS -- /dev/sdb6 /mnt/sdb6

166 - output mismatch (see 166.out.bad)
2,6c2,7
< 0: [0..31]: XX..YY AG (AA..BB) 32
< 1: [32..127]: XX..YY AG (AA..BB) 96 10000
< 2: [128..159]: XX..YY AG (AA..BB) 32
< 3: [160..223]: XX..YY AG (AA..BB) 64 10000
< 4: [224..255]: XX..YY AG (AA..BB) 32
---

Quoted 8 lines
Quoted 6 lines
Quoted 6 lines

/mnt/sdb6/test_file: XX..YY AG (AA..BB) 0: [0..7]: XX..YY AG (AA..BB) 8
1: [8..127]: XX..YY AG (AA..BB) 120 10000 2: [128..135]: XX..YY AG (AA..BB) 8 3: [136..247]: XX..YY AG (AA..BB) 112 10000 4: [248..255]: XX..YY AG (AA..BB) 8

When you post the failure, someone will say, "Duh, that's obvious".

fine, fine ;-)

Quoted 6 lines

Well: Duh! That's obvious. ;)

You've got 3x4k written blocks in the file which is *correct*. There's nothing wrong with the kernel code. It's just that the test is expecting 3x16k extents to be marked written.

Quoted 8 lines

See the problem yet?

mmap dirties entire pages. page size differs between platforms - ia64 = 16k, x86 = 4k - so the size of the extent allocated is different. Guess what platform I wrote the test on and use as my primary platform?

The output needs better filtering, methinks.

yeah, figured it had something to do w/ the ia64 weenies when I saw the 32 vs. 8, factor of 4 ... fine, fine, my bad. :)

Thanks,

-Eric

Quoted 3 lines

Cheers,

Dave.

#3 lore raw
Quoted 2 lines · Eric Sandeen

yeah, figured it had something to do w/ the ia64 weenies when I saw the 32 vs. 8, factor of 4 ... fine, fine, my bad. :)

No, not your bad. Mine if anyones because I wrote the test.

Cheers,

Dave.

Signature · 3 lines
Dave Chinner
Principal Engineer
SGI Australian Software Group
author
#4 lore raw
Quoted 6 lines · David Chinner
Quoted 3 lines · Eric Sandeen

yeah, figured it had something to do w/ the ia64 weenies when I saw the 32 vs. 8, factor of 4 ... fine, fine, my bad. :)

Quoted 6 lines

No, not your bad. Mine if anyones because I wrote the test.

Cheers,

Dave.

Well, I was going to look at it more closely before I sent it off :)

What do you think of a patch like this, to munmap 16k chunks regardless of page size:

unwritten_mmap.c renamed from unwritten_mmap.c.orig +11 -1
@@ -12,6 +12,7 @@
  */
 int main(int argc, char **argv) {
 	unsigned long long o;
+	int minsize;
 	int fd, i;
 	struct xfs_flock64 space;
 	unsigned char *buf;

@@ -23,6 +24,13 @@
 
 	errno = 0;
 	o = strtoull(argv[1], NULL, 0);
+
+	minsize = 3*16384; /* 3 ia64 pages */
+	if (o < minsize) {
+		fprintf(stderr, "count must be at least %d\n", minsize);
+		exit(1);
+	}
+
 	if (errno) {
 		perror("strtoull");
 		exit(errno);

@@ -55,9 +63,9 @@
 			perror("mmap()");
 			exit(5);
 		} else {
+			memset(buf, 0, 16384);
+			memset(buf+o/2, 0, 16384);
+			memset(buf+o-16384, 0, 16384);
 			munmap(buf, o);
 		}
 
-Eric
author
#5 lore raw
Quoted 14 lines · Eric Sandeen
Quoted 11 lines · David Chinner
Quoted 4 lines · Eric Sandeen

yeah, figured it had something to do w/ the ia64 weenies when I saw the 32 vs. 8, factor of 4 ... fine, fine, my bad. :)

No, not your bad. Mine if anyones because I wrote the test.

Cheers,

Quoted 8 lines
Quoted 3 lines

Dave.

Well, I was going to look at it more closely before I sent it off :)

What do you think of a patch like this, to munmap 16k chunks regardless of page size:

Er, bad patch-ninja-boy, don't hand-edit :)

unwritten_mmap.c renamed from unwritten_mmap.c.orig +11 -3
@@ -12,6 +12,7 @@
  */
 int main(int argc, char **argv) {
 	unsigned long long o;
+	int minsize;
 	int fd, i;
 	struct xfs_flock64 space;
 	unsigned char *buf;

@@ -23,6 +24,13 @@
 
 	errno = 0;
 	o = strtoull(argv[1], NULL, 0);
+
+	minsize = 3*16384; /* 3 ia64 pages */
+	if (o < minsize) {
+		fprintf(stderr, "count must be at least %d\n", minsize);
+		exit(1);
+	}
+
 	if (errno) {
 		perror("strtoull");
 		exit(errno);

@@ -55,9 +63,9 @@
 			perror("mmap()");
 			exit(5);
 		} else {
-			buf[o-1] = 0;
-			buf[o/2] = 0;
-			buf[0] = 0;
+			memset(buf, 0, 16384);
+			memset(buf+o/2, 0, 16384);
+			memset(buf+o-16384, 0, 16384);
 			munmap(buf, o);
 		}
#6 lore raw
In reply to Eric Sandeen
Quoted 10 lines · Eric Sandeen
Quoted 8 lines · David Chinner
Quoted 3 lines · Eric Sandeen

yeah, figured it had something to do w/ the ia64 weenies when I saw the 32 vs. 8, factor of 4 ... fine, fine, my bad. :)

No, not your bad. Mine if anyones because I wrote the test.

Quoted 8 lines
Quoted 4 lines

Cheers,

Dave.

Well, I was going to look at it more closely before I sent it off :)

What do you think of a patch like this, to munmap 16k chunks regardless of page size:

That could be done, but will break on 64k page size machines ;)

Perhaps the filter needs to use getconf PAGE_SIZE to filter the output down:

ia64:

% getconf PAGE_SIZE
16384

x86_64:

% getconf PAGE_SIZE
4096

So the written extent sizes are output as a multiple of page size rather than basic blocks....

Cheers,

Dave.

Signature · 3 lines
Dave Chinner
Principal Engineer
SGI Australian Software Group
author
#7 lore raw
Quoted 10 lines · David Chinner

That could be done, but will break on 64k page size machines ;)

Perhaps the filter needs to use getconf PAGE_SIZE to filter the output down:

ia64:

% getconf PAGE_SIZE
16384

Quoted 7 lines

x86_64:

% getconf PAGE_SIZE
4096

So the written extent sizes are output as a multiple of page size rather than basic blocks....

yeah, computing all the numbers for the filter seemed nasty.

Maybe just a dumbed-down filter that expects

written
unwritten
written
unwritten
written

would be sufficient for this test.

166 2007-08-07 23:21:11.111409407 -0500 renamed from 166.orig 2007-08-07 23:19:33.391317217 -0500 +2 -2
@@ -31,9 +31,9 @@
 {
        $AWK_PROG '/[0-9]/ {
                if ($7)
-                       print $1, $2, "XX..YY", "AG", "(AA..BB)", $6, $7;
+                       print $7
                else
-                       print $1, $2, "XX..YY", "AG", "(AA..BB)", $6;
+                       print "no flags"
        }'
 }
166.out 2007-08-07 23:21:45.706131763 -0500 renamed from 166.out.orig 2007-08-07 23:21:25.299244961 -0500 +5 -5
@@ -1,6 +1,6 @@
 QA output created by 166
-0: [0..31]: XX..YY AG (AA..BB) 32
-1: [32..127]: XX..YY AG (AA..BB) 96 10000
-2: [128..159]: XX..YY AG (AA..BB) 32
-3: [160..223]: XX..YY AG (AA..BB) 64 10000
-4: [224..255]: XX..YY AG (AA..BB) 32
+no flags
+10000
+no flags
+10000
+no flags

-Eric

#8 lore raw
Quoted 55 lines · Eric Sandeen
Quoted 10 lines · David Chinner

That could be done, but will break on 64k page size machines ;)

Perhaps the filter needs to use getconf PAGE_SIZE to filter the output down:

ia64:

% getconf PAGE_SIZE
16384

Quoted 7 lines

x86_64:

% getconf PAGE_SIZE
4096

So the written extent sizes are output as a multiple of page size rather than basic blocks....

yeah, computing all the numbers for the filter seemed nasty.

Maybe just a dumbed-down filter that expects

written
unwritten
written
unwritten
written

would be sufficient for this test.

166 2007-08-07 23:21:11.111409407 -0500 renamed from 166.orig 2007-08-07 23:19:33.391317217 -0500 +2 -2
@@ -31,9 +31,9 @@
 {
        $AWK_PROG '/[0-9]/ {
                if ($7)
-                       print $1, $2, "XX..YY", "AG", "(AA..BB)", $6, $7;
+                       print $7
                else
-                       print $1, $2, "XX..YY", "AG", "(AA..BB)", $6;
+                       print "no flags"
        }'
 }
166.out 2007-08-07 23:21:45.706131763 -0500 renamed from 166.out.orig 2007-08-07 23:21:25.299244961 -0500 +0 -3
@@ -1,6 +1,6 @@
 QA output created by 166
-0: [0..31]: XX..YY AG (AA..BB) 32
-1: [32..127]: XX..YY AG (AA..BB) 96 10000
-2: [128..159]: XX..YY AG (AA..BB) 32

166.out 2007-08-07 23:21:45.706131763 -0500

Quoted 10 lines
166.out 2007-08-07 23:21:45.706131763 -0500 renamed from 166.out.orig 2007-08-07 23:21:25.299244961 -0500 +5 -3
-3: [160..223]: XX..YY AG (AA..BB) 64 10000
-4: [224..255]: XX..YY AG (AA..BB) 32
+no flags
+10000
+no flags
+10000
+no flags
 
 
-Eric

Instead of "no flags", how about "00000" so the test doesn't break with xfsprogs 2.9.x and later.

Quoted 1 line
author
#9 lore raw
Quoted 4 lines · Barry Naujok

Instead of "no flags", how about "00000" so the test doesn't break with xfsprogs 2.9.x and later.

[root@inode xfstests]# rpm -q xfsprogs
xfsprogs-2.9.3-1

I have that... but... whatever works :)

-Eric

#10 lore raw
In reply to David Chinner
Quoted 5 lines · David Chinner
Quoted 2 lines · Eric Sandeen

yeah, figured it had something to do w/ the ia64 weenies when I saw the 32 vs. 8, factor of 4 ... fine, fine, my bad. :)

No, not your bad. Mine if anyones because I wrote the test.

Ia64 supports 4k, 8k, 16k and 64k base pagesize. Time to add a little script to make the pagesize random for your kernel builds?

#11 lore raw
Quoted 9 lines · Christoph Hellwig
Quoted 5 lines · David Chinner
Quoted 2 lines · Eric Sandeen

yeah, figured it had something to do w/ the ia64 weenies when I saw the 32 vs. 8, factor of 4 ... fine, fine, my bad. :)

No, not your bad. Mine if anyones because I wrote the test.

Ia64 supports 4k, 8k, 16k and 64k base pagesize. Time to add a little script to make the pagesize random for your kernel builds?

Interesting idea. Definitely worth considering for the randomised mkfs/mount option config QA runs we already do....

Cheers,

Dave.

Signature · 3 lines
Dave Chinner
Principal Engineer
SGI Australian Software Group
#12 lore raw
In reply to David Chinner
Quoted 7 lines

From: [email protected]
[mailto:[email protected]]On Behalf Of
David Chinner
Sent: Tuesday, August 07, 2007 9:56 PM
To: Eric Sandeen
Cc: xfs-oss
Subject: Re: qa 166 failure on f8 kernel

...

Quoted 6 lines

mmap dirties entire pages. page size differs between platforms - ia64 = 16k, x86 = 4k - so the size of the extent allocated is different. Guess what platform I wrote the test on and use as my primary platform?

The output needs better filtering, methinks.

Could the test be changed so the output is pagesize independent rather than filtering the output? Or maybe the test should use a buffer size that's the LCM of all supported page sizes.

I'm not actually familiar with the test, so forgive me if I'm way off...

-Alex

A message that is not in the archive (sent off-list or not yet received): D59801BCA402F5418C040937230AF7E00B9A8E@mtv-amer002e--3.americas.sgi.c om

#14 lore raw
Quoted 13 lines
Quoted 7 lines

From: [email protected]
[mailto:[email protected]]On Behalf Of
David Chinner
Sent: Tuesday, August 07, 2007 9:56 PM
To: Eric Sandeen
Cc: xfs-oss
Subject: Re: qa 166 failure on f8 kernel

...

Quoted 5 lines

mmap dirties entire pages. page size differs between platforms - ia64 = 16k, x86 = 4k - so the size of the extent allocated is different. Guess what platform I wrote the test on and use as my primary platform?

Quoted 6 lines
Quoted 1 line

The output needs better filtering, methinks.

Could the test be changed so the output is pagesize independent rather than filtering the output? Or maybe the test should use a buffer size that's the LCM of all supported page sizes.

FWIW, xfs-cmds/xfstests/src/feature.c will tell you the page size (-s); with that you can accurately filter the output and not lose any info.

cheers.

--
Nathan

#15 lore raw
In reply to Eric Sandeen
Quoted 9 lines · Eric Sandeen

Posting this just in case it rings any bells, though I plan to investigate...

[root@inode xfstests]# ./check 166

FSTYP         -- xfs (non-debug)
PLATFORM      -- Linux/x86_64 inode 2.6.23-0.71.rc2.fc8
MKFS_OPTIONS  -- -f -bsize=4096 /dev/sdb6

MOUNT_OPTIONS -- /dev/sdb6 /mnt/sdb6

Quoted 9 lines

166 - output mismatch (see 166.out.bad)
2,6c2,7
< 0: [0..31]: XX..YY AG (AA..BB) 32
< 1: [32..127]: XX..YY AG (AA..BB) 96 10000
< 2: [128..159]: XX..YY AG (AA..BB) 32
< 3: [160..223]: XX..YY AG (AA..BB) 64 10000
< 4: [224..255]: XX..YY AG (AA..BB) 32
---

Quoted 1 line

/mnt/sdb6/test_file: XX..YY AG (AA..BB)

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

I suspect this line is appearing in the output due to a number in the mountpoint. I think the _filter_blocks() is only extracting lines with numbers in it.

Quoted 14 lines
Quoted 5 lines

0: [0..7]: XX..YY AG (AA..BB) 8
1: [8..127]: XX..YY AG (AA..BB) 120 10000 2: [128..135]: XX..YY AG (AA..BB) 8 3: [136..247]: XX..YY AG (AA..BB) 112 10000 4: [248..255]: XX..YY AG (AA..BB) 8

Failures: 166
Failed 1 of 1 tests
[root@inode xfstests]# uname -a
Linux inode.lab.msp.redhat.com 2.6.23-0.71.rc2.fc8 #1 SMP Sat Aug 4 01:21:06 EDT 2007 x86_64 x86_64 x86_64 GNU/Linux

-Eric

author
#16 lore raw
Quoted 20 lines · Barry Naujok
Quoted 17 lines · Eric Sandeen

Posting this just in case it rings any bells, though I plan to investigate...

[root@inode xfstests]# ./check 166

FSTYP         -- xfs (non-debug)
PLATFORM      -- Linux/x86_64 inode 2.6.23-0.71.rc2.fc8
MKFS_OPTIONS  -- -f -bsize=4096 /dev/sdb6

MOUNT_OPTIONS -- /dev/sdb6 /mnt/sdb6

166 - output mismatch (see 166.out.bad)
2,6c2,7
< 0: [0..31]: XX..YY AG (AA..BB) 32
< 1: [32..127]: XX..YY AG (AA..BB) 96 10000
< 2: [128..159]: XX..YY AG (AA..BB) 32
< 3: [160..223]: XX..YY AG (AA..BB) 64 10000
< 4: [224..255]: XX..YY AG (AA..BB) 32
---

Quoted 6 lines
Quoted 1 line
Quoted 1 line

/mnt/sdb6/test_file: XX..YY AG (AA..BB)

    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

I suspect this line is appearing in the output due to a number in the mountpoint. I think the _filter_blocks() is only extracting lines with numbers in it.

Bingo. Yep, changing it to /mnt/foo, plus my change to the test, passes. thanks!

-Eric