status of userspace release

We're working toward a userspace release this month. There are several patches that need to go in first, including backing out the xfsdump format version bump from Eric, fixes for the makefiles from Mike, and the Polish language update for xfsdump from Jakub. If anyone knows of something else we need, now is the time to flame about it. I will take a look around for other important patches too.

This time I'm going to tag an -rc1 (probably later today or tomorrow). We'll give everyone a few working days to do a final test and/or pipe up if we have missed something important. Then if all goes well we'll cut the release next Tuesday.

Hi,

Quoted 11 lines · Ben Myers

We're working toward a userspace release this month. There are several patches that need to go in first, including backing out the xfsdump format version bump from Eric, fixes for the makefiles from Mike, and the Polish language update for xfsdump from Jakub. If anyone knows of something else we need, now is the time to flame about it. I will take a look around for other important patches too.

This time I'm going to tag an -rc1 (probably later today or tomorrow). We'll give everyone a few working days to do a final test and/or pipe up if we have missed something important. Then if all goes well we'll cut the release next Tuesday.

I've tagged -rc1 for the upcoming releases of dmapi, xfsprogs, and xfsdump. If we missed something important now is the time to speak up.

Currently there are two items of which I'm aware:

1) the Polish (and German) translations of xfsdump are not working. I'll start a separate thread for that issue.

2) Christoph has pointed out this patch series:

http://oss.sgi.com/archives/xfs/2012-05/msg00323.html
http://oss.sgi.com/archives/xfs/2012-10/msg00541.html

Quoted 20 lines · Ben Myers

Hi,

Quoted 11 lines · Ben Myers

We're working toward a userspace release this month. There are several patches that need to go in first, including backing out the xfsdump format version bump from Eric, fixes for the makefiles from Mike, and the Polish language update for xfsdump from Jakub. If anyone knows of something else we need, now is the time to flame about it. I will take a look around for other important patches too.

This time I'm going to tag an -rc1 (probably later today or tomorrow). We'll give everyone a few working days to do a final test and/or pipe up if we have missed something important. Then if all goes well we'll cut the release next Tuesday.

I've tagged -rc1 for the upcoming releases of dmapi, xfsprogs, and xfsdump. If we missed something important now is the time to speak up.

Currently there are two items of which I'm aware:

Quoted 7 lines

1) the Polish (and German) translations of xfsdump are not working. I'll start a separate thread for that issue.

2) Christoph has pointed out this patch series:

http://oss.sgi.com/archives/xfs/2012-05/msg00323.html
http://oss.sgi.com/archives/xfs/2012-10/msg00541.html

I agree with christoph - the kernel side intptr/uintptr stuff that Jan did needs to go into the kernel first, and then brought to libxfs via a kernel code sync. A kernel code resync needs to be done immediately after the xfsprogs release so that usrspace is using the same base code for the CRC work, so splitting them to before/after the relases makes sense IMO....

Hi Folks,

We're working toward a userspace release this month. There are several patches that need to go in first, including backing out the xfsdump format version bump from Eric, fixes for the makefiles from Mike, and the Polish language update for xfsdump from Jakub. If anyone knows of something else we need, now is the time to flame about it. I will take a look around for other important patches too.

This time I'm going to tag an -rc1 (probably later today or tomorrow). We'll give everyone a few working days to do a final test and/or pipe up if we have missed something important. Then if all goes well we'll cut the release next Tuesday.

Quoted 9 lines · Ben Myers

Hi Folks,

We're working toward a userspace release this month. There are several patches that need to go in first, including backing out the xfsdump format version bump from Eric, fixes for the makefiles from Mike, and the Polish language update for xfsdump from Jakub. If anyone knows of something else we need, now is the time to flame about it. I will take a look around for other important patches too.

Quoted 4 lines

This time I'm going to tag an -rc1 (probably later today or tomorrow). We'll give everyone a few working days to do a final test and/or pipe up if we have missed something important. Then if all goes well we'll cut the release next Tuesday.

I think that dump/restore need more work/testing.

I've already pointed Eric to the header checksum failures (forkoff patch being needed), and that fixes the failures I've been seeing on normal xfstests runs.

Running some large filesystem testing, however, I see more problems. I'm using a 17TB filesytsem and the --largefs patch series. This results in a futex hang in 059 like so:

[ 4770.007858] xfsrestore      S ffff88021fc52d40  5504  3926   3487 0x00000000
[ 4770.007858]  ffff880212ea9c68 0000000000000082 ffff880207830140 ffff880212ea9fd8
[ 4770.007858]  ffff880212ea9fd8 ffff880212ea9fd8 ffff880216cec2c0 ffff880207830140
[ 4770.007858]  ffff880212ea9d08 ffff880212ea9d58 ffff880207830140 0000000000000000
[ 4770.007858] Call Trace:
[ 4770.007858]  [<ffffffff81b8a009>] schedule+0x29/0x70
[ 4770.007858]  [<ffffffff810db089>] futex_wait_queue_me+0xc9/0x100
[ 4770.007858]  [<ffffffff810db809>] futex_wait+0x189/0x290
[ 4770.007858]  [<ffffffff8113acf7>] ? __free_pages+0x47/0x70
[ 4770.007858]  [<ffffffff810dd41c>] do_futex+0x11c/0xa80
[ 4770.007858]  [<ffffffff810abbd5>] ? hrtimer_try_to_cancel+0x55/0x110
[ 4770.007858]  [<ffffffff810abcb2>] ? hrtimer_cancel+0x22/0x30
[ 4770.007858]  [<ffffffff81b88f44>] ? do_nanosleep+0xa4/0xd0
[ 4770.007858]  [<ffffffff810dde0d>] sys_futex+0x8d/0x1b0
[ 4770.007858]  [<ffffffff810ab6e0>] ? update_rmtp+0x80/0x80
[ 4770.007858]  [<ffffffff81b93a99>] system_call_fastpath+0x16/0x1b
[ 4770.007858] xfsrestore      S ffff88021fc52d40  5656  3927   3487 0x00000000
[ 4770.007858]  ffff880208f29c68 0000000000000082 ffff880208f84180 ffff880208f29fd8
[ 4770.007858]  ffff880208f29fd8 ffff880208f29fd8 ffff880216cec2c0 ffff880208f84180
[ 4770.007858]  ffff880208f29d08 ffff880208f29d58 ffff880208f84180 0000000000000000
[ 4770.007858] Call Trace:
[ 4770.007858]  [<ffffffff81b8a009>] schedule+0x29/0x70
[ 4770.007858]  [<ffffffff810db089>] futex_wait_queue_me+0xc9/0x100
[ 4770.007858]  [<ffffffff810db809>] futex_wait+0x189/0x290
[ 4770.007858]  [<ffffffff810dd41c>] do_futex+0x11c/0xa80
[ 4770.007858]  [<ffffffff810abbd5>] ? hrtimer_try_to_cancel+0x55/0x110
[ 4770.007858]  [<ffffffff810abcb2>] ? hrtimer_cancel+0x22/0x30
[ 4770.007858]  [<ffffffff81b88f44>] ? do_nanosleep+0xa4/0xd0
[ 4770.007858]  [<ffffffff810dde0d>] sys_futex+0x8d/0x1b0
[ 4770.007858]  [<ffffffff810ab6e0>] ? update_rmtp+0x80/0x80
[ 4770.007858]  [<ffffffff81b93a99>] system_call_fastpath+0x16/0x1b
[ 4770.007858] xfsrestore      S ffff88021fc92d40  5848  3928   3487 0x00000000
[ 4770.007858]  ffff880212d0dc68 0000000000000082 ffff880208e76240 ffff880212d0dfd8
[ 4770.007858]  ffff880212d0dfd8 ffff880212d0dfd8 ffff880216cf2300 ffff880208e76240
[ 4770.007858]  ffff880212d0dd08 ffff880212d0dd58 ffff880208e76240 0000000000000000
[ 4770.007858] Call Trace:
[ 4770.007858]  [<ffffffff81b8a009>] schedule+0x29/0x70
[ 4770.007858]  [<ffffffff810db089>] futex_wait_queue_me+0xc9/0x100
[ 4770.007858]  [<ffffffff810db809>] futex_wait+0x189/0x290
[ 4770.007858]  [<ffffffff810dd41c>] do_futex+0x11c/0xa80
[ 4770.007858]  [<ffffffff810abbd5>] ? hrtimer_try_to_cancel+0x55/0x110
[ 4770.007858]  [<ffffffff810abcb2>] ? hrtimer_cancel+0x22/0x30
[ 4770.007858]  [<ffffffff81b88f44>] ? do_nanosleep+0xa4/0xd0
[ 4770.007858]  [<ffffffff810dde0d>] sys_futex+0x8d/0x1b0
[ 4770.007858]  [<ffffffff810ab6e0>] ? update_rmtp+0x80/0x80
[ 4770.007858]  [<ffffffff81b93a99>] system_call_fastpath+0x16/0x1b

I can't reliably reproduce it at this point, but there does appear to be some kind of locking problem in the multistream support.

Speaking of which, most large filesystems dump/restore tests are failing because of this output:

026 20s ... - output mismatch (see 026.out.bad)

026.out.bad 2012-11-02 16:20:17.000000000 +1100 renamed from 026.out 2012-10-05 11:37:51.000000000 +1000 +2 -0

@@ -20,6 +20,7 @@
 xfsdump: media file size NUM bytes
 xfsdump: dump size (non-dir files) : NUM bytes
 xfsdump: dump complete: SECS seconds elapsed
+xfsdump:   stream 0 DUMP_FILE OK (success)
 xfsdump: Dump Status: SUCCESS
 Restoring from file...
 xfsrestore  -f DUMP_FILE  -L stress_026 RESTORE_DIR

@@ -32,6 +33,7 @@
 xfsrestore: directory post-processing
 xfsrestore: restoring non-directory files
 xfsrestore: restore complete: SECS seconds elapsed
+xfsrestore:   stream 0 DUMP_FILE OK (success)
 xfsrestore: Restore Status: SUCCESS
 Comparing dump directory with restore directory
 Files DUMP_DIR/big and RESTORE_DIR/DUMP_SUBDIR/big are identical

Which looks like output from the multistream code. Why it is emitting this for large filesystem testing and not for small filesystems, I'm not sure yet.

In fact, with --largefs, I see this for the dump group:

Failures: 026 028 046 047 056 059 060 061 063 064 065 066 266 281 282 283
Failed 16 of 19 tests

And this for the normal sized (10GB) scratch device:

Passed all 18 tests

So there's something funky going on here....

Hi Dave,

Quoted 10 lines · Dave Chinner

Quoted 9 lines · Ben Myers

Hi Folks,

We're working toward a userspace release this month. There are several patches that need to go in first, including backing out the xfsdump format version bump from Eric, fixes for the makefiles from Mike, and the Polish language update for xfsdump from Jakub. If anyone knows of something else we need, now is the time to flame about it. I will take a look around for other important patches too.

Quoted 6 lines

Quoted 4 lines

This time I'm going to tag an -rc1 (probably later today or tomorrow). We'll give everyone a few working days to do a final test and/or pipe up if we have missed something important. Then if all goes well we'll cut the release next Tuesday.

I think that dump/restore need more work/testing.

Sounds good. AFAIK there is no blazing hurry to release immediately.

Quoted 3 lines

Running some large filesystem testing, however, I see more problems. I'm using a 17TB filesytsem and the --largefs patch series. This results in a futex hang in 059 like so:

Quoted 2 lines

I can't reliably reproduce it at this point, but there does appear to be some kind of locking problem in the multistream support.

One of my machines hit this overnight without --largefs. I wasn't able to get a dump though. Just another data point.

Quoted 26 lines

Speaking of which, most large filesystems dump/restore tests are failing because of this output:

026 20s ... - output mismatch (see 026.out.bad)

026.out.bad 2012-11-02 16:20:17.000000000 +1100 renamed from 026.out 2012-10-05 11:37:51.000000000 +1000 +2 -0

@@ -20,6 +20,7 @@
 xfsdump: media file size NUM bytes
 xfsdump: dump size (non-dir files) : NUM bytes
 xfsdump: dump complete: SECS seconds elapsed
+xfsdump:   stream 0 DUMP_FILE OK (success)
 xfsdump: Dump Status: SUCCESS
 Restoring from file...
 xfsrestore  -f DUMP_FILE  -L stress_026 RESTORE_DIR

@@ -32,6 +33,7 @@
 xfsrestore: directory post-processing
 xfsrestore: restoring non-directory files
 xfsrestore: restore complete: SECS seconds elapsed
+xfsrestore:   stream 0 DUMP_FILE OK (success)
 xfsrestore: Restore Status: SUCCESS
 Comparing dump directory with restore directory
 Files DUMP_DIR/big and RESTORE_DIR/DUMP_SUBDIR/big are identical

Which looks like output from the multistream code. Why it is emitting this for large filesystem testing and not for small filesystems, I'm not sure yet.

Rich also reported some golden output related changes with --largefs awhile back. I don't think he saw this one though.

The TODO list for userspace release currently stands at:

1) fix the header checksum failures... which is resolved
2) fix a futex hang in 059
3) fix the golden output changes related to multistream support in xfsdump and --largefs

4) test on more platforms

Quoted 12 lines · Ben Myers

Hi Dave,

Quoted 9 lines · Dave Chinner

Quoted 8 lines · Ben Myers

Hi Folks,

We're working toward a userspace release this month. There are several patches that need to go in first, including backing out the xfsdump format version bump from Eric, fixes for the makefiles from Mike, and the Polish language update for xfsdump from Jakub. If anyone knows of something else we need, now is the time to flame about it. I will take a look around for other important patches too.

Quoted 6 lines

The TODO list for userspace release currently stands at:

1) fix the header checksum failures... which is resolved
2) fix a futex hang in 059
3) fix the golden output changes related to multistream support in xfsdump and --largefs

Well, understand them first, then fix ;)

Quoted 1 line

4) test on more platforms

Quoted 13 lines · Dave Chinner

Quoted 12 lines · Ben Myers

Hi Dave,

Quoted 9 lines · Dave Chinner

Quoted 8 lines · Ben Myers

Hi Folks,

We're working toward a userspace release this month. There are several patches that need to go in first, including backing out the xfsdump format version bump from Eric, fixes for the makefiles from Mike, and the Polish language update for xfsdump from Jakub. If anyone knows of something else we need, now is the time to flame about it. I will take a look around for other important patches too.

....

Quoted 10 lines

Quoted 6 lines

The TODO list for userspace release currently stands at:

1) fix the header checksum failures... which is resolved
2) fix a futex hang in 059
3) fix the golden output changes related to multistream support in xfsdump and --largefs

Well, understand them first, then fix ;)

Quoted 1 line

4) test on more platforms

Another:

$ sudo xfs_info /mnt/scratch/
meta-data=/dev/vdc               isize=256    agcount=4, agsize=12800 blks
         =                       sectsz=512   attr=2
data     =                       bsize=4096   blocks=51200, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0
log      =internal               bsize=4096   blocks=1200, version=2
         =                       sectsz=512   sunit=0 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0
$ sudo xfs_db -r -c "sb 0" -c "version" /dev/vdc
versionnum [0xb4a4+0x8a] = V4,NLINK,ALIGN,DIRV2,LOGV2,EXTFLG,MOREBITS,ATTR2,LAZYSBCOUNT,PROJID32BIT
$

xfs_info is not reporting the 32 bit project ID status.

Yes, I know this requires the XFS_IOC_FSGEOM support for it, but I'd like it this release to at least say "off or unknown" here.

I say this, because this is the first thing I noticed when having a look at a test 287 failure:

7 10s ... - output mismatch (see 287.out.bad)

287.out.bad 2012-11-03 10:55:15.000000000 +1100 renamed from 287.out 2012-10-05 11:38:08.000000000 +1000 +12 -10

@@ -2,22 +2,24 @@
 No 32bit project quotas:
 projid = 1234
 projid = 0
+xfs_quota: cannot set project on /mnt/scratch/pquota/32bit: Invalid argument
 With 32bit project quota support:
 projid = 1234
-projid = 2123456789
+projid = 0
+xfs_quota: cannot set project on /mnt/scratch/restore/pquota/32bitv2: Invalid argument
 The restored file system + one additional file:
 projid = 1234
-projid = 2123456789
-projid = 2123456789
+projid = 0
+projid = 0
 These two values of 16bit project quota ids shall be the same
-core.projid_lo = 1234
+core.projid_lo = 0
 core.projid_hi = 0
 core.projid_lo = 1234
 core.projid_hi = 0
 These three values of 32bit project quota ids shall be the same
-core.projid_lo = 24853
-core.projid_hi = 32401
-core.projid_lo = 24853
-core.projid_hi = 32401
-core.projid_lo = 24853
-core.projid_hi = 32401
+core.projid_lo = 0
+core.projid_hi = 0
+core.projid_lo = 0
+core.projid_hi = 0
+core.projid_lo = 0
+core.projid_hi = 0

Here's what's curious - this is failing on the 17TB filesystem, but is not failing on 10-20GB filesystems. There seems to be a pattern here....

Note that I only recently updated xfstests on the VM with the 17TB filesystem (i.e. on wednesday), so this is probably the first time I have run test 287 on a large filesystem like this. Same goes for much of the other problems I'm reporting - xfstests on this machine has been running out of dev branch I hadn't updated for a while, so these problems might have been around for a while on large filesystems...

Cheers,

Dave.

Quoted 6 lines · Ben Myers

The TODO list for userspace release currently stands at:

1) fix the header checksum failures... which is resolved
2) fix a futex hang in 059
3) fix the golden output changes related to multistream support in xfsdump and --largefs

Well, understand them first, then fix ;)

Quoted 41 lines · Dave Chinner

Quoted 13 lines · Dave Chinner

Quoted 12 lines · Ben Myers

Hi Dave,

Quoted 9 lines · Dave Chinner

Quoted 8 lines · Ben Myers

Hi Folks,

We're working toward a userspace release this month. There are several patches that need to go in first, including backing out the xfsdump format version bump from Eric, fixes for the makefiles from Mike, and the Polish language update for xfsdump from Jakub. If anyone knows of something else we need, now is the time to flame about it. I will take a look around for other important patches too.

....

Quoted 10 lines

Quoted 6 lines

The TODO list for userspace release currently stands at:

1) fix the header checksum failures... which is resolved
2) fix a futex hang in 059
3) fix the golden output changes related to multistream support in xfsdump and --largefs

Well, understand them first, then fix ;)

Quoted 1 line

4) test on more platforms

Another:

$ sudo xfs_info /mnt/scratch/
meta-data=/dev/vdc               isize=256    agcount=4, agsize=12800 blks
         =                       sectsz=512   attr=2
data     =                       bsize=4096   blocks=51200, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0
log      =internal               bsize=4096   blocks=1200, version=2
         =                       sectsz=512   sunit=0 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0
$ sudo xfs_db -r -c "sb 0" -c "version" /dev/vdc
versionnum [0xb4a4+0x8a] = V4,NLINK,ALIGN,DIRV2,LOGV2,EXTFLG,MOREBITS,ATTR2,LAZYSBCOUNT,PROJID32BIT
$

Quoted 1 line

xfs_info is not reporting the 32 bit project ID status.

Weird, I didn't realize that
[PATCH 2/2] xfsprogs: report projid32 status in growfs output hadn't been pulled in.

Quoted 2 lines

Yes, I know this requires the XFS_IOC_FSGEOM support for it, but I'd like it this release to at least say "off or unknown" here.

Heh, ok, when you reviewed you said it was no big deal ;) but I guess we can add the "or unknown" if you like.

Quoted 2 lines

I say this, because this is the first thing I noticed when having a look at a test 287 failure:

Hm that's pretty odd.

I say this, because this is the first thing I noticed when having a look at a test 287 failure:

7 10s ... - output mismatch (see 287.out.bad)

287.out.bad 2012-11-03 10:55:15.000000000 +1100 renamed from 287.out 2012-10-05 11:38:08.000000000 +1000 +12 -10

@@ -2,22 +2,24 @@
 No 32bit project quotas:
 projid = 1234
 projid = 0
+xfs_quota: cannot set project on /mnt/scratch/pquota/32bit: Invalid argument
 With 32bit project quota support:
 projid = 1234
-projid = 2123456789
+projid = 0
+xfs_quota: cannot set project on /mnt/scratch/restore/pquota/32bitv2: Invalid argument
 The restored file system + one additional file:
 projid = 1234
-projid = 2123456789
-projid = 2123456789
+projid = 0
+projid = 0
 These two values of 16bit project quota ids shall be the same
-core.projid_lo = 1234
+core.projid_lo = 0
 core.projid_hi = 0
 core.projid_lo = 1234
 core.projid_hi = 0
 These three values of 32bit project quota ids shall be the same
-core.projid_lo = 24853
-core.projid_hi = 32401
-core.projid_lo = 24853
-core.projid_hi = 32401
-core.projid_lo = 24853
-core.projid_hi = 32401
+core.projid_lo = 0
+core.projid_hi = 0
+core.projid_lo = 0
+core.projid_hi = 0
+core.projid_lo = 0
+core.projid_hi = 0

Here's what's curious - this is failing on the 17TB filesystem, but is not failing on 10-20GB filesystems. There seems to be a pattern here....