Skip to contentSkip to conversation

fstests: sector size fixes and whiteouts...

PATCH In review Dave Chinner 3 participants 25 messages Feb 24, 2015 – Mar 2, 2015

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

v2 cover fstests: sector size fixes and whiteouts...

Hi folks,

Version 2 of the patchset I sent yesterday here:

http://thread.gmane.org/gmane.comp.file-systems.fstests/293

V2:
- log size too small for other tests as well (e.g. xfs/297) so added them to patch 1/4
- realised I'd only updated the external log file output for xfs/096 in patch 2/4, so fixed that and updated the filtering to handle differences in log version as crc enabled ffilesystems override the log version given on the CLI.

#1 lore raw

PATCH v2 1/4 xfs/104: log size too small for 4k sector drives

From: Dave Chinner <[email protected]>

xfs/104, xfs/119, xfs/291 and xfs/297 have small fixed log sizes. A recent change to the kernel ramdisk changed it's physical sector size from 512B to 4kB, and this results in mkfs calculating a log size larger than the fixed test size and hence the tests fail.

Change the log size to a larger size that works with 4k sectors, and also increase the size of the filesystem being created so that the amount of data space in the filesystem does not change and hence does not perturb the rest of the test.

---
 tests/xfs/104     |  8 ++++----
 tests/xfs/104.out | 46 +++++++++++++++++++++++-----------------------
 tests/xfs/119     |  2 +-
 tests/xfs/291     |  2 +-
 tests/xfs/297     |  2 +-
 5 files changed, 30 insertions(+), 30 deletions(-)
tests/xfs/104 +4 -4
index 69fcc69..ca2ae21 100755
@@ -81,10 +81,10 @@ modsize=`expr   4 \* $incsize`	# pause after this many increments
 [ `expr $endsize / $dbsize` -lt $dblocks ] || _notrun "Scratch device too small"
 
 nags=4
-size=`expr 120 \* 1048576`	# 120 megabytes initially
+size=`expr 125 \* 1048576`	# 120 megabytes initially
 sizeb=`expr $size / $dbsize`	# in data blocks
 echo "*** creating scratch filesystem"
-_create_scratch -lsize=5m -dsize=${size} -dagcount=${nags}
+_create_scratch -lsize=10m -dsize=${size} -dagcount=${nags}
 
 fillsize=`expr 110 \* 1048576`	# 110 megabytes of filling
 echo "*** using some initial space on scratch filesystem"
@@ -95,13 +95,13 @@ _fill_scratch $fillsize
 # Kick off more stress threads on each iteration, grow; repeat.
 #
 while [ $size -le $endsize ]; do
-	echo "*** stressing a ${size} byte filesystem"
+	echo "*** stressing filesystem"
 	echo "*** stressing a ${sizeb} block filesystem" >> $seqres.full
 	_stress_scratch
 	sleep 1
 	size=`expr $size + $incsize`
 	sizeb=`expr $size / $dbsize`	# in data blocks
-	echo "*** growing to a ${size} byte filesystem"
+	echo "*** growing filesystem"
 	echo "*** growing to a ${sizeb} block filesystem" >> $seqres.full
 	xfs_growfs -D ${sizeb} $SCRATCH_MNT \
 		| tee -a $seqres.full | _filter_mkfs 2>$tmp.growfs
tests/xfs/104.out +23 -23

113 lines not on this page · raw message

tests/xfs/119 +1 -1
index c7c46d9..490495b 100755
@@ -54,7 +54,7 @@ _require_scratch
 # this may hang
 sync
 
-export MKFS_OPTIONS="-l version=2,size=1200b,su=64k" 
+export MKFS_OPTIONS="-l version=2,size=2500b,su=64k" 
 export MOUNT_OPTIONS="-o logbsize=64k"
 _scratch_mkfs_xfs >/dev/null
 
tests/xfs/291 +1 -1
index fbf9c51..c226e65 100755
@@ -46,7 +46,7 @@ _supported_os IRIX Linux
 # real QA test starts here
 rm -f $seqres.full
 _require_scratch
-_scratch_mkfs_xfs -n size=16k -l size=5m -d size=128m >> $seqres.full 2>&1
+_scratch_mkfs_xfs -n size=16k -l size=10m -d size=133m >> $seqres.full 2>&1
 _scratch_mount
 
 # First we cause very badly fragmented freespace, then
tests/xfs/297 +1 -1
index 1cdbbb9..25b597e 100755
@@ -50,7 +50,7 @@ _require_scratch
 _require_freeze
 
 rm -f $seqres.full
-_scratch_mkfs_xfs -d agcount=16,su=256k,sw=12 -l su=256k,size=2560b >/dev/null 2>&1
+_scratch_mkfs_xfs -d agcount=16,su=256k,sw=12 -l su=256k,size=5120b >/dev/null 2>&1
 _scratch_mount >/dev/null 2>&1
 
 STRESS_DIR="$SCRATCH_MNT/testdir"
Signature · 1 line
2.0.0
#2 lore raw

Re: [PATCH 1/4] xfs/104: log size too small for 4k sector drives

Quoted 7 lines · Dave Chinner

From: Dave Chinner <[email protected]>

xfs/104, xfs/119, xfs/291 and xfs/297 have small fixed log sizes. A recent change to the kernel ramdisk changed it's physical sector size from 512B to 4kB, and this results in mkfs calculating a log size larger than the fixed test size and hence the tests fail.

Quoted 7 lines

Change the log size to a larger size that works with 4k sectors, and also increase the size of the filesystem being created so that the amount of data space in the filesystem does not change and hence does not perturb the rest of the test.

---

Well for some reason I can't mount a ramdisk on the current tot to test this. In fact, I can't mount _anything_ after the ramdisk mount attempt. The mount actually reports success too, but there's nothing there... :/

# modprobe brd
# mkfs.xfs -f /dev/ram0 
meta-data=/dev/ram0              isize=256    agcount=1, agsize=4096
blks
         =                       sectsz=4096  attr=2, projid32bit=1
         =                       crc=0        finobt=0
data     =                       bsize=4096   blocks=4096, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0 ftype=0
log      =internal log           bsize=4096   blocks=1605, version=2
         =                       sectsz=4096  sunit=1 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0
# mount /dev/ram0 /mnt/
# mount | grep mnt
# umount  /mnt/
umount: /mnt/: not mounted

... and then I can't even mount my normal scratch device until after a reboot:

# mount /dev/test/scratch /mnt/
# mount | grep mnt
# umount  /mnt/
umount: /mnt/: not mounted

I see some general flakiness with loop devices as well (re: patch 3/4). Anyways... until I get a chance to look at that, a couple nits to follow. Otherwise this looks Ok to me.

Quoted 12 lines
 tests/xfs/104     |  8 ++++----
 tests/xfs/104.out | 46 +++++++++++++++++++++++-----------------------
 tests/xfs/119     |  2 +-
 tests/xfs/291     |  2 +-
 tests/xfs/297     |  2 +-
 5 files changed, 30 insertions(+), 30 deletions(-)
tests/xfs/104 +0 -0
index 69fcc69..ca2ae21 100755

tests/xfs/104 modsize=`expr 4 \* $incsize` # pause after this many increments · line 84

Quoted 5 lines
tests/xfs/104 +1 -1
@@ -81,10 +81,10 @@ modsize=`expr   4 \* $incsize`	# pause after this many increments
 [ `expr $endsize / $dbsize` -lt $dblocks ] || _notrun "Scratch device too small"
 
 nags=4
-size=`expr 120 \* 1048576`	# 120 megabytes initially
+size=`expr 125 \* 1048576`	# 120 megabytes initially

The comment is wrong now.

Quoted 155 lines
tests/xfs/104 +3 -3
 sizeb=`expr $size / $dbsize`	# in data blocks
 echo "*** creating scratch filesystem"
-_create_scratch -lsize=5m -dsize=${size} -dagcount=${nags}
+_create_scratch -lsize=10m -dsize=${size} -dagcount=${nags}
 
 fillsize=`expr 110 \* 1048576`	# 110 megabytes of filling
 echo "*** using some initial space on scratch filesystem"
@@ -95,13 +95,13 @@ _fill_scratch $fillsize
 # Kick off more stress threads on each iteration, grow; repeat.
 #
 while [ $size -le $endsize ]; do
-	echo "*** stressing a ${size} byte filesystem"
+	echo "*** stressing filesystem"
 	echo "*** stressing a ${sizeb} block filesystem" >> $seqres.full
 	_stress_scratch
 	sleep 1
 	size=`expr $size + $incsize`
 	sizeb=`expr $size / $dbsize`	# in data blocks
-	echo "*** growing to a ${size} byte filesystem"
+	echo "*** growing filesystem"
 	echo "*** growing to a ${sizeb} block filesystem" >> $seqres.full
 	xfs_growfs -D ${sizeb} $SCRATCH_MNT \
 		| tee -a $seqres.full | _filter_mkfs 2>$tmp.growfs
tests/xfs/104.out +23 -23
index f237e5e..de6c7f2 100644
@@ -15,8 +15,8 @@ log      =LDEV bsize=XXX blocks=XXX
 realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
 *** mount
 *** using some initial space on scratch filesystem
-*** stressing a 125829120 byte filesystem
-*** growing to a 169869312 byte filesystem
+*** stressing filesystem
+*** growing filesystem
 meta-data=DDEV isize=XXX agcount=N, agsize=XXX blks
 data     = bsize=XXX blocks=XXX, imaxpct=PCT
          = sunit=XXX swidth=XXX, unwritten=X
@@ -25,8 +25,8 @@ log      =LDEV bsize=XXX blocks=XXX
 realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
 AGCOUNT=4
 
-*** stressing a 169869312 byte filesystem
-*** growing to a 213909504 byte filesystem
+*** stressing filesystem
+*** growing filesystem
 meta-data=DDEV isize=XXX agcount=N, agsize=XXX blks
 data     = bsize=XXX blocks=XXX, imaxpct=PCT
          = sunit=XXX swidth=XXX, unwritten=X
@@ -35,8 +35,8 @@ log      =LDEV bsize=XXX blocks=XXX
 realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
 AGCOUNT=6
 
-*** stressing a 213909504 byte filesystem
-*** growing to a 257949696 byte filesystem
+*** stressing filesystem
+*** growing filesystem
 meta-data=DDEV isize=XXX agcount=N, agsize=XXX blks
 data     = bsize=XXX blocks=XXX, imaxpct=PCT
          = sunit=XXX swidth=XXX, unwritten=X
@@ -45,8 +45,8 @@ log      =LDEV bsize=XXX blocks=XXX
 realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
 AGCOUNT=7
 
-*** stressing a 257949696 byte filesystem
-*** growing to a 301989888 byte filesystem
+*** stressing filesystem
+*** growing filesystem
 meta-data=DDEV isize=XXX agcount=N, agsize=XXX blks
 data     = bsize=XXX blocks=XXX, imaxpct=PCT
          = sunit=XXX swidth=XXX, unwritten=X
@@ -55,8 +55,8 @@ log      =LDEV bsize=XXX blocks=XXX
 realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
 AGCOUNT=9
 
-*** stressing a 301989888 byte filesystem
-*** growing to a 346030080 byte filesystem
+*** stressing filesystem
+*** growing filesystem
 meta-data=DDEV isize=XXX agcount=N, agsize=XXX blks
 data     = bsize=XXX blocks=XXX, imaxpct=PCT
          = sunit=XXX swidth=XXX, unwritten=X
@@ -65,8 +65,8 @@ log      =LDEV bsize=XXX blocks=XXX
 realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
 AGCOUNT=10
 
-*** stressing a 346030080 byte filesystem
-*** growing to a 390070272 byte filesystem
+*** stressing filesystem
+*** growing filesystem
 meta-data=DDEV isize=XXX agcount=N, agsize=XXX blks
 data     = bsize=XXX blocks=XXX, imaxpct=PCT
          = sunit=XXX swidth=XXX, unwritten=X
@@ -75,8 +75,8 @@ log      =LDEV bsize=XXX blocks=XXX
 realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
 AGCOUNT=11
 
-*** stressing a 390070272 byte filesystem
-*** growing to a 434110464 byte filesystem
+*** stressing filesystem
+*** growing filesystem
 meta-data=DDEV isize=XXX agcount=N, agsize=XXX blks
 data     = bsize=XXX blocks=XXX, imaxpct=PCT
          = sunit=XXX swidth=XXX, unwritten=X
@@ -85,8 +85,8 @@ log      =LDEV bsize=XXX blocks=XXX
 realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
 AGCOUNT=13
 
-*** stressing a 434110464 byte filesystem
-*** growing to a 478150656 byte filesystem
+*** stressing filesystem
+*** growing filesystem
 meta-data=DDEV isize=XXX agcount=N, agsize=XXX blks
 data     = bsize=XXX blocks=XXX, imaxpct=PCT
          = sunit=XXX swidth=XXX, unwritten=X
@@ -95,18 +95,18 @@ log      =LDEV bsize=XXX blocks=XXX
 realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
 AGCOUNT=14
 
-*** stressing a 478150656 byte filesystem
-*** growing to a 522190848 byte filesystem
+*** stressing filesystem
+*** growing filesystem
 meta-data=DDEV isize=XXX agcount=N, agsize=XXX blks
 data     = bsize=XXX blocks=XXX, imaxpct=PCT
          = sunit=XXX swidth=XXX, unwritten=X
 naming   =VERN bsize=XXX
 log      =LDEV bsize=XXX blocks=XXX
 realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
-AGCOUNT=16
+AGCOUNT=15
 
-*** stressing a 522190848 byte filesystem
-*** growing to a 566231040 byte filesystem
+*** stressing filesystem
+*** growing filesystem
 meta-data=DDEV isize=XXX agcount=N, agsize=XXX blks
 data     = bsize=XXX blocks=XXX, imaxpct=PCT
          = sunit=XXX swidth=XXX, unwritten=X
@@ -115,8 +115,8 @@ log      =LDEV bsize=XXX blocks=XXX
 realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
 AGCOUNT=17
 
-*** stressing a 566231040 byte filesystem
-*** growing to a 610271232 byte filesystem
+*** stressing filesystem
+*** growing filesystem
 meta-data=DDEV isize=XXX agcount=N, agsize=XXX blks
 data     = bsize=XXX blocks=XXX, imaxpct=PCT
          = sunit=XXX swidth=XXX, unwritten=X
tests/xfs/119 +0 -0
index c7c46d9..490495b 100755

tests/xfs/119 _require_scratch · line 57

Quoted 5 lines
tests/xfs/119 +1 -1
@@ -54,7 +54,7 @@ _require_scratch
 # this may hang
 sync
 
-export MKFS_OPTIONS="-l version=2,size=1200b,su=64k" 
+export MKFS_OPTIONS="-l version=2,size=2500b,su=64k" 

Trailing space here.

Brian

Quoted 36 lines
tests/xfs/119 +0 -0
 export MOUNT_OPTIONS="-o logbsize=64k"
 _scratch_mkfs_xfs >/dev/null
 
tests/xfs/291 +1 -1
index fbf9c51..c226e65 100755
@@ -46,7 +46,7 @@ _supported_os IRIX Linux
 # real QA test starts here
 rm -f $seqres.full
 _require_scratch
-_scratch_mkfs_xfs -n size=16k -l size=5m -d size=128m >> $seqres.full 2>&1
+_scratch_mkfs_xfs -n size=16k -l size=10m -d size=133m >> $seqres.full 2>&1
 _scratch_mount
 
 # First we cause very badly fragmented freespace, then
tests/xfs/297 +1 -1
index 1cdbbb9..25b597e 100755
@@ -50,7 +50,7 @@ _require_scratch
 _require_freeze
 
 rm -f $seqres.full
-_scratch_mkfs_xfs -d agcount=16,su=256k,sw=12 -l su=256k,size=2560b >/dev/null 2>&1
+_scratch_mkfs_xfs -d agcount=16,su=256k,sw=12 -l su=256k,size=5120b >/dev/null 2>&1
 _scratch_mount >/dev/null 2>&1
 
 STRESS_DIR="$SCRATCH_MNT/testdir"
Signature · 6 lines
2.0.0

--
To unsubscribe from this list: send the line "unsubscribe fstests" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#3 lore raw

ramdisk problems in 4.0-rc1? (was Re: [PATCH 1/4] xfs/104: log size too small for 4k sector drives)

[cc linux-fsdevel, Boaz and others]

Quoted 37 lines · Brian Foster
Quoted 7 lines · Dave Chinner

From: Dave Chinner <[email protected]>

xfs/104, xfs/119, xfs/291 and xfs/297 have small fixed log sizes. A recent change to the kernel ramdisk changed it's physical sector size from 512B to 4kB, and this results in mkfs calculating a log size larger than the fixed test size and hence the tests fail.

Quoted 7 lines

Change the log size to a larger size that works with 4k sectors, and also increase the size of the filesystem being created so that the amount of data space in the filesystem does not change and hence does not perturb the rest of the test.

---

Well for some reason I can't mount a ramdisk on the current tot to test this. In fact, I can't mount _anything_ after the ramdisk mount attempt. The mount actually reports success too, but there's nothing there... :/

# modprobe brd
# mkfs.xfs -f /dev/ram0 
meta-data=/dev/ram0              isize=256    agcount=1, agsize=4096
blks
         =                       sectsz=4096  attr=2, projid32bit=1
         =                       crc=0        finobt=0
data     =                       bsize=4096   blocks=4096, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0 ftype=0
log      =internal log           bsize=4096   blocks=1605, version=2
         =                       sectsz=4096  sunit=1 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0
# mount /dev/ram0 /mnt/
# mount | grep mnt
# umount  /mnt/
umount: /mnt/: not mounted
Quoted 7 lines

... and then I can't even mount my normal scratch device until after a reboot:

# mount /dev/test/scratch /mnt/
# mount | grep mnt
# umount  /mnt/
umount: /mnt/: not mounted

Ok, so that's just plain broken. What's in dmesg?

As it is, I'm seeing plenty of weirdness in 4.0-rc1 on ramdisks as well. Apart from the change to 4k physical sector size causing all sorts of chaos with xfstests results due to it changing mkfs.xfs behaviour, I'm also seeing this happen randomly:

....

Feb 25 11:48:35 test4 dave: run xfstest generic/083
Feb 25 11:48:37 test4 kernel: [ 8732.316223] XFS (ram1): Mounting V5 Filesystem
Feb 25 11:48:37 test4 kernel: [ 8732.318904] XFS (ram1): Ending clean mount
Feb 25 11:48:40 test4 kernel: [ 8735.871968] XFS (ram1): Unmounting Filesystem
Feb 25 11:48:40 test4 kernel: [ 8735.930160]  ram1: [POWERTEC] p1 p2 p3 p4 p5 p6 p7
Feb 25 11:48:40 test4 kernel: [ 8735.932081] ram1: p2 start 3158599292 is beyond EOD, truncated
Feb 25 11:48:40 test4 kernel: [ 8735.933983] ram1: p3 size 1627389952 extends beyond EOD, truncated
Feb 25 11:48:40 test4 kernel: [ 8735.936177] ram1: p4 size 1158021120 extends beyond EOD, truncated
Feb 25 11:48:40 test4 kernel: [ 8735.938269] ram1: p5 start 50924556 is beyond EOD, truncated
Feb 25 11:48:40 test4 kernel: [ 8735.940103] ram1: p6 size 67108864 extends beyond EOD, truncated
Feb 25 11:48:40 test4 kernel: [ 8735.942101] ram1: p7 start 4294967295 is beyond EOD, truncated
Feb 25 11:48:40 test4 dave: run xfstest generic/088

....

Something is causing partition rescans on ram devices that don't have partitions, and this is new behaviour. Boaz, your commit 937af5ecd05 ("brd: Fix all partitions BUGs") seems the likely cause of this problem I'm seeing - looks likea behaviour regression to me as no other block device I have on any machine running the same kernel throws these strange warnings from partition probing...

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]
#4 lore raw

Re: ramdisk problems in 4.0-rc1? (was Re: [PATCH 1/4] xfs/104: log size too small for 4k sector drives)

Quoted 43 lines · Dave Chinner

[cc linux-fsdevel, Boaz and others]

Quoted 40 lines · Brian Foster
Quoted 7 lines · Dave Chinner

From: Dave Chinner <[email protected]>

xfs/104, xfs/119, xfs/291 and xfs/297 have small fixed log sizes. A recent change to the kernel ramdisk changed it's physical sector size from 512B to 4kB, and this results in mkfs calculating a log size larger than the fixed test size and hence the tests fail.

Quoted 7 lines

Change the log size to a larger size that works with 4k sectors, and also increase the size of the filesystem being created so that the amount of data space in the filesystem does not change and hence does not perturb the rest of the test.

---

Well for some reason I can't mount a ramdisk on the current tot to test this. In fact, I can't mount _anything_ after the ramdisk mount attempt. The mount actually reports success too, but there's nothing there... :/

# modprobe brd
# mkfs.xfs -f /dev/ram0 
meta-data=/dev/ram0              isize=256    agcount=1, agsize=4096
blks
         =                       sectsz=4096  attr=2, projid32bit=1
         =                       crc=0        finobt=0
data     =                       bsize=4096   blocks=4096, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0 ftype=0
log      =internal log           bsize=4096   blocks=1605, version=2
         =                       sectsz=4096  sunit=1 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0
# mount /dev/ram0 /mnt/
# mount | grep mnt
# umount  /mnt/
umount: /mnt/: not mounted

... and then I can't even mount my normal scratch device until after a reboot:

Quoted 7 lines
Quoted 4 lines
# mount /dev/test/scratch /mnt/
# mount | grep mnt
# umount  /mnt/
umount: /mnt/: not mounted

Ok, so that's just plain broken. What's in dmesg?

Once I got back to this I found that for some reason systemd is immediately invoking a umount on the mount. :/ No idea why or how to stop it, but if I do something like this:

mount /dev/ram0 /mnt; cd /mnt

... I can occasionally win the race and get systemd to spin in a umount() cycle trying to undo the mount. I haven't gone back to confirm it's the same behavior with the normal devices at that point, but I suspect it is, perhaps due to getting into some kind of bad state.

So fyi that this particular problem doesn't appear to be directly kernel related...

Brian

Quoted 33 lines

As it is, I'm seeing plenty of weirdness in 4.0-rc1 on ramdisks as well. Apart from the change to 4k physical sector size causing all sorts of chaos with xfstests results due to it changing mkfs.xfs behaviour, I'm also seeing this happen randomly:

....

Feb 25 11:48:35 test4 dave: run xfstest generic/083
Feb 25 11:48:37 test4 kernel: [ 8732.316223] XFS (ram1): Mounting V5 Filesystem
Feb 25 11:48:37 test4 kernel: [ 8732.318904] XFS (ram1): Ending clean mount
Feb 25 11:48:40 test4 kernel: [ 8735.871968] XFS (ram1): Unmounting Filesystem
Feb 25 11:48:40 test4 kernel: [ 8735.930160]  ram1: [POWERTEC] p1 p2 p3 p4 p5 p6 p7
Feb 25 11:48:40 test4 kernel: [ 8735.932081] ram1: p2 start 3158599292 is beyond EOD, truncated
Feb 25 11:48:40 test4 kernel: [ 8735.933983] ram1: p3 size 1627389952 extends beyond EOD, truncated
Feb 25 11:48:40 test4 kernel: [ 8735.936177] ram1: p4 size 1158021120 extends beyond EOD, truncated
Feb 25 11:48:40 test4 kernel: [ 8735.938269] ram1: p5 start 50924556 is beyond EOD, truncated
Feb 25 11:48:40 test4 kernel: [ 8735.940103] ram1: p6 size 67108864 extends beyond EOD, truncated
Feb 25 11:48:40 test4 kernel: [ 8735.942101] ram1: p7 start 4294967295 is beyond EOD, truncated
Feb 25 11:48:40 test4 dave: run xfstest generic/088

....

Something is causing partition rescans on ram devices that don't have partitions, and this is new behaviour. Boaz, your commit 937af5ecd05 ("brd: Fix all partitions BUGs") seems the likely cause of this problem I'm seeing - looks likea behaviour regression to me as no other block device I have on any machine running the same kernel throws these strange warnings from partition probing...

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]
#5 lore raw

Re: ramdisk problems in 4.0-rc1? (was Re: [PATCH 1/4] xfs/104: log size too small for 4k sector drives)

Quoted 58 lines · Brian Foster
Quoted 43 lines · Dave Chinner

[cc linux-fsdevel, Boaz and others]

Quoted 40 lines · Brian Foster
Quoted 7 lines · Dave Chinner

From: Dave Chinner <[email protected]>

xfs/104, xfs/119, xfs/291 and xfs/297 have small fixed log sizes. A recent change to the kernel ramdisk changed it's physical sector size from 512B to 4kB, and this results in mkfs calculating a log size larger than the fixed test size and hence the tests fail.

Quoted 7 lines

Change the log size to a larger size that works with 4k sectors, and also increase the size of the filesystem being created so that the amount of data space in the filesystem does not change and hence does not perturb the rest of the test.

---

Well for some reason I can't mount a ramdisk on the current tot to test this. In fact, I can't mount _anything_ after the ramdisk mount attempt. The mount actually reports success too, but there's nothing there... :/

# modprobe brd
# mkfs.xfs -f /dev/ram0 
meta-data=/dev/ram0              isize=256    agcount=1, agsize=4096
blks
         =                       sectsz=4096  attr=2, projid32bit=1
         =                       crc=0        finobt=0
data     =                       bsize=4096   blocks=4096, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0 ftype=0
log      =internal log           bsize=4096   blocks=1605, version=2
         =                       sectsz=4096  sunit=1 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0
# mount /dev/ram0 /mnt/
# mount | grep mnt
# umount  /mnt/
umount: /mnt/: not mounted

... and then I can't even mount my normal scratch device until after a reboot:

Quoted 7 lines
Quoted 4 lines
# mount /dev/test/scratch /mnt/
# mount | grep mnt
# umount  /mnt/
umount: /mnt/: not mounted

Ok, so that's just plain broken. What's in dmesg?

Once I got back to this I found that for some reason systemd is immediately invoking a umount on the mount. :/ No idea why or how to stop it, but if I do something like this:

mount /dev/ram0 /mnt; cd /mnt

Quoted 7 lines

... I can occasionally win the race and get systemd to spin in a umount() cycle trying to undo the mount. I haven't gone back to confirm it's the same behavior with the normal devices at that point, but I suspect it is, perhaps due to getting into some kind of bad state.

So fyi that this particular problem doesn't appear to be directly kernel related...

It may still be related to the kernel changes e.g. by triggering udev events when they didn't previously. The only machine I have that is triggering the partition probing is also the only test machine that I have that runs systemd and it didn't have this problem on 3.19.

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]
#6 lore raw

Re: ramdisk problems in 4.0-rc1? (was Re: [PATCH 1/4] xfs/104: log size too small for 4k sector drives)

Quoted 64 lines · Dave Chinner
Quoted 63 lines · Brian Foster
Quoted 43 lines · Dave Chinner

[cc linux-fsdevel, Boaz and others]

Quoted 40 lines · Brian Foster
Quoted 7 lines · Dave Chinner

From: Dave Chinner <[email protected]>

xfs/104, xfs/119, xfs/291 and xfs/297 have small fixed log sizes. A recent change to the kernel ramdisk changed it's physical sector size from 512B to 4kB, and this results in mkfs calculating a log size larger than the fixed test size and hence the tests fail.

Quoted 7 lines

Change the log size to a larger size that works with 4k sectors, and also increase the size of the filesystem being created so that the amount of data space in the filesystem does not change and hence does not perturb the rest of the test.

---

Well for some reason I can't mount a ramdisk on the current tot to test this. In fact, I can't mount _anything_ after the ramdisk mount attempt. The mount actually reports success too, but there's nothing there... :/

# modprobe brd
# mkfs.xfs -f /dev/ram0 
meta-data=/dev/ram0              isize=256    agcount=1, agsize=4096
blks
         =                       sectsz=4096  attr=2, projid32bit=1
         =                       crc=0        finobt=0
data     =                       bsize=4096   blocks=4096, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0 ftype=0
log      =internal log           bsize=4096   blocks=1605, version=2
         =                       sectsz=4096  sunit=1 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0
# mount /dev/ram0 /mnt/
# mount | grep mnt
# umount  /mnt/
umount: /mnt/: not mounted

... and then I can't even mount my normal scratch device until after a reboot:

Quoted 7 lines
Quoted 4 lines
# mount /dev/test/scratch /mnt/
# mount | grep mnt
# umount  /mnt/
umount: /mnt/: not mounted

Ok, so that's just plain broken. What's in dmesg?

Once I got back to this I found that for some reason systemd is immediately invoking a umount on the mount. :/ No idea why or how to stop it, but if I do something like this:

mount /dev/ram0 /mnt; cd /mnt

... I can occasionally win the race and get systemd to spin in a umount() cycle trying to undo the mount. I haven't gone back to confirm it's the same behavior with the normal devices at that point, but I suspect it is, perhaps due to getting into some kind of bad state.

Quoted 9 lines
Quoted 2 lines

So fyi that this particular problem doesn't appear to be directly kernel related...

It may still be related to the kernel changes e.g. by triggering udev events when they didn't previously. The only machine I have that is triggering the partition probing is also the only test machine that I have that runs systemd and it didn't have this problem on 3.19.

Sigh, thanks Dave. Yes you are correct my patch enabled the udev events, as part of fixing ramdisk with partitions. This is because if you do not enable them then mount by UUID and all sort of lsblk and friends do not work.

I did try to test this in all kind of ways, xfstest+ext4 as well, and ran with it on Fedora 20 for a while, sorry about that.

It looks like the system anticipates that ramdisk "should not have these events"

I will send a patch ASAP that re-instates the module_parameter for enabling notification, and leaving the default off. It should be easy to set the param if one intends to use these utilities.

That said, please do agree with me that there is brokenness in systemd?

BTW: You also said something about the 4k sectors thing, It looks like we are pulled in two different directions here. If you will want to use DAX on ramdisk then you want it on, if you are not using DAX, and wants to use smaller-then-page_size FS blocks than you do not want it.

Please advise what we should do? Maybe only do 4k if BLK_DEV_RAM_DAX is set in Kconfig ?

Sorry for the mess, I'll send a fix ASAP

Quoted 3 lines

Cheers,
Dave.

Thanks
Boaz

#7 lore raw

Re: ramdisk problems in 4.0-rc1? (was Re: [PATCH 1/4] xfs/104: log size too small for 4k sector drives)

Quoted 84 lines · Boaz Harrosh
Quoted 64 lines · Dave Chinner
Quoted 63 lines · Brian Foster
Quoted 43 lines · Dave Chinner

[cc linux-fsdevel, Boaz and others]

Quoted 40 lines · Brian Foster
Quoted 7 lines · Dave Chinner

From: Dave Chinner <[email protected]>

xfs/104, xfs/119, xfs/291 and xfs/297 have small fixed log sizes. A recent change to the kernel ramdisk changed it's physical sector size from 512B to 4kB, and this results in mkfs calculating a log size larger than the fixed test size and hence the tests fail.

Quoted 7 lines

Change the log size to a larger size that works with 4k sectors, and also increase the size of the filesystem being created so that the amount of data space in the filesystem does not change and hence does not perturb the rest of the test.

---

Well for some reason I can't mount a ramdisk on the current tot to test this. In fact, I can't mount _anything_ after the ramdisk mount attempt. The mount actually reports success too, but there's nothing there... :/

# modprobe brd
# mkfs.xfs -f /dev/ram0 
meta-data=/dev/ram0              isize=256    agcount=1, agsize=4096
blks
         =                       sectsz=4096  attr=2, projid32bit=1
         =                       crc=0        finobt=0
data     =                       bsize=4096   blocks=4096, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0 ftype=0
log      =internal log           bsize=4096   blocks=1605, version=2
         =                       sectsz=4096  sunit=1 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0
# mount /dev/ram0 /mnt/
# mount | grep mnt
# umount  /mnt/
umount: /mnt/: not mounted

... and then I can't even mount my normal scratch device until after a reboot:

Quoted 7 lines
Quoted 4 lines
# mount /dev/test/scratch /mnt/
# mount | grep mnt
# umount  /mnt/
umount: /mnt/: not mounted

Ok, so that's just plain broken. What's in dmesg?

Once I got back to this I found that for some reason systemd is immediately invoking a umount on the mount. :/ No idea why or how to stop it, but if I do something like this:

mount /dev/ram0 /mnt; cd /mnt

... I can occasionally win the race and get systemd to spin in a umount() cycle trying to undo the mount. I haven't gone back to confirm it's the same behavior with the normal devices at that point, but I suspect it is, perhaps due to getting into some kind of bad state.

Quoted 9 lines
Quoted 2 lines

So fyi that this particular problem doesn't appear to be directly kernel related...

It may still be related to the kernel changes e.g. by triggering udev events when they didn't previously. The only machine I have that is triggering the partition probing is also the only test machine that I have that runs systemd and it didn't have this problem on 3.19.

Sigh, thanks Dave. Yes you are correct my patch enabled the udev events, as part of fixing ramdisk with partitions. This is because if you do not enable them then mount by UUID and all sort of lsblk and friends do not work.

I did try to test this in all kind of ways, xfstest+ext4 as well, and ran with it on Fedora 20 for a while, sorry about that.

Quoted 7 lines

It looks like the system anticipates that ramdisk "should not have these events"

I will send a patch ASAP that re-instates the module_parameter for enabling notification, and leaving the default off. It should be easy to set the param if one intends to use these utilities.

Thanks Boaz, but I still see the same behavior with the part_show patch. It seems to be something that broke in systemd on Fedora between versions systemd-218 and systemd-219. The latter is broken on a 3.19 kernel as well.

I've filed a systemd bug so we'll see what comes of it from that end:

https://bugzilla.redhat.com/show_bug.cgi?id=1196452

Brian

Quoted 22 lines

That said, please do agree with me that there is brokenness in systemd?

BTW: You also said something about the 4k sectors thing, It looks like we are pulled in two different directions here. If you will want to use DAX on ramdisk then you want it on, if you are not using DAX, and wants to use smaller-then-page_size FS blocks than you do not want it.

Please advise what we should do? Maybe only do 4k if BLK_DEV_RAM_DAX is set in Kconfig ?

Sorry for the mess, I'll send a fix ASAP

Quoted 3 lines

Cheers,
Dave.

Thanks
Boaz

#8 lore raw

Re: ramdisk problems in 4.0-rc1? (was Re: [PATCH 1/4] xfs/104: log size too small for 4k sector drives)

On 02/26/2015 07:23 PM, Brian Foster wrote:
<>

Quoted 6 lines

Thanks Boaz, but I still see the same behavior with the part_show patch. It seems to be something that broke in systemd on Fedora between versions systemd-218 and systemd-219. The latter is broken on a 3.19 kernel as well.

Quoted 6 lines

I've filed a systemd bug so we'll see what comes of it from that end:

https://bugzilla.redhat.com/show_bug.cgi?id=1196452

Brian

Hi Brian

It says in bugzilla (link above) that this issue is "fixed in git" so I guess we should be fine ?

Jens does *not* need to take
[PATCH] brd: Re-instate ram disk visibility option (part_show)

Please confirm.

Please tell me if there is anything I can help with?

Thanks
Boaz

#9 lore raw

Re: ramdisk problems in 4.0-rc1? (was Re: [PATCH 1/4] xfs/104: log size too small for 4k sector drives)

Quoted 12 lines · Boaz Harrosh

On 02/26/2015 07:23 PM, Brian Foster wrote:
<>

Quoted 10 lines

Thanks Boaz, but I still see the same behavior with the part_show patch. It seems to be something that broke in systemd on Fedora between versions systemd-218 and systemd-219. The latter is broken on a 3.19 kernel as well.

I've filed a systemd bug so we'll see what comes of it from that end:

https://bugzilla.redhat.com/show_bug.cgi?id=1196452

Quoted 8 lines
Quoted 2 lines

Brian

Hi Brian

It says in bugzilla (link above) that this issue is "fixed in git" so I guess we should be fine ?

Yes, I picked up a more recent systemd version and it seems to work fine now without the patch referenced below.

Brian

Quoted 10 lines

Jens does *not* need to take
[PATCH] brd: Re-instate ram disk visibility option (part_show)

Please confirm.

Please tell me if there is anything I can help with?

Thanks
Boaz

#10 lore raw

Re: ramdisk problems in 4.0-rc1? (was Re: [PATCH 1/4] xfs/104: log size too small for 4k sector drives)

In reply to Boaz Harrosh
Quoted 75 lines · Boaz Harrosh
Quoted 64 lines · Dave Chinner
Quoted 63 lines · Brian Foster
Quoted 43 lines · Dave Chinner

[cc linux-fsdevel, Boaz and others]

Quoted 40 lines · Brian Foster
Quoted 7 lines · Dave Chinner

From: Dave Chinner <[email protected]>

xfs/104, xfs/119, xfs/291 and xfs/297 have small fixed log sizes. A recent change to the kernel ramdisk changed it's physical sector size from 512B to 4kB, and this results in mkfs calculating a log size larger than the fixed test size and hence the tests fail.

Quoted 7 lines

Change the log size to a larger size that works with 4k sectors, and also increase the size of the filesystem being created so that the amount of data space in the filesystem does not change and hence does not perturb the rest of the test.

---

Well for some reason I can't mount a ramdisk on the current tot to test this. In fact, I can't mount _anything_ after the ramdisk mount attempt. The mount actually reports success too, but there's nothing there... :/

# modprobe brd
# mkfs.xfs -f /dev/ram0 
meta-data=/dev/ram0              isize=256    agcount=1, agsize=4096
blks
         =                       sectsz=4096  attr=2, projid32bit=1
         =                       crc=0        finobt=0
data     =                       bsize=4096   blocks=4096, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0 ftype=0
log      =internal log           bsize=4096   blocks=1605, version=2
         =                       sectsz=4096  sunit=1 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0
# mount /dev/ram0 /mnt/
# mount | grep mnt
# umount  /mnt/
umount: /mnt/: not mounted

... and then I can't even mount my normal scratch device until after a reboot:

Quoted 7 lines
Quoted 4 lines
# mount /dev/test/scratch /mnt/
# mount | grep mnt
# umount  /mnt/
umount: /mnt/: not mounted

Ok, so that's just plain broken. What's in dmesg?

Once I got back to this I found that for some reason systemd is immediately invoking a umount on the mount. :/ No idea why or how to stop it, but if I do something like this:

mount /dev/ram0 /mnt; cd /mnt

... I can occasionally win the race and get systemd to spin in a umount() cycle trying to undo the mount. I haven't gone back to confirm it's the same behavior with the normal devices at that point, but I suspect it is, perhaps due to getting into some kind of bad state.

Quoted 9 lines
Quoted 2 lines

So fyi that this particular problem doesn't appear to be directly kernel related...

It may still be related to the kernel changes e.g. by triggering udev events when they didn't previously. The only machine I have that is triggering the partition probing is also the only test machine that I have that runs systemd and it didn't have this problem on 3.19.

Quoted 4 lines

Sigh, thanks Dave. Yes you are correct my patch enabled the udev events, as part of fixing ramdisk with partitions. This is because if you do not enable them then mount by UUID and all sort of lsblk and friends do not work.

Sure, that's what the gendisk abstraction just you. But why am I seeing random partition probes on a ramdisk that *isn't using partitions*?

Quoted 6 lines

I did try to test this in all kind of ways, xfstest+ext4 as well, and ran with it on Fedora 20 for a while, sorry about that.

It looks like the system anticipates that ramdisk "should not have these events"

Right, but not because it's a ramdisk. Those events should not be occurring because I'm not creating or destroying devices, I'm not changing partition tables, I'm not resizing ramdisks or partitions, and so on. I'm simply mkfs'ing, mounting and unmounting filesystems on the ramdisks - nothing should be generating device based udev events...

Finding the trigger that is causing these events will tell us what the bug is - restricting the config won't help, especially as DAX will *always* be enabled on my test machines as it's something needed in my test matrix. I'm not sure how to go about finding that trigger right now and as such I won't really have time to look at it until after lsfmm/vault...

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]
#11 lore raw

Re: ramdisk problems in 4.0-rc1? (was Re: [PATCH 1/4] xfs/104: log size too small for 4k sector drives)

On 02/27/2015 02:58 AM, Dave Chinner wrote:
<>

Quoted 10 lines
Quoted 5 lines

Sigh, thanks Dave. Yes you are correct my patch enabled the udev events, as part of fixing ramdisk with partitions. This is because if you do not enable them then mount by UUID and all sort of lsblk and friends do not work.

Sure, that's what the gendisk abstraction just you. But why am I seeing random partition probes on a ramdisk that *isn't using partitions*?

Yes, There should be the one new event on create (modprobe or mknod) which was not there before. Perhaps it triggers a systemd process that never used to run before. (And is now sitting there and making a mess)

<>

Quoted 10 lines
Quoted 2 lines

It looks like the system anticipates that ramdisk "should not have these events"

Right, but not because it's a ramdisk. Those events should not be occurring because I'm not creating or destroying devices, I'm not changing partition tables, I'm not resizing ramdisks or partitions, and so on. I'm simply mkfs'ing, mounting and unmounting filesystems on the ramdisks - nothing should be generating device based udev events...

Quoted 6 lines

Finding the trigger that is causing these events will tell us what the bug is -

restricting the config won't help, especially as DAX will *always* be enabled on my test machines as it's something needed in my test matrix.

No the "if DAX" is for the 4k thing. The enablement of the uevents is with a new "part_show" module parameter (See patch-1).

Quoted 4 lines

I'm not sure how to go about finding that
trigger right now and as such I won't really have time to look at it until after lsfmm/vault...

I'll try to reproduce this here. What Fedora version do I need?

Quoted 3 lines

Cheers,
Dave.

Thanks
Boaz

#12 lore raw

Re: ramdisk problems in 4.0-rc1? (was Re: [PATCH 1/4] xfs/104: log size too small for 4k sector drives)

Quoted 14 lines · Boaz Harrosh
Quoted 12 lines · Dave Chinner
Quoted 2 lines

It looks like the system anticipates that ramdisk "should not have these events"

Right, but not because it's a ramdisk. Those events should not be occurring because I'm not creating or destroying devices, I'm not changing partition tables, I'm not resizing ramdisks or partitions, and so on. I'm simply mkfs'ing, mounting and unmounting filesystems on the ramdisks - nothing should be generating device based udev events...

Finding the trigger that is causing these events will tell us what the bug is -

Quoted 6 lines
Quoted 3 lines

restricting the config won't help, especially as DAX will *always* be enabled on my test machines as it's something needed in my test matrix.

No the "if DAX" is for the 4k thing. The enablement of the uevents is with a new "part_show" module parameter (See patch-1).

Sure, but that doesn't answer my question: what is generating device level uevents when all I'm doing is mkfs/mount/umount on the device?

Quoted 6 lines
Quoted 4 lines

I'm not sure how to go about finding that
trigger right now and as such I won't really have time to look at it until after lsfmm/vault...

I'll try to reproduce this here. What Fedora version do I need?

I'm running debian unstable w/ systemd-215 on the particular test machine that is hitting this problem.

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]
#13 lore raw

Re: ramdisk problems in 4.0-rc1? (was Re: [PATCH 1/4] xfs/104: log size too small for 4k sector drives)

Quoted 2 lines · Dave Chinner
Quoted 1 line · Boaz Harrosh

On 02/27/2015 02:58 AM, Dave Chinner wrote:

<>

Quoted 6 lines
Quoted 2 lines

No the "if DAX" is for the 4k thing. The enablement of the uevents is with a new "part_show" module parameter (See patch-1).

Sure, but that doesn't answer my question: what is generating device level uevents when all I'm doing is mkfs/mount/umount on the device?

I was suspecting it is this systemd bug which keeps trying to tier-down the devices.

Quoted 10 lines
Quoted 6 lines
Quoted 4 lines

I'm not sure how to go about finding that
trigger right now and as such I won't really have time to look at it until after lsfmm/vault...

I'll try to reproduce this here. What Fedora version do I need?

I'm running debian unstable w/ systemd-215 on the particular test machine that is hitting this problem.

Oooff, On my fedora 20 I'm at systemd 208. I'll see if I have time to install an fc21 vm or maybe upgrade from source. (Any easy way?)

I have setup my xfs rig and ran "./check -g auto" by now. I tried both part_show=1/0 and both look working as expected.
(Do I need any special $MKFS_OPTIONS or anything else)

I'll probably be giving up soon, and will just wait for more reports.

With the patch-1 I sent I am reverting to old behavior so I need a reproducer to try and run with patch-1 and part_show=1 should show the problem and part_show=0 should not. Else this is something else

Quoted 3 lines

Cheers,
Dave.

Thanks Dave, sorry for trapping you in this boring mess, life at Kernel the one reproducing the problem needs to help fix it ;-)

Have an enjoyable and productive LSF
Thanks
Boaz

#14 lore raw

PATCH 1/1 brd: Re-instate ram disk visibility option (part_show)

In reply to Dave Chinner

This re-enables part_show option, so we can keep it false by default.

Here is what Dmitry said in the original patch:
[aeac318] brd: add ram disk visibility option
Currently ram disk is not visible inside /proc/partitions. This was done for compatibility reasons here: 53978d0a7a27. But some utilities expect disk presents in /proc/partitions. Let's add module's option and let's administrator chose visibility behavior. By default, old behavior preserved.

Dave Chinner and other have reported problems with current system when udev events start firing for ramdisk:

	http://marc.info/?l=linux-fsdevel&m=142490357918672&w=2

It was me who enabled these notifications through this patch:
[937af5e] brd: Fix all partitions BUGs
Sorry for the mess.

CC Dave Chinner <[email protected]>
CC Dmitry Monakhov <[email protected]>
---
 drivers/block/brd.c | 6 ++++++
 1 file changed, 6 insertions(+)
drivers/block/brd.c +6 -0
index 64ab495..6e0775b 100644
@@ -450,6 +450,10 @@ static int max_part = 1;
 module_param(max_part, int, S_IRUGO);
 MODULE_PARM_DESC(max_part, "Num Minors to reserve between devices");
 
+static int part_show = 0;
+module_param(part_show, int, S_IRUGO);
+MODULE_PARM_DESC(part_show, "Control RAM disk visibility in /proc/partitions");
+
 MODULE_LICENSE("GPL");
 MODULE_ALIAS_BLOCKDEV_MAJOR(RAMDISK_MAJOR);
 MODULE_ALIAS("rd");
@@ -513,6 +517,8 @@ static struct brd_device *brd_alloc(int i)
 	disk->private_data	= brd;
 	disk->queue		= brd->brd_queue;
 	disk->flags		= GENHD_FL_EXT_DEVT;
+	if (!part_show)
+		disk->flags |= GENHD_FL_SUPPRESS_PARTITION_INFO;
 	sprintf(disk->disk_name, "ram%d", i);
 	set_capacity(disk, rd_size * 2);
 
Signature · 1 line
1.9.3
#15 lore raw

PATCH 1/1 brd: Only request 4K sectors if DAX is enabled

People systems have been using ramdisk with
smaller-than-page-size blocks in their mkfs.

The 4K sectors is only important if we will be
using brd with a DAX filesystem.

So only enable 4K sectors if DAX is configured

---
 drivers/block/brd.c | 2 ++
 1 file changed, 2 insertions(+)
drivers/block/brd.c +2 -0
index 6e0775b..e875f12 100644
@@ -495,6 +495,7 @@ static struct brd_device *brd_alloc(int i)
 	blk_queue_max_hw_sectors(brd->brd_queue, 1024);
 	blk_queue_bounce_limit(brd->brd_queue, BLK_BOUNCE_ANY);
 
+#ifdef CONFIG_BLK_DEV_RAM_DAX
 	/* This is so fdisk will align partitions on 4k, because of
 	 * direct_access API needing 4k alignment, returning a PFN
 	 * (This is only a problem on very small devices <= 4M,
@@ -502,6 +503,7 @@ static struct brd_device *brd_alloc(int i)
 	 *  is harmless)
 	 */
 	blk_queue_physical_block_size(brd->brd_queue, PAGE_SIZE);
+#endif
 
 	brd->brd_queue->limits.discard_granularity = PAGE_SIZE;
 	brd->brd_queue->limits.max_discard_sectors = UINT_MAX;
Signature · 1 line
1.9.3
#16 lore raw

Re: [PATCH] brd: Only request 4K sectors if DAX is enabled

Quoted 27 lines · Boaz Harrosh

People systems have been using ramdisk with
smaller-than-page-size blocks in their mkfs.

The 4K sectors is only important if we will be
using brd with a DAX filesystem.

So only enable 4K sectors if DAX is configured

---
 drivers/block/brd.c | 2 ++
 1 file changed, 2 insertions(+)
drivers/block/brd.c +1 -0
index 6e0775b..e875f12 100644
@@ -495,6 +495,7 @@ static struct brd_device *brd_alloc(int i)
 	blk_queue_max_hw_sectors(brd->brd_queue, 1024);
 	blk_queue_bounce_limit(brd->brd_queue, BLK_BOUNCE_ANY);
 
+#ifdef CONFIG_BLK_DEV_RAM_DAX
 	/* This is so fdisk will align partitions on 4k, because of
 	 * direct_access API needing 4k alignment, returning a PFN
 	 * (This is only a problem on very small devices <= 4M,

drivers/block/brd.c static struct brd_device *brd_alloc(int i) · line 504

Quoted 2 lines
drivers/block/brd.c +0 -0
@@ -502,6 +503,7 @@ static struct brd_device *brd_alloc(int i)
 	 *  is harmless)
 	 */

Hi Dave

I was just thinking should we also do

+ if (part_show)

drivers/block/brd.c static struct brd_device *brd_alloc(int i) · line 505

Quoted 1 line
drivers/block/brd.c +0 -0
 		blk_queue_physical_block_size(brd->brd_queue, PAGE_SIZE);

The rational is that if part_show is off then we are not using partitions at all and then there can be no problems?

I'll try to run your tests here. I never test xfs always ext4, maybe I should start to ;-). Actually I do have xfsprogs so why not. But if you could run a quick test as well it could be great.

Thanks
Boaz

<>

#17 lore raw

Re: [PATCH] brd: Only request 4K sectors if DAX is enabled

In reply to Boaz Harrosh
Quoted 8 lines · Boaz Harrosh

People systems have been using ramdisk with
smaller-than-page-size blocks in their mkfs.

The 4K sectors is only important if we will be
using brd with a DAX filesystem.

So only enable 4K sectors if DAX is configured

4k sectors are not a problem - we should be handling them fine and because it makes the ramdisk look like a 512e drive, no applications should fail, either.

The main "unexpected" part about it was how much of xfstests didn't handle 4k sectors in mkfs output properly. This isn't a problem with the kernel change and so doesn't need fixing.

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]
#18 lore raw

Re: [PATCH] brd: Only request 4K sectors if DAX is enabled

Quoted 10 lines · Dave Chinner
Quoted 8 lines · Boaz Harrosh

People systems have been using ramdisk with
smaller-than-page-size blocks in their mkfs.

The 4K sectors is only important if we will be
using brd with a DAX filesystem.

So only enable 4K sectors if DAX is configured

Quoted 8 lines

4k sectors are not a problem - we should be handling them fine and because it makes the ramdisk look like a 512e drive, no applications should fail, either.

The main "unexpected" part about it was how much of xfstests didn't handle 4k sectors in mkfs output properly. This isn't a problem with the kernel change and so doesn't need fixing.

Glad to be of service ;-)

Please tell me if there is anything I can help with

Quoted 3 lines

Cheers,
Dave.

Thanks
Boaz

#19 lore raw

PATCH v2 2/4 xfs: don't output mkfs sector sizes into golden output

In reply to Dave Chinner

From: Dave Chinner <[email protected]>

The mkfs sector sizes are dependent on the underlying device in use, and so is not fixed. hence it needs to be filtered from any golden output file, otherwise tests that just differ by sector size will fail.

---
 tests/xfs/096          |  8 ++++++--
 tests/xfs/096.external | 15 +++++++--------
 tests/xfs/096.internal | 16 ++++++++--------
 tests/xfs/119          |  2 +-
 tests/xfs/206          | 33 +++++++++++++++------------------
 5 files changed, 37 insertions(+), 37 deletions(-)
tests/xfs/096 +6 -2
index 0ecf88f..2da8fa1 100755
@@ -47,6 +47,8 @@ _cleanup()
 #
 # filter out counts which will vary
 #   - extsz, blocks, agsize, agcount, device name, rtextents
+#   - log version varies for crc enabled fs
+#   - lsunit varies for 512/4k sector devices
 # filter out differences between linux and irix:
 #   - sectsz on Linux
 #   - mmr, mixed-case on IRIX
@@ -63,7 +65,7 @@ _cleanup()
 #           =                       sunit=0 blks
 #  realtime =none                   extsz=65536  blocks=N, rtextents=N
 #
-_mkfs_filter()
+mkfs_filter()
 {
    tee -a $seqres.full | \
    sed \
@@ -80,6 +82,8 @@ _mkfs_filter()
 	-e '/ *= *mmr=[0-9][0-9]* *$/d' \
 	-e 's/ *mixed-case=[YN]//' \
 	-e 's/ *ascii-ci=[01]//' \
+	-e 's/\(version=\)\([12]\)/\1N/' \
+	-e 's/\(sunit=\)\([018] blks\)/\1N blks/' \
 	-e 's/sectsz=[0-9][0-9]* *//' \
 	-e 's/, lazy-count.*//' \
 	-e '/inode-paths/d' \
@@ -145,7 +149,7 @@ do
     fi
     echo "--- mkfs=$mkfs ---"
     export MKFS_OPTIONS="$mkfs"
-    _scratch_mkfs_xfs 2>&1 | _mkfs_filter
+    _scratch_mkfs_xfs 2>&1 | mkfs_filter
     echo ""
     echo ""
 done
tests/xfs/096.external +7 -8
index 95833c8..7923340 100644
@@ -11,8 +11,7 @@ meta-data=DEV isize=256    agcount=N, agsize=N blks
 data     =                       bsize=4096   blocks=N, imaxpct=N
          =                       sunit=0      swidth=0 blks, unwritten=1
 naming   =version 2              bsize=4096
-log      =LOG                    bsize=4096   blocks=N, version=2
-         =                       sunit=8 blks
+log      =LOG                    bsize=4096   blocks=N, version=N
 realtime =REALTIME               extsz=N, blocks=N, rtextents=N
 
 
@@ -22,8 +21,8 @@ meta-data=DEV isize=256    agcount=N, agsize=N blks
 data     =                       bsize=4096   blocks=N, imaxpct=N
          =                       sunit=65     swidth=65 blks, unwritten=1
 naming   =version 2              bsize=4096
-log      =LOG                    bsize=4096   blocks=N, version=2
-         =                       sunit=0 blks
+log      =LOG                    bsize=4096   blocks=N, version=N
+         =                       sunit=N blks
 realtime =REALTIME               extsz=N, blocks=N, rtextents=N
 
 
@@ -33,8 +32,8 @@ meta-data=DEV isize=256    agcount=N, agsize=N blks
 data     =                       bsize=4096   blocks=N, imaxpct=N
          =                       sunit=65     swidth=65 blks, unwritten=1
 naming   =version 2              bsize=4096
-log      =LOG                    bsize=4096   blocks=N, version=1
-         =                       sunit=0 blks
+log      =LOG                    bsize=4096   blocks=N, version=N
+         =                       sunit=N blks
 realtime =REALTIME               extsz=N, blocks=N, rtextents=N
 
 
@@ -44,8 +43,8 @@ meta-data=DEV isize=256    agcount=N, agsize=N blks
 data     =                       bsize=4096   blocks=N, imaxpct=N
          =                       sunit=65     swidth=65 blks, unwritten=1
 naming   =version 2              bsize=4096
-log      =LOG                    bsize=4096   blocks=N, version=1
-         =                       sunit=0 blks
+log      =LOG                    bsize=4096   blocks=N, version=N
+         =                       sunit=N blks
 realtime =REALTIME               extsz=N, blocks=N, rtextents=N
 
 
tests/xfs/096.internal +8 -8
index 7bf848c..eaba13f 100644
@@ -11,8 +11,8 @@ meta-data=DEV isize=256    agcount=N, agsize=N blks
 data     =                       bsize=4096   blocks=N, imaxpct=N
          =                       sunit=0      swidth=0 blks, unwritten=1
 naming   =version 2              bsize=4096
-log      =LOG                    bsize=4096   blocks=N, version=2
-         =                       sunit=8 blks
+log      =LOG                    bsize=4096   blocks=N, version=N
+         =                       sunit=N blks
 realtime =REALTIME               extsz=N, blocks=N, rtextents=N
 
 
@@ -22,8 +22,8 @@ meta-data=DEV isize=256    agcount=N, agsize=N blks
 data     =                       bsize=4096   blocks=N, imaxpct=N
          =                       sunit=65     swidth=65 blks, unwritten=1
 naming   =version 2              bsize=4096
-log      =LOG                    bsize=4096   blocks=N, version=2
-         =                       sunit=8 blks
+log      =LOG                    bsize=4096   blocks=N, version=N
+         =                       sunit=N blks
 realtime =REALTIME               extsz=N, blocks=N, rtextents=N
 
 
@@ -33,8 +33,8 @@ meta-data=DEV isize=256    agcount=N, agsize=N blks
 data     =                       bsize=4096   blocks=N, imaxpct=N
          =                       sunit=65     swidth=65 blks, unwritten=1
 naming   =version 2              bsize=4096
-log      =LOG                    bsize=4096   blocks=N, version=1
-         =                       sunit=0 blks
+log      =LOG                    bsize=4096   blocks=N, version=N
+         =                       sunit=N blks
 realtime =REALTIME               extsz=N, blocks=N, rtextents=N
 
 
@@ -44,8 +44,8 @@ meta-data=DEV isize=256    agcount=N, agsize=N blks
 data     =                       bsize=4096   blocks=N, imaxpct=N
          =                       sunit=65     swidth=65 blks, unwritten=1
 naming   =version 2              bsize=4096
-log      =LOG                    bsize=4096   blocks=N, version=1
-         =                       sunit=0 blks
+log      =LOG                    bsize=4096   blocks=N, version=N
+         =                       sunit=N blks
 realtime =REALTIME               extsz=N, blocks=N, rtextents=N
 
 
tests/xfs/119 +1 -1
index 490495b..156d5e4 100755
@@ -54,7 +54,7 @@ _require_scratch
 # this may hang
 sync
 
-export MKFS_OPTIONS="-l version=2,size=2500b,su=64k" 
+export MKFS_OPTIONS="-l version=2,size=2560b,su=64k" 
 export MOUNT_OPTIONS="-o logbsize=64k"
 _scratch_mkfs_xfs >/dev/null
 
tests/xfs/206 +15 -18
index f6dcca6..198e413 100755
@@ -73,34 +73,31 @@ echo "=== truncate file ==="
 dd if=/dev/zero of=$tmpfile bs=1 seek=19998630180864 count=1 >/dev/null 2>&1 \
 	|| _fail "!!! failed to truncate loopback file to correct size"
 
+mkfs_filter()
+{
+	sed -e "s,^meta-data=.*isize,meta-data=FILE                   isize,g" \
+	    -e "s/\(^log.*blocks=\)\([0-9]*,\)/\1XXXXX,/" \
+	    -e "s/, projid32bit=[0-9]//" \
+	    -e "s/ ftype=[0-9]//" \
+	    -e "s/\(sectsz\)\(=[0-9]* *\)/\1=512   /" \
+	    -e "s/\(sunit=\)\([0-9]* blks,\)/\10 blks,/" \
+	    -e "/.*crc=/d"
+}
+
 # mkfs slightly smaller than that
 echo "=== mkfs.xfs ==="
-mkfs.xfs -f -bsize=4096 -dagsize=76288719b,size=3905982455b -llazy-count=0 $tmpfile \
-	| sed -e "s,^meta-data=.*isize,meta-data=FILE                   isize,g" \
-		-e "s/\(^log.*blocks=\)\([0-9]*,\)/\1XXXXX,/" \
-		-e "s/, projid32bit=[0-9]//" \
-		-e "s/ ftype=[0-9]//" \
-		-e "/.*crc=/d"
+mkfs.xfs -f -bsize=4096 -dagsize=76288719b,size=3905982455b \
+	 -llazy-count=0 $tmpfile  | mkfs_filter
 
 mount -o loop $tmpfile $tmpdir || _fail "!!! failed to loopback mount"
 
 # see what happens when we growfs it
 echo "=== xfs_growfs ==="
-xfs_growfs $tmpdir \
-	| sed -e "s,^meta-data=.*isize,meta-data=FILE                   isize,g" \
-		-e "s/\(^log.*blocks=\)\([0-9]*,\)/\1XXXXX,/" \
-		-e "s/, projid32bit=[0-9]//" \
-		-e "s/ ftype=[0-9]//" \
-		-e "/.*crc=/d"
+xfs_growfs $tmpdir  | mkfs_filter
 
 # and double-check the new geometry
 echo "=== xfs_info ==="
-xfs_info $tmpdir \
-	| sed -e "s,^meta-data=.*isize,meta-data=FILE                   isize,g" \
-		-e "s/\(^log.*blocks=\)\([0-9]*,\)/\1XXXXX,/" \
-		-e "s/, projid32bit=[0-9]//" \
-		-e "s/ ftype=[0-9]//" \
-		-e "/.*crc=/d"
+xfs_info $tmpdir | mkfs_filter
 
 # _cleanup cleans up for us
 
Signature · 1 line
2.0.0
Reviewed-by on v2 2/4
#20 lore raw

Re: [PATCH 2/4] xfs: don't output mkfs sector sizes into golden output

Quoted 156 lines · Dave Chinner

From: Dave Chinner <[email protected]>

The mkfs sector sizes are dependent on the underlying device in use, and so is not fixed. hence it needs to be filtered from any golden output file, otherwise tests that just differ by sector size will fail.

---
 tests/xfs/096          |  8 ++++++--
 tests/xfs/096.external | 15 +++++++--------
 tests/xfs/096.internal | 16 ++++++++--------
 tests/xfs/119          |  2 +-
 tests/xfs/206          | 33 +++++++++++++++------------------
 5 files changed, 37 insertions(+), 37 deletions(-)
tests/xfs/096 +6 -2
index 0ecf88f..2da8fa1 100755
@@ -47,6 +47,8 @@ _cleanup()
 #
 # filter out counts which will vary
 #   - extsz, blocks, agsize, agcount, device name, rtextents
+#   - log version varies for crc enabled fs
+#   - lsunit varies for 512/4k sector devices
 # filter out differences between linux and irix:
 #   - sectsz on Linux
 #   - mmr, mixed-case on IRIX
@@ -63,7 +65,7 @@ _cleanup()
 #           =                       sunit=0 blks
 #  realtime =none                   extsz=65536  blocks=N, rtextents=N
 #
-_mkfs_filter()
+mkfs_filter()
 {
    tee -a $seqres.full | \
    sed \
@@ -80,6 +82,8 @@ _mkfs_filter()
 	-e '/ *= *mmr=[0-9][0-9]* *$/d' \
 	-e 's/ *mixed-case=[YN]//' \
 	-e 's/ *ascii-ci=[01]//' \
+	-e 's/\(version=\)\([12]\)/\1N/' \
+	-e 's/\(sunit=\)\([018] blks\)/\1N blks/' \
 	-e 's/sectsz=[0-9][0-9]* *//' \
 	-e 's/, lazy-count.*//' \
 	-e '/inode-paths/d' \
@@ -145,7 +149,7 @@ do
     fi
     echo "--- mkfs=$mkfs ---"
     export MKFS_OPTIONS="$mkfs"
-    _scratch_mkfs_xfs 2>&1 | _mkfs_filter
+    _scratch_mkfs_xfs 2>&1 | mkfs_filter
     echo ""
     echo ""
 done
tests/xfs/096.external +7 -8
index 95833c8..7923340 100644
@@ -11,8 +11,7 @@ meta-data=DEV isize=256    agcount=N, agsize=N blks
 data     =                       bsize=4096   blocks=N, imaxpct=N
          =                       sunit=0      swidth=0 blks, unwritten=1
 naming   =version 2              bsize=4096
-log      =LOG                    bsize=4096   blocks=N, version=2
-         =                       sunit=8 blks
+log      =LOG                    bsize=4096   blocks=N, version=N
 realtime =REALTIME               extsz=N, blocks=N, rtextents=N
 
 
@@ -22,8 +21,8 @@ meta-data=DEV isize=256    agcount=N, agsize=N blks
 data     =                       bsize=4096   blocks=N, imaxpct=N
          =                       sunit=65     swidth=65 blks, unwritten=1
 naming   =version 2              bsize=4096
-log      =LOG                    bsize=4096   blocks=N, version=2
-         =                       sunit=0 blks
+log      =LOG                    bsize=4096   blocks=N, version=N
+         =                       sunit=N blks
 realtime =REALTIME               extsz=N, blocks=N, rtextents=N
 
 
@@ -33,8 +32,8 @@ meta-data=DEV isize=256    agcount=N, agsize=N blks
 data     =                       bsize=4096   blocks=N, imaxpct=N
          =                       sunit=65     swidth=65 blks, unwritten=1
 naming   =version 2              bsize=4096
-log      =LOG                    bsize=4096   blocks=N, version=1
-         =                       sunit=0 blks
+log      =LOG                    bsize=4096   blocks=N, version=N
+         =                       sunit=N blks
 realtime =REALTIME               extsz=N, blocks=N, rtextents=N
 
 
@@ -44,8 +43,8 @@ meta-data=DEV isize=256    agcount=N, agsize=N blks
 data     =                       bsize=4096   blocks=N, imaxpct=N
          =                       sunit=65     swidth=65 blks, unwritten=1
 naming   =version 2              bsize=4096
-log      =LOG                    bsize=4096   blocks=N, version=1
-         =                       sunit=0 blks
+log      =LOG                    bsize=4096   blocks=N, version=N
+         =                       sunit=N blks
 realtime =REALTIME               extsz=N, blocks=N, rtextents=N
 
 
tests/xfs/096.internal +8 -8
index 7bf848c..eaba13f 100644
@@ -11,8 +11,8 @@ meta-data=DEV isize=256    agcount=N, agsize=N blks
 data     =                       bsize=4096   blocks=N, imaxpct=N
          =                       sunit=0      swidth=0 blks, unwritten=1
 naming   =version 2              bsize=4096
-log      =LOG                    bsize=4096   blocks=N, version=2
-         =                       sunit=8 blks
+log      =LOG                    bsize=4096   blocks=N, version=N
+         =                       sunit=N blks
 realtime =REALTIME               extsz=N, blocks=N, rtextents=N
 
 
@@ -22,8 +22,8 @@ meta-data=DEV isize=256    agcount=N, agsize=N blks
 data     =                       bsize=4096   blocks=N, imaxpct=N
          =                       sunit=65     swidth=65 blks, unwritten=1
 naming   =version 2              bsize=4096
-log      =LOG                    bsize=4096   blocks=N, version=2
-         =                       sunit=8 blks
+log      =LOG                    bsize=4096   blocks=N, version=N
+         =                       sunit=N blks
 realtime =REALTIME               extsz=N, blocks=N, rtextents=N
 
 
@@ -33,8 +33,8 @@ meta-data=DEV isize=256    agcount=N, agsize=N blks
 data     =                       bsize=4096   blocks=N, imaxpct=N
          =                       sunit=65     swidth=65 blks, unwritten=1
 naming   =version 2              bsize=4096
-log      =LOG                    bsize=4096   blocks=N, version=1
-         =                       sunit=0 blks
+log      =LOG                    bsize=4096   blocks=N, version=N
+         =                       sunit=N blks
 realtime =REALTIME               extsz=N, blocks=N, rtextents=N
 
 
@@ -44,8 +44,8 @@ meta-data=DEV isize=256    agcount=N, agsize=N blks
 data     =                       bsize=4096   blocks=N, imaxpct=N
          =                       sunit=65     swidth=65 blks, unwritten=1
 naming   =version 2              bsize=4096
-log      =LOG                    bsize=4096   blocks=N, version=1
-         =                       sunit=0 blks
+log      =LOG                    bsize=4096   blocks=N, version=N
+         =                       sunit=N blks
 realtime =REALTIME               extsz=N, blocks=N, rtextents=N
 
 
tests/xfs/119 +0 -0
index 490495b..156d5e4 100755

tests/xfs/119 _require_scratch · line 57

Quoted 5 lines
tests/xfs/119 +1 -1
@@ -54,7 +54,7 @@ _require_scratch
 # this may hang
 sync
 
-export MKFS_OPTIONS="-l version=2,size=2500b,su=64k" 
+export MKFS_OPTIONS="-l version=2,size=2560b,su=64k" 

Not clear if this was intended to be part of the previous patch? Anyways, still a trailing space here. Otherwise this looks fine to me:

Quoted 64 lines
tests/xfs/119 +0 -0
 export MOUNT_OPTIONS="-o logbsize=64k"
 _scratch_mkfs_xfs >/dev/null
 
tests/xfs/206 +15 -18
index f6dcca6..198e413 100755
@@ -73,34 +73,31 @@ echo "=== truncate file ==="
 dd if=/dev/zero of=$tmpfile bs=1 seek=19998630180864 count=1 >/dev/null 2>&1 \
 	|| _fail "!!! failed to truncate loopback file to correct size"
 
+mkfs_filter()
+{
+	sed -e "s,^meta-data=.*isize,meta-data=FILE                   isize,g" \
+	    -e "s/\(^log.*blocks=\)\([0-9]*,\)/\1XXXXX,/" \
+	    -e "s/, projid32bit=[0-9]//" \
+	    -e "s/ ftype=[0-9]//" \
+	    -e "s/\(sectsz\)\(=[0-9]* *\)/\1=512   /" \
+	    -e "s/\(sunit=\)\([0-9]* blks,\)/\10 blks,/" \
+	    -e "/.*crc=/d"
+}
+
 # mkfs slightly smaller than that
 echo "=== mkfs.xfs ==="
-mkfs.xfs -f -bsize=4096 -dagsize=76288719b,size=3905982455b -llazy-count=0 $tmpfile \
-	| sed -e "s,^meta-data=.*isize,meta-data=FILE                   isize,g" \
-		-e "s/\(^log.*blocks=\)\([0-9]*,\)/\1XXXXX,/" \
-		-e "s/, projid32bit=[0-9]//" \
-		-e "s/ ftype=[0-9]//" \
-		-e "/.*crc=/d"
+mkfs.xfs -f -bsize=4096 -dagsize=76288719b,size=3905982455b \
+	 -llazy-count=0 $tmpfile  | mkfs_filter
 
 mount -o loop $tmpfile $tmpdir || _fail "!!! failed to loopback mount"
 
 # see what happens when we growfs it
 echo "=== xfs_growfs ==="
-xfs_growfs $tmpdir \
-	| sed -e "s,^meta-data=.*isize,meta-data=FILE                   isize,g" \
-		-e "s/\(^log.*blocks=\)\([0-9]*,\)/\1XXXXX,/" \
-		-e "s/, projid32bit=[0-9]//" \
-		-e "s/ ftype=[0-9]//" \
-		-e "/.*crc=/d"
+xfs_growfs $tmpdir  | mkfs_filter
 
 # and double-check the new geometry
 echo "=== xfs_info ==="
-xfs_info $tmpdir \
-	| sed -e "s,^meta-data=.*isize,meta-data=FILE                   isize,g" \
-		-e "s/\(^log.*blocks=\)\([0-9]*,\)/\1XXXXX,/" \
-		-e "s/, projid32bit=[0-9]//" \
-		-e "s/ ftype=[0-9]//" \
-		-e "/.*crc=/d"
+xfs_info $tmpdir | mkfs_filter
 
 # _cleanup cleans up for us
 
Signature · 6 lines
2.0.0

--
To unsubscribe from this list: send the line "unsubscribe fstests" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#21 lore raw

PATCH v2 3/4 xfs/049: umount -d fails when kernel wins teardown race

In reply to Dave Chinner

From: Dave Chinner <[email protected]>

When /etc/mtab is linked to /proc/mounts and we are using mount time created loop devices (i.e. mount -o loop), the unmount can fail with this amazingly informative error message:

umount: /mnt/scratch/test2: filesystem was unmounted, but mount(8) failed: Invalid argument

What it actually means in this case is that the kernel tore down the loop device when the last reference went away, and it did it so fast that mount was not able to find it in /etc/mtab after the unmount syscall. Hence it could not find the loop device it was supposed to tear down and has a hissy fit.

This is simple to fix: mount does not need to tear down the loop device as the kernel does it automatically. Remove the "-d" from the umount command, and the test passes again.

There's quite a few other tests that also use umount -d - fix them as well.

---
 tests/xfs/049 | 8 ++++----
 tests/xfs/073 | 6 +++---
 tests/xfs/078 | 4 ++--
 tests/xfs/216 | 2 +-
 tests/xfs/217 | 2 +-
 tests/xfs/250 | 4 ++--
 6 files changed, 13 insertions(+), 13 deletions(-)
tests/xfs/049 +4 -4
index 04c2c75..8d4e074 100755
@@ -29,8 +29,8 @@ echo "QA output created by $seq"
 _cleanup()
 {
     cd /
-    umount -d $SCRATCH_MNT/test2 > /dev/null 2>&1
-    umount -d $SCRATCH_MNT/test > /dev/null 2>&1
+    umount $SCRATCH_MNT/test2 > /dev/null 2>&1
+    umount $SCRATCH_MNT/test > /dev/null 2>&1
     rm -f $tmp.*
 
     if [ -w $seqres.full ]
@@ -123,11 +123,11 @@ rm -rf $SCRATCH_MNT/test/* >> $seqres.full 2>&1 \
     || _fail "!!! clean failed"
 
 _log "umount ext2 on xfs"
-umount -d $SCRATCH_MNT/test2 >> $seqres.full 2>&1 \
+umount $SCRATCH_MNT/test2 >> $seqres.full 2>&1 \
     || _fail "!!! umount ext2 failed"
 
 _log "umount xfs"
-umount -d $SCRATCH_MNT/test >> $seqres.full 2>&1 \
+umount $SCRATCH_MNT/test >> $seqres.full 2>&1 \
     || _fail "!!! umount xfs failed"
 
 echo "--- mounts at end (before cleanup)" >> $seqres.full
tests/xfs/073 +3 -3
index f955771..38ed2cb 100755
@@ -41,9 +41,9 @@ _cleanup()
 {
 	cd /
 	umount $SCRATCH_MNT 2>/dev/null
-	umount -d $imgs.loop 2>/dev/null
+	umount $imgs.loop 2>/dev/null
 	[ -d $imgs.loop ] && rmdir $imgs.loop
-	umount -d $imgs.source_dir 2>/dev/null
+	umount $imgs.source_dir 2>/dev/null
 	[ -d $imgs.source_dir ] && rm -rf $imgs.source_dir
 	rm -f $imgs.* $tmp.* /var/tmp/xfs_copy.log.*
 }
@@ -119,7 +119,7 @@ _verify_copy()
 
 	echo unmounting and removing new image
 	umount $source_dir
-	umount -d $target_dir > /dev/null 2>&1
+	umount $target_dir > /dev/null 2>&1
 	rm -f $target
 }
 
tests/xfs/078 +2 -2
index f859efc..d8cb919 100755
@@ -36,7 +36,7 @@ _cleanup()
 {
     cd /
     rm -f $tmp.*
-    umount -d $LOOP_MNT 2>/dev/null
+    umount $LOOP_MNT 2>/dev/null
     rmdir $LOOP_MNT
 }
 
@@ -97,7 +97,7 @@ _grow_loop()
 	$XFS_GROWFS_PROG $LOOP_MNT 2>&1 |  _filter_growfs 2>&1
 
 	echo "*** unmount"
-	umount -d $LOOP_MNT > /dev/null 2>&1
+	umount $LOOP_MNT > /dev/null 2>&1
 
 	# Large grows takes forever to check..
 	if [ "$check" -gt "0" ]
tests/xfs/216 +1 -1
index 8513479..76f79ca 100755
@@ -60,7 +60,7 @@ _do_mkfs()
 			-d name=$LOOP_DEV,size=${i}g |grep log
 		mount -o loop -t xfs $LOOP_DEV $LOOP_MNT
 		echo "test write" > $LOOP_MNT/test
-		umount -d $LOOP_MNT > /dev/null 2>&1
+		umount $LOOP_MNT > /dev/null 2>&1
 	done
 }
 # make large holey file
tests/xfs/217 +1 -1
index ab55a30..8aacdf9 100755
@@ -62,7 +62,7 @@ _do_mkfs()
 			-d name=$LOOP_DEV,size=${i}g |grep log
 		mount -o loop -t xfs $LOOP_DEV $LOOP_MNT
 		echo "test write" > $LOOP_MNT/test
-		umount -d $LOOP_MNT > /dev/null 2>&1
+		umount $LOOP_MNT > /dev/null 2>&1
 
 		# punch out the previous blocks so that we keep the amount of
 		# disk space the test requires down to a minimum.
tests/xfs/250 +2 -2
index c1622a4..0cdc382 100755
@@ -33,7 +33,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 _cleanup()
 {
 	cd /
-	umount -d $LOOP_MNT 2>/dev/null
+	umount $LOOP_MNT 2>/dev/null
 	rm -f $LOOP_DEV
 	rmdir $LOOP_MNT
 }
@@ -84,7 +84,7 @@ _test_loop()
 	xfs_io -f -c "resvsp 0 $fsize" $LOOP_MNT/foo | _filter_io
 
 	echo "*** unmount loop filesystem"
-	umount -d $LOOP_MNT > /dev/null 2>&1
+	umount $LOOP_MNT > /dev/null 2>&1
 
 	echo "*** check loop filesystem"
 	 _check_xfs_filesystem $LOOP_DEV none none
Signature · 1 line
2.0.0
Reviewed-by on v2 3/4
#22 lore raw

Re: [PATCH 3/4] xfs/049: umount -d fails when kernel wins teardown race

Quoted 18 lines · Dave Chinner

From: Dave Chinner <[email protected]>

When /etc/mtab is linked to /proc/mounts and we are using mount time created loop devices (i.e. mount -o loop), the unmount can fail with this amazingly informative error message:

umount: /mnt/scratch/test2: filesystem was unmounted, but mount(8) failed: Invalid argument

What it actually means in this case is that the kernel tore down the loop device when the last reference went away, and it did it so fast that mount was not able to find it in /etc/mtab after the unmount syscall. Hence it could not find the loop device it was supposed to tear down and has a hissy fit.

This is simple to fix: mount does not need to tear down the loop device as the kernel does it automatically. Remove the "-d" from the umount command, and the test passes again.

Quoted 5 lines

There's quite a few other tests that also use umount -d - fix them as well.

---

Quoted 139 lines
 tests/xfs/049 | 8 ++++----
 tests/xfs/073 | 6 +++---
 tests/xfs/078 | 4 ++--
 tests/xfs/216 | 2 +-
 tests/xfs/217 | 2 +-
 tests/xfs/250 | 4 ++--
 6 files changed, 13 insertions(+), 13 deletions(-)
tests/xfs/049 +4 -4
index 04c2c75..8d4e074 100755
@@ -29,8 +29,8 @@ echo "QA output created by $seq"
 _cleanup()
 {
     cd /
-    umount -d $SCRATCH_MNT/test2 > /dev/null 2>&1
-    umount -d $SCRATCH_MNT/test > /dev/null 2>&1
+    umount $SCRATCH_MNT/test2 > /dev/null 2>&1
+    umount $SCRATCH_MNT/test > /dev/null 2>&1
     rm -f $tmp.*
 
     if [ -w $seqres.full ]
@@ -123,11 +123,11 @@ rm -rf $SCRATCH_MNT/test/* >> $seqres.full 2>&1 \
     || _fail "!!! clean failed"
 
 _log "umount ext2 on xfs"
-umount -d $SCRATCH_MNT/test2 >> $seqres.full 2>&1 \
+umount $SCRATCH_MNT/test2 >> $seqres.full 2>&1 \
     || _fail "!!! umount ext2 failed"
 
 _log "umount xfs"
-umount -d $SCRATCH_MNT/test >> $seqres.full 2>&1 \
+umount $SCRATCH_MNT/test >> $seqres.full 2>&1 \
     || _fail "!!! umount xfs failed"
 
 echo "--- mounts at end (before cleanup)" >> $seqres.full
tests/xfs/073 +3 -3
index f955771..38ed2cb 100755
@@ -41,9 +41,9 @@ _cleanup()
 {
 	cd /
 	umount $SCRATCH_MNT 2>/dev/null
-	umount -d $imgs.loop 2>/dev/null
+	umount $imgs.loop 2>/dev/null
 	[ -d $imgs.loop ] && rmdir $imgs.loop
-	umount -d $imgs.source_dir 2>/dev/null
+	umount $imgs.source_dir 2>/dev/null
 	[ -d $imgs.source_dir ] && rm -rf $imgs.source_dir
 	rm -f $imgs.* $tmp.* /var/tmp/xfs_copy.log.*
 }
@@ -119,7 +119,7 @@ _verify_copy()
 
 	echo unmounting and removing new image
 	umount $source_dir
-	umount -d $target_dir > /dev/null 2>&1
+	umount $target_dir > /dev/null 2>&1
 	rm -f $target
 }
 
tests/xfs/078 +2 -2
index f859efc..d8cb919 100755
@@ -36,7 +36,7 @@ _cleanup()
 {
     cd /
     rm -f $tmp.*
-    umount -d $LOOP_MNT 2>/dev/null
+    umount $LOOP_MNT 2>/dev/null
     rmdir $LOOP_MNT
 }
 
@@ -97,7 +97,7 @@ _grow_loop()
 	$XFS_GROWFS_PROG $LOOP_MNT 2>&1 |  _filter_growfs 2>&1
 
 	echo "*** unmount"
-	umount -d $LOOP_MNT > /dev/null 2>&1
+	umount $LOOP_MNT > /dev/null 2>&1
 
 	# Large grows takes forever to check..
 	if [ "$check" -gt "0" ]
tests/xfs/216 +1 -1
index 8513479..76f79ca 100755
@@ -60,7 +60,7 @@ _do_mkfs()
 			-d name=$LOOP_DEV,size=${i}g |grep log
 		mount -o loop -t xfs $LOOP_DEV $LOOP_MNT
 		echo "test write" > $LOOP_MNT/test
-		umount -d $LOOP_MNT > /dev/null 2>&1
+		umount $LOOP_MNT > /dev/null 2>&1
 	done
 }
 # make large holey file
tests/xfs/217 +1 -1
index ab55a30..8aacdf9 100755
@@ -62,7 +62,7 @@ _do_mkfs()
 			-d name=$LOOP_DEV,size=${i}g |grep log
 		mount -o loop -t xfs $LOOP_DEV $LOOP_MNT
 		echo "test write" > $LOOP_MNT/test
-		umount -d $LOOP_MNT > /dev/null 2>&1
+		umount $LOOP_MNT > /dev/null 2>&1
 
 		# punch out the previous blocks so that we keep the amount of
 		# disk space the test requires down to a minimum.
tests/xfs/250 +2 -2
index c1622a4..0cdc382 100755
@@ -33,7 +33,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 _cleanup()
 {
 	cd /
-	umount -d $LOOP_MNT 2>/dev/null
+	umount $LOOP_MNT 2>/dev/null
 	rm -f $LOOP_DEV
 	rmdir $LOOP_MNT
 }
@@ -84,7 +84,7 @@ _test_loop()
 	xfs_io -f -c "resvsp 0 $fsize" $LOOP_MNT/foo | _filter_io
 
 	echo "*** unmount loop filesystem"
-	umount -d $LOOP_MNT > /dev/null 2>&1
+	umount $LOOP_MNT > /dev/null 2>&1
 
 	echo "*** check loop filesystem"
 	 _check_xfs_filesystem $LOOP_DEV none none
Signature · 6 lines
2.0.0

--
To unsubscribe from this list: send the line "unsubscribe fstests" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#23 lore raw

PATCH v2 4/4 generic: Add rudimetary RENAME_WHITEOUT test

In reply to Dave Chinner

From: Dave Chinner <[email protected]>

There is no API documentation for RENAME_WHITEOUT. There is no developer documentation for RENAME_WHITEOUT. There are not comments in the overlayfs or ext4 implementation of RENAME_WHITEOUT.

Hence, this test simply tries to expose basic RENAME_WHITEOUT behaviour from ext4 so we can reverse-engineer and verify bug-for-bug renameat2(RENAME_WHITEOUT) ext4 compatibility.

Note: uses generic/078 just to keep out of the way of the 6-7 other pending new tests.

---
 src/renameat2.c       |  4 ++--
 tests/generic/078     | 66 +++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/generic/078.out | 51 +++++++++++++++++++++++++++++++++++++++
 tests/generic/group   |  1 +
 4 files changed, 120 insertions(+), 2 deletions(-)
 create mode 100755 tests/generic/078
 create mode 100644 tests/generic/078.out
src/renameat2.c +2 -2
index 5ac0936..c59ce65 100644
@@ -96,9 +96,9 @@ int main(int argc, char *argv[])
 		 * Turn EEXIST into ENOTEMPTY.  E.g. XFS uses EEXIST, and that
 		 * is also accepted by the standards.
 		 *
-		 * This applies only to plain rename (flags == 0).
+		 * This applies only to plain rename and RENAME_WHITEOUT
 		 */
-		if (!flags && errno == EEXIST)
+		if (errno == EEXIST && (!flags || (flags & RENAME_WHITEOUT)))
 			errno = ENOTEMPTY;
 
 		perror("");
tests/generic/078 new file +66 -0

66 lines not on this page · raw message

tests/generic/078.out new file +51 -0

51 lines not on this page · raw message

tests/generic/group +1 -0
index f2eb87a..0f274a6 100644
@@ -69,6 +69,7 @@
 075 rw udf auto quick
 076 metadata rw udf auto quick stress
 077 acl attr auto enospc
+078 auto quick metadata
 079 acl attr ioctl metadata auto quick
 083 rw auto enospc stress
 088 perms auto quick
Signature · 1 line
2.0.0
Reviewed-by on v2 4/4
#24 lore raw

Re: [PATCH 4/4] generic: Add rudimetary RENAME_WHITEOUT test

Quoted 10 lines · Dave Chinner

From: Dave Chinner <[email protected]>

There is no API documentation for RENAME_WHITEOUT. There is no developer documentation for RENAME_WHITEOUT. There are not comments in the overlayfs or ext4 implementation of RENAME_WHITEOUT.

Hence, this test simply tries to expose basic RENAME_WHITEOUT behaviour from ext4 so we can reverse-engineer and verify bug-for-bug renameat2(RENAME_WHITEOUT) ext4 compatibility.

Quoted 5 lines

Note: uses generic/078 just to keep out of the way of the 6-7 other pending new tests.

---

Quoted 172 lines
 src/renameat2.c       |  4 ++--
 tests/generic/078     | 66 +++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/generic/078.out | 51 +++++++++++++++++++++++++++++++++++++++
 tests/generic/group   |  1 +
 4 files changed, 120 insertions(+), 2 deletions(-)
 create mode 100755 tests/generic/078
 create mode 100644 tests/generic/078.out
src/renameat2.c +2 -2
index 5ac0936..c59ce65 100644
@@ -96,9 +96,9 @@ int main(int argc, char *argv[])
 		 * Turn EEXIST into ENOTEMPTY.  E.g. XFS uses EEXIST, and that
 		 * is also accepted by the standards.
 		 *
-		 * This applies only to plain rename (flags == 0).
+		 * This applies only to plain rename and RENAME_WHITEOUT
 		 */
-		if (!flags && errno == EEXIST)
+		if (errno == EEXIST && (!flags || (flags & RENAME_WHITEOUT)))
 			errno = ENOTEMPTY;
 
 		perror("");
tests/generic/078 new file +66 -0
new file mode 100755
index 0000000..92ece0c
@@ -0,0 +1,66 @@
+#! /bin/bash
+# FS QA Test No. generic/078
+#
+# Check renameat2 syscall with RENAME_WHITEOUT flag
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2014 Miklos Szeredi.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#-----------------------------------------------------------------------
+#
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+    cd /
+    rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/renameat2
+
+_supported_fs generic
+_supported_os Linux
+
+_require_test
+_requires_renameat2
+_require_test_symlinks
+
+rename_dir=$TEST_DIR/$$
+mkdir $rename_dir
+touch $rename_dir/foo $rename_dir/bar
+if ! src/renameat2 -t -w $rename_dir/foo $rename_dir/bar; then
+    rm -f $rename_dir/foo $rename_dir/bar; rmdir $rename_dir
+    _notrun "fs doesn't support RENAME_WHITEOUT"
+fi
+rm -f $rename_dir/foo $rename_dir/bar
+
+# real QA test starts here
+
+_rename_tests $rename_dir -w
+rmdir $rename_dir
+
+# success, all done
+status=0
+exit
tests/generic/078.out new file +51 -0
new file mode 100644
index 0000000..5d5e3a0
@@ -0,0 +1,51 @@
+QA output created by 078
+samedir  none/none -> No such file or directory
+samedir  none/regu -> No such file or directory
+samedir  none/symb -> No such file or directory
+samedir  none/dire -> No such file or directory
+samedir  none/tree -> No such file or directory
+samedir  regu/none -> char/regu.
+samedir  regu/regu -> char/regu.
+samedir  regu/symb -> char/regu.
+samedir  regu/dire -> Is a directory
+samedir  regu/tree -> Is a directory
+samedir  symb/none -> char/symb.
+samedir  symb/regu -> char/symb.
+samedir  symb/symb -> char/symb.
+samedir  symb/dire -> Is a directory
+samedir  symb/tree -> Is a directory
+samedir  dire/none -> char/dire.
+samedir  dire/regu -> Not a directory
+samedir  dire/symb -> Not a directory
+samedir  dire/dire -> char/dire.
+samedir  dire/tree -> Directory not empty
+samedir  tree/none -> char/tree.
+samedir  tree/regu -> Not a directory
+samedir  tree/symb -> Not a directory
+samedir  tree/dire -> char/tree.
+samedir  tree/tree -> Directory not empty
+crossdir none/none -> No such file or directory
+crossdir none/regu -> No such file or directory
+crossdir none/symb -> No such file or directory
+crossdir none/dire -> No such file or directory
+crossdir none/tree -> No such file or directory
+crossdir regu/none -> char/regu.
+crossdir regu/regu -> char/regu.
+crossdir regu/symb -> char/regu.
+crossdir regu/dire -> Is a directory
+crossdir regu/tree -> Is a directory
+crossdir symb/none -> char/symb.
+crossdir symb/regu -> char/symb.
+crossdir symb/symb -> char/symb.
+crossdir symb/dire -> Is a directory
+crossdir symb/tree -> Is a directory
+crossdir dire/none -> char/dire.
+crossdir dire/regu -> Not a directory
+crossdir dire/symb -> Not a directory
+crossdir dire/dire -> char/dire.
+crossdir dire/tree -> Directory not empty
+crossdir tree/none -> char/tree.
+crossdir tree/regu -> Not a directory
+crossdir tree/symb -> Not a directory
+crossdir tree/dire -> char/tree.
+crossdir tree/tree -> Directory not empty
tests/generic/group +1 -0
index f2eb87a..0f274a6 100644
@@ -69,6 +69,7 @@
 075 rw udf auto quick
 076 metadata rw udf auto quick stress
 077 acl attr auto enospc
+078 auto quick metadata
 079 acl attr ioctl metadata auto quick
 083 rw auto enospc stress
 088 perms auto quick
Signature · 6 lines
2.0.0

--
To unsubscribe from this list: send the line "unsubscribe fstests" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html