Skip to contentSkip to conversation

xfstests failures

DISCUSSION Dave Jiang 6 participants 13 messages Feb 5, 2018 – Feb 6, 2018

lore
Share & tools
lore

Dave Jiang

author
#0 lore raw

Eryu,
I've noticed that these tests fails under what I think is a normal config (BRD of 48G). We have an expectation that for simple configs all tests in the 'auto' group should pass, and these ones don't. Are these false positive failures? If so, what do we need to do to remove these false positives? a) fix the tests to handle these cases b) remove the tests from the 'auto' group? Something else? Attached file with test outputs. I think some if not all of these failures have lasted many kernel versions.

# xfs
generic/009
generic/012
generic/016
generic/021
generic/022
generic/058
generic/060
generic/061
generic/063
generic/092
generic/255
xfs/167
xfs/191-input-validation
xfs/242
xfs/252
xfs/432
# ext4
generic/388
Signature · 4 lines
Dave Jiang
Software Engineer, SSG/OTC
Intel Corp.
[email protected]

# XFS failures

# ./check generic/009
FSTYP         -- xfs (non-debug)
PLATFORM      -- Linux/x86_64 skx-ntbusd 4.15.0+
MKFS_OPTIONS  -- -f -bsize=4096 /dev/ram0p2
MOUNT_OPTIONS -- -o context=system_u:object_r:root_t:s0 /dev/ram0p2 /mnt/xfstests_scratch

generic/009 - output mismatch (see /root/xfstests/xfstests-dev/results//generic/009.out.bad)

    --- tests/generic/009.out	2016-09-09 09:30:36.006800609 -0700
    +++ /root/xfstests/xfstests-dev/results//generic/009.out.bad	2018-02-05 13:24:23.702640408 -0700
    @@ -1,79 +1,75 @@
     QA output created by 009
     	1. into a hole
    -0: [0..7]: hole
    -1: [8..23]: unwritten
    -2: [24..39]: hole
    +0: [0..4095]: unwritten
     daa100df6e6711906b61c9ab5aa16032
    ...
    (Run 'diff -u tests/generic/009.out /root/xfstests/xfstests-dev/results//generic/009.out.bad'  to see the entire diff)

Ran: generic/009
Failures: generic/009
Failed 1 of 1 tests

# ./check generic/012
FSTYP         -- xfs (non-debug)
PLATFORM      -- Linux/x86_64 skx-ntbusd 4.15.0+
MKFS_OPTIONS  -- -f -bsize=4096 /dev/ram0p2
MOUNT_OPTIONS -- -o context=system_u:object_r:root_t:s0 /dev/ram0p2 /mnt/xfstests_scratch

generic/012 - output mismatch (see /root/xfstests/xfstests-dev/results//generic/012.out.bad)

    --- tests/generic/012.out	2016-09-09 09:30:36.007800624 -0700
    +++ /root/xfstests/xfstests-dev/results//generic/012.out.bad	2018-02-05 13:25:20.226260972 -0700
    @@ -8,28 +8,24 @@
     0: [0..95]: extent
     f07217d5ac7ffa15dd8910c4aa912674
     	4. hole -> data
    -0: [0..63]: extent
    -1: [64..95]: hole
    +0: [0..95]: extent
     e5c94f6299822646f9f57aeacd8bdc01
    ...
    (Run 'diff -u tests/generic/012.out /root/xfstests/xfstests-dev/results//generic/012.out.bad'  to see the entire diff)

Ran: generic/012
Failures: generic/012
Failed 1 of 1 tests

# ./check generic/016
FSTYP         -- xfs (non-debug)
PLATFORM      -- Linux/x86_64 skx-ntbusd 4.15.0+
MKFS_OPTIONS  -- -f -bsize=4096 /dev/ram0p2
MOUNT_OPTIONS -- -o context=system_u:object_r:root_t:s0 /dev/ram0p2 /mnt/xfstests_scratch

generic/016 - output mismatch (see /root/xfstests/xfstests-dev/results//generic/016.out.bad)

    --- tests/generic/016.out	2016-09-09 09:30:36.007800624 -0700
    +++ /root/xfstests/xfstests-dev/results//generic/016.out.bad	2018-02-05 13:25:39.645923737 -0700
    @@ -8,28 +8,24 @@
     0: [0..95]: extent
     f07217d5ac7ffa15dd8910c4aa912674
     	4. hole -> data
    -0: [0..63]: extent
    -1: [64..95]: hole
    +0: [0..95]: extent
     e5c94f6299822646f9f57aeacd8bdc01
    ...
    (Run 'diff -u tests/generic/016.out /root/xfstests/xfstests-dev/results//generic/016.out.bad'  to see the entire diff)

Ran: generic/016
Failures: generic/016
Failed 1 of 1 tests

# ./check generic/021
FSTYP         -- xfs (non-debug)
PLATFORM      -- Linux/x86_64 skx-ntbusd 4.15.0+
MKFS_OPTIONS  -- -f -bsize=4096 /dev/ram0p2
MOUNT_OPTIONS -- -o context=system_u:object_r:root_t:s0 /dev/ram0p2 /mnt/xfstests_scratch

generic/021 - output mismatch (see /root/xfstests/xfstests-dev/results//generic/021.out.bad)

    --- tests/generic/021.out	2016-09-09 09:30:36.008800639 -0700
    +++ /root/xfstests/xfstests-dev/results//generic/021.out.bad	2018-02-05 13:25:51.877341178 -0700
    @@ -8,32 +8,25 @@
     0: [0..95]: extent
     f4f35d60b3cc18aaa6d8d92f0cd3708a
     	4. hole -> data
    -0: [0..31]: hole
    -1: [32..63]: extent
    -2: [64..95]: hole
    +0: [0..95]: extent
    ...
    (Run 'diff -u tests/generic/021.out /root/xfstests/xfstests-dev/results//generic/021.out.bad'  to see the entire diff)

Ran: generic/021
Failures: generic/021
Failed 1 of 1 tests

# ./check generic/022
FSTYP         -- xfs (non-debug)
PLATFORM      -- Linux/x86_64 skx-ntbusd 4.15.0+
MKFS_OPTIONS  -- -f -bsize=4096 /dev/ram0p2
MOUNT_OPTIONS -- -o context=system_u:object_r:root_t:s0 /dev/ram0p2 /mnt/xfstests_scratch

generic/022 - output mismatch (see /root/xfstests/xfstests-dev/results//generic/022.out.bad)

    --- tests/generic/022.out	2016-09-09 09:30:36.008800639 -0700
    +++ /root/xfstests/xfstests-dev/results//generic/022.out.bad	2018-02-05 13:26:26.001510568 -0700
    @@ -8,32 +8,25 @@
     0: [0..95]: extent
     f4f35d60b3cc18aaa6d8d92f0cd3708a
     	4. hole -> data
    -0: [0..31]: hole
    -1: [32..63]: extent
    -2: [64..95]: hole
    +0: [0..95]: extent
    ...
    (Run 'diff -u tests/generic/022.out /root/xfstests/xfstests-dev/results//generic/022.out.bad'  to see the entire diff)

Ran: generic/022
Failures: generic/022
Failed 1 of 1 tests

# ./check generic/058
FSTYP         -- xfs (non-debug)
PLATFORM      -- Linux/x86_64 skx-ntbusd 4.15.0+
MKFS_OPTIONS  -- -f -bsize=4096 /dev/ram0p2
MOUNT_OPTIONS -- -o context=system_u:object_r:root_t:s0 /dev/ram0p2 /mnt/xfstests_scratch

generic/058 - output mismatch (see /root/xfstests/xfstests-dev/results//generic/058.out.bad)

    --- tests/generic/058.out	2016-09-09 09:30:36.014800728 -0700
    +++ /root/xfstests/xfstests-dev/results//generic/058.out.bad	2018-02-05 13:26:44.932170828 -0700
    @@ -4,75 +4,71 @@
     	2. into allocated space
     0: [0..7]: extent
     1: [8..23]: hole
    -2: [24..55]: extent
    +2: [24..71]: extent
     64e72217eebcbdf31b1b058f9f5f476a
     	3. into unwritten space
    ...
    (Run 'diff -u tests/generic/058.out /root/xfstests/xfstests-dev/results//generic/058.out.bad'  to see the entire diff)

Ran: generic/058
Failures: generic/058
Failed 1 of 1 tests

# ./check generic/060
FSTYP         -- xfs (non-debug)
PLATFORM      -- Linux/x86_64 skx-ntbusd 4.15.0+
MKFS_OPTIONS  -- -f -bsize=4096 /dev/ram0p2
MOUNT_OPTIONS -- -o context=system_u:object_r:root_t:s0 /dev/ram0p2 /mnt/xfstests_scratch

generic/060 - output mismatch (see /root/xfstests/xfstests-dev/results//generic/060.out.bad)

    --- tests/generic/060.out	2016-09-09 09:30:36.014800728 -0700
    +++ /root/xfstests/xfstests-dev/results//generic/060.out.bad	2018-02-05 13:27:01.555750621 -0700
    @@ -4,75 +4,71 @@
     	2. into allocated space
     0: [0..7]: extent
     1: [8..23]: hole
    -2: [24..55]: extent
    +2: [24..71]: extent
     64e72217eebcbdf31b1b058f9f5f476a
     	3. into unwritten space
    ...
    (Run 'diff -u tests/generic/060.out /root/xfstests/xfstests-dev/results//generic/060.out.bad'  to see the entire diff)

Ran: generic/060
Failures: generic/060
Failed 1 of 1 tests

# ./check generic/061
FSTYP         -- xfs (non-debug)
PLATFORM      -- Linux/x86_64 skx-ntbusd 4.15.0+
MKFS_OPTIONS  -- -f -bsize=4096 /dev/ram0p2
MOUNT_OPTIONS -- -o context=system_u:object_r:root_t:s0 /dev/ram0p2 /mnt/xfstests_scratch

generic/061 - output mismatch (see /root/xfstests/xfstests-dev/results//generic/061.out.bad)

    --- tests/generic/061.out	2016-09-09 09:30:36.014800728 -0700
    +++ /root/xfstests/xfstests-dev/results//generic/061.out.bad	2018-02-05 13:27:20.943426820 -0700
    @@ -4,7 +4,7 @@
     	2. into allocated space
     0: [0..7]: extent
     1: [8..23]: hole
    -2: [24..55]: extent
    +2: [24..71]: extent
     64e72217eebcbdf31b1b058f9f5f476a
     	3. into unwritten space
    ...
    (Run 'diff -u tests/generic/061.out /root/xfstests/xfstests-dev/results//generic/061.out.bad'  to see the entire diff)

Ran: generic/061
Failures: generic/061
Failed 1 of 1 tests

# ./check generic/063
FSTYP         -- xfs (non-debug)
PLATFORM      -- Linux/x86_64 skx-ntbusd 4.15.0+
MKFS_OPTIONS  -- -f -bsize=4096 /dev/ram0p2
MOUNT_OPTIONS -- -o context=system_u:object_r:root_t:s0 /dev/ram0p2 /mnt/xfstests_scratch

generic/063 - output mismatch (see /root/xfstests/xfstests-dev/results//generic/063.out.bad)

    --- tests/generic/063.out	2016-09-09 09:30:36.015800743 -0700
    +++ /root/xfstests/xfstests-dev/results//generic/063.out.bad	2018-02-05 13:27:44.177252365 -0700
    @@ -4,7 +4,7 @@
     	2. into allocated space
     0: [0..7]: extent
     1: [8..23]: hole
    -2: [24..55]: extent
    +2: [24..71]: extent
     64e72217eebcbdf31b1b058f9f5f476a
     	3. into unwritten space
    ...
    (Run 'diff -u tests/generic/063.out /root/xfstests/xfstests-dev/results//generic/063.out.bad'  to see the entire diff)

Ran: generic/063
Failures: generic/063
Failed 1 of 1 tests

# ./check generic/092
FSTYP         -- xfs (non-debug)
PLATFORM      -- Linux/x86_64 skx-ntbusd 4.15.0+
MKFS_OPTIONS  -- -f -bsize=4096 /dev/ram0p2
MOUNT_OPTIONS -- -o context=system_u:object_r:root_t:s0 /dev/ram0p2 /mnt/xfstests_scratch

generic/092 - output mismatch (see /root/xfstests/xfstests-dev/results//generic/092.out.bad)

    --- tests/generic/092.out	2016-09-09 09:30:36.022800847 -0700
    +++ /root/xfstests/xfstests-dev/results//generic/092.out.bad	2018-02-05 13:28:07.683089807 -0700
    @@ -3,4 +3,4 @@
     XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
     0: [0..10239]: data
     0: [0..10239]: data
    -1: [10240..20479]: unwritten
    +1: [10240..22527]: unwritten
    ...
    (Run 'diff -u tests/generic/092.out /root/xfstests/xfstests-dev/results//generic/092.out.bad'  to see the entire diff)

Ran: generic/092
Failures: generic/092
Failed 1 of 1 tests

# ./check generic/255
FSTYP         -- xfs (non-debug)
PLATFORM      -- Linux/x86_64 skx-ntbusd 4.15.0+
MKFS_OPTIONS  -- -f -bsize=4096 /dev/ram0p2
MOUNT_OPTIONS -- -o context=system_u:object_r:root_t:s0 /dev/ram0p2 /mnt/xfstests_scratch

generic/255 - output mismatch (see /root/xfstests/xfstests-dev/results//generic/255.out.bad)

    --- tests/generic/255.out	2016-09-09 09:30:36.265804467 -0700
    +++ /root/xfstests/xfstests-dev/results//generic/255.out.bad	2018-02-05 13:28:32.766049923 -0700
    @@ -4,71 +4,75 @@
     	2. into allocated space
     0: [0..7]: extent
     1: [8..23]: hole
    -2: [24..39]: extent
    +2: [24..4095]: extent
     cc58a7417c2d7763adc45b6fcd3fa024
     	3. into unwritten space
    ...
    (Run 'diff -u tests/generic/255.out /root/xfstests/xfstests-dev/results//generic/255.out.bad'  to see the entire diff)

Ran: generic/255
Failures: generic/255
Failed 1 of 1 tests

# ./check xfs/167
FSTYP         -- xfs (non-debug)
PLATFORM      -- Linux/x86_64 skx-ntbusd 4.15.0+
MKFS_OPTIONS  -- -f -bsize=4096 /dev/ram0p2
MOUNT_OPTIONS -- -o context=system_u:object_r:root_t:s0 /dev/ram0p2 /mnt/xfstests_scratch

xfs/167 - output mismatch (see /root/xfstests/xfstests-dev/results//xfs/167.out.bad)
--- tests/xfs/167.out 2016-09-09 09:30:36.483807714 -0700
+++ /root/xfstests/xfstests-dev/results//xfs/167.out.bad 2018-02-05 13:29:28.210828339 -0700
@@ -1,3 +1,4 @@
QA output created by 167
*** test unwritten extent conversion under heavy I/O
+xfsctl : No space left on device
*** test done
...
(Run 'diff -u tests/xfs/167.out /root/xfstests/xfstests-dev/results//xfs/167.out.bad' to see the entire diff)
Ran: xfs/167
Failures: xfs/167
Failed 1 of 1 tests

# ./check xfs/191-input-validation
FSTYP         -- xfs (non-debug)
PLATFORM      -- Linux/x86_64 skx-ntbusd 4.15.0+
MKFS_OPTIONS  -- -f -bsize=4096 /dev/ram0p2
MOUNT_OPTIONS -- -o context=system_u:object_r:root_t:s0 /dev/ram0p2 /mnt/xfstests_scratch

xfs/191-input-validation - output mismatch (see /root/xfstests/xfstests-dev/results//xfs/191-input-validation.out.bad)

    --- tests/xfs/191-input-validation.out	2016-09-09 09:30:36.496807908 -0700
    +++ /root/xfstests/xfstests-dev/results//xfs/191-input-validation.out.bad	2018-02-05 13:30:03.702874689 -0700
    @@ -1,2 +1,3 @@
     QA output created by 191-input-validation
     silence is golden
    +pass -l version=1 -m crc=1 /dev/ram0p2
    ...
    (Run 'diff -u tests/xfs/191-input-validation.out /root/xfstests/xfstests-dev/results//xfs/191-input-validation.out.bad'  to see the entire diff)

Ran: xfs/191-input-validation
Failures: xfs/191-input-validation
Failed 1 of 1 tests

# ./check xfs/242
FSTYP         -- xfs (non-debug)
PLATFORM      -- Linux/x86_64 skx-ntbusd 4.15.0+
MKFS_OPTIONS  -- -f -bsize=4096 /dev/ram0p2
MOUNT_OPTIONS -- -o context=system_u:object_r:root_t:s0 /dev/ram0p2 /mnt/xfstests_scratch

xfs/242 - output mismatch (see /root/xfstests/xfstests-dev/results//xfs/242.out.bad)

    --- tests/xfs/242.out	2016-09-09 09:30:36.523808310 -0700
    +++ /root/xfstests/xfstests-dev/results//xfs/242.out.bad	2018-02-05 13:30:36.511748601 -0700
    @@ -1,79 +1,75 @@
     QA output created by 242
     	1. into a hole
    -0: [0..7]: hole
    -1: [8..23]: unwritten
    -2: [24..39]: hole
    +0: [0..4095]: unwritten
     daa100df6e6711906b61c9ab5aa16032
    ...
    (Run 'diff -u tests/xfs/242.out /root/xfstests/xfstests-dev/results//xfs/242.out.bad'  to see the entire diff)

Ran: xfs/242
Failures: xfs/242
Failed 1 of 1 tests

# ./check xfs/252
FSTYP         -- xfs (non-debug)
PLATFORM      -- Linux/x86_64 skx-ntbusd 4.15.0+
MKFS_OPTIONS  -- -f -bsize=4096 /dev/ram0p2
MOUNT_OPTIONS -- -o context=system_u:object_r:root_t:s0 /dev/ram0p2 /mnt/xfstests_scratch

xfs/252 - output mismatch (see /root/xfstests/xfstests-dev/results//xfs/252.out.bad)

    --- tests/xfs/252.out	2016-09-09 09:30:36.531808429 -0700
    +++ /root/xfstests/xfstests-dev/results//xfs/252.out.bad	2018-02-05 13:30:51.608198508 -0700
    @@ -5,73 +5,85 @@
     0: [0..7]: data
     1: [8..23]: hole
     2: [24..39]: data
    +3: [40..4095]: unwritten
     cc58a7417c2d7763adc45b6fcd3fa024
     	3. into unwritten space
     0: [0..7]: unwritten
    ...
    (Run 'diff -u tests/xfs/252.out /root/xfstests/xfstests-dev/results//xfs/252.out.bad'  to see the entire diff)

Ran: xfs/252
Failures: xfs/252
Failed 1 of 1 tests

# EXT4 failures

# ./check generic/388
FSTYP         -- ext4
PLATFORM      -- Linux/x86_64 skx-ntbusd 4.15.0+
MKFS_OPTIONS  -- /dev/ram0p2
MOUNT_OPTIONS -- -o acl,user_xattr -o context=system_u:object_r:root_t:s0 /dev/ram0p2 /mnt/xfstests_scratch

generic/388 68s ... 70s
_check_generic_filesystem: filesystem on /dev/ram0p2 is inconsistent (see /root/xfstests/xfstests-dev/results//generic/388.full for details)
Ran: generic/388
Failures: generic/388
Failed 1 of 1 tests

#1 lore raw
Quoted 19 lines · Dave Jiang

Eryu,
I've noticed that these tests fails under what I think is a normal config (BRD of 48G). We have an expectation that for simple configs all tests in the 'auto' group should pass, and these ones don't. Are these false positive failures? If so, what do we need to do to remove these false positives? a) fix the tests to handle these cases b) remove the tests from the 'auto' group? Something else? Attached file with test outputs. I think some if not all of these failures have lasted many kernel versions.

# xfs
generic/009
generic/012
generic/016
generic/021
generic/022
generic/058
generic/060
generic/061
Quoted 8 lines
generic/063
generic/092
generic/255
xfs/167
xfs/191-input-validation
xfs/242
xfs/252
xfs/432

Except for xfs/191, these all look to be extent mapping failures. i.e. there's one bug or config issue that is causing them all.

Quoted 29 lines
# ext4
generic/388
Signature · 23 lines
Dave Jiang
Software Engineer, SSG/OTC
Intel Corp.
[email protected]

# XFS failures

# ./check generic/009
FSTYP         -- xfs (non-debug)
PLATFORM      -- Linux/x86_64 skx-ntbusd 4.15.0+
MKFS_OPTIONS  -- -f -bsize=4096 /dev/ram0p2
MOUNT_OPTIONS -- -o context=system_u:object_r:root_t:s0 /dev/ram0p2 /mnt/xfstests_scratch

generic/009	 - output mismatch (see /root/xfstests/xfstests-dev/results//generic/009.out.bad)
    --- tests/generic/009.out	2016-09-09 09:30:36.006800609 -0700
    +++ /root/xfstests/xfstests-dev/results//generic/009.out.bad	2018-02-05 13:24:23.702640408 -0700
    @@ -1,79 +1,75 @@
     QA output created by 009
     	1. into a hole
    -0: [0..7]: hole
    -1: [8..23]: unwritten
    -2: [24..39]: hole
    +0: [0..4095]: unwritten

You're getting a 2MB extent allocated here. I'm guessing your testdev is configured with a 2MB extent size hint or something similar left over from trying to test DAX w/ 2MB huge pages?

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]
author
#2 lore raw
Quoted 62 lines · Dave Chinner
Quoted 19 lines · Dave Jiang

Eryu,
I've noticed that these tests fails under what I think is a normal config (BRD of 48G). We have an expectation that for simple configs all tests in the 'auto' group should pass, and these ones don't. Are these false positive failures? If so, what do we need to do to remove these false positives? a) fix the tests to handle these cases b) remove the tests from the 'auto' group? Something else? Attached file with test outputs. I think some if not all of these failures have lasted many kernel versions.

# xfs
generic/009
generic/012
generic/016
generic/021
generic/022
generic/058
generic/060
generic/061
Quoted 8 lines
generic/063
generic/092
generic/255
xfs/167
xfs/191-input-validation
xfs/242
xfs/252
xfs/432

Except for xfs/191, these all look to be extent mapping failures. i.e. there's one bug or config issue that is causing them all.

Quoted 29 lines
# ext4
generic/388
Signature · 23 lines
Dave Jiang
Software Engineer, SSG/OTC
Intel Corp.
[email protected]

# XFS failures

# ./check generic/009
FSTYP         -- xfs (non-debug)
PLATFORM      -- Linux/x86_64 skx-ntbusd 4.15.0+
MKFS_OPTIONS  -- -f -bsize=4096 /dev/ram0p2
MOUNT_OPTIONS -- -o context=system_u:object_r:root_t:s0 /dev/ram0p2 /mnt/xfstests_scratch

generic/009	 - output mismatch (see /root/xfstests/xfstests-dev/results//generic/009.out.bad)
    --- tests/generic/009.out	2016-09-09 09:30:36.006800609 -0700
    +++ /root/xfstests/xfstests-dev/results//generic/009.out.bad	2018-02-05 13:24:23.702640408 -0700
    @@ -1,79 +1,75 @@
     QA output created by 009
     	1. into a hole
    -0: [0..7]: hole
    -1: [8..23]: unwritten
    -2: [24..39]: hole
    +0: [0..4095]: unwritten
Quoted 3 lines

You're getting a 2MB extent allocated here. I'm guessing your testdev is configured with a 2MB extent size hint or something similar left over from trying to test DAX w/ 2MB huge pages?

Yes. Looks like the config script was setting 2M extent. After removing and retesting xfs/191 and xfs/432 fails.

# ./check xfs/432
FSTYP         -- xfs (non-debug)
PLATFORM      -- Linux/x86_64 skx-ntbusd 4.15.0+
MKFS_OPTIONS  -- -f -bsize=4096 /dev/ram0p2
MOUNT_OPTIONS -- -o context=system_u:object_r:root_t:s0 /dev/ram0p2
/mnt/xfstests_scratch

xfs/432 - output mismatch (see
/root/xfstests/xfstests-dev/results//xfs/432.out.bad)

    --- tests/xfs/432.out	2017-10-19 10:57:22.562819579 -0700
    +++ /root/xfstests/xfstests-dev/results//xfs/432.out.bad	2018-02-05

15:57:29.673255360 -0700
@@ -3,4 +3,5 @@
Create huge dir
Check for > 1000 block extent?
Try to metadump
+xfs_metadump: suspicious count 1088 in bmap extent 1 in dir3 ino 35
Check restored metadump image
...
(Run 'diff -u tests/xfs/432.out
/root/xfstests/xfstests-dev/results//xfs/432.out.bad' to see the entire diff)
Ran: xfs/432
Failures: xfs/432
Failed 1 of 1 tests

Quoted 5 lines

Cheers,

Dave.

#3 lore raw
Quoted 80 lines · Dave Jiang
Quoted 62 lines · Dave Chinner
Quoted 19 lines · Dave Jiang

Eryu,
I've noticed that these tests fails under what I think is a normal config (BRD of 48G). We have an expectation that for simple configs all tests in the 'auto' group should pass, and these ones don't. Are these false positive failures? If so, what do we need to do to remove these false positives? a) fix the tests to handle these cases b) remove the tests from the 'auto' group? Something else? Attached file with test outputs. I think some if not all of these failures have lasted many kernel versions.

# xfs
generic/009
generic/012
generic/016
generic/021
generic/022
generic/058
generic/060
generic/061
Quoted 8 lines
generic/063
generic/092
generic/255
xfs/167
xfs/191-input-validation
xfs/242
xfs/252
xfs/432

Except for xfs/191, these all look to be extent mapping failures. i.e. there's one bug or config issue that is causing them all.

Quoted 29 lines
# ext4
generic/388
Signature · 23 lines
Dave Jiang
Software Engineer, SSG/OTC
Intel Corp.
[email protected]

# XFS failures

# ./check generic/009
FSTYP         -- xfs (non-debug)
PLATFORM      -- Linux/x86_64 skx-ntbusd 4.15.0+
MKFS_OPTIONS  -- -f -bsize=4096 /dev/ram0p2
MOUNT_OPTIONS -- -o context=system_u:object_r:root_t:s0 /dev/ram0p2 /mnt/xfstests_scratch

generic/009	 - output mismatch (see /root/xfstests/xfstests-dev/results//generic/009.out.bad)
    --- tests/generic/009.out	2016-09-09 09:30:36.006800609 -0700
    +++ /root/xfstests/xfstests-dev/results//generic/009.out.bad	2018-02-05 13:24:23.702640408 -0700
    @@ -1,79 +1,75 @@
     QA output created by 009
     	1. into a hole
    -0: [0..7]: hole
    -1: [8..23]: unwritten
    -2: [24..39]: hole
    +0: [0..4095]: unwritten
Quoted 3 lines

You're getting a 2MB extent allocated here. I'm guessing your testdev is configured with a 2MB extent size hint or something similar left over from trying to test DAX w/ 2MB huge pages?

Yes. Looks like the config script was setting 2M extent. After removing and retesting xfs/191 and xfs/432 fails.

# ./check xfs/432
FSTYP         -- xfs (non-debug)
PLATFORM      -- Linux/x86_64 skx-ntbusd 4.15.0+
MKFS_OPTIONS  -- -f -bsize=4096 /dev/ram0p2
MOUNT_OPTIONS -- -o context=system_u:object_r:root_t:s0 /dev/ram0p2
/mnt/xfstests_scratch

xfs/432 - output mismatch (see
/root/xfstests/xfstests-dev/results//xfs/432.out.bad)

Quoted 8 lines
    --- tests/xfs/432.out	2017-10-19 10:57:22.562819579 -0700
    +++ /root/xfstests/xfstests-dev/results//xfs/432.out.bad	2018-02-05

15:57:29.673255360 -0700
@@ -3,4 +3,5 @@
Create huge dir
Check for > 1000 block extent?
Try to metadump
+xfs_metadump: suspicious count 1088 in bmap extent 1 in dir3 ino 35

What version of xfsprogs are you running? I fixed that a while ago... I think.

--D

Quoted 19 lines
     Check restored metadump image
    ...
    (Run 'diff -u tests/xfs/432.out

/root/xfstests/xfstests-dev/results//xfs/432.out.bad' to see the entire diff)
Ran: xfs/432
Failures: xfs/432
Failed 1 of 1 tests

Quoted 5 lines

Cheers,

Dave.

--
To unsubscribe from this list: send the line "unsubscribe fstests" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

author
#4 lore raw
Quoted 90 lines · Darrick J. Wong
Quoted 80 lines · Dave Jiang
Quoted 62 lines · Dave Chinner
Quoted 19 lines · Dave Jiang

Eryu,
I've noticed that these tests fails under what I think is a normal config (BRD of 48G). We have an expectation that for simple configs all tests in the 'auto' group should pass, and these ones don't. Are these false positive failures? If so, what do we need to do to remove these false positives? a) fix the tests to handle these cases b) remove the tests from the 'auto' group? Something else? Attached file with test outputs. I think some if not all of these failures have lasted many kernel versions.

# xfs
generic/009
generic/012
generic/016
generic/021
generic/022
generic/058
generic/060
generic/061
Quoted 8 lines
generic/063
generic/092
generic/255
xfs/167
xfs/191-input-validation
xfs/242
xfs/252
xfs/432

Except for xfs/191, these all look to be extent mapping failures. i.e. there's one bug or config issue that is causing them all.

Quoted 29 lines
# ext4
generic/388
Signature · 23 lines
Dave Jiang
Software Engineer, SSG/OTC
Intel Corp.
[email protected]

# XFS failures

# ./check generic/009
FSTYP         -- xfs (non-debug)
PLATFORM      -- Linux/x86_64 skx-ntbusd 4.15.0+
MKFS_OPTIONS  -- -f -bsize=4096 /dev/ram0p2
MOUNT_OPTIONS -- -o context=system_u:object_r:root_t:s0 /dev/ram0p2 /mnt/xfstests_scratch

generic/009	 - output mismatch (see /root/xfstests/xfstests-dev/results//generic/009.out.bad)
    --- tests/generic/009.out	2016-09-09 09:30:36.006800609 -0700
    +++ /root/xfstests/xfstests-dev/results//generic/009.out.bad	2018-02-05 13:24:23.702640408 -0700
    @@ -1,79 +1,75 @@
     QA output created by 009
     	1. into a hole
    -0: [0..7]: hole
    -1: [8..23]: unwritten
    -2: [24..39]: hole
    +0: [0..4095]: unwritten
Quoted 3 lines

You're getting a 2MB extent allocated here. I'm guessing your testdev is configured with a 2MB extent size hint or something similar left over from trying to test DAX w/ 2MB huge pages?

Yes. Looks like the config script was setting 2M extent. After removing and retesting xfs/191 and xfs/432 fails.

# ./check xfs/432
FSTYP         -- xfs (non-debug)
PLATFORM      -- Linux/x86_64 skx-ntbusd 4.15.0+
MKFS_OPTIONS  -- -f -bsize=4096 /dev/ram0p2
MOUNT_OPTIONS -- -o context=system_u:object_r:root_t:s0 /dev/ram0p2
/mnt/xfstests_scratch

xfs/432 - output mismatch (see
/root/xfstests/xfstests-dev/results//xfs/432.out.bad)

Quoted 8 lines
    --- tests/xfs/432.out	2017-10-19 10:57:22.562819579 -0700
    +++ /root/xfstests/xfstests-dev/results//xfs/432.out.bad	2018-02-05

15:57:29.673255360 -0700
@@ -3,4 +3,5 @@
Create huge dir
Check for > 1000 block extent?
Try to metadump
+xfs_metadump: suspicious count 1088 in bmap extent 1 in dir3 ino 35

Quoted 2 lines

What version of xfsprogs are you running? I fixed that a while ago... I think.

xfsprogs-4.12.0-4.fc27.x86_64

Quoted 18 lines

--D

Quoted 15 lines
     Check restored metadump image
    ...
    (Run 'diff -u tests/xfs/432.out

/root/xfstests/xfstests-dev/results//xfs/432.out.bad' to see the entire diff)
Ran: xfs/432
Failures: xfs/432
Failed 1 of 1 tests

Quoted 5 lines

Cheers,

Dave.

Reviewed-by Carlos Maiolino
#5 lore raw
Quoted 93 lines · Dave Jiang
Quoted 90 lines · Darrick J. Wong
Quoted 88 lines · Dave Jiang
Quoted 62 lines · Dave Chinner
Quoted 19 lines · Dave Jiang

Eryu,
I've noticed that these tests fails under what I think is a normal config (BRD of 48G). We have an expectation that for simple configs all tests in the 'auto' group should pass, and these ones don't. Are these false positive failures? If so, what do we need to do to remove these false positives? a) fix the tests to handle these cases b) remove the tests from the 'auto' group? Something else? Attached file with test outputs. I think some if not all of these failures have lasted many kernel versions.

# xfs
generic/009
generic/012
generic/016
generic/021
generic/022
generic/058
generic/060
generic/061
Quoted 8 lines
generic/063
generic/092
generic/255
xfs/167
xfs/191-input-validation
xfs/242
xfs/252
xfs/432

Except for xfs/191, these all look to be extent mapping failures. i.e. there's one bug or config issue that is causing them all.

Quoted 29 lines
# ext4
generic/388
Signature · 23 lines
Dave Jiang
Software Engineer, SSG/OTC
Intel Corp.
[email protected]

# XFS failures

# ./check generic/009
FSTYP         -- xfs (non-debug)
PLATFORM      -- Linux/x86_64 skx-ntbusd 4.15.0+
MKFS_OPTIONS  -- -f -bsize=4096 /dev/ram0p2
MOUNT_OPTIONS -- -o context=system_u:object_r:root_t:s0 /dev/ram0p2 /mnt/xfstests_scratch

generic/009	 - output mismatch (see /root/xfstests/xfstests-dev/results//generic/009.out.bad)
    --- tests/generic/009.out	2016-09-09 09:30:36.006800609 -0700
    +++ /root/xfstests/xfstests-dev/results//generic/009.out.bad	2018-02-05 13:24:23.702640408 -0700
    @@ -1,79 +1,75 @@
     QA output created by 009
     	1. into a hole
    -0: [0..7]: hole
    -1: [8..23]: unwritten
    -2: [24..39]: hole
    +0: [0..4095]: unwritten
Quoted 3 lines

You're getting a 2MB extent allocated here. I'm guessing your testdev is configured with a 2MB extent size hint or something similar left over from trying to test DAX w/ 2MB huge pages?

Yes. Looks like the config script was setting 2M extent. After removing and retesting xfs/191 and xfs/432 fails.

# ./check xfs/432
FSTYP         -- xfs (non-debug)
PLATFORM      -- Linux/x86_64 skx-ntbusd 4.15.0+
MKFS_OPTIONS  -- -f -bsize=4096 /dev/ram0p2
MOUNT_OPTIONS -- -o context=system_u:object_r:root_t:s0 /dev/ram0p2
/mnt/xfstests_scratch

xfs/432 - output mismatch (see
/root/xfstests/xfstests-dev/results//xfs/432.out.bad)

    --- tests/xfs/432.out	2017-10-19 10:57:22.562819579 -0700
    +++ /root/xfstests/xfstests-dev/results//xfs/432.out.bad	2018-02-05

15:57:29.673255360 -0700
@@ -3,4 +3,5 @@
Create huge dir
Check for > 1000 block extent?
Try to metadump
+xfs_metadump: suspicious count 1088 in bmap extent 1 in dir3 ino 35

Quoted 4 lines
Quoted 2 lines

What version of xfsprogs are you running? I fixed that a while ago... I think.

xfsprogs-4.12.0-4.fc27.x86_64

This hit 4.14:

commit 921c30674e9bc719e7c2747deb6deb04be2adb4b

Author: Darrick J. Wong <[email protected]>
Date:   Thu Nov 9 11:35:22 2017 -0600

db: increase metadump's default overly long extent discard threshold

Back in 88b8e1d6d7 ("Make xfs_metadump more robust against bad data"), metadump grew the ability to ignore a directory extent if it was longer than 20 blocks. Presumably this was to protect metadump from dumping absurdly long extents resulting from bmbt corruption, but it's certainly possible to create a directory with an extent longer than 20 blocks. Hilariously, the discards happen with no warning unless the caller explicitly set -w.

This was raised to 1000 blocks in 7431d134fe8 ("Increase default maximum extent size for xfs_metadump when copying..."), but it's still possible to create a directory with an extent longer than 1000 blocks.

Increase the threshold to MAXEXTLEN blocks because it's totally valid for the filesystem to create extents up to that length.

Usually best to run latest xfsprogs esp. for xfstests testing; while xfsprogs is usually very stable, corner cases do get added to xfstests as they are fixed and so you're likely to run into failures on new tests like this.

(Or to put a finer point on it, the test you failed on is something which is specifically testing the fix above).

-Eric

#6 lore raw
In reply to Dave Jiang
Quoted 4 lines · Dave Jiang

Eryu,
I've noticed that these tests fails under what I think is a normal config (BRD of 48G). We have an expectation that for simple configs all tests in the 'auto' group should pass, and these ones don't. Are these

No, 'auto' group doesn't mean the test should pass, we do add tests that're known to fail to auto group, but with the expectation that the failures will be fixed in the near future.

Quoted 2 lines

false positive failures? If so, what do we need to do to remove these false positives? a) fix the tests to handle these cases b) remove the

And some tests do fail on unusual configs/setups, e.g. 2M extent setting in your case, and some tests may not work well with 4k-sector disks. We'd want to fix the tests but sometimes it's hard to make test work with all configs, perhahs it's just not worth it if the config is strange enough and no one cares about it.. But I do like to see the easy ones get fixed.

Quoted 4 lines

tests from the 'auto' group? Something else? Attached file with test outputs. I think some if not all of these failures have lasted many kernel versions.

Quoted 14 lines
# xfs
generic/009
generic/012
generic/016
generic/021
generic/022
generic/058
generic/060
generic/061
generic/063
generic/092
generic/255
xfs/167
xfs/191-input-validation

This tests mkfs.xfs behavior and AFAIK it fails after Dave's mkfs refactor patchset, the test itself requires some fixes.

Quoted 6 lines

xfs/242
xfs/252
xfs/432

# ext4
generic/388

This is a known issue on ext4, there's a discussion thread on ext4 list.
https://marc.info/?l=linux-ext4&m=151629719004002&w=2

Thanks,
Eryu

#7 lore raw
Quoted 5 lines · Eryu Guan
Quoted 1 line · Dave Jiang

xfs/191-input-validation

This tests mkfs.xfs behavior and AFAIK it fails after Dave's mkfs refactor patchset, the test itself requires some fixes.

It failed before it, too. I'd just mark this test as broken and remove it for the auto group, because it isn't doing what I originally intended it to be used for.

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]
#8 lore raw
Quoted 8 lines · Dave Chinner
Quoted 5 lines · Eryu Guan
Quoted 1 line · Dave Jiang

xfs/191-input-validation

This tests mkfs.xfs behavior and AFAIK it fails after Dave's mkfs refactor patchset, the test itself requires some fixes.

It failed before it, too. I'd just mark this test as broken and

It failed only with 4k-sector disk, but my memory may not be accurate..

Quoted 2 lines

remove it for the auto group, because it isn't doing what I originally intended it to be used for.

Thanks for the info! Would you mind sending a patch for this?

Thanks,
Eryu

#9 lore raw
In reply to Eryu Guan
Quoted 46 lines · Eryu Guan
Quoted 4 lines · Dave Jiang

Eryu,
I've noticed that these tests fails under what I think is a normal config (BRD of 48G). We have an expectation that for simple configs all tests in the 'auto' group should pass, and these ones don't. Are these

No, 'auto' group doesn't mean the test should pass, we do add tests that're known to fail to auto group, but with the expectation that the failures will be fixed in the near future.

Quoted 2 lines

false positive failures? If so, what do we need to do to remove these false positives? a) fix the tests to handle these cases b) remove the

And some tests do fail on unusual configs/setups, e.g. 2M extent setting in your case, and some tests may not work well with 4k-sector disks. We'd want to fix the tests but sometimes it's hard to make test work with all configs, perhahs it's just not worth it if the config is strange enough and no one cares about it.. But I do like to see the easy ones get fixed.

Quoted 4 lines

tests from the 'auto' group? Something else? Attached file with test outputs. I think some if not all of these failures have lasted many kernel versions.

Quoted 14 lines
# xfs
generic/009
generic/012
generic/016
generic/021
generic/022
generic/058
generic/060
generic/061
generic/063
generic/092
generic/255
xfs/167
xfs/191-input-validation

This tests mkfs.xfs behavior and AFAIK it fails after Dave's mkfs refactor patchset, the test itself requires some fixes.

Quoted 4 lines

xfs/242
xfs/252
xfs/432

Quoted 6 lines
Quoted 2 lines
# ext4
generic/388

This is a known issue on ext4, there's a discussion thread on ext4 list.
https://marc.info/?l=linux-ext4&m=151629719004002&w=2

Eryu,

This short thread has shown how much the information about failing tests is not documented. And that the information is stored only in our collective brain hive.

For example, how many people out there are able to dig through the list the find out
the following information about overlay test failures:
016 - long standing known issue - time for a fix unknown 041,42,43 - known issue - resolved by RFC "xino" patches -
time for merge unknown
049 - mainline issue - resolved by today's overlayfs-next merge 054,55 - new failing tests with today's mainline - fix patch posted

I know even you did not have the correct information about 017 and thought it should have been failing...

How about maintaining an "expunge" format file with all failing test in the repository with comment like the above?
We could have 2 files, one for long standing issues like 016,041,042,043 and one for temporary failures, like 054,055

We can debate whether long standing issues should be in auto or not, but it doesn't hurt to mark them with some group (e.g. "known") that will point users at the expunge.known file for a reason and for easy way to exclude those tests.

If you agree to this concept, please start a file and I will add the overlay failures info to that file.

Thanks,
Amir.

#10 lore raw
Quoted 69 lines · Amir Goldstein
Quoted 46 lines · Eryu Guan
Quoted 4 lines · Dave Jiang

Eryu,
I've noticed that these tests fails under what I think is a normal config (BRD of 48G). We have an expectation that for simple configs all tests in the 'auto' group should pass, and these ones don't. Are these

No, 'auto' group doesn't mean the test should pass, we do add tests that're known to fail to auto group, but with the expectation that the failures will be fixed in the near future.

Quoted 2 lines

false positive failures? If so, what do we need to do to remove these false positives? a) fix the tests to handle these cases b) remove the

And some tests do fail on unusual configs/setups, e.g. 2M extent setting in your case, and some tests may not work well with 4k-sector disks. We'd want to fix the tests but sometimes it's hard to make test work with all configs, perhahs it's just not worth it if the config is strange enough and no one cares about it.. But I do like to see the easy ones get fixed.

Quoted 4 lines

tests from the 'auto' group? Something else? Attached file with test outputs. I think some if not all of these failures have lasted many kernel versions.

Quoted 14 lines
# xfs
generic/009
generic/012
generic/016
generic/021
generic/022
generic/058
generic/060
generic/061
generic/063
generic/092
generic/255
xfs/167
xfs/191-input-validation

This tests mkfs.xfs behavior and AFAIK it fails after Dave's mkfs refactor patchset, the test itself requires some fixes.

Quoted 4 lines

xfs/242
xfs/252
xfs/432

Quoted 6 lines
Quoted 2 lines
# ext4
generic/388

This is a known issue on ext4, there's a discussion thread on ext4 list.
https://marc.info/?l=linux-ext4&m=151629719004002&w=2

Eryu,

This short thread has shown how much the information about failing tests is not documented. And that the information is stored only in our collective brain hive.

For example, how many people out there are able to dig through the list the find out
the following information about overlay test failures:
016 - long standing known issue - time for a fix unknown 041,42,43 - known issue - resolved by RFC "xino" patches -
time for merge unknown
049 - mainline issue - resolved by today's overlayfs-next merge 054,55 - new failing tests with today's mainline - fix patch posted

Quoted 5 lines

I know even you did not have the correct information about 017 and thought it should have been failing...

How about maintaining an "expunge" format file with all failing test in the repository with comment like the above?

I'm fine with providing a mechanism to let fstests know which tests are known failures, like what Andreas just proposed just 3 days ago

check: Add -F option to specify failing tests
https://www.spinics.net/lists/fstests/msg08901.html

Quoted 7 lines

We could have 2 files, one for long standing issues like 016,041,042,043 and one for temporary failures, like 054,055

We can debate whether long standing issues should be in auto or not, but it doesn't hurt to mark them with some group (e.g. "known") that will point users at the expunge.known file for a reason and for easy way to exclude those tests.

But I agreed with Dave[1] here that we should not maintain known failures in fstests itself, because the list can vary based on testers' configurations and hardware/software environments. I think the list should be maintained by testers according to their test configs and environments.

I know Dave and Josef discussed the idea of making perf results public somewhere online in your "[LSF/MM TOPIC] Filesystem performance regression tests" thread[2]. Perhaps each filesystem maintainer could public their "official" results there too when the infrastructure is available?

Thanks,
Eryu

[1] https://patchwork.kernel.org/patch/8077361/
[2] https://www.spinics.net/lists/fstests/msg08898.html

#11 lore raw
Quoted 2 lines · Eryu Guan
Quoted 1 line · Amir Goldstein

On Tue, Feb 6, 2018 at 5:54 AM, Eryu Guan <[email protected]> wrote:

[...]

Quoted 42 lines
Quoted 16 lines

Eryu,

This short thread has shown how much the information about failing tests is not documented. And that the information is stored only in our collective brain hive.

For example, how many people out there are able to dig through the list the find out
the following information about overlay test failures:
016 - long standing known issue - time for a fix unknown 041,42,43 - known issue - resolved by RFC "xino" patches -
time for merge unknown
049 - mainline issue - resolved by today's overlayfs-next merge 054,55 - new failing tests with today's mainline - fix patch posted

Quoted 5 lines

I know even you did not have the correct information about 017 and thought it should have been failing...

How about maintaining an "expunge" format file with all failing test in the repository with comment like the above?

I'm fine with providing a mechanism to let fstests know which tests are known failures, like what Andreas just proposed just 3 days ago

check: Add -F option to specify failing tests
https://www.spinics.net/lists/fstests/msg08901.html

Quoted 7 lines

We could have 2 files, one for long standing issues like 016,041,042,043 and one for temporary failures, like 054,055

We can debate whether long standing issues should be in auto or not, but it doesn't hurt to mark them with some group (e.g. "known") that will point users at the expunge.known file for a reason and for easy way to exclude those tests.

But I agreed with Dave[1] here that we should not maintain known failures in fstests itself, because the list can vary based on testers' configurations and hardware/software environments. I think the list should be maintained by testers according to their test configs and environments.

Quoted 6 lines

I know Dave and Josef discussed the idea of making perf results public somewhere online in your "[LSF/MM TOPIC] Filesystem performance regression tests" thread[2]. Perhaps each filesystem maintainer could public their "official" results there too when the infrastructure is available?

Hmm, yeh, maintaining failure information per configuration is a challenge. Creating a subset of tests that could be run on stable kernels/stable xfsprogs is another worthy yet controversial goal. I read that Linaro have started an effort to run LTP on stable kernels.

But how about long standing issues. Maybe overlay/016 is the only example of a long standing issue?
When I posted the test, over a year ago the commit message read "This issue is about to be fixed with a patch set by Miklos Szeredi" Said patch did not sit well with Al, but the issue is still at the top of the agenda
and has just been discussed last week [1].

How about a statement in the test _expect_failure that will automatically add the test to the -F list?
Alternatively, or in addition to the statement we can add the list to "failing" group, so that the -F code can easily count the expected failures on run start.

Thanks,
Amir.

[1] https://marc.info/?l=linux-unionfs&m=151707113630958&w=2

#12 lore raw
Quoted 53 lines · Amir Goldstein
Quoted 2 lines · Eryu Guan
Quoted 1 line · Amir Goldstein

On Tue, Feb 6, 2018 at 5:54 AM, Eryu Guan <[email protected]> wrote:

[...]

Quoted 42 lines
Quoted 16 lines

Eryu,

This short thread has shown how much the information about failing tests is not documented. And that the information is stored only in our collective brain hive.

For example, how many people out there are able to dig through the list the find out
the following information about overlay test failures:
016 - long standing known issue - time for a fix unknown 041,42,43 - known issue - resolved by RFC "xino" patches -
time for merge unknown
049 - mainline issue - resolved by today's overlayfs-next merge 054,55 - new failing tests with today's mainline - fix patch posted

Quoted 5 lines

I know even you did not have the correct information about 017 and thought it should have been failing...

How about maintaining an "expunge" format file with all failing test in the repository with comment like the above?

I'm fine with providing a mechanism to let fstests know which tests are known failures, like what Andreas just proposed just 3 days ago

check: Add -F option to specify failing tests
https://www.spinics.net/lists/fstests/msg08901.html

Quoted 7 lines

We could have 2 files, one for long standing issues like 016,041,042,043 and one for temporary failures, like 054,055

We can debate whether long standing issues should be in auto or not, but it doesn't hurt to mark them with some group (e.g. "known") that will point users at the expunge.known file for a reason and for easy way to exclude those tests.

But I agreed with Dave[1] here that we should not maintain known failures in fstests itself, because the list can vary based on testers' configurations and hardware/software environments. I think the list should be maintained by testers according to their test configs and environments.

Quoted 6 lines

I know Dave and Josef discussed the idea of making perf results public somewhere online in your "[LSF/MM TOPIC] Filesystem performance regression tests" thread[2]. Perhaps each filesystem maintainer could public their "official" results there too when the infrastructure is available?

Quoted 7 lines

Hmm, yeh, maintaining failure information per configuration is a challenge. Creating a subset of tests that could be run on stable kernels/stable xfsprogs is another worthy yet controversial goal. I read that Linaro have started an effort to run LTP on stable kernels.

But how about long standing issues. Maybe overlay/016 is the only example of a long standing issue?

Just removing it from auto and quick group should be sufficient (with good commit log, so we have some clues when looking back at the git history). Another example in similar case is generic/042, it's known to fail and we didn't know when there'll be a fix, so it's been removed from auto group.

Quoted 11 lines

When I posted the test, over a year ago the commit message read "This issue is about to be fixed with a patch set by Miklos Szeredi" Said patch did not sit well with Al, but the issue is still at the top of the agenda
and has just been discussed last week [1].

How about a statement in the test _expect_failure that will automatically add the test to the -F list?
Alternatively, or in addition to the statement we can add the list to "failing" group, so that the -F code can easily count the expected failures on run start.

I don't think such annotations for tests should be part of fstests either..

Thanks,
Eryu