update test_dummy_encryption testing in ext4/053

The kernel patch "ext4: only allow test_dummy_encryption when supported" will tighten the requirements on when the test_dummy_encryption mount option will be accepted. Update ext4/053 accordingly.

Move the test cases to later in the file to group them with the other test cases that use do_mkfs to add custom mkfs options instead of using the "default" filesystem that the test creates at the beginning.

Quoted 4 lines · Eric Biggers

The kernel patch "ext4: only allow test_dummy_encryption when supported" will tighten the requirements on when the test_dummy_encryption mount option will be accepted. Update ext4/053 accordingly.

One of the problems with ext4/053 is that it is very implementation dependent. It was useful when we were making the change to the new mount API, but the problem is any future changes to the mount option handling is going to break the patch.

So for example, the kernel patch which Eric has proposed, "ext4: only allow test_dummy_encryption when supported", breaks ext4/053, which I noted in the review the patch. But then this patch will break kernels as they currently stand without this patch, and for kernels that haven't moved to the new mount API, fixing it is going to be a mess.

Perhaps ext4/053 is still useful in that it will flag changes that might unintentionally make user-visible changes mount options handling in ext4, but for cases like this one, where we are changing a mount option which is really intended for kernel developers, perhaps the right approach here is to just remove the parts of ext4/053 that are testing the behaviour of test_dummy_encryption in such a super-nit-picky way?

What do folks think?

Quoted 4 lines

Move the test cases to later in the file to group them with the other test cases that use do_mkfs to add custom mkfs options instead of using the "default" filesystem that the test creates at the beginning.

Note: this patch doesn't apply because ext4/053 currently has this line:

not_mnt test_dummy_encryption=v3

and the patch is trying to remove this line in the first patch chunk:

mnt test_dummy_encryption=v3 ^test_dummy_encryption=v3

I checked the upstream version of ext4/053 just in case I had some local modification of ext4/053 in my tree via "git diff -r origin/master tests/ext4/053" but that returned no deltas.

Eric, do you have a local modification to this test in your tree already, perhaps?

We could gate them on the kernel version, similar to the whole ext4/053 which already only runs on kernel version 5.12. (Kernel versions checks suck, but maybe it's the right choice for this very-nit-picky test.) Alternatively, I could just backport "ext4: only allow test_dummy_encryption when supported" to 5.15, which would be the only relevant LTS kernel version.

Quoted 6 lines · Eric Biggers

We could gate them on the kernel version, similar to the whole ext4/053 which already only runs on kernel version 5.12. (Kernel versions checks suck, but maybe it's the right choice for this very-nit-picky test.) Alternatively, I could just backport "ext4: only allow test_dummy_encryption when supported" to 5.15, which would be the only relevant LTS kernel version.

If we don't need the "only allow test_dummy_encryption when supported" in any Android, Distro, or LTS kernel --- which seems to be a reasonable assumption, that seems to be OK. Lukas, do you agree?

In the long term I suspect there will be times when we want to backport mount option handling changes to older kernels, and we're going to be hit this issue again, but as the saying goes, "sufficient unto the day is the evil thereof".

- Ted

This series updates the testing of the test_dummy_encryption mount option in ext4/053.

The first patch will be needed for the test to pass if the kernel patch "ext4: only allow test_dummy_encryption when supported"
(https://lore.kernel.org/r/[email protected])
is applied.

The second patch starts testing a case that previously wasn't tested. It reproduces a bug that was introduced in the v5.17 kernel and will be fixed by the kernel patch
"ext4: fix up test_dummy_encryption handling for new mount API"
(https://lore.kernel.org/r/[email protected]).

This applies on top of my recent patch
"ext4/053: fix the rejected mount option testing"
(https://lore.kernel.org/r/[email protected]).

Eric Biggers (2):
ext4/053: update the test_dummy_encryption tests ext4/053: test changing test_dummy_encryption on remount

 tests/ext4/053 | 38 ++++++++++++++++++++++++--------------
 1 file changed, 24 insertions(+), 14 deletions(-)

Signature · 1 line

2.36.0

Quoted 14 lines · Eric Biggers

This series updates the testing of the test_dummy_encryption mount option in ext4/053.

The first patch will be needed for the test to pass if the kernel patch "ext4: only allow test_dummy_encryption when supported"
(https://lore.kernel.org/r/[email protected])
is applied.

The second patch starts testing a case that previously wasn't tested. It reproduces a bug that was introduced in the v5.17 kernel and will be fixed by the kernel patch
"ext4: fix up test_dummy_encryption handling for new mount API"
(https://lore.kernel.org/r/[email protected]).

Quoted 3 lines

This applies on top of my recent patch
"ext4/053: fix the rejected mount option testing"
(https://lore.kernel.org/r/[email protected]).

Hi Eric,

Your "ext4/053: fix the rejected mount option testing" has been merged. As the two kernel patches haven't been merged by upstream linux, I'd like to merge this patchset after the kernel patches be merged. (feel free to ping me, if I forget this:)

And I saw some discussion under this patchset, and no any RVB, so I'm wondering if you are still working/changing on it?

Quoted 19 lines · Zorro Lang

Quoted 17 lines · Eric Biggers

This series updates the testing of the test_dummy_encryption mount option in ext4/053.

The first patch will be needed for the test to pass if the kernel patch "ext4: only allow test_dummy_encryption when supported"
(https://lore.kernel.org/r/[email protected])
is applied.

The second patch starts testing a case that previously wasn't tested. It reproduces a bug that was introduced in the v5.17 kernel and will be fixed by the kernel patch
"ext4: fix up test_dummy_encryption handling for new mount API"
(https://lore.kernel.org/r/[email protected]).

This applies on top of my recent patch
"ext4/053: fix the rejected mount option testing"
(https://lore.kernel.org/r/[email protected]).

Quoted 6 lines

Hi Eric,

Your "ext4/053: fix the rejected mount option testing" has been merged. As the two kernel patches haven't been merged by upstream linux, I'd like to merge this patchset after the kernel patches be merged. (feel free to ping me, if I forget this:)

Yes, I'm waiting for them to be applied.

Quoted 5 lines

And I saw some discussion under this patchset, and no any RVB, so I'm wondering if you are still working/changing on it?

I might add a check for kernel version >= 5.19 in patch 1. Otherwise I'm not planning any more changes.

- Eric

Quoted 7 lines

Quoted 4 lines

And I saw some discussion under this patchset, and no any RVB, so I'm wondering if you are still working/changing on it?

I might add a check for kernel version >= 5.19 in patch 1. Otherwise I'm not planning any more changes.

Actually I don't think the kernel version check (in fstests) is a good method. Better to check a behavior/feature directly likes those "_require_*" functions.

Why ext4/053 need >=5.12 or even >=5.19, what features restrict that? If some features testing might break the garden image (.out file), we can refer to _link_out_file(). Or even split this case to several small cases, make ext4/053 only test old stable behaviors. Then use other cases to test new features, and use _require_$feature_you_test for them (avoid the kernel version restriction).

Zorro, can you fix your email configuration? Your emails have a Mail-Followup-To header that excludes you, so replying doesn't work correctly; I had to manually fix the recipients list. If you're using mutt, you need to add 'set followup_to = no' to your muttrc.

Quoted 11 lines

Quoted 7 lines

Quoted 4 lines

And I saw some discussion under this patchset, and no any RVB, so I'm wondering if you are still working/changing on it?

I might add a check for kernel version >= 5.19 in patch 1. Otherwise I'm not planning any more changes.

Actually I don't think the kernel version check (in fstests) is a good method. Better to check a behavior/feature directly likes those "_require_*" functions.

Quoted 6 lines

Why ext4/053 need >=5.12 or even >=5.19, what features restrict that? If some features testing might break the garden image (.out file), we can refer to _link_out_file(). Or even split this case to several small cases, make ext4/053 only test old stable behaviors. Then use other cases to test new features, and use _require_$feature_you_test for them (avoid the kernel version restriction).

This has been discussed earlier in this thread as well as on the patch that added ext4/053 originally. ext4/053 has been gated on version >= 5.12 since the beginning. Kernel version checks are certainly bad in general, but ext4/053 is a very nit-picky test intended to detect if anything changed, where a change does not necessarily mean a bug. So maybe the kernel version check makes sense

there. Lukas, any thoughts about the issues you encountered when running ext4/053 on older kernels?

If you don't want a >= 5.19 version check for the test_dummy_encryption test case as well, then I'd rather treat the kernel patch
"ext4: only allow test_dummy_encryption when supported" as a bug fix and backport it to the LTS kernels. The patch is fixing the mount option to work the way it should have worked originally. Either that or we just remove the test_dummy_encryption test case as Ted suggested.

- Eric

Quoted 4 lines · Eric Biggers

Zorro, can you fix your email configuration? Your emails have a Mail-Followup-To header that excludes you, so replying doesn't work correctly; I had to manually fix the recipients list. If you're using mutt, you need to add 'set followup_to = no' to your muttrc.

Oh, I didn't notice that, I use neomutt, it might enable the followup_to by default. OK, I've set followup_to = no, and restart my neomutt. Hope it helps:)

Quoted 20 lines · Zorro Lang

Quoted 11 lines

Quoted 7 lines

Quoted 4 lines

And I saw some discussion under this patchset, and no any RVB, so I'm wondering if you are still working/changing on it?

I might add a check for kernel version >= 5.19 in patch 1. Otherwise I'm not planning any more changes.

Actually I don't think the kernel version check (in fstests) is a good method. Better to check a behavior/feature directly likes those "_require_*" functions.

Quoted 6 lines

Why ext4/053 need >=5.12 or even >=5.19, what features restrict that? If some features testing might break the garden image (.out file), we can refer to _link_out_file(). Or even split this case to several small cases, make ext4/053 only test old stable behaviors. Then use other cases to test new features, and use _require_$feature_you_test for them (avoid the kernel version restriction).

Quoted 5 lines

This has been discussed earlier in this thread as well as on the patch that added ext4/053 originally. ext4/053 has been gated on version >= 5.12 since the beginning. Kernel version checks are certainly bad in general, but ext4/053 is a very nit-picky test intended to detect if anything changed, where a change does not necessarily mean a bug. So maybe the kernel version check makes sense

Even on old RHEL-8 system (with a variant of kernel 3.10), the ext4/053 fails as [1]. Most of mount options test passed, only a few options (inlinecrypt, test_dummy_encryption, prefetch_block_bitmaps, dioread_lock) might not be supported.

I think it's not necessary to mix all old and new ext4 mount options test into one single test cause. If it's too complicated, we can move some functions into common/ext4 (or others you like), split ext4/053 to several cases. Let ext4/053 test stable enough mount option (supported from an old enough kernel). Then let other newer mount options in different single cases.

For example, make those CONFIG_FS_ENCRYPTION* tests into a seperated case, and add something likes require_(fs_encryption?), and src/feature.c can be used too. Then about dioread_lock and prefetch_block_bitmaps things, we can deal with them specially, or split them out from ext4/053. I even don't mind if you test ext2 and ext3/4 in separate case.

That's my personal opinion, I can try to help to do that after merging this patchset, if ext4 forks agree and would like to give me some supports (review and Q&A). Anyway, as it's an ext4 specific testing, I respect the opinion from ext4 list particularly.

[1]

(quoted diff) +21 -0

+SHOULD FAIL remounting ext2 "commit=7" (remount unexpectedly succeeded) FAILED
+mounting ext2 "test_dummy_encryption=v1" (failed mount) FAILED
+mounting ext2 "test_dummy_encryption=v2" (failed mount) FAILED
+mounting ext2 "test_dummy_encryption=v3" (failed mount) FAILED
+mounting ext2 "inlinecrypt" (failed mount) FAILED
+mounting ext2 "prefetch_block_bitmaps" (failed mount) FAILED
+mounting ext2 "no_prefetch_block_bitmaps" (failed mount) FAILED
+mounting ext3 "test_dummy_encryption=v1" (failed mount) FAILED
+mounting ext3 "test_dummy_encryption=v2" (failed mount) FAILED
+mounting ext3 "test_dummy_encryption=v3" (failed mount) FAILED
+mounting ext3 "inlinecrypt" (failed mount) FAILED
+mounting ext3 "prefetch_block_bitmaps" (failed mount) FAILED
+mounting ext3 "no_prefetch_block_bitmaps" (failed mount) FAILED
+mounting ext4 "nodioread_nolock" (failed mount) FAILED
+mounting ext4 "dioread_lock" checking "nodioread_nolock" (not found) FAILED
+mounting ext4 "test_dummy_encryption=v1" (failed mount) FAILED
+mounting ext4 "test_dummy_encryption=v2" (failed mount) FAILED
+mounting ext4 "test_dummy_encryption=v3" (failed mount) FAILED
+mounting ext4 "inlinecrypt" (failed mount) FAILED
+mounting ext4 "prefetch_block_bitmaps" (failed mount) FAILED
+mounting ext4 "no_prefetch_block_bitmaps" (failed mount) FAILED

Quoted 10 lines

there. Lukas, any thoughts about the issues you encountered when running ext4/053 on older kernels?

If you don't want a >= 5.19 version check for the test_dummy_encryption test case as well, then I'd rather treat the kernel patch
"ext4: only allow test_dummy_encryption when supported" as a bug fix and backport it to the LTS kernels. The patch is fixing the mount option to work the way it should have worked originally. Either that or we just remove the test_dummy_encryption test case as Ted suggested.

Oh, I'd not like to push anyone to do more jobs:) And there're many Linux distributions with their downstream kernel, not only LTS kernel project. So I don't mean to make fstests' cases support the oldest existing kernel version, just hope some common cases try not *only* work for the latest one, if they have the chance :)

Thanks,
Zorro

Quoted 3 lines

- Eric

Quoted 35 lines · Zorro Lang

Quoted 4 lines · Eric Biggers

Zorro, can you fix your email configuration? Your emails have a Mail-Followup-To header that excludes you, so replying doesn't work correctly; I had to manually fix the recipients list. If you're using mutt, you need to add 'set followup_to = no' to your muttrc.

Oh, I didn't notice that, I use neomutt, it might enable the followup_to by default. OK, I've set followup_to = no, and restart my neomutt. Hope it helps:)

Quoted 25 lines · Zorro Lang

Quoted 11 lines

Quoted 7 lines

Quoted 4 lines

And I saw some discussion under this patchset, and no any RVB, so I'm wondering if you are still working/changing on it?

I might add a check for kernel version >= 5.19 in patch 1. Otherwise I'm not planning any more changes.

Actually I don't think the kernel version check (in fstests) is a good method. Better to check a behavior/feature directly likes those "_require_*" functions.

Quoted 6 lines

Why ext4/053 need >=5.12 or even >=5.19, what features restrict that? If some features testing might break the garden image (.out file), we can refer to _link_out_file(). Or even split this case to several small cases, make ext4/053 only test old stable behaviors. Then use other cases to test new features, and use _require_$feature_you_test for them (avoid the kernel version restriction).

This has been discussed earlier in this thread as well as on the patch that added ext4/053 originally. ext4/053 has been gated on version >= 5.12 since the beginning. Kernel version checks are certainly bad in general, but ext4/053 is a very nit-picky test intended to detect if anything changed, where a change does not necessarily mean a bug. So maybe the kernel version check makes sense

Quoted 4 lines

Even on old RHEL-8 system (with a variant of kernel 3.10), the ext4/053 fails as [1]. Most of mount options test passed, only a few options (inlinecrypt, test_dummy_encryption, prefetch_block_bitmaps, dioread_lock) might not be supported.

No it does not. On RHEL-8 system the test will not run because of kernel version test. It will be skipped.

Quoted 8 lines

I think it's not necessary to mix all old and new ext4 mount options test into one single test cause. If it's too complicated, we can move some functions into common/ext4 (or others you like), split ext4/053 to several cases. Let ext4/053 test stable enough mount option (supported from an old enough kernel). Then let other newer mount options in different single cases.

Quoted 5 lines

For example, make those CONFIG_FS_ENCRYPTION* tests into a seperated case, and add something likes require_(fs_encryption?), and src/feature.c can be used too. Then about dioread_lock and prefetch_block_bitmaps things, we can deal with them specially, or split them out from ext4/053. I even don't mind if you test ext2 and ext3/4 in separate case.

Sure, but why to split it? It all should be stable enough, it's user facing interface, that's the whole point of the test. I certainly see the benefit of having the test for all ext4 mount option in one test - it's faster and it's easier to see what's there. I would be agains splitting it.

As it is now there is only one kernel_gte() check to avoid testing the entire history. With any new mount option as a separate test we would still need kernel_gte test to avoid failing on kernels that don't have the mount option. At least until kernel gains ability to list supported mount options it's the only test we have.

On the other hand I do see some value in making a new test for a new mount option. But I don't have a strong opinion about that.

As for the original topic of the discussion, as I said in previous reply, maybe the right solution here is to treat the change as a bug fix which is arguably is and let it fail on old behavior.

Thanks!
-Lukas

Quoted 52 lines

That's my personal opinion, I can try to help to do that after merging this patchset, if ext4 forks agree and would like to give me some supports (review and Q&A). Anyway, as it's an ext4 specific testing, I respect the opinion from ext4 list particularly.

[1]

(quoted diff) +21 -0

+SHOULD FAIL remounting ext2 "commit=7" (remount unexpectedly succeeded) FAILED
+mounting ext2 "test_dummy_encryption=v1" (failed mount) FAILED
+mounting ext2 "test_dummy_encryption=v2" (failed mount) FAILED
+mounting ext2 "test_dummy_encryption=v3" (failed mount) FAILED
+mounting ext2 "inlinecrypt" (failed mount) FAILED
+mounting ext2 "prefetch_block_bitmaps" (failed mount) FAILED
+mounting ext2 "no_prefetch_block_bitmaps" (failed mount) FAILED
+mounting ext3 "test_dummy_encryption=v1" (failed mount) FAILED
+mounting ext3 "test_dummy_encryption=v2" (failed mount) FAILED
+mounting ext3 "test_dummy_encryption=v3" (failed mount) FAILED
+mounting ext3 "inlinecrypt" (failed mount) FAILED
+mounting ext3 "prefetch_block_bitmaps" (failed mount) FAILED
+mounting ext3 "no_prefetch_block_bitmaps" (failed mount) FAILED
+mounting ext4 "nodioread_nolock" (failed mount) FAILED
+mounting ext4 "dioread_lock" checking "nodioread_nolock" (not found) FAILED
+mounting ext4 "test_dummy_encryption=v1" (failed mount) FAILED
+mounting ext4 "test_dummy_encryption=v2" (failed mount) FAILED
+mounting ext4 "test_dummy_encryption=v3" (failed mount) FAILED
+mounting ext4 "inlinecrypt" (failed mount) FAILED
+mounting ext4 "prefetch_block_bitmaps" (failed mount) FAILED
+mounting ext4 "no_prefetch_block_bitmaps" (failed mount) FAILED

Quoted 10 lines

there. Lukas, any thoughts about the issues you encountered when running ext4/053 on older kernels?

If you don't want a >= 5.19 version check for the test_dummy_encryption test case as well, then I'd rather treat the kernel patch
"ext4: only allow test_dummy_encryption when supported" as a bug fix and backport it to the LTS kernels. The patch is fixing the mount option to work the way it should have worked originally. Either that or we just remove the test_dummy_encryption test case as Ted suggested.

Oh, I'd not like to push anyone to do more jobs:) And there're many Linux distributions with their downstream kernel, not only LTS kernel project. So I don't mean to make fstests' cases support the oldest existing kernel version, just hope some common cases try not *only* work for the latest one, if they have the chance :)

Thanks,
Zorro

Quoted 3 lines

- Eric

Zorro, can you fix your email configuration? Your emails have a Mail-Followup-To header that excludes you, so replying doesn't work correctly; I had to manually fix the recipients list. If you're using mutt, you need to add 'set followup_to = no' to your muttrc.

Quoted 11 lines · Zorro Lang

Quoted 7 lines

Quoted 4 lines

And I saw some discussion under this patchset, and no any RVB, so I'm wondering if you are still working/changing on it?

I might add a check for kernel version >= 5.19 in patch 1. Otherwise I'm not planning any more changes.

Actually I don't think the kernel version check (in fstests) is a good method. Better to check a behavior/feature directly likes those "_require_*" functions.

Quoted 6 lines

Why ext4/053 need >=5.12 or even >=5.19, what features restrict that? If some features testing might break the garden image (.out file), we can refer to _link_out_file(). Or even split this case to several small cases, make ext4/053 only test old stable behaviors. Then use other cases to test new features, and use _require_$feature_you_test for them (avoid the kernel version restriction).

This has been discussed earlier in this thread as well as on the patch that added ext4/053 originally. ext4/053 has been gated on version >= 5.12 since the beginning. Kernel version checks are certainly bad in general, but ext4/053 is a very nit-picky test intended to detect if anything changed, where a change does not necessarily mean a bug. So maybe the kernel version check makes sense there. Lukas, any thoughts about the issues you encountered when running ext4/053 on older kernels?

If you don't want a >= 5.19 version check for the test_dummy_encryption test case as well, then I'd rather treat the kernel patch
"ext4: only allow test_dummy_encryption when supported" as a bug fix and backport it to the LTS kernels. The patch is fixing the mount option to work the way it should have worked originally. Either that or we just remove the test_dummy_encryption test case as Ted suggested.

- Eric

This series updates the testing of the test_dummy_encryption mount option in ext4/053.

The first patch will be needed for the test to pass if the kernel patch "ext4: only allow test_dummy_encryption when supported"
(https://lore.kernel.org/r/[email protected])
is applied.

The second patch starts testing a case that previously wasn't tested. It reproduces a bug that was introduced in the v5.17 kernel and will be fixed by the kernel patch
"ext4: fix up test_dummy_encryption handling for new mount API"
(https://lore.kernel.org/r/[email protected]).

This applies on top of my recent patch
"ext4/053: fix the rejected mount option testing"
(https://lore.kernel.org/r/[email protected]).

Eric Biggers (2):
ext4/053: update the test_dummy_encryption tests ext4/053: test changing test_dummy_encryption on remount

 tests/ext4/053 | 38 ++++++++++++++++++++++++--------------
 1 file changed, 24 insertions(+), 14 deletions(-)

Signature · 1 line

2.36.0