Skip to contentSkip to conversation

Split some fanotify tests

LTP Petr Vorel 4 participants 4 messages Jan 24, 2025

lore
Share & tools
lore

Petr Vorel

author
#0 lore raw

Hi Amir,

FYI we were thinking with Martin about splitting some of the fanotify tests. Why?

Some of the fanotify tests are hard to review (e.g. fanotify{10,13,14}.c,), because the have a lot of results.

Also other problem is that some enterprise kernels does not backport minor fixes and the usual approach is to whitelist the issue based on exit code. But this cannot be done on the test which has many results. We could workaround it with tst_kvercmp2(), but that's kind of a last result.

Kind regards,
Petr

Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
#1 lore raw
Quoted 8 lines · Petr Vorel

Hi Amir,

FYI we were thinking with Martin about splitting some of the fanotify tests. Why?

Some of the fanotify tests are hard to review (e.g. fanotify{10,13,14}.c,), because the have a lot of results.

I assume you are referring to test cases that are declared as regression tests for a fix commit.
Do you intend to split every such test case into its own test?

I agree that the FAN_DELETE_SELF test case is a good candidate
for splitting, if not for anything else, because the ugliness of requiring this test case to be the last test case.

Quoted 5 lines

Also other problem is that some enterprise kernels does not backport minor fixes and the usual approach is to whitelist the issue based on exit code. But this cannot be done on the test which has many results. We could workaround it with tst_kvercmp2(), but that's kind of a last result.

As a general rule, not testing a know bug because it is not backported sounds bad to me, although I do agree that there could be exceptions, like:

        if (tc->ignored_flags & FAN_EVENT_ON_CHILD && tst_kvercmp(5,
9, 0) < 0) {
                tst_res(TCONF, "ignored mask in combination with flag
FAN_EVENT_ON_CHILD"
                                " has undefined behavior on kernel < 5.9");

If the test runner whitelists a test it's up to them.
It would be nice if LTP could add the ability to whitelist a test case/variant or even a specific filesystem for a specific all_filesystems test, but it's up to
you guys to decide if it is worth it.

The reason I tend to add test cases is to save code duplication between tests, because there is a lot of boilerplate code for event verification, which often has bugs and needs fixing in multiple tests.
There is another solution for code duplication - to write generic fanotify event verification helpers.

It's up to you to decide if there is room for some helpers for fanotify event verification or if you want to fork the tests with code duplication. WDYT?

Thanks,
Amir.

Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
#2 lore raw

Hi!

Quoted 9 lines

The reason I tend to add test cases is to save code duplication between tests, because there is a lot of boilerplate code for event verification, which often has bugs and needs fixing in multiple tests.
There is another solution for code duplication - to write generic fanotify event verification helpers.

It's up to you to decide if there is room for some helpers for fanotify event verification or if you want to fork the tests with code duplication. WDYT?

Sharing code is almost never a mistake. We usually put static inline helpers into a common.h header and include that in tests.

Signature · 5 lines
Cyril Hrubis
[email protected]

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp
#3 lore raw
In reply to Amir Goldstein

Hi!

Quoted 10 lines · Amir Goldstein
Quoted 8 lines · Petr Vorel

Hi Amir,

FYI we were thinking with Martin about splitting some of the fanotify tests. Why?

Some of the fanotify tests are hard to review (e.g. fanotify{10,13,14}.c,), because the have a lot of results.

Quoted 7 lines

I assume you are referring to test cases that are declared as regression tests for a fix commit.
Do you intend to split every such test case into its own test?

I agree that the FAN_DELETE_SELF test case is a good candidate
for splitting, if not for anything else, because the ugliness of requiring this test case to be the last test case.

Yes, that would be a good candidate for its own test. But in general, one test program should have at most 5-10 subtests so that when you need to whitelist it as a whole, you don't lose too much test coverage due to one failing subtest. There are of course exceptions, closely related subtests (e.g. a single set of init and mark flags tested on different types of filesystem objects) can be in the same test program even if there's way more than 10 of them.

Quoted 5 lines

If the test runner whitelists a test it's up to them.
It would be nice if LTP could add the ability to whitelist a test case/variant or even a specific filesystem for a specific all_filesystems test, but it's up to
you guys to decide if it is worth it.

Exiting a subtest with TCONF is the only way of whitelisting issues inside LTP at the moment. The main concern in this thread is the difficulty and trade-offs of whitelisting at the test runner level.

Quoted 9 lines

The reason I tend to add test cases is to save code duplication between tests, because there is a lot of boilerplate code for event verification, which often has bugs and needs fixing in multiple tests.
There is another solution for code duplication - to write generic fanotify event verification helpers.

It's up to you to decide if there is room for some helpers for fanotify event verification or if you want to fork the tests with code duplication. WDYT?

Shared file with fanotify verification helper code would be great. Thanks in advance.

Signature · 10 lines
Martin Doucha   [email protected]
SW Quality Engineer
SUSE LINUX, s.r.o.
CORSO IIa
Krizikova 148/34
186 00 Prague 8
Czech Republic

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp