Skip to contentSkip to conversation

Suggestion to use python-parameterized for QA

DISCUSSION Vyacheslav Yurkov 2 participants 7 messages Nov 15, 2021 – Nov 16, 2021

lore
Share & tools
lore
#0 lore raw

Hi guys,
I have a suggestion to add https://pypi.org/project/parameterized/ as a required package for QA selftests. Together with python unittest module it can be useful to increase code coverage and reduce duplication in case you want to do the same test with different input values.

Anything speaks against it?

Cheers,
Vyacheslav

#1 lore raw

It would be helpful to provide an example of improvements that can be enabled with it.

Alex

Quoted 22 lines · Vyacheslav Yurkov

Hi guys,
I have a suggestion to add https://pypi.org/project/parameterized/ as a required package for QA selftests. Together with python unittest module it can be useful to increase code coverage and reduce duplication in case you want to do the same test with different input values.

Anything speaks against it?

Cheers,
Vyacheslav

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#158309):
https://lists.openembedded.org/g/openembedded-core/message/158309
Mute This Topic: https://lists.openembedded.org/mt/87078063/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ [email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

#2 lore raw

This is still work-in-progress, but I'm working on overlayfs-etc class, which would have different behavior depending on an input variable. You can see the example here

https://github.com/UVV-gh/openembedded-core/commit/ea424efb7780dc3c85e329b36f4a675b56e86877#diff-9980b9625344e11f15451ab7c97fb1f137e94b68fa27df737c576b75e313996aR215

Regardless of the input parameter the test would be the same and should work.

What do you think?

Vyacheslav

Quoted 3 lines · Alexander Kanavin

It would be helpful to provide an example of improvements that can be enabled with it.

Alex

#3 lore raw

I can't understand what is supposed to happen, perhaps because it's incomplete? There's

@parameterized.expand([
("original", True),
("preinit", False),
])

but where is that used?

Alex

Quoted 21 lines · Vyacheslav Yurkov

This is still work-in-progress, but I'm working on overlayfs-etc class, which would have different behavior depending on an input variable. You can see the example here

https://github.com/UVV-gh/openembedded-core/commit/ea424efb7780dc3c85e329b36f4a675b56e86877#diff-9980b9625344e11f15451ab7c97fb1f137e94b68fa27df737c576b75e313996aR215

Regardless of the input parameter the test would be the same and should work.

What do you think?

Vyacheslav

Quoted 4 lines · Alexander Kanavin

It would be helpful to provide an example of improvements that can be enabled with it.

Alex

#4 lore raw

It's substituted further down into
def test_sbin_init(self, _, origInit):

parameterized.expand generates two tests out of it. The string parameter is used as a suffix for the test, the rest is used as input parameters, i.e. the first test would be called with origInit=True, the second one with origInit=False.

The sample output looks like this:

2021-11-15 20:21:15,740 - oe-selftest - INFO - RESULTS - overlayfs.OverlayFSEtcRunTimeTests.test_sbin_init_0_original: PASSED (295.73s)
2021-11-15 20:21:15,740 - oe-selftest - INFO - RESULTS - overlayfs.OverlayFSEtcRunTimeTests.test_sbin_init_1_preinit: PASSED (177.97s)

Vyacheslav

Quoted 14 lines · Alexander Kanavin

I can't understand what is supposed to happen, perhaps because it's incomplete? There's

@parameterized.expand([

("original", True),

("preinit", False),

])

but where is that used?

Alex

#5 lore raw

Can't we achieve the same by abstracting the common part into a function and defining the two tests explicitly?

Alex

Quoted 35 lines · Vyacheslav Yurkov

It's substituted further down into
def test_sbin_init(self, _, origInit):

parameterized.expand generates two tests out of it. The string parameter is used as a suffix for the test, the rest is used as input parameters, i.e. the first test would be called with origInit=True, the second one with origInit=False.

The sample output looks like this:

2021-11-15 20:21:15,740 - oe-selftest - INFO - RESULTS -
overlayfs.OverlayFSEtcRunTimeTests.test_sbin_init_0_original: PASSED
(295.73s)
2021-11-15 20:21:15,740 - oe-selftest - INFO - RESULTS -

overlayfs.OverlayFSEtcRunTimeTests.test_sbin_init_1_preinit: PASSED (177.97s)

Vyacheslav

On 15.11.2021 21:20, Alexander Kanavin wrote:

I can't understand what is supposed to happen, perhaps because it's incomplete? There's

@parameterized.expand([
("original", True),
("preinit", False),
])

but where is that used?

Alex

#6 lore raw

Yes, technically it would be the same. Thanks for the hint.

Vyacheslav

Quoted 25 lines · Alexander Kanavin

Can't we achieve the same by abstracting the common part into a function and defining the two tests explicitly?

Alex

On Mon, 15 Nov 2021 at 21:26, Vyacheslav Yurkov <[email protected]> wrote:

It's substituted further down into
def test_sbin_init(self, _, origInit):

parameterized.expand generates two tests out of it. The string parameter is used as a suffix for the test, the rest is used as input parameters, i.e. the first test would be called with origInit=True, the second one with origInit=False.

The sample output looks like this:

2021-11-15 20:21:15,740 - oe-selftest - INFO - RESULTS - overlayfs.OverlayFSEtcRunTimeTests.test_sbin_init_0_original:
PASSED (295.73s)
2021-11-15 20:21:15,740 - oe-selftest - INFO - RESULTS - overlayfs.OverlayFSEtcRunTimeTests.test_sbin_init_1_preinit:
PASSED (177.97s)

Vyacheslav