Skip to contentSkip to conversation

test/py: Provide custom IDs when parametrizing tests

PATCH New Stephen Warren 2 participants 4 messages Jan 26, 2016 – Jan 29, 2016

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 test/py: Provide custom IDs when parametrizing tests

From: Stephen Warren <[email protected]>

When pytest generates the name for parametrized tests, simple parameter values (ints, strings) get used directly, but more complex values such as dicts are not handled. This yields test names such as:

    dfu[env__usb_dev_port0-env__dfu_config0]
    dfu[env__usb_dev_port0-env__dfu_config1]

Add some code to extract a custom fixture ID from the fixture values, so that we end up with meaningful names such as:

dfu[micro_b-emmc]
dfu[devport2-ram]

If the boardenv file doesn't define custom names, the code falls back to the old algorithm.

---
 test/py/conftest.py       | 8 +++++++-
 test/py/tests/test_dfu.py | 3 +++
 test/py/tests/test_ums.py | 3 +++
 3 files changed, 13 insertions(+), 1 deletion(-)
test/py/conftest.py +7 -1
index a4e54c66ceda..9c9426aebe10 100644
@@ -225,7 +225,13 @@ def pytest_generate_tests(metafunc):
             # ... otherwise, see if there's a key that contains a list of
             # values to use instead.
             vals = subconfig.get(fn + 's', [])
-        metafunc.parametrize(fn, vals)
+        def fixture_id(index, val):
+            try:
+                return val["fixture_id"]
+            except:
+                return fn + str(index)
+        ids = [fixture_id(index, val) for (index, val) in enumerate(vals)]
+        metafunc.parametrize(fn, vals, ids=ids)
 
 @pytest.fixture(scope='function')
 def u_boot_console(request):
test/py/tests/test_dfu.py +3 -0
index c09b90278dca..bb70008af300 100644
@@ -21,6 +21,7 @@ For example:
 
 env__usb_dev_ports = (
     {
+        "fixture_id": "micro_b",
         "tgt_usb_ctlr": "0",
         "host_usb_dev_node": "/dev/usbdev-p2371-2180",
         # This parameter is optional /if/ you only have a single board
@@ -32,10 +33,12 @@ env__usb_dev_ports = (
 env__dfu_configs = (
     # eMMC, partition 1
     {
+        "fixture_id": "emmc",
         "alt_info": "/dfu_test.bin ext4 0 1;/dfu_dummy.bin ext4 0 1",
         "cmd_params": "mmc 0",
     },
 )
+
 b) udev rules to set permissions on devices nodes, so that sudo is not
 required. For example:
 
test/py/tests/test_ums.py +3 -0
index 21d40a972581..8c3ee2b053c4 100644
@@ -29,6 +29,7 @@ env__mount_points = (
 
 env__usb_dev_ports = (
     {
+        "fixture_id": "micro_b",
         "tgt_usb_ctlr": "0",
         "host_ums_dev_node": "/dev/disk/by-path/pci-0000:00:14.0-usb-0:13:1.0-scsi-0:0:0:0",
     },
@@ -37,6 +38,7 @@ env__usb_dev_ports = (
 env__block_devs = (
     # eMMC; always present
     {
+        "fixture_id": "emmc",
         "type": "mmc",
         "id": "0",
         # The following two properties are optional.
@@ -48,6 +50,7 @@ env__block_devs = (
     },
     # SD card; present since I plugged one in
     {
+        "fixture_id": "sd",
         "type": "mmc",
         "id": "1"
     },
Signature · 1 line
2.7.0
#1 lore raw

PATCH 1/1 test/py: Provide custom IDs when parametrizing tests

Hi Stephen,

Quoted 18 lines · Stephen Warren

From: Stephen Warren <[email protected]>

When pytest generates the name for parametrized tests, simple parameter values (ints, strings) get used directly, but more complex values such as dicts are not handled. This yields test names such as:

    dfu[env__usb_dev_port0-env__dfu_config0]
    dfu[env__usb_dev_port0-env__dfu_config1]

Add some code to extract a custom fixture ID from the fixture values, so that we end up with meaningful names such as:

dfu[micro_b-emmc]
dfu[devport2-ram]

If the boardenv file doesn't define custom names, the code falls back to the old algorithm.

Quoted 6 lines
---
 test/py/conftest.py       | 8 +++++++-
 test/py/tests/test_dfu.py | 3 +++
 test/py/tests/test_ums.py | 3 +++
 3 files changed, 13 insertions(+), 1 deletion(-)

test/py/conftest.py def pytest_generate_tests(metafunc): · line 226

Quoted 8 lines
test/py/conftest.py +0 -0
index a4e54c66ceda..9c9426aebe10 100644
@@ -225,7 +225,13 @@ def pytest_generate_tests(metafunc):
             # ... otherwise, see if there's a key that contains a list of
             # values to use instead.

Is this function comment still accurate?

Quoted 66 lines
test/py/conftest.py +7 -1
             vals = subconfig.get(fn + 's', [])
-        metafunc.parametrize(fn, vals)
+        def fixture_id(index, val):
+            try:
+                return val["fixture_id"]
+            except:
+                return fn + str(index)
+        ids = [fixture_id(index, val) for (index, val) in enumerate(vals)]
+        metafunc.parametrize(fn, vals, ids=ids)
 
 @pytest.fixture(scope='function')
 def u_boot_console(request):
test/py/tests/test_dfu.py +3 -0
index c09b90278dca..bb70008af300 100644
@@ -21,6 +21,7 @@ For example:
 
 env__usb_dev_ports = (
     {
+        "fixture_id": "micro_b",
         "tgt_usb_ctlr": "0",
         "host_usb_dev_node": "/dev/usbdev-p2371-2180",
         # This parameter is optional /if/ you only have a single board
@@ -32,10 +33,12 @@ env__usb_dev_ports = (
 env__dfu_configs = (
     # eMMC, partition 1
     {
+        "fixture_id": "emmc",
         "alt_info": "/dfu_test.bin ext4 0 1;/dfu_dummy.bin ext4 0 1",
         "cmd_params": "mmc 0",
     },
 )
+
 b) udev rules to set permissions on devices nodes, so that sudo is not
 required. For example:
 
test/py/tests/test_ums.py +3 -1
index 21d40a972581..8c3ee2b053c4 100644
@@ -29,6 +29,7 @@ env__mount_points = (
 
 env__usb_dev_ports = (
     {
+        "fixture_id": "micro_b",
         "tgt_usb_ctlr": "0",
         "host_ums_dev_node": "/dev/disk/by-path/pci-0000:00:14.0-usb-0:13:1.0-scsi-0:0:0:0",
     },
@@ -37,6 +38,7 @@ env__usb_dev_ports = (
 env__block_devs = (
     # eMMC; always present
     {
+        "fixture_id": "emmc",
         "type": "mmc",
         "id": "0",
         # The following two properties are optional.
@@ -48,6 +50,7 @@ env__block_devs = (
     },
     # SD card; present since I plugged one in
     {
+        "fixture_id": "sd",
         "type": "mmc",
         "id": "1"
     },
--
Quoted 2 lines

2.7.0

Regards,
Simon

#2 lore raw

PATCH 1/1 test/py: Provide custom IDs when parametrizing tests

Quoted 30 lines · Simon Glass

Hi Stephen,

Quoted 18 lines · Stephen Warren

From: Stephen Warren <[email protected]>

When pytest generates the name for parametrized tests, simple parameter values (ints, strings) get used directly, but more complex values such as dicts are not handled. This yields test names such as:

     dfu[env__usb_dev_port0-env__dfu_config0]
     dfu[env__usb_dev_port0-env__dfu_config1]

Add some code to extract a custom fixture ID from the fixture values, so that we end up with meaningful names such as:

dfu[micro_b-emmc]
dfu[devport2-ram]

If the boardenv file doesn't define custom names, the code falls back to the old algorithm.

Quoted 6 lines
---
  test/py/conftest.py       | 8 +++++++-
  test/py/tests/test_dfu.py | 3 +++
  test/py/tests/test_ums.py | 3 +++
  3 files changed, 13 insertions(+), 1 deletion(-)
Quoted 7 lines

test/py/conftest.py def pytest_generate_tests(metafunc): · line 226

Quoted 5 lines
test/py/conftest.py +0 -0
@@ -225,7 +225,13 @@ def pytest_generate_tests(metafunc):
              # ... otherwise, see if there's a key that contains a list of
              # values to use instead.

Is this function comment still accurate?

Yes; that comment is referring to the list of values that the relevant test function iterates over, not the names/IDs of the values or anything like that.

#3 lore raw

PATCH 1/1 test/py: Provide custom IDs when parametrizing tests

Quoted 46 lines · Stephen Warren
Quoted 36 lines · Simon Glass

Hi Stephen,

Quoted 19 lines · Stephen Warren

From: Stephen Warren <[email protected]>

When pytest generates the name for parametrized tests, simple parameter values (ints, strings) get used directly, but more complex values such as dicts are not handled. This yields test names such as:

     dfu[env__usb_dev_port0-env__dfu_config0]
     dfu[env__usb_dev_port0-env__dfu_config1]

Add some code to extract a custom fixture ID from the fixture values, so that we end up with meaningful names such as:

dfu[micro_b-emmc]
dfu[devport2-ram]

If the boardenv file doesn't define custom names, the code falls back to the old algorithm.

Quoted 6 lines
---
  test/py/conftest.py       | 8 +++++++-
  test/py/tests/test_dfu.py | 3 +++
  test/py/tests/test_ums.py | 3 +++
  3 files changed, 13 insertions(+), 1 deletion(-)
Quoted 1 line
test/py/conftest.py +0 -0
Quoted 7 lines
Quoted 4 lines
test/py/conftest.py +0 -0
@@ -225,7 +225,13 @@ def pytest_generate_tests(metafunc):
              # ... otherwise, see if there's a key that contains a list

of

              # values to use instead.

Is this function comment still accurate?

Quoted 4 lines

Yes; that comment is referring to the list of values that the relevant test function iterates over, not the names/IDs of the values or anything like that.

Applied to u-boot-dm, thanks!