iotests: Run pylint and mypy in a testcase
cover iotests: Run pylint and mypy in a testcase
Kevin Wolf (2):
iotests: Fix incomplete type declarations
iotests: Run pylint and mypy in a testcase
tests/qemu-iotests/iotests.py | 8 +++---- tests/qemu-iotests/297 | 44 +++++++++++++++++++++++++++++++++++ tests/qemu-iotests/297.out | 3 +++ tests/qemu-iotests/group | 1 + 4 files changed, 52 insertions(+), 4 deletions(-) create mode 100755 tests/qemu-iotests/297 create mode 100644 tests/qemu-iotests/297.out
Signature · 1 line
2.25.3
PATCH 1/2 iotests: Fix incomplete type declarations
We need to fix only a few places so that iotests.py can pass mypy --disallow-incomplete-defs, which seems to be a desirable option to have enabled in the long run.
--- tests/qemu-iotests/iotests.py | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
tests/qemu-iotests/iotests.py +4 -4
@@ -1040,7 +1040,7 @@ def _verify_cache_mode(supported_cache_modes: Sequence[str] = ()) -> None: if supported_cache_modes and (cachemode not in supported_cache_modes): notrun('not suitable for this cache mode: %s' % cachemode) -def _verify_aio_mode(supported_aio_modes: Sequence[str] = ()): +def _verify_aio_mode(supported_aio_modes: Sequence[str] = ()) -> None: if supported_aio_modes and (aiomode not in supported_aio_modes): notrun('not suitable for this aio mode: %s' % aiomode)
@@ -1087,7 +1087,8 @@ def skip_if_unsupported(required_formats=(), read_only=False): '''Skip Test Decorator Runs the test if all the required formats are whitelisted''' def skip_test_decorator(func): - def func_wrapper(test_case: QMPTestCase, *args, **kwargs): + def func_wrapper(test_case: QMPTestCase, *args: List[Any], + **kwargs: Dict[str, Any]) -> None: if callable(required_formats): fmts = required_formats(test_case) else:
@@ -1097,9 +1098,8 @@ def skip_if_unsupported(required_formats=(), read_only=False): if usf_list: msg = f'{test_case}: formats {usf_list} are not whitelisted' test_case.case_skip(msg) - return None else: - return func(test_case, *args, **kwargs) + func(test_case, *args, **kwargs) return func_wrapper return skip_test_decorator
Signature · 1 line
2.25.3
Re: [PATCH 1/2] iotests: Fix incomplete type declarations
Quoted 8 lines · Kevin Wolf
We need to fix only a few places so that
iotests.pycan pass mypy --disallow-incomplete-defs, which seems to be a desirable option to have enabled in the long run.Signed-off-by Kevin Wolf <[email protected]>--- tests/qemu-iotests/iotests.py | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
PATCH 2/2 iotests: Run pylint and mypy in a testcase
In reply to Kevin WolfWe made sure that iotests.py passes pylint. It would be a shame if we allowed new patches in that break this again, so let's just add a meta-test case that runs pylint on it.
While we don't pass mypy --strict yet, we can already run it with a few options that would be part of --strict to make sure that we won't regress on these aspects at least until we can enable the full thing.
--- tests/qemu-iotests/297 | 44 ++++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/297.out | 3 +++ tests/qemu-iotests/group | 1 + 3 files changed, 48 insertions(+) create mode 100755 tests/qemu-iotests/297 create mode 100644 tests/qemu-iotests/297.out
tests/qemu-iotests/297 new file +44 -0
@@ -0,0 +1,44 @@ +#!/usr/bin/env bash +# +# Copyright (C) 2020 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. + +seq=$(basename $0) +echo "QA output created by $seq" + +status=1 # failure is the default! + +# get standard environment +. ./common.rc + +if ! type -p "pylint-3" > /dev/null; then + _notrun "pylint-3 not found" +fi +if ! type -p "mypy" > /dev/null; then + _notrun "mypy not found" +fi + +pylint-3 --score=n iotests.py + +MYPYPATH=../../python/ mypy --warn-unused-configs --disallow-subclassing-any \ + --disallow-any-generics --disallow-incomplete-defs \ + --disallow-untyped-decorators --no-implicit-optional \ + --warn-redundant-casts --warn-unused-ignores \ + --no-implicit-reexport iotests.py + +# success, all done +echo "*** done" +rm -f $seq.full +status=0
tests/qemu-iotests/297.out new file +3 -0
@@ -0,0 +1,3 @@ +QA output created by 297 +Success: no issues found in 1 source file +*** done
tests/qemu-iotests/group +1 -0
@@ -299,3 +299,4 @@ 289 rw quick 290 rw auto quick 292 rw auto quick +297 meta
Signature · 1 line
2.25.3
Quoted 8 lines · Kevin Wolf
We made sure that
iotests.pypasses pylint. It would be a shame if we allowed new patches in that break this again, so let's just add a meta-test case that runs pylint on it.While we don't pass mypy --strict yet, we can already run it with a few options that would be part of --strict to make sure that we won't regress on these aspects at least until we can enable the full thing.
Quoted 8 lines
Signed-off-by Kevin Wolf <[email protected]>--- tests/qemu-iotests/297 | 44 ++++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/297.out | 3 +++ tests/qemu-iotests/group | 1 + 3 files changed, 48 insertions(+) create mode 100755 tests/qemu-iotests/297 create mode 100644 tests/qemu-iotests/297.out
Bit of a shame that this takes 8 s (on my machine at least) and will run with every format/protocol combination unless I explictly exclude it with -x meta... But I suppose the actual problem here is the fact that tests still can’t just define a “This is the format/protocol combination I require” and then you can just let all tests run once with that default combination. (And maybe afterwards run all tests again with some custom combinations, but only when that makes sense.)
Well. Not a new problem.
Am 13.05.2020 um 14:14 hat Max Reitz geschrieben:
Quoted 18 lines · Kevin Wolf
Quoted 8 lines
We made sure that
iotests.pypasses pylint. It would be a shame if we allowed new patches in that break this again, so let's just add a meta-test case that runs pylint on it.While we don't pass mypy --strict yet, we can already run it with a few options that would be part of --strict to make sure that we won't regress on these aspects at least until we can enable the full thing.
Quoted 8 lines
Signed-off-by Kevin Wolf <[email protected]>--- tests/qemu-iotests/297 | 44 ++++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/297.out | 3 +++ tests/qemu-iotests/group | 1 + 3 files changed, 48 insertions(+) create mode 100755 tests/qemu-iotests/297 create mode 100644 tests/qemu-iotests/297.out
Quoted 3 lines
Bit of a shame that this takes 8 s (on my machine at least) and will run with every format/protocol combination unless I explictly exclude it with -x meta...
Yes, it's surprising how slow these tools are. At least mypy caches some stuff, so the second run is considerably faster, but pylint doesn't do that.
I wonder if there is some overlap between mypy and pylint that we could configure away in pylint to speed it up.
Quoted 5 lines
But I suppose the actual problem here is the fact that
tests still can’t just define a “This is the format/protocol combination I require” and then you can just let all tests run once with that default combination. (And maybe afterwards run all tests again with some custom combinations, but only when that makes sense.)
It's probably not hard to find more "actual problems" in the test harness...
Kevin
Quoted 4 lines · Kevin Wolf
Kevin Wolf (2):
iotests: Fix incomplete type declarations
iotests: Run pylint and mypy in a testcase
Quoted 8 lines
tests/qemu-iotests/iotests.py | 8 +++---- tests/qemu-iotests/297 | 44 +++++++++++++++++++++++++++++++++++ tests/qemu-iotests/297.out | 3 +++ tests/qemu-iotests/group | 1 + 4 files changed, 52 insertions(+), 4 deletions(-) create mode 100755 tests/qemu-iotests/297 create mode 100644 tests/qemu-iotests/297.out
Cool!
Maybe the approach of putting this in 'make check' somewhere can alleviate that expensive test being performed in iotests directly (as Max notes), but, uhm. whatever.
For now, this will help prevent the backslide.