selftests: alsa - add PCM test
Subsystems (3)
Jaroslav Kysela
maintainerRFC PATCH v2 1/1 selftests: alsa - add PCM test
This initial code does a simple sample transfer tests. By default, all PCM devices are detected and tested with short and long buffering parameters for 4 seconds. If the sample transfer timing is not in a +-100ms boundary, the test fails. Only the interleaved buffering scheme is supported in this version.
The configuration may be modified with the configuration files. A specific hardware configuration is detected and activated using the sysfs regex matching. This allows to use the DMI string (/sys/class/dmi/id/* tree) or any other system parameters exposed in sysfs for the matching for the CI automation.
The configuration file may also specify the PCM device list to detect the missing PCM devices.
v1..v2:
- added missing alsa-local.h header file
--- tools/testing/selftests/alsa/Makefile | 4 +- tools/testing/selftests/alsa/alsa-local.h | 22 + tools/testing/selftests/alsa/conf.c | 379 +++++++++++++ .../alsa/conf.d/Lenovo_ThinkPad_P1_Gen2.conf | 76 +++ tools/testing/selftests/alsa/pcm-test.c | 523 ++++++++++++++++++ 5 files changed, 1003 insertions(+), 1 deletion(-) create mode 100644 tools/testing/selftests/alsa/alsa-local.h create mode 100644 tools/testing/selftests/alsa/conf.c create mode 100644 tools/testing/selftests/alsa/conf.d/Lenovo_ThinkPad_P1_Gen2.conf create mode 100644 tools/testing/selftests/alsa/pcm-test.c
tools/testing/selftests/alsa/Makefile +3 -1
@@ -7,6 +7,8 @@ ifeq ($(LDLIBS),) LDLIBS += -lasound endif -TEST_GEN_PROGS := mixer-test +TEST_GEN_PROGS := mixer-test pcm-test + +pcm-test: pcm-test.c conf.c include ../lib.mk
tools/testing/selftests/alsa/alsa-local.h new file +22 -0
@@ -0,0 +1,22 @@ +// SPDX-License-Identifier: GPL-2.0 +// +// kselftest configuration helpers for the hw specific configuration +// +// Original author: Jaroslav Kysela <[email protected]> +// Copyright (c) 2022 Red Hat Inc. + +#ifndef __ALSA_LOCAL_H +#define __ALSA_LOCAL_H + +#include <alsa/asoundlib.h> + +void conf_load (void ); +void conf_free (void ); +snd_config_t *conf_by_card (int card); +snd_config_t *conf_get_subtree (snd_config_t *root,const char *key1,const char *key2); +int conf_get_count (snd_config_t *root,const char *key1,const char *key2); +const char *conf_get_string (snd_config_t *root,const char *key1,const char *key2,const char *def); +long conf_get_long (snd_config_t *root,const char *key1,const char *key2,long def); +int conf_get_bool (snd_config_t *root,const char *key1,const char *key2,int def); + +#endif /* __ALSA_LOCAL_H */
tools/testing/selftests/alsa/conf.c new file +379 -0
379 lines not on this page · raw message
tools/testing/selftests/alsa/conf.d/Lenovo_ThinkPad_P1_Gen2.conf new file +76 -0
76 lines not on this page · raw message
tools/testing/selftests/alsa/pcm-test.c new file +523 -0
523 lines not on this page · raw message
Signature · 1 line
2.35.3
Quoted 5 lines · Jaroslav Kysela
This initial code does a simple sample transfer tests. By default, all PCM devices are detected and tested with short and long buffering parameters for 4 seconds. If the sample transfer timing is not in a +-100ms boundary, the test fails. Only the interleaved buffering scheme is supported in this version.
Oh, thanks for picking this up - something like this has been on my mind for ages! This should probably be copied to Shuah and the kselftest list as well, I've added them. This looks basically good to me, I've got a bunch of comments below but I'm not sure any of them except possibly the one about not putting values in the configuration file by default should block getting this merged so:
Quoted 6 lines
The configuration may be modified with the configuration files. A specific hardware configuration is detected and activated using the sysfs regex matching. This allows to use the DMI string (
/sys/class/dmi/id/*tree) or any other system parameters exposed in sysfs for the matching for the CI automation.
Quoted 7 lines
The configuration file may also specify the PCM device list to detect the missing PCM devices.
create mode 100644 tools/testing/selftests/alsa/alsa-local.h create mode 100644 tools/testing/selftests/alsa/conf.c create mode 100644 tools/testing/selftests/alsa/conf.d/Lenovo_ThinkPad_P1_Gen2.conf create mode 100644 tools/testing/selftests/alsa/pcm-test.c
This is a bit unusual for kselftest and might create a bit of churn but does seem sensible and reasonable to me, it's on the edge of what kselftest usually covers but seems close enough in scope. I worry a bit about ending up needing to add a config fragment as a result but perhaps we can get away without.
tools/testing/selftests/alsa/conf.d/Lenovo_ThinkPad_P1_Gen2.conf line 48
Quoted 13 lines
index 000000000000..0a83f35d43eb
tools/testing/selftests/alsa/conf.d/Lenovo_ThinkPad_P1_Gen2.conf new file +9
-0+ pcm.0.0 { + PLAYBACK { + test.time1 { + access RW_INTERLEAVED # can be omitted - default + format S16_LE # can be omitted - default + rate 48000 # can be omitted - default + channels 2 # can be omitted - default + period_size 512 + buffer_size 4096
I think it'd be better to leave these commented by default, especially if/once we improve the enumeration. That way the coverage will default to whatever the tool does by default on the system (including any checking of constraints for example). I guess we might want to add a way of saying "here's what I expect the constraints to be" but that's very much future work.
tools/testing/selftests/alsa/conf.d/Lenovo_ThinkPad_P1_Gen2.conf
Quoted 10 lines
tools/testing/selftests/alsa/conf.d/Lenovo_ThinkPad_P1_Gen2.conf new file +10
-0+#ifdef SND_LIB_VER +#if SND_LIB_VERSION >= SND_LIB_VER(1, 2, 6) +#define LIB_HAS_LOAD_STRING +#endif +#endif + +#ifndef LIB_HAS_LOAD_STRING +static int snd_config_load_string(snd_config_t **config, const char *s, + size_t size) +{
This is also in mixer-test, we should pull it into a helper library too. Something that could be done separately/incrementally.
tools/testing/selftests/alsa/conf.d/Lenovo_ThinkPad_P1_Gen2.conf
Quoted 6 lines
tools/testing/selftests/alsa/conf.d/Lenovo_ThinkPad_P1_Gen2.conf new file +5
-0+ for (i = 0; i < 4; i++) { + + snd_pcm_drain(handle); + ms = timestamp_diff_ms(&tstamp); + if (ms < 3900 || ms > 4100) {
It feels like the runtime might be usefully parameterised here - there's a tradeoff with detecting inaccurate clocks and runtime that people might want to make.
tools/testing/selftests/alsa/conf.d/Lenovo_ThinkPad_P1_Gen2.conf
Quoted 7 lines
tools/testing/selftests/alsa/conf.d/Lenovo_ThinkPad_P1_Gen2.conf new file +6
-0+ ksft_set_plan(num_missing + num_pcms * TESTS_PER_PCM); + for (pcm = pcm_missing; pcm != NULL; pcm = pcm->next) { + ksft_test_result(false, "test.missing.%d.%d.%d.%s\n", + pcm->card, pcm->device, pcm->subdevice, + snd_pcm_stream_name(pcm->stream)); + }
We don't seem to report a successful test.missing anywhere like find_pcms() so if we ever hit a test.missing then it'll look like a new test, old test runs won't have logged the failure. That can change how people look at any failures that crop up, "it's new and never worked" is different to "this used to work" and people are likely to just be running kselftest rather than specifically know this test. It'd be better if we counted the cards in the config and used that for our expected number of test.missings, logging cards that we find here as well.
tools/testing/selftests/alsa/conf.d/Lenovo_ThinkPad_P1_Gen2.conf
Quoted 4 lines
tools/testing/selftests/alsa/conf.d/Lenovo_ThinkPad_P1_Gen2.conf new file +4
-0+ for (pcm = pcm_list; pcm != NULL; pcm = pcm->next) { + test_pcm_time1(pcm, "test.time1", "S16_LE", 48000, 2, 512, 4096); + test_pcm_time1(pcm, "test.time2", "S16_LE", 48000, 2, 24000, 192000); + }
It does feel like especially in the case where no configuration is specified we should be eumerating what the card can do and both potentially doing more tests (though there's obviously an execution time tradeoff with going overboard there) and skipping configurations that the card never claimed to support in the first place. In particular I'm expecting we'll see some cards that only do either 44.1kHz or 48kHz and will get spurious fails by default, and I'd like to see coverage of mono playback on cards that claim to support it because I suspect there's a bunch of them that don't actually do the right thing.
Like I say most of this could be done incrementally if we decide it needs to get done at all though, we shouldn't let perfect be the enemy of good.
Quoted 19 lines · Jaroslav Kysela
This initial code does a simple sample transfer tests. By default, all PCM devices are detected and tested with short and long buffering parameters for 4 seconds. If the sample transfer timing is not in a +-100ms boundary, the test fails. Only the interleaved buffering scheme is supported in this version.
The configuration may be modified with the configuration files. A specific hardware configuration is detected and activated using the sysfs regex matching. This allows to use the DMI string (
/sys/class/dmi/id/*tree) or any other system parameters exposed in sysfs for the matching for the CI automation.The configuration file may also specify the PCM device list to detect the missing PCM devices.
v1..v2:
- added missingalsa-local.hheader file
Quoted 8 lines
Cc Mark Brown <[email protected]>Cc Pierre-Louis Bossart <[email protected]>Cc Liam Girdwood <[email protected]>Cc Jesse Barnes <[email protected]>Cc Jimmy Cheng-Yi Chiang <[email protected]>Cc Curtis Malainey <[email protected]>Cc Brian Norris <[email protected]>Signed-off-by Jaroslav Kysela <[email protected]>
Sorry for the late reply.
The patch looks good and works fine on the smoke test, so I applied it now. There are a few issues Mark already commented on, and we can fix or improve them in incremental manner later.
thanks,
Takashi
Jaroslav Kysela
maintainerQuoted 30 lines · Takashi Iwai
Quoted 19 lines · Jaroslav Kysela
This initial code does a simple sample transfer tests. By default, all PCM devices are detected and tested with short and long buffering parameters for 4 seconds. If the sample transfer timing is not in a +-100ms boundary, the test fails. Only the interleaved buffering scheme is supported in this version.
The configuration may be modified with the configuration files. A specific hardware configuration is detected and activated using the sysfs regex matching. This allows to use the DMI string (
/sys/class/dmi/id/*tree) or any other system parameters exposed in sysfs for the matching for the CI automation.The configuration file may also specify the PCM device list to detect the missing PCM devices.
v1..v2:
- added missingalsa-local.hheader fileQuoted 8 lines
Cc Mark Brown <[email protected]>Cc Pierre-Louis Bossart <[email protected]>Cc Liam Girdwood <[email protected]>Cc Jesse Barnes <[email protected]>Cc Jimmy Cheng-Yi Chiang <[email protected]>Cc Curtis Malainey <[email protected]>Cc Brian Norris <[email protected]>Signed-off-by Jaroslav Kysela <[email protected]>
Quoted 5 lines
Sorry for the late reply.
The patch looks good and works fine on the smoke test, so I applied it now. There are a few issues Mark already commented on, and we can fix or improve them in incremental manner later.
Thanks. I will try to work the Mark's comments when I find a free time.
Jaroslav
Signature · 2 lines
Jaroslav Kysela <[email protected]> Linux Sound Maintainer; ALSA Project; Red Hat, Inc.
Mark Brown
maintainertools/testing/selftests/alsa/Makefile ifeq ($(LDLIBS),) · line 14
Quoted 12 lines · Jaroslav Kysela
tools/testing/selftests/alsa/Makefile +3
-1@@ -7,6 +7,8 @@ ifeq ($(LDLIBS),) LDLIBS += -lasound endif -TEST_GEN_PROGS := mixer-test +TEST_GEN_PROGS := mixer-test pcm-test + +pcm-test: pcm-test.c conf.c include ../lib.mk
This breaks out of tree kselftest builds - if you specify an output directory with O= (as all the main automated test systems do) then any target specific rules need to prepend $(OUTPUT)/ to specify the right location for the output (this also works fine with in tree builds, a default OUTPUT is generated). Unfortunately looking at this just now simply fixing that is causing only pcm-test to be built and not the existing mixer-test, and explicitly specifying commands for mixer-test causes that to be built and not pcm-test even explicitly asking to build "all" as the target for reasons I'm not yet clear on.
You can see an example failure in:
https://storage.kernelci.org/next/master/next-20221128/arm64/defconfig/gcc-10/logs/kselftest.log
The whole kselftest build setup is rather fragile and frustrating to work with unfortunately, especially with tests built from multiple objects. The only fix I've come up with thus far is to inline conf.c into pcm-test.c (a #include will do the trick) which isn't great.
Jaroslav Kysela
maintainerQuoted 25 lines · Mark Brown
tools/testing/selftests/alsa/Makefile ifeq ($(LDLIBS),) · line 14
Quoted 12 lines · Jaroslav Kysela
tools/testing/selftests/alsa/Makefile +3
-1@@ -7,6 +7,8 @@ ifeq ($(LDLIBS),) LDLIBS += -lasound endif -TEST_GEN_PROGS := mixer-test +TEST_GEN_PROGS := mixer-test pcm-test + +pcm-test: pcm-test.c conf.c include ../lib.mkThis breaks out of tree kselftest builds - if you specify an output directory with O= (as all the main automated test systems do) then any target specific rules need to prepend $(OUTPUT)/ to specify the right location for the output (this also works fine with in tree builds, a default OUTPUT is generated). Unfortunately looking at this just now simply fixing that is causing only pcm-test to be built and not the existing mixer-test, and explicitly specifying commands for mixer-test causes that to be built and not pcm-test even explicitly asking to build "all" as the target for reasons I'm not yet clear on.
Quoted 8 lines
You can see an example failure in:
https://storage.kernelci.org/next/master/next-20221128/arm64/defconfig/gcc-10/logs/kselftest.logThe whole kselftest build setup is rather fragile and frustrating to work with unfortunately, especially with tests built from multiple objects. The only fix I've come up with thus far is to inline
conf.cintopcm-test.c(a #include will do the trick) which isn't great.
Thanks Mark for the feedback. I tried to resolve this issue using the rules found in the rseq tests programs - create a shared library and link both test programs to this library. It seems working for me. Patch [1].
Jaroslav
[1] https://lore.kernel.org/alsa-devel/[email protected]/
Signature · 2 lines
Jaroslav Kysela <[email protected]> Linux Sound Maintainer; ALSA Project; Red Hat, Inc.