ASoC: Intel: avs: Conditional path support
Subsystems (3)
Cezary Rojewski
maintainercover ASoC: Intel: avs: Conditional path support
The code presented here is a lighter version of what was initially discussed in 2021 [1]. Later, we've had a more general discussion [2] "what if the feature was part of the ASoC core instead?". The discussion turned towards leaving a simplified version within the avs/, so here is the final outcome.
What's removed when compared to the older [1] version?
- conditional path priorities
- conditional path overriding
The background:
Conditional path (condpath) helps facilitate modern audio usecases such as Echo Cancellations and Noise Reduction. These are not invoked by the means of userspace application opening an endpoint (FrontEnd) but are a "side effect" of selected PCMs running simultaneously e.g.: if both Speaker (source) and Microphone Array (sink) are running, reference data from the Speaker and take it into account when processing capture for better voice command detection ratio.
Which PCMs are needed for given conditional path to be spawned is determinated by the driver when registering the condpath.
Two patches implement the feature:
1) update the topology parser to acknowledge the new tuples
2) update the path-creation and functions that participate in the PCM-runtime e.g.: run/pause.
The runtime update is needed to keep streaming sane - all the path operations shall be synchronized when a condpath is being spawned or its status is about to change.
[1]: https://lore.kernel.org/alsa-devel/[email protected]/
[2]: https://lore.kernel.org/linux-sound/[email protected]/
Cezary Rojewski (2):
ASoC: Intel: avs: Parse conditional path tuples
ASoC: Intel: avs: Conditional-path support
include/uapi/sound/intel/avs/tokens.h | 15 ++ sound/soc/intel/avs/path.c | 267 ++++++++++++++++++++++++++ sound/soc/intel/avs/path.h | 13 ++ sound/soc/intel/avs/topology.c | 110 +++++++++++ sound/soc/intel/avs/topology.h | 8 + 5 files changed, 413 insertions(+)
Signature · 1 line
2.25.1
Cezary Rojewski
maintainerPATCH 1/2 ASoC: Intel: avs: Parse conditional path tuples
Conditional paths need information about their source and sink paths to be created which is then stored to keep track of who their parents are. That information allows to change their state accordingly to what is currently happening to their parent paths.
--- include/uapi/sound/intel/avs/tokens.h | 15 ++++ sound/soc/intel/avs/topology.c | 110 ++++++++++++++++++++++++++ sound/soc/intel/avs/topology.h | 8 ++ 3 files changed, 133 insertions(+)
include/uapi/sound/intel/avs/tokens.h +15 -0
@@ -133,6 +133,21 @@ enum avs_tplg_token {AVS_TKN_PATH_FE_FMT_ID_U32 =1902 ,AVS_TKN_PATH_BE_FMT_ID_U32 =1903 , +/* struct avs_tplg_path_template (conditional) */ +AVS_TKN_CONDPATH_TMPL_ID_U32 =1801 , +AVS_TKN_CONDPATH_TMPL_SOURCE_TPLG_NAME_STRING =2002 , +AVS_TKN_CONDPATH_TMPL_SOURCE_PATH_TMPL_ID_U32 =2003 , +AVS_TKN_CONDPATH_TMPL_SINK_TPLG_NAME_STRING =2004 , +AVS_TKN_CONDPATH_TMPL_SINK_PATH_TMPL_ID_U32 =2005 , +AVS_TKN_CONDPATH_TMPL_COND_TYPE_U32 =2006 , +AVS_TKN_CONDPATH_TMPL_OVERRIDABLE_BOOL =2007 , +AVS_TKN_CONDPATH_TMPL_PRIORITY_U8 =2008 , + +/* struct avs_tplg_path (conditional) */ +AVS_TKN_CONDPATH_ID_U32 =1901 , +AVS_TKN_CONDPATH_SOURCE_PATH_ID_U32 =2102 , +AVS_TKN_CONDPATH_SINK_PATH_ID_U32 =2103 , +/* struct avs_tplg_pin_format */ AVS_TKN_PIN_FMT_INDEX_U32 =2201 , AVS_TKN_PIN_FMT_IOBS_U32 = 2202,
sound/soc/intel/avs/topology.c +110 -0
134 lines not on this page · raw message
sound/soc/intel/avs/topology.h +8 -0
@@ -33,6 +33,7 @@ struct avs_tplg {u32 num_pplcfgs;struct avs_tplg_binding *bindings;u32 num_bindings; +struct avs_tplg_path_template *condpath_tmpls;u32 num_condpath_tmpls;struct avs_tplg_init_config *init_configs;u32 num_init_configs;
@@ -155,6 +156,10 @@ struct avs_tplg_path_template {struct snd_soc_dapm_widget *w; +/* Conditional path. */ +struct avs_tplg_path_template_id source; +struct avs_tplg_path_template_id sink; +struct list_head path_list;struct avs_tplg *owner;
@@ -176,6 +181,9 @@ struct avs_tplg_path {/* Path format requirements. */ struct avs_audio_format *fe_fmt;struct avs_audio_format *be_fmt; +/* Condpath path-variant requirements. */ +u32 source_path_id; +u32 sink_path_id;struct list_head ppl_list;
Signature · 1 line
2.25.1
Cezary Rojewski
maintainerPATCH 2/2 ASoC: Intel: avs: Conditional-path support
In reply to Cezary RojewskiCompared to standard paths, conditional paths are not invoked by the means of FE being opened by userspace and thus their variant selection is not FE/BE audio format based. These are a side effect of standard path creation if specific criteria are met.
Algorithm is implemented to walk on all existing runtime paths and match them against conditions provided by topology. These conditions are based on source and sink path formats, rather than formats provided from userspace app or present on the codec. If match is found, new path is created and tied to those which brought it into existence: source and sink path. If any of its parents perishes, so does the conditional path.
Conditional paths are used to enable any complex, modern audio scenario which involves usage of KPB, AEC and WoV modules and more.
--- sound/soc/intel/avs/path.c | 267 +++++++++++++++++++++++++++++++++++++ sound/soc/intel/avs/path.h | 13 ++ 2 files changed, 280 insertions(+)
sound/soc/intel/avs/path.c +267 -0
@@ -115,6 +115,55 @@ avs_path_find_variant(struct avs_dev *adev,return NULL ; } +static struct avs_tplg_path *avs_condpath_find_variant (struct avs_dev *adev, +struct avs_tplg_path_template *template, +struct avs_path *source, +struct avs_path *sink) +{ +struct avs_tplg_path *variant; + +list_for_each_entry (variant, &template->path_list, node) { +if (variant->source_path_id == source->template->id && + variant->sink_path_id == sink->template->id) +return variant; + } + +return NULL ; +} + +static bool avs_tplg_path_template_id_equal (struct avs_tplg_path_template_id *id, +struct avs_tplg_path_template_id *id2) +{ +return id->id == id2->id && !strcmp (id->tplg_name, id2->tplg_name); +} + +static struct avs_path *avs_condpath_find_match (struct avs_dev *adev, +struct avs_tplg_path_template *template, +struct avs_path *path,int dir) +{ +struct avs_tplg_path_template_id *id, *id2; + +if (dir) { + id = &template->source; + id2 = &template->sink; + }else { + id = &template->sink; + id2 = &template->source; + } + +/* Check whether this path is either source or sink of condpath template. */ +if (id->id != path->template->owner->id || +strcmp (id->tplg_name, path->template->owner->owner->name)) +return NULL ; + +/* Unidirectional condpaths are allowed. */ +if (avs_tplg_path_template_id_equal (id, id2)) +return path; + +/* Now find the counterpart. */ +return avs_path_find_path (adev, id2->tplg_name, id2->id); +} +static struct acpi_nhlt_config *avs_nhlt_config_or_default (struct avs_dev *adev,struct avs_tplg_module *t);
@@ -1051,6 +1100,10 @@ static int avs_path_init(struct avs_dev *adev, struct avs_path *path, path->dma_id = dma_id;INIT_LIST_HEAD (&path->ppl_list);INIT_LIST_HEAD (&path->node); +INIT_LIST_HEAD (&path->source_list); +INIT_LIST_HEAD (&path->sink_list); +INIT_LIST_HEAD (&path->source_node); +INIT_LIST_HEAD (&path->sink_node);/* create all the pipelines */ list_for_each_entry (tppl, &template->ppl_list, node) {
@@ -1134,12 +1187,129 @@ static struct avs_path *avs_path_create_unlocked(struct avs_dev *adev, u32 dma_ireturn ERR_PTR (ret); } +static void avs_condpath_free (struct avs_dev *adev,struct avs_path *path) +{ +int ret; + +list_del (&path->source_node); +list_del (&path->sink_node); + + ret =avs_path_reset (path); +if (ret <0 ) +dev_err (adev->dev,"reset condpath failed: %d\n" , ret); + + ret =avs_path_unbind (path); +if (ret <0 ) +dev_err (adev->dev,"unbind condpath failed: %d\n" , ret); + +avs_path_free_unlocked (path); +} + +static struct avs_path *avs_condpath_create (struct avs_dev *adev, +struct avs_tplg_path *template, +struct avs_path *source, +struct avs_path *sink) +{ +struct avs_path *path; +int ret; + + path =avs_path_create_unlocked (adev,0 , template); +if (IS_ERR (path)) +return path; + + ret =avs_path_bind (path); +if (ret) + goto err_bind; + + ret =avs_path_reset (path); +if (ret) + goto err_reset; + + path->source = source; + path->sink = sink; +list_add_tail (&path->source_node, &source->source_list); +list_add_tail (&path->sink_node, &sink->sink_list); + +return path; + +err_reset: +avs_path_unbind (path); +err_bind: +avs_path_free_unlocked (path); +return ERR_PTR (ret); +} + +static int avs_condpaths_walk (struct avs_dev *adev,struct avs_path *path,int dir) +{ +struct avs_soc_component *acomp; +struct avs_path *source, *sink; +struct avs_path **other; + +if (dir) { + source = path; + other = &sink; + }else { + sink = path; + other = &source; + } + +list_for_each_entry (acomp, &adev->comp_list, node) { +for (int i =0 ; i < acomp->tplg->num_condpath_tmpls; i++) { +struct avs_tplg_path_template *template; +struct avs_tplg_path *variant; +struct avs_path *cpath; + + template = &acomp->tplg->condpath_tmpls[i]; + +/* Do not create unidirectional condpaths twice. */ +if (avs_tplg_path_template_id_equal (&template->source, + &template->sink) && dir) +continue ; + + *other =avs_condpath_find_match (adev, template, path, dir); +if (!*other) +continue ; + + variant =avs_condpath_find_variant (adev, template, source, sink); +if (!variant) +continue ; + + cpath =avs_condpath_create (adev, variant, source, sink); +if (IS_ERR (cpath)) +return PTR_ERR (cpath); + } + } + +return 0 ; +} + +/* Caller responsible for holding adev->path_mutex. */ +static int avs_condpaths_walk_all (struct avs_dev *adev,struct avs_path *path) +{ +int ret; + + ret =avs_condpaths_walk (adev, path,SNDRV_PCM_STREAM_CAPTURE ); +if (ret) +return ret; + +return avs_condpaths_walk (adev, path,SNDRV_PCM_STREAM_PLAYBACK ); +} +void avs_path_free (struct avs_path *path) { +struct avs_path *cpath, *csave;struct avs_dev *adev = path->owner;mutex_lock (&adev->path_mutex); + +/* Free all condpaths this path spawned. */ +list_for_each_entry_safe (cpath, csave, &path->source_list, source_node) +avs_condpath_free (path->owner, cpath); +list_for_each_entry_safe (cpath, csave, &path->sink_list, sink_node) +avs_condpath_free (path->owner, cpath); +avs_path_free_unlocked (path); +mutex_unlock (&adev->path_mutex); }
@@ -1150,6 +1320,7 @@ struct avs_path *avs_path_create(struct avs_dev *adev, u32 dma_id, {struct avs_tplg_path *variant;struct avs_path *path; +int ret; variant =avs_path_find_variant (adev, template, fe_params, be_params);if (!variant) {
@@ -1163,7 +1334,16 @@ struct avs_path *avs_path_create(struct avs_dev *adev, u32 dma_id,mutex_lock (&adev->comp_list_mutex); path =avs_path_create_unlocked (adev, dma_id, variant); +if (IS_ERR (path)) + goto exit; + ret =avs_condpaths_walk_all (adev, path); +if (ret) { +avs_path_free_unlocked (path); + path =ERR_PTR (ret); + } + +exit:mutex_unlock (&adev->comp_list_mutex);mutex_unlock (&adev->path_mutex);
@@ -1286,6 +1466,42 @@ int avs_path_reset(struct avs_path *path)return 0 ; } +static int avs_condpath_pause (struct avs_dev *adev,struct avs_path *cpath) +{ +struct avs_path_pipeline *ppl; +int ret; + +if (cpath->state ==AVS_PPL_STATE_PAUSED ) +return 0 ; + +list_for_each_entry_reverse (ppl, &cpath->ppl_list, node) { + ret =avs_ipc_set_pipeline_state (adev, ppl->instance_id,AVS_PPL_STATE_PAUSED ); +if (ret) { +dev_err (adev->dev,"pause cpath failed: %d\n" , ret); + cpath->state =AVS_PPL_STATE_INVALID ; +return AVS_IPC_RET (ret); + } + } + + cpath->state =AVS_PPL_STATE_PAUSED ; +return 0 ; +} + +static void avs_condpaths_pause (struct avs_dev *adev,struct avs_path *path) +{ +struct avs_path *cpath; + +mutex_lock (&adev->path_mutex); + +/* If either source or sink stops, so do the attached conditional paths. */ +list_for_each_entry (cpath, &path->source_list, source_node) +avs_condpath_pause (adev, cpath); +list_for_each_entry (cpath, &path->sink_list, sink_node) +avs_condpath_pause (adev, cpath); + +mutex_unlock (&adev->path_mutex); +} +int avs_path_pause (struct avs_path *path) {struct avs_path_pipeline *ppl;
@@ -1295,6 +1511,8 @@ int avs_path_pause(struct avs_path *path)if (path->state ==AVS_PPL_STATE_PAUSED )return 0 ; +avs_condpaths_pause (adev, path); +list_for_each_entry_reverse (ppl, &path->ppl_list, node) { ret =avs_ipc_set_pipeline_state (adev, ppl->instance_id,AVS_PPL_STATE_PAUSED );
@@ -1309,6 +1527,50 @@ int avs_path_pause(struct avs_path *path)return 0 ; } +static int avs_condpath_run (struct avs_dev *adev,struct avs_path *cpath,int trigger) +{ +struct avs_path_pipeline *ppl; +int ret; + +if (cpath->state ==AVS_PPL_STATE_RUNNING ) +return 0 ; + +list_for_each_entry (ppl, &cpath->ppl_list, node) { +if (ppl->template->cfg->trigger != trigger) +continue ; + + ret =avs_ipc_set_pipeline_state (adev, ppl->instance_id,AVS_PPL_STATE_RUNNING ); +if (ret) { +dev_err (adev->dev,"run cpath failed: %d\n" , ret); + cpath->state =AVS_PPL_STATE_INVALID ; +return AVS_IPC_RET (ret); + } + } + + cpath->state =AVS_PPL_STATE_RUNNING ; +return 0 ; +} + +static void avs_condpaths_run (struct avs_dev *adev,struct avs_path *path,int trigger) +{ +struct avs_path *cpath; + +mutex_lock (&adev->path_mutex); + +/* Run conditional paths only if source and sink are both running. */ +list_for_each_entry (cpath, &path->source_list, source_node) +if (cpath->source->state ==AVS_PPL_STATE_RUNNING && + cpath->sink->state ==AVS_PPL_STATE_RUNNING ) +avs_condpath_run (adev, cpath, trigger); + +list_for_each_entry (cpath, &path->sink_list, sink_node) +if (cpath->source->state ==AVS_PPL_STATE_RUNNING && + cpath->sink->state ==AVS_PPL_STATE_RUNNING ) +avs_condpath_run (adev, cpath, trigger); + +mutex_unlock (&adev->path_mutex); +} +int avs_path_run (struct avs_path *path,int trigger) {struct avs_path_pipeline *ppl;
@@ -1332,5 +1594,10 @@ int avs_path_run(struct avs_path *path, int trigger) } path->state =AVS_PPL_STATE_RUNNING ; + +/* Granular pipeline triggering not intended for conditional paths. */ +if (trigger ==AVS_TPLG_TRIGGER_AUTO ) +avs_condpaths_run (adev, path, trigger); +return 0 ; }
sound/soc/intel/avs/path.h +13 -0
@@ -13,11 +13,24 @@#include "avs.h" #include "topology.h" +#define AVS_COND_TYPE_NONE 0 +#define AVS_COND_TYPE_AECREF 1 +struct avs_path {u32 dma_id;struct list_head ppl_list;u32 state; +/* condpath navigation for standard paths */ +struct list_head source_list; +struct list_head sink_list; + +/* conditional path fields */ +struct avs_path *source; +struct avs_path *sink; +struct list_head source_node; +struct list_head sink_node; +struct avs_tplg_path *template;struct avs_dev *owner;/* device path management */
Signature · 1 line
2.25.1
Mark Brown
maintainerQuoted 11 lines · Cezary Rojewski
The code presented here is a lighter version of what was initially discussed in 2021 [1]. Later, we've had a more general discussion [2] "what if the feature was part of the ASoC core instead?". The discussion turned towards leaving a simplified version within the avs/, so here is the final outcome.
What's removed when compared to the older [1] version?
- conditional path priorities
- conditional path overriding[...]
Applied to
https://git.kernel.org/pub/scm/linux/kernel/git/broonie/sound.git for-next
Thanks!
[1/2] ASoC: Intel: avs: Parse conditional path tuples
commit: 8bcfcb3bd3e38b8f3bb7e5eb3acb4120500994a0
[2/2] ASoC: Intel: avs: Conditional-path support
commit: 595b7f155b926460a00776cc581e4dcd01220006All being well this means that it will be integrated into the linux-next tree (usually sometime in the next 24 hours) and sent to Linus during the next merge window (or sooner if it is a bug fix), however if problems are discovered then the patch may be dropped or reverted.
You may get further e-mails resulting from automated or manual testing and review of the tree, please engage with people reporting problems and send followup patches addressing any issues that are reported if needed.
If any updates are required or you are submitting further changes they should be sent as incremental updates against current git, existing patches will not be replaced.
Please add any relevant lists and maintainers to the CCs when replying to this mail.
Thanks,
Mark