soc-dsp programming model for loopbacks
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 10 lines · Pierre-Louis Bossart
Hi,
I've been doing some work on audio loopback (FM radio, BT, Modem -> audio codec), and I am somewhat confused by the soc-dsp programming model. Let me take the FM radio example:If the FM radio is directly connected to the audio codec, my understanding is that changing the routing with codec controls will trigger the DAPM logic, which will turn on everything that needs to be on. Very simple, no matter if the FM-codec link is analog or digital.
Yep, this is correct. soc-dsp (now know as Dynamic PCM) takes care of any digital links that require the host CPU to perform any PCM or DAI operations (based again on DAPM graph status). i.e. on OMAP we can start the McBSP to FM Digital DAI based on the DAPM graph status and then configure the link format, speed etc to optimal values for the ABE and FM.
It's not required for the analog path as DAPM will switch on the correct components for FM analog to be played on the CODEC Speaker output.
Quoted 11 lines
Now if the FM radio routing is handled with a digital loopback on the application processor audio dsp (omap-abe, Intel SST, etc), then soc-dsp will need to be used. And for a simple FM playback, I need to
1. configure the audio codec routing for output selection
2. open a virtual front-end for FM capture
3. configure the DSP routing to link capture front-end to I2S1 backend (FM interface)
4. open a virtual front-end for FM playback
5. configure the DSP routing to link playback front-end to I2S2 backend (codec interface)
Using Dynamic PCM here really depends on your hardware capabilities. On OMAP, the host CPU configures and manages all DAIs so Dynamic PCM manages the DAI and PCM operations. We also have the need to re-write the hw_params and DAI formats between the FEs and BEs for optimum performance.
Quoted 7 lines
That seems complicated. To some extent, having the ability to have two back-ends connected together would make more sense, and would simplify the programming model a great deal. User-space code would be similar for loopbacks internal to the codecs or handled on the application processor. This would apply to Bluetooth and modem connections as well. Without this capability, we will end-up with multiple 'virtual' front-ends (6 in my case), making user-space code quite complex.
It's not really that much extra code, it's really just open(), hw_params() and pcm_start() on the FE. Steps 1,3 & 5 are compulsory anyway for any audio device, but we need to configure format, rate, DAI clocking for the loopback. Having the FE PCM gives us this capability, although we dont transfer any data to or from the host with the FE so it's use is only for configuration.
We could eventually remove steps 2 and 4 for the FE DAI link, and look at hard coding the hw_params() in the mach driver for the loopback link. That way the only user space driven actions would be to configure the mixers in the CODEC and DSP for the correct route. DAPM would then detect the path and Dynamic PCM would use the hard coded configuration or bespoke mach driver logic to configure the loopback DAI link based on use case. This would have to be done after the basic Dynamic PCM infrastructure was upstream though (unless you have a patch atm).
Quoted 5 lines
Looking at the current soc-dsp code, I saw that each back-end is supposed to have at least one front-end client, and the impact of my proposal seems fairly important. Before I start looking further into code changes, I wanted to see if my understanding is correct and if there are other ideas to simplify loopbacks.
Atm with the current code base we can support routing many FE PCMs to many BE PCMs. We can support the BE loopback, but this does require the FE to be opened atm in order to configure the DAI link.
Regards
Liam
Quoted 4 lines · Liam Girdwood
On Tue, 2012-01-24 at 13:36 -0800, Pierre-Louis Bossart wrote:
We could eventually remove steps 2 and 4 for the FE DAI link, and look at hard coding the
hw_params()in the mach driver for the loopback link.
That's what the CODEC<->CODEC stuff is all about, hiding the link from userspace.
Quoted 6 lines
That way the only user space driven actions would be to configure the mixers in the CODEC and DSP for the correct route. DAPM would then detect the path and Dynamic PCM would use the hard coded configuration or bespoke mach driver logic to configure the loopback DAI link based on use case. This would have to be done after the basic Dynamic PCM infrastructure was upstream though (unless you have a patch atm).
I've sort of got one, tidying it up and pushing it out keeps on being very near the top of my list but I've never actually got it looking nice enough for me to actually want to send it out.
Quoted 9 lines · Mark Brown
Quoted 4 lines · Liam Girdwood
On Tue, 2012-01-24 at 13:36 -0800, Pierre-Louis Bossart wrote:
We could eventually remove steps 2 and 4 for the FE DAI link, and look at hard coding the
hw_params()in the mach driver for the loopback link.That's what the CODEC<->CODEC stuff is all about, hiding the link from userspace.
Ok, cool - Is the formatting and params hard coded ?
Liam
Quoted 6 lines · Liam Girdwood
Quoted 2 lines · Mark Brown
That's what the CODEC<->CODEC stuff is all about, hiding the link from userspace.
Ok, cool - Is the formatting and params hard coded ?
My implementations thus far have been able to infer these from the capabilities of the devices they're connecting (one end of the link is usually a device with a fixed configuration) and therefore not needed to worry about this. However this has always been the most painful bit of the code so I'm probably going to make it hard coded or something.
Quoted 2 lines
That's what the CODEC<->CODEC stuff is all about, hiding the link from userspace.
I can see how this would simplify the loopback part, but doesn't it make the 'regular' playback/capture functionality more complex? If my DSP is seen as a CODEC, how do I represent the CPU-DSP interface (DMA or shared memory)?
A message that is not in the archive (sent off-list or not yet received): 002d01ccdb9c$7099f1e0$51cdd5a0$@[email protected]
Quoted 6 lines · Pierre-Louis Bossart
Quoted 2 lines
That's what the CODEC<->CODEC stuff is all about, hiding the link from userspace.
I can see how this would simplify the loopback part, but doesn't it make the 'regular' playback/capture functionality more complex? If my DSP is seen as a CODEC, how do I represent the CPU-DSP interface (DMA or shared memory)?
You'd need a DAI for that interface but you'll need that no matter what you do, it'll just be a very simple DAI that needs no configuration for things like AIF format. With the soc-pcm setup you also need the front and back end DAIs so it's about the same, the benefit comes at runtime. My primary focus for the CODEC<->CODEC links is actually things like basebands rather than on-SoC DSPs, it's just that if the hardware can be thought of in the same way it should simplify things a little.
Quoted 5 lines
Quoted 4 lines
I can see how this would simplify the loopback part, but doesn't it make the 'regular' playback/capture functionality more complex? If my DSP is seen as a CODEC, how do I represent the CPU-DSP interface (DMA or shared memory)?
Quoted 7 lines
You'd need a DAI for that interface but you'll need that no matter what you do, it'll just be a very simple DAI that needs no configuration for things like AIF format. With the soc-pcm setup you also need the front and back end DAIs so it's about the same, the benefit comes at runtime. My primary focus for the CODEC<->CODEC links is actually things like basebands rather than on-SoC DSPs, it's just that if the hardware can be thought of in the same way it should simplify things a little.
Bear with me here, I am still somewhat confused.
- Liam suggested either a hard-coded DAI configuration or machine driver logic to remove the need for virtual front-ends
- Mark suggested to consider the DSP as a CODEC with some work to re-implement the host-dsp interface.
Is this CODEC<->CODEC solution on top of the Dynamic PCM patches? Or are we talking about two completely different approaches?
Thanks for your feedback
-Pierre
Quoted 7 lines · Pierre-Louis Bossart
Bear with me here, I am still somewhat confused.
- Liam suggested either a hard-coded DAI configuration or machine driver logic to remove the need for virtual front-ends
- Mark suggested to consider the DSP as a CODEC with some work to re-implement the host-dsp interface.
Is this CODEC<->CODEC solution on top of the Dynamic PCM patches? Or are we talking about two completely different approaches?
The two are orthogonal to each other but complimenary. Like I say it really depends on how your hardware is wired up - it depends on the level of isolation that the DSP provides between the CPU and the outside world.
If the system looks like:
CPU <-> DSP <-> External
you should model the DSP as a separate device. If on the other hand it looks more like this:
DSP External
^----v----^
CPUthen soc-pcm is what you're looking for.
Quoted 23 lines · Mark Brown
Quoted 7 lines · Pierre-Louis Bossart
Bear with me here, I am still somewhat confused.
- Liam suggested either a hard-coded DAI configuration or machine driver logic to remove the need for virtual front-ends
- Mark suggested to consider the DSP as a CODEC with some work to re-implement the host-dsp interface.
Is this CODEC<->CODEC solution on top of the Dynamic PCM patches? Or are we talking about two completely different approaches?The two are orthogonal to each other but complimenary. Like I say it really depends on how your hardware is wired up - it depends on the level of isolation that the DSP provides between the CPU and the outside world.
If the system looks like:
CPU <-> DSP <-> Externalyou should model the DSP as a separate device. If on the other hand it looks more like this:
Quoted 5 lines
DSP External ^----v----^ CPUthen soc-pcm is what you're looking for.
Fwiw, the OMAP ABE architecture mixes both above diagrams. We are using dynamic PCM for managing all the DAI links and will add in support to use the CODEC <-> CODEC stuff for the loopback DAI interfaces (saving the need to open()), hw_oparams() etc).
Regards
Liam
Quoted 4 lines · Liam Girdwood
Quoted 1 line · Mark Brown
CPU <-> DSP <-> External
Quoted 8 lines
Quoted 3 lines
DSP External ^----v----^ CPUFwiw, the OMAP ABE architecture mixes both above diagrams. We are using dynamic PCM for managing all the DAI links and will add in support to use the CODEC <-> CODEC stuff for the loopback DAI interfaces (saving the need to
open()),hw_oparams()etc).
In terms of the data path I think it's pretty much always like the second path with the DMA controller owned by Linux driving the data between the various components?
Quoted 9 lines · Mark Brown
Quoted 7 lines · Liam Girdwood
Quoted 5 lines · Mark Brown
CPU <-> DSP <-> ExternalDSP External ^----v----^ CPU
Quoted 8 lines
Quoted 4 lines
Fwiw, the OMAP ABE architecture mixes both above diagrams. We are using dynamic PCM for managing all the DAI links and will add in support to use the CODEC <-> CODEC stuff for the loopback DAI interfaces (saving the need to
open()),hw_oparams()etc).In terms of the data path I think it's pretty much always like the second path with the DMA controller owned by Linux driving the data between the various components?
Wrt OMAP, the DMA and the ABE can drive data to the component drivers. Both DMA and ABE only drive data at the request of the CPU (via PCM and DAI ops).
Regards
Liam
Quoted 10 lines · Liam Girdwood
We could eventually remove steps 2 and 4 for the FE DAI link, and look at hard coding the
hw_params()in the mach driver for the loopback link.
That way the only user space driven actions would be to configure the mixers in the CODEC and DSP for the correct route. DAPM would then detect the path and Dynamic PCM would use the hard coded configuration or bespoke mach driver logic to configure the loopback DAI link based on
use case. This would have to be done after the basic Dynamic PCM infrastructure was upstream though (unless you have a patch atm).
Hi,
Sorry to join the party late :-)
Liam, if we configure the hw_params in the machine driver statically, represent the DSP using a map along with a codec kind of modeling.
Would the BEs be triggered on from soc_dsp_runtime_update() when the loopback is established thru the DSP.
This way we avoid all the "virtual" FEs. Use loopback to turn on codec and DSP (thru BEs)
Would this make sense, or I need more coffee :)
Signature · 1 line
~Vinod
A message that is not in the archive (sent off-list or not yet received): CABS+qY2GADPO+wkmDf5hzT7uaH7v1zNrdAGKnLbip-QAL8V_RQ@mail.gmail.com
Quoted 12 lines · Liam
Quoted 11 lines · Vinod Koul
Quoted 10 lines · Liam Girdwood
We could eventually remove steps 2 and 4 for the FE DAI link, and look at hard coding the
hw_params()in the mach driver for the loopback link.
That way the only user space driven actions would be to configure the mixers in the CODEC and DSP for the correct route. DAPM would then detect the path and Dynamic PCM would use the hard coded configuration or bespoke mach driver logic to configure the loopback DAI link based on
use case. This would have to be done after the basic Dynamic PCM infrastructure was upstream though (unless you have a patch atm).
Quoted 4 lines
Quoted 3 lines
Hi,
Sorry to join the party late :-)
No worries ;-)
Quoted 4 lines
Quoted 4 lines
Liam, if we configure the
hw_paramsin the machine driver statically, represent the DSP using a map along with a codec kind of modeling.
In OMAP we reconfigure (or fixup) some of the hw_params() in the machine driver for a BE DAI link that cant operate at the requested FE params. The ABE takes care of any conversion though.
Quoted 6 lines
Quoted 6 lines
Would the BEs be triggered on from
soc_dsp_runtime_update()when the loopback is established thru the DSP.
This way we avoid all the "virtual" FEs. Use loopback to turn on codec and DSP (thru BEs)Would this make sense, or I need more coffee :)
In the code base atm we have to use a FE pcm to trigger the BE loopback DAIs. however, the intention is to use Mark's CODEC <-> CODEC DAI link work to support the loopback with hard coded params in the machine driver. This would save the need for any FE pcm operations.
Regards
Liam
Quoted 5 lines
Quoted 4 lines
--
~Vinod
Quoted 17 lines · Liam Girdwood
Quoted 5 lines · Liam
On 1 February 2012 09:07, Vinod Koul <[email protected]> wrote
Quoted 3 lines
Liam, if we configure the
hw_paramsin the machine driver statically, represent the DSP using a map along with a codec kind of modeling.In OMAP we reconfigure (or fixup) some of the
hw_params()in the machine driver for a BE DAI link that cant operate at the requested FE params. The ABE takes care of any conversion though.Quoted 5 lines
Quoted 5 lines
Would the BEs be triggered on from
soc_dsp_runtime_update()when the loopback is established thru the DSP.
This way we avoid all the "virtual" FEs. Use loopback to turn on codec and DSP (thru BEs)
Quoted 5 lines
Quoted 1 line
Quoted 1 line
Would this make sense, or I need more coffee :)
In the code base atm we have to use a FE pcm to trigger the BE loopback DAIs. however, the intention is to use Mark's CODEC <-> CODEC DAI link work to support the loopback with hard coded params in the machine driver. This would save the need for any FE pcm operations.
precisely my thoughts :)
Assuming we dont have to worry about hw_params (hard coded in machine), then establishing a loopback with two BEs should automatically trigger DAPM to detect a loopback, and by principle start these BEs and any associated codec DAIs. Does this idea sound good?
Now to actual changes, when will we see dynamic-pcm in asoc-next :-)
And Mark, can we see your wip changes??
Signature · 1 line
~Vinod
Quoted 1 line · Vinod Koul
And Mark, can we see your wip changes??
No. I essentially never have any WIP changes, anything I do to the point where it's actually useful gets submitted to mainline pretty much immediately. I did do some stuff but it was too horrible as it was trying to figure out the hw_params automatically and this was causing appauling pain, underlying changes have bitrotted that away.
Quoted 9 lines · Mark Brown
Quoted 1 line · Vinod Koul
And Mark, can we see your wip changes??
No. I essentially never have any WIP changes, anything I do to the point where it's actually useful gets submitted to mainline pretty much immediately. I did do some stuff but it was too horrible as it was trying to figure out the
hw_paramsautomatically and this was causing appauling pain, underlying changes have bitrotted that away.
Okay thanks.
your thoughts on how this should be done?
Signature · 1 line
~Vinod
Quoted 1 line · Vinod Koul
your thoughts on how this should be done?
Well, the most pressing things are to do something about the hard coded` codec_dai/cpu_dai in the rtd structure, and to make the mechanism for starting up streams a bit more integrated with the rest of the world. Once those are in place it should be a simple matter of programming.
Quoted 3 lines · Vinod Koul
Now to actual changes, when will we see dynamic-pcm in asoc-next :-)
My apologies, customer work has been getting priority over upstreaming for me in the last 6 months or so. We should see another batch of patches next week.
Liam