ASoC: snd_soc_jack for HDMI audio: does it make sense?

Hello!

I have been working on prototypes for the ASoC OMAP HDMI audio driver to propagate events from the HDMI output (e.g., display getting enabled/disabled/suspended). This for the users of the driver to react to such events. For instance, if the display is disabled or disconected, audio could be stopped, rerouted or whatever other decision the user makes. This is needed because, if, for instance, the HDMI IP goes off, audio will stall and the audio users will only see a "playback write error (DMA or IRQ trouble?)"

In my prototypes I have used snd_soc_jack for this purpose and I have some questions:

*I see snd_soc_jack is used mostly for headsets and microphones with actual external mechanical connections. Strictly, in my case I propagate events originated by the OMAP display driver (changes in the power state), and not from external events. Some of these events are generated from an actual HDMI cable connection/disconnection, though.

*Maybe the event should be propagated by omapdss/omapdrm/drm and the entity in charge of the audio policy should listen those events instead.

*I do see SND_JACK_VIDEOOUT and SND_JACK_AVOUT types so maybe it is feasible for an audio driver to report events from an AV output.

I was wondering about how much sense does it make to you guys use a snd_soc_jack in this case?

Quoted 2 lines · Ricardo Neri

I was wondering about how much sense does it make to you guys use a snd_soc_jack in this case?

HD-audio already uses the generic jack event for the HDMI/DP connection change notification as well, so I think it would make sense in general.

Quoted 8 lines · Takashi Iwai

Quoted 2 lines · Ricardo Neri

I was wondering about how much sense does it make to you guys use a snd_soc_jack in this case?

HD-audio already uses the generic jack event for the HDMI/DP connection change notification as well, so I think it would make sense in general.

The whole problem here is that we don't *have* a generic jack interface. We've got:

- sound/core/jack.c which was written to be a generic API and is used by everything that does jack support currently.

- sound/core/ctljack.c which was added later and provides separate in-kernel and userspace APIs and is currently only used by HDA.

- extcon which does have a good reason to be a separate API since that it's not audio specific (and is likely to be picked up by Android as the code was originally taken from there); it's currently not supported by the frameworks in ALSA. I'd suggest Pulse should be using it too.

This is a complete shambles for both driver authors and userspace, the ABI varies randomly with drivers and in theory driver authors have to implement everything three times which is just nuts.

What I'd like to see happening is that we merge ctljack into jack (since only HDA is going to be affected by that change it seems like the right direction to make the merge) and also add extcon support, I have looked at the extcon support.

Short term for drivers used on embedded systems I'd have to recommend extcon rather than anything ALSA-specific.

Quoted 2 lines · Mark Brown

- sound/core/ctljack.c which was added later and provides separate in-kernel and userspace APIs and is currently only used by HDA.

This is wrong. PulseAudio uses the new ctljack API. I started out with an implementation which used the input device API (sound/core/jack.c), but since some people did not like this API, the ctljack API was designed and implemented for PA to use, which it does (since PulseAudio 2.0 - the input device API implementation in PulseAudio was never merged upstream).

Quoted 4 lines

What I'd like to see happening is that we merge ctljack into jack (since only HDA is going to be affected by that change it seems like the right direction to make the merge) and also add extcon support, I have looked at the extcon support.

I don't know either why we have two different interfaces for jack detection (not counting extcon) for driver writers. I think we should merge jack and ctljack, as long as that does not mean you break anything I'm relying on in ctljack.

Maybe something for discussion at Plumbers?

Quoted 2 lines

Short term for drivers used on embedded systems I'd have to recommend extcon rather than anything ALSA-specific.

I thought that when Takashi did the new ctljack interface, that would effectively deprecate the old input device interface, and that ctljack was the new recommendation.

Quoted 11 lines · David Henningsson

Quoted 2 lines · Mark Brown

- sound/core/ctljack.c which was added later and provides separate in-kernel and userspace APIs and is currently only used by HDA.

This is wrong. PulseAudio uses the new ctljack API. I started out with an implementation which used the input device API (sound/core/jack.c), but since some people did not like this API, the ctljack API was designed and implemented for PA to use, which it does (since PulseAudio 2.0 - the input device API implementation in PulseAudio was never merged upstream).

...and the only thing using this API to generate events is HDA which is what I was talking about here given that this is a question from a driver point of view.

Quoted 11 lines

Quoted 4 lines

What I'd like to see happening is that we merge ctljack into jack (since only HDA is going to be affected by that change it seems like the right direction to make the merge) and also add extcon support, I have looked at the extcon support.

I don't know either why we have two different interfaces for jack detection (not counting extcon) for driver writers. I think we should merge jack and ctljack, as long as that does not mean you break anything I'm relying on in ctljack.

Maybe something for discussion at Plumbers?

We discussed this last time we all met and came to the above conclusion :/

Quoted 2 lines

Quoted 2 lines

Short term for drivers used on embedded systems I'd have to recommend extcon rather than anything ALSA-specific.

Quoted 3 lines

I thought that when Takashi did the new ctljack interface, that would effectively deprecate the old input device interface, and that ctljack was the new recommendation.

Android currently uses the out of tree version of the extcon ABI (like I say it's where the code originated), and some OSs do use the input ABI also but realistically if you've got to pick one Android is where the market is at.

To my knowledge no embedded OS is using the control jacks, it's possible that there's something using them as a function of using PulseAudio but the ones I'm famililar with currently ignore PulseAudio routing and just use the mixing facilities. Given the dependency on alsa-lib it'd be an inconvenient ABI to adopt for most of them.

So, it seems that the way to go is extcon. I guess that ALSA will use extcon just like today snd_jack uses the input driver. is this correct? Is there any chance that ctrljack will propagate the events through extcon? Is there any early implementation that I could look at? I am asking to know how feasible is to use ctljack today and be compatible with extcon in the future.

Quoted 8 lines · Takashi Iwai

Quoted 2 lines · Ricardo Neri

I was wondering about how much sense does it make to you guys use a snd_soc_jack in this case?

HD-audio already uses the generic jack event for the HDMI/DP connection change notification as well, so I think it would make sense in general.

The whole problem here is that we don't *have* a generic jack interface. We've got:

- sound/core/jack.c which was written to be a generic API and is used by everything that does jack support currently.

- sound/core/ctljack.c which was added later and provides separate in-kernel and userspace APIs and is currently only used by HDA.

- extcon which does have a good reason to be a separate API since that it's not audio specific (and is likely to be picked up by Android as the code was originally taken from there); it's currently not supported by the frameworks in ALSA. I'd suggest Pulse should be using it too.

This is a complete shambles for both driver authors and userspace, the ABI varies randomly with drivers and in theory driver authors have to implement everything three times which is just nuts.

What I'd like to see happening is that we merge ctljack into jack (since only HDA is going to be affected by that change it seems like the right direction to make the merge) and also add extcon support, I have looked at the extcon support.

Short term for drivers used on embedded systems I'd have to recommend extcon rather than anything ALSA-specific.

It's been in shambles for an absurdly long time. Would be good to actually try to tackle it again at Plumbers or sth.

Hello!

I have been working on prototypes for the ASoC OMAP HDMI audio driver to propagate events from the HDMI output (e.g., display getting enabled/disabled/suspended). This for the users of the driver to react to such events. For instance, if the display is disabled or disconected, audio could be stopped, rerouted or whatever other decision the user makes. This is needed because, if, for instance, the HDMI IP goes off, audio will stall and the audio users will only see a "playback write error (DMA or IRQ trouble?)"

In my prototypes I have used snd_soc_jack for this purpose and I have some questions:

*I see snd_soc_jack is used mostly for headsets and microphones with actual external mechanical connections. Strictly, in my case I propagate events originated by the OMAP display driver (changes in the power state), and not from external events. Some of these events are generated from an actual HDMI cable connection/disconnection, though.

*Maybe the event should be propagated by omapdss/omapdrm/drm and the entity in charge of the audio policy should listen those events instead.

*I do see SND_JACK_VIDEOOUT and SND_JACK_AVOUT types so maybe it is feasible for an audio driver to report events from an AV output.

I was wondering about how much sense does it make to you guys use a snd_soc_jack in this case?

Quoted 23 lines · Ricardo Neri

Hello!

I have been working on prototypes for the ASoC OMAP HDMI audio driver to propagate events from the HDMI output (e.g., display getting enabled/disabled/suspended). This for the users of the driver to react to such events. For instance, if the display is disabled or disconected, audio could be stopped, rerouted or whatever other decision the user makes. This is needed because, if, for instance, the HDMI IP goes off, audio will stall and the audio users will only see a "playback write error (DMA or IRQ trouble?)"

In my prototypes I have used snd_soc_jack for this purpose and I have some questions:

*I see snd_soc_jack is used mostly for headsets and microphones with actual external mechanical connections. Strictly, in my case I propagate events originated by the OMAP display driver (changes in the power state), and not from external events. Some of these events are generated from an actual HDMI cable connection/disconnection, though.

*Maybe the event should be propagated by omapdss/omapdrm/drm and the entity in charge of the audio policy should listen those events instead.

Quoted 5 lines

*I do see SND_JACK_VIDEOOUT and SND_JACK_AVOUT types so maybe it is feasible for an audio driver to report events from an AV output.

I was wondering about how much sense does it make to you guys use a snd_soc_jack in this case?

How does DRM handle audio? I made a quick grep, but I see the drm drivers only enabling the audio in the HW, nothing else.

From a quick look, it seems most of what the existing drm drivers are

If there's a common generic way to handle this, we should obviously use that. But if we need to choose between doing something custom or doing it in omapdrm driver, I think we should go for drm the only solution and forget about audio with omapfb.

Tomi

Does ASoC support 'hotplug' of audio devices? If so, maybe it makes some sense to have some support in drm core. At least all the edid parsing stuff to determine if the display supports audio should be generic and not driver specific.

Quoted 27 lines · Tomi Valkeinen

Quoted 26 lines · Ricardo Neri

Hello!

I have been working on prototypes for the ASoC OMAP HDMI audio driver to propagate events from the HDMI output (e.g., display getting enabled/disabled/suspended). This for the users of the driver to react to such events. For instance, if the display is disabled or disconected, audio could be stopped, rerouted or whatever other decision the user makes. This is needed because, if, for instance, the HDMI IP goes off, audio will stall and the audio users will only see a "playback write error (DMA or IRQ trouble?)"

In my prototypes I have used snd_soc_jack for this purpose and I have some questions:

*I see snd_soc_jack is used mostly for headsets and microphones with actual external mechanical connections. Strictly, in my case I propagate events originated by the OMAP display driver (changes in the power state), and not from external events. Some of these events are generated from an actual HDMI cable connection/disconnection, though.

*Maybe the event should be propagated by omapdss/omapdrm/drm and the entity in charge of the audio policy should listen those events instead.

*I do see SND_JACK_VIDEOOUT and SND_JACK_AVOUT types so maybe it is feasible for an audio driver to report events from an AV output.

Quoted 5 lines

Quoted 2 lines

I was wondering about how much sense does it make to you guys use a snd_soc_jack in this case?

How does DRM handle audio? I made a quick grep, but I see the drm drivers only enabling the audio in the HW, nothing else.

I confess to not knowing too much about audio/alsa, but what does audio driver need from hdmi? Just hotplug events?

From a quick look, it seems most of what the existing drm drivers are
doing is detecting if the display supports audio, and then turning on/off the hw.. (and some infoframe stuff in some cases).

Does ASoC support 'hotplug' of audio devices? If so, maybe it makes some sense to have some support in drm core. At least all the edid parsing stuff to determine if the display supports audio should be generic and not driver specific.

BR,
-R

Quoted 7 lines

If there's a common generic way to handle this, we should obviously use that. But if we need to choose between doing something custom or doing it in omapdrm driver, I think we should go for drm the only solution and forget about audio with omapfb.

Tomi

Quoted 28 lines · Rob

Quoted 27 lines · Tomi Valkeinen

Quoted 26 lines · Ricardo Neri

Hello!

I have been working on prototypes for the ASoC OMAP HDMI audio driver to propagate events from the HDMI output (e.g., display getting enabled/disabled/suspended). This for the users of the driver to react to such events. For instance, if the display is disabled or disconected, audio could be stopped, rerouted or whatever other decision the user makes. This is needed because, if, for instance, the HDMI IP goes off, audio will stall and the audio users will only see a "playback write error (DMA or IRQ trouble?)"

In my prototypes I have used snd_soc_jack for this purpose and I have some questions:

*I see snd_soc_jack is used mostly for headsets and microphones with actual external mechanical connections. Strictly, in my case I propagate events originated by the OMAP display driver (changes in the power state), and not from external events. Some of these events are generated from an actual HDMI cable connection/disconnection, though.

*Maybe the event should be propagated by omapdss/omapdrm/drm and the entity in charge of the audio policy should listen those events instead.

*I do see SND_JACK_VIDEOOUT and SND_JACK_AVOUT types so maybe it is feasible for an audio driver to report events from an AV output.

Quoted 8 lines

Quoted 5 lines

Quoted 2 lines

I was wondering about how much sense does it make to you guys use a snd_soc_jack in this case?

How does DRM handle audio? I made a quick grep, but I see the drm drivers only enabling the audio in the HW, nothing else.

I confess to not knowing too much about audio/alsa, but what does audio driver need from hdmi? Just hotplug events?

At least for the case of the ASoC HDMI audio driver (but hopefully for other audio drivers as well), it needs to detect whether an HDMI output is available, if the display's current configuration supports audio (e.g., a 1080p display configured as VGA should be reported as not supporting audio). It also needs interfaces to configure/prepare/start/stop audio. Also, of course, needs to know if the display is off/on/suspended and when transitions occur. For OMAP, omapdss provide an interface for this functionality for ALSA or any other interested user.

Quoted 4 lines

From a quick look, it seems most of what the existing drm drivers are
doing is detecting if the display supports audio, and then turning on/off the hw.. (and some infoframe stuff in some cases).

Yes, it seems to me that every driver makes its own audio implementation, mainly focused on configuration. I could not find any audio common interface so that users like ALSA can take advantage of.

Also, I could not see any ALSA driver using functionality provided by a drm driver.

Maybe the lack of audio support in drm is because the audio users should not talk to drm directly but to a lower level component (omapdrm, omapdss?). However, today there exists video technology supports audio as well, such as DisplayPort or HDMI. Could it make more sense now to provide audio support?

At Tue, 21 Aug 2012 19:58:02 -0500,

Quoted 30 lines · Ricardo Neri

Quoted 26 lines · Rob

Quoted 25 lines · Tomi Valkeinen

Quoted 24 lines · Ricardo Neri

Hello!

I have been working on prototypes for the ASoC OMAP HDMI audio driver to propagate events from the HDMI output (e.g., display getting enabled/disabled/suspended). This for the users of the driver to react to such events. For instance, if the display is disabled or disconected, audio could be stopped, rerouted or whatever other decision the user makes. This is needed because, if, for instance, the HDMI IP goes off, audio will stall and the audio users will only see a "playback write error (DMA or IRQ trouble?)"

In my prototypes I have used snd_soc_jack for this purpose and I have some questions:

*I see snd_soc_jack is used mostly for headsets and microphones with actual external mechanical connections. Strictly, in my case I propagate events originated by the OMAP display driver (changes in the power state), and not from external events. Some of these events are generated from an actual HDMI cable connection/disconnection, though.

*Maybe the event should be propagated by omapdss/omapdrm/drm and the entity in charge of the audio policy should listen those events instead.

*I do see SND_JACK_VIDEOOUT and SND_JACK_AVOUT types so maybe it ie · · Il y a 12 minutes ·

Quoted 32 lines

Quoted 10 lines

Quoted 7 lines

Quoted 4 lines

feasible for an audio driver to report events from an AV output.

I was wondering about how much sense does it make to you guys use a snd_soc_jack in this case?

How does DRM handle audio? I made a quick grep, but I see the drm drivers only enabling the audio in the HW, nothing else.

I confess to not knowing too much about audio/alsa, but what does audio driver need from hdmi? Just hotplug events?

At least for the case of the ASoC HDMI audio driver (but hopefully for other audio drivers as well), it needs to detect whether an HDMI output is available, if the display's current configuration supports audio (e.g., a 1080p display configured as VGA should be reported as not supporting audio). It also needs interfaces to configure/prepare/start/stop audio. Also, of course, needs to know if the display is off/on/suspended and when transitions occur. For OMAP, omapdss provide an interface for this functionality for ALSA or any other interested user.

Quoted 4 lines

From a quick look, it seems most of what the existing drm drivers are
doing is detecting if the display supports audio, and then turning on/off the hw.. (and some infoframe stuff in some cases).

Yes, it seems to me that every driver makes its own audio implementation, mainly focused on configuration. I could not find any audio common interface so that users like ALSA can take advantage of.

Also, I could not see any ALSA driver using functionality provided by a drm driver.

Quoted 5 lines

Maybe the lack of audio support in drm is because the audio users should not talk to drm directly but to a lower level component (omapdrm, omapdss?). However, today there exists video technology supports audio as well, such as DisplayPort or HDMI. Could it make more sense now to provide audio support?

The reason is that the audio and video handling are already separated in the hardware level, at least, for desktop graphics.

The audio infoframe is passed via ELD to the audio controller part upon plug/unplugging via HD-audio unsolicited event, and the audio driver sets up the stuff according to the given ELD. Thus no extra interface between drm and ALSA was required in the kernel API level, so far.

Takashi

Quoted 2 lines · Takashi Iwai

At Tue, 21 Aug 2012 19:58:02 -0500,
Ricardo Neri wrote:

Quoted 8 lines

Quoted 5 lines

Maybe the lack of audio support in drm is because the audio users should not talk to drm directly but to a lower level component (omapdrm, omapdss?). However, today there exists video technology supports audio as well, such as DisplayPort or HDMI. Could it make more sense now to provide audio support?

The reason is that the audio and video handling are already separated in the hardware level, at least, for desktop graphics.

Separated in what sense? Do they have separate register banks in

independent power domains?

Can audio an video work with complete
independence. What happens if someone decides to power off video. Is the audio able to continue if required?

Quoted 5 lines

The audio infoframe is passed via ELD to the audio controller part upon plug/unplugging via HD-audio unsolicited event, and the audio driver sets up the stuff according to the given ELD. Thus no extra interface between drm and ALSA was required in the kernel API level, so far.

I see that the unsolicited event is used only to parse the EDID, correct? It also notifies about the jack status. Hence, if there the cable is disconnected the application will know and act accordingly. Is this understanding correct?

Hi Takashi,

Quoted 32 lines · Takashi Iwai

At Tue, 21 Aug 2012 19:58:02 -0500,

Quoted 30 lines · Ricardo Neri

Quoted 26 lines · Rob

Quoted 25 lines · Tomi Valkeinen

Quoted 24 lines · Ricardo Neri

Hello!

I have been working on prototypes for the ASoC OMAP HDMI audio driver to propagate events from the HDMI output (e.g., display getting enabled/disabled/suspended). This for the users of the driver to react to such events. For instance, if the display is disabled or disconected, audio could be stopped, rerouted or whatever other decision the user makes. This is needed because, if, for instance, the HDMI IP goes off, audio will stall and the audio users will only see a "playback write error (DMA or IRQ trouble?)"

In my prototypes I have used snd_soc_jack for this purpose and I have some questions:

*I see snd_soc_jack is used mostly for headsets and microphones with actual external mechanical connections. Strictly, in my case I propagate events originated by the OMAP display driver (changes in the power state), and not from external events. Some of these events are generated from an actual HDMI cable connection/disconnection, though.

*Maybe the event should be propagated by omapdss/omapdrm/drm and the entity in charge of the audio policy should listen those events instead.

*I do see SND_JACK_VIDEOOUT and SND_JACK_AVOUT types so maybe it ie · · Il y a 12 minutes ·

s

Quoted 38 lines

Quoted 37 lines

Quoted 10 lines

Quoted 7 lines

Quoted 4 lines

feasible for an audio driver to report events from an AV output.

I was wondering about how much sense does it make to you guys use a snd_soc_jack in this case?

How does DRM handle audio? I made a quick grep, but I see the drm drivers only enabling the audio in the HW, nothing else.

I confess to not knowing too much about audio/alsa, but what does audio driver need from hdmi? Just hotplug events?

At least for the case of the ASoC HDMI audio driver (but hopefully for other audio drivers as well), it needs to detect whether an HDMI output is available, if the display's current configuration supports audio (e.g., a 1080p display configured as VGA should be reported as not supporting audio). It also needs interfaces to configure/prepare/start/stop audio. Also, of course, needs to know if the display is off/on/suspended and when transitions occur. For OMAP, omapdss provide an interface for this functionality for ALSA or any other interested user.

Quoted 4 lines

From a quick look, it seems most of what the existing drm drivers are
doing is detecting if the display supports audio, and then turning on/off the hw.. (and some infoframe stuff in some cases).

Yes, it seems to me that every driver makes its own audio implementation, mainly focused on configuration. I could not find any audio common interface so that users like ALSA can take advantage of.

Also, I could not see any ALSA driver using functionality provided by a drm driver.

Maybe the lack of audio support in drm is because the audio users should not talk to drm directly but to a lower level component (omapdrm, omapdss?). However, today there exists video technology supports audio as well, such as DisplayPort or HDMI. Could it make more sense now to provide audio support?

Quoted 2 lines

The reason is that the audio and video handling are already separated in the hardware level, at least, for desktop graphics.

Separated in what sense? Do they have separate register banks in independent power domains? Can audio an video work with complete independence. What happens if someone decides to power off video. Is the audio able to continue if required?

Quoted 5 lines

The audio infoframe is passed via ELD to the audio controller part upon plug/unplugging via HD-audio unsolicited event, and the audio driver sets up the stuff according to the given ELD. Thus no extra interface between drm and ALSA was required in the kernel API level, so far.

I see that the unsolicited event is used only to parse the EDID, correct? It also notifies about the jack status. Hence, if there the cable is disconnected the application will know and act accordingly. Is this understanding correct?

Also, I see that hda_pcm_ops does not have a trigger or start/stop functions, how does it know when to start/stop audio. Maybe with azx_pcm_trigger?