Skip to content

RFC ALSA/ASoC: USB Audio Offload

Cezary Rojewski linux-sound 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 RFC Latest15 patches 12 repliesIn review

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/15ALSA: usb: Move media-filters to the media codesound/usb/card.c sound/usb/media.c+9 -100 replies
2/15ALSA: usb: Drop private_free() usage for card and pcmssound/usb/card.c sound/usb/stream.c sound/usb/stream.h+31 -340 replies
3/15ALSA: usb: Relocate the usbaudio header fileinclude/sound/usb.h sound/usb/caiaq/device.h sound/usb/card.c +27 more+44 -360 replies
4/15ALSA: usb: Implement two-stage quirk applying mechanismsound/usb/quirks.c sound/usb/quirks.h+179 -420 replies
5/15ALSA: usb: Implement two-stage stream creation mechanisminclude/sound/usb.h sound/usb/card.c sound/usb/stream.c +1 more+319 -00 replies
6/15ALSA: usb: Implement two-stage chip probing mechanisminclude/linux/usb.h include/linux/usb/ch9.h include/sound/usb.h +1 more+316 -10 replies
7/15ALSA: usb: Switch to the two-stage chip probingsound/usb/card.c+45 -2360 replies
8/15ALSA: usb: Switch to the two-stage stream creationsound/usb/card.c sound/usb/stream.c+29 -1870 replies
9/15ALSA: usb: Switch to the two-stage quirk applyingsound/usb/quirks.c+2 -400 replies
10/15ALSA: usb: Export PCM operationsinclude/sound/usb.h sound/usb/pcm.c+120 -480 replies
11/15ALSA: usb: Export usb_interface driver operationsinclude/sound/usb.h sound/usb/card.c+85 -110 replies
12/15ALSA: usb: Export card-naming procedureinclude/sound/usb.h sound/usb/card.c+39 -400 replies
13/15ALSA: usb: Add getters to obtain endpoint informationinclude/sound/usb.h sound/usb/card.h sound/usb/stream.c+57 -20 replies
14/15ASoC: codecs: Add USB-Audio driverinclude/sound/usb_offload.h sound/soc/codecs/Kconfig sound/soc/codecs/Makefile +1 more+495 -00 replies
15/15ASoC: Intel: avs: Add USB machine boardsound/soc/intel/avs/boards/Kconfig sound/soc/intel/avs/boards/Makefile sound/soc/intel/avs/boards/usb.c+125 -00 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.

Read the v1 thread

Cover letter [RFC 00/15] ALSA/ASoC: USB Audio Offload

Note: this series is based on Mark's broonie/for-next. The xHCI dependency is missing so it won't compile. Goal of this RFC is discussing the direction of sound/usb changes.

Note #2: Why exclude xHCI? Current form of xhci-sideband.c [1] does not fare well with Intel's hardware design for USB Audio Offload feature. The production shape for usb/xHCI subjects is being discussed with Mathias. Once we're ready, I'll share the rest.

Note #3: this series does _NOT_ aim to block QCOM's equivalent series [2]. The team does acknowledge that we came the "table" late. At the same time, we're prepared to help QCOM switch to the presented sound/usb approach if that would benefit the framework and its users as a whole. Make it part of this very series if need be.

Apart from changes to sound/usb, the patchset contains exemplary ASoC-based, offload-aware USB driver and a small sound card on the avs-driver side to show how card/dai_link initialization looks like.

In short, USB Audio Offload functionality lowers CPU usage when streaming PCM over a USB Audio-Class device. It has been first introduced in xHCI 1.2 release and is described in section 7.9 [3]. Once hardware is prepared with hw_params(), all the USB endpoints operations e.g.: start, stop, submitting URBs, are performed internally, by the hardware and AudioDSP firmware. Software driver shall not intervene.

Startup flow from:

	usb_pcm_open()
	usb_pcm_hw_params()
		snd_usb_endpoint_open()
	usb_pcm_prepare()
		usb_set_interface()
		snd_usb_endpoint_start()
	usb_pcm_trigger(cmd: START/STOP etc.)

reduced to:

	usb_pcm_open()
	usb_pcm_hw_params()
		snd_usb_endpoint_open()
	usb_pcm_prepare()
		usb_set_interface()

Handlers such as ack(), sync_stop(), pointer(), delay() are not used here too. The AudioDSP driver will handle pointer(), rest is firmware/hardware responsibility.

There's a hefty number of limitations, most importantly:

1) typically 2 USB devices tops, rest go the classic (non-offload) path
2) AUDIOSTREAMING interfaces only, MIDI not. While not an interface type, Media (sound/usb/media) unsupported either
3) simple PCM, UAC_FORMAT_TYPE_I_PCM only

Current patchset shows this in form of 'udev->audsb_capable' field. True if xHCI sideband reasource has been assigned to the device. The filtering is code is not part of the patchset.

Important to highlight, 1) means both, offload-aware and offload-unaware drivers could be utilized simultaneously on the system in runtime. Opportunistically few devices would be controlled by the offload-aware driver, whereas everything else by the offload-unaware one. This differs from existing HDAudio Controller driver situation where either classic, snd_hda_intel driver takes complete control -or- the offload-aware snd_soc_avs driver.
In short, once all Audio Sideband resources are depleted, classic sound/usb/card.c driver manages whatever comes next:

snd_usb_audio (offload un-aware, sound/usb/card.c) snd_soc_usb_codec (offload aware, sound/soc/codecs/usb.c)

The design goals:
- make ASoC first class citizen of sound/usb
- re-use code found in sound/usb, mimic HDAudio integration in ASoC:
small sound/soc/codecs/hda.c driver leveraging power of entire sound/pci/hda/
- no shared control over a USB device, either snd_usb_audio or its ASoC equivalent takes control of the device

To do that, major tasks are identified:

a) On ASoC side 'struct snd_card' is part of 'struct snd_soc_card' and is managed by the framework. Similar situation with 'struct snd_pcm' and rtd->pcm. To keep the teardown path sane, drop card->private_free() and pcm->private_free() usage.

b) To initialize ASoC components/DAI properly, PCM capabilities should be known up-front. To do that, existing USB card probe() has to be split. From one-stage to two-stage process:

- look ahead and parse usb_interface descriptors for PCM endpoints but do not create any PCMs (sound devices) yet
- create all PCMs based on obtained ->pcm_list and follow with MIDI/mixers/media

Such approach allows to feed DAIs proper data even when a valid sound-card pointer is not yet present - the initialization occurs before snd_soc_bind_card() is called.

Point a) is scaled for all three "domains" of sound/usb: chip, stream and quirks. That's why there total of 6 commits doing that job. First implement, then switch. Everything that follows is, in my opinion, self-explanatory. No need to repeat commit messages.

[1]: https://lore.kernel.org/all/[email protected]/
[2]: https://lore.kernel.org/all/[email protected]/
[3]: https://www.intel.com/content/dam/www/public/us/en/documents/technical-specifications/extensible-host-controler-interface-usb-xhci.pdf

Cezary Rojewski (15):
ALSA: usb: Move media-filters to the media code
ALSA: usb: Drop private_free() usage for card and pcms
ALSA: usb: Relocate the usbaudio header file
ALSA: usb: Implement two-stage quirk applying mechanism
ALSA: usb: Implement two-stage stream creation mechanism
ALSA: usb: Implement two-stage chip probing mechanism
ALSA: usb: Switch to the two-stage chip probing
ALSA: usb: Switch to the two-stage stream creation
ALSA: usb: Switch to the two-stage quirk applying
ALSA: usb: Export PCM operations
ALSA: usb: Export usb_interface driver operations
ALSA: usb: Export card-naming procedure
ALSA: usb: Add getters to obtain endpoint information
ASoC: codecs: Add USB-Audio driver
ASoC: Intel: avs: Add USB machine board

 include/linux/usb.h                         |  15 +
 include/linux/usb/ch9.h                     |  11 +
 sound/usb/usbaudio.h => include/sound/usb.h |  53 +-
 include/sound/usb_offload.h                 |  46 +
 sound/soc/codecs/Kconfig                    |   6 +
 sound/soc/codecs/Makefile                   |   2 +
 sound/soc/codecs/usb.c                      | 441 +++++++++
 sound/soc/intel/avs/boards/Kconfig          |   8 +
 sound/soc/intel/avs/boards/Makefile         |   2 +
 sound/soc/intel/avs/boards/usb.c            | 115 +++
 sound/usb/caiaq/device.h                    |   2 +-
 sound/usb/card.c                            | 946 +++++++++++++-------
 sound/usb/card.h                            |   4 +-
 sound/usb/clock.c                           |   2 +-
 sound/usb/endpoint.c                        |   2 +-
 sound/usb/format.c                          |   2 +-
 sound/usb/helper.c                          |   2 +-
 sound/usb/implicit.c                        |   2 +-
 sound/usb/media.c                           |   8 +-
 sound/usb/midi.c                            |   2 +-
 sound/usb/midi.h                            |   2 +
 sound/usb/midi2.c                           |   2 +-
 sound/usb/midi2.h                           |   1 +
 sound/usb/misc/ua101.c                      |   2 +-
 sound/usb/mixer.c                           |   2 +-
 sound/usb/mixer_quirks.c                    |   2 +-
 sound/usb/mixer_s1810c.c                    |   2 +-
 sound/usb/mixer_scarlett.c                  |   2 +-
 sound/usb/mixer_scarlett2.c                 |   2 +-
 sound/usb/mixer_us16x08.c                   |   2 +-
 sound/usb/pcm.c                             | 161 +++-
 sound/usb/power.c                           |   2 +-
 sound/usb/proc.c                            |   2 +-
 sound/usb/quirks.c                          | 203 +++--
 sound/usb/quirks.h                          |   6 +
 sound/usb/stream.c                          | 217 +++--
 sound/usb/stream.h                          |   2 +
 sound/usb/usx2y/us122l.c                    |   2 +-
 sound/usb/usx2y/usX2Yhwdep.c                |   1 +
 sound/usb/usx2y/usbusx2y.h                  |   2 +-
 sound/usb/validate.c                        |   2 +-
 41 files changed, 1749 insertions(+), 541 deletions(-)
 rename sound/usb/usbaudio.h => include/sound/usb.h (82%)
 create mode 100644 include/sound/usb_offload.h
 create mode 100644 sound/soc/codecs/usb.c
 create mode 100644 sound/soc/intel/avs/boards/usb.c
Signature · 1 line
2.25.1