Media Device Allocator API

From: Shuah Khan <[email protected]>

Media Device Allocator API to allows multiple drivers share a media device. This API solves a very common use-case for media devices where one physical device (an USB stick) provides both audio and video. When such media device exposes a standard USB Audio class, a proprietary Video class, two or more independent drivers will share a single physical USB bridge. In such cases, it is necessary to coordinate access to the shared resource.

Using this API, drivers can allocate a media device with the shared struct device as the key. Once the media device is allocated by a driver, other drivers can get a reference to it. The media device is released when all the references are released.

Change the ALSA driver to use the Media Controller API to share media resources with DVB, and V4L2 drivers on a AU0828 media device.

The Media Controller specific initialization is done after sound card is registered. ALSA creates Media interface and entity function graph nodes for Control, Mixer, PCM Playback, and PCM Capture devices.

snd_usb_hw_params() will call Media Controller enable source handler interface to request the media resource. If resource request is granted, it will release it from snd_usb_hw_free(). If resource is busy, -EBUSY is returned.

Media specific cleanup is done in usb_audio_disconnect().

Signed-off-by Shuah Khan <[email protected]>

Quoted 22 lines · On Tue, 18 Dec 2018 18:59:39 +0100, [email protected] wrote:

From: Shuah Khan <[email protected]>

Media Device Allocator API to allows multiple drivers share a media device. This API solves a very common use-case for media devices where one physical device (an USB stick) provides both audio and video. When such media device exposes a standard USB Audio class, a proprietary Video class, two or more independent drivers will share a single physical USB bridge. In such cases, it is necessary to coordinate access to the shared resource.

Using this API, drivers can allocate a media device with the shared struct device as the key. Once the media device is allocated by a driver, other drivers can get a reference to it. The media device is released when all the references are released.

Change the ALSA driver to use the Media Controller API to share media resources with DVB, and V4L2 drivers on a AU0828 media device.

The Media Controller specific initialization is done after sound card is registered. ALSA creates Media interface and entity function graph nodes for Control, Mixer, PCM Playback, and PCM Capture devices.

Quoted 8 lines

snd_usb_hw_params() will call Media Controller enable source handler interface to request the media resource. If resource request is granted, it will release it from snd_usb_hw_free(). If resource is busy, -EBUSY is returned.

Media specific cleanup is done in usb_audio_disconnect().

Signed-off-by Shuah Khan <[email protected]>

Feel free to take my ack regarding the sound stuff:

Reviewed-by Takashi Iwai <[email protected]>

Thanks!

Takashi

Quoted 37 lines · Takashi Iwai

Quoted 22 lines · On Tue, 18 Dec 2018 18:59:39 +0100, [email protected] wrote:

From: Shuah Khan <[email protected]>

Media Device Allocator API to allows multiple drivers share a media device. This API solves a very common use-case for media devices where one physical device (an USB stick) provides both audio and video. When such media device exposes a standard USB Audio class, a proprietary Video class, two or more independent drivers will share a single physical USB bridge. In such cases, it is necessary to coordinate access to the shared resource.

Using this API, drivers can allocate a media device with the shared struct device as the key. Once the media device is allocated by a driver, other drivers can get a reference to it. The media device is released when all the references are released.

Change the ALSA driver to use the Media Controller API to share media resources with DVB, and V4L2 drivers on a AU0828 media device.

The Media Controller specific initialization is done after sound card is registered. ALSA creates Media interface and entity function graph nodes for Control, Mixer, PCM Playback, and PCM Capture devices.

Quoted 8 lines

snd_usb_hw_params() will call Media Controller enable source handler interface to request the media resource. If resource request is granted, it will release it from snd_usb_hw_free(). If resource is busy, -EBUSY is returned.

Media specific cleanup is done in usb_audio_disconnect().

Signed-off-by Shuah Khan <[email protected]>

Feel free to take my ack regarding the sound stuff:

Reviewed-by Takashi Iwai <[email protected]>

Quoted 4 lines

Thanks!

Takashi

Hi Mauro,

Any update on this patch series?

Quoted 38 lines · shuah

Quoted 37 lines · Takashi Iwai

Quoted 22 lines · On Tue, 18 Dec 2018 18:59:39 +0100, [email protected] wrote:

From: Shuah Khan <[email protected]>

Media Device Allocator API to allows multiple drivers share a media device. This API solves a very common use-case for media devices where one physical device (an USB stick) provides both audio and video. When such media device exposes a standard USB Audio class, a proprietary Video class, two or more independent drivers will share a single physical USB bridge. In such cases, it is necessary to coordinate access to the shared resource.

Using this API, drivers can allocate a media device with the shared struct device as the key. Once the media device is allocated by a driver, other drivers can get a reference to it. The media device is released when all the references are released.

Change the ALSA driver to use the Media Controller API to share media resources with DVB, and V4L2 drivers on a AU0828 media device.

The Media Controller specific initialization is done after sound card is registered. ALSA creates Media interface and entity function graph nodes for Control, Mixer, PCM Playback, and PCM Capture devices.

Quoted 8 lines

snd_usb_hw_params() will call Media Controller enable source handler interface to request the media resource. If resource request is granted, it will release it from snd_usb_hw_free(). If resource is busy, -EBUSY is returned.

Media specific cleanup is done in usb_audio_disconnect().

Signed-off-by Shuah Khan <[email protected]>

Feel free to take my ack regarding the sound stuff:

Reviewed-by Takashi Iwai <[email protected]>

Quoted 8 lines

Quoted 4 lines

Thanks!

Takashi

Hi Mauro,

Any update on this patch series?

I'm planning to process this series for 5.1. Haven't gotten around to it yet, but I expect to do this next week.

Still going through all the pending patches after the Christmas period :-)

Regards,

Hans

From: Shuah Khan <[email protected]>

Media Device Allocator API to allows multiple drivers share a media device. This API solves a very common use-case for media devices where one physical device (an USB stick) provides both audio and video. When such media device exposes a standard USB Audio class, a proprietary Video class, two or more independent drivers will share a single physical USB bridge. In such cases, it is necessary to coordinate access to the shared resource.

Using this API, drivers can allocate a media device with the shared struct device as the key. Once the media device is allocated by a driver, other drivers can get a reference to it. The media device is released when all the references are released.

Change the ALSA driver to use the Media Controller API to share media resources with DVB, and V4L2 drivers on a AU0828 media device.

The Media Controller specific initialization is done after sound card is registered. ALSA creates Media interface and entity function graph nodes for Control, Mixer, PCM Playback, and PCM Capture devices.

snd_usb_hw_params() will call Media Controller enable source handler interface to request the media resource. If resource request is granted, it will release it from snd_usb_hw_free(). If resource is busy, -EBUSY is returned.

Media specific cleanup is done in usb_audio_disconnect().

Signed-off-by Shuah Khan <[email protected]>

---
 sound/usb/Kconfig        |   4 +
 sound/usb/Makefile       |   2 +
 sound/usb/card.c         |  14 ++
 sound/usb/card.h         |   3 +
 sound/usb/media.c        | 321 +++++++++++++++++++++++++++++++++++++++
 sound/usb/media.h        |  74 +++++++++
 sound/usb/mixer.h        |   3 +
 sound/usb/pcm.c          |  29 +++-
 sound/usb/quirks-table.h |   1 +
 sound/usb/stream.c       |   2 +
 sound/usb/usbaudio.h     |   6 +
 11 files changed, 455 insertions(+), 4 deletions(-)
 create mode 100644 sound/usb/media.c
 create mode 100644 sound/usb/media.h

sound/usb/media.c new file +10 -0

+int snd_media_device_create(struct snd_usb_audio *chip,
+			struct usb_interface *iface)
+{
+	struct media_device *mdev;
+	struct usb_device *usbdev = interface_to_usbdev(iface);
+	int ret;
+
+	/* usb-audio driver is probed for each usb interface, and
+	 * there are multiple interfaces per device. Avoid calling
+	 * media_device_usb_allocate() each time usb_audio_probe()

sound/usb/media.c new file +19 -0

+		/* register media_device */
+		ret = media_device_register(mdev);
+		if (ret) {
+			media_device_delete(mdev, KBUILD_MODNAME);
+			dev_err(&usbdev->dev,
+				"Couldn't register media device. Error: %d\n",
+				ret);
+			return ret;
+		}
+	}
+
+	/* save media device - avoid lookups */
+	chip->media_dev = mdev;
+
+snd_mixer_init:
+	/* Create media entities for mixer and control dev */
+	ret = snd_media_mixer_init(chip);
+	if (ret) {
+		dev_err(&usbdev->dev,

Quoted 45 lines · On 12/18/18 6:59 PM, [email protected] wrote:

From: Shuah Khan <[email protected]>

Media Device Allocator API to allows multiple drivers share a media device. This API solves a very common use-case for media devices where one physical device (an USB stick) provides both audio and video. When such media device exposes a standard USB Audio class, a proprietary Video class, two or more independent drivers will share a single physical USB bridge. In such cases, it is necessary to coordinate access to the shared resource.

Using this API, drivers can allocate a media device with the shared struct device as the key. Once the media device is allocated by a driver, other drivers can get a reference to it. The media device is released when all the references are released.

Change the ALSA driver to use the Media Controller API to share media resources with DVB, and V4L2 drivers on a AU0828 media device.

The Media Controller specific initialization is done after sound card is registered. ALSA creates Media interface and entity function graph nodes for Control, Mixer, PCM Playback, and PCM Capture devices.

snd_usb_hw_params() will call Media Controller enable source handler interface to request the media resource. If resource request is granted, it will release it from snd_usb_hw_free(). If resource is busy, -EBUSY is returned.

Media specific cleanup is done in usb_audio_disconnect().

Signed-off-by Shuah Khan <[email protected]>

---
  sound/usb/Kconfig        |   4 +
  sound/usb/Makefile       |   2 +
  sound/usb/card.c         |  14 ++
  sound/usb/card.h         |   3 +
  sound/usb/media.c        | 321 +++++++++++++++++++++++++++++++++++++++
  sound/usb/media.h        |  74 +++++++++
  sound/usb/mixer.h        |   3 +
  sound/usb/pcm.c          |  29 +++-
  sound/usb/quirks-table.h |   1 +
  sound/usb/stream.c       |   2 +
  sound/usb/usbaudio.h     |   6 +
  11 files changed, 455 insertions(+), 4 deletions(-)

create mode 100644 sound/usb/media.c
create mode 100644 sound/usb/media.h

<snip>

Quoted 10 lines

sound/usb/media.c new file +10 -0

+int snd_media_device_create(struct snd_usb_audio *chip,
+			struct usb_interface *iface)
+{
+	struct media_device *mdev;
+	struct usb_device *usbdev = interface_to_usbdev(iface);
+	int ret;
+
+	/* usb-audio driver is probed for each usb interface, and
+	 * there are multiple interfaces per device. Avoid calling
+	 * media_device_usb_allocate() each time usb_audio_probe()

sound/usb/media.c line 273

Quoted 10 lines

sound/usb/media.c new file +10 -0

+	 * is called. Do it only once.
+	 */
+	if (chip->media_dev)
+		goto snd_mixer_init;
+
+	mdev = media_device_usb_allocate(usbdev, KBUILD_MODNAME);
+	if (!mdev)
+		return -ENOMEM;
+
+	if (!media_devnode_is_registered(mdev->devnode)) {

It looks like you missed my comment for v8:

"You should first configure the media device before registering it."

In other words, first create the media entities, and only then do you register the media device. Otherwise it will come up without any alsa entities, which are then added. So an application that immediately opens the media device upon creation will see a topology that is still in flux.

Quoted 75 lines · Hans Verkuil

Quoted 46 lines · On 12/18/18 6:59 PM, [email protected] wrote:

From: Shuah Khan <[email protected]>

Media Device Allocator API to allows multiple drivers share a media device. This API solves a very common use-case for media devices where one physical device (an USB stick) provides both audio and video. When such media device exposes a standard USB Audio class, a proprietary Video class, two or more independent drivers will share a single physical USB bridge. In such cases, it is necessary to coordinate access to the shared resource.

Using this API, drivers can allocate a media device with the shared struct device as the key. Once the media device is allocated by a driver, other drivers can get a reference to it. The media device is released when all the references are released.

Change the ALSA driver to use the Media Controller API to share media resources with DVB, and V4L2 drivers on a AU0828 media device.

The Media Controller specific initialization is done after sound card is registered. ALSA creates Media interface and entity function graph nodes for Control, Mixer, PCM Playback, and PCM Capture devices.

snd_usb_hw_params() will call Media Controller enable source handler interface to request the media resource. If resource request is granted, it will release it from snd_usb_hw_free(). If resource is busy, -EBUSY is
returned.

Media specific cleanup is done in usb_audio_disconnect().

Signed-off-by Shuah Khan <[email protected]>

---
  sound/usb/Kconfig        |   4 +
  sound/usb/Makefile       |   2 +
  sound/usb/card.c         |  14 ++
  sound/usb/card.h         |   3 +

  sound/usb/media.c        | 321 +++++++++++++++++++++++++++++++++++++++
  sound/usb/media.h        |  74 +++++++++
  sound/usb/mixer.h        |   3 +

  sound/usb/pcm.c          |  29 +++-
  sound/usb/quirks-table.h |   1 +
  sound/usb/stream.c       |   2 +
  sound/usb/usbaudio.h     |   6 +

  11 files changed, 455 insertions(+), 4 deletions(-)

  create mode 100644 sound/usb/media.c
  create mode 100644 sound/usb/media.h

<snip>

Quoted 10 lines

sound/usb/media.c new file +10 -0

+int snd_media_device_create(struct snd_usb_audio *chip,
+            struct usb_interface *iface)
+{
+    struct media_device *mdev;
+    struct usb_device *usbdev = interface_to_usbdev(iface);
+    int ret;
+
+    /* usb-audio driver is probed for each usb interface, and
+     * there are multiple interfaces per device. Avoid calling
+     * media_device_usb_allocate() each time usb_audio_probe()

sound/usb/media.c line 273

Quoted 10 lines

sound/usb/media.c new file +10 -0

+     * is called. Do it only once.
+     */
+    if (chip->media_dev)
+        goto snd_mixer_init;
+
+    mdev = media_device_usb_allocate(usbdev, KBUILD_MODNAME);
+    if (!mdev)
+        return -ENOMEM;
+
+    if (!media_devnode_is_registered(mdev->devnode)) {

It looks like you missed my comment for v8:

"You should first configure the media device before registering it."

Quoted 5 lines

In other words, first create the media entities, and only then do you register the media device. Otherwise it will come up without any alsa entities, which are then added. So an application that immediately opens the media device upon creation will see a topology that is still in flux.

Yes. You are right. I saw your comment and thought I got it addressed. I will fix it. I have the logic correct in au0828, but not here.

Quoted 88 lines · shuah

Quoted 86 lines · Hans Verkuil

Quoted 52 lines · On 12/18/18 6:59 PM, [email protected] wrote:

From: Shuah Khan <[email protected]>

Media Device Allocator API to allows multiple drivers share a media device.
This API solves a very common use-case for media devices where one physical
device (an USB stick) provides both audio and video. When such media device
exposes a standard USB Audio class, a proprietary Video class, two or more
independent drivers will share a single physical USB bridge. In such cases,
it is necessary to coordinate access to the shared resource.

Using this API, drivers can allocate a media device with the shared struct
device as the key. Once the media device is allocated by a driver, other drivers can get a reference to it. The media device is released when all the references are released.

Change the ALSA driver to use the Media Controller API to share media resources with DVB, and V4L2 drivers on a AU0828 media device.

The Media Controller specific initialization is done after sound card is registered. ALSA creates Media interface and entity function graph nodes for Control, Mixer, PCM Playback, and PCM Capture devices.

snd_usb_hw_params() will call Media Controller enable source handler interface to request the media resource. If resource request is granted, it will release it from snd_usb_hw_free(). If resource is busy, -EBUSY is
returned.

Media specific cleanup is done in usb_audio_disconnect().

Signed-off-by Shuah Khan <[email protected]>

---
  sound/usb/Kconfig        |   4 +
  sound/usb/Makefile       |   2 +
  sound/usb/card.c         |  14 ++
  sound/usb/card.h         |   3 +
  sound/usb/media.c        | 321 +++++++++++++++++++++++++++++++++++++++
  sound/usb/media.h        |  74 +++++++++
  sound/usb/mixer.h        |   3 +
  sound/usb/pcm.c          |  29 +++-
  sound/usb/quirks-table.h |   1 +
  sound/usb/stream.c       |   2 +
  sound/usb/usbaudio.h     |   6 +
  11 files changed, 455 insertions(+), 4 deletions(-)

create mode 100644 sound/usb/media.c
create mode 100644 sound/usb/media.h

<snip>

Quoted 10 lines

sound/usb/media.c new file +10 -0

+int snd_media_device_create(struct snd_usb_audio *chip,
+            struct usb_interface *iface)
+{
+    struct media_device *mdev;
+    struct usb_device *usbdev = interface_to_usbdev(iface);
+    int ret;
+
+    /* usb-audio driver is probed for each usb interface, and
+     * there are multiple interfaces per device. Avoid calling
+     * media_device_usb_allocate() each time usb_audio_probe()

sound/usb/media.c line 273

Quoted 10 lines

sound/usb/media.c new file +10 -0

+     * is called. Do it only once.
+     */
+    if (chip->media_dev)
+        goto snd_mixer_init;
+
+    mdev = media_device_usb_allocate(usbdev, KBUILD_MODNAME);
+    if (!mdev)
+        return -ENOMEM;
+
+    if (!media_devnode_is_registered(mdev->devnode)) {

It looks like you missed my comment for v8:

"You should first configure the media device before registering it."

In other words, first create the media entities, and only then do you register the media device. Otherwise it will come up without any alsa entities, which are then added. So an application that immediately opens the media device upon creation will see a topology that is still in flux.

Quoted 3 lines

Yes. You are right. I saw your comment and thought I got it addressed. I will fix it. I have the logic correct in au0828, but not here.

One thing to mention here is some ALSA entities get created dynamically during PCM open when stream is initialized. This happens after media device is registered. These get deleted when pcm close happens. There is no way to avoid creating entities after media device register.

snd_media_device_create() could get called after media_device_register() happens, if au0828 registers it first.

I did find a problem in snd_media_device_create() and I will send a v10 with that fix.

thanks,
-- Shuah

Quoted 89 lines · shuah

Quoted 88 lines · shuah

Quoted 86 lines · Hans Verkuil

Quoted 52 lines · On 12/18/18 6:59 PM, [email protected] wrote:

From: Shuah Khan <[email protected]>

Media Device Allocator API to allows multiple drivers share a media device.
This API solves a very common use-case for media devices where one physical
device (an USB stick) provides both audio and video. When such media device
exposes a standard USB Audio class, a proprietary Video class, two or more
independent drivers will share a single physical USB bridge. In such cases,
it is necessary to coordinate access to the shared resource.

Using this API, drivers can allocate a media device with the shared struct
device as the key. Once the media device is allocated by a driver, other drivers can get a reference to it. The media device is released when all the references are released.

Change the ALSA driver to use the Media Controller API to share media resources with DVB, and V4L2 drivers on a AU0828 media device.

The Media Controller specific initialization is done after sound card is registered. ALSA creates Media interface and entity function graph nodes for Control, Mixer, PCM Playback, and PCM Capture devices.

snd_usb_hw_params() will call Media Controller enable source handler interface to request the media resource. If resource request is granted, it will release it from snd_usb_hw_free(). If resource is busy, -EBUSY is
returned.

Media specific cleanup is done in usb_audio_disconnect().

Signed-off-by Shuah Khan <[email protected]>

---
   sound/usb/Kconfig        |   4 +
   sound/usb/Makefile       |   2 +
   sound/usb/card.c         |  14 ++
   sound/usb/card.h         |   3 +
   sound/usb/media.c        | 321 +++++++++++++++++++++++++++++++++++++++
   sound/usb/media.h        |  74 +++++++++
   sound/usb/mixer.h        |   3 +
   sound/usb/pcm.c          |  29 +++-
   sound/usb/quirks-table.h |   1 +
   sound/usb/stream.c       |   2 +
   sound/usb/usbaudio.h     |   6 +
   11 files changed, 455 insertions(+), 4 deletions(-)

create mode 100644 sound/usb/media.c
create mode 100644 sound/usb/media.h

<snip>

Quoted 10 lines

sound/usb/media.c new file +10 -0

+int snd_media_device_create(struct snd_usb_audio *chip,
+            struct usb_interface *iface)
+{
+    struct media_device *mdev;
+    struct usb_device *usbdev = interface_to_usbdev(iface);
+    int ret;
+
+    /* usb-audio driver is probed for each usb interface, and
+     * there are multiple interfaces per device. Avoid calling
+     * media_device_usb_allocate() each time usb_audio_probe()

sound/usb/media.c line 273

Quoted 10 lines

sound/usb/media.c new file +10 -0

+     * is called. Do it only once.
+     */
+    if (chip->media_dev)
+        goto snd_mixer_init;
+
+    mdev = media_device_usb_allocate(usbdev, KBUILD_MODNAME);
+    if (!mdev)
+        return -ENOMEM;
+
+    if (!media_devnode_is_registered(mdev->devnode)) {

It looks like you missed my comment for v8:

"You should first configure the media device before registering it."

In other words, first create the media entities, and only then do you register the media device. Otherwise it will come up without any alsa entities, which are then added. So an application that immediately opens the media device upon creation will see a topology that is still in flux.

Quoted 8 lines

Quoted 3 lines

Yes. You are right. I saw your comment and thought I got it addressed. I will fix it. I have the logic correct in au0828, but not here.

One thing to mention here is some ALSA entities get created dynamically during PCM open when stream is initialized. This happens after media device is registered. These get deleted when pcm close happens. There is no way to avoid creating entities after media device register.

That's OK. We're missing some infrastructure (media events) to inform the application about topology changes.

But when registering the media device for the first time it is good practice to do that after creating all the entities that you can.

From: Shuah Khan <[email protected]>

Media Device Allocator API to allows multiple drivers share a media device. This API solves a very common use-case for media devices where one physical device (an USB stick) provides both audio and video. When such media device exposes a standard USB Audio class, a proprietary Video class, two or more independent drivers will share a single physical USB bridge. In such cases, it is necessary to coordinate access to the shared resource.

Using this API, drivers can allocate a media device with the shared struct device as the key. Once the media device is allocated by a driver, other drivers can get a reference to it. The media device is released when all the references are released.

- Tested sharing resources with kaffeine, vlc, xawtv, tvtime, and arecord. When analog is streaming, digital and audio user-space applications detect that the tuner is busy and exit. When digital is streaming, analog and audio applications detect that the tuner is busy and exit. When arecord is owns the tuner, digital and analog detect that the tuner is busy and exit.
- Tested media device allocator API with bind/unbind testing on snd-usb-audio and au0828 drivers to make sure /dev/mediaX is released only when the last driver is unbound.
- This patch series is tested on 4.20-rc6
- Addressed review comments from Hans on the RFC v8 (rebased on 4.19)
- Updated change log to describe the use-case more clearly.
- No changes to 0001,0002 code since the v7 referenced below.
- 0003 is a new patch to enable ALSA defines that have been disabled for kernel between 4.9 and 4.19.
- Minor merge conflict resolution in 0004.
- Added SPDX to new files.

References:
https://lkml.org/lkml/2018/11/2/169
https://www.mail-archive.com/[email protected]/msg105854.html

Hi Shuah,

Quoted 32 lines · On 12/18/2018 06:59 PM, [email protected] wrote:

From: Shuah Khan <[email protected]>

Media Device Allocator API to allows multiple drivers share a media device. This API solves a very common use-case for media devices where one physical device (an USB stick) provides both audio and video. When such media device exposes a standard USB Audio class, a proprietary Video class, two or more independent drivers will share a single physical USB bridge. In such cases, it is necessary to coordinate access to the shared resource.

Using this API, drivers can allocate a media device with the shared struct device as the key. Once the media device is allocated by a driver, other drivers can get a reference to it. The media device is released when all the references are released.

- Tested sharing resources with kaffeine, vlc, xawtv, tvtime, and arecord. When analog is streaming, digital and audio user-space applications detect that the tuner is busy and exit. When digital is streaming, analog and audio applications detect that the tuner is busy and exit. When arecord is owns the tuner, digital and analog detect that the tuner is busy and exit.
- Tested media device allocator API with bind/unbind testing on snd-usb-audio and au0828 drivers to make sure /dev/mediaX is released only when the last driver is unbound.
- This patch series is tested on 4.20-rc6
- Addressed review comments from Hans on the RFC v8 (rebased on 4.19)
- Updated change log to describe the use-case more clearly.
- No changes to 0001,0002 code since the v7 referenced below.
- 0003 is a new patch to enable ALSA defines that have been disabled for kernel between 4.9 and 4.19.
- Minor merge conflict resolution in 0004.
- Added SPDX to new files.

Quoted 3 lines

References:
https://lkml.org/lkml/2018/11/2/169
https://www.mail-archive.com/[email protected]/msg105854.html

When I connect my au0828 to my laptop with your v9 patch series applied I get these warnings:

[   45.416047] xhci_hcd 0000:39:00.0: xHCI Host Controller
[   45.417882] xhci_hcd 0000:39:00.0: new USB bus registered, assigned bus number 3
[   45.419292] xhci_hcd 0000:39:00.0: hcc params 0x200077c1 hci version 0x110 quirks 0x0000000200009810
[   45.420835] usb usb3: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 5.00
[   45.420893] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[   45.420899] usb usb3: Product: xHCI Host Controller
[   45.420905] usb usb3: Manufacturer: Linux 5.0.0-rc1-zen xhci-hcd
[   45.420911] usb usb3: SerialNumber: 0000:39:00.0
[   45.424290] hub 3-0:1.0: USB hub found
[   45.425274] hub 3-0:1.0: 2 ports detected
[   45.431061] xhci_hcd 0000:39:00.0: xHCI Host Controller
[   45.432436] xhci_hcd 0000:39:00.0: new USB bus registered, assigned bus number 4
[   45.432448] xhci_hcd 0000:39:00.0: Host supports USB 3.1 Enhanced SuperSpeed
[   45.433299] usb usb4: New USB device found, idVendor=1d6b, idProduct=0003, bcdDevice= 5.00
[   45.433354] usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[   45.433360] usb usb4: Product: xHCI Host Controller
[   45.433365] usb usb4: Manufacturer: Linux 5.0.0-rc1-zen xhci-hcd
[   45.433370] usb usb4: SerialNumber: 0000:39:00.0
[   45.436134] hub 4-0:1.0: USB hub found
[   45.436576] hub 4-0:1.0: 2 ports detected
[   45.750940] usb 3-1: new high-speed USB device number 2 using xhci_hcd
[   45.899927] usb 3-1: New USB device found, idVendor=2040, idProduct=721e, bcdDevice= 0.05
[   45.899949] usb 3-1: New USB device strings: Mfr=1, Product=2, SerialNumber=10
[   45.899960] usb 3-1: Product: WinTV Aero-A
[   45.899970] usb 3-1: Manufacturer: Hauppauge
[   45.899979] usb 3-1: SerialNumber: 4033622430
[   46.053476] au0828: au0828 driver loaded
[   46.053726] WARNING: CPU: 1 PID: 1824 at kernel/module.c:262 module_assert_mutex+0x20/0x30
[   46.053818] Modules linked in: au0828(+) tveeprom dvb_core v4l2_common rfcomm bnep snd_hda_codec_hdmi snd_hda_codec_realtek

snd_hda_codec_generic uvcvideo videobuf2_vmalloc videobuf2_memops videobuf2_v4l2 videobuf2_common videodev media btusb btintel bluetooth snd_soc_skl snd_soc_skl_ipc snd_soc_sst_ipc snd_soc_sst_dsp snd_soc_acpi_intel_match snd_soc_acpi snd_hda_ext_core x86_pkg_temp_thermal snd_soc_core snd_compress ac97_bus snd_pcm_dmaengine snd_hda_intel iwlmvm snd_hda_codec snd_hda_core snd_hwdep snd_pcm snd_timer snd iwlwifi i915 intel_gtt battery ac pcc_cpufreq thermal

[   46.053853] CPU: 1 PID: 1824 Comm: systemd-udevd Not tainted 5.0.0-rc1-zen #85
[   46.053856] Hardware name: ASUSTeK COMPUTER INC. UX490UA/UX490UA, BIOS UX490UA.312 04/09/2018
[   46.053862] RIP: 0010:module_assert_mutex+0x20/0x30
[   46.053867] Code: 5d c3 e8 f3 68 f5 ff 0f 1f 00 8b 05 d2 5a 7c 01 85 c0 75 01 c3 be ff ff ff ff 48 c7 c7 80 bc 86 82 e8 e4 27 fb ff 85 c0
75 ea <0f> 0b c3 66 66 2e 0f 1f 84 00 00 00 00 00 66 90 53 48 89 fb e8 c7
[   46.053871] RSP: 0018:ffffc90002aa3ac8 EFLAGS: 00010246
[   46.053876] RAX: 0000000000000000 RBX: ffffffffa0573abf RCX: 0000000000000000
[   46.053880] RDX: 0000000000000000 RSI: ffffffff8286bc80 RDI: ffff8882a517d570
[   46.053883] RBP: ffff8882880e6000 R08: 0000000000000000 R09: ffff8882b1f5d000
[   46.053886] R10: 0000000000000001 R11: 0000000000000003 R12: ffffffffa0573abf
[   46.053890] R13: ffff8882880e60a0 R14: ffff8882880e6000 R15: ffffffffa0577200
[   46.053894] FS:  00007f0ac7d318c0(0000) GS:ffff8882b6a80000(0000) knlGS:0000000000000000
[   46.053898] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   46.053902] CR2: 0000560d354ed800 CR3: 00000002880f9003 CR4: 00000000003606e0
[   46.053905] Call Trace:
[   46.053910]  find_module+0x9/0x20

My .config is available upon request.

Quoted 92 lines · Hans Verkuil

Hi Shuah,

Quoted 32 lines · On 12/18/2018 06:59 PM, [email protected] wrote:

From: Shuah Khan <[email protected]>

Media Device Allocator API to allows multiple drivers share a media device. This API solves a very common use-case for media devices where one physical device (an USB stick) provides both audio and video. When such media device exposes a standard USB Audio class, a proprietary Video class, two or more independent drivers will share a single physical USB bridge. In such cases, it is necessary to coordinate access to the shared resource.

Using this API, drivers can allocate a media device with the shared struct device as the key. Once the media device is allocated by a driver, other drivers can get a reference to it. The media device is released when all the references are released.

- Tested sharing resources with kaffeine, vlc, xawtv, tvtime, and arecord. When analog is streaming, digital and audio user-space applications detect that the tuner is busy and exit. When digital is streaming, analog and audio applications detect that the tuner is busy and exit. When arecord is owns the tuner, digital and analog detect that the tuner is busy and exit.
- Tested media device allocator API with bind/unbind testing on snd-usb-audio and au0828 drivers to make sure /dev/mediaX is released only when the last driver is unbound.
- This patch series is tested on 4.20-rc6
- Addressed review comments from Hans on the RFC v8 (rebased on 4.19)
- Updated change log to describe the use-case more clearly.
- No changes to 0001,0002 code since the v7 referenced below.
- 0003 is a new patch to enable ALSA defines that have been disabled for kernel between 4.9 and 4.19.
- Minor merge conflict resolution in 0004.
- Added SPDX to new files.

Quoted 3 lines

References:
https://lkml.org/lkml/2018/11/2/169
https://www.mail-archive.com/[email protected]/msg105854.html

When I connect my au0828 to my laptop with your v9 patch series applied I get these warnings:

[   45.416047] xhci_hcd 0000:39:00.0: xHCI Host Controller
[   45.417882] xhci_hcd 0000:39:00.0: new USB bus registered, assigned bus number 3
[   45.419292] xhci_hcd 0000:39:00.0: hcc params 0x200077c1 hci version 0x110 quirks 0x0000000200009810
[   45.420835] usb usb3: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 5.00
[   45.420893] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[   45.420899] usb usb3: Product: xHCI Host Controller
[   45.420905] usb usb3: Manufacturer: Linux 5.0.0-rc1-zen xhci-hcd
[   45.420911] usb usb3: SerialNumber: 0000:39:00.0
[   45.424290] hub 3-0:1.0: USB hub found
[   45.425274] hub 3-0:1.0: 2 ports detected
[   45.431061] xhci_hcd 0000:39:00.0: xHCI Host Controller
[   45.432436] xhci_hcd 0000:39:00.0: new USB bus registered, assigned bus number 4
[   45.432448] xhci_hcd 0000:39:00.0: Host supports USB 3.1 Enhanced SuperSpeed
[   45.433299] usb usb4: New USB device found, idVendor=1d6b, idProduct=0003, bcdDevice= 5.00
[   45.433354] usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[   45.433360] usb usb4: Product: xHCI Host Controller
[   45.433365] usb usb4: Manufacturer: Linux 5.0.0-rc1-zen xhci-hcd
[   45.433370] usb usb4: SerialNumber: 0000:39:00.0
[   45.436134] hub 4-0:1.0: USB hub found
[   45.436576] hub 4-0:1.0: 2 ports detected
[   45.750940] usb 3-1: new high-speed USB device number 2 using xhci_hcd
[   45.899927] usb 3-1: New USB device found, idVendor=2040, idProduct=721e, bcdDevice= 0.05
[   45.899949] usb 3-1: New USB device strings: Mfr=1, Product=2, SerialNumber=10
[   45.899960] usb 3-1: Product: WinTV Aero-A
[   45.899970] usb 3-1: Manufacturer: Hauppauge
[   45.899979] usb 3-1: SerialNumber: 4033622430
[   46.053476] au0828: au0828 driver loaded
[   46.053726] WARNING: CPU: 1 PID: 1824 at kernel/module.c:262 module_assert_mutex+0x20/0x30
[   46.053818] Modules linked in: au0828(+) tveeprom dvb_core v4l2_common rfcomm bnep snd_hda_codec_hdmi snd_hda_codec_realtek

snd_hda_codec_generic uvcvideo videobuf2_vmalloc videobuf2_memops videobuf2_v4l2 videobuf2_common videodev media btusb btintel bluetooth snd_soc_skl snd_soc_skl_ipc snd_soc_sst_ipc snd_soc_sst_dsp snd_soc_acpi_intel_match snd_soc_acpi snd_hda_ext_core x86_pkg_temp_thermal snd_soc_core snd_compress ac97_bus snd_pcm_dmaengine snd_hda_intel iwlmvm snd_hda_codec snd_hda_core snd_hwdep snd_pcm snd_timer snd iwlwifi i915 intel_gtt battery ac pcc_cpufreq thermal

[   46.053853] CPU: 1 PID: 1824 Comm: systemd-udevd Not tainted 5.0.0-rc1-zen #85
[   46.053856] Hardware name: ASUSTeK COMPUTER INC. UX490UA/UX490UA, BIOS UX490UA.312 04/09/2018
[   46.053862] RIP: 0010:module_assert_mutex+0x20/0x30
[   46.053867] Code: 5d c3 e8 f3 68 f5 ff 0f 1f 00 8b 05 d2 5a 7c 01 85 c0 75 01 c3 be ff ff ff ff 48 c7 c7 80 bc 86 82 e8 e4 27 fb ff 85 c0
75 ea <0f> 0b c3 66 66 2e 0f 1f 84 00 00 00 00 00 66 90 53 48 89 fb e8 c7
[   46.053871] RSP: 0018:ffffc90002aa3ac8 EFLAGS: 00010246
[   46.053876] RAX: 0000000000000000 RBX: ffffffffa0573abf RCX: 0000000000000000
[   46.053880] RDX: 0000000000000000 RSI: ffffffff8286bc80 RDI: ffff8882a517d570
[   46.053883] RBP: ffff8882880e6000 R08: 0000000000000000 R09: ffff8882b1f5d000
[   46.053886] R10: 0000000000000001 R11: 0000000000000003 R12: ffffffffa0573abf
[   46.053890] R13: ffff8882880e60a0 R14: ffff8882880e6000 R15: ffffffffa0577200
[   46.053894] FS:  00007f0ac7d318c0(0000) GS:ffff8882b6a80000(0000) knlGS:0000000000000000
[   46.053898] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   46.053902] CR2: 0000560d354ed800 CR3: 00000002880f9003 CR4: 00000000003606e0
[   46.053905] Call Trace:
[   46.053910]  find_module+0x9/0x20

Quoted 2 lines

My .config is available upon request.

Thanks for the config.

Thanks for finding this. find_module() should be called with module_mutex since 4.15. The media allocator patch was written before that.

I didn't realize I don't have CONFIG_DEBUG_MUTEXES=y when I switched to a new system.

I reproduced the problem and re-worked patch 1 and patch 4 in this series. I will resend the v10 for just those two patches tomorrow.

Please let me know if you want me to send the entire series.