Skip to contentSkip to conversation

Draft agenda for the kernel summit

Ksummit-discuss Theodore Ts'o 19 participants 37 messages Oct 12, 2015 – Oct 22, 2015

lore
Share & tools
lore
#0 lore raw

This is an initial draft for the upcoming kernel summit.

It is still very drafty; in particular, just because a topic has been listed, it may turn out that the topic has been resolved off-line, or otherwise overtaken by events, in which case we will drop it. In addition, we're also in the process of sending queries to the proposed topic leaders to see if they are willing to kick off the discussion, and so that is subject to change as well.

I'd appreciate comments about any topics that you think might be missing and that would be worth our discussing. In addition, if you have strong opinions moving a topic from the technical session day to the core day or vice versa, please make the case for the change.

There will be some TBD slots which can be scheduled at the last minute, but I'd like to get at least one full track of the technical sessions and most of the core sessions scheduled.

Many thanks,

- Ted

Kernel Summit Agenda
October 26-28, 2015
DRAFT

Web Version Agenda:

* Monday: Workshops and break out sessions (overlap with Korea Linux Forum)
* Tuesday: Dual-track technical sessions
* Wednesday: Invite-only core attendees' plenary sessions

Dual-Track Technical sessions (Oct. 27th)
=========================================

9:00 - Welcome and agenda bashing
9:30 - Topic A1 | Topic B1
10:00 - Topic A2 | Topic B2
10:30 - Break
11:00 - Topic A3 | Topic B3

11:30   - Topic A4                   | Topic B4
12:00   - Topic A5                   | Topic B5
12:30   - Lunch
1:30    - Topic A6                   | Topic B6
2:00    - Topic A7                   | Topic B7
2:30    - Break
3:00    - Topic A8                   | Topic B8
3:30    - Topic A9                   | Topic B9
4:00    - Topic A10                  | Topic B10
4:30    - Break
5:00    - Kernel Summit Tech Day -- What went well, what can we do better? 
5:30    - Group Photograph (for core day attendees)
6:30    - Dinner

Potential Tech topics
=====================

1. Mainline kernel on a cellphone (need someone to lead?)
2. Semantics of MMIO mapping attributes across archs (Benjamin Herrenschmidt)
3. GPIO API/ABI discussion (need someone to lead?)
4. System-wide interface to specify the level of PM tuning (Rafael J. Wysocki)
5. Giving freezer well-defined semantics (Jiri Kosina)
6. IRQ affinity (Christoph Hellwig)
7. benchmarking and performance trends (Chris Mason)
8. FPGAs and how to program them from kernel (need somone to lead?
9. Overlays and file(system) unioning issues (David Howells)
10. Firmware signing (David Howells)
11. Context tracking / nohz / RCU state (Andy Lutomirski)
12. userspace infrastructure services (Stephen Hemminger)
13. Kernel support for compute-offload devices (Joerg Roedel)
14. Improving Kernel Security
15. Developer Workflow Security (Panel)

Core Day (Oct. 28th)
====================

9:00 - Welcome and agenda bashing
9:30 - Live patching (still needed?)
10:00 - Testing (Masahami, Shuah)
10:30 - Break
11:00 - Kernel Security - Readout and further discussion

11:30   - Recruitment; Outreach Programmes (Greg K-H)
12:00   - Lightweight per-cpu locks / restartable sequences (Andy Lutomirski)
12:30   - Lunch
1:30    - TBD
2:00    - TBD
2:30    - Break
3:00    - Kernel Development Process (Is Linus happy?)
3:30    - Issues with stable process (Sasha Levin)
4:00    - TBD
4:30    - Break
5:00    - Kernel Summit Core Day -- What went well, what can we do better? 
5:30    - Group Photograph (for core day attendees)
6:30    - Dinner
#1 lore raw
Quoted 8 lines · Theodore Ts'o

Dual-Track Technical sessions (Oct. 27th)
=========================================

9:00 - Welcome and agenda bashing
9:30 - Topic A1 | Topic B1
10:00 - Topic A2 | Topic B2
10:30 - Break
11:00 - Topic A3 | Topic B3

Quoted 12 lines
11:30   - Topic A4                   | Topic B4
12:00   - Topic A5                   | Topic B5
12:30   - Lunch
1:30    - Topic A6                   | Topic B6
2:00    - Topic A7                   | Topic B7
2:30    - Break
3:00    - Topic A8                   | Topic B8
3:30    - Topic A9                   | Topic B9
4:00    - Topic A10                  | Topic B10
4:30    - Break
5:00    - Kernel Summit Tech Day -- What went well, what can we do better? 
5:30    - Group Photograph (for core day attendees)

Should this say "for tech day attendees"?

- Josh Triplett

#2 lore raw
In reply to Theodore Ts'o
Quoted 1 line · Theodore Ts'o

3. GPIO API/ABI discussion (need someone to lead?)

Unfortunately I will not be at the KS, but I definately want to know the outcome of this discussion for natural reasons.

Johan Hovold has been raising issues with the current ABI on the list.

I am now trying to get a chardev prototype up for a richer/saner userspace ABI for GPIO but other contributions and ideas are welcome.

Yours,
Linus Walleij

#3 lore raw
In reply to Theodore Ts'o
Quoted 1 line · Theodore Ts'o

This is an initial draft for the upcoming kernel summit.

[...]

Quoted 22 lines

Dual-Track Technical sessions (Oct. 27th)
=========================================

9:00    - Welcome and agenda bashing
9:30    - Topic A1                   | Topic B1
10:00   - Topic A2                   | Topic B2
10:30   - Break
11:00   - Topic A3                   | Topic B3
11:30   - Topic A4                   | Topic B4
12:00   - Topic A5                   | Topic B5
12:30   - Lunch
1:30    - Topic A6                   | Topic B6
2:00    - Topic A7                   | Topic B7
2:30    - Break
3:00    - Topic A8                   | Topic B8
3:30    - Topic A9                   | Topic B9
4:00    - Topic A10                  | Topic B10
4:30    - Break
5:00    - Kernel Summit Tech Day -- What went well, what can we do better?
5:30    - Group Photograph (for core day attendees)
6:30    - Dinner
Quoted 4 lines

Potential Tech topics
=====================

1. Mainline kernel on a cellphone (need someone to lead?)

I can lead this.

This is a kitchen sink of a topic though and something that's been discussed several times recently at ELC and Plumbers. There's not much point in rehashing the problem (go watch the conference videos), but instead we should perhaps drill down into specific issues where we have the right people present. Here's my list:

- Gaps in driver subsystems:
- WiFi - Android gaps: mode switching, P2P, Chromecast, roaming,
ePNO, Passpoint
- DRM/Graphics
- BT/GPS/NFC - proper kernel drivers for serial devices
- LEDs - tri-color support, defining roles (e.g. Android attention LED).
- H/W video codecs
- Camera
- Battery/charger issues
- Android Memory management - What to do with ION?
- Power management - ?
- How to speed up mainlining process?

Other topics?

Rob

#4 lore raw
Quoted 4 lines · Rob Herring

- Gaps in driver subsystems:
- WiFi - Android gaps: mode switching, P2P, Chromecast, roaming,
ePNO, Passpoint

I will be there, I can talk about this if desired, or you can poke me on the hallway to talk about it :)

The overwhelming problem isn't our unwillingness to upstream this work though, it's the unwillingness of Google to release requirements early enough to make that possible, along with the unwillingness of chip vendors (hi Broadcom, Qualcomm, MediaTek) to have an upstream driver that they actually use (some of those do support an upstream driver, but don't ship that one and it doesn't nearly have the necessary features, while the actually used (still open source [1]) driver is a pile of spaghetti code that we'd probably never merge upstream...)

[1] except for MediaTek, I think, who seem to never be releasing source code for any of their kernels

johannes

#5 lore raw
Quoted 11 lines · On Friday 16 October 2015 08:52:37 Johannes Berg wrote:

On Thu, 2015-10-15 at 19:41 -0500, Rob Herring wrote:
the unwillingness of chip
vendors (hi Broadcom, Qualcomm, MediaTek) to have an upstream driver that they actually use (some of those do support an upstream driver, but don't ship that one and it doesn't nearly have the necessary features, while the actually used (still open source [1]) driver is a pile of spaghetti code that we'd probably never merge upstream...)

[1] except for MediaTek, I think, who seem to never be releasing source code for any of their kernels

(getting offtopic here)

My understanding is that MediaTek has improved much recently and their mt76 wireless driver source is available and getting upstreamed now and integrated into openwrt now. OTOH, Broadcom apparently regressed and is no longer releasing any source code to their newer softmac parts (bcm4360, bcm4352) or patching brcmsmac to support them, while they on the other hand got better at upstreaming their SoC support recently.

Arnd

#6 lore raw
Quoted 5 lines · Arnd Bergmann

(getting offtopic here)

My understanding is that MediaTek has improved much recently and their mt76 wireless driver source is available and getting upstreamed now and integrated into openwrt now.

Yes, mostly a community effort from what I can tell, although surely they released something. However, it's a different product line as far as I know.

johannes

#7 lore raw
In reply to Arnd Bergmann

Hi Arnd,

Quoted 12 lines
Quoted 11 lines · Rob Herring

the unwillingness of chip
vendors (hi Broadcom, Qualcomm, MediaTek) to have an upstream driver that they actually use (some of those do support an upstream driver, but don't ship that one and it doesn't nearly have the necessary features, while the actually used (still open source [1]) driver is a pile of spaghetti code that we'd probably never merge upstream...)

[1] except for MediaTek, I think, who seem to never be releasing source code for any of their kernels

Quoted 8 lines

(getting offtopic here)

My understanding is that MediaTek has improved much recently and their mt76 wireless driver source is available and getting upstreamed now and integrated into openwrt now. OTOH, Broadcom apparently regressed and is no longer releasing any source code to their newer softmac parts (bcm4360, bcm4352) or patching brcmsmac to support them, while they on the other hand got better at upstreaming their SoC support recently.

are sure that Mediatek got better? I am not convinced by that.

I think there is also a large difference in their MiniPCI cards compared to their connectivity hub in their SoC. The Bluetooth side for example is largely copying existing drivers, hacking their vendor specific behavior in and then throwing it over the wall.

Regards

Marcel

#8 lore raw
Quoted 22 lines · On Friday 16 October 2015 10:00:18 Marcel Holtmann wrote:
Quoted 13 lines
Quoted 9 lines · Rob Herring

the unwillingness of chip
vendors (hi Broadcom, Qualcomm, MediaTek) to have an upstream driver that they actually use (some of those do support an upstream driver, but don't ship that one and it doesn't nearly have the necessary features, while the actually used (still open source [1]) driver is a pile of spaghetti code that we'd probably never merge upstream...)

Quoted 3 lines

[1] except for MediaTek, I think, who seem to never be releasing source code for any of their kernels

Quoted 8 lines

(getting offtopic here)

My understanding is that MediaTek has improved much recently and their mt76 wireless driver source is available and getting upstreamed now and integrated into openwrt now. OTOH, Broadcom apparently regressed and is no longer releasing any source code to their newer softmac parts (bcm4360, bcm4352) or patching brcmsmac to support them, while they on the other hand got better at upstreaming their SoC support recently.

Quoted 6 lines

are sure that Mediatek got better? I am not convinced by that.

I think there is also a large difference in their MiniPCI cards compared to their connectivity hub in their SoC. The Bluetooth side for example is largely copying existing drivers, hacking their vendor specific behavior in and then throwing it over the wall.

It's of course a large company, and there are several business units that are not all improving equally. There is however a big difference between basically ignoring upstream as of a few years ago and now being as good as any other SoC vendor that works with us on arm-soc.

The areas that I have observed improving most are:

- support for their wireless AP solutions (mtk76xx, formerly ralink) is getting much better. While this is mainly driven by OpenWRT on the upstream side, they are working together. All there is today is the MIPS based support, but I'm sure that the recently announced ARM based parts will get upstreamed quickly too as they are quite similar.

- Mobile phone/tablet SoC support is getting merged upstream for both ARM32 (mt65xx, mt81xx) and ARM64 (mt67xx, mt81xx too) in a good way. They are still doing groundwork at the moment, so I'm not surprised that the people that rewrite the code into upstreamable drivers have not arrived at bluetooth yet, but it seems likely that they will get there as they do one subsystem at a time.

The main concern that is still valid for all their SoCs is whether they eventually get to the point where full support for any product is in mainline by the time that it gets into user's hands, which is really what our KS session is about.

Arnd

#9 lore raw
In reply to Marcel Holtmann
Quoted 11 lines · Marcel Holtmann

Arnd:

Quoted 2 lines

My understanding is that MediaTek has improved much recently and their mt76 wireless driver source is available and getting upstreamed (...)

are sure that Mediatek got better? I am not convinced by that.

I think there is also a large difference in their MiniPCI cards compared to their connectivity hub in their SoC. The Bluetooth side for example is largely copying existing drivers, hacking their vendor specific behavior in and then throwing it over the wall.

When discussing the sadly unsupported MT7630E MiniPCI card which has both WiFi and Bluetooth Jakub Kicinski elaborated a bit on the story there:
http://marc.info/?l=linux-wireless&m=143815707108278&w=2

It seems Mediatek got a bit of internal struggle, the kernel doing three different drivers sharing aspects of the same chip(s) and general mishmash due to a cocktail of factors: organizational, company consolidation etc, but also due to a few chips, especially MT7630E, ending up in a technical middle-ground between two families of chips.

The person writing the MT7630E driver was obviously intending to upstream it, I can see the beginnings of an upstream attempt in the source code, but was apparently removed from that task before s/he could complete it.

Sadly there is a bunch of users stuck with this card. They (including my daughter) survive by using a forwardport of the code drop from Mediatek, but who knows how long that thing will even compile.

Linus Walleij

#10 lore raw
In reply to Rob Herring

Rob,

Quoted 5 lines · Rob Herring

- Gaps in driver subsystems:
- WiFi - Android gaps: mode switching, P2P, Chromecast, roaming,
ePNO, Passpoint
- DRM/Graphics
- BT/GPS/NFC - proper kernel drivers for serial devices

Could you elaborate by what you mean by "proper" drivers in that context?

Cheers,
Samuel.

#11 lore raw

Hi Samuel,

Quoted 8 lines · Samuel Ortiz
Quoted 5 lines · Rob Herring

- Gaps in driver subsystems:
- WiFi - Android gaps: mode switching, P2P, Chromecast, roaming,
ePNO, Passpoint
- DRM/Graphics
- BT/GPS/NFC - proper kernel drivers for serial devices

Could you elaborate by what you mean by "proper" drivers in that context?

I think this is about the ongoing work to represent serial devices as a bus, so serial slaves can connect to them using a standard API.

Gr{oetje,eeting}s,

Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]

In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds

#12 lore raw
In reply to Samuel Ortiz
Quoted 10 lines · Samuel Ortiz

Rob,

Quoted 5 lines · Rob Herring

- Gaps in driver subsystems:
- WiFi - Android gaps: mode switching, P2P, Chromecast, roaming,
ePNO, Passpoint
- DRM/Graphics
- BT/GPS/NFC - proper kernel drivers for serial devices

Could you elaborate by what you mean by "proper" drivers in that context?

Exactly what was discussed on the thread early for this topic. Creating in kernel drivers for serial attached devices like BT to handle all the chip and board specific bits rather than do this in userspace.

Rob

#13 lore raw
In reply to Rob Herring
Quoted 3 lines · Rob Herring

On Mon, Oct 12, 2015 at 2:01 PM, Theodore Ts'o <[email protected]> wrote:

- Battery/charger issues

That's as much the whole situation with support for new USB standards as anything specific to batteries, considerations around USB are what's causing disrupton here.

#14 lore raw
In reply to Theodore Ts'o

Am 12.10.2015 um 21:01 schrieb Theodore Ts'o:

Quoted 23 lines

This is an initial draft for the upcoming kernel summit.

It is still very drafty; in particular, just because a topic has been listed, it may turn out that the topic has been resolved off-line, or otherwise overtaken by events, in which case we will drop it. In addition, we're also in the process of sending queries to the proposed topic leaders to see if they are willing to kick off the discussion, and so that is subject to change as well.

I'd appreciate comments about any topics that you think might be missing and that would be worth our discussing. In addition, if you have strong opinions moving a topic from the technical session day to the core day or vice versa, please make the case for the change.

There will be some TBD slots which can be scheduled at the last minute, but I'd like to get at least one full track of the technical sessions and most of the core sessions scheduled.

Many thanks,

- Ted

Quoted 7 lines

Kernel Summit Agenda
October 26-28, 2015
DRAFT

Web Version Agenda:

* Monday: Workshops and break out sessions (overlap with Korea Linux Forum)

Might be a dumb questions, but are the Monday workshops planned via the Korea Linux Forum or do we have a separate list of planned workshops?

Christian

#15 lore raw
In reply to Theodore Ts'o

Em Mon, 12 Oct 2015 15:01:37 -0400
Theodore Ts'o <[email protected]> escreveu:

Quoted 58 lines

This is an initial draft for the upcoming kernel summit.

It is still very drafty; in particular, just because a topic has been listed, it may turn out that the topic has been resolved off-line, or otherwise overtaken by events, in which case we will drop it. In addition, we're also in the process of sending queries to the proposed topic leaders to see if they are willing to kick off the discussion, and so that is subject to change as well.

I'd appreciate comments about any topics that you think might be missing and that would be worth our discussing. In addition, if you have strong opinions moving a topic from the technical session day to the core day or vice versa, please make the case for the change.

There will be some TBD slots which can be scheduled at the last minute, but I'd like to get at least one full track of the technical sessions and most of the core sessions scheduled.

Many thanks,

- Ted

Kernel Summit Agenda
October 26-28, 2015
DRAFT

Web Version Agenda:

* Monday: Workshops and break out sessions (overlap with Korea Linux Forum)
* Tuesday: Dual-track technical sessions
* Wednesday: Invite-only core attendees' plenary sessions

Dual-Track Technical sessions (Oct. 27th)
=========================================

9:00 - Welcome and agenda bashing
9:30 - Topic A1 | Topic B1
10:00 - Topic A2 | Topic B2
10:30 - Break
11:00 - Topic A3 | Topic B3

11:30   - Topic A4                   | Topic B4
12:00   - Topic A5                   | Topic B5
12:30   - Lunch
1:30    - Topic A6                   | Topic B6
2:00    - Topic A7                   | Topic B7
2:30    - Break
3:00    - Topic A8                   | Topic B8
3:30    - Topic A9                   | Topic B9
4:00    - Topic A10                  | Topic B10
4:30    - Break
5:00    - Kernel Summit Tech Day -- What went well, what can we do better? 
5:30    - Group Photograph (for core day attendees)
6:30    - Dinner

Potential Tech topics
=====================

Quoted 15 lines

1. Mainline kernel on a cellphone (need someone to lead?)
2. Semantics of MMIO mapping attributes across archs (Benjamin Herrenschmidt)
3. GPIO API/ABI discussion (need someone to lead?)
4. System-wide interface to specify the level of PM tuning (Rafael J. Wysocki)
5. Giving freezer well-defined semantics (Jiri Kosina)
6. IRQ affinity (Christoph Hellwig)
7. benchmarking and performance trends (Chris Mason)
8. FPGAs and how to program them from kernel (need somone to lead?
9. Overlays and file(system) unioning issues (David Howells)
10. Firmware signing (David Howells)
11. Context tracking / nohz / RCU state (Andy Lutomirski)
12. userspace infrastructure services (Stephen Hemminger)
13. Kernel support for compute-offload devices (Joerg Roedel)
14. Improving Kernel Security
15. Developer Workflow Security (Panel)

Could you please add a new topic:

16. The Next Generation of the Media Controller (Mauro Carvalho Chehab)

Along the years, we've been developing a mechanism at the Kernel to store and present graphs to userspace, called Media Controller. The original scope of the Media Controller were to store and represent pipeline connections between hardware components on complex streaming devices, like the ones that are used for webcams on cell phones, where there are typically two camera sensors, and lots of processing units to enhance the image and to convert it into different formats.

Since the beginning, we found that it would be useful to use it some day on other subsystems, but we were too busy doing it at V4L2.

At the end of the last year, we added support for the Media Controller on DVB, using the existing API, in order to represent hybrid TV devices with radio, analog TV (video and audio) and digital TV. However, as it didn't fit quite well, the DVB MC support was marked as broken before reaching upstream, and we designed and implemented a next generation of the API, together with userspace applications to test it.

The ultimate goal will be to support the pipelines on a TV set and Set Top Boxes. On such devices, the streaming data pipeline can cover camera sensors, analog TV, digital TV, crypto/decrypto modules, audio and GPU. Also, they may have network interfaces that may be provided via DVB (with is very useful on Cable TV).

So, as we want/need to represent and control the full pipeline, the media controller has to have support on different subsystems: V4L2, DVB, ALSA, DRM and Network. It also needs a way to share a common struct between those subsystems without making them depend on each other. So, it should use devres.

We did a Media Controller Summit in July. During the summit, it was pointed that other subsystems like IIO (Linux Industrial I/O) have similar needs and also need to track complex pipelines.

The goal of this tech topic is to present the MC next generation concepts and how the patches to support MC was written, showing some pipeline examples.

The next gen support patches were written in August, with some additional patches written after that, and it has support for DVB and for one ALSA driver (snd-usb-audio).

The patches are not merged yet at the media development tree and at -next, as one of the core developers of the media controller is missing time to review patches. So, while it was originally targeted to be merged on Kernel 4.4, it is likely that this will be merged only for Kernel 4.5, in order to give him more time to review the patchset.

PS.: I send already a [TECH TOPIC] about that on Aug, 6, and some people seemed to be interested.

#16 lore raw
Quoted 12 lines · Mauro Carvalho Chehab

We did a Media Controller Summit in July. During the summit, it was pointed that other subsystems like IIO (Linux Industrial I/O) have similar needs and also need to track complex pipelines.

The goal of this tech topic is to present the MC next generation concepts and how the patches to support MC was written, showing some pipeline examples.

The next gen support patches were written in August, with some additional patches written after that, and it has support for DVB and for one ALSA driver (snd-usb-audio).

Quoted 6 lines

The patches are not merged yet at the media development tree and at -next, as one of the core developers of the media controller is missing time to review patches. So, while it was originally targeted to be merged on Kernel 4.4, it is likely that this will be merged only for Kernel 4.5, in order to give him more time to review the patchset.

One fun thing on this that came up at the audio meeting we had in Dublin after ELC-E is that there is some desire to export the internal topology of audio devices to userspace using the same format that we're using to load topology information for DSP firmwares (see include/sound/soc-topoloy.h). If we go ahead with that it'd create an alternative topology ABI, it seems like we should at least consider if it's worth trying to share things here.

I've not had a chance to look at the new media controller work yet, I've CCed Lars and Liam who have more knowledge in this area. Unfortunately I don't think any of the people working directly on this will be at KS this year.

#17 lore raw
Quoted 29 lines · Mark Brown
Quoted 12 lines · Mauro Carvalho Chehab

We did a Media Controller Summit in July. During the summit, it was pointed that other subsystems like IIO (Linux Industrial I/O) have similar needs and also need to track complex pipelines.

The goal of this tech topic is to present the MC next generation concepts and how the patches to support MC was written, showing some pipeline examples.

The next gen support patches were written in August, with some additional patches written after that, and it has support for DVB and for one ALSA driver (snd-usb-audio).

Quoted 6 lines

The patches are not merged yet at the media development tree and at -next, as one of the core developers of the media controller is missing time to review patches. So, while it was originally targeted to be merged on Kernel 4.4, it is likely that this will be merged only for Kernel 4.5, in order to give him more time to review the patchset.

One fun thing on this that came up at the audio meeting we had in Dublin after ELC-E is that there is some desire to export the internal topology of audio devices to userspace using the same format that we're using to load topology information for DSP firmwares (see include/sound/soc-topoloy.h). If we go ahead with that it'd create an alternative topology ABI, it seems like we should at least consider if it's worth trying to share things here.

Quoted 4 lines

I've not had a chance to look at the new media controller work yet, I've CCed Lars and Liam who have more knowledge in this area. Unfortunately I don't think any of the people working directly on this will be at KS this year.

Just to give a little more background (for any non audio folks), there are many audio DSPs today that can load their topology and other capabilities at runtime depending on the target device (phone, laptop, PC, etc) and the audio use cases.

Our intention is to also have a single audio driver per DSP family (i.e. single skylake audio DSP driver to cover all skylake based devices). The DSP driver would load it's topology data alongside the DSP firmware so that the driver topology (i.e. graph, controls, capabilities) would be aligned with the DSP firmware topology.

The process for creating the topology data involves either compiling a text topology file to the binary topology file (used by the kernel) or using a new ALSA C API to generate the binary topology file (as part of a firmware toolchain). The topology text compiler and C API are both upstream.

We are currently implementing de-compiler support to the userspace compiler in order to convert the binary topology files into text files for debug and development purposes.

Ideally I'd like to be able to dump the topology data using the same format that it used for topology loading. That way we could easily feed them into the de-compiler for development/debug.

Having a simple mechanism for dumping topology data is important as we will need to be able to easily dump this data on Android and Chrome alongside regular Linux for development and debug purposes. i.e. cat /sys/some/dump/file > audio_topology

I'm also supportive of having a higher level and more elegant topology query interface for non kernel/firmware developers using media controller.

What I'd like to propose is that we support both mechanisms for dumping the audio topology data. :-

1) Simple file dump using same format that topology is loaded. Used by kernel/firmware developers only when media controller userspace not available. Enabled by kernel Kconfig debug option.

2) Media controller API used by everyone else.

Liam

#18 lore raw

Hi Mark/Liam,

Em Mon, 19 Oct 2015 17:27:56 +0100
Liam Girdwood <[email protected]> escreveu:

Quoted 22 lines · Mark Brown
Quoted 21 lines · Mauro Carvalho Chehab
Quoted 19 lines

We did a Media Controller Summit in July. During the summit, it was pointed that other subsystems like IIO (Linux Industrial I/O) have similar needs and also need to track complex pipelines.

The goal of this tech topic is to present the MC next generation concepts and how the patches to support MC was written, showing some pipeline examples.

The next gen support patches were written in August, with some additional patches written after that, and it has support for DVB and for one ALSA driver (snd-usb-audio).

The patches are not merged yet at the media development tree and at -next, as one of the core developers of the media controller is missing time to review patches. So, while it was originally targeted to be merged on Kernel 4.4, it is likely that this will be merged only for Kernel 4.5, in order to give him more time to review the patchset.

Quoted 7 lines
Quoted 7 lines

One fun thing on this that came up at the audio meeting we had in Dublin after ELC-E is that there is some desire to export the internal topology of audio devices to userspace using the same format that we're using to load topology information for DSP firmwares (see include/sound/soc-topoloy.h). If we go ahead with that it'd create an alternative topology ABI, it seems like we should at least consider if it's worth trying to share things here.

Thanks for the feedback. I was wanting to go to ELC-E this year due to ALSA summit, but, unfortunately, I had 3 other trips happening on a one month interval (two of them are 28+28 hours of trip). So, I had to give up going to ELC-E.

Quoted 27 lines
Quoted 5 lines

I've not had a chance to look at the new media controller work yet, I've CCed Lars and Liam who have more knowledge in this area. Unfortunately I don't think any of the people working directly on this will be at KS this year.

Just to give a little more background (for any non audio folks), there are many audio DSPs today that can load their topology and other capabilities at runtime depending on the target device (phone, laptop, PC, etc) and the audio use cases.

Our intention is to also have a single audio driver per DSP family (i.e. single skylake audio DSP driver to cover all skylake based devices). The DSP driver would load it's topology data alongside the DSP firmware so that the driver topology (i.e. graph, controls, capabilities) would be aligned with the DSP firmware topology.

The process for creating the topology data involves either compiling a text topology file to the binary topology file (used by the kernel) or using a new ALSA C API to generate the binary topology file (as part of a firmware toolchain). The topology text compiler and C API are both upstream.

We are currently implementing de-compiler support to the userspace compiler in order to convert the binary topology files into text files for debug and development purposes.

Quoted 8 lines

Ideally I'd like to be able to dump the topology data using the same format that it used for topology loading. That way we could easily feed them into the de-compiler for development/debug.

Having a simple mechanism for dumping topology data is important as we will need to be able to easily dump this data on Android and Chrome alongside regular Linux for development and debug purposes. i.e. cat /sys/some/dump/file > audio_topology

Well, it is not hard to get the topology using the media controller. I wrote a testing program using it at:
http://git.linuxtv.org/cgit.cgi/mchehab/experimental-v4l-utils.git/tree/contrib/test/mc_nextgen_test.c

The only library optional dependency is libudev, used to get the device node names. The code falls back to use /sys/dev/char/ and /dev to find the device names when compiled without libudev.

So, it should be easy to provide a cat-like program that would be dumping the audio topology on some file.

Quoted 3 lines

I'm also supportive of having a higher level and more elegant topology query interface for non kernel/firmware developers using media controller.

The media controller also allows runtime topology changes. This is important specially on SoC devices that allow runtime changes.

One of the advanced use cases of the media controller API is on SoC that have dynamic pipelines on more than one subsystem that could be wired on a single circuit.

One such example would be to setup a TV decoding pipeline that starts on a TV tuner, goes though DVB, V4L, DRM and ALSA subsystems and the sink nodes would be the audio speaker and the TV display.

As the TV channel is changed, different codecs may be needed for video and/or audio. Also, as the program may be encrypted (or not), it may be needed to add Conditional Access Modules at the pipelines and/or other hardware IP blocks in order to decrypt the contents.

Quoted 7 lines

What I'd like to propose is that we support both mechanisms for dumping the audio topology data. :-

1) Simple file dump using same format that topology is loaded. Used by kernel/firmware developers only when media controller userspace not available. Enabled by kernel Kconfig debug option.

Yeah, it sounds reasonable to have a mechanism like that just for debug purposes.

Quoted 1 line

2) Media controller API used by everyone else.

Makes sense to me.

Regards,
Mauro

#19 lore raw
Quoted 71 lines · Mauro Carvalho Chehab

Hi Mark/Liam,

Em Mon, 19 Oct 2015 17:27:56 +0100
Liam Girdwood <[email protected]> escreveu:

Quoted 22 lines · Mark Brown
Quoted 21 lines · Mauro Carvalho Chehab
Quoted 19 lines

We did a Media Controller Summit in July. During the summit, it was pointed that other subsystems like IIO (Linux Industrial I/O) have similar needs and also need to track complex pipelines.

The goal of this tech topic is to present the MC next generation concepts and how the patches to support MC was written, showing some pipeline examples.

The next gen support patches were written in August, with some additional patches written after that, and it has support for DVB and for one ALSA driver (snd-usb-audio).

The patches are not merged yet at the media development tree and at -next, as one of the core developers of the media controller is missing time to review patches. So, while it was originally targeted to be merged on Kernel 4.4, it is likely that this will be merged only for Kernel 4.5, in order to give him more time to review the patchset.

Quoted 7 lines
Quoted 7 lines

One fun thing on this that came up at the audio meeting we had in Dublin after ELC-E is that there is some desire to export the internal topology of audio devices to userspace using the same format that we're using to load topology information for DSP firmwares (see include/sound/soc-topoloy.h). If we go ahead with that it'd create an alternative topology ABI, it seems like we should at least consider if it's worth trying to share things here.

Thanks for the feedback. I was wanting to go to ELC-E this year due to ALSA summit, but, unfortunately, I had 3 other trips happening on a one month interval (two of them are 28+28 hours of trip). So, I had to give up going to ELC-E.

Quoted 31 lines
Quoted 5 lines

I've not had a chance to look at the new media controller work yet, I've CCed Lars and Liam who have more knowledge in this area. Unfortunately I don't think any of the people working directly on this will be at KS this year.

Just to give a little more background (for any non audio folks), there are many audio DSPs today that can load their topology and other capabilities at runtime depending on the target device (phone, laptop, PC, etc) and the audio use cases.

Our intention is to also have a single audio driver per DSP family (i.e. single skylake audio DSP driver to cover all skylake based devices). The DSP driver would load it's topology data alongside the DSP firmware so that the driver topology (i.e. graph, controls, capabilities) would be aligned with the DSP firmware topology.

The process for creating the topology data involves either compiling a text topology file to the binary topology file (used by the kernel) or using a new ALSA C API to generate the binary topology file (as part of a firmware toolchain). The topology text compiler and C API are both upstream.

We are currently implementing de-compiler support to the userspace compiler in order to convert the binary topology files into text files for debug and development purposes.

Ideally I'd like to be able to dump the topology data using the same format that it used for topology loading. That way we could easily feed them into the de-compiler for development/debug.

Quoted 9 lines
Quoted 4 lines

Having a simple mechanism for dumping topology data is important as we will need to be able to easily dump this data on Android and Chrome alongside regular Linux for development and debug purposes. i.e. cat /sys/some/dump/file > audio_topology

Well, it is not hard to get the topology using the media controller. I wrote a testing program using it at:
http://git.linuxtv.org/cgit.cgi/mchehab/experimental-v4l-utils.git/tree/contrib/test/mc_nextgen_test.c

Adding to what Mauro said, I have a patch series out that updates ASLA and AU0828 drivers (Hauppauge Win TV HVR 950Q Hybrid USB TV Stick) to share tuner. I am speaking about this work at the KLF on Oct 26th.

http://korealinuxforum2015.sched.org/event/f59175cc49f866dbc9bebc36f86fe663#.ViVWjGwVhBd

You can find the media graph that includes ALSA Mixer Function and Control Interface (Device: /dev/snd/controlC2), PCM Audio Capture Function and PCM Audio Capture interface (Device: /dev/snd/pcmC2D0c) at:

https://drive.google.com/open?id=0B0NIL0BQg-AlbjZkcElJb2RtVk0

thanks,
-- Shuah

#20 lore raw
Quoted 88 lines · Shuah Khan
Quoted 71 lines · Mauro Carvalho Chehab

Hi Mark/Liam,

Em Mon, 19 Oct 2015 17:27:56 +0100
Liam Girdwood <[email protected]> escreveu:

Quoted 22 lines · Mark Brown
Quoted 21 lines · Mauro Carvalho Chehab
Quoted 19 lines

We did a Media Controller Summit in July. During the summit, it was pointed that other subsystems like IIO (Linux Industrial I/O) have similar needs and also need to track complex pipelines.

The goal of this tech topic is to present the MC next generation concepts and how the patches to support MC was written, showing some pipeline examples.

The next gen support patches were written in August, with some additional patches written after that, and it has support for DVB and for one ALSA driver (snd-usb-audio).

The patches are not merged yet at the media development tree and at -next, as one of the core developers of the media controller is missing time to review patches. So, while it was originally targeted to be merged on Kernel 4.4, it is likely that this will be merged only for Kernel 4.5, in order to give him more time to review the patchset.

Quoted 7 lines
Quoted 7 lines

One fun thing on this that came up at the audio meeting we had in Dublin after ELC-E is that there is some desire to export the internal topology of audio devices to userspace using the same format that we're using to load topology information for DSP firmwares (see include/sound/soc-topoloy.h). If we go ahead with that it'd create an alternative topology ABI, it seems like we should at least consider if it's worth trying to share things here.

Thanks for the feedback. I was wanting to go to ELC-E this year due to ALSA summit, but, unfortunately, I had 3 other trips happening on a one month interval (two of them are 28+28 hours of trip). So, I had to give up going to ELC-E.

Quoted 31 lines
Quoted 5 lines

I've not had a chance to look at the new media controller work yet, I've CCed Lars and Liam who have more knowledge in this area. Unfortunately I don't think any of the people working directly on this will be at KS this year.

Just to give a little more background (for any non audio folks), there are many audio DSPs today that can load their topology and other capabilities at runtime depending on the target device (phone, laptop, PC, etc) and the audio use cases.

Our intention is to also have a single audio driver per DSP family (i.e. single skylake audio DSP driver to cover all skylake based devices). The DSP driver would load it's topology data alongside the DSP firmware so that the driver topology (i.e. graph, controls, capabilities) would be aligned with the DSP firmware topology.

The process for creating the topology data involves either compiling a text topology file to the binary topology file (used by the kernel) or using a new ALSA C API to generate the binary topology file (as part of a firmware toolchain). The topology text compiler and C API are both upstream.

We are currently implementing de-compiler support to the userspace compiler in order to convert the binary topology files into text files for debug and development purposes.

Ideally I'd like to be able to dump the topology data using the same format that it used for topology loading. That way we could easily feed them into the de-compiler for development/debug.

Quoted 9 lines
Quoted 4 lines

Having a simple mechanism for dumping topology data is important as we will need to be able to easily dump this data on Android and Chrome alongside regular Linux for development and debug purposes. i.e. cat /sys/some/dump/file > audio_topology

Well, it is not hard to get the topology using the media controller. I wrote a testing program using it at:
http://git.linuxtv.org/cgit.cgi/mchehab/experimental-v4l-utils.git/tree/contrib/test/mc_nextgen_test.c

Adding to what Mauro said, I have a patch series out that updates ASLA and AU0828 drivers (Hauppauge Win TV HVR 950Q Hybrid USB TV Stick) to share tuner. I am speaking about this work at the KLF on Oct 26th.

Quoted 8 lines

http://korealinuxforum2015.sched.org/event/f59175cc49f866dbc9bebc36f86fe663#.ViVWjGwVhBd

You can find the media graph that includes ALSA Mixer Function and Control Interface (Device: /dev/snd/controlC2), PCM Audio Capture Function and PCM Audio Capture interface (Device: /dev/snd/pcmC2D0c) at:

https://drive.google.com/open?id=0B0NIL0BQg-AlbjZkcElJb2RtVk0

Looks nice. Fwiw, the graph will probably get a lot bigger when we show the audio DSP and codec paths (as DSPs and codecs have multiple muxes and mixers). It may be good to have a cmd line option to stop at certain nodes on the graph to avoid over populated/complex graphs ?

Liam

Quoted 3 lines

thanks,
-- Shuah

#21 lore raw
Quoted 7 lines · Liam Girdwood
Quoted 4 lines · Shuah Khan

You can find the media graph that includes ALSA Mixer Function and Control Interface (Device: /dev/snd/controlC2), PCM Audio Capture Function and PCM Audio Capture interface (Device: /dev/snd/pcmC2D0c) at:

Quoted 6 lines
Quoted 1 line

https://drive.google.com/open?id=0B0NIL0BQg-AlbjZkcElJb2RtVk0

Looks nice. Fwiw, the graph will probably get a lot bigger when we show the audio DSP and codec paths (as DSPs and codecs have multiple muxes and mixers). It may be good to have a cmd line option to stop at certain nodes on the graph to avoid over populated/complex graphs ?

Having the ability to describe which physical device things are part of would probably help a lot with the rendering here - I'm guessing that a lot of filtering is likely to be things like collapsing down devices not being looked at right now into a single block in the graph and it'd probably help with user comprehesibilty provide a mapping back to things they can see in the physical world.

This doesn't entirely correspond with struct device due to MFD and multi-function USB stuff but it'd be a good start I expect.

#22 lore raw

Em Tue, 20 Oct 2015 14:22:09 +0100
Mark Brown <[email protected]> escreveu:

Quoted 22 lines · Liam Girdwood
Quoted 7 lines · Shuah Khan
Quoted 5 lines

You can find the media graph that includes ALSA Mixer Function and Control Interface (Device: /dev/snd/controlC2), PCM Audio Capture Function and PCM Audio Capture interface (Device: /dev/snd/pcmC2D0c) at:

Quoted 6 lines
Quoted 1 line

https://drive.google.com/open?id=0B0NIL0BQg-AlbjZkcElJb2RtVk0

Looks nice. Fwiw, the graph will probably get a lot bigger when we show the audio DSP and codec paths (as DSPs and codecs have multiple muxes and mixers). It may be good to have a cmd line option to stop at certain nodes on the graph to avoid over populated/complex graphs ?

Having the ability to describe which physical device things are part of would probably help a lot with the rendering here - I'm guessing that a lot of filtering is likely to be things like collapsing down devices not being looked at right now into a single block in the graph and it'd probably help with user comprehesibilty provide a mapping back to things they can see in the physical world.

Quoted 2 lines

This doesn't entirely correspond with struct device due to MFD and multi-function USB stuff but it'd be a good start I expect.

Yeah, collapsing devices make sense.

Sakari is working on a properties API to be added to Media Controller entities/interfaces/links. Perhaps he could add this requirement to the API he's working with.

Regards,
Mauro

#23 lore raw

Hi Mauro and Mark,

Quoted 25 lines · Mauro Carvalho Chehab

Em Tue, 20 Oct 2015 14:22:09 +0100
Mark Brown <[email protected]> escreveu:

Quoted 22 lines · Liam Girdwood
Quoted 7 lines · Shuah Khan
Quoted 5 lines

You can find the media graph that includes ALSA Mixer Function and Control Interface (Device: /dev/snd/controlC2), PCM Audio Capture Function and PCM Audio Capture interface (Device: /dev/snd/pcmC2D0c) at:

Quoted 6 lines
Quoted 1 line

https://drive.google.com/open?id=0B0NIL0BQg-AlbjZkcElJb2RtVk0

Looks nice. Fwiw, the graph will probably get a lot bigger when we show the audio DSP and codec paths (as DSPs and codecs have multiple muxes and mixers). It may be good to have a cmd line option to stop at certain nodes on the graph to avoid over populated/complex graphs ?

Having the ability to describe which physical device things are part of would probably help a lot with the rendering here - I'm guessing that a lot of filtering is likely to be things like collapsing down devices not being looked at right now into a single block in the graph and it'd probably help with user comprehesibilty provide a mapping back to things they can see in the physical world.

Quoted 8 lines
Quoted 2 lines

This doesn't entirely correspond with struct device due to MFD and multi-function USB stuff but it'd be a good start I expect.

Yeah, collapsing devices make sense.

Sakari is working on a properties API to be added to Media Controller entities/interfaces/links. Perhaps he could add this requirement to the API he's working with.

I think that's entirely possible. It's up to how we want to describe devices. I wonder if we could e.g. add a sysfs path as a property there. The user would then need to check which entities have equal sysfs paths.

Signature · 4 lines
Regards,

Sakari Ailus
e-mail: [email protected]	XMPP: [email protected]
#24 lore raw
In reply to Mauro Carvalho Chehab

Hi Mauro and Mark,

(Fixed Mark's e-mail.)

Quoted 25 lines · Mauro Carvalho Chehab

Em Tue, 20 Oct 2015 14:22:09 +0100
Mark Brown <[email protected]> escreveu:

Quoted 22 lines · Liam Girdwood
Quoted 7 lines · Shuah Khan
Quoted 5 lines

You can find the media graph that includes ALSA Mixer Function and Control Interface (Device: /dev/snd/controlC2), PCM Audio Capture Function and PCM Audio Capture interface (Device: /dev/snd/pcmC2D0c) at:

Quoted 6 lines
Quoted 1 line

https://drive.google.com/open?id=0B0NIL0BQg-AlbjZkcElJb2RtVk0

Looks nice. Fwiw, the graph will probably get a lot bigger when we show the audio DSP and codec paths (as DSPs and codecs have multiple muxes and mixers). It may be good to have a cmd line option to stop at certain nodes on the graph to avoid over populated/complex graphs ?

Having the ability to describe which physical device things are part of would probably help a lot with the rendering here - I'm guessing that a lot of filtering is likely to be things like collapsing down devices not being looked at right now into a single block in the graph and it'd probably help with user comprehesibilty provide a mapping back to things they can see in the physical world.

Quoted 8 lines
Quoted 2 lines

This doesn't entirely correspond with struct device due to MFD and multi-function USB stuff but it'd be a good start I expect.

Yeah, collapsing devices make sense.

Sakari is working on a properties API to be added to Media Controller entities/interfaces/links. Perhaps he could add this requirement to the API he's working with.

I think that's entirely possible. It's up to how we want to describe devices. I wonder if we could e.g. add a sysfs path as a property there. The user would then need to check which entities have equal sysfs paths.

Signature · 4 lines
Regards,

Sakari Ailus
e-mail: [email protected]	XMPP: [email protected]
#25 lore raw
In reply to Liam Girdwood

Em Tue, 20 Oct 2015 12:55:37 +0100
Liam Girdwood <[email protected]> escreveu:

Quoted 89 lines · Shuah Khan
Quoted 88 lines
Quoted 71 lines · Mauro Carvalho Chehab

Hi Mark/Liam,

Em Mon, 19 Oct 2015 17:27:56 +0100
Liam Girdwood <[email protected]> escreveu:

Quoted 22 lines · Mark Brown
Quoted 21 lines · Mauro Carvalho Chehab
Quoted 19 lines

We did a Media Controller Summit in July. During the summit, it was pointed that other subsystems like IIO (Linux Industrial I/O) have similar needs and also need to track complex pipelines.

The goal of this tech topic is to present the MC next generation concepts and how the patches to support MC was written, showing some pipeline examples.

The next gen support patches were written in August, with some additional patches written after that, and it has support for DVB and for one ALSA driver (snd-usb-audio).

The patches are not merged yet at the media development tree and at -next, as one of the core developers of the media controller is missing time to review patches. So, while it was originally targeted to be merged on Kernel 4.4, it is likely that this will be merged only for Kernel 4.5, in order to give him more time to review the patchset.

Quoted 7 lines
Quoted 7 lines

One fun thing on this that came up at the audio meeting we had in Dublin after ELC-E is that there is some desire to export the internal topology of audio devices to userspace using the same format that we're using to load topology information for DSP firmwares (see include/sound/soc-topoloy.h). If we go ahead with that it'd create an alternative topology ABI, it seems like we should at least consider if it's worth trying to share things here.

Thanks for the feedback. I was wanting to go to ELC-E this year due to ALSA summit, but, unfortunately, I had 3 other trips happening on a one month interval (two of them are 28+28 hours of trip). So, I had to give up going to ELC-E.

Quoted 31 lines
Quoted 5 lines

I've not had a chance to look at the new media controller work yet, I've CCed Lars and Liam who have more knowledge in this area. Unfortunately I don't think any of the people working directly on this will be at KS this year.

Just to give a little more background (for any non audio folks), there are many audio DSPs today that can load their topology and other capabilities at runtime depending on the target device (phone, laptop, PC, etc) and the audio use cases.

Our intention is to also have a single audio driver per DSP family (i.e. single skylake audio DSP driver to cover all skylake based devices). The DSP driver would load it's topology data alongside the DSP firmware so that the driver topology (i.e. graph, controls, capabilities) would be aligned with the DSP firmware topology.

The process for creating the topology data involves either compiling a text topology file to the binary topology file (used by the kernel) or using a new ALSA C API to generate the binary topology file (as part of a firmware toolchain). The topology text compiler and C API are both upstream.

We are currently implementing de-compiler support to the userspace compiler in order to convert the binary topology files into text files for debug and development purposes.

Ideally I'd like to be able to dump the topology data using the same format that it used for topology loading. That way we could easily feed them into the de-compiler for development/debug.

Quoted 9 lines
Quoted 4 lines

Having a simple mechanism for dumping topology data is important as we will need to be able to easily dump this data on Android and Chrome alongside regular Linux for development and debug purposes. i.e. cat /sys/some/dump/file > audio_topology

Well, it is not hard to get the topology using the media controller. I wrote a testing program using it at:
http://git.linuxtv.org/cgit.cgi/mchehab/experimental-v4l-utils.git/tree/contrib/test/mc_nextgen_test.c

Adding to what Mauro said, I have a patch series out that updates ASLA and AU0828 drivers (Hauppauge Win TV HVR 950Q Hybrid USB TV Stick) to share tuner. I am speaking about this work at the KLF on Oct 26th.

Quoted 8 lines
Quoted 8 lines

http://korealinuxforum2015.sched.org/event/f59175cc49f866dbc9bebc36f86fe663#.ViVWjGwVhBd

You can find the media graph that includes ALSA Mixer Function and Control Interface (Device: /dev/snd/controlC2), PCM Audio Capture Function and PCM Audio Capture interface (Device: /dev/snd/pcmC2D0c) at:

https://drive.google.com/open?id=0B0NIL0BQg-AlbjZkcElJb2RtVk0

Shuah,

I guess the links for the ALSA part are not ok. On your graph, it is showing:

	ATV decoder [PAD 3] --> Audio Mixer
			    --> Audio Capture

I suspect that the right graph would be, instead:

	ATV decoder [PAD 3] --> Audio Mixer --> Audio Capture
Quoted 4 lines

Looks nice. Fwiw, the graph will probably get a lot bigger when we show the audio DSP and codec paths (as DSPs and codecs have multiple muxes and mixers).

Yeah, the graph is simplified, as for now we only needed to represent the parts of the graph that are associated with the ALSA interfaces.

Quoted 2 lines

It may be good to have a cmd line option to stop at certain nodes on the graph to avoid over populated/complex graphs ?

Yes, it makes sense to have some ways to simplify the graph plot. Yet, I'm not sure yet what would be the best ways to simplify the graph when plotted, as this is not a simple task.

For example, on the au0828/snd-usb-audio graph, the dvb-demux entity actually has a lot more than 5 source PADs. The demux basically gets a MPEG-TS stream and splits audio, video and data channels on different I/O outputs. Most drivers (including au0828) have 256 outputs. Each node connected to two or more I/O entities. On an USB driver like this one, those outputs are seen via two device nodes, but TV sets and Set Top Boxes provide ways to wire the output of the demux to the aSoC and to the DRM driver, in order to be able to output sound and video directly. On some hardware, it is even possible to reboot the CPU keeping those wires while the CPU reboots. So, the user may not even notice if the CPU reboots due to some trouble.

So, we need all outputs of the demux to be controlled by the Media Controller, for it to allow dynamically route audio and video to the right entities at the ALSA and DRM parts of the graph.

The au0828 graphs that Shuah and I have were generated with a Kernel hack patch that reduces the number of outputs to just 5, just to make the graph visually readable. Of course, such patch should not be sent upstream, and we need to find a solution on userspace to simplify the graph on plots.

There are some ways that we could do that:

1) we could group entities/interfaces per subsystem. So, if all the user wants is to see ALSA-related nodes, it will filter out the other nodes (eventually, keeping the nodes that are directly connected to ALSA but belongs to the other subsystems);

2) Group the entities that have the same routes (like DVB demux output PADs that are routed to the same userspace interfaces);

3) Hide unused entities;

4) Have a property describing the "level" of the entity. By default, the tool would only show entities that are above a certain level.

We could, of course, add knowledge at the userspace tool for it to simplify some specific entity types, but I would prefer to keep the userspace tools more generic and add some properties for each entity/interface that would allow the tool to automatically simplify the graph based on the hints provided by the Kernel.

For now, we're more focused on adding the proper Kernel support for MC, but I'm sure we'll need to work at an userspace library afterwards, providing such capabilities.

Regards,
Mauro

#26 lore raw
Quoted 6 lines · Mauro Carvalho Chehab
Quoted 3 lines

Looks nice. Fwiw, the graph will probably get a lot bigger when we show the audio DSP and codec paths (as DSPs and codecs have multiple muxes and mixers).

Yeah, the graph is simplified, as for now we only needed to represent the parts of the graph that are associated with the ALSA interfaces.

So this doesn't expose the graphs inside USB-audio chip but rather only the mixer elements parsed by ALSA driver, right? That's what I understood from Shuah's patchset, so far.

Quoted 29 lines
Quoted 2 lines

It may be good to have a cmd line option to stop at certain nodes on the graph to avoid over populated/complex graphs ?

Yes, it makes sense to have some ways to simplify the graph plot. Yet, I'm not sure yet what would be the best ways to simplify the graph when plotted, as this is not a simple task.

For example, on the au0828/snd-usb-audio graph, the dvb-demux entity actually has a lot more than 5 source PADs. The demux basically gets a MPEG-TS stream and splits audio, video and data channels on different I/O outputs. Most drivers (including au0828) have 256 outputs. Each node connected to two or more I/O entities. On an USB driver like this one, those outputs are seen via two device nodes, but TV sets and Set Top Boxes provide ways to wire the output of the demux to the aSoC and to the DRM driver, in order to be able to output sound and video directly. On some hardware, it is even possible to reboot the CPU keeping those wires while the CPU reboots. So, the user may not even notice if the CPU reboots due to some trouble.

So, we need all outputs of the demux to be controlled by the Media Controller, for it to allow dynamically route audio and video to the right entities at the ALSA and DRM parts of the graph.

The au0828 graphs that Shuah and I have were generated with a Kernel hack patch that reduces the number of outputs to just 5, just to make the graph visually readable. Of course, such patch should not be sent upstream, and we need to find a solution on userspace to simplify the graph on plots.

Quoted 6 lines

There are some ways that we could do that:

1) we could group entities/interfaces per subsystem. So, if all the user wants is to see ALSA-related nodes, it will filter out the other nodes (eventually, keeping the nodes that are directly connected to ALSA but belongs to the other subsystems);

This actually made me wonder how all things are managed. Are all mapped into the same topology, or they are individual trees (e.g. per device object)?

thanks,

Takashi

#27 lore raw

Em Tue, 20 Oct 2015 17:47:24 +0200
Takashi Iwai <[email protected]> escreveu:

Quoted 6 lines
Quoted 4 lines · Mauro Carvalho Chehab
Quoted 3 lines

Looks nice. Fwiw, the graph will probably get a lot bigger when we show the audio DSP and codec paths (as DSPs and codecs have multiple muxes and mixers).

Quoted 6 lines
Quoted 2 lines

Yeah, the graph is simplified, as for now we only needed to represent the parts of the graph that are associated with the ALSA interfaces.

So this doesn't expose the graphs inside USB-audio chip but rather only the mixer elements parsed by ALSA driver, right? That's what I understood from Shuah's patchset, so far.

Yes, Shuah's patch set is only exposing the minimal stuff that it is currently needed to identify if there are shared resources with the DVB and V4L2/ALSA usage.

Adding support for the graphs inside the USB-audio chip is simple, though. Basically, one call to create the entities and another one to create the links between two entities.

Quoted 32 lines
Quoted 32 lines
Quoted 3 lines

It may be good to have a cmd line option to stop at certain nodes on the graph to avoid over populated/complex graphs ?

Yes, it makes sense to have some ways to simplify the graph plot. Yet, I'm not sure yet what would be the best ways to simplify the graph when plotted, as this is not a simple task.

For example, on the au0828/snd-usb-audio graph, the dvb-demux entity actually has a lot more than 5 source PADs. The demux basically gets a MPEG-TS stream and splits audio, video and data channels on different I/O outputs. Most drivers (including au0828) have 256 outputs. Each node connected to two or more I/O entities. On an USB driver like this one, those outputs are seen via two device nodes, but TV sets and Set Top Boxes provide ways to wire the output of the demux to the aSoC and to the DRM driver, in order to be able to output sound and video directly. On some hardware, it is even possible to reboot the CPU keeping those wires while the CPU reboots. So, the user may not even notice if the CPU reboots due to some trouble.

So, we need all outputs of the demux to be controlled by the Media Controller, for it to allow dynamically route audio and video to the right entities at the ALSA and DRM parts of the graph.

The au0828 graphs that Shuah and I have were generated with a Kernel hack patch that reduces the number of outputs to just 5, just to make the graph visually readable. Of course, such patch should not be sent upstream, and we need to find a solution on userspace to simplify the graph on plots.

There are some ways that we could do that:

Quoted 8 lines
Quoted 4 lines

1) we could group entities/interfaces per subsystem. So, if all the user wants is to see ALSA-related nodes, it will filter out the other nodes (eventually, keeping the nodes that are directly connected to ALSA but belongs to the other subsystems);

This actually made me wonder how all things are managed. Are all mapped into the same topology, or they are individual trees (e.g. per device object)?

It was mapped as one tree by device. That makes easier to handle the links between two entities that may belong to different subsystems.

In other words, if you have two USB sticks, each one will have its own media controller. All stuff inside each USB stick will belong to the same graph.

Regards,
Mauro

#28 lore raw
In reply to Mauro Carvalho Chehab
Quoted 7 lines · Mauro Carvalho Chehab

Em Tue, 20 Oct 2015 12:55:37 +0100

Shuah,

I guess the links for the ALSA part are not ok. On your graph, it is showing:

Quoted 7 lines
        ATV decoder [PAD 3] --> Audio Mixer
                            --> Audio Capture

I suspect that the right graph would be, instead:

        ATV decoder [PAD 3] --> Audio Mixer --> Audio Capture

Mauro,

Thanks for the find Fixed now in patch v2 series. Please find the media graphs at

https://drive.google.com/open?id=0B0NIL0BQg-AlLWE3SzAxazBJWm8

thanks,
-- Shuah

#29 lore raw
In reply to Mauro Carvalho Chehab
Quoted 11 lines · Mauro Carvalho Chehab

Liam Girdwood <[email protected]> escreveu:

Quoted 4 lines

Having a simple mechanism for dumping topology data is important as we will need to be able to easily dump this data on Android and Chrome alongside regular Linux for development and debug purposes. i.e. cat /sys/some/dump/file > audio_topology

Well, it is not hard to get the topology using the media controller. I wrote a testing program using it at:
http://git.linuxtv.org/cgit.cgi/mchehab/experimental-v4l-utils.git/tree/contrib/test/mc_nextgen_test.c

Quoted 6 lines

The only library optional dependency is libudev, used to get the device node names. The code falls back to use /sys/dev/char/ and /dev to find the device names when compiled without libudev.

So, it should be easy to provide a cat-like program that would be dumping the audio topology on some file.

It's still a program that has to be built for and installed on the device under test which is the big bar for a lot of users (consider the case where an audio expert is doing system tuning on a device using a firmware image supplied from elsewhere, it may be difficult for them to build new programs for the image or request that they be included in the image).

Quoted 7 lines
Quoted 6 lines

What I'd like to propose is that we support both mechanisms for dumping the audio topology data. :-

1) Simple file dump using same format that topology is loaded. Used by kernel/firmware developers only when media controller userspace not available. Enabled by kernel Kconfig debug option.

Quoted 6 lines

Yeah, it sounds reasonable to have a mechanism like that just for debug purposes.

Quoted 1 line

2) Media controller API used by everyone else.

Makes sense to me.

My main concern here is ending up with two different machine parsable formats for exporting the topology information to userspace - it's potentially a bit confusing for people to know which one to pick and might end up needing multiple tools and libraries to parse depending on the situation. It would be much nicer if we could get a consistent format between the two.

#30 lore raw

Em Tue, 20 Oct 2015 14:13:30 +0100
Mark Brown <[email protected]> escreveu:

Quoted 19 lines · Mauro Carvalho Chehab
Quoted 11 lines

Liam Girdwood <[email protected]> escreveu:

Quoted 4 lines

Having a simple mechanism for dumping topology data is important as we will need to be able to easily dump this data on Android and Chrome alongside regular Linux for development and debug purposes. i.e. cat /sys/some/dump/file > audio_topology

Well, it is not hard to get the topology using the media controller. I wrote a testing program using it at:
http://git.linuxtv.org/cgit.cgi/mchehab/experimental-v4l-utils.git/tree/contrib/test/mc_nextgen_test.c

Quoted 6 lines

The only library optional dependency is libudev, used to get the device node names. The code falls back to use /sys/dev/char/ and /dev to find the device names when compiled without libudev.

So, it should be easy to provide a cat-like program that would be dumping the audio topology on some file.

Quoted 6 lines

It's still a program that has to be built for and installed on the device under test which is the big bar for a lot of users (consider the case where an audio expert is doing system tuning on a device using a firmware image supplied from elsewhere, it may be difficult for them to build new programs for the image or request that they be included in the image).

Well, with such assumption, even having a debug option would be a problem, as the firmware image may not be compiled with such option.

I think that it is actually easier to cross-compile a program with all libraries required statically linked and then copy it to the target than convincing the firmware image manufacturer to send an image compiled with some additional options.

That's said, it shouldn't be hard to add some debugfs module that would output the MC topology.

Quoted 12 lines
Quoted 11 lines
Quoted 6 lines

What I'd like to propose is that we support both mechanisms for dumping the audio topology data. :-

1) Simple file dump using same format that topology is loaded. Used by kernel/firmware developers only when media controller userspace not available. Enabled by kernel Kconfig debug option.

Yeah, it sounds reasonable to have a mechanism like that just for debug purposes.

Quoted 1 line

2) Media controller API used by everyone else.

Quoted 8 lines
Quoted 1 line

Makes sense to me.

My main concern here is ending up with two different machine parsable formats for exporting the topology information to userspace - it's potentially a bit confusing for people to know which one to pick and might end up needing multiple tools and libraries to parse depending on the situation. It would be much nicer if we could get a consistent format between the two.

I'm open to suggestions. The advantage of the MC next gen API is that it is subsystem independent, and we're adding some ways for an already existing MC graph to be used by other subsystems. So, it can provide an end-to-end graph that represents how the stream is wired on the entire Kernel.

This is a requirement for some sorts of usages, where userspace may need to change the stream path on more than one subsystem.

It also helps to lock resources that may be shared by more than one subsystem. There are such cases between V4L2 and DVB subsystems, for things like TV tuner. There are also such cases between V4L2 and DRM subsystems, where the same video codec could be used by both subsystems, but not at the same time.

So, I guess it is easier to extend the MC to fulfill the needs for ALSA than doing the reverse or writing some other solution.

Regards,
Mauro

#31 lore raw
Quoted 9 lines · Mauro Carvalho Chehab

Mark Brown <[email protected]> escreveu:

Quoted 6 lines

It's still a program that has to be built for and installed on the device under test which is the big bar for a lot of users (consider the case where an audio expert is doing system tuning on a device using a firmware image supplied from elsewhere, it may be difficult for them to build new programs for the image or request that they be included in the image).

Quoted 7 lines

Well, with such assumption, even having a debug option would be a problem, as the firmware image may not be compiled with such option.

I think that it is actually easier to cross-compile a program with all libraries required statically linked and then copy it to the target than convincing the firmware image manufacturer to send an image compiled with some additional options.

No, not normally - this is usually within OEMs and in places where this is a problem you've typically got separate teams looking after userspace and kernel delivery. The kernel team is providing the drivers, the userspace team isn't really involved and so it's more natural to work with the kernel people. It's kind of a specific need but it's surprisingly common, and if it's just a debugfs thing there's a reasonable chance someone wanted debugfs anyway.

Quoted 7 lines
Quoted 6 lines

My main concern here is ending up with two different machine parsable formats for exporting the topology information to userspace - it's potentially a bit confusing for people to know which one to pick and might end up needing multiple tools and libraries to parse depending on the situation. It would be much nicer if we could get a consistent format between the two.

Quoted 5 lines

I'm open to suggestions. The advantage of the MC next gen API is that it is subsystem independent, and we're adding some ways for an already existing MC graph to be used by other subsystems. So, it can provide an end-to-end graph that represents how the stream is wired on the entire Kernel.

Yeah, I like the idea behind MC - I think what I'm suggesting is mainly looking at the topology binary format that has been defined for the DSPs and seeing if it can be reused in the MC ABI (or if there's some blockers to reuse which might also impact the topology code in its current use cases which would be just as valuable).

#32 lore raw
In reply to Liam Girdwood

Hi Liam,

Quoted 68 lines · On Monday 19 October 2015 17:27:56 Liam Girdwood wrote:
Quoted 28 lines · Mark Brown
Quoted 12 lines · Mauro Carvalho Chehab

We did a Media Controller Summit in July. During the summit, it was pointed that other subsystems like IIO (Linux Industrial I/O) have similar needs and also need to track complex pipelines.

The goal of this tech topic is to present the MC next generation concepts and how the patches to support MC was written, showing some pipeline examples.

The next gen support patches were written in August, with some additional patches written after that, and it has support for DVB and for one ALSA driver (snd-usb-audio).

Quoted 6 lines

The patches are not merged yet at the media development tree and at -next, as one of the core developers of the media controller is missing time to review patches. So, while it was originally targeted to be merged on Kernel 4.4, it is likely that this will be merged only for Kernel 4.5, in order to give him more time to review the patchset.

One fun thing on this that came up at the audio meeting we had in Dublin after ELC-E is that there is some desire to export the internal topology of audio devices to userspace using the same format that we're using to load topology information for DSP firmwares (see include/sound/soc-topoloy.h). If we go ahead with that it'd create an alternative topology ABI, it seems like we should at least consider if it's worth trying to share things here.

Quoted 4 lines

I've not had a chance to look at the new media controller work yet, I've CCed Lars and Liam who have more knowledge in this area. Unfortunately I don't think any of the people working directly on this will be at KS this year.

Just to give a little more background (for any non audio folks), there are many audio DSPs today that can load their topology and other capabilities at runtime depending on the target device (phone, laptop, PC, etc) and the audio use cases.

Our intention is to also have a single audio driver per DSP family (i.e. single skylake audio DSP driver to cover all skylake based devices). The DSP driver would load it's topology data alongside the DSP firmware so that the driver topology (i.e. graph, controls, capabilities) would be aligned with the DSP firmware topology.

The process for creating the topology data involves either compiling a text topology file to the binary topology file (used by the kernel) or using a new ALSA C API to generate the binary topology file (as part of a firmware toolchain). The topology text compiler and C API are both upstream.

We are currently implementing de-compiler support to the userspace compiler in order to convert the binary topology files into text files for debug and development purposes.

Ideally I'd like to be able to dump the topology data using the same format that it used for topology loading. That way we could easily feed them into the de-compiler for development/debug.

Having a simple mechanism for dumping topology data is important as we will need to be able to easily dump this data on Android and Chrome alongside regular Linux for development and debug purposes. i.e. cat /sys/some/dump/file > audio_topology

I'm also supportive of having a higher level and more elegant topology query interface for non kernel/firmware developers using media controller.

Quoted 6 lines

What I'd like to propose is that we support both mechanisms for dumping the audio topology data. :-

1) Simple file dump using same format that topology is loaded. Used by kernel/firmware developers only when media controller userspace not available. Enabled by kernel Kconfig debug option.

I wouldn't be against an option to dump the topology of MC devices through a file to capture the topology on a remote system that doesn't have the necessary tools installed and analyze it on a development machine. I'm however not saying that such a feature should be a hard requirement for MC as I haven't evaluated the complexity yet. Using the same format as the ALSA DSP firmware topology binary would probably not be possible though, as I expect that format to be quite ALSA-specific.

Quoted 1 line

2) Media controller API used by everyone else.

Signature · 3 lines
Regards,

Laurent Pinchart
#33 lore raw
Quoted 78 lines · Laurent Pinchart

Hi Liam,

Quoted 68 lines · On Monday 19 October 2015 17:27:56 Liam Girdwood wrote:
Quoted 28 lines · Mark Brown
Quoted 12 lines · Mauro Carvalho Chehab

We did a Media Controller Summit in July. During the summit, it was pointed that other subsystems like IIO (Linux Industrial I/O) have similar needs and also need to track complex pipelines.

The goal of this tech topic is to present the MC next generation concepts and how the patches to support MC was written, showing some pipeline examples.

The next gen support patches were written in August, with some additional patches written after that, and it has support for DVB and for one ALSA driver (snd-usb-audio).

Quoted 6 lines

The patches are not merged yet at the media development tree and at -next, as one of the core developers of the media controller is missing time to review patches. So, while it was originally targeted to be merged on Kernel 4.4, it is likely that this will be merged only for Kernel 4.5, in order to give him more time to review the patchset.

One fun thing on this that came up at the audio meeting we had in Dublin after ELC-E is that there is some desire to export the internal topology of audio devices to userspace using the same format that we're using to load topology information for DSP firmwares (see include/sound/soc-topoloy.h). If we go ahead with that it'd create an alternative topology ABI, it seems like we should at least consider if it's worth trying to share things here.

Quoted 4 lines

I've not had a chance to look at the new media controller work yet, I've CCed Lars and Liam who have more knowledge in this area. Unfortunately I don't think any of the people working directly on this will be at KS this year.

Just to give a little more background (for any non audio folks), there are many audio DSPs today that can load their topology and other capabilities at runtime depending on the target device (phone, laptop, PC, etc) and the audio use cases.

Our intention is to also have a single audio driver per DSP family (i.e. single skylake audio DSP driver to cover all skylake based devices). The DSP driver would load it's topology data alongside the DSP firmware so that the driver topology (i.e. graph, controls, capabilities) would be aligned with the DSP firmware topology.

The process for creating the topology data involves either compiling a text topology file to the binary topology file (used by the kernel) or using a new ALSA C API to generate the binary topology file (as part of a firmware toolchain). The topology text compiler and C API are both upstream.

We are currently implementing de-compiler support to the userspace compiler in order to convert the binary topology files into text files for debug and development purposes.

Ideally I'd like to be able to dump the topology data using the same format that it used for topology loading. That way we could easily feed them into the de-compiler for development/debug.

Having a simple mechanism for dumping topology data is important as we will need to be able to easily dump this data on Android and Chrome alongside regular Linux for development and debug purposes. i.e. cat /sys/some/dump/file > audio_topology

I'm also supportive of having a higher level and more elegant topology query interface for non kernel/firmware developers using media controller.

Quoted 6 lines

What I'd like to propose is that we support both mechanisms for dumping the audio topology data. :-

1) Simple file dump using same format that topology is loaded. Used by kernel/firmware developers only when media controller userspace not available. Enabled by kernel Kconfig debug option.

Quoted 7 lines

I wouldn't be against an option to dump the topology of MC devices through a file to capture the topology on a remote system that doesn't have the necessary tools installed and analyze it on a development machine. I'm however not saying that such a feature should be a hard requirement for MC as I haven't evaluated the complexity yet. Using the same format as the ALSA DSP firmware topology binary would probably not be possible though, as I expect that format to be quite ALSA-specific.

Yeah, it is quite ALSA specific. It may just be best to just dump this data format for driver/firmware developers only if it's going to be difficult to align with MC. The internal ALSA code used to gather this data will be mostly similar for this simple dump and MC though, so we will still have a lot of codec reuse.

I think we may just have to support both formats, but stick a big fat warning about the simple developer format is not for general use and will change without warning etc. It's intended that the topology compiler will be the only tool that will read this format (when it de-compiles ALSA topology data).

Liam

Quoted 3 lines
Quoted 2 lines

2) Media controller API used by everyone else.

#34 lore raw
In reply to Laurent Pinchart
Quoted 4 lines · Laurent Pinchart

not saying that such a feature should be a hard requirement for MC as I haven't evaluated the complexity yet. Using the same format as the ALSA DSP firmware topology binary would probably not be possible though, as I expect that format to be quite ALSA-specific.

It'd definitely need extension but hopefully there's enough space in there for doing that.

#35 lore raw
In reply to Liam Girdwood
Quoted 3 lines · Liam Girdwood

We are currently implementing de-compiler support to the userspace compiler in order to convert the binary topology files into text files for debug and development purposes.

Just wondering: could this topology be represented/dumped as a DT?

Gr{oetje,eeting}s,

Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]

In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds

#36 lore raw
In reply to Mauro Carvalho Chehab
Quoted 122 lines · Mauro Carvalho Chehab

Em Mon, 12 Oct 2015 15:01:37 -0400
Theodore Ts'o <[email protected]> escreveu:

Quoted 6 lines

This is an initial draft for the upcoming kernel summit.

It is still very drafty; in particular, just because a topic has been listed, it may turn out that the topic has been resolved off-line, or otherwise overtaken by events, in which case we will drop it. In addition, we're also in the process of sending queries to the

proposed

Quoted 17 lines

topic leaders to see if they are willing to kick off the discussion, and so that is subject to change as well.

I'd appreciate comments about any topics that you think might be missing and that would be worth our discussing. In addition, if you have strong opinions moving a topic from the technical session day to the core day or vice versa, please make the case for the change.

There will be some TBD slots which can be scheduled at the last minute, but I'd like to get at least one full track of the technical sessions and most of the core sessions scheduled.

Many thanks,

- Ted

Quoted 7 lines

Kernel Summit Agenda
October 26-28, 2015
DRAFT

Web Version Agenda:

* Monday: Workshops and break out sessions (overlap with Korea Linux

Forum)

Quoted 11 lines

* Tuesday: Dual-track technical sessions
* Wednesday: Invite-only core attendees' plenary sessions

Dual-Track Technical sessions (Oct. 27th)
=========================================

9:00 - Welcome and agenda bashing
9:30 - Topic A1 | Topic B1
10:00 - Topic A2 | Topic B2
10:30 - Break
11:00 - Topic A3 | Topic B3

Quoted 11 lines
11:30   - Topic A4                   | Topic B4
12:00   - Topic A5                   | Topic B5
12:30   - Lunch
1:30    - Topic A6                   | Topic B6
2:00    - Topic A7                   | Topic B7
2:30    - Break
3:00    - Topic A8                   | Topic B8
3:30    - Topic A9                   | Topic B9
4:00    - Topic A10                  | Topic B10
4:30    - Break
5:00    - Kernel Summit Tech Day -- What went well, what can we do

better?

Quoted 8 lines

5:30 - Group Photograph (for core day attendees)
6:30 - Dinner

Potential Tech topics
=====================

1. Mainline kernel on a cellphone (need someone to lead?)
2. Semantics of MMIO mapping attributes across archs (Benjamin

Herrenschmidt)

Quoted 2 lines

3. GPIO API/ABI discussion (need someone to lead?)
4. System-wide interface to specify the level of PM tuning (Rafael J.

Wysocki)

Quoted 11 lines

5. Giving freezer well-defined semantics (Jiri Kosina)
6. IRQ affinity (Christoph Hellwig)
7. benchmarking and performance trends (Chris Mason)
8. FPGAs and how to program them from kernel (need somone to lead?
9. Overlays and file(system) unioning issues (David Howells)
10. Firmware signing (David Howells)
11. Context tracking / nohz / RCU state (Andy Lutomirski)
12. userspace infrastructure services (Stephen Hemminger)
13. Kernel support for compute-offload devices (Joerg Roedel)
14. Improving Kernel Security
15. Developer Workflow Security (Panel)

Could you please add a new topic:

16. The Next Generation of the Media Controller (Mauro Carvalho Chehab)

Along the years, we've been developing a mechanism at the Kernel to store
and present graphs to userspace, called Media Controller. The original scope of the Media Controller were to store and represent pipeline connections between hardware components on complex streaming devices, like the ones that are used for webcams on cell phones, where there are typically two camera sensors, and lots of processing units to enhance the image and to convert it into different formats.

Since the beginning, we found that it would be useful to use it some day
on other subsystems, but we were too busy doing it at V4L2.

At the end of the last year, we added support for the Media Controller on DVB, using the existing API, in order to represent hybrid TV devices with radio, analog TV (video and audio) and digital TV. However, as it didn't fit quite well, the DVB MC support was marked as broken before reaching upstream, and we designed and implemented a next generation of the API, together with userspace applications to test it.

The ultimate goal will be to support the pipelines on a TV set and Set Top Boxes. On such devices, the streaming data pipeline can cover camera
sensors, analog TV, digital TV, crypto/decrypto modules, audio and GPU. Also, they may have network interfaces that may be provided via DVB (with is very useful on Cable TV).

So, as we want/need to represent and control the full pipeline, the media
controller has to have support on different subsystems: V4L2, DVB, ALSA,
DRM and Network. It also needs a way to share a common struct between those subsystems without making them depend on each other. So, it should
use devres.

Quoted 3 lines

We did a Media Controller Summit in July. During the summit, it was pointed that other subsystems like IIO (Linux Industrial I/O) have similar needs and also need to track complex pipelines.

I was one who expressed interest as IIO maintainer. Unfortunately I am not going
to make it to the KS this year. The obvious IIO person for any discussions on this is Lars-Peter Clausen (he gets about!)

Sorry Mauro, this is something I would very much have like to hear more on/get
more involved with. Just a spot of unfortunate timing!

Quoted 24 lines

The goal of this tech topic is to present the MC next generation concepts and how the patches to support MC was written, showing some pipeline examples.

The next gen support patches were written in August, with some additional patches written after that, and it has support for DVB and for one ALSA driver (snd-usb-audio).

The patches are not merged yet at the media development tree and at -next, as one of the core developers of the media controller is missing time to review patches. So, while it was originally targeted to be merged on Kernel 4.4, it is likely that this will be merged only for Kernel 4.5, in order to give him more time to review the patchset.

PS.: I send already a [TECH TOPIC] about that on Aug, 6, and some people seemed to be interested.

_______________________________________________
Ksummit-discuss mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss

Signature · 1 line
Sent from my Android device with K-9 Mail. Please excuse my brevity.