Skip to contentSkip to conversation

PM dependencies

Ksummit-discuss Laurent Pinchart 13 participants 48 messages May 12, 2014 – May 26, 2014

lore
Share & tools
lore
#0 lore raw

Hello,

On modern systems many PM dependencies don't follow the Linux kernel device model based on parent-child relationships from a control bus point of view. For instance a GPU will need the IOMMU that services its memory requests to be powered on to perform DMA operations.

Ad-hoc solutions with subsystem-specific or even driver-specific APIs have been implemented (see omap_iommu_save_ctx and omap_iommu_restore_ctx in include/linux/omap-iommu.h for instance, that push the burden of saving and restoring the IOMMU registers to bus master drivers), but they make automatic handling of hardware resources difficult. In the IOMMU case again, the goal is to hide IOMMU handling inside the DMA mapping API and make its usage completely transparent to bus master drivers in most of the cases. An IOMMU- specific API to explicitly control IOMMU power from the bus master driver would make that goal impossible to reach.

The problem is not limited to IOMMUs. We have similar dependencies at suspend/resume time with camera interfaces for instance, where two completely unrelated device in the Linux device hierarchy (a camera interface platform device in the SoC and an I2C camera sensor) need to be suspended and resumed in a controlled order. I'm sure many more use cases exist.

Whether we can (partly) reuse existing infrastructure for this is not clear. I'd like to gather use cases, to first find out exactly how widespread the need is, and whether we can come up with a common solution or we are really faced with different classes of similarly looking but distinct issues.

Tomasz Figa (CC'ed) has recently expressed interest for this topic, if he's still interested I'd like to nominate him.

Signature · 3 lines
Regards,

Laurent Pinchart
#1 lore raw
Quoted 17 lines · Laurent Pinchart

Hello,

On modern systems many PM dependencies don't follow the Linux kernel device model based on parent-child relationships from a control bus point of view. For instance a GPU will need the IOMMU that services its memory requests to be powered on to perform DMA operations.

Ad-hoc solutions with subsystem-specific or even driver-specific APIs have been implemented (see omap_iommu_save_ctx and omap_iommu_restore_ctx in include/linux/omap-iommu.h for instance, that push the burden of saving and restoring the IOMMU registers to bus master drivers), but they make automatic handling of hardware resources difficult. In the IOMMU case again, the goal is to hide IOMMU handling inside the DMA mapping API and make its usage completely transparent to bus master drivers in most of the cases. An IOMMU- specific API to explicitly control IOMMU power from the bus master driver would make that goal impossible to reach.

Quoted 5 lines

The problem is not limited to IOMMUs. We have similar dependencies at suspend/resume time with camera interfaces for instance, where two completely unrelated device in the Linux device hierarchy (a camera interface platform device in the SoC and an I2C camera sensor) need to be suspended and resumed in a controlled order. I'm sure many more use cases exist.

We discussed this topic briefly at the Media Mini-summit. I looking into similar
use-cases in media devices. Suspend/resume of a tuner for example should only be done once as opposed multiple times by drivers that think they have exclusive access to the tuner on a media device.

I am interested in this discussion and would like to participate. Requesting nomination.

-- Shuah

Quoted 4 lines

_______________________________________________
Ksummit-discuss mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss

#2 lore raw

Em Mon, 12 May 2014 11:51:54 -0600
Shuah Khan <[email protected]> escreveu:

Quoted 25 lines
Quoted 17 lines · Laurent Pinchart

Hello,

On modern systems many PM dependencies don't follow the Linux kernel device model based on parent-child relationships from a control bus point of view. For instance a GPU will need the IOMMU that services its memory requests to be powered on to perform DMA operations.

Ad-hoc solutions with subsystem-specific or even driver-specific APIs have been implemented (see omap_iommu_save_ctx and omap_iommu_restore_ctx in include/linux/omap-iommu.h for instance, that push the burden of saving and restoring the IOMMU registers to bus master drivers), but they make automatic handling of hardware resources difficult. In the IOMMU case again, the goal is to hide IOMMU handling inside the DMA mapping API and make its usage completely transparent to bus master drivers in most of the cases. An IOMMU- specific API to explicitly control IOMMU power from the bus master driver would make that goal impossible to reach.

Quoted 5 lines

The problem is not limited to IOMMUs. We have similar dependencies at suspend/resume time with camera interfaces for instance, where two completely unrelated device in the Linux device hierarchy (a camera interface platform device in the SoC and an I2C camera sensor) need to be suspended and resumed in a controlled order. I'm sure many more use cases exist.

Quoted 8 lines

We discussed this topic briefly at the Media Mini-summit. I looking into similar
use-cases in media devices. Suspend/resume of a tuner for example should only be done once as opposed multiple times by drivers that think they have exclusive access to the tuner on a media device.

I am interested in this discussion and would like to participate. Requesting nomination.

PM on media devices is very complex, and this is broken since... ever.

From time to time, someone comes with some patch that fixes it for some specific drivers, but, after some Kernel cycles, things get broken again, because the approach taken by those patches were to add a logic inside the specific PCI (or USB) bridge driver that simply unbinds all other drivers at suspend, and re-initialize all of them at resume.

Since the end of last year, both me and Shuah are working together in order to find some strategies to fix it.

One of the problems with media devices is that they are typically multi function devices that share some resources on very complex ways. So, both suspend and resume operations should be serialized in a way that will warrant that the devices internal clocks, GPIOs, I2C buses, I2C, switches, IOMMUs etc are initialized before talking to each component of the hardware that depends on it.

On several such hardware, there are components that are optional, like IR receivers, IR transmitters, etc, and are only known to be available at probe time.

Also, some devices like hybrid TV ones disables part of the hardware when in usage, as they can't typically stream analog TV (V4L2 API and v4l subsystem) while an ATSC stream is displayed (DVB API and dvb subsystem).

In other words, the driver would need to build a PM dependency tree in probe time, adjusting it in runtime, depending on what sub-devices are in use, in order to assure that the stream will continue being played after resume.

I'd like to nominate myself and Shuah for this topic.

#3 lore raw
In reply to Laurent Pinchart

Hi,

Quoted 23 lines · Laurent Pinchart

Hello,

On modern systems many PM dependencies don't follow the Linux kernel device model based on parent-child relationships from a control bus point of view. For instance a GPU will need the IOMMU that services its memory requests to be powered on to perform DMA operations.

Ad-hoc solutions with subsystem-specific or even driver-specific APIs have been implemented (see omap_iommu_save_ctx and omap_iommu_restore_ctx in include/linux/omap-iommu.h for instance, that push the burden of saving and restoring the IOMMU registers to bus master drivers), but they make automatic handling of hardware resources difficult. In the IOMMU case again, the goal is to hide IOMMU handling inside the DMA mapping API and make its usage completely transparent to bus master drivers in most of the cases. An IOMMU- specific API to explicitly control IOMMU power from the bus master driver would make that goal impossible to reach.

The problem is not limited to IOMMUs. We have similar dependencies at suspend/resume time with camera interfaces for instance, where two completely unrelated device in the Linux device hierarchy (a camera interface platform device in the SoC and an I2C camera sensor) need to be suspended and resumed in a controlled order. I'm sure many more use cases exist.

Quoted 7 lines

Whether we can (partly) reuse existing infrastructure for this is not clear. I'd like to gather use cases, to first find out exactly how widespread the need is, and whether we can come up with a common solution or we are really faced with different classes of similarly looking but distinct issues.

Tomasz Figa (CC'ed) has recently expressed interest for this topic, if he's still interested I'd like to nominate him.

Yes, I'm still interested.

IOMMUs are the use case I've been wondering about recently, but I'd like to take some time to analyze other use cases useful for Samsung platforms. I will post them in another reply to this thread as soon as I get to some interesting conclusions.

Best regards,
Tomasz

#4 lore raw
In reply to Laurent Pinchart
Quoted 10 lines · Laurent Pinchart

The problem is not limited to IOMMUs. We have similar dependencies at suspend/resume time with camera interfaces for instance, where two completely unrelated device in the Linux device hierarchy (a camera interface platform device in the SoC and an I2C camera sensor) need to be suspended and resumed in a controlled order. I'm sure many more use cases exist.

Whether we can (partly) reuse existing infrastructure for this is not clear. I'd like to gather use cases, to first find out exactly how widespread the need is, and whether we can come up with a common solution or we are really faced with different classes of similarly looking but distinct issues.

This is something ASoC has been resolving since forever with the machine drivers, Russell King recently created a generic version of the code that does the dependency deferral glue (see linux/component.h). It's not a model of elegance but it ends up getting the job done.

Anyway, I'm definitely interested in this - a couple of the subsystems I maintain are affected and it's something I've thought about a bit.

#5 lore raw

Hi Mark,

Quoted 14 lines · On Monday 12 May 2014 21:14:38 Mark Brown wrote:
Quoted 7 lines · Laurent Pinchart

The problem is not limited to IOMMUs. We have similar dependencies at suspend/resume time with camera interfaces for instance, where two completely unrelated device in the Linux device hierarchy (a camera interface platform device in the SoC and an I2C camera sensor) need to be suspended and resumed in a controlled order. I'm sure many more use cases exist.

Quoted 5 lines

Whether we can (partly) reuse existing infrastructure for this is not clear. I'd like to gather use cases, to first find out exactly how widespread the need is, and whether we can come up with a common solution or we are really faced with different classes of similarly looking but distinct issues.

Quoted 4 lines

This is something ASoC has been resolving since forever with the machine drivers, Russell King recently created a generic version of the code that does the dependency deferral glue (see linux/component.h). It's not a model of elegance but it ends up getting the job done.

That solves the probe time dependencies, but unfortunately not the runtime PM dependencies.

Quoted 2 lines

Anyway, I'm definitely interested in this - a couple of the subsystems I maintain are affected and it's something I've thought about a bit.

Signature · 3 lines
Regards,

Laurent Pinchart
#6 lore raw
Quoted 9 lines · Laurent Pinchart
Quoted 4 lines · On Monday 12 May 2014 21:14:38 Mark Brown wrote:

This is something ASoC has been resolving since forever with the machine drivers, Russell King recently created a generic version of the code that does the dependency deferral glue (see linux/component.h). It's not a model of elegance but it ends up getting the job done.

That solves the probe time dependencies, but unfortunately not the runtime PM dependencies.

It also solves the system suspend dependencies. Why don't the runtime PM dependencies just work with reference counting?

#7 lore raw

Hi Mark,

Quoted 9 lines · Mark Brown
Quoted 7 lines · Laurent Pinchart
Quoted 5 lines · On Monday 12 May 2014 21:14:38 Mark Brown wrote:

This is something ASoC has been resolving since forever with the machine drivers, Russell King recently created a generic version of the code that does the dependency deferral glue (see linux/component.h). It's not a model of elegance but it ends up getting the job done.

Quoted 6 lines
Quoted 2 lines

That solves the probe time dependencies, but unfortunately not the runtime PM dependencies.

It also solves the system suspend dependencies. Why don't the runtime PM dependencies just work with reference counting?

Runtime PM dependencies work with reference counting just fine, but only for topologies matching Linux driver model, e.g. devices with exactly one device they depend on, e.g. SPI controller and SPI devices on the bus driven by it. Add there an IOMMU and other various strange things that should be transparent to the drivers and it stops working.

I'm still investigating this issue, so more uses cases are yet to be found, but I also guess this is the purpose of this thread. Anyway, for some reason .suspend_late() and .resume_early() callbacks exist in dev_pm_ops struct and I believe that at least some of the cases "solved" by them might be related to the issue being discussed here.

Best regards,
Tomasz

#8 lore raw
Quoted 10 lines · Tomasz Figa
Quoted 2 lines · Mark Brown

It also solves the system suspend dependencies. Why don't the runtime PM dependencies just work with reference counting?

Runtime PM dependencies work with reference counting just fine, but only for topologies matching Linux driver model, e.g. devices with exactly one device they depend on, e.g. SPI controller and SPI devices on the bus driven by it. Add there an IOMMU and other various strange things that should be transparent to the drivers and it stops working.

There's no reason why runtime PM references have to follow the topology
- you do get a default reference count up to any parent (though we break that sometimes, as is the case with SPI controllers being suspended even though the devices below them are active) but there's nothing stopping references being taken outside the topology.

Quoted 5 lines

I'm still investigating this issue, so more uses cases are yet to be found, but I also guess this is the purpose of this thread. Anyway, for some reason .suspend_late() and .resume_early() callbacks exist in dev_pm_ops struct and I believe that at least some of the cases "solved" by them might be related to the issue being discussed here.

Yes, they're partly solving a particular common case for this sort of interdependency (though I guess they do also do things like allow us to make sure the hardware came back in a state where it won't be harmful to the rest of the system if we start enabling things).

#9 lore raw
Quoted 11 lines · Mark Brown
Quoted 9 lines · Tomasz Figa
Quoted 2 lines · Mark Brown

It also solves the system suspend dependencies. Why don't the runtime PM dependencies just work with reference counting?

Runtime PM dependencies work with reference counting just fine, but only for topologies matching Linux driver model, e.g. devices with exactly one device they depend on, e.g. SPI controller and SPI devices on the bus driven by it. Add there an IOMMU and other various strange things that should be transparent to the drivers and it stops working.

Quoted 5 lines

There's no reason why runtime PM references have to follow the topology
- you do get a default reference count up to any parent (though we break that sometimes, as is the case with SPI controllers being suspended even though the devices below them are active) but there's nothing stopping references being taken outside the topology.

I guess some helpers to grab/drop runtime PM references on all parts of a componentized device should resolve this? We propably don't want to bake this into the driver core since often the driver will know that it e.g. doesn't need a specific encoder block (if it's not enabled), so doesn't to grab a reference. But for simple drivers, and to get runtime PM off the ground when enabling new hardware this could be rather useful.
-Daniel

Signature · 3 lines
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch
#10 lore raw
Quoted 12 lines · On Tuesday 13 May 2014 09:43:05 Daniel Vetter wrote:
Quoted 11 lines · Mark Brown
Quoted 9 lines · Tomasz Figa
Quoted 2 lines · Mark Brown

It also solves the system suspend dependencies. Why don't the runtime PM dependencies just work with reference counting?

Runtime PM dependencies work with reference counting just fine, but only for topologies matching Linux driver model, e.g. devices with exactly one device they depend on, e.g. SPI controller and SPI devices on the bus driven by it. Add there an IOMMU and other various strange things that should be transparent to the drivers and it stops working.

Quoted 5 lines
Quoted 5 lines

There's no reason why runtime PM references have to follow the topology
- you do get a default reference count up to any parent (though we break that sometimes, as is the case with SPI controllers being suspended even though the devices below them are active) but there's nothing stopping references being taken outside the topology.

One of the issues is that we need API(s) to take/release those references. In some cases the depender doesn't have direct access to the dependee, like in the IOMMU case where the DMA mapping API hides the IOMMU. One of the questions here is whether this kind of problems should be solved with ad-hoc solutions (for instance adding PM support to the DMA mapping API in this specific case), or in a more generic fashion. Answering that will require a comprehensive list of use cases to detect common patterns.

Quoted 2 lines

I guess some helpers to grab/drop runtime PM references on all parts of a componentized device should resolve this?

That could help in some cases, but there might be ordering issues.

Quoted 5 lines

We propably don't want to bake this into the driver core since often the driver will know that it e.g. doesn't need a specific encoder block (if it's not enabled), so doesn't to grab a reference. But for simple drivers, and to get runtime PM off the ground when enabling new hardware this could be rather useful.

This won't solve the IOMMU problem, as the component framework isn't used there. I'm not advocating for a common solution to all non parent/child PM dependencies just for the sake of it, we could find out that several distinct solutions to distinct classes of issues is the way to go.

Signature · 3 lines
Regards,

Laurent Pinchart
#11 lore raw
Quoted 27 lines · Laurent Pinchart
Quoted 12 lines · On Tuesday 13 May 2014 09:43:05 Daniel Vetter wrote:
Quoted 11 lines · Mark Brown
Quoted 9 lines · Tomasz Figa
Quoted 2 lines · Mark Brown

It also solves the system suspend dependencies. Why don't the runtime PM dependencies just work with reference counting?

Runtime PM dependencies work with reference counting just fine, but only for topologies matching Linux driver model, e.g. devices with exactly one device they depend on, e.g. SPI controller and SPI devices on the bus driven by it. Add there an IOMMU and other various strange things that should be transparent to the drivers and it stops working.

Quoted 5 lines
Quoted 5 lines

There's no reason why runtime PM references have to follow the topology
- you do get a default reference count up to any parent (though we break that sometimes, as is the case with SPI controllers being suspended even though the devices below them are active) but there's nothing stopping references being taken outside the topology.

One of the issues is that we need API(s) to take/release those references. In some cases the depender doesn't have direct access to the dependee, like in the IOMMU case where the DMA mapping API hides the IOMMU. One of the questions here is whether this kind of problems should be solved with ad-hoc solutions (for instance adding PM support to the DMA mapping API in this specific case), or in a more generic fashion. Answering that will require a comprehensive list of use cases to detect common patterns.

Quoted 4 lines
Quoted 2 lines

I guess some helpers to grab/drop runtime PM references on all parts of a componentized device should resolve this?

That could help in some cases, but there might be ordering issues.

Can we use pm domains concept to solve the problem? As it exists today, it probably
can't support loosely associated devices, however, could it be extended to support
handling device groups that don't necessarily share power source and/or clock.

I started looking at this as a way to solve media device PM issues, however I haven't
had a chance to experiment with this idea.

-- Shuah

Quoted 5 lines

_______________________________________________
Ksummit-discuss mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss

#12 lore raw
Quoted 31 lines · On Tuesday 13 May 2014 08:26:09 Shuah Khan wrote:
Quoted 30 lines · Laurent Pinchart
Quoted 12 lines · On Tuesday 13 May 2014 09:43:05 Daniel Vetter wrote:
Quoted 11 lines · Mark Brown
Quoted 9 lines · Tomasz Figa
Quoted 2 lines · Mark Brown

It also solves the system suspend dependencies. Why don't the runtime PM dependencies just work with reference counting?

Runtime PM dependencies work with reference counting just fine, but only for topologies matching Linux driver model, e.g. devices with exactly one device they depend on, e.g. SPI controller and SPI devices on the bus driven by it. Add there an IOMMU and other various strange things that should be transparent to the drivers and it stops working.

Quoted 5 lines
Quoted 5 lines

There's no reason why runtime PM references have to follow the topology
- you do get a default reference count up to any parent (though we break that sometimes, as is the case with SPI controllers being suspended even though the devices below them are active) but there's nothing stopping references being taken outside the topology.

One of the issues is that we need API(s) to take/release those references. In some cases the depender doesn't have direct access to the dependee, like in the IOMMU case where the DMA mapping API hides the IOMMU. One of the questions here is whether this kind of problems should be solved with ad-hoc solutions (for instance adding PM support to the DMA mapping API in this specific case), or in a more generic fashion. Answering that will require a comprehensive list of use cases to detect common patterns.

Quoted 2 lines

I guess some helpers to grab/drop runtime PM references on all parts of a componentized device should resolve this?

Quoted 6 lines
Quoted 1 line

That could help in some cases, but there might be ordering issues.

Can we use pm domains concept to solve the problem? As it exists today, it probably can't support loosely associated devices, however, could it be extended to support handling device groups that don't necessarily share power source and/or clock.

If I understand them properly, power domains model groups of devices that share common power handling. I'm not sure how they could be used to model PM dependencies between devices. Feel free to prove me wrong though :-)

Quoted 2 lines

I started looking at this as a way to solve media device PM issues, however I haven't had a chance to experiment with this idea.

Signature · 3 lines
Regards,

Laurent Pinchart
#13 lore raw
Quoted 13 lines · Laurent Pinchart
Quoted 5 lines · On Tuesday 13 May 2014 08:26:09 Shuah Khan wrote:

Can we use pm domains concept to solve the problem? As it exists today, it probably can't support loosely associated devices, however, could it be extended to support handling device groups that don't necessarily share power source and/or clock.

If I understand them properly, power domains model groups of devices that share common power handling. I'm not sure how they could be used to model PM dependencies between devices. Feel free to prove me wrong though :-)

Quoted 2 lines

I started looking at this as a way to solve media device PM issues, however I haven't had a chance to experiment with this idea.

Right now, PM domains currently support grouping of devices that share power source. What if we extended PM domains to support a concept of logical grouping of devices that have PM dependencies with a way to specify dependency ordering?

For example, a media usb device example:
The main driver could create a logical power domain and rest of the drivers add their devices to the same domain specifying their dependencies. logical power domain pm handlers can handle ordering. In some cases, there is a need to save and restore state of tuner i2c and the existing save_state and restore_state could be used to within the scope of this logical domain.

I think PM domains can definitely be used for media device case, but I am not sure about the specific case you are looking at where there is a clearly identified master driver/device.

These are at the moment ideas and I need to do some work to test these theories. :)

-- Shuah

#14 lore raw
Quoted 13 lines · Shuah Khan
Quoted 11 lines · Laurent Pinchart
Quoted 5 lines · On Tuesday 13 May 2014 08:26:09 Shuah Khan wrote:

Can we use pm domains concept to solve the problem? As it exists today, it probably can't support loosely associated devices, however, could it be extended to support handling device groups that don't necessarily share power source and/or clock.

If I understand them properly, power domains model groups of devices that share common power handling. I'm not sure how they could be used to model PM dependencies between devices. Feel free to prove me wrong though :-)

Quoted 5 lines
Quoted 2 lines
Quoted 2 lines

I started looking at this as a way to solve media device PM issues, however I haven't had a chance to experiment with this idea.

Right now, PM domains currently support grouping of devices that share power source. What if we extended PM domains to support a concept of

Physical power domain...

Quoted 7 lines

logical grouping of devices that have PM dependencies with a way to specify dependency ordering?

For example, a media usb device example:
The main driver could create a logical power domain and rest of the drivers add their devices to the same domain specifying their dependencies. logical power domain pm handlers can handle ordering. In

... and logical power domain...

Quoted 7 lines

some cases, there is a need to save and restore state of tuner i2c and the existing save_state and restore_state could be used to within the scope of this logical domain.

I think PM domains can definitely be used for media device case, but I am not sure about the specific case you are looking at where there is a clearly identified master driver/device.

Sure, it can be done.

Quoted 2 lines

These are at the moment ideas and I need to do some work to test these theories. :)

.. and we also have clock power domains (cfr. pm_clk_notify()).

Now we do have hardware that would benefit from all three of the above. AFAIK a device can be a member of only one power domain, through the struct device.pm_domain pointer. Can/should this be extended, cfr. the RPM_GET_CALLBACK() stack for Runtime PM (but we do want to call into more than one of them).

I'm still too pm_domain-illiterate to say much more about this though...

Gr{oetje,eeting}s,

Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]

In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds

#15 lore raw
In reply to Mark Brown
Quoted 12 lines · Mark Brown
Quoted 10 lines · Tomasz Figa
Quoted 2 lines · Mark Brown

It also solves the system suspend dependencies. Why don't the runtime PM dependencies just work with reference counting?

Runtime PM dependencies work with reference counting just fine, but only for topologies matching Linux driver model, e.g. devices with exactly one device they depend on, e.g. SPI controller and SPI devices on the bus driven by it. Add there an IOMMU and other various strange things that should be transparent to the drivers and it stops working.

Quoted 5 lines

There's no reason why runtime PM references have to follow the topology
- you do get a default reference count up to any parent (though we break that sometimes, as is the case with SPI controllers being suspended even though the devices below them are active) but there's nothing stopping references being taken outside the topology.

Precisely.

Quoted 10 lines
Quoted 5 lines

I'm still investigating this issue, so more uses cases are yet to be found, but I also guess this is the purpose of this thread. Anyway, for some reason .suspend_late() and .resume_early() callbacks exist in dev_pm_ops struct and I believe that at least some of the cases "solved" by them might be related to the issue being discussed here.

Yes, they're partly solving a particular common case for this sort of interdependency (though I guess they do also do things like allow us to make sure the hardware came back in a state where it won't be harmful to the rest of the system if we start enabling things).

The original reason why .suspend_late/.resume_early were introduced was to allow the same callback routines to be run for runtime and for system-wide PM. Specifically, the idea was that .suspend_late may point to the same routine as .runtime_suspend, for example, because of the analogy of the conditions in which they are called (the device is known to be not in use).

That didn't work out perfectly, but those callbacks turn out to be useful nonetheless.

And yes, I'm interested in this topic too.

Rafael (who wonders why he wasn't CCed in this thread to start with)

#16 lore raw
Quoted 13 lines · Rafael J. Wysocki
Quoted 12 lines · Mark Brown
Quoted 10 lines · Tomasz Figa
Quoted 2 lines · Mark Brown

It also solves the system suspend dependencies. Why don't the runtime PM dependencies just work with reference counting?

Runtime PM dependencies work with reference counting just fine, but only for topologies matching Linux driver model, e.g. devices with exactly one device they depend on, e.g. SPI controller and SPI devices on the bus driven by it. Add there an IOMMU and other various strange things that should be transparent to the drivers and it stops working.

Quoted 7 lines
Quoted 5 lines

There's no reason why runtime PM references have to follow the topology
- you do get a default reference count up to any parent (though we break that sometimes, as is the case with SPI controllers being suspended even though the devices below them are active) but there's nothing stopping references being taken outside the topology.

Precisely.

BTW, I guess that the problem is resume and specifically the fact that if a child device resumes, the parent will also resume automatically, but the other devices the child may depend on will not (the child's resume may need to resume them directly).

But I'm not sure why that is a problem, so can anyone please share some details?

Rafael

#17 lore raw
Quoted 20 lines · Rafael J. Wysocki
Quoted 19 lines · Rafael J. Wysocki
Quoted 17 lines · Mark Brown
Quoted 10 lines · Tomasz Figa
Quoted 2 lines · Mark Brown

It also solves the system suspend dependencies. Why don't the runtime PM dependencies just work with reference counting?

Runtime PM dependencies work with reference counting just fine, but only for topologies matching Linux driver model, e.g. devices with exactly one device they depend on, e.g. SPI controller and SPI devices on the bus driven by it. Add there an IOMMU and other various strange things that should be transparent to the drivers and it stops working.

There's no reason why runtime PM references have to follow the topology
- you do get a default reference count up to any parent (though we break that sometimes, as is the case with SPI controllers being suspended even though the devices below them are active) but there's nothing stopping references being taken outside the topology.

Quoted 8 lines
Quoted 1 line

Precisely.

BTW, I guess that the problem is resume and specifically the fact that if a child device resumes, the parent will also resume automatically, but the other devices the child may depend on will not (the child's resume may need to resume them directly).

But I'm not sure why that is a problem, so can anyone please share some details?

Thinking more about this, there seem to be two cases:

(a) The driver knows that its device (X) depends on another device (Y).
(b) The driver doesn't know that there is the dependency (there is a framework having that information, but drivers only access it indirectly, for example).

In principle (a) may be addressed by the driver itself if it is careful enough to reference count Y in addition to X during runtime PM operations. The async (system) suspend case is more problematic, because it also requies the driver of Y to wait for X to suspend (that is not required if both X and Y are "sync", but that case is kind of suboptimal).

However, (b) cannot be taken care of by the driver itself and it looks like some coordination mechanism would be necessary to address it.

I proposed something like that a few years ago (https://lkml.org/lkml/2009/8/26/485),
but people generally didn't like it and it has locking problems. Admittedly, I'd do it differently today, but if there's interest, I can refresh that approach.

Rafael

#18 lore raw
In reply to Rafael J. Wysocki
Quoted 21 lines · On Wednesday 14 May 2014 00:34:56 Rafael J. Wysocki wrote:
Quoted 12 lines · Rafael J. Wysocki
Quoted 11 lines · Mark Brown
Quoted 9 lines · Tomasz Figa
Quoted 2 lines · Mark Brown

It also solves the system suspend dependencies. Why don't the runtime PM dependencies just work with reference counting?

Runtime PM dependencies work with reference counting just fine, but only for topologies matching Linux driver model, e.g. devices with exactly one device they depend on, e.g. SPI controller and SPI devices on the bus driven by it. Add there an IOMMU and other various strange things that should be transparent to the drivers and it stops working.

Quoted 7 lines
Quoted 5 lines

There's no reason why runtime PM references have to follow the topology
- you do get a default reference count up to any parent (though we break that sometimes, as is the case with SPI controllers being suspended even though the devices below them are active) but there's nothing stopping references being taken outside the topology.

Precisely.

Quoted 7 lines

BTW, I guess that the problem is resume and specifically the fact that if a child device resumes, the parent will also resume automatically, but the other devices the child may depend on will not (the child's resume may need to resume them directly).

But I'm not sure why that is a problem, so can anyone please share some details?

Here are two real life examples.

1. IOMMU and bus master

Bus master devices connected to an IOMMU need the IOMMU to be powered on in order to access memory. In order to save power the IOMMU should of course be powered off as much as possible.

The tricky part here comes from the fact that the IOMMU is hidden behind the DMA API. The bus master driver can't manage the IOMMU power state eplicitly by taking/releasing references to the IOMMU device.

This could easily be solved in an ad-hoc fashion by extending the DMA mapping API, but I'm wondering whether similar issues wouldn't benefit from a common solution (I'm not sure yet what all the similar issues are, hence the topic proposal to try and gather use cases).

2. Composite devices

I'll take an embedded camera devices as an example as that's the case I know better, but the same problem occurs on the display side as well.

Camera devices on embedded systems are usually made of a camera interface inside an SoC (possibly split into several IP cores, such as a CSI - Camera Serial Interface - receiver and a backend) and one or more external devices, such as sensors, flash controller, lens controller or sometimes dedicated video processing accelerators.

The external devices are most of the time controlled through I2C (SPI is an option as well, the exact interface doesn't matter much). Those that handle video data streams (sensor, video processor) use dedicated parallel or high speed serial busses.

Most of the external devices require an input clock. The clock can be fixed (easy), provided by a dedicated chip or IP core in the SoC (easy as well), sometimes by the SoC camera interface or even by one of the other external devices (for instance the lens controller could be clocked by the pixel clock output by the sensor - that's a bit far-fetched - or, more boring but equally annoying, I've seen a camera interface in an SoC that required the pixel clock output by the sensor to be running in order to complete its reset sequence).

On OMAP3 systems implementing a camera the image sensor is usually supplied with a clock output by the ISP (Image Signal Processor, the camera interface). When resuming the system from suspend (assuming the ISP was capturing video when the system got suspended), it's important to restart the ISP first and only then restart the sensor. To ensure that the order is followed the ISP driver resume path first restarts the ISP and then calls the sensor driver to start the video stream. The sensor driver will need to send I2C messages to the device in order to start it, which requires the I2C controller to be resumed. As the ISP is a platform device at the same level as the I2C controller, that ordering is not guaranteed.

Furthermore, if the sensor is resumed first, it might try to access the device, which requires the clock output by the ISP to be available, and thus requires the ISP to be resumed. To solve this problem the ISP driver only restarts the clocks in its PM resume callback, and restarts the video stream (following the sequence described above) in its PM complete callback.

When adding more external devices to the mix the problem just becomes more complex, especially when the devices are chained (for instance sensor -> video processor -> ISP). The problem is similar on the display side, possibly with a different resume ordering (it should be noted that the external devices vs. internal device ordering might vary even inside the same class of devices - camera or display).

Signature · 3 lines
Regards,

Laurent Pinchart
#19 lore raw
Quoted 38 lines · Laurent Pinchart
Quoted 21 lines · On Wednesday 14 May 2014 00:34:56 Rafael J. Wysocki wrote:
Quoted 12 lines · Rafael J. Wysocki
Quoted 11 lines · Mark Brown
Quoted 9 lines · Tomasz Figa
Quoted 2 lines · Mark Brown

It also solves the system suspend dependencies. Why don't the runtime PM dependencies just work with reference counting?

Runtime PM dependencies work with reference counting just fine, but only for topologies matching Linux driver model, e.g. devices with exactly one device they depend on, e.g. SPI controller and SPI devices on the bus driven by it. Add there an IOMMU and other various strange things that should be transparent to the drivers and it stops working.

Quoted 7 lines
Quoted 5 lines

There's no reason why runtime PM references have to follow the topology
- you do get a default reference count up to any parent (though we break that sometimes, as is the case with SPI controllers being suspended even though the devices below them are active) but there's nothing stopping references being taken outside the topology.

Precisely.

Quoted 7 lines

BTW, I guess that the problem is resume and specifically the fact that if a child device resumes, the parent will also resume automatically, but the other devices the child may depend on will not (the child's resume may need to resume them directly).

But I'm not sure why that is a problem, so can anyone please share some details?

Here are two real life examples.

1. IOMMU and bus master

Bus master devices connected to an IOMMU need the IOMMU to be powered on in order to access memory. In order to save power the IOMMU should of course be powered off as much as possible.

Quoted 8 lines

The tricky part here comes from the fact that the IOMMU is hidden behind the DMA API. The bus master driver can't manage the IOMMU power state eplicitly by taking/releasing references to the IOMMU device.

This could easily be solved in an ad-hoc fashion by extending the DMA mapping API, but I'm wondering whether similar issues wouldn't benefit from a common solution (I'm not sure yet what all the similar issues are, hence the topic proposal to try and gather use cases).

I'll continue to beat the runtime PM drum...

This would also easily be solved if the bus master device, the IOMMU and the dmaengine (or the platform-specific dma driver) were all using runtime PM. e.g. bus master device is a user of dmaengine, which is a user of the IOMMU. If all are using runtime PM, the fact that a device is "in use", means runtime PM would keep them active when needed. In this example, it wouldn't matter that the bus master device doesn't know about the IOMMU. It suffices that the dmaengine driver knows about the IOMMU.

Quoted 24 lines

2. Composite devices

I'll take an embedded camera devices as an example as that's the case I know better, but the same problem occurs on the display side as well.

Camera devices on embedded systems are usually made of a camera interface inside an SoC (possibly split into several IP cores, such as a CSI - Camera Serial Interface - receiver and a backend) and one or more external devices, such as sensors, flash controller, lens controller or sometimes dedicated video processing accelerators.

The external devices are most of the time controlled through I2C (SPI is an option as well, the exact interface doesn't matter much). Those that handle video data streams (sensor, video processor) use dedicated parallel or high speed serial busses.

Most of the external devices require an input clock. The clock can be fixed (easy), provided by a dedicated chip or IP core in the SoC (easy as well), sometimes by the SoC camera interface or even by one of the other external devices (for instance the lens controller could be clocked by the pixel clock output by the sensor - that's a bit far-fetched - or, more boring but equally annoying, I've seen a camera interface in an SoC that required the pixel clock output by the sensor to be running in order to complete its reset sequence).

Quoted 10 lines

On OMAP3 systems implementing a camera the image sensor is usually supplied with a clock output by the ISP (Image Signal Processor, the camera interface). When resuming the system from suspend (assuming the ISP was capturing video when the system got suspended), it's important to restart the ISP first and only then restart the sensor. To ensure that the order is followed the ISP driver resume path first restarts the ISP and then calls the sensor driver to start the video stream. The sensor driver will need to send I2C messages to the device in order to start it, which requires the I2C controller to be resumed. As the ISP is a platform device at the same level as the I2C controller, that ordering is not guaranteed.

As you likely already know, at least on OMAP, the I2C ordering would work just fine because the I2C driver is (what I like to call) runtime PM centric. IOW, it doesn't really implement suspend/resume, instead it only does runtime PM "on demand" on a per-xfer basis. That means that the I2C driver can be used anytime during the [runtime] suspend/resume of any other device without any ordering problems because it's activated on demand.

Extending this same "runtime PM centric" view of the world to the other devices, I think that the runtime PM reference counting invovled when all the components are using runtime PM would solve this problem.

(I realize that this I2C solution doesn't solve the more general "resume ISP before sensor" problem though, but with better modeling, I think it can. More on that below...)

Quoted 5 lines

Furthermore, if the sensor is resumed first, it might try to access the device, which requires the clock output by the ISP to be available, and thus requires the ISP to be resumed. To solve this problem the ISP driver only restarts the clocks in its PM resume callback, and restarts the video stream (following the sequence described above) in its PM complete callback.

For most devices, input clocks are modeled by the clock framework (or managed by the SoC's runtime PM core), and therefore, a pm_runtime_get() (or possibly an explict clk_enable()) is used to ensure the input clock is running. In this external device example, it sounds to me like the sensor driver has no knowledge of its input clock so it has to rely on some other layer to resume things in the right order for correct functionality.

Maybe I'm wrong here (likely, since I haven't looked at the code, and am admittedly very ignorant of the camera and display subsystems) but it sounds to me like what's missing is the sensor driver having knowledge of it's input clock and/or a way for it to request it's input clock to be enabled (e.g. clk_get/clk_enable.)

Alternatively, what would proably be even better would be that the sensor driver has a reference to the actual device that provides its input clock (possibly via a DT phandle?) so that the sensor driver can simply do a pm_runtime_get() on the device providing the clock.

Quoted 6 lines

When adding more external devices to the mix the problem just becomes more complex, especially when the devices are chained (for instance sensor -> video processor -> ISP). The problem is similar on the display side, possibly with a different resume ordering (it should be noted that the external devices vs. internal device ordering might vary even inside the same class of devices - camera or display).

IMO, I still think that properly modeling the device dependenies combined with a "runtime PM centric" view of suspend/resume should allow the dependencies to be handled correctly for system suspend/resume and runtime PM.

I think what complicates things here is not the PM specifics but probably the fact that the device hierarchy (and dependencies) may be dynamic depending on many factors like which sensors are in use, post-processing, etc. etc.

Above, I suggested possibly using DT phandles to model these non parent/child relationships. That's all fine if the dependencies are not changing, but if they are dynamic, we'll probably need something different.

At least for starters though, static dependencies like this should be (relatively) easy to model in DT and combined with runtime PM refcounding on the dependencies, should be able to address the problem.

Kevin

#20 lore raw
Quoted 14 lines · Kevin Hilman

For most devices, input clocks are modeled by the clock framework (or managed by the SoC's runtime PM core), and therefore, a pm_runtime_get() (or possibly an explict clk_enable()) is used to ensure the input clock is running. In this external device example, it sounds to me like the sensor driver has no knowledge of its input clock so it has to rely on some other layer to resume things in the right order for correct functionality.

Maybe I'm wrong here (likely, since I haven't looked at the code, and am admittedly very ignorant of the camera and display subsystems) but it sounds to me like what's missing is the sensor driver having knowledge of it's input clock and/or a way for it to request it's input clock to be enabled (e.g. clk_get/clk_enable.)

Quoted 4 lines

Alternatively, what would proably be even better would be that the sensor driver has a reference to the actual device that provides its input clock (possibly via a DT phandle?) so that the sensor driver can simply do a pm_runtime_get() on the device providing the clock.

We have this problem on ASoC too and for DT converted devices we've been doing exactly as you suggest and just teaching the device it has a clock and letting it work out what to do with it. Ideally we'd be able to rely on having a clock API (better yet, the common clock API implementation) and just put the entire clock tree in there. It's not been causing us any problems thus far but it's not that widely rolled out yet either.

#21 lore raw
In reply to Kevin Hilman
Quoted 103 lines · Kevin Hilman
Quoted 38 lines · Laurent Pinchart
Quoted 21 lines · On Wednesday 14 May 2014 00:34:56 Rafael J. Wysocki wrote:
Quoted 12 lines · Rafael J. Wysocki
Quoted 11 lines · Mark Brown
Quoted 9 lines · Tomasz Figa
Quoted 2 lines · Mark Brown

It also solves the system suspend dependencies. Why don't the runtime PM dependencies just work with reference counting?

Runtime PM dependencies work with reference counting just fine, but only for topologies matching Linux driver model, e.g. devices with exactly one device they depend on, e.g. SPI controller and SPI devices on the bus driven by it. Add there an IOMMU and other various strange things that should be transparent to the drivers and it stops working.

Quoted 7 lines
Quoted 5 lines

There's no reason why runtime PM references have to follow the topology
- you do get a default reference count up to any parent (though we break that sometimes, as is the case with SPI controllers being suspended even though the devices below them are active) but there's nothing stopping references being taken outside the topology.

Precisely.

Quoted 7 lines

BTW, I guess that the problem is resume and specifically the fact that if a child device resumes, the parent will also resume automatically, but the other devices the child may depend on will not (the child's resume may need to resume them directly).

But I'm not sure why that is a problem, so can anyone please share some details?

Here are two real life examples.

1. IOMMU and bus master

Bus master devices connected to an IOMMU need the IOMMU to be powered on in order to access memory. In order to save power the IOMMU should of course be powered off as much as possible.

Quoted 8 lines

The tricky part here comes from the fact that the IOMMU is hidden behind the DMA API. The bus master driver can't manage the IOMMU power state eplicitly by taking/releasing references to the IOMMU device.

This could easily be solved in an ad-hoc fashion by extending the DMA mapping API, but I'm wondering whether similar issues wouldn't benefit from a common solution (I'm not sure yet what all the similar issues are, hence the topic proposal to try and gather use cases).

I'll continue to beat the runtime PM drum...

This would also easily be solved if the bus master device, the IOMMU and the dmaengine (or the platform-specific dma driver) were all using runtime PM. e.g. bus master device is a user of dmaengine, which is a user of the IOMMU. If all are using runtime PM, the fact that a device is "in use", means runtime PM would keep them active when needed. In this example, it wouldn't matter that the bus master device doesn't know about the IOMMU. It suffices that the dmaengine driver knows about the IOMMU.

Quoted 24 lines

2. Composite devices

I'll take an embedded camera devices as an example as that's the case I know better, but the same problem occurs on the display side as well.

Camera devices on embedded systems are usually made of a camera interface inside an SoC (possibly split into several IP cores, such as a CSI - Camera Serial Interface - receiver and a backend) and one or more external devices, such as sensors, flash controller, lens controller or sometimes dedicated video processing accelerators.

The external devices are most of the time controlled through I2C (SPI is an option as well, the exact interface doesn't matter much). Those that handle video data streams (sensor, video processor) use dedicated parallel or high speed serial busses.

Most of the external devices require an input clock. The clock can be fixed (easy), provided by a dedicated chip or IP core in the SoC (easy as well), sometimes by the SoC camera interface or even by one of the other external devices (for instance the lens controller could be clocked by the pixel clock output by the sensor - that's a bit far-fetched - or, more boring but equally annoying, I've seen a camera interface in an SoC that required the pixel clock output by the sensor to be running in order to complete its reset sequence).

Quoted 10 lines

On OMAP3 systems implementing a camera the image sensor is usually supplied with a clock output by the ISP (Image Signal Processor, the camera interface). When resuming the system from suspend (assuming the ISP was capturing video when the system got suspended), it's important to restart the ISP first and only then restart the sensor. To ensure that the order is followed the ISP driver resume path first restarts the ISP and then calls the sensor driver to start the video stream. The sensor driver will need to send I2C messages to the device in order to start it, which requires the I2C controller to be resumed. As the ISP is a platform device at the same level as the I2C controller, that ordering is not guaranteed.

As you likely already know, at least on OMAP, the I2C ordering would work just fine because the I2C driver is (what I like to call) runtime PM centric. IOW, it doesn't really implement suspend/resume, instead it only does runtime PM "on demand" on a per-xfer basis. That means that the I2C driver can be used anytime during the [runtime] suspend/resume of any other device without any ordering problems because it's activated on demand.

Quoted 3 lines

Extending this same "runtime PM centric" view of the world to the other devices, I think that the runtime PM reference counting invovled when all the components are using runtime PM would solve this problem.

I like this approach, but we should know that there are some complex issues you need to address when using it it.

For reasons, the PM core prevent runtime PM suspend during system suspend, (it invokes pm_runtime_get_noresume() for each device). Especially, this means you can't guarantee the I2C driver to hold it's device in in-activated state while the platform is in system suspend state - unless you explicitly takes care of it from a system PM callback, somehow and somewhere.

One potential help here, could be to use the new runtime PM helper functions, pm_runtime_force_suspend|resume().

Kind regards
Ulf Hansson

#22 lore raw
In reply to Kevin Hilman
Quoted 14 lines · Kevin Hilman
Quoted 5 lines

Furthermore, if the sensor is resumed first, it might try to access the device, which requires the clock output by the ISP to be available, and thus requires the ISP to be resumed. To solve this problem the ISP driver only restarts the clocks in its PM resume callback, and restarts the video stream (following the sequence described above) in its PM complete callback.

For most devices, input clocks are modeled by the clock framework (or managed by the SoC's runtime PM core), and therefore, a pm_runtime_get() (or possibly an explict clk_enable()) is used to ensure the input clock is running. In this external device example, it sounds to me like the sensor driver has no knowledge of its input clock so it has to rely on some other layer to resume things in the right order for correct functionality.

Quoted 5 lines

Maybe I'm wrong here (likely, since I haven't looked at the code, and am admittedly very ignorant of the camera and display subsystems) but it sounds to me like what's missing is the sensor driver having knowledge of it's input clock and/or a way for it to request it's input clock to be enabled (e.g. clk_get/clk_enable.)

For static configuration, this could indeed be put in DT.

Quoted 4 lines

Alternatively, what would proably be even better would be that the sensor driver has a reference to the actual device that provides its input clock (possibly via a DT phandle?) so that the sensor driver can simply do a pm_runtime_get() on the device providing the clock.

Why not a clocks property instead of a phandle?
Of course the actual device providing the clock needs to advertise itself as a clock (e.g. in DT).

Quoted 17 lines
Quoted 6 lines

When adding more external devices to the mix the problem just becomes more complex, especially when the devices are chained (for instance sensor -> video processor -> ISP). The problem is similar on the display side, possibly with a different resume ordering (it should be noted that the external devices vs. internal device ordering might vary even inside the same class of devices - camera or display).

IMO, I still think that properly modeling the device dependenies combined with a "runtime PM centric" view of suspend/resume should allow the dependencies to be handled correctly for system suspend/resume and runtime PM.

I think what complicates things here is not the PM specifics but probably the fact that the device hierarchy (and dependencies) may be dynamic depending on many factors like which sensors are in use, post-processing, etc. etc.

Quoted 8 lines

Above, I suggested possibly using DT phandles to model these non parent/child relationships. That's all fine if the dependencies are not changing, but if they are dynamic, we'll probably need something different.

At least for starters though, static dependencies like this should be (relatively) easy to model in DT and combined with runtime PM refcounding on the dependencies, should be able to address the problem.

I'm still looking for a more generic way to handle such clocks, perhaps in the driver core. Right now the driver is supposed to handle this, so it needs to be aware of the existence of the clock, which may be optional. E.g. if device modules in a new SoC suddenly gets gateable clocks, do we have to modify all affected drivers? It would be nice if it would "just work" by adding the clocks to DT.

Gr{oetje,eeting}s,

Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]

In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds

#23 lore raw
In reply to Kevin Hilman
Quoted 44 lines · Kevin Hilman
Quoted 42 lines · Laurent Pinchart
Quoted 21 lines · On Wednesday 14 May 2014 00:34:56 Rafael J. Wysocki wrote:
Quoted 12 lines · Rafael J. Wysocki
Quoted 11 lines · Mark Brown
Quoted 9 lines · Tomasz Figa
Quoted 2 lines · Mark Brown

It also solves the system suspend dependencies. Why don't the runtime PM dependencies just work with reference counting?

Runtime PM dependencies work with reference counting just fine, but only for topologies matching Linux driver model, e.g. devices with exactly one device they depend on, e.g. SPI controller and SPI devices on the bus driven by it. Add there an IOMMU and other various strange things that should be transparent to the drivers and it stops working.

Quoted 7 lines
Quoted 5 lines

There's no reason why runtime PM references have to follow the topology
- you do get a default reference count up to any parent (though we break that sometimes, as is the case with SPI controllers being suspended even though the devices below them are active) but there's nothing stopping references being taken outside the topology.

Precisely.

Quoted 7 lines

BTW, I guess that the problem is resume and specifically the fact that if a child device resumes, the parent will also resume automatically, but the other devices the child may depend on will not (the child's resume may need to resume them directly).

But I'm not sure why that is a problem, so can anyone please share some details?

Here are two real life examples.

1. IOMMU and bus master

Bus master devices connected to an IOMMU need the IOMMU to be powered on in order to access memory. In order to save power the IOMMU should of course be powered off as much as possible.

The tricky part here comes from the fact that the IOMMU is hidden behind the DMA API. The bus master driver can't manage the IOMMU power state eplicitly by taking/releasing references to the IOMMU device.

Quoted 6 lines
Quoted 4 lines

This could easily be solved in an ad-hoc fashion by extending the DMA mapping API, but I'm wondering whether similar issues wouldn't benefit from a common solution (I'm not sure yet what all the similar issues are, hence the topic proposal to try and gather use cases).

I'll continue to beat the runtime PM drum...

What about async system suspend/resume, then?

Quoted 8 lines

This would also easily be solved if the bus master device, the IOMMU and the dmaengine (or the platform-specific dma driver) were all using runtime PM. e.g. bus master device is a user of dmaengine, which is a user of the IOMMU. If all are using runtime PM, the fact that a device is "in use", means runtime PM would keep them active when needed. In this example, it wouldn't matter that the bus master device doesn't know about the IOMMU. It suffices that the dmaengine driver knows about the IOMMU.

That requires it to be able to say "now I'm not using you any more" whenever the device in question suspend and "now I'm going to use you again" when that device resumes. Does something like that happen today?

There are more weird cases still. For example, we have the _DEP object in ACPI that basically says "this device depends on that one" and there may be no other relationship between the two whatsoever. How are we supposed to implement this within the existing frameworks?

Rafael

#24 lore raw
Quoted 11 lines · Rafael J. Wysocki
Quoted 8 lines · Kevin Hilman

This would also easily be solved if the bus master device, the IOMMU and the dmaengine (or the platform-specific dma driver) were all using runtime PM. e.g. bus master device is a user of dmaengine, which is a user of the IOMMU. If all are using runtime PM, the fact that a device is "in use", means runtime PM would keep them active when needed. In this example, it wouldn't matter that the bus master device doesn't know about the IOMMU. It suffices that the dmaengine driver knows about the IOMMU.

Quoted 3 lines

That requires it to be able to say "now I'm not using you any more" whenever the device in question suspend and "now I'm going to use you again" when that device resumes. Does something like that happen today?

The easiest thing to do with this sort of thing is usually to put the referencing into the API that abstracts things (it really ought to know when things are actually being used anyway after all) - there are some APIs that do this already.

Quoted 4 lines

There are more weird cases still. For example, we have the _DEP object in ACPI that basically says "this device depends on that one" and there may be no other relationship between the two whatsoever. How are we supposed to implement this within the existing frameworks?

That sounds like something that should be baked into however ACPI is hooked in already?

#25 lore raw
Quoted 26 lines · Mark Brown

--msuMhlVPES9Pz2Dp
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline

Quoted 11 lines · Rafael J. Wysocki
Quoted 8 lines · Kevin Hilman

This would also easily be solved if the bus master device, the IOMMU and the dmaengine (or the platform-specific dma driver) were all using runtime PM. e.g. bus master device is a user of dmaengine, which is a user of the IOMMU. If all are using runtime PM, the fact that a device is "in use", means runtime PM would keep them active when needed. In this example, it wouldn't matter that the bus master device doesn't know about the IOMMU. It suffices that the dmaengine driver knows about the IOMMU.

Quoted 3 lines

That requires it to be able to say "now I'm not using you any more" whenever the device in question suspend and "now I'm going to use you again" when that device resumes. Does something like that happen today?

The easiest thing to do with this sort of thing is usually to put the referencing into the API that abstracts things (it really ought to know when things are actually being used anyway after all) - there are some APIs that do this already.

Quoted 7 lines
Quoted 4 lines

There are more weird cases still. For example, we have the _DEP object in ACPI that basically says "this device depends on that one" and there may be no other relationship between the two whatsoever. How are we supposed to implement this within the existing frameworks?

That sounds like something that should be baked into however ACPI is hooked in already?

No, it is not. It is a relatively new addition to ACPI and we don't support it today. That's because it hasn't been present in the ACPI tables of any production systems until recently.

However, today we have systems with it shipping and we need to add support for it.

Rafael

#26 lore raw
Quoted 10 lines · Rafael J. Wysocki
Quoted 7 lines · Mark Brown
Quoted 4 lines

There are more weird cases still. For example, we have the _DEP object in ACPI that basically says "this device depends on that one" and there may be no other relationship between the two whatsoever. How are we supposed to implement this within the existing frameworks?

That sounds like something that should be baked into however ACPI is hooked in already?

Quoted 6 lines

No, it is not. It is a relatively new addition to ACPI and we don't support it today. That's because it hasn't been present in the ACPI tables of any production systems until recently.

However, today we have systems with it shipping and we need to add support for it.

Sorry, I wasn't clear - what I meant was that it should be something that can be handled based on information the ACPI implementation already has, I had been under the impression it got notification of all the basic PM transitions already?

#27 lore raw
Quoted 20 lines · Mark Brown

--6o5AAeie4QQ7Mfd3
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline

Quoted 13 lines · Rafael J. Wysocki
Quoted 7 lines · Mark Brown
Quoted 4 lines

There are more weird cases still. For example, we have the _DEP object in ACPI that basically says "this device depends on that one" and there may be no other relationship between the two whatsoever. How are we supposed to implement this within the existing frameworks?

That sounds like something that should be baked into however ACPI is hooked in already?

No, it is not. It is a relatively new addition to ACPI and we don't support it today. That's because it hasn't been present in the ACPI tables of any production systems until recently.

Quoted 7 lines
Quoted 2 lines

However, today we have systems with it shipping and we need to add support for it.

Sorry, I wasn't clear - what I meant was that it should be something that can be handled based on information the ACPI implementation already has, I had been under the impression it got notification of all the basic PM transitions already?

It is called to do some low-level manipulations of device PM settings during those transitions, but that may be after the driver's suspend callback has run (like for PCI, for example).

Rafael

#28 lore raw
In reply to Rafael J. Wysocki
Quoted 45 lines · Rafael J. Wysocki
Quoted 44 lines · Kevin Hilman
Quoted 42 lines · Laurent Pinchart
Quoted 21 lines · On Wednesday 14 May 2014 00:34:56 Rafael J. Wysocki wrote:
Quoted 12 lines · Rafael J. Wysocki
Quoted 11 lines · Mark Brown
Quoted 9 lines · Tomasz Figa
Quoted 2 lines · Mark Brown

It also solves the system suspend dependencies. Why don't the runtime PM dependencies just work with reference counting?

Runtime PM dependencies work with reference counting just fine, but only for topologies matching Linux driver model, e.g. devices with exactly one device they depend on, e.g. SPI controller and SPI devices on the bus driven by it. Add there an IOMMU and other various strange things that should be transparent to the drivers and it stops working.

Quoted 7 lines
Quoted 5 lines

There's no reason why runtime PM references have to follow the topology
- you do get a default reference count up to any parent (though we break that sometimes, as is the case with SPI controllers being suspended even though the devices below them are active) but there's nothing stopping references being taken outside the topology.

Precisely.

Quoted 7 lines

BTW, I guess that the problem is resume and specifically the fact that if a child device resumes, the parent will also resume automatically, but the other devices the child may depend on will not (the child's resume may need to resume them directly).

But I'm not sure why that is a problem, so can anyone please share some details?

Here are two real life examples.

1. IOMMU and bus master

Bus master devices connected to an IOMMU need the IOMMU to be powered on in order to access memory. In order to save power the IOMMU should of course be powered off as much as possible.

The tricky part here comes from the fact that the IOMMU is hidden behind the DMA API. The bus master driver can't manage the IOMMU power state eplicitly by taking/releasing references to the IOMMU device.

Quoted 8 lines
Quoted 6 lines
Quoted 4 lines

This could easily be solved in an ad-hoc fashion by extending the DMA mapping API, but I'm wondering whether similar issues wouldn't benefit from a common solution (I'm not sure yet what all the similar issues are, hence the topic proposal to try and gather use cases).

I'll continue to beat the runtime PM drum...

What about async system suspend/resume, then?

Not sure what you're asking here. I don't see any (new) conflicts with async system suspend/resume when combined with runtime PM.

Quoted 9 lines
Quoted 8 lines

This would also easily be solved if the bus master device, the IOMMU and the dmaengine (or the platform-specific dma driver) were all using runtime PM. e.g. bus master device is a user of dmaengine, which is a user of the IOMMU. If all are using runtime PM, the fact that a device is "in use", means runtime PM would keep them active when needed. In this example, it wouldn't matter that the bus master device doesn't know about the IOMMU. It suffices that the dmaengine driver knows about the IOMMU.

Quoted 3 lines

That requires it to be able to say "now I'm not using you any more" whenever the device in question suspend and "now I'm going to use you again" when that device resumes. Does something like that happen today?

I don't know the specifics of the dmaengine or IOMMU APIs, but they are certainly request based, so I suspect the infrastructure exist already. And in the case of DMAs, the core code knows when xfers are done, so having it request/release the underlying IOMMUs (or other dependencies) seems rather straight forward.

Even if the APIs dont exist today, instrumenting the frameworks themsleves seems better to me than trying to invent a generic new dependency structure, when runtime PM is most of the way there already.

Quoted 4 lines

There are more weird cases still. For example, we have the _DEP object in ACPI that basically says "this device depends on that one" and there may be no other relationship between the two whatsoever. How are we supposed to implement this within the existing frameworks?

Sounds like something for the ACPI PM domain to sort out, and it really depends on what what "this device depends on that one" means. Does it mean device Y has to be on whenever device X is on? Does it mean device Y has to be on just for device X to suspend/resume? Does it mean device Y has to be on for X to have certain functionality?

If it's something like device Y has to be on whenever device X is on, the PM domain would just need to track the dependencies and ensure that it does a pm_runtime_get() on device Y whenever device X is enabled.

Kevin

#29 lore raw
Quoted 52 lines · Kevin Hilman
Quoted 50 lines · Rafael J. Wysocki
Quoted 49 lines · Kevin Hilman
Quoted 46 lines · Laurent Pinchart
Quoted 21 lines · On Wednesday 14 May 2014 00:34:56 Rafael J. Wysocki wrote:
Quoted 12 lines · Rafael J. Wysocki
Quoted 11 lines · Mark Brown
Quoted 9 lines · Tomasz Figa
Quoted 2 lines · Mark Brown

It also solves the system suspend dependencies. Why don't the runtime PM dependencies just work with reference counting?

Runtime PM dependencies work with reference counting just fine, but only for topologies matching Linux driver model, e.g. devices with exactly one device they depend on, e.g. SPI controller and SPI devices on the bus driven by it. Add there an IOMMU and other various strange things that should be transparent to the drivers and it stops working.

Quoted 7 lines
Quoted 5 lines

There's no reason why runtime PM references have to follow the topology
- you do get a default reference count up to any parent (though we break that sometimes, as is the case with SPI controllers being suspended even though the devices below them are active) but there's nothing stopping references being taken outside the topology.

Precisely.

Quoted 7 lines

BTW, I guess that the problem is resume and specifically the fact that if a child device resumes, the parent will also resume automatically, but the other devices the child may depend on will not (the child's resume may need to resume them directly).

But I'm not sure why that is a problem, so can anyone please share some details?

Here are two real life examples.

1. IOMMU and bus master

Bus master devices connected to an IOMMU need the IOMMU to be powered on in order to access memory. In order to save power the IOMMU should of course be powered off as much as possible.

The tricky part here comes from the fact that the IOMMU is hidden behind the DMA API. The bus master driver can't manage the IOMMU power state eplicitly by taking/releasing references to the IOMMU device.

This could easily be solved in an ad-hoc fashion by extending the DMA mapping API, but I'm wondering whether similar issues wouldn't benefit from a common solution (I'm not sure yet what all the similar issues are, hence the topic proposal to try and gather use cases).

Quoted 6 lines
Quoted 3 lines
Quoted 1 line

I'll continue to beat the runtime PM drum...

What about async system suspend/resume, then?

Not sure what you're asking here. I don't see any (new) conflicts with async system suspend/resume when combined with runtime PM.

I don't mean conflicts, but how runtime PM can help in that area. :-)

[cut]

Quoted 11 lines
Quoted 4 lines

There are more weird cases still. For example, we have the _DEP object in ACPI that basically says "this device depends on that one" and there may be no other relationship between the two whatsoever. How are we supposed to implement this within the existing frameworks?

Sounds like something for the ACPI PM domain to sort out, and it really depends on what what "this device depends on that one" means. Does it mean device Y has to be on whenever device X is on? Does it mean device Y has to be on just for device X to suspend/resume? Does it mean device Y has to be on for X to have certain functionality?

Quoted 3 lines

If it's something like device Y has to be on whenever device X is on, the PM domain would just need to track the dependencies and ensure that it does a pm_runtime_get() on device Y whenever device X is enabled.

It is kind of hard to explain without going into details about how certain things in AML work, but roughly an ACPI method for one device may use resources provided by another device (for example, an ACPI method for device X uses an I2C connection with controller Y). In that case the "resource provider" has to be functional whenever the "consumer" method is executed.

Rafael

#30 lore raw
In reply to Kevin Hilman

Hi Kevin,

Quoted 44 lines · On Tuesday 20 May 2014 09:57:14 Kevin Hilman wrote:
Quoted 43 lines · Laurent Pinchart
Quoted 22 lines · On Wednesday 14 May 2014 00:34:56 Rafael J. Wysocki wrote:
Quoted 13 lines · Rafael J. Wysocki
Quoted 12 lines · Mark Brown
Quoted 10 lines · Tomasz Figa
Quoted 2 lines · Mark Brown

It also solves the system suspend dependencies. Why don't the runtime PM dependencies just work with reference counting?

Runtime PM dependencies work with reference counting just fine, but only for topologies matching Linux driver model, e.g. devices with exactly one device they depend on, e.g. SPI controller and SPI devices on the bus driven by it. Add there an IOMMU and other various strange things that should be transparent to the drivers and it stops working.

Quoted 7 lines
Quoted 5 lines

There's no reason why runtime PM references have to follow the topology - you do get a default reference count up to any parent (though we break that sometimes, as is the case with SPI controllers being suspended even though the devices below them are active) but there's nothing stopping references being taken outside the topology.

Precisely.

Quoted 7 lines

BTW, I guess that the problem is resume and specifically the fact that if a child device resumes, the parent will also resume automatically, but the other devices the child may depend on will not (the child's resume may need to resume them directly).

But I'm not sure why that is a problem, so can anyone please share some details?

Here are two real life examples.

1. IOMMU and bus master

Bus master devices connected to an IOMMU need the IOMMU to be powered on in order to access memory. In order to save power the IOMMU should of course be powered off as much as possible.

The tricky part here comes from the fact that the IOMMU is hidden behind the DMA API. The bus master driver can't manage the IOMMU power state eplicitly by taking/releasing references to the IOMMU device.

Quoted 6 lines
Quoted 4 lines

This could easily be solved in an ad-hoc fashion by extending the DMA mapping API, but I'm wondering whether similar issues wouldn't benefit from a common solution (I'm not sure yet what all the similar issues are, hence the topic proposal to try and gather use cases).

I'll continue to beat the runtime PM drum...

And I'll continue to sing a slightly different song :-)

Quoted 7 lines

This would also easily be solved if the bus master device, the IOMMU and the dmaengine (or the platform-specific dma driver) were all using runtime PM. e.g. bus master device is a user of dmaengine, which is a user of the IOMMU. If all are using runtime PM, the fact that a device is "in use", means runtime PM would keep them active when needed. In this example, it wouldn't matter that the bus master device doesn't know about the IOMMU. It suffices that the dmaengine driver knows about the IOMMU.

You're confusing the DMA engine and DMA mapping APIs here. IOMMU integration occurs in the DMA mapping API, which lets the bus master map/unmap memory regions for DMA. The DMA mapping API itself doesn't expose any underlying device (when no IOMMU is used there's no underlying device to be exposed), so the bus master can't take a PM reference on the IOMMU device neither directly nor indirectly at the moment.

The DMA mapping API is not request-based, so we can't rely on requests to manage the IOMMU power. There's a concept of mappings being created and torn down, where no memory access can happen without a mapping. A first step would be to keep the IOMMU powered on while mappings exist. However, as creating a mapping can be expensive, devices usually keep mappings around even when they don't need to access memory.

This could be solved pretty easily (at least in my mind, coding a solution would likely reveal issues) by extending the DMA mapping API, but what I'm wondering is whether that ad-hoc solution is the right way to go, or if something more generic could be useful. I don't think this falls in the same PM dependency category as the problem below, but it might still benefit from a more generic solution than just a DMA mapping API extension. If nobody can see a real added value here I would be fine with just extending the DMA mapping API.

Quoted 37 lines
Quoted 25 lines

2. Composite devices

I'll take an embedded camera devices as an example as that's the case I know better, but the same problem occurs on the display side as well.

Camera devices on embedded systems are usually made of a camera interface inside an SoC (possibly split into several IP cores, such as a CSI - Camera Serial Interface - receiver and a backend) and one or more external devices, such as sensors, flash controller, lens controller or sometimes dedicated video processing accelerators.

The external devices are most of the time controlled through I2C (SPI is an option as well, the exact interface doesn't matter much). Those that handle video data streams (sensor, video processor) use dedicated parallel or high speed serial busses.

Most of the external devices require an input clock. The clock can be fixed (easy), provided by a dedicated chip or IP core in the SoC (easy as well), sometimes by the SoC camera interface or even by one of the other external devices (for instance the lens controller could be clocked by the pixel clock output by the sensor - that's a bit far-fetched - or, more boring but equally annoying, I've seen a camera interface in an SoC that required the pixel clock output by the sensor to be running in order to complete its reset sequence).

Quoted 11 lines

On OMAP3 systems implementing a camera the image sensor is usually supplied with a clock output by the ISP (Image Signal Processor, the camera interface). When resuming the system from suspend (assuming the ISP was capturing video when the system got suspended), it's important to restart the ISP first and only then restart the sensor. To ensure that the order is followed the ISP driver resume path first restarts the ISP and then calls the sensor driver to start the video stream. The sensor driver will need to send I2C messages to the device in order to start it, which requires the I2C controller to be resumed. As the ISP is a platform device at the same level as the I2C controller, that ordering is not guaranteed.

Quoted 7 lines

As you likely already know, at least on OMAP, the I2C ordering would work just fine because the I2C driver is (what I like to call) runtime PM centric. IOW, it doesn't really implement suspend/resume, instead it only does runtime PM "on demand" on a per-xfer basis. That means that the I2C driver can be used anytime during the [runtime] suspend/resume of any other device without any ordering problems because it's activated on demand.

That's nice. As long as all devices up the chain can work that way (if the bus the I2C master sits on is suspended, we would just be moving the problem one square up) that should fix at least part of the problem

Quoted 23 lines

Extending this same "runtime PM centric" view of the world to the other devices, I think that the runtime PM reference counting invovled when all the components are using runtime PM would solve this problem.

(I realize that this I2C solution doesn't solve the more general "resume ISP before sensor" problem though, but with better modeling, I think it can. More on that below...)

Quoted 6 lines

Furthermore, if the sensor is resumed first, it might try to access the device, which requires the clock output by the ISP to be available, and thus requires the ISP to be resumed. To solve this problem the ISP driver only restarts the clocks in its PM resume callback, and restarts the video stream (following the sequence described above) in its PM complete callback.

For most devices, input clocks are modeled by the clock framework (or managed by the SoC's runtime PM core), and therefore, a pm_runtime_get() (or possibly an explict clk_enable()) is used to ensure the input clock is running. In this external device example, it sounds to me like the sensor driver has no knowledge of its input clock so it has to rely on some other layer to resume things in the right order for correct functionality.

Quoted 5 lines

Maybe I'm wrong here (likely, since I haven't looked at the code, and am admittedly very ignorant of the camera and display subsystems) but it sounds to me like what's missing is the sensor driver having knowledge of it's input clock and/or a way for it to request it's input clock to be enabled (e.g. clk_get/clk_enable.)

I'm glad that you point out likely being wrong yourself, as you are ;-) The sensor driver manages the sensor input clock explicitly through CCF (or at least it should, not all drivers do their homework properly, but that's just a matter of fixing them), so it can enable/disable the clock when needed. If the clock provider driver becomes runtime PM centric then part of the problem would be fixed.

Quoted 4 lines

Alternatively, what would proably be even better would be that the sensor driver has a reference to the actual device that provides its input clock (possibly via a DT phandle?) so that the sensor driver can simply do a pm_runtime_get() on the device providing the clock.

Isn't it better for the sensor DT node to reference its input clock through the clocks property and enable/disable the clock on demand instead of explicitly calling pm_runtime_(get|put) on the clock provider device ?

Quoted 11 lines
Quoted 6 lines

When adding more external devices to the mix the problem just becomes more complex, especially when the devices are chained (for instance sensor -> video processor -> ISP). The problem is similar on the display side, possibly with a different resume ordering (it should be noted that the external devices vs. internal device ordering might vary even inside the same class of devices - camera or display).

IMO, I still think that properly modeling the device dependenies combined with a "runtime PM centric" view of suspend/resume should allow the dependencies to be handled correctly for system suspend/resume and runtime PM.

I definitely need to give this a bit more thought. I agree that it would likely solve part of the issue, but I'm not sure whether the rest is solvable with the infrastructure we have now.

Quoted 9 lines

I think what complicates things here is not the PM specifics but probably the fact that the device hierarchy (and dependencies) may be dynamic depending on many factors like which sensors are in use, post-processing, etc. etc.

Above, I suggested possibly using DT phandles to model these non parent/child relationships. That's all fine if the dependencies are not changing, but if they are dynamic, we'll probably need something different.

They can be dynamic, yes. However, we already model the video data streams dependencies using phandles in the V4L2 bindings, so the required information is there. It would thus "just" be a matter of orchestrating all the involved components. If we go the runtime PM way the problem might be simplified, but as Ulf Hansson mentioned interactions between runtime PM and system suspend/resume need to be taken care of.

Quoted 3 lines

At least for starters though, static dependencies like this should be (relatively) easy to model in DT and combined with runtime PM refcounding on the dependencies, should be able to address the problem.

Signature · 3 lines
Regards,

Laurent Pinchart
#31 lore raw
Quoted 4 lines · Laurent Pinchart

Hi Kevin,

Quoted 1 line · On Tuesday 20 May 2014 09:57:14 Kevin Hilman wrote:

Laurent Pinchart <[email protected]> writes:

[...]

I'll respond to the DMA/IOMMU part separately, as I need some time to digest it.

Quoted 15 lines
Quoted 15 lines
Quoted 6 lines

Furthermore, if the sensor is resumed first, it might try to access the device, which requires the clock output by the ISP to be available, and thus requires the ISP to be resumed. To solve this problem the ISP driver only restarts the clocks in its PM resume callback, and restarts the video stream (following the sequence described above) in its PM complete callback.

For most devices, input clocks are modeled by the clock framework (or managed by the SoC's runtime PM core), and therefore, a pm_runtime_get() (or possibly an explict clk_enable()) is used to ensure the input clock is running. In this external device example, it sounds to me like the sensor driver has no knowledge of its input clock so it has to rely on some other layer to resume things in the right order for correct functionality.

Quoted 7 lines
Quoted 5 lines

Maybe I'm wrong here (likely, since I haven't looked at the code, and am admittedly very ignorant of the camera and display subsystems) but it sounds to me like what's missing is the sensor driver having knowledge of it's input clock and/or a way for it to request it's input clock to be enabled (e.g. clk_get/clk_enable.)

I'm glad that you point out likely being wrong yourself, as you are ;-)

Touché.

It's good to have a low standard for oneself. ;)

Quoted 5 lines

The sensor driver manages the sensor input clock explicitly through CCF (or at least it should, not all drivers do their homework properly, but that's just a matter of fixing them), so it can enable/disable the clock when needed. If the clock provider driver becomes runtime PM centric then part of the problem would be fixed.

Great, that simplifies things a bit, except...

Quoted 8 lines
Quoted 4 lines

Alternatively, what would proably be even better would be that the sensor driver has a reference to the actual device that provides its input clock (possibly via a DT phandle?) so that the sensor driver can simply do a pm_runtime_get() on the device providing the clock.

Isn't it better for the sensor DT node to reference its input clock through the clocks property and enable/disable the clock on demand instead of explicitly calling pm_runtime_(get|put) on the clock provider device ?

Possibly, but I suspect that's not going to be good enough.

As I mentioned in an earlier response to Geert, I don't think managing clocks alone is enough. I can imagine many platforms where a simple clk_enable() of an input clock is not enough to bring the device providing that clock out of a low power state. For this to be generic enough to handle those cases, I suspect runtime PM get/put is the right way to go.

Of course, for many "simple" platforms, runtime PM get/put just ends up doing a clk_enable/disable, but on the platforms where runtime PM is slightly more... um, "interesting"... just doing a clk enable/disable won't do what you hope.

Quoted 7 lines
Quoted 7 lines
Quoted 6 lines

When adding more external devices to the mix the problem just becomes more complex, especially when the devices are chained (for instance sensor -> video processor -> ISP). The problem is similar on the display side, possibly with a different resume ordering (it should be noted that the external devices vs. internal device ordering might vary even inside the same class of devices - camera or display).

Quoted 8 lines
Quoted 4 lines

IMO, I still think that properly modeling the device dependenies combined with a "runtime PM centric" view of suspend/resume should allow the dependencies to be handled correctly for system suspend/resume and runtime PM.

I definitely need to give this a bit more thought. I agree that it would likely solve part of the issue, but I'm not sure whether the rest is solvable with the infrastructure we have now.

Not without some enhancments, but IMO ensuring the frameworks are runtime PM centric, and possibly extending runtime PM slightly where needed is a better route than creating something new. Whatever that "something new" would be would end up duplicating much of the use counting already done by runtime PM anyways.

Quoted 10 lines
Quoted 9 lines

I think what complicates things here is not the PM specifics but probably the fact that the device hierarchy (and dependencies) may be dynamic depending on many factors like which sensors are in use, post-processing, etc. etc.

Above, I suggested possibly using DT phandles to model these non parent/child relationships. That's all fine if the dependencies are not changing, but if they are dynamic, we'll probably need something different.

Quoted 6 lines

They can be dynamic, yes. However, we already model the video data streams dependencies using phandles in the V4L2 bindings, so the required information is there. It would thus "just" be a matter of orchestrating all the involved components. If we go the runtime PM way the problem might be simplified, but as Ulf Hansson mentioned interactions between runtime PM and system suspend/resume need to be taken care of.

I'm unfortunately all too familiar with the interactions between system PM and runtime PM, but I believe we have a pretty good grip on that now, and the infrastructure is in place to solve those problems.

Kevin

#32 lore raw
Quoted 4 lines · Kevin Hilman

Of course, for many "simple" platforms, runtime PM get/put just ends up doing a clk_enable/disable, but on the platforms where runtime PM is slightly more... um, "interesting"... just doing a clk enable/disable won't do what you hope.

And drivers/base/power/clock_ops.c as invoked from drivers/sh/pm_runtime.c does exactly that simple thing using PM domains. Quite elegantly too, I'd say.

But now I get curious - can we define the "interesting PM" more generally, like get the details out? And I mean like for a few different platforms, I know OMAP with its hwmod is one such thing, but how much interesting stuff do we have out there that we know we must be able to handle?

Cc:ing in Peter De Schrijver who might have some interesting PM to tell us about :-)

Yours,
Linus Walleij

#33 lore raw
Quoted 11 lines · Linus Walleij
Quoted 4 lines · Kevin Hilman

Of course, for many "simple" platforms, runtime PM get/put just ends up doing a clk_enable/disable, but on the platforms where runtime PM is slightly more... um, "interesting"... just doing a clk enable/disable won't do what you hope.

And drivers/base/power/clock_ops.c as invoked from drivers/sh/pm_runtime.c does exactly that simple thing using PM domains. Quite elegantly too, I'd say.

Quoted 5 lines

But now I get curious - can we define the "interesting PM" more generally, like get the details out? And I mean like for a few different platforms, I know OMAP with its hwmod is one such thing, but how much interesting stuff do we have out there that we know we must be able to handle?

By "interesting", I was referring to some of the more complicated ones we have, such as OMAP (though I suspect other SoCs are equally complicated but just don't have their stuff documented and/or upstream yet so we don't know.)

That being said, I think any SoC that requires more than a clk_enable() to bring an IP block out of a low-power state, or clk_disable() to put it into a low-power state qualifies as "interesting" because it can no longer rely on the "simple" clock_ops.c you refer to above.

Stated differently, for most SoCs, fine-grained PM is a lot more than just clock management. As soon as you have to do more than manage clocks, the generic clock_ops.c isn't good enough and you need your own PM domain (or use genpd.)

Kevin

#34 lore raw
Quoted 29 lines · Kevin Hilman
Quoted 11 lines · Linus Walleij
Quoted 4 lines · Kevin Hilman

Of course, for many "simple" platforms, runtime PM get/put just ends up doing a clk_enable/disable, but on the platforms where runtime PM is slightly more... um, "interesting"... just doing a clk enable/disable won't do what you hope.

And drivers/base/power/clock_ops.c as invoked from drivers/sh/pm_runtime.c does exactly that simple thing using PM domains. Quite elegantly too, I'd say.

Quoted 5 lines

But now I get curious - can we define the "interesting PM" more generally, like get the details out? And I mean like for a few different platforms, I know OMAP with its hwmod is one such thing, but how much interesting stuff do we have out there that we know we must be able to handle?

By "interesting", I was referring to some of the more complicated ones we have, such as OMAP (though I suspect other SoCs are equally complicated but just don't have their stuff documented and/or upstream yet so we don't know.)

That being said, I think any SoC that requires more than a clk_enable() to bring an IP block out of a low-power state, or clk_disable() to put it into a low-power state qualifies as "interesting" because it can no longer rely on the "simple" clock_ops.c you refer to above.

Quoted 4 lines

Stated differently, for most SoCs, fine-grained PM is a lot more than just clock management. As soon as you have to do more than manage clocks, the generic clock_ops.c isn't good enough and you need your own PM domain (or use genpd.)

Well stated.

Also, as I said in one of the previous messages, if you consider a driver as a piece of code that should work with different platforms (or different SoCs if you will, but there's more to that), then it's quite clear that the driver is probably not the right place to do all PM that's needed.

Rafael

#35 lore raw
In reply to Linus Walleij
Quoted 10 lines · Linus Walleij
Quoted 4 lines · Kevin Hilman

Of course, for many "simple" platforms, runtime PM get/put just ends up doing a clk_enable/disable, but on the platforms where runtime PM is slightly more... um, "interesting"... just doing a clk enable/disable won't do what you hope.

And drivers/base/power/clock_ops.c as invoked from drivers/sh/pm_runtime.c does exactly that simple thing using PM domains. Quite elegantly too, I'd say.

Well, thanks! :-)

Quoted 8 lines

But now I get curious - can we define the "interesting PM" more generally, like get the details out? And I mean like for a few different platforms, I know OMAP with its hwmod is one such thing, but how much interesting stuff do we have out there that we know we must be able to handle?

Cc:ing in Peter De Schrijver who might have some interesting PM to tell us about :-)

I generally like to split PM into things that can be done by the driver regardless of the underlying platform and things that are platform-dependent. It generally is better to abstract the latter away from drivers in my opinion.

So if a given clock is guaranteed to be manageable in the same way on every platform where the driver will ever run, that clock may be managed by the driver itself. However, if that clock is correlated with something else on one of the platforms and the driver would need to take that into account, things can get quite ugly relatively quickly. And if one of those platforms is ACPI, they are ugly right to start with. ;-)

Rafael

#36 lore raw
In reply to Linus Walleij
Quoted 11 lines · Linus Walleij
Quoted 4 lines · Kevin Hilman

Of course, for many "simple" platforms, runtime PM get/put just ends up doing a clk_enable/disable, but on the platforms where runtime PM is slightly more... um, "interesting"... just doing a clk enable/disable won't do what you hope.

And drivers/base/power/clock_ops.c as invoked from drivers/sh/pm_runtime.c does exactly that simple thing using PM domains. Quite elegantly too, I'd say.

Quoted 9 lines

But now I get curious - can we define the "interesting PM" more generally, like get the details out? And I mean like for a few different platforms, I know OMAP with its hwmod is one such thing, but how much interesting stuff do we have out there that we know we must be able to handle?

Cc:ing in Peter De Schrijver who might have some interesting PM to tell us about :-)

On Tegra to powergate a domain you have to:

1) wait until all outstanding memory accesses by DMA clients in the domain have finished and assert the reset of the DMA clients. This is because DMA to/from memory goes via a seperate path compared to control MMIO (register accesses).
2) assert reset of the modules in the domain
3) disable the clocks of the modules in the domain
4) enable clamps
5) turn off power

to powerungate a domain you basically have to do:

1) enable power
2) enable all clocks of the modules in the domain
3) unclamp the domain
4) deassert reset (this is why the clocks have to be on, otherwise the reset signals wouldn't propagate)
6) deassert reset of the DMA clients
5) disable all module clocks again

My guess is that OMAP does most of this sequencing in hw (probably done by the PRCM). Also I guess that OMAP doesn't need the special handling of the DMA clients as both control MMIO and DMA goes via the same interconnect.

Cheers,

Peter.

-----------------------------------------------------------------------------------
This email message is for the sole use of the intended recipient(s) and may contain confidential information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message.
-----------------------------------------------------------------------------------

#37 lore raw
In reply to Laurent Pinchart
Quoted 10 lines · Laurent Pinchart
Quoted 4 lines · On Tuesday 20 May 2014 09:57:14 Kevin Hilman wrote:

Alternatively, what would proably be even better would be that the sensor driver has a reference to the actual device that provides its input clock (possibly via a DT phandle?) so that the sensor driver can simply do a pm_runtime_get() on the device providing the clock.

Isn't it better for the sensor DT node to reference its input clock through the clocks property and enable/disable the clock on demand instead of explicitly calling pm_runtime_(get|put) on the clock provider device ?

This is tangent to another discussion we've had off and on whether a device's resources (clocks, regulators, GPIOs, pins, pwms, D/As...) should be handled explicitly by the driver or implicitly by e.g. a PM domain.

It appears that currently we have bias such that for a discrete component (such as some sensor) soldered on a board with rails and stuff, these generally do explicit resource management with clk_prepare_enable() and friends.

On the other hand there are some SoCs (OMAP especially but also some Reneasa SH stuff IIRC!) that does implicit resource management of devices on the SoC using PM domains.

This is a typical example of implicit resource management:
drivers/base/power/clock_ops.c

As PM domains are really not just for SoCs the broader question of whether runtime resource management should be explicit using handles in the drivers or implicit by referencing PM domain pops up.

The mixture of both that we have right now is a bit confusing, admittedly.

Yours,
Linus Walleij

#38 lore raw
Quoted 30 lines · Linus Walleij
Quoted 10 lines · Laurent Pinchart
Quoted 4 lines · On Tuesday 20 May 2014 09:57:14 Kevin Hilman wrote:

Alternatively, what would proably be even better would be that the sensor driver has a reference to the actual device that provides its input clock (possibly via a DT phandle?) so that the sensor driver can simply do a pm_runtime_get() on the device providing the clock.

Isn't it better for the sensor DT node to reference its input clock through the clocks property and enable/disable the clock on demand instead of explicitly calling pm_runtime_(get|put) on the clock provider device ?

This is tangent to another discussion we've had off and on whether a device's resources (clocks, regulators, GPIOs, pins, pwms, D/As...) should be handled explicitly by the driver or implicitly by e.g. a PM domain.

It appears that currently we have bias such that for a discrete component (such as some sensor) soldered on a board with rails and stuff, these generally do explicit resource management with clk_prepare_enable() and friends.

On the other hand there are some SoCs (OMAP especially but also some Reneasa SH stuff IIRC!) that does implicit resource management of devices on the SoC using PM domains.

This is a typical example of implicit resource management:
drivers/base/power/clock_ops.c

Quoted 6 lines

As PM domains are really not just for SoCs the broader question of whether runtime resource management should be explicit using handles in the drivers or implicit by referencing PM domain pops up.

The mixture of both that we have right now is a bit confusing, admittedly.

This discussion, touches several unresolved runtime PM and system PM issues in general. I think it's good we bring this all together, because certainly there still some pieces missing in the puzzle.

As we have brought up pm_domains in the discussion, we might as well add the "generic power domain" here.

Currently I am working on simplifying genpd. I am trying to iron out what responsibilities actually should reside in it. Like what SOC specific actions should it control, how should "pm_runtime_irq_safe" devices be handled, etc.

Kind regards
Ulf Hansson

Quoted 7 lines

Yours,
Linus Walleij
_______________________________________________
Ksummit-discuss mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss

#39 lore raw
In reply to Linus Walleij
Quoted 27 lines · Linus Walleij
Quoted 10 lines · Laurent Pinchart
Quoted 4 lines · On Tuesday 20 May 2014 09:57:14 Kevin Hilman wrote:

Alternatively, what would proably be even better would be that the sensor driver has a reference to the actual device that provides its input clock (possibly via a DT phandle?) so that the sensor driver can simply do a pm_runtime_get() on the device providing the clock.

Isn't it better for the sensor DT node to reference its input clock through the clocks property and enable/disable the clock on demand instead of explicitly calling pm_runtime_(get|put) on the clock provider device ?

This is tangent to another discussion we've had off and on whether a device's resources (clocks, regulators, GPIOs, pins, pwms, D/As...) should be handled explicitly by the driver or implicitly by e.g. a PM domain.

It appears that currently we have bias such that for a discrete component (such as some sensor) soldered on a board with rails and stuff, these generally do explicit resource management with clk_prepare_enable() and friends.

On the other hand there are some SoCs (OMAP especially but also some Reneasa SH stuff IIRC!) that does implicit resource management of devices on the SoC using PM domains.

Quoted 6 lines

This is a typical example of implicit resource management:
drivers/base/power/clock_ops.c

As PM domains are really not just for SoCs the broader question of whether runtime resource management should be explicit using handles in the drivers or implicit by referencing PM domain pops up.

When I was adding PM domains to the PM core one of the concerns was that the PM resources the drivers can access may not be independent and using them would require synchronization between drivers. Using PM domains is one of the ways to achieve that.

Also using PM domains helps to address the case when the same driver needs to be used on different platforms with different PM configurations.

Quoted 2 lines

The mixture of both that we have right now is a bit confusing, admittedly.

I suppose that this is for historical reasons mostly (different platforms chose to do things differently and code duplication between them related to handling analogous IP blocks in slightly different ways was less of a concern in the past).

For driver portability, however, there has to be a level of indirection between the driver and the PM features of the platform and PM domains can be used for providing something like that.

Rafael

#40 lore raw
In reply to Rafael J. Wysocki
Quoted 37 lines · On Wednesday 14 May 2014 00:27:47 Rafael J. Wysocki wrote:
Quoted 11 lines · Mark Brown
Quoted 9 lines · Tomasz Figa
Quoted 2 lines · Mark Brown

It also solves the system suspend dependencies. Why don't the runtime PM dependencies just work with reference counting?

Runtime PM dependencies work with reference counting just fine, but only for topologies matching Linux driver model, e.g. devices with exactly one device they depend on, e.g. SPI controller and SPI devices on the bus driven by it. Add there an IOMMU and other various strange things that should be transparent to the drivers and it stops working.

Quoted 5 lines

There's no reason why runtime PM references have to follow the topology
- you do get a default reference count up to any parent (though we break that sometimes, as is the case with SPI controllers being suspended even though the devices below them are active) but there's nothing stopping references being taken outside the topology.

Precisely.

Quoted 10 lines
Quoted 5 lines

I'm still investigating this issue, so more uses cases are yet to be found, but I also guess this is the purpose of this thread. Anyway, for some reason .suspend_late() and .resume_early() callbacks exist in dev_pm_ops struct and I believe that at least some of the cases "solved" by them might be related to the issue being discussed here.

Yes, they're partly solving a particular common case for this sort of interdependency (though I guess they do also do things like allow us to make sure the hardware came back in a state where it won't be harmful to the rest of the system if we start enabling things).

The original reason why .suspend_late/.resume_early were introduced was to allow the same callback routines to be run for runtime and for system-wide PM. Specifically, the idea was that .suspend_late may point to the same routine as .runtime_suspend, for example, because of the analogy of the conditions in which they are called (the device is known to be not in use).

Quoted 6 lines

That didn't work out perfectly, but those callbacks turn out to be useful nonetheless.

And yes, I'm interested in this topic too.

Rafael (who wonders why he wasn't CCed in this thread to start with)

It was a honest mistake, I'm sorry about that.

Signature · 3 lines
Regards,

Laurent Pinchart
#41 lore raw
In reply to Mark Brown
Quoted 12 lines · Mark Brown
Quoted 10 lines · Tomasz Figa
Quoted 2 lines · Mark Brown

It also solves the system suspend dependencies. Why don't the runtime PM dependencies just work with reference counting?

Runtime PM dependencies work with reference counting just fine, but only for topologies matching Linux driver model, e.g. devices with exactly one device they depend on, e.g. SPI controller and SPI devices on the bus driven by it. Add there an IOMMU and other various strange things that should be transparent to the drivers and it stops working.

Quoted 5 lines

There's no reason why runtime PM references have to follow the topology
- you do get a default reference count up to any parent (though we break that sometimes, as is the case with SPI controllers being suspended even though the devices below them are active) but there's nothing stopping references being taken outside the topology.

I'm very interested in this topic as well.

Though, I'm reluctant to see new APIs invented when I think already have the infrastructure to handle this (though admittedly, I haven't seen all the use-cases where this is problematic either.)

In my experience with this, it seems to me the root cause here is simply that there are still lots of drivers/subsytems that are not runtime PM adapted.

IOW, even if device X has a non-parent/child relationship with device Y, if both are runtime PM adapted, the simple fact of X *using* Y (or using the framework hiding Y) will be enough to ensure Y is powered up when used.

Yes, we could hack up the ability for X to directly refcount Y, but why should we do that instead of adapdting Y to use runtime PM itself so that when ever it's requested/used, it's powered up by the runtime PM infrastructure.

What am I missing ?

Quoted 10 lines
Quoted 5 lines

I'm still investigating this issue, so more uses cases are yet to be found, but I also guess this is the purpose of this thread. Anyway, for some reason .suspend_late() and .resume_early() callbacks exist in dev_pm_ops struct and I believe that at least some of the cases "solved" by them might be related to the issue being discussed here.

Yes, they're partly solving a particular common case for this sort of interdependency (though I guess they do also do things like allow us to make sure the hardware came back in a state where it won't be harmful to the rest of the system if we start enabling things).

IMO, the suspend_late/resume_early callbacks are useful for turning drivers/subsystems into what I call "runtime PM centric". That means the driver/subsytem can focus on a runtime PM centric view of the world, and then implement suspend/resume using the late/early callbacks to call the same functions as the runtime suspend/resume.

Kevin

#42 lore raw
In reply to Mark Brown
Quoted 8 lines · Mark Brown
Quoted 6 lines · Laurent Pinchart
Quoted 4 lines · On Monday 12 May 2014 21:14:38 Mark Brown wrote:

This is something ASoC has been resolving since forever with the machine drivers, Russell King recently created a generic version of the code that does the dependency deferral glue (see linux/component.h). It's not a model of elegance but it ends up getting the job done.

Quoted 4 lines
Quoted 2 lines

That solves the probe time dependencies, but unfortunately not the runtime PM dependencies.

It also solves the system suspend dependencies.

Well, I'm not sure about that. How does it do that, exactly?

Rafael

#43 lore raw
Quoted 9 lines · Rafael J. Wysocki
Quoted 6 lines · Mark Brown
Quoted 2 lines · Laurent Pinchart

That solves the probe time dependencies, but unfortunately not the runtime PM dependencies.

It also solves the system suspend dependencies.

Well, I'm not sure about that. How does it do that, exactly?

Since the core device probe is deferred until everything it depends on is ready it'll get pushed along the device list so that it appears after everything it needs and hence is suspended first.

#44 lore raw
Quoted 7 lines · Mark Brown
Quoted 5 lines · Rafael J. Wysocki
Quoted 4 lines · Mark Brown
Quoted 2 lines · Laurent Pinchart

That solves the probe time dependencies, but unfortunately not the runtime PM dependencies.

Quoted 7 lines
Quoted 3 lines
Quoted 1 line

It also solves the system suspend dependencies.

Well, I'm not sure about that. How does it do that, exactly?

Since the core device probe is deferred until everything it depends on is ready it'll get pushed along the device list so that it appears after everything it needs and hence is suspended first.

Except for async suspend, that is?

Rafael

#45 lore raw
Quoted 7 lines · Rafael J. Wysocki
Quoted 3 lines · Mark Brown

Since the core device probe is deferred until everything it depends on is ready it'll get pushed along the device list so that it appears after everything it needs and hence is suspended first.

Except for async suspend, that is?

Gah, I'd not actually seen that before - few of the systems I've worked on would have changed it anyway. Yes, that'll break things too.

#46 lore raw
In reply to Mark Brown
Quoted 6 lines · On Wednesday 14 May 2014 12:57:45 Mark Brown wrote:
Quoted 5 lines · Rafael J. Wysocki
Quoted 4 lines · Mark Brown
Quoted 2 lines · Laurent Pinchart

That solves the probe time dependencies, but unfortunately not the runtime PM dependencies.

Quoted 7 lines
Quoted 3 lines
Quoted 1 line

It also solves the system suspend dependencies.

Well, I'm not sure about that. How does it do that, exactly?

Since the core device probe is deferred until everything it depends on is ready it'll get pushed along the device list so that it appears after everything it needs and hence is suspended first.

That might solve (part of) the master vs. components dependencies (and even in that case I'm pretty sure that there will be cases where the master needs to be suspended first and cases where th slave(s) need to be suspended first), but it doesn't handle inter-slaves dependencies.

In the V4L subsystem we've implemented an explicit power management API that "slaves" (called subdevices in V4L) must implement, to be called by the master. That's of course more of a workaround than a perfect (or even good) solution.

Signature · 3 lines
Regards,

Laurent Pinchart
#47 lore raw
Quoted 11 lines · Laurent Pinchart
Quoted 3 lines · On Wednesday 14 May 2014 12:57:45 Mark Brown wrote:

Since the core device probe is deferred until everything it depends on is ready it'll get pushed along the device list so that it appears after everything it needs and hence is suspended first.

That might solve (part of) the master vs. components dependencies (and even in that case I'm pretty sure that there will be cases where the master needs to be suspended first and cases where th slave(s) need to be suspended first), but it doesn't handle inter-slaves dependencies.

Quoted 4 lines

In the V4L subsystem we've implemented an explicit power management API that "slaves" (called subdevices in V4L) must implement, to be called by the master. That's of course more of a workaround than a perfect (or even good) solution.

Right, that's what the function of the master device is in suspend for this model - it quiesces the compound device as a whole so that any interdependencies between the devices have been unwound (including giving callbacks to the slaves where appropriate). It gives us a place to hang everything associated with having the devices so closely interrelated.

Though as Rafael says with async suspend that gets broken.