Skip to contentSkip to conversation

remoteproc: qcom: Venus firmware loader support for msm8996

PATCH RESEND In review Avaneesh Kumar Dwivedi 4 participants 14 messages Nov 29, 2016 – Jan 25, 2017

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH RESEND v2 1/1 remoteproc: qcom: Venus firmware loader support for msm8996

This Patch is based on

        https://patchwork.kernel.org/patch/9415627/
        https://patchwork.kernel.org/patch/9415651/

This patch add necessary routine and data structure to support standalone venus firmware load on msm8996. Below are brief of changes.

1- Add private data structure which provide string-name and rate of clock
on msm8996 platform for venus.
2- Provide clock initialization and enable/disable functionality.

below is console log on msm8996 platform with above change, this is standalone test log without video driver enablement.

[    2.612011]  remoteproc0: soc:vidc_tzpil@0 is available
[    2.616939]  remoteproc0: Note: remoteproc is still under development and considered experimental.
[    2.621963]  remoteproc0: THE BINARY FORMAT IS NOT YET FINALIZED, and backward compatibility isn't yet guaranteed.
[    2.631037] qcom-tz-pil soc:vidc_tzpil@0: Venus rproc probe done
[    2.641463]  remoteproc0: powering up soc:vidc_tzpil@0
[    2.641468]  remoteproc0: Booting fw image venus.mdt, size 6812
[    2.698127]  remoteproc0: remote processor soc:vidc_tzpil@0 is now up

Changes w.r.t. patchset 1
reorganization and cleanup of added code.
Avaneesh Kumar Dwivedi (1):
remoteproc: qcom: Add venus rproc support on msm8996 platform.

 .../devicetree/bindings/remoteproc/qcom,venus.txt  |  26 ++++-
 drivers/remoteproc/qcom_venus_pil.c                | 116 ++++++++++++++++++++-
 2 files changed, 140 insertions(+), 2 deletions(-)
Signature · 2 lines
Qualcomm India Private Limited, on behalf of Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project.
#1 lore raw

PATCH RESEND v2 1/1 remoteproc: qcom: Add venus rproc support on msm8996 platform.

This patch is based on

	https://patchwork.kernel.org/patch/9415627/
	https://patchwork.kernel.org/patch/9415651/

This patch add clock initialization, enable and disable support. Required resource name string and rating are differentiated based on compatible string. Also added documentation for venus pil on msm8996.

---
 .../devicetree/bindings/remoteproc/qcom,venus.txt  |  26 ++++-
 drivers/remoteproc/qcom_venus_pil.c                | 116 ++++++++++++++++++++-
 2 files changed, 140 insertions(+), 2 deletions(-)
Documentation/devicetree/bindings/remoteproc/qcom,venus.txt +25 -1
index 2d73ba1..417026b 100644
@@ -6,13 +6,30 @@ on the Qualcomm Venus remote processor core.
 - compatible:
 	Usage: required
 	Value type: <string>
-	Definition: must contain "qcom,venus-pil"
+	Definition: must contain "qcom,venus-pil" or
+				"qcom,venus-msm8996-pil"
 
 - memory-region:
 	Usage: required
 	Value type: <phandle>
 	Definition: a phandle to a node describing reserved memory
 
+- clocks:
+	Usage: required
+	Value type: <prop-encoded-array>
+	Definition: reference to the core, iface and bus and maxi clocks to be held on
+		    behalf of the booting of the venus core
+
+- clock-names:
+	Usage: required
+	Value type: <stringlist>
+	Definition: should be "core_clk", "iface_clk", "bus_clk", "maxi_clk"
+
+- power-domains:
+	Usage: required
+	Value type: <prop-encoded-array>
+	Definition: reference to the venus gdsc to be turned on before booting venus core
+
 * An example
 	reserved-memory {
 		#address-cells = <2>;
@@ -29,5 +46,12 @@ on the Qualcomm Venus remote processor core.
 
 	rproc_venus@0 {
 		compatible = "qcom,venus-pil";
+		clocks = <&mmcc VIDEO_CORE_CLK>,
+				<&mmcc VIDEO_AHB_CLK>,
+				<&mmcc VIDEO_AXI_CLK>,
+				<&mmcc VIDEO_MAXI_CLK>;
+
+		clock-names = "core_clk", "iface_clk", "bus_clk", "maxi_clk";
+		power-domains = <&mmcc VENUS_GDSC>;
 		memory-region = <&venus_mem>;
 	};
drivers/remoteproc/qcom_venus_pil.c +115 -1

179 lines not on this page · raw message

Signature · 2 lines
Qualcomm India Private Limited, on behalf of Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project.
#12 lore raw
In reply to Avaneesh Kumar Dwivedi
Quoted 10 lines · Avaneesh Kumar Dwivedi

This Patch is based on

        https://patchwork.kernel.org/patch/9415627/
        https://patchwork.kernel.org/patch/9415651/

This patch add necessary routine and data structure to support standalone venus firmware load on msm8996. Below are brief of changes.

1- Add private data structure which provide string-name and rate of clock
on msm8996 platform for venus.
2- Provide clock initialization and enable/disable functionality.

For one patch we typically don't send cover letters. These sorts of details just go below the dashed lines after the commit text of the single patch.

Signature · 2 lines
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
a Linux Foundation Collaborative Project
#2 lore raw

Re: [RESEND PATCH v2] remoteproc: qcom: Add venus rproc support on msm8996 platform.

In reply to Avaneesh Kumar Dwivedi
Quoted 32 lines · Avaneesh Kumar Dwivedi

This patch is based on

	https://patchwork.kernel.org/patch/9415627/
	https://patchwork.kernel.org/patch/9415651/

This patch add clock initialization, enable and disable support. Required resource name string and rating are differentiated based on compatible string. Also added documentation for venus pil on msm8996.

---
 .../devicetree/bindings/remoteproc/qcom,venus.txt  |  26 ++++-
 drivers/remoteproc/qcom_venus_pil.c                | 116 ++++++++++++++++++++-
 2 files changed, 140 insertions(+), 2 deletions(-)
Documentation/devicetree/bindings/remoteproc/qcom,venus.txt +2 -1
index 2d73ba1..417026b 100644
@@ -6,13 +6,30 @@ on the Qualcomm Venus remote processor core.
 - compatible:
 	Usage: required
 	Value type: <string>
-	Definition: must contain "qcom,venus-pil"
+	Definition: must contain "qcom,venus-pil" or
+				"qcom,venus-msm8996-pil"
 
 - memory-region:
 	Usage: required
 	Value type: <phandle>
 	Definition: a phandle to a node describing reserved memory
 

Documentation/devicetree/bindings/remoteproc/qcom,venus.txt on the Qualcomm Venus remote processor core. · line 26

Quoted 10 lines
Documentation/devicetree/bindings/remoteproc/qcom,venus.txt +10 -0
+- clocks:
+	Usage: required
+	Value type: <prop-encoded-array>
+	Definition: reference to the core, iface and bus and maxi clocks to be held on
+		    behalf of the booting of the venus core
+
+- clock-names:
+	Usage: required
+	Value type: <stringlist>
+	Definition: should be "core_clk", "iface_clk", "bus_clk", "maxi_clk"

Please drop _clk from all clock names.

Documentation/devicetree/bindings/remoteproc/qcom,venus.txt on the Qualcomm Venus remote processor core. · line 31

Quoted 5 lines
Documentation/devicetree/bindings/remoteproc/qcom,venus.txt +5 -0
+
+- power-domains:
+	Usage: required
+	Value type: <prop-encoded-array>
+	Definition: reference to the venus gdsc to be turned on before booting venus core

All these new properties can't be required if the original compatible is used, right?

Quoted 17 lines
Documentation/devicetree/bindings/remoteproc/qcom,venus.txt +8 -0
+
 * An example
 	reserved-memory {
 		#address-cells = <2>;
@@ -29,5 +46,12 @@ on the Qualcomm Venus remote processor core.
 
 	rproc_venus@0 {
 		compatible = "qcom,venus-pil";
+		clocks = <&mmcc VIDEO_CORE_CLK>,
+				<&mmcc VIDEO_AHB_CLK>,
+				<&mmcc VIDEO_AXI_CLK>,
+				<&mmcc VIDEO_MAXI_CLK>;
+
+		clock-names = "core_clk", "iface_clk", "bus_clk", "maxi_clk";
+		power-domains = <&mmcc VENUS_GDSC>;
 		memory-region = <&venus_mem>;
 	};
Signature · 2 lines
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
a Linux Foundation Collaborative Project
#13 lore raw
In reply to Stephen Boyd
Quoted 8 lines · Stephen Boyd
Quoted 7 lines · Avaneesh Kumar Dwivedi

This Patch is based on

         https://patchwork.kernel.org/patch/9415627/
         https://patchwork.kernel.org/patch/9415651/

This patch add necessary routine and data structure to support standalone venus firmware load on msm8996. Below are brief of changes.

Quoted 6 lines
Quoted 3 lines

1- Add private data structure which provide string-name and rate of clock
on msm8996 platform for venus.
2- Provide clock initialization and enable/disable functionality.

For one patch we typically don't send cover letters. These sorts of details just go below the dashed lines after the commit text of the single patch.

OK

Signature · 2 lines
Qualcomm India Private Limited, on behalf of Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project.
#3 lore raw

Re: [RESEND PATCH v2] remoteproc: qcom: Add venus rproc support on msm8996 platform.

In reply to Stephen Boyd
Quoted 33 lines · Stephen Boyd
Quoted 32 lines · Avaneesh Kumar Dwivedi

This patch is based on

	https://patchwork.kernel.org/patch/9415627/
	https://patchwork.kernel.org/patch/9415651/

This patch add clock initialization, enable and disable support. Required resource name string and rating are differentiated based on compatible string. Also added documentation for venus pil on msm8996.

---
  .../devicetree/bindings/remoteproc/qcom,venus.txt  |  26 ++++-
  drivers/remoteproc/qcom_venus_pil.c                | 116 ++++++++++++++++++++-
  2 files changed, 140 insertions(+), 2 deletions(-)
Documentation/devicetree/bindings/remoteproc/qcom,venus.txt +2 -1
index 2d73ba1..417026b 100644
@@ -6,13 +6,30 @@ on the Qualcomm Venus remote processor core.
  - compatible:
  	Usage: required
  	Value type: <string>
-	Definition: must contain "qcom,venus-pil"
+	Definition: must contain "qcom,venus-pil" or
+				"qcom,venus-msm8996-pil"
 
  - memory-region:
  	Usage: required
  	Value type: <phandle>
  	Definition: a phandle to a node describing reserved memory
 
Quoted 11 lines

Documentation/devicetree/bindings/remoteproc/qcom,venus.txt on the Qualcomm Venus remote processor core. · line 26

Quoted 10 lines
Documentation/devicetree/bindings/remoteproc/qcom,venus.txt +10 -0
+- clocks:
+	Usage: required
+	Value type: <prop-encoded-array>
+	Definition: reference to the core, iface and bus and maxi clocks to be held on
+		    behalf of the booting of the venus core
+
+- clock-names:
+	Usage: required
+	Value type: <stringlist>
+	Definition: should be "core_clk", "iface_clk", "bus_clk", "maxi_clk"

Please drop _clk from all clock names.

OK

Quoted 8 lines

Documentation/devicetree/bindings/remoteproc/qcom,venus.txt on the Qualcomm Venus remote processor core. · line 31

Quoted 6 lines
Documentation/devicetree/bindings/remoteproc/qcom,venus.txt +5 -0
+
+- power-domains:
+	Usage: required
+	Value type: <prop-encoded-array>
+	Definition: reference to the venus gdsc to be turned on before booting venus core

All these new properties can't be required if the original compatible is used, right?

gdsc were earlier handled via regulator framework, but now gdsc's are being handled via power domain framework. so any driver which require gdsc to be enabled need to use this property on mainline kernel. so these properties will even be applicable for 8916 venus rproc on mainline kernel.

Quoted 18 lines
Quoted 18 lines
Documentation/devicetree/bindings/remoteproc/qcom,venus.txt +8 -0
+
  * An example
  	reserved-memory {
  		#address-cells = <2>;
@@ -29,5 +46,12 @@ on the Qualcomm Venus remote processor core.
 
  	rproc_venus@0 {
  		compatible = "qcom,venus-pil";
+		clocks = <&mmcc VIDEO_CORE_CLK>,
+				<&mmcc VIDEO_AHB_CLK>,
+				<&mmcc VIDEO_AXI_CLK>,
+				<&mmcc VIDEO_MAXI_CLK>;
+
+		clock-names = "core_clk", "iface_clk", "bus_clk", "maxi_clk";
+		power-domains = <&mmcc VENUS_GDSC>;
  		memory-region = <&venus_mem>;
  	};
Signature · 2 lines
Qualcomm India Private Limited, on behalf of Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project.
#4 lore raw

Re: [RESEND PATCH v2] remoteproc: qcom: Add venus rproc support on msm8996 platform.

Hi,

Quoted 58 lines · Avaneesh Kumar (avani)
Quoted 39 lines · Stephen Boyd
Quoted 38 lines · Avaneesh Kumar Dwivedi

This patch is based on

    https://patchwork.kernel.org/patch/9415627/
    https://patchwork.kernel.org/patch/9415651/

This patch add clock initialization, enable and disable support. Required resource name string and rating are differentiated based on compatible string. Also added documentation for venus pil on msm8996.

---
  .../devicetree/bindings/remoteproc/qcom,venus.txt  |  26 ++++-
  drivers/remoteproc/qcom_venus_pil.c                | 116
++++++++++++++++++++-
  2 files changed, 140 insertions(+), 2 deletions(-)

diff --git a/Documentation/devicetree/bindings/remoteproc/qcom,venus.txt b/Documentation/devicetree/bindings/remoteproc/qcom,venus.txt index 2d73ba1..417026b 100644

Documentation/devicetree/bindings/remoteproc/qcom,venus.txt +5 -1
@@ -6,13 +6,30 @@ on the Qualcomm Venus remote processor core.
  - compatible:
      Usage: required
      Value type: <string>
-    Definition: must contain "qcom,venus-pil"
+    Definition: must contain "qcom,venus-pil" or
+                "qcom,venus-msm8996-pil"
    - memory-region:
      Usage: required
      Value type: <phandle>
      Definition: a phandle to a node describing reserved memory
  +- clocks:
+    Usage: required
+    Value type: <prop-encoded-array>
+    Definition: reference to the core, iface and bus and maxi clocks

to be held on

Quoted 8 lines
Quoted 7 lines
Documentation/devicetree/bindings/remoteproc/qcom,venus.txt +6 -0
+            behalf of the booting of the venus core
+
+- clock-names:
+    Usage: required
+    Value type: <stringlist>
+    Definition: should be "core_clk", "iface_clk", "bus_clk",

"maxi_clk"

Please drop _clk from all clock names.

OK

Quoted 7 lines
Quoted 7 lines
Documentation/devicetree/bindings/remoteproc/qcom,venus.txt +5 -0
+
+- power-domains:
+    Usage: required
+    Value type: <prop-encoded-array>
+    Definition: reference to the venus gdsc to be turned on before

booting venus core

Quoted 8 lines
Quoted 2 lines

All these new properties can't be required if the original compatible is used, right?

gdsc were earlier handled via regulator framework, but now gdsc's are being handled via power domain framework.
so any driver which require gdsc to be enabled need to use this property on mainline kernel.
so these properties will even be applicable for 8916 venus rproc on mainline kernel.

Then I'd propose to add compatible string for msm8916 too. We will need to distinguish between SoCs in order to select proper Venus firmware file too.

I think using power domain in remoteproc driver will break power management of the v4l2 venus driver. Presently I use runtime pm to switch ON/OFF GDSC and enable/disable Venus clocks.

When the .probe of venus remoteproc driver is called the power domain will become ON and never switched OFF (except when platform driver .remove method is called). One possible solution would be to add runtime pm in venus remoreproc driver.

IMO we should think more about that.

Bjorn, Stephen what are your opinions?

Signature · 2 lines
regards,
Stan
#5 lore raw

Re: [RESEND PATCH v2] remoteproc: qcom: Add venus rproc support on msm8996 platform.

Quoted 75 lines · Stanimir Varbanov

Hi,

Quoted 56 lines · Avaneesh Kumar (avani)
Quoted 39 lines · Stephen Boyd
Quoted 38 lines · Avaneesh Kumar Dwivedi

This patch is based on

     https://patchwork.kernel.org/patch/9415627/
     https://patchwork.kernel.org/patch/9415651/

This patch add clock initialization, enable and disable support. Required resource name string and rating are differentiated based on compatible string. Also added documentation for venus pil on msm8996.

---
   .../devicetree/bindings/remoteproc/qcom,venus.txt  |  26 ++++-
   drivers/remoteproc/qcom_venus_pil.c                | 116
++++++++++++++++++++-
   2 files changed, 140 insertions(+), 2 deletions(-)

diff --git a/Documentation/devicetree/bindings/remoteproc/qcom,venus.txt b/Documentation/devicetree/bindings/remoteproc/qcom,venus.txt index 2d73ba1..417026b 100644

Documentation/devicetree/bindings/remoteproc/qcom,venus.txt +5 -1
@@ -6,13 +6,30 @@ on the Qualcomm Venus remote processor core.
   - compatible:
       Usage: required
       Value type: <string>
-    Definition: must contain "qcom,venus-pil"
+    Definition: must contain "qcom,venus-pil" or
+                "qcom,venus-msm8996-pil"
     - memory-region:
       Usage: required
       Value type: <phandle>
       Definition: a phandle to a node describing reserved memory
   +- clocks:
+    Usage: required
+    Value type: <prop-encoded-array>
+    Definition: reference to the core, iface and bus and maxi clocks

to be held on

Quoted 8 lines
Quoted 7 lines
Documentation/devicetree/bindings/remoteproc/qcom,venus.txt +6 -0
+            behalf of the booting of the venus core
+
+- clock-names:
+    Usage: required
+    Value type: <stringlist>
+    Definition: should be "core_clk", "iface_clk", "bus_clk",

"maxi_clk"

Please drop _clk from all clock names.

OK

Quoted 6 lines
Quoted 6 lines
Documentation/devicetree/bindings/remoteproc/qcom,venus.txt +5 -0
+
+- power-domains:
+    Usage: required
+    Value type: <prop-encoded-array>
+    Definition: reference to the venus gdsc to be turned on before

booting venus core

Quoted 8 lines
Quoted 2 lines

All these new properties can't be required if the original compatible is used, right?

gdsc were earlier handled via regulator framework, but now gdsc's are being handled via power domain framework.
so any driver which require gdsc to be enabled need to use this property on mainline kernel.
so these properties will even be applicable for 8916 venus rproc on mainline kernel.

Then I'd propose to add compatible string for msm8916 too. We will need to distinguish between SoCs in order to select proper Venus firmware file too.

I think using power domain in remoteproc driver will break power management of the v4l2 venus driver. Presently I use runtime pm to switch ON/OFF GDSC and enable/disable Venus clocks.

Quoted 9 lines

When the .probe of venus remoteproc driver is called the power domain will become ON and never switched OFF (except when platform driver .remove method is called). One possible solution would be to add runtime pm in venus remoreproc driver.

IMO we should think more about that.

Bjorn, Stephen what are your opinions?

Shall I go ahead and add runtime PM API calls during shutdown/start call along with other changes to address other comments? i was waiting any comment from Sboyed and Bjorn.

Signature · 2 lines
Qualcomm India Private Limited, on behalf of Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project.
#6 lore raw

Re: [RESEND PATCH v2] remoteproc: qcom: Add venus rproc support on msm8996 platform.

Quoted 83 lines · Avaneesh Kumar (avani)
Quoted 80 lines · Stanimir Varbanov

Hi,

Quoted 56 lines · Avaneesh Kumar (avani)
Quoted 39 lines · Stephen Boyd
Quoted 38 lines · Avaneesh Kumar Dwivedi

This patch is based on

     https://patchwork.kernel.org/patch/9415627/
     https://patchwork.kernel.org/patch/9415651/

This patch add clock initialization, enable and disable support. Required resource name string and rating are differentiated based on compatible string. Also added documentation for venus pil on msm8996.

---
   .../devicetree/bindings/remoteproc/qcom,venus.txt  |  26 ++++-
   drivers/remoteproc/qcom_venus_pil.c                | 116
++++++++++++++++++++-
   2 files changed, 140 insertions(+), 2 deletions(-)

diff --git a/Documentation/devicetree/bindings/remoteproc/qcom,venus.txt b/Documentation/devicetree/bindings/remoteproc/qcom,venus.txt index 2d73ba1..417026b 100644

Documentation/devicetree/bindings/remoteproc/qcom,venus.txt +5 -1
@@ -6,13 +6,30 @@ on the Qualcomm Venus remote processor core.
   - compatible:
       Usage: required
       Value type: <string>
-    Definition: must contain "qcom,venus-pil"
+    Definition: must contain "qcom,venus-pil" or
+                "qcom,venus-msm8996-pil"
     - memory-region:
       Usage: required
       Value type: <phandle>
       Definition: a phandle to a node describing reserved memory
   +- clocks:
+    Usage: required
+    Value type: <prop-encoded-array>
+    Definition: reference to the core, iface and bus and maxi clocks

to be held on

Quoted 8 lines
Quoted 7 lines
Documentation/devicetree/bindings/remoteproc/qcom,venus.txt +6 -0
+            behalf of the booting of the venus core
+
+- clock-names:
+    Usage: required
+    Value type: <stringlist>
+    Definition: should be "core_clk", "iface_clk", "bus_clk",

"maxi_clk"

Please drop _clk from all clock names.

OK

Quoted 6 lines
Quoted 6 lines
Documentation/devicetree/bindings/remoteproc/qcom,venus.txt +5 -0
+
+- power-domains:
+    Usage: required
+    Value type: <prop-encoded-array>
+    Definition: reference to the venus gdsc to be turned on before

booting venus core

Quoted 8 lines
Quoted 2 lines

All these new properties can't be required if the original compatible is used, right?

gdsc were earlier handled via regulator framework, but now gdsc's are being handled via power domain framework.
so any driver which require gdsc to be enabled need to use this property on mainline kernel.
so these properties will even be applicable for 8916 venus rproc on mainline kernel.

Then I'd propose to add compatible string for msm8916 too. We will need to distinguish between SoCs in order to select proper Venus firmware file too.

I think using power domain in remoteproc driver will break power management of the v4l2 venus driver. Presently I use runtime pm to switch ON/OFF GDSC and enable/disable Venus clocks.

When the .probe of venus remoteproc driver is called the power domain will become ON and never switched OFF (except when platform driver .remove method is called). One possible solution would be to add runtime pm in venus remoreproc driver.

Quoted 8 lines
Quoted 4 lines

IMO we should think more about that.

Bjorn, Stephen what are your opinions?

Shall I go ahead and add runtime PM API calls during shutdown/start call along with other changes to address other comments? i was waiting any comment from Sboyed and Bjorn.

IMO runtime should be added in remoteproc core, so you can start with an RFC patch to see how is going.

Signature · 2 lines
regards,
Stan
#7 lore raw

Re: [RESEND PATCH v2] remoteproc: qcom: Add venus rproc support on msm8996 platform.

In reply to Stanimir Varbanov

On Wed 30 Nov 01:22 PST 2016, Stanimir Varbanov wrote:
[..]

Quoted 5 lines

Then I'd propose to add compatible string for msm8916 too. We will need to distinguish between SoCs in order to select proper Venus firmware file too.

This does make sense.

Quoted 4 lines

I think using power domain in remoteproc driver will break power management of the v4l2 venus driver. Presently I use runtime pm to switch ON/OFF GDSC and enable/disable Venus clocks.

Quoted 9 lines

When the .probe of venus remoteproc driver is called the power domain will become ON and never switched OFF (except when platform driver .remove method is called). One possible solution would be to add runtime pm in venus remoreproc driver.

IMO we should think more about that.

Bjorn, Stephen what are your opinions?

Sorry for not getting back to you on this after our earlier discussion on the topic.

Modelling the remoteproc and codec drivers as completely separate entities (with only the rproc phandle) gives us issues with the timing between the two. As I pulled down and started playing with [1] I noticed that I was not able to get the codec driver to probe unless I compiled it as a module which I insmoded after my /lib/firmware became available.

In addition to this, there's no way for the remoteproc driver to inform the codec driver when it's actually shutting down and booting back up during a crash recovery (an async operation).

And lastly when I talked to Rob about the DT binding the obvious question of why one piece of hardware have two disjunct nodes in the DT.

So I believe we should represent the codec driver as a child of the remoteproc driver, utilizing the newly introduced "remoteproc subdevices" to probe and remove the codec driver based on the state of the remoteproc driver. This relationship will also allow us to tie certain resources (e.g. the clocks and power-domain) to the remoteproc driver and use pm_runtime in either driver to make sure the resources are enabled appropriately.

I did backport some patches from my v4.10 remoteproc pull request into
[2] and merged this into [1] and made a prototype of this. You can find it here [3], I did test that I can boot the remoteproc and run v4l2-compliance, shut it down, boot it back up and re-run v4l2-compliance, and it seems to work fine.

With these patches you can compile the remoteproc driver builtin to the kernel and once your system is booted you can issue:

echo start > /sys/class/remoteproc/remoteproc0/state

and venus should boot and the codec should be probed.

echo stop > /sys/class/remoteproc/remoteproc0/state

will shut down the core after removing the codec.

Please let me know if you think this is a viable solution.

(This is only tested on db410c so far, looks like we might need a few more clocks for 8996 and we should probably be explicit about the noc and its resources)

[1] https://git.linaro.org/people/stanimir.varbanov/linux.git/log/?h=release/qcomlt-4.9-venus-wip
[2] https://github.com/andersson/kernel/commits/qcomlt/v4.9-rproc-backport
[3] https://github.com/andersson/kernel/commits/qcomlt/venus-wip

Regards,
Bjorn

#8 lore raw

Re: [RESEND PATCH v2] remoteproc: qcom: Add venus rproc support on msm8996 platform.

Quoted 42 lines · Bjorn Andersson

On Wed 30 Nov 01:22 PST 2016, Stanimir Varbanov wrote:
[..]

Quoted 4 lines

I think using power domain in remoteproc driver will break power management of the v4l2 venus driver. Presently I use runtime pm to switch ON/OFF GDSC and enable/disable Venus clocks.

Quoted 9 lines

When the .probe of venus remoteproc driver is called the power domain will become ON and never switched OFF (except when platform driver .remove method is called). One possible solution would be to add runtime pm in venus remoreproc driver.

IMO we should think more about that.

Bjorn, Stephen what are your opinions?

Sorry for not getting back to you on this after our earlier discussion on the topic.

Modelling the remoteproc and codec drivers as completely separate entities (with only the rproc phandle) gives us issues with the timing between the two. As I pulled down and started playing with [1] I noticed that I was not able to get the codec driver to probe unless I compiled it as a module which I insmoded after my /lib/firmware became available.

In addition to this, there's no way for the remoteproc driver to inform the codec driver when it's actually shutting down and booting back up during a crash recovery (an async operation).

And lastly when I talked to Rob about the DT binding the obvious question of why one piece of hardware have two disjunct nodes in the DT.

So I believe we should represent the codec driver as a child of the remoteproc driver, utilizing the newly introduced "remoteproc subdevices" to probe and remove the codec driver based on the state of the remoteproc driver. This relationship will also allow us to tie certain resources (e.g. the clocks and power-domain) to the remoteproc driver and use pm_runtime in either driver to make sure the resources are enabled appropriately.

Quoted 8 lines

I did backport some patches from my v4.10 remoteproc pull request into
[2] and merged this into [1] and made a prototype of this. You can find it here [3], I did test that I can boot the remoteproc and run v4l2-compliance, shut it down, boot it back up and re-run v4l2-compliance, and it seems to work fine.

From a DT perspective this seems backwards. We shouldn't be

putting something with a reg property underneath a "virtual" node that doesn't have a reg property. It's already causing pain, evident by these sorts of patches, so something seems wrong.

Are we gaining anything by having a remoteproc driver and device for the video hardware here? Why can't the video driver load/unload firmware with the mdt loader code by itself when it wants to boot the processor? That seems like a much simpler design and it nicely avoids this DT confusion.

Signature · 2 lines
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
a Linux Foundation Collaborative Project
#9 lore raw

Re: [RESEND PATCH v2] remoteproc: qcom: Add venus rproc support on msm8996 platform.

Quoted 50 lines · Stephen Boyd
Quoted 42 lines · Bjorn Andersson

On Wed 30 Nov 01:22 PST 2016, Stanimir Varbanov wrote:
[..]

Quoted 4 lines

I think using power domain in remoteproc driver will break power management of the v4l2 venus driver. Presently I use runtime pm to switch ON/OFF GDSC and enable/disable Venus clocks.

Quoted 9 lines

When the .probe of venus remoteproc driver is called the power domain will become ON and never switched OFF (except when platform driver .remove method is called). One possible solution would be to add runtime pm in venus remoreproc driver.

IMO we should think more about that.

Bjorn, Stephen what are your opinions?

Sorry for not getting back to you on this after our earlier discussion on the topic.

Modelling the remoteproc and codec drivers as completely separate entities (with only the rproc phandle) gives us issues with the timing between the two. As I pulled down and started playing with [1] I noticed that I was not able to get the codec driver to probe unless I compiled it as a module which I insmoded after my /lib/firmware became available.

In addition to this, there's no way for the remoteproc driver to inform the codec driver when it's actually shutting down and booting back up during a crash recovery (an async operation).

And lastly when I talked to Rob about the DT binding the obvious question of why one piece of hardware have two disjunct nodes in the DT.

So I believe we should represent the codec driver as a child of the remoteproc driver, utilizing the newly introduced "remoteproc subdevices" to probe and remove the codec driver based on the state of the remoteproc driver. This relationship will also allow us to tie certain resources (e.g. the clocks and power-domain) to the remoteproc driver and use pm_runtime in either driver to make sure the resources are enabled appropriately.

Quoted 6 lines

I did backport some patches from my v4.10 remoteproc pull request into
[2] and merged this into [1] and made a prototype of this. You can find it here [3], I did test that I can boot the remoteproc and run v4l2-compliance, shut it down, boot it back up and re-run v4l2-compliance, and it seems to work fine.

Quoted 5 lines

From a DT perspective this seems backwards. We shouldn't be putting something with a reg property underneath a "virtual" node that doesn't have a reg property. It's already causing pain, evident by these sorts of patches, so something seems wrong.

I agree.

Quoted 2 lines

Are we gaining anything by having a remoteproc driver and device for the video hardware here?

After discussing the matter with Avaneesh I realized that the downstream driver powers the venus core up and down based on the presence of clients. I expect that this means we have to do the same to be able to meet our power KPIs.

With this in mind the remoteproc driver becomes a wrapper for a mdt loader and the scm pas interface - and brings with it a bunch of other things, expectations and challenges.

Quoted 2 lines

Why can't the video driver load/unload firmware with the mdt loader code by itself when it wants to boot the processor?

Providing a saner api around the mdt-loader and pas would allow venus and Adreno to reuse the code, without the remoteproc bloat.

Quoted 3 lines

That seems like a much simpler design and it nicely avoids this DT confusion.

Agreed.

I was under the impression that the venus core was always on, but after re-reading the downstream venus code a few more times I think this would be much cleaner to do so, both implementation and binding wise.

Regards,
Bjorn

#10 lore raw

Re: [RESEND PATCH v2] remoteproc: qcom: Add venus rproc support on msm8996 platform.

Quoted 73 lines · Bjorn Andersson
Quoted 48 lines · Stephen Boyd
Quoted 41 lines · Bjorn Andersson

On Wed 30 Nov 01:22 PST 2016, Stanimir Varbanov wrote:
[..]

Quoted 4 lines

I think using power domain in remoteproc driver will break power management of the v4l2 venus driver. Presently I use runtime pm to switch ON/OFF GDSC and enable/disable Venus clocks.

Quoted 9 lines

When the .probe of venus remoteproc driver is called the power domain will become ON and never switched OFF (except when platform driver .remove method is called). One possible solution would be to add runtime pm in venus remoreproc driver.

IMO we should think more about that.

Bjorn, Stephen what are your opinions?

Sorry for not getting back to you on this after our earlier discussion on the topic.

Modelling the remoteproc and codec drivers as completely separate entities (with only the rproc phandle) gives us issues with the timing between the two. As I pulled down and started playing with [1] I noticed that I was not able to get the codec driver to probe unless I compiled it as a module which I insmoded after my /lib/firmware became available.

In addition to this, there's no way for the remoteproc driver to inform the codec driver when it's actually shutting down and booting back up during a crash recovery (an async operation).

And lastly when I talked to Rob about the DT binding the obvious question of why one piece of hardware have two disjunct nodes in the DT.

So I believe we should represent the codec driver as a child of the remoteproc driver, utilizing the newly introduced "remoteproc subdevices" to probe and remove the codec driver based on the state of the remoteproc driver. This relationship will also allow us to tie certain resources (e.g. the clocks and power-domain) to the remoteproc driver and use pm_runtime in either driver to make sure the resources are enabled appropriately.

Quoted 6 lines

I did backport some patches from my v4.10 remoteproc pull request into
[2] and merged this into [1] and made a prototype of this. You can find it here [3], I did test that I can boot the remoteproc and run v4l2-compliance, shut it down, boot it back up and re-run v4l2-compliance, and it seems to work fine.

Quoted 5 lines

From a DT perspective this seems backwards. We shouldn't be
putting something with a reg property underneath a "virtual" node that doesn't have a reg property. It's already causing pain, evident by these sorts of patches, so something seems wrong.

I agree.

Quoted 2 lines

Are we gaining anything by having a remoteproc driver and device for the video hardware here?

After discussing the matter with Avaneesh I realized that the downstream driver powers the venus core up and down based on the presence of clients. I expect that this means we have to do the same to be able to meet our power KPIs.

With this in mind the remoteproc driver becomes a wrapper for a mdt loader and the scm pas interface - and brings with it a bunch of other things, expectations and challenges.

Quoted 2 lines

Why can't the video driver load/unload firmware with the mdt loader code by itself when it wants to boot the processor?

Providing a saner api around the mdt-loader and pas would allow venus and Adreno to reuse the code, without the remoteproc bloat.

Quoted 8 lines
Quoted 3 lines

That seems like a much simpler design and it nicely avoids this DT confusion.

Agreed.

I was under the impression that the venus core was always on, but after re-reading the downstream venus code a few more times I think this would be much cleaner to do so, both implementation and binding wise.

Does It mean that, you are thinking to abolish venus rproc driver? and video driver will use mdt loader and scm interface to boot venus core as clients make call? effectively avoiding any requirement for separate DT node for venus rproc?

Quoted 3 lines

Regards,
Bjorn

Signature · 2 lines
Qualcomm India Private Limited, on behalf of Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project.
#11 lore raw

Re: [RESEND PATCH v2] remoteproc: qcom: Add venus rproc support on msm8996 platform.

In reply to Bjorn Andersson

Hi Bjorn,

Quoted 72 lines · Bjorn Andersson
Quoted 50 lines · Stephen Boyd
Quoted 42 lines · Bjorn Andersson

On Wed 30 Nov 01:22 PST 2016, Stanimir Varbanov wrote:
[..]

Quoted 4 lines

I think using power domain in remoteproc driver will break power management of the v4l2 venus driver. Presently I use runtime pm to switch ON/OFF GDSC and enable/disable Venus clocks.

Quoted 9 lines

When the .probe of venus remoteproc driver is called the power domain will become ON and never switched OFF (except when platform driver .remove method is called). One possible solution would be to add runtime pm in venus remoreproc driver.

IMO we should think more about that.

Bjorn, Stephen what are your opinions?

Sorry for not getting back to you on this after our earlier discussion on the topic.

Modelling the remoteproc and codec drivers as completely separate entities (with only the rproc phandle) gives us issues with the timing between the two. As I pulled down and started playing with [1] I noticed that I was not able to get the codec driver to probe unless I compiled it as a module which I insmoded after my /lib/firmware became available.

In addition to this, there's no way for the remoteproc driver to inform the codec driver when it's actually shutting down and booting back up during a crash recovery (an async operation).

And lastly when I talked to Rob about the DT binding the obvious question of why one piece of hardware have two disjunct nodes in the DT.

So I believe we should represent the codec driver as a child of the remoteproc driver, utilizing the newly introduced "remoteproc subdevices" to probe and remove the codec driver based on the state of the remoteproc driver. This relationship will also allow us to tie certain resources (e.g. the clocks and power-domain) to the remoteproc driver and use pm_runtime in either driver to make sure the resources are enabled appropriately.

Quoted 6 lines

I did backport some patches from my v4.10 remoteproc pull request into
[2] and merged this into [1] and made a prototype of this. You can find it here [3], I did test that I can boot the remoteproc and run v4l2-compliance, shut it down, boot it back up and re-run v4l2-compliance, and it seems to work fine.

Quoted 5 lines

From a DT perspective this seems backwards. We shouldn't be putting something with a reg property underneath a "virtual" node that doesn't have a reg property. It's already causing pain, evident by these sorts of patches, so something seems wrong.

I agree.

Quoted 2 lines

Are we gaining anything by having a remoteproc driver and device for the video hardware here?

After discussing the matter with Avaneesh I realized that the downstream driver powers the venus core up and down based on the presence of clients. I expect that this means we have to do the same to be able to meet our power KPIs.

With this in mind the remoteproc driver becomes a wrapper for a mdt loader and the scm pas interface - and brings with it a bunch of other things, expectations and challenges.

Quoted 5 lines
Quoted 2 lines

Why can't the video driver load/unload firmware with the mdt loader code by itself when it wants to boot the processor?

Providing a saner api around the mdt-loader and pas would allow venus and Adreno to reuse the code, without the remoteproc bloat.

Does it make sense if we move mdt parser in drivers/firmware ?

Signature · 2 lines
regards,
Stan