remoteproc: qcom: Venus firmware loader support for msm8996
PATCH RESEND v2 1/1 remoteproc: qcom: Venus firmware loader support for msm8996
This Patch is based on
https://patchwork.kernel.org/patch/9415627/ https://patchwork.kernel.org/patch/9415651/
This patch add necessary routine and data structure to support standalone venus firmware load on msm8996. Below are brief of changes.
1- Add private data structure which provide string-name and rate of clock
on msm8996 platform for venus.
2- Provide clock initialization and enable/disable functionality.
below is console log on msm8996 platform with above change, this is standalone test log without video driver enablement.
[ 2.612011] remoteproc0: soc:vidc_tzpil@0 is available [ 2.616939] remoteproc0: Note: remoteproc is still under development and considered experimental. [ 2.621963] remoteproc0: THE BINARY FORMAT IS NOT YET FINALIZED, and backward compatibility isn't yet guaranteed. [ 2.631037] qcom-tz-pil soc:vidc_tzpil@0: Venus rproc probe done [ 2.641463] remoteproc0: powering up soc:vidc_tzpil@0 [ 2.641468] remoteproc0: Booting fw image venus.mdt, size 6812 [ 2.698127] remoteproc0: remote processor soc:vidc_tzpil@0 is now up
Changes w.r.t. patchset 1
reorganization and cleanup of added code.
Avaneesh Kumar Dwivedi (1):
remoteproc: qcom: Add venus rproc support on msm8996 platform.
.../devicetree/bindings/remoteproc/qcom,venus.txt | 26 ++++- drivers/remoteproc/qcom_venus_pil.c | 116 ++++++++++++++++++++- 2 files changed, 140 insertions(+), 2 deletions(-)
Signature · 2 lines
Qualcomm India Private Limited, on behalf of Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum, a Linux Foundation Collaborative Project.
PATCH RESEND v2 1/1 remoteproc: qcom: Add venus rproc support on msm8996 platform.
This patch is based on
https://patchwork.kernel.org/patch/9415627/ https://patchwork.kernel.org/patch/9415651/
This patch add clock initialization, enable and disable support. Required resource name string and rating are differentiated based on compatible string. Also added documentation for venus pil on msm8996.
--- .../devicetree/bindings/remoteproc/qcom,venus.txt | 26 ++++- drivers/remoteproc/qcom_venus_pil.c | 116 ++++++++++++++++++++- 2 files changed, 140 insertions(+), 2 deletions(-)
Documentation/devicetree/bindings/remoteproc/qcom,venus.txt +25 -1
@@ -6,13 +6,30 @@ on the Qualcomm Venus remote processor core. - compatible: Usage: required Value type: <string> - Definition: must contain "qcom,venus-pil" + Definition: must contain "qcom,venus-pil" or + "qcom,venus-msm8996-pil" - memory-region: Usage: required Value type: <phandle> Definition: a phandle to a node describing reserved memory +- clocks: + Usage: required + Value type: <prop-encoded-array> + Definition: reference to the core, iface and bus and maxi clocks to be held on + behalf of the booting of the venus core + +- clock-names: + Usage: required + Value type: <stringlist> + Definition: should be "core_clk", "iface_clk", "bus_clk", "maxi_clk" + +- power-domains: + Usage: required + Value type: <prop-encoded-array> + Definition: reference to the venus gdsc to be turned on before booting venus core + * An example reserved-memory { #address-cells = <2>;
@@ -29,5 +46,12 @@ on the Qualcomm Venus remote processor core. rproc_venus@0 { compatible = "qcom,venus-pil"; + clocks = <&mmcc VIDEO_CORE_CLK>, + <&mmcc VIDEO_AHB_CLK>, + <&mmcc VIDEO_AXI_CLK>, + <&mmcc VIDEO_MAXI_CLK>; + + clock-names = "core_clk", "iface_clk", "bus_clk", "maxi_clk"; + power-domains = <&mmcc VENUS_GDSC>; memory-region = <&venus_mem>; };
drivers/remoteproc/qcom_venus_pil.c +115 -1
179 lines not on this page · raw message
Signature · 2 lines
Qualcomm India Private Limited, on behalf of Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum, a Linux Foundation Collaborative Project.
Quoted 10 lines · Avaneesh Kumar Dwivedi
This Patch is based on
https://patchwork.kernel.org/patch/9415627/ https://patchwork.kernel.org/patch/9415651/This patch add necessary routine and data structure to support standalone venus firmware load on msm8996. Below are brief of changes.
1- Add private data structure which provide string-name and rate of clock
on msm8996 platform for venus.
2- Provide clock initialization and enable/disable functionality.
For one patch we typically don't send cover letters. These sorts of details just go below the dashed lines after the commit text of the single patch.
Signature · 2 lines
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project
Re: [RESEND PATCH v2] remoteproc: qcom: Add venus rproc support on msm8996 platform.
In reply to Avaneesh Kumar DwivediQuoted 32 lines · Avaneesh Kumar Dwivedi
This patch is based on
https://patchwork.kernel.org/patch/9415627/ https://patchwork.kernel.org/patch/9415651/This patch add clock initialization, enable and disable support. Required resource name string and rating are differentiated based on compatible string. Also added documentation for venus pil on msm8996.
Signed-off-by Avaneesh Kumar Dwivedi <[email protected]>--- .../devicetree/bindings/remoteproc/qcom,venus.txt | 26 ++++- drivers/remoteproc/qcom_venus_pil.c | 116 ++++++++++++++++++++- 2 files changed, 140 insertions(+), 2 deletions(-)Documentation/devicetree/bindings/remoteproc/qcom,venus.txt +2
-1@@ -6,13 +6,30 @@ on the Qualcomm Venus remote processor core. - compatible: Usage: required Value type: <string> - Definition: must contain "qcom,venus-pil" + Definition: must contain "qcom,venus-pil" or + "qcom,venus-msm8996-pil" - memory-region: Usage: required Value type: <phandle> Definition: a phandle to a node describing reserved memory
Documentation/devicetree/bindings/remoteproc/qcom,venus.txt on the Qualcomm Venus remote processor core. · line 26
Quoted 10 lines
Documentation/devicetree/bindings/remoteproc/qcom,venus.txt +10
-0+- clocks: + Usage: required + Value type: <prop-encoded-array> + Definition: reference to the core, iface and bus and maxi clocks to be held on + behalf of the booting of the venus core + +- clock-names: + Usage: required + Value type: <stringlist> + Definition: should be "core_clk", "iface_clk", "bus_clk", "maxi_clk"
Please drop _clk from all clock names.
Documentation/devicetree/bindings/remoteproc/qcom,venus.txt on the Qualcomm Venus remote processor core. · line 31
Quoted 5 lines
Documentation/devicetree/bindings/remoteproc/qcom,venus.txt +5
-0+ +- power-domains: + Usage: required + Value type: <prop-encoded-array> + Definition: reference to the venus gdsc to be turned on before booting venus core
All these new properties can't be required if the original compatible is used, right?
Quoted 17 lines
Documentation/devicetree/bindings/remoteproc/qcom,venus.txt +8
-0+ * An example reserved-memory { #address-cells = <2>;@@ -29,5 +46,12 @@ on the Qualcomm Venus remote processor core. rproc_venus@0 { compatible = "qcom,venus-pil"; + clocks = <&mmcc VIDEO_CORE_CLK>, + <&mmcc VIDEO_AHB_CLK>, + <&mmcc VIDEO_AXI_CLK>, + <&mmcc VIDEO_MAXI_CLK>; + + clock-names = "core_clk", "iface_clk", "bus_clk", "maxi_clk"; + power-domains = <&mmcc VENUS_GDSC>; memory-region = <&venus_mem>; };
Signature · 2 lines
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project
Quoted 8 lines · Stephen Boyd
Quoted 7 lines · Avaneesh Kumar Dwivedi
This Patch is based on
https://patchwork.kernel.org/patch/9415627/ https://patchwork.kernel.org/patch/9415651/This patch add necessary routine and data structure to support standalone venus firmware load on msm8996. Below are brief of changes.
Quoted 6 lines
Quoted 3 lines
1- Add private data structure which provide string-name and rate of clock
on msm8996 platform for venus.
2- Provide clock initialization and enable/disable functionality.For one patch we typically don't send cover letters. These sorts of details just go below the dashed lines after the commit text of the single patch.
OK
Signature · 2 lines
Qualcomm India Private Limited, on behalf of Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum, a Linux Foundation Collaborative Project.
Re: [RESEND PATCH v2] remoteproc: qcom: Add venus rproc support on msm8996 platform.
In reply to Stephen BoydQuoted 33 lines · Stephen Boyd
Quoted 32 lines · Avaneesh Kumar Dwivedi
This patch is based on
https://patchwork.kernel.org/patch/9415627/ https://patchwork.kernel.org/patch/9415651/This patch add clock initialization, enable and disable support. Required resource name string and rating are differentiated based on compatible string. Also added documentation for venus pil on msm8996.
Signed-off-by Avaneesh Kumar Dwivedi <[email protected]>--- .../devicetree/bindings/remoteproc/qcom,venus.txt | 26 ++++- drivers/remoteproc/qcom_venus_pil.c | 116 ++++++++++++++++++++- 2 files changed, 140 insertions(+), 2 deletions(-)Documentation/devicetree/bindings/remoteproc/qcom,venus.txt +2
-1@@ -6,13 +6,30 @@ on the Qualcomm Venus remote processor core. - compatible: Usage: required Value type: <string> - Definition: must contain "qcom,venus-pil" + Definition: must contain "qcom,venus-pil" or + "qcom,venus-msm8996-pil" - memory-region: Usage: required Value type: <phandle> Definition: a phandle to a node describing reserved memory
Quoted 11 lines
Documentation/devicetree/bindings/remoteproc/qcom,venus.txt on the Qualcomm Venus remote processor core. · line 26
Quoted 10 lines
Documentation/devicetree/bindings/remoteproc/qcom,venus.txt +10
-0+- clocks: + Usage: required + Value type: <prop-encoded-array> + Definition: reference to the core, iface and bus and maxi clocks to be held on + behalf of the booting of the venus core + +- clock-names: + Usage: required + Value type: <stringlist> + Definition: should be "core_clk", "iface_clk", "bus_clk", "maxi_clk"Please drop
_clkfrom all clock names.
OK
Quoted 8 lines
Documentation/devicetree/bindings/remoteproc/qcom,venus.txt on the Qualcomm Venus remote processor core. · line 31
Quoted 6 lines
Documentation/devicetree/bindings/remoteproc/qcom,venus.txt +5
-0+ +- power-domains: + Usage: required + Value type: <prop-encoded-array> + Definition: reference to the venus gdsc to be turned on before booting venus coreAll these new properties can't be required if the original compatible is used, right?
gdsc were earlier handled via regulator framework, but now gdsc's are being handled via power domain framework. so any driver which require gdsc to be enabled need to use this property on mainline kernel. so these properties will even be applicable for 8916 venus rproc on mainline kernel.
Quoted 18 lines
Quoted 18 lines
Documentation/devicetree/bindings/remoteproc/qcom,venus.txt +8
-0+ * An example reserved-memory { #address-cells = <2>;@@ -29,5 +46,12 @@ on the Qualcomm Venus remote processor core. rproc_venus@0 { compatible = "qcom,venus-pil"; + clocks = <&mmcc VIDEO_CORE_CLK>, + <&mmcc VIDEO_AHB_CLK>, + <&mmcc VIDEO_AXI_CLK>, + <&mmcc VIDEO_MAXI_CLK>; + + clock-names = "core_clk", "iface_clk", "bus_clk", "maxi_clk"; + power-domains = <&mmcc VENUS_GDSC>; memory-region = <&venus_mem>; };
Signature · 2 lines
Qualcomm India Private Limited, on behalf of Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum, a Linux Foundation Collaborative Project.
Re: [RESEND PATCH v2] remoteproc: qcom: Add venus rproc support on msm8996 platform.
Hi,
Quoted 58 lines · Avaneesh Kumar (avani)
Quoted 39 lines · Stephen Boyd
Quoted 38 lines · Avaneesh Kumar Dwivedi
This patch is based on
https://patchwork.kernel.org/patch/9415627/ https://patchwork.kernel.org/patch/9415651/This patch add clock initialization, enable and disable support. Required resource name string and rating are differentiated based on compatible string. Also added documentation for venus pil on msm8996.
Signed-off-by Avaneesh Kumar Dwivedi <[email protected]>--- .../devicetree/bindings/remoteproc/qcom,venus.txt | 26 ++++- drivers/remoteproc/qcom_venus_pil.c | 116 ++++++++++++++++++++- 2 files changed, 140 insertions(+), 2 deletions(-)diff --git
a/Documentation/devicetree/bindings/remoteproc/qcom,venus.txtb/Documentation/devicetree/bindings/remoteproc/qcom,venus.txtindex 2d73ba1..417026b 100644Documentation/devicetree/bindings/remoteproc/qcom,venus.txt +5
-1@@ -6,13 +6,30 @@ on the Qualcomm Venus remote processor core. - compatible: Usage: required Value type: <string> - Definition: must contain "qcom,venus-pil" + Definition: must contain "qcom,venus-pil" or + "qcom,venus-msm8996-pil" - memory-region: Usage: required Value type: <phandle> Definition: a phandle to a node describing reserved memory +- clocks: + Usage: required + Value type: <prop-encoded-array> + Definition: reference to the core, iface and bus and maxi clocksto be held on
Quoted 8 lines
Quoted 7 lines
Documentation/devicetree/bindings/remoteproc/qcom,venus.txt +6
-0+ behalf of the booting of the venus core + +- clock-names: + Usage: required + Value type: <stringlist> + Definition: should be "core_clk", "iface_clk", "bus_clk","
maxi_clk"Please drop
_clkfrom all clock names.OK
Quoted 7 lines
Quoted 7 lines
Documentation/devicetree/bindings/remoteproc/qcom,venus.txt +5
-0+ +- power-domains: + Usage: required + Value type: <prop-encoded-array> + Definition: reference to the venus gdsc to be turned on beforebooting venus core
Quoted 8 lines
Quoted 2 lines
All these new properties can't be required if the original compatible is used, right?
gdsc were earlier handled via regulator framework, but now gdsc's are being handled via power domain framework.
so any driver which require gdsc to be enabled need to use this property on mainline kernel.
so these properties will even be applicable for 8916 venus rproc on mainline kernel.
Then I'd propose to add compatible string for msm8916 too. We will need to distinguish between SoCs in order to select proper Venus firmware file too.
I think using power domain in remoteproc driver will break power management of the v4l2 venus driver. Presently I use runtime pm to switch ON/OFF GDSC and enable/disable Venus clocks.
When the .probe of venus remoteproc driver is called the power domain will become ON and never switched OFF (except when platform driver .remove method is called). One possible solution would be to add runtime pm in venus remoreproc driver.
IMO we should think more about that.
Bjorn, Stephen what are your opinions?
Signature · 2 lines
regards, Stan
Re: [RESEND PATCH v2] remoteproc: qcom: Add venus rproc support on msm8996 platform.
Quoted 75 lines · Stanimir Varbanov
Hi,
Quoted 56 lines · Avaneesh Kumar (avani)
Quoted 39 lines · Stephen Boyd
Quoted 38 lines · Avaneesh Kumar Dwivedi
This patch is based on
https://patchwork.kernel.org/patch/9415627/ https://patchwork.kernel.org/patch/9415651/This patch add clock initialization, enable and disable support. Required resource name string and rating are differentiated based on compatible string. Also added documentation for venus pil on msm8996.
Signed-off-by Avaneesh Kumar Dwivedi <[email protected]>--- .../devicetree/bindings/remoteproc/qcom,venus.txt | 26 ++++- drivers/remoteproc/qcom_venus_pil.c | 116 ++++++++++++++++++++- 2 files changed, 140 insertions(+), 2 deletions(-)diff --git
a/Documentation/devicetree/bindings/remoteproc/qcom,venus.txtb/Documentation/devicetree/bindings/remoteproc/qcom,venus.txtindex 2d73ba1..417026b 100644Documentation/devicetree/bindings/remoteproc/qcom,venus.txt +5
-1@@ -6,13 +6,30 @@ on the Qualcomm Venus remote processor core. - compatible: Usage: required Value type: <string> - Definition: must contain "qcom,venus-pil" + Definition: must contain "qcom,venus-pil" or + "qcom,venus-msm8996-pil" - memory-region: Usage: required Value type: <phandle> Definition: a phandle to a node describing reserved memory +- clocks: + Usage: required + Value type: <prop-encoded-array> + Definition: reference to the core, iface and bus and maxi clocksto be held on
Quoted 8 lines
Quoted 7 lines
Documentation/devicetree/bindings/remoteproc/qcom,venus.txt +6
-0+ behalf of the booting of the venus core + +- clock-names: + Usage: required + Value type: <stringlist> + Definition: should be "core_clk", "iface_clk", "bus_clk","
maxi_clk"Please drop
_clkfrom all clock names.OK
Quoted 6 lines
Quoted 6 lines
Documentation/devicetree/bindings/remoteproc/qcom,venus.txt +5
-0+ +- power-domains: + Usage: required + Value type: <prop-encoded-array> + Definition: reference to the venus gdsc to be turned on beforebooting venus core
Quoted 8 lines
Quoted 2 lines
All these new properties can't be required if the original compatible is used, right?
gdsc were earlier handled via regulator framework, but now gdsc's are being handled via power domain framework.
so any driver which require gdsc to be enabled need to use this property on mainline kernel.
so these properties will even be applicable for 8916 venus rproc on mainline kernel.Then I'd propose to add compatible string for msm8916 too. We will need to distinguish between SoCs in order to select proper Venus firmware file too.
I think using power domain in remoteproc driver will break power management of the v4l2 venus driver. Presently I use runtime pm to switch ON/OFF GDSC and enable/disable Venus clocks.
Quoted 9 lines
When the .probe of venus remoteproc driver is called the power domain will become ON and never switched OFF (except when platform driver .remove method is called). One possible solution would be to add runtime pm in venus remoreproc driver.
IMO we should think more about that.
Bjorn, Stephen what are your opinions?
Shall I go ahead and add runtime PM API calls during shutdown/start call along with other changes to address other comments? i was waiting any comment from Sboyed and Bjorn.
Signature · 2 lines
Qualcomm India Private Limited, on behalf of Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum, a Linux Foundation Collaborative Project.
Re: [RESEND PATCH v2] remoteproc: qcom: Add venus rproc support on msm8996 platform.
Quoted 83 lines · Avaneesh Kumar (avani)
Quoted 80 lines · Stanimir Varbanov
Hi,
Quoted 56 lines · Avaneesh Kumar (avani)
Quoted 39 lines · Stephen Boyd
Quoted 38 lines · Avaneesh Kumar Dwivedi
This patch is based on
https://patchwork.kernel.org/patch/9415627/ https://patchwork.kernel.org/patch/9415651/This patch add clock initialization, enable and disable support. Required resource name string and rating are differentiated based on compatible string. Also added documentation for venus pil on msm8996.
Signed-off-by Avaneesh Kumar Dwivedi <[email protected]>--- .../devicetree/bindings/remoteproc/qcom,venus.txt | 26 ++++- drivers/remoteproc/qcom_venus_pil.c | 116 ++++++++++++++++++++- 2 files changed, 140 insertions(+), 2 deletions(-)diff --git
a/Documentation/devicetree/bindings/remoteproc/qcom,venus.txtb/Documentation/devicetree/bindings/remoteproc/qcom,venus.txtindex 2d73ba1..417026b 100644Documentation/devicetree/bindings/remoteproc/qcom,venus.txt +5
-1@@ -6,13 +6,30 @@ on the Qualcomm Venus remote processor core. - compatible: Usage: required Value type: <string> - Definition: must contain "qcom,venus-pil" + Definition: must contain "qcom,venus-pil" or + "qcom,venus-msm8996-pil" - memory-region: Usage: required Value type: <phandle> Definition: a phandle to a node describing reserved memory +- clocks: + Usage: required + Value type: <prop-encoded-array> + Definition: reference to the core, iface and bus and maxi clocksto be held on
Quoted 8 lines
Quoted 7 lines
Documentation/devicetree/bindings/remoteproc/qcom,venus.txt +6
-0+ behalf of the booting of the venus core + +- clock-names: + Usage: required + Value type: <stringlist> + Definition: should be "core_clk", "iface_clk", "bus_clk","
maxi_clk"Please drop
_clkfrom all clock names.OK
Quoted 6 lines
Quoted 6 lines
Documentation/devicetree/bindings/remoteproc/qcom,venus.txt +5
-0+ +- power-domains: + Usage: required + Value type: <prop-encoded-array> + Definition: reference to the venus gdsc to be turned on beforebooting venus core
Quoted 8 lines
Quoted 2 lines
All these new properties can't be required if the original compatible is used, right?
gdsc were earlier handled via regulator framework, but now gdsc's are being handled via power domain framework.
so any driver which require gdsc to be enabled need to use this property on mainline kernel.
so these properties will even be applicable for 8916 venus rproc on mainline kernel.Then I'd propose to add compatible string for msm8916 too. We will need to distinguish between SoCs in order to select proper Venus firmware file too.
I think using power domain in remoteproc driver will break power management of the v4l2 venus driver. Presently I use runtime pm to switch ON/OFF GDSC and enable/disable Venus clocks.
When the .probe of venus remoteproc driver is called the power domain will become ON and never switched OFF (except when platform driver .remove method is called). One possible solution would be to add runtime pm in venus remoreproc driver.
Quoted 8 lines
Quoted 4 lines
IMO we should think more about that.
Bjorn, Stephen what are your opinions?
Shall I go ahead and add runtime PM API calls during shutdown/start call along with other changes to address other comments? i was waiting any comment from Sboyed and Bjorn.
IMO runtime should be added in remoteproc core, so you can start with an RFC patch to see how is going.
Signature · 2 lines
regards, Stan
Re: [RESEND PATCH v2] remoteproc: qcom: Add venus rproc support on msm8996 platform.
In reply to Stanimir VarbanovOn Wed 30 Nov 01:22 PST 2016, Stanimir Varbanov wrote:
[..]
Quoted 5 lines
Then I'd propose to add compatible string for msm8916 too. We will need to distinguish between SoCs in order to select proper Venus firmware file too.
This does make sense.
Quoted 4 lines
I think using power domain in remoteproc driver will break power management of the v4l2 venus driver. Presently I use runtime pm to switch ON/OFF GDSC and enable/disable Venus clocks.
Quoted 9 lines
When the .probe of venus remoteproc driver is called the power domain will become ON and never switched OFF (except when platform driver .remove method is called). One possible solution would be to add runtime pm in venus remoreproc driver.
IMO we should think more about that.
Bjorn, Stephen what are your opinions?
Sorry for not getting back to you on this after our earlier discussion on the topic.
Modelling the remoteproc and codec drivers as completely separate entities (with only the rproc phandle) gives us issues with the timing between the two. As I pulled down and started playing with [1] I noticed that I was not able to get the codec driver to probe unless I compiled it as a module which I insmoded after my /lib/firmware became available.
In addition to this, there's no way for the remoteproc driver to inform the codec driver when it's actually shutting down and booting back up during a crash recovery (an async operation).
And lastly when I talked to Rob about the DT binding the obvious question of why one piece of hardware have two disjunct nodes in the DT.
So I believe we should represent the codec driver as a child of the remoteproc driver, utilizing the newly introduced "remoteproc subdevices" to probe and remove the codec driver based on the state of the remoteproc driver. This relationship will also allow us to tie certain resources (e.g. the clocks and power-domain) to the remoteproc driver and use pm_runtime in either driver to make sure the resources are enabled appropriately.
I did backport some patches from my v4.10 remoteproc pull request into
[2] and merged this into [1] and made a prototype of this. You can find it here [3], I did test that I can boot the remoteproc and run v4l2-compliance, shut it down, boot it back up and re-run v4l2-compliance, and it seems to work fine.
With these patches you can compile the remoteproc driver builtin to the kernel and once your system is booted you can issue:
echo start > /sys/class/remoteproc/remoteproc0/state
and venus should boot and the codec should be probed.
echo stop > /sys/class/remoteproc/remoteproc0/state
will shut down the core after removing the codec.
Please let me know if you think this is a viable solution.
(This is only tested on db410c so far, looks like we might need a few more clocks for 8996 and we should probably be explicit about the noc and its resources)
[1] https://git.linaro.org/people/stanimir.varbanov/linux.git/log/?h=release/qcomlt-4.9-venus-wip
[2] https://github.com/andersson/kernel/commits/qcomlt/v4.9-rproc-backport
[3] https://github.com/andersson/kernel/commits/qcomlt/venus-wip
Regards,
Bjorn
Re: [RESEND PATCH v2] remoteproc: qcom: Add venus rproc support on msm8996 platform.
Quoted 42 lines · Bjorn Andersson
On Wed 30 Nov 01:22 PST 2016, Stanimir Varbanov wrote:
[..]Quoted 4 lines
I think using power domain in remoteproc driver will break power management of the v4l2 venus driver. Presently I use runtime pm to switch ON/OFF GDSC and enable/disable Venus clocks.
Quoted 9 lines
When the .probe of venus remoteproc driver is called the power domain will become ON and never switched OFF (except when platform driver .remove method is called). One possible solution would be to add runtime pm in venus remoreproc driver.
IMO we should think more about that.
Bjorn, Stephen what are your opinions?
Sorry for not getting back to you on this after our earlier discussion on the topic.
Modelling the remoteproc and codec drivers as completely separate entities (with only the rproc phandle) gives us issues with the timing between the two. As I pulled down and started playing with [1] I noticed that I was not able to get the codec driver to probe unless I compiled it as a module which I insmoded after my
/lib/firmwarebecame available.In addition to this, there's no way for the remoteproc driver to inform the codec driver when it's actually shutting down and booting back up during a crash recovery (an async operation).
And lastly when I talked to Rob about the DT binding the obvious question of why one piece of hardware have two disjunct nodes in the DT.
So I believe we should represent the codec driver as a child of the remoteproc driver, utilizing the newly introduced "remoteproc subdevices" to probe and remove the codec driver based on the state of the remoteproc driver. This relationship will also allow us to tie certain resources (e.g. the clocks and power-domain) to the remoteproc driver and use
pm_runtimein either driver to make sure the resources are enabled appropriately.
Quoted 8 lines
I did backport some patches from my v4.10 remoteproc pull request into
[2] and merged this into [1] and made a prototype of this. You can find it here [3], I did test that I can boot the remoteproc and run v4l2-compliance, shut it down, boot it back up and re-run v4l2-compliance, and it seems to work fine.From a DT perspective this seems backwards. We shouldn't be
putting something with a reg property underneath a "virtual" node that doesn't have a reg property. It's already causing pain, evident by these sorts of patches, so something seems wrong.
Are we gaining anything by having a remoteproc driver and device for the video hardware here? Why can't the video driver load/unload firmware with the mdt loader code by itself when it wants to boot the processor? That seems like a much simpler design and it nicely avoids this DT confusion.
Signature · 2 lines
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project
Re: [RESEND PATCH v2] remoteproc: qcom: Add venus rproc support on msm8996 platform.
Quoted 50 lines · Stephen Boyd
Quoted 42 lines · Bjorn Andersson
On Wed 30 Nov 01:22 PST 2016, Stanimir Varbanov wrote:
[..]Quoted 4 lines
I think using power domain in remoteproc driver will break power management of the v4l2 venus driver. Presently I use runtime pm to switch ON/OFF GDSC and enable/disable Venus clocks.
Quoted 9 lines
When the .probe of venus remoteproc driver is called the power domain will become ON and never switched OFF (except when platform driver .remove method is called). One possible solution would be to add runtime pm in venus remoreproc driver.
IMO we should think more about that.
Bjorn, Stephen what are your opinions?
Sorry for not getting back to you on this after our earlier discussion on the topic.
Modelling the remoteproc and codec drivers as completely separate entities (with only the rproc phandle) gives us issues with the timing between the two. As I pulled down and started playing with [1] I noticed that I was not able to get the codec driver to probe unless I compiled it as a module which I insmoded after my
/lib/firmwarebecame available.In addition to this, there's no way for the remoteproc driver to inform the codec driver when it's actually shutting down and booting back up during a crash recovery (an async operation).
And lastly when I talked to Rob about the DT binding the obvious question of why one piece of hardware have two disjunct nodes in the DT.
So I believe we should represent the codec driver as a child of the remoteproc driver, utilizing the newly introduced "remoteproc subdevices" to probe and remove the codec driver based on the state of the remoteproc driver. This relationship will also allow us to tie certain resources (e.g. the clocks and power-domain) to the remoteproc driver and use
pm_runtimein either driver to make sure the resources are enabled appropriately.Quoted 6 lines
I did backport some patches from my v4.10 remoteproc pull request into
[2] and merged this into [1] and made a prototype of this. You can find it here [3], I did test that I can boot the remoteproc and run v4l2-compliance, shut it down, boot it back up and re-run v4l2-compliance, and it seems to work fine.
Quoted 5 lines
From a DT perspective this seems backwards. We shouldn't be putting something with a reg property underneath a "virtual" node that doesn't have a reg property. It's already causing pain, evident by these sorts of patches, so something seems wrong.
I agree.
Quoted 2 lines
Are we gaining anything by having a remoteproc driver and device for the video hardware here?
After discussing the matter with Avaneesh I realized that the downstream driver powers the venus core up and down based on the presence of clients. I expect that this means we have to do the same to be able to meet our power KPIs.
With this in mind the remoteproc driver becomes a wrapper for a mdt loader and the scm pas interface - and brings with it a bunch of other things, expectations and challenges.
Quoted 2 lines
Why can't the video driver load/unload firmware with the mdt loader code by itself when it wants to boot the processor?
Providing a saner api around the mdt-loader and pas would allow venus and Adreno to reuse the code, without the remoteproc bloat.
Quoted 3 lines
That seems like a much simpler design and it nicely avoids this DT confusion.
Agreed.
I was under the impression that the venus core was always on, but after re-reading the downstream venus code a few more times I think this would be much cleaner to do so, both implementation and binding wise.
Regards,
Bjorn
Re: [RESEND PATCH v2] remoteproc: qcom: Add venus rproc support on msm8996 platform.
Quoted 73 lines · Bjorn Andersson
Quoted 48 lines · Stephen Boyd
Quoted 41 lines · Bjorn Andersson
On Wed 30 Nov 01:22 PST 2016, Stanimir Varbanov wrote:
[..]Quoted 4 lines
I think using power domain in remoteproc driver will break power management of the v4l2 venus driver. Presently I use runtime pm to switch ON/OFF GDSC and enable/disable Venus clocks.
Quoted 9 lines
When the .probe of venus remoteproc driver is called the power domain will become ON and never switched OFF (except when platform driver .remove method is called). One possible solution would be to add runtime pm in venus remoreproc driver.
IMO we should think more about that.
Bjorn, Stephen what are your opinions?
Sorry for not getting back to you on this after our earlier discussion on the topic.
Modelling the remoteproc and codec drivers as completely separate entities (with only the rproc phandle) gives us issues with the timing between the two. As I pulled down and started playing with [1] I noticed that I was not able to get the codec driver to probe unless I compiled it as a module which I insmoded after my
/lib/firmwarebecame available.In addition to this, there's no way for the remoteproc driver to inform the codec driver when it's actually shutting down and booting back up during a crash recovery (an async operation).
And lastly when I talked to Rob about the DT binding the obvious question of why one piece of hardware have two disjunct nodes in the DT.
So I believe we should represent the codec driver as a child of the remoteproc driver, utilizing the newly introduced "remoteproc subdevices" to probe and remove the codec driver based on the state of the remoteproc driver. This relationship will also allow us to tie certain resources (e.g. the clocks and power-domain) to the remoteproc driver and use
pm_runtimein either driver to make sure the resources are enabled appropriately.Quoted 6 lines
I did backport some patches from my v4.10 remoteproc pull request into
[2] and merged this into [1] and made a prototype of this. You can find it here [3], I did test that I can boot the remoteproc and run v4l2-compliance, shut it down, boot it back up and re-run v4l2-compliance, and it seems to work fine.Quoted 5 lines
From a DT perspective this seems backwards. We shouldn't be
putting something with a reg property underneath a "virtual" node that doesn't have a reg property. It's already causing pain, evident by these sorts of patches, so something seems wrong.I agree.
Quoted 2 lines
Are we gaining anything by having a remoteproc driver and device for the video hardware here?
After discussing the matter with Avaneesh I realized that the downstream driver powers the venus core up and down based on the presence of clients. I expect that this means we have to do the same to be able to meet our power KPIs.
With this in mind the remoteproc driver becomes a wrapper for a mdt loader and the scm pas interface - and brings with it a bunch of other things, expectations and challenges.
Quoted 2 lines
Why can't the video driver load/unload firmware with the mdt loader code by itself when it wants to boot the processor?
Providing a saner api around the mdt-loader and pas would allow venus and Adreno to reuse the code, without the remoteproc bloat.
Quoted 8 lines
Quoted 3 lines
That seems like a much simpler design and it nicely avoids this DT confusion.
Agreed.
I was under the impression that the venus core was always on, but after re-reading the downstream venus code a few more times I think this would be much cleaner to do so, both implementation and binding wise.
Does It mean that, you are thinking to abolish venus rproc driver? and video driver will use mdt loader and scm interface to boot venus core as clients make call? effectively avoiding any requirement for separate DT node for venus rproc?
Quoted 3 lines
Regards,
Bjorn
Signature · 2 lines
Qualcomm India Private Limited, on behalf of Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum, a Linux Foundation Collaborative Project.
Re: [RESEND PATCH v2] remoteproc: qcom: Add venus rproc support on msm8996 platform.
In reply to Bjorn AnderssonHi Bjorn,
Quoted 72 lines · Bjorn Andersson
Quoted 50 lines · Stephen Boyd
Quoted 42 lines · Bjorn Andersson
On Wed 30 Nov 01:22 PST 2016, Stanimir Varbanov wrote:
[..]Quoted 4 lines
I think using power domain in remoteproc driver will break power management of the v4l2 venus driver. Presently I use runtime pm to switch ON/OFF GDSC and enable/disable Venus clocks.
Quoted 9 lines
When the .probe of venus remoteproc driver is called the power domain will become ON and never switched OFF (except when platform driver .remove method is called). One possible solution would be to add runtime pm in venus remoreproc driver.
IMO we should think more about that.
Bjorn, Stephen what are your opinions?
Sorry for not getting back to you on this after our earlier discussion on the topic.
Modelling the remoteproc and codec drivers as completely separate entities (with only the rproc phandle) gives us issues with the timing between the two. As I pulled down and started playing with [1] I noticed that I was not able to get the codec driver to probe unless I compiled it as a module which I insmoded after my
/lib/firmwarebecame available.In addition to this, there's no way for the remoteproc driver to inform the codec driver when it's actually shutting down and booting back up during a crash recovery (an async operation).
And lastly when I talked to Rob about the DT binding the obvious question of why one piece of hardware have two disjunct nodes in the DT.
So I believe we should represent the codec driver as a child of the remoteproc driver, utilizing the newly introduced "remoteproc subdevices" to probe and remove the codec driver based on the state of the remoteproc driver. This relationship will also allow us to tie certain resources (e.g. the clocks and power-domain) to the remoteproc driver and use
pm_runtimein either driver to make sure the resources are enabled appropriately.Quoted 6 lines
I did backport some patches from my v4.10 remoteproc pull request into
[2] and merged this into [1] and made a prototype of this. You can find it here [3], I did test that I can boot the remoteproc and run v4l2-compliance, shut it down, boot it back up and re-run v4l2-compliance, and it seems to work fine.Quoted 5 lines
From a DT perspective this seems backwards. We shouldn't be putting something with a reg property underneath a "virtual" node that doesn't have a reg property. It's already causing pain, evident by these sorts of patches, so something seems wrong.
I agree.
Quoted 2 lines
Are we gaining anything by having a remoteproc driver and device for the video hardware here?
After discussing the matter with Avaneesh I realized that the downstream driver powers the venus core up and down based on the presence of clients. I expect that this means we have to do the same to be able to meet our power KPIs.
With this in mind the remoteproc driver becomes a wrapper for a mdt loader and the scm pas interface - and brings with it a bunch of other things, expectations and challenges.
Quoted 5 lines
Quoted 2 lines
Why can't the video driver load/unload firmware with the mdt loader code by itself when it wants to boot the processor?
Providing a saner api around the mdt-loader and pas would allow venus and Adreno to reuse the code, without the remoteproc bloat.
Does it make sense if we move mdt parser in drivers/firmware ?
Signature · 2 lines
regards, Stan