Skip to contentSkip to conversation

Modularize RPMH driver

PATCH Superseded Maulik Shah 4 participants 9 messages Aug 19, 2020 – Aug 24, 2020

Subsystems (2)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

cover Modularize RPMH driver

This series is to modularize RPMH driver

The tracepoint in RPMH driver was changed to _rcuidle variant based on the test results of unmerged series [1] where .power_off callback from genpd reported RCU warnings.

The series which finally got merged [2] uses CPU PM notifications and genpd .power_off callback is not implemented in RPMH driver to invoke rpmh_flush(). The CPU PM notifications are done with RCU non idle in kernel (see cpu_pm_notify() uses rcu_irq_enter_irqson() before notifications)

However using _rcuidle variant prevented RPMH driver to compile as module since these _rcuidle are not exported symbols for tracepoints.

This seris reverts the change [3] to remove _rcuidle variant for tracepoint as its no more valid test case (genpd .power_off is not implemented) and bring backs the change [4] that was reverted due to _rcuidle preventing to become modular.

[1] https://patchwork.kernel.org/project/linux-arm-msm/list/?series=243931
[2] https://patchwork.kernel.org/project/linux-arm-msm/list/?series=269733
[3] https://lore.kernel.org/r/[email protected]
[4] https://lore.kernel.org/r/[email protected]

Maulik Shah (2):
Revert "drivers: qcom: rpmh-rsc: Use rcuidle tracepoints for rpmh" Revert "Revert "soc: qcom: rpmh: Allow RPMH driver to be loaded as a
module""

 drivers/soc/qcom/Kconfig    | 2 +-
 drivers/soc/qcom/rpmh-rsc.c | 7 ++++++-
 2 files changed, 7 insertions(+), 2 deletions(-)
Signature · 2 lines
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member
of Code Aurora Forum, hosted by The Linux Foundation
#1 lore raw

PATCH 1/2 Revert "drivers: qcom: rpmh-rsc: Use rcuidle tracepoints for rpmh"

This change was done based on an test results of unmerged series of adding RSC power domain and using .power_off callback of genpd to invoke rpmh_flush().

     Call trace:
      dump_backtrace+0x0/0x174
      show_stack+0x20/0x2c
      dump_stack+0xc8/0x124
      lockdep_rcu_suspicious+0xe4/0x104
      __tcs_buffer_write+0x230/0x2d0
      rpmh_rsc_write_ctrl_data+0x210/0x270
      rpmh_flush+0x84/0x24c
      rpmh_domain_power_off+0x78/0x98
      _genpd_power_off+0x40/0xc0
      genpd_power_off+0x168/0x208

Later the final merged solution is to use CPU PM notification to invoke rpmh_flush() and .power_off callback of genpd is not implemented in the driver.

Remove this change since RCU will not be idle during CPU PM notifications hence not required to use _rcuidle tracepoint. Using _rcuidle tracepoint prevented rpmh driver to be loadable module as these are not exported symbols.

This reverts commit efde2659b0fe835732047357b2902cca14f054d9.

Cc Sai Prakash Ranjan <[email protected]>
Cc John Stultz <[email protected]>
Cc Stephen Rothwell <[email protected]>
---
 drivers/soc/qcom/rpmh-rsc.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/soc/qcom/rpmh-rsc.c +1 -1
index ae66757..fabe390d 100644
@@ -495,7 +495,7 @@ static void __tcs_buffer_write(struct rsc_drv *drv, int tcs_id, int cmd_id,
 		write_tcs_cmd(drv, RSC_DRV_CMD_MSGID, tcs_id, j, msgid);
 		write_tcs_cmd(drv, RSC_DRV_CMD_ADDR, tcs_id, j, cmd->addr);
 		write_tcs_cmd(drv, RSC_DRV_CMD_DATA, tcs_id, j, cmd->data);
-		trace_rpmh_send_msg_rcuidle(drv, tcs_id, j, msgid, cmd);
+		trace_rpmh_send_msg(drv, tcs_id, j, msgid, cmd);
 	}
 
 	write_tcs_reg(drv, RSC_DRV_CMD_WAIT_FOR_CMPL, tcs_id, cmd_complete);
Signature · 2 lines
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member
of Code Aurora Forum, hosted by The Linux Foundation
Reviewed-by on v1 1/2
#2 lore raw

Re: [PATCH 1/2] Revert "drivers: qcom: rpmh-rsc: Use rcuidle tracepoints for rpmh"

Quoting Maulik Shah (2020-08-19 03:07:49)

Quoted 3 lines

This change was done based on an test results of unmerged series of adding RSC power domain and using .power_off callback of genpd to invoke rpmh_flush().

Perhaps:

Commit efde2659b0fe ("drivers: qcom: rpmh-rsc: Use rcuidle tracepoints for rpmh") was written to fix a bug seen in an unmerged series that implemented a struct generic_pm_domain::power_off() callback calling rpmh_flush(). See stack trace below.

Quoted 13 lines
     Call trace:
      dump_backtrace+0x0/0x174
      show_stack+0x20/0x2c
      dump_stack+0xc8/0x124
      lockdep_rcu_suspicious+0xe4/0x104
      __tcs_buffer_write+0x230/0x2d0
      rpmh_rsc_write_ctrl_data+0x210/0x270
      rpmh_flush+0x84/0x24c
      rpmh_domain_power_off+0x78/0x98
      _genpd_power_off+0x40/0xc0
      genpd_power_off+0x168/0x208
Quoted 3 lines

Later the final merged solution is to use CPU PM notification to invoke rpmh_flush() and .power_off callback of genpd is not implemented in the driver.

I'd also add:

CPU PM notifiers are run with RCU enabled/watching (see cpu_pm_notify() and how it calls rcu_irq_enter_irqson() before calling the notifiers).

Quoted 6 lines

Remove this change since RCU will not be idle during CPU PM notifications hence not required to use _rcuidle tracepoint. Using _rcuidle tracepoint prevented rpmh driver to be loadable module as these are not exported symbols.

Quoted 7 lines

This reverts commit efde2659b0fe835732047357b2902cca14f054d9.

Cc Sai Prakash Ranjan <[email protected]>
Cc John Stultz <[email protected]>
Cc Stephen Rothwell <[email protected]>

---

Otherwise

#3 lore raw

Re: [PATCH 1/2] Revert "drivers: qcom: rpmh-rsc: Use rcuidle tracepoints for rpmh"

Hi,

Quoted 10 lines · Stephen Boyd

Quoting Maulik Shah (2020-08-19 03:07:49)

Quoted 3 lines

This change was done based on an test results of unmerged series of adding RSC power domain and using .power_off callback of genpd to invoke rpmh_flush().

Perhaps:

Commit efde2659b0fe ("drivers: qcom: rpmh-rsc: Use rcuidle tracepoints for rpmh") was written to fix a bug seen in an unmerged series that implemented a struct generic_pm_domain::power_off() callback calling rpmh_flush(). See stack trace below.

Sure i will update in v2.

Quoted 13 lines
Quoted 13 lines
      Call trace:
       dump_backtrace+0x0/0x174
       show_stack+0x20/0x2c
       dump_stack+0xc8/0x124
       lockdep_rcu_suspicious+0xe4/0x104
       __tcs_buffer_write+0x230/0x2d0
       rpmh_rsc_write_ctrl_data+0x210/0x270
       rpmh_flush+0x84/0x24c
       rpmh_domain_power_off+0x78/0x98
       _genpd_power_off+0x40/0xc0
       genpd_power_off+0x168/0x208
Quoted 7 lines
Quoted 3 lines

Later the final merged solution is to use CPU PM notification to invoke rpmh_flush() and .power_off callback of genpd is not implemented in the driver.

I'd also add:

CPU PM notifiers are run with RCU enabled/watching (see cpu_pm_notify() and how it calls rcu_irq_enter_irqson() before calling the notifiers).

Sure i will update in v2.

Quoted 8 lines
Quoted 8 lines

Remove this change since RCU will not be idle during CPU PM notifications hence not required to use _rcuidle tracepoint. Using _rcuidle tracepoint prevented rpmh driver to be loadable module as these are not exported symbols.

This reverts commit efde2659b0fe835732047357b2902cca14f054d9.

Quoted 8 lines
Quoted 5 lines
Cc Sai Prakash Ranjan <[email protected]>
Cc John Stultz <[email protected]>
Cc Stephen Rothwell <[email protected]>

---

Otherwise

Thanks for the review.

Thanks,
Maulik

Signature · 1 line
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation
Reviewed-by on v1 1/2
#4 lore raw

Re: [PATCH 1/2] Revert "drivers: qcom: rpmh-rsc: Use rcuidle tracepoints for rpmh"

In reply to Maulik Shah
Quoted 25 lines · Maulik Shah

This change was done based on an test results of unmerged series of adding RSC power domain and using .power_off callback of genpd to invoke rpmh_flush().

     Call trace:
      dump_backtrace+0x0/0x174
      show_stack+0x20/0x2c
      dump_stack+0xc8/0x124
      lockdep_rcu_suspicious+0xe4/0x104
      __tcs_buffer_write+0x230/0x2d0
      rpmh_rsc_write_ctrl_data+0x210/0x270
      rpmh_flush+0x84/0x24c
      rpmh_domain_power_off+0x78/0x98
      _genpd_power_off+0x40/0xc0
      genpd_power_off+0x168/0x208

Later the final merged solution is to use CPU PM notification to invoke rpmh_flush() and .power_off callback of genpd is not implemented in the driver.

Remove this change since RCU will not be idle during CPU PM notifications hence not required to use _rcuidle tracepoint. Using _rcuidle tracepoint prevented rpmh driver to be loadable module as these are not exported symbols.

Quoted 7 lines

This reverts commit efde2659b0fe835732047357b2902cca14f054d9.

Cc Sai Prakash Ranjan <[email protected]>
Cc John Stultz <[email protected]>
Cc Stephen Rothwell <[email protected]>

---

Thanks,
Sai

Signature · 2 lines
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member
of Code Aurora Forum, hosted by The Linux Foundation
Reviewed-by on v1 1/2
#5 lore raw

Re: [PATCH 1/2] Revert "drivers: qcom: rpmh-rsc: Use rcuidle tracepoints for rpmh"

In reply to Maulik Shah
Quoted 26 lines · Maulik Shah

This change was done based on an test results of unmerged series of adding RSC power domain and using .power_off callback of genpd to invoke rpmh_flush().

     Call trace:
      dump_backtrace+0x0/0x174
      show_stack+0x20/0x2c
      dump_stack+0xc8/0x124
      lockdep_rcu_suspicious+0xe4/0x104
      __tcs_buffer_write+0x230/0x2d0
      rpmh_rsc_write_ctrl_data+0x210/0x270
      rpmh_flush+0x84/0x24c
      rpmh_domain_power_off+0x78/0x98
      _genpd_power_off+0x40/0xc0
      genpd_power_off+0x168/0x208

Later the final merged solution is to use CPU PM notification to invoke rpmh_flush() and .power_off callback of genpd is not implemented in the driver.

Remove this change since RCU will not be idle during CPU PM notifications hence not required to use _rcuidle tracepoint. Using _rcuidle tracepoint prevented rpmh driver to be loadable module as these are not exported symbols.

Quoted 6 lines

This reverts commit efde2659b0fe835732047357b2902cca14f054d9.

Cc Sai Prakash Ranjan <[email protected]>
Cc John Stultz <[email protected]>
Cc Stephen Rothwell <[email protected]>

Along with Stephen's comments, feel free to add:

Also note, when using the PSCI OSI mode, I am proposing to use "genpd on/off notifiers" [1]. This may lead to similar RCU issues, but let's come back to adress that later on, when/if needed.

Kind regards
Uffe

[1]
https://www.spinics.net/lists/kernel/msg3629443.html

Quoted 21 lines
---
 drivers/soc/qcom/rpmh-rsc.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/soc/qcom/rpmh-rsc.c +1 -2
index ae66757..fabe390d 100644
@@ -495,7 +495,7 @@ static void __tcs_buffer_write(struct rsc_drv *drv, int tcs_id, int cmd_id,
                write_tcs_cmd(drv, RSC_DRV_CMD_MSGID, tcs_id, j, msgid);
                write_tcs_cmd(drv, RSC_DRV_CMD_ADDR, tcs_id, j, cmd->addr);
                write_tcs_cmd(drv, RSC_DRV_CMD_DATA, tcs_id, j, cmd->data);
-               trace_rpmh_send_msg_rcuidle(drv, tcs_id, j, msgid, cmd);
+               trace_rpmh_send_msg(drv, tcs_id, j, msgid, cmd);
        }
 
        write_tcs_reg(drv, RSC_DRV_CMD_WAIT_FOR_CMPL, tcs_id, cmd_complete);
--

QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation

#6 lore raw

PATCH 2/2 Revert "Revert "soc: qcom: rpmh: Allow RPMH driver to be loaded as a module""

In reply to Maulik Shah

The _rcuidle tracepoints are removed from RPMH driver which prevented to compile it as module. Bring back the change to make it module.

This reverts commit 1f7a3eb785e4a4e196729cd3d5ec97bd5f9f2940.

Cc John Stultz <[email protected]>
Cc Stephen Rothwell <[email protected]>
Cc Todd Kjos <[email protected]>
Cc Saravana Kannan <[email protected]>
---
 drivers/soc/qcom/Kconfig    | 2 +-
 drivers/soc/qcom/rpmh-rsc.c | 5 +++++
 2 files changed, 6 insertions(+), 1 deletion(-)
drivers/soc/qcom/Kconfig +1 -1
index 3dc3e3d..892bdc7 100644
@@ -92,7 +92,7 @@ config QCOM_RMTFS_MEM
 	  Say y here if you intend to boot the modem remoteproc.
 
 config QCOM_RPMH
-	bool "Qualcomm RPM-Hardened (RPMH) Communication"
+	tristate "Qualcomm RPM-Hardened (RPMH) Communication"
 	depends on ARCH_QCOM || COMPILE_TEST
 	help
 	  Support for communication with the hardened-RPM blocks in
drivers/soc/qcom/rpmh-rsc.c +5 -0
index fabe390d..5fce87c 100644
@@ -13,6 +13,7 @@
 #include <linux/iopoll.h>
 #include <linux/kernel.h>
 #include <linux/list.h>
+#include <linux/module.h>
 #include <linux/of.h>
 #include <linux/of_irq.h>
 #include <linux/of_platform.h>
@@ -1025,6 +1026,7 @@ static const struct of_device_id rpmh_drv_match[] = {
 	{ .compatible = "qcom,rpmh-rsc", },
 	{ }
 };
+MODULE_DEVICE_TABLE(of, rpmh_drv_match);
 
 static struct platform_driver rpmh_driver = {
 	.probe = rpmh_rsc_probe,
@@ -1040,3 +1042,6 @@ static int __init rpmh_driver_init(void)
 	return platform_driver_register(&rpmh_driver);
 }
 arch_initcall(rpmh_driver_init);
+
+MODULE_DESCRIPTION("Qualcomm Technologies, Inc. RPMh Driver");
+MODULE_LICENSE("GPL v2");
Signature · 2 lines
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member
of Code Aurora Forum, hosted by The Linux Foundation
#7 lore raw

Re: [PATCH 2/2] Revert "Revert "soc: qcom: rpmh: Allow RPMH driver to be loaded as a module""

Quoting Maulik Shah (2020-08-19 03:07:50)

Quoted 11 lines

The _rcuidle tracepoints are removed from RPMH driver which prevented to compile it as module. Bring back the change to make it module.

This reverts commit 1f7a3eb785e4a4e196729cd3d5ec97bd5f9f2940.

Cc John Stultz <[email protected]>
Cc Stephen Rothwell <[email protected]>
Cc Todd Kjos <[email protected]>
Cc Saravana Kannan <[email protected]>

---

Please send the original patch and add your SoB to it. Reverting a patch out of the history works, but doesn't do justice to the authorship of the patch.

#8 lore raw

Re: [PATCH 2/2] Revert "Revert "soc: qcom: rpmh: Allow RPMH driver to be loaded as a module""

Hi,

Quoted 12 lines · Stephen Boyd

Quoting Maulik Shah (2020-08-19 03:07:50)

Quoted 11 lines

The _rcuidle tracepoints are removed from RPMH driver which prevented to compile it as module. Bring back the change to make it module.

This reverts commit 1f7a3eb785e4a4e196729cd3d5ec97bd5f9f2940.

Cc John Stultz <[email protected]>
Cc Stephen Rothwell <[email protected]>
Cc Todd Kjos <[email protected]>
Cc Saravana Kannan <[email protected]>

---

Quoted 3 lines

Please send the original patch and add your SoB to it. Reverting a patch out of the history works, but doesn't do justice to the authorship of the patch.

Sure, i will re-send the original patch in v2 instead of doing revert's revert.

Thanks,
Maulik

Signature · 1 line
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation