Skip to contentSkip to conversation

Modularize RPMH driver

PATCH Superseded Maulik Shah 4 participants 8 messages Aug 25, 2020 – Sep 29, 2020

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

v2 cover Modularize RPMH driver

Changes in v2:
- Update commit message in patch 1
- send [4] again instead of revert's revert in patch 2.

This series is to modularize RPMH driver

The tracepoint in RPMH driver was changed to _rcuidle variant based on the test results of unmerged series [1] where .power_off callback from genpd reported RCU warnings.

The series which finally got merged [2] uses CPU PM notifications and genpd .power_off callback is not implemented in RPMH driver to invoke rpmh_flush(). The CPU PM notifications are done with RCU non idle in kernel (see cpu_pm_notify() uses rcu_irq_enter_irqson() before notifications)

However using _rcuidle variant prevented RPMH driver to compile as module since these _rcuidle are not exported symbols for tracepoints.

This seris reverts the change [3] to remove _rcuidle variant for tracepoint as its no more valid test case (genpd .power_off is not implemented) and bring backs the change [4] that was reverted due to _rcuidle preventing to become modular.

[1] https://patchwork.kernel.org/project/linux-arm-msm/list/?series=243931
[2] https://patchwork.kernel.org/project/linux-arm-msm/list/?series=269733
[3] https://lore.kernel.org/r/[email protected]
[4] https://lore.kernel.org/r/[email protected]

John Stultz (1):
soc: qcom: rpmh: Allow RPMH driver to be loaded as a module

Maulik Shah (1):
Revert "drivers: qcom: rpmh-rsc: Use rcuidle tracepoints for rpmh"

 drivers/soc/qcom/Kconfig    | 2 +-
 drivers/soc/qcom/rpmh-rsc.c | 7 ++++++-
 2 files changed, 7 insertions(+), 2 deletions(-)
Signature · 2 lines
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member
of Code Aurora Forum, hosted by The Linux Foundation
#1 lore raw

PATCH v2 1/2 Revert "drivers: qcom: rpmh-rsc: Use rcuidle tracepoints for rpmh"

Commit efde2659b0fe ("drivers: qcom: rpmh-rsc: Use rcuidle tracepoints for rpmh") was written to fix a bug seen in an unmerged series that implemented a struct generic_pm_domain::power_off() callback calling rpmh_flush(). See stack trace below.

     Call trace:
      dump_backtrace+0x0/0x174
      show_stack+0x20/0x2c
      dump_stack+0xc8/0x124
      lockdep_rcu_suspicious+0xe4/0x104
      __tcs_buffer_write+0x230/0x2d0
      rpmh_rsc_write_ctrl_data+0x210/0x270
      rpmh_flush+0x84/0x24c
      rpmh_domain_power_off+0x78/0x98
      _genpd_power_off+0x40/0xc0
      genpd_power_off+0x168/0x208

Later the final merged solution is to use CPU PM notification to invoke rpmh_flush() and power_off() callback of genpd is not implemented in the driver.

CPU PM notifiers are run with RCU enabled/watching (see cpu_pm_notify() and how it calls rcu_irq_enter_irqson() before calling the notifiers).

Remove this change since RCU will not be idle during CPU PM notifications hence not required to use _rcuidle tracepoint. Using _rcuidle tracepoint prevented rpmh driver to be loadable module as these are not exported symbols.

This reverts commit efde2659b0fe835732047357b2902cca14f054d9.

---
 drivers/soc/qcom/rpmh-rsc.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/soc/qcom/rpmh-rsc.c +1 -1
index ae66757..fabe390d 100644
@@ -495,7 +495,7 @@ static void __tcs_buffer_write(struct rsc_drv *drv, int tcs_id, int cmd_id,
 		write_tcs_cmd(drv, RSC_DRV_CMD_MSGID, tcs_id, j, msgid);
 		write_tcs_cmd(drv, RSC_DRV_CMD_ADDR, tcs_id, j, cmd->addr);
 		write_tcs_cmd(drv, RSC_DRV_CMD_DATA, tcs_id, j, cmd->data);
-		trace_rpmh_send_msg_rcuidle(drv, tcs_id, j, msgid, cmd);
+		trace_rpmh_send_msg(drv, tcs_id, j, msgid, cmd);
 	}
 
 	write_tcs_reg(drv, RSC_DRV_CMD_WAIT_FOR_CMPL, tcs_id, cmd_complete);
Signature · 2 lines
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member
of Code Aurora Forum, hosted by The Linux Foundation
Reviewed-by on v2 1/2
#2 lore raw

Re: [PATCH v2 1/2] Revert "drivers: qcom: rpmh-rsc: Use rcuidle tracepoints for rpmh"

Quoted 29 lines · Maulik Shah

Commit efde2659b0fe ("drivers: qcom: rpmh-rsc: Use rcuidle tracepoints for rpmh") was written to fix a bug seen in an unmerged series that implemented a struct generic_pm_domain::power_off() callback calling rpmh_flush(). See stack trace below.

     Call trace:
      dump_backtrace+0x0/0x174
      show_stack+0x20/0x2c
      dump_stack+0xc8/0x124
      lockdep_rcu_suspicious+0xe4/0x104
      __tcs_buffer_write+0x230/0x2d0
      rpmh_rsc_write_ctrl_data+0x210/0x270
      rpmh_flush+0x84/0x24c
      rpmh_domain_power_off+0x78/0x98
      _genpd_power_off+0x40/0xc0
      genpd_power_off+0x168/0x208

Later the final merged solution is to use CPU PM notification to invoke rpmh_flush() and power_off() callback of genpd is not implemented in the driver.

CPU PM notifiers are run with RCU enabled/watching (see cpu_pm_notify() and how it calls rcu_irq_enter_irqson() before calling the notifiers).

Remove this change since RCU will not be idle during CPU PM notifications hence not required to use _rcuidle tracepoint. Using _rcuidle tracepoint prevented rpmh driver to be loadable module as these are not exported symbols.

Quoted 8 lines

This reverts commit efde2659b0fe835732047357b2902cca14f054d9.

Regards,
Bjorn

Quoted 22 lines
---
 drivers/soc/qcom/rpmh-rsc.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/soc/qcom/rpmh-rsc.c +1 -1
index ae66757..fabe390d 100644
@@ -495,7 +495,7 @@ static void __tcs_buffer_write(struct rsc_drv *drv, int tcs_id, int cmd_id,
 		write_tcs_cmd(drv, RSC_DRV_CMD_MSGID, tcs_id, j, msgid);
 		write_tcs_cmd(drv, RSC_DRV_CMD_ADDR, tcs_id, j, cmd->addr);
 		write_tcs_cmd(drv, RSC_DRV_CMD_DATA, tcs_id, j, cmd->data);
-		trace_rpmh_send_msg_rcuidle(drv, tcs_id, j, msgid, cmd);
+		trace_rpmh_send_msg(drv, tcs_id, j, msgid, cmd);
 	}
 
 	write_tcs_reg(drv, RSC_DRV_CMD_WAIT_FOR_CMPL, tcs_id, cmd_complete);
Signature · 2 lines
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member
of Code Aurora Forum, hosted by The Linux Foundation
#3 lore raw

Re: [PATCH v2 1/2] Revert "drivers: qcom: rpmh-rsc: Use rcuidle tracepoints for rpmh"

In reply to Maulik Shah
Quoted 32 lines · Maulik Shah

Commit efde2659b0fe ("drivers: qcom: rpmh-rsc: Use rcuidle tracepoints for rpmh") was written to fix a bug seen in an unmerged series that implemented a struct generic_pm_domain::power_off() callback calling rpmh_flush(). See stack trace below.

     Call trace:
      dump_backtrace+0x0/0x174
      show_stack+0x20/0x2c
      dump_stack+0xc8/0x124
      lockdep_rcu_suspicious+0xe4/0x104
      __tcs_buffer_write+0x230/0x2d0
      rpmh_rsc_write_ctrl_data+0x210/0x270
      rpmh_flush+0x84/0x24c
      rpmh_domain_power_off+0x78/0x98
      _genpd_power_off+0x40/0xc0
      genpd_power_off+0x168/0x208

Later the final merged solution is to use CPU PM notification to invoke rpmh_flush() and power_off() callback of genpd is not implemented in the driver.

CPU PM notifiers are run with RCU enabled/watching (see cpu_pm_notify() and how it calls rcu_irq_enter_irqson() before calling the notifiers).

Remove this change since RCU will not be idle during CPU PM notifications hence not required to use _rcuidle tracepoint. Using _rcuidle tracepoint prevented rpmh driver to be loadable module as these are not exported symbols.

This reverts commit efde2659b0fe835732047357b2902cca14f054d9.

Quoted 7 lines

Hey Maulik!
Thanks so much for sending out this series! I noticed this hasn't
made it to -next yet, so would it be good to resubmit it?

thanks
-john

#4 lore raw

Re: [PATCH v2 1/2] Revert "drivers: qcom: rpmh-rsc: Use rcuidle tracepoints for rpmh"

Hi,

Quoted 39 lines · John Stultz
Quoted 31 lines · Maulik Shah

Commit efde2659b0fe ("drivers: qcom: rpmh-rsc: Use rcuidle tracepoints for rpmh") was written to fix a bug seen in an unmerged series that implemented a struct generic_pm_domain::power_off() callback calling rpmh_flush(). See stack trace below.

      Call trace:
       dump_backtrace+0x0/0x174
       show_stack+0x20/0x2c
       dump_stack+0xc8/0x124
       lockdep_rcu_suspicious+0xe4/0x104
       __tcs_buffer_write+0x230/0x2d0
       rpmh_rsc_write_ctrl_data+0x210/0x270
       rpmh_flush+0x84/0x24c
       rpmh_domain_power_off+0x78/0x98
       _genpd_power_off+0x40/0xc0
       genpd_power_off+0x168/0x208

Later the final merged solution is to use CPU PM notification to invoke rpmh_flush() and power_off() callback of genpd is not implemented in the driver.

CPU PM notifiers are run with RCU enabled/watching (see cpu_pm_notify() and how it calls rcu_irq_enter_irqson() before calling the notifiers).

Remove this change since RCU will not be idle during CPU PM notifications hence not required to use _rcuidle tracepoint. Using _rcuidle tracepoint prevented rpmh driver to be loadable module as these are not exported symbols.

This reverts commit efde2659b0fe835732047357b2902cca14f054d9.

Quoted 7 lines
Quoted 6 lines

Hey Maulik!
Thanks so much for sending out this series! I noticed this hasn't
made it to -next yet, so would it be good to resubmit it?

thanks
-john

Sure i will resend the series.

Thanks,
Maulik

Signature · 1 line
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation
#5 lore raw

PATCH v2 2/2 soc: qcom: rpmh: Allow RPMH driver to be loaded as a module

In reply to Maulik Shah

From: John Stultz <[email protected]>

This patch allow the rpmh driver to be loaded as a permenent module. Meaning it can be loaded from a module, but then cannot be unloaded.

Ideally, it would include a remove hook and related logic, but the rpmh driver is fairly core to the system, so once its loaded with almost anything else to get the system to go, the dependencies are not likely to ever also be removed.

So making it a permanent module at least improves things slightly over requiring it to be a built in driver.

Cc Todd Kjos <[email protected]>
Cc Saravana Kannan <[email protected]>
Cc Andy Gross <[email protected]>
Cc Bjorn Andersson <[email protected]>
Cc Rajendra Nayak <[email protected]>
[mkshah: Fix typos in commit message, send after removing _rcuidle trace]
---
 drivers/soc/qcom/Kconfig    | 2 +-
 drivers/soc/qcom/rpmh-rsc.c | 5 +++++
 2 files changed, 6 insertions(+), 1 deletion(-)
drivers/soc/qcom/Kconfig +1 -1
index 3dc3e3d..892bdc7 100644
@@ -92,7 +92,7 @@ config QCOM_RMTFS_MEM
 	  Say y here if you intend to boot the modem remoteproc.
 
 config QCOM_RPMH
-	bool "Qualcomm RPM-Hardened (RPMH) Communication"
+	tristate "Qualcomm RPM-Hardened (RPMH) Communication"
 	depends on ARCH_QCOM || COMPILE_TEST
 	help
 	  Support for communication with the hardened-RPM blocks in
drivers/soc/qcom/rpmh-rsc.c +5 -0
index fabe390d..5fce87c 100644
@@ -13,6 +13,7 @@
 #include <linux/iopoll.h>
 #include <linux/kernel.h>
 #include <linux/list.h>
+#include <linux/module.h>
 #include <linux/of.h>
 #include <linux/of_irq.h>
 #include <linux/of_platform.h>
@@ -1025,6 +1026,7 @@ static const struct of_device_id rpmh_drv_match[] = {
 	{ .compatible = "qcom,rpmh-rsc", },
 	{ }
 };
+MODULE_DEVICE_TABLE(of, rpmh_drv_match);
 
 static struct platform_driver rpmh_driver = {
 	.probe = rpmh_rsc_probe,
@@ -1040,3 +1042,6 @@ static int __init rpmh_driver_init(void)
 	return platform_driver_register(&rpmh_driver);
 }
 arch_initcall(rpmh_driver_init);
+
+MODULE_DESCRIPTION("Qualcomm Technologies, Inc. RPMh Driver");
+MODULE_LICENSE("GPL v2");
Signature · 2 lines
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member
of Code Aurora Forum, hosted by The Linux Foundation
Reviewed-by on v2 2/2
#6 lore raw

Re: [PATCH v2 2/2] soc: qcom: rpmh: Allow RPMH driver to be loaded as a module

Quoting Maulik Shah (2020-08-25 04:21:22)

Quoted 24 lines

From: John Stultz <[email protected]>

This patch allow the rpmh driver to be loaded as a permenent module. Meaning it can be loaded from a module, but then cannot be unloaded.

Ideally, it would include a remove hook and related logic, but the rpmh driver is fairly core to the system, so once its loaded with almost anything else to get the system to go, the dependencies are not likely to ever also be removed.

So making it a permanent module at least improves things slightly over requiring it to be a built in driver.

Cc Todd Kjos <[email protected]>
Cc Saravana Kannan <[email protected]>
Cc Andy Gross <[email protected]>
Cc Bjorn Andersson <[email protected]>
Cc Rajendra Nayak <[email protected]>
[mkshah: Fix typos in commit message, send after removing _rcuidle trace]
Quoted 1 line

---

Reviewed-by on v2 2/2
#7 lore raw

Re: [PATCH v2 2/2] soc: qcom: rpmh: Allow RPMH driver to be loaded as a module

In reply to Maulik Shah
Quoted 14 lines · Maulik Shah

From: John Stultz <[email protected]>

This patch allow the rpmh driver to be loaded as a permenent module. Meaning it can be loaded from a module, but then cannot be unloaded.

Ideally, it would include a remove hook and related logic, but the rpmh driver is fairly core to the system, so once its loaded with almost anything else to get the system to go, the dependencies are not likely to ever also be removed.

So making it a permanent module at least improves things slightly over requiring it to be a built in driver.

Quoted 10 lines
Cc Todd Kjos <[email protected]>
Cc Saravana Kannan <[email protected]>
Cc Andy Gross <[email protected]>
Cc Bjorn Andersson <[email protected]>
Cc Rajendra Nayak <[email protected]>
[mkshah: Fix typos in commit message, send after removing _rcuidle trace]

Regards,
Bjorn

Quoted 49 lines
---
 drivers/soc/qcom/Kconfig    | 2 +-
 drivers/soc/qcom/rpmh-rsc.c | 5 +++++
 2 files changed, 6 insertions(+), 1 deletion(-)
drivers/soc/qcom/Kconfig +1 -1
index 3dc3e3d..892bdc7 100644
@@ -92,7 +92,7 @@ config QCOM_RMTFS_MEM
 	  Say y here if you intend to boot the modem remoteproc.
 
 config QCOM_RPMH
-	bool "Qualcomm RPM-Hardened (RPMH) Communication"
+	tristate "Qualcomm RPM-Hardened (RPMH) Communication"
 	depends on ARCH_QCOM || COMPILE_TEST
 	help
 	  Support for communication with the hardened-RPM blocks in
drivers/soc/qcom/rpmh-rsc.c +5 -0
index fabe390d..5fce87c 100644
@@ -13,6 +13,7 @@
 #include <linux/iopoll.h>
 #include <linux/kernel.h>
 #include <linux/list.h>
+#include <linux/module.h>
 #include <linux/of.h>
 #include <linux/of_irq.h>
 #include <linux/of_platform.h>
@@ -1025,6 +1026,7 @@ static const struct of_device_id rpmh_drv_match[] = {
 	{ .compatible = "qcom,rpmh-rsc", },
 	{ }
 };
+MODULE_DEVICE_TABLE(of, rpmh_drv_match);
 
 static struct platform_driver rpmh_driver = {
 	.probe = rpmh_rsc_probe,
@@ -1040,3 +1042,6 @@ static int __init rpmh_driver_init(void)
 	return platform_driver_register(&rpmh_driver);
 }
 arch_initcall(rpmh_driver_init);
+
+MODULE_DESCRIPTION("Qualcomm Technologies, Inc. RPMh Driver");
+MODULE_LICENSE("GPL v2");
Signature · 2 lines
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member
of Code Aurora Forum, hosted by The Linux Foundation