Skip to contentSkip to conversation

remoteproc: qcom: q6v5: Update running state before requesting stop

PATCH In review Sibi Sankar 3 participants 9 messages Jun 2, 2020 – Jul 28, 2020

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH 1/2 remoteproc: qcom: q6v5: Update running state before requesting stop

Sometimes the stop triggers a watchdog rather than a stop-ack. Update the running state to false on requesting stop to skip the watchdog instead.

Error Logs:

$ echo stop > /sys/class/remoteproc/remoteproc0/state
ipa 1e40000.ipa: received modem stopping event
remoteproc-modem: watchdog received: sys_m_smsm_mpss.c:291:APPS force stop
qcom-q6v5-mss 4080000.remoteproc-modem: port failed halt
ipa 1e40000.ipa: received modem offline event
remoteproc0: stopped remote processor 4080000.remoteproc-modem
Fixes 3b415c8fb263 ("remoteproc: q6v5: Extract common resource handling")
---
 drivers/remoteproc/qcom_q6v5.c | 2 ++
 1 file changed, 2 insertions(+)
drivers/remoteproc/qcom_q6v5.c +2 -0
index 111a442c993c4..fd6fd36268d93 100644
@@ -153,6 +153,8 @@ int qcom_q6v5_request_stop(struct qcom_q6v5 *q6v5)
 {
 	int ret;
 
+	q6v5->running = false;
+
 	qcom_smem_state_update_bits(q6v5->state,
 				    BIT(q6v5->stop_bit), BIT(q6v5->stop_bit));
 
Signature · 2 lines
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project
#1 lore raw

PATCH 2/2 remoteproc: qcom_q6v5_mss: Remove redundant running state

Remove the redundant running state, as an equivalent is maintained in the common q6v5 resource handling helpers.

---
 drivers/remoteproc/qcom_q6v5_mss.c | 5 -----
 1 file changed, 5 deletions(-)
drivers/remoteproc/qcom_q6v5_mss.c +0 -5
index feb70283b6a21..702352cd66188 100644
@@ -178,8 +178,6 @@ struct q6v5 {
 	int active_reg_count;
 	int proxy_reg_count;
 
-	bool running;
-
 	bool dump_mba_loaded;
 	unsigned long dump_segment_mask;
 	unsigned long dump_complete_mask;
@@ -1275,7 +1273,6 @@ static int q6v5_start(struct rproc *rproc)
 
 	/* Reset Dump Segment Mask */
 	qproc->dump_segment_mask = 0;
-	qproc->running = true;
 
 	return 0;
 
@@ -1290,8 +1287,6 @@ static int q6v5_stop(struct rproc *rproc)
 	struct q6v5 *qproc = (struct q6v5 *)rproc->priv;
 	int ret;
 
-	qproc->running = false;
-
 	ret = qcom_q6v5_request_stop(&qproc->q6v5);
 	if (ret == -ETIMEDOUT)
 		dev_err(qproc->dev, "timed out on wait\n");
Signature · 2 lines
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project
Reviewed-by on v1 2/2
#2 lore raw

Re: [PATCH 2/2] remoteproc: qcom_q6v5_mss: Remove redundant running state

Quoted 5 lines · Sibi Sankar

Remove the redundant running state, as an equivalent is maintained in the common q6v5 resource handling helpers.

This variable was written to and never read, sigh. Thanks for cleaning it up.

Reviewed-by on v1 2/2
#3 lore raw

Re: [PATCH 2/2] remoteproc: qcom_q6v5_mss: Remove redundant running state

In reply to Sibi Sankar
Quoted 4 lines · Sibi Sankar

Remove the redundant running state, as an equivalent is maintained in the common q6v5 resource handling helpers.

Quoted 38 lines
---
 drivers/remoteproc/qcom_q6v5_mss.c | 5 -----
 1 file changed, 5 deletions(-)
drivers/remoteproc/qcom_q6v5_mss.c +0 -5
index feb70283b6a21..702352cd66188 100644
@@ -178,8 +178,6 @@ struct q6v5 {
 	int active_reg_count;
 	int proxy_reg_count;
 
-	bool running;
-
 	bool dump_mba_loaded;
 	unsigned long dump_segment_mask;
 	unsigned long dump_complete_mask;
@@ -1275,7 +1273,6 @@ static int q6v5_start(struct rproc *rproc)
 
 	/* Reset Dump Segment Mask */
 	qproc->dump_segment_mask = 0;
-	qproc->running = true;
 
 	return 0;
 
@@ -1290,8 +1287,6 @@ static int q6v5_stop(struct rproc *rproc)
 	struct q6v5 *qproc = (struct q6v5 *)rproc->priv;
 	int ret;
 
-	qproc->running = false;
-
 	ret = qcom_q6v5_request_stop(&qproc->q6v5);
 	if (ret == -ETIMEDOUT)
 		dev_err(qproc->dev, "timed out on wait\n");
Signature · 2 lines
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project
#4 lore raw
In reply to Sibi Sankar
Quoted 13 lines · Sibi Sankar

Sometimes the stop triggers a watchdog rather than a stop-ack. Update the running state to false on requesting stop to skip the watchdog instead.

Error Logs:

$ echo stop > /sys/class/remoteproc/remoteproc0/state
ipa 1e40000.ipa: received modem stopping event
remoteproc-modem: watchdog received: sys_m_smsm_mpss.c:291:APPS force stop
qcom-q6v5-mss 4080000.remoteproc-modem: port failed halt
ipa 1e40000.ipa: received modem offline event
remoteproc0: stopped remote processor 4080000.remoteproc-modem
Quoted 4 lines
Fixes 3b415c8fb263 ("remoteproc: q6v5: Extract common resource handling")

---

Are you sure you want to tolerate this behavior from MSS? This is a graceful shutdown, modem shouldn't have a problem completing the proper handshake. If they do, isn't that a bug on the modem side?

I just worry this will mask real issues that happen during graceful shutdown. -Evan

#5 lore raw

Evan,
Thanks for taking time to review the series.

Quoted 14 lines · Evan Green
Quoted 13 lines · Sibi Sankar

Sometimes the stop triggers a watchdog rather than a stop-ack. Update the running state to false on requesting stop to skip the watchdog instead.

Error Logs:

$ echo stop > /sys/class/remoteproc/remoteproc0/state
ipa 1e40000.ipa: received modem stopping event
remoteproc-modem: watchdog received: sys_m_smsm_mpss.c:291:APPS force stop
qcom-q6v5-mss 4080000.remoteproc-modem: port failed halt
ipa 1e40000.ipa: received modem offline event
remoteproc0: stopped remote processor 4080000.remoteproc-modem
Quoted 8 lines
Quoted 4 lines
Fixes 3b415c8fb263 ("remoteproc: q6v5: Extract common resource handling")

---

Are you sure you want to tolerate this behavior from MSS? This is a graceful shutdown, modem shouldn't have a problem completing the proper handshake. If they do, isn't that a bug on the modem side?

The graceful shutdown is achieved though sysmon (enabled using CONFIG_QCOM_SYSMON). When sysmon is enabled we get a shutdown-ack when we try to stop the modem, post which request stop is a basically a nop. Request stop is done to force stop the modem during failure cases (like rmtfs is not running and so on) and we do want to mask the wdog that we get during this scenario ( The locking already prevents the servicing of the wdog during shutdown, the check just prevents the scheduling of crash handler and err messages associated with it). Also this check was always present and was missed during common q6v5 resource helper migration, hence the unused running state in mss driver.

Quoted 3 lines

I just worry this will mask real issues that happen during graceful shutdown. -Evan

Signature · 2 lines
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
a Linux Foundation Collaborative Project.
Reviewed-by
#6 lore raw
Quoted 43 lines · Sibi Sankar

Evan,
Thanks for taking time to review the series.

Quoted 22 lines · Evan Green
Quoted 14 lines · Sibi Sankar

Sometimes the stop triggers a watchdog rather than a stop-ack. Update the running state to false on requesting stop to skip the watchdog instead.

Error Logs:

$ echo stop > /sys/class/remoteproc/remoteproc0/state
ipa 1e40000.ipa: received modem stopping event
remoteproc-modem: watchdog received: sys_m_smsm_mpss.c:291:APPS force
stop
qcom-q6v5-mss 4080000.remoteproc-modem: port failed halt
ipa 1e40000.ipa: received modem offline event
remoteproc0: stopped remote processor 4080000.remoteproc-modem
Quoted 5 lines
Fixes 3b415c8fb263 ("remoteproc: q6v5: Extract common resource

handling")

---

Quoted 3 lines

Are you sure you want to tolerate this behavior from MSS? This is a graceful shutdown, modem shouldn't have a problem completing the proper handshake. If they do, isn't that a bug on the modem side?

The graceful shutdown is achieved though sysmon (enabled using CONFIG_QCOM_SYSMON). When sysmon is enabled we get a shutdown-ack when we try to stop the modem, post which request stop is a basically a nop. Request stop is done to force stop the modem during failure cases (like rmtfs is not running and so on) and we do want to mask the wdog that we get during this scenario ( The locking

Quoted 8 lines

already prevents the servicing of the wdog during shutdown, the check just prevents the scheduling of crash handler and err messages associated with it). Also this check was always present and was missed during common q6v5 resource helper migration, hence the unused running state in mss driver.

So you're saying that the intention of the ->running check already in q6v5_wdog_interrupt() was to allow either the stop-ack or wdog interrupt to complete the stop. This patch just fixes a regression introduced during the refactor.
This patch seems ok to me then. It still sort of seems like a bug that the modem responds arbitrarily in one of two ways, even to a "harsh" shutdown request.

I wasn't aware of QCOM_SYSMON. Reading it now, It seems like kind of a lot... do I really need all this? Can I get by with just remoteproc stops?
Anyway, for this patch:

#7 lore raw
Quoted 61 lines · Evan Green
Quoted 43 lines · Sibi Sankar

Evan,
Thanks for taking time to review the series.

Quoted 22 lines · Evan Green
Quoted 14 lines · Sibi Sankar

Sometimes the stop triggers a watchdog rather than a stop-ack. Update the running state to false on requesting stop to skip the watchdog instead.

Error Logs:

$ echo stop > /sys/class/remoteproc/remoteproc0/state
ipa 1e40000.ipa: received modem stopping event
remoteproc-modem: watchdog received: sys_m_smsm_mpss.c:291:APPS force
stop
qcom-q6v5-mss 4080000.remoteproc-modem: port failed halt
ipa 1e40000.ipa: received modem offline event
remoteproc0: stopped remote processor 4080000.remoteproc-modem
Quoted 5 lines
Fixes 3b415c8fb263 ("remoteproc: q6v5: Extract common resource

handling")

---

Quoted 3 lines

Are you sure you want to tolerate this behavior from MSS? This is a graceful shutdown, modem shouldn't have a problem completing the proper handshake. If they do, isn't that a bug on the modem side?

The graceful shutdown is achieved though sysmon (enabled using CONFIG_QCOM_SYSMON). When sysmon is enabled we get a shutdown-ack when we try to stop the modem, post which request stop is a basically a nop. Request stop is done to force stop the modem during failure cases (like rmtfs is not running and so on) and we do want to mask the wdog that we get during this scenario ( The locking

Quoted 8 lines

already prevents the servicing of the wdog during shutdown, the check just prevents the scheduling of crash handler and err messages associated with it). Also this check was always present and was missed during common q6v5 resource helper migration, hence the unused running state in mss driver.

So you're saying that the intention of the ->running check already in q6v5_wdog_interrupt() was to allow either the stop-ack or wdog interrupt to complete the stop. This patch just fixes a regression introduced during the refactor.
This patch seems ok to me then. It still sort of seems like a bug that the modem responds arbitrarily in one of two ways, even to a "harsh" shutdown request.

Quoted 1 line

I wasn't aware of QCOM_SYSMON. Reading it now, It seems like kind of a

TL;DR
Sysmon when enabled adds a lookup
for qmi service 43 (Subsystem
control service). When we shutdown
the modem, we send a SSCTL_SHUTDOWN_REQ
to the service and the modem responds
with a shutdown-ack interrupt. If you
have rmtfs running with -v turned on
you can notice pending efs transactions
being completed followed by a bye I guess.

Quoted 5 lines

lot... do I really need all this? Can I get by with just remoteproc stops?
Anyway, for this patch:

Thanks for the review!

Signature · 2 lines
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
a Linux Foundation Collaborative Project.
#8 lore raw
In reply to Evan Green
Quoted 53 lines · Evan Green
Quoted 43 lines · Sibi Sankar

Evan,
Thanks for taking time to review the series.

Quoted 22 lines · Evan Green
Quoted 14 lines · Sibi Sankar

Sometimes the stop triggers a watchdog rather than a stop-ack. Update the running state to false on requesting stop to skip the watchdog instead.

Error Logs:

$ echo stop > /sys/class/remoteproc/remoteproc0/state
ipa 1e40000.ipa: received modem stopping event
remoteproc-modem: watchdog received: sys_m_smsm_mpss.c:291:APPS force
stop
qcom-q6v5-mss 4080000.remoteproc-modem: port failed halt
ipa 1e40000.ipa: received modem offline event
remoteproc0: stopped remote processor 4080000.remoteproc-modem
Quoted 5 lines
Fixes 3b415c8fb263 ("remoteproc: q6v5: Extract common resource

handling")

---

Quoted 3 lines

Are you sure you want to tolerate this behavior from MSS? This is a graceful shutdown, modem shouldn't have a problem completing the proper handshake. If they do, isn't that a bug on the modem side?

The graceful shutdown is achieved though sysmon (enabled using CONFIG_QCOM_SYSMON). When sysmon is enabled we get a shutdown-ack when we try to stop the modem, post which request stop is a basically a nop. Request stop is done to force stop the modem during failure cases (like rmtfs is not running and so on) and we do want to mask the wdog that we get during this scenario ( The locking

Quoted 8 lines

already prevents the servicing of the wdog during shutdown, the check just prevents the scheduling of crash handler and err messages associated with it). Also this check was always present and was missed during common q6v5 resource helper migration, hence the unused running state in mss driver.

Quoted 8 lines

So you're saying that the intention of the ->running check already in q6v5_wdog_interrupt() was to allow either the stop-ack or wdog interrupt to complete the stop. This patch just fixes a regression introduced during the refactor.
This patch seems ok to me then. It still sort of seems like a bug that the modem responds arbitrarily in one of two ways, even to a "harsh" shutdown request.

I think the patch properly fixes this regression, but I share your concern about the fact that we omit an entire category of shutdown-related crashes from being reported.

The problem I presume with the current behavior is that the shutdown will race against the crash handler - which might boot the remoteproc up again while the shutdown is progressing.

So I'm okay with this fix for the immediate problem, but think it would be nice if we could report the issue appropriately and then finalize the shutdown.

Quoted 3 lines

I wasn't aware of QCOM_SYSMON. Reading it now, It seems like kind of a lot... do I really need all this? Can I get by with just remoteproc stops?

It used to be that we set one bit in shared memory and sent an interrupt and the modem would set another bit and interrupt us back when it was done shutting down...

Quoted 3 lines

Anyway, for this patch:

Thanks,
Bjorn