remoteproc: qcom: q6v5: Update running state before requesting stop
PATCH 1/2 remoteproc: qcom: q6v5: Update running state before requesting stop
Sometimes the stop triggers a watchdog rather than a stop-ack. Update the running state to false on requesting stop to skip the watchdog instead.
Error Logs:
$ echo stop > /sys/class/remoteproc/remoteproc0/state ipa 1e40000.ipa: received modem stopping event remoteproc-modem: watchdog received: sys_m_smsm_mpss.c:291:APPS force stop qcom-q6v5-mss 4080000.remoteproc-modem: port failed halt ipa 1e40000.ipa: received modem offline event remoteproc0: stopped remote processor 4080000.remoteproc-modem
--- drivers/remoteproc/qcom_q6v5.c | 2 ++ 1 file changed, 2 insertions(+)
drivers/remoteproc/qcom_q6v5.c +2 -0
@@ -153,6 +153,8 @@ int qcom_q6v5_request_stop(struct qcom_q6v5 *q6v5) {int ret; + q6v5->running = false; +qcom_smem_state_update_bits (q6v5->state,BIT (q6v5->stop_bit),BIT (q6v5->stop_bit));
Signature · 2 lines
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum, a Linux Foundation Collaborative Project
PATCH 2/2 remoteproc: qcom_q6v5_mss: Remove redundant running state
Remove the redundant running state, as an equivalent is maintained in the common q6v5 resource handling helpers.
--- drivers/remoteproc/qcom_q6v5_mss.c | 5 ----- 1 file changed, 5 deletions(-)
drivers/remoteproc/qcom_q6v5_mss.c +0 -5
@@ -178,8 +178,6 @@ struct q6v5 {int active_reg_count;int proxy_reg_count; -bool running; -bool dump_mba_loaded;unsigned long dump_segment_mask;unsigned long dump_complete_mask;
@@ -1275,7 +1273,6 @@ static int q6v5_start(struct rproc *rproc)/* Reset Dump Segment Mask */ qproc->dump_segment_mask =0 ; - qproc->running = true;return 0 ;
@@ -1290,8 +1287,6 @@ static int q6v5_stop(struct rproc *rproc)struct q6v5 *qproc = (struct q6v5 *)rproc->priv;int ret; - qproc->running = false; - ret =qcom_q6v5_request_stop (&qproc->q6v5);if (ret == -ETIMEDOUT )dev_err (qproc->dev,"timed out on wait\n" );
Signature · 2 lines
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum, a Linux Foundation Collaborative Project
Re: [PATCH 2/2] remoteproc: qcom_q6v5_mss: Remove redundant running state
Quoted 5 lines · Sibi Sankar
Remove the redundant running state, as an equivalent is maintained in the common q6v5 resource handling helpers.
Signed-off-by Sibi Sankar <[email protected]>
This variable was written to and never read, sigh. Thanks for cleaning it up.
Re: [PATCH 2/2] remoteproc: qcom_q6v5_mss: Remove redundant running state
In reply to Sibi SankarQuoted 4 lines · Sibi Sankar
Remove the redundant running state, as an equivalent is maintained in the common q6v5 resource handling helpers.
Signed-off-by Sibi Sankar <[email protected]>
Quoted 38 lines
--- drivers/remoteproc/qcom_q6v5_mss.c | 5 ----- 1 file changed, 5 deletions(-)drivers/remoteproc/qcom_q6v5_mss.c +0
-5@@ -178,8 +178,6 @@ struct q6v5 {int active_reg_count;int proxy_reg_count; -bool running; -bool dump_mba_loaded;unsigned long dump_segment_mask;unsigned long dump_complete_mask;@@ -1275,7 +1273,6 @@ static int q6v5_start(struct rproc *rproc)/* Reset Dump Segment Mask */ qproc->dump_segment_mask =0 ; - qproc->running = true;return 0 ;@@ -1290,8 +1287,6 @@ static int q6v5_stop(struct rproc *rproc)struct q6v5 *qproc = (struct q6v5 *)rproc->priv;int ret; - qproc->running = false; - ret =qcom_q6v5_request_stop (&qproc->q6v5);if (ret == -ETIMEDOUT )dev_err (qproc->dev,"timed out on wait\n" );Signature · 2 lines
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum, a Linux Foundation Collaborative Project
Quoted 13 lines · Sibi Sankar
Sometimes the stop triggers a watchdog rather than a stop-ack. Update the running state to false on requesting stop to skip the watchdog instead.
Error Logs:
$ echo stop > /sys/class/remoteproc/remoteproc0/state ipa 1e40000.ipa: received modem stopping event remoteproc-modem: watchdog received: sys_m_smsm_mpss.c:291:APPS force stop qcom-q6v5-mss 4080000.remoteproc-modem: port failed halt ipa 1e40000.ipa: received modem offline event remoteproc0: stopped remote processor 4080000.remoteproc-modem
Quoted 4 lines
Fixes 3b415c8fb263 ("remoteproc: q6v5: Extract common resource handling")Signed-off-by Sibi Sankar <[email protected]>---
Are you sure you want to tolerate this behavior from MSS? This is a graceful shutdown, modem shouldn't have a problem completing the proper handshake. If they do, isn't that a bug on the modem side?
I just worry this will mask real issues that happen during graceful shutdown. -Evan
Evan,
Thanks for taking time to review the series.
Quoted 14 lines · Evan Green
Quoted 13 lines · Sibi Sankar
Sometimes the stop triggers a watchdog rather than a stop-ack. Update the running state to false on requesting stop to skip the watchdog instead.
Error Logs:
$ echo stop > /sys/class/remoteproc/remoteproc0/state ipa 1e40000.ipa: received modem stopping event remoteproc-modem: watchdog received: sys_m_smsm_mpss.c:291:APPS force stop qcom-q6v5-mss 4080000.remoteproc-modem: port failed halt ipa 1e40000.ipa: received modem offline event remoteproc0: stopped remote processor 4080000.remoteproc-modem
Quoted 8 lines
Quoted 4 lines
Fixes 3b415c8fb263 ("remoteproc: q6v5: Extract common resource handling")Signed-off-by Sibi Sankar <[email protected]>---
Are you sure you want to tolerate this behavior from MSS? This is a graceful shutdown, modem shouldn't have a problem completing the proper handshake. If they do, isn't that a bug on the modem side?
The graceful shutdown is achieved though sysmon (enabled using CONFIG_QCOM_SYSMON). When sysmon is enabled we get a shutdown-ack when we try to stop the modem, post which request stop is a basically a nop. Request stop is done to force stop the modem during failure cases (like rmtfs is not running and so on) and we do want to mask the wdog that we get during this scenario ( The locking already prevents the servicing of the wdog during shutdown, the check just prevents the scheduling of crash handler and err messages associated with it). Also this check was always present and was missed during common q6v5 resource helper migration, hence the unused running state in mss driver.
Quoted 3 lines
I just worry this will mask real issues that happen during graceful shutdown. -Evan
Signature · 2 lines
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project.
Evan Green
Reviewed-byQuoted 43 lines · Sibi Sankar
Evan,
Thanks for taking time to review the series.Quoted 22 lines · Evan Green
Quoted 14 lines · Sibi Sankar
Sometimes the stop triggers a watchdog rather than a stop-ack. Update the running state to false on requesting stop to skip the watchdog instead.
Error Logs:
$ echo stop > /sys/class/remoteproc/remoteproc0/state ipa 1e40000.ipa: received modem stopping event remoteproc-modem: watchdog received: sys_m_smsm_mpss.c:291:APPS force stop qcom-q6v5-mss 4080000.remoteproc-modem: port failed halt ipa 1e40000.ipa: received modem offline event remoteproc0: stopped remote processor 4080000.remoteproc-modemQuoted 5 lines
Fixes 3b415c8fb263 ("remoteproc: q6v5: Extract common resourcehandling")
Signed-off-by Sibi Sankar <[email protected]>---
Quoted 3 lines
Are you sure you want to tolerate this behavior from MSS? This is a graceful shutdown, modem shouldn't have a problem completing the proper handshake. If they do, isn't that a bug on the modem side?
The graceful shutdown is achieved though sysmon (enabled using
CONFIG_QCOM_SYSMON). When sysmon is enabled we get a shutdown-ack when we try to stop the modem, post which request stop is a basically a nop. Request stop is done to force stop the modem during failure cases (like rmtfs is not running and so on) and we do want to mask the wdog that we get during this scenario ( The locking
Quoted 8 lines
already prevents the servicing of the wdog during shutdown, the check just prevents the scheduling of crash handler and err messages associated with it). Also this check was always present and was missed during common q6v5 resource helper migration, hence the unused running state in mss driver.
So you're saying that the intention of the ->running check already in q6v5_wdog_interrupt() was to allow either the stop-ack or wdog interrupt to complete the stop. This patch just fixes a regression introduced during the refactor.
This patch seems ok to me then. It still sort of seems like a bug that the modem responds arbitrarily in one of two ways, even to a "harsh" shutdown request.
I wasn't aware of QCOM_SYSMON. Reading it now, It seems like kind of a lot... do I really need all this? Can I get by with just remoteproc stops?
Anyway, for this patch:
Quoted 61 lines · Evan Green
Quoted 43 lines · Sibi Sankar
Evan,
Thanks for taking time to review the series.Quoted 22 lines · Evan Green
Quoted 14 lines · Sibi Sankar
Sometimes the stop triggers a watchdog rather than a stop-ack. Update the running state to false on requesting stop to skip the watchdog instead.
Error Logs:
$ echo stop > /sys/class/remoteproc/remoteproc0/state ipa 1e40000.ipa: received modem stopping event remoteproc-modem: watchdog received: sys_m_smsm_mpss.c:291:APPS force stop qcom-q6v5-mss 4080000.remoteproc-modem: port failed halt ipa 1e40000.ipa: received modem offline event remoteproc0: stopped remote processor 4080000.remoteproc-modemQuoted 5 lines
Fixes 3b415c8fb263 ("remoteproc: q6v5: Extract common resourcehandling")
Signed-off-by Sibi Sankar <[email protected]>---
Quoted 3 lines
Are you sure you want to tolerate this behavior from MSS? This is a graceful shutdown, modem shouldn't have a problem completing the proper handshake. If they do, isn't that a bug on the modem side?
The graceful shutdown is achieved though sysmon (enabled using
CONFIG_QCOM_SYSMON). When sysmon is enabled we get a shutdown-ack when we try to stop the modem, post which request stop is a basically a nop. Request stop is done to force stop the modem during failure cases (like rmtfs is not running and so on) and we do want to mask the wdog that we get during this scenario ( The lockingQuoted 8 lines
already prevents the servicing of the wdog during shutdown, the check just prevents the scheduling of crash handler and err messages associated with it). Also this check was always present and was missed during common q6v5 resource helper migration, hence the unused running state in mss driver.
So you're saying that the intention of the ->running check already in
q6v5_wdog_interrupt()was to allow either the stop-ack or wdog interrupt to complete the stop. This patch just fixes a regression introduced during the refactor.
This patch seems ok to me then. It still sort of seems like a bug that the modem responds arbitrarily in one of two ways, even to a "harsh" shutdown request.
Quoted 1 line
I wasn't aware of
QCOM_SYSMON. Reading it now, It seems like kind of a
TL;DR
Sysmon when enabled adds a lookup
for qmi service 43 (Subsystem
control service). When we shutdown
the modem, we send a SSCTL_SHUTDOWN_REQ
to the service and the modem responds
with a shutdown-ack interrupt. If you
have rmtfs running with -v turned on
you can notice pending efs transactions
being completed followed by a bye I guess.
Quoted 5 lines
lot... do I really need all this? Can I get by with just remoteproc stops?
Anyway, for this patch:Reviewed-by Evan Green <[email protected]>
Thanks for the review!
Signature · 2 lines
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project.
Quoted 53 lines · Evan Green
Quoted 43 lines · Sibi Sankar
Evan,
Thanks for taking time to review the series.Quoted 22 lines · Evan Green
Quoted 14 lines · Sibi Sankar
Sometimes the stop triggers a watchdog rather than a stop-ack. Update the running state to false on requesting stop to skip the watchdog instead.
Error Logs:
$ echo stop > /sys/class/remoteproc/remoteproc0/state ipa 1e40000.ipa: received modem stopping event remoteproc-modem: watchdog received: sys_m_smsm_mpss.c:291:APPS force stop qcom-q6v5-mss 4080000.remoteproc-modem: port failed halt ipa 1e40000.ipa: received modem offline event remoteproc0: stopped remote processor 4080000.remoteproc-modemQuoted 5 lines
Fixes 3b415c8fb263 ("remoteproc: q6v5: Extract common resourcehandling")
Signed-off-by Sibi Sankar <[email protected]>---
Quoted 3 lines
Are you sure you want to tolerate this behavior from MSS? This is a graceful shutdown, modem shouldn't have a problem completing the proper handshake. If they do, isn't that a bug on the modem side?
The graceful shutdown is achieved though sysmon (enabled using
CONFIG_QCOM_SYSMON). When sysmon is enabled we get a shutdown-ack when we try to stop the modem, post which request stop is a basically a nop. Request stop is done to force stop the modem during failure cases (like rmtfs is not running and so on) and we do want to mask the wdog that we get during this scenario ( The lockingQuoted 8 lines
already prevents the servicing of the wdog during shutdown, the check just prevents the scheduling of crash handler and err messages associated with it). Also this check was always present and was missed during common q6v5 resource helper migration, hence the unused running state in mss driver.
Quoted 8 lines
So you're saying that the intention of the ->running check already in
q6v5_wdog_interrupt()was to allow either the stop-ack or wdog interrupt to complete the stop. This patch just fixes a regression introduced during the refactor.
This patch seems ok to me then. It still sort of seems like a bug that the modem responds arbitrarily in one of two ways, even to a "harsh" shutdown request.
I think the patch properly fixes this regression, but I share your concern about the fact that we omit an entire category of shutdown-related crashes from being reported.
The problem I presume with the current behavior is that the shutdown will race against the crash handler - which might boot the remoteproc up again while the shutdown is progressing.
So I'm okay with this fix for the immediate problem, but think it would be nice if we could report the issue appropriately and then finalize the shutdown.
Quoted 3 lines
I wasn't aware of
QCOM_SYSMON. Reading it now, It seems like kind of a lot... do I really need all this? Can I get by with just remoteproc stops?
It used to be that we set one bit in shared memory and sent an interrupt and the modem would set another bit and interrupt us back when it was done shutting down...
Quoted 3 lines
Anyway, for this patch:
Reviewed-by Evan Green <[email protected]>
Thanks,
Bjorn